2024-05-11T11:57:39.443 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-11T11:57:39.444 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/sessionmap} --name rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi --verbose --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702355 -- /home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702355/orig.config.yaml 2024-05-11T11:57:39.491 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-11T11:57:39.585 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702355 branch: wip-rishabh-testing-20240510.053205 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/sessionmap} email: null first_in_suite: false job_id: '7702355' kernel: client: branch: testing flavor: default kdb: true sha1: fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240510.053205 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - client session with non-allowable root sha1: 308506802b0bdea4df7050cfc1fcbdc04fea6a6e ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 308506802b0bdea4df7050cfc1fcbdc04fea6a6e kclient: syntax: v2 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240510.053205 sha1: 308506802b0bdea4df7050cfc1fcbdc04fea6a6e owner: scheduled_rishabh@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 4085 sha1: 308506802b0bdea4df7050cfc1fcbdc04fea6a6e sleep_before_teardown: 0 subset: 100/512 suite: fs suite_branch: wip-rishabh-testing-20240510.053205 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_308506802b0bdea4df7050cfc1fcbdc04fea6a6e/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 308506802b0bdea4df7050cfc1fcbdc04fea6a6e syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi167.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLcCJrL5x4XtKZnV4OkwoYGHTMYytpMjb27CDYxENeJ7Wuf1CQjNeczJsGnxPKceq1BaYiNVM+W+OPWUVeWKPvk= smithi204.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF+c6w6v8BIiyverlkjQMjsNd8TOMDnjSzdFfa1mdxBkGmGMndfO6h/EX48QI5zQxOaxZ1sa08je1A/k05pDU5E= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_sessionmap teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-11_05:57:22 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-11T11:57:39.585 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_308506802b0bdea4df7050cfc1fcbdc04fea6a6e/qa; will attempt to use it 2024-05-11T11:57:39.586 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_308506802b0bdea4df7050cfc1fcbdc04fea6a6e/qa/tasks 2024-05-11T11:57:39.586 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-11T11:57:39.587 INFO:teuthology.task.internal:Checking packages... 2024-05-11T11:57:39.611 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '308506802b0bdea4df7050cfc1fcbdc04fea6a6e' 2024-05-11T11:57:39.612 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-11T11:57:39.612 INFO:teuthology.packaging:ref: None 2024-05-11T11:57:39.612 INFO:teuthology.packaging:tag: None 2024-05-11T11:57:39.612 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240510.053205 2024-05-11T11:57:39.612 INFO:teuthology.packaging:sha1: 308506802b0bdea4df7050cfc1fcbdc04fea6a6e 2024-05-11T11:57:39.612 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240510.053205 2024-05-11T11:57:39.754 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3601.g30850680 2024-05-11T11:57:39.756 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-11T11:57:39.763 INFO:teuthology.task.internal:no buildpackages task found 2024-05-11T11:57:39.763 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-11T11:57:39.805 INFO:teuthology.task.internal:Saving configuration 2024-05-11T11:57:39.822 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-11T11:57:39.839 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-11T11:57:39.861 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi167.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702355', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-11 11:52:13.423279', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLcCJrL5x4XtKZnV4OkwoYGHTMYytpMjb27CDYxENeJ7Wuf1CQjNeczJsGnxPKceq1BaYiNVM+W+OPWUVeWKPvk='} 2024-05-11T11:57:39.881 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi204.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702355', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-11 11:52:13.425060', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF+c6w6v8BIiyverlkjQMjsNd8TOMDnjSzdFfa1mdxBkGmGMndfO6h/EX48QI5zQxOaxZ1sa08je1A/k05pDU5E='} 2024-05-11T11:57:39.881 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-11T11:57:39.888 INFO:teuthology.task.internal:roles: ubuntu@smithi167.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-05-11T11:57:39.888 INFO:teuthology.task.internal:roles: ubuntu@smithi204.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-05-11T11:57:39.888 INFO:teuthology.run_tasks:Running task console_log... 2024-05-11T11:57:39.984 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f21f880f160>, signals=[15]) 2024-05-11T11:57:39.985 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-11T11:57:39.991 INFO:teuthology.task.internal:Opening connections... 2024-05-11T11:57:39.991 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi167.front.sepia.ceph.com 2024-05-11T11:57:39.993 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T11:57:40.071 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi204.front.sepia.ceph.com 2024-05-11T11:57:40.072 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T11:57:40.145 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-11T11:57:40.152 DEBUG:teuthology.orchestra.run.smithi167:> uname -m 2024-05-11T11:57:40.169 INFO:teuthology.orchestra.run.smithi167.stdout:x86_64 2024-05-11T11:57:40.170 DEBUG:teuthology.orchestra.run.smithi167:> cat /etc/os-release 2024-05-11T11:57:40.226 INFO:teuthology.orchestra.run.smithi167.stdout:NAME="CentOS Stream" 2024-05-11T11:57:40.226 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION="9" 2024-05-11T11:57:40.227 INFO:teuthology.orchestra.run.smithi167.stdout:ID="centos" 2024-05-11T11:57:40.227 INFO:teuthology.orchestra.run.smithi167.stdout:ID_LIKE="rhel fedora" 2024-05-11T11:57:40.227 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION_ID="9" 2024-05-11T11:57:40.227 INFO:teuthology.orchestra.run.smithi167.stdout:PLATFORM_ID="platform:el9" 2024-05-11T11:57:40.227 INFO:teuthology.orchestra.run.smithi167.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-11T11:57:40.227 INFO:teuthology.orchestra.run.smithi167.stdout:ANSI_COLOR="0;31" 2024-05-11T11:57:40.227 INFO:teuthology.orchestra.run.smithi167.stdout:LOGO="fedora-logo-icon" 2024-05-11T11:57:40.227 INFO:teuthology.orchestra.run.smithi167.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-11T11:57:40.227 INFO:teuthology.orchestra.run.smithi167.stdout:HOME_URL="https://centos.org/" 2024-05-11T11:57:40.227 INFO:teuthology.orchestra.run.smithi167.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-11T11:57:40.227 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-11T11:57:40.227 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-11T11:57:40.228 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server 2024-05-11T11:57:40.256 DEBUG:teuthology.orchestra.run.smithi204:> uname -m 2024-05-11T11:57:40.273 INFO:teuthology.orchestra.run.smithi204.stdout:x86_64 2024-05-11T11:57:40.273 DEBUG:teuthology.orchestra.run.smithi204:> cat /etc/os-release 2024-05-11T11:57:40.331 INFO:teuthology.orchestra.run.smithi204.stdout:NAME="CentOS Stream" 2024-05-11T11:57:40.331 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION="9" 2024-05-11T11:57:40.331 INFO:teuthology.orchestra.run.smithi204.stdout:ID="centos" 2024-05-11T11:57:40.331 INFO:teuthology.orchestra.run.smithi204.stdout:ID_LIKE="rhel fedora" 2024-05-11T11:57:40.331 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION_ID="9" 2024-05-11T11:57:40.331 INFO:teuthology.orchestra.run.smithi204.stdout:PLATFORM_ID="platform:el9" 2024-05-11T11:57:40.331 INFO:teuthology.orchestra.run.smithi204.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-11T11:57:40.331 INFO:teuthology.orchestra.run.smithi204.stdout:ANSI_COLOR="0;31" 2024-05-11T11:57:40.331 INFO:teuthology.orchestra.run.smithi204.stdout:LOGO="fedora-logo-icon" 2024-05-11T11:57:40.331 INFO:teuthology.orchestra.run.smithi204.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-11T11:57:40.332 INFO:teuthology.orchestra.run.smithi204.stdout:HOME_URL="https://centos.org/" 2024-05-11T11:57:40.332 INFO:teuthology.orchestra.run.smithi204.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-11T11:57:40.332 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-11T11:57:40.332 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-11T11:57:40.332 INFO:teuthology.lock.ops:Updating smithi204.front.sepia.ceph.com on lock server 2024-05-11T11:57:40.348 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-11T11:57:40.357 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-11T11:57:40.380 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-11T11:57:40.380 DEBUG:teuthology.orchestra.run.smithi167:> test '!' -e /home/ubuntu/cephtest 2024-05-11T11:57:40.383 DEBUG:teuthology.orchestra.run.smithi204:> test '!' -e /home/ubuntu/cephtest 2024-05-11T11:57:40.399 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-11T11:57:40.404 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-11T11:57:40.404 DEBUG:teuthology.orchestra.run.smithi167:> test -z $(ls -A /var/lib/ceph) 2024-05-11T11:57:40.441 DEBUG:teuthology.orchestra.run.smithi204:> test -z $(ls -A /var/lib/ceph) 2024-05-11T11:57:40.531 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-11T11:57:40.588 INFO:teuthology.run_tasks:Running task kernel... 2024-05-11T11:57:40.602 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': 'fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e'} 2024-05-11T11:57:40.603 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-05-11T11:57:40.603 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-05-11T11:57:40.603 DEBUG:teuthology.orchestra.run.smithi167:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-11T11:57:40.604 DEBUG:teuthology.orchestra.run.smithi204:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-11T11:57:40.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T11:57:40.621 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-05-11T11:57:40.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T11:57:40.625 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-05-11T11:57:40.859 DEBUG:teuthology.task.kernel:sha1 for client.1 is fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e 2024-05-11T11:57:40.859 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e"... 2024-05-11T11:57:40.859 DEBUG:teuthology.orchestra.run.smithi204:> uname -r 2024-05-11T11:57:40.863 DEBUG:teuthology.task.kernel:sha1 for client.0 is fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e 2024-05-11T11:57:40.864 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e"... 2024-05-11T11:57:40.864 DEBUG:teuthology.orchestra.run.smithi167:> uname -r 2024-05-11T11:57:40.876 INFO:teuthology.orchestra.run.smithi204.stdout:5.14.0-437.el9.x86_64 2024-05-11T11:57:40.877 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e 2024-05-11T11:57:40.877 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-11T11:57:40.877 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum upgrade -y linux-firmware 2024-05-11T11:57:40.881 INFO:teuthology.orchestra.run.smithi167.stdout:5.14.0-437.el9.x86_64 2024-05-11T11:57:40.882 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e 2024-05-11T11:57:40.882 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-11T11:57:40.882 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum upgrade -y linux-firmware 2024-05-11T11:57:43.201 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.1 MB 00:00 2024-05-11T11:57:43.449 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - BaseOS 9.3 MB/s | 8.1 MB 00:00 2024-05-11T11:57:45.202 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-05-11T11:57:45.759 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - AppStream 16 MB/s | 19 MB 00:01 2024-05-11T11:57:49.453 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - CRB 9.5 MB/s | 6.2 MB 00:00 2024-05-11T11:57:50.926 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - CRB 4.0 MB/s | 6.2 MB 00:01 2024-05-11T11:57:51.262 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 16 kB 00:00 2024-05-11T11:57:52.601 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - Extras packages 51 kB/s | 16 kB 00:00 2024-05-11T11:57:52.930 INFO:teuthology.orchestra.run.smithi204.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-05-11T11:57:53.803 INFO:teuthology.orchestra.run.smithi167.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-05-11T11:57:58.064 INFO:teuthology.orchestra.run.smithi204.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-05-11T11:57:58.953 INFO:teuthology.orchestra.run.smithi167.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-05-11T11:58:00.181 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-05-11T11:58:00.182 INFO:teuthology.orchestra.run.smithi204.stdout:Nothing to do. 2024-05-11T11:58:00.182 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-05-11T11:58:00.287 INFO:teuthology.task.kernel:Downloading kernel fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e on client.1... 2024-05-11T11:58:00.288 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e 2024-05-11T11:58:00.513 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e/centos/9/flavors/default/x86_64/ 2024-05-11T11:58:00.513 DEBUG:teuthology.orchestra.run.smithi204:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_gfb9a6a44107e-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e/centos/9/flavors/default/x86_64/ --input-file=- 2024-05-11T11:58:00.516 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi204.front.sepia.ceph.com to complete... 2024-05-11T11:58:01.055 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-05-11T11:58:01.056 INFO:teuthology.orchestra.run.smithi167.stdout:Nothing to do. 2024-05-11T11:58:01.056 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2024-05-11T11:58:01.157 INFO:teuthology.task.kernel:Downloading kernel fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e on client.0... 2024-05-11T11:58:01.158 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e 2024-05-11T11:58:01.303 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e/centos/9/flavors/default/x86_64/ 2024-05-11T11:58:01.303 DEBUG:teuthology.orchestra.run.smithi167:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_gfb9a6a44107e-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e/centos/9/flavors/default/x86_64/ --input-file=- 2024-05-11T11:58:01.306 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi167.front.sepia.ceph.com to complete... 2024-05-11T11:58:13.605 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11 11:58:13 URL:https://2.chacra.ceph.com/r/kernel/testing/fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_gfb9a6a44107e-1.x86_64.rpm [357387737/357387737] -> "/tmp/kernel.x86_64.rpm" [1] 2024-05-11T11:58:13.606 INFO:teuthology.orchestra.run.smithi204.stderr:FINISHED --2024-05-11 11:58:13-- 2024-05-11T11:58:13.606 INFO:teuthology.orchestra.run.smithi204.stderr:Total wall clock time: 13s 2024-05-11T11:58:13.606 INFO:teuthology.orchestra.run.smithi204.stderr:Downloaded: 1 files, 341M in 13s (26.8 MB/s) 2024-05-11T11:58:13.608 INFO:teuthology.task.kernel:Installing kernel fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e on client.1... 2024-05-11T11:58:13.609 DEBUG:teuthology.orchestra.run.smithi204:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-05-11T11:58:15.747 INFO:teuthology.orchestra.run.smithi204.stdout:Verifying... ######################################## 2024-05-11T11:58:16.000 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing... ######################################## 2024-05-11T11:58:16.119 INFO:teuthology.orchestra.run.smithi204.stdout:Updating / installing... 2024-05-11T11:58:24.081 INFO:teuthology.orchestra.run.smithi204.stdout:kernel-6.9.0_rc3_gfb9a6a44107e-1 ######################################## 2024-05-11T11:58:24.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11 11:58:24 URL:https://2.chacra.ceph.com/r/kernel/testing/fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_gfb9a6a44107e-1.x86_64.rpm [357387737/357387737] -> "/tmp/kernel.x86_64.rpm" [1] 2024-05-11T11:58:24.972 INFO:teuthology.orchestra.run.smithi167.stderr:FINISHED --2024-05-11 11:58:24-- 2024-05-11T11:58:24.972 INFO:teuthology.orchestra.run.smithi167.stderr:Total wall clock time: 24s 2024-05-11T11:58:24.972 INFO:teuthology.orchestra.run.smithi167.stderr:Downloaded: 1 files, 341M in 23s (14.7 MB/s) 2024-05-11T11:58:24.975 INFO:teuthology.task.kernel:Installing kernel fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e on client.0... 2024-05-11T11:58:24.975 DEBUG:teuthology.orchestra.run.smithi167:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-05-11T11:58:27.149 INFO:teuthology.orchestra.run.smithi167.stdout:Verifying... ######################################## 2024-05-11T11:58:27.282 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing... ######################################## 2024-05-11T11:58:27.418 INFO:teuthology.orchestra.run.smithi167.stdout:Updating / installing... 2024-05-11T11:58:35.505 INFO:teuthology.orchestra.run.smithi167.stdout:kernel-6.9.0_rc3_gfb9a6a44107e-1 ######################################## 2024-05-11T11:59:03.807 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi204.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-05-11T11:59:03.807 DEBUG:teuthology.orchestra.run.smithi204:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-05-11T11:59:03.880 INFO:teuthology.orchestra.run.smithi204.stdout:/boot/System.map-6.9.0-rc3-gfb9a6a44107e 2024-05-11T11:59:03.880 INFO:teuthology.orchestra.run.smithi204.stdout:/boot/config-6.9.0-rc3-gfb9a6a44107e 2024-05-11T11:59:03.881 INFO:teuthology.orchestra.run.smithi204.stdout:/boot/vmlinuz-6.9.0-rc3-gfb9a6a44107e 2024-05-11T11:59:03.881 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/System.map 2024-05-11T11:59:03.881 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/config 2024-05-11T11:59:03.881 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel 2024-05-11T11:59:03.881 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch 2024-05-11T11:59:03.881 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86 2024-05-11T11:59:03.881 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto 2024-05-11T11:59:03.881 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-11T11:59:03.881 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-11T11:59:03.881 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-11T11:59:03.881 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-11T11:59:03.881 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-11T11:59:03.881 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-11T11:59:03.882 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-11T11:59:03.882 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/crc32c-intel.ko 2024-05-11T11:59:03.882 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-11T11:59:03.882 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-11T11:59:03.882 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-11T11:59:03.882 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-11T11:59:03.882 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-11T11:59:03.882 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-11T11:59:03.882 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-11T11:59:03.882 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-11T11:59:03.882 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-11T11:59:03.882 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-11T11:59:03.883 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-11T11:59:03.883 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-11T11:59:03.883 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kernel 2024-05-11T11:59:03.883 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kernel/cpu 2024-05-11T11:59:03.883 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kernel/cpu/mce 2024-05-11T11:59:03.883 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-11T11:59:03.883 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kvm 2024-05-11T11:59:03.883 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-11T11:59:03.883 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-11T11:59:03.883 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kvm/kvm.ko 2024-05-11T11:59:03.883 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto 2024-05-11T11:59:03.883 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/ansi_cprng.ko 2024-05-11T11:59:03.884 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/anubis.ko 2024-05-11T11:59:03.884 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/arc4.ko 2024-05-11T11:59:03.884 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx 2024-05-11T11:59:03.884 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_memcpy.ko 2024-05-11T11:59:03.884 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_pq.ko 2024-05-11T11:59:03.884 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-11T11:59:03.885 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_tx.ko 2024-05-11T11:59:03.885 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_xor.ko 2024-05-11T11:59:03.885 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/raid6test.ko 2024-05-11T11:59:03.885 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/authenc.ko 2024-05-11T11:59:03.885 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/authencesn.ko 2024-05-11T11:59:03.885 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/blake2b_generic.ko 2024-05-11T11:59:03.885 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/blowfish_common.ko 2024-05-11T11:59:03.885 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/blowfish_generic.ko 2024-05-11T11:59:03.885 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/camellia_generic.ko 2024-05-11T11:59:03.885 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cast5_generic.ko 2024-05-11T11:59:03.885 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cast6_generic.ko 2024-05-11T11:59:03.885 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cast_common.ko 2024-05-11T11:59:03.885 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/ccm.ko 2024-05-11T11:59:03.885 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cmac.ko 2024-05-11T11:59:03.885 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/crc32_generic.ko 2024-05-11T11:59:03.886 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/crypto_engine.ko 2024-05-11T11:59:03.886 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/crypto_user.ko 2024-05-11T11:59:03.886 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cts.ko 2024-05-11T11:59:03.886 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/des_generic.ko 2024-05-11T11:59:03.886 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/dh_generic.ko 2024-05-11T11:59:03.886 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/echainiv.ko 2024-05-11T11:59:03.886 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/essiv.ko 2024-05-11T11:59:03.886 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/fcrypt.ko 2024-05-11T11:59:03.886 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/khazad.ko 2024-05-11T11:59:03.886 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/lz4.ko 2024-05-11T11:59:03.886 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/lz4hc.ko 2024-05-11T11:59:03.886 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/md4.ko 2024-05-11T11:59:03.886 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/michael_mic.ko 2024-05-11T11:59:03.887 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/pcbc.ko 2024-05-11T11:59:03.887 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/pcrypt.ko 2024-05-11T11:59:03.887 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/rmd160.ko 2024-05-11T11:59:03.887 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/seed.ko 2024-05-11T11:59:03.887 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/serpent_generic.ko 2024-05-11T11:59:03.887 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/tcrypt.ko 2024-05-11T11:59:03.887 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/tea.ko 2024-05-11T11:59:03.887 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/twofish_common.ko 2024-05-11T11:59:03.887 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/twofish_generic.ko 2024-05-11T11:59:03.887 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/vmac.ko 2024-05-11T11:59:03.887 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/wp512.ko 2024-05-11T11:59:03.887 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/xcbc.ko 2024-05-11T11:59:03.887 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/xor.ko 2024-05-11T11:59:03.887 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/xxhash_generic.ko 2024-05-11T11:59:03.887 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers 2024-05-11T11:59:03.887 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi 2024-05-11T11:59:03.888 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-11T11:59:03.888 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/acpi_pad.ko 2024-05-11T11:59:03.888 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/ec_sys.ko 2024-05-11T11:59:03.888 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/platform_profile.ko 2024-05-11T11:59:03.888 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/sbs.ko 2024-05-11T11:59:03.888 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/sbshc.ko 2024-05-11T11:59:03.888 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/video.ko 2024-05-11T11:59:03.888 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata 2024-05-11T11:59:03.888 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/acard-ahci.ko 2024-05-11T11:59:03.888 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/ahci_platform.ko 2024-05-11T11:59:03.888 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/ata_generic.ko 2024-05-11T11:59:03.888 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/libahci_platform.ko 2024-05-11T11:59:03.888 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_acpi.ko 2024-05-11T11:59:03.888 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_ali.ko 2024-05-11T11:59:03.888 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_amd.ko 2024-05-11T11:59:03.889 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_artop.ko 2024-05-11T11:59:03.889 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_atiixp.ko 2024-05-11T11:59:03.889 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_atp867x.ko 2024-05-11T11:59:03.889 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_cmd640.ko 2024-05-11T11:59:03.889 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_cmd64x.ko 2024-05-11T11:59:03.889 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_cypress.ko 2024-05-11T11:59:03.889 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_efar.ko 2024-05-11T11:59:03.889 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_hpt366.ko 2024-05-11T11:59:03.889 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_hpt37x.ko 2024-05-11T11:59:03.889 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-11T11:59:03.889 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-11T11:59:03.889 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_it8213.ko 2024-05-11T11:59:03.889 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_it821x.ko 2024-05-11T11:59:03.889 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_jmicron.ko 2024-05-11T11:59:03.889 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_marvell.ko 2024-05-11T11:59:03.890 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_mpiix.ko 2024-05-11T11:59:03.890 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_netcell.ko 2024-05-11T11:59:03.890 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_ninja32.ko 2024-05-11T11:59:03.890 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_ns87410.ko 2024-05-11T11:59:03.890 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_ns87415.ko 2024-05-11T11:59:03.890 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_oldpiix.ko 2024-05-11T11:59:03.890 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_opti.ko 2024-05-11T11:59:03.890 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_optidma.ko 2024-05-11T11:59:03.890 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_pcmcia.ko 2024-05-11T11:59:03.890 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-11T11:59:03.890 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-11T11:59:03.890 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_piccolo.ko 2024-05-11T11:59:03.891 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_rdc.ko 2024-05-11T11:59:03.891 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_sch.ko 2024-05-11T11:59:03.891 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_serverworks.ko 2024-05-11T11:59:03.891 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_sil680.ko 2024-05-11T11:59:03.891 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_sis.ko 2024-05-11T11:59:03.891 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_sl82c105.ko 2024-05-11T11:59:03.891 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_triflex.ko 2024-05-11T11:59:03.891 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_via.ko 2024-05-11T11:59:03.891 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pdc_adma.ko 2024-05-11T11:59:03.891 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_inic162x.ko 2024-05-11T11:59:03.891 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_mv.ko 2024-05-11T11:59:03.891 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_nv.ko 2024-05-11T11:59:03.891 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_promise.ko 2024-05-11T11:59:03.891 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_qstor.ko 2024-05-11T11:59:03.891 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_sil.ko 2024-05-11T11:59:03.892 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_sil24.ko 2024-05-11T11:59:03.892 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_sis.ko 2024-05-11T11:59:03.892 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_svw.ko 2024-05-11T11:59:03.892 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_sx4.ko 2024-05-11T11:59:03.892 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_uli.ko 2024-05-11T11:59:03.892 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_via.ko 2024-05-11T11:59:03.892 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_vsc.ko 2024-05-11T11:59:03.892 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/auxdisplay 2024-05-11T11:59:03.892 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/auxdisplay/ks0108.ko 2024-05-11T11:59:03.892 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/base 2024-05-11T11:59:03.892 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/base/regmap 2024-05-11T11:59:03.892 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/base/regmap/regmap-i2c.ko 2024-05-11T11:59:03.892 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/base/regmap/regmap-mmio.ko 2024-05-11T11:59:03.892 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/bcma 2024-05-11T11:59:03.893 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/bcma/bcma.ko 2024-05-11T11:59:03.893 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block 2024-05-11T11:59:03.893 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/aoe 2024-05-11T11:59:03.893 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/aoe/aoe.ko 2024-05-11T11:59:03.893 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/brd.ko 2024-05-11T11:59:03.893 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/drbd 2024-05-11T11:59:03.893 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/drbd/drbd.ko 2024-05-11T11:59:03.893 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/floppy.ko 2024-05-11T11:59:03.893 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/loop.ko 2024-05-11T11:59:03.893 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/mtip32xx 2024-05-11T11:59:03.893 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-11T11:59:03.893 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/nbd.ko 2024-05-11T11:59:03.893 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/null_blk 2024-05-11T11:59:03.893 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/null_blk/null_blk.ko 2024-05-11T11:59:03.894 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/pktcdvd.ko 2024-05-11T11:59:03.894 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/rbd.ko 2024-05-11T11:59:03.894 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/virtio_blk.ko 2024-05-11T11:59:03.894 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/xen-blkback 2024-05-11T11:59:03.894 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-11T11:59:03.894 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/xen-blkfront.ko 2024-05-11T11:59:03.894 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/zram 2024-05-11T11:59:03.894 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/zram/zram.ko 2024-05-11T11:59:03.894 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char 2024-05-11T11:59:03.894 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hangcheck-timer.ko 2024-05-11T11:59:03.894 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random 2024-05-11T11:59:03.894 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-11T11:59:03.894 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-11T11:59:03.895 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-11T11:59:03.895 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/via-rng.ko 2024-05-11T11:59:03.895 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-11T11:59:03.895 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi 2024-05-11T11:59:03.895 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-11T11:59:03.895 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-11T11:59:03.895 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-11T11:59:03.895 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-11T11:59:03.895 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-11T11:59:03.895 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-11T11:59:03.895 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/lp.ko 2024-05-11T11:59:03.895 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/mwave 2024-05-11T11:59:03.895 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/mwave/mwave.ko 2024-05-11T11:59:03.895 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ppdev.ko 2024-05-11T11:59:03.895 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tlclk.ko 2024-05-11T11:59:03.896 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm 2024-05-11T11:59:03.896 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm.ko 2024-05-11T11:59:03.896 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-11T11:59:03.896 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-11T11:59:03.896 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-11T11:59:03.896 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_tis.ko 2024-05-11T11:59:03.896 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-05-11T11:59:03.896 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/uv_mmtimer.ko 2024-05-11T11:59:03.896 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/virtio_console.ko 2024-05-11T11:59:03.896 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq 2024-05-11T11:59:03.896 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-05-11T11:59:03.896 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-11T11:59:03.896 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-11T11:59:03.896 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-05-11T11:59:03.897 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/powernow-k8.ko 2024-05-11T11:59:03.897 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-11T11:59:03.897 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto 2024-05-11T11:59:03.897 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/ccp 2024-05-11T11:59:03.897 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-11T11:59:03.897 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/ccp/ccp.ko 2024-05-11T11:59:03.897 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel 2024-05-11T11:59:03.897 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat 2024-05-11T11:59:03.897 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat/qat_common 2024-05-11T11:59:03.897 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-11T11:59:03.897 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-05-11T11:59:03.897 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-11T11:59:03.897 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/padlock-aes.ko 2024-05-11T11:59:03.897 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/padlock-sha.ko 2024-05-11T11:59:03.898 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/virtio 2024-05-11T11:59:03.898 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-11T11:59:03.898 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dca 2024-05-11T11:59:03.898 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dca/dca.ko 2024-05-11T11:59:03.898 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/devfreq 2024-05-11T11:59:03.898 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-05-11T11:59:03.898 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma 2024-05-11T11:59:03.898 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma/dw 2024-05-11T11:59:03.898 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-11T11:59:03.898 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma/ioat 2024-05-11T11:59:03.898 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-11T11:59:03.898 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac 2024-05-11T11:59:03.898 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/amd64_edac.ko 2024-05-11T11:59:03.899 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/e752x_edac.ko 2024-05-11T11:59:03.899 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/edac_mce_amd.ko 2024-05-11T11:59:03.899 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i3000_edac.ko 2024-05-11T11:59:03.899 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i3200_edac.ko 2024-05-11T11:59:03.899 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i5100_edac.ko 2024-05-11T11:59:03.899 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i5400_edac.ko 2024-05-11T11:59:03.899 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i7300_edac.ko 2024-05-11T11:59:03.899 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i7core_edac.ko 2024-05-11T11:59:03.899 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i82975x_edac.ko 2024-05-11T11:59:03.899 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/ie31200_edac.ko 2024-05-11T11:59:03.899 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/sb_edac.ko 2024-05-11T11:59:03.899 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/x38_edac.ko 2024-05-11T11:59:03.899 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/firmware 2024-05-11T11:59:03.899 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/firmware/edd.ko 2024-05-11T11:59:03.900 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-11T11:59:03.900 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio 2024-05-11T11:59:03.900 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-ich.ko 2024-05-11T11:59:03.900 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-11T11:59:03.900 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid 2024-05-11T11:59:03.900 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-appleir.ko 2024-05-11T11:59:03.900 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-aureal.ko 2024-05-11T11:59:03.900 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-axff.ko 2024-05-11T11:59:03.900 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-dr.ko 2024-05-11T11:59:03.900 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-elecom.ko 2024-05-11T11:59:03.900 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-elo.ko 2024-05-11T11:59:03.900 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-emsff.ko 2024-05-11T11:59:03.900 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-gaff.ko 2024-05-11T11:59:03.900 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-gt683r.ko 2024-05-11T11:59:03.901 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-gyration.ko 2024-05-11T11:59:03.901 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-11T11:59:03.901 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-11T11:59:03.901 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-holtekff.ko 2024-05-11T11:59:03.901 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-hyperv.ko 2024-05-11T11:59:03.901 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-icade.ko 2024-05-11T11:59:03.901 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-keytouch.ko 2024-05-11T11:59:03.901 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-kye.ko 2024-05-11T11:59:03.901 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-lcpower.ko 2024-05-11T11:59:03.901 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-led.ko 2024-05-11T11:59:03.901 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-lenovo.ko 2024-05-11T11:59:03.901 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-11T11:59:03.901 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-11T11:59:03.901 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-multitouch.ko 2024-05-11T11:59:03.901 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-ortek.ko 2024-05-11T11:59:03.902 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-penmount.ko 2024-05-11T11:59:03.902 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-petalynx.ko 2024-05-11T11:59:03.902 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-picolcd.ko 2024-05-11T11:59:03.902 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-pl.ko 2024-05-11T11:59:03.902 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-plantronics.ko 2024-05-11T11:59:03.902 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-primax.ko 2024-05-11T11:59:03.902 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-rmi.ko 2024-05-11T11:59:03.902 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-11T11:59:03.902 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-common.ko 2024-05-11T11:59:03.902 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-11T11:59:03.902 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-11T11:59:03.902 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-11T11:59:03.902 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-11T11:59:03.902 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-11T11:59:03.903 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-11T11:59:03.903 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-11T11:59:03.903 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-11T11:59:03.903 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-11T11:59:03.903 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat.ko 2024-05-11T11:59:03.903 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-saitek.ko 2024-05-11T11:59:03.903 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-samsung.ko 2024-05-11T11:59:03.903 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-11T11:59:03.903 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-sjoy.ko 2024-05-11T11:59:03.903 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-sony.ko 2024-05-11T11:59:03.903 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-speedlink.ko 2024-05-11T11:59:03.903 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-steelseries.ko 2024-05-11T11:59:03.903 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-sunplus.ko 2024-05-11T11:59:03.903 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-11T11:59:03.904 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-tivo.ko 2024-05-11T11:59:03.904 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-tmff.ko 2024-05-11T11:59:03.904 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-topseed.ko 2024-05-11T11:59:03.904 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-twinhan.ko 2024-05-11T11:59:03.904 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-uclogic.ko 2024-05-11T11:59:03.904 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-waltop.ko 2024-05-11T11:59:03.904 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-wiimote.ko 2024-05-11T11:59:03.904 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-xinmo.ko 2024-05-11T11:59:03.904 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-zpff.ko 2024-05-11T11:59:03.904 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-zydacron.ko 2024-05-11T11:59:03.904 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/uhid.ko 2024-05-11T11:59:03.904 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/wacom.ko 2024-05-11T11:59:03.904 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hv 2024-05-11T11:59:03.904 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hv/hv_balloon.ko 2024-05-11T11:59:03.904 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hv/hv_utils.ko 2024-05-11T11:59:03.905 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hv/hv_vmbus.ko 2024-05-11T11:59:03.905 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon 2024-05-11T11:59:03.905 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/abituguru.ko 2024-05-11T11:59:03.905 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/abituguru3.ko 2024-05-11T11:59:03.905 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-11T11:59:03.905 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ad7414.ko 2024-05-11T11:59:03.905 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ad7418.ko 2024-05-11T11:59:03.905 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adc128d818.ko 2024-05-11T11:59:03.905 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm1025.ko 2024-05-11T11:59:03.905 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm1026.ko 2024-05-11T11:59:03.905 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm1029.ko 2024-05-11T11:59:03.905 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm1031.ko 2024-05-11T11:59:03.905 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm9240.ko 2024-05-11T11:59:03.906 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ads7828.ko 2024-05-11T11:59:03.906 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7410.ko 2024-05-11T11:59:03.906 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7411.ko 2024-05-11T11:59:03.906 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7462.ko 2024-05-11T11:59:03.906 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7470.ko 2024-05-11T11:59:03.906 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7475.ko 2024-05-11T11:59:03.906 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7x10.ko 2024-05-11T11:59:03.906 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/amc6821.ko 2024-05-11T11:59:03.906 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/applesmc.ko 2024-05-11T11:59:03.906 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/asb100.ko 2024-05-11T11:59:03.906 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/asc7621.ko 2024-05-11T11:59:03.906 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-11T11:59:03.906 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/atxp1.ko 2024-05-11T11:59:03.907 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/coretemp.ko 2024-05-11T11:59:03.907 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-11T11:59:03.907 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/dme1737.ko 2024-05-11T11:59:03.907 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ds1621.ko 2024-05-11T11:59:03.907 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ds620.ko 2024-05-11T11:59:03.907 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/emc1403.ko 2024-05-11T11:59:03.907 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/emc6w201.ko 2024-05-11T11:59:03.907 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/f71805f.ko 2024-05-11T11:59:03.907 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/f71882fg.ko 2024-05-11T11:59:03.907 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/f75375s.ko 2024-05-11T11:59:03.907 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/fam15h_power.ko 2024-05-11T11:59:03.907 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/fschmd.ko 2024-05-11T11:59:03.907 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/g760a.ko 2024-05-11T11:59:03.908 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/g762.ko 2024-05-11T11:59:03.908 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/gl518sm.ko 2024-05-11T11:59:03.908 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/gl520sm.ko 2024-05-11T11:59:03.908 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-11T11:59:03.908 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/i5500_temp.ko 2024-05-11T11:59:03.908 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/i5k_amb.ko 2024-05-11T11:59:03.908 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ibmaem.ko 2024-05-11T11:59:03.908 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ibmpex.ko 2024-05-11T11:59:03.908 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ina209.ko 2024-05-11T11:59:03.908 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ina2xx.ko 2024-05-11T11:59:03.908 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/it87.ko 2024-05-11T11:59:03.908 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/k10temp.ko 2024-05-11T11:59:03.908 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/k8temp.ko 2024-05-11T11:59:03.908 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lineage-pem.ko 2024-05-11T11:59:03.909 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm63.ko 2024-05-11T11:59:03.909 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm73.ko 2024-05-11T11:59:03.909 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm75.ko 2024-05-11T11:59:03.909 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm77.ko 2024-05-11T11:59:03.909 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm78.ko 2024-05-11T11:59:03.909 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm80.ko 2024-05-11T11:59:03.909 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm83.ko 2024-05-11T11:59:03.909 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm85.ko 2024-05-11T11:59:03.909 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm87.ko 2024-05-11T11:59:03.909 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm90.ko 2024-05-11T11:59:03.909 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm92.ko 2024-05-11T11:59:03.909 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm93.ko 2024-05-11T11:59:03.909 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm95234.ko 2024-05-11T11:59:03.909 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm95241.ko 2024-05-11T11:59:03.910 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm95245.ko 2024-05-11T11:59:03.910 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc2945.ko 2024-05-11T11:59:03.910 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4151.ko 2024-05-11T11:59:03.910 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4215.ko 2024-05-11T11:59:03.910 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4222.ko 2024-05-11T11:59:03.910 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4245.ko 2024-05-11T11:59:03.910 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4260.ko 2024-05-11T11:59:03.910 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4261.ko 2024-05-11T11:59:03.910 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max16065.ko 2024-05-11T11:59:03.910 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max1619.ko 2024-05-11T11:59:03.910 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max1668.ko 2024-05-11T11:59:03.910 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max197.ko 2024-05-11T11:59:03.910 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max6639.ko 2024-05-11T11:59:03.910 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max6650.ko 2024-05-11T11:59:03.911 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max6697.ko 2024-05-11T11:59:03.911 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/mcp3021.ko 2024-05-11T11:59:03.911 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/nct6683.ko 2024-05-11T11:59:03.911 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/nct6775-core.ko 2024-05-11T11:59:03.911 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/nct6775.ko 2024-05-11T11:59:03.911 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/nct7802.ko 2024-05-11T11:59:03.911 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pc87360.ko 2024-05-11T11:59:03.911 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pc87427.ko 2024-05-11T11:59:03.911 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pcf8591.ko 2024-05-11T11:59:03.911 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus 2024-05-11T11:59:03.911 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-11T11:59:03.911 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-11T11:59:03.911 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-11T11:59:03.911 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-11T11:59:03.912 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-11T11:59:03.912 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-11T11:59:03.912 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-11T11:59:03.912 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-11T11:59:03.912 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-11T11:59:03.912 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-11T11:59:03.912 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-11T11:59:03.912 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-11T11:59:03.912 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/powr1220.ko 2024-05-11T11:59:03.912 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sch5627.ko 2024-05-11T11:59:03.912 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sch5636.ko 2024-05-11T11:59:03.912 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-11T11:59:03.912 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sht15.ko 2024-05-11T11:59:03.912 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sht21.ko 2024-05-11T11:59:03.913 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/shtc1.ko 2024-05-11T11:59:03.913 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sis5595.ko 2024-05-11T11:59:03.913 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/smsc47b397.ko 2024-05-11T11:59:03.913 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/smsc47m1.ko 2024-05-11T11:59:03.913 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/smsc47m192.ko 2024-05-11T11:59:03.913 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/thmc50.ko 2024-05-11T11:59:03.913 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/tmp102.ko 2024-05-11T11:59:03.913 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/tmp103.ko 2024-05-11T11:59:03.913 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/tmp401.ko 2024-05-11T11:59:03.913 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/tmp421.ko 2024-05-11T11:59:03.913 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/via-cputemp.ko 2024-05-11T11:59:03.913 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/via686a.ko 2024-05-11T11:59:03.913 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/vt1211.ko 2024-05-11T11:59:03.914 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/vt8231.ko 2024-05-11T11:59:03.914 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83627ehf.ko 2024-05-11T11:59:03.914 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83627hf.ko 2024-05-11T11:59:03.914 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83781d.ko 2024-05-11T11:59:03.914 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83791d.ko 2024-05-11T11:59:03.914 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83792d.ko 2024-05-11T11:59:03.914 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83793.ko 2024-05-11T11:59:03.914 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83795.ko 2024-05-11T11:59:03.914 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83l785ts.ko 2024-05-11T11:59:03.914 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83l786ng.ko 2024-05-11T11:59:03.914 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c 2024-05-11T11:59:03.914 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/algos 2024-05-11T11:59:03.914 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-11T11:59:03.915 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-11T11:59:03.915 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses 2024-05-11T11:59:03.915 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-11T11:59:03.915 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-11T11:59:03.915 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-11T11:59:03.915 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-11T11:59:03.915 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-11T11:59:03.915 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-11T11:59:03.915 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-11T11:59:03.915 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-11T11:59:03.915 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-11T11:59:03.915 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-11T11:59:03.915 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-11T11:59:03.916 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-11T11:59:03.916 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-11T11:59:03.916 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-11T11:59:03.916 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-11T11:59:03.916 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-11T11:59:03.916 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-11T11:59:03.916 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-11T11:59:03.916 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-11T11:59:03.916 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-11T11:59:03.916 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-11T11:59:03.916 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-11T11:59:03.916 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-11T11:59:03.916 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-dev.ko 2024-05-11T11:59:03.916 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-mux.ko 2024-05-11T11:59:03.916 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-05-11T11:59:03.917 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-smbus.ko 2024-05-11T11:59:03.917 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-stub.ko 2024-05-11T11:59:03.917 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband 2024-05-11T11:59:03.917 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core 2024-05-11T11:59:03.917 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-11T11:59:03.917 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/ib_core.ko 2024-05-11T11:59:03.917 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-11T11:59:03.917 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-11T11:59:03.917 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-11T11:59:03.917 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-11T11:59:03.917 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-11T11:59:03.917 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw 2024-05-11T11:59:03.917 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/cxgb4 2024-05-11T11:59:03.917 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-11T11:59:03.918 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mlx4 2024-05-11T11:59:03.918 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-11T11:59:03.918 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mlx5 2024-05-11T11:59:03.918 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-11T11:59:03.918 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mthca 2024-05-11T11:59:03.918 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-11T11:59:03.918 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp 2024-05-11T11:59:03.918 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/ipoib 2024-05-11T11:59:03.918 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-11T11:59:03.918 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/iser 2024-05-11T11:59:03.918 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-11T11:59:03.918 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/isert 2024-05-11T11:59:03.918 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-11T11:59:03.919 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/srp 2024-05-11T11:59:03.919 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-11T11:59:03.919 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/srpt 2024-05-11T11:59:03.919 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-11T11:59:03.919 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input 2024-05-11T11:59:03.919 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/gameport 2024-05-11T11:59:03.919 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-11T11:59:03.919 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-11T11:59:03.919 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/gameport/gameport.ko 2024-05-11T11:59:03.919 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/joydev.ko 2024-05-11T11:59:03.919 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard 2024-05-11T11:59:03.919 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-11T11:59:03.919 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse 2024-05-11T11:59:03.919 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/appletouch.ko 2024-05-11T11:59:03.920 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/bcm5974.ko 2024-05-11T11:59:03.920 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/cyapatp.ko 2024-05-11T11:59:03.920 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-11T11:59:03.920 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/sermouse.ko 2024-05-11T11:59:03.920 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-11T11:59:03.920 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-11T11:59:03.920 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-11T11:59:03.920 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/rmi4 2024-05-11T11:59:03.920 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-11T11:59:03.920 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio 2024-05-11T11:59:03.920 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/altera_ps2.ko 2024-05-11T11:59:03.920 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/arc_ps2.ko 2024-05-11T11:59:03.920 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-11T11:59:03.920 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/serio_raw.ko 2024-05-11T11:59:03.921 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/sparse-keymap.ko 2024-05-11T11:59:03.921 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds 2024-05-11T11:59:03.921 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-blinkm.ko 2024-05-11T11:59:03.921 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-lm3530.ko 2024-05-11T11:59:03.921 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-lp3944.ko 2024-05-11T11:59:03.921 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-ss4200.ko 2024-05-11T11:59:03.921 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger 2024-05-11T11:59:03.921 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-11T11:59:03.921 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-11T11:59:03.921 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-11T11:59:03.921 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-11T11:59:03.921 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-11T11:59:03.921 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-11T11:59:03.922 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-11T11:59:03.922 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-11T11:59:03.922 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-11T11:59:03.922 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md 2024-05-11T11:59:03.922 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/bcache 2024-05-11T11:59:03.922 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/bcache/bcache.ko 2024-05-11T11:59:03.922 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-bio-prison.ko 2024-05-11T11:59:03.922 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-cache-smq.ko 2024-05-11T11:59:03.922 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-cache.ko 2024-05-11T11:59:03.922 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-crypt.ko 2024-05-11T11:59:03.922 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-delay.ko 2024-05-11T11:59:03.922 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-flakey.ko 2024-05-11T11:59:03.922 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-log-userspace.ko 2024-05-11T11:59:03.923 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-multipath.ko 2024-05-11T11:59:03.923 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-queue-length.ko 2024-05-11T11:59:03.923 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-raid.ko 2024-05-11T11:59:03.923 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-round-robin.ko 2024-05-11T11:59:03.923 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-service-time.ko 2024-05-11T11:59:03.923 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-switch.ko 2024-05-11T11:59:03.923 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-thin-pool.ko 2024-05-11T11:59:03.923 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-verity.ko 2024-05-11T11:59:03.923 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/persistent-data 2024-05-11T11:59:03.923 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-11T11:59:03.923 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/raid0.ko 2024-05-11T11:59:03.923 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/raid1.ko 2024-05-11T11:59:03.923 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/raid10.ko 2024-05-11T11:59:03.924 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/raid456.ko 2024-05-11T11:59:03.924 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message 2024-05-11T11:59:03.924 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion 2024-05-11T11:59:03.924 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptbase.ko 2024-05-11T11:59:03.924 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptctl.ko 2024-05-11T11:59:03.924 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptfc.ko 2024-05-11T11:59:03.924 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptlan.ko 2024-05-11T11:59:03.924 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptsas.ko 2024-05-11T11:59:03.924 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptscsih.ko 2024-05-11T11:59:03.924 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptspi.ko 2024-05-11T11:59:03.924 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd 2024-05-11T11:59:03.924 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/lpc_ich.ko 2024-05-11T11:59:03.924 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/lpc_sch.ko 2024-05-11T11:59:03.924 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/mfd-core.ko 2024-05-11T11:59:03.925 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/sm501.ko 2024-05-11T11:59:03.925 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/viperboard.ko 2024-05-11T11:59:03.925 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/vx855.ko 2024-05-11T11:59:03.925 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/wl1273-core.ko 2024-05-11T11:59:03.925 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc 2024-05-11T11:59:03.925 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/altera-stapl 2024-05-11T11:59:03.925 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-11T11:59:03.925 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/apds9802als.ko 2024-05-11T11:59:03.925 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/apds990x.ko 2024-05-11T11:59:03.925 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/bh1770glc.ko 2024-05-11T11:59:03.925 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/cb710 2024-05-11T11:59:03.925 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/cb710/cb710.ko 2024-05-11T11:59:03.925 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/eeprom 2024-05-11T11:59:03.925 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/eeprom/at24.ko 2024-05-11T11:59:03.926 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-11T11:59:03.926 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/eeprom/max6875.ko 2024-05-11T11:59:03.926 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/enclosure.ko 2024-05-11T11:59:03.926 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/hpilo.ko 2024-05-11T11:59:03.926 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/isl29003.ko 2024-05-11T11:59:03.926 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/isl29020.ko 2024-05-11T11:59:03.926 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/lis3lv02d 2024-05-11T11:59:03.926 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-11T11:59:03.926 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-11T11:59:03.926 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/mei 2024-05-11T11:59:03.926 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/mei/mei-me.ko 2024-05-11T11:59:03.926 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/mei/mei-txe.ko 2024-05-11T11:59:03.926 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/mei/mei.ko 2024-05-11T11:59:03.926 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-gru 2024-05-11T11:59:03.926 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-gru/gru.ko 2024-05-11T11:59:03.927 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-xp 2024-05-11T11:59:03.927 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-xp/xp.ko 2024-05-11T11:59:03.927 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-xp/xpc.ko 2024-05-11T11:59:03.927 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-05-11T11:59:03.927 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/tifm_7xx1.ko 2024-05-11T11:59:03.927 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/tifm_core.ko 2024-05-11T11:59:03.927 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/tsl2550.ko 2024-05-11T11:59:03.927 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/vmw_balloon.ko 2024-05-11T11:59:03.927 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/vmw_vmci 2024-05-11T11:59:03.927 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-11T11:59:03.927 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net 2024-05-11T11:59:03.927 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/bonding 2024-05-11T11:59:03.927 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/bonding/bonding.ko 2024-05-11T11:59:03.927 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa 2024-05-11T11:59:03.927 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/b53 2024-05-11T11:59:03.928 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-11T11:59:03.928 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-11T11:59:03.928 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-11T11:59:03.928 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/mv88e6xxx 2024-05-11T11:59:03.928 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-11T11:59:03.928 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dummy.ko 2024-05-11T11:59:03.928 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/eql.ko 2024-05-11T11:59:03.928 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet 2024-05-11T11:59:03.928 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com 2024-05-11T11:59:03.928 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-11T11:59:03.928 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-11T11:59:03.928 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-11T11:59:03.928 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-11T11:59:03.928 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390 2024-05-11T11:59:03.928 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-11T11:59:03.929 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-11T11:59:03.929 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-11T11:59:03.929 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-11T11:59:03.929 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/adaptec 2024-05-11T11:59:03.929 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-11T11:59:03.929 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/agere 2024-05-11T11:59:03.929 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-11T11:59:03.929 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/alteon 2024-05-11T11:59:03.929 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-11T11:59:03.929 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/altera 2024-05-11T11:59:03.929 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-11T11:59:03.929 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/amd 2024-05-11T11:59:03.929 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-11T11:59:03.929 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-11T11:59:03.929 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-11T11:59:03.930 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros 2024-05-11T11:59:03.930 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/alx 2024-05-11T11:59:03.930 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-11T11:59:03.930 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atl1c 2024-05-11T11:59:03.930 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-11T11:59:03.930 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atl1e 2024-05-11T11:59:03.930 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-11T11:59:03.930 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atlx 2024-05-11T11:59:03.930 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-11T11:59:03.930 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-11T11:59:03.930 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom 2024-05-11T11:59:03.930 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-11T11:59:03.930 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-11T11:59:03.931 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-05-11T11:59:03.931 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-11T11:59:03.931 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-11T11:59:03.931 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/genet 2024-05-11T11:59:03.931 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-11T11:59:03.931 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-11T11:59:03.931 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/brocade 2024-05-11T11:59:03.931 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/brocade/bna 2024-05-11T11:59:03.931 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-11T11:59:03.931 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio 2024-05-11T11:59:03.931 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb 2024-05-11T11:59:03.931 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-11T11:59:03.931 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-05-11T11:59:03.931 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-11T11:59:03.932 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-05-11T11:59:03.932 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-11T11:59:03.932 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-05-11T11:59:03.932 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-11T11:59:03.932 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-05-11T11:59:03.932 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-11T11:59:03.932 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/cisco 2024-05-11T11:59:03.932 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/cisco/enic 2024-05-11T11:59:03.932 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-11T11:59:03.932 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec 2024-05-11T11:59:03.932 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip 2024-05-11T11:59:03.932 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-11T11:59:03.932 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-11T11:59:03.933 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-11T11:59:03.933 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-11T11:59:03.933 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-11T11:59:03.933 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-11T11:59:03.933 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dlink 2024-05-11T11:59:03.933 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-11T11:59:03.933 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-11T11:59:03.933 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dnet.ko 2024-05-11T11:59:03.933 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/emulex 2024-05-11T11:59:03.933 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/emulex/benet 2024-05-11T11:59:03.933 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-11T11:59:03.933 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/ethoc.ko 2024-05-11T11:59:03.933 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/fealnx.ko 2024-05-11T11:59:03.933 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel 2024-05-11T11:59:03.934 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-11T11:59:03.934 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e1000 2024-05-11T11:59:03.934 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-11T11:59:03.934 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e1000e 2024-05-11T11:59:03.934 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-11T11:59:03.934 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/fm10k 2024-05-11T11:59:03.934 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-11T11:59:03.934 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/i40e 2024-05-11T11:59:03.934 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-11T11:59:03.934 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/iavf 2024-05-11T11:59:03.934 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-11T11:59:03.934 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/igb 2024-05-11T11:59:03.934 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-11T11:59:03.934 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/igbvf 2024-05-11T11:59:03.934 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-11T11:59:03.935 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/ixgbe 2024-05-11T11:59:03.935 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-11T11:59:03.935 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/ixgbevf 2024-05-11T11:59:03.935 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-11T11:59:03.935 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/jme.ko 2024-05-11T11:59:03.935 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/marvell 2024-05-11T11:59:03.935 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-11T11:59:03.935 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-11T11:59:03.935 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-11T11:59:03.935 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox 2024-05-11T11:59:03.935 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx4 2024-05-11T11:59:03.935 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-11T11:59:03.935 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-11T11:59:03.936 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx5 2024-05-11T11:59:03.936 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-05-11T11:59:03.936 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-11T11:59:03.936 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/micrel 2024-05-11T11:59:03.936 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-11T11:59:03.936 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/myricom 2024-05-11T11:59:03.936 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/myricom/myri10ge 2024-05-11T11:59:03.936 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-11T11:59:03.936 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/natsemi 2024-05-11T11:59:03.936 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-11T11:59:03.936 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-11T11:59:03.936 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/neterion 2024-05-11T11:59:03.936 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-11T11:59:03.936 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/nvidia 2024-05-11T11:59:03.936 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-11T11:59:03.937 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/packetengines 2024-05-11T11:59:03.937 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-11T11:59:03.937 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-11T11:59:03.937 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic 2024-05-11T11:59:03.937 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/netxen 2024-05-11T11:59:03.937 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-11T11:59:03.937 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-11T11:59:03.937 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-05-11T11:59:03.937 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-11T11:59:03.937 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/rdc 2024-05-11T11:59:03.937 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-11T11:59:03.937 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek 2024-05-11T11:59:03.937 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-11T11:59:03.937 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-11T11:59:03.937 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-11T11:59:03.938 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-11T11:59:03.938 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/rocker 2024-05-11T11:59:03.938 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-05-11T11:59:03.938 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sfc 2024-05-11T11:59:03.938 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-11T11:59:03.938 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/silan 2024-05-11T11:59:03.938 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-11T11:59:03.938 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sis 2024-05-11T11:59:03.938 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-11T11:59:03.938 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-11T11:59:03.938 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc 2024-05-11T11:59:03.938 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-11T11:59:03.938 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-11T11:59:03.939 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-11T11:59:03.939 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-11T11:59:03.939 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/stmicro 2024-05-11T11:59:03.939 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/stmicro/stmmac 2024-05-11T11:59:03.939 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-11T11:59:03.939 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-11T11:59:03.939 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun 2024-05-11T11:59:03.939 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-11T11:59:03.939 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-11T11:59:03.939 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-11T11:59:03.939 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-11T11:59:03.939 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/tehuti 2024-05-11T11:59:03.939 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-11T11:59:03.939 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/ti 2024-05-11T11:59:03.939 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-11T11:59:03.940 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/via 2024-05-11T11:59:03.940 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-11T11:59:03.940 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-11T11:59:03.940 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/wiznet 2024-05-11T11:59:03.940 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-11T11:59:03.940 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-11T11:59:03.940 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/xircom 2024-05-11T11:59:03.940 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-11T11:59:03.940 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/geneve.ko 2024-05-11T11:59:03.940 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/hyperv 2024-05-11T11:59:03.940 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-11T11:59:03.940 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ieee802154 2024-05-11T11:59:03.940 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-11T11:59:03.940 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ifb.ko 2024-05-11T11:59:03.941 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ipvlan 2024-05-11T11:59:03.941 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-11T11:59:03.941 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/macvlan.ko 2024-05-11T11:59:03.941 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/macvtap.ko 2024-05-11T11:59:03.941 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio 2024-05-11T11:59:03.941 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio.ko 2024-05-11T11:59:03.941 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-11T11:59:03.941 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-11T11:59:03.941 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-11T11:59:03.941 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mii.ko 2024-05-11T11:59:03.941 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/net_failover.ko 2024-05-11T11:59:03.941 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/netconsole.ko 2024-05-11T11:59:03.941 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/nlmon.ko 2024-05-11T11:59:03.941 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/pcs 2024-05-11T11:59:03.942 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-11T11:59:03.942 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-11T11:59:03.942 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy 2024-05-11T11:59:03.942 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/amd.ko 2024-05-11T11:59:03.942 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-11T11:59:03.942 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-05-11T11:59:03.942 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-11T11:59:03.942 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/bcm87xx.ko 2024-05-11T11:59:03.942 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/broadcom.ko 2024-05-11T11:59:03.942 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/cicada.ko 2024-05-11T11:59:03.942 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/davicom.ko 2024-05-11T11:59:03.942 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/dp83640.ko 2024-05-11T11:59:03.942 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/et1011c.ko 2024-05-11T11:59:03.942 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/icplus.ko 2024-05-11T11:59:03.943 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/lxt.ko 2024-05-11T11:59:03.943 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/marvell.ko 2024-05-11T11:59:03.943 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/micrel.ko 2024-05-11T11:59:03.943 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/national.ko 2024-05-11T11:59:03.943 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/phylink.ko 2024-05-11T11:59:03.943 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/qsemi.ko 2024-05-11T11:59:03.945 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/realtek.ko 2024-05-11T11:59:03.945 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/smsc.ko 2024-05-11T11:59:03.945 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/ste10Xp.ko 2024-05-11T11:59:03.946 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/vitesse.ko 2024-05-11T11:59:03.946 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp 2024-05-11T11:59:03.946 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-11T11:59:03.946 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_async.ko 2024-05-11T11:59:03.946 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-11T11:59:03.946 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_generic.ko 2024-05-11T11:59:03.946 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-11T11:59:03.946 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-11T11:59:03.946 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/pppoe.ko 2024-05-11T11:59:03.946 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/pppox.ko 2024-05-11T11:59:03.946 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/pptp.ko 2024-05-11T11:59:03.946 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/slip 2024-05-11T11:59:03.946 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/slip/slhc.ko 2024-05-11T11:59:03.946 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/slip/slip.ko 2024-05-11T11:59:03.947 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/sungem_phy.ko 2024-05-11T11:59:03.947 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/tap.ko 2024-05-11T11:59:03.947 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team 2024-05-11T11:59:03.947 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team.ko 2024-05-11T11:59:03.947 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-11T11:59:03.947 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-11T11:59:03.947 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-11T11:59:03.947 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_random.ko 2024-05-11T11:59:03.947 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-11T11:59:03.947 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/tun.ko 2024-05-11T11:59:03.947 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/veth.ko 2024-05-11T11:59:03.947 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/virtio_net.ko 2024-05-11T11:59:03.947 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/vmxnet3 2024-05-11T11:59:03.947 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-11T11:59:03.948 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/vxlan 2024-05-11T11:59:03.948 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/vxlan/vxlan.ko 2024-05-11T11:59:03.948 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/xen-netback 2024-05-11T11:59:03.948 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-11T11:59:03.948 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/xen-netfront.ko 2024-05-11T11:59:03.948 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ntb 2024-05-11T11:59:03.948 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ntb/ntb.ko 2024-05-11T11:59:03.948 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/nvme 2024-05-11T11:59:03.948 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/nvme/host 2024-05-11T11:59:03.948 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/nvme/host/nvme-core.ko 2024-05-11T11:59:03.948 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/nvme/host/nvme.ko 2024-05-11T11:59:03.948 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport 2024-05-11T11:59:03.948 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport/parport.ko 2024-05-11T11:59:03.948 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport/parport_cs.ko 2024-05-11T11:59:03.949 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport/parport_pc.ko 2024-05-11T11:59:03.949 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport/parport_serial.ko 2024-05-11T11:59:03.949 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci 2024-05-11T11:59:03.949 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/hotplug 2024-05-11T11:59:03.949 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-11T11:59:03.949 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/pcie 2024-05-11T11:59:03.949 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/pcie/aer_inject.ko 2024-05-11T11:59:03.949 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/xen-pcifront.ko 2024-05-11T11:59:03.949 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia 2024-05-11T11:59:03.949 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia/i82092.ko 2024-05-11T11:59:03.949 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia/pd6729.ko 2024-05-11T11:59:03.949 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-11T11:59:03.949 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pinctrl 2024-05-11T11:59:03.950 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pinctrl/intel 2024-05-11T11:59:03.950 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-11T11:59:03.950 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform 2024-05-11T11:59:03.950 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86 2024-05-11T11:59:03.950 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-11T11:59:03.950 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/acerhdf.ko 2024-05-11T11:59:03.950 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-11T11:59:03.950 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-11T11:59:03.950 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-11T11:59:03.950 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-11T11:59:03.950 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-11T11:59:03.950 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-11T11:59:03.950 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-11T11:59:03.950 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-11T11:59:03.950 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-11T11:59:03.951 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/hdaps.ko 2024-05-11T11:59:03.951 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/intel 2024-05-11T11:59:03.951 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-11T11:59:03.951 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/intel_ips.ko 2024-05-11T11:59:03.951 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-11T11:59:03.951 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-11T11:59:03.951 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-11T11:59:03.951 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-11T11:59:03.951 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-11T11:59:03.951 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-11T11:59:03.951 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-11T11:59:03.951 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-11T11:59:03.951 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-11T11:59:03.951 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/wmi.ko 2024-05-11T11:59:03.951 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/powercap 2024-05-11T11:59:03.952 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-11T11:59:03.952 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-11T11:59:03.952 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps 2024-05-11T11:59:03.952 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/clients 2024-05-11T11:59:03.952 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-11T11:59:03.952 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-11T11:59:03.952 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/clients/pps_parport.ko 2024-05-11T11:59:03.952 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/pps_core.ko 2024-05-11T11:59:03.952 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ptp 2024-05-11T11:59:03.952 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ptp/ptp.ko 2024-05-11T11:59:03.952 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ptp/ptp_kvm.ko 2024-05-11T11:59:03.952 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras 2024-05-11T11:59:03.952 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras/amd 2024-05-11T11:59:03.952 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras/amd/atl 2024-05-11T11:59:03.952 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-11T11:59:03.953 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras/amd/fmpm.ko 2024-05-11T11:59:03.953 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc 2024-05-11T11:59:03.953 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-11T11:59:03.953 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-11T11:59:03.953 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-11T11:59:03.953 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-11T11:59:03.953 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-11T11:59:03.953 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-11T11:59:03.953 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-11T11:59:03.953 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-11T11:59:03.953 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-11T11:59:03.953 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-11T11:59:03.953 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-em3027.ko 2024-05-11T11:59:03.953 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-11T11:59:03.954 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-11T11:59:03.954 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-11T11:59:03.954 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-11T11:59:03.954 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-11T11:59:03.954 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-11T11:59:03.954 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-max6900.ko 2024-05-11T11:59:03.954 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-11T11:59:03.954 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-11T11:59:03.954 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-11T11:59:03.954 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-11T11:59:03.954 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-11T11:59:03.954 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-11T11:59:03.954 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-11T11:59:03.955 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-11T11:59:03.955 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-11T11:59:03.955 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-11T11:59:03.955 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-11T11:59:03.955 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-11T11:59:03.955 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-x1205.ko 2024-05-11T11:59:03.955 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi 2024-05-11T11:59:03.955 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/3w-9xxx.ko 2024-05-11T11:59:03.955 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/3w-sas.ko 2024-05-11T11:59:03.955 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/3w-xxxx.ko 2024-05-11T11:59:03.955 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/BusLogic.ko 2024-05-11T11:59:03.955 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/a100u2w.ko 2024-05-11T11:59:03.955 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aacraid 2024-05-11T11:59:03.955 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-11T11:59:03.955 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/advansys.ko 2024-05-11T11:59:03.956 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aic7xxx 2024-05-11T11:59:03.956 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-11T11:59:03.956 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-11T11:59:03.956 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/am53c974.ko 2024-05-11T11:59:03.956 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/arcmsr 2024-05-11T11:59:03.956 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-11T11:59:03.956 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/atp870u.ko 2024-05-11T11:59:03.956 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/be2iscsi 2024-05-11T11:59:03.956 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-11T11:59:03.956 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bfa 2024-05-11T11:59:03.956 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bfa/bfa.ko 2024-05-11T11:59:03.956 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bnx2fc 2024-05-11T11:59:03.956 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-11T11:59:03.956 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bnx2i 2024-05-11T11:59:03.957 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-11T11:59:03.957 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/ch.ko 2024-05-11T11:59:03.957 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/csiostor 2024-05-11T11:59:03.957 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-11T11:59:03.957 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi 2024-05-11T11:59:03.957 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/cxgb3i 2024-05-11T11:59:03.957 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-11T11:59:03.957 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/cxgb4i 2024-05-11T11:59:03.957 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-11T11:59:03.957 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-11T11:59:03.957 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/dc395x.ko 2024-05-11T11:59:03.957 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/device_handler 2024-05-11T11:59:03.957 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-05-11T11:59:03.957 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-05-11T11:59:03.957 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-05-11T11:59:03.958 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-05-11T11:59:03.958 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/dmx3191d.ko 2024-05-11T11:59:03.958 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/esas2r 2024-05-11T11:59:03.958 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-11T11:59:03.958 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/esp_scsi.ko 2024-05-11T11:59:03.958 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fcoe 2024-05-11T11:59:03.958 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-11T11:59:03.958 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-11T11:59:03.958 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fnic 2024-05-11T11:59:03.958 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fnic/fnic.ko 2024-05-11T11:59:03.958 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/hpsa.ko 2024-05-11T11:59:03.958 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/hptiop.ko 2024-05-11T11:59:03.958 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/hv_storvsc.ko 2024-05-11T11:59:03.959 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/initio.ko 2024-05-11T11:59:03.959 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/ipr.ko 2024-05-11T11:59:03.959 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/ips.ko 2024-05-11T11:59:03.959 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/isci 2024-05-11T11:59:03.959 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/isci/isci.ko 2024-05-11T11:59:03.959 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-11T11:59:03.959 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-11T11:59:03.959 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libfc 2024-05-11T11:59:03.959 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libfc/libfc.ko 2024-05-11T11:59:03.959 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libiscsi.ko 2024-05-11T11:59:03.959 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-11T11:59:03.959 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libsas 2024-05-11T11:59:03.959 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libsas/libsas.ko 2024-05-11T11:59:03.959 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/lpfc 2024-05-11T11:59:03.960 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-11T11:59:03.960 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid 2024-05-11T11:59:03.960 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid.ko 2024-05-11T11:59:03.960 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-11T11:59:03.960 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-11T11:59:03.960 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-11T11:59:03.960 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mpt3sas 2024-05-11T11:59:03.960 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-11T11:59:03.960 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mvsas 2024-05-11T11:59:03.960 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-11T11:59:03.960 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mvumi.ko 2024-05-11T11:59:03.960 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/pm8001 2024-05-11T11:59:03.960 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-11T11:59:03.960 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/pmcraid.ko 2024-05-11T11:59:03.960 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla1280.ko 2024-05-11T11:59:03.961 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla2xxx 2024-05-11T11:59:03.961 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-11T11:59:03.961 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-11T11:59:03.961 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla4xxx 2024-05-11T11:59:03.961 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-11T11:59:03.961 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/raid_class.ko 2024-05-11T11:59:03.961 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_debug.ko 2024-05-11T11:59:03.961 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-11T11:59:03.961 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-11T11:59:03.961 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-11T11:59:03.961 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-11T11:59:03.961 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-11T11:59:03.961 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/ses.ko 2024-05-11T11:59:03.961 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/st.ko 2024-05-11T11:59:03.961 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/stex.ko 2024-05-11T11:59:03.962 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/sym53c8xx_2 2024-05-11T11:59:03.962 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-11T11:59:03.962 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/virtio_scsi.ko 2024-05-11T11:59:03.962 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-11T11:59:03.962 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/wd719x.ko 2024-05-11T11:59:03.962 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ssb 2024-05-11T11:59:03.962 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ssb/ssb.ko 2024-05-11T11:59:03.962 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target 2024-05-11T11:59:03.962 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/iscsi 2024-05-11T11:59:03.962 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-11T11:59:03.962 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/loopback 2024-05-11T11:59:03.962 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-11T11:59:03.962 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_file.ko 2024-05-11T11:59:03.962 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_iblock.ko 2024-05-11T11:59:03.963 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_mod.ko 2024-05-11T11:59:03.963 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_pscsi.ko 2024-05-11T11:59:03.963 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_user.ko 2024-05-11T11:59:03.963 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/tcm_fc 2024-05-11T11:59:03.963 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-11T11:59:03.963 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal 2024-05-11T11:59:03.963 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel 2024-05-11T11:59:03.963 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal 2024-05-11T11:59:03.963 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-11T11:59:03.963 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-11T11:59:03.963 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-11T11:59:03.963 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-11T11:59:03.963 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-11T11:59:03.963 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-11T11:59:03.964 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-11T11:59:03.964 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-11T11:59:03.964 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-11T11:59:03.964 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-11T11:59:03.964 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-11T11:59:03.964 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-11T11:59:03.964 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-11T11:59:03.964 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-11T11:59:03.964 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-11T11:59:03.964 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-11T11:59:03.964 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-11T11:59:03.964 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-11T11:59:03.964 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty 2024-05-11T11:59:03.964 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/ipwireless 2024-05-11T11:59:03.964 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-11T11:59:03.965 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/n_gsm.ko 2024-05-11T11:59:03.965 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/n_hdlc.ko 2024-05-11T11:59:03.965 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/nozomi.ko 2024-05-11T11:59:03.965 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial 2024-05-11T11:59:03.965 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/8250 2024-05-11T11:59:03.965 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-11T11:59:03.965 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/arc_uart.ko 2024-05-11T11:59:03.965 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/jsm 2024-05-11T11:59:03.965 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-11T11:59:03.965 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs 2024-05-11T11:59:03.965 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs/core 2024-05-11T11:59:03.965 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-11T11:59:03.965 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs/host 2024-05-11T11:59:03.965 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-11T11:59:03.966 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio 2024-05-11T11:59:03.966 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio.ko 2024-05-11T11:59:03.966 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_aec.ko 2024-05-11T11:59:03.966 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_cif.ko 2024-05-11T11:59:03.966 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_pci_generic.ko 2024-05-11T11:59:03.966 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_sercos3.ko 2024-05-11T11:59:03.966 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb 2024-05-11T11:59:03.966 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class 2024-05-11T11:59:03.966 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class/cdc-acm.ko 2024-05-11T11:59:03.966 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-11T11:59:03.966 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class/usblp.ko 2024-05-11T11:59:03.966 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class/usbtmc.ko 2024-05-11T11:59:03.966 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/host 2024-05-11T11:59:03.966 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-11T11:59:03.967 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/image 2024-05-11T11:59:03.967 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/image/mdc800.ko 2024-05-11T11:59:03.967 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/image/microtek.ko 2024-05-11T11:59:03.967 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc 2024-05-11T11:59:03.967 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/adutux.ko 2024-05-11T11:59:03.967 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/appledisplay.ko 2024-05-11T11:59:03.967 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/emi26.ko 2024-05-11T11:59:03.967 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/emi62.ko 2024-05-11T11:59:03.967 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/ezusb.ko 2024-05-11T11:59:03.967 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/idmouse.ko 2024-05-11T11:59:03.967 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/iowarrior.ko 2024-05-11T11:59:03.967 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-11T11:59:03.967 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/ldusb.ko 2024-05-11T11:59:03.967 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/legousbtower.ko 2024-05-11T11:59:03.968 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/sisusbvga 2024-05-11T11:59:03.968 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-11T11:59:03.968 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-11T11:59:03.968 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/usb3503.ko 2024-05-11T11:59:03.968 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/usblcd.ko 2024-05-11T11:59:03.968 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-11T11:59:03.968 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/uss720.ko 2024-05-11T11:59:03.968 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/yurex.ko 2024-05-11T11:59:03.968 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/phy 2024-05-11T11:59:03.968 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/phy/phy-generic.ko 2024-05-11T11:59:03.968 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial 2024-05-11T11:59:03.968 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/aircable.ko 2024-05-11T11:59:03.968 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ark3116.ko 2024-05-11T11:59:03.968 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-11T11:59:03.969 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ch341.ko 2024-05-11T11:59:03.969 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/cp210x.ko 2024-05-11T11:59:03.969 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/cyberjack.ko 2024-05-11T11:59:03.969 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-11T11:59:03.969 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-11T11:59:03.969 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/empeg.ko 2024-05-11T11:59:03.969 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-11T11:59:03.969 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-11T11:59:03.969 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-11T11:59:03.969 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/io_ti.ko 2024-05-11T11:59:03.969 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ipaq.ko 2024-05-11T11:59:03.969 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ipw.ko 2024-05-11T11:59:03.969 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ir-usb.ko 2024-05-11T11:59:03.969 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-11T11:59:03.969 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/keyspan.ko 2024-05-11T11:59:03.970 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-11T11:59:03.970 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-11T11:59:03.970 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-11T11:59:03.970 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/mct_u232.ko 2024-05-11T11:59:03.970 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/mos7720.ko 2024-05-11T11:59:03.970 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/mos7840.ko 2024-05-11T11:59:03.970 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/navman.ko 2024-05-11T11:59:03.970 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/omninet.ko 2024-05-11T11:59:03.970 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/opticon.ko 2024-05-11T11:59:03.970 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/option.ko 2024-05-11T11:59:03.970 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/oti6858.ko 2024-05-11T11:59:03.970 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/pl2303.ko 2024-05-11T11:59:03.970 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/qcaux.ko 2024-05-11T11:59:03.970 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/qcserial.ko 2024-05-11T11:59:03.970 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/quatech2.ko 2024-05-11T11:59:03.971 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/safe_serial.ko 2024-05-11T11:59:03.971 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/sierra.ko 2024-05-11T11:59:03.971 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-11T11:59:03.971 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ssu100.ko 2024-05-11T11:59:03.971 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/symbolserial.ko 2024-05-11T11:59:03.971 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-11T11:59:03.971 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-11T11:59:03.971 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/usb_debug.ko 2024-05-11T11:59:03.971 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-11T11:59:03.971 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/visor.ko 2024-05-11T11:59:03.971 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/whiteheat.ko 2024-05-11T11:59:03.971 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-11T11:59:03.971 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage 2024-05-11T11:59:03.971 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/uas.ko 2024-05-11T11:59:03.971 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-11T11:59:03.972 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-11T11:59:03.972 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-11T11:59:03.972 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-11T11:59:03.972 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-11T11:59:03.972 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-11T11:59:03.972 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-11T11:59:03.972 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-karma.ko 2024-05-11T11:59:03.972 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-11T11:59:03.972 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-11T11:59:03.972 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-11T11:59:03.972 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-11T11:59:03.972 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-11T11:59:03.972 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/usb-storage.ko 2024-05-11T11:59:03.972 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/usbip 2024-05-11T11:59:03.973 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-11T11:59:03.973 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-11T11:59:03.973 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-11T11:59:03.973 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio 2024-05-11T11:59:03.973 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/pci 2024-05-11T11:59:03.973 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-11T11:59:03.973 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-11T11:59:03.973 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/vfio.ko 2024-05-11T11:59:03.973 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-11T11:59:03.973 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost 2024-05-11T11:59:03.973 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost/vhost.ko 2024-05-11T11:59:03.973 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-11T11:59:03.973 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost/vhost_net.ko 2024-05-11T11:59:03.973 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost/vhost_scsi.ko 2024-05-11T11:59:03.974 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video 2024-05-11T11:59:03.974 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight 2024-05-11T11:59:03.974 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/apple_bl.ko 2024-05-11T11:59:03.974 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/lcd.ko 2024-05-11T11:59:03.974 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-11T11:59:03.974 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio 2024-05-11T11:59:03.974 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio.ko 2024-05-11T11:59:03.974 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_balloon.ko 2024-05-11T11:59:03.974 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_mmio.ko 2024-05-11T11:59:03.974 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_pci.ko 2024-05-11T11:59:03.974 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-05-11T11:59:03.974 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-05-11T11:59:03.974 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_ring.ko 2024-05-11T11:59:03.974 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1 2024-05-11T11:59:03.975 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/masters 2024-05-11T11:59:03.975 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/masters/ds2482.ko 2024-05-11T11:59:03.975 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/masters/ds2490.ko 2024-05-11T11:59:03.975 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves 2024-05-11T11:59:03.975 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-11T11:59:03.975 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-11T11:59:03.975 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-11T11:59:03.975 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-11T11:59:03.975 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-11T11:59:03.975 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-11T11:59:03.975 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-11T11:59:03.975 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-11T11:59:03.975 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-11T11:59:03.975 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-11T11:59:03.975 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-11T11:59:03.976 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/wire.ko 2024-05-11T11:59:03.976 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog 2024-05-11T11:59:03.976 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-11T11:59:03.976 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-11T11:59:03.976 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-11T11:59:03.976 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/hpwdt.ko 2024-05-11T11:59:03.976 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/i6300esb.ko 2024-05-11T11:59:03.976 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-11T11:59:03.976 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-11T11:59:03.976 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/ib700wdt.ko 2024-05-11T11:59:03.976 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/ibmasr.ko 2024-05-11T11:59:03.976 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-11T11:59:03.976 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-11T11:59:03.976 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/it87_wdt.ko 2024-05-11T11:59:03.976 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/machzwd.ko 2024-05-11T11:59:03.977 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/nv_tco.ko 2024-05-11T11:59:03.977 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-11T11:59:03.977 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-11T11:59:03.977 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-11T11:59:03.977 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-11T11:59:03.977 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/softdog.ko 2024-05-11T11:59:03.977 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-11T11:59:03.977 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/via_wdt.ko 2024-05-11T11:59:03.977 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-11T11:59:03.977 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-11T11:59:03.977 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-11T11:59:03.977 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/wdt_pci.ko 2024-05-11T11:59:03.977 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/xen_wdt.ko 2024-05-11T11:59:03.977 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen 2024-05-11T11:59:03.977 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-acpi-processor.ko 2024-05-11T11:59:03.978 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-evtchn.ko 2024-05-11T11:59:03.978 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-gntalloc.ko 2024-05-11T11:59:03.978 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-gntdev.ko 2024-05-11T11:59:03.978 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-pciback 2024-05-11T11:59:03.978 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-11T11:59:03.978 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-privcmd.ko 2024-05-11T11:59:03.978 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xenfs 2024-05-11T11:59:03.978 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-11T11:59:03.978 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs 2024-05-11T11:59:03.978 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/9p 2024-05-11T11:59:03.978 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/9p/9p.ko 2024-05-11T11:59:03.978 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/affs 2024-05-11T11:59:03.978 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/affs/affs.ko 2024-05-11T11:59:03.978 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/befs 2024-05-11T11:59:03.979 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/befs/befs.ko 2024-05-11T11:59:03.979 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/binfmt_misc.ko 2024-05-11T11:59:03.979 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/btrfs 2024-05-11T11:59:03.979 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/btrfs/btrfs.ko 2024-05-11T11:59:03.979 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/cachefiles 2024-05-11T11:59:03.979 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/cachefiles/cachefiles.ko 2024-05-11T11:59:03.979 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ceph 2024-05-11T11:59:03.979 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ceph/ceph.ko 2024-05-11T11:59:03.979 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/coda 2024-05-11T11:59:03.979 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/coda/coda.ko 2024-05-11T11:59:03.979 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/cramfs 2024-05-11T11:59:03.979 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/cramfs/cramfs.ko 2024-05-11T11:59:03.979 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/dlm 2024-05-11T11:59:03.979 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/dlm/dlm.ko 2024-05-11T11:59:03.980 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ecryptfs 2024-05-11T11:59:03.980 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ecryptfs/ecryptfs.ko 2024-05-11T11:59:03.980 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/f2fs 2024-05-11T11:59:03.980 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/f2fs/f2fs.ko 2024-05-11T11:59:03.980 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fat 2024-05-11T11:59:03.980 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fat/fat.ko 2024-05-11T11:59:03.980 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fat/msdos.ko 2024-05-11T11:59:03.980 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fat/vfat.ko 2024-05-11T11:59:03.980 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fuse 2024-05-11T11:59:03.980 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fuse/cuse.ko 2024-05-11T11:59:03.980 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fuse/fuse.ko 2024-05-11T11:59:03.980 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/gfs2 2024-05-11T11:59:03.980 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/gfs2/gfs2.ko 2024-05-11T11:59:03.980 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/hfs 2024-05-11T11:59:03.980 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/hfs/hfs.ko 2024-05-11T11:59:03.981 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/hfsplus 2024-05-11T11:59:03.981 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/hfsplus/hfsplus.ko 2024-05-11T11:59:03.981 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/isofs 2024-05-11T11:59:03.981 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/isofs/isofs.ko 2024-05-11T11:59:03.981 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/jfs 2024-05-11T11:59:03.981 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/jfs/jfs.ko 2024-05-11T11:59:03.981 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/lockd 2024-05-11T11:59:03.981 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/lockd/lockd.ko 2024-05-11T11:59:03.981 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/minix 2024-05-11T11:59:03.981 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/minix/minix.ko 2024-05-11T11:59:03.981 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/netfs 2024-05-11T11:59:03.981 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/netfs/netfs.ko 2024-05-11T11:59:03.981 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs 2024-05-11T11:59:03.982 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/blocklayout 2024-05-11T11:59:03.982 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-11T11:59:03.982 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/filelayout 2024-05-11T11:59:03.982 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-11T11:59:03.982 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/flexfilelayout 2024-05-11T11:59:03.982 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-11T11:59:03.982 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/nfs.ko 2024-05-11T11:59:03.982 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/nfsv3.ko 2024-05-11T11:59:03.982 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/nfsv4.ko 2024-05-11T11:59:03.982 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs_common 2024-05-11T11:59:03.982 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs_common/grace.ko 2024-05-11T11:59:03.982 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs_common/nfs_acl.ko 2024-05-11T11:59:03.982 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfsd 2024-05-11T11:59:03.982 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfsd/nfsd.ko 2024-05-11T11:59:03.983 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nilfs2 2024-05-11T11:59:03.983 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nilfs2/nilfs2.ko 2024-05-11T11:59:03.983 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls 2024-05-11T11:59:03.983 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-celtic.ko 2024-05-11T11:59:03.983 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-centeuro.ko 2024-05-11T11:59:03.983 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-croatian.ko 2024-05-11T11:59:03.983 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-cyrillic.ko 2024-05-11T11:59:03.983 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-gaelic.ko 2024-05-11T11:59:03.983 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-greek.ko 2024-05-11T11:59:03.983 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-iceland.ko 2024-05-11T11:59:03.983 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-inuit.ko 2024-05-11T11:59:03.983 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-roman.ko 2024-05-11T11:59:03.983 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-romanian.ko 2024-05-11T11:59:03.983 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-turkish.ko 2024-05-11T11:59:03.984 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp1250.ko 2024-05-11T11:59:03.984 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp1251.ko 2024-05-11T11:59:03.984 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp1255.ko 2024-05-11T11:59:03.984 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp737.ko 2024-05-11T11:59:03.984 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp775.ko 2024-05-11T11:59:03.984 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp850.ko 2024-05-11T11:59:03.984 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp852.ko 2024-05-11T11:59:03.984 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp855.ko 2024-05-11T11:59:03.984 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp857.ko 2024-05-11T11:59:03.984 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp860.ko 2024-05-11T11:59:03.984 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp861.ko 2024-05-11T11:59:03.984 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp862.ko 2024-05-11T11:59:03.984 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp863.ko 2024-05-11T11:59:03.984 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp864.ko 2024-05-11T11:59:03.985 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp865.ko 2024-05-11T11:59:03.985 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp866.ko 2024-05-11T11:59:03.985 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp869.ko 2024-05-11T11:59:03.985 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp874.ko 2024-05-11T11:59:03.985 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp932.ko 2024-05-11T11:59:03.985 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp936.ko 2024-05-11T11:59:03.985 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp949.ko 2024-05-11T11:59:03.985 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp950.ko 2024-05-11T11:59:03.985 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_euc-jp.ko 2024-05-11T11:59:03.985 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-1.ko 2024-05-11T11:59:03.985 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-13.ko 2024-05-11T11:59:03.985 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-14.ko 2024-05-11T11:59:03.985 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-15.ko 2024-05-11T11:59:03.985 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-2.ko 2024-05-11T11:59:03.986 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-3.ko 2024-05-11T11:59:03.986 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-4.ko 2024-05-11T11:59:03.986 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-5.ko 2024-05-11T11:59:03.986 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-6.ko 2024-05-11T11:59:03.986 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-7.ko 2024-05-11T11:59:03.986 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-9.ko 2024-05-11T11:59:03.986 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_koi8-r.ko 2024-05-11T11:59:03.986 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_koi8-ru.ko 2024-05-11T11:59:03.986 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_koi8-u.ko 2024-05-11T11:59:03.986 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-11T11:59:03.986 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_utf8.ko 2024-05-11T11:59:03.986 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2 2024-05-11T11:59:03.986 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/cluster 2024-05-11T11:59:03.987 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-11T11:59:03.987 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/dlm 2024-05-11T11:59:03.987 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-11T11:59:03.987 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/dlmfs 2024-05-11T11:59:03.987 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-11T11:59:03.987 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/ocfs2.ko 2024-05-11T11:59:03.987 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-11T11:59:03.987 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-11T11:59:03.987 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-11T11:59:03.987 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/overlayfs 2024-05-11T11:59:03.987 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/overlayfs/overlay.ko 2024-05-11T11:59:03.987 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/pstore 2024-05-11T11:59:03.987 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/pstore/ramoops.ko 2024-05-11T11:59:03.987 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/reiserfs 2024-05-11T11:59:03.988 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/reiserfs/reiserfs.ko 2024-05-11T11:59:03.988 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/romfs 2024-05-11T11:59:03.988 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/romfs/romfs.ko 2024-05-11T11:59:03.988 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb 2024-05-11T11:59:03.988 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/client 2024-05-11T11:59:03.988 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/client/cifs.ko 2024-05-11T11:59:03.988 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/common 2024-05-11T11:59:03.988 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/common/cifs_arc4.ko 2024-05-11T11:59:03.988 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/common/cifs_md4.ko 2024-05-11T11:59:03.988 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/squashfs 2024-05-11T11:59:03.988 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/squashfs/squashfs.ko 2024-05-11T11:59:03.988 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/sysv 2024-05-11T11:59:03.988 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/sysv/sysv.ko 2024-05-11T11:59:03.988 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/udf 2024-05-11T11:59:03.988 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/udf/udf.ko 2024-05-11T11:59:03.989 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ufs 2024-05-11T11:59:03.989 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ufs/ufs.ko 2024-05-11T11:59:03.989 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/xfs 2024-05-11T11:59:03.989 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/xfs/xfs.ko 2024-05-11T11:59:03.989 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel 2024-05-11T11:59:03.989 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/rcu 2024-05-11T11:59:03.989 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/rcu/rcutorture.ko 2024-05-11T11:59:03.989 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/torture.ko 2024-05-11T11:59:03.989 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/trace 2024-05-11T11:59:03.989 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-05-11T11:59:03.989 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib 2024-05-11T11:59:03.989 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/asn1_encoder.ko 2024-05-11T11:59:03.989 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crc-itu-t.ko 2024-05-11T11:59:03.989 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crc8.ko 2024-05-11T11:59:03.990 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crypto 2024-05-11T11:59:03.990 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crypto/libarc4.ko 2024-05-11T11:59:03.990 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crypto/libdes.ko 2024-05-11T11:59:03.990 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/libcrc32c.ko 2024-05-11T11:59:03.990 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/lru_cache.ko 2024-05-11T11:59:03.990 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/lz4 2024-05-11T11:59:03.990 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/lz4/lz4_compress.ko 2024-05-11T11:59:03.990 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/lz4/lz4hc_compress.ko 2024-05-11T11:59:03.990 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/math 2024-05-11T11:59:03.990 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/math/cordic.ko 2024-05-11T11:59:03.990 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/raid6 2024-05-11T11:59:03.990 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/raid6/raid6_pq.ko 2024-05-11T11:59:03.990 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/reed_solomon 2024-05-11T11:59:03.990 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-11T11:59:03.991 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/ts_bm.ko 2024-05-11T11:59:03.991 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/ts_fsm.ko 2024-05-11T11:59:03.991 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/ts_kmp.ko 2024-05-11T11:59:03.991 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/zstd 2024-05-11T11:59:03.991 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/zstd/zstd_compress.ko 2024-05-11T11:59:03.991 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/mm 2024-05-11T11:59:03.991 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/mm/hwpoison-inject.ko 2024-05-11T11:59:03.991 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net 2024-05-11T11:59:03.991 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan 2024-05-11T11:59:03.991 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/6lowpan.ko 2024-05-11T11:59:03.991 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_dest.ko 2024-05-11T11:59:03.991 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_fragment.ko 2024-05-11T11:59:03.991 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_hop.ko 2024-05-11T11:59:03.991 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-11T11:59:03.992 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_mobility.ko 2024-05-11T11:59:03.992 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_routing.ko 2024-05-11T11:59:03.992 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_udp.ko 2024-05-11T11:59:03.992 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802 2024-05-11T11:59:03.992 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/garp.ko 2024-05-11T11:59:03.992 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/mrp.ko 2024-05-11T11:59:03.992 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/p8022.ko 2024-05-11T11:59:03.992 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/psnap.ko 2024-05-11T11:59:03.992 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/stp.ko 2024-05-11T11:59:03.992 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/8021q 2024-05-11T11:59:03.992 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/8021q/8021q.ko 2024-05-11T11:59:03.992 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p 2024-05-11T11:59:03.992 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p/9pnet.ko 2024-05-11T11:59:03.992 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p/9pnet_fd.ko 2024-05-11T11:59:03.993 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p/9pnet_rdma.ko 2024-05-11T11:59:03.993 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p/9pnet_virtio.ko 2024-05-11T11:59:03.993 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/batman-adv 2024-05-11T11:59:03.993 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/batman-adv/batman-adv.ko 2024-05-11T11:59:03.993 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge 2024-05-11T11:59:03.993 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/br_netfilter.ko 2024-05-11T11:59:03.993 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/bridge.ko 2024-05-11T11:59:03.993 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter 2024-05-11T11:59:03.993 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-11T11:59:03.993 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-11T11:59:03.993 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-11T11:59:03.993 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-11T11:59:03.993 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-11T11:59:03.993 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-11T11:59:03.994 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-11T11:59:03.994 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-11T11:59:03.994 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-11T11:59:03.994 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-11T11:59:03.994 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-11T11:59:03.997 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-11T11:59:03.997 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-11T11:59:03.997 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-11T11:59:03.998 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-11T11:59:03.998 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-11T11:59:03.998 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-11T11:59:03.998 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-11T11:59:03.998 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-11T11:59:03.998 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-11T11:59:03.998 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebtables.ko 2024-05-11T11:59:03.998 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-05-11T11:59:03.998 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-11T11:59:03.998 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-11T11:59:03.998 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ceph 2024-05-11T11:59:03.998 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ceph/libceph.ko 2024-05-11T11:59:03.998 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/core 2024-05-11T11:59:03.999 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/core/failover.ko 2024-05-11T11:59:03.999 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/core/pktgen.ko 2024-05-11T11:59:03.999 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp 2024-05-11T11:59:03.999 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp/dccp.ko 2024-05-11T11:59:03.999 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp/dccp_diag.ko 2024-05-11T11:59:03.999 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp/dccp_ipv4.ko 2024-05-11T11:59:03.999 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp/dccp_ipv6.ko 2024-05-11T11:59:03.999 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dns_resolver 2024-05-11T11:59:03.999 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dns_resolver/dns_resolver.ko 2024-05-11T11:59:03.999 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa 2024-05-11T11:59:03.999 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/dsa_core.ko 2024-05-11T11:59:03.999 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/tag_brcm.ko 2024-05-11T11:59:03.999 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/tag_dsa.ko 2024-05-11T11:59:03.999 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/tag_none.ko 2024-05-11T11:59:04.000 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/tag_trailer.ko 2024-05-11T11:59:04.000 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154 2024-05-11T11:59:04.000 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154/6lowpan 2024-05-11T11:59:04.000 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-11T11:59:04.000 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154/ieee802154.ko 2024-05-11T11:59:04.000 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-11T11:59:04.000 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4 2024-05-11T11:59:04.000 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ah4.ko 2024-05-11T11:59:04.000 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/esp4.ko 2024-05-11T11:59:04.000 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/fou.ko 2024-05-11T11:59:04.000 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/gre.ko 2024-05-11T11:59:04.000 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/inet_diag.ko 2024-05-11T11:59:04.000 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ip_gre.ko 2024-05-11T11:59:04.000 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ip_tunnel.ko 2024-05-11T11:59:04.001 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ip_vti.ko 2024-05-11T11:59:04.001 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ipcomp.ko 2024-05-11T11:59:04.001 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ipip.ko 2024-05-11T11:59:04.001 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter 2024-05-11T11:59:04.001 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-11T11:59:04.001 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-11T11:59:04.001 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-11T11:59:04.001 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-11T11:59:04.001 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-11T11:59:04.001 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-11T11:59:04.001 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-11T11:59:04.001 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-11T11:59:04.001 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-11T11:59:04.001 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-11T11:59:04.001 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-11T11:59:04.001 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-11T11:59:04.002 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-11T11:59:04.002 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-11T11:59:04.002 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-11T11:59:04.002 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-11T11:59:04.002 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-11T11:59:04.002 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-11T11:59:04.002 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-11T11:59:04.002 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-11T11:59:04.002 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-11T11:59:04.002 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_bic.ko 2024-05-11T11:59:04.002 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_dctcp.ko 2024-05-11T11:59:04.002 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_diag.ko 2024-05-11T11:59:04.002 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_highspeed.ko 2024-05-11T11:59:04.003 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_htcp.ko 2024-05-11T11:59:04.003 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_hybla.ko 2024-05-11T11:59:04.003 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_illinois.ko 2024-05-11T11:59:04.003 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_lp.ko 2024-05-11T11:59:04.003 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_scalable.ko 2024-05-11T11:59:04.003 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_vegas.ko 2024-05-11T11:59:04.003 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_veno.ko 2024-05-11T11:59:04.003 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_westwood.ko 2024-05-11T11:59:04.003 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_yeah.ko 2024-05-11T11:59:04.003 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tunnel4.ko 2024-05-11T11:59:04.003 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/udp_diag.ko 2024-05-11T11:59:04.003 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/udp_tunnel.ko 2024-05-11T11:59:04.003 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-11T11:59:04.003 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6 2024-05-11T11:59:04.003 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ah6.ko 2024-05-11T11:59:04.004 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/esp6.ko 2024-05-11T11:59:04.004 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/fou6.ko 2024-05-11T11:59:04.004 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ip6_tunnel.ko 2024-05-11T11:59:04.004 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-11T11:59:04.004 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ip6_vti.ko 2024-05-11T11:59:04.004 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ipcomp6.ko 2024-05-11T11:59:04.004 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter 2024-05-11T11:59:04.004 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-11T11:59:04.004 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-11T11:59:04.004 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-11T11:59:04.004 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-11T11:59:04.004 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-11T11:59:04.004 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-11T11:59:04.004 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-11T11:59:04.005 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-11T11:59:04.005 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-11T11:59:04.005 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-11T11:59:04.005 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-11T11:59:04.005 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-11T11:59:04.005 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-11T11:59:04.005 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-11T11:59:04.005 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-11T11:59:04.005 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-11T11:59:04.005 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-11T11:59:04.005 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-11T11:59:04.005 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-11T11:59:04.005 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-11T11:59:04.005 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-11T11:59:04.005 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-11T11:59:04.006 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-11T11:59:04.006 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-11T11:59:04.006 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/sit.ko 2024-05-11T11:59:04.006 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/tunnel6.ko 2024-05-11T11:59:04.006 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-11T11:59:04.006 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/key 2024-05-11T11:59:04.006 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/key/af_key.ko 2024-05-11T11:59:04.006 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp 2024-05-11T11:59:04.006 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_core.ko 2024-05-11T11:59:04.006 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-11T11:59:04.006 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_eth.ko 2024-05-11T11:59:04.006 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_ip.ko 2024-05-11T11:59:04.006 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_ip6.ko 2024-05-11T11:59:04.007 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_netlink.ko 2024-05-11T11:59:04.007 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_ppp.ko 2024-05-11T11:59:04.007 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/llc 2024-05-11T11:59:04.007 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/llc/llc.ko 2024-05-11T11:59:04.007 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/mac802154 2024-05-11T11:59:04.007 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/mac802154/mac802154.ko 2024-05-11T11:59:04.007 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/mpls 2024-05-11T11:59:04.007 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/mpls/mpls_gso.ko 2024-05-11T11:59:04.007 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter 2024-05-11T11:59:04.007 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset 2024-05-11T11:59:04.007 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set.ko 2024-05-11T11:59:04.007 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-11T11:59:04.007 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-11T11:59:04.007 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-11T11:59:04.007 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-11T11:59:04.008 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-11T11:59:04.008 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-11T11:59:04.008 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-11T11:59:04.008 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-11T11:59:04.008 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-11T11:59:04.008 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-11T11:59:04.008 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-11T11:59:04.008 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-11T11:59:04.008 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-11T11:59:04.008 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-11T11:59:04.008 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-11T11:59:04.008 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs 2024-05-11T11:59:04.008 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-11T11:59:04.008 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-11T11:59:04.009 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-11T11:59:04.009 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-11T11:59:04.009 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-11T11:59:04.009 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-11T11:59:04.009 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-11T11:59:04.009 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-11T11:59:04.009 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-11T11:59:04.009 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-11T11:59:04.009 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-11T11:59:04.009 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-11T11:59:04.009 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-11T11:59:04.009 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-11T11:59:04.009 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conncount.ko 2024-05-11T11:59:04.009 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack.ko 2024-05-11T11:59:04.009 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-11T11:59:04.010 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-11T11:59:04.010 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-11T11:59:04.010 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-11T11:59:04.010 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-11T11:59:04.010 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-11T11:59:04.010 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-11T11:59:04.010 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-11T11:59:04.010 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-11T11:59:04.010 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-11T11:59:04.010 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-11T11:59:04.010 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-11T11:59:04.010 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-11T11:59:04.010 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_flow_table.ko 2024-05-11T11:59:04.011 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-11T11:59:04.011 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_log_syslog.ko 2024-05-11T11:59:04.011 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat.ko 2024-05-11T11:59:04.011 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-11T11:59:04.011 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-11T11:59:04.011 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_irc.ko 2024-05-11T11:59:04.011 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_sip.ko 2024-05-11T11:59:04.011 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-11T11:59:04.011 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-11T11:59:04.011 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_tables.ko 2024-05-11T11:59:04.011 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink.ko 2024-05-11T11:59:04.011 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-11T11:59:04.011 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink_log.ko 2024-05-11T11:59:04.011 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-11T11:59:04.012 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-11T11:59:04.012 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_chain_nat.ko 2024-05-11T11:59:04.012 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_compat.ko 2024-05-11T11:59:04.012 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_ct.ko 2024-05-11T11:59:04.012 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-11T11:59:04.012 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_fib.ko 2024-05-11T11:59:04.012 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-11T11:59:04.012 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_hash.ko 2024-05-11T11:59:04.012 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_limit.ko 2024-05-11T11:59:04.012 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_log.ko 2024-05-11T11:59:04.012 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_masq.ko 2024-05-11T11:59:04.012 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_nat.ko 2024-05-11T11:59:04.012 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_numgen.ko 2024-05-11T11:59:04.012 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_queue.ko 2024-05-11T11:59:04.013 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_quota.ko 2024-05-11T11:59:04.013 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_redir.ko 2024-05-11T11:59:04.013 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_reject.ko 2024-05-11T11:59:04.013 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_reject_inet.ko 2024-05-11T11:59:04.013 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_socket.ko 2024-05-11T11:59:04.013 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_tunnel.ko 2024-05-11T11:59:04.013 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_AUDIT.ko 2024-05-11T11:59:04.013 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-11T11:59:04.013 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-11T11:59:04.013 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-11T11:59:04.013 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_CT.ko 2024-05-11T11:59:04.013 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_DSCP.ko 2024-05-11T11:59:04.013 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_HL.ko 2024-05-11T11:59:04.013 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_HMARK.ko 2024-05-11T11:59:04.014 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-11T11:59:04.014 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_LED.ko 2024-05-11T11:59:04.014 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_LOG.ko 2024-05-11T11:59:04.014 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-11T11:59:04.014 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_NETMAP.ko 2024-05-11T11:59:04.014 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_NFLOG.ko 2024-05-11T11:59:04.014 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-11T11:59:04.014 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_RATEEST.ko 2024-05-11T11:59:04.014 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-11T11:59:04.014 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_SECMARK.ko 2024-05-11T11:59:04.014 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-11T11:59:04.014 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-11T11:59:04.014 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TEE.ko 2024-05-11T11:59:04.014 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TPROXY.ko 2024-05-11T11:59:04.015 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TRACE.ko 2024-05-11T11:59:04.015 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_addrtype.ko 2024-05-11T11:59:04.015 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_bpf.ko 2024-05-11T11:59:04.015 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_cgroup.ko 2024-05-11T11:59:04.015 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_cluster.ko 2024-05-11T11:59:04.015 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_comment.ko 2024-05-11T11:59:04.015 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_connbytes.ko 2024-05-11T11:59:04.015 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_connlabel.ko 2024-05-11T11:59:04.015 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_connlimit.ko 2024-05-11T11:59:04.015 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_connmark.ko 2024-05-11T11:59:04.015 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_conntrack.ko 2024-05-11T11:59:04.015 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_cpu.ko 2024-05-11T11:59:04.015 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_dccp.ko 2024-05-11T11:59:04.015 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_devgroup.ko 2024-05-11T11:59:04.016 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_dscp.ko 2024-05-11T11:59:04.016 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_ecn.ko 2024-05-11T11:59:04.016 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_esp.ko 2024-05-11T11:59:04.016 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_hashlimit.ko 2024-05-11T11:59:04.016 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_helper.ko 2024-05-11T11:59:04.016 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_hl.ko 2024-05-11T11:59:04.016 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_ipcomp.ko 2024-05-11T11:59:04.016 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_iprange.ko 2024-05-11T11:59:04.016 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_ipvs.ko 2024-05-11T11:59:04.016 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_l2tp.ko 2024-05-11T11:59:04.016 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_length.ko 2024-05-11T11:59:04.016 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_limit.ko 2024-05-11T11:59:04.016 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_mac.ko 2024-05-11T11:59:04.016 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_mark.ko 2024-05-11T11:59:04.017 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_multiport.ko 2024-05-11T11:59:04.017 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_nat.ko 2024-05-11T11:59:04.017 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_nfacct.ko 2024-05-11T11:59:04.017 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_osf.ko 2024-05-11T11:59:04.017 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_owner.ko 2024-05-11T11:59:04.017 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_physdev.ko 2024-05-11T11:59:04.017 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_pkttype.ko 2024-05-11T11:59:04.017 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_policy.ko 2024-05-11T11:59:04.017 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_quota.ko 2024-05-11T11:59:04.017 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_rateest.ko 2024-05-11T11:59:04.017 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_realm.ko 2024-05-11T11:59:04.017 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_recent.ko 2024-05-11T11:59:04.017 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_sctp.ko 2024-05-11T11:59:04.017 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_set.ko 2024-05-11T11:59:04.017 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_socket.ko 2024-05-11T11:59:04.018 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_state.ko 2024-05-11T11:59:04.018 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_statistic.ko 2024-05-11T11:59:04.018 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_string.ko 2024-05-11T11:59:04.018 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_tcpmss.ko 2024-05-11T11:59:04.018 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_time.ko 2024-05-11T11:59:04.018 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_u32.ko 2024-05-11T11:59:04.018 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netlink 2024-05-11T11:59:04.018 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netlink/netlink_diag.ko 2024-05-11T11:59:04.018 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/nsh 2024-05-11T11:59:04.018 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/nsh/nsh.ko 2024-05-11T11:59:04.018 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch 2024-05-11T11:59:04.018 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch/openvswitch.ko 2024-05-11T11:59:04.018 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch/vport-geneve.ko 2024-05-11T11:59:04.018 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch/vport-gre.ko 2024-05-11T11:59:04.019 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch/vport-vxlan.ko 2024-05-11T11:59:04.019 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/packet 2024-05-11T11:59:04.019 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/packet/af_packet_diag.ko 2024-05-11T11:59:04.019 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/rds 2024-05-11T11:59:04.019 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/rds/rds.ko 2024-05-11T11:59:04.019 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/rds/rds_rdma.ko 2024-05-11T11:59:04.019 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/rds/rds_tcp.ko 2024-05-11T11:59:04.019 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched 2024-05-11T11:59:04.019 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_csum.ko 2024-05-11T11:59:04.019 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_gact.ko 2024-05-11T11:59:04.019 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_mirred.ko 2024-05-11T11:59:04.019 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_nat.ko 2024-05-11T11:59:04.019 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_pedit.ko 2024-05-11T11:59:04.019 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_police.ko 2024-05-11T11:59:04.019 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_simple.ko 2024-05-11T11:59:04.020 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_skbedit.ko 2024-05-11T11:59:04.020 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_vlan.ko 2024-05-11T11:59:04.020 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_basic.ko 2024-05-11T11:59:04.020 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_bpf.ko 2024-05-11T11:59:04.020 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_flow.ko 2024-05-11T11:59:04.020 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_fw.ko 2024-05-11T11:59:04.020 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_route.ko 2024-05-11T11:59:04.020 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_u32.ko 2024-05-11T11:59:04.020 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_cmp.ko 2024-05-11T11:59:04.020 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_ipset.ko 2024-05-11T11:59:04.020 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_meta.ko 2024-05-11T11:59:04.020 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_nbyte.ko 2024-05-11T11:59:04.020 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_text.ko 2024-05-11T11:59:04.020 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_u32.ko 2024-05-11T11:59:04.021 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_choke.ko 2024-05-11T11:59:04.021 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_codel.ko 2024-05-11T11:59:04.021 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_drr.ko 2024-05-11T11:59:04.021 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_fq.ko 2024-05-11T11:59:04.021 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_gred.ko 2024-05-11T11:59:04.021 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_hfsc.ko 2024-05-11T11:59:04.021 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_hhf.ko 2024-05-11T11:59:04.021 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_htb.ko 2024-05-11T11:59:04.021 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_ingress.ko 2024-05-11T11:59:04.021 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_mqprio.ko 2024-05-11T11:59:04.021 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_mqprio_lib.ko 2024-05-11T11:59:04.021 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_multiq.ko 2024-05-11T11:59:04.021 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_netem.ko 2024-05-11T11:59:04.021 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_pie.ko 2024-05-11T11:59:04.021 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_plug.ko 2024-05-11T11:59:04.022 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_prio.ko 2024-05-11T11:59:04.022 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_qfq.ko 2024-05-11T11:59:04.022 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_red.ko 2024-05-11T11:59:04.022 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_sfb.ko 2024-05-11T11:59:04.022 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_sfq.ko 2024-05-11T11:59:04.022 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_tbf.ko 2024-05-11T11:59:04.022 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_teql.ko 2024-05-11T11:59:04.022 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sctp 2024-05-11T11:59:04.022 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sctp/sctp.ko 2024-05-11T11:59:04.022 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sctp/sctp_diag.ko 2024-05-11T11:59:04.022 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc 2024-05-11T11:59:04.022 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/auth_gss 2024-05-11T11:59:04.022 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-11T11:59:04.022 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-11T11:59:04.023 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/sunrpc.ko 2024-05-11T11:59:04.023 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/xprtrdma 2024-05-11T11:59:04.023 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-11T11:59:04.023 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/tipc 2024-05-11T11:59:04.023 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/tipc/tipc.ko 2024-05-11T11:59:04.023 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/tipc/tipc_diag.ko 2024-05-11T11:59:04.023 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/unix 2024-05-11T11:59:04.023 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/unix/unix_diag.ko 2024-05-11T11:59:04.023 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock 2024-05-11T11:59:04.023 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-11T11:59:04.023 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-11T11:59:04.023 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vsock.ko 2024-05-11T11:59:04.023 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-11T11:59:04.023 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-11T11:59:04.023 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/xfrm 2024-05-11T11:59:04.024 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-11T11:59:04.024 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security 2024-05-11T11:59:04.024 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security/keys 2024-05-11T11:59:04.024 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security/keys/encrypted-keys 2024-05-11T11:59:04.024 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-05-11T11:59:04.024 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security/keys/trusted-keys 2024-05-11T11:59:04.024 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security/keys/trusted-keys/trusted.ko 2024-05-11T11:59:04.024 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.alias 2024-05-11T11:59:04.024 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.alias.bin 2024-05-11T11:59:04.024 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.builtin 2024-05-11T11:59:04.024 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.builtin.alias.bin 2024-05-11T11:59:04.024 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.builtin.bin 2024-05-11T11:59:04.024 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.builtin.modinfo 2024-05-11T11:59:04.024 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.dep 2024-05-11T11:59:04.025 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.dep.bin 2024-05-11T11:59:04.025 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.devname 2024-05-11T11:59:04.025 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.order 2024-05-11T11:59:04.025 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.softdep 2024-05-11T11:59:04.025 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.symbols 2024-05-11T11:59:04.025 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.symbols.bin 2024-05-11T11:59:04.025 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/vmlinuz 2024-05-11T11:59:04.029 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-gfb9a6a44107e 2024-05-11T11:59:04.029 DEBUG:teuthology.orchestra.run.smithi204:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-05-11T11:59:04.052 INFO:teuthology.orchestra.run.smithi204.stdout:postinstall scriptlet (using /bin/sh): 2024-05-11T11:59:04.052 INFO:teuthology.orchestra.run.smithi204.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-11T11:59:04.052 INFO:teuthology.orchestra.run.smithi204.stdout: /usr/bin/kernel-install add 6.9.0-rc3-gfb9a6a44107e /lib/modules/6.9.0-rc3-gfb9a6a44107e/vmlinuz 2024-05-11T11:59:04.052 INFO:teuthology.orchestra.run.smithi204.stdout:fi 2024-05-11T11:59:04.052 INFO:teuthology.orchestra.run.smithi204.stdout:for file in vmlinuz System.map config; do 2024-05-11T11:59:04.052 INFO:teuthology.orchestra.run.smithi204.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-gfb9a6a44107e/${file}" "/boot/${file}-6.9.0-rc3-gfb9a6a44107e"; then 2024-05-11T11:59:04.052 INFO:teuthology.orchestra.run.smithi204.stdout: cp "/lib/modules/6.9.0-rc3-gfb9a6a44107e/${file}" "/boot/${file}-6.9.0-rc3-gfb9a6a44107e" 2024-05-11T11:59:04.053 INFO:teuthology.orchestra.run.smithi204.stdout: fi 2024-05-11T11:59:04.053 INFO:teuthology.orchestra.run.smithi204.stdout:done 2024-05-11T11:59:04.053 INFO:teuthology.orchestra.run.smithi204.stdout:if [ -d "/lib/modules/6.9.0-rc3-gfb9a6a44107e/dtb" ] && \ 2024-05-11T11:59:04.053 INFO:teuthology.orchestra.run.smithi204.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-gfb9a6a44107e/dtb" "/boot/dtb-6.9.0-rc3-gfb9a6a44107e" >/dev/null 2>&1; then 2024-05-11T11:59:04.053 INFO:teuthology.orchestra.run.smithi204.stdout: rm -rf "/boot/dtb-6.9.0-rc3-gfb9a6a44107e" 2024-05-11T11:59:04.053 INFO:teuthology.orchestra.run.smithi204.stdout: cp -r "/lib/modules/6.9.0-rc3-gfb9a6a44107e/dtb" "/boot/dtb-6.9.0-rc3-gfb9a6a44107e" 2024-05-11T11:59:04.053 INFO:teuthology.orchestra.run.smithi204.stdout:fi 2024-05-11T11:59:04.053 INFO:teuthology.orchestra.run.smithi204.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.dep" ]; then 2024-05-11T11:59:04.053 INFO:teuthology.orchestra.run.smithi204.stdout: /usr/sbin/depmod 6.9.0-rc3-gfb9a6a44107e 2024-05-11T11:59:04.053 INFO:teuthology.orchestra.run.smithi204.stdout:fi 2024-05-11T11:59:04.053 INFO:teuthology.orchestra.run.smithi204.stdout:preuninstall scriptlet (using /bin/sh): 2024-05-11T11:59:04.053 INFO:teuthology.orchestra.run.smithi204.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-11T11:59:04.053 INFO:teuthology.orchestra.run.smithi204.stdout:kernel-install remove 6.9.0-rc3-gfb9a6a44107e 2024-05-11T11:59:04.053 INFO:teuthology.orchestra.run.smithi204.stdout:fi 2024-05-11T11:59:04.054 DEBUG:teuthology.orchestra.run.smithi204:> sudo rpm -qi grub2-tools 2024-05-11T11:59:04.128 INFO:teuthology.orchestra.run.smithi204.stdout:Name : grub2-tools 2024-05-11T11:59:04.128 INFO:teuthology.orchestra.run.smithi204.stdout:Epoch : 1 2024-05-11T11:59:04.128 INFO:teuthology.orchestra.run.smithi204.stdout:Version : 2.06 2024-05-11T11:59:04.128 INFO:teuthology.orchestra.run.smithi204.stdout:Release : 68.el9 2024-05-11T11:59:04.128 INFO:teuthology.orchestra.run.smithi204.stdout:Architecture: x86_64 2024-05-11T11:59:04.128 INFO:teuthology.orchestra.run.smithi204.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-11T11:59:04.128 INFO:teuthology.orchestra.run.smithi204.stdout:Group : Unspecified 2024-05-11T11:59:04.128 INFO:teuthology.orchestra.run.smithi204.stdout:Size : 8264254 2024-05-11T11:59:04.129 INFO:teuthology.orchestra.run.smithi204.stdout:License : GPLv3+ 2024-05-11T11:59:04.129 INFO:teuthology.orchestra.run.smithi204.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-11T11:59:04.129 INFO:teuthology.orchestra.run.smithi204.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-11T11:59:04.129 INFO:teuthology.orchestra.run.smithi204.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-11T11:59:04.129 INFO:teuthology.orchestra.run.smithi204.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-11T11:59:04.129 INFO:teuthology.orchestra.run.smithi204.stdout:Packager : builder@centos.org 2024-05-11T11:59:04.129 INFO:teuthology.orchestra.run.smithi204.stdout:Vendor : CentOS 2024-05-11T11:59:04.129 INFO:teuthology.orchestra.run.smithi204.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-11T11:59:04.129 INFO:teuthology.orchestra.run.smithi204.stdout:Summary : Support tools for GRUB. 2024-05-11T11:59:04.129 INFO:teuthology.orchestra.run.smithi204.stdout:Description : 2024-05-11T11:59:04.129 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T11:59:04.129 INFO:teuthology.orchestra.run.smithi204.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-11T11:59:04.129 INFO:teuthology.orchestra.run.smithi204.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-11T11:59:04.129 INFO:teuthology.orchestra.run.smithi204.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-11T11:59:04.129 INFO:teuthology.orchestra.run.smithi204.stdout:hardware devices. 2024-05-11T11:59:04.130 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T11:59:04.130 INFO:teuthology.orchestra.run.smithi204.stdout:This subpackage provides tools for support of all platforms. 2024-05-11T11:59:04.130 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-11T11:59:04.131 INFO:teuthology.task.kernel:Updating grub on smithi204 to boot 6.9.0-rc3-gfb9a6a44107e 2024-05-11T11:59:04.131 DEBUG:teuthology.orchestra.run.smithi204:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-11T11:59:04.703 INFO:teuthology.orchestra.run.smithi204.stderr:Generating grub configuration file ... 2024-05-11T11:59:06.023 INFO:teuthology.orchestra.run.smithi204.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-11T11:59:06.053 INFO:teuthology.orchestra.run.smithi204.stderr:done 2024-05-11T11:59:06.055 DEBUG:teuthology.orchestra.run.smithi204:> mktemp 2024-05-11T11:59:06.086 INFO:teuthology.orchestra.run.smithi204.stdout:/tmp/tmp.8KjkMrAd1s 2024-05-11T11:59:06.087 DEBUG:teuthology.orchestra.run.smithi204:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.8KjkMrAd1s 2024-05-11T11:59:06.152 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 0666 /tmp/tmp.8KjkMrAd1s 2024-05-11T11:59:06.292 DEBUG:teuthology.orchestra.remote:smithi204:/tmp/tmp.8KjkMrAd1s is 6KB 2024-05-11T11:59:06.343 DEBUG:teuthology.orchestra.run.smithi204:> rm -fr /tmp/tmp.8KjkMrAd1s 2024-05-11T11:59:06.357 DEBUG:teuthology.orchestra.run.smithi204:> sudo /bin/ls /boot/loader/entries || true 2024-05-11T11:59:06.426 INFO:teuthology.orchestra.run.smithi204.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-11T11:59:06.426 INFO:teuthology.orchestra.run.smithi204.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-11T11:59:06.426 INFO:teuthology.orchestra.run.smithi204.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-11T11:59:06.426 INFO:teuthology.orchestra.run.smithi204.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-gfb9a6a44107e.conf 2024-05-11T11:59:06.428 DEBUG:teuthology.orchestra.run.smithi204:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-gfb9a6a44107e 2024-05-11T11:59:06.591 DEBUG:teuthology.orchestra.run.smithi204:> sudo shutdown -r now 2024-05-11T11:59:13.081 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi167.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-05-11T11:59:13.081 DEBUG:teuthology.orchestra.run.smithi167:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-05-11T11:59:13.156 INFO:teuthology.orchestra.run.smithi167.stdout:/boot/System.map-6.9.0-rc3-gfb9a6a44107e 2024-05-11T11:59:13.157 INFO:teuthology.orchestra.run.smithi167.stdout:/boot/config-6.9.0-rc3-gfb9a6a44107e 2024-05-11T11:59:13.157 INFO:teuthology.orchestra.run.smithi167.stdout:/boot/vmlinuz-6.9.0-rc3-gfb9a6a44107e 2024-05-11T11:59:13.157 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/System.map 2024-05-11T11:59:13.157 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/config 2024-05-11T11:59:13.157 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel 2024-05-11T11:59:13.157 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch 2024-05-11T11:59:13.157 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86 2024-05-11T11:59:13.157 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto 2024-05-11T11:59:13.157 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-11T11:59:13.157 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-11T11:59:13.157 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-11T11:59:13.157 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-11T11:59:13.158 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-11T11:59:13.158 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-11T11:59:13.158 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-11T11:59:13.158 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/crc32c-intel.ko 2024-05-11T11:59:13.158 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-11T11:59:13.158 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-11T11:59:13.158 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-11T11:59:13.158 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-11T11:59:13.158 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-11T11:59:13.158 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-11T11:59:13.158 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-11T11:59:13.158 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-11T11:59:13.159 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-11T11:59:13.159 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-11T11:59:13.159 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-11T11:59:13.159 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-11T11:59:13.159 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kernel 2024-05-11T11:59:13.159 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kernel/cpu 2024-05-11T11:59:13.159 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kernel/cpu/mce 2024-05-11T11:59:13.159 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-11T11:59:13.159 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kvm 2024-05-11T11:59:13.159 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-11T11:59:13.159 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-11T11:59:13.159 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kvm/kvm.ko 2024-05-11T11:59:13.159 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto 2024-05-11T11:59:13.159 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/ansi_cprng.ko 2024-05-11T11:59:13.160 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/anubis.ko 2024-05-11T11:59:13.160 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/arc4.ko 2024-05-11T11:59:13.160 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx 2024-05-11T11:59:13.160 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_memcpy.ko 2024-05-11T11:59:13.160 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_pq.ko 2024-05-11T11:59:13.160 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-11T11:59:13.160 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_tx.ko 2024-05-11T11:59:13.160 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_xor.ko 2024-05-11T11:59:13.160 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/raid6test.ko 2024-05-11T11:59:13.160 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/authenc.ko 2024-05-11T11:59:13.160 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/authencesn.ko 2024-05-11T11:59:13.160 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/blake2b_generic.ko 2024-05-11T11:59:13.160 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/blowfish_common.ko 2024-05-11T11:59:13.160 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/blowfish_generic.ko 2024-05-11T11:59:13.161 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/camellia_generic.ko 2024-05-11T11:59:13.161 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cast5_generic.ko 2024-05-11T11:59:13.161 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cast6_generic.ko 2024-05-11T11:59:13.161 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cast_common.ko 2024-05-11T11:59:13.161 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/ccm.ko 2024-05-11T11:59:13.161 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cmac.ko 2024-05-11T11:59:13.161 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/crc32_generic.ko 2024-05-11T11:59:13.161 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/crypto_engine.ko 2024-05-11T11:59:13.161 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/crypto_user.ko 2024-05-11T11:59:13.161 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cts.ko 2024-05-11T11:59:13.161 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/des_generic.ko 2024-05-11T11:59:13.161 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/dh_generic.ko 2024-05-11T11:59:13.161 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/echainiv.ko 2024-05-11T11:59:13.161 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/essiv.ko 2024-05-11T11:59:13.161 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/fcrypt.ko 2024-05-11T11:59:13.162 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/khazad.ko 2024-05-11T11:59:13.162 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/lz4.ko 2024-05-11T11:59:13.162 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/lz4hc.ko 2024-05-11T11:59:13.162 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/md4.ko 2024-05-11T11:59:13.162 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/michael_mic.ko 2024-05-11T11:59:13.162 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/pcbc.ko 2024-05-11T11:59:13.162 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/pcrypt.ko 2024-05-11T11:59:13.162 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/rmd160.ko 2024-05-11T11:59:13.162 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/seed.ko 2024-05-11T11:59:13.162 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/serpent_generic.ko 2024-05-11T11:59:13.162 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/tcrypt.ko 2024-05-11T11:59:13.162 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/tea.ko 2024-05-11T11:59:13.162 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/twofish_common.ko 2024-05-11T11:59:13.162 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/twofish_generic.ko 2024-05-11T11:59:13.163 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/vmac.ko 2024-05-11T11:59:13.163 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/wp512.ko 2024-05-11T11:59:13.163 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/xcbc.ko 2024-05-11T11:59:13.163 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/xor.ko 2024-05-11T11:59:13.163 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/xxhash_generic.ko 2024-05-11T11:59:13.163 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers 2024-05-11T11:59:13.163 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi 2024-05-11T11:59:13.163 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-11T11:59:13.163 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/acpi_pad.ko 2024-05-11T11:59:13.163 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/ec_sys.ko 2024-05-11T11:59:13.163 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/platform_profile.ko 2024-05-11T11:59:13.163 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/sbs.ko 2024-05-11T11:59:13.163 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/sbshc.ko 2024-05-11T11:59:13.163 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/video.ko 2024-05-11T11:59:13.164 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata 2024-05-11T11:59:13.164 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/acard-ahci.ko 2024-05-11T11:59:13.164 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/ahci_platform.ko 2024-05-11T11:59:13.164 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/ata_generic.ko 2024-05-11T11:59:13.164 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/libahci_platform.ko 2024-05-11T11:59:13.164 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_acpi.ko 2024-05-11T11:59:13.164 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_ali.ko 2024-05-11T11:59:13.164 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_amd.ko 2024-05-11T11:59:13.164 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_artop.ko 2024-05-11T11:59:13.164 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_atiixp.ko 2024-05-11T11:59:13.164 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_atp867x.ko 2024-05-11T11:59:13.164 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_cmd640.ko 2024-05-11T11:59:13.165 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_cmd64x.ko 2024-05-11T11:59:13.165 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_cypress.ko 2024-05-11T11:59:13.165 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_efar.ko 2024-05-11T11:59:13.165 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_hpt366.ko 2024-05-11T11:59:13.165 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_hpt37x.ko 2024-05-11T11:59:13.165 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-11T11:59:13.165 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-11T11:59:13.165 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_it8213.ko 2024-05-11T11:59:13.165 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_it821x.ko 2024-05-11T11:59:13.165 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_jmicron.ko 2024-05-11T11:59:13.165 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_marvell.ko 2024-05-11T11:59:13.165 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_mpiix.ko 2024-05-11T11:59:13.165 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_netcell.ko 2024-05-11T11:59:13.166 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_ninja32.ko 2024-05-11T11:59:13.166 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_ns87410.ko 2024-05-11T11:59:13.166 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_ns87415.ko 2024-05-11T11:59:13.166 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_oldpiix.ko 2024-05-11T11:59:13.166 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_opti.ko 2024-05-11T11:59:13.166 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_optidma.ko 2024-05-11T11:59:13.166 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_pcmcia.ko 2024-05-11T11:59:13.166 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-11T11:59:13.166 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-11T11:59:13.166 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_piccolo.ko 2024-05-11T11:59:13.166 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_rdc.ko 2024-05-11T11:59:13.167 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_sch.ko 2024-05-11T11:59:13.167 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_serverworks.ko 2024-05-11T11:59:13.167 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_sil680.ko 2024-05-11T11:59:13.167 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_sis.ko 2024-05-11T11:59:13.167 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_sl82c105.ko 2024-05-11T11:59:13.167 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_triflex.ko 2024-05-11T11:59:13.167 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_via.ko 2024-05-11T11:59:13.167 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pdc_adma.ko 2024-05-11T11:59:13.167 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_inic162x.ko 2024-05-11T11:59:13.167 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_mv.ko 2024-05-11T11:59:13.167 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_nv.ko 2024-05-11T11:59:13.167 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_promise.ko 2024-05-11T11:59:13.167 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_qstor.ko 2024-05-11T11:59:13.167 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_sil.ko 2024-05-11T11:59:13.168 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_sil24.ko 2024-05-11T11:59:13.168 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_sis.ko 2024-05-11T11:59:13.168 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_svw.ko 2024-05-11T11:59:13.168 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_sx4.ko 2024-05-11T11:59:13.168 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_uli.ko 2024-05-11T11:59:13.168 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_via.ko 2024-05-11T11:59:13.168 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_vsc.ko 2024-05-11T11:59:13.168 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/auxdisplay 2024-05-11T11:59:13.168 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/auxdisplay/ks0108.ko 2024-05-11T11:59:13.168 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/base 2024-05-11T11:59:13.168 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/base/regmap 2024-05-11T11:59:13.168 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/base/regmap/regmap-i2c.ko 2024-05-11T11:59:13.168 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/base/regmap/regmap-mmio.ko 2024-05-11T11:59:13.168 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/bcma 2024-05-11T11:59:13.168 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/bcma/bcma.ko 2024-05-11T11:59:13.169 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block 2024-05-11T11:59:13.169 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/aoe 2024-05-11T11:59:13.169 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/aoe/aoe.ko 2024-05-11T11:59:13.169 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/brd.ko 2024-05-11T11:59:13.169 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/drbd 2024-05-11T11:59:13.169 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/drbd/drbd.ko 2024-05-11T11:59:13.169 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/floppy.ko 2024-05-11T11:59:13.170 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/loop.ko 2024-05-11T11:59:13.170 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/mtip32xx 2024-05-11T11:59:13.170 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-11T11:59:13.170 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/nbd.ko 2024-05-11T11:59:13.170 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/null_blk 2024-05-11T11:59:13.170 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/null_blk/null_blk.ko 2024-05-11T11:59:13.170 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/pktcdvd.ko 2024-05-11T11:59:13.170 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/rbd.ko 2024-05-11T11:59:13.170 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/virtio_blk.ko 2024-05-11T11:59:13.170 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/xen-blkback 2024-05-11T11:59:13.170 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-11T11:59:13.171 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/xen-blkfront.ko 2024-05-11T11:59:13.171 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/zram 2024-05-11T11:59:13.171 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/zram/zram.ko 2024-05-11T11:59:13.171 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char 2024-05-11T11:59:13.171 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hangcheck-timer.ko 2024-05-11T11:59:13.171 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random 2024-05-11T11:59:13.171 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-11T11:59:13.171 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-11T11:59:13.171 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-11T11:59:13.171 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/via-rng.ko 2024-05-11T11:59:13.171 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-11T11:59:13.171 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi 2024-05-11T11:59:13.171 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-11T11:59:13.171 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-11T11:59:13.172 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-11T11:59:13.172 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-11T11:59:13.172 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-11T11:59:13.172 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-11T11:59:13.172 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/lp.ko 2024-05-11T11:59:13.172 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/mwave 2024-05-11T11:59:13.172 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/mwave/mwave.ko 2024-05-11T11:59:13.172 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ppdev.ko 2024-05-11T11:59:13.172 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tlclk.ko 2024-05-11T11:59:13.172 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm 2024-05-11T11:59:13.172 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm.ko 2024-05-11T11:59:13.172 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-11T11:59:13.172 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-11T11:59:13.172 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-11T11:59:13.173 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_tis.ko 2024-05-11T11:59:13.173 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-05-11T11:59:13.173 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/uv_mmtimer.ko 2024-05-11T11:59:13.173 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/virtio_console.ko 2024-05-11T11:59:13.173 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq 2024-05-11T11:59:13.173 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-05-11T11:59:13.173 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-11T11:59:13.173 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-11T11:59:13.173 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-05-11T11:59:13.173 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/powernow-k8.ko 2024-05-11T11:59:13.173 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-11T11:59:13.173 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto 2024-05-11T11:59:13.174 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/ccp 2024-05-11T11:59:13.174 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-11T11:59:13.174 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/ccp/ccp.ko 2024-05-11T11:59:13.174 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel 2024-05-11T11:59:13.174 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat 2024-05-11T11:59:13.174 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat/qat_common 2024-05-11T11:59:13.174 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-11T11:59:13.174 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-05-11T11:59:13.174 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-11T11:59:13.174 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/padlock-aes.ko 2024-05-11T11:59:13.174 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/padlock-sha.ko 2024-05-11T11:59:13.174 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/virtio 2024-05-11T11:59:13.174 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-11T11:59:13.174 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dca 2024-05-11T11:59:13.175 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dca/dca.ko 2024-05-11T11:59:13.175 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/devfreq 2024-05-11T11:59:13.175 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-05-11T11:59:13.175 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma 2024-05-11T11:59:13.175 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma/dw 2024-05-11T11:59:13.175 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-11T11:59:13.175 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma/ioat 2024-05-11T11:59:13.175 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-11T11:59:13.175 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac 2024-05-11T11:59:13.175 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/amd64_edac.ko 2024-05-11T11:59:13.175 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/e752x_edac.ko 2024-05-11T11:59:13.175 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/edac_mce_amd.ko 2024-05-11T11:59:13.175 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i3000_edac.ko 2024-05-11T11:59:13.175 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i3200_edac.ko 2024-05-11T11:59:13.176 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i5100_edac.ko 2024-05-11T11:59:13.176 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i5400_edac.ko 2024-05-11T11:59:13.176 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i7300_edac.ko 2024-05-11T11:59:13.176 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i7core_edac.ko 2024-05-11T11:59:13.176 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i82975x_edac.ko 2024-05-11T11:59:13.176 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/ie31200_edac.ko 2024-05-11T11:59:13.176 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/sb_edac.ko 2024-05-11T11:59:13.176 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/x38_edac.ko 2024-05-11T11:59:13.176 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/firmware 2024-05-11T11:59:13.176 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/firmware/edd.ko 2024-05-11T11:59:13.176 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-11T11:59:13.176 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio 2024-05-11T11:59:13.176 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-ich.ko 2024-05-11T11:59:13.176 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-11T11:59:13.177 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid 2024-05-11T11:59:13.177 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-appleir.ko 2024-05-11T11:59:13.177 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-aureal.ko 2024-05-11T11:59:13.177 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-axff.ko 2024-05-11T11:59:13.177 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-dr.ko 2024-05-11T11:59:13.177 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-elecom.ko 2024-05-11T11:59:13.177 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-elo.ko 2024-05-11T11:59:13.177 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-emsff.ko 2024-05-11T11:59:13.177 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-gaff.ko 2024-05-11T11:59:13.177 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-gt683r.ko 2024-05-11T11:59:13.177 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-gyration.ko 2024-05-11T11:59:13.177 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-11T11:59:13.177 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-11T11:59:13.177 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-holtekff.ko 2024-05-11T11:59:13.178 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-hyperv.ko 2024-05-11T11:59:13.178 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-icade.ko 2024-05-11T11:59:13.178 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-keytouch.ko 2024-05-11T11:59:13.178 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-kye.ko 2024-05-11T11:59:13.178 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-lcpower.ko 2024-05-11T11:59:13.178 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-led.ko 2024-05-11T11:59:13.178 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-lenovo.ko 2024-05-11T11:59:13.178 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-11T11:59:13.178 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-11T11:59:13.178 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-multitouch.ko 2024-05-11T11:59:13.178 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-ortek.ko 2024-05-11T11:59:13.178 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-penmount.ko 2024-05-11T11:59:13.179 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-petalynx.ko 2024-05-11T11:59:13.179 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-picolcd.ko 2024-05-11T11:59:13.179 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-pl.ko 2024-05-11T11:59:13.179 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-plantronics.ko 2024-05-11T11:59:13.179 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-primax.ko 2024-05-11T11:59:13.179 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-rmi.ko 2024-05-11T11:59:13.179 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-11T11:59:13.179 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-common.ko 2024-05-11T11:59:13.179 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-11T11:59:13.179 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-11T11:59:13.179 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-11T11:59:13.179 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-11T11:59:13.179 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-11T11:59:13.179 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-11T11:59:13.179 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-11T11:59:13.180 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-11T11:59:13.180 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-11T11:59:13.180 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat.ko 2024-05-11T11:59:13.180 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-saitek.ko 2024-05-11T11:59:13.180 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-samsung.ko 2024-05-11T11:59:13.180 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-11T11:59:13.180 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-sjoy.ko 2024-05-11T11:59:13.180 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-sony.ko 2024-05-11T11:59:13.180 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-speedlink.ko 2024-05-11T11:59:13.180 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-steelseries.ko 2024-05-11T11:59:13.180 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-sunplus.ko 2024-05-11T11:59:13.180 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-11T11:59:13.180 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-tivo.ko 2024-05-11T11:59:13.180 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-tmff.ko 2024-05-11T11:59:13.181 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-topseed.ko 2024-05-11T11:59:13.181 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-twinhan.ko 2024-05-11T11:59:13.181 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-uclogic.ko 2024-05-11T11:59:13.181 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-waltop.ko 2024-05-11T11:59:13.181 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-wiimote.ko 2024-05-11T11:59:13.181 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-xinmo.ko 2024-05-11T11:59:13.181 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-zpff.ko 2024-05-11T11:59:13.181 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-zydacron.ko 2024-05-11T11:59:13.181 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/uhid.ko 2024-05-11T11:59:13.181 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/wacom.ko 2024-05-11T11:59:13.181 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hv 2024-05-11T11:59:13.182 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hv/hv_balloon.ko 2024-05-11T11:59:13.182 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hv/hv_utils.ko 2024-05-11T11:59:13.182 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hv/hv_vmbus.ko 2024-05-11T11:59:13.182 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon 2024-05-11T11:59:13.182 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/abituguru.ko 2024-05-11T11:59:13.182 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/abituguru3.ko 2024-05-11T11:59:13.183 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-11T11:59:13.183 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ad7414.ko 2024-05-11T11:59:13.183 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ad7418.ko 2024-05-11T11:59:13.183 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adc128d818.ko 2024-05-11T11:59:13.183 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm1025.ko 2024-05-11T11:59:13.183 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm1026.ko 2024-05-11T11:59:13.183 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm1029.ko 2024-05-11T11:59:13.183 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm1031.ko 2024-05-11T11:59:13.183 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm9240.ko 2024-05-11T11:59:13.183 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ads7828.ko 2024-05-11T11:59:13.183 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7410.ko 2024-05-11T11:59:13.183 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7411.ko 2024-05-11T11:59:13.183 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7462.ko 2024-05-11T11:59:13.183 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7470.ko 2024-05-11T11:59:13.184 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7475.ko 2024-05-11T11:59:13.184 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7x10.ko 2024-05-11T11:59:13.184 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/amc6821.ko 2024-05-11T11:59:13.184 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/applesmc.ko 2024-05-11T11:59:13.184 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/asb100.ko 2024-05-11T11:59:13.184 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/asc7621.ko 2024-05-11T11:59:13.184 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-11T11:59:13.184 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/atxp1.ko 2024-05-11T11:59:13.184 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/coretemp.ko 2024-05-11T11:59:13.184 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-11T11:59:13.184 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/dme1737.ko 2024-05-11T11:59:13.184 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ds1621.ko 2024-05-11T11:59:13.184 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ds620.ko 2024-05-11T11:59:13.184 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/emc1403.ko 2024-05-11T11:59:13.184 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/emc6w201.ko 2024-05-11T11:59:13.185 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/f71805f.ko 2024-05-11T11:59:13.185 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/f71882fg.ko 2024-05-11T11:59:13.185 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/f75375s.ko 2024-05-11T11:59:13.185 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/fam15h_power.ko 2024-05-11T11:59:13.185 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/fschmd.ko 2024-05-11T11:59:13.185 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/g760a.ko 2024-05-11T11:59:13.185 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/g762.ko 2024-05-11T11:59:13.185 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/gl518sm.ko 2024-05-11T11:59:13.185 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/gl520sm.ko 2024-05-11T11:59:13.185 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-11T11:59:13.185 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/i5500_temp.ko 2024-05-11T11:59:13.185 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/i5k_amb.ko 2024-05-11T11:59:13.185 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ibmaem.ko 2024-05-11T11:59:13.185 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ibmpex.ko 2024-05-11T11:59:13.186 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ina209.ko 2024-05-11T11:59:13.186 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ina2xx.ko 2024-05-11T11:59:13.186 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/it87.ko 2024-05-11T11:59:13.186 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/k10temp.ko 2024-05-11T11:59:13.186 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/k8temp.ko 2024-05-11T11:59:13.186 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lineage-pem.ko 2024-05-11T11:59:13.186 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm63.ko 2024-05-11T11:59:13.186 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm73.ko 2024-05-11T11:59:13.186 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm75.ko 2024-05-11T11:59:13.186 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm77.ko 2024-05-11T11:59:13.186 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm78.ko 2024-05-11T11:59:13.186 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm80.ko 2024-05-11T11:59:13.186 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm83.ko 2024-05-11T11:59:13.186 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm85.ko 2024-05-11T11:59:13.187 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm87.ko 2024-05-11T11:59:13.187 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm90.ko 2024-05-11T11:59:13.187 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm92.ko 2024-05-11T11:59:13.187 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm93.ko 2024-05-11T11:59:13.187 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm95234.ko 2024-05-11T11:59:13.187 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm95241.ko 2024-05-11T11:59:13.187 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm95245.ko 2024-05-11T11:59:13.187 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc2945.ko 2024-05-11T11:59:13.187 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4151.ko 2024-05-11T11:59:13.187 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4215.ko 2024-05-11T11:59:13.187 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4222.ko 2024-05-11T11:59:13.187 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4245.ko 2024-05-11T11:59:13.188 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4260.ko 2024-05-11T11:59:13.188 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4261.ko 2024-05-11T11:59:13.188 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max16065.ko 2024-05-11T11:59:13.188 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max1619.ko 2024-05-11T11:59:13.188 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max1668.ko 2024-05-11T11:59:13.188 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max197.ko 2024-05-11T11:59:13.188 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max6639.ko 2024-05-11T11:59:13.188 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max6650.ko 2024-05-11T11:59:13.188 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max6697.ko 2024-05-11T11:59:13.188 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/mcp3021.ko 2024-05-11T11:59:13.188 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/nct6683.ko 2024-05-11T11:59:13.188 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/nct6775-core.ko 2024-05-11T11:59:13.188 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/nct6775.ko 2024-05-11T11:59:13.188 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/nct7802.ko 2024-05-11T11:59:13.189 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pc87360.ko 2024-05-11T11:59:13.189 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pc87427.ko 2024-05-11T11:59:13.189 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pcf8591.ko 2024-05-11T11:59:13.189 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus 2024-05-11T11:59:13.189 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-11T11:59:13.189 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-11T11:59:13.189 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-11T11:59:13.189 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-11T11:59:13.189 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-11T11:59:13.189 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-11T11:59:13.189 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-11T11:59:13.189 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-11T11:59:13.189 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-11T11:59:13.189 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-11T11:59:13.190 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-11T11:59:13.190 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-11T11:59:13.190 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/powr1220.ko 2024-05-11T11:59:13.190 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sch5627.ko 2024-05-11T11:59:13.190 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sch5636.ko 2024-05-11T11:59:13.190 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-11T11:59:13.190 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sht15.ko 2024-05-11T11:59:13.190 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sht21.ko 2024-05-11T11:59:13.190 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/shtc1.ko 2024-05-11T11:59:13.190 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sis5595.ko 2024-05-11T11:59:13.190 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/smsc47b397.ko 2024-05-11T11:59:13.190 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/smsc47m1.ko 2024-05-11T11:59:13.190 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/smsc47m192.ko 2024-05-11T11:59:13.190 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/thmc50.ko 2024-05-11T11:59:13.190 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/tmp102.ko 2024-05-11T11:59:13.191 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/tmp103.ko 2024-05-11T11:59:13.191 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/tmp401.ko 2024-05-11T11:59:13.191 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/tmp421.ko 2024-05-11T11:59:13.191 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/via-cputemp.ko 2024-05-11T11:59:13.191 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/via686a.ko 2024-05-11T11:59:13.191 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/vt1211.ko 2024-05-11T11:59:13.191 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/vt8231.ko 2024-05-11T11:59:13.191 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83627ehf.ko 2024-05-11T11:59:13.191 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83627hf.ko 2024-05-11T11:59:13.191 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83781d.ko 2024-05-11T11:59:13.191 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83791d.ko 2024-05-11T11:59:13.191 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83792d.ko 2024-05-11T11:59:13.191 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83793.ko 2024-05-11T11:59:13.192 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83795.ko 2024-05-11T11:59:13.192 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83l785ts.ko 2024-05-11T11:59:13.192 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83l786ng.ko 2024-05-11T11:59:13.192 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c 2024-05-11T11:59:13.192 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/algos 2024-05-11T11:59:13.192 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-11T11:59:13.192 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-11T11:59:13.192 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses 2024-05-11T11:59:13.192 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-11T11:59:13.192 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-11T11:59:13.192 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-11T11:59:13.192 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-11T11:59:13.192 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-11T11:59:13.192 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-11T11:59:13.193 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-11T11:59:13.193 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-11T11:59:13.193 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-11T11:59:13.193 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-11T11:59:13.193 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-11T11:59:13.193 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-11T11:59:13.193 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-11T11:59:13.193 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-11T11:59:13.193 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-11T11:59:13.193 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-11T11:59:13.193 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-11T11:59:13.193 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-11T11:59:13.193 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-11T11:59:13.193 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-11T11:59:13.194 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-11T11:59:13.194 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-11T11:59:13.194 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-11T11:59:13.194 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-dev.ko 2024-05-11T11:59:13.194 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-mux.ko 2024-05-11T11:59:13.194 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-05-11T11:59:13.194 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-smbus.ko 2024-05-11T11:59:13.194 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-stub.ko 2024-05-11T11:59:13.194 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband 2024-05-11T11:59:13.194 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core 2024-05-11T11:59:13.194 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-11T11:59:13.195 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/ib_core.ko 2024-05-11T11:59:13.195 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-11T11:59:13.195 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-11T11:59:13.195 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-11T11:59:13.195 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-11T11:59:13.195 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-11T11:59:13.195 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw 2024-05-11T11:59:13.195 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/cxgb4 2024-05-11T11:59:13.195 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-11T11:59:13.195 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mlx4 2024-05-11T11:59:13.195 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-11T11:59:13.195 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mlx5 2024-05-11T11:59:13.195 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-11T11:59:13.195 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mthca 2024-05-11T11:59:13.196 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-11T11:59:13.196 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp 2024-05-11T11:59:13.196 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/ipoib 2024-05-11T11:59:13.196 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-11T11:59:13.196 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/iser 2024-05-11T11:59:13.196 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-11T11:59:13.196 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/isert 2024-05-11T11:59:13.196 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-11T11:59:13.196 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/srp 2024-05-11T11:59:13.196 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-11T11:59:13.196 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/srpt 2024-05-11T11:59:13.196 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-11T11:59:13.196 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input 2024-05-11T11:59:13.197 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/gameport 2024-05-11T11:59:13.197 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-11T11:59:13.197 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-11T11:59:13.197 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/gameport/gameport.ko 2024-05-11T11:59:13.197 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/joydev.ko 2024-05-11T11:59:13.197 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard 2024-05-11T11:59:13.197 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-11T11:59:13.197 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse 2024-05-11T11:59:13.197 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/appletouch.ko 2024-05-11T11:59:13.197 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/bcm5974.ko 2024-05-11T11:59:13.197 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/cyapatp.ko 2024-05-11T11:59:13.197 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-11T11:59:13.197 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/sermouse.ko 2024-05-11T11:59:13.197 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-11T11:59:13.198 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-11T11:59:13.198 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-11T11:59:13.198 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/rmi4 2024-05-11T11:59:13.198 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-11T11:59:13.198 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio 2024-05-11T11:59:13.198 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/altera_ps2.ko 2024-05-11T11:59:13.198 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/arc_ps2.ko 2024-05-11T11:59:13.198 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-11T11:59:13.198 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/serio_raw.ko 2024-05-11T11:59:13.198 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/sparse-keymap.ko 2024-05-11T11:59:13.198 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds 2024-05-11T11:59:13.198 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-blinkm.ko 2024-05-11T11:59:13.198 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-lm3530.ko 2024-05-11T11:59:13.198 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-lp3944.ko 2024-05-11T11:59:13.199 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-ss4200.ko 2024-05-11T11:59:13.199 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger 2024-05-11T11:59:13.199 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-11T11:59:13.199 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-11T11:59:13.199 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-11T11:59:13.199 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-11T11:59:13.199 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-11T11:59:13.199 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-11T11:59:13.199 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-11T11:59:13.199 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-11T11:59:13.199 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-11T11:59:13.199 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md 2024-05-11T11:59:13.199 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/bcache 2024-05-11T11:59:13.199 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/bcache/bcache.ko 2024-05-11T11:59:13.200 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-bio-prison.ko 2024-05-11T11:59:13.200 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-cache-smq.ko 2024-05-11T11:59:13.200 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-cache.ko 2024-05-11T11:59:13.200 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-crypt.ko 2024-05-11T11:59:13.200 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-delay.ko 2024-05-11T11:59:13.200 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-flakey.ko 2024-05-11T11:59:13.200 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-log-userspace.ko 2024-05-11T11:59:13.200 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-multipath.ko 2024-05-11T11:59:13.200 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-queue-length.ko 2024-05-11T11:59:13.200 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-raid.ko 2024-05-11T11:59:13.200 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-round-robin.ko 2024-05-11T11:59:13.200 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-service-time.ko 2024-05-11T11:59:13.200 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-switch.ko 2024-05-11T11:59:13.200 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-thin-pool.ko 2024-05-11T11:59:13.200 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-verity.ko 2024-05-11T11:59:13.201 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/persistent-data 2024-05-11T11:59:13.201 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-11T11:59:13.201 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/raid0.ko 2024-05-11T11:59:13.201 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/raid1.ko 2024-05-11T11:59:13.201 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/raid10.ko 2024-05-11T11:59:13.201 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/raid456.ko 2024-05-11T11:59:13.201 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message 2024-05-11T11:59:13.201 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion 2024-05-11T11:59:13.201 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptbase.ko 2024-05-11T11:59:13.201 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptctl.ko 2024-05-11T11:59:13.201 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptfc.ko 2024-05-11T11:59:13.201 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptlan.ko 2024-05-11T11:59:13.201 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptsas.ko 2024-05-11T11:59:13.201 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptscsih.ko 2024-05-11T11:59:13.202 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptspi.ko 2024-05-11T11:59:13.202 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd 2024-05-11T11:59:13.202 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/lpc_ich.ko 2024-05-11T11:59:13.202 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/lpc_sch.ko 2024-05-11T11:59:13.202 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/mfd-core.ko 2024-05-11T11:59:13.202 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/sm501.ko 2024-05-11T11:59:13.202 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/viperboard.ko 2024-05-11T11:59:13.202 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/vx855.ko 2024-05-11T11:59:13.202 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/wl1273-core.ko 2024-05-11T11:59:13.202 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc 2024-05-11T11:59:13.202 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/altera-stapl 2024-05-11T11:59:13.202 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-11T11:59:13.202 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/apds9802als.ko 2024-05-11T11:59:13.203 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/apds990x.ko 2024-05-11T11:59:13.203 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/bh1770glc.ko 2024-05-11T11:59:13.203 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/cb710 2024-05-11T11:59:13.203 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/cb710/cb710.ko 2024-05-11T11:59:13.203 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/eeprom 2024-05-11T11:59:13.203 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/eeprom/at24.ko 2024-05-11T11:59:13.203 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-11T11:59:13.203 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/eeprom/max6875.ko 2024-05-11T11:59:13.203 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/enclosure.ko 2024-05-11T11:59:13.203 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/hpilo.ko 2024-05-11T11:59:13.203 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/isl29003.ko 2024-05-11T11:59:13.203 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/isl29020.ko 2024-05-11T11:59:13.203 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/lis3lv02d 2024-05-11T11:59:13.203 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-11T11:59:13.204 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-11T11:59:13.204 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/mei 2024-05-11T11:59:13.204 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/mei/mei-me.ko 2024-05-11T11:59:13.204 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/mei/mei-txe.ko 2024-05-11T11:59:13.204 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/mei/mei.ko 2024-05-11T11:59:13.204 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-gru 2024-05-11T11:59:13.204 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-gru/gru.ko 2024-05-11T11:59:13.204 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-xp 2024-05-11T11:59:13.204 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-xp/xp.ko 2024-05-11T11:59:13.204 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-xp/xpc.ko 2024-05-11T11:59:13.204 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-05-11T11:59:13.204 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/tifm_7xx1.ko 2024-05-11T11:59:13.204 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/tifm_core.ko 2024-05-11T11:59:13.204 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/tsl2550.ko 2024-05-11T11:59:13.205 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/vmw_balloon.ko 2024-05-11T11:59:13.205 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/vmw_vmci 2024-05-11T11:59:13.205 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-11T11:59:13.205 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net 2024-05-11T11:59:13.205 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/bonding 2024-05-11T11:59:13.205 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/bonding/bonding.ko 2024-05-11T11:59:13.205 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa 2024-05-11T11:59:13.205 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/b53 2024-05-11T11:59:13.205 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-11T11:59:13.205 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-11T11:59:13.205 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-11T11:59:13.205 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/mv88e6xxx 2024-05-11T11:59:13.205 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-11T11:59:13.206 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dummy.ko 2024-05-11T11:59:13.206 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/eql.ko 2024-05-11T11:59:13.206 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet 2024-05-11T11:59:13.206 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com 2024-05-11T11:59:13.206 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-11T11:59:13.206 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-11T11:59:13.206 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-11T11:59:13.206 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-11T11:59:13.206 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390 2024-05-11T11:59:13.206 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-11T11:59:13.206 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-11T11:59:13.206 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-11T11:59:13.206 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-11T11:59:13.206 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/adaptec 2024-05-11T11:59:13.207 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-11T11:59:13.207 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/agere 2024-05-11T11:59:13.207 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-11T11:59:13.207 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/alteon 2024-05-11T11:59:13.207 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-11T11:59:13.207 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/altera 2024-05-11T11:59:13.207 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-11T11:59:13.207 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/amd 2024-05-11T11:59:13.207 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-11T11:59:13.207 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-11T11:59:13.207 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-11T11:59:13.207 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros 2024-05-11T11:59:13.207 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/alx 2024-05-11T11:59:13.207 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-11T11:59:13.208 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atl1c 2024-05-11T11:59:13.208 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-11T11:59:13.208 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atl1e 2024-05-11T11:59:13.208 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-11T11:59:13.208 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atlx 2024-05-11T11:59:13.208 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-11T11:59:13.208 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-11T11:59:13.208 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom 2024-05-11T11:59:13.208 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-11T11:59:13.209 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-11T11:59:13.209 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-05-11T11:59:13.209 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-11T11:59:13.209 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-11T11:59:13.209 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/genet 2024-05-11T11:59:13.209 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-11T11:59:13.209 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-11T11:59:13.209 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/brocade 2024-05-11T11:59:13.209 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/brocade/bna 2024-05-11T11:59:13.209 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-11T11:59:13.209 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio 2024-05-11T11:59:13.209 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb 2024-05-11T11:59:13.210 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-11T11:59:13.210 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-05-11T11:59:13.210 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-11T11:59:13.210 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-05-11T11:59:13.210 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-11T11:59:13.210 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-05-11T11:59:13.210 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-11T11:59:13.210 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-05-11T11:59:13.210 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-11T11:59:13.210 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/cisco 2024-05-11T11:59:13.210 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/cisco/enic 2024-05-11T11:59:13.210 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-11T11:59:13.211 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec 2024-05-11T11:59:13.211 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip 2024-05-11T11:59:13.211 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-11T11:59:13.211 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-11T11:59:13.211 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-11T11:59:13.211 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-11T11:59:13.211 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-11T11:59:13.211 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-11T11:59:13.211 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dlink 2024-05-11T11:59:13.211 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-11T11:59:13.211 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-11T11:59:13.211 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dnet.ko 2024-05-11T11:59:13.211 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/emulex 2024-05-11T11:59:13.211 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/emulex/benet 2024-05-11T11:59:13.212 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-11T11:59:13.212 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/ethoc.ko 2024-05-11T11:59:13.212 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/fealnx.ko 2024-05-11T11:59:13.212 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel 2024-05-11T11:59:13.212 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-11T11:59:13.212 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e1000 2024-05-11T11:59:13.212 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-11T11:59:13.212 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e1000e 2024-05-11T11:59:13.212 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-11T11:59:13.212 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/fm10k 2024-05-11T11:59:13.212 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-11T11:59:13.212 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/i40e 2024-05-11T11:59:13.212 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-11T11:59:13.212 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/iavf 2024-05-11T11:59:13.213 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-11T11:59:13.213 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/igb 2024-05-11T11:59:13.213 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-11T11:59:13.213 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/igbvf 2024-05-11T11:59:13.213 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-11T11:59:13.213 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/ixgbe 2024-05-11T11:59:13.213 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-11T11:59:13.213 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/ixgbevf 2024-05-11T11:59:13.213 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-11T11:59:13.213 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/jme.ko 2024-05-11T11:59:13.213 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/marvell 2024-05-11T11:59:13.213 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-11T11:59:13.213 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-11T11:59:13.214 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-11T11:59:13.214 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox 2024-05-11T11:59:13.214 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx4 2024-05-11T11:59:13.214 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-11T11:59:13.214 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-11T11:59:13.214 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx5 2024-05-11T11:59:13.214 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-05-11T11:59:13.214 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-11T11:59:13.214 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/micrel 2024-05-11T11:59:13.214 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-11T11:59:13.214 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/myricom 2024-05-11T11:59:13.215 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/myricom/myri10ge 2024-05-11T11:59:13.215 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-11T11:59:13.215 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/natsemi 2024-05-11T11:59:13.215 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-11T11:59:13.215 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-11T11:59:13.215 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/neterion 2024-05-11T11:59:13.215 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-11T11:59:13.215 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/nvidia 2024-05-11T11:59:13.215 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-11T11:59:13.215 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/packetengines 2024-05-11T11:59:13.215 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-11T11:59:13.215 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-11T11:59:13.215 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic 2024-05-11T11:59:13.216 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/netxen 2024-05-11T11:59:13.216 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-11T11:59:13.216 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-11T11:59:13.216 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-05-11T11:59:13.216 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-11T11:59:13.216 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/rdc 2024-05-11T11:59:13.216 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-11T11:59:13.216 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek 2024-05-11T11:59:13.216 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-11T11:59:13.216 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-11T11:59:13.216 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-11T11:59:13.216 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-11T11:59:13.216 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/rocker 2024-05-11T11:59:13.216 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-05-11T11:59:13.217 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sfc 2024-05-11T11:59:13.217 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-11T11:59:13.217 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/silan 2024-05-11T11:59:13.217 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-11T11:59:13.217 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sis 2024-05-11T11:59:13.217 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-11T11:59:13.217 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-11T11:59:13.217 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc 2024-05-11T11:59:13.217 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-11T11:59:13.217 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-11T11:59:13.217 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-11T11:59:13.217 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-11T11:59:13.217 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/stmicro 2024-05-11T11:59:13.217 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/stmicro/stmmac 2024-05-11T11:59:13.218 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-11T11:59:13.218 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-11T11:59:13.218 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun 2024-05-11T11:59:13.218 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-11T11:59:13.218 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-11T11:59:13.218 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-11T11:59:13.218 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-11T11:59:13.218 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/tehuti 2024-05-11T11:59:13.218 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-11T11:59:13.218 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/ti 2024-05-11T11:59:13.218 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-11T11:59:13.218 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/via 2024-05-11T11:59:13.218 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-11T11:59:13.219 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-11T11:59:13.219 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/wiznet 2024-05-11T11:59:13.219 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-11T11:59:13.219 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-11T11:59:13.219 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/xircom 2024-05-11T11:59:13.219 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-11T11:59:13.219 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/geneve.ko 2024-05-11T11:59:13.219 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/hyperv 2024-05-11T11:59:13.219 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-11T11:59:13.219 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ieee802154 2024-05-11T11:59:13.219 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-11T11:59:13.219 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ifb.ko 2024-05-11T11:59:13.219 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ipvlan 2024-05-11T11:59:13.219 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-11T11:59:13.219 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/macvlan.ko 2024-05-11T11:59:13.220 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/macvtap.ko 2024-05-11T11:59:13.220 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio 2024-05-11T11:59:13.220 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio.ko 2024-05-11T11:59:13.220 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-11T11:59:13.220 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-11T11:59:13.220 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-11T11:59:13.220 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mii.ko 2024-05-11T11:59:13.220 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/net_failover.ko 2024-05-11T11:59:13.220 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/netconsole.ko 2024-05-11T11:59:13.220 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/nlmon.ko 2024-05-11T11:59:13.220 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/pcs 2024-05-11T11:59:13.220 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-11T11:59:13.220 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-11T11:59:13.220 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy 2024-05-11T11:59:13.220 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/amd.ko 2024-05-11T11:59:13.220 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-11T11:59:13.221 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-05-11T11:59:13.221 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-11T11:59:13.221 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/bcm87xx.ko 2024-05-11T11:59:13.221 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/broadcom.ko 2024-05-11T11:59:13.221 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/cicada.ko 2024-05-11T11:59:13.221 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/davicom.ko 2024-05-11T11:59:13.221 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/dp83640.ko 2024-05-11T11:59:13.221 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/et1011c.ko 2024-05-11T11:59:13.221 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/icplus.ko 2024-05-11T11:59:13.221 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/lxt.ko 2024-05-11T11:59:13.221 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/marvell.ko 2024-05-11T11:59:13.221 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/micrel.ko 2024-05-11T11:59:13.221 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/national.ko 2024-05-11T11:59:13.221 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/phylink.ko 2024-05-11T11:59:13.221 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/qsemi.ko 2024-05-11T11:59:13.222 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/realtek.ko 2024-05-11T11:59:13.222 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/smsc.ko 2024-05-11T11:59:13.222 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/ste10Xp.ko 2024-05-11T11:59:13.222 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/vitesse.ko 2024-05-11T11:59:13.222 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp 2024-05-11T11:59:13.222 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-11T11:59:13.222 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_async.ko 2024-05-11T11:59:13.222 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-11T11:59:13.222 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_generic.ko 2024-05-11T11:59:13.222 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-11T11:59:13.222 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-11T11:59:13.222 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/pppoe.ko 2024-05-11T11:59:13.223 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/pppox.ko 2024-05-11T11:59:13.223 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/pptp.ko 2024-05-11T11:59:13.223 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/slip 2024-05-11T11:59:13.223 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/slip/slhc.ko 2024-05-11T11:59:13.223 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/slip/slip.ko 2024-05-11T11:59:13.223 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/sungem_phy.ko 2024-05-11T11:59:13.223 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/tap.ko 2024-05-11T11:59:13.223 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team 2024-05-11T11:59:13.224 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team.ko 2024-05-11T11:59:13.224 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-11T11:59:13.224 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-11T11:59:13.224 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-11T11:59:13.224 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_random.ko 2024-05-11T11:59:13.224 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-11T11:59:13.224 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/tun.ko 2024-05-11T11:59:13.224 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/veth.ko 2024-05-11T11:59:13.225 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/virtio_net.ko 2024-05-11T11:59:13.225 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/vmxnet3 2024-05-11T11:59:13.225 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-11T11:59:13.225 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/vxlan 2024-05-11T11:59:13.225 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/vxlan/vxlan.ko 2024-05-11T11:59:13.225 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/xen-netback 2024-05-11T11:59:13.225 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-11T11:59:13.225 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/xen-netfront.ko 2024-05-11T11:59:13.225 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ntb 2024-05-11T11:59:13.226 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ntb/ntb.ko 2024-05-11T11:59:13.226 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/nvme 2024-05-11T11:59:13.226 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/nvme/host 2024-05-11T11:59:13.226 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/nvme/host/nvme-core.ko 2024-05-11T11:59:13.226 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/nvme/host/nvme.ko 2024-05-11T11:59:13.226 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport 2024-05-11T11:59:13.226 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport/parport.ko 2024-05-11T11:59:13.226 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport/parport_cs.ko 2024-05-11T11:59:13.227 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport/parport_pc.ko 2024-05-11T11:59:13.227 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport/parport_serial.ko 2024-05-11T11:59:13.227 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci 2024-05-11T11:59:13.227 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/hotplug 2024-05-11T11:59:13.227 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-11T11:59:13.228 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/pcie 2024-05-11T11:59:13.228 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/pcie/aer_inject.ko 2024-05-11T11:59:13.228 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/xen-pcifront.ko 2024-05-11T11:59:13.228 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia 2024-05-11T11:59:13.228 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia/i82092.ko 2024-05-11T11:59:13.228 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia/pd6729.ko 2024-05-11T11:59:13.229 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-11T11:59:13.229 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pinctrl 2024-05-11T11:59:13.229 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pinctrl/intel 2024-05-11T11:59:13.229 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-11T11:59:13.229 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform 2024-05-11T11:59:13.229 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86 2024-05-11T11:59:13.229 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-11T11:59:13.229 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/acerhdf.ko 2024-05-11T11:59:13.230 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-11T11:59:13.230 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-11T11:59:13.230 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-11T11:59:13.230 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-11T11:59:13.230 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-11T11:59:13.230 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-11T11:59:13.230 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-11T11:59:13.230 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-11T11:59:13.231 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-11T11:59:13.231 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/hdaps.ko 2024-05-11T11:59:13.231 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/intel 2024-05-11T11:59:13.231 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-11T11:59:13.231 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/intel_ips.ko 2024-05-11T11:59:13.231 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-11T11:59:13.231 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-11T11:59:13.232 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-11T11:59:13.232 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-11T11:59:13.232 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-11T11:59:13.232 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-11T11:59:13.232 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-11T11:59:13.232 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-11T11:59:13.232 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-11T11:59:13.233 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/wmi.ko 2024-05-11T11:59:13.233 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/powercap 2024-05-11T11:59:13.233 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-11T11:59:13.233 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-11T11:59:13.233 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps 2024-05-11T11:59:13.233 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/clients 2024-05-11T11:59:13.234 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-11T11:59:13.234 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-11T11:59:13.234 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/clients/pps_parport.ko 2024-05-11T11:59:13.234 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/pps_core.ko 2024-05-11T11:59:13.234 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ptp 2024-05-11T11:59:13.234 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ptp/ptp.ko 2024-05-11T11:59:13.234 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ptp/ptp_kvm.ko 2024-05-11T11:59:13.234 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras 2024-05-11T11:59:13.234 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras/amd 2024-05-11T11:59:13.234 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras/amd/atl 2024-05-11T11:59:13.235 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-11T11:59:13.235 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras/amd/fmpm.ko 2024-05-11T11:59:13.235 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc 2024-05-11T11:59:13.235 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-11T11:59:13.235 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-11T11:59:13.235 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-11T11:59:13.235 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-11T11:59:13.235 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-11T11:59:13.235 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-11T11:59:13.235 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-11T11:59:13.236 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-11T11:59:13.236 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-11T11:59:13.236 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-11T11:59:13.236 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-em3027.ko 2024-05-11T11:59:13.236 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-11T11:59:13.236 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-11T11:59:13.236 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-11T11:59:13.236 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-11T11:59:13.236 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-11T11:59:13.236 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-11T11:59:13.236 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-max6900.ko 2024-05-11T11:59:13.236 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-11T11:59:13.236 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-11T11:59:13.236 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-11T11:59:13.237 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-11T11:59:13.237 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-11T11:59:13.237 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-11T11:59:13.237 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-11T11:59:13.237 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-11T11:59:13.237 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-11T11:59:13.237 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-11T11:59:13.237 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-11T11:59:13.237 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-11T11:59:13.237 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-x1205.ko 2024-05-11T11:59:13.237 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi 2024-05-11T11:59:13.237 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/3w-9xxx.ko 2024-05-11T11:59:13.237 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/3w-sas.ko 2024-05-11T11:59:13.237 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/3w-xxxx.ko 2024-05-11T11:59:13.237 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/BusLogic.ko 2024-05-11T11:59:13.238 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/a100u2w.ko 2024-05-11T11:59:13.238 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aacraid 2024-05-11T11:59:13.238 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-11T11:59:13.238 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/advansys.ko 2024-05-11T11:59:13.238 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aic7xxx 2024-05-11T11:59:13.238 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-11T11:59:13.238 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-11T11:59:13.238 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/am53c974.ko 2024-05-11T11:59:13.238 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/arcmsr 2024-05-11T11:59:13.238 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-11T11:59:13.239 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/atp870u.ko 2024-05-11T11:59:13.239 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/be2iscsi 2024-05-11T11:59:13.239 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-11T11:59:13.239 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bfa 2024-05-11T11:59:13.239 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bfa/bfa.ko 2024-05-11T11:59:13.239 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bnx2fc 2024-05-11T11:59:13.239 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-11T11:59:13.239 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bnx2i 2024-05-11T11:59:13.239 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-11T11:59:13.239 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/ch.ko 2024-05-11T11:59:13.239 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/csiostor 2024-05-11T11:59:13.239 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-11T11:59:13.239 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi 2024-05-11T11:59:13.239 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/cxgb3i 2024-05-11T11:59:13.240 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-11T11:59:13.240 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/cxgb4i 2024-05-11T11:59:13.240 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-11T11:59:13.240 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-11T11:59:13.240 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/dc395x.ko 2024-05-11T11:59:13.240 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/device_handler 2024-05-11T11:59:13.240 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-05-11T11:59:13.240 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-05-11T11:59:13.240 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-05-11T11:59:13.240 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-05-11T11:59:13.240 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/dmx3191d.ko 2024-05-11T11:59:13.240 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/esas2r 2024-05-11T11:59:13.240 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-11T11:59:13.240 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/esp_scsi.ko 2024-05-11T11:59:13.240 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fcoe 2024-05-11T11:59:13.241 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-11T11:59:13.241 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-11T11:59:13.241 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fnic 2024-05-11T11:59:13.241 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fnic/fnic.ko 2024-05-11T11:59:13.241 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/hpsa.ko 2024-05-11T11:59:13.241 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/hptiop.ko 2024-05-11T11:59:13.241 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/hv_storvsc.ko 2024-05-11T11:59:13.241 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/initio.ko 2024-05-11T11:59:13.241 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/ipr.ko 2024-05-11T11:59:13.241 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/ips.ko 2024-05-11T11:59:13.241 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/isci 2024-05-11T11:59:13.241 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/isci/isci.ko 2024-05-11T11:59:13.241 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-11T11:59:13.241 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-11T11:59:13.241 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libfc 2024-05-11T11:59:13.241 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libfc/libfc.ko 2024-05-11T11:59:13.242 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libiscsi.ko 2024-05-11T11:59:13.242 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-11T11:59:13.242 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libsas 2024-05-11T11:59:13.242 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libsas/libsas.ko 2024-05-11T11:59:13.242 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/lpfc 2024-05-11T11:59:13.242 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-11T11:59:13.242 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid 2024-05-11T11:59:13.242 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid.ko 2024-05-11T11:59:13.242 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-11T11:59:13.242 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-11T11:59:13.242 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-11T11:59:13.242 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mpt3sas 2024-05-11T11:59:13.242 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-11T11:59:13.243 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mvsas 2024-05-11T11:59:13.243 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-11T11:59:13.243 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mvumi.ko 2024-05-11T11:59:13.243 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/pm8001 2024-05-11T11:59:13.243 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-11T11:59:13.243 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/pmcraid.ko 2024-05-11T11:59:13.243 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla1280.ko 2024-05-11T11:59:13.243 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla2xxx 2024-05-11T11:59:13.243 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-11T11:59:13.243 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-11T11:59:13.243 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla4xxx 2024-05-11T11:59:13.243 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-11T11:59:13.244 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/raid_class.ko 2024-05-11T11:59:13.244 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_debug.ko 2024-05-11T11:59:13.244 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-11T11:59:13.244 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-11T11:59:13.244 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-11T11:59:13.244 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-11T11:59:13.244 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-11T11:59:13.244 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/ses.ko 2024-05-11T11:59:13.245 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/st.ko 2024-05-11T11:59:13.245 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/stex.ko 2024-05-11T11:59:13.245 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/sym53c8xx_2 2024-05-11T11:59:13.245 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-11T11:59:13.245 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/virtio_scsi.ko 2024-05-11T11:59:13.245 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-11T11:59:13.245 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/wd719x.ko 2024-05-11T11:59:13.245 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ssb 2024-05-11T11:59:13.245 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ssb/ssb.ko 2024-05-11T11:59:13.245 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target 2024-05-11T11:59:13.245 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/iscsi 2024-05-11T11:59:13.245 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-11T11:59:13.245 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/loopback 2024-05-11T11:59:13.245 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-11T11:59:13.245 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_file.ko 2024-05-11T11:59:13.246 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_iblock.ko 2024-05-11T11:59:13.246 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_mod.ko 2024-05-11T11:59:13.246 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_pscsi.ko 2024-05-11T11:59:13.246 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_user.ko 2024-05-11T11:59:13.246 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/tcm_fc 2024-05-11T11:59:13.246 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-11T11:59:13.246 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal 2024-05-11T11:59:13.246 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel 2024-05-11T11:59:13.246 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal 2024-05-11T11:59:13.246 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-11T11:59:13.246 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-11T11:59:13.246 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-11T11:59:13.246 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-11T11:59:13.247 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-11T11:59:13.247 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-11T11:59:13.247 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-11T11:59:13.247 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-11T11:59:13.247 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-11T11:59:13.247 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-11T11:59:13.247 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-11T11:59:13.247 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-11T11:59:13.247 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-11T11:59:13.247 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-11T11:59:13.247 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-11T11:59:13.247 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-11T11:59:13.247 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-11T11:59:13.247 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-11T11:59:13.248 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty 2024-05-11T11:59:13.248 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/ipwireless 2024-05-11T11:59:13.248 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-11T11:59:13.248 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/n_gsm.ko 2024-05-11T11:59:13.248 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/n_hdlc.ko 2024-05-11T11:59:13.248 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/nozomi.ko 2024-05-11T11:59:13.248 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial 2024-05-11T11:59:13.248 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/8250 2024-05-11T11:59:13.248 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-11T11:59:13.248 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/arc_uart.ko 2024-05-11T11:59:13.248 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/jsm 2024-05-11T11:59:13.248 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-11T11:59:13.248 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs 2024-05-11T11:59:13.248 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs/core 2024-05-11T11:59:13.248 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-11T11:59:13.249 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs/host 2024-05-11T11:59:13.249 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-11T11:59:13.249 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio 2024-05-11T11:59:13.249 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio.ko 2024-05-11T11:59:13.249 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_aec.ko 2024-05-11T11:59:13.249 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_cif.ko 2024-05-11T11:59:13.249 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_pci_generic.ko 2024-05-11T11:59:13.249 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_sercos3.ko 2024-05-11T11:59:13.249 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb 2024-05-11T11:59:13.249 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class 2024-05-11T11:59:13.249 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class/cdc-acm.ko 2024-05-11T11:59:13.249 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-11T11:59:13.249 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class/usblp.ko 2024-05-11T11:59:13.249 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class/usbtmc.ko 2024-05-11T11:59:13.250 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/host 2024-05-11T11:59:13.250 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-11T11:59:13.250 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/image 2024-05-11T11:59:13.250 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/image/mdc800.ko 2024-05-11T11:59:13.250 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/image/microtek.ko 2024-05-11T11:59:13.250 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc 2024-05-11T11:59:13.250 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/adutux.ko 2024-05-11T11:59:13.250 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/appledisplay.ko 2024-05-11T11:59:13.250 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/emi26.ko 2024-05-11T11:59:13.250 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/emi62.ko 2024-05-11T11:59:13.250 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/ezusb.ko 2024-05-11T11:59:13.251 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/idmouse.ko 2024-05-11T11:59:13.251 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/iowarrior.ko 2024-05-11T11:59:13.251 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-11T11:59:13.251 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/ldusb.ko 2024-05-11T11:59:13.251 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/legousbtower.ko 2024-05-11T11:59:13.251 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/sisusbvga 2024-05-11T11:59:13.251 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-11T11:59:13.251 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-11T11:59:13.251 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/usb3503.ko 2024-05-11T11:59:13.251 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/usblcd.ko 2024-05-11T11:59:13.251 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-11T11:59:13.251 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/uss720.ko 2024-05-11T11:59:13.251 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/yurex.ko 2024-05-11T11:59:13.251 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/phy 2024-05-11T11:59:13.252 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/phy/phy-generic.ko 2024-05-11T11:59:13.252 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial 2024-05-11T11:59:13.252 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/aircable.ko 2024-05-11T11:59:13.252 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ark3116.ko 2024-05-11T11:59:13.252 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-11T11:59:13.252 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ch341.ko 2024-05-11T11:59:13.252 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/cp210x.ko 2024-05-11T11:59:13.252 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/cyberjack.ko 2024-05-11T11:59:13.252 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-11T11:59:13.252 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-11T11:59:13.252 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/empeg.ko 2024-05-11T11:59:13.252 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-11T11:59:13.252 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-11T11:59:13.253 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-11T11:59:13.253 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/io_ti.ko 2024-05-11T11:59:13.253 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ipaq.ko 2024-05-11T11:59:13.253 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ipw.ko 2024-05-11T11:59:13.253 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ir-usb.ko 2024-05-11T11:59:13.253 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-11T11:59:13.253 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/keyspan.ko 2024-05-11T11:59:13.253 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-11T11:59:13.253 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-11T11:59:13.253 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-11T11:59:13.253 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/mct_u232.ko 2024-05-11T11:59:13.253 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/mos7720.ko 2024-05-11T11:59:13.253 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/mos7840.ko 2024-05-11T11:59:13.254 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/navman.ko 2024-05-11T11:59:13.254 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/omninet.ko 2024-05-11T11:59:13.254 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/opticon.ko 2024-05-11T11:59:13.254 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/option.ko 2024-05-11T11:59:13.254 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/oti6858.ko 2024-05-11T11:59:13.254 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/pl2303.ko 2024-05-11T11:59:13.254 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/qcaux.ko 2024-05-11T11:59:13.254 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/qcserial.ko 2024-05-11T11:59:13.254 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/quatech2.ko 2024-05-11T11:59:13.254 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/safe_serial.ko 2024-05-11T11:59:13.255 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/sierra.ko 2024-05-11T11:59:13.255 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-11T11:59:13.255 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ssu100.ko 2024-05-11T11:59:13.255 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/symbolserial.ko 2024-05-11T11:59:13.255 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-11T11:59:13.255 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-11T11:59:13.255 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/usb_debug.ko 2024-05-11T11:59:13.255 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-11T11:59:13.255 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/visor.ko 2024-05-11T11:59:13.255 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/whiteheat.ko 2024-05-11T11:59:13.255 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-11T11:59:13.255 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage 2024-05-11T11:59:13.255 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/uas.ko 2024-05-11T11:59:13.255 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-11T11:59:13.256 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-11T11:59:13.256 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-11T11:59:13.256 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-11T11:59:13.256 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-11T11:59:13.256 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-11T11:59:13.256 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-11T11:59:13.256 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-karma.ko 2024-05-11T11:59:13.256 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-11T11:59:13.256 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-11T11:59:13.256 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-11T11:59:13.256 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-11T11:59:13.256 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-11T11:59:13.256 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/usb-storage.ko 2024-05-11T11:59:13.256 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/usbip 2024-05-11T11:59:13.257 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-11T11:59:13.257 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-11T11:59:13.257 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-11T11:59:13.257 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio 2024-05-11T11:59:13.257 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/pci 2024-05-11T11:59:13.257 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-11T11:59:13.257 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-11T11:59:13.257 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/vfio.ko 2024-05-11T11:59:13.258 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-11T11:59:13.258 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost 2024-05-11T11:59:13.258 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost/vhost.ko 2024-05-11T11:59:13.258 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-11T11:59:13.258 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost/vhost_net.ko 2024-05-11T11:59:13.258 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost/vhost_scsi.ko 2024-05-11T11:59:13.258 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video 2024-05-11T11:59:13.258 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight 2024-05-11T11:59:13.258 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/apple_bl.ko 2024-05-11T11:59:13.258 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/lcd.ko 2024-05-11T11:59:13.258 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-11T11:59:13.258 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio 2024-05-11T11:59:13.258 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio.ko 2024-05-11T11:59:13.258 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_balloon.ko 2024-05-11T11:59:13.259 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_mmio.ko 2024-05-11T11:59:13.259 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_pci.ko 2024-05-11T11:59:13.259 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-05-11T11:59:13.259 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-05-11T11:59:13.259 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_ring.ko 2024-05-11T11:59:13.259 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1 2024-05-11T11:59:13.259 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/masters 2024-05-11T11:59:13.259 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/masters/ds2482.ko 2024-05-11T11:59:13.260 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/masters/ds2490.ko 2024-05-11T11:59:13.260 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves 2024-05-11T11:59:13.260 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-11T11:59:13.260 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-11T11:59:13.260 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-11T11:59:13.260 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-11T11:59:13.261 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-11T11:59:13.261 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-11T11:59:13.261 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-11T11:59:13.261 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-11T11:59:13.261 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-11T11:59:13.261 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-11T11:59:13.261 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-11T11:59:13.262 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/wire.ko 2024-05-11T11:59:13.262 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog 2024-05-11T11:59:13.262 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-11T11:59:13.262 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-11T11:59:13.262 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-11T11:59:13.262 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/hpwdt.ko 2024-05-11T11:59:13.262 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/i6300esb.ko 2024-05-11T11:59:13.262 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-11T11:59:13.262 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-11T11:59:13.262 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/ib700wdt.ko 2024-05-11T11:59:13.263 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/ibmasr.ko 2024-05-11T11:59:13.263 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-11T11:59:13.263 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-11T11:59:13.263 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/it87_wdt.ko 2024-05-11T11:59:13.263 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/machzwd.ko 2024-05-11T11:59:13.263 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/nv_tco.ko 2024-05-11T11:59:13.263 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-11T11:59:13.263 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-11T11:59:13.264 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-11T11:59:13.264 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-11T11:59:13.264 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/softdog.ko 2024-05-11T11:59:13.264 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-11T11:59:13.264 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/via_wdt.ko 2024-05-11T11:59:13.264 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-11T11:59:13.264 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-11T11:59:13.264 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-11T11:59:13.264 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/wdt_pci.ko 2024-05-11T11:59:13.264 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/xen_wdt.ko 2024-05-11T11:59:13.264 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen 2024-05-11T11:59:13.265 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-acpi-processor.ko 2024-05-11T11:59:13.265 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-evtchn.ko 2024-05-11T11:59:13.265 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-gntalloc.ko 2024-05-11T11:59:13.265 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-gntdev.ko 2024-05-11T11:59:13.265 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-pciback 2024-05-11T11:59:13.265 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-11T11:59:13.265 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-privcmd.ko 2024-05-11T11:59:13.265 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xenfs 2024-05-11T11:59:13.265 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-11T11:59:13.265 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs 2024-05-11T11:59:13.266 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/9p 2024-05-11T11:59:13.266 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/9p/9p.ko 2024-05-11T11:59:13.266 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/affs 2024-05-11T11:59:13.266 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/affs/affs.ko 2024-05-11T11:59:13.266 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/befs 2024-05-11T11:59:13.266 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/befs/befs.ko 2024-05-11T11:59:13.266 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/binfmt_misc.ko 2024-05-11T11:59:13.266 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/btrfs 2024-05-11T11:59:13.266 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/btrfs/btrfs.ko 2024-05-11T11:59:13.266 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/cachefiles 2024-05-11T11:59:13.266 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/cachefiles/cachefiles.ko 2024-05-11T11:59:13.266 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ceph 2024-05-11T11:59:13.266 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ceph/ceph.ko 2024-05-11T11:59:13.266 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/coda 2024-05-11T11:59:13.267 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/coda/coda.ko 2024-05-11T11:59:13.267 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/cramfs 2024-05-11T11:59:13.267 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/cramfs/cramfs.ko 2024-05-11T11:59:13.267 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/dlm 2024-05-11T11:59:13.267 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/dlm/dlm.ko 2024-05-11T11:59:13.267 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ecryptfs 2024-05-11T11:59:13.267 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ecryptfs/ecryptfs.ko 2024-05-11T11:59:13.267 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/f2fs 2024-05-11T11:59:13.267 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/f2fs/f2fs.ko 2024-05-11T11:59:13.267 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fat 2024-05-11T11:59:13.267 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fat/fat.ko 2024-05-11T11:59:13.267 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fat/msdos.ko 2024-05-11T11:59:13.267 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fat/vfat.ko 2024-05-11T11:59:13.267 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fuse 2024-05-11T11:59:13.267 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fuse/cuse.ko 2024-05-11T11:59:13.268 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fuse/fuse.ko 2024-05-11T11:59:13.268 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/gfs2 2024-05-11T11:59:13.268 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/gfs2/gfs2.ko 2024-05-11T11:59:13.268 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/hfs 2024-05-11T11:59:13.268 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/hfs/hfs.ko 2024-05-11T11:59:13.268 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/hfsplus 2024-05-11T11:59:13.268 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/hfsplus/hfsplus.ko 2024-05-11T11:59:13.268 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/isofs 2024-05-11T11:59:13.268 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/isofs/isofs.ko 2024-05-11T11:59:13.268 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/jfs 2024-05-11T11:59:13.268 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/jfs/jfs.ko 2024-05-11T11:59:13.268 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/lockd 2024-05-11T11:59:13.268 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/lockd/lockd.ko 2024-05-11T11:59:13.268 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/minix 2024-05-11T11:59:13.269 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/minix/minix.ko 2024-05-11T11:59:13.269 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/netfs 2024-05-11T11:59:13.269 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/netfs/netfs.ko 2024-05-11T11:59:13.269 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs 2024-05-11T11:59:13.269 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/blocklayout 2024-05-11T11:59:13.269 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-11T11:59:13.269 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/filelayout 2024-05-11T11:59:13.269 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-11T11:59:13.269 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/flexfilelayout 2024-05-11T11:59:13.269 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-11T11:59:13.269 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/nfs.ko 2024-05-11T11:59:13.269 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/nfsv3.ko 2024-05-11T11:59:13.269 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/nfsv4.ko 2024-05-11T11:59:13.269 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs_common 2024-05-11T11:59:13.269 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs_common/grace.ko 2024-05-11T11:59:13.270 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs_common/nfs_acl.ko 2024-05-11T11:59:13.270 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfsd 2024-05-11T11:59:13.270 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfsd/nfsd.ko 2024-05-11T11:59:13.270 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nilfs2 2024-05-11T11:59:13.270 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nilfs2/nilfs2.ko 2024-05-11T11:59:13.270 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls 2024-05-11T11:59:13.270 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-celtic.ko 2024-05-11T11:59:13.270 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-centeuro.ko 2024-05-11T11:59:13.270 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-croatian.ko 2024-05-11T11:59:13.271 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-cyrillic.ko 2024-05-11T11:59:13.271 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-gaelic.ko 2024-05-11T11:59:13.271 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-greek.ko 2024-05-11T11:59:13.271 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-iceland.ko 2024-05-11T11:59:13.271 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-inuit.ko 2024-05-11T11:59:13.271 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-roman.ko 2024-05-11T11:59:13.271 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-romanian.ko 2024-05-11T11:59:13.271 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-turkish.ko 2024-05-11T11:59:13.271 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp1250.ko 2024-05-11T11:59:13.271 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp1251.ko 2024-05-11T11:59:13.271 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp1255.ko 2024-05-11T11:59:13.272 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp737.ko 2024-05-11T11:59:13.272 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp775.ko 2024-05-11T11:59:13.272 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp850.ko 2024-05-11T11:59:13.272 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp852.ko 2024-05-11T11:59:13.272 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp855.ko 2024-05-11T11:59:13.272 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp857.ko 2024-05-11T11:59:13.272 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp860.ko 2024-05-11T11:59:13.272 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp861.ko 2024-05-11T11:59:13.272 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp862.ko 2024-05-11T11:59:13.272 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp863.ko 2024-05-11T11:59:13.272 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp864.ko 2024-05-11T11:59:13.272 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp865.ko 2024-05-11T11:59:13.272 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp866.ko 2024-05-11T11:59:13.272 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp869.ko 2024-05-11T11:59:13.272 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp874.ko 2024-05-11T11:59:13.273 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp932.ko 2024-05-11T11:59:13.273 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp936.ko 2024-05-11T11:59:13.273 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp949.ko 2024-05-11T11:59:13.273 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp950.ko 2024-05-11T11:59:13.273 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_euc-jp.ko 2024-05-11T11:59:13.273 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-1.ko 2024-05-11T11:59:13.273 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-13.ko 2024-05-11T11:59:13.273 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-14.ko 2024-05-11T11:59:13.273 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-15.ko 2024-05-11T11:59:13.273 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-2.ko 2024-05-11T11:59:13.273 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-3.ko 2024-05-11T11:59:13.273 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-4.ko 2024-05-11T11:59:13.273 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-5.ko 2024-05-11T11:59:13.273 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-6.ko 2024-05-11T11:59:13.273 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-7.ko 2024-05-11T11:59:13.274 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-9.ko 2024-05-11T11:59:13.274 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_koi8-r.ko 2024-05-11T11:59:13.274 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_koi8-ru.ko 2024-05-11T11:59:13.274 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_koi8-u.ko 2024-05-11T11:59:13.274 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-11T11:59:13.274 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_utf8.ko 2024-05-11T11:59:13.274 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2 2024-05-11T11:59:13.274 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/cluster 2024-05-11T11:59:13.274 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-11T11:59:13.274 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/dlm 2024-05-11T11:59:13.275 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-11T11:59:13.275 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/dlmfs 2024-05-11T11:59:13.275 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-11T11:59:13.275 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/ocfs2.ko 2024-05-11T11:59:13.275 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-11T11:59:13.275 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-11T11:59:13.275 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-11T11:59:13.275 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/overlayfs 2024-05-11T11:59:13.275 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/overlayfs/overlay.ko 2024-05-11T11:59:13.275 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/pstore 2024-05-11T11:59:13.275 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/pstore/ramoops.ko 2024-05-11T11:59:13.275 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/reiserfs 2024-05-11T11:59:13.275 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/reiserfs/reiserfs.ko 2024-05-11T11:59:13.275 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/romfs 2024-05-11T11:59:13.275 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/romfs/romfs.ko 2024-05-11T11:59:13.276 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb 2024-05-11T11:59:13.276 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/client 2024-05-11T11:59:13.276 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/client/cifs.ko 2024-05-11T11:59:13.276 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/common 2024-05-11T11:59:13.276 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/common/cifs_arc4.ko 2024-05-11T11:59:13.276 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/common/cifs_md4.ko 2024-05-11T11:59:13.276 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/squashfs 2024-05-11T11:59:13.276 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/squashfs/squashfs.ko 2024-05-11T11:59:13.276 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/sysv 2024-05-11T11:59:13.276 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/sysv/sysv.ko 2024-05-11T11:59:13.276 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/udf 2024-05-11T11:59:13.276 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/udf/udf.ko 2024-05-11T11:59:13.276 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ufs 2024-05-11T11:59:13.276 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ufs/ufs.ko 2024-05-11T11:59:13.276 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/xfs 2024-05-11T11:59:13.277 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/xfs/xfs.ko 2024-05-11T11:59:13.277 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel 2024-05-11T11:59:13.277 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/rcu 2024-05-11T11:59:13.277 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/rcu/rcutorture.ko 2024-05-11T11:59:13.277 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/torture.ko 2024-05-11T11:59:13.277 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/trace 2024-05-11T11:59:13.277 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-05-11T11:59:13.277 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib 2024-05-11T11:59:13.277 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/asn1_encoder.ko 2024-05-11T11:59:13.277 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crc-itu-t.ko 2024-05-11T11:59:13.277 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crc8.ko 2024-05-11T11:59:13.277 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crypto 2024-05-11T11:59:13.277 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crypto/libarc4.ko 2024-05-11T11:59:13.277 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crypto/libdes.ko 2024-05-11T11:59:13.277 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/libcrc32c.ko 2024-05-11T11:59:13.278 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/lru_cache.ko 2024-05-11T11:59:13.278 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/lz4 2024-05-11T11:59:13.278 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/lz4/lz4_compress.ko 2024-05-11T11:59:13.278 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/lz4/lz4hc_compress.ko 2024-05-11T11:59:13.278 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/math 2024-05-11T11:59:13.278 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/math/cordic.ko 2024-05-11T11:59:13.278 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/raid6 2024-05-11T11:59:13.278 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/raid6/raid6_pq.ko 2024-05-11T11:59:13.278 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/reed_solomon 2024-05-11T11:59:13.278 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-11T11:59:13.278 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/ts_bm.ko 2024-05-11T11:59:13.278 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/ts_fsm.ko 2024-05-11T11:59:13.278 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/ts_kmp.ko 2024-05-11T11:59:13.278 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/zstd 2024-05-11T11:59:13.279 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/zstd/zstd_compress.ko 2024-05-11T11:59:13.279 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/mm 2024-05-11T11:59:13.279 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/mm/hwpoison-inject.ko 2024-05-11T11:59:13.279 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net 2024-05-11T11:59:13.279 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan 2024-05-11T11:59:13.279 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/6lowpan.ko 2024-05-11T11:59:13.279 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_dest.ko 2024-05-11T11:59:13.279 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_fragment.ko 2024-05-11T11:59:13.279 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_hop.ko 2024-05-11T11:59:13.279 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-11T11:59:13.279 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_mobility.ko 2024-05-11T11:59:13.279 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_routing.ko 2024-05-11T11:59:13.279 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_udp.ko 2024-05-11T11:59:13.279 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802 2024-05-11T11:59:13.280 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/garp.ko 2024-05-11T11:59:13.280 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/mrp.ko 2024-05-11T11:59:13.280 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/p8022.ko 2024-05-11T11:59:13.280 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/psnap.ko 2024-05-11T11:59:13.280 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/stp.ko 2024-05-11T11:59:13.280 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/8021q 2024-05-11T11:59:13.280 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/8021q/8021q.ko 2024-05-11T11:59:13.280 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p 2024-05-11T11:59:13.280 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p/9pnet.ko 2024-05-11T11:59:13.280 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p/9pnet_fd.ko 2024-05-11T11:59:13.280 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p/9pnet_rdma.ko 2024-05-11T11:59:13.280 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p/9pnet_virtio.ko 2024-05-11T11:59:13.280 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/batman-adv 2024-05-11T11:59:13.280 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/batman-adv/batman-adv.ko 2024-05-11T11:59:13.280 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge 2024-05-11T11:59:13.281 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/br_netfilter.ko 2024-05-11T11:59:13.281 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/bridge.ko 2024-05-11T11:59:13.281 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter 2024-05-11T11:59:13.281 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-11T11:59:13.281 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-11T11:59:13.281 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-11T11:59:13.281 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-11T11:59:13.281 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-11T11:59:13.281 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-11T11:59:13.281 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-11T11:59:13.281 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-11T11:59:13.281 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-11T11:59:13.281 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-11T11:59:13.281 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-11T11:59:13.281 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-11T11:59:13.282 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-11T11:59:13.282 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-11T11:59:13.282 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-11T11:59:13.282 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-11T11:59:13.282 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-11T11:59:13.282 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-11T11:59:13.282 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-11T11:59:13.282 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-11T11:59:13.282 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebtables.ko 2024-05-11T11:59:13.282 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-05-11T11:59:13.282 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-11T11:59:13.282 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-11T11:59:13.283 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ceph 2024-05-11T11:59:13.283 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ceph/libceph.ko 2024-05-11T11:59:13.283 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/core 2024-05-11T11:59:13.283 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/core/failover.ko 2024-05-11T11:59:13.283 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/core/pktgen.ko 2024-05-11T11:59:13.283 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp 2024-05-11T11:59:13.283 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp/dccp.ko 2024-05-11T11:59:13.283 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp/dccp_diag.ko 2024-05-11T11:59:13.283 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp/dccp_ipv4.ko 2024-05-11T11:59:13.283 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp/dccp_ipv6.ko 2024-05-11T11:59:13.283 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dns_resolver 2024-05-11T11:59:13.283 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dns_resolver/dns_resolver.ko 2024-05-11T11:59:13.283 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa 2024-05-11T11:59:13.283 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/dsa_core.ko 2024-05-11T11:59:13.283 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/tag_brcm.ko 2024-05-11T11:59:13.284 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/tag_dsa.ko 2024-05-11T11:59:13.284 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/tag_none.ko 2024-05-11T11:59:13.284 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/tag_trailer.ko 2024-05-11T11:59:13.284 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154 2024-05-11T11:59:13.284 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154/6lowpan 2024-05-11T11:59:13.284 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-11T11:59:13.284 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154/ieee802154.ko 2024-05-11T11:59:13.284 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-11T11:59:13.284 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4 2024-05-11T11:59:13.284 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ah4.ko 2024-05-11T11:59:13.284 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/esp4.ko 2024-05-11T11:59:13.284 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/fou.ko 2024-05-11T11:59:13.284 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/gre.ko 2024-05-11T11:59:13.284 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/inet_diag.ko 2024-05-11T11:59:13.284 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ip_gre.ko 2024-05-11T11:59:13.285 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ip_tunnel.ko 2024-05-11T11:59:13.285 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ip_vti.ko 2024-05-11T11:59:13.285 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ipcomp.ko 2024-05-11T11:59:13.285 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ipip.ko 2024-05-11T11:59:13.285 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter 2024-05-11T11:59:13.285 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-11T11:59:13.285 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-11T11:59:13.285 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-11T11:59:13.285 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-11T11:59:13.285 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-11T11:59:13.285 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-11T11:59:13.285 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-11T11:59:13.285 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-11T11:59:13.285 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-11T11:59:13.285 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-11T11:59:13.286 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-11T11:59:13.286 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-11T11:59:13.286 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-11T11:59:13.286 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-11T11:59:13.286 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-11T11:59:13.286 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-11T11:59:13.286 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-11T11:59:13.286 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-11T11:59:13.286 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-11T11:59:13.286 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-11T11:59:13.286 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-11T11:59:13.286 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_bic.ko 2024-05-11T11:59:13.286 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_dctcp.ko 2024-05-11T11:59:13.287 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_diag.ko 2024-05-11T11:59:13.287 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_highspeed.ko 2024-05-11T11:59:13.287 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_htcp.ko 2024-05-11T11:59:13.287 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_hybla.ko 2024-05-11T11:59:13.287 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_illinois.ko 2024-05-11T11:59:13.287 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_lp.ko 2024-05-11T11:59:13.287 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_scalable.ko 2024-05-11T11:59:13.287 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_vegas.ko 2024-05-11T11:59:13.287 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_veno.ko 2024-05-11T11:59:13.287 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_westwood.ko 2024-05-11T11:59:13.287 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_yeah.ko 2024-05-11T11:59:13.287 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tunnel4.ko 2024-05-11T11:59:13.287 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/udp_diag.ko 2024-05-11T11:59:13.287 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/udp_tunnel.ko 2024-05-11T11:59:13.288 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-11T11:59:13.288 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6 2024-05-11T11:59:13.288 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ah6.ko 2024-05-11T11:59:13.288 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/esp6.ko 2024-05-11T11:59:13.288 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/fou6.ko 2024-05-11T11:59:13.288 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ip6_tunnel.ko 2024-05-11T11:59:13.288 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-11T11:59:13.288 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ip6_vti.ko 2024-05-11T11:59:13.288 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ipcomp6.ko 2024-05-11T11:59:13.288 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter 2024-05-11T11:59:13.288 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-11T11:59:13.289 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-11T11:59:13.289 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-11T11:59:13.289 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-11T11:59:13.289 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-11T11:59:13.289 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-11T11:59:13.289 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-11T11:59:13.289 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-11T11:59:13.289 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-11T11:59:13.289 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-11T11:59:13.289 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-11T11:59:13.289 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-11T11:59:13.290 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-11T11:59:13.290 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-11T11:59:13.290 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-11T11:59:13.290 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-11T11:59:13.290 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-11T11:59:13.290 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-11T11:59:13.290 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-11T11:59:13.290 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-11T11:59:13.290 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-11T11:59:13.290 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-11T11:59:13.290 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-11T11:59:13.290 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-11T11:59:13.290 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/sit.ko 2024-05-11T11:59:13.290 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/tunnel6.ko 2024-05-11T11:59:13.291 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-11T11:59:13.291 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/key 2024-05-11T11:59:13.291 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/key/af_key.ko 2024-05-11T11:59:13.291 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp 2024-05-11T11:59:13.291 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_core.ko 2024-05-11T11:59:13.291 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-11T11:59:13.291 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_eth.ko 2024-05-11T11:59:13.291 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_ip.ko 2024-05-11T11:59:13.291 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_ip6.ko 2024-05-11T11:59:13.291 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_netlink.ko 2024-05-11T11:59:13.291 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_ppp.ko 2024-05-11T11:59:13.292 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/llc 2024-05-11T11:59:13.292 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/llc/llc.ko 2024-05-11T11:59:13.292 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/mac802154 2024-05-11T11:59:13.292 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/mac802154/mac802154.ko 2024-05-11T11:59:13.292 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/mpls 2024-05-11T11:59:13.292 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/mpls/mpls_gso.ko 2024-05-11T11:59:13.292 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter 2024-05-11T11:59:13.292 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset 2024-05-11T11:59:13.292 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set.ko 2024-05-11T11:59:13.292 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-11T11:59:13.292 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-11T11:59:13.292 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-11T11:59:13.292 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-11T11:59:13.293 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-11T11:59:13.293 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-11T11:59:13.293 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-11T11:59:13.293 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-11T11:59:13.293 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-11T11:59:13.293 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-11T11:59:13.293 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-11T11:59:13.293 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-11T11:59:13.293 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-11T11:59:13.293 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-11T11:59:13.294 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-11T11:59:13.294 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs 2024-05-11T11:59:13.294 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-11T11:59:13.294 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-11T11:59:13.294 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-11T11:59:13.294 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-11T11:59:13.294 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-11T11:59:13.294 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-11T11:59:13.294 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-11T11:59:13.294 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-11T11:59:13.294 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-11T11:59:13.295 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-11T11:59:13.295 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-11T11:59:13.295 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-11T11:59:13.295 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-11T11:59:13.295 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-11T11:59:13.295 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conncount.ko 2024-05-11T11:59:13.295 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack.ko 2024-05-11T11:59:13.295 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-11T11:59:13.295 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-11T11:59:13.295 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-11T11:59:13.296 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-11T11:59:13.296 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-11T11:59:13.296 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-11T11:59:13.296 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-11T11:59:13.296 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-11T11:59:13.296 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-11T11:59:13.296 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-11T11:59:13.296 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-11T11:59:13.296 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-11T11:59:13.296 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-11T11:59:13.296 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_flow_table.ko 2024-05-11T11:59:13.296 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-11T11:59:13.296 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_log_syslog.ko 2024-05-11T11:59:13.297 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat.ko 2024-05-11T11:59:13.297 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-11T11:59:13.297 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-11T11:59:13.297 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_irc.ko 2024-05-11T11:59:13.297 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_sip.ko 2024-05-11T11:59:13.297 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-11T11:59:13.297 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-11T11:59:13.297 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_tables.ko 2024-05-11T11:59:13.297 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink.ko 2024-05-11T11:59:13.297 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-11T11:59:13.297 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink_log.ko 2024-05-11T11:59:13.297 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-11T11:59:13.297 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-11T11:59:13.298 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_chain_nat.ko 2024-05-11T11:59:13.298 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_compat.ko 2024-05-11T11:59:13.298 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_ct.ko 2024-05-11T11:59:13.298 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-11T11:59:13.298 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_fib.ko 2024-05-11T11:59:13.298 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-11T11:59:13.298 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_hash.ko 2024-05-11T11:59:13.298 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_limit.ko 2024-05-11T11:59:13.298 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_log.ko 2024-05-11T11:59:13.298 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_masq.ko 2024-05-11T11:59:13.298 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_nat.ko 2024-05-11T11:59:13.298 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_numgen.ko 2024-05-11T11:59:13.298 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_queue.ko 2024-05-11T11:59:13.299 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_quota.ko 2024-05-11T11:59:13.299 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_redir.ko 2024-05-11T11:59:13.299 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_reject.ko 2024-05-11T11:59:13.299 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_reject_inet.ko 2024-05-11T11:59:13.299 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_socket.ko 2024-05-11T11:59:13.299 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_tunnel.ko 2024-05-11T11:59:13.299 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_AUDIT.ko 2024-05-11T11:59:13.299 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-11T11:59:13.299 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-11T11:59:13.299 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-11T11:59:13.299 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_CT.ko 2024-05-11T11:59:13.299 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_DSCP.ko 2024-05-11T11:59:13.299 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_HL.ko 2024-05-11T11:59:13.299 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_HMARK.ko 2024-05-11T11:59:13.299 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-11T11:59:13.300 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_LED.ko 2024-05-11T11:59:13.300 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_LOG.ko 2024-05-11T11:59:13.300 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-11T11:59:13.300 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_NETMAP.ko 2024-05-11T11:59:13.300 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_NFLOG.ko 2024-05-11T11:59:13.300 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-11T11:59:13.300 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_RATEEST.ko 2024-05-11T11:59:13.300 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-11T11:59:13.300 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_SECMARK.ko 2024-05-11T11:59:13.300 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-11T11:59:13.300 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-11T11:59:13.300 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TEE.ko 2024-05-11T11:59:13.300 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TPROXY.ko 2024-05-11T11:59:13.300 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TRACE.ko 2024-05-11T11:59:13.300 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_addrtype.ko 2024-05-11T11:59:13.301 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_bpf.ko 2024-05-11T11:59:13.301 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_cgroup.ko 2024-05-11T11:59:13.301 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_cluster.ko 2024-05-11T11:59:13.301 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_comment.ko 2024-05-11T11:59:13.301 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_connbytes.ko 2024-05-11T11:59:13.301 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_connlabel.ko 2024-05-11T11:59:13.301 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_connlimit.ko 2024-05-11T11:59:13.301 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_connmark.ko 2024-05-11T11:59:13.301 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_conntrack.ko 2024-05-11T11:59:13.301 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_cpu.ko 2024-05-11T11:59:13.301 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_dccp.ko 2024-05-11T11:59:13.301 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_devgroup.ko 2024-05-11T11:59:13.301 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_dscp.ko 2024-05-11T11:59:13.301 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_ecn.ko 2024-05-11T11:59:13.301 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_esp.ko 2024-05-11T11:59:13.301 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_hashlimit.ko 2024-05-11T11:59:13.301 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_helper.ko 2024-05-11T11:59:13.302 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_hl.ko 2024-05-11T11:59:13.302 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_ipcomp.ko 2024-05-11T11:59:13.302 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_iprange.ko 2024-05-11T11:59:13.302 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_ipvs.ko 2024-05-11T11:59:13.302 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_l2tp.ko 2024-05-11T11:59:13.302 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_length.ko 2024-05-11T11:59:13.302 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_limit.ko 2024-05-11T11:59:13.302 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_mac.ko 2024-05-11T11:59:13.302 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_mark.ko 2024-05-11T11:59:13.302 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_multiport.ko 2024-05-11T11:59:13.302 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_nat.ko 2024-05-11T11:59:13.302 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_nfacct.ko 2024-05-11T11:59:13.303 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_osf.ko 2024-05-11T11:59:13.303 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_owner.ko 2024-05-11T11:59:13.303 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_physdev.ko 2024-05-11T11:59:13.303 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_pkttype.ko 2024-05-11T11:59:13.303 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_policy.ko 2024-05-11T11:59:13.303 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_quota.ko 2024-05-11T11:59:13.303 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_rateest.ko 2024-05-11T11:59:13.303 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_realm.ko 2024-05-11T11:59:13.303 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_recent.ko 2024-05-11T11:59:13.303 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_sctp.ko 2024-05-11T11:59:13.303 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_set.ko 2024-05-11T11:59:13.303 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_socket.ko 2024-05-11T11:59:13.303 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_state.ko 2024-05-11T11:59:13.303 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_statistic.ko 2024-05-11T11:59:13.304 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_string.ko 2024-05-11T11:59:13.304 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_tcpmss.ko 2024-05-11T11:59:13.304 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_time.ko 2024-05-11T11:59:13.304 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_u32.ko 2024-05-11T11:59:13.304 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netlink 2024-05-11T11:59:13.304 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netlink/netlink_diag.ko 2024-05-11T11:59:13.304 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/nsh 2024-05-11T11:59:13.304 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/nsh/nsh.ko 2024-05-11T11:59:13.304 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch 2024-05-11T11:59:13.304 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch/openvswitch.ko 2024-05-11T11:59:13.304 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch/vport-geneve.ko 2024-05-11T11:59:13.304 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch/vport-gre.ko 2024-05-11T11:59:13.304 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch/vport-vxlan.ko 2024-05-11T11:59:13.304 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/packet 2024-05-11T11:59:13.304 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/packet/af_packet_diag.ko 2024-05-11T11:59:13.305 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/rds 2024-05-11T11:59:13.305 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/rds/rds.ko 2024-05-11T11:59:13.305 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/rds/rds_rdma.ko 2024-05-11T11:59:13.305 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/rds/rds_tcp.ko 2024-05-11T11:59:13.305 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched 2024-05-11T11:59:13.305 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_csum.ko 2024-05-11T11:59:13.305 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_gact.ko 2024-05-11T11:59:13.305 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_mirred.ko 2024-05-11T11:59:13.305 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_nat.ko 2024-05-11T11:59:13.305 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_pedit.ko 2024-05-11T11:59:13.305 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_police.ko 2024-05-11T11:59:13.305 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_simple.ko 2024-05-11T11:59:13.305 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_skbedit.ko 2024-05-11T11:59:13.305 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_vlan.ko 2024-05-11T11:59:13.306 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_basic.ko 2024-05-11T11:59:13.306 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_bpf.ko 2024-05-11T11:59:13.306 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_flow.ko 2024-05-11T11:59:13.306 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_fw.ko 2024-05-11T11:59:13.306 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_route.ko 2024-05-11T11:59:13.306 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_u32.ko 2024-05-11T11:59:13.306 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_cmp.ko 2024-05-11T11:59:13.306 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_ipset.ko 2024-05-11T11:59:13.306 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_meta.ko 2024-05-11T11:59:13.306 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_nbyte.ko 2024-05-11T11:59:13.306 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_text.ko 2024-05-11T11:59:13.306 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_u32.ko 2024-05-11T11:59:13.306 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_choke.ko 2024-05-11T11:59:13.306 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_codel.ko 2024-05-11T11:59:13.307 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_drr.ko 2024-05-11T11:59:13.307 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_fq.ko 2024-05-11T11:59:13.307 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_gred.ko 2024-05-11T11:59:13.307 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_hfsc.ko 2024-05-11T11:59:13.307 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_hhf.ko 2024-05-11T11:59:13.307 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_htb.ko 2024-05-11T11:59:13.307 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_ingress.ko 2024-05-11T11:59:13.307 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_mqprio.ko 2024-05-11T11:59:13.307 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_mqprio_lib.ko 2024-05-11T11:59:13.307 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_multiq.ko 2024-05-11T11:59:13.307 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_netem.ko 2024-05-11T11:59:13.307 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_pie.ko 2024-05-11T11:59:13.307 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_plug.ko 2024-05-11T11:59:13.307 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_prio.ko 2024-05-11T11:59:13.308 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_qfq.ko 2024-05-11T11:59:13.308 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_red.ko 2024-05-11T11:59:13.308 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_sfb.ko 2024-05-11T11:59:13.308 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_sfq.ko 2024-05-11T11:59:13.308 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_tbf.ko 2024-05-11T11:59:13.308 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_teql.ko 2024-05-11T11:59:13.308 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sctp 2024-05-11T11:59:13.308 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sctp/sctp.ko 2024-05-11T11:59:13.308 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sctp/sctp_diag.ko 2024-05-11T11:59:13.308 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc 2024-05-11T11:59:13.308 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/auth_gss 2024-05-11T11:59:13.308 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-11T11:59:13.308 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-11T11:59:13.308 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/sunrpc.ko 2024-05-11T11:59:13.308 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/xprtrdma 2024-05-11T11:59:13.309 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-11T11:59:13.309 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/tipc 2024-05-11T11:59:13.309 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/tipc/tipc.ko 2024-05-11T11:59:13.309 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/tipc/tipc_diag.ko 2024-05-11T11:59:13.309 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/unix 2024-05-11T11:59:13.309 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/unix/unix_diag.ko 2024-05-11T11:59:13.309 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock 2024-05-11T11:59:13.309 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-11T11:59:13.309 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-11T11:59:13.309 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vsock.ko 2024-05-11T11:59:13.309 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-11T11:59:13.309 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-11T11:59:13.309 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/xfrm 2024-05-11T11:59:13.309 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-11T11:59:13.309 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security 2024-05-11T11:59:13.310 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security/keys 2024-05-11T11:59:13.310 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security/keys/encrypted-keys 2024-05-11T11:59:13.310 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-05-11T11:59:13.310 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security/keys/trusted-keys 2024-05-11T11:59:13.310 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security/keys/trusted-keys/trusted.ko 2024-05-11T11:59:13.310 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.alias 2024-05-11T11:59:13.310 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.alias.bin 2024-05-11T11:59:13.310 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.builtin 2024-05-11T11:59:13.310 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.builtin.alias.bin 2024-05-11T11:59:13.310 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.builtin.bin 2024-05-11T11:59:13.310 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.builtin.modinfo 2024-05-11T11:59:13.310 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.dep 2024-05-11T11:59:13.310 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.dep.bin 2024-05-11T11:59:13.311 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.devname 2024-05-11T11:59:13.311 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.order 2024-05-11T11:59:13.311 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.softdep 2024-05-11T11:59:13.311 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.symbols 2024-05-11T11:59:13.311 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.symbols.bin 2024-05-11T11:59:13.311 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/vmlinuz 2024-05-11T11:59:13.315 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-gfb9a6a44107e 2024-05-11T11:59:13.315 DEBUG:teuthology.orchestra.run.smithi167:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-05-11T11:59:13.338 INFO:teuthology.orchestra.run.smithi167.stdout:postinstall scriptlet (using /bin/sh): 2024-05-11T11:59:13.338 INFO:teuthology.orchestra.run.smithi167.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-11T11:59:13.338 INFO:teuthology.orchestra.run.smithi167.stdout: /usr/bin/kernel-install add 6.9.0-rc3-gfb9a6a44107e /lib/modules/6.9.0-rc3-gfb9a6a44107e/vmlinuz 2024-05-11T11:59:13.338 INFO:teuthology.orchestra.run.smithi167.stdout:fi 2024-05-11T11:59:13.338 INFO:teuthology.orchestra.run.smithi167.stdout:for file in vmlinuz System.map config; do 2024-05-11T11:59:13.338 INFO:teuthology.orchestra.run.smithi167.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-gfb9a6a44107e/${file}" "/boot/${file}-6.9.0-rc3-gfb9a6a44107e"; then 2024-05-11T11:59:13.339 INFO:teuthology.orchestra.run.smithi167.stdout: cp "/lib/modules/6.9.0-rc3-gfb9a6a44107e/${file}" "/boot/${file}-6.9.0-rc3-gfb9a6a44107e" 2024-05-11T11:59:13.339 INFO:teuthology.orchestra.run.smithi167.stdout: fi 2024-05-11T11:59:13.339 INFO:teuthology.orchestra.run.smithi167.stdout:done 2024-05-11T11:59:13.339 INFO:teuthology.orchestra.run.smithi167.stdout:if [ -d "/lib/modules/6.9.0-rc3-gfb9a6a44107e/dtb" ] && \ 2024-05-11T11:59:13.339 INFO:teuthology.orchestra.run.smithi167.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-gfb9a6a44107e/dtb" "/boot/dtb-6.9.0-rc3-gfb9a6a44107e" >/dev/null 2>&1; then 2024-05-11T11:59:13.339 INFO:teuthology.orchestra.run.smithi167.stdout: rm -rf "/boot/dtb-6.9.0-rc3-gfb9a6a44107e" 2024-05-11T11:59:13.339 INFO:teuthology.orchestra.run.smithi167.stdout: cp -r "/lib/modules/6.9.0-rc3-gfb9a6a44107e/dtb" "/boot/dtb-6.9.0-rc3-gfb9a6a44107e" 2024-05-11T11:59:13.339 INFO:teuthology.orchestra.run.smithi167.stdout:fi 2024-05-11T11:59:13.339 INFO:teuthology.orchestra.run.smithi167.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.dep" ]; then 2024-05-11T11:59:13.339 INFO:teuthology.orchestra.run.smithi167.stdout: /usr/sbin/depmod 6.9.0-rc3-gfb9a6a44107e 2024-05-11T11:59:13.339 INFO:teuthology.orchestra.run.smithi167.stdout:fi 2024-05-11T11:59:13.339 INFO:teuthology.orchestra.run.smithi167.stdout:preuninstall scriptlet (using /bin/sh): 2024-05-11T11:59:13.339 INFO:teuthology.orchestra.run.smithi167.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-11T11:59:13.340 INFO:teuthology.orchestra.run.smithi167.stdout:kernel-install remove 6.9.0-rc3-gfb9a6a44107e 2024-05-11T11:59:13.340 INFO:teuthology.orchestra.run.smithi167.stdout:fi 2024-05-11T11:59:13.340 DEBUG:teuthology.orchestra.run.smithi167:> sudo rpm -qi grub2-tools 2024-05-11T11:59:13.413 INFO:teuthology.orchestra.run.smithi167.stdout:Name : grub2-tools 2024-05-11T11:59:13.413 INFO:teuthology.orchestra.run.smithi167.stdout:Epoch : 1 2024-05-11T11:59:13.413 INFO:teuthology.orchestra.run.smithi167.stdout:Version : 2.06 2024-05-11T11:59:13.413 INFO:teuthology.orchestra.run.smithi167.stdout:Release : 68.el9 2024-05-11T11:59:13.413 INFO:teuthology.orchestra.run.smithi167.stdout:Architecture: x86_64 2024-05-11T11:59:13.413 INFO:teuthology.orchestra.run.smithi167.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-11T11:59:13.413 INFO:teuthology.orchestra.run.smithi167.stdout:Group : Unspecified 2024-05-11T11:59:13.413 INFO:teuthology.orchestra.run.smithi167.stdout:Size : 8264254 2024-05-11T11:59:13.413 INFO:teuthology.orchestra.run.smithi167.stdout:License : GPLv3+ 2024-05-11T11:59:13.414 INFO:teuthology.orchestra.run.smithi167.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-11T11:59:13.414 INFO:teuthology.orchestra.run.smithi167.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-11T11:59:13.414 INFO:teuthology.orchestra.run.smithi167.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-11T11:59:13.414 INFO:teuthology.orchestra.run.smithi167.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-11T11:59:13.414 INFO:teuthology.orchestra.run.smithi167.stdout:Packager : builder@centos.org 2024-05-11T11:59:13.414 INFO:teuthology.orchestra.run.smithi167.stdout:Vendor : CentOS 2024-05-11T11:59:13.414 INFO:teuthology.orchestra.run.smithi167.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-11T11:59:13.414 INFO:teuthology.orchestra.run.smithi167.stdout:Summary : Support tools for GRUB. 2024-05-11T11:59:13.414 INFO:teuthology.orchestra.run.smithi167.stdout:Description : 2024-05-11T11:59:13.414 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T11:59:13.414 INFO:teuthology.orchestra.run.smithi167.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-11T11:59:13.414 INFO:teuthology.orchestra.run.smithi167.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-11T11:59:13.414 INFO:teuthology.orchestra.run.smithi167.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-11T11:59:13.415 INFO:teuthology.orchestra.run.smithi167.stdout:hardware devices. 2024-05-11T11:59:13.415 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T11:59:13.415 INFO:teuthology.orchestra.run.smithi167.stdout:This subpackage provides tools for support of all platforms. 2024-05-11T11:59:13.416 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-11T11:59:13.416 INFO:teuthology.task.kernel:Updating grub on smithi167 to boot 6.9.0-rc3-gfb9a6a44107e 2024-05-11T11:59:13.416 DEBUG:teuthology.orchestra.run.smithi167:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-11T11:59:13.943 INFO:teuthology.orchestra.run.smithi167.stderr:Generating grub configuration file ... 2024-05-11T11:59:15.319 INFO:teuthology.orchestra.run.smithi167.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-11T11:59:15.347 INFO:teuthology.orchestra.run.smithi167.stderr:done 2024-05-11T11:59:15.350 DEBUG:teuthology.orchestra.run.smithi167:> mktemp 2024-05-11T11:59:15.365 INFO:teuthology.orchestra.run.smithi167.stdout:/tmp/tmp.Uat2ANab11 2024-05-11T11:59:15.365 DEBUG:teuthology.orchestra.run.smithi167:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Uat2ANab11 2024-05-11T11:59:15.446 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 0666 /tmp/tmp.Uat2ANab11 2024-05-11T11:59:15.588 DEBUG:teuthology.orchestra.remote:smithi167:/tmp/tmp.Uat2ANab11 is 6KB 2024-05-11T11:59:15.636 DEBUG:teuthology.orchestra.run.smithi167:> rm -fr /tmp/tmp.Uat2ANab11 2024-05-11T11:59:15.650 DEBUG:teuthology.orchestra.run.smithi167:> sudo /bin/ls /boot/loader/entries || true 2024-05-11T11:59:15.715 INFO:teuthology.orchestra.run.smithi167.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-11T11:59:15.715 INFO:teuthology.orchestra.run.smithi167.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-11T11:59:15.715 INFO:teuthology.orchestra.run.smithi167.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-11T11:59:15.715 INFO:teuthology.orchestra.run.smithi167.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-gfb9a6a44107e.conf 2024-05-11T11:59:15.717 DEBUG:teuthology.orchestra.run.smithi167:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-gfb9a6a44107e 2024-05-11T11:59:15.855 DEBUG:teuthology.orchestra.run.smithi167:> sudo shutdown -r now 2024-05-11T11:59:36.606 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-05-11T11:59:36.606 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-05-11T11:59:36.607 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T11:59:45.861 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-11T11:59:45.861 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-05-11T11:59:45.862 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T12:00:11.295 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.204 2024-05-11T12:00:20.304 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-05-11T12:00:20.306 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T12:00:20.510 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.167 2024-05-11T12:00:29.519 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-05-11T12:00:29.520 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T12:00:47.902 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.167 2024-05-11T12:00:59.917 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-05-11T12:00:59.918 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T12:01:02.974 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.167 2024-05-11T12:01:17.986 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-05-11T12:01:18.676 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T12:01:19.139 DEBUG:teuthology.orchestra.run.smithi167:> true 2024-05-11T12:01:19.801 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-05-11T12:01:19.801 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-gfb9a6a44107e"... 2024-05-11T12:01:19.801 DEBUG:teuthology.orchestra.run.smithi167:> uname -r 2024-05-11T12:01:19.838 INFO:teuthology.orchestra.run.smithi167.stdout:6.9.0-rc3-gfb9a6a44107e 2024-05-11T12:01:19.839 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-gfb9a6a44107e vs 6.9.0-rc3-gfb9a6a44107e 2024-05-11T12:01:19.839 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-11T12:01:19.839 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-11T12:01:20.309 DEBUG:teuthology.orchestra.remote:timed out 2024-05-11T12:01:20.840 DEBUG:teuthology.parallel:result is None 2024-05-11T12:01:32.310 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-05-11T12:01:32.311 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T12:01:32.684 DEBUG:teuthology.orchestra.run.smithi204:> true 2024-05-11T12:01:33.363 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-05-11T12:01:33.363 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.9.0-rc3-gfb9a6a44107e"... 2024-05-11T12:01:33.363 DEBUG:teuthology.orchestra.run.smithi204:> uname -r 2024-05-11T12:01:33.413 INFO:teuthology.orchestra.run.smithi204.stdout:6.9.0-rc3-gfb9a6a44107e 2024-05-11T12:01:33.413 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-gfb9a6a44107e vs 6.9.0-rc3-gfb9a6a44107e 2024-05-11T12:01:33.413 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-11T12:01:33.413 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-11T12:01:34.414 DEBUG:teuthology.parallel:result is None 2024-05-11T12:01:34.414 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-11T12:01:34.425 INFO:teuthology.task.internal:Creating test directory... 2024-05-11T12:01:34.425 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-11T12:01:34.433 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-11T12:01:34.490 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-11T12:01:34.497 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-11T12:01:34.528 INFO:teuthology.task.internal:Creating archive directory... 2024-05-11T12:01:34.529 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-11T12:01:34.534 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-11T12:01:34.601 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-11T12:01:34.608 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-11T12:01:34.608 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-11T12:01:34.615 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-11T12:01:34.742 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-11T12:01:34.758 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-11T12:01:34.802 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-11T12:01:34.827 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-11T12:01:34.832 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-11T12:01:34.840 INFO:teuthology.task.internal:Configuring sudo... 2024-05-11T12:01:34.840 DEBUG:teuthology.orchestra.run.smithi167:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-11T12:01:34.845 DEBUG:teuthology.orchestra.run.smithi204:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-11T12:01:34.935 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-11T12:01:34.944 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-11T12:01:34.944 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-11T12:01:34.951 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-11T12:01:35.003 DEBUG:teuthology.orchestra.run.smithi167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-11T12:01:35.080 DEBUG:teuthology.orchestra.run.smithi167:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-11T12:01:35.190 DEBUG:teuthology.orchestra.run.smithi167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-11T12:01:35.272 DEBUG:teuthology.orchestra.run.smithi167:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-11T12:01:35.357 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:01:35.357 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-11T12:01:35.440 DEBUG:teuthology.orchestra.run.smithi204:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-11T12:01:35.523 DEBUG:teuthology.orchestra.run.smithi204:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-11T12:01:35.622 DEBUG:teuthology.orchestra.run.smithi204:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-11T12:01:35.705 DEBUG:teuthology.orchestra.run.smithi204:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-11T12:01:35.789 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:01:35.789 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-11T12:01:35.870 DEBUG:teuthology.orchestra.run.smithi167:> sudo service rsyslog restart 2024-05-11T12:01:35.877 DEBUG:teuthology.orchestra.run.smithi204:> sudo service rsyslog restart 2024-05-11T12:01:35.989 INFO:teuthology.orchestra.run.smithi167.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-11T12:01:36.024 INFO:teuthology.orchestra.run.smithi204.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-11T12:01:36.529 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-11T12:01:36.537 INFO:teuthology.task.internal:Starting timer... 2024-05-11T12:01:36.537 INFO:teuthology.run_tasks:Running task pcp... 2024-05-11T12:01:36.589 INFO:teuthology.run_tasks:Running task selinux... 2024-05-11T12:01:36.629 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-05-11T12:01:36.630 DEBUG:teuthology.orchestra.run.smithi167:> sudo service auditd rotate 2024-05-11T12:01:36.749 INFO:teuthology.orchestra.run.smithi167.stdout:Rotating logs: 2024-05-11T12:01:36.755 DEBUG:teuthology.orchestra.run.smithi204:> sudo service auditd rotate 2024-05-11T12:01:36.875 INFO:teuthology.orchestra.run.smithi204.stdout:Rotating logs: 2024-05-11T12:01:36.879 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-11T12:01:36.879 DEBUG:teuthology.orchestra.run.smithi167:> /usr/sbin/getenforce 2024-05-11T12:01:36.944 INFO:teuthology.orchestra.run.smithi167.stdout:Permissive 2024-05-11T12:01:36.944 DEBUG:teuthology.orchestra.run.smithi204:> /usr/sbin/getenforce 2024-05-11T12:01:37.003 INFO:teuthology.orchestra.run.smithi204.stdout:Permissive 2024-05-11T12:01:37.003 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi167.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi204.front.sepia.ceph.com': 'permissive'} 2024-05-11T12:01:37.003 DEBUG:teuthology.orchestra.run.smithi167:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-11T12:01:37.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T12:01:37.073 DEBUG:teuthology.orchestra.run.smithi204:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-11T12:01:37.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T12:01:37.142 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-11T12:01:37.142 DEBUG:teuthology.orchestra.run.smithi167:> sudo /usr/sbin/setenforce permissive 2024-05-11T12:01:37.250 DEBUG:teuthology.orchestra.run.smithi204:> sudo /usr/sbin/setenforce permissive 2024-05-11T12:01:37.328 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-11T12:01:37.338 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-11T12:01:37.347 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-11T12:01:37.491 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-11T12:01:37.507 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-11T12:01:37.508 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi167.front.sepia.ceph.com,smithi204.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-11T12:07:16.265 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi167.front.sepia.ceph.com'), Remote(name='ubuntu@smithi204.front.sepia.ceph.com')] 2024-05-11T12:07:16.266 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-05-11T12:07:16.267 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T12:07:16.381 DEBUG:teuthology.orchestra.run.smithi167:> true 2024-05-11T12:07:16.540 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-05-11T12:07:16.540 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-05-11T12:07:16.541 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T12:07:16.642 DEBUG:teuthology.orchestra.run.smithi204:> true 2024-05-11T12:07:16.798 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-05-11T12:07:16.798 INFO:teuthology.run_tasks:Running task clock... 2024-05-11T12:07:16.810 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-11T12:07:16.810 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-11T12:07:16.811 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-11T12:07:16.818 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-11T12:07:16.818 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-11T12:07:16.903 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-11T12:07:16.907 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-11T12:07:16.944 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-11T12:07:16.948 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-11T12:07:17.016 INFO:teuthology.orchestra.run.smithi204.stderr:sudo: ntpd: command not found 2024-05-11T12:07:17.019 INFO:teuthology.orchestra.run.smithi167.stderr:sudo: ntpd: command not found 2024-05-11T12:07:17.047 INFO:teuthology.orchestra.run.smithi204.stdout:506 Cannot talk to daemon 2024-05-11T12:07:17.050 INFO:teuthology.orchestra.run.smithi167.stdout:506 Cannot talk to daemon 2024-05-11T12:07:17.089 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-11T12:07:17.089 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-11T12:07:17.130 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-11T12:07:17.131 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-11T12:07:17.262 INFO:teuthology.orchestra.run.smithi167.stderr:bash: line 1: ntpq: command not found 2024-05-11T12:07:17.263 INFO:teuthology.orchestra.run.smithi204.stderr:bash: line 1: ntpq: command not found 2024-05-11T12:07:17.271 INFO:teuthology.orchestra.run.smithi167.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-11T12:07:17.271 INFO:teuthology.orchestra.run.smithi167.stdout:=============================================================================== 2024-05-11T12:07:17.271 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T12:07:17.272 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T12:07:17.272 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T12:07:17.272 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T12:07:17.272 INFO:teuthology.orchestra.run.smithi204.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-11T12:07:17.272 INFO:teuthology.orchestra.run.smithi204.stdout:=============================================================================== 2024-05-11T12:07:17.272 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T12:07:17.272 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T12:07:17.272 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T12:07:17.272 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T12:07:17.274 INFO:teuthology.run_tasks:Running task install... 2024-05-11T12:07:17.283 DEBUG:teuthology.task.install:project ceph 2024-05-11T12:07:17.283 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '308506802b0bdea4df7050cfc1fcbdc04fea6a6e'}} 2024-05-11T12:07:17.283 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '308506802b0bdea4df7050cfc1fcbdc04fea6a6e'} 2024-05-11T12:07:17.283 INFO:teuthology.task.install:Using flavor: default 2024-05-11T12:07:17.288 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-11T12:07:17.289 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-11T12:07:17.289 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '308506802b0bdea4df7050cfc1fcbdc04fea6a6e', 'tag': None, 'wait_for_package': False} 2024-05-11T12:07:17.289 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=308506802b0bdea4df7050cfc1fcbdc04fea6a6e 2024-05-11T12:07:17.291 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '308506802b0bdea4df7050cfc1fcbdc04fea6a6e', 'tag': None, 'wait_for_package': False} 2024-05-11T12:07:17.292 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=308506802b0bdea4df7050cfc1fcbdc04fea6a6e 2024-05-11T12:07:17.503 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/flavors/default/ 2024-05-11T12:07:17.503 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3601.g30850680 2024-05-11T12:07:17.514 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/flavors/default/ 2024-05-11T12:07:17.514 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3601.g30850680 2024-05-11T12:07:17.615 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-11T12:07:17.615 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:07:17.615 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-11T12:07:17.681 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-11T12:07:17.681 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:07:17.681 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-11T12:07:17.706 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-11T12:07:17.707 DEBUG:teuthology.orchestra.run.smithi167:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-11T12:07:17.769 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-11T12:07:17.769 DEBUG:teuthology.orchestra.run.smithi204:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-11T12:07:17.835 DEBUG:teuthology.orchestra.run.smithi167:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-11T12:07:17.896 DEBUG:teuthology.orchestra.run.smithi204:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-11T12:07:17.946 DEBUG:teuthology.orchestra.run.smithi167:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-11T12:07:18.009 DEBUG:teuthology.orchestra.run.smithi204:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-11T12:07:18.021 INFO:teuthology.orchestra.run.smithi167.stdout:check_obsoletes = 1 2024-05-11T12:07:18.025 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum clean all 2024-05-11T12:07:18.091 INFO:teuthology.orchestra.run.smithi204.stdout:check_obsoletes = 1 2024-05-11T12:07:18.096 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum clean all 2024-05-11T12:07:18.347 INFO:teuthology.orchestra.run.smithi167.stdout:54 files removed 2024-05-11T12:07:18.387 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-11T12:07:18.425 INFO:teuthology.orchestra.run.smithi204.stdout:54 files removed 2024-05-11T12:07:18.464 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-11T12:07:19.412 INFO:teuthology.orchestra.run.smithi204.stdout:ceph packages for x86_64 164 kB/s | 84 kB 00:00 2024-05-11T12:07:19.612 INFO:teuthology.orchestra.run.smithi167.stdout:ceph packages for x86_64 106 kB/s | 84 kB 00:00 2024-05-11T12:07:20.021 INFO:teuthology.orchestra.run.smithi204.stdout:ceph noarch packages 23 kB/s | 12 kB 00:00 2024-05-11T12:07:20.237 INFO:teuthology.orchestra.run.smithi167.stdout:ceph noarch packages 22 kB/s | 12 kB 00:00 2024-05-11T12:07:20.464 INFO:teuthology.orchestra.run.smithi204.stdout:ceph source packages 5.3 kB/s | 1.9 kB 00:00 2024-05-11T12:07:20.779 INFO:teuthology.orchestra.run.smithi167.stdout:ceph source packages 4.1 kB/s | 1.9 kB 00:00 2024-05-11T12:07:21.079 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.1 MB 00:00 2024-05-11T12:07:22.086 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - BaseOS 6.6 MB/s | 8.1 MB 00:01 2024-05-11T12:07:23.239 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-05-11T12:07:27.643 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-05-11T12:07:30.182 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - Extras packages 14 kB/s | 16 kB 00:01 2024-05-11T12:07:35.575 INFO:teuthology.orchestra.run.smithi204.stdout:Extra Packages for Enterprise Linux 4.1 MB/s | 21 MB 00:05 2024-05-11T12:07:39.636 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 19 MB 00:16 2024-05-11T12:07:41.792 INFO:teuthology.orchestra.run.smithi204.stdout:lab-extras 18 kB/s | 1.7 kB 00:00 2024-05-11T12:07:44.385 INFO:teuthology.orchestra.run.smithi204.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-11T12:07:44.387 INFO:teuthology.orchestra.run.smithi204.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-11T12:07:44.407 INFO:teuthology.orchestra.run.smithi204.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-11T12:07:44.408 INFO:teuthology.orchestra.run.smithi204.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-11T12:07:44.410 INFO:teuthology.orchestra.run.smithi204.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-11T12:07:44.411 INFO:teuthology.orchestra.run.smithi204.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-11T12:07:44.414 INFO:teuthology.orchestra.run.smithi204.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-11T12:07:44.417 INFO:teuthology.orchestra.run.smithi204.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-11T12:07:44.418 INFO:teuthology.orchestra.run.smithi204.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-11T12:07:44.488 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-05-11T12:07:44.525 INFO:teuthology.orchestra.run.smithi204.stdout:======================================================================================= 2024-05-11T12:07:44.526 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repository Size 2024-05-11T12:07:44.526 INFO:teuthology.orchestra.run.smithi204.stdout:======================================================================================= 2024-05-11T12:07:44.526 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-05-11T12:07:44.526 INFO:teuthology.orchestra.run.smithi204.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-11T12:07:44.526 INFO:teuthology.orchestra.run.smithi204.stdout: ceph x86_64 2:19.0.0-3601.g30850680.el9 ceph 6.6 k 2024-05-11T12:07:44.526 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-base x86_64 2:19.0.0-3601.g30850680.el9 ceph 5.4 M 2024-05-11T12:07:44.526 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-fuse x86_64 2:19.0.0-3601.g30850680.el9 ceph 890 k 2024-05-11T12:07:44.526 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3601.g30850680.el9 ceph 145 k 2024-05-11T12:07:44.526 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr x86_64 2:19.0.0-3601.g30850680.el9 ceph 1.5 M 2024-05-11T12:07:44.526 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 142 k 2024-05-11T12:07:44.526 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 3.5 M 2024-05-11T12:07:44.527 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 7.4 M 2024-05-11T12:07:44.527 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-rook noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 49 k 2024-05-11T12:07:44.527 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-radosgw x86_64 2:19.0.0-3601.g30850680.el9 ceph 11 M 2024-05-11T12:07:44.527 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-test x86_64 2:19.0.0-3601.g30850680.el9 ceph 49 M 2024-05-11T12:07:44.527 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-volume noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 273 k 2024-05-11T12:07:44.527 INFO:teuthology.orchestra.run.smithi204.stdout: cephadm noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 764 k 2024-05-11T12:07:44.527 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-mirror x86_64 2:19.0.0-3601.g30850680.el9 ceph 225 k 2024-05-11T12:07:44.527 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-shell x86_64 2:19.0.0-3601.g30850680.el9 ceph 27 k 2024-05-11T12:07:44.527 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-top noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 18 k 2024-05-11T12:07:44.527 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-11T12:07:44.527 INFO:teuthology.orchestra.run.smithi204.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-11T12:07:44.527 INFO:teuthology.orchestra.run.smithi204.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-11T12:07:44.527 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs-devel x86_64 2:19.0.0-3601.g30850680.el9 ceph 32 k 2024-05-11T12:07:44.527 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs2 x86_64 2:19.0.0-3601.g30850680.el9 ceph 746 k 2024-05-11T12:07:44.527 INFO:teuthology.orchestra.run.smithi204.stdout: librados-devel x86_64 2:19.0.0-3601.g30850680.el9 ceph 127 k 2024-05-11T12:07:44.528 INFO:teuthology.orchestra.run.smithi204.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-11T12:07:44.528 INFO:teuthology.orchestra.run.smithi204.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-11T12:07:44.528 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-11T12:07:44.528 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-11T12:07:44.528 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-11T12:07:44.528 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-11T12:07:44.528 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cephfs x86_64 2:19.0.0-3601.g30850680.el9 ceph 161 k 2024-05-11T12:07:44.528 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rados x86_64 2:19.0.0-3601.g30850680.el9 ceph 321 k 2024-05-11T12:07:44.528 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rbd x86_64 2:19.0.0-3601.g30850680.el9 ceph 300 k 2024-05-11T12:07:44.528 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rgw x86_64 2:19.0.0-3601.g30850680.el9 ceph 100 k 2024-05-11T12:07:44.528 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-fuse x86_64 2:19.0.0-3601.g30850680.el9 ceph 86 k 2024-05-11T12:07:44.528 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-mirror x86_64 2:19.0.0-3601.g30850680.el9 ceph 3.1 M 2024-05-11T12:07:44.528 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-nbd x86_64 2:19.0.0-3601.g30850680.el9 ceph 171 k 2024-05-11T12:07:44.529 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-11T12:07:44.529 INFO:teuthology.orchestra.run.smithi204.stdout:Upgrading: 2024-05-11T12:07:44.529 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-11T12:07:44.529 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-11T12:07:44.529 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-11T12:07:44.529 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-11T12:07:44.529 INFO:teuthology.orchestra.run.smithi204.stdout: librados2 x86_64 2:19.0.0-3601.g30850680.el9 ceph 3.4 M 2024-05-11T12:07:44.529 INFO:teuthology.orchestra.run.smithi204.stdout: librbd1 x86_64 2:19.0.0-3601.g30850680.el9 ceph 3.1 M 2024-05-11T12:07:44.529 INFO:teuthology.orchestra.run.smithi204.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-11T12:07:44.529 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-11T12:07:44.529 INFO:teuthology.orchestra.run.smithi204.stdout:Installing dependencies: 2024-05-11T12:07:44.529 INFO:teuthology.orchestra.run.smithi204.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-11T12:07:44.529 INFO:teuthology.orchestra.run.smithi204.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-11T12:07:44.529 INFO:teuthology.orchestra.run.smithi204.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-11T12:07:44.529 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-common x86_64 2:19.0.0-3601.g30850680.el9 ceph 21 M 2024-05-11T12:07:44.530 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 34 k 2024-05-11T12:07:44.530 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mds x86_64 2:19.0.0-3601.g30850680.el9 ceph 2.4 M 2024-05-11T12:07:44.530 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 274 k 2024-05-11T12:07:44.530 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mon x86_64 2:19.0.0-3601.g30850680.el9 ceph 4.7 M 2024-05-11T12:07:44.530 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-osd x86_64 2:19.0.0-3601.g30850680.el9 ceph 17 M 2024-05-11T12:07:44.530 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 16 k 2024-05-11T12:07:44.530 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-selinux x86_64 2:19.0.0-3601.g30850680.el9 ceph 25 k 2024-05-11T12:07:44.530 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-11T12:07:44.530 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-11T12:07:44.530 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-11T12:07:44.530 INFO:teuthology.orchestra.run.smithi204.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-11T12:07:44.530 INFO:teuthology.orchestra.run.smithi204.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-11T12:07:44.530 INFO:teuthology.orchestra.run.smithi204.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-11T12:07:44.530 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-11T12:07:44.531 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-11T12:07:44.531 INFO:teuthology.orchestra.run.smithi204.stdout: libcephsqlite x86_64 2:19.0.0-3601.g30850680.el9 ceph 163 k 2024-05-11T12:07:44.531 INFO:teuthology.orchestra.run.smithi204.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-11T12:07:44.531 INFO:teuthology.orchestra.run.smithi204.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-11T12:07:44.531 INFO:teuthology.orchestra.run.smithi204.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-11T12:07:44.531 INFO:teuthology.orchestra.run.smithi204.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-11T12:07:44.531 INFO:teuthology.orchestra.run.smithi204.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-11T12:07:44.531 INFO:teuthology.orchestra.run.smithi204.stdout: libradosstriper1 x86_64 2:19.0.0-3601.g30850680.el9 ceph 501 k 2024-05-11T12:07:44.531 INFO:teuthology.orchestra.run.smithi204.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-11T12:07:44.531 INFO:teuthology.orchestra.run.smithi204.stdout: librgw2 x86_64 2:19.0.0-3601.g30850680.el9 ceph 5.4 M 2024-05-11T12:07:44.531 INFO:teuthology.orchestra.run.smithi204.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-11T12:07:44.531 INFO:teuthology.orchestra.run.smithi204.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-11T12:07:44.531 INFO:teuthology.orchestra.run.smithi204.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-11T12:07:44.531 INFO:teuthology.orchestra.run.smithi204.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-11T12:07:44.531 INFO:teuthology.orchestra.run.smithi204.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-11T12:07:44.532 INFO:teuthology.orchestra.run.smithi204.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-11T12:07:44.532 INFO:teuthology.orchestra.run.smithi204.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-11T12:07:44.532 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-11T12:07:44.532 INFO:teuthology.orchestra.run.smithi204.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-11T12:07:44.532 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-11T12:07:44.532 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-11T12:07:44.532 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-11T12:07:44.532 INFO:teuthology.orchestra.run.smithi204.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-11T12:07:44.532 INFO:teuthology.orchestra.run.smithi204.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-11T12:07:44.532 INFO:teuthology.orchestra.run.smithi204.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-11T12:07:44.532 INFO:teuthology.orchestra.run.smithi204.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-11T12:07:44.532 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-11T12:07:44.532 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-argparse x86_64 2:19.0.0-3601.g30850680.el9 ceph 45 k 2024-05-11T12:07:44.532 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-common x86_64 2:19.0.0-3601.g30850680.el9 ceph 135 k 2024-05-11T12:07:44.533 INFO:teuthology.orchestra.run.smithi204.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-11T12:07:44.533 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-11T12:07:44.533 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-11T12:07:44.533 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-11T12:07:44.533 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-11T12:07:44.533 INFO:teuthology.orchestra.run.smithi204.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-11T12:07:44.533 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-11T12:07:44.533 INFO:teuthology.orchestra.run.smithi204.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-11T12:07:44.533 INFO:teuthology.orchestra.run.smithi204.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-11T12:07:44.533 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-11T12:07:44.533 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-11T12:07:44.533 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-11T12:07:44.533 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-11T12:07:44.533 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-11T12:07:44.533 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-11T12:07:44.534 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-11T12:07:44.534 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-11T12:07:44.534 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-11T12:07:44.534 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-11T12:07:44.534 INFO:teuthology.orchestra.run.smithi204.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-11T12:07:44.534 INFO:teuthology.orchestra.run.smithi204.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-11T12:07:44.534 INFO:teuthology.orchestra.run.smithi204.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-11T12:07:44.534 INFO:teuthology.orchestra.run.smithi204.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-11T12:07:44.534 INFO:teuthology.orchestra.run.smithi204.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-11T12:07:44.534 INFO:teuthology.orchestra.run.smithi204.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-11T12:07:44.534 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-11T12:07:44.534 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-11T12:07:44.534 INFO:teuthology.orchestra.run.smithi204.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-11T12:07:44.534 INFO:teuthology.orchestra.run.smithi204.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-11T12:07:44.534 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-11T12:07:44.535 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-11T12:07:44.535 INFO:teuthology.orchestra.run.smithi204.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-11T12:07:44.535 INFO:teuthology.orchestra.run.smithi204.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-11T12:07:44.535 INFO:teuthology.orchestra.run.smithi204.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-11T12:07:44.535 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-11T12:07:44.535 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-11T12:07:44.535 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-11T12:07:44.535 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-11T12:07:44.535 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-11T12:07:44.535 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-11T12:07:44.535 INFO:teuthology.orchestra.run.smithi204.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-11T12:07:44.535 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-11T12:07:44.535 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-11T12:07:44.535 INFO:teuthology.orchestra.run.smithi204.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-11T12:07:44.535 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-11T12:07:44.535 INFO:teuthology.orchestra.run.smithi204.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-11T12:07:44.536 INFO:teuthology.orchestra.run.smithi204.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-11T12:07:44.536 INFO:teuthology.orchestra.run.smithi204.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-11T12:07:44.536 INFO:teuthology.orchestra.run.smithi204.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-11T12:07:44.536 INFO:teuthology.orchestra.run.smithi204.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-11T12:07:44.536 INFO:teuthology.orchestra.run.smithi204.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-11T12:07:44.536 INFO:teuthology.orchestra.run.smithi204.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-11T12:07:44.536 INFO:teuthology.orchestra.run.smithi204.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-11T12:07:44.536 INFO:teuthology.orchestra.run.smithi204.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-11T12:07:44.536 INFO:teuthology.orchestra.run.smithi204.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-11T12:07:44.536 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-11T12:07:44.536 INFO:teuthology.orchestra.run.smithi204.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-11T12:07:44.536 INFO:teuthology.orchestra.run.smithi204.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-11T12:07:44.536 INFO:teuthology.orchestra.run.smithi204.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-11T12:07:44.536 INFO:teuthology.orchestra.run.smithi204.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-11T12:07:44.536 INFO:teuthology.orchestra.run.smithi204.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-11T12:07:44.537 INFO:teuthology.orchestra.run.smithi204.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-11T12:07:44.537 INFO:teuthology.orchestra.run.smithi204.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-11T12:07:44.537 INFO:teuthology.orchestra.run.smithi204.stdout:Installing weak dependencies: 2024-05-11T12:07:44.537 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-11T12:07:44.537 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:07:44.537 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-05-11T12:07:44.537 INFO:teuthology.orchestra.run.smithi204.stdout:======================================================================================= 2024-05-11T12:07:44.537 INFO:teuthology.orchestra.run.smithi204.stdout:Install 148 Packages 2024-05-11T12:07:44.537 INFO:teuthology.orchestra.run.smithi204.stdout:Upgrade 8 Packages 2024-05-11T12:07:44.537 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:07:44.538 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 226 M 2024-05-11T12:07:44.538 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-05-11T12:07:45.444 INFO:teuthology.orchestra.run.smithi204.stdout:(1/156): ceph-19.0.0-3601.g30850680.el9.x86_64. 19 kB/s | 6.6 kB 00:00 2024-05-11T12:07:45.761 INFO:teuthology.orchestra.run.smithi204.stdout:(2/156): ceph-fuse-19.0.0-3601.g30850680.el9.x8 2.8 MB/s | 890 kB 00:00 2024-05-11T12:07:45.803 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - CRB 2.7 MB/s | 6.2 MB 00:02 2024-05-11T12:07:45.870 INFO:teuthology.orchestra.run.smithi204.stdout:(3/156): ceph-base-19.0.0-3601.g30850680.el9.x8 6.9 MB/s | 5.4 MB 00:00 2024-05-11T12:07:45.928 INFO:teuthology.orchestra.run.smithi204.stdout:(4/156): ceph-immutable-object-cache-19.0.0-360 872 kB/s | 145 kB 00:00 2024-05-11T12:07:46.145 INFO:teuthology.orchestra.run.smithi204.stdout:(5/156): ceph-mds-19.0.0-3601.g30850680.el9.x86 8.6 MB/s | 2.4 MB 00:00 2024-05-11T12:07:46.262 INFO:teuthology.orchestra.run.smithi204.stdout:(6/156): ceph-mgr-19.0.0-3601.g30850680.el9.x86 4.5 MB/s | 1.5 MB 00:00 2024-05-11T12:07:46.571 INFO:teuthology.orchestra.run.smithi204.stdout:(7/156): ceph-mon-19.0.0-3601.g30850680.el9.x86 11 MB/s | 4.7 MB 00:00 2024-05-11T12:07:47.580 INFO:teuthology.orchestra.run.smithi204.stdout:(8/156): ceph-radosgw-19.0.0-3601.g30850680.el9 11 MB/s | 11 MB 00:01 2024-05-11T12:07:47.604 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-05-11T12:07:47.638 INFO:teuthology.orchestra.run.smithi204.stdout:(9/156): ceph-selinux-19.0.0-3601.g30850680.el9 440 kB/s | 25 kB 00:00 2024-05-11T12:07:47.980 INFO:teuthology.orchestra.run.smithi204.stdout:(10/156): ceph-common-19.0.0-3601.g30850680.el9 7.3 MB/s | 21 MB 00:02 2024-05-11T12:07:48.064 INFO:teuthology.orchestra.run.smithi204.stdout:(11/156): cephfs-mirror-19.0.0-3601.g30850680.e 2.7 MB/s | 225 kB 00:00 2024-05-11T12:07:48.122 INFO:teuthology.orchestra.run.smithi204.stdout:(12/156): cephfs-shell-19.0.0-3601.g30850680.el 474 kB/s | 27 kB 00:00 2024-05-11T12:07:48.197 INFO:teuthology.orchestra.run.smithi204.stdout:(13/156): libcephfs-devel-19.0.0-3601.g30850680 427 kB/s | 32 kB 00:00 2024-05-11T12:07:48.382 INFO:teuthology.orchestra.run.smithi204.stdout:(14/156): libcephfs2-19.0.0-3601.g30850680.el9. 4.0 MB/s | 746 kB 00:00 2024-05-11T12:07:48.506 INFO:teuthology.orchestra.run.smithi204.stdout:(15/156): libcephsqlite-19.0.0-3601.g30850680.e 1.3 MB/s | 163 kB 00:00 2024-05-11T12:07:48.598 INFO:teuthology.orchestra.run.smithi204.stdout:(16/156): librados-devel-19.0.0-3601.g30850680. 1.4 MB/s | 127 kB 00:00 2024-05-11T12:07:48.816 INFO:teuthology.orchestra.run.smithi204.stdout:(17/156): ceph-osd-19.0.0-3601.g30850680.el9.x8 6.6 MB/s | 17 MB 00:02 2024-05-11T12:07:48.866 INFO:teuthology.orchestra.run.smithi204.stdout:(18/156): libradosstriper1-19.0.0-3601.g3085068 1.8 MB/s | 501 kB 00:00 2024-05-11T12:07:48.873 INFO:teuthology.orchestra.run.smithi167.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-05-11T12:07:48.933 INFO:teuthology.orchestra.run.smithi204.stdout:(19/156): python3-ceph-argparse-19.0.0-3601.g30 689 kB/s | 45 kB 00:00 2024-05-11T12:07:49.033 INFO:teuthology.orchestra.run.smithi204.stdout:(20/156): python3-ceph-common-19.0.0-3601.g3085 1.3 MB/s | 135 kB 00:00 2024-05-11T12:07:49.125 INFO:teuthology.orchestra.run.smithi204.stdout:(21/156): python3-cephfs-19.0.0-3601.g30850680. 1.7 MB/s | 161 kB 00:00 2024-05-11T12:07:49.234 INFO:teuthology.orchestra.run.smithi204.stdout:(22/156): python3-rados-19.0.0-3601.g30850680.e 2.9 MB/s | 321 kB 00:00 2024-05-11T12:07:49.342 INFO:teuthology.orchestra.run.smithi204.stdout:(23/156): python3-rbd-19.0.0-3601.g30850680.el9 2.7 MB/s | 300 kB 00:00 2024-05-11T12:07:49.426 INFO:teuthology.orchestra.run.smithi204.stdout:(24/156): python3-rgw-19.0.0-3601.g30850680.el9 1.2 MB/s | 100 kB 00:00 2024-05-11T12:07:49.551 INFO:teuthology.orchestra.run.smithi204.stdout:(25/156): librgw2-19.0.0-3601.g30850680.el9.x86 7.3 MB/s | 5.4 MB 00:00 2024-05-11T12:07:49.585 INFO:teuthology.orchestra.run.smithi204.stdout:(26/156): rbd-fuse-19.0.0-3601.g30850680.el9.x8 542 kB/s | 86 kB 00:00 2024-05-11T12:07:49.661 INFO:teuthology.orchestra.run.smithi204.stdout:(27/156): rbd-nbd-19.0.0-3601.g30850680.el9.x86 2.3 MB/s | 171 kB 00:00 2024-05-11T12:07:49.744 INFO:teuthology.orchestra.run.smithi204.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3601.g 415 kB/s | 34 kB 00:00 2024-05-11T12:07:49.845 INFO:teuthology.orchestra.run.smithi204.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3601.g3085068 1.4 MB/s | 142 kB 00:00 2024-05-11T12:07:50.012 INFO:teuthology.orchestra.run.smithi204.stdout:(30/156): rbd-mirror-19.0.0-3601.g30850680.el9. 6.7 MB/s | 3.1 MB 00:00 2024-05-11T12:07:50.862 INFO:teuthology.orchestra.run.smithi204.stdout:(31/156): ceph-test-19.0.0-3601.g30850680.el9.x 15 MB/s | 49 MB 00:03 2024-05-11T12:07:50.937 INFO:teuthology.orchestra.run.smithi204.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3601.g30 3.6 MB/s | 274 kB 00:00 2024-05-11T12:07:51.037 INFO:teuthology.orchestra.run.smithi204.stdout:(33/156): ceph-mgr-dashboard-19.0.0-3601.g30850 3.0 MB/s | 3.5 MB 00:01 2024-05-11T12:07:51.054 INFO:teuthology.orchestra.run.smithi204.stdout:(34/156): ceph-mgr-rook-19.0.0-3601.g30850680.e 426 kB/s | 49 kB 00:00 2024-05-11T12:07:51.096 INFO:teuthology.orchestra.run.smithi204.stdout:(35/156): ceph-prometheus-alerts-19.0.0-3601.g3 286 kB/s | 16 kB 00:00 2024-05-11T12:07:51.130 INFO:teuthology.orchestra.run.smithi204.stdout:(36/156): ceph-volume-19.0.0-3601.g30850680.el9 3.6 MB/s | 273 kB 00:00 2024-05-11T12:07:51.180 INFO:teuthology.orchestra.run.smithi204.stdout:(37/156): cephfs-top-19.0.0-3601.g30850680.el9. 371 kB/s | 18 kB 00:00 2024-05-11T12:07:51.230 INFO:teuthology.orchestra.run.smithi204.stdout:(38/156): cephadm-19.0.0-3601.g30850680.el9.noa 5.6 MB/s | 764 kB 00:00 2024-05-11T12:07:51.373 INFO:teuthology.orchestra.run.smithi204.stdout:(39/156): ceph-mgr-diskprediction-local-19.0.0- 5.4 MB/s | 7.4 MB 00:01 2024-05-11T12:07:51.398 INFO:teuthology.orchestra.run.smithi204.stdout:(40/156): fuse-2.9.9-15.el9.x86_64.rpm 371 kB/s | 80 kB 00:00 2024-05-11T12:07:51.515 INFO:teuthology.orchestra.run.smithi204.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 253 kB 00:00 2024-05-11T12:07:51.548 INFO:teuthology.orchestra.run.smithi204.stdout:(42/156): libquadmath-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 195 kB 00:00 2024-05-11T12:07:51.590 INFO:teuthology.orchestra.run.smithi204.stdout:(43/156): libgfortran-11.4.1-3.el9.x86_64.rpm 2.2 MB/s | 803 kB 00:00 2024-05-11T12:07:51.615 INFO:teuthology.orchestra.run.smithi204.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-05-11T12:07:51.641 INFO:teuthology.orchestra.run.smithi204.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-05-11T12:07:51.666 INFO:teuthology.orchestra.run.smithi204.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.5 MB/s | 125 kB 00:00 2024-05-11T12:07:51.700 INFO:teuthology.orchestra.run.smithi204.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-05-11T12:07:51.750 INFO:teuthology.orchestra.run.smithi204.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 5.2 MB/s | 1.2 MB 00:00 2024-05-11T12:07:51.867 INFO:teuthology.orchestra.run.smithi204.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 924 kB/s | 107 kB 00:00 2024-05-11T12:07:51.926 INFO:teuthology.orchestra.run.smithi204.stdout:(50/156): elfutils-libelf-devel-0.190-2.el9.x86 375 kB/s | 21 kB 00:00 2024-05-11T12:07:51.967 INFO:teuthology.orchestra.run.smithi204.stdout:(51/156): automake-1.16.2-8.el9.noarch.rpm 2.2 MB/s | 689 kB 00:00 2024-05-11T12:07:52.051 INFO:teuthology.orchestra.run.smithi204.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 2.6 MB/s | 943 kB 00:00 2024-05-11T12:07:52.076 INFO:teuthology.orchestra.run.smithi204.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 304 kB/s | 33 kB 00:00 2024-05-11T12:07:52.135 INFO:teuthology.orchestra.run.smithi204.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 313 kB/s | 18 kB 00:00 2024-05-11T12:07:52.169 INFO:teuthology.orchestra.run.smithi204.stdout:(55/156): flex-2.6.4-9.el9.x86_64.rpm 1.3 MB/s | 313 kB 00:00 2024-05-11T12:07:52.228 INFO:teuthology.orchestra.run.smithi204.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 122 kB/s | 11 kB 00:00 2024-05-11T12:07:52.286 INFO:teuthology.orchestra.run.smithi204.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-05-11T12:07:52.312 INFO:teuthology.orchestra.run.smithi204.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 548 kB/s | 45 kB 00:00 2024-05-11T12:07:52.437 INFO:teuthology.orchestra.run.smithi204.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 662 kB 00:00 2024-05-11T12:07:52.522 INFO:teuthology.orchestra.run.smithi204.stdout:(60/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 306 kB/s | 25 kB 00:00 2024-05-11T12:07:52.596 INFO:teuthology.orchestra.run.smithi204.stdout:(61/156): libtool-2.4.6-45.el9.x86_64.rpm 2.0 MB/s | 582 kB 00:00 2024-05-11T12:07:52.621 INFO:teuthology.orchestra.run.smithi204.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 479 kB/s | 47 kB 00:00 2024-05-11T12:07:52.705 INFO:teuthology.orchestra.run.smithi204.stdout:(63/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.6 MB/s | 3.0 MB 00:00 2024-05-11T12:07:52.763 INFO:teuthology.orchestra.run.smithi204.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 1.3 MB/s | 188 kB 00:00 2024-05-11T12:07:52.780 INFO:teuthology.orchestra.run.smithi204.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 475 kB/s | 35 kB 00:00 2024-05-11T12:07:52.814 INFO:teuthology.orchestra.run.smithi204.stdout:(66/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.3 MB/s | 292 kB 00:00 2024-05-11T12:07:53.364 INFO:teuthology.orchestra.run.smithi204.stdout:(67/156): openblas-openmp-0.3.21-2.el9.x86_64.r 7.9 MB/s | 4.7 MB 00:00 2024-05-11T12:07:53.664 INFO:teuthology.orchestra.run.smithi204.stdout:(68/156): postgresql-contrib-13.14-1.el9.x86_64 3.0 MB/s | 911 kB 00:00 2024-05-11T12:07:53.932 INFO:teuthology.orchestra.run.smithi204.stdout:(69/156): postgresql-13.14-1.el9.x86_64.rpm 1.4 MB/s | 1.6 MB 00:01 2024-05-11T12:07:53.965 INFO:teuthology.orchestra.run.smithi204.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 445 kB/s | 133 kB 00:00 2024-05-11T12:07:54.116 INFO:teuthology.orchestra.run.smithi204.stdout:(71/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 3.3 MB/s | 4.4 MB 00:01 2024-05-11T12:07:54.241 INFO:teuthology.orchestra.run.smithi204.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 707 kB/s | 87 kB 00:00 2024-05-11T12:07:54.300 INFO:teuthology.orchestra.run.smithi204.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 3.0 MB/s | 1.0 MB 00:00 2024-05-11T12:07:55.008 INFO:teuthology.orchestra.run.smithi204.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 345 kB/s | 244 kB 00:00 2024-05-11T12:07:55.072 INFO:teuthology.orchestra.run.smithi167.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-05-11T12:07:55.175 INFO:teuthology.orchestra.run.smithi204.stdout:(75/156): python3-jinja2-2.11.3-5.el9.noarch.rp 1.5 MB/s | 248 kB 00:00 2024-05-11T12:07:55.242 INFO:teuthology.orchestra.run.smithi204.stdout:(76/156): python3-jsonpatch-1.21-16.el9.noarch. 387 kB/s | 26 kB 00:00 2024-05-11T12:07:55.342 INFO:teuthology.orchestra.run.smithi204.stdout:(77/156): python3-jsonpointer-2.0-4.el9.noarch. 193 kB/s | 19 kB 00:00 2024-05-11T12:07:55.485 INFO:teuthology.orchestra.run.smithi204.stdout:(78/156): postgresql-server-13.14-1.el9.x86_64. 3.7 MB/s | 5.8 MB 00:01 2024-05-11T12:07:55.518 INFO:teuthology.orchestra.run.smithi204.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 988 kB/s | 172 kB 00:00 2024-05-11T12:07:55.585 INFO:teuthology.orchestra.run.smithi204.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 349 kB/s | 35 kB 00:00 2024-05-11T12:07:55.702 INFO:teuthology.orchestra.run.smithi204.stdout:(81/156): python3-babel-2.9.1-2.el9.noarch.rpm 4.1 MB/s | 6.0 MB 00:01 2024-05-11T12:07:55.836 INFO:teuthology.orchestra.run.smithi204.stdout:(82/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.6 MB/s | 222 kB 00:00 2024-05-11T12:07:55.944 INFO:teuthology.orchestra.run.smithi204.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 700 kB/s | 250 kB 00:00 2024-05-11T12:07:56.095 INFO:teuthology.orchestra.run.smithi204.stdout:(84/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 8.9 MB/s | 5.1 MB 00:00 2024-05-11T12:07:56.120 INFO:teuthology.orchestra.run.smithi204.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 273 kB/s | 77 kB 00:00 2024-05-11T12:07:56.145 INFO:teuthology.orchestra.run.smithi204.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 212 kB/s | 42 kB 00:00 2024-05-11T12:07:56.221 INFO:teuthology.orchestra.run.smithi204.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-05-11T12:07:56.304 INFO:teuthology.orchestra.run.smithi204.stdout:(88/156): python3-pytz-2021.1-5.el9.noarch.rpm 620 kB/s | 51 kB 00:00 2024-05-11T12:07:56.371 INFO:teuthology.orchestra.run.smithi204.stdout:(89/156): python3-requests-oauthlib-1.3.0-12.el 817 kB/s | 54 kB 00:00 2024-05-11T12:07:56.405 INFO:teuthology.orchestra.run.smithi204.stdout:(90/156): python3-protobuf-3.14.0-13.el9.noarch 872 kB/s | 269 kB 00:00 2024-05-11T12:07:56.438 INFO:teuthology.orchestra.run.smithi204.stdout:(91/156): python3-pyasn1-modules-0.4.8-6.el9.no 957 kB/s | 279 kB 00:00 2024-05-11T12:07:56.489 INFO:teuthology.orchestra.run.smithi204.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 504 kB/s | 42 kB 00:00 2024-05-11T12:07:56.514 INFO:teuthology.orchestra.run.smithi204.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 595 kB/s | 44 kB 00:00 2024-05-11T12:07:56.606 INFO:teuthology.orchestra.run.smithi204.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 396 kB/s | 36 kB 00:00 2024-05-11T12:07:56.681 INFO:teuthology.orchestra.run.smithi204.stdout:(95/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 641 kB/s | 47 kB 00:00 2024-05-11T12:07:56.706 INFO:teuthology.orchestra.run.smithi204.stdout:(96/156): qatlib-24.02.0-1.el9.x86_64.rpm 1.0 MB/s | 221 kB 00:00 2024-05-11T12:07:56.781 INFO:teuthology.orchestra.run.smithi204.stdout:(97/156): uuid-1.6.2-55.el9.x86_64.rpm 783 kB/s | 58 kB 00:00 2024-05-11T12:07:56.815 INFO:teuthology.orchestra.run.smithi204.stdout:(98/156): socat-1.7.4.1-5.el9.x86_64.rpm 2.2 MB/s | 305 kB 00:00 2024-05-11T12:07:57.091 INFO:teuthology.orchestra.run.smithi204.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 304 kB/s | 94 kB 00:00 2024-05-11T12:07:57.107 INFO:teuthology.orchestra.run.smithi204.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 219 kB/s | 64 kB 00:00 2024-05-11T12:07:57.166 INFO:teuthology.orchestra.run.smithi204.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 603 kB/s | 45 kB 00:00 2024-05-11T12:07:57.629 INFO:teuthology.orchestra.run.smithi167.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-11T12:07:57.630 INFO:teuthology.orchestra.run.smithi167.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-11T12:07:57.649 INFO:teuthology.orchestra.run.smithi167.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-11T12:07:57.651 INFO:teuthology.orchestra.run.smithi167.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-11T12:07:57.652 INFO:teuthology.orchestra.run.smithi167.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-11T12:07:57.653 INFO:teuthology.orchestra.run.smithi167.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-11T12:07:57.655 INFO:teuthology.orchestra.run.smithi167.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-11T12:07:57.659 INFO:teuthology.orchestra.run.smithi167.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-11T12:07:57.660 INFO:teuthology.orchestra.run.smithi167.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-11T12:07:57.730 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-05-11T12:07:57.744 INFO:teuthology.orchestra.run.smithi167.stdout:======================================================================================= 2024-05-11T12:07:57.744 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2024-05-11T12:07:57.744 INFO:teuthology.orchestra.run.smithi167.stdout:======================================================================================= 2024-05-11T12:07:57.744 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2024-05-11T12:07:57.745 INFO:teuthology.orchestra.run.smithi167.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-11T12:07:57.745 INFO:teuthology.orchestra.run.smithi167.stdout: ceph x86_64 2:19.0.0-3601.g30850680.el9 ceph 6.6 k 2024-05-11T12:07:57.745 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-base x86_64 2:19.0.0-3601.g30850680.el9 ceph 5.4 M 2024-05-11T12:07:57.745 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-fuse x86_64 2:19.0.0-3601.g30850680.el9 ceph 890 k 2024-05-11T12:07:57.745 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3601.g30850680.el9 ceph 145 k 2024-05-11T12:07:57.745 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr x86_64 2:19.0.0-3601.g30850680.el9 ceph 1.5 M 2024-05-11T12:07:57.745 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 142 k 2024-05-11T12:07:57.745 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 3.5 M 2024-05-11T12:07:57.745 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 7.4 M 2024-05-11T12:07:57.745 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-rook noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 49 k 2024-05-11T12:07:57.745 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-radosgw x86_64 2:19.0.0-3601.g30850680.el9 ceph 11 M 2024-05-11T12:07:57.745 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-test x86_64 2:19.0.0-3601.g30850680.el9 ceph 49 M 2024-05-11T12:07:57.745 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-volume noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 273 k 2024-05-11T12:07:57.745 INFO:teuthology.orchestra.run.smithi167.stdout: cephadm noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 764 k 2024-05-11T12:07:57.746 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-mirror x86_64 2:19.0.0-3601.g30850680.el9 ceph 225 k 2024-05-11T12:07:57.746 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-shell x86_64 2:19.0.0-3601.g30850680.el9 ceph 27 k 2024-05-11T12:07:57.746 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-top noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 18 k 2024-05-11T12:07:57.746 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-11T12:07:57.746 INFO:teuthology.orchestra.run.smithi167.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-11T12:07:57.746 INFO:teuthology.orchestra.run.smithi167.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-11T12:07:57.746 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs-devel x86_64 2:19.0.0-3601.g30850680.el9 ceph 32 k 2024-05-11T12:07:57.746 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs2 x86_64 2:19.0.0-3601.g30850680.el9 ceph 746 k 2024-05-11T12:07:57.746 INFO:teuthology.orchestra.run.smithi167.stdout: librados-devel x86_64 2:19.0.0-3601.g30850680.el9 ceph 127 k 2024-05-11T12:07:57.746 INFO:teuthology.orchestra.run.smithi167.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-11T12:07:57.746 INFO:teuthology.orchestra.run.smithi167.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-11T12:07:57.746 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-11T12:07:57.746 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-11T12:07:57.747 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-11T12:07:57.747 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-11T12:07:57.747 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cephfs x86_64 2:19.0.0-3601.g30850680.el9 ceph 161 k 2024-05-11T12:07:57.747 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rados x86_64 2:19.0.0-3601.g30850680.el9 ceph 321 k 2024-05-11T12:07:57.747 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rbd x86_64 2:19.0.0-3601.g30850680.el9 ceph 300 k 2024-05-11T12:07:57.747 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rgw x86_64 2:19.0.0-3601.g30850680.el9 ceph 100 k 2024-05-11T12:07:57.747 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-fuse x86_64 2:19.0.0-3601.g30850680.el9 ceph 86 k 2024-05-11T12:07:57.747 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-mirror x86_64 2:19.0.0-3601.g30850680.el9 ceph 3.1 M 2024-05-11T12:07:57.747 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-nbd x86_64 2:19.0.0-3601.g30850680.el9 ceph 171 k 2024-05-11T12:07:57.747 INFO:teuthology.orchestra.run.smithi167.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-11T12:07:57.747 INFO:teuthology.orchestra.run.smithi167.stdout:Upgrading: 2024-05-11T12:07:57.747 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-11T12:07:57.747 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-11T12:07:57.748 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-11T12:07:57.748 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-11T12:07:57.748 INFO:teuthology.orchestra.run.smithi167.stdout: librados2 x86_64 2:19.0.0-3601.g30850680.el9 ceph 3.4 M 2024-05-11T12:07:57.748 INFO:teuthology.orchestra.run.smithi167.stdout: librbd1 x86_64 2:19.0.0-3601.g30850680.el9 ceph 3.1 M 2024-05-11T12:07:57.748 INFO:teuthology.orchestra.run.smithi167.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-11T12:07:57.748 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-11T12:07:57.748 INFO:teuthology.orchestra.run.smithi167.stdout:Installing dependencies: 2024-05-11T12:07:57.748 INFO:teuthology.orchestra.run.smithi167.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-11T12:07:57.748 INFO:teuthology.orchestra.run.smithi167.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-11T12:07:57.748 INFO:teuthology.orchestra.run.smithi167.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-11T12:07:57.748 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-common x86_64 2:19.0.0-3601.g30850680.el9 ceph 21 M 2024-05-11T12:07:57.748 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 34 k 2024-05-11T12:07:57.748 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mds x86_64 2:19.0.0-3601.g30850680.el9 ceph 2.4 M 2024-05-11T12:07:57.749 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 274 k 2024-05-11T12:07:57.749 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mon x86_64 2:19.0.0-3601.g30850680.el9 ceph 4.7 M 2024-05-11T12:07:57.749 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-osd x86_64 2:19.0.0-3601.g30850680.el9 ceph 17 M 2024-05-11T12:07:57.749 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 16 k 2024-05-11T12:07:57.749 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-selinux x86_64 2:19.0.0-3601.g30850680.el9 ceph 25 k 2024-05-11T12:07:57.749 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-11T12:07:57.749 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-11T12:07:57.749 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-11T12:07:57.749 INFO:teuthology.orchestra.run.smithi167.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-11T12:07:57.749 INFO:teuthology.orchestra.run.smithi167.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-11T12:07:57.749 INFO:teuthology.orchestra.run.smithi167.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-11T12:07:57.749 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-11T12:07:57.749 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-11T12:07:57.749 INFO:teuthology.orchestra.run.smithi167.stdout: libcephsqlite x86_64 2:19.0.0-3601.g30850680.el9 ceph 163 k 2024-05-11T12:07:57.750 INFO:teuthology.orchestra.run.smithi167.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-11T12:07:57.750 INFO:teuthology.orchestra.run.smithi167.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-11T12:07:57.750 INFO:teuthology.orchestra.run.smithi167.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-11T12:07:57.750 INFO:teuthology.orchestra.run.smithi167.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-11T12:07:57.750 INFO:teuthology.orchestra.run.smithi167.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-11T12:07:57.750 INFO:teuthology.orchestra.run.smithi167.stdout: libradosstriper1 x86_64 2:19.0.0-3601.g30850680.el9 ceph 501 k 2024-05-11T12:07:57.750 INFO:teuthology.orchestra.run.smithi167.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-11T12:07:57.750 INFO:teuthology.orchestra.run.smithi167.stdout: librgw2 x86_64 2:19.0.0-3601.g30850680.el9 ceph 5.4 M 2024-05-11T12:07:57.750 INFO:teuthology.orchestra.run.smithi167.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-11T12:07:57.750 INFO:teuthology.orchestra.run.smithi167.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-11T12:07:57.750 INFO:teuthology.orchestra.run.smithi167.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-11T12:07:57.750 INFO:teuthology.orchestra.run.smithi167.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-11T12:07:57.751 INFO:teuthology.orchestra.run.smithi167.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-11T12:07:57.751 INFO:teuthology.orchestra.run.smithi167.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-11T12:07:57.751 INFO:teuthology.orchestra.run.smithi167.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-11T12:07:57.751 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-11T12:07:57.751 INFO:teuthology.orchestra.run.smithi167.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-11T12:07:57.751 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-11T12:07:57.751 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-11T12:07:57.751 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-11T12:07:57.751 INFO:teuthology.orchestra.run.smithi167.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-11T12:07:57.751 INFO:teuthology.orchestra.run.smithi167.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-11T12:07:57.751 INFO:teuthology.orchestra.run.smithi167.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-11T12:07:57.751 INFO:teuthology.orchestra.run.smithi167.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-11T12:07:57.751 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-11T12:07:57.752 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-argparse x86_64 2:19.0.0-3601.g30850680.el9 ceph 45 k 2024-05-11T12:07:57.752 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-common x86_64 2:19.0.0-3601.g30850680.el9 ceph 135 k 2024-05-11T12:07:57.752 INFO:teuthology.orchestra.run.smithi167.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-11T12:07:57.752 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-11T12:07:57.752 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-11T12:07:57.752 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-11T12:07:57.752 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-11T12:07:57.752 INFO:teuthology.orchestra.run.smithi167.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-11T12:07:57.752 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-11T12:07:57.752 INFO:teuthology.orchestra.run.smithi167.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-11T12:07:57.752 INFO:teuthology.orchestra.run.smithi167.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-11T12:07:57.752 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-11T12:07:57.752 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-11T12:07:57.752 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-11T12:07:57.753 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-11T12:07:57.753 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-11T12:07:57.753 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-11T12:07:57.753 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-11T12:07:57.753 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-11T12:07:57.753 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-11T12:07:57.753 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-11T12:07:57.753 INFO:teuthology.orchestra.run.smithi167.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-11T12:07:57.753 INFO:teuthology.orchestra.run.smithi167.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-11T12:07:57.753 INFO:teuthology.orchestra.run.smithi167.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-11T12:07:57.753 INFO:teuthology.orchestra.run.smithi167.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-11T12:07:57.753 INFO:teuthology.orchestra.run.smithi167.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-11T12:07:57.753 INFO:teuthology.orchestra.run.smithi167.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-11T12:07:57.754 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-11T12:07:57.754 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-11T12:07:57.754 INFO:teuthology.orchestra.run.smithi167.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-11T12:07:57.754 INFO:teuthology.orchestra.run.smithi167.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-11T12:07:57.754 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-11T12:07:57.754 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-11T12:07:57.754 INFO:teuthology.orchestra.run.smithi167.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-11T12:07:57.754 INFO:teuthology.orchestra.run.smithi167.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-11T12:07:57.754 INFO:teuthology.orchestra.run.smithi167.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-11T12:07:57.754 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-11T12:07:57.754 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-11T12:07:57.754 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-11T12:07:57.754 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-11T12:07:57.755 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-11T12:07:57.755 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-11T12:07:57.755 INFO:teuthology.orchestra.run.smithi167.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-11T12:07:57.755 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-11T12:07:57.755 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-11T12:07:57.755 INFO:teuthology.orchestra.run.smithi167.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-11T12:07:57.755 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-11T12:07:57.755 INFO:teuthology.orchestra.run.smithi167.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-11T12:07:57.755 INFO:teuthology.orchestra.run.smithi167.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-11T12:07:57.755 INFO:teuthology.orchestra.run.smithi167.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-11T12:07:57.755 INFO:teuthology.orchestra.run.smithi167.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-11T12:07:57.755 INFO:teuthology.orchestra.run.smithi167.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-11T12:07:57.755 INFO:teuthology.orchestra.run.smithi167.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-11T12:07:57.755 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-11T12:07:57.755 INFO:teuthology.orchestra.run.smithi167.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-11T12:07:57.756 INFO:teuthology.orchestra.run.smithi167.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-11T12:07:57.756 INFO:teuthology.orchestra.run.smithi167.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-11T12:07:57.756 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-11T12:07:57.756 INFO:teuthology.orchestra.run.smithi167.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-11T12:07:57.756 INFO:teuthology.orchestra.run.smithi167.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-11T12:07:57.756 INFO:teuthology.orchestra.run.smithi167.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-11T12:07:57.756 INFO:teuthology.orchestra.run.smithi167.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-11T12:07:57.756 INFO:teuthology.orchestra.run.smithi167.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-11T12:07:57.756 INFO:teuthology.orchestra.run.smithi167.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-11T12:07:57.756 INFO:teuthology.orchestra.run.smithi167.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-11T12:07:57.756 INFO:teuthology.orchestra.run.smithi167.stdout:Installing weak dependencies: 2024-05-11T12:07:57.756 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-11T12:07:57.756 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:07:57.756 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2024-05-11T12:07:57.757 INFO:teuthology.orchestra.run.smithi167.stdout:======================================================================================= 2024-05-11T12:07:57.757 INFO:teuthology.orchestra.run.smithi167.stdout:Install 148 Packages 2024-05-11T12:07:57.757 INFO:teuthology.orchestra.run.smithi167.stdout:Upgrade 8 Packages 2024-05-11T12:07:57.757 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:07:57.757 INFO:teuthology.orchestra.run.smithi204.stdout:(102/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 12 MB/s | 16 MB 00:01 2024-05-11T12:07:57.758 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 226 M 2024-05-11T12:07:57.758 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2024-05-11T12:07:57.775 INFO:teuthology.orchestra.run.smithi204.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 33 kB/s | 22 kB 00:00 2024-05-11T12:07:58.276 INFO:teuthology.orchestra.run.smithi204.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.0 MB/s | 548 kB 00:00 2024-05-11T12:07:58.317 INFO:teuthology.orchestra.run.smithi204.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 571 kB/s | 309 kB 00:00 2024-05-11T12:07:58.351 INFO:teuthology.orchestra.run.smithi204.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 262 kB/s | 19 kB 00:00 2024-05-11T12:07:58.401 INFO:teuthology.orchestra.run.smithi204.stdout:(107/156): protobuf-compiler-3.14.0-13.el9.x86_ 700 kB/s | 863 kB 00:01 2024-05-11T12:07:58.427 INFO:teuthology.orchestra.run.smithi204.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 347 kB/s | 26 kB 00:00 2024-05-11T12:07:58.544 INFO:teuthology.orchestra.run.smithi204.stdout:(109/156): libunwind-1.6.2-1.el9.x86_64.rpm 582 kB/s | 67 kB 00:00 2024-05-11T12:07:58.628 INFO:teuthology.orchestra.run.smithi204.stdout:(110/156): luarocks-3.9.2-1.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-05-11T12:07:58.727 INFO:teuthology.orchestra.run.smithi167.stdout:(1/156): ceph-19.0.0-3601.g30850680.el9.x86_64. 22 kB/s | 6.6 kB 00:00 2024-05-11T12:07:58.728 INFO:teuthology.orchestra.run.smithi204.stdout:(111/156): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-05-11T12:07:58.753 INFO:teuthology.orchestra.run.smithi204.stdout:(112/156): liboath-2.6.7-2.el9.x86_64.rpm 139 kB/s | 49 kB 00:00 2024-05-11T12:07:58.811 INFO:teuthology.orchestra.run.smithi204.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.5 MB/s | 839 kB 00:00 2024-05-11T12:07:58.904 INFO:teuthology.orchestra.run.smithi204.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 3.1 MB/s | 548 kB 00:00 2024-05-11T12:07:58.959 INFO:teuthology.orchestra.run.smithi167.stdout:(2/156): ceph-fuse-19.0.0-3601.g30850680.el9.x8 3.8 MB/s | 890 kB 00:00 2024-05-11T12:07:59.009 INFO:teuthology.orchestra.run.smithi167.stdout:(3/156): ceph-immutable-object-cache-19.0.0-360 2.9 MB/s | 145 kB 00:00 2024-05-11T12:07:59.118 INFO:teuthology.orchestra.run.smithi204.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 119 kB/s | 43 kB 00:00 2024-05-11T12:07:59.146 INFO:teuthology.orchestra.run.smithi204.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 97 kB/s | 32 kB 00:00 2024-05-11T12:07:59.171 INFO:teuthology.orchestra.run.smithi204.stdout:(117/156): python3-certifi-2021.10.8-2.el9.noar 56 kB/s | 15 kB 00:00 2024-05-11T12:07:59.205 INFO:teuthology.orchestra.run.smithi204.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-05-11T12:07:59.255 INFO:teuthology.orchestra.run.smithi204.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 4.3 MB/s | 358 kB 00:00 2024-05-11T12:07:59.280 INFO:teuthology.orchestra.run.smithi204.stdout:(120/156): python3-colorama-0.4.4-12.el9.noarch 473 kB/s | 35 kB 00:00 2024-05-11T12:07:59.326 INFO:teuthology.orchestra.run.smithi167.stdout:(4/156): ceph-mds-19.0.0-3601.g30850680.el9.x86 7.5 MB/s | 2.4 MB 00:00 2024-05-11T12:07:59.339 INFO:teuthology.orchestra.run.smithi204.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 2.4 MB/s | 206 kB 00:00 2024-05-11T12:07:59.373 INFO:teuthology.orchestra.run.smithi204.stdout:(122/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.5 MB/s | 303 kB 00:00 2024-05-11T12:07:59.415 INFO:teuthology.orchestra.run.smithi204.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-05-11T12:07:59.426 INFO:teuthology.orchestra.run.smithi167.stdout:(5/156): ceph-base-19.0.0-3601.g30850680.el9.x8 5.4 MB/s | 5.4 MB 00:00 2024-05-11T12:07:59.448 INFO:teuthology.orchestra.run.smithi204.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 144 kB/s | 11 kB 00:00 2024-05-11T12:07:59.490 INFO:teuthology.orchestra.run.smithi204.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 237 kB/s | 18 kB 00:00 2024-05-11T12:07:59.543 INFO:teuthology.orchestra.run.smithi167.stdout:(6/156): ceph-mgr-19.0.0-3601.g30850680.el9.x86 7.0 MB/s | 1.5 MB 00:00 2024-05-11T12:07:59.566 INFO:teuthology.orchestra.run.smithi204.stdout:(126/156): python3-grpcio-1.46.7-10.el9.x86_64. 7.2 MB/s | 2.0 MB 00:00 2024-05-11T12:07:59.591 INFO:teuthology.orchestra.run.smithi204.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 163 kB/s | 23 kB 00:00 2024-05-11T12:07:59.641 INFO:teuthology.orchestra.run.smithi204.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 130 kB/s | 19 kB 00:00 2024-05-11T12:07:59.666 INFO:teuthology.orchestra.run.smithi204.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 198 kB/s | 20 kB 00:00 2024-05-11T12:07:59.717 INFO:teuthology.orchestra.run.smithi204.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 627 kB/s | 46 kB 00:00 2024-05-11T12:07:59.750 INFO:teuthology.orchestra.run.smithi204.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 950 kB/s | 79 kB 00:00 2024-05-11T12:07:59.801 INFO:teuthology.orchestra.run.smithi204.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 6.5 MB/s | 1.0 MB 00:00 2024-05-11T12:07:59.826 INFO:teuthology.orchestra.run.smithi204.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 535 kB/s | 58 kB 00:00 2024-05-11T12:07:59.859 INFO:teuthology.orchestra.run.smithi204.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.5 MB/s | 272 kB 00:00 2024-05-11T12:07:59.877 INFO:teuthology.orchestra.run.smithi204.stdout:(135/156): python3-portend-3.1.0-2.el9.noarch.r 219 kB/s | 16 kB 00:00 2024-05-11T12:07:59.910 INFO:teuthology.orchestra.run.smithi204.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-05-11T12:07:59.936 INFO:teuthology.orchestra.run.smithi204.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 376 kB/s | 28 kB 00:00 2024-05-11T12:07:59.953 INFO:teuthology.orchestra.run.smithi204.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 412 kB/s | 31 kB 00:00 2024-05-11T12:07:59.968 INFO:teuthology.orchestra.run.smithi167.stdout:(7/156): ceph-mon-19.0.0-3601.g30850680.el9.x86 8.7 MB/s | 4.7 MB 00:00 2024-05-11T12:08:00.003 INFO:teuthology.orchestra.run.smithi204.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-05-11T12:08:00.028 INFO:teuthology.orchestra.run.smithi204.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 478 kB/s | 36 kB 00:00 2024-05-11T12:08:00.054 INFO:teuthology.orchestra.run.smithi204.stdout:(141/156): python3-rsa-4.9-2.el9.noarch.rpm 506 kB/s | 59 kB 00:00 2024-05-11T12:08:00.154 INFO:teuthology.orchestra.run.smithi204.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 684 kB/s | 51 kB 00:00 2024-05-11T12:08:00.154 INFO:teuthology.orchestra.run.smithi204.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-05-11T12:08:00.162 INFO:teuthology.orchestra.run.smithi204.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 830 kB/s | 90 kB 00:00 2024-05-11T12:08:00.213 INFO:teuthology.orchestra.run.smithi204.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-05-11T12:08:00.238 INFO:teuthology.orchestra.run.smithi204.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 191 kB/s | 20 kB 00:00 2024-05-11T12:08:00.272 INFO:teuthology.orchestra.run.smithi204.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-05-11T12:08:00.414 INFO:teuthology.orchestra.run.smithi204.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 7.9 MB/s | 1.6 MB 00:00 2024-05-11T12:08:00.636 INFO:teuthology.orchestra.run.smithi204.stdout:(149/156): NetworkManager-1.47.5-1.el9.x86_64.r 10 MB/s | 2.3 MB 00:00 2024-05-11T12:08:00.739 INFO:teuthology.orchestra.run.smithi204.stdout:(150/156): NetworkManager-libnm-1.47.5-1.el9.x8 18 MB/s | 1.8 MB 00:00 2024-05-11T12:08:00.781 INFO:teuthology.orchestra.run.smithi204.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 1.0 MB/s | 42 kB 00:00 2024-05-11T12:08:00.832 INFO:teuthology.orchestra.run.smithi204.stdout:(152/156): NetworkManager-tui-1.47.5-1.el9.x86_ 4.9 MB/s | 248 kB 00:00 2024-05-11T12:08:00.915 INFO:teuthology.orchestra.run.smithi204.stdout:(153/156): openssl-3.2.1-1.el9.x86_64.rpm 16 MB/s | 1.3 MB 00:00 2024-05-11T12:08:01.016 INFO:teuthology.orchestra.run.smithi204.stdout:(154/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 24 MB/s | 2.4 MB 00:00 2024-05-11T12:08:01.241 INFO:teuthology.orchestra.run.smithi204.stdout:(155/156): librbd1-19.0.0-3601.g30850680.el9.x8 3.2 MB/s | 3.1 MB 00:00 2024-05-11T12:08:01.432 INFO:teuthology.orchestra.run.smithi204.stdout:(156/156): librados2-19.0.0-3601.g30850680.el9. 2.8 MB/s | 3.4 MB 00:01 2024-05-11T12:08:01.445 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-05-11T12:08:01.445 INFO:teuthology.orchestra.run.smithi204.stdout:Total 13 MB/s | 226 MB 00:16 2024-05-11T12:08:01.577 INFO:teuthology.orchestra.run.smithi167.stdout:(8/156): ceph-osd-19.0.0-3601.g30850680.el9.x86 8.3 MB/s | 17 MB 00:02 2024-05-11T12:08:01.752 INFO:teuthology.orchestra.run.smithi167.stdout:(9/156): ceph-radosgw-19.0.0-3601.g30850680.el9 5.9 MB/s | 11 MB 00:01 2024-05-11T12:08:01.777 INFO:teuthology.orchestra.run.smithi167.stdout:(10/156): ceph-selinux-19.0.0-3601.g30850680.el 126 kB/s | 25 kB 00:00 2024-05-11T12:08:02.069 INFO:teuthology.orchestra.run.smithi167.stdout:(11/156): ceph-common-19.0.0-3601.g30850680.el9 5.8 MB/s | 21 MB 00:03 2024-05-11T12:08:02.102 INFO:teuthology.orchestra.run.smithi167.stdout:(12/156): cephfs-mirror-19.0.0-3601.g30850680.e 693 kB/s | 225 kB 00:00 2024-05-11T12:08:02.136 INFO:teuthology.orchestra.run.smithi167.stdout:(13/156): cephfs-shell-19.0.0-3601.g30850680.el 412 kB/s | 27 kB 00:00 2024-05-11T12:08:02.162 INFO:teuthology.orchestra.run.smithi167.stdout:(14/156): libcephfs-devel-19.0.0-3601.g30850680 545 kB/s | 32 kB 00:00 2024-05-11T12:08:02.237 INFO:teuthology.orchestra.run.smithi167.stdout:(15/156): libcephsqlite-19.0.0-3601.g30850680.e 2.2 MB/s | 163 kB 00:00 2024-05-11T12:08:02.304 INFO:teuthology.orchestra.run.smithi167.stdout:(16/156): librados-devel-19.0.0-3601.g30850680. 1.9 MB/s | 127 kB 00:00 2024-05-11T12:08:02.346 INFO:teuthology.orchestra.run.smithi167.stdout:(17/156): libcephfs2-19.0.0-3601.g30850680.el9. 3.5 MB/s | 746 kB 00:00 2024-05-11T12:08:02.405 INFO:teuthology.orchestra.run.smithi167.stdout:(18/156): libradosstriper1-19.0.0-3601.g3085068 4.9 MB/s | 501 kB 00:00 2024-05-11T12:08:02.472 INFO:teuthology.orchestra.run.smithi167.stdout:(19/156): python3-ceph-argparse-19.0.0-3601.g30 686 kB/s | 45 kB 00:00 2024-05-11T12:08:02.538 INFO:teuthology.orchestra.run.smithi167.stdout:(20/156): python3-ceph-common-19.0.0-3601.g3085 2.0 MB/s | 135 kB 00:00 2024-05-11T12:08:02.605 INFO:teuthology.orchestra.run.smithi167.stdout:(21/156): python3-cephfs-19.0.0-3601.g30850680. 2.4 MB/s | 161 kB 00:00 2024-05-11T12:08:02.697 INFO:teuthology.orchestra.run.smithi167.stdout:(22/156): python3-rados-19.0.0-3601.g30850680.e 3.4 MB/s | 321 kB 00:00 2024-05-11T12:08:02.772 INFO:teuthology.orchestra.run.smithi167.stdout:(23/156): python3-rbd-19.0.0-3601.g30850680.el9 4.0 MB/s | 300 kB 00:00 2024-05-11T12:08:02.832 INFO:teuthology.orchestra.run.smithi167.stdout:(24/156): python3-rgw-19.0.0-3601.g30850680.el9 1.7 MB/s | 100 kB 00:00 2024-05-11T12:08:02.899 INFO:teuthology.orchestra.run.smithi167.stdout:(25/156): rbd-fuse-19.0.0-3601.g30850680.el9.x8 1.3 MB/s | 86 kB 00:00 2024-05-11T12:08:03.066 INFO:teuthology.orchestra.run.smithi167.stdout:(26/156): librgw2-19.0.0-3601.g30850680.el9.x86 7.5 MB/s | 5.4 MB 00:00 2024-05-11T12:08:03.149 INFO:teuthology.orchestra.run.smithi167.stdout:(27/156): rbd-nbd-19.0.0-3601.g30850680.el9.x86 2.0 MB/s | 171 kB 00:00 2024-05-11T12:08:03.216 INFO:teuthology.orchestra.run.smithi167.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3601.g 521 kB/s | 34 kB 00:00 2024-05-11T12:08:03.291 INFO:teuthology.orchestra.run.smithi167.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3601.g3085068 1.9 MB/s | 142 kB 00:00 2024-05-11T12:08:03.367 INFO:teuthology.orchestra.run.smithi167.stdout:(30/156): rbd-mirror-19.0.0-3601.g30850680.el9. 6.6 MB/s | 3.1 MB 00:00 2024-05-11T12:08:03.750 INFO:teuthology.orchestra.run.smithi167.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3601.g30850 7.7 MB/s | 3.5 MB 00:00 2024-05-11T12:08:03.834 INFO:teuthology.orchestra.run.smithi167.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3601.g30 3.2 MB/s | 274 kB 00:00 2024-05-11T12:08:03.934 INFO:teuthology.orchestra.run.smithi167.stdout:(33/156): ceph-mgr-rook-19.0.0-3601.g30850680.e 498 kB/s | 49 kB 00:00 2024-05-11T12:08:04.001 INFO:teuthology.orchestra.run.smithi167.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3601.g3 250 kB/s | 16 kB 00:00 2024-05-11T12:08:04.093 INFO:teuthology.orchestra.run.smithi167.stdout:(35/156): ceph-volume-19.0.0-3601.g30850680.el9 2.9 MB/s | 273 kB 00:00 2024-05-11T12:08:04.319 INFO:teuthology.orchestra.run.smithi167.stdout:(36/156): ceph-mgr-diskprediction-local-19.0.0- 7.8 MB/s | 7.4 MB 00:00 2024-05-11T12:08:04.352 INFO:teuthology.orchestra.run.smithi167.stdout:(37/156): cephadm-19.0.0-3601.g30850680.el9.noa 2.9 MB/s | 764 kB 00:00 2024-05-11T12:08:04.408 INFO:teuthology.orchestra.run.smithi167.stdout:(38/156): cephfs-top-19.0.0-3601.g30850680.el9. 208 kB/s | 18 kB 00:00 2024-05-11T12:08:04.514 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-05-11T12:08:04.758 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-05-11T12:08:04.759 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-05-11T12:08:05.203 INFO:teuthology.orchestra.run.smithi167.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 95 kB/s | 80 kB 00:00 2024-05-11T12:08:05.395 INFO:teuthology.orchestra.run.smithi167.stdout:(40/156): libgfortran-11.4.1-3.el9.x86_64.rpm 814 kB/s | 803 kB 00:00 2024-05-11T12:08:06.420 INFO:teuthology.orchestra.run.smithi167.stdout:(41/156): ceph-test-19.0.0-3601.g30850680.el9.x 11 MB/s | 49 MB 00:04 2024-05-11T12:08:06.537 INFO:teuthology.orchestra.run.smithi167.stdout:(42/156): libquadmath-11.4.1-3.el9.x86_64.rpm 147 kB/s | 195 kB 00:01 2024-05-11T12:08:06.804 INFO:teuthology.orchestra.run.smithi167.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 180 kB/s | 253 kB 00:01 2024-05-11T12:08:07.371 INFO:teuthology.orchestra.run.smithi167.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 128 kB/s | 106 kB 00:00 2024-05-11T12:08:07.371 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-05-11T12:08:07.372 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-05-11T12:08:07.488 INFO:teuthology.orchestra.run.smithi167.stdout:(45/156): python3-cryptography-36.0.1-4.el9.x86 1.1 MB/s | 1.2 MB 00:01 2024-05-11T12:08:07.788 INFO:teuthology.orchestra.run.smithi167.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 148 kB/s | 135 kB 00:00 2024-05-11T12:08:08.005 INFO:teuthology.orchestra.run.smithi167.stdout:(47/156): automake-1.16.2-8.el9.noarch.rpm 3.1 MB/s | 689 kB 00:00 2024-05-11T12:08:08.097 INFO:teuthology.orchestra.run.smithi167.stdout:(48/156): bison-3.7.4-5.el9.x86_64.rpm 10 MB/s | 943 kB 00:00 2024-05-11T12:08:08.138 INFO:teuthology.orchestra.run.smithi167.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 2.6 MB/s | 107 kB 00:00 2024-05-11T12:08:08.189 INFO:teuthology.orchestra.run.smithi167.stdout:(50/156): elfutils-libelf-devel-0.190-2.el9.x86 431 kB/s | 21 kB 00:00 2024-05-11T12:08:08.223 INFO:teuthology.orchestra.run.smithi167.stdout:(51/156): python3-requests-2.25.1-8.el9.noarch. 147 kB/s | 125 kB 00:00 2024-05-11T12:08:08.257 INFO:teuthology.orchestra.run.smithi167.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 4.6 MB/s | 313 kB 00:00 2024-05-11T12:08:08.332 INFO:teuthology.orchestra.run.smithi167.stdout:(53/156): python3-urllib3-1.26.5-5.el9.noarch.r 255 kB/s | 215 kB 00:00 2024-05-11T12:08:08.366 INFO:teuthology.orchestra.run.smithi167.stdout:(54/156): flexiblas-3.0.4-8.el9.x86_64.rpm 231 kB/s | 33 kB 00:00 2024-05-11T12:08:08.391 INFO:teuthology.orchestra.run.smithi167.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 313 kB/s | 18 kB 00:00 2024-05-11T12:08:08.441 INFO:teuthology.orchestra.run.smithi167.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 150 kB/s | 11 kB 00:00 2024-05-11T12:08:08.525 INFO:teuthology.orchestra.run.smithi167.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2024-05-11T12:08:08.550 INFO:teuthology.orchestra.run.smithi167.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 160 kB 00:00 2024-05-11T12:08:08.575 INFO:teuthology.orchestra.run.smithi167.stdout:(59/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 340 kB/s | 45 kB 00:00 2024-05-11T12:08:08.617 INFO:teuthology.orchestra.run.smithi167.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 7.1 MB/s | 662 kB 00:00 2024-05-11T12:08:08.634 INFO:teuthology.orchestra.run.smithi167.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 441 kB/s | 25 kB 00:00 2024-05-11T12:08:08.660 INFO:teuthology.orchestra.run.smithi167.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-05-11T12:08:08.701 INFO:teuthology.orchestra.run.smithi167.stdout:(63/156): libtool-2.4.6-45.el9.x86_64.rpm 3.8 MB/s | 582 kB 00:00 2024-05-11T12:08:08.760 INFO:teuthology.orchestra.run.smithi167.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-05-11T12:08:08.785 INFO:teuthology.orchestra.run.smithi167.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 424 kB/s | 35 kB 00:00 2024-05-11T12:08:08.819 INFO:teuthology.orchestra.run.smithi167.stdout:(66/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.6 MB/s | 292 kB 00:00 2024-05-11T12:08:08.945 INFO:teuthology.orchestra.run.smithi167.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-05-11T12:08:09.020 INFO:teuthology.orchestra.run.smithi167.stdout:(68/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 19 MB/s | 4.4 MB 00:00 2024-05-11T12:08:09.112 INFO:teuthology.orchestra.run.smithi167.stdout:(69/156): openblas-openmp-0.3.21-2.el9.x86_64.r 14 MB/s | 4.7 MB 00:00 2024-05-11T12:08:09.145 INFO:teuthology.orchestra.run.smithi167.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 1.0 MB/s | 133 kB 00:00 2024-05-11T12:08:09.196 INFO:teuthology.orchestra.run.smithi167.stdout:(71/156): postgresql-contrib-13.14-1.el9.x86_64 3.6 MB/s | 911 kB 00:00 2024-05-11T12:08:09.304 INFO:teuthology.orchestra.run.smithi167.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 6.4 MB/s | 1.0 MB 00:00 2024-05-11T12:08:09.329 INFO:teuthology.orchestra.run.smithi167.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 659 kB/s | 87 kB 00:00 2024-05-11T12:08:09.463 INFO:teuthology.orchestra.run.smithi167.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 1.8 MB/s | 244 kB 00:00 2024-05-11T12:08:09.530 INFO:teuthology.orchestra.run.smithi167.stdout:(75/156): python3-jinja2-2.11.3-5.el9.noarch.rp 3.7 MB/s | 248 kB 00:00 2024-05-11T12:08:09.580 INFO:teuthology.orchestra.run.smithi167.stdout:(76/156): python3-jsonpatch-1.21-16.el9.noarch. 522 kB/s | 26 kB 00:00 2024-05-11T12:08:09.731 INFO:teuthology.orchestra.run.smithi167.stdout:(77/156): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2024-05-11T12:08:09.756 INFO:teuthology.orchestra.run.smithi167.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 110 kB/s | 19 kB 00:00 2024-05-11T12:08:09.807 INFO:teuthology.orchestra.run.smithi167.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.3 MB/s | 172 kB 00:00 2024-05-11T12:08:09.832 INFO:teuthology.orchestra.run.smithi167.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 465 kB/s | 35 kB 00:00 2024-05-11T12:08:09.890 INFO:teuthology.orchestra.run.smithi167.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.3 MB/s | 250 kB 00:00 2024-05-11T12:08:09.999 INFO:teuthology.orchestra.run.smithi167.stdout:(82/156): postgresql-server-13.14-1.el9.x86_64. 6.5 MB/s | 5.8 MB 00:00 2024-05-11T12:08:10.033 INFO:teuthology.orchestra.run.smithi167.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.5 MB/s | 222 kB 00:00 2024-05-11T12:08:10.058 INFO:teuthology.orchestra.run.smithi167.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-05-11T12:08:10.083 INFO:teuthology.orchestra.run.smithi167.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 850 kB/s | 42 kB 00:00 2024-05-11T12:08:10.117 INFO:teuthology.orchestra.run.smithi167.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 269 kB 00:00 2024-05-11T12:08:10.171 INFO:teuthology.orchestra.run.smithi167.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.8 MB/s | 159 kB 00:00 2024-05-11T12:08:10.209 INFO:teuthology.orchestra.run.smithi167.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 3.0 MB/s | 279 kB 00:00 2024-05-11T12:08:10.234 INFO:teuthology.orchestra.run.smithi167.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 814 kB/s | 51 kB 00:00 2024-05-11T12:08:10.260 INFO:teuthology.orchestra.run.smithi167.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.1 MB/s | 54 kB 00:00 2024-05-11T12:08:10.302 INFO:teuthology.orchestra.run.smithi167.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-05-11T12:08:10.419 INFO:teuthology.orchestra.run.smithi167.stdout:(92/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 8.3 MB/s | 5.1 MB 00:00 2024-05-11T12:08:10.436 INFO:teuthology.orchestra.run.smithi167.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 333 kB/s | 44 kB 00:00 2024-05-11T12:08:10.494 INFO:teuthology.orchestra.run.smithi167.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 626 kB/s | 36 kB 00:00 2024-05-11T12:08:10.528 INFO:teuthology.orchestra.run.smithi167.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.0 MB/s | 221 kB 00:00 2024-05-11T12:08:10.545 INFO:teuthology.orchestra.run.smithi167.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 950 kB/s | 47 kB 00:00 2024-05-11T12:08:10.587 INFO:teuthology.orchestra.run.smithi167.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 5.2 MB/s | 305 kB 00:00 2024-05-11T12:08:10.612 INFO:teuthology.orchestra.run.smithi167.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 876 kB/s | 58 kB 00:00 2024-05-11T12:08:10.663 INFO:teuthology.orchestra.run.smithi167.stdout:(99/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2024-05-11T12:08:10.688 INFO:teuthology.orchestra.run.smithi167.stdout:(100/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 937 kB/s | 94 kB 00:00 2024-05-11T12:08:10.713 INFO:teuthology.orchestra.run.smithi167.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 904 kB/s | 45 kB 00:00 2024-05-11T12:08:10.772 INFO:teuthology.orchestra.run.smithi167.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 269 kB/s | 22 kB 00:00 2024-05-11T12:08:10.880 INFO:teuthology.orchestra.run.smithi167.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 5.1 MB/s | 863 kB 00:00 2024-05-11T12:08:11.065 INFO:teuthology.orchestra.run.smithi167.stdout:(104/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 20 MB/s | 16 MB 00:00 2024-05-11T12:08:11.331 INFO:teuthology.orchestra.run.smithi167.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 73 kB/s | 19 kB 00:00 2024-05-11T12:08:11.498 INFO:teuthology.orchestra.run.smithi167.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 500 kB/s | 309 kB 00:00 2024-05-11T12:08:11.612 INFO:teuthology.orchestra.run.smithi167.stdout:(107/156): abseil-cpp-20211102.0-3.el9.x86_64.r 654 kB/s | 548 kB 00:00 2024-05-11T12:08:11.807 INFO:teuthology.orchestra.run.smithi167.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 84 kB/s | 26 kB 00:00 2024-05-11T12:08:11.966 INFO:teuthology.orchestra.run.smithi167.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 138 kB/s | 49 kB 00:00 2024-05-11T12:08:12.158 INFO:teuthology.orchestra.run.smithi167.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 193 kB/s | 67 kB 00:00 2024-05-11T12:08:12.350 INFO:teuthology.orchestra.run.smithi167.stdout:(111/156): libarrow-9.0.0-11.el9.x86_64.rpm 4.3 MB/s | 4.4 MB 00:01 2024-05-11T12:08:12.393 INFO:teuthology.orchestra.run.smithi167.stdout:(112/156): luarocks-3.9.2-1.el9.noarch.rpm 355 kB/s | 151 kB 00:00 2024-05-11T12:08:12.742 INFO:teuthology.orchestra.run.smithi167.stdout:(113/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 124 kB/s | 43 kB 00:00 2024-05-11T12:08:12.834 INFO:teuthology.orchestra.run.smithi167.stdout:(114/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 1.2 MB/s | 839 kB 00:00 2024-05-11T12:08:12.951 INFO:teuthology.orchestra.run.smithi167.stdout:(115/156): python3-asyncssh-2.13.2-5.el9.noarch 914 kB/s | 548 kB 00:00 2024-05-11T12:08:13.040 INFO:teuthology.orchestra.run.smithi167.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 108 kB/s | 32 kB 00:00 2024-05-11T12:08:13.101 INFO:teuthology.orchestra.run.smithi167.stdout:(117/156): python3-certifi-2021.10.8-2.el9.noar 56 kB/s | 15 kB 00:00 2024-05-11T12:08:13.393 INFO:teuthology.orchestra.run.smithi167.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 389 kB/s | 172 kB 00:00 2024-05-11T12:08:13.552 INFO:teuthology.orchestra.run.smithi167.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 701 kB/s | 358 kB 00:00 2024-05-11T12:08:13.611 INFO:teuthology.orchestra.run.smithi167.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 596 kB/s | 303 kB 00:00 2024-05-11T12:08:13.661 INFO:teuthology.orchestra.run.smithi167.stdout:(121/156): python3-colorama-0.4.4-12.el9.noarch 132 kB/s | 35 kB 00:00 2024-05-11T12:08:14.094 INFO:teuthology.orchestra.run.smithi167.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 379 kB/s | 206 kB 00:00 2024-05-11T12:08:14.095 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-05-11T12:08:14.178 INFO:teuthology.orchestra.run.smithi167.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 280 kB/s | 144 kB 00:00 2024-05-11T12:08:14.207 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-11T12:08:14.287 INFO:teuthology.orchestra.run.smithi167.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 56 kB/s | 11 kB 00:00 2024-05-11T12:08:14.404 INFO:teuthology.orchestra.run.smithi167.stdout:(125/156): python3-grpcio-1.46.7-10.el9.x86_64. 2.6 MB/s | 2.0 MB 00:00 2024-05-11T12:08:14.425 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-11T12:08:14.465 INFO:teuthology.orchestra.run.smithi167.stdout:(126/156): python3-jaraco-classes-3.2.1-5.el9.n 62 kB/s | 18 kB 00:00 2024-05-11T12:08:14.547 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-11T12:08:14.604 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-11T12:08:14.604 INFO:teuthology.orchestra.run.smithi167.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 73 kB/s | 23 kB 00:00 2024-05-11T12:08:14.642 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-11T12:08:14.671 INFO:teuthology.orchestra.run.smithi167.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 73 kB/s | 19 kB 00:00 2024-05-11T12:08:14.738 INFO:teuthology.orchestra.run.smithi167.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 72 kB/s | 20 kB 00:00 2024-05-11T12:08:14.739 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-11T12:08:15.030 INFO:teuthology.orchestra.run.smithi167.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 129 kB/s | 46 kB 00:00 2024-05-11T12:08:15.072 INFO:teuthology.orchestra.run.smithi167.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 237 kB/s | 79 kB 00:00 2024-05-11T12:08:15.164 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-11T12:08:15.265 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : librados2-2:19.0.0-3601.g30850680.el9.x86_64 8/164 2024-05-11T12:08:15.281 INFO:teuthology.orchestra.run.smithi167.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 1.5 MB/s | 1.0 MB 00:00 2024-05-11T12:08:15.381 INFO:teuthology.orchestra.run.smithi167.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 165 kB/s | 58 kB 00:00 2024-05-11T12:08:15.474 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librados2-2:19.0.0-3601.g30850680.el9.x86_64 8/164 2024-05-11T12:08:15.507 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephfs2-2:19.0.0-3601.g30850680.el9.x86_64 9/164 2024-05-11T12:08:15.573 INFO:teuthology.orchestra.run.smithi167.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 56 kB/s | 16 kB 00:00 2024-05-11T12:08:15.607 INFO:teuthology.orchestra.run.smithi167.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 510 kB/s | 272 kB 00:00 2024-05-11T12:08:15.640 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libcephfs2-2:19.0.0-3601.g30850680.el9.x86_64 9/164 2024-05-11T12:08:15.674 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rados-2:19.0.0-3601.g30850680.el9.x86_64 10/164 2024-05-11T12:08:15.732 INFO:teuthology.orchestra.run.smithi167.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 257 kB/s | 90 kB 00:00 2024-05-11T12:08:15.832 INFO:teuthology.orchestra.run.smithi167.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 109 kB/s | 28 kB 00:00 2024-05-11T12:08:15.891 INFO:teuthology.orchestra.run.smithi167.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 108 kB/s | 31 kB 00:00 2024-05-11T12:08:15.971 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-11T12:08:16.029 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-11T12:08:16.054 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephsqlite-2:19.0.0-3601.g30850680.el9.x86_64 13/164 2024-05-11T12:08:16.166 INFO:teuthology.orchestra.run.smithi167.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 130 kB/s | 36 kB 00:00 2024-05-11T12:08:16.191 INFO:teuthology.orchestra.run.smithi167.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 411 kB/s | 188 kB 00:00 2024-05-11T12:08:16.198 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3601.g30850680.el9.x86_64 13/164 2024-05-11T12:08:16.208 INFO:teuthology.orchestra.run.smithi167.stdout:(141/156): python3-rsa-4.9-2.el9.noarch.rpm 158 kB/s | 59 kB 00:00 2024-05-11T12:08:16.230 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libradosstriper1-2:19.0.0-3601.g30850680.el9.x86 14/164 2024-05-11T12:08:16.316 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3601.g30850680.el9.x86 14/164 2024-05-11T12:08:16.348 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-11T12:08:16.455 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-11T12:08:16.509 INFO:teuthology.orchestra.run.smithi167.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 149 kB/s | 51 kB 00:00 2024-05-11T12:08:16.550 INFO:teuthology.orchestra.run.smithi167.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 263 kB/s | 90 kB 00:00 2024-05-11T12:08:16.749 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-11T12:08:16.847 INFO:teuthology.orchestra.run.smithi167.stdout:(144/156): python3-webob-1.8.7-6.el9.noarch.rpm 360 kB/s | 230 kB 00:00 2024-05-11T12:08:16.877 INFO:teuthology.orchestra.run.smithi167.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 61 kB/s | 20 kB 00:00 2024-05-11T12:08:17.063 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-11T12:08:17.194 INFO:teuthology.orchestra.run.smithi167.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 625 kB/s | 427 kB 00:00 2024-05-11T12:08:17.229 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-11T12:08:17.277 INFO:teuthology.orchestra.run.smithi167.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 446 kB/s | 192 kB 00:00 2024-05-11T12:08:17.326 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-11T12:08:17.516 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-11T12:08:17.586 INFO:teuthology.orchestra.run.smithi167.stdout:(148/156): librados2-19.0.0-3601.g30850680.el9. 8.7 MB/s | 3.4 MB 00:00 2024-05-11T12:08:17.625 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-11T12:08:17.652 INFO:teuthology.orchestra.run.smithi167.stdout:(149/156): thrift-0.15.0-2.el9.x86_64.rpm 2.0 MB/s | 1.6 MB 00:00 2024-05-11T12:08:17.719 INFO:teuthology.orchestra.run.smithi167.stdout:(150/156): librbd1-19.0.0-3601.g30850680.el9.x8 7.1 MB/s | 3.1 MB 00:00 2024-05-11T12:08:17.925 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-11T12:08:18.122 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-11T12:08:18.164 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-11T12:08:18.217 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-11T12:08:18.320 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-11T12:08:18.368 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-11T12:08:18.472 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-11T12:08:18.512 INFO:teuthology.orchestra.run.smithi167.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 53 kB/s | 42 kB 00:00 2024-05-11T12:08:18.524 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-11T12:08:18.563 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-11T12:08:18.680 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-11T12:08:18.752 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-11T12:08:18.790 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-11T12:08:18.846 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-11T12:08:18.854 INFO:teuthology.orchestra.run.smithi167.stdout:(152/156): NetworkManager-1.47.5-1.el9.x86_64.r 1.8 MB/s | 2.3 MB 00:01 2024-05-11T12:08:18.912 INFO:teuthology.orchestra.run.smithi167.stdout:(153/156): NetworkManager-libnm-1.47.5-1.el9.x8 1.5 MB/s | 1.8 MB 00:01 2024-05-11T12:08:18.985 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-11T12:08:19.029 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ceph-common-2:19.0.0-3601.g30850680.el9. 32/164 2024-05-11T12:08:19.075 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ceph-argparse-2:19.0.0-3601.g30850680.el 33/164 2024-05-11T12:08:19.108 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cephfs-2:19.0.0-3601.g30850680.el9.x86_6 34/164 2024-05-11T12:08:19.227 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-11T12:08:19.288 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-11T12:08:19.428 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-11T12:08:19.505 INFO:teuthology.orchestra.run.smithi167.stdout:(154/156): NetworkManager-tui-1.47.5-1.el9.x86_ 251 kB/s | 248 kB 00:00 2024-05-11T12:08:19.883 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-11T12:08:19.948 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-11T12:08:19.963 INFO:teuthology.orchestra.run.smithi167.stdout:(155/156): openssl-3.2.1-1.el9.x86_64.rpm 1.2 MB/s | 1.3 MB 00:01 2024-05-11T12:08:20.088 INFO:teuthology.orchestra.run.smithi167.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.0 MB/s | 2.4 MB 00:01 2024-05-11T12:08:20.096 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2024-05-11T12:08:20.098 INFO:teuthology.orchestra.run.smithi167.stdout:Total 10 MB/s | 226 MB 00:22 2024-05-11T12:08:20.643 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-11T12:08:20.760 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-11T12:08:20.799 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-11T12:08:20.834 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-11T12:08:20.875 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librados-devel-2:19.0.0-3601.g30850680.el9.x86_6 44/164 2024-05-11T12:08:20.947 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-11T12:08:21.173 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-11T12:08:21.284 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-11T12:08:21.405 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-11T12:08:21.478 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-11T12:08:21.562 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-11T12:08:21.611 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-11T12:08:21.649 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-11T12:08:21.714 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-11T12:08:21.758 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-11T12:08:21.820 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-11T12:08:21.879 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-11T12:08:22.072 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-11T12:08:22.131 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-11T12:08:22.177 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-11T12:08:22.234 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-11T12:08:22.362 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-11T12:08:22.410 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-11T12:08:22.459 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-11T12:08:22.492 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-11T12:08:22.525 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-11T12:08:22.775 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-11T12:08:22.961 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-11T12:08:23.007 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-11T12:08:23.054 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2024-05-11T12:08:23.076 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-11T12:08:23.125 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-11T12:08:23.182 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-11T12:08:23.235 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-11T12:08:23.288 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-11T12:08:23.296 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2024-05-11T12:08:23.296 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2024-05-11T12:08:23.332 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-11T12:08:23.382 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-11T12:08:23.456 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-11T12:08:23.505 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-11T12:08:23.648 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-11T12:08:23.709 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-11T12:08:23.753 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-11T12:08:23.824 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-11T12:08:23.970 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-11T12:08:24.033 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-11T12:08:25.024 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-11T12:08:25.099 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-11T12:08:25.133 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-11T12:08:25.260 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-11T12:08:25.319 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-11T12:08:25.417 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-11T12:08:25.468 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-11T12:08:25.703 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-11T12:08:25.740 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-11T12:08:25.846 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-11T12:08:25.913 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2024-05-11T12:08:25.914 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2024-05-11T12:08:26.686 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-11T12:08:26.822 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-11T12:08:28.461 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-11T12:08:28.555 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-11T12:08:28.590 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-11T12:08:28.651 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-11T12:08:29.020 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-11T12:08:29.068 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : librbd1-2:19.0.0-3601.g30850680.el9.x86_64 101/164 2024-05-11T12:08:29.199 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librbd1-2:19.0.0-3601.g30850680.el9.x86_64 101/164 2024-05-11T12:08:29.234 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rbd-2:19.0.0-3601.g30850680.el9.x86_64 102/164 2024-05-11T12:08:29.291 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-11T12:08:29.836 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-11T12:08:29.871 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librgw2-2:19.0.0-3601.g30850680.el9.x86_64 105/164 2024-05-11T12:08:29.972 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librgw2-2:19.0.0-3601.g30850680.el9.x86_64 105/164 2024-05-11T12:08:29.998 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rgw-2:19.0.0-3601.g30850680.el9.x86_64 106/164 2024-05-11T12:08:32.705 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2024-05-11T12:08:32.709 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 107/164 2024-05-11T12:08:32.745 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 107/164 2024-05-11T12:08:32.794 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-11T12:08:32.858 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-11T12:08:32.998 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-11T12:08:33.050 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-11T12:08:33.081 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-11T12:08:33.183 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-11T12:08:33.391 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 107/164 2024-05-11T12:08:33.432 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-base-2:19.0.0-3601.g30850680.el9.x86_64 108/164 2024-05-11T12:08:33.588 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-base-2:19.0.0-3601.g30850680.el9.x86_64 108/164 2024-05-11T12:08:33.589 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-11T12:08:33.589 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-11T12:08:33.589 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:08:33.605 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-11T12:08:33.668 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-selinux-2:19.0.0-3601.g30850680.el9.x86_64 109/164 2024-05-11T12:08:33.712 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : librados2-2:19.0.0-3601.g30850680.el9.x86_64 8/164 2024-05-11T12:08:33.878 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librados2-2:19.0.0-3601.g30850680.el9.x86_64 8/164 2024-05-11T12:08:33.913 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephfs2-2:19.0.0-3601.g30850680.el9.x86_64 9/164 2024-05-11T12:08:34.039 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libcephfs2-2:19.0.0-3601.g30850680.el9.x86_64 9/164 2024-05-11T12:08:34.071 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rados-2:19.0.0-3601.g30850680.el9.x86_64 10/164 2024-05-11T12:08:34.352 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-11T12:08:34.426 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-11T12:08:34.451 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephsqlite-2:19.0.0-3601.g30850680.el9.x86_64 13/164 2024-05-11T12:08:34.598 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3601.g30850680.el9.x86_64 13/164 2024-05-11T12:08:34.627 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libradosstriper1-2:19.0.0-3601.g30850680.el9.x86 14/164 2024-05-11T12:08:34.721 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3601.g30850680.el9.x86 14/164 2024-05-11T12:08:34.784 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-11T12:08:34.855 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-11T12:08:35.139 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-11T12:08:35.487 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-11T12:08:35.668 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-11T12:08:35.773 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-11T12:08:35.978 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-11T12:08:36.080 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-11T12:08:36.411 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-11T12:08:36.605 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-11T12:08:36.678 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-11T12:08:36.732 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-11T12:08:36.853 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-11T12:08:36.915 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-11T12:08:36.981 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-11T12:08:37.029 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-11T12:08:37.068 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-11T12:08:37.185 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-11T12:08:37.253 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-11T12:08:37.286 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-11T12:08:37.347 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-11T12:08:37.449 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-11T12:08:37.489 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ceph-common-2:19.0.0-3601.g30850680.el9. 32/164 2024-05-11T12:08:37.541 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ceph-argparse-2:19.0.0-3601.g30850680.el 33/164 2024-05-11T12:08:37.572 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cephfs-2:19.0.0-3601.g30850680.el9.x86_6 34/164 2024-05-11T12:08:37.712 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-11T12:08:37.769 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-11T12:08:37.877 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-11T12:08:38.051 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-11T12:08:38.120 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-11T12:08:38.800 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-11T12:08:38.909 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-11T12:08:38.953 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-11T12:08:39.030 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-11T12:08:39.082 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librados-devel-2:19.0.0-3601.g30850680.el9.x86_6 44/164 2024-05-11T12:08:39.150 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-11T12:08:39.359 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-11T12:08:39.422 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-11T12:08:39.536 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-11T12:08:39.608 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-11T12:08:39.692 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-11T12:08:39.731 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-11T12:08:39.760 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-11T12:08:39.824 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-11T12:08:39.862 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-11T12:08:39.920 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-11T12:08:40.005 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-11T12:08:40.206 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-11T12:08:40.260 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-11T12:08:40.297 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-11T12:08:40.342 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-11T12:08:40.468 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-11T12:08:40.515 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-11T12:08:40.563 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-11T12:08:40.596 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-11T12:08:40.629 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-11T12:08:40.811 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-11T12:08:40.990 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-11T12:08:41.037 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-11T12:08:41.073 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-11T12:08:41.119 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-11T12:08:41.175 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-11T12:08:41.230 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-11T12:08:41.272 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-11T12:08:41.326 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-11T12:08:41.402 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-11T12:08:41.480 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-11T12:08:41.526 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-11T12:08:41.667 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-11T12:08:41.738 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-11T12:08:41.783 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-11T12:08:41.881 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-11T12:08:42.038 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-11T12:08:42.130 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-11T12:08:43.087 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-11T12:08:43.162 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-11T12:08:43.196 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-11T12:08:43.359 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-11T12:08:43.417 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-11T12:08:43.522 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-11T12:08:43.566 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-11T12:08:43.800 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-11T12:08:43.838 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-11T12:08:43.934 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-11T12:08:44.753 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-11T12:08:44.901 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-11T12:08:46.476 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-11T12:08:46.517 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3601.g30850680.el9.x86_64 109/164 2024-05-11T12:08:46.517 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /sys 2024-05-11T12:08:46.517 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /proc 2024-05-11T12:08:46.517 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /mnt 2024-05-11T12:08:46.517 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /var/tmp 2024-05-11T12:08:46.517 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /home 2024-05-11T12:08:46.517 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /root 2024-05-11T12:08:46.518 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /tmp 2024-05-11T12:08:46.518 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:08:46.587 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-11T12:08:46.624 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-11T12:08:46.658 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-11T12:08:46.736 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-volume-2:19.0.0-3601.g30850680.el9.noarch 110/164 2024-05-11T12:08:46.784 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-volume-2:19.0.0-3601.g30850680.el9.noarch 110/164 2024-05-11T12:08:46.784 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T12:08:46.784 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-11T12:08:46.784 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:08:47.031 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-11T12:08:47.074 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : librbd1-2:19.0.0-3601.g30850680.el9.x86_64 101/164 2024-05-11T12:08:47.196 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librbd1-2:19.0.0-3601.g30850680.el9.x86_64 101/164 2024-05-11T12:08:47.230 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rbd-2:19.0.0-3601.g30850680.el9.x86_64 102/164 2024-05-11T12:08:47.287 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-11T12:08:47.832 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-11T12:08:47.869 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librgw2-2:19.0.0-3601.g30850680.el9.x86_64 105/164 2024-05-11T12:08:47.986 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librgw2-2:19.0.0-3601.g30850680.el9.x86_64 105/164 2024-05-11T12:08:48.012 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rgw-2:19.0.0-3601.g30850680.el9.x86_64 106/164 2024-05-11T12:08:48.419 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-osd-2:19.0.0-3601.g30850680.el9.x86_64 111/164 2024-05-11T12:08:48.499 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-osd-2:19.0.0-3601.g30850680.el9.x86_64 111/164 2024-05-11T12:08:48.500 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T12:08:48.500 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-11T12:08:48.500 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-11T12:08:48.500 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-11T12:08:48.500 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:08:48.764 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mds-2:19.0.0-3601.g30850680.el9.x86_64 112/164 2024-05-11T12:08:48.833 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mds-2:19.0.0-3601.g30850680.el9.x86_64 112/164 2024-05-11T12:08:48.833 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T12:08:48.833 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-11T12:08:48.834 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-11T12:08:48.834 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-11T12:08:48.834 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:08:49.315 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mon-2:19.0.0-3601.g30850680.el9.x86_64 113/164 2024-05-11T12:08:49.384 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mon-2:19.0.0-3601.g30850680.el9.x86_64 113/164 2024-05-11T12:08:49.385 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T12:08:49.385 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-11T12:08:49.385 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-11T12:08:49.385 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-11T12:08:49.385 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:08:49.524 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-11T12:08:49.588 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-11T12:08:49.662 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-11T12:08:49.898 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-11T12:08:49.970 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-11T12:08:50.056 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-11T12:08:50.165 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-11T12:08:50.332 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-11T12:08:50.391 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-11T12:08:50.469 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-11T12:08:50.501 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-11T12:08:50.745 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 107/164 2024-05-11T12:08:50.868 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 107/164 2024-05-11T12:08:51.291 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: cephadm-2:19.0.0-3601.g30850680.el9.noarch 125/164 2024-05-11T12:08:51.322 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephadm-2:19.0.0-3601.g30850680.el9.noarch 125/164 2024-05-11T12:08:51.377 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3601.g30850680.e 126/164 2024-05-11T12:08:51.486 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3601.g30850680. 127/164 2024-05-11T12:08:51.532 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3601.g30850680.el9.noa 128/164 2024-05-11T12:08:51.549 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 107/164 2024-05-11T12:08:51.597 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-base-2:19.0.0-3601.g30850680.el9.x86_64 108/164 2024-05-11T12:08:51.767 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-base-2:19.0.0-3601.g30850680.el9.x86_64 108/164 2024-05-11T12:08:51.767 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-11T12:08:51.767 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-11T12:08:51.768 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:08:51.831 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-selinux-2:19.0.0-3601.g30850680.el9.x86_64 109/164 2024-05-11T12:08:52.837 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3601.g30850680.el9.noa 128/164 2024-05-11T12:08:52.892 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3601.g30850680.el9.n 129/164 2024-05-11T12:08:54.003 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3601.g30850680.el9.n 129/164 2024-05-11T12:08:54.034 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3601.g308 130/164 2024-05-11T12:08:54.193 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3601.g308 130/164 2024-05-11T12:08:54.365 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3601.g30850680.el 131/164 2024-05-11T12:08:54.403 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-2:19.0.0-3601.g30850680.el9.x86_64 132/164 2024-05-11T12:08:54.473 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3601.g30850680.el9.x86_64 132/164 2024-05-11T12:08:54.473 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T12:08:54.474 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-11T12:08:54.474 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-11T12:08:54.474 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-11T12:08:54.474 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:08:54.537 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-rook-2:19.0.0-3601.g30850680.el9.noarch 133/164 2024-05-11T12:08:54.574 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3601.g30850680.el9.noarch 133/164 2024-05-11T12:08:54.719 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-2:19.0.0-3601.g30850680.el9.x86_64 134/164 2024-05-11T12:08:54.799 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-fuse-2:19.0.0-3601.g30850680.el9.x86_64 135/164 2024-05-11T12:08:55.815 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-11T12:08:55.856 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-radosgw-2:19.0.0-3601.g30850680.el9.x86_64 137/164 2024-05-11T12:08:55.928 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3601.g30850680.el9.x86_64 137/164 2024-05-11T12:08:55.928 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T12:08:55.928 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-11T12:08:55.928 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-11T12:08:55.929 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-11T12:08:55.929 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:08:56.007 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3601.g30850 138/164 2024-05-11T12:08:56.076 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3601.g30850 138/164 2024-05-11T12:08:56.077 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T12:08:56.077 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-11T12:08:56.077 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:08:56.149 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-mirror-2:19.0.0-3601.g30850680.el9.x86_64 139/164 2024-05-11T12:08:56.224 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3601.g30850680.el9.x86_64 139/164 2024-05-11T12:08:56.225 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T12:08:56.225 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-11T12:08:56.225 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-11T12:08:56.225 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-11T12:08:56.225 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:08:56.592 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-mirror-2:19.0.0-3601.g30850680.el9.x86_64 140/164 2024-05-11T12:08:56.668 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3601.g30850680.el9.x86_64 140/164 2024-05-11T12:08:56.669 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T12:08:56.669 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-11T12:08:56.669 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-11T12:08:56.669 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-11T12:08:56.669 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:09:01.740 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-test-2:19.0.0-3601.g30850680.el9.x86_64 141/164 2024-05-11T12:09:01.795 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-fuse-2:19.0.0-3601.g30850680.el9.x86_64 142/164 2024-05-11T12:09:01.882 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-nbd-2:19.0.0-3601.g30850680.el9.x86_64 143/164 2024-05-11T12:09:01.931 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-11T12:09:01.978 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-11T12:09:02.322 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-shell-2:19.0.0-3601.g30850680.el9.x86_64 146/164 2024-05-11T12:09:02.379 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-11T12:09:02.408 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephfs-devel-2:19.0.0-3601.g30850680.el9.x86_ 148/164 2024-05-11T12:09:02.453 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-11T12:09:02.501 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-11T12:09:03.413 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-11T12:09:03.456 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-11T12:09:03.520 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-11T12:09:04.857 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3601.g30850680.el9.x86_64 109/164 2024-05-11T12:09:04.857 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /sys 2024-05-11T12:09:04.857 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /proc 2024-05-11T12:09:04.857 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /mnt 2024-05-11T12:09:04.857 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /var/tmp 2024-05-11T12:09:04.857 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /home 2024-05-11T12:09:04.857 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /root 2024-05-11T12:09:04.857 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /tmp 2024-05-11T12:09:04.857 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:09:04.870 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-top-2:19.0.0-3601.g30850680.el9.noarch 152/164 2024-05-11T12:09:04.980 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-11T12:09:05.047 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-11T12:09:05.104 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-volume-2:19.0.0-3601.g30850680.el9.noarch 110/164 2024-05-11T12:09:05.158 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-volume-2:19.0.0-3601.g30850680.el9.noarch 110/164 2024-05-11T12:09:05.158 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T12:09:05.158 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-11T12:09:05.158 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:09:05.219 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-11T12:09:05.256 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-11T12:09:05.294 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-11T12:09:05.294 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-11T12:09:05.422 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-11T12:09:05.477 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-11T12:09:05.478 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-11T12:09:05.632 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-11T12:09:05.670 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-11T12:09:05.702 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-11T12:09:05.703 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-11T12:09:06.025 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-11T12:09:06.025 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-11T12:09:06.178 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-11T12:09:06.308 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-11T12:09:06.581 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-11T12:09:06.581 INFO:teuthology.orchestra.run.smithi204.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-05-11T12:09:06.581 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:09:06.799 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-osd-2:19.0.0-3601.g30850680.el9.x86_64 111/164 2024-05-11T12:09:06.877 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-osd-2:19.0.0-3601.g30850680.el9.x86_64 111/164 2024-05-11T12:09:06.877 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T12:09:06.877 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-11T12:09:06.877 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-11T12:09:06.878 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-11T12:09:06.878 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:09:07.124 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mds-2:19.0.0-3601.g30850680.el9.x86_64 112/164 2024-05-11T12:09:07.201 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mds-2:19.0.0-3601.g30850680.el9.x86_64 112/164 2024-05-11T12:09:07.202 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T12:09:07.202 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-11T12:09:07.202 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-11T12:09:07.202 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-11T12:09:07.202 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:09:07.683 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mon-2:19.0.0-3601.g30850680.el9.x86_64 113/164 2024-05-11T12:09:07.777 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mon-2:19.0.0-3601.g30850680.el9.x86_64 113/164 2024-05-11T12:09:07.777 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T12:09:07.777 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-11T12:09:07.777 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-11T12:09:07.778 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-11T12:09:07.778 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:09:07.909 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-11T12:09:07.971 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-11T12:09:08.044 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-11T12:09:08.267 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-11T12:09:08.341 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-11T12:09:08.431 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-11T12:09:08.540 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-11T12:09:08.704 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-11T12:09:08.760 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-11T12:09:08.830 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-11T12:09:08.861 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-11T12:09:09.544 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: cephadm-2:19.0.0-3601.g30850680.el9.noarch 125/164 2024-05-11T12:09:09.582 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephadm-2:19.0.0-3601.g30850680.el9.noarch 125/164 2024-05-11T12:09:09.640 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3601.g30850680.e 126/164 2024-05-11T12:09:09.745 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3601.g30850680. 127/164 2024-05-11T12:09:09.800 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3601.g30850680.el9.noa 128/164 2024-05-11T12:09:11.126 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3601.g30850680.el9.noa 128/164 2024-05-11T12:09:11.193 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3601.g30850680.el9.n 129/164 2024-05-11T12:09:12.302 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3601.g30850680.el9.n 129/164 2024-05-11T12:09:12.326 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3601.g308 130/164 2024-05-11T12:09:12.482 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3601.g308 130/164 2024-05-11T12:09:12.554 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-2:19.0.0-3601.g30850680.el9.x86_64 1/164 2024-05-11T12:09:12.554 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-base-2:19.0.0-3601.g30850680.el9.x86_64 2/164 2024-05-11T12:09:12.554 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 3/164 2024-05-11T12:09:12.554 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-fuse-2:19.0.0-3601.g30850680.el9.x86_64 4/164 2024-05-11T12:09:12.554 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3601.g30850 5/164 2024-05-11T12:09:12.554 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mds-2:19.0.0-3601.g30850680.el9.x86_64 6/164 2024-05-11T12:09:12.554 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-2:19.0.0-3601.g30850680.el9.x86_64 7/164 2024-05-11T12:09:12.554 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mon-2:19.0.0-3601.g30850680.el9.x86_64 8/164 2024-05-11T12:09:12.555 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-osd-2:19.0.0-3601.g30850680.el9.x86_64 9/164 2024-05-11T12:09:12.555 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-radosgw-2:19.0.0-3601.g30850680.el9.x86_64 10/164 2024-05-11T12:09:12.555 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-selinux-2:19.0.0-3601.g30850680.el9.x86_64 11/164 2024-05-11T12:09:12.555 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-test-2:19.0.0-3601.g30850680.el9.x86_64 12/164 2024-05-11T12:09:12.555 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-mirror-2:19.0.0-3601.g30850680.el9.x86_64 13/164 2024-05-11T12:09:12.555 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-shell-2:19.0.0-3601.g30850680.el9.x86_64 14/164 2024-05-11T12:09:12.555 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephfs-devel-2:19.0.0-3601.g30850680.el9.x86_ 15/164 2024-05-11T12:09:12.555 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephfs2-2:19.0.0-3601.g30850680.el9.x86_64 16/164 2024-05-11T12:09:12.555 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephsqlite-2:19.0.0-3601.g30850680.el9.x86_64 17/164 2024-05-11T12:09:12.555 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados-devel-2:19.0.0-3601.g30850680.el9.x86_6 18/164 2024-05-11T12:09:12.555 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libradosstriper1-2:19.0.0-3601.g30850680.el9.x86 19/164 2024-05-11T12:09:12.555 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librgw2-2:19.0.0-3601.g30850680.el9.x86_64 20/164 2024-05-11T12:09:12.555 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3601.g30850680.el 21/164 2024-05-11T12:09:12.555 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ceph-common-2:19.0.0-3601.g30850680.el9. 22/164 2024-05-11T12:09:12.555 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cephfs-2:19.0.0-3601.g30850680.el9.x86_6 23/164 2024-05-11T12:09:12.556 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rados-2:19.0.0-3601.g30850680.el9.x86_64 24/164 2024-05-11T12:09:12.556 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rbd-2:19.0.0-3601.g30850680.el9.x86_64 25/164 2024-05-11T12:09:12.556 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rgw-2:19.0.0-3601.g30850680.el9.x86_64 26/164 2024-05-11T12:09:12.556 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-fuse-2:19.0.0-3601.g30850680.el9.x86_64 27/164 2024-05-11T12:09:12.568 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-mirror-2:19.0.0-3601.g30850680.el9.x86_64 28/164 2024-05-11T12:09:12.568 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-nbd-2:19.0.0-3601.g30850680.el9.x86_64 29/164 2024-05-11T12:09:12.569 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3601.g30850680. 30/164 2024-05-11T12:09:12.569 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3601.g30850680.el9.noa 31/164 2024-05-11T12:09:12.569 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3601.g30850680.el9.n 32/164 2024-05-11T12:09:12.569 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3601.g308 33/164 2024-05-11T12:09:12.569 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3601.g30850680.el 34/164 2024-05-11T12:09:12.569 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3601.g30850680.el9.noarch 35/164 2024-05-11T12:09:12.569 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3601.g30850680.e 36/164 2024-05-11T12:09:12.569 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-volume-2:19.0.0-3601.g30850680.el9.noarch 37/164 2024-05-11T12:09:12.569 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephadm-2:19.0.0-3601.g30850680.el9.noarch 38/164 2024-05-11T12:09:12.569 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-top-2:19.0.0-3601.g30850680.el9.noarch 39/164 2024-05-11T12:09:12.569 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-11T12:09:12.569 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-11T12:09:12.569 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-11T12:09:12.569 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-11T12:09:12.570 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-11T12:09:12.570 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-11T12:09:12.570 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-11T12:09:12.570 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-11T12:09:12.570 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-11T12:09:12.570 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-11T12:09:12.570 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-11T12:09:12.570 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-11T12:09:12.570 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-11T12:09:12.570 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-11T12:09:12.570 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-11T12:09:12.570 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-11T12:09:12.570 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-11T12:09:12.571 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-11T12:09:12.571 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-11T12:09:12.571 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-11T12:09:12.571 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-11T12:09:12.571 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-11T12:09:12.571 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-11T12:09:12.571 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-11T12:09:12.571 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-11T12:09:12.571 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-11T12:09:12.571 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-11T12:09:12.571 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-11T12:09:12.571 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-11T12:09:12.571 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-11T12:09:12.571 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-11T12:09:12.572 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-11T12:09:12.572 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-11T12:09:12.572 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-11T12:09:12.572 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-11T12:09:12.572 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-11T12:09:12.572 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-11T12:09:12.572 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-11T12:09:12.572 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-11T12:09:12.572 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-11T12:09:12.572 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-11T12:09:12.572 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-11T12:09:12.572 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-11T12:09:12.572 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-11T12:09:12.572 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-11T12:09:12.573 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-11T12:09:12.573 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-11T12:09:12.573 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-11T12:09:12.573 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-11T12:09:12.573 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-11T12:09:12.573 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-11T12:09:12.573 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-11T12:09:12.573 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-11T12:09:12.573 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-11T12:09:12.573 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-11T12:09:12.573 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-11T12:09:12.573 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-11T12:09:12.574 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-11T12:09:12.574 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-11T12:09:12.574 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-11T12:09:12.574 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-11T12:09:12.574 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-11T12:09:12.574 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-11T12:09:12.574 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-11T12:09:12.574 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-11T12:09:12.574 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-11T12:09:12.574 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-11T12:09:12.574 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-11T12:09:12.574 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-11T12:09:12.574 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-11T12:09:12.574 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-11T12:09:12.575 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-11T12:09:12.575 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-11T12:09:12.575 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-11T12:09:12.575 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-11T12:09:12.575 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-11T12:09:12.575 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-11T12:09:12.575 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-11T12:09:12.575 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-11T12:09:12.575 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-11T12:09:12.575 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-11T12:09:12.575 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-11T12:09:12.575 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-11T12:09:12.575 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-11T12:09:12.575 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-11T12:09:12.576 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-11T12:09:12.576 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-11T12:09:12.576 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-11T12:09:12.576 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-11T12:09:12.576 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-11T12:09:12.576 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-11T12:09:12.576 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-11T12:09:12.576 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-11T12:09:12.576 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-11T12:09:12.576 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-11T12:09:12.576 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-11T12:09:12.576 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-11T12:09:12.576 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-11T12:09:12.576 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-11T12:09:12.577 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-11T12:09:12.577 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-11T12:09:12.577 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-11T12:09:12.577 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-11T12:09:12.577 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-11T12:09:12.577 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-11T12:09:12.577 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-11T12:09:12.577 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-11T12:09:12.577 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-11T12:09:12.577 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-11T12:09:12.577 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados2-2:19.0.0-3601.g30850680.el9.x86_64 149/164 2024-05-11T12:09:12.577 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-11T12:09:12.577 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librbd1-2:19.0.0-3601.g30850680.el9.x86_64 151/164 2024-05-11T12:09:12.577 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-11T12:09:12.577 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-11T12:09:12.578 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-11T12:09:12.578 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-11T12:09:12.578 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-11T12:09:12.578 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-11T12:09:12.578 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-11T12:09:12.578 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-11T12:09:12.578 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-11T12:09:12.578 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-11T12:09:12.578 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-11T12:09:12.578 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-11T12:09:12.658 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3601.g30850680.el 131/164 2024-05-11T12:09:12.695 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-2:19.0.0-3601.g30850680.el9.x86_64 132/164 2024-05-11T12:09:12.765 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3601.g30850680.el9.x86_64 132/164 2024-05-11T12:09:12.765 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T12:09:12.765 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-11T12:09:12.765 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-11T12:09:12.765 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-11T12:09:12.766 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:09:12.821 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-rook-2:19.0.0-3601.g30850680.el9.noarch 133/164 2024-05-11T12:09:12.858 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3601.g30850680.el9.noarch 133/164 2024-05-11T12:09:13.000 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-2:19.0.0-3601.g30850680.el9.x86_64 134/164 2024-05-11T12:09:13.078 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-fuse-2:19.0.0-3601.g30850680.el9.x86_64 135/164 2024-05-11T12:09:14.109 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-11T12:09:14.148 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-radosgw-2:19.0.0-3601.g30850680.el9.x86_64 137/164 2024-05-11T12:09:14.217 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3601.g30850680.el9.x86_64 137/164 2024-05-11T12:09:14.217 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T12:09:14.217 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-11T12:09:14.217 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-11T12:09:14.217 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-11T12:09:14.217 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:09:14.257 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3601.g30850 138/164 2024-05-11T12:09:14.321 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3601.g30850 138/164 2024-05-11T12:09:14.321 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T12:09:14.321 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-11T12:09:14.321 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:09:14.392 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephfs-mirror-2:19.0.0-3601.g30850680.el9.x86_64 139/164 2024-05-11T12:09:14.466 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3601.g30850680.el9.x86_64 139/164 2024-05-11T12:09:14.466 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T12:09:14.466 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-11T12:09:14.466 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-11T12:09:14.466 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-11T12:09:14.466 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:09:14.790 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-11T12:09:14.790 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:09:14.790 INFO:teuthology.orchestra.run.smithi204.stdout:Upgraded: 2024-05-11T12:09:14.790 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-11T12:09:14.790 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-11T12:09:14.790 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-11T12:09:14.791 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-11T12:09:14.791 INFO:teuthology.orchestra.run.smithi204.stdout: librados2-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:14.791 INFO:teuthology.orchestra.run.smithi204.stdout: librbd1-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:14.791 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-11T12:09:14.791 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-11T12:09:14.791 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-05-11T12:09:14.791 INFO:teuthology.orchestra.run.smithi204.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-11T12:09:14.791 INFO:teuthology.orchestra.run.smithi204.stdout: automake-1.16.2-8.el9.noarch 2024-05-11T12:09:14.791 INFO:teuthology.orchestra.run.smithi204.stdout: bison-3.7.4-5.el9.x86_64 2024-05-11T12:09:14.791 INFO:teuthology.orchestra.run.smithi204.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-11T12:09:14.791 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:14.791 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-base-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:14.791 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:14.791 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-fuse-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:14.791 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-grafana-dashboards-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:09:14.791 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-immutable-object-cache-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:14.792 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mds-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:14.792 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:14.792 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-cephadm-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:09:14.792 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-dashboard-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:09:14.792 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:09:14.792 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-modules-core-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:09:14.792 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-rook-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:09:14.792 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mon-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:14.792 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-osd-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:14.792 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-prometheus-alerts-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:09:14.792 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-radosgw-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:14.792 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-selinux-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:14.792 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-test-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:14.792 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-volume-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:09:14.792 INFO:teuthology.orchestra.run.smithi204.stdout: cephadm-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:09:14.792 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-mirror-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:14.793 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-shell-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:14.793 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-top-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:09:14.793 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-11T12:09:14.793 INFO:teuthology.orchestra.run.smithi204.stdout: flex-2.6.4-9.el9.x86_64 2024-05-11T12:09:14.793 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-11T12:09:14.793 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-11T12:09:14.793 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-11T12:09:14.793 INFO:teuthology.orchestra.run.smithi204.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-11T12:09:14.793 INFO:teuthology.orchestra.run.smithi204.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-11T12:09:14.793 INFO:teuthology.orchestra.run.smithi204.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-11T12:09:14.793 INFO:teuthology.orchestra.run.smithi204.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-11T12:09:14.793 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-11T12:09:14.793 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-11T12:09:14.793 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs-devel-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:14.793 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs2-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:14.794 INFO:teuthology.orchestra.run.smithi204.stdout: libcephsqlite-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:14.794 INFO:teuthology.orchestra.run.smithi204.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-11T12:09:14.794 INFO:teuthology.orchestra.run.smithi204.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-11T12:09:14.794 INFO:teuthology.orchestra.run.smithi204.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-11T12:09:14.794 INFO:teuthology.orchestra.run.smithi204.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-11T12:09:14.794 INFO:teuthology.orchestra.run.smithi204.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-11T12:09:14.794 INFO:teuthology.orchestra.run.smithi204.stdout: librados-devel-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:14.794 INFO:teuthology.orchestra.run.smithi204.stdout: libradosstriper1-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:14.794 INFO:teuthology.orchestra.run.smithi204.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-11T12:09:14.794 INFO:teuthology.orchestra.run.smithi204.stdout: librgw2-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:14.794 INFO:teuthology.orchestra.run.smithi204.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-11T12:09:14.794 INFO:teuthology.orchestra.run.smithi204.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-11T12:09:14.794 INFO:teuthology.orchestra.run.smithi204.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-11T12:09:14.794 INFO:teuthology.orchestra.run.smithi204.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-11T12:09:14.794 INFO:teuthology.orchestra.run.smithi204.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-11T12:09:14.794 INFO:teuthology.orchestra.run.smithi204.stdout: lua-5.4.4-4.el9.x86_64 2024-05-11T12:09:14.795 INFO:teuthology.orchestra.run.smithi204.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-11T12:09:14.795 INFO:teuthology.orchestra.run.smithi204.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-11T12:09:14.795 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-11T12:09:14.795 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-11T12:09:14.795 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-11T12:09:14.795 INFO:teuthology.orchestra.run.smithi204.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-11T12:09:14.795 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-13.14-1.el9.x86_64 2024-05-11T12:09:14.795 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-11T12:09:14.795 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-11T12:09:14.795 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-11T12:09:14.795 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-11T12:09:14.795 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-11T12:09:14.795 INFO:teuthology.orchestra.run.smithi204.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-11T12:09:14.795 INFO:teuthology.orchestra.run.smithi204.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-11T12:09:14.795 INFO:teuthology.orchestra.run.smithi204.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-11T12:09:14.795 INFO:teuthology.orchestra.run.smithi204.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-11T12:09:14.796 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-11T12:09:14.796 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-argparse-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:14.796 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:14.796 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cephfs-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:14.796 INFO:teuthology.orchestra.run.smithi204.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-11T12:09:14.796 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-11T12:09:14.796 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-11T12:09:14.796 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-11T12:09:14.796 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-11T12:09:14.796 INFO:teuthology.orchestra.run.smithi204.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-11T12:09:14.796 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-11T12:09:14.796 INFO:teuthology.orchestra.run.smithi204.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-11T12:09:14.796 INFO:teuthology.orchestra.run.smithi204.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-11T12:09:14.796 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-11T12:09:14.796 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-11T12:09:14.797 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-11T12:09:14.797 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-11T12:09:14.797 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-11T12:09:14.797 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-11T12:09:14.797 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-11T12:09:14.797 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-11T12:09:14.797 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-11T12:09:14.797 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-11T12:09:14.797 INFO:teuthology.orchestra.run.smithi204.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-11T12:09:14.797 INFO:teuthology.orchestra.run.smithi204.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-11T12:09:14.797 INFO:teuthology.orchestra.run.smithi204.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-11T12:09:14.797 INFO:teuthology.orchestra.run.smithi204.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-11T12:09:14.797 INFO:teuthology.orchestra.run.smithi204.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-11T12:09:14.797 INFO:teuthology.orchestra.run.smithi204.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-11T12:09:14.797 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-11T12:09:14.797 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-11T12:09:14.798 INFO:teuthology.orchestra.run.smithi204.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-11T12:09:14.798 INFO:teuthology.orchestra.run.smithi204.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-11T12:09:14.798 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-11T12:09:14.798 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ply-3.11-14.el9.noarch 2024-05-11T12:09:14.798 INFO:teuthology.orchestra.run.smithi204.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-11T12:09:14.798 INFO:teuthology.orchestra.run.smithi204.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-11T12:09:14.798 INFO:teuthology.orchestra.run.smithi204.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-11T12:09:14.798 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-11T12:09:14.798 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-11T12:09:14.798 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-11T12:09:14.798 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-11T12:09:14.798 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-11T12:09:14.798 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-11T12:09:14.798 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rados-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:14.798 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rbd-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:14.799 INFO:teuthology.orchestra.run.smithi204.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-11T12:09:14.799 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-11T12:09:14.799 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-11T12:09:14.799 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rgw-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:14.799 INFO:teuthology.orchestra.run.smithi204.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-11T12:09:14.799 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-11T12:09:14.799 INFO:teuthology.orchestra.run.smithi204.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-11T12:09:14.799 INFO:teuthology.orchestra.run.smithi204.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-11T12:09:14.799 INFO:teuthology.orchestra.run.smithi204.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-11T12:09:14.799 INFO:teuthology.orchestra.run.smithi204.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-11T12:09:14.799 INFO:teuthology.orchestra.run.smithi204.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-11T12:09:14.799 INFO:teuthology.orchestra.run.smithi204.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-11T12:09:14.799 INFO:teuthology.orchestra.run.smithi204.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-11T12:09:14.799 INFO:teuthology.orchestra.run.smithi204.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-11T12:09:14.799 INFO:teuthology.orchestra.run.smithi204.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-11T12:09:14.799 INFO:teuthology.orchestra.run.smithi204.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-11T12:09:14.800 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-11T12:09:14.800 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-11T12:09:14.800 INFO:teuthology.orchestra.run.smithi204.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-11T12:09:14.800 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-fuse-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:14.800 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-mirror-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:14.800 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-nbd-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:14.800 INFO:teuthology.orchestra.run.smithi204.stdout: re2-1:20211101-3.el9.x86_64 2024-05-11T12:09:14.800 INFO:teuthology.orchestra.run.smithi204.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-11T12:09:14.800 INFO:teuthology.orchestra.run.smithi204.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-11T12:09:14.800 INFO:teuthology.orchestra.run.smithi204.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-11T12:09:14.800 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-11T12:09:14.800 INFO:teuthology.orchestra.run.smithi204.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-11T12:09:14.800 INFO:teuthology.orchestra.run.smithi204.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-11T12:09:14.801 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:09:14.801 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-05-11T12:09:14.825 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-mirror-2:19.0.0-3601.g30850680.el9.x86_64 140/164 2024-05-11T12:09:14.898 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3601.g30850680.el9.x86_64 140/164 2024-05-11T12:09:14.898 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T12:09:14.898 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-11T12:09:14.898 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-11T12:09:14.899 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-11T12:09:14.899 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:09:15.449 DEBUG:teuthology.parallel:result is None 2024-05-11T12:09:19.976 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-test-2:19.0.0-3601.g30850680.el9.x86_64 141/164 2024-05-11T12:09:20.033 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-fuse-2:19.0.0-3601.g30850680.el9.x86_64 142/164 2024-05-11T12:09:20.096 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-nbd-2:19.0.0-3601.g30850680.el9.x86_64 143/164 2024-05-11T12:09:20.133 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-11T12:09:20.196 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-11T12:09:20.505 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephfs-shell-2:19.0.0-3601.g30850680.el9.x86_64 146/164 2024-05-11T12:09:20.563 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-11T12:09:20.592 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephfs-devel-2:19.0.0-3601.g30850680.el9.x86_ 148/164 2024-05-11T12:09:20.662 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-11T12:09:20.710 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-11T12:09:21.641 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-11T12:09:21.690 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-11T12:09:21.745 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-11T12:09:23.085 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephfs-top-2:19.0.0-3601.g30850680.el9.noarch 152/164 2024-05-11T12:09:23.189 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-11T12:09:23.258 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-11T12:09:23.399 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-11T12:09:23.463 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-11T12:09:23.503 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-11T12:09:23.503 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-11T12:09:23.627 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-11T12:09:23.670 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-11T12:09:23.670 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-11T12:09:23.807 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-11T12:09:23.845 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-11T12:09:23.878 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-11T12:09:23.878 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-11T12:09:24.184 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-11T12:09:24.184 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-11T12:09:24.320 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-11T12:09:24.450 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-11T12:09:24.703 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-11T12:09:24.703 INFO:teuthology.orchestra.run.smithi167.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-05-11T12:09:24.703 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:09:30.646 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-2:19.0.0-3601.g30850680.el9.x86_64 1/164 2024-05-11T12:09:30.646 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-base-2:19.0.0-3601.g30850680.el9.x86_64 2/164 2024-05-11T12:09:30.646 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 3/164 2024-05-11T12:09:30.647 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-fuse-2:19.0.0-3601.g30850680.el9.x86_64 4/164 2024-05-11T12:09:30.647 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3601.g30850 5/164 2024-05-11T12:09:30.647 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mds-2:19.0.0-3601.g30850680.el9.x86_64 6/164 2024-05-11T12:09:30.647 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-2:19.0.0-3601.g30850680.el9.x86_64 7/164 2024-05-11T12:09:30.647 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mon-2:19.0.0-3601.g30850680.el9.x86_64 8/164 2024-05-11T12:09:30.647 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-osd-2:19.0.0-3601.g30850680.el9.x86_64 9/164 2024-05-11T12:09:30.647 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-radosgw-2:19.0.0-3601.g30850680.el9.x86_64 10/164 2024-05-11T12:09:30.647 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-selinux-2:19.0.0-3601.g30850680.el9.x86_64 11/164 2024-05-11T12:09:30.647 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-test-2:19.0.0-3601.g30850680.el9.x86_64 12/164 2024-05-11T12:09:30.647 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephfs-mirror-2:19.0.0-3601.g30850680.el9.x86_64 13/164 2024-05-11T12:09:30.647 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephfs-shell-2:19.0.0-3601.g30850680.el9.x86_64 14/164 2024-05-11T12:09:30.647 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephfs-devel-2:19.0.0-3601.g30850680.el9.x86_ 15/164 2024-05-11T12:09:30.647 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephfs2-2:19.0.0-3601.g30850680.el9.x86_64 16/164 2024-05-11T12:09:30.648 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephsqlite-2:19.0.0-3601.g30850680.el9.x86_64 17/164 2024-05-11T12:09:30.648 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados-devel-2:19.0.0-3601.g30850680.el9.x86_6 18/164 2024-05-11T12:09:30.648 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libradosstriper1-2:19.0.0-3601.g30850680.el9.x86 19/164 2024-05-11T12:09:30.648 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librgw2-2:19.0.0-3601.g30850680.el9.x86_64 20/164 2024-05-11T12:09:30.648 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3601.g30850680.el 21/164 2024-05-11T12:09:30.648 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ceph-common-2:19.0.0-3601.g30850680.el9. 22/164 2024-05-11T12:09:30.648 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cephfs-2:19.0.0-3601.g30850680.el9.x86_6 23/164 2024-05-11T12:09:30.648 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rados-2:19.0.0-3601.g30850680.el9.x86_64 24/164 2024-05-11T12:09:30.648 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rbd-2:19.0.0-3601.g30850680.el9.x86_64 25/164 2024-05-11T12:09:30.648 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rgw-2:19.0.0-3601.g30850680.el9.x86_64 26/164 2024-05-11T12:09:30.648 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-fuse-2:19.0.0-3601.g30850680.el9.x86_64 27/164 2024-05-11T12:09:30.661 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-mirror-2:19.0.0-3601.g30850680.el9.x86_64 28/164 2024-05-11T12:09:30.661 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-nbd-2:19.0.0-3601.g30850680.el9.x86_64 29/164 2024-05-11T12:09:30.661 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3601.g30850680. 30/164 2024-05-11T12:09:30.661 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3601.g30850680.el9.noa 31/164 2024-05-11T12:09:30.661 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3601.g30850680.el9.n 32/164 2024-05-11T12:09:30.661 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3601.g308 33/164 2024-05-11T12:09:30.661 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3601.g30850680.el 34/164 2024-05-11T12:09:30.661 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3601.g30850680.el9.noarch 35/164 2024-05-11T12:09:30.661 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3601.g30850680.e 36/164 2024-05-11T12:09:30.662 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-volume-2:19.0.0-3601.g30850680.el9.noarch 37/164 2024-05-11T12:09:30.662 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephadm-2:19.0.0-3601.g30850680.el9.noarch 38/164 2024-05-11T12:09:30.662 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephfs-top-2:19.0.0-3601.g30850680.el9.noarch 39/164 2024-05-11T12:09:30.662 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-11T12:09:30.662 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-11T12:09:30.662 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-11T12:09:30.662 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-11T12:09:30.662 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-11T12:09:30.662 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-11T12:09:30.662 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-11T12:09:30.662 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-11T12:09:30.662 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-11T12:09:30.662 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-11T12:09:30.662 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-11T12:09:30.663 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-11T12:09:30.663 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-11T12:09:30.663 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-11T12:09:30.663 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-11T12:09:30.663 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-11T12:09:30.663 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-11T12:09:30.663 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-11T12:09:30.663 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-11T12:09:30.663 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-11T12:09:30.663 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-11T12:09:30.663 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-11T12:09:30.663 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-11T12:09:30.663 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-11T12:09:30.664 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-11T12:09:30.664 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-11T12:09:30.664 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-11T12:09:30.664 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-11T12:09:30.664 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-11T12:09:30.664 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-11T12:09:30.664 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-11T12:09:30.664 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-11T12:09:30.664 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-11T12:09:30.664 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-11T12:09:30.664 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-11T12:09:30.664 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-11T12:09:30.664 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-11T12:09:30.665 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-11T12:09:30.665 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-11T12:09:30.665 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-11T12:09:30.665 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-11T12:09:30.665 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-11T12:09:30.665 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-11T12:09:30.665 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-11T12:09:30.665 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-11T12:09:30.665 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-11T12:09:30.665 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-11T12:09:30.665 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-11T12:09:30.665 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-11T12:09:30.665 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-11T12:09:30.666 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-11T12:09:30.666 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-11T12:09:30.666 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-11T12:09:30.666 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-11T12:09:30.666 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-11T12:09:30.666 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-11T12:09:30.666 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-11T12:09:30.666 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-11T12:09:30.666 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-11T12:09:30.666 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-11T12:09:30.666 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-11T12:09:30.666 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-11T12:09:30.666 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-11T12:09:30.667 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-11T12:09:30.667 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-11T12:09:30.667 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-11T12:09:30.667 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-11T12:09:30.667 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-11T12:09:30.667 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-11T12:09:30.667 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-11T12:09:30.667 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-11T12:09:30.667 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-11T12:09:30.667 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-11T12:09:30.667 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-11T12:09:30.667 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-11T12:09:30.667 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-11T12:09:30.667 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-11T12:09:30.667 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-11T12:09:30.668 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-11T12:09:30.668 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-11T12:09:30.668 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-11T12:09:30.668 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-11T12:09:30.668 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-11T12:09:30.668 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-11T12:09:30.668 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-11T12:09:30.668 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-11T12:09:30.668 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-11T12:09:30.668 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-11T12:09:30.668 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-11T12:09:30.668 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-11T12:09:30.668 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-11T12:09:30.668 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-11T12:09:30.669 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-11T12:09:30.669 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-11T12:09:30.669 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-11T12:09:30.669 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-11T12:09:30.669 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-11T12:09:30.669 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-11T12:09:30.669 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-11T12:09:30.669 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-11T12:09:30.669 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-11T12:09:30.669 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-11T12:09:30.669 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-11T12:09:30.669 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-11T12:09:30.669 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-11T12:09:30.669 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-11T12:09:30.670 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-11T12:09:30.670 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-11T12:09:30.670 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-11T12:09:30.670 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados2-2:19.0.0-3601.g30850680.el9.x86_64 149/164 2024-05-11T12:09:30.670 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-11T12:09:30.670 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librbd1-2:19.0.0-3601.g30850680.el9.x86_64 151/164 2024-05-11T12:09:30.670 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-11T12:09:30.670 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-11T12:09:30.670 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-11T12:09:30.670 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-11T12:09:30.670 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-11T12:09:30.670 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-11T12:09:30.670 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-11T12:09:30.671 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-11T12:09:30.671 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-11T12:09:30.671 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-11T12:09:30.671 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-11T12:09:30.671 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-11T12:09:32.840 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-11T12:09:32.840 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:09:32.840 INFO:teuthology.orchestra.run.smithi167.stdout:Upgraded: 2024-05-11T12:09:32.840 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-11T12:09:32.840 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-11T12:09:32.840 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-11T12:09:32.841 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-11T12:09:32.841 INFO:teuthology.orchestra.run.smithi167.stdout: librados2-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:32.841 INFO:teuthology.orchestra.run.smithi167.stdout: librbd1-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:32.841 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-11T12:09:32.841 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-11T12:09:32.841 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2024-05-11T12:09:32.841 INFO:teuthology.orchestra.run.smithi167.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-11T12:09:32.841 INFO:teuthology.orchestra.run.smithi167.stdout: automake-1.16.2-8.el9.noarch 2024-05-11T12:09:32.841 INFO:teuthology.orchestra.run.smithi167.stdout: bison-3.7.4-5.el9.x86_64 2024-05-11T12:09:32.841 INFO:teuthology.orchestra.run.smithi167.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-11T12:09:32.841 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:32.841 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-base-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:32.841 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:32.842 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-fuse-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:32.842 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-grafana-dashboards-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:09:32.842 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-immutable-object-cache-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:32.842 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mds-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:32.842 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:32.842 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-cephadm-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:09:32.842 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-dashboard-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:09:32.842 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:09:32.842 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-modules-core-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:09:32.842 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-rook-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:09:32.842 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mon-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:32.842 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-osd-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:32.842 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-prometheus-alerts-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:09:32.843 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-radosgw-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:32.843 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-selinux-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:32.843 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-test-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:32.843 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-volume-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:09:32.843 INFO:teuthology.orchestra.run.smithi167.stdout: cephadm-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:09:32.843 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-mirror-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:32.843 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-shell-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:32.843 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-top-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:09:32.843 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-11T12:09:32.843 INFO:teuthology.orchestra.run.smithi167.stdout: flex-2.6.4-9.el9.x86_64 2024-05-11T12:09:32.843 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-11T12:09:32.843 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-11T12:09:32.843 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-11T12:09:32.843 INFO:teuthology.orchestra.run.smithi167.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-11T12:09:32.843 INFO:teuthology.orchestra.run.smithi167.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-11T12:09:32.844 INFO:teuthology.orchestra.run.smithi167.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-11T12:09:32.844 INFO:teuthology.orchestra.run.smithi167.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-11T12:09:32.844 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-11T12:09:32.844 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-11T12:09:32.844 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs-devel-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:32.844 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs2-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:32.844 INFO:teuthology.orchestra.run.smithi167.stdout: libcephsqlite-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:32.844 INFO:teuthology.orchestra.run.smithi167.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-11T12:09:32.844 INFO:teuthology.orchestra.run.smithi167.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-11T12:09:32.844 INFO:teuthology.orchestra.run.smithi167.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-11T12:09:32.844 INFO:teuthology.orchestra.run.smithi167.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-11T12:09:32.844 INFO:teuthology.orchestra.run.smithi167.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-11T12:09:32.844 INFO:teuthology.orchestra.run.smithi167.stdout: librados-devel-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:32.844 INFO:teuthology.orchestra.run.smithi167.stdout: libradosstriper1-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:32.845 INFO:teuthology.orchestra.run.smithi167.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-11T12:09:32.845 INFO:teuthology.orchestra.run.smithi167.stdout: librgw2-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:32.845 INFO:teuthology.orchestra.run.smithi167.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-11T12:09:32.845 INFO:teuthology.orchestra.run.smithi167.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-11T12:09:32.845 INFO:teuthology.orchestra.run.smithi167.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-11T12:09:32.845 INFO:teuthology.orchestra.run.smithi167.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-11T12:09:32.845 INFO:teuthology.orchestra.run.smithi167.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-11T12:09:32.845 INFO:teuthology.orchestra.run.smithi167.stdout: lua-5.4.4-4.el9.x86_64 2024-05-11T12:09:32.845 INFO:teuthology.orchestra.run.smithi167.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-11T12:09:32.845 INFO:teuthology.orchestra.run.smithi167.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-11T12:09:32.845 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-11T12:09:32.845 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-11T12:09:32.845 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-11T12:09:32.845 INFO:teuthology.orchestra.run.smithi167.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-11T12:09:32.845 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-13.14-1.el9.x86_64 2024-05-11T12:09:32.846 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-11T12:09:32.846 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-11T12:09:32.846 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-11T12:09:32.846 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-11T12:09:32.846 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-11T12:09:32.846 INFO:teuthology.orchestra.run.smithi167.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-11T12:09:32.846 INFO:teuthology.orchestra.run.smithi167.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-11T12:09:32.846 INFO:teuthology.orchestra.run.smithi167.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-11T12:09:32.846 INFO:teuthology.orchestra.run.smithi167.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-11T12:09:32.846 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-11T12:09:32.846 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-argparse-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:32.846 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:32.846 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cephfs-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:32.846 INFO:teuthology.orchestra.run.smithi167.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-11T12:09:32.846 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-11T12:09:32.847 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-11T12:09:32.847 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-11T12:09:32.847 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-11T12:09:32.847 INFO:teuthology.orchestra.run.smithi167.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-11T12:09:32.847 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-11T12:09:32.847 INFO:teuthology.orchestra.run.smithi167.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-11T12:09:32.847 INFO:teuthology.orchestra.run.smithi167.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-11T12:09:32.847 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-11T12:09:32.847 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-11T12:09:32.847 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-11T12:09:32.847 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-11T12:09:32.847 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-11T12:09:32.847 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-11T12:09:32.847 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-11T12:09:32.848 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-11T12:09:32.848 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-11T12:09:32.848 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-11T12:09:32.848 INFO:teuthology.orchestra.run.smithi167.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-11T12:09:32.848 INFO:teuthology.orchestra.run.smithi167.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-11T12:09:32.848 INFO:teuthology.orchestra.run.smithi167.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-11T12:09:32.848 INFO:teuthology.orchestra.run.smithi167.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-11T12:09:32.848 INFO:teuthology.orchestra.run.smithi167.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-11T12:09:32.848 INFO:teuthology.orchestra.run.smithi167.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-11T12:09:32.848 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-11T12:09:32.848 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-11T12:09:32.848 INFO:teuthology.orchestra.run.smithi167.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-11T12:09:32.848 INFO:teuthology.orchestra.run.smithi167.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-11T12:09:32.848 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-11T12:09:32.848 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ply-3.11-14.el9.noarch 2024-05-11T12:09:32.849 INFO:teuthology.orchestra.run.smithi167.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-11T12:09:32.849 INFO:teuthology.orchestra.run.smithi167.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-11T12:09:32.849 INFO:teuthology.orchestra.run.smithi167.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-11T12:09:32.849 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-11T12:09:32.849 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-11T12:09:32.849 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-11T12:09:32.849 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-11T12:09:32.849 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-11T12:09:32.849 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-11T12:09:32.849 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rados-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:32.849 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rbd-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:32.849 INFO:teuthology.orchestra.run.smithi167.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-11T12:09:32.849 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-11T12:09:32.849 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-11T12:09:32.849 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rgw-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:32.850 INFO:teuthology.orchestra.run.smithi167.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-11T12:09:32.850 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-11T12:09:32.850 INFO:teuthology.orchestra.run.smithi167.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-11T12:09:32.850 INFO:teuthology.orchestra.run.smithi167.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-11T12:09:32.850 INFO:teuthology.orchestra.run.smithi167.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-11T12:09:32.850 INFO:teuthology.orchestra.run.smithi167.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-11T12:09:32.850 INFO:teuthology.orchestra.run.smithi167.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-11T12:09:32.850 INFO:teuthology.orchestra.run.smithi167.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-11T12:09:32.850 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-11T12:09:32.850 INFO:teuthology.orchestra.run.smithi167.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-11T12:09:32.850 INFO:teuthology.orchestra.run.smithi167.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-11T12:09:32.850 INFO:teuthology.orchestra.run.smithi167.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-11T12:09:32.850 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-11T12:09:32.850 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-11T12:09:32.850 INFO:teuthology.orchestra.run.smithi167.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-11T12:09:32.851 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-fuse-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:32.851 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-mirror-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:32.851 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-nbd-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:09:32.851 INFO:teuthology.orchestra.run.smithi167.stdout: re2-1:20211101-3.el9.x86_64 2024-05-11T12:09:32.851 INFO:teuthology.orchestra.run.smithi167.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-11T12:09:32.851 INFO:teuthology.orchestra.run.smithi167.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-11T12:09:32.851 INFO:teuthology.orchestra.run.smithi167.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-11T12:09:32.851 INFO:teuthology.orchestra.run.smithi167.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-11T12:09:32.851 INFO:teuthology.orchestra.run.smithi167.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-11T12:09:32.851 INFO:teuthology.orchestra.run.smithi167.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-11T12:09:32.851 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:09:32.851 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2024-05-11T12:09:33.478 DEBUG:teuthology.parallel:result is None 2024-05-11T12:09:33.478 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=308506802b0bdea4df7050cfc1fcbdc04fea6a6e 2024-05-11T12:09:33.727 DEBUG:teuthology.orchestra.run.smithi167:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-11T12:09:33.788 INFO:teuthology.orchestra.run.smithi167.stdout:19.0.0-3601.g30850680.el9 2024-05-11T12:09:33.788 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3601.g30850680.el9 2024-05-11T12:09:33.789 INFO:teuthology.task.install:The correct ceph version 19.0.0-3601.g30850680 is installed. 2024-05-11T12:09:33.790 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=308506802b0bdea4df7050cfc1fcbdc04fea6a6e 2024-05-11T12:09:34.010 DEBUG:teuthology.orchestra.run.smithi204:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-11T12:09:34.083 INFO:teuthology.orchestra.run.smithi204.stdout:19.0.0-3601.g30850680.el9 2024-05-11T12:09:34.084 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3601.g30850680.el9 2024-05-11T12:09:34.084 INFO:teuthology.task.install:The correct ceph version 19.0.0-3601.g30850680 is installed. 2024-05-11T12:09:34.085 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-11T12:09:34.085 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:09:34.086 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-11T12:09:34.159 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:09:34.159 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-11T12:09:34.251 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-11T12:09:34.251 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:09:34.251 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/daemon-helper 2024-05-11T12:09:34.320 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-11T12:09:34.430 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:09:34.430 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/daemon-helper 2024-05-11T12:09:34.520 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-11T12:09:34.606 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-11T12:09:34.606 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:09:34.606 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-11T12:09:34.678 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-11T12:09:34.789 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:09:34.789 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-11T12:09:34.870 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-11T12:09:34.998 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-11T12:09:34.998 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:09:34.998 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/stdin-killer 2024-05-11T12:09:35.069 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-11T12:09:35.175 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:09:35.176 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/stdin-killer 2024-05-11T12:09:35.265 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-11T12:09:35.350 INFO:teuthology.run_tasks:Running task ceph... 2024-05-11T12:09:35.430 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-05-11T12:09:35.430 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 777 /var/log/ceph 2024-05-11T12:09:35.435 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 777 /var/log/ceph 2024-05-11T12:09:35.523 INFO:tasks.ceph:Disabling ceph logrotate... 2024-05-11T12:09:35.523 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-11T12:09:35.528 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-11T12:09:35.611 INFO:tasks.ceph:Creating extra log directories... 2024-05-11T12:09:35.611 DEBUG:teuthology.orchestra.run.smithi167:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-11T12:09:35.615 DEBUG:teuthology.orchestra.run.smithi204:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-11T12:09:35.704 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-05-11T12:09:35.704 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-05-11T12:09:35.704 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-05-11T12:09:35.705 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:09:35.705 DEBUG:teuthology.orchestra.run.smithi167:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-11T12:09:35.746 DEBUG:teuthology.orchestra.run.smithi167:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-11T12:09:35.941 DEBUG:teuthology.orchestra.run.smithi167:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-11T12:09:36.007 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:09:36.007 DEBUG:teuthology.orchestra.run.smithi204:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-11T12:09:36.065 DEBUG:teuthology.orchestra.run.smithi204:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-11T12:09:36.278 DEBUG:teuthology.orchestra.run.smithi204:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-11T12:09:36.369 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-05-11T12:09:36.369 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'client session with non-allowable root'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-05-11T12:09:36.369 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702355', 'branch': 'wip-rishabh-testing-20240510.053205', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/sessionmap}', 'email': None, 'first_in_suite': False, 'job_id': '7702355', 'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': 'fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-testing-20240510.053205'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'client session with non-allowable root'], 'sha1': '308506802b0bdea4df7050cfc1fcbdc04fea6a6e'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '308506802b0bdea4df7050cfc1fcbdc04fea6a6e'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-testing-20240510.053205', 'sha1': '308506802b0bdea4df7050cfc1fcbdc04fea6a6e'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 75, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 4085, 'sha1': '308506802b0bdea4df7050cfc1fcbdc04fea6a6e', 'sleep_before_teardown': 0, 'subset': '100/512', 'suite': 'fs', 'suite_branch': 'wip-rishabh-testing-20240510.053205', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_308506802b0bdea4df7050cfc1fcbdc04fea6a6e/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '308506802b0bdea4df7050cfc1fcbdc04fea6a6e', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi167.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLcCJrL5x4XtKZnV4OkwoYGHTMYytpMjb27CDYxENeJ7Wuf1CQjNeczJsGnxPKceq1BaYiNVM+W+OPWUVeWKPvk=', 'smithi204.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF+c6w6v8BIiyverlkjQMjsNd8TOMDnjSzdFfa1mdxBkGmGMndfO6h/EX48QI5zQxOaxZ1sa08je1A/k05pDU5E='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': 'fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '308506802b0bdea4df7050cfc1fcbdc04fea6a6e'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'client session with non-allowable root'], 'sha1': '308506802b0bdea4df7050cfc1fcbdc04fea6a6e', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_sessionmap']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '1ae7ad82388e92a475afff437d49054826c019a1', 'timestamp': '2024-05-11_05:57:22', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-05-11T12:09:36.370 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-11T12:09:36.376 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-11T12:09:36.429 DEBUG:teuthology.orchestra.run.smithi167:> sudo install -d -m0777 -- /var/run/ceph 2024-05-11T12:09:36.434 DEBUG:teuthology.orchestra.run.smithi204:> sudo install -d -m0777 -- /var/run/ceph 2024-05-11T12:09:36.517 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:09:36.518 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/scratch_devs of=/dev/stdout 2024-05-11T12:09:36.559 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-11T12:09:36.559 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_1 2024-05-11T12:09:36.642 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-11T12:09:36.642 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T12:09:36.642 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 6h/6d Inode: 633 Links: 1 2024-05-11T12:09:36.642 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T12:09:36.642 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T12:09:36.642 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-05-11 12:09:25.126410852 +0000 2024-05-11T12:09:36.642 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-05-11 12:06:48.795786443 +0000 2024-05-11T12:09:36.643 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-05-11 12:06:48.795786443 +0000 2024-05-11T12:09:36.643 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-05-11 12:06:48.795786443 +0000 2024-05-11T12:09:36.643 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-11T12:09:36.745 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-05-11T12:09:36.745 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-05-11T12:09:36.745 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000423661 s, 1.2 MB/s 2024-05-11T12:09:36.748 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-11T12:09:36.793 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_2 2024-05-11T12:09:36.876 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-11T12:09:36.876 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T12:09:36.876 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-05-11T12:09:36.876 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T12:09:36.876 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T12:09:36.876 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-05-11 12:09:25.127410835 +0000 2024-05-11T12:09:36.877 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-05-11 12:06:49.417775495 +0000 2024-05-11T12:09:36.877 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-05-11 12:06:49.417775495 +0000 2024-05-11T12:09:36.877 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-05-11 12:06:49.417775495 +0000 2024-05-11T12:09:36.877 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-11T12:09:36.981 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-05-11T12:09:36.982 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-05-11T12:09:36.982 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000279147 s, 1.8 MB/s 2024-05-11T12:09:36.984 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-11T12:09:37.027 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_3 2024-05-11T12:09:37.107 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-11T12:09:37.107 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T12:09:37.107 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-05-11T12:09:37.107 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T12:09:37.107 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T12:09:37.107 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-05-11 12:09:25.127410835 +0000 2024-05-11T12:09:37.107 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-05-11 12:06:49.994765339 +0000 2024-05-11T12:09:37.107 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-05-11 12:06:49.994765339 +0000 2024-05-11T12:09:37.107 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-05-11 12:06:49.994765339 +0000 2024-05-11T12:09:37.107 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-11T12:09:37.207 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-05-11T12:09:37.207 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-05-11T12:09:37.207 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000380456 s, 1.3 MB/s 2024-05-11T12:09:37.210 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-11T12:09:37.250 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_4 2024-05-11T12:09:37.331 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-11T12:09:37.331 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T12:09:37.331 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 6h/6d Inode: 689 Links: 1 2024-05-11T12:09:37.331 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T12:09:37.331 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T12:09:37.331 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-05-11 12:09:25.128410818 +0000 2024-05-11T12:09:37.331 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-05-11 12:06:50.576755096 +0000 2024-05-11T12:09:37.332 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-05-11 12:06:50.576755096 +0000 2024-05-11T12:09:37.332 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-05-11 12:06:50.576755096 +0000 2024-05-11T12:09:37.332 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-11T12:09:37.432 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-05-11T12:09:37.432 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-05-11T12:09:37.432 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000305459 s, 1.7 MB/s 2024-05-11T12:09:37.435 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-11T12:09:37.475 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-11T12:09:37.476 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:09:37.476 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/scratch_devs of=/dev/stdout 2024-05-11T12:09:37.533 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-11T12:09:37.533 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_1 2024-05-11T12:09:37.585 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-11T12:09:37.585 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T12:09:37.585 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 6h/6d Inode: 633 Links: 1 2024-05-11T12:09:37.585 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T12:09:37.585 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T12:09:37.585 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-05-11 12:09:07.026789089 +0000 2024-05-11T12:09:37.585 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-05-11 12:06:46.482247506 +0000 2024-05-11T12:09:37.586 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-05-11 12:06:46.482247506 +0000 2024-05-11T12:09:37.586 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-05-11 12:06:46.482247506 +0000 2024-05-11T12:09:37.586 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-11T12:09:37.666 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-05-11T12:09:37.666 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-05-11T12:09:37.666 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000367975 s, 1.4 MB/s 2024-05-11T12:09:37.670 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-11T12:09:37.726 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_2 2024-05-11T12:09:37.775 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-11T12:09:37.775 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T12:09:37.775 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-05-11T12:09:37.775 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T12:09:37.776 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T12:09:37.776 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-05-11 12:09:07.027789076 +0000 2024-05-11T12:09:37.776 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-05-11 12:06:47.076239898 +0000 2024-05-11T12:09:37.776 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-05-11 12:06:47.076239898 +0000 2024-05-11T12:09:37.776 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-05-11 12:06:47.076239898 +0000 2024-05-11T12:09:37.776 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-11T12:09:37.852 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-05-11T12:09:37.852 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-05-11T12:09:37.852 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.00034905 s, 1.5 MB/s 2024-05-11T12:09:37.857 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-11T12:09:37.914 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_3 2024-05-11T12:09:37.966 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-11T12:09:37.966 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T12:09:37.966 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 6h/6d Inode: 671 Links: 1 2024-05-11T12:09:37.966 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T12:09:37.966 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T12:09:37.966 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-05-11 12:09:07.027789076 +0000 2024-05-11T12:09:37.966 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-05-11 12:06:47.635232738 +0000 2024-05-11T12:09:37.967 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-05-11 12:06:47.635232738 +0000 2024-05-11T12:09:37.967 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-05-11 12:06:47.635232738 +0000 2024-05-11T12:09:37.967 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-11T12:09:38.044 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-05-11T12:09:38.044 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-05-11T12:09:38.044 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000342683 s, 1.5 MB/s 2024-05-11T12:09:38.048 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-11T12:09:38.104 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_4 2024-05-11T12:09:38.153 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-11T12:09:38.153 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T12:09:38.153 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-05-11T12:09:38.153 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T12:09:38.153 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T12:09:38.153 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-05-11 12:09:07.028789063 +0000 2024-05-11T12:09:38.153 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-05-11 12:06:48.243224951 +0000 2024-05-11T12:09:38.153 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-05-11 12:06:48.243224951 +0000 2024-05-11T12:09:38.154 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-05-11 12:06:48.243224951 +0000 2024-05-11T12:09:38.154 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-11T12:09:38.231 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-05-11T12:09:38.231 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-05-11T12:09:38.231 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000346993 s, 1.5 MB/s 2024-05-11T12:09:38.235 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-11T12:09:38.291 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-11T12:09:38.291 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi167.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi204.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-11T12:09:38.291 INFO:tasks.ceph:Generating config... 2024-05-11T12:09:38.292 INFO:tasks.ceph:[client] client mount timeout = 600 2024-05-11T12:09:38.292 INFO:tasks.ceph:[client] debug client = 20 2024-05-11T12:09:38.292 INFO:tasks.ceph:[client] debug ms = 1 2024-05-11T12:09:38.292 INFO:tasks.ceph:[client] pid file = 2024-05-11T12:09:38.292 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-05-11T12:09:38.292 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-05-11T12:09:38.293 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-05-11T12:09:38.293 INFO:tasks.ceph:[mds] debug mds = 20 2024-05-11T12:09:38.293 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-05-11T12:09:38.293 INFO:tasks.ceph:[mds] debug ms = 1 2024-05-11T12:09:38.293 INFO:tasks.ceph:[mds] mds debug frag = True 2024-05-11T12:09:38.293 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-05-11T12:09:38.293 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-05-11T12:09:38.293 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-05-11T12:09:38.293 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-05-11T12:09:38.293 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-05-11T12:09:38.293 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-05-11T12:09:38.293 INFO:tasks.ceph:[mgr] debug client = 20 2024-05-11T12:09:38.293 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-05-11T12:09:38.293 INFO:tasks.ceph:[mgr] debug ms = 1 2024-05-11T12:09:38.293 INFO:tasks.ceph:[mon] debug mon = 20 2024-05-11T12:09:38.293 INFO:tasks.ceph:[mon] debug ms = 1 2024-05-11T12:09:38.294 INFO:tasks.ceph:[mon] debug paxos = 20 2024-05-11T12:09:38.294 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-05-11T12:09:38.294 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-05-11T12:09:38.294 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-05-11T12:09:38.294 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-05-11T12:09:38.294 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-05-11T12:09:38.294 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-05-11T12:09:38.294 INFO:tasks.ceph:[osd] debug ms = 1 2024-05-11T12:09:38.294 INFO:tasks.ceph:[osd] debug osd = 20 2024-05-11T12:09:38.294 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-05-11T12:09:38.294 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-05-11T12:09:38.294 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-05-11T12:09:38.295 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-05-11T12:09:38.295 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-05-11T12:09:38.295 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-05-11T12:09:38.295 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-05-11T12:09:38.295 INFO:tasks.ceph:Setting up mon.a... 2024-05-11T12:09:38.295 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-05-11T12:09:38.440 INFO:teuthology.orchestra.run.smithi167.stdout:creating /etc/ceph/ceph.keyring 2024-05-11T12:09:38.450 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-05-11T12:09:38.569 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-11T12:09:38.638 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.167'), ('mon.b', '172.21.15.204'), ('mon.c', '[v2:172.21.15.204:3301,v1:172.21.15.204:6790]')] 2024-05-11T12:09:38.638 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.167,172.21.15.204,[v2:172.21.15.204:3301,v1:172.21.15.204:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-05-11T12:09:38.639 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:09:38.639 DEBUG:teuthology.orchestra.run.smithi167:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-05-11T12:09:38.692 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.167 --add b 172.21.15.204 --addv c '[v2:172.21.15.204:3301,v1:172.21.15.204:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-05-11T12:09:38.844 INFO:teuthology.orchestra.run.smithi167.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-05-11T12:09:38.845 INFO:teuthology.orchestra.run.smithi167.stdout:monmaptool: generated fsid 5f32d344-7289-4c27-881b-1533419ab678 2024-05-11T12:09:38.845 INFO:teuthology.orchestra.run.smithi167.stdout:setting min_mon_release = quincy 2024-05-11T12:09:38.845 INFO:teuthology.orchestra.run.smithi167.stdout:epoch 0 2024-05-11T12:09:38.845 INFO:teuthology.orchestra.run.smithi167.stdout:fsid 5f32d344-7289-4c27-881b-1533419ab678 2024-05-11T12:09:38.845 INFO:teuthology.orchestra.run.smithi167.stdout:last_changed 2024-05-11T12:09:38.845893+0000 2024-05-11T12:09:38.845 INFO:teuthology.orchestra.run.smithi167.stdout:created 2024-05-11T12:09:38.845893+0000 2024-05-11T12:09:38.845 INFO:teuthology.orchestra.run.smithi167.stdout:min_mon_release 17 (quincy) 2024-05-11T12:09:38.845 INFO:teuthology.orchestra.run.smithi167.stdout:election_strategy: 1 2024-05-11T12:09:38.845 INFO:teuthology.orchestra.run.smithi167.stdout:0: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.a 2024-05-11T12:09:38.845 INFO:teuthology.orchestra.run.smithi167.stdout:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-11T12:09:38.845 INFO:teuthology.orchestra.run.smithi167.stdout:2: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.c 2024-05-11T12:09:38.845 INFO:teuthology.orchestra.run.smithi167.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-05-11T12:09:38.848 DEBUG:teuthology.orchestra.run.smithi167:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-05-11T12:09:38.899 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 5f32d344-7289-4c27-881b-1533419ab678... 2024-05-11T12:09:38.900 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-11T12:09:38.905 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-11T12:09:39.036 INFO:teuthology.orchestra.run.smithi167.stdout:[global] 2024-05-11T12:09:39.037 INFO:teuthology.orchestra.run.smithi167.stdout: chdir = "" 2024-05-11T12:09:39.037 INFO:teuthology.orchestra.run.smithi167.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-11T12:09:39.037 INFO:teuthology.orchestra.run.smithi167.stdout: auth supported = cephx 2024-05-11T12:09:39.037 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:09:39.037 INFO:teuthology.orchestra.run.smithi167.stdout: filestore xattr use omap = true 2024-05-11T12:09:39.037 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:09:39.037 INFO:teuthology.orchestra.run.smithi167.stdout: mon clock drift allowed = 1.000 2024-05-11T12:09:39.037 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:09:39.037 INFO:teuthology.orchestra.run.smithi167.stdout: osd crush chooseleaf type = 0 2024-05-11T12:09:39.037 INFO:teuthology.orchestra.run.smithi167.stdout: auth debug = true 2024-05-11T12:09:39.037 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:09:39.037 INFO:teuthology.orchestra.run.smithi167.stdout: ms die on old message = true 2024-05-11T12:09:39.037 INFO:teuthology.orchestra.run.smithi167.stdout: ms die on bug = true 2024-05-11T12:09:39.037 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:09:39.037 INFO:teuthology.orchestra.run.smithi167.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-11T12:09:39.038 INFO:teuthology.orchestra.run.smithi167.stdout: mon pg warn max object skew = 0 2024-05-11T12:09:39.038 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:09:39.038 INFO:teuthology.orchestra.run.smithi167.stdout: # disable pg_autoscaler by default for new pools 2024-05-11T12:09:39.038 INFO:teuthology.orchestra.run.smithi167.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-11T12:09:39.038 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:09:39.038 INFO:teuthology.orchestra.run.smithi167.stdout: osd pool default size = 2 2024-05-11T12:09:39.038 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:09:39.038 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd allow primary affinity = true 2024-05-11T12:09:39.038 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd allow pg remap = true 2024-05-11T12:09:39.038 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on legacy crush tunables = false 2024-05-11T12:09:39.038 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on crush straw calc version zero = false 2024-05-11T12:09:39.038 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on no sortbitwise = false 2024-05-11T12:09:39.038 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on osd down out interval zero = false 2024-05-11T12:09:39.038 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on too few osds = false 2024-05-11T12:09:39.039 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-11T12:09:39.039 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-11T12:09:39.039 INFO:teuthology.orchestra.run.smithi167.stdout: mon_allow_pool_size_one = true 2024-05-11T12:09:39.039 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:09:39.039 INFO:teuthology.orchestra.run.smithi167.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-11T12:09:39.039 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:09:39.039 INFO:teuthology.orchestra.run.smithi167.stdout: osd default data pool replay window = 5 2024-05-11T12:09:39.039 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:09:39.039 INFO:teuthology.orchestra.run.smithi167.stdout: mon allow pool delete = true 2024-05-11T12:09:39.039 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:09:39.039 INFO:teuthology.orchestra.run.smithi167.stdout: mon cluster log file level = debug 2024-05-11T12:09:39.039 INFO:teuthology.orchestra.run.smithi167.stdout: debug asserts on shutdown = true 2024-05-11T12:09:39.039 INFO:teuthology.orchestra.run.smithi167.stdout: mon health detail to clog = false 2024-05-11T12:09:39.039 INFO:teuthology.orchestra.run.smithi167.stdout: mon host = "172.21.15.167,172.21.15.204,[v2:172.21.15.204:3301,v1:172.21.15.204:6790]" 2024-05-11T12:09:39.039 INFO:teuthology.orchestra.run.smithi167.stdout: ms die on skipped message = False 2024-05-11T12:09:39.040 INFO:teuthology.orchestra.run.smithi167.stdout: fsid = 5f32d344-7289-4c27-881b-1533419ab678 2024-05-11T12:09:39.040 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:09:39.040 INFO:teuthology.orchestra.run.smithi167.stdout:[osd] 2024-05-11T12:09:39.040 INFO:teuthology.orchestra.run.smithi167.stdout: osd journal size = 100 2024-05-11T12:09:39.040 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:09:39.040 INFO:teuthology.orchestra.run.smithi167.stdout: osd scrub load threshold = 5.0 2024-05-11T12:09:39.040 INFO:teuthology.orchestra.run.smithi167.stdout: osd scrub max interval = 600 2024-05-11T12:09:39.040 INFO:teuthology.orchestra.run.smithi167.stdout: osd mclock profile = high_recovery_ops 2024-05-11T12:09:39.040 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:09:39.040 INFO:teuthology.orchestra.run.smithi167.stdout: osd recover clone overlap = true 2024-05-11T12:09:39.040 INFO:teuthology.orchestra.run.smithi167.stdout: osd recovery max chunk = 1048576 2024-05-11T12:09:39.040 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:09:39.040 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug shutdown = true 2024-05-11T12:09:39.040 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug op order = true 2024-05-11T12:09:39.040 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug verify stray on activate = true 2024-05-11T12:09:39.040 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug trim objects = true 2024-05-11T12:09:39.041 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:09:39.041 INFO:teuthology.orchestra.run.smithi167.stdout: osd open classes on start = true 2024-05-11T12:09:39.041 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug pg log writeout = true 2024-05-11T12:09:39.041 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:09:39.041 INFO:teuthology.orchestra.run.smithi167.stdout: osd deep scrub update digest min age = 30 2024-05-11T12:09:39.041 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:09:39.041 INFO:teuthology.orchestra.run.smithi167.stdout: osd map max advance = 10 2024-05-11T12:09:39.041 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:09:39.041 INFO:teuthology.orchestra.run.smithi167.stdout: journal zero on create = true 2024-05-11T12:09:39.041 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:09:39.041 INFO:teuthology.orchestra.run.smithi167.stdout: filestore ondisk finisher threads = 3 2024-05-11T12:09:39.041 INFO:teuthology.orchestra.run.smithi167.stdout: filestore apply finisher threads = 3 2024-05-11T12:09:39.041 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:09:39.041 INFO:teuthology.orchestra.run.smithi167.stdout: bdev debug aio = true 2024-05-11T12:09:39.041 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug misdirected ops = true 2024-05-11T12:09:39.042 INFO:teuthology.orchestra.run.smithi167.stdout: bluestore block size = 96636764160 2024-05-11T12:09:39.042 INFO:teuthology.orchestra.run.smithi167.stdout: bluestore fsck on mount = True 2024-05-11T12:09:39.042 INFO:teuthology.orchestra.run.smithi167.stdout: debug bluefs = 20 2024-05-11T12:09:39.042 INFO:teuthology.orchestra.run.smithi167.stdout: debug bluestore = 20 2024-05-11T12:09:39.042 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2024-05-11T12:09:39.042 INFO:teuthology.orchestra.run.smithi167.stdout: debug osd = 20 2024-05-11T12:09:39.042 INFO:teuthology.orchestra.run.smithi167.stdout: debug rocksdb = 10 2024-05-11T12:09:39.042 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd backfillfull_ratio = 0.85 2024-05-11T12:09:39.042 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd full ratio = 0.9 2024-05-11T12:09:39.042 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd nearfull ratio = 0.8 2024-05-11T12:09:39.042 INFO:teuthology.orchestra.run.smithi167.stdout: osd failsafe full ratio = 0.95 2024-05-11T12:09:39.042 INFO:teuthology.orchestra.run.smithi167.stdout: osd objectstore = bluestore 2024-05-11T12:09:39.042 INFO:teuthology.orchestra.run.smithi167.stdout: osd op complaint time = 180 2024-05-11T12:09:39.042 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:09:39.042 INFO:teuthology.orchestra.run.smithi167.stdout:[mgr] 2024-05-11T12:09:39.043 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2024-05-11T12:09:39.043 INFO:teuthology.orchestra.run.smithi167.stdout: debug mgr = 20 2024-05-11T12:09:39.043 INFO:teuthology.orchestra.run.smithi167.stdout: debug mon = 20 2024-05-11T12:09:39.043 INFO:teuthology.orchestra.run.smithi167.stdout: debug auth = 20 2024-05-11T12:09:39.043 INFO:teuthology.orchestra.run.smithi167.stdout: mon reweight min pgs per osd = 4 2024-05-11T12:09:39.043 INFO:teuthology.orchestra.run.smithi167.stdout: mon reweight min bytes per osd = 10 2024-05-11T12:09:39.043 INFO:teuthology.orchestra.run.smithi167.stdout: mgr/telemetry/nag = false 2024-05-11T12:09:39.043 INFO:teuthology.orchestra.run.smithi167.stdout: debug client = 20 2024-05-11T12:09:39.043 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:09:39.043 INFO:teuthology.orchestra.run.smithi167.stdout:[mon] 2024-05-11T12:09:39.043 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2024-05-11T12:09:39.043 INFO:teuthology.orchestra.run.smithi167.stdout: debug mon = 20 2024-05-11T12:09:39.043 INFO:teuthology.orchestra.run.smithi167.stdout: debug paxos = 20 2024-05-11T12:09:39.043 INFO:teuthology.orchestra.run.smithi167.stdout: debug auth = 20 2024-05-11T12:09:39.043 INFO:teuthology.orchestra.run.smithi167.stdout: mon data avail warn = 5 2024-05-11T12:09:39.044 INFO:teuthology.orchestra.run.smithi167.stdout: mon mgr mkfs grace = 240 2024-05-11T12:09:39.044 INFO:teuthology.orchestra.run.smithi167.stdout: mon reweight min pgs per osd = 4 2024-05-11T12:09:39.044 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd reporter subtree level = osd 2024-05-11T12:09:39.044 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd prime pg temp = true 2024-05-11T12:09:39.044 INFO:teuthology.orchestra.run.smithi167.stdout: mon reweight min bytes per osd = 10 2024-05-11T12:09:39.044 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:09:39.044 INFO:teuthology.orchestra.run.smithi167.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-11T12:09:39.044 INFO:teuthology.orchestra.run.smithi167.stdout: auth mon ticket ttl = 660 # 11m 2024-05-11T12:09:39.044 INFO:teuthology.orchestra.run.smithi167.stdout: auth service ticket ttl = 240 # 4m 2024-05-11T12:09:39.044 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:09:39.044 INFO:teuthology.orchestra.run.smithi167.stdout: # don't complain about insecure global_id in the test suite 2024-05-11T12:09:39.044 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-11T12:09:39.044 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-11T12:09:39.044 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:09:39.044 INFO:teuthology.orchestra.run.smithi167.stdout: # 1m isn't quite enough 2024-05-11T12:09:39.045 INFO:teuthology.orchestra.run.smithi167.stdout: mon_down_mkfs_grace = 2m 2024-05-11T12:09:39.045 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:09:39.045 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_filestore_osds = false 2024-05-11T12:09:39.045 INFO:teuthology.orchestra.run.smithi167.stdout: mon down mkfs grace = 300 2024-05-11T12:09:39.045 INFO:teuthology.orchestra.run.smithi167.stdout: mon op complaint time = 120 2024-05-11T12:09:39.045 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:09:39.045 INFO:teuthology.orchestra.run.smithi167.stdout:[client] 2024-05-11T12:09:39.045 INFO:teuthology.orchestra.run.smithi167.stdout: rgw cache enabled = true 2024-05-11T12:09:39.045 INFO:teuthology.orchestra.run.smithi167.stdout: rgw enable ops log = true 2024-05-11T12:09:39.045 INFO:teuthology.orchestra.run.smithi167.stdout: rgw enable usage log = true 2024-05-11T12:09:39.045 INFO:teuthology.orchestra.run.smithi167.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-11T12:09:39.045 INFO:teuthology.orchestra.run.smithi167.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-11T12:09:39.045 INFO:teuthology.orchestra.run.smithi167.stdout: client mount timeout = 600 2024-05-11T12:09:39.045 INFO:teuthology.orchestra.run.smithi167.stdout: debug client = 20 2024-05-11T12:09:39.045 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2024-05-11T12:09:39.046 INFO:teuthology.orchestra.run.smithi167.stdout: pid file = "" 2024-05-11T12:09:39.046 INFO:teuthology.orchestra.run.smithi167.stdout: rados mon op timeout = 900 2024-05-11T12:09:39.046 INFO:teuthology.orchestra.run.smithi167.stdout: rados osd op timeout = 900 2024-05-11T12:09:39.046 INFO:teuthology.orchestra.run.smithi167.stdout:[mon.a] 2024-05-11T12:09:39.046 INFO:teuthology.orchestra.run.smithi167.stdout:[mon.b] 2024-05-11T12:09:39.046 INFO:teuthology.orchestra.run.smithi167.stdout:[mon.c] 2024-05-11T12:09:39.046 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.a] 2024-05-11T12:09:39.046 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.c] 2024-05-11T12:09:39.046 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.b] 2024-05-11T12:09:39.046 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.d] 2024-05-11T12:09:39.046 INFO:teuthology.orchestra.run.smithi167.stdout:[mds] 2024-05-11T12:09:39.046 INFO:teuthology.orchestra.run.smithi167.stdout: debug mds = 20 2024-05-11T12:09:39.046 INFO:teuthology.orchestra.run.smithi167.stdout: debug mds balancer = 20 2024-05-11T12:09:39.046 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2024-05-11T12:09:39.047 INFO:teuthology.orchestra.run.smithi167.stdout: mds debug frag = True 2024-05-11T12:09:39.047 INFO:teuthology.orchestra.run.smithi167.stdout: mds debug scatterstat = True 2024-05-11T12:09:39.047 INFO:teuthology.orchestra.run.smithi167.stdout: mds op complaint time = 180 2024-05-11T12:09:39.047 INFO:teuthology.orchestra.run.smithi167.stdout: mds verify scatter = True 2024-05-11T12:09:39.047 INFO:teuthology.orchestra.run.smithi167.stdout: osd op complaint time = 180 2024-05-11T12:09:39.047 INFO:teuthology.orchestra.run.smithi167.stdout: rados mon op timeout = 900 2024-05-11T12:09:39.047 INFO:teuthology.orchestra.run.smithi167.stdout: rados osd op timeout = 900 2024-05-11T12:09:39.048 INFO:teuthology.orchestra.run.smithi204.stdout:[global] 2024-05-11T12:09:39.048 INFO:teuthology.orchestra.run.smithi204.stdout: chdir = "" 2024-05-11T12:09:39.048 INFO:teuthology.orchestra.run.smithi204.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-11T12:09:39.048 INFO:teuthology.orchestra.run.smithi204.stdout: auth supported = cephx 2024-05-11T12:09:39.048 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:09:39.048 INFO:teuthology.orchestra.run.smithi204.stdout: filestore xattr use omap = true 2024-05-11T12:09:39.048 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:09:39.048 INFO:teuthology.orchestra.run.smithi204.stdout: mon clock drift allowed = 1.000 2024-05-11T12:09:39.049 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:09:39.049 INFO:teuthology.orchestra.run.smithi204.stdout: osd crush chooseleaf type = 0 2024-05-11T12:09:39.049 INFO:teuthology.orchestra.run.smithi204.stdout: auth debug = true 2024-05-11T12:09:39.049 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:09:39.049 INFO:teuthology.orchestra.run.smithi204.stdout: ms die on old message = true 2024-05-11T12:09:39.049 INFO:teuthology.orchestra.run.smithi204.stdout: ms die on bug = true 2024-05-11T12:09:39.049 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:09:39.049 INFO:teuthology.orchestra.run.smithi204.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-11T12:09:39.049 INFO:teuthology.orchestra.run.smithi204.stdout: mon pg warn max object skew = 0 2024-05-11T12:09:39.049 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:09:39.049 INFO:teuthology.orchestra.run.smithi204.stdout: # disable pg_autoscaler by default for new pools 2024-05-11T12:09:39.049 INFO:teuthology.orchestra.run.smithi204.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-11T12:09:39.049 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:09:39.049 INFO:teuthology.orchestra.run.smithi204.stdout: osd pool default size = 2 2024-05-11T12:09:39.049 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:09:39.050 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd allow primary affinity = true 2024-05-11T12:09:39.050 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd allow pg remap = true 2024-05-11T12:09:39.050 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on legacy crush tunables = false 2024-05-11T12:09:39.050 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on crush straw calc version zero = false 2024-05-11T12:09:39.050 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on no sortbitwise = false 2024-05-11T12:09:39.050 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on osd down out interval zero = false 2024-05-11T12:09:39.050 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on too few osds = false 2024-05-11T12:09:39.050 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-11T12:09:39.050 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-11T12:09:39.050 INFO:teuthology.orchestra.run.smithi204.stdout: mon_allow_pool_size_one = true 2024-05-11T12:09:39.050 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:09:39.050 INFO:teuthology.orchestra.run.smithi204.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-11T12:09:39.050 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:09:39.050 INFO:teuthology.orchestra.run.smithi204.stdout: osd default data pool replay window = 5 2024-05-11T12:09:39.050 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:09:39.051 INFO:teuthology.orchestra.run.smithi204.stdout: mon allow pool delete = true 2024-05-11T12:09:39.051 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:09:39.051 INFO:teuthology.orchestra.run.smithi204.stdout: mon cluster log file level = debug 2024-05-11T12:09:39.051 INFO:teuthology.orchestra.run.smithi204.stdout: debug asserts on shutdown = true 2024-05-11T12:09:39.051 INFO:teuthology.orchestra.run.smithi204.stdout: mon health detail to clog = false 2024-05-11T12:09:39.051 INFO:teuthology.orchestra.run.smithi204.stdout: mon host = "172.21.15.167,172.21.15.204,[v2:172.21.15.204:3301,v1:172.21.15.204:6790]" 2024-05-11T12:09:39.051 INFO:teuthology.orchestra.run.smithi204.stdout: ms die on skipped message = False 2024-05-11T12:09:39.051 INFO:teuthology.orchestra.run.smithi204.stdout: fsid = 5f32d344-7289-4c27-881b-1533419ab678 2024-05-11T12:09:39.051 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:09:39.051 INFO:teuthology.orchestra.run.smithi204.stdout:[osd] 2024-05-11T12:09:39.051 INFO:teuthology.orchestra.run.smithi204.stdout: osd journal size = 100 2024-05-11T12:09:39.051 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:09:39.051 INFO:teuthology.orchestra.run.smithi204.stdout: osd scrub load threshold = 5.0 2024-05-11T12:09:39.051 INFO:teuthology.orchestra.run.smithi204.stdout: osd scrub max interval = 600 2024-05-11T12:09:39.051 INFO:teuthology.orchestra.run.smithi204.stdout: osd mclock profile = high_recovery_ops 2024-05-11T12:09:39.052 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:09:39.052 INFO:teuthology.orchestra.run.smithi204.stdout: osd recover clone overlap = true 2024-05-11T12:09:39.052 INFO:teuthology.orchestra.run.smithi204.stdout: osd recovery max chunk = 1048576 2024-05-11T12:09:39.052 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:09:39.052 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug shutdown = true 2024-05-11T12:09:39.052 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug op order = true 2024-05-11T12:09:39.052 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug verify stray on activate = true 2024-05-11T12:09:39.052 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug trim objects = true 2024-05-11T12:09:39.052 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:09:39.052 INFO:teuthology.orchestra.run.smithi204.stdout: osd open classes on start = true 2024-05-11T12:09:39.052 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug pg log writeout = true 2024-05-11T12:09:39.052 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:09:39.052 INFO:teuthology.orchestra.run.smithi204.stdout: osd deep scrub update digest min age = 30 2024-05-11T12:09:39.052 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:09:39.053 INFO:teuthology.orchestra.run.smithi204.stdout: osd map max advance = 10 2024-05-11T12:09:39.053 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:09:39.053 INFO:teuthology.orchestra.run.smithi204.stdout: journal zero on create = true 2024-05-11T12:09:39.053 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:09:39.053 INFO:teuthology.orchestra.run.smithi204.stdout: filestore ondisk finisher threads = 3 2024-05-11T12:09:39.053 INFO:teuthology.orchestra.run.smithi204.stdout: filestore apply finisher threads = 3 2024-05-11T12:09:39.053 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:09:39.053 INFO:teuthology.orchestra.run.smithi204.stdout: bdev debug aio = true 2024-05-11T12:09:39.053 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug misdirected ops = true 2024-05-11T12:09:39.053 INFO:teuthology.orchestra.run.smithi204.stdout: bluestore block size = 96636764160 2024-05-11T12:09:39.053 INFO:teuthology.orchestra.run.smithi204.stdout: bluestore fsck on mount = True 2024-05-11T12:09:39.053 INFO:teuthology.orchestra.run.smithi204.stdout: debug bluefs = 20 2024-05-11T12:09:39.053 INFO:teuthology.orchestra.run.smithi204.stdout: debug bluestore = 20 2024-05-11T12:09:39.053 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-05-11T12:09:39.053 INFO:teuthology.orchestra.run.smithi204.stdout: debug osd = 20 2024-05-11T12:09:39.054 INFO:teuthology.orchestra.run.smithi204.stdout: debug rocksdb = 10 2024-05-11T12:09:39.054 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd backfillfull_ratio = 0.85 2024-05-11T12:09:39.054 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd full ratio = 0.9 2024-05-11T12:09:39.054 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd nearfull ratio = 0.8 2024-05-11T12:09:39.054 INFO:teuthology.orchestra.run.smithi204.stdout: osd failsafe full ratio = 0.95 2024-05-11T12:09:39.054 INFO:teuthology.orchestra.run.smithi204.stdout: osd objectstore = bluestore 2024-05-11T12:09:39.054 INFO:teuthology.orchestra.run.smithi204.stdout: osd op complaint time = 180 2024-05-11T12:09:39.054 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:09:39.054 INFO:teuthology.orchestra.run.smithi204.stdout:[mgr] 2024-05-11T12:09:39.054 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-05-11T12:09:39.054 INFO:teuthology.orchestra.run.smithi204.stdout: debug mgr = 20 2024-05-11T12:09:39.054 INFO:teuthology.orchestra.run.smithi204.stdout: debug mon = 20 2024-05-11T12:09:39.054 INFO:teuthology.orchestra.run.smithi204.stdout: debug auth = 20 2024-05-11T12:09:39.054 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min pgs per osd = 4 2024-05-11T12:09:39.055 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min bytes per osd = 10 2024-05-11T12:09:39.055 INFO:teuthology.orchestra.run.smithi204.stdout: mgr/telemetry/nag = false 2024-05-11T12:09:39.055 INFO:teuthology.orchestra.run.smithi204.stdout: debug client = 20 2024-05-11T12:09:39.055 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:09:39.055 INFO:teuthology.orchestra.run.smithi204.stdout:[mon] 2024-05-11T12:09:39.055 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-05-11T12:09:39.055 INFO:teuthology.orchestra.run.smithi204.stdout: debug mon = 20 2024-05-11T12:09:39.055 INFO:teuthology.orchestra.run.smithi204.stdout: debug paxos = 20 2024-05-11T12:09:39.055 INFO:teuthology.orchestra.run.smithi204.stdout: debug auth = 20 2024-05-11T12:09:39.055 INFO:teuthology.orchestra.run.smithi204.stdout: mon data avail warn = 5 2024-05-11T12:09:39.055 INFO:teuthology.orchestra.run.smithi204.stdout: mon mgr mkfs grace = 240 2024-05-11T12:09:39.055 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min pgs per osd = 4 2024-05-11T12:09:39.055 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd reporter subtree level = osd 2024-05-11T12:09:39.055 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd prime pg temp = true 2024-05-11T12:09:39.055 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min bytes per osd = 10 2024-05-11T12:09:39.055 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:09:39.056 INFO:teuthology.orchestra.run.smithi204.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-11T12:09:39.056 INFO:teuthology.orchestra.run.smithi204.stdout: auth mon ticket ttl = 660 # 11m 2024-05-11T12:09:39.056 INFO:teuthology.orchestra.run.smithi204.stdout: auth service ticket ttl = 240 # 4m 2024-05-11T12:09:39.056 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:09:39.056 INFO:teuthology.orchestra.run.smithi204.stdout: # don't complain about insecure global_id in the test suite 2024-05-11T12:09:39.056 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-11T12:09:39.056 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-11T12:09:39.056 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:09:39.056 INFO:teuthology.orchestra.run.smithi204.stdout: # 1m isn't quite enough 2024-05-11T12:09:39.056 INFO:teuthology.orchestra.run.smithi204.stdout: mon_down_mkfs_grace = 2m 2024-05-11T12:09:39.056 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:09:39.056 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_filestore_osds = false 2024-05-11T12:09:39.056 INFO:teuthology.orchestra.run.smithi204.stdout: mon down mkfs grace = 300 2024-05-11T12:09:39.057 INFO:teuthology.orchestra.run.smithi204.stdout: mon op complaint time = 120 2024-05-11T12:09:39.057 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:09:39.057 INFO:teuthology.orchestra.run.smithi204.stdout:[client] 2024-05-11T12:09:39.057 INFO:teuthology.orchestra.run.smithi204.stdout: rgw cache enabled = true 2024-05-11T12:09:39.057 INFO:teuthology.orchestra.run.smithi204.stdout: rgw enable ops log = true 2024-05-11T12:09:39.057 INFO:teuthology.orchestra.run.smithi204.stdout: rgw enable usage log = true 2024-05-11T12:09:39.057 INFO:teuthology.orchestra.run.smithi204.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-11T12:09:39.057 INFO:teuthology.orchestra.run.smithi204.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-11T12:09:39.057 INFO:teuthology.orchestra.run.smithi204.stdout: client mount timeout = 600 2024-05-11T12:09:39.057 INFO:teuthology.orchestra.run.smithi204.stdout: debug client = 20 2024-05-11T12:09:39.057 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-05-11T12:09:39.057 INFO:teuthology.orchestra.run.smithi204.stdout: pid file = "" 2024-05-11T12:09:39.057 INFO:teuthology.orchestra.run.smithi204.stdout: rados mon op timeout = 900 2024-05-11T12:09:39.057 INFO:teuthology.orchestra.run.smithi204.stdout: rados osd op timeout = 900 2024-05-11T12:09:39.058 INFO:teuthology.orchestra.run.smithi204.stdout:[mon.a] 2024-05-11T12:09:39.058 INFO:teuthology.orchestra.run.smithi204.stdout:[mon.b] 2024-05-11T12:09:39.058 INFO:teuthology.orchestra.run.smithi204.stdout:[mon.c] 2024-05-11T12:09:39.058 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.a] 2024-05-11T12:09:39.058 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.c] 2024-05-11T12:09:39.058 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.b] 2024-05-11T12:09:39.058 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.d] 2024-05-11T12:09:39.058 INFO:teuthology.orchestra.run.smithi204.stdout:[mds] 2024-05-11T12:09:39.058 INFO:teuthology.orchestra.run.smithi204.stdout: debug mds = 20 2024-05-11T12:09:39.058 INFO:teuthology.orchestra.run.smithi204.stdout: debug mds balancer = 20 2024-05-11T12:09:39.058 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-05-11T12:09:39.058 INFO:teuthology.orchestra.run.smithi204.stdout: mds debug frag = True 2024-05-11T12:09:39.058 INFO:teuthology.orchestra.run.smithi204.stdout: mds debug scatterstat = True 2024-05-11T12:09:39.058 INFO:teuthology.orchestra.run.smithi204.stdout: mds op complaint time = 180 2024-05-11T12:09:39.059 INFO:teuthology.orchestra.run.smithi204.stdout: mds verify scatter = True 2024-05-11T12:09:39.059 INFO:teuthology.orchestra.run.smithi204.stdout: osd op complaint time = 180 2024-05-11T12:09:39.059 INFO:teuthology.orchestra.run.smithi204.stdout: rados mon op timeout = 900 2024-05-11T12:09:39.059 INFO:teuthology.orchestra.run.smithi204.stdout: rados osd op timeout = 900 2024-05-11T12:09:39.082 INFO:tasks.ceph:Creating admin key on mon.a... 2024-05-11T12:09:39.082 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-05-11T12:09:39.216 INFO:tasks.ceph:Copying monmap to all nodes... 2024-05-11T12:09:39.216 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:09:39.216 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-05-11T12:09:39.262 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:09:39.262 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-05-11T12:09:39.355 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi167.front.sepia.ceph.com 2024-05-11T12:09:39.355 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:09:39.355 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-11T12:09:39.355 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-11T12:09:39.476 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:09:39.476 DEBUG:teuthology.orchestra.run.smithi167:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-11T12:09:39.569 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi204.front.sepia.ceph.com 2024-05-11T12:09:39.570 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:09:39.570 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-11T12:09:39.570 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-11T12:09:39.692 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:09:39.692 DEBUG:teuthology.orchestra.run.smithi204:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-11T12:09:39.788 INFO:tasks.ceph:Setting up mon nodes... 2024-05-11T12:09:39.788 INFO:tasks.ceph:Setting up mgr nodes... 2024-05-11T12:09:39.788 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-05-11T12:09:39.960 INFO:teuthology.orchestra.run.smithi167.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-05-11T12:09:39.971 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-05-11T12:09:40.162 INFO:teuthology.orchestra.run.smithi204.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-05-11T12:09:40.174 INFO:tasks.ceph:Setting up mds nodes... 2024-05-11T12:09:40.175 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-05-11T12:09:40.344 INFO:teuthology.orchestra.run.smithi167.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-05-11T12:09:40.353 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-05-11T12:09:40.434 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-05-11T12:09:40.591 INFO:teuthology.orchestra.run.smithi167.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-05-11T12:09:40.599 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-05-11T12:09:40.683 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-05-11T12:09:40.848 INFO:teuthology.orchestra.run.smithi204.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-05-11T12:09:40.856 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-05-11T12:09:40.938 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-05-11T12:09:41.097 INFO:teuthology.orchestra.run.smithi204.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-05-11T12:09:41.106 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-05-11T12:09:41.192 INFO:tasks.ceph_client:Setting up client nodes... 2024-05-11T12:09:41.192 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-11T12:09:41.315 INFO:teuthology.orchestra.run.smithi167.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-05-11T12:09:41.368 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-05-11T12:09:41.483 INFO:teuthology.orchestra.run.smithi167.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-05-11T12:09:41.523 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-05-11T12:09:41.646 INFO:teuthology.orchestra.run.smithi204.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-05-11T12:09:41.683 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-05-11T12:09:41.798 INFO:teuthology.orchestra.run.smithi204.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-05-11T12:09:41.836 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-05-11T12:09:41.836 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi167.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi204.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-11T12:09:41.836 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-05-11T12:09:41.924 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-11T12:09:41.924 INFO:tasks.ceph:role: osd.0 2024-05-11T12:09:41.924 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi167.front.sepia.ceph.com 2024-05-11T12:09:41.924 DEBUG:teuthology.orchestra.run.smithi167:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-11T12:09:42.013 INFO:teuthology.orchestra.run.smithi167.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-11T12:09:42.013 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-11T12:09:42.013 INFO:teuthology.orchestra.run.smithi167.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-11T12:09:42.013 INFO:teuthology.orchestra.run.smithi167.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-11T12:09:42.013 INFO:teuthology.orchestra.run.smithi167.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-11T12:09:42.013 INFO:teuthology.orchestra.run.smithi167.stdout: = sunit=0 swidth=0 blks 2024-05-11T12:09:42.013 INFO:teuthology.orchestra.run.smithi167.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-11T12:09:42.013 INFO:teuthology.orchestra.run.smithi167.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-11T12:09:42.013 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-11T12:09:42.013 INFO:teuthology.orchestra.run.smithi167.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-11T12:09:42.428 INFO:teuthology.orchestra.run.smithi167.stdout:Discarding blocks...Done. 2024-05-11T12:09:42.434 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi167.front.sepia.ceph.com -o noatime 2024-05-11T12:09:42.435 DEBUG:teuthology.orchestra.run.smithi167:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-05-11T12:09:42.611 DEBUG:teuthology.orchestra.run.smithi167:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-05-11T12:09:42.703 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-05-11T12:09:42.780 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-11T12:09:42.781 INFO:tasks.ceph:role: osd.1 2024-05-11T12:09:42.781 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi167.front.sepia.ceph.com 2024-05-11T12:09:42.781 DEBUG:teuthology.orchestra.run.smithi167:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-11T12:09:42.859 INFO:teuthology.orchestra.run.smithi167.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-11T12:09:42.859 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-11T12:09:42.859 INFO:teuthology.orchestra.run.smithi167.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-11T12:09:42.859 INFO:teuthology.orchestra.run.smithi167.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-11T12:09:42.859 INFO:teuthology.orchestra.run.smithi167.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-11T12:09:42.859 INFO:teuthology.orchestra.run.smithi167.stdout: = sunit=0 swidth=0 blks 2024-05-11T12:09:42.859 INFO:teuthology.orchestra.run.smithi167.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-11T12:09:42.859 INFO:teuthology.orchestra.run.smithi167.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-11T12:09:42.859 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-11T12:09:42.859 INFO:teuthology.orchestra.run.smithi167.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-11T12:09:43.214 INFO:teuthology.orchestra.run.smithi167.stdout:Discarding blocks...Done. 2024-05-11T12:09:43.218 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi167.front.sepia.ceph.com -o noatime 2024-05-11T12:09:43.218 DEBUG:teuthology.orchestra.run.smithi167:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-05-11T12:09:43.320 DEBUG:teuthology.orchestra.run.smithi167:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-05-11T12:09:43.399 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-05-11T12:09:43.463 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-11T12:09:43.463 INFO:tasks.ceph:role: osd.2 2024-05-11T12:09:43.463 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi167.front.sepia.ceph.com 2024-05-11T12:09:43.463 DEBUG:teuthology.orchestra.run.smithi167:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-11T12:09:43.540 INFO:teuthology.orchestra.run.smithi167.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-11T12:09:43.540 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-11T12:09:43.540 INFO:teuthology.orchestra.run.smithi167.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-11T12:09:43.540 INFO:teuthology.orchestra.run.smithi167.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-11T12:09:43.540 INFO:teuthology.orchestra.run.smithi167.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-11T12:09:43.540 INFO:teuthology.orchestra.run.smithi167.stdout: = sunit=0 swidth=0 blks 2024-05-11T12:09:43.540 INFO:teuthology.orchestra.run.smithi167.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-11T12:09:43.541 INFO:teuthology.orchestra.run.smithi167.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-11T12:09:43.541 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-11T12:09:43.541 INFO:teuthology.orchestra.run.smithi167.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-11T12:09:43.934 INFO:teuthology.orchestra.run.smithi167.stdout:Discarding blocks...Done. 2024-05-11T12:09:43.940 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi167.front.sepia.ceph.com -o noatime 2024-05-11T12:09:43.940 DEBUG:teuthology.orchestra.run.smithi167:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-05-11T12:09:44.072 DEBUG:teuthology.orchestra.run.smithi167:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-05-11T12:09:44.149 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-05-11T12:09:44.218 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-11T12:09:44.218 INFO:tasks.ceph:role: osd.3 2024-05-11T12:09:44.218 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi167.front.sepia.ceph.com 2024-05-11T12:09:44.218 DEBUG:teuthology.orchestra.run.smithi167:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-11T12:09:44.291 INFO:teuthology.orchestra.run.smithi167.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-11T12:09:44.291 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-11T12:09:44.291 INFO:teuthology.orchestra.run.smithi167.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-11T12:09:44.291 INFO:teuthology.orchestra.run.smithi167.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-11T12:09:44.291 INFO:teuthology.orchestra.run.smithi167.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-11T12:09:44.291 INFO:teuthology.orchestra.run.smithi167.stdout: = sunit=0 swidth=0 blks 2024-05-11T12:09:44.291 INFO:teuthology.orchestra.run.smithi167.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-11T12:09:44.292 INFO:teuthology.orchestra.run.smithi167.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-11T12:09:44.292 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-11T12:09:44.292 INFO:teuthology.orchestra.run.smithi167.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-11T12:09:44.707 INFO:teuthology.orchestra.run.smithi167.stdout:Discarding blocks...Done. 2024-05-11T12:09:44.713 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi167.front.sepia.ceph.com -o noatime 2024-05-11T12:09:44.713 DEBUG:teuthology.orchestra.run.smithi167:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-05-11T12:09:44.882 DEBUG:teuthology.orchestra.run.smithi167:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-05-11T12:09:44.969 DEBUG:teuthology.orchestra.run.smithi167:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-11T12:09:45.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:09:45.123+0000 7f47ef214640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-05-11T12:09:45.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:09:45.124+0000 7f47ef214640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-05-11T12:09:45.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:09:45.124+0000 7f47ef214640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-11T12:09:45.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:09:45.124+0000 7f47ef214640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-11T12:09:45.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:09:45.124+0000 7f47ef214640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-11T12:09:45.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:09:45.128+0000 7f47ef214640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-05-11T12:09:48.096 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-11T12:09:48.184 DEBUG:teuthology.orchestra.run.smithi167:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-11T12:09:48.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:09:48.309+0000 7f8569215640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-05-11T12:09:48.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:09:48.310+0000 7f8569215640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-05-11T12:09:48.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:09:48.310+0000 7f8569215640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-11T12:09:48.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:09:48.310+0000 7f8569215640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-11T12:09:48.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:09:48.310+0000 7f8569215640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-11T12:09:48.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:09:48.312+0000 7f8569215640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-05-11T12:09:51.515 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-11T12:09:51.603 DEBUG:teuthology.orchestra.run.smithi167:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-11T12:09:51.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:09:51.729+0000 7f087940d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-05-11T12:09:51.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:09:51.730+0000 7f087940d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-05-11T12:09:51.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:09:51.730+0000 7f087940d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-11T12:09:51.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:09:51.730+0000 7f087940d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-11T12:09:51.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:09:51.730+0000 7f087940d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-11T12:09:51.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:09:51.732+0000 7f087940d640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-05-11T12:09:54.708 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-11T12:09:54.794 DEBUG:teuthology.orchestra.run.smithi167:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-11T12:09:54.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:09:54.916+0000 7f36b9a0f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-05-11T12:09:54.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:09:54.917+0000 7f36b9a0f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-05-11T12:09:54.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:09:54.917+0000 7f36b9a0f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-11T12:09:54.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:09:54.917+0000 7f36b9a0f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-11T12:09:54.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:09:54.917+0000 7f36b9a0f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-11T12:09:54.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:09:54.920+0000 7f36b9a0f640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-05-11T12:09:58.159 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-11T12:09:58.246 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-05-11T12:09:58.336 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-11T12:09:58.336 INFO:tasks.ceph:role: osd.4 2024-05-11T12:09:58.336 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi204.front.sepia.ceph.com 2024-05-11T12:09:58.336 DEBUG:teuthology.orchestra.run.smithi204:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-11T12:09:58.424 INFO:teuthology.orchestra.run.smithi204.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-11T12:09:58.425 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-11T12:09:58.425 INFO:teuthology.orchestra.run.smithi204.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-11T12:09:58.425 INFO:teuthology.orchestra.run.smithi204.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-11T12:09:58.425 INFO:teuthology.orchestra.run.smithi204.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-11T12:09:58.425 INFO:teuthology.orchestra.run.smithi204.stdout: = sunit=0 swidth=0 blks 2024-05-11T12:09:58.425 INFO:teuthology.orchestra.run.smithi204.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-11T12:09:58.425 INFO:teuthology.orchestra.run.smithi204.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-11T12:09:58.425 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-11T12:09:58.425 INFO:teuthology.orchestra.run.smithi204.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-11T12:09:58.832 INFO:teuthology.orchestra.run.smithi204.stdout:Discarding blocks...Done. 2024-05-11T12:09:58.839 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi204.front.sepia.ceph.com -o noatime 2024-05-11T12:09:58.839 DEBUG:teuthology.orchestra.run.smithi204:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-05-11T12:09:58.993 DEBUG:teuthology.orchestra.run.smithi204:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-05-11T12:09:59.083 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-05-11T12:09:59.162 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-11T12:09:59.162 INFO:tasks.ceph:role: osd.5 2024-05-11T12:09:59.162 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi204.front.sepia.ceph.com 2024-05-11T12:09:59.162 DEBUG:teuthology.orchestra.run.smithi204:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-11T12:09:59.241 INFO:teuthology.orchestra.run.smithi204.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-11T12:09:59.241 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-11T12:09:59.241 INFO:teuthology.orchestra.run.smithi204.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-11T12:09:59.241 INFO:teuthology.orchestra.run.smithi204.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-11T12:09:59.242 INFO:teuthology.orchestra.run.smithi204.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-11T12:09:59.242 INFO:teuthology.orchestra.run.smithi204.stdout: = sunit=0 swidth=0 blks 2024-05-11T12:09:59.242 INFO:teuthology.orchestra.run.smithi204.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-11T12:09:59.242 INFO:teuthology.orchestra.run.smithi204.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-11T12:09:59.242 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-11T12:09:59.242 INFO:teuthology.orchestra.run.smithi204.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-11T12:09:59.598 INFO:teuthology.orchestra.run.smithi204.stdout:Discarding blocks...Done. 2024-05-11T12:09:59.603 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi204.front.sepia.ceph.com -o noatime 2024-05-11T12:09:59.603 DEBUG:teuthology.orchestra.run.smithi204:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-05-11T12:09:59.725 DEBUG:teuthology.orchestra.run.smithi204:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-05-11T12:09:59.810 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-05-11T12:09:59.883 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-11T12:09:59.883 INFO:tasks.ceph:role: osd.6 2024-05-11T12:09:59.883 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi204.front.sepia.ceph.com 2024-05-11T12:09:59.883 DEBUG:teuthology.orchestra.run.smithi204:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-11T12:09:59.970 INFO:teuthology.orchestra.run.smithi204.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-11T12:09:59.970 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-11T12:09:59.970 INFO:teuthology.orchestra.run.smithi204.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-11T12:09:59.970 INFO:teuthology.orchestra.run.smithi204.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-11T12:09:59.970 INFO:teuthology.orchestra.run.smithi204.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-11T12:09:59.970 INFO:teuthology.orchestra.run.smithi204.stdout: = sunit=0 swidth=0 blks 2024-05-11T12:09:59.970 INFO:teuthology.orchestra.run.smithi204.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-11T12:09:59.970 INFO:teuthology.orchestra.run.smithi204.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-11T12:09:59.970 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-11T12:09:59.970 INFO:teuthology.orchestra.run.smithi204.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-11T12:10:00.354 INFO:teuthology.orchestra.run.smithi204.stdout:Discarding blocks...Done. 2024-05-11T12:10:00.360 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi204.front.sepia.ceph.com -o noatime 2024-05-11T12:10:00.360 DEBUG:teuthology.orchestra.run.smithi204:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-05-11T12:10:00.502 DEBUG:teuthology.orchestra.run.smithi204:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-05-11T12:10:00.585 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-05-11T12:10:00.659 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-11T12:10:00.659 INFO:tasks.ceph:role: osd.7 2024-05-11T12:10:00.659 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi204.front.sepia.ceph.com 2024-05-11T12:10:00.659 DEBUG:teuthology.orchestra.run.smithi204:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-11T12:10:00.741 INFO:teuthology.orchestra.run.smithi204.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-11T12:10:00.741 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-11T12:10:00.741 INFO:teuthology.orchestra.run.smithi204.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-11T12:10:00.741 INFO:teuthology.orchestra.run.smithi204.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-11T12:10:00.741 INFO:teuthology.orchestra.run.smithi204.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-11T12:10:00.741 INFO:teuthology.orchestra.run.smithi204.stdout: = sunit=0 swidth=0 blks 2024-05-11T12:10:00.741 INFO:teuthology.orchestra.run.smithi204.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-11T12:10:00.741 INFO:teuthology.orchestra.run.smithi204.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-11T12:10:00.741 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-11T12:10:00.741 INFO:teuthology.orchestra.run.smithi204.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-11T12:10:01.161 INFO:teuthology.orchestra.run.smithi204.stdout:Discarding blocks...Done. 2024-05-11T12:10:01.168 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi204.front.sepia.ceph.com -o noatime 2024-05-11T12:10:01.168 DEBUG:teuthology.orchestra.run.smithi204:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-05-11T12:10:01.322 DEBUG:teuthology.orchestra.run.smithi204:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-05-11T12:10:01.410 DEBUG:teuthology.orchestra.run.smithi204:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-11T12:10:01.538 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:01.539+0000 7f4993a3c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-05-11T12:10:01.539 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:01.540+0000 7f4993a3c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-05-11T12:10:01.539 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:01.540+0000 7f4993a3c640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-11T12:10:01.539 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:01.540+0000 7f4993a3c640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-11T12:10:01.539 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:01.540+0000 7f4993a3c640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-11T12:10:01.542 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:01.543+0000 7f4993a3c640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-05-11T12:10:04.775 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-11T12:10:04.858 DEBUG:teuthology.orchestra.run.smithi204:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-11T12:10:04.988 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:04.989+0000 7fdf9eb4c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-05-11T12:10:04.989 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:04.990+0000 7fdf9eb4c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-05-11T12:10:04.989 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:04.990+0000 7fdf9eb4c640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-11T12:10:04.989 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:04.990+0000 7fdf9eb4c640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-11T12:10:04.989 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:04.990+0000 7fdf9eb4c640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-11T12:10:04.992 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:04.993+0000 7fdf9eb4c640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-05-11T12:10:06.371 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T12:10:06.380 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T12:10:07.955 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-11T12:10:08.039 DEBUG:teuthology.orchestra.run.smithi204:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-11T12:10:08.161 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:08.162+0000 7f91e700d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-05-11T12:10:08.162 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:08.163+0000 7f91e700d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-05-11T12:10:08.162 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:08.163+0000 7f91e700d640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-11T12:10:08.162 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:08.163+0000 7f91e700d640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-11T12:10:08.162 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:08.163+0000 7f91e700d640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-11T12:10:08.165 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:08.166+0000 7f91e700d640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-05-11T12:10:11.138 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-05-11T12:10:11.222 DEBUG:teuthology.orchestra.run.smithi204:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-11T12:10:11.354 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:11.354+0000 7f0fce43a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-05-11T12:10:11.354 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:11.354+0000 7f0fce43a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-05-11T12:10:11.354 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:11.354+0000 7f0fce43a640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-11T12:10:11.355 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:11.354+0000 7f0fce43a640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-11T12:10:11.355 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:11.354+0000 7f0fce43a640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-11T12:10:11.357 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:11.357+0000 7f0fce43a640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-05-11T12:10:14.567 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-05-11T12:10:14.654 INFO:tasks.ceph:Reading keys from all nodes... 2024-05-11T12:10:14.654 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:10:14.654 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-05-11T12:10:14.745 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:10:14.746 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-05-11T12:10:14.830 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:10:14.830 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-05-11T12:10:14.913 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:10:14.913 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-05-11T12:10:14.994 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:10:14.994 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-05-11T12:10:15.078 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:10:15.078 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-05-11T12:10:15.152 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:10:15.153 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-05-11T12:10:15.227 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:10:15.227 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-05-11T12:10:15.316 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:10:15.316 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-05-11T12:10:15.401 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:10:15.401 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-05-11T12:10:15.486 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:10:15.486 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-05-11T12:10:15.567 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:10:15.567 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-05-11T12:10:15.649 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:10:15.649 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-05-11T12:10:15.729 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:10:15.729 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-05-11T12:10:15.809 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:10:15.809 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-05-11T12:10:15.865 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:10:15.865 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-05-11T12:10:15.918 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:10:15.918 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-05-11T12:10:15.975 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:10:15.975 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-05-11T12:10:16.026 INFO:tasks.ceph:Adding keys to all mons... 2024-05-11T12:10:16.026 DEBUG:teuthology.orchestra.run.smithi167:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-11T12:10:16.033 DEBUG:teuthology.orchestra.run.smithi204:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-11T12:10:16.107 INFO:teuthology.orchestra.run.smithi204.stdout:[mgr.y] 2024-05-11T12:10:16.107 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQADYD9m2cZRORAAX9kYcT5/t4f5jQRZwXHHzw== 2024-05-11T12:10:16.107 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.a] 2024-05-11T12:10:16.107 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQAEYD9msMGnFBAAamqokXQ+znxPHtHhhs611g== 2024-05-11T12:10:16.107 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.c] 2024-05-11T12:10:16.107 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQAEYD9mhylUIxAAqrIRYuNIHKfoI00zx+Gh1Q== 2024-05-11T12:10:16.107 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.0] 2024-05-11T12:10:16.107 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQAJYD9mb71mBxAAKMSzkfppZVbEevwELKrqvw== 2024-05-11T12:10:16.107 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.1] 2024-05-11T12:10:16.107 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQAMYD9m/qx7EhAAUcZntottFNBwqjdc/CqFbw== 2024-05-11T12:10:16.107 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.2] 2024-05-11T12:10:16.107 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQAPYD9m5PKRKxAAFQuzh7dAN8PVEHBq+UdQvg== 2024-05-11T12:10:16.107 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.3] 2024-05-11T12:10:16.108 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQASYD9mZAy+NhAA49k6QbSvtSxA/5t8bWbPuQ== 2024-05-11T12:10:16.108 INFO:teuthology.orchestra.run.smithi204.stdout:[mgr.x] 2024-05-11T12:10:16.108 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQAEYD9mAZTKCRAAJbY821O+4pUMC0DgxRa3og== 2024-05-11T12:10:16.108 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.b] 2024-05-11T12:10:16.108 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQAEYD9mqbCgMhAA5CNUf1cz77nUqLli2bTFcQ== 2024-05-11T12:10:16.108 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.d] 2024-05-11T12:10:16.108 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQAFYD9mcUjpBRAAbx54iDHcqexfNXGKk/avlw== 2024-05-11T12:10:16.108 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.4] 2024-05-11T12:10:16.108 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQAZYD9mEbsvIBAA3nkQXQTbIf7ubq1SJtUeIA== 2024-05-11T12:10:16.108 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.5] 2024-05-11T12:10:16.108 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQAcYD9mtdsHOxAAvNUSkMgQlDc2n1ak/lfRgg== 2024-05-11T12:10:16.108 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.6] 2024-05-11T12:10:16.108 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQAgYD9mr3u8CRAAm2ShwV0lJpIVudodmviusg== 2024-05-11T12:10:16.108 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.7] 2024-05-11T12:10:16.108 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQAjYD9m+0oyFRAAUVQW0kQmvtU8IE599nrWJQ== 2024-05-11T12:10:16.109 INFO:teuthology.orchestra.run.smithi204.stdout:[client.0] 2024-05-11T12:10:16.109 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQAFYD9mLK/lEhAAyERTYMl250KkBvIrAXWYSw== 2024-05-11T12:10:16.109 INFO:teuthology.orchestra.run.smithi204.stdout:[client.2] 2024-05-11T12:10:16.109 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQAFYD9mMzXnHBAAmJ//hwSofaZpk+6ykxVM/A== 2024-05-11T12:10:16.109 INFO:teuthology.orchestra.run.smithi204.stdout:[client.1] 2024-05-11T12:10:16.109 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQAFYD9m2oSjJhAAz53H1SvPStkxGq5V+2VFeA== 2024-05-11T12:10:16.109 INFO:teuthology.orchestra.run.smithi204.stdout:[client.3] 2024-05-11T12:10:16.109 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQAFYD9muOmsLxAA/tYFT4YNy3adENgVNLljdw== 2024-05-11T12:10:16.110 INFO:teuthology.orchestra.run.smithi167.stdout:[mgr.y] 2024-05-11T12:10:16.110 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQADYD9m2cZRORAAX9kYcT5/t4f5jQRZwXHHzw== 2024-05-11T12:10:16.110 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.a] 2024-05-11T12:10:16.111 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQAEYD9msMGnFBAAamqokXQ+znxPHtHhhs611g== 2024-05-11T12:10:16.111 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.c] 2024-05-11T12:10:16.111 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQAEYD9mhylUIxAAqrIRYuNIHKfoI00zx+Gh1Q== 2024-05-11T12:10:16.111 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.0] 2024-05-11T12:10:16.111 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQAJYD9mb71mBxAAKMSzkfppZVbEevwELKrqvw== 2024-05-11T12:10:16.111 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.1] 2024-05-11T12:10:16.111 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQAMYD9m/qx7EhAAUcZntottFNBwqjdc/CqFbw== 2024-05-11T12:10:16.111 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.2] 2024-05-11T12:10:16.111 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQAPYD9m5PKRKxAAFQuzh7dAN8PVEHBq+UdQvg== 2024-05-11T12:10:16.111 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.3] 2024-05-11T12:10:16.111 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQASYD9mZAy+NhAA49k6QbSvtSxA/5t8bWbPuQ== 2024-05-11T12:10:16.111 INFO:teuthology.orchestra.run.smithi167.stdout:[mgr.x] 2024-05-11T12:10:16.111 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQAEYD9mAZTKCRAAJbY821O+4pUMC0DgxRa3og== 2024-05-11T12:10:16.111 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.b] 2024-05-11T12:10:16.111 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQAEYD9mqbCgMhAA5CNUf1cz77nUqLli2bTFcQ== 2024-05-11T12:10:16.112 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.d] 2024-05-11T12:10:16.112 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQAFYD9mcUjpBRAAbx54iDHcqexfNXGKk/avlw== 2024-05-11T12:10:16.112 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.4] 2024-05-11T12:10:16.112 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQAZYD9mEbsvIBAA3nkQXQTbIf7ubq1SJtUeIA== 2024-05-11T12:10:16.112 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.5] 2024-05-11T12:10:16.112 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQAcYD9mtdsHOxAAvNUSkMgQlDc2n1ak/lfRgg== 2024-05-11T12:10:16.112 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.6] 2024-05-11T12:10:16.112 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQAgYD9mr3u8CRAAm2ShwV0lJpIVudodmviusg== 2024-05-11T12:10:16.112 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.7] 2024-05-11T12:10:16.112 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQAjYD9m+0oyFRAAUVQW0kQmvtU8IE599nrWJQ== 2024-05-11T12:10:16.112 INFO:teuthology.orchestra.run.smithi167.stdout:[client.0] 2024-05-11T12:10:16.112 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQAFYD9mLK/lEhAAyERTYMl250KkBvIrAXWYSw== 2024-05-11T12:10:16.112 INFO:teuthology.orchestra.run.smithi167.stdout:[client.2] 2024-05-11T12:10:16.112 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQAFYD9mMzXnHBAAmJ//hwSofaZpk+6ykxVM/A== 2024-05-11T12:10:16.112 INFO:teuthology.orchestra.run.smithi167.stdout:[client.1] 2024-05-11T12:10:16.113 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQAFYD9m2oSjJhAAz53H1SvPStkxGq5V+2VFeA== 2024-05-11T12:10:16.113 INFO:teuthology.orchestra.run.smithi167.stdout:[client.3] 2024-05-11T12:10:16.113 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQAFYD9muOmsLxAA/tYFT4YNy3adENgVNLljdw== 2024-05-11T12:10:16.115 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-11T12:10:16.163 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-11T12:10:16.299 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-11T12:10:16.307 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-11T12:10:16.443 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-11T12:10:16.449 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-11T12:10:16.583 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T12:10:16.590 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T12:10:16.724 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T12:10:16.729 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T12:10:16.863 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T12:10:16.868 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T12:10:16.993 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T12:10:17.000 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T12:10:17.133 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-11T12:10:17.138 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-11T12:10:17.266 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-11T12:10:17.270 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-11T12:10:17.397 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-11T12:10:17.403 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-11T12:10:17.534 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T12:10:17.541 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T12:10:17.678 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T12:10:17.684 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T12:10:17.808 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T12:10:17.814 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T12:10:17.945 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T12:10:17.952 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T12:10:18.088 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-11T12:10:18.095 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-11T12:10:18.228 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-11T12:10:18.235 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-11T12:10:18.373 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-11T12:10:18.378 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-11T12:10:18.511 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-11T12:10:18.517 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-11T12:10:18.647 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-05-11T12:10:18.648 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-05-11T12:10:18.732 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-11T12:10:18.884 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-05-11T12:10:18.951 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-05-11T12:10:19.039 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-11T12:10:19.188 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-05-11T12:10:19.260 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-05-11T12:10:19.342 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-11T12:10:19.486 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-05-11T12:10:19.558 DEBUG:teuthology.orchestra.run.smithi167:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-11T12:10:19.565 DEBUG:teuthology.orchestra.run.smithi204:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-11T12:10:19.617 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-05-11T12:10:19.618 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-11T12:10:19.618 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-11T12:10:19.623 INFO:tasks.ceph.mon.a:Started 2024-05-11T12:10:19.624 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-11T12:10:19.624 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-11T12:10:19.631 INFO:tasks.ceph.mon.b:Started 2024-05-11T12:10:19.631 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-11T12:10:19.631 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-11T12:10:19.637 INFO:tasks.ceph.mon.c:Started 2024-05-11T12:10:19.637 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-05-11T12:10:19.637 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-05-11T12:10:19.637 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-05-11T12:10:19.642 INFO:tasks.ceph.mgr.y:Started 2024-05-11T12:10:19.643 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-05-11T12:10:19.643 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-05-11T12:10:19.648 INFO:tasks.ceph.mgr.x:Started 2024-05-11T12:10:19.648 INFO:tasks.ceph:Setting crush tunables to default 2024-05-11T12:10:19.648 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd crush tunables default 2024-05-11T12:10:19.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:19.816+0000 7fa700000640 1 Processor -- start 2024-05-11T12:10:19.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:19.816+0000 7fa700000640 1 -- start start 2024-05-11T12:10:19.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:19.818+0000 7fa700000640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa6f815ac80 0x7fa6f8074260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:19.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:19.818+0000 7fa700000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa6f8161380 0x7fa6f8072950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:19.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:19.818+0000 7fa700000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6f80b65e0 con 0x7fa6f80b3d50 2024-05-11T12:10:19.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:19.818+0000 7fa700000640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fa6f806e3c0 con 0x7fa6f815ac80 2024-05-11T12:10:19.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:19.818+0000 7fa700000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fa6f815b060 con 0x7fa6f8161380 2024-05-11T12:10:19.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:19.818+0000 7fa6fd600640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa6f815ac80 0x7fa6f8074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:19.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:19.818+0000 7fa6fd600640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa6f815ac80 0x7fa6f8074260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:51840/0 (socket says 172.21.15.167:51840) 2024-05-11T12:10:19.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:19.818+0000 7fa6fd600640 1 -- 172.21.15.167:0/26074721 learned_addr learned my addr 172.21.15.167:0/26074721 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:10:19.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:19.818+0000 7fa6fd600640 1 -- 172.21.15.167:0/26074721 >> v2:172.21.15.167:3300/0 conn(0x7fa6f815ac80 msgr2=0x7fa6f8074260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-11T12:10:19.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:19.818+0000 7fa6fd600640 1 -- 172.21.15.167:0/26074721 >> v2:172.21.15.167:3300/0 conn(0x7fa6f815ac80 msgr2=0x7fa6f8074260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-11T12:10:19.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:19.818+0000 7fa6fd600640 1 --2- 172.21.15.167:0/26074721 >> v2:172.21.15.167:3300/0 conn(0x7fa6f815ac80 0x7fa6f8074260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-11T12:10:19.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:19.818+0000 7fa6fd600640 1 --2- 172.21.15.167:0/26074721 >> v2:172.21.15.167:3300/0 conn(0x7fa6f815ac80 0x7fa6f8074260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-11T12:10:19.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:19.818+0000 7fa6fe000640 1 --2- 172.21.15.167:0/26074721 >> v2:172.21.15.204:3300/0 conn(0x7fa6f8161380 0x7fa6f8072950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:19.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:19.819+0000 7fa6fe000640 1 -- 172.21.15.167:0/26074721 >> v2:172.21.15.204:3300/0 conn(0x7fa6f8161380 msgr2=0x7fa6f8072950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-11T12:10:19.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:19.819+0000 7fa6fe000640 1 -- 172.21.15.167:0/26074721 >> v2:172.21.15.204:3300/0 conn(0x7fa6f8161380 msgr2=0x7fa6f8072950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-11T12:10:19.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:19.819+0000 7fa6fe000640 1 --2- 172.21.15.167:0/26074721 >> v2:172.21.15.204:3300/0 conn(0x7fa6f8161380 0x7fa6f8072950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-11T12:10:19.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:19.819+0000 7fa6fe000640 1 --2- 172.21.15.167:0/26074721 >> v2:172.21.15.204:3300/0 conn(0x7fa6f8161380 0x7fa6f8072950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-11T12:10:20.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.019+0000 7fa6fd600640 1 --2- 172.21.15.167:0/26074721 >> v2:172.21.15.167:3300/0 conn(0x7fa6f815ac80 0x7fa6f8074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:20.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.020+0000 7fa6fd600640 1 -- 172.21.15.167:0/26074721 >> v2:172.21.15.167:3300/0 conn(0x7fa6f815ac80 msgr2=0x7fa6f8074260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-11T12:10:20.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.020+0000 7fa6fd600640 1 -- 172.21.15.167:0/26074721 >> v2:172.21.15.167:3300/0 conn(0x7fa6f815ac80 msgr2=0x7fa6f8074260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-11T12:10:20.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.020+0000 7fa6fd600640 1 --2- 172.21.15.167:0/26074721 >> v2:172.21.15.167:3300/0 conn(0x7fa6f815ac80 0x7fa6f8074260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-11T12:10:20.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.020+0000 7fa6fd600640 1 --2- 172.21.15.167:0/26074721 >> v2:172.21.15.167:3300/0 conn(0x7fa6f815ac80 0x7fa6f8074260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-11T12:10:20.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.020+0000 7fa6fe000640 1 --2- 172.21.15.167:0/26074721 >> v2:172.21.15.204:3300/0 conn(0x7fa6f8161380 0x7fa6f8072950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:20.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.021+0000 7fa6fe000640 1 -- 172.21.15.167:0/26074721 >> v2:172.21.15.204:3300/0 conn(0x7fa6f8161380 msgr2=0x7fa6f8072950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-11T12:10:20.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.021+0000 7fa6fe000640 1 -- 172.21.15.167:0/26074721 >> v2:172.21.15.204:3300/0 conn(0x7fa6f8161380 msgr2=0x7fa6f8072950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-11T12:10:20.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.021+0000 7fa6fe000640 1 --2- 172.21.15.167:0/26074721 >> v2:172.21.15.204:3300/0 conn(0x7fa6f8161380 0x7fa6f8072950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-11T12:10:20.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.021+0000 7fa6fe000640 1 --2- 172.21.15.167:0/26074721 >> v2:172.21.15.204:3300/0 conn(0x7fa6f8161380 0x7fa6f8072950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-11T12:10:20.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.070+0000 7fa6f7e00640 1 -- 172.21.15.167:0/26074721 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3310784164 0 0) 0x7fa6f80b65e0 con 0x7fa6f80b3d50 2024-05-11T12:10:20.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.070+0000 7fa6f7e00640 1 -- 172.21.15.167:0/26074721 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa6e0003520 con 0x7fa6f80b3d50 2024-05-11T12:10:20.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.071+0000 7fa6f7e00640 1 -- 172.21.15.167:0/26074721 <== mon.1 v1:172.21.15.204:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3047682159 0 0) 0x7fa6e0003520 con 0x7fa6f80b3d50 2024-05-11T12:10:20.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.071+0000 7fa6f7e00640 1 -- 172.21.15.167:0/26074721 >> v2:172.21.15.204:3300/0 conn(0x7fa6f8161380 msgr2=0x7fa6f8072950 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-11T12:10:20.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.071+0000 7fa6f7e00640 1 --2- 172.21.15.167:0/26074721 >> v2:172.21.15.204:3300/0 conn(0x7fa6f8161380 0x7fa6f8072950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:20.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.071+0000 7fa6f7e00640 1 -- 172.21.15.167:0/26074721 >> v2:172.21.15.167:3300/0 conn(0x7fa6f815ac80 msgr2=0x7fa6f8074260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-11T12:10:20.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.071+0000 7fa6f7e00640 1 --2- 172.21.15.167:0/26074721 >> v2:172.21.15.167:3300/0 conn(0x7fa6f815ac80 0x7fa6f8074260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:20.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.071+0000 7fa6f7e00640 1 -- 172.21.15.167:0/26074721 --> v1:172.21.15.204:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f80b68c0 con 0x7fa6f80b3d50 2024-05-11T12:10:20.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.071+0000 7fa6f7e00640 1 -- 172.21.15.167:0/26074721 <== mon.1 v1:172.21.15.204:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7fa6e40016f0 con 0x7fa6f80b3d50 2024-05-11T12:10:20.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.071+0000 7fa6f7e00640 1 -- 172.21.15.167:0/26074721 >> v1:172.21.15.204:6789/0 conn(0x7fa6f80b3d50 legacy=0x7fa6f8160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:20.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.071+0000 7fa6f7e00640 1 --2- 172.21.15.167:0/26074721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6e0003a50 0x7fa6e0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:20.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.071+0000 7fa6f7e00640 1 --2- 172.21.15.167:0/26074721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6e0004800 0x7fa6e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:20.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.071+0000 7fa6f7e00640 1 --2- 172.21.15.167:0/26074721 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6e00051c0 0x7fa6e000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:20.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.071+0000 7fa6f7e00640 1 -- 172.21.15.167:0/26074721 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f815b060 con 0x7fa6e0004800 2024-05-11T12:10:20.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.071+0000 7fa6f7e00640 1 -- 172.21.15.167:0/26074721 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e000c0f0 con 0x7fa6e0003a50 2024-05-11T12:10:20.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.071+0000 7fa6f7e00640 1 -- 172.21.15.167:0/26074721 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa6e000c2f0 con 0x7fa6e00051c0 2024-05-11T12:10:20.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.072+0000 7fa6fcc00640 1 --2- 172.21.15.167:0/26074721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6e0004800 0x7fa6e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:20.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.072+0000 7fa6fd600640 1 --2- 172.21.15.167:0/26074721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6e0003a50 0x7fa6e0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:20.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.072+0000 7fa6fe000640 1 --2- 172.21.15.167:0/26074721 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6e00051c0 0x7fa6e000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:20.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.073+0000 7fa6fcc00640 1 -- 172.21.15.167:0/26074721 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6e00051c0 msgr2=0x7fa6e000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:20.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.073+0000 7fa6fcc00640 1 --2- 172.21.15.167:0/26074721 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6e00051c0 0x7fa6e000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:20.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.073+0000 7fa6fcc00640 1 -- 172.21.15.167:0/26074721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6e0003a50 msgr2=0x7fa6e0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:20.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.073+0000 7fa6fcc00640 1 --2- 172.21.15.167:0/26074721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6e0003a50 0x7fa6e0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:20.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.073+0000 7fa6fcc00640 1 -- 172.21.15.167:0/26074721 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6e0003520 con 0x7fa6e0004800 2024-05-11T12:10:20.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.074+0000 7fa6fd600640 1 --2- 172.21.15.167:0/26074721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6e0003a50 0x7fa6e0003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:10:20.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.074+0000 7fa6fe000640 1 -- 172.21.15.167:0/26074721 reap_dead start 2024-05-11T12:10:20.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.076+0000 7fa6fcc00640 1 --2- 172.21.15.167:0/26074721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6e0004800 0x7fa6e0004c80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400c160 tx=0x7fa6e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53f995017d6633ab server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:20.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.079+0000 7fa6f7e00640 1 -- 172.21.15.167:0/26074721 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e4036070 con 0x7fa6e0004800 2024-05-11T12:10:20.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.079+0000 7fa6f7e00640 1 -- 172.21.15.167:0/26074721 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6e4031600 con 0x7fa6e0004800 2024-05-11T12:10:20.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.079+0000 7fa6f7e00640 1 -- 172.21.15.167:0/26074721 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e4007c60 con 0x7fa6e0004800 2024-05-11T12:10:20.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.080+0000 7fa700000640 1 -- 172.21.15.167:0/26074721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6e0004800 msgr2=0x7fa6e0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:20.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.080+0000 7fa700000640 1 --2- 172.21.15.167:0/26074721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6e0004800 0x7fa6e0004c80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400c160 tx=0x7fa6e0005830 comp rx=0 tx=0).stop 2024-05-11T12:10:20.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.080+0000 7fa700000640 1 -- 172.21.15.167:0/26074721 shutdown_connections 2024-05-11T12:10:20.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.080+0000 7fa700000640 1 --2- 172.21.15.167:0/26074721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6e0004800 0x7fa6e0004c80 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:20.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.080+0000 7fa700000640 1 -- 172.21.15.167:0/26074721 >> 172.21.15.167:0/26074721 conn(0x7fa6f806c450 msgr2=0x7fa6f806c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:20.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.081+0000 7fa700000640 1 -- 172.21.15.167:0/26074721 shutdown_connections 2024-05-11T12:10:20.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.081+0000 7fa700000640 1 -- 172.21.15.167:0/26074721 wait complete. 2024-05-11T12:10:20.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.086+0000 7fa700000640 1 Processor -- start 2024-05-11T12:10:20.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.086+0000 7fa700000640 1 -- start start 2024-05-11T12:10:20.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.087+0000 7fa700000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6e0004800 0x7fa6f813bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:20.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.087+0000 7fa700000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6f80ce8f0 0x7fa6f80c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:20.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.087+0000 7fa700000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6f8143f60 0x7fa6f815c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:20.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.087+0000 7fa700000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f80b44f0 con 0x7fa6f8143f60 2024-05-11T12:10:20.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.087+0000 7fa700000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f80b41e0 con 0x7fa6f80ce8f0 2024-05-11T12:10:20.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.087+0000 7fa700000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa6f8074260 con 0x7fa6e0004800 2024-05-11T12:10:20.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.088+0000 7fa6fd600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6e0004800 0x7fa6f813bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:20.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.088+0000 7fa6fcc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6f80ce8f0 0x7fa6f80c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:20.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.088+0000 7fa6fd600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6e0004800 0x7fa6f813bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:47532/0 (socket says 172.21.15.167:47532) 2024-05-11T12:10:20.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.088+0000 7fa6fd600640 1 -- 172.21.15.167:0/3308619130 learned_addr learned my addr 172.21.15.167:0/3308619130 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:10:20.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.088+0000 7fa6fe000640 1 --2- 172.21.15.167:0/3308619130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6f8143f60 0x7fa6f815c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:20.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.088+0000 7fa6fcc00640 1 -- 172.21.15.167:0/3308619130 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6e0004800 msgr2=0x7fa6f813bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:20.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.088+0000 7fa6fcc00640 1 --2- 172.21.15.167:0/3308619130 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6e0004800 0x7fa6f813bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:20.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.088+0000 7fa6fcc00640 1 -- 172.21.15.167:0/3308619130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6f8143f60 msgr2=0x7fa6f815c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:20.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.088+0000 7fa6fcc00640 1 --2- 172.21.15.167:0/3308619130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6f8143f60 0x7fa6f815c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:20.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.088+0000 7fa6fcc00640 1 -- 172.21.15.167:0/3308619130 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f8139580 con 0x7fa6f80ce8f0 2024-05-11T12:10:20.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.089+0000 7fa6fcc00640 1 --2- 172.21.15.167:0/3308619130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6f80ce8f0 0x7fa6f80c5cc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa6e40018e0 tx=0x7fa6f80b6e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:20.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.090+0000 7fa6f5600640 1 -- 172.21.15.167:0/3308619130 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e4041070 con 0x7fa6f80ce8f0 2024-05-11T12:10:20.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.090+0000 7fa6f5600640 1 -- 172.21.15.167:0/3308619130 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6e4007c40 con 0x7fa6f80ce8f0 2024-05-11T12:10:20.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.090+0000 7fa6f5600640 1 -- 172.21.15.167:0/3308619130 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e4036040 con 0x7fa6f80ce8f0 2024-05-11T12:10:20.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.090+0000 7fa700000640 1 -- 172.21.15.167:0/3308619130 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6f8139860 con 0x7fa6f80ce8f0 2024-05-11T12:10:20.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.090+0000 7fa700000640 1 -- 172.21.15.167:0/3308619130 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6f81443e0 con 0x7fa6f80ce8f0 2024-05-11T12:10:20.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.091+0000 7fa6f5600640 1 -- 172.21.15.167:0/3308619130 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa6e4044070 con 0x7fa6f80ce8f0 2024-05-11T12:10:20.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.091+0000 7fa6f5600640 1 -- 172.21.15.167:0/3308619130 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa6e4006b10 con 0x7fa6f80ce8f0 2024-05-11T12:10:20.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.092+0000 7fa700000640 1 -- 172.21.15.167:0/3308619130 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6f81445b0 con 0x7fa6f80ce8f0 2024-05-11T12:10:20.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.096+0000 7fa6f5600640 1 -- 172.21.15.167:0/3308619130 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fa6e4006d60 con 0x7fa6f80ce8f0 2024-05-11T12:10:20.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.172+0000 7fa700000640 1 -- 172.21.15.167:0/3308619130 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fa6f8069520 con 0x7fa6f80ce8f0 2024-05-11T12:10:20.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.179+0000 7fa6f5600640 1 -- 172.21.15.167:0/3308619130 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7fa6e4005330 con 0x7fa6f80ce8f0 2024-05-11T12:10:20.179 INFO:teuthology.orchestra.run.smithi167.stderr:adjusted tunables profile to default 2024-05-11T12:10:20.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.180+0000 7fa700000640 1 -- 172.21.15.167:0/3308619130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6f80ce8f0 msgr2=0x7fa6f80c5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:20.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.180+0000 7fa700000640 1 --2- 172.21.15.167:0/3308619130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6f80ce8f0 0x7fa6f80c5cc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa6e40018e0 tx=0x7fa6f80b6e80 comp rx=0 tx=0).stop 2024-05-11T12:10:20.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.181+0000 7fa700000640 1 -- 172.21.15.167:0/3308619130 shutdown_connections 2024-05-11T12:10:20.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.181+0000 7fa700000640 1 --2- 172.21.15.167:0/3308619130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6f8143f60 0x7fa6f815c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:20.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.181+0000 7fa700000640 1 --2- 172.21.15.167:0/3308619130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6f80ce8f0 0x7fa6f80c5cc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:20.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.181+0000 7fa700000640 1 --2- 172.21.15.167:0/3308619130 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6e0004800 0x7fa6f813bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:20.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.181+0000 7fa700000640 1 -- 172.21.15.167:0/3308619130 >> 172.21.15.167:0/3308619130 conn(0x7fa6f806c450 msgr2=0x7fa6f806f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:20.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.181+0000 7fa700000640 1 -- 172.21.15.167:0/3308619130 shutdown_connections 2024-05-11T12:10:20.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:20.181+0000 7fa700000640 1 -- 172.21.15.167:0/3308619130 wait complete. 2024-05-11T12:10:20.198 INFO:tasks.ceph:check_enable_crimson: False 2024-05-11T12:10:20.198 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-05-11T12:10:20.198 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:10:20.199 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-05-11T12:10:20.232 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T12:10:20.232+0000 7fa9e4ddc040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-11T12:10:20.235 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T12:10:20.235+0000 7f98b0141040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-11T12:10:20.265 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:10:20.266 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-05-11T12:10:20.384 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:10:20.384 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-05-11T12:10:20.417 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T12:10:20.417+0000 7fa9e4ddc040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-11T12:10:20.423 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T12:10:20.423+0000 7f98b0141040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-11T12:10:20.461 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:10:20.461 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-05-11T12:10:20.484 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T12:10:20.483+0000 7fa9e4ddc040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-11T12:10:20.490 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T12:10:20.489+0000 7f98b0141040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-11T12:10:20.565 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:10:20.566 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-05-11T12:10:20.640 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:10:20.640 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-05-11T12:10:20.761 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:10:20.761 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-05-11T12:10:20.765 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T12:10:20.765+0000 7f98b0141040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-11T12:10:20.778 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T12:10:20.778+0000 7fa9e4ddc040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-11T12:10:20.818 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T12:10:20.818+0000 7f98b0141040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-11T12:10:20.836 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T12:10:20.836+0000 7fa9e4ddc040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-11T12:10:20.852 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:10:20.852 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-05-11T12:10:20.970 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new f30405d9-0563-4bb3-be12-ec51d5932be7 0 2024-05-11T12:10:21.152 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.152+0000 7f1a3a200640 1 Processor -- start 2024-05-11T12:10:21.152 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.152+0000 7f1a3a200640 1 -- start start 2024-05-11T12:10:21.154 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.153+0000 7f1a3a200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a3415f7a0 0x7f1a3415fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:21.154 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.153+0000 7f1a3a200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1a3406d360 0x7f1a34168d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:21.154 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.153+0000 7f1a3a200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a3406d980 con 0x7f1a341600c0 2024-05-11T12:10:21.155 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.153+0000 7f1a3a200640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f1a341608f0 con 0x7f1a3406d360 2024-05-11T12:10:21.155 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.153+0000 7f1a3a200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1a3406df40 con 0x7f1a3415f7a0 2024-05-11T12:10:21.155 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.153+0000 7f1a38e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a3415f7a0 0x7f1a3415fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:21.155 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.154+0000 7f1a38e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a3415f7a0 0x7f1a3415fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:42960/0 (socket says 172.21.15.204:42960) 2024-05-11T12:10:21.155 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.154+0000 7f1a38e00640 1 -- 172.21.15.204:0/2299828403 learned_addr learned my addr 172.21.15.204:0/2299828403 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-11T12:10:21.155 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.154+0000 7f1a39800640 1 --2- 172.21.15.204:0/2299828403 >> v2:172.21.15.167:3300/0 conn(0x7f1a3406d360 0x7f1a34168d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:21.155 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.154+0000 7f1a38e00640 1 -- 172.21.15.204:0/2299828403 >> v2:172.21.15.167:3300/0 conn(0x7f1a3406d360 msgr2=0x7f1a34168d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:21.155 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.154+0000 7f1a38e00640 1 --2- 172.21.15.204:0/2299828403 >> v2:172.21.15.167:3300/0 conn(0x7f1a3406d360 0x7f1a34168d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:21.155 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.154+0000 7f1a33e00640 1 -- 172.21.15.204:0/2299828403 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2266875246 0 0) 0x7f1a3406d980 con 0x7f1a341600c0 2024-05-11T12:10:21.155 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.154+0000 7f1a38e00640 1 -- 172.21.15.204:0/2299828403 >> v1:172.21.15.204:6789/0 conn(0x7f1a341600c0 legacy=0x7f1a3406cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:21.155 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.154+0000 7f1a38e00640 1 -- 172.21.15.204:0/2299828403 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a3406dc60 con 0x7f1a3415f7a0 2024-05-11T12:10:21.156 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.155+0000 7f1a38e00640 1 --2- 172.21.15.204:0/2299828403 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a3415f7a0 0x7f1a3415fb80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1a24009870 tx=0x7f1a2402a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ac783d63623f129 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:21.157 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.155+0000 7f1a33e00640 1 -- 172.21.15.204:0/2299828403 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a24037070 con 0x7f1a3415f7a0 2024-05-11T12:10:21.157 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.155+0000 7f1a33e00640 1 -- 172.21.15.204:0/2299828403 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a24032440 con 0x7f1a3415f7a0 2024-05-11T12:10:21.157 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.155+0000 7f1a33e00640 1 -- 172.21.15.204:0/2299828403 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a24032810 con 0x7f1a3415f7a0 2024-05-11T12:10:21.157 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.156+0000 7f1a3a200640 1 -- 172.21.15.204:0/2299828403 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a3415f7a0 msgr2=0x7f1a3415fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:21.157 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.156+0000 7f1a3a200640 1 --2- 172.21.15.204:0/2299828403 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a3415f7a0 0x7f1a3415fb80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1a24009870 tx=0x7f1a2402a250 comp rx=0 tx=0).stop 2024-05-11T12:10:21.157 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.156+0000 7f1a3a200640 1 -- 172.21.15.204:0/2299828403 shutdown_connections 2024-05-11T12:10:21.157 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.156+0000 7f1a3a200640 1 --2- 172.21.15.204:0/2299828403 >> v2:172.21.15.167:3300/0 conn(0x7f1a3406d360 0x7f1a34168d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:21.157 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.156+0000 7f1a3a200640 1 --2- 172.21.15.204:0/2299828403 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a3415f7a0 0x7f1a3415fb80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:21.157 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.156+0000 7f1a3a200640 1 -- 172.21.15.204:0/2299828403 >> 172.21.15.204:0/2299828403 conn(0x7f1a340b6ee0 msgr2=0x7f1a340b72f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:21.157 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.156+0000 7f1a3a200640 1 -- 172.21.15.204:0/2299828403 shutdown_connections 2024-05-11T12:10:21.157 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.156+0000 7f1a3a200640 1 -- 172.21.15.204:0/2299828403 wait complete. 2024-05-11T12:10:21.158 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.158+0000 7f1a3a200640 1 Processor -- start 2024-05-11T12:10:21.158 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.158+0000 7f1a3a200640 1 -- start start 2024-05-11T12:10:21.160 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.159+0000 7f1a3a200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a3406d360 0x7f1a34152780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:21.160 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.159+0000 7f1a3a200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a3415f7a0 0x7f1a340caf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:21.161 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.159+0000 7f1a3a200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a341600c0 0x7f1a341461d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:21.161 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.159+0000 7f1a3a200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1a340d3450 con 0x7f1a341600c0 2024-05-11T12:10:21.161 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.159+0000 7f1a3a200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1a340d2f20 con 0x7f1a3415f7a0 2024-05-11T12:10:21.161 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.159+0000 7f1a3a200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1a34160520 con 0x7f1a3406d360 2024-05-11T12:10:21.161 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.159+0000 7f1a2be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a3415f7a0 0x7f1a340caf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:21.161 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.159+0000 7f1a38e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a3406d360 0x7f1a34152780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:21.161 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.159+0000 7f1a38e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a3406d360 0x7f1a34152780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:42970/0 (socket says 172.21.15.204:42970) 2024-05-11T12:10:21.161 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.159+0000 7f1a2be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a3415f7a0 0x7f1a340caf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:39598/0 (socket says 172.21.15.204:39598) 2024-05-11T12:10:21.161 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.159+0000 7f1a38e00640 1 -- 172.21.15.204:0/408127720 learned_addr learned my addr 172.21.15.204:0/408127720 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-11T12:10:21.161 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.159+0000 7f1a39800640 1 --2- 172.21.15.204:0/408127720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a341600c0 0x7f1a341461d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:21.161 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.159+0000 7f1a2be00640 1 -- 172.21.15.204:0/408127720 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a3406d360 msgr2=0x7f1a34152780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:21.161 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.159+0000 7f1a2be00640 1 --2- 172.21.15.204:0/408127720 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a3406d360 0x7f1a34152780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:21.161 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.159+0000 7f1a2be00640 1 -- 172.21.15.204:0/408127720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a341600c0 msgr2=0x7f1a341461d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:21.162 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.159+0000 7f1a2be00640 1 --2- 172.21.15.204:0/408127720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a341600c0 0x7f1a341461d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:21.162 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.159+0000 7f1a2be00640 1 -- 172.21.15.204:0/408127720 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a341e6340 con 0x7f1a3415f7a0 2024-05-11T12:10:21.162 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.159+0000 7f1a38e00640 1 --2- 172.21.15.204:0/408127720 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a3406d360 0x7f1a34152780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:10:21.162 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.159+0000 7f1a39800640 1 --2- 172.21.15.204:0/408127720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a341600c0 0x7f1a341461d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:10:21.162 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.160+0000 7f1a2be00640 1 --2- 172.21.15.204:0/408127720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a3415f7a0 0x7f1a340caf90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c00c3c0 tx=0x7f1a1c00c7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:21.162 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.160+0000 7f1a31600640 1 -- 172.21.15.204:0/408127720 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c017070 con 0x7f1a3415f7a0 2024-05-11T12:10:21.162 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.160+0000 7f1a31600640 1 -- 172.21.15.204:0/408127720 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a1c0035f0 con 0x7f1a3415f7a0 2024-05-11T12:10:21.162 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.160+0000 7f1a31600640 1 -- 172.21.15.204:0/408127720 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c003a20 con 0x7f1a3415f7a0 2024-05-11T12:10:21.162 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.160+0000 7f1a3a200640 1 -- 172.21.15.204:0/408127720 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a341e65e0 con 0x7f1a3415f7a0 2024-05-11T12:10:21.162 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.160+0000 7f1a3a200640 1 -- 172.21.15.204:0/408127720 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a34146710 con 0x7f1a3415f7a0 2024-05-11T12:10:21.162 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.160+0000 7f1a31600640 1 -- 172.21.15.204:0/408127720 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1a1c01a070 con 0x7f1a3415f7a0 2024-05-11T12:10:21.162 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.160+0000 7f1a31600640 1 -- 172.21.15.204:0/408127720 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f1a1c012740 con 0x7f1a3415f7a0 2024-05-11T12:10:21.163 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.161+0000 7f1a3a200640 1 -- 172.21.15.204:0/408127720 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a34069bc0 con 0x7f1a3415f7a0 2024-05-11T12:10:21.164 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.164+0000 7f1a31600640 1 -- 172.21.15.204:0/408127720 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f1a1c021020 con 0x7f1a3415f7a0 2024-05-11T12:10:21.243 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.243+0000 7f1a3a200640 1 -- 172.21.15.204:0/408127720 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f30405d9-0563-4bb3-be12-ec51d5932be7", "id": 0} v 0) -- 0x7f1a340cbe70 con 0x7f1a3415f7a0 2024-05-11T12:10:21.250 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.250+0000 7f1a31600640 1 -- 172.21.15.204:0/408127720 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f30405d9-0563-4bb3-be12-ec51d5932be7", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f1a1c003d60 con 0x7f1a3415f7a0 2024-05-11T12:10:21.251 INFO:teuthology.orchestra.run.smithi204.stdout:0 2024-05-11T12:10:21.252 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.252+0000 7f1a3a200640 1 -- 172.21.15.204:0/408127720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a3415f7a0 msgr2=0x7f1a340caf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:21.252 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.252+0000 7f1a3a200640 1 --2- 172.21.15.204:0/408127720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a3415f7a0 0x7f1a340caf90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c00c3c0 tx=0x7f1a1c00c7c0 comp rx=0 tx=0).stop 2024-05-11T12:10:21.253 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.252+0000 7f1a3a200640 1 -- 172.21.15.204:0/408127720 shutdown_connections 2024-05-11T12:10:21.253 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.252+0000 7f1a3a200640 1 --2- 172.21.15.204:0/408127720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a341600c0 0x7f1a341461d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:21.253 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.252+0000 7f1a3a200640 1 --2- 172.21.15.204:0/408127720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a3415f7a0 0x7f1a340caf90 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:21.253 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.252+0000 7f1a3a200640 1 --2- 172.21.15.204:0/408127720 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a3406d360 0x7f1a34152780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:21.253 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.252+0000 7f1a3a200640 1 -- 172.21.15.204:0/408127720 >> 172.21.15.204:0/408127720 conn(0x7f1a340b6ee0 msgr2=0x7f1a340b9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:21.253 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.252+0000 7f1a3a200640 1 -- 172.21.15.204:0/408127720 shutdown_connections 2024-05-11T12:10:21.253 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.252+0000 7f1a3a200640 1 -- 172.21.15.204:0/408127720 wait complete. 2024-05-11T12:10:21.268 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new 139c6ac5-9615-41f4-8ec1-f409b6b68f6f 1 2024-05-11T12:10:21.447 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.447+0000 7fe8ed600640 1 Processor -- start 2024-05-11T12:10:21.447 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.447+0000 7fe8ed600640 1 -- start start 2024-05-11T12:10:21.449 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.448+0000 7fe8ed600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8e806d200 0x7fe8e806d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:21.449 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.448+0000 7fe8ed600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe8e80d2db0 0x7fe8e80d3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:21.449 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.449+0000 7fe8ed600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8e8159d50 con 0x7fe8e8159970 2024-05-11T12:10:21.450 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.449+0000 7fe8ed600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fe8e80d04b0 con 0x7fe8e80d2db0 2024-05-11T12:10:21.450 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.449+0000 7fe8ed600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe8e80d0790 con 0x7fe8e806d200 2024-05-11T12:10:21.450 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.449+0000 7fe8dfe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8e806d200 0x7fe8e806d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:21.450 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.449+0000 7fe8e7400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe8e80d2db0 0x7fe8e80d3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:21.450 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.449+0000 7fe8e7400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe8e80d2db0 0x7fe8e80d3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:39604/0 (socket says 172.21.15.204:39604) 2024-05-11T12:10:21.450 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.449+0000 7fe8dfe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8e806d200 0x7fe8e806d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:42976/0 (socket says 172.21.15.204:42976) 2024-05-11T12:10:21.450 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.449+0000 7fe8e7400640 1 -- 172.21.15.204:0/509206724 learned_addr learned my addr 172.21.15.204:0/509206724 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-11T12:10:21.450 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.449+0000 7fe8e6a00640 1 --1- 172.21.15.204:0/509206724 >> v1:172.21.15.204:6789/0 conn(0x7fe8e8159970 0x7fe8e80d1270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:60364/0 (socket says 172.21.15.204:60364) 2024-05-11T12:10:21.450 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.449+0000 7fe8dfe00640 1 -- 172.21.15.204:0/509206724 >> v2:172.21.15.204:3300/0 conn(0x7fe8e80d2db0 msgr2=0x7fe8e80d3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:21.450 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.449+0000 7fe8dfe00640 1 --2- 172.21.15.204:0/509206724 >> v2:172.21.15.204:3300/0 conn(0x7fe8e80d2db0 0x7fe8e80d3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:21.450 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.449+0000 7fe8dfe00640 1 -- 172.21.15.204:0/509206724 >> v1:172.21.15.204:6789/0 conn(0x7fe8e8159970 legacy=0x7fe8e80d1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:21.450 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.449+0000 7fe8dfe00640 1 -- 172.21.15.204:0/509206724 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e80d0a70 con 0x7fe8e806d200 2024-05-11T12:10:21.450 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.449+0000 7fe8e7400640 1 --2- 172.21.15.204:0/509206724 >> v2:172.21.15.204:3300/0 conn(0x7fe8e80d2db0 0x7fe8e80d3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:10:21.451 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.451+0000 7fe8dfe00640 1 --2- 172.21.15.204:0/509206724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8e806d200 0x7fe8e806d660 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe8d00097d0 tx=0x7fe8d002a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bfd33dec8bc8a3c server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:21.451 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.451+0000 7fe8e6000640 1 -- 172.21.15.204:0/509206724 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d0037070 con 0x7fe8e806d200 2024-05-11T12:10:21.451 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.451+0000 7fe8e6000640 1 -- 172.21.15.204:0/509206724 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8d0032440 con 0x7fe8e806d200 2024-05-11T12:10:21.451 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.451+0000 7fe8e6000640 1 -- 172.21.15.204:0/509206724 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d0032810 con 0x7fe8e806d200 2024-05-11T12:10:21.452 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.452+0000 7fe8ed600640 1 -- 172.21.15.204:0/509206724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8e806d200 msgr2=0x7fe8e806d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:21.452 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.452+0000 7fe8ed600640 1 --2- 172.21.15.204:0/509206724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8e806d200 0x7fe8e806d660 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe8d00097d0 tx=0x7fe8d002a1c0 comp rx=0 tx=0).stop 2024-05-11T12:10:21.452 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.452+0000 7fe8ed600640 1 -- 172.21.15.204:0/509206724 shutdown_connections 2024-05-11T12:10:21.452 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.452+0000 7fe8ed600640 1 --2- 172.21.15.204:0/509206724 >> v2:172.21.15.204:3300/0 conn(0x7fe8e80d2db0 0x7fe8e80d3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:21.453 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.452+0000 7fe8ed600640 1 --2- 172.21.15.204:0/509206724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8e806d200 0x7fe8e806d660 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:21.453 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.452+0000 7fe8ed600640 1 -- 172.21.15.204:0/509206724 >> 172.21.15.204:0/509206724 conn(0x7fe8e80d51e0 msgr2=0x7fe8e80d55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:21.453 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.452+0000 7fe8ed600640 1 -- 172.21.15.204:0/509206724 shutdown_connections 2024-05-11T12:10:21.453 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.452+0000 7fe8ed600640 1 -- 172.21.15.204:0/509206724 wait complete. 2024-05-11T12:10:21.454 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.454+0000 7fe8ed600640 1 Processor -- start 2024-05-11T12:10:21.454 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.454+0000 7fe8ed600640 1 -- start start 2024-05-11T12:10:21.457 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.455+0000 7fe8ed600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8e806d200 0x7fe8e815f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:21.457 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.455+0000 7fe8ed600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8e80d2db0 0x7fe8e80caef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:21.457 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.455+0000 7fe8ed600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8e8159970 0x7fe8e815a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:21.457 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.455+0000 7fe8ed600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e80d1980 con 0x7fe8e80d2db0 2024-05-11T12:10:21.457 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.455+0000 7fe8ed600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e8163d00 con 0x7fe8e8159970 2024-05-11T12:10:21.457 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.455+0000 7fe8ed600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe8e8162f10 con 0x7fe8e806d200 2024-05-11T12:10:21.457 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.455+0000 7fe8e6a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8e806d200 0x7fe8e815f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:21.457 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.455+0000 7fe8e6a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8e806d200 0x7fe8e815f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:42980/0 (socket says 172.21.15.204:42980) 2024-05-11T12:10:21.458 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.455+0000 7fe8e6a00640 1 -- 172.21.15.204:0/1437080339 learned_addr learned my addr 172.21.15.204:0/1437080339 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-11T12:10:21.458 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.455+0000 7fe8e7400640 1 --2- 172.21.15.204:0/1437080339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8e8159970 0x7fe8e815a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:21.458 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.455+0000 7fe8dfe00640 1 --2- 172.21.15.204:0/1437080339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8e80d2db0 0x7fe8e80caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:21.458 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.455+0000 7fe8e6a00640 1 -- 172.21.15.204:0/1437080339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8e8159970 msgr2=0x7fe8e815a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:21.458 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.455+0000 7fe8e6a00640 1 --2- 172.21.15.204:0/1437080339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8e8159970 0x7fe8e815a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:21.458 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.455+0000 7fe8e6a00640 1 -- 172.21.15.204:0/1437080339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8e80d2db0 msgr2=0x7fe8e80caef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:21.458 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.455+0000 7fe8e6a00640 1 --2- 172.21.15.204:0/1437080339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8e80d2db0 0x7fe8e80caef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:21.458 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.455+0000 7fe8e6a00640 1 -- 172.21.15.204:0/1437080339 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e8158ed0 con 0x7fe8e806d200 2024-05-11T12:10:21.458 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.456+0000 7fe8e7400640 1 --2- 172.21.15.204:0/1437080339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8e8159970 0x7fe8e815a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:10:21.458 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.456+0000 7fe8dfe00640 1 --2- 172.21.15.204:0/1437080339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8e80d2db0 0x7fe8e80caef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:10:21.458 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.456+0000 7fe8e6a00640 1 --2- 172.21.15.204:0/1437080339 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8e806d200 0x7fe8e815f9e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8001eb0 tx=0x7fe8d800c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:21.458 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.456+0000 7fe8dea00640 1 -- 172.21.15.204:0/1437080339 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d80074e0 con 0x7fe8e806d200 2024-05-11T12:10:21.458 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.456+0000 7fe8dea00640 1 -- 172.21.15.204:0/1437080339 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8d8007680 con 0x7fe8e806d200 2024-05-11T12:10:21.459 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.456+0000 7fe8dea00640 1 -- 172.21.15.204:0/1437080339 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d80049e0 con 0x7fe8e806d200 2024-05-11T12:10:21.459 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.456+0000 7fe8ed600640 1 -- 172.21.15.204:0/1437080339 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8e815ffc0 con 0x7fe8e806d200 2024-05-11T12:10:21.459 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.456+0000 7fe8ed600640 1 -- 172.21.15.204:0/1437080339 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8e815d4d0 con 0x7fe8e806d200 2024-05-11T12:10:21.459 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.456+0000 7fe8dea00640 1 -- 172.21.15.204:0/1437080339 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe8d801a020 con 0x7fe8e806d200 2024-05-11T12:10:21.459 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.457+0000 7fe8dea00640 1 -- 172.21.15.204:0/1437080339 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fe8d801c950 con 0x7fe8e806d200 2024-05-11T12:10:21.459 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.457+0000 7fe8ed600640 1 -- 172.21.15.204:0/1437080339 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8e8069520 con 0x7fe8e806d200 2024-05-11T12:10:21.460 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.460+0000 7fe8dea00640 1 -- 172.21.15.204:0/1437080339 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fe8d8021070 con 0x7fe8e806d200 2024-05-11T12:10:21.541 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.541+0000 7fe8ed600640 1 -- 172.21.15.204:0/1437080339 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd new", "uuid": "139c6ac5-9615-41f4-8ec1-f409b6b68f6f", "id": 1} v 0) -- 0x7fe8e80b8400 con 0x7fe8e806d200 2024-05-11T12:10:21.548 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.548+0000 7fe8dea00640 1 -- 172.21.15.204:0/1437080339 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "139c6ac5-9615-41f4-8ec1-f409b6b68f6f", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fe8e8158ed0 con 0x7fe8e806d200 2024-05-11T12:10:21.548 INFO:teuthology.orchestra.run.smithi204.stdout:1 2024-05-11T12:10:21.549 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.549+0000 7fe8ed600640 1 -- 172.21.15.204:0/1437080339 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8e806d200 msgr2=0x7fe8e815f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:21.549 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.549+0000 7fe8ed600640 1 --2- 172.21.15.204:0/1437080339 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8e806d200 0x7fe8e815f9e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8001eb0 tx=0x7fe8d800c890 comp rx=0 tx=0).stop 2024-05-11T12:10:21.549 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.549+0000 7fe8ed600640 1 -- 172.21.15.204:0/1437080339 shutdown_connections 2024-05-11T12:10:21.549 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.549+0000 7fe8ed600640 1 --2- 172.21.15.204:0/1437080339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8e8159970 0x7fe8e815a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:21.550 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.549+0000 7fe8ed600640 1 --2- 172.21.15.204:0/1437080339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8e80d2db0 0x7fe8e80caef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:21.550 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.549+0000 7fe8ed600640 1 --2- 172.21.15.204:0/1437080339 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8e806d200 0x7fe8e815f9e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:21.550 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.549+0000 7fe8ed600640 1 -- 172.21.15.204:0/1437080339 >> 172.21.15.204:0/1437080339 conn(0x7fe8e80d51e0 msgr2=0x7fe8e80ce2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:21.550 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.549+0000 7fe8ed600640 1 -- 172.21.15.204:0/1437080339 shutdown_connections 2024-05-11T12:10:21.550 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.549+0000 7fe8ed600640 1 -- 172.21.15.204:0/1437080339 wait complete. 2024-05-11T12:10:21.565 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6 2 2024-05-11T12:10:21.731 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.731+0000 7fc8fc800640 1 Processor -- start 2024-05-11T12:10:21.731 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.731+0000 7fc8fc800640 1 -- start start 2024-05-11T12:10:21.732 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.732+0000 7fc8fc800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc8f415f910 0x7fc8f406c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:21.733 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.732+0000 7fc8fc800640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8f406dcc0 con 0x7fc8f40d40a0 2024-05-11T12:10:21.733 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.732+0000 7fc8fc800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8f406d140 con 0x7fc8f406cc30 2024-05-11T12:10:21.733 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.732+0000 7fc8fc800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc8f4156820 con 0x7fc8f415f910 2024-05-11T12:10:21.733 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.733+0000 7fc8f9e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc8f415f910 0x7fc8f406c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:21.733 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.733+0000 7fc8f9e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc8f415f910 0x7fc8f406c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:42988/0 (socket says 172.21.15.204:42988) 2024-05-11T12:10:21.733 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.733+0000 7fc8f9400640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fc8f406cc30 0x7fc8f415fcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:60368/0 (socket says 172.21.15.204:60368) 2024-05-11T12:10:21.733 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.733+0000 7fc8f9e00640 1 -- 172.21.15.204:0/1699038540 learned_addr learned my addr 172.21.15.204:0/1699038540 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-11T12:10:21.734 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.733+0000 7fc8f8a00640 1 -- 172.21.15.204:0/1699038540 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3144680938 0 0) 0x7fc8f406d140 con 0x7fc8f406cc30 2024-05-11T12:10:21.734 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.733+0000 7fc8f8a00640 1 -- 172.21.15.204:0/1699038540 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc8dc003520 con 0x7fc8f406cc30 2024-05-11T12:10:21.734 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.733+0000 7fc8f9e00640 1 -- 172.21.15.204:0/1699038540 >> v1:172.21.15.204:6789/0 conn(0x7fc8f406cc30 legacy=0x7fc8f415fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:21.734 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.733+0000 7fc8f9e00640 1 -- 172.21.15.204:0/1699038540 >> v1:172.21.15.167:6789/0 conn(0x7fc8f40d40a0 legacy=0x7fc8f4168ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:21.734 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.733+0000 7fc8f9e00640 1 -- 172.21.15.204:0/1699038540 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8f406d420 con 0x7fc8f415f910 2024-05-11T12:10:21.735 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.734+0000 7fc8f9e00640 1 --2- 172.21.15.204:0/1699038540 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc8f415f910 0x7fc8f406c6f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc8e80098a0 tx=0x7fc8e802a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54d791f2d60270d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:21.735 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.734+0000 7fc8f8a00640 1 -- 172.21.15.204:0/1699038540 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e8037070 con 0x7fc8f415f910 2024-05-11T12:10:21.735 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.735+0000 7fc8f8a00640 1 -- 172.21.15.204:0/1699038540 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8e8032440 con 0x7fc8f415f910 2024-05-11T12:10:21.735 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.735+0000 7fc8f8a00640 1 -- 172.21.15.204:0/1699038540 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e80327f0 con 0x7fc8f415f910 2024-05-11T12:10:21.736 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.735+0000 7fc8fc800640 1 -- 172.21.15.204:0/1699038540 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc8f415f910 msgr2=0x7fc8f406c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:21.736 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.735+0000 7fc8fc800640 1 --2- 172.21.15.204:0/1699038540 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc8f415f910 0x7fc8f406c6f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc8e80098a0 tx=0x7fc8e802a250 comp rx=0 tx=0).stop 2024-05-11T12:10:21.736 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.735+0000 7fc8fc800640 1 -- 172.21.15.204:0/1699038540 shutdown_connections 2024-05-11T12:10:21.736 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.735+0000 7fc8fc800640 1 --2- 172.21.15.204:0/1699038540 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc8f415f910 0x7fc8f406c6f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:21.736 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.735+0000 7fc8fc800640 1 -- 172.21.15.204:0/1699038540 >> 172.21.15.204:0/1699038540 conn(0x7fc8f40d51e0 msgr2=0x7fc8f40d55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:21.736 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.735+0000 7fc8fc800640 1 -- 172.21.15.204:0/1699038540 shutdown_connections 2024-05-11T12:10:21.736 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.735+0000 7fc8fc800640 1 -- 172.21.15.204:0/1699038540 wait complete. 2024-05-11T12:10:21.737 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.737+0000 7fc8fc800640 1 Processor -- start 2024-05-11T12:10:21.737 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.737+0000 7fc8fc800640 1 -- start start 2024-05-11T12:10:21.740 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.738+0000 7fc8fc800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8f406cc30 0x7fc8f41521c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:21.740 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.738+0000 7fc8fc800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8f40d40a0 0x7fc8f40ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:21.740 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.738+0000 7fc8fc800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc8f415f910 0x7fc8f40cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:21.740 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.738+0000 7fc8fc800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f40d0060 con 0x7fc8f40d40a0 2024-05-11T12:10:21.740 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.738+0000 7fc8fc800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f406d9e0 con 0x7fc8f406cc30 2024-05-11T12:10:21.740 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.738+0000 7fc8fc800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc8f40d01e0 con 0x7fc8f415f910 2024-05-11T12:10:21.740 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.738+0000 7fc8f9e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8f406cc30 0x7fc8f41521c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:21.740 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.738+0000 7fc8fa800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc8f415f910 0x7fc8f40cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:21.741 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.738+0000 7fc8f9e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8f406cc30 0x7fc8f41521c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:39614/0 (socket says 172.21.15.204:39614) 2024-05-11T12:10:21.741 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.738+0000 7fc8f9e00640 1 -- 172.21.15.204:0/1943143967 learned_addr learned my addr 172.21.15.204:0/1943143967 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-11T12:10:21.741 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.739+0000 7fc8f9400640 1 --2- 172.21.15.204:0/1943143967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8f40d40a0 0x7fc8f40ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:21.741 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.739+0000 7fc8fa800640 1 -- 172.21.15.204:0/1943143967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8f406cc30 msgr2=0x7fc8f41521c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:21.741 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.739+0000 7fc8fa800640 1 --2- 172.21.15.204:0/1943143967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8f406cc30 0x7fc8f41521c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:21.741 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.739+0000 7fc8fa800640 1 -- 172.21.15.204:0/1943143967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8f40d40a0 msgr2=0x7fc8f40ca0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:21.741 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.739+0000 7fc8fa800640 1 --2- 172.21.15.204:0/1943143967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8f40d40a0 0x7fc8f40ca0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:21.741 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.739+0000 7fc8fa800640 1 -- 172.21.15.204:0/1943143967 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8f41f4d90 con 0x7fc8f415f910 2024-05-11T12:10:21.741 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.739+0000 7fc8f9e00640 1 --2- 172.21.15.204:0/1943143967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8f406cc30 0x7fc8f41521c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:10:21.741 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.739+0000 7fc8f9400640 1 --2- 172.21.15.204:0/1943143967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8f40d40a0 0x7fc8f40ca0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:10:21.741 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.739+0000 7fc8fa800640 1 --2- 172.21.15.204:0/1943143967 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc8f415f910 0x7fc8f40cecb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc8f000c760 tx=0x7fc8f000cc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:21.741 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.739+0000 7fc8ee000640 1 -- 172.21.15.204:0/1943143967 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f0003b70 con 0x7fc8f415f910 2024-05-11T12:10:21.741 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.739+0000 7fc8ee000640 1 -- 172.21.15.204:0/1943143967 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8f0003d10 con 0x7fc8f415f910 2024-05-11T12:10:21.741 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.739+0000 7fc8ee000640 1 -- 172.21.15.204:0/1943143967 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f000aa70 con 0x7fc8f415f910 2024-05-11T12:10:21.742 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.739+0000 7fc8fc800640 1 -- 172.21.15.204:0/1943143967 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8f41433f0 con 0x7fc8f415f910 2024-05-11T12:10:21.742 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.740+0000 7fc8fc800640 1 -- 172.21.15.204:0/1943143967 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8f41439b0 con 0x7fc8f415f910 2024-05-11T12:10:21.742 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.740+0000 7fc8ee000640 1 -- 172.21.15.204:0/1943143967 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc8f000ac10 con 0x7fc8f415f910 2024-05-11T12:10:21.742 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.740+0000 7fc8ee000640 1 -- 172.21.15.204:0/1943143967 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7fc8f00054f0 con 0x7fc8f415f910 2024-05-11T12:10:21.742 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.741+0000 7fc8fc800640 1 -- 172.21.15.204:0/1943143967 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8f4069500 con 0x7fc8f415f910 2024-05-11T12:10:21.743 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.743+0000 7fc8ee000640 1 -- 172.21.15.204:0/1943143967 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fc8f00050d0 con 0x7fc8f415f910 2024-05-11T12:10:21.787 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T12:10:21.787+0000 7fa9e4ddc040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-11T12:10:21.791 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T12:10:21.790+0000 7f98b0141040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-11T12:10:21.820 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.820+0000 7fc8fc800640 1 -- 172.21.15.204:0/1943143967 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6", "id": 2} v 0) -- 0x7fc8f4155e60 con 0x7fc8f415f910 2024-05-11T12:10:21.828 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.828+0000 7fc8ee000640 1 -- 172.21.15.204:0/1943143967 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7fc8f0025480 con 0x7fc8f415f910 2024-05-11T12:10:21.828 INFO:teuthology.orchestra.run.smithi204.stdout:2 2024-05-11T12:10:21.830 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.829+0000 7fc8fc800640 1 -- 172.21.15.204:0/1943143967 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc8f415f910 msgr2=0x7fc8f40cecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:21.830 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.829+0000 7fc8fc800640 1 --2- 172.21.15.204:0/1943143967 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc8f415f910 0x7fc8f40cecb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc8f000c760 tx=0x7fc8f000cc20 comp rx=0 tx=0).stop 2024-05-11T12:10:21.830 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.829+0000 7fc8fc800640 1 -- 172.21.15.204:0/1943143967 shutdown_connections 2024-05-11T12:10:21.830 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.829+0000 7fc8fc800640 1 --2- 172.21.15.204:0/1943143967 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc8f415f910 0x7fc8f40cecb0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:21.830 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.829+0000 7fc8fc800640 1 --2- 172.21.15.204:0/1943143967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8f40d40a0 0x7fc8f40ca0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:21.830 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.829+0000 7fc8fc800640 1 --2- 172.21.15.204:0/1943143967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8f406cc30 0x7fc8f41521c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:21.830 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.829+0000 7fc8fc800640 1 -- 172.21.15.204:0/1943143967 >> 172.21.15.204:0/1943143967 conn(0x7fc8f40d51e0 msgr2=0x7fc8f40d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:21.830 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.830+0000 7fc8fc800640 1 -- 172.21.15.204:0/1943143967 shutdown_connections 2024-05-11T12:10:21.830 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:21.830+0000 7fc8fc800640 1 -- 172.21.15.204:0/1943143967 wait complete. 2024-05-11T12:10:21.845 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new 2e95ae7d-d784-47ab-a14a-e74331ca8821 3 2024-05-11T12:10:21.896 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T12:10:21.896+0000 7f98b0141040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-11T12:10:21.899 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T12:10:21.899+0000 7fa9e4ddc040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-11T12:10:21.946 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T12:10:21.946+0000 7f98b0141040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-11T12:10:21.953 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T12:10:21.953+0000 7fa9e4ddc040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-11T12:10:22.023 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.023+0000 7f524be00640 1 Processor -- start 2024-05-11T12:10:22.024 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.023+0000 7f524be00640 1 -- start start 2024-05-11T12:10:22.025 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.025+0000 7f524be00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f524c15f220 0x7f524c0c7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:22.025 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.025+0000 7f524be00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f524c15f600 con 0x7f524c074730 2024-05-11T12:10:22.026 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.025+0000 7f524be00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f524c070700 con 0x7f524c06ff20 2024-05-11T12:10:22.026 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.025+0000 7f524be00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f524c0cf930 con 0x7f524c15f220 2024-05-11T12:10:22.026 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.025+0000 7f524aa00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f524c15f220 0x7f524c0c7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:22.026 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.025+0000 7f524aa00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f524c15f220 0x7f524c0c7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:43008/0 (socket says 172.21.15.204:43008) 2024-05-11T12:10:22.026 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.025+0000 7f524aa00640 1 -- 172.21.15.204:0/3418410921 learned_addr learned my addr 172.21.15.204:0/3418410921 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-11T12:10:22.027 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.026+0000 7f524aa00640 1 -- 172.21.15.204:0/3418410921 >> v1:172.21.15.204:6789/0 conn(0x7f524c06ff20 legacy=0x7f524c074020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:22.027 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.026+0000 7f524aa00640 1 -- 172.21.15.204:0/3418410921 >> v1:172.21.15.167:6789/0 conn(0x7f524c074730 legacy=0x7f524c0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:22.027 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.026+0000 7f524aa00640 1 -- 172.21.15.204:0/3418410921 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f524c0cfc10 con 0x7f524c15f220 2024-05-11T12:10:22.029 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.027+0000 7f524aa00640 1 --2- 172.21.15.204:0/3418410921 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f524c15f220 0x7f524c0c7790 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f523c0098a0 tx=0x7f523c02a2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a1145feffd06729 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:22.029 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.027+0000 7f5249600640 1 -- 172.21.15.204:0/3418410921 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c037070 con 0x7f524c15f220 2024-05-11T12:10:22.030 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.027+0000 7f5249600640 1 -- 172.21.15.204:0/3418410921 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f523c032640 con 0x7f524c15f220 2024-05-11T12:10:22.030 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.027+0000 7f5249600640 1 -- 172.21.15.204:0/3418410921 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c032a10 con 0x7f524c15f220 2024-05-11T12:10:22.030 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.028+0000 7f524be00640 1 -- 172.21.15.204:0/3418410921 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f524c15f220 msgr2=0x7f524c0c7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:22.030 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.028+0000 7f524be00640 1 --2- 172.21.15.204:0/3418410921 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f524c15f220 0x7f524c0c7790 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f523c0098a0 tx=0x7f523c02a2e0 comp rx=0 tx=0).stop 2024-05-11T12:10:22.030 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.028+0000 7f524be00640 1 -- 172.21.15.204:0/3418410921 shutdown_connections 2024-05-11T12:10:22.030 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.028+0000 7f524be00640 1 --2- 172.21.15.204:0/3418410921 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f524c15f220 0x7f524c0c7790 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:22.030 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.028+0000 7f524be00640 1 -- 172.21.15.204:0/3418410921 >> 172.21.15.204:0/3418410921 conn(0x7f524c0ce740 msgr2=0x7f524c0ceb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:22.031 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.028+0000 7f524be00640 1 -- 172.21.15.204:0/3418410921 shutdown_connections 2024-05-11T12:10:22.031 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.028+0000 7f524be00640 1 -- 172.21.15.204:0/3418410921 wait complete. 2024-05-11T12:10:22.033 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.030+0000 7f524be00640 1 Processor -- start 2024-05-11T12:10:22.033 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.030+0000 7f524be00640 1 -- start start 2024-05-11T12:10:22.033 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.031+0000 7f524be00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f524c06ff20 0x7f524c0b7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:22.033 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.031+0000 7f524be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f524c074730 0x7f524c0cb210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:22.033 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.031+0000 7f524be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f524c15f220 0x7f524c0cb750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:22.034 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.031+0000 7f524be00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f524c070380 con 0x7f524c15f220 2024-05-11T12:10:22.034 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.031+0000 7f524be00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f524c0d01d0 con 0x7f524c074730 2024-05-11T12:10:22.034 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.031+0000 7f524be00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f524c0d0e00 con 0x7f524c06ff20 2024-05-11T12:10:22.034 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.031+0000 7f524aa00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f524c06ff20 0x7f524c0b7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:22.034 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.031+0000 7f524aa00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f524c06ff20 0x7f524c0b7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:43010/0 (socket says 172.21.15.204:43010) 2024-05-11T12:10:22.034 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.031+0000 7f524aa00640 1 -- 172.21.15.204:0/49704470 learned_addr learned my addr 172.21.15.204:0/49704470 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-11T12:10:22.034 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.031+0000 7f524a000640 1 --2- 172.21.15.204:0/49704470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f524c074730 0x7f524c0cb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:22.034 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.031+0000 7f524b400640 1 --2- 172.21.15.204:0/49704470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f524c15f220 0x7f524c0cb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:22.034 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.031+0000 7f524aa00640 1 -- 172.21.15.204:0/49704470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f524c074730 msgr2=0x7f524c0cb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:22.034 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.031+0000 7f524aa00640 1 --2- 172.21.15.204:0/49704470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f524c074730 0x7f524c0cb210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:22.034 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.031+0000 7f524aa00640 1 -- 172.21.15.204:0/49704470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f524c15f220 msgr2=0x7f524c0cb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:22.034 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.031+0000 7f524aa00640 1 --2- 172.21.15.204:0/49704470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f524c15f220 0x7f524c0cb750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:22.034 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.031+0000 7f524aa00640 1 -- 172.21.15.204:0/49704470 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f524c0c9b50 con 0x7f524c06ff20 2024-05-11T12:10:22.035 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.031+0000 7f524b400640 1 --2- 172.21.15.204:0/49704470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f524c15f220 0x7f524c0cb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:10:22.035 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.032+0000 7f524aa00640 1 --2- 172.21.15.204:0/49704470 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f524c06ff20 0x7f524c0b7490 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f523c0098a0 tx=0x7f523c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:22.035 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.032+0000 7f5242a00640 1 -- 172.21.15.204:0/49704470 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c037040 con 0x7f524c06ff20 2024-05-11T12:10:22.035 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.032+0000 7f5242a00640 1 -- 172.21.15.204:0/49704470 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f523c030770 con 0x7f524c06ff20 2024-05-11T12:10:22.035 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.032+0000 7f524be00640 1 -- 172.21.15.204:0/49704470 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f524c0c9e30 con 0x7f524c06ff20 2024-05-11T12:10:22.035 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.032+0000 7f5242a00640 1 -- 172.21.15.204:0/49704470 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c030b40 con 0x7f524c06ff20 2024-05-11T12:10:22.035 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.032+0000 7f524be00640 1 -- 172.21.15.204:0/49704470 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f524c0ca3f0 con 0x7f524c06ff20 2024-05-11T12:10:22.035 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.032+0000 7f5242a00640 1 -- 172.21.15.204:0/49704470 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f523c044030 con 0x7f524c06ff20 2024-05-11T12:10:22.035 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.032+0000 7f5242a00640 1 -- 172.21.15.204:0/49704470 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f523c030d80 con 0x7f524c06ff20 2024-05-11T12:10:22.035 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.033+0000 7f524be00640 1 -- 172.21.15.204:0/49704470 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f524c069bc0 con 0x7f524c06ff20 2024-05-11T12:10:22.036 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.035+0000 7f5242a00640 1 -- 172.21.15.204:0/49704470 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f523c03b950 con 0x7f524c06ff20 2024-05-11T12:10:22.114 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.114+0000 7f524be00640 1 -- 172.21.15.204:0/49704470 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd new", "uuid": "2e95ae7d-d784-47ab-a14a-e74331ca8821", "id": 3} v 0) -- 0x7f524c0c7ba0 con 0x7f524c06ff20 2024-05-11T12:10:22.120 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.120+0000 7f5242a00640 1 -- 172.21.15.204:0/49704470 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2e95ae7d-d784-47ab-a14a-e74331ca8821", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f523c03bbd0 con 0x7f524c06ff20 2024-05-11T12:10:22.120 INFO:teuthology.orchestra.run.smithi204.stdout:3 2024-05-11T12:10:22.122 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.122+0000 7f524be00640 1 -- 172.21.15.204:0/49704470 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f524c06ff20 msgr2=0x7f524c0b7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:22.122 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.122+0000 7f524be00640 1 --2- 172.21.15.204:0/49704470 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f524c06ff20 0x7f524c0b7490 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f523c0098a0 tx=0x7f523c0046c0 comp rx=0 tx=0).stop 2024-05-11T12:10:22.123 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.122+0000 7f524be00640 1 -- 172.21.15.204:0/49704470 shutdown_connections 2024-05-11T12:10:22.123 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.122+0000 7f524be00640 1 --2- 172.21.15.204:0/49704470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f524c15f220 0x7f524c0cb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:22.123 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.122+0000 7f524be00640 1 --2- 172.21.15.204:0/49704470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f524c074730 0x7f524c0cb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:22.123 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.122+0000 7f524be00640 1 --2- 172.21.15.204:0/49704470 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f524c06ff20 0x7f524c0b7490 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:22.123 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.122+0000 7f524be00640 1 -- 172.21.15.204:0/49704470 >> 172.21.15.204:0/49704470 conn(0x7f524c0ce740 msgr2=0x7f524c0b0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:22.123 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.122+0000 7f524be00640 1 -- 172.21.15.204:0/49704470 shutdown_connections 2024-05-11T12:10:22.123 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.122+0000 7f524be00640 1 -- 172.21.15.204:0/49704470 wait complete. 2024-05-11T12:10:22.139 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new 3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb 4 2024-05-11T12:10:22.141 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T12:10:22.141+0000 7f98b0141040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-11T12:10:22.168 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T12:10:22.168+0000 7fa9e4ddc040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-11T12:10:22.196 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T12:10:22.196+0000 7f98b0141040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-11T12:10:22.236 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T12:10:22.236+0000 7fa9e4ddc040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-11T12:10:22.259 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T12:10:22.259+0000 7f98b0141040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-11T12:10:22.301 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T12:10:22.301+0000 7fa9e4ddc040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-11T12:10:22.308 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.308+0000 7f628e400640 1 Processor -- start 2024-05-11T12:10:22.308 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.308+0000 7f628e400640 1 -- start start 2024-05-11T12:10:22.310 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.310+0000 7f628e400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f628815ef60 0x7f62880d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:22.310 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.310+0000 7f628e400640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62880b78b0 con 0x7f62880b23c0 2024-05-11T12:10:22.310 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.310+0000 7f628e400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62880d2570 con 0x7f628815f340 2024-05-11T12:10:22.311 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.310+0000 7f628e400640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f62880d2850 con 0x7f628815ef60 2024-05-11T12:10:22.311 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.310+0000 7f6287e00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f628815f340 0x7f62880b71a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:60384/0 (socket says 172.21.15.204:60384) 2024-05-11T12:10:22.311 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.310+0000 7f6287e00640 1 -- 172.21.15.204:0/1499661717 learned_addr learned my addr 172.21.15.204:0/1499661717 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-05-11T12:10:22.311 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.310+0000 7f628d000640 1 --2- 172.21.15.204:0/1499661717 >> v2:172.21.15.167:3300/0 conn(0x7f628815ef60 0x7f62880d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:22.311 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.311+0000 7f6287400640 1 -- 172.21.15.204:0/1499661717 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2352021137 0 0) 0x7f62880d2570 con 0x7f628815f340 2024-05-11T12:10:22.312 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.311+0000 7f6287400640 1 -- 172.21.15.204:0/1499661717 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6270003520 con 0x7f628815f340 2024-05-11T12:10:22.312 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.311+0000 7f628d000640 1 -- 172.21.15.204:0/1499661717 >> v1:172.21.15.204:6789/0 conn(0x7f628815f340 legacy=0x7f62880b71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:22.312 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.311+0000 7f628d000640 1 -- 172.21.15.204:0/1499661717 >> v1:172.21.15.167:6789/0 conn(0x7f62880b23c0 legacy=0x7f62880cd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:22.312 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.311+0000 7f628d000640 1 -- 172.21.15.204:0/1499661717 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62880ceaf0 con 0x7f628815ef60 2024-05-11T12:10:22.312 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.312+0000 7f628d000640 1 --2- 172.21.15.204:0/1499661717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f628815ef60 0x7f62880d12b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f627c0097d0 tx=0x7f627c02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77c2205ff4b03ed6 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:22.313 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.312+0000 7f6287400640 1 -- 172.21.15.204:0/1499661717 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627c037070 con 0x7f628815ef60 2024-05-11T12:10:22.313 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.312+0000 7f6287400640 1 -- 172.21.15.204:0/1499661717 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f627c032440 con 0x7f628815ef60 2024-05-11T12:10:22.313 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.312+0000 7f6287400640 1 -- 172.21.15.204:0/1499661717 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627c0327f0 con 0x7f628815ef60 2024-05-11T12:10:22.314 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.313+0000 7f628e400640 1 -- 172.21.15.204:0/1499661717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f628815ef60 msgr2=0x7f62880d12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:22.314 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.313+0000 7f628e400640 1 --2- 172.21.15.204:0/1499661717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f628815ef60 0x7f62880d12b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f627c0097d0 tx=0x7f627c02a250 comp rx=0 tx=0).stop 2024-05-11T12:10:22.314 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.313+0000 7f628e400640 1 -- 172.21.15.204:0/1499661717 shutdown_connections 2024-05-11T12:10:22.314 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.313+0000 7f628e400640 1 --2- 172.21.15.204:0/1499661717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f628815ef60 0x7f62880d12b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:22.314 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.313+0000 7f628e400640 1 -- 172.21.15.204:0/1499661717 >> 172.21.15.204:0/1499661717 conn(0x7f6288074540 msgr2=0x7f62880b4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:22.314 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.313+0000 7f628e400640 1 -- 172.21.15.204:0/1499661717 shutdown_connections 2024-05-11T12:10:22.314 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.313+0000 7f628e400640 1 -- 172.21.15.204:0/1499661717 wait complete. 2024-05-11T12:10:22.315 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.315+0000 7f628e400640 1 Processor -- start 2024-05-11T12:10:22.315 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.315+0000 7f628e400640 1 -- start start 2024-05-11T12:10:22.317 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.316+0000 7f628e400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62880b23c0 0x7f628815d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:22.318 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.316+0000 7f628e400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f628815ef60 0x7f6288070c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:22.318 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.316+0000 7f628e400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f628806e970 0x7f62880cc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:22.318 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.316+0000 7f628e400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f62880d17f0 con 0x7f62880b23c0 2024-05-11T12:10:22.318 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.316+0000 7f628e400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f62880cf0b0 con 0x7f628806e970 2024-05-11T12:10:22.318 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.316+0000 7f628e400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f62880cda90 con 0x7f628815ef60 2024-05-11T12:10:22.318 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.316+0000 7f6287e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f628815ef60 0x7f6288070c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:22.318 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.316+0000 7f628da00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f628806e970 0x7f62880cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:22.318 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.316+0000 7f6287e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f628815ef60 0x7f6288070c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:43014/0 (socket says 172.21.15.204:43014) 2024-05-11T12:10:22.318 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.316+0000 7f6287e00640 1 -- 172.21.15.204:0/1219995089 learned_addr learned my addr 172.21.15.204:0/1219995089 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-11T12:10:22.318 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.316+0000 7f628d000640 1 --2- 172.21.15.204:0/1219995089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62880b23c0 0x7f628815d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:22.319 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.317+0000 7f6287e00640 1 -- 172.21.15.204:0/1219995089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f628806e970 msgr2=0x7f62880cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:22.319 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.317+0000 7f6287e00640 1 --2- 172.21.15.204:0/1219995089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f628806e970 0x7f62880cc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:22.319 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.317+0000 7f6287e00640 1 -- 172.21.15.204:0/1219995089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62880b23c0 msgr2=0x7f628815d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:22.319 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.317+0000 7f6287e00640 1 --2- 172.21.15.204:0/1219995089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62880b23c0 0x7f628815d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:22.319 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.317+0000 7f6287e00640 1 -- 172.21.15.204:0/1219995089 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62880715e0 con 0x7f628815ef60 2024-05-11T12:10:22.319 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.317+0000 7f6287e00640 1 --2- 172.21.15.204:0/1219995089 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f628815ef60 0x7f6288070c90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f627400c420 tx=0x7f627400c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:22.319 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.317+0000 7f6284c00640 1 -- 172.21.15.204:0/1219995089 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6274017070 con 0x7f628815ef60 2024-05-11T12:10:22.319 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.317+0000 7f6284c00640 1 -- 172.21.15.204:0/1219995089 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6274005ce0 con 0x7f628815ef60 2024-05-11T12:10:22.319 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.317+0000 7f628e400640 1 -- 172.21.15.204:0/1219995089 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62880718c0 con 0x7f628815ef60 2024-05-11T12:10:22.319 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.317+0000 7f6284c00640 1 -- 172.21.15.204:0/1219995089 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62740072d0 con 0x7f628815ef60 2024-05-11T12:10:22.319 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.317+0000 7f628e400640 1 -- 172.21.15.204:0/1219995089 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f62880c8330 con 0x7f628815ef60 2024-05-11T12:10:22.319 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.318+0000 7f6284c00640 1 -- 172.21.15.204:0/1219995089 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f627401a070 con 0x7f628815ef60 2024-05-11T12:10:22.319 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.318+0000 7f6284c00640 1 -- 172.21.15.204:0/1219995089 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f627401cbc0 con 0x7f628815ef60 2024-05-11T12:10:22.320 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.319+0000 7f628e400640 1 -- 172.21.15.204:0/1219995089 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6288071de0 con 0x7f628815ef60 2024-05-11T12:10:22.321 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.321+0000 7f6284c00640 1 -- 172.21.15.204:0/1219995089 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f6274007470 con 0x7f628815ef60 2024-05-11T12:10:22.389 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T12:10:22.389+0000 7f98b0141040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-11T12:10:22.398 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.398+0000 7f628e400640 1 -- 172.21.15.204:0/1219995089 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd new", "uuid": "3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb", "id": 4} v 0) -- 0x7f6288069bc0 con 0x7f628815ef60 2024-05-11T12:10:22.404 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.404+0000 7f6284c00640 1 -- 172.21.15.204:0/1219995089 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f6274007650 con 0x7f628815ef60 2024-05-11T12:10:22.404 INFO:teuthology.orchestra.run.smithi204.stdout:4 2024-05-11T12:10:22.406 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.406+0000 7f628e400640 1 -- 172.21.15.204:0/1219995089 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f628815ef60 msgr2=0x7f6288070c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:22.406 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.406+0000 7f628e400640 1 --2- 172.21.15.204:0/1219995089 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f628815ef60 0x7f6288070c90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f627400c420 tx=0x7f627400c8e0 comp rx=0 tx=0).stop 2024-05-11T12:10:22.406 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.406+0000 7f628e400640 1 -- 172.21.15.204:0/1219995089 shutdown_connections 2024-05-11T12:10:22.406 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.406+0000 7f628e400640 1 --2- 172.21.15.204:0/1219995089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f628806e970 0x7f62880cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:22.407 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.406+0000 7f628e400640 1 --2- 172.21.15.204:0/1219995089 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f628815ef60 0x7f6288070c90 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:22.407 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.406+0000 7f628e400640 1 --2- 172.21.15.204:0/1219995089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62880b23c0 0x7f628815d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:22.407 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.406+0000 7f628e400640 1 -- 172.21.15.204:0/1219995089 >> 172.21.15.204:0/1219995089 conn(0x7f6288074540 msgr2=0x7f62880cd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:22.407 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.406+0000 7f628e400640 1 -- 172.21.15.204:0/1219995089 shutdown_connections 2024-05-11T12:10:22.407 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.406+0000 7f628e400640 1 -- 172.21.15.204:0/1219995089 wait complete. 2024-05-11T12:10:22.423 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new 12519311-12a1-4b83-b835-9af10f397c8e 5 2024-05-11T12:10:22.439 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T12:10:22.439+0000 7fa9e4ddc040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-11T12:10:22.618 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.617+0000 7f73a5200640 1 Processor -- start 2024-05-11T12:10:22.618 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.618+0000 7f73a5200640 1 -- start start 2024-05-11T12:10:22.620 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.619+0000 7f73a5200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f73a015ac80 0x7f73a0074260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:22.620 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.619+0000 7f73a5200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f73a0161380 0x7f73a0072950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:22.620 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.619+0000 7f73a5200640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73a00b65e0 con 0x7f73a00b3d50 2024-05-11T12:10:22.620 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.619+0000 7f73a5200640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f73a006e3c0 con 0x7f73a015ac80 2024-05-11T12:10:22.620 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.619+0000 7f73a5200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f73a015b060 con 0x7f73a0161380 2024-05-11T12:10:22.620 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.619+0000 7f739e800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f73a0161380 0x7f73a0072950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:22.621 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.620+0000 7f739e800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f73a0161380 0x7f73a0072950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:39636/0 (socket says 172.21.15.204:39636) 2024-05-11T12:10:22.621 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.620+0000 7f739e800640 1 -- 172.21.15.204:0/1458965174 learned_addr learned my addr 172.21.15.204:0/1458965174 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-11T12:10:22.621 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.620+0000 7f739de00640 1 --2- 172.21.15.204:0/1458965174 >> v2:172.21.15.167:3300/0 conn(0x7f73a015ac80 0x7f73a0074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:22.621 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.620+0000 7f739e800640 1 -- 172.21.15.204:0/1458965174 >> v2:172.21.15.167:3300/0 conn(0x7f73a015ac80 msgr2=0x7f73a0074260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:22.621 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.620+0000 7f739e800640 1 --2- 172.21.15.204:0/1458965174 >> v2:172.21.15.167:3300/0 conn(0x7f73a015ac80 0x7f73a0074260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:22.621 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.620+0000 7f739e800640 1 -- 172.21.15.204:0/1458965174 >> v1:172.21.15.167:6789/0 conn(0x7f73a00b3d50 legacy=0x7f73a0160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:22.621 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.620+0000 7f739e800640 1 -- 172.21.15.204:0/1458965174 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a00b68c0 con 0x7f73a0161380 2024-05-11T12:10:22.621 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.620+0000 7f739de00640 1 --2- 172.21.15.204:0/1458965174 >> v2:172.21.15.167:3300/0 conn(0x7f73a015ac80 0x7f73a0074260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:10:22.622 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.621+0000 7f739e800640 1 --2- 172.21.15.204:0/1458965174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73a0161380 0x7f73a0072950 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7398009e50 tx=0x7f739802a6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bc4650706792dc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:22.622 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.621+0000 7f739ca00640 1 -- 172.21.15.204:0/1458965174 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7398036070 con 0x7f73a0161380 2024-05-11T12:10:22.622 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.621+0000 7f739ca00640 1 -- 172.21.15.204:0/1458965174 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7398031600 con 0x7f73a0161380 2024-05-11T12:10:22.622 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.621+0000 7f739ca00640 1 -- 172.21.15.204:0/1458965174 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73980319d0 con 0x7f73a0161380 2024-05-11T12:10:22.623 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.622+0000 7f73a5200640 1 -- 172.21.15.204:0/1458965174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73a0161380 msgr2=0x7f73a0072950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:22.623 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.622+0000 7f73a5200640 1 --2- 172.21.15.204:0/1458965174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73a0161380 0x7f73a0072950 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7398009e50 tx=0x7f739802a6c0 comp rx=0 tx=0).stop 2024-05-11T12:10:22.623 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.622+0000 7f73a5200640 1 -- 172.21.15.204:0/1458965174 shutdown_connections 2024-05-11T12:10:22.623 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.622+0000 7f73a5200640 1 --2- 172.21.15.204:0/1458965174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73a0161380 0x7f73a0072950 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:22.623 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.622+0000 7f73a5200640 1 --2- 172.21.15.204:0/1458965174 >> v2:172.21.15.167:3300/0 conn(0x7f73a015ac80 0x7f73a0074260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:22.623 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.622+0000 7f73a5200640 1 -- 172.21.15.204:0/1458965174 >> 172.21.15.204:0/1458965174 conn(0x7f73a006c450 msgr2=0x7f73a006c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:22.623 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.622+0000 7f73a5200640 1 -- 172.21.15.204:0/1458965174 shutdown_connections 2024-05-11T12:10:22.624 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.622+0000 7f73a5200640 1 -- 172.21.15.204:0/1458965174 wait complete. 2024-05-11T12:10:22.625 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.625+0000 7f73a5200640 1 Processor -- start 2024-05-11T12:10:22.625 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.625+0000 7f73a5200640 1 -- start start 2024-05-11T12:10:22.628 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.627+0000 7f73a5200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73a00b3d50 0x7f73a0143870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:22.628 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.627+0000 7f73a5200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73a015ac80 0x7f73a013f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:22.629 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.627+0000 7f73a5200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f73a0161380 0x7f73a015f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:22.629 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.627+0000 7f73a5200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f73a00b44f0 con 0x7f73a015ac80 2024-05-11T12:10:22.629 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.627+0000 7f73a5200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f73a00b41e0 con 0x7f73a00b3d50 2024-05-11T12:10:22.629 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.627+0000 7f73a5200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f73a0072e90 con 0x7f73a0161380 2024-05-11T12:10:22.629 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.627+0000 7f739de00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73a00b3d50 0x7f73a0143870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:22.629 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.627+0000 7f739de00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73a00b3d50 0x7f73a0143870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:39640/0 (socket says 172.21.15.204:39640) 2024-05-11T12:10:22.629 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.627+0000 7f739de00640 1 -- 172.21.15.204:0/1226545627 learned_addr learned my addr 172.21.15.204:0/1226545627 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-11T12:10:22.629 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.627+0000 7f739e800640 1 --2- 172.21.15.204:0/1226545627 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f73a0161380 0x7f73a015f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:22.629 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.627+0000 7f739d400640 1 --2- 172.21.15.204:0/1226545627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73a015ac80 0x7f73a013f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:22.629 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.627+0000 7f739de00640 1 -- 172.21.15.204:0/1226545627 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f73a0161380 msgr2=0x7f73a015f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:22.629 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.627+0000 7f739de00640 1 --2- 172.21.15.204:0/1226545627 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f73a0161380 0x7f73a015f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:22.629 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.627+0000 7f739de00640 1 -- 172.21.15.204:0/1226545627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73a015ac80 msgr2=0x7f73a013f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:22.629 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.627+0000 7f739de00640 1 --2- 172.21.15.204:0/1226545627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73a015ac80 0x7f73a013f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:22.629 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.627+0000 7f739de00640 1 -- 172.21.15.204:0/1226545627 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a01f8e30 con 0x7f73a00b3d50 2024-05-11T12:10:22.629 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.627+0000 7f739d400640 1 --2- 172.21.15.204:0/1226545627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73a015ac80 0x7f73a013f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:10:22.630 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.628+0000 7f739de00640 1 --2- 172.21.15.204:0/1226545627 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73a00b3d50 0x7f73a0143870 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f739000e7e0 tx=0x7f739000eca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:22.630 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.628+0000 7f7396000640 1 -- 172.21.15.204:0/1226545627 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739000f040 con 0x7f73a00b3d50 2024-05-11T12:10:22.630 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.628+0000 7f7396000640 1 -- 172.21.15.204:0/1226545627 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7390004030 con 0x7f73a00b3d50 2024-05-11T12:10:22.630 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.628+0000 7f73a5200640 1 -- 172.21.15.204:0/1226545627 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73a01f9110 con 0x7f73a00b3d50 2024-05-11T12:10:22.630 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.628+0000 7f7396000640 1 -- 172.21.15.204:0/1226545627 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7390004400 con 0x7f73a00b3d50 2024-05-11T12:10:22.630 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.628+0000 7f73a5200640 1 -- 172.21.15.204:0/1226545627 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73a01f96d0 con 0x7f73a00b3d50 2024-05-11T12:10:22.630 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.628+0000 7f7396000640 1 -- 172.21.15.204:0/1226545627 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f739000ca60 con 0x7f73a00b3d50 2024-05-11T12:10:22.630 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.628+0000 7f7396000640 1 -- 172.21.15.204:0/1226545627 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f739000cd50 con 0x7f73a00b3d50 2024-05-11T12:10:22.630 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.629+0000 7f73a5200640 1 -- 172.21.15.204:0/1226545627 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73a0069520 con 0x7f73a00b3d50 2024-05-11T12:10:22.632 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.632+0000 7f7396000640 1 -- 172.21.15.204:0/1226545627 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f7390017020 con 0x7f73a00b3d50 2024-05-11T12:10:22.707 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.706+0000 7f73a5200640 1 -- 172.21.15.204:0/1226545627 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd new", "uuid": "12519311-12a1-4b83-b835-9af10f397c8e", "id": 5} v 0) -- 0x7f73a0069cd0 con 0x7f73a00b3d50 2024-05-11T12:10:22.716 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.715+0000 7f7396000640 1 -- 172.21.15.204:0/1226545627 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "12519311-12a1-4b83-b835-9af10f397c8e", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f7390003670 con 0x7f73a00b3d50 2024-05-11T12:10:22.716 INFO:teuthology.orchestra.run.smithi204.stdout:5 2024-05-11T12:10:22.717 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.717+0000 7f73a5200640 1 -- 172.21.15.204:0/1226545627 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73a00b3d50 msgr2=0x7f73a0143870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:22.717 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.717+0000 7f73a5200640 1 --2- 172.21.15.204:0/1226545627 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73a00b3d50 0x7f73a0143870 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f739000e7e0 tx=0x7f739000eca0 comp rx=0 tx=0).stop 2024-05-11T12:10:22.717 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.717+0000 7f73a5200640 1 -- 172.21.15.204:0/1226545627 shutdown_connections 2024-05-11T12:10:22.717 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.717+0000 7f73a5200640 1 --2- 172.21.15.204:0/1226545627 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f73a0161380 0x7f73a015f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:22.717 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.717+0000 7f73a5200640 1 --2- 172.21.15.204:0/1226545627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73a015ac80 0x7f73a013f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:22.718 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.717+0000 7f73a5200640 1 --2- 172.21.15.204:0/1226545627 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73a00b3d50 0x7f73a0143870 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:22.718 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.717+0000 7f73a5200640 1 -- 172.21.15.204:0/1226545627 >> 172.21.15.204:0/1226545627 conn(0x7f73a006c450 msgr2=0x7f73a0073270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:22.718 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.717+0000 7f73a5200640 1 -- 172.21.15.204:0/1226545627 shutdown_connections 2024-05-11T12:10:22.718 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.717+0000 7f73a5200640 1 -- 172.21.15.204:0/1226545627 wait complete. 2024-05-11T12:10:22.734 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new 1ed30e71-408f-4586-869b-ce90d9bc76c4 6 2024-05-11T12:10:22.846 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T12:10:22.846+0000 7f98b0141040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-11T12:10:22.904 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.904+0000 7fa772400640 1 Processor -- start 2024-05-11T12:10:22.904 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.904+0000 7fa772400640 1 -- start start 2024-05-11T12:10:22.906 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.905+0000 7fa772400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa76c15f7a0 0x7fa76c15fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:22.907 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.905+0000 7fa772400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa76c1600c0 0x7fa76c06cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:22.907 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.906+0000 7fa772400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa76c06d980 con 0x7fa76c06d190 2024-05-11T12:10:22.907 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.906+0000 7fa772400640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fa76c1608f0 con 0x7fa76c1600c0 2024-05-11T12:10:22.907 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.906+0000 7fa772400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fa76c06df40 con 0x7fa76c15f7a0 2024-05-11T12:10:22.907 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.906+0000 7fa771000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa76c15f7a0 0x7fa76c15fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:22.907 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.906+0000 7fa771a00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fa76c06d190 0x7fa76c0d3970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:37106/0 (socket says 172.21.15.204:37106) 2024-05-11T12:10:22.907 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.906+0000 7fa771a00640 1 -- 172.21.15.204:0/2194463760 learned_addr learned my addr 172.21.15.204:0/2194463760 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-05-11T12:10:22.907 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.906+0000 7fa76be00640 1 --2- 172.21.15.204:0/2194463760 >> v2:172.21.15.167:3300/0 conn(0x7fa76c1600c0 0x7fa76c06cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:22.907 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.906+0000 7fa771000640 1 -- 172.21.15.204:0/2194463760 >> v2:172.21.15.167:3300/0 conn(0x7fa76c1600c0 msgr2=0x7fa76c06cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:22.907 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.906+0000 7fa771000640 1 --2- 172.21.15.204:0/2194463760 >> v2:172.21.15.167:3300/0 conn(0x7fa76c1600c0 0x7fa76c06cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:22.907 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.906+0000 7fa771000640 1 -- 172.21.15.204:0/2194463760 >> v1:172.21.15.204:6789/0 conn(0x7fa76c06d190 legacy=0x7fa76c0d3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:22.907 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.906+0000 7fa771000640 1 -- 172.21.15.204:0/2194463760 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa76c06dc60 con 0x7fa76c15f7a0 2024-05-11T12:10:22.908 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.907+0000 7fa771000640 1 --2- 172.21.15.204:0/2194463760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa76c15f7a0 0x7fa76c15fb80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa76c0cfd40 tx=0x7fa75c02a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c679d659b7ebe37 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:22.908 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.908+0000 7fa76b400640 1 -- 172.21.15.204:0/2194463760 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75c007740 con 0x7fa76c15f7a0 2024-05-11T12:10:22.908 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.908+0000 7fa76b400640 1 -- 172.21.15.204:0/2194463760 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa75c0078e0 con 0x7fa76c15f7a0 2024-05-11T12:10:22.908 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.908+0000 7fa76b400640 1 -- 172.21.15.204:0/2194463760 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75c007cb0 con 0x7fa76c15f7a0 2024-05-11T12:10:22.909 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.909+0000 7fa772400640 1 -- 172.21.15.204:0/2194463760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa76c15f7a0 msgr2=0x7fa76c15fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:22.909 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.909+0000 7fa772400640 1 --2- 172.21.15.204:0/2194463760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa76c15f7a0 0x7fa76c15fb80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa76c0cfd40 tx=0x7fa75c02a680 comp rx=0 tx=0).stop 2024-05-11T12:10:22.909 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.909+0000 7fa772400640 1 -- 172.21.15.204:0/2194463760 shutdown_connections 2024-05-11T12:10:22.909 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.909+0000 7fa772400640 1 --2- 172.21.15.204:0/2194463760 >> v2:172.21.15.167:3300/0 conn(0x7fa76c1600c0 0x7fa76c06cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:22.909 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.909+0000 7fa772400640 1 --2- 172.21.15.204:0/2194463760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa76c15f7a0 0x7fa76c15fb80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:22.910 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.909+0000 7fa772400640 1 -- 172.21.15.204:0/2194463760 >> 172.21.15.204:0/2194463760 conn(0x7fa76c0b6ee0 msgr2=0x7fa76c0b72f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:22.910 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.909+0000 7fa772400640 1 -- 172.21.15.204:0/2194463760 shutdown_connections 2024-05-11T12:10:22.910 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.909+0000 7fa772400640 1 -- 172.21.15.204:0/2194463760 wait complete. 2024-05-11T12:10:22.911 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.911+0000 7fa772400640 1 Processor -- start 2024-05-11T12:10:22.911 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.911+0000 7fa772400640 1 -- start start 2024-05-11T12:10:22.914 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.912+0000 7fa772400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa76c06d190 0x7fa76c14c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:22.914 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.912+0000 7fa772400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa76c15f7a0 0x7fa76c14f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:22.914 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.912+0000 7fa772400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa76c1600c0 0x7fa76c15e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:22.914 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.912+0000 7fa772400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c0d3450 con 0x7fa76c1600c0 2024-05-11T12:10:22.914 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.912+0000 7fa772400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c0d2f20 con 0x7fa76c06d190 2024-05-11T12:10:22.914 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.912+0000 7fa772400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa76c160550 con 0x7fa76c15f7a0 2024-05-11T12:10:22.915 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.913+0000 7fa771000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa76c06d190 0x7fa76c14c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:22.915 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.913+0000 7fa76be00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa76c15f7a0 0x7fa76c14f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:22.915 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.913+0000 7fa76be00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa76c15f7a0 0x7fa76c14f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:39718/0 (socket says 172.21.15.204:39718) 2024-05-11T12:10:22.915 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.913+0000 7fa76be00640 1 -- 172.21.15.204:0/184778596 learned_addr learned my addr 172.21.15.204:0/184778596 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-11T12:10:22.915 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.913+0000 7fa771a00640 1 --2- 172.21.15.204:0/184778596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa76c1600c0 0x7fa76c15e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:22.915 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.913+0000 7fa76be00640 1 -- 172.21.15.204:0/184778596 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa76c06d190 msgr2=0x7fa76c14c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:22.915 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.913+0000 7fa76be00640 1 --2- 172.21.15.204:0/184778596 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa76c06d190 0x7fa76c14c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:22.915 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.913+0000 7fa76be00640 1 -- 172.21.15.204:0/184778596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa76c1600c0 msgr2=0x7fa76c15e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:22.915 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.913+0000 7fa76be00640 1 --2- 172.21.15.204:0/184778596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa76c1600c0 0x7fa76c15e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:22.915 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.913+0000 7fa76be00640 1 -- 172.21.15.204:0/184778596 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa76c0cf420 con 0x7fa76c15f7a0 2024-05-11T12:10:22.915 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.913+0000 7fa76be00640 1 --2- 172.21.15.204:0/184778596 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa76c15f7a0 0x7fa76c14f950 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa75400d620 tx=0x7fa75400dae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:22.915 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.914+0000 7fa768c00640 1 -- 172.21.15.204:0/184778596 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75400b620 con 0x7fa76c15f7a0 2024-05-11T12:10:22.915 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.914+0000 7fa768c00640 1 -- 172.21.15.204:0/184778596 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa754019070 con 0x7fa76c15f7a0 2024-05-11T12:10:22.915 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.914+0000 7fa768c00640 1 -- 172.21.15.204:0/184778596 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75400ba00 con 0x7fa76c15f7a0 2024-05-11T12:10:22.916 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.914+0000 7fa772400640 1 -- 172.21.15.204:0/184778596 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa76c0ce8a0 con 0x7fa76c15f7a0 2024-05-11T12:10:22.916 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.914+0000 7fa772400640 1 -- 172.21.15.204:0/184778596 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa76c0cee60 con 0x7fa76c15f7a0 2024-05-11T12:10:22.916 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.914+0000 7fa768c00640 1 -- 172.21.15.204:0/184778596 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa75400bba0 con 0x7fa76c15f7a0 2024-05-11T12:10:22.916 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.914+0000 7fa768c00640 1 -- 172.21.15.204:0/184778596 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7fa75401ea50 con 0x7fa76c15f7a0 2024-05-11T12:10:22.916 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.915+0000 7fa772400640 1 -- 172.21.15.204:0/184778596 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa76c069bc0 con 0x7fa76c15f7a0 2024-05-11T12:10:22.917 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.917+0000 7fa768c00640 1 -- 172.21.15.204:0/184778596 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fa75401e460 con 0x7fa76c15f7a0 2024-05-11T12:10:22.926 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T12:10:22.926+0000 7fa9e4ddc040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-11T12:10:22.993 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:22.993+0000 7fa772400640 1 -- 172.21.15.204:0/184778596 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd new", "uuid": "1ed30e71-408f-4586-869b-ce90d9bc76c4", "id": 6} v 0) -- 0x7fa76c0b7ec0 con 0x7fa76c15f7a0 2024-05-11T12:10:23.001 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T12:10:23.001+0000 7f98b0141040 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-11T12:10:23.002 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.002+0000 7fa768c00640 1 -- 172.21.15.204:0/184778596 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1ed30e71-408f-4586-869b-ce90d9bc76c4", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7fa75401e640 con 0x7fa76c15f7a0 2024-05-11T12:10:23.002 INFO:teuthology.orchestra.run.smithi204.stdout:6 2024-05-11T12:10:23.003 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.003+0000 7fa772400640 1 -- 172.21.15.204:0/184778596 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa76c15f7a0 msgr2=0x7fa76c14f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:23.003 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.003+0000 7fa772400640 1 --2- 172.21.15.204:0/184778596 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa76c15f7a0 0x7fa76c14f950 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa75400d620 tx=0x7fa75400dae0 comp rx=0 tx=0).stop 2024-05-11T12:10:23.003 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.003+0000 7fa772400640 1 -- 172.21.15.204:0/184778596 shutdown_connections 2024-05-11T12:10:23.004 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.003+0000 7fa772400640 1 --2- 172.21.15.204:0/184778596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa76c1600c0 0x7fa76c15e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:23.004 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.003+0000 7fa772400640 1 --2- 172.21.15.204:0/184778596 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa76c15f7a0 0x7fa76c14f950 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:23.004 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.003+0000 7fa772400640 1 --2- 172.21.15.204:0/184778596 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa76c06d190 0x7fa76c14c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:23.004 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.003+0000 7fa772400640 1 -- 172.21.15.204:0/184778596 >> 172.21.15.204:0/184778596 conn(0x7fa76c0b6ee0 msgr2=0x7fa76c15a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:23.004 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.003+0000 7fa772400640 1 -- 172.21.15.204:0/184778596 shutdown_connections 2024-05-11T12:10:23.004 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.003+0000 7fa772400640 1 -- 172.21.15.204:0/184778596 wait complete. 2024-05-11T12:10:23.020 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new 37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6 7 2024-05-11T12:10:23.091 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T12:10:23.090+0000 7fa9e4ddc040 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-11T12:10:23.201 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.201+0000 7efdd7200640 1 Processor -- start 2024-05-11T12:10:23.201 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.201+0000 7efdd7200640 1 -- start start 2024-05-11T12:10:23.203 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.202+0000 7efdd7200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdd815ac60 0x7efdd80d0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:23.203 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.203+0000 7efdd7200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7efdd80b3e90 0x7efdd80d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:23.204 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.203+0000 7efdd7200640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdd815d6d0 con 0x7efdd80739b0 2024-05-11T12:10:23.204 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.203+0000 7efdd7200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7efdd815d9b0 con 0x7efdd80b3e90 2024-05-11T12:10:23.204 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.203+0000 7efdd7200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7efdd815dc90 con 0x7efdd815ac60 2024-05-11T12:10:23.204 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.203+0000 7efdd5e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdd815ac60 0x7efdd80d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:23.204 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.203+0000 7efdd5e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdd815ac60 0x7efdd80d0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:39724/0 (socket says 172.21.15.204:39724) 2024-05-11T12:10:23.204 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.203+0000 7efdd6800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7efdd80b3e90 0x7efdd80d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:23.204 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.203+0000 7efdd5e00640 1 -- 172.21.15.204:0/987233660 learned_addr learned my addr 172.21.15.204:0/987233660 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-11T12:10:23.204 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.203+0000 7efdd5e00640 1 -- 172.21.15.204:0/987233660 >> v2:172.21.15.204:3300/0 conn(0x7efdd80b3e90 msgr2=0x7efdd80d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:23.204 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.203+0000 7efdd5e00640 1 --2- 172.21.15.204:0/987233660 >> v2:172.21.15.204:3300/0 conn(0x7efdd80b3e90 0x7efdd80d0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:23.204 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.203+0000 7efdd5e00640 1 -- 172.21.15.204:0/987233660 >> v1:172.21.15.167:6789/0 conn(0x7efdd80739b0 legacy=0x7efdd8160bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:23.204 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.203+0000 7efdd5e00640 1 -- 172.21.15.204:0/987233660 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdd815df70 con 0x7efdd815ac60 2024-05-11T12:10:23.205 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.204+0000 7efdd5e00640 1 --2- 172.21.15.204:0/987233660 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdd815ac60 0x7efdd80d0100 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7efdc8009870 tx=0x7efdc802a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94acee3877e629e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:23.205 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.205+0000 7efdd4a00640 1 -- 172.21.15.204:0/987233660 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc8037070 con 0x7efdd815ac60 2024-05-11T12:10:23.205 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.205+0000 7efdd4a00640 1 -- 172.21.15.204:0/987233660 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdc8032440 con 0x7efdd815ac60 2024-05-11T12:10:23.205 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.205+0000 7efdd4a00640 1 -- 172.21.15.204:0/987233660 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc8032810 con 0x7efdd815ac60 2024-05-11T12:10:23.206 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.205+0000 7efdd7200640 1 -- 172.21.15.204:0/987233660 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdd815ac60 msgr2=0x7efdd80d0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:23.206 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.205+0000 7efdd7200640 1 --2- 172.21.15.204:0/987233660 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdd815ac60 0x7efdd80d0100 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7efdc8009870 tx=0x7efdc802a250 comp rx=0 tx=0).stop 2024-05-11T12:10:23.206 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.205+0000 7efdd7200640 1 -- 172.21.15.204:0/987233660 shutdown_connections 2024-05-11T12:10:23.206 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.205+0000 7efdd7200640 1 --2- 172.21.15.204:0/987233660 >> v2:172.21.15.204:3300/0 conn(0x7efdd80b3e90 0x7efdd80d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:23.206 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.205+0000 7efdd7200640 1 --2- 172.21.15.204:0/987233660 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdd815ac60 0x7efdd80d0100 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:23.206 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.205+0000 7efdd7200640 1 -- 172.21.15.204:0/987233660 >> 172.21.15.204:0/987233660 conn(0x7efdd8070f20 msgr2=0x7efdd80b65b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:23.206 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.206+0000 7efdd7200640 1 -- 172.21.15.204:0/987233660 shutdown_connections 2024-05-11T12:10:23.206 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.206+0000 7efdd7200640 1 -- 172.21.15.204:0/987233660 wait complete. 2024-05-11T12:10:23.207 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.207+0000 7efdd7200640 1 Processor -- start 2024-05-11T12:10:23.207 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.207+0000 7efdd7200640 1 -- start start 2024-05-11T12:10:23.210 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.208+0000 7efdd7200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdd80739b0 0x7efdd814eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:23.210 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.208+0000 7efdd7200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efdd80b3e90 0x7efdd81e8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:23.210 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.208+0000 7efdd7200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efdd815ac60 0x7efdd80cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:23.210 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.208+0000 7efdd7200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efdd80c69d0 con 0x7efdd815ac60 2024-05-11T12:10:23.210 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.208+0000 7efdd7200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7efdd806d460 con 0x7efdd80b3e90 2024-05-11T12:10:23.210 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.208+0000 7efdd7200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7efdd80c6410 con 0x7efdd80739b0 2024-05-11T12:10:23.210 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.209+0000 7efdd5e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdd80739b0 0x7efdd814eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:23.211 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.209+0000 7efdd5e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdd80739b0 0x7efdd814eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:39730/0 (socket says 172.21.15.204:39730) 2024-05-11T12:10:23.211 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.209+0000 7efdd5e00640 1 -- 172.21.15.204:0/2546351916 learned_addr learned my addr 172.21.15.204:0/2546351916 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-11T12:10:23.211 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.209+0000 7efdd5400640 1 --2- 172.21.15.204:0/2546351916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efdd80b3e90 0x7efdd81e8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:23.211 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.209+0000 7efdd6800640 1 --2- 172.21.15.204:0/2546351916 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efdd815ac60 0x7efdd80cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:23.211 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.209+0000 7efdd5e00640 1 -- 172.21.15.204:0/2546351916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efdd80b3e90 msgr2=0x7efdd81e8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:23.211 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.209+0000 7efdd5e00640 1 --2- 172.21.15.204:0/2546351916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efdd80b3e90 0x7efdd81e8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:23.211 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.209+0000 7efdd5e00640 1 -- 172.21.15.204:0/2546351916 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efdd815ac60 msgr2=0x7efdd80cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:23.211 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.209+0000 7efdd5e00640 1 --2- 172.21.15.204:0/2546351916 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efdd815ac60 0x7efdd80cc2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:23.211 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.209+0000 7efdd5e00640 1 -- 172.21.15.204:0/2546351916 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdd81e9400 con 0x7efdd80739b0 2024-05-11T12:10:23.211 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.209+0000 7efdd5e00640 1 --2- 172.21.15.204:0/2546351916 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdd80739b0 0x7efdd814eca0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efdc8000c00 tx=0x7efdc80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:23.211 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.209+0000 7efdce000640 1 -- 172.21.15.204:0/2546351916 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc8037040 con 0x7efdd80739b0 2024-05-11T12:10:23.211 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.210+0000 7efdce000640 1 -- 172.21.15.204:0/2546351916 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdc8042070 con 0x7efdd80739b0 2024-05-11T12:10:23.211 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.210+0000 7efdce000640 1 -- 172.21.15.204:0/2546351916 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc8030880 con 0x7efdd80739b0 2024-05-11T12:10:23.211 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.210+0000 7efdd7200640 1 -- 172.21.15.204:0/2546351916 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdd80ced50 con 0x7efdd80739b0 2024-05-11T12:10:23.212 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.210+0000 7efdd7200640 1 -- 172.21.15.204:0/2546351916 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7efdd80ce270 con 0x7efdd80739b0 2024-05-11T12:10:23.212 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.210+0000 7efdce000640 1 -- 172.21.15.204:0/2546351916 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7efdc8030a20 con 0x7efdd80739b0 2024-05-11T12:10:23.212 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.210+0000 7efdce000640 1 -- 172.21.15.204:0/2546351916 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7efdc8030d10 con 0x7efdd80739b0 2024-05-11T12:10:23.212 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.211+0000 7efdd7200640 1 -- 172.21.15.204:0/2546351916 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdd80b3c80 con 0x7efdd80739b0 2024-05-11T12:10:23.213 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.213+0000 7efdce000640 1 -- 172.21.15.204:0/2546351916 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7efdd80b3c80 con 0x7efdd80739b0 2024-05-11T12:10:23.289 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.289+0000 7efdd7200640 1 -- 172.21.15.204:0/2546351916 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd new", "uuid": "37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6", "id": 7} v 0) -- 0x7efdd8161950 con 0x7efdd80739b0 2024-05-11T12:10:23.299 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.299+0000 7efdce000640 1 -- 172.21.15.204:0/2546351916 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7efdc8047460 con 0x7efdd80739b0 2024-05-11T12:10:23.299 INFO:teuthology.orchestra.run.smithi204.stdout:7 2024-05-11T12:10:23.301 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.300+0000 7efdd7200640 1 -- 172.21.15.204:0/2546351916 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdd80739b0 msgr2=0x7efdd814eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:23.301 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.301+0000 7efdd7200640 1 --2- 172.21.15.204:0/2546351916 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdd80739b0 0x7efdd814eca0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efdc8000c00 tx=0x7efdc80046c0 comp rx=0 tx=0).stop 2024-05-11T12:10:23.301 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.301+0000 7efdd7200640 1 -- 172.21.15.204:0/2546351916 shutdown_connections 2024-05-11T12:10:23.301 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.301+0000 7efdd7200640 1 --2- 172.21.15.204:0/2546351916 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efdd815ac60 0x7efdd80cc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:23.302 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.301+0000 7efdd7200640 1 --2- 172.21.15.204:0/2546351916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efdd80b3e90 0x7efdd81e8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:23.302 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.301+0000 7efdd7200640 1 --2- 172.21.15.204:0/2546351916 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdd80739b0 0x7efdd814eca0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:23.302 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.301+0000 7efdd7200640 1 -- 172.21.15.204:0/2546351916 >> 172.21.15.204:0/2546351916 conn(0x7efdd8070f20 msgr2=0x7efdd8158c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:23.302 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.301+0000 7efdd7200640 1 -- 172.21.15.204:0/2546351916 shutdown_connections 2024-05-11T12:10:23.302 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:10:23.301+0000 7efdd7200640 1 -- 172.21.15.204:0/2546351916 wait complete. 2024-05-11T12:10:23.303 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T12:10:23.303+0000 7f98b0141040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-11T12:10:23.318 INFO:tasks.ceph.osd.0:Restarting daemon 2024-05-11T12:10:23.318 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-05-11T12:10:23.325 INFO:tasks.ceph.osd.0:Started 2024-05-11T12:10:23.325 INFO:tasks.ceph.osd.1:Restarting daemon 2024-05-11T12:10:23.325 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-05-11T12:10:23.332 INFO:tasks.ceph.osd.1:Started 2024-05-11T12:10:23.332 INFO:tasks.ceph.osd.2:Restarting daemon 2024-05-11T12:10:23.332 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-05-11T12:10:23.338 INFO:tasks.ceph.osd.2:Started 2024-05-11T12:10:23.338 INFO:tasks.ceph.osd.3:Restarting daemon 2024-05-11T12:10:23.338 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-05-11T12:10:23.343 INFO:tasks.ceph.osd.3:Started 2024-05-11T12:10:23.343 INFO:tasks.ceph.osd.4:Restarting daemon 2024-05-11T12:10:23.343 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-05-11T12:10:23.350 INFO:tasks.ceph.osd.4:Started 2024-05-11T12:10:23.350 INFO:tasks.ceph.osd.5:Restarting daemon 2024-05-11T12:10:23.350 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-05-11T12:10:23.357 INFO:tasks.ceph.osd.5:Started 2024-05-11T12:10:23.358 INFO:tasks.ceph.osd.6:Restarting daemon 2024-05-11T12:10:23.358 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-05-11T12:10:23.364 INFO:tasks.ceph.osd.6:Started 2024-05-11T12:10:23.364 INFO:tasks.ceph.osd.7:Restarting daemon 2024-05-11T12:10:23.364 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-05-11T12:10:23.369 INFO:tasks.ceph.osd.7:Started 2024-05-11T12:10:23.369 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:10:23.375 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T12:10:23.373+0000 7f98b0141040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-11T12:10:23.414 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T12:10:23.414+0000 7fa9e4ddc040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-11T12:10:23.481 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T12:10:23.481+0000 7fa9e4ddc040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-11T12:10:23.564 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T12:10:23.563+0000 7f98b0141040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-11T12:10:23.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.586+0000 7f6c95a00640 1 Processor -- start 2024-05-11T12:10:23.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.586+0000 7f6c95a00640 1 -- start start 2024-05-11T12:10:23.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.587+0000 7f6c95a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c9015f460 0x7f6c900b83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:23.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.587+0000 7f6c95a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6c900d42c0 0x7f6c900d0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:23.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.587+0000 7f6c95a00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c900cf990 con 0x7f6c900b88e0 2024-05-11T12:10:23.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.587+0000 7f6c95a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f6c900cfc70 con 0x7f6c900d42c0 2024-05-11T12:10:23.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.587+0000 7f6c95a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6c900cff50 con 0x7f6c9015f460 2024-05-11T12:10:23.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.587+0000 7f6c8f400640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f6c900b88e0 0x7f6c900d3bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:57678/0 (socket says 172.21.15.167:57678) 2024-05-11T12:10:23.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.587+0000 7f6c8f400640 1 -- 172.21.15.167:0/2941346448 learned_addr learned my addr 172.21.15.167:0/2941346448 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:10:23.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.587+0000 7f6c8fe00640 1 --2- 172.21.15.167:0/2941346448 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c9015f460 0x7f6c900b83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:23.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.587+0000 7f6c95000640 1 --2- 172.21.15.167:0/2941346448 >> v2:172.21.15.204:3300/0 conn(0x7f6c900d42c0 0x7f6c900d0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:23.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.588+0000 7f6c8ea00640 1 -- 172.21.15.167:0/2941346448 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4094676950 0 0) 0x7f6c900cf990 con 0x7f6c900b88e0 2024-05-11T12:10:23.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.588+0000 7f6c8ea00640 1 -- 172.21.15.167:0/2941346448 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c74003520 con 0x7f6c900b88e0 2024-05-11T12:10:23.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.588+0000 7f6c8fe00640 1 -- 172.21.15.167:0/2941346448 >> v2:172.21.15.204:3300/0 conn(0x7f6c900d42c0 msgr2=0x7f6c900d0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:23.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.588+0000 7f6c8fe00640 1 --2- 172.21.15.167:0/2941346448 >> v2:172.21.15.204:3300/0 conn(0x7f6c900d42c0 0x7f6c900d0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:23.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.588+0000 7f6c8fe00640 1 -- 172.21.15.167:0/2941346448 >> v1:172.21.15.167:6789/0 conn(0x7f6c900b88e0 legacy=0x7f6c900d3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:23.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.588+0000 7f6c8fe00640 1 -- 172.21.15.167:0/2941346448 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c900d0230 con 0x7f6c9015f460 2024-05-11T12:10:23.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.589+0000 7f6c8fe00640 1 --2- 172.21.15.167:0/2941346448 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c9015f460 0x7f6c900b83a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6c800097a0 tx=0x7f6c8002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4c536a28bf9a8ad server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:23.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.589+0000 7f6c8ea00640 1 -- 172.21.15.167:0/2941346448 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c80037070 con 0x7f6c9015f460 2024-05-11T12:10:23.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.589+0000 7f6c8ea00640 1 -- 172.21.15.167:0/2941346448 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c80032440 con 0x7f6c9015f460 2024-05-11T12:10:23.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.589+0000 7f6c8ea00640 1 -- 172.21.15.167:0/2941346448 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c80032810 con 0x7f6c9015f460 2024-05-11T12:10:23.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.590+0000 7f6c95a00640 1 -- 172.21.15.167:0/2941346448 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c9015f460 msgr2=0x7f6c900b83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:23.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.590+0000 7f6c95a00640 1 --2- 172.21.15.167:0/2941346448 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c9015f460 0x7f6c900b83a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6c800097a0 tx=0x7f6c8002a180 comp rx=0 tx=0).stop 2024-05-11T12:10:23.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.590+0000 7f6c95a00640 1 -- 172.21.15.167:0/2941346448 shutdown_connections 2024-05-11T12:10:23.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.590+0000 7f6c95a00640 1 --2- 172.21.15.167:0/2941346448 >> v2:172.21.15.204:3300/0 conn(0x7f6c900d42c0 0x7f6c900d0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:23.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.590+0000 7f6c95a00640 1 --2- 172.21.15.167:0/2941346448 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c9015f460 0x7f6c900b83a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:23.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.590+0000 7f6c95a00640 1 -- 172.21.15.167:0/2941346448 >> 172.21.15.167:0/2941346448 conn(0x7f6c900d1a50 msgr2=0x7f6c900d1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:23.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.590+0000 7f6c95a00640 1 -- 172.21.15.167:0/2941346448 shutdown_connections 2024-05-11T12:10:23.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.590+0000 7f6c95a00640 1 -- 172.21.15.167:0/2941346448 wait complete. 2024-05-11T12:10:23.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.592+0000 7f6c95a00640 1 Processor -- start 2024-05-11T12:10:23.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.592+0000 7f6c95a00640 1 -- start start 2024-05-11T12:10:23.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.593+0000 7f6c95a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c900b88e0 0x7f6c901505b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:23.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.593+0000 7f6c95a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c900d42c0 0x7f6c9015f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:23.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.593+0000 7f6c95a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c9015f460 0x7f6c9015fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:23.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.593+0000 7f6c95a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9006cc40 con 0x7f6c900b88e0 2024-05-11T12:10:23.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.593+0000 7f6c95a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6c900cf620 con 0x7f6c900d42c0 2024-05-11T12:10:23.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.593+0000 7f6c95a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6c9006cdc0 con 0x7f6c9015f460 2024-05-11T12:10:23.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.593+0000 7f6c8fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c900b88e0 0x7f6c901505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:23.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.593+0000 7f6c8fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c900b88e0 0x7f6c901505b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:51938/0 (socket says 172.21.15.167:51938) 2024-05-11T12:10:23.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.593+0000 7f6c8fe00640 1 -- 172.21.15.167:0/513290027 learned_addr learned my addr 172.21.15.167:0/513290027 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:10:23.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.593+0000 7f6c95000640 1 --2- 172.21.15.167:0/513290027 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c9015f460 0x7f6c9015fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:23.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.593+0000 7f6c8f400640 1 --2- 172.21.15.167:0/513290027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c900d42c0 0x7f6c9015f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:23.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.594+0000 7f6c8fe00640 1 -- 172.21.15.167:0/513290027 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c9015f460 msgr2=0x7f6c9015fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:23.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.594+0000 7f6c8fe00640 1 --2- 172.21.15.167:0/513290027 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c9015f460 0x7f6c9015fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:23.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.594+0000 7f6c8fe00640 1 -- 172.21.15.167:0/513290027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c900d42c0 msgr2=0x7f6c9015f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:23.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.594+0000 7f6c8fe00640 1 --2- 172.21.15.167:0/513290027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c900d42c0 0x7f6c9015f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:23.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.594+0000 7f6c8fe00640 1 -- 172.21.15.167:0/513290027 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c90151ca0 con 0x7f6c900b88e0 2024-05-11T12:10:23.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.594+0000 7f6c8fe00640 1 --2- 172.21.15.167:0/513290027 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c900b88e0 0x7f6c901505b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6c80035040 tx=0x7f6c800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:23.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.594+0000 7f6c87e00640 1 -- 172.21.15.167:0/513290027 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c80041030 con 0x7f6c900b88e0 2024-05-11T12:10:23.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.594+0000 7f6c87e00640 1 -- 172.21.15.167:0/513290027 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c8003d4e0 con 0x7f6c900b88e0 2024-05-11T12:10:23.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.594+0000 7f6c95a00640 1 -- 172.21.15.167:0/513290027 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c90160460 con 0x7f6c900b88e0 2024-05-11T12:10:23.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.594+0000 7f6c87e00640 1 -- 172.21.15.167:0/513290027 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c80037040 con 0x7f6c900b88e0 2024-05-11T12:10:23.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.594+0000 7f6c95a00640 1 -- 172.21.15.167:0/513290027 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c90160a20 con 0x7f6c900b88e0 2024-05-11T12:10:23.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.595+0000 7f6c87e00640 1 -- 172.21.15.167:0/513290027 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6c80044070 con 0x7f6c900b88e0 2024-05-11T12:10:23.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.595+0000 7f6c87e00640 1 -- 172.21.15.167:0/513290027 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f6c80046db0 con 0x7f6c900b88e0 2024-05-11T12:10:23.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.595+0000 7f6c95a00640 1 -- 172.21.15.167:0/513290027 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c900694b0 con 0x7f6c900b88e0 2024-05-11T12:10:23.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.598+0000 7f6c87e00640 1 -- 172.21.15.167:0/513290027 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f6c800466e0 con 0x7f6c900b88e0 2024-05-11T12:10:23.643 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T12:10:23.643+0000 7fa9e4ddc040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-11T12:10:23.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.665+0000 7f6c95a00640 1 -- 172.21.15.167:0/513290027 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6c900d4900 con 0x7f6c900b88e0 2024-05-11T12:10:23.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.666+0000 7f6c87e00640 1 -- 172.21.15.167:0/513290027 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f6c800468c0 con 0x7f6c900b88e0 2024-05-11T12:10:23.666 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:10:23.667 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":10,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:10:23.292769+0000","last_up_change":"0.000000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:10:23.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.667+0000 7f6c95a00640 1 -- 172.21.15.167:0/513290027 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c900b88e0 msgr2=0x7f6c901505b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:23.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.667+0000 7f6c95a00640 1 --2- 172.21.15.167:0/513290027 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c900b88e0 0x7f6c901505b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6c80035040 tx=0x7f6c800046c0 comp rx=0 tx=0).stop 2024-05-11T12:10:23.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.667+0000 7f6c95a00640 1 -- 172.21.15.167:0/513290027 shutdown_connections 2024-05-11T12:10:23.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.667+0000 7f6c95a00640 1 --2- 172.21.15.167:0/513290027 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c9015f460 0x7f6c9015fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:23.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.667+0000 7f6c95a00640 1 --2- 172.21.15.167:0/513290027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c900d42c0 0x7f6c9015f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:23.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.667+0000 7f6c95a00640 1 --2- 172.21.15.167:0/513290027 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c900b88e0 0x7f6c901505b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:23.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.667+0000 7f6c95a00640 1 -- 172.21.15.167:0/513290027 >> 172.21.15.167:0/513290027 conn(0x7f6c900d1a50 msgr2=0x7f6c900ba5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:23.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.668+0000 7f6c95a00640 1 -- 172.21.15.167:0/513290027 shutdown_connections 2024-05-11T12:10:23.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:23.668+0000 7f6c95a00640 1 -- 172.21.15.167:0/513290027 wait complete. 2024-05-11T12:10:23.684 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-05-11T12:10:23.685 INFO:tasks.ceph:Waiting for OSDs to come up 2024-05-11T12:10:23.906 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T12:10:23.906+0000 7f98b0141040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-11T12:10:23.962 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T12:10:23.962+0000 7f98b0141040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-11T12:10:23.974 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T12:10:23.974+0000 7fa9e4ddc040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-11T12:10:24.024 INFO:tasks.ceph.osd.1.smithi167.stderr:2024-05-11T12:10:24.024+0000 7ff8d37ad640 -1 Falling back to public interface 2024-05-11T12:10:24.028 INFO:tasks.ceph.osd.0.smithi167.stderr:2024-05-11T12:10:24.028+0000 7f5b4331d640 -1 Falling back to public interface 2024-05-11T12:10:24.030 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T12:10:24.030+0000 7fa9e4ddc040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-11T12:10:24.048 INFO:tasks.ceph.osd.3.smithi167.stderr:2024-05-11T12:10:24.049+0000 7f8d90fd4640 -1 Falling back to public interface 2024-05-11T12:10:24.049 INFO:tasks.ceph.osd.6.smithi204.stderr:2024-05-11T12:10:24.049+0000 7f41d9144640 -1 Falling back to public interface 2024-05-11T12:10:24.051 INFO:tasks.ceph.osd.2.smithi167.stderr:2024-05-11T12:10:24.052+0000 7f539440c640 -1 Falling back to public interface 2024-05-11T12:10:24.056 INFO:tasks.ceph.osd.7.smithi204.stderr:2024-05-11T12:10:24.057+0000 7f325c788640 -1 Falling back to public interface 2024-05-11T12:10:24.061 INFO:tasks.ceph.osd.4.smithi204.stderr:2024-05-11T12:10:24.061+0000 7f6b88263640 -1 Falling back to public interface 2024-05-11T12:10:24.071 INFO:tasks.ceph.osd.5.smithi204.stderr:2024-05-11T12:10:24.071+0000 7f903e21c640 -1 Falling back to public interface 2024-05-11T12:10:24.155 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T12:10:24.155+0000 7f98b0141040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-11T12:10:24.222 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T12:10:24.222+0000 7fa9e4ddc040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-11T12:10:24.323 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T12:10:24.323+0000 7f98b0141040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-11T12:10:24.383 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T12:10:24.383+0000 7f98b0141040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-11T12:10:24.390 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T12:10:24.390+0000 7fa9e4ddc040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-11T12:10:24.450 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T12:10:24.449+0000 7fa9e4ddc040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-11T12:10:24.464 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T12:10:24.464+0000 7f98b0141040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-11T12:10:24.490 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-11T12:10:24.518 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T12:10:24.518+0000 7f98b0141040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-11T12:10:24.530 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T12:10:24.530+0000 7fa9e4ddc040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-11T12:10:24.583 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T12:10:24.583+0000 7fa9e4ddc040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-11T12:10:24.655 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.654+0000 7f2a1f000640 1 Processor -- start 2024-05-11T12:10:24.655 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.655+0000 7f2a1f000640 1 -- start start 2024-05-11T12:10:24.656 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.656+0000 7f2a1f000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2a18162800 0x7f2a180cf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:24.656 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.656+0000 7f2a1f000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2a180d1270 0x7f2a180d3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:24.656 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.656+0000 7f2a1f000640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2a180d4120 0x7f2a1806c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:24.656 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.656+0000 7f2a1f000640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f2a180d07f0 con 0x7f2a180d4120 2024-05-11T12:10:24.656 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.656+0000 7f2a1f000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f2a180b8230 con 0x7f2a180d1270 2024-05-11T12:10:24.656 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.656+0000 7f2a1f000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2a180cff50 con 0x7f2a18162800 2024-05-11T12:10:24.656 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.656+0000 7f2a1e600640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2a180d4120 0x7f2a1806c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:24.657 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.656+0000 7f2a1e600640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2a180d4120 0x7f2a1806c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:51940/0 (socket says 172.21.15.167:51940) 2024-05-11T12:10:24.657 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.656+0000 7f2a1e600640 1 -- 172.21.15.167:0/948115365 learned_addr learned my addr 172.21.15.167:0/948115365 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:10:24.657 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.656+0000 7f2a1dc00640 1 --2- 172.21.15.167:0/948115365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2a18162800 0x7f2a180cf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:24.657 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.656+0000 7f2a1d200640 1 --2- 172.21.15.167:0/948115365 >> v2:172.21.15.204:3300/0 conn(0x7f2a180d1270 0x7f2a180d3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:24.657 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.656+0000 7f2a1e600640 1 -- 172.21.15.167:0/948115365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2a18162800 msgr2=0x7f2a180cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:24.658 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.656+0000 7f2a1e600640 1 --2- 172.21.15.167:0/948115365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2a18162800 0x7f2a180cf650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:24.658 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.656+0000 7f2a1e600640 1 -- 172.21.15.167:0/948115365 >> v2:172.21.15.204:3300/0 conn(0x7f2a180d1270 msgr2=0x7f2a180d3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:24.658 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.656+0000 7f2a1e600640 1 --2- 172.21.15.167:0/948115365 >> v2:172.21.15.204:3300/0 conn(0x7f2a180d1270 0x7f2a180d3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:24.658 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.657+0000 7f2a1e600640 1 -- 172.21.15.167:0/948115365 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a180d0230 con 0x7f2a180d4120 2024-05-11T12:10:24.658 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.657+0000 7f2a1e600640 1 --2- 172.21.15.167:0/948115365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a180d4120 0x7f2a1806c320 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2a18075ed0 tx=0x7f2a1002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bc6667e8dcd3d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:24.659 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.658+0000 7f2a17e00640 1 -- 172.21.15.167:0/948115365 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a10037070 con 0x7f2a180d4120 2024-05-11T12:10:24.659 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.658+0000 7f2a17e00640 1 -- 172.21.15.167:0/948115365 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a10032440 con 0x7f2a180d4120 2024-05-11T12:10:24.659 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.658+0000 7f2a17e00640 1 -- 172.21.15.167:0/948115365 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a10032810 con 0x7f2a180d4120 2024-05-11T12:10:24.659 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.658+0000 7f2a1f000640 1 -- 172.21.15.167:0/948115365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a180d4120 msgr2=0x7f2a1806c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:24.659 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.658+0000 7f2a1f000640 1 --2- 172.21.15.167:0/948115365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a180d4120 0x7f2a1806c320 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2a18075ed0 tx=0x7f2a1002a1e0 comp rx=0 tx=0).stop 2024-05-11T12:10:24.659 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.658+0000 7f2a1f000640 1 -- 172.21.15.167:0/948115365 shutdown_connections 2024-05-11T12:10:24.660 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.658+0000 7f2a1f000640 1 --2- 172.21.15.167:0/948115365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a180d4120 0x7f2a1806c320 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:24.660 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.658+0000 7f2a1f000640 1 --2- 172.21.15.167:0/948115365 >> v2:172.21.15.204:3300/0 conn(0x7f2a180d1270 0x7f2a180d3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:24.660 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.658+0000 7f2a1f000640 1 --2- 172.21.15.167:0/948115365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2a18162800 0x7f2a180cf650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:24.660 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.658+0000 7f2a1f000640 1 -- 172.21.15.167:0/948115365 >> 172.21.15.167:0/948115365 conn(0x7f2a180b6af0 msgr2=0x7f2a180b6f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:24.660 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.659+0000 7f2a1f000640 1 -- 172.21.15.167:0/948115365 shutdown_connections 2024-05-11T12:10:24.660 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.659+0000 7f2a1f000640 1 -- 172.21.15.167:0/948115365 wait complete. 2024-05-11T12:10:24.660 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.660+0000 7f2a1f000640 1 Processor -- start 2024-05-11T12:10:24.660 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.660+0000 7f2a1f000640 1 -- start start 2024-05-11T12:10:24.661 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.661+0000 7f2a1f000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2a180d1270 0x7f2a18158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:24.662 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.661+0000 7f2a1f000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a180d4120 0x7f2a18155d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:24.662 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.661+0000 7f2a1f000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a18162800 0x7f2a181536a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:24.662 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.661+0000 7f2a1f000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2a180cfb90 con 0x7f2a180d4120 2024-05-11T12:10:24.662 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.661+0000 7f2a1f000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2a18162be0 con 0x7f2a18162800 2024-05-11T12:10:24.662 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.661+0000 7f2a1f000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2a180cfd10 con 0x7f2a180d1270 2024-05-11T12:10:24.662 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.661+0000 7f2a1d200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a180d4120 0x7f2a18155d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:24.662 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.662+0000 7f2a1d200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a180d4120 0x7f2a18155d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:51950/0 (socket says 172.21.15.167:51950) 2024-05-11T12:10:24.662 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.662+0000 7f2a1d200640 1 -- 172.21.15.167:0/580705402 learned_addr learned my addr 172.21.15.167:0/580705402 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:10:24.662 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.662+0000 7f2a1dc00640 1 --2- 172.21.15.167:0/580705402 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2a180d1270 0x7f2a18158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:24.662 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.662+0000 7f2a1e600640 1 --2- 172.21.15.167:0/580705402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a18162800 0x7f2a181536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:24.664 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.662+0000 7f2a1d200640 1 -- 172.21.15.167:0/580705402 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2a180d1270 msgr2=0x7f2a18158cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:24.664 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.662+0000 7f2a1d200640 1 --2- 172.21.15.167:0/580705402 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2a180d1270 0x7f2a18158cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:24.664 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.662+0000 7f2a1d200640 1 -- 172.21.15.167:0/580705402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a18162800 msgr2=0x7f2a181536a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:24.664 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.662+0000 7f2a1d200640 1 --2- 172.21.15.167:0/580705402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a18162800 0x7f2a181536a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:24.664 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.662+0000 7f2a1d200640 1 -- 172.21.15.167:0/580705402 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a180cebb0 con 0x7f2a180d4120 2024-05-11T12:10:24.664 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.662+0000 7f2a1d200640 1 --2- 172.21.15.167:0/580705402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a180d4120 0x7f2a18155d50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2a0400e860 tx=0x7f2a0400ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:24.664 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.662+0000 7f2a15600640 1 -- 172.21.15.167:0/580705402 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a04017070 con 0x7f2a180d4120 2024-05-11T12:10:24.664 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.663+0000 7f2a15600640 1 -- 172.21.15.167:0/580705402 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a0400c940 con 0x7f2a180d4120 2024-05-11T12:10:24.664 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.663+0000 7f2a1f000640 1 -- 172.21.15.167:0/580705402 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a180cee90 con 0x7f2a180d4120 2024-05-11T12:10:24.664 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.663+0000 7f2a15600640 1 -- 172.21.15.167:0/580705402 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0400cd10 con 0x7f2a180d4120 2024-05-11T12:10:24.664 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.663+0000 7f2a1f000640 1 -- 172.21.15.167:0/580705402 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a181584b0 con 0x7f2a180d4120 2024-05-11T12:10:24.665 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.663+0000 7f2a15600640 1 -- 172.21.15.167:0/580705402 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2a0401a070 con 0x7f2a180d4120 2024-05-11T12:10:24.665 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.663+0000 7f2a15600640 1 -- 172.21.15.167:0/580705402 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f2a040065f0 con 0x7f2a180d4120 2024-05-11T12:10:24.665 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.664+0000 7f2a1f000640 1 -- 172.21.15.167:0/580705402 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a180507f0 con 0x7f2a180d4120 2024-05-11T12:10:24.666 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.666+0000 7f2a15600640 1 -- 172.21.15.167:0/580705402 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f2a04005990 con 0x7f2a180d4120 2024-05-11T12:10:24.734 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.734+0000 7f2a1f000640 1 -- 172.21.15.167:0/580705402 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a1806c7a0 con 0x7f2a180d4120 2024-05-11T12:10:24.735 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.735+0000 7f2a15600640 1 -- 172.21.15.167:0/580705402 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f2a04005b70 con 0x7f2a180d4120 2024-05-11T12:10:24.736 INFO:teuthology.misc.health.smithi167.stdout: 2024-05-11T12:10:24.736 INFO:teuthology.misc.health.smithi167.stdout:{"epoch":10,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:10:23.292769+0000","last_up_change":"0.000000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:10:24.737 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.737+0000 7f2a1f000640 1 -- 172.21.15.167:0/580705402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a180d4120 msgr2=0x7f2a18155d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:24.737 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.737+0000 7f2a1f000640 1 --2- 172.21.15.167:0/580705402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a180d4120 0x7f2a18155d50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2a0400e860 tx=0x7f2a0400ed20 comp rx=0 tx=0).stop 2024-05-11T12:10:24.737 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.737+0000 7f2a1f000640 1 -- 172.21.15.167:0/580705402 shutdown_connections 2024-05-11T12:10:24.737 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.737+0000 7f2a1f000640 1 --2- 172.21.15.167:0/580705402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a18162800 0x7f2a181536a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:24.737 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.737+0000 7f2a1f000640 1 --2- 172.21.15.167:0/580705402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a180d4120 0x7f2a18155d50 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:24.737 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.737+0000 7f2a1f000640 1 --2- 172.21.15.167:0/580705402 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2a180d1270 0x7f2a18158cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:24.738 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.737+0000 7f2a1f000640 1 -- 172.21.15.167:0/580705402 >> 172.21.15.167:0/580705402 conn(0x7f2a180b6af0 msgr2=0x7f2a181612a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:24.738 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.737+0000 7f2a1f000640 1 -- 172.21.15.167:0/580705402 shutdown_connections 2024-05-11T12:10:24.738 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:24.737+0000 7f2a1f000640 1 -- 172.21.15.167:0/580705402 wait complete. 2024-05-11T12:10:24.749 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-11T12:10:25.072 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T12:10:25.071+0000 7f98b0141040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-11T12:10:25.131 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T12:10:25.131+0000 7fa9e4ddc040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-11T12:10:25.207 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T12:10:25.207+0000 7f98b0141040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-11T12:10:25.273 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T12:10:25.273+0000 7fa9e4ddc040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-11T12:10:26.245 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T12:10:26.244+0000 7f9850000640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-05-11T12:10:26.245 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T12:10:26.245+0000 7f9850000640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-11T12:10:27.041 INFO:tasks.ceph.osd.2.smithi167.stderr:2024-05-11T12:10:27.041+0000 7f539440c640 -1 osd.2 0 log_to_monitors true 2024-05-11T12:10:27.686 INFO:tasks.ceph.osd.6.smithi204.stderr:2024-05-11T12:10:27.686+0000 7f41d9144640 -1 osd.6 0 log_to_monitors true 2024-05-11T12:10:27.740 INFO:tasks.ceph.osd.3.smithi167.stderr:2024-05-11T12:10:27.741+0000 7f8d90fd4640 -1 osd.3 0 log_to_monitors true 2024-05-11T12:10:27.745 INFO:tasks.ceph.osd.5.smithi204.stderr:2024-05-11T12:10:27.744+0000 7f903e21c640 -1 osd.5 0 log_to_monitors true 2024-05-11T12:10:27.916 INFO:tasks.ceph.osd.0.smithi167.stderr:2024-05-11T12:10:27.917+0000 7f5b4331d640 -1 osd.0 0 log_to_monitors true 2024-05-11T12:10:27.921 INFO:tasks.ceph.osd.1.smithi167.stderr:2024-05-11T12:10:27.922+0000 7ff8d37ad640 -1 osd.1 0 log_to_monitors true 2024-05-11T12:10:27.954 INFO:tasks.ceph.osd.7.smithi204.stderr:2024-05-11T12:10:27.954+0000 7f325c788640 -1 osd.7 0 log_to_monitors true 2024-05-11T12:10:27.958 INFO:tasks.ceph.osd.4.smithi204.stderr:2024-05-11T12:10:27.958+0000 7f6b88263640 -1 osd.4 0 log_to_monitors true 2024-05-11T12:10:29.403 INFO:tasks.ceph.osd.2.smithi167.stderr:2024-05-11T12:10:29.403+0000 7f538ec00640 -1 osd.2 0 waiting for initial osdmap 2024-05-11T12:10:29.512 INFO:tasks.ceph.osd.2.smithi167.stderr:2024-05-11T12:10:29.513+0000 7f5384c00640 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T12:10:31.108 INFO:tasks.ceph.osd.1.smithi167.stderr:2024-05-11T12:10:31.109+0000 7ff8cf000640 -1 osd.1 0 waiting for initial osdmap 2024-05-11T12:10:31.114 INFO:tasks.ceph.osd.0.smithi167.stderr:2024-05-11T12:10:31.116+0000 7f5b3e200640 -1 osd.0 0 waiting for initial osdmap 2024-05-11T12:10:31.120 INFO:tasks.ceph.osd.1.smithi167.stderr:2024-05-11T12:10:31.120+0000 7ff8c4600640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T12:10:31.123 INFO:tasks.ceph.osd.0.smithi167.stderr:2024-05-11T12:10:31.124+0000 7f5b34200640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T12:10:31.189 INFO:tasks.ceph.osd.3.smithi167.stderr:2024-05-11T12:10:31.190+0000 7f8d8be00640 -1 osd.3 0 waiting for initial osdmap 2024-05-11T12:10:31.197 INFO:tasks.ceph.osd.3.smithi167.stderr:2024-05-11T12:10:31.198+0000 7f8d81e00640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T12:10:31.556 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-11T12:10:31.588 INFO:tasks.ceph.osd.5.smithi204.stderr:2024-05-11T12:10:31.587+0000 7f9038a00640 -1 osd.5 0 waiting for initial osdmap 2024-05-11T12:10:31.595 INFO:tasks.ceph.osd.5.smithi204.stderr:2024-05-11T12:10:31.595+0000 7f902ea00640 -1 osd.5 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T12:10:31.600 INFO:tasks.ceph.osd.6.smithi204.stderr:2024-05-11T12:10:31.601+0000 7f41d4a00640 -1 osd.6 0 waiting for initial osdmap 2024-05-11T12:10:31.608 INFO:tasks.ceph.osd.6.smithi204.stderr:2024-05-11T12:10:31.608+0000 7f41ca000640 -1 osd.6 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T12:10:31.649 INFO:tasks.ceph.osd.4.smithi204.stderr:2024-05-11T12:10:31.649+0000 7f6b83200640 -1 osd.4 0 waiting for initial osdmap 2024-05-11T12:10:31.660 INFO:tasks.ceph.osd.4.smithi204.stderr:2024-05-11T12:10:31.660+0000 7f6b79200640 -1 osd.4 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T12:10:31.663 INFO:tasks.ceph.osd.7.smithi204.stderr:2024-05-11T12:10:31.663+0000 7f3258000640 -1 osd.7 0 waiting for initial osdmap 2024-05-11T12:10:31.674 INFO:tasks.ceph.osd.7.smithi204.stderr:2024-05-11T12:10:31.674+0000 7f324d600640 -1 osd.7 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T12:10:31.722 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.723+0000 7f58e1000640 1 Processor -- start 2024-05-11T12:10:31.722 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.723+0000 7f58e1000640 1 -- start start 2024-05-11T12:10:31.724 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.724+0000 7f58e1000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f58dc145870 0x7f58dc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:31.724 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.724+0000 7f58e1000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f58dc160360 0x7f58dc06d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:31.725 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.724+0000 7f58e1000640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58dc0cbd90 con 0x7f58dc0ccb10 2024-05-11T12:10:31.725 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.724+0000 7f58e1000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f58dc0d0500 con 0x7f58dc160360 2024-05-11T12:10:31.725 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.724+0000 7f58e1000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f58dc145c50 con 0x7f58dc145870 2024-05-11T12:10:31.725 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.725+0000 7f58d1400640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f58dc0ccb10 0x7f58dc15fc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:43576/0 (socket says 172.21.15.167:43576) 2024-05-11T12:10:31.725 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.725+0000 7f58d1400640 1 -- 172.21.15.167:0/3493555104 learned_addr learned my addr 172.21.15.167:0/3493555104 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:10:31.725 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.725+0000 7f58da800640 1 --2- 172.21.15.167:0/3493555104 >> v2:172.21.15.204:3300/0 conn(0x7f58dc160360 0x7f58dc06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:31.725 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.725+0000 7f58d9e00640 1 --2- 172.21.15.167:0/3493555104 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f58dc145870 0x7f58dc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:31.725 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.725+0000 7f58d9400640 1 -- 172.21.15.167:0/3493555104 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4146558744 0 0) 0x7f58dc0cbd90 con 0x7f58dc0ccb10 2024-05-11T12:10:31.725 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.725+0000 7f58d9400640 1 -- 172.21.15.167:0/3493555104 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f58c0003520 con 0x7f58dc0ccb10 2024-05-11T12:10:31.725 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.725+0000 7f58d9e00640 1 -- 172.21.15.167:0/3493555104 >> v2:172.21.15.204:3300/0 conn(0x7f58dc160360 msgr2=0x7f58dc06d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:31.725 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.725+0000 7f58d9e00640 1 --2- 172.21.15.167:0/3493555104 >> v2:172.21.15.204:3300/0 conn(0x7f58dc160360 0x7f58dc06d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:31.725 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.725+0000 7f58d9e00640 1 -- 172.21.15.167:0/3493555104 >> v1:172.21.15.167:6789/0 conn(0x7f58dc0ccb10 legacy=0x7f58dc15fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:31.725 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.725+0000 7f58d9e00640 1 -- 172.21.15.167:0/3493555104 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58dc0cc030 con 0x7f58dc145870 2024-05-11T12:10:31.725 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.725+0000 7f58da800640 1 --2- 172.21.15.167:0/3493555104 >> v2:172.21.15.204:3300/0 conn(0x7f58dc160360 0x7f58dc06d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:10:31.727 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.726+0000 7f58d9e00640 1 --2- 172.21.15.167:0/3493555104 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f58dc145870 0x7f58dc06d330 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f58cc0097a0 tx=0x7f58cc02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5990bec0f5aa1ce server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:31.727 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.727+0000 7f58d9400640 1 -- 172.21.15.167:0/3493555104 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc037070 con 0x7f58dc145870 2024-05-11T12:10:31.727 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.727+0000 7f58d9400640 1 -- 172.21.15.167:0/3493555104 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58cc032440 con 0x7f58dc145870 2024-05-11T12:10:31.727 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.727+0000 7f58d9400640 1 -- 172.21.15.167:0/3493555104 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc0327f0 con 0x7f58dc145870 2024-05-11T12:10:31.727 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.727+0000 7f58e1000640 1 -- 172.21.15.167:0/3493555104 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f58dc145870 msgr2=0x7f58dc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:31.727 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.727+0000 7f58e1000640 1 --2- 172.21.15.167:0/3493555104 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f58dc145870 0x7f58dc06d330 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f58cc0097a0 tx=0x7f58cc02a180 comp rx=0 tx=0).stop 2024-05-11T12:10:31.727 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.728+0000 7f58e1000640 1 -- 172.21.15.167:0/3493555104 shutdown_connections 2024-05-11T12:10:31.727 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.728+0000 7f58e1000640 1 --2- 172.21.15.167:0/3493555104 >> v2:172.21.15.204:3300/0 conn(0x7f58dc160360 0x7f58dc06d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:31.727 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.728+0000 7f58e1000640 1 --2- 172.21.15.167:0/3493555104 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f58dc145870 0x7f58dc06d330 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:31.728 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.728+0000 7f58e1000640 1 -- 172.21.15.167:0/3493555104 >> 172.21.15.167:0/3493555104 conn(0x7f58dc0cd780 msgr2=0x7f58dc0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:31.728 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.728+0000 7f58e1000640 1 -- 172.21.15.167:0/3493555104 shutdown_connections 2024-05-11T12:10:31.728 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.728+0000 7f58e1000640 1 -- 172.21.15.167:0/3493555104 wait complete. 2024-05-11T12:10:31.729 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.730+0000 7f58e1000640 1 Processor -- start 2024-05-11T12:10:31.729 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.730+0000 7f58e1000640 1 -- start start 2024-05-11T12:10:31.732 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.731+0000 7f58e1000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58dc0ccb10 0x7f58dc15adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:31.732 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.731+0000 7f58e1000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58dc145870 0x7f58dc0b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:31.732 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.731+0000 7f58e1000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f58dc160360 0x7f58dc0b6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:31.732 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.731+0000 7f58e1000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc1609b0 con 0x7f58dc145870 2024-05-11T12:10:31.732 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.731+0000 7f58e1000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc0cc5f0 con 0x7f58dc0ccb10 2024-05-11T12:10:31.732 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.731+0000 7f58e1000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f58dc0cf4f0 con 0x7f58dc160360 2024-05-11T12:10:31.732 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.731+0000 7f58d1400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58dc145870 0x7f58dc0b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:31.732 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.731+0000 7f58d1400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58dc145870 0x7f58dc0b61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:38680/0 (socket says 172.21.15.167:38680) 2024-05-11T12:10:31.732 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.731+0000 7f58d1400640 1 -- 172.21.15.167:0/3119890388 learned_addr learned my addr 172.21.15.167:0/3119890388 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:10:31.732 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.732+0000 7f58d9e00640 1 --2- 172.21.15.167:0/3119890388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58dc0ccb10 0x7f58dc15adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:31.732 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.732+0000 7f58d1400640 1 -- 172.21.15.167:0/3119890388 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f58dc160360 msgr2=0x7f58dc0b6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:31.733 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.732+0000 7f58d1400640 1 --2- 172.21.15.167:0/3119890388 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f58dc160360 0x7f58dc0b6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:31.733 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.732+0000 7f58d1400640 1 -- 172.21.15.167:0/3119890388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58dc0ccb10 msgr2=0x7f58dc15adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:31.733 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.732+0000 7f58d1400640 1 --2- 172.21.15.167:0/3119890388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58dc0ccb10 0x7f58dc15adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:31.733 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.732+0000 7f58d1400640 1 -- 172.21.15.167:0/3119890388 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58dc15b2f0 con 0x7f58dc145870 2024-05-11T12:10:31.733 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.732+0000 7f58d9e00640 1 --2- 172.21.15.167:0/3119890388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58dc0ccb10 0x7f58dc15adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:10:31.733 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.732+0000 7f58d1400640 1 --2- 172.21.15.167:0/3119890388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58dc145870 0x7f58dc0b61f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f58c400c500 tx=0x7f58c400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:31.733 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.732+0000 7f58d2a00640 1 -- 172.21.15.167:0/3119890388 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c4017070 con 0x7f58dc145870 2024-05-11T12:10:31.733 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.732+0000 7f58d2a00640 1 -- 172.21.15.167:0/3119890388 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58c4005ce0 con 0x7f58dc145870 2024-05-11T12:10:31.733 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.732+0000 7f58d2a00640 1 -- 172.21.15.167:0/3119890388 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c4007270 con 0x7f58dc145870 2024-05-11T12:10:31.733 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.732+0000 7f58e1000640 1 -- 172.21.15.167:0/3119890388 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58dc159d10 con 0x7f58dc145870 2024-05-11T12:10:31.733 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.733+0000 7f58e1000640 1 -- 172.21.15.167:0/3119890388 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58dc15a2d0 con 0x7f58dc145870 2024-05-11T12:10:31.734 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.733+0000 7f58e1000640 1 -- 172.21.15.167:0/3119890388 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58dc069de0 con 0x7f58dc145870 2024-05-11T12:10:31.734 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.735+0000 7f58d2a00640 1 -- 172.21.15.167:0/3119890388 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 4) ==== 98145+0+0 (secure 0 0 0) 0x7f58c401a070 con 0x7f58dc145870 2024-05-11T12:10:31.735 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.735+0000 7f58d2a00640 1 --2- 172.21.15.167:0/3119890388 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f58b8076240 0x7f58b8078700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:31.735 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.736+0000 7f58d2a00640 1 -- 172.21.15.167:0/3119890388 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3351+0+0 (secure 0 0 0) 0x7f58c4095e10 con 0x7f58dc145870 2024-05-11T12:10:31.735 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.736+0000 7f58d9e00640 1 --2- 172.21.15.167:0/3119890388 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f58b8076240 0x7f58b8078700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:31.735 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.736+0000 7f58d9e00640 1 --2- 172.21.15.167:0/3119890388 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f58b8076240 0x7f58b8078700 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f58cc034000 tx=0x7f58cc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:31.740 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.741+0000 7f58d2a00640 1 -- 172.21.15.167:0/3119890388 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f58dc15b2f0 con 0x7f58dc145870 2024-05-11T12:10:31.930 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.931+0000 7f58e1000640 1 -- 172.21.15.167:0/3119890388 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f58dc0d0840 con 0x7f58dc145870 2024-05-11T12:10:31.932 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.932+0000 7f58d2a00640 1 -- 172.21.15.167:0/3119890388 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) ==== 74+0+9048 (secure 0 0 0) 0x7f58c405fb20 con 0x7f58dc145870 2024-05-11T12:10:31.932 INFO:teuthology.misc.health.smithi167.stdout: 2024-05-11T12:10:31.932 INFO:teuthology.misc.health.smithi167.stdout:{"epoch":15,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:10:31.547366+0000","last_up_change":"2024-05-11T12:10:31.547366+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:10:31.935 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.935+0000 7f58e1000640 1 -- 172.21.15.167:0/3119890388 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f58b8076240 msgr2=0x7f58b8078700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:31.935 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.935+0000 7f58e1000640 1 --2- 172.21.15.167:0/3119890388 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f58b8076240 0x7f58b8078700 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f58cc034000 tx=0x7f58cc02e000 comp rx=0 tx=0).stop 2024-05-11T12:10:31.935 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.935+0000 7f58e1000640 1 -- 172.21.15.167:0/3119890388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58dc145870 msgr2=0x7f58dc0b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:31.935 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.935+0000 7f58e1000640 1 --2- 172.21.15.167:0/3119890388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58dc145870 0x7f58dc0b61f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f58c400c500 tx=0x7f58c400c9c0 comp rx=0 tx=0).stop 2024-05-11T12:10:31.935 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.935+0000 7f58e1000640 1 -- 172.21.15.167:0/3119890388 shutdown_connections 2024-05-11T12:10:31.936 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.935+0000 7f58e1000640 1 --2- 172.21.15.167:0/3119890388 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f58dc160360 0x7f58dc0b6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:31.936 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.935+0000 7f58e1000640 1 --2- 172.21.15.167:0/3119890388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58dc145870 0x7f58dc0b61f0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:31.936 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.935+0000 7f58e1000640 1 --2- 172.21.15.167:0/3119890388 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f58b8076240 0x7f58b8078700 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:31.936 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.935+0000 7f58e1000640 1 --2- 172.21.15.167:0/3119890388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58dc0ccb10 0x7f58dc15adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:31.936 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.935+0000 7f58e1000640 1 -- 172.21.15.167:0/3119890388 >> 172.21.15.167:0/3119890388 conn(0x7f58dc0cd780 msgr2=0x7f58dc06d610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:31.936 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.936+0000 7f58e1000640 1 -- 172.21.15.167:0/3119890388 shutdown_connections 2024-05-11T12:10:31.936 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:31.936+0000 7f58e1000640 1 -- 172.21.15.167:0/3119890388 wait complete. 2024-05-11T12:10:31.948 DEBUG:teuthology.misc:4 of 8 OSDs are up 2024-05-11T12:10:36.528 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T12:10:36.537 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T12:10:38.755 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-11T12:10:38.990 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.990+0000 7f2c4fe00640 1 Processor -- start 2024-05-11T12:10:38.990 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.991+0000 7f2c4fe00640 1 -- start start 2024-05-11T12:10:38.991 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.992+0000 7f2c4fe00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2c500b6190 0x7f2c5006f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:38.992 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.992+0000 7f2c4fe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c5006dc40 0x7f2c500d11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:38.992 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.992+0000 7f2c4fe00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c500d0410 con 0x7f2c500b6570 2024-05-11T12:10:38.992 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.992+0000 7f2c4fe00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f2c50167f60 con 0x7f2c500b6190 2024-05-11T12:10:38.992 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.992+0000 7f2c4fe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2c500d01c0 con 0x7f2c5006dc40 2024-05-11T12:10:38.992 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.993+0000 7f2c47e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f2c500b6570 0x7f2c5006d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:51300/0 (socket says 172.21.15.167:51300) 2024-05-11T12:10:38.992 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.993+0000 7f2c47e00640 1 -- 172.21.15.167:0/3107401415 learned_addr learned my addr 172.21.15.167:0/3107401415 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:10:38.994 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.993+0000 7f2c4ea00640 1 --2- 172.21.15.167:0/3107401415 >> v2:172.21.15.204:3300/0 conn(0x7f2c500b6190 0x7f2c5006f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:38.994 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.993+0000 7f2c4f400640 1 --2- 172.21.15.167:0/3107401415 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c5006dc40 0x7f2c500d11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:38.994 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.993+0000 7f2c4e000640 1 -- 172.21.15.167:0/3107401415 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1060498086 0 0) 0x7f2c500d0410 con 0x7f2c500b6570 2024-05-11T12:10:38.994 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.993+0000 7f2c4e000640 1 -- 172.21.15.167:0/3107401415 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2c34003520 con 0x7f2c500b6570 2024-05-11T12:10:38.994 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.994+0000 7f2c4e000640 1 -- 172.21.15.167:0/3107401415 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 514264592 0 0) 0x7f2c34003520 con 0x7f2c500b6570 2024-05-11T12:10:38.994 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.994+0000 7f2c4e000640 1 -- 172.21.15.167:0/3107401415 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c5006dc40 msgr2=0x7f2c500d11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:38.995 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.994+0000 7f2c4e000640 1 --2- 172.21.15.167:0/3107401415 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c5006dc40 0x7f2c500d11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:38.995 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.994+0000 7f2c4e000640 1 -- 172.21.15.167:0/3107401415 >> v2:172.21.15.204:3300/0 conn(0x7f2c500b6190 msgr2=0x7f2c5006f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:38.995 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.994+0000 7f2c4e000640 1 --2- 172.21.15.167:0/3107401415 >> v2:172.21.15.204:3300/0 conn(0x7f2c500b6190 0x7f2c5006f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:38.995 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.994+0000 7f2c4e000640 1 -- 172.21.15.167:0/3107401415 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c500d06f0 con 0x7f2c500b6570 2024-05-11T12:10:38.995 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.994+0000 7f2c4e000640 1 -- 172.21.15.167:0/3107401415 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f2c380016f0 con 0x7f2c500b6570 2024-05-11T12:10:38.995 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.994+0000 7f2c4e000640 1 -- 172.21.15.167:0/3107401415 >> v1:172.21.15.167:6789/0 conn(0x7f2c500b6570 legacy=0x7f2c5006d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:38.995 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.994+0000 7f2c4e000640 1 --2- 172.21.15.167:0/3107401415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c34003a70 0x7f2c34003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:38.995 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.994+0000 7f2c4e000640 1 --2- 172.21.15.167:0/3107401415 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c5006dc40 0x7f2c34004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:38.995 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.994+0000 7f2c4e000640 1 --2- 172.21.15.167:0/3107401415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c34004d40 0x7f2c340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:38.995 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.994+0000 7f2c4e000640 1 -- 172.21.15.167:0/3107401415 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2c500d01c0 con 0x7f2c34004d40 2024-05-11T12:10:38.995 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.994+0000 7f2c4e000640 1 -- 172.21.15.167:0/3107401415 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3400ba70 con 0x7f2c34003a70 2024-05-11T12:10:38.995 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.994+0000 7f2c4e000640 1 -- 172.21.15.167:0/3107401415 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2c3400bc40 con 0x7f2c5006dc40 2024-05-11T12:10:38.995 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.994+0000 7f2c4f400640 1 --2- 172.21.15.167:0/3107401415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c34004d40 0x7f2c340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:38.995 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.994+0000 7f2c47e00640 1 --2- 172.21.15.167:0/3107401415 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c5006dc40 0x7f2c34004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:38.996 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.994+0000 7f2c4ea00640 1 --2- 172.21.15.167:0/3107401415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c34003a70 0x7f2c34003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:38.996 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.994+0000 7f2c4f400640 1 -- 172.21.15.167:0/3107401415 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c5006dc40 msgr2=0x7f2c34004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:38.996 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.994+0000 7f2c4f400640 1 --2- 172.21.15.167:0/3107401415 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c5006dc40 0x7f2c34004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:38.996 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.994+0000 7f2c4f400640 1 -- 172.21.15.167:0/3107401415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c34003a70 msgr2=0x7f2c34003e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:38.996 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.994+0000 7f2c4f400640 1 --2- 172.21.15.167:0/3107401415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c34003a70 0x7f2c34003e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:38.996 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.995+0000 7f2c4f400640 1 -- 172.21.15.167:0/3107401415 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c34003520 con 0x7f2c34004d40 2024-05-11T12:10:38.996 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.995+0000 7f2c47e00640 1 --2- 172.21.15.167:0/3107401415 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c5006dc40 0x7f2c34004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:10:38.996 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.995+0000 7f2c4ea00640 1 --2- 172.21.15.167:0/3107401415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c34003a70 0x7f2c34003e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:10:38.997 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.996+0000 7f2c4f400640 1 --2- 172.21.15.167:0/3107401415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c34004d40 0x7f2c340051c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2c48009fc0 tx=0x7f2c34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1836a85c77f45aba server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:38.997 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.996+0000 7f2c4e000640 1 -- 172.21.15.167:0/3107401415 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c48009c10 con 0x7f2c34004d40 2024-05-11T12:10:38.998 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.996+0000 7f2c4e000640 1 -- 172.21.15.167:0/3107401415 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c48036070 con 0x7f2c34004d40 2024-05-11T12:10:38.998 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.996+0000 7f2c4e000640 1 -- 172.21.15.167:0/3107401415 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c48031880 con 0x7f2c34004d40 2024-05-11T12:10:38.998 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.996+0000 7f2c4fe00640 1 -- 172.21.15.167:0/3107401415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c34004d40 msgr2=0x7f2c340051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:38.998 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.996+0000 7f2c4fe00640 1 --2- 172.21.15.167:0/3107401415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c34004d40 0x7f2c340051c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2c48009fc0 tx=0x7f2c34005830 comp rx=0 tx=0).stop 2024-05-11T12:10:38.998 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.996+0000 7f2c4f400640 1 -- 172.21.15.167:0/3107401415 reap_dead start 2024-05-11T12:10:38.998 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.997+0000 7f2c4fe00640 1 -- 172.21.15.167:0/3107401415 shutdown_connections 2024-05-11T12:10:38.998 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.997+0000 7f2c4fe00640 1 -- 172.21.15.167:0/3107401415 >> 172.21.15.167:0/3107401415 conn(0x7f2c500733e0 msgr2=0x7f2c500737c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:38.998 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.997+0000 7f2c4fe00640 1 -- 172.21.15.167:0/3107401415 shutdown_connections 2024-05-11T12:10:38.998 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.997+0000 7f2c4fe00640 1 -- 172.21.15.167:0/3107401415 wait complete. 2024-05-11T12:10:38.998 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.998+0000 7f2c4fe00640 1 Processor -- start 2024-05-11T12:10:38.998 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.999+0000 7f2c4fe00640 1 -- start start 2024-05-11T12:10:39.000 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.999+0000 7f2c4fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c34004d40 0x7f2c5015a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:39.000 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:38.999+0000 7f2c4fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c50143df0 0x7f2c5015c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:39.000 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:39.000+0000 7f2c4fe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c50159190 0x7f2c500ced10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:39.000 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:39.000+0000 7f2c4fe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5006fa10 con 0x7f2c34004d40 2024-05-11T12:10:39.000 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:39.000+0000 7f2c4fe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2c500d0cb0 con 0x7f2c50143df0 2024-05-11T12:10:39.000 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:39.000+0000 7f2c4fe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2c5006fb90 con 0x7f2c50159190 2024-05-11T12:10:39.000 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:39.000+0000 7f2c4ea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c34004d40 0x7f2c5015a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:39.000 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:39.000+0000 7f2c4ea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c34004d40 0x7f2c5015a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58836/0 (socket says 172.21.15.167:58836) 2024-05-11T12:10:39.001 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:39.000+0000 7f2c4ea00640 1 -- 172.21.15.167:0/650930838 learned_addr learned my addr 172.21.15.167:0/650930838 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:10:39.001 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:39.000+0000 7f2c47e00640 1 --2- 172.21.15.167:0/650930838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c50143df0 0x7f2c5015c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:39.001 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:39.000+0000 7f2c4f400640 1 --2- 172.21.15.167:0/650930838 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c50159190 0x7f2c500ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:39.001 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:39.000+0000 7f2c4ea00640 1 -- 172.21.15.167:0/650930838 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c50159190 msgr2=0x7f2c500ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:39.001 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:39.000+0000 7f2c4ea00640 1 --2- 172.21.15.167:0/650930838 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c50159190 0x7f2c500ced10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:39.001 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:39.000+0000 7f2c4ea00640 1 -- 172.21.15.167:0/650930838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c50143df0 msgr2=0x7f2c5015c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:39.001 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:39.000+0000 7f2c4ea00640 1 --2- 172.21.15.167:0/650930838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c50143df0 0x7f2c5015c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:39.001 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:39.000+0000 7f2c4ea00640 1 -- 172.21.15.167:0/650930838 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c50144270 con 0x7f2c34004d40 2024-05-11T12:10:39.001 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:39.001+0000 7f2c4ea00640 1 --2- 172.21.15.167:0/650930838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c34004d40 0x7f2c5015a9a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2c40010a00 tx=0x7f2c40010ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:39.001 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:39.001+0000 7f2c46a00640 1 -- 172.21.15.167:0/650930838 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4000f040 con 0x7f2c34004d40 2024-05-11T12:10:39.001 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:39.001+0000 7f2c46a00640 1 -- 172.21.15.167:0/650930838 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c4000ec20 con 0x7f2c34004d40 2024-05-11T12:10:39.001 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:39.001+0000 7f2c46a00640 1 -- 172.21.15.167:0/650930838 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4001c400 con 0x7f2c34004d40 2024-05-11T12:10:39.001 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:39.001+0000 7f2c4fe00640 1 -- 172.21.15.167:0/650930838 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c50159900 con 0x7f2c34004d40 2024-05-11T12:10:39.001 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:39.001+0000 7f2c4fe00640 1 -- 172.21.15.167:0/650930838 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c50159ec0 con 0x7f2c34004d40 2024-05-11T12:10:39.002 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:39.002+0000 7f2c4fe00640 1 -- 172.21.15.167:0/650930838 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c500b3400 con 0x7f2c34004d40 2024-05-11T12:10:39.002 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:39.003+0000 7f2c46a00640 1 -- 172.21.15.167:0/650930838 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2c4001c5a0 con 0x7f2c34004d40 2024-05-11T12:10:39.003 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:39.004+0000 7f2c46a00640 1 --2- 172.21.15.167:0/650930838 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2c2c0762d0 0x7f2c2c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:39.003 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:39.004+0000 7f2c46a00640 1 -- 172.21.15.167:0/650930838 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f2c40097a50 con 0x7f2c34004d40 2024-05-11T12:10:39.003 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:39.005+0000 7f2c47e00640 1 --2- 172.21.15.167:0/650930838 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2c2c0762d0 0x7f2c2c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:39.004 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:39.005+0000 7f2c47e00640 1 --2- 172.21.15.167:0/650930838 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2c2c0762d0 0x7f2c2c078790 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2c501fc8c0 tx=0x7f2c38009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:39.009 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:39.010+0000 7f2c46a00640 1 -- 172.21.15.167:0/650930838 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2c500b3400 con 0x7f2c34004d40 2024-05-11T12:10:39.224 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:39.225+0000 7f2c4fe00640 1 -- 172.21.15.167:0/650930838 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2c500507f0 con 0x7f2c34004d40 2024-05-11T12:10:39.225 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:39.226+0000 7f2c46a00640 1 -- 172.21.15.167:0/650930838 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+12932 (secure 0 0 0) 0x7f2c40018030 con 0x7f2c34004d40 2024-05-11T12:10:39.226 INFO:teuthology.misc.health.smithi167.stdout: 2024-05-11T12:10:39.226 INFO:teuthology.misc.health.smithi167.stdout:{"epoch":19,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:10:35.627747+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:10:39.228 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:39.229+0000 7f2c4fe00640 1 -- 172.21.15.167:0/650930838 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2c2c0762d0 msgr2=0x7f2c2c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:39.228 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:39.229+0000 7f2c4fe00640 1 --2- 172.21.15.167:0/650930838 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2c2c0762d0 0x7f2c2c078790 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2c501fc8c0 tx=0x7f2c38009040 comp rx=0 tx=0).stop 2024-05-11T12:10:39.228 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:39.229+0000 7f2c4fe00640 1 -- 172.21.15.167:0/650930838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c34004d40 msgr2=0x7f2c5015a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:39.229 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:39.229+0000 7f2c4fe00640 1 --2- 172.21.15.167:0/650930838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c34004d40 0x7f2c5015a9a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2c40010a00 tx=0x7f2c40010ec0 comp rx=0 tx=0).stop 2024-05-11T12:10:39.229 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:39.229+0000 7f2c4fe00640 1 -- 172.21.15.167:0/650930838 shutdown_connections 2024-05-11T12:10:39.229 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:39.229+0000 7f2c4fe00640 1 --2- 172.21.15.167:0/650930838 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c50159190 0x7f2c500ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:39.229 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:39.229+0000 7f2c4fe00640 1 --2- 172.21.15.167:0/650930838 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2c2c0762d0 0x7f2c2c078790 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:39.229 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:39.229+0000 7f2c4fe00640 1 --2- 172.21.15.167:0/650930838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c50143df0 0x7f2c5015c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:39.229 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:39.229+0000 7f2c4fe00640 1 --2- 172.21.15.167:0/650930838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c34004d40 0x7f2c5015a9a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:39.229 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:39.229+0000 7f2c4fe00640 1 -- 172.21.15.167:0/650930838 >> 172.21.15.167:0/650930838 conn(0x7f2c500733e0 msgr2=0x7f2c500b5780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:39.229 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:39.229+0000 7f2c4fe00640 1 -- 172.21.15.167:0/650930838 shutdown_connections 2024-05-11T12:10:39.229 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T12:10:39.229+0000 7f2c4fe00640 1 -- 172.21.15.167:0/650930838 wait complete. 2024-05-11T12:10:39.244 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-05-11T12:10:39.244 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-05-11T12:10:39.244 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-11T12:10:39.244 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-11T12:10:39.249 INFO:tasks.ceph.mds.a:Started 2024-05-11T12:10:39.250 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-11T12:10:39.250 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-11T12:10:39.254 INFO:tasks.ceph.mds.c:Started 2024-05-11T12:10:39.254 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-11T12:10:39.254 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-11T12:10:39.262 INFO:tasks.ceph.mds.b:Started 2024-05-11T12:10:39.262 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-11T12:10:39.262 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-11T12:10:39.270 INFO:tasks.ceph.mds.d:Started 2024-05-11T12:10:39.270 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-05-11T12:10:39.270 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:10:39.393 INFO:tasks.ceph.mds.a.smithi167.stdout:starting mds.a at 2024-05-11T12:10:39.396 INFO:tasks.ceph.mds.c.smithi167.stdout:starting mds.c at 2024-05-11T12:10:39.417 INFO:tasks.ceph.mds.d.smithi204.stdout:starting mds.d at 2024-05-11T12:10:39.418 INFO:tasks.ceph.mds.b.smithi204.stdout:starting mds.b at 2024-05-11T12:10:39.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.447+0000 7fdb5aa00640 1 Processor -- start 2024-05-11T12:10:39.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.448+0000 7fdb5aa00640 1 -- start start 2024-05-11T12:10:39.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.449+0000 7fdb5aa00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fdb540b3ea0 0x7fdb54161490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:39.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.449+0000 7fdb5aa00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fdb541619d0 0x7fdb5406d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:39.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.449+0000 7fdb5aa00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb5415df30 con 0x7fdb54157750 2024-05-11T12:10:39.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.449+0000 7fdb5aa00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fdb54161000 con 0x7fdb541619d0 2024-05-11T12:10:39.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.449+0000 7fdb5aa00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fdb5415e210 con 0x7fdb540b3ea0 2024-05-11T12:10:39.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.449+0000 7fdb53e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fdb54157750 0x7fdb5415d740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:51314/0 (socket says 172.21.15.167:51314) 2024-05-11T12:10:39.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.449+0000 7fdb53e00640 1 -- 172.21.15.167:0/57114354 learned_addr learned my addr 172.21.15.167:0/57114354 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:10:39.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.449+0000 7fdb58a00640 1 --2- 172.21.15.167:0/57114354 >> v2:172.21.15.167:3300/0 conn(0x7fdb541619d0 0x7fdb5406d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:39.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.449+0000 7fdb4b400640 1 --2- 172.21.15.167:0/57114354 >> v2:172.21.15.204:3300/0 conn(0x7fdb540b3ea0 0x7fdb54161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:39.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.449+0000 7fdb53400640 1 -- 172.21.15.167:0/57114354 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 694625409 0 0) 0x7fdb5415df30 con 0x7fdb54157750 2024-05-11T12:10:39.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.450+0000 7fdb53400640 1 -- 172.21.15.167:0/57114354 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb38003520 con 0x7fdb54157750 2024-05-11T12:10:39.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.450+0000 7fdb58a00640 1 -- 172.21.15.167:0/57114354 >> v2:172.21.15.204:3300/0 conn(0x7fdb540b3ea0 msgr2=0x7fdb54161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:39.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.450+0000 7fdb58a00640 1 --2- 172.21.15.167:0/57114354 >> v2:172.21.15.204:3300/0 conn(0x7fdb540b3ea0 0x7fdb54161490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:39.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.450+0000 7fdb58a00640 1 -- 172.21.15.167:0/57114354 >> v1:172.21.15.167:6789/0 conn(0x7fdb54157750 legacy=0x7fdb5415d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:39.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.450+0000 7fdb58a00640 1 -- 172.21.15.167:0/57114354 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb5415e4f0 con 0x7fdb541619d0 2024-05-11T12:10:39.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.450+0000 7fdb4b400640 1 --2- 172.21.15.167:0/57114354 >> v2:172.21.15.204:3300/0 conn(0x7fdb540b3ea0 0x7fdb54161490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:10:39.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.451+0000 7fdb58a00640 1 --2- 172.21.15.167:0/57114354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb541619d0 0x7fdb5406d460 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdb540702f0 tx=0x7fdb4c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb4a9ad630d00758 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:39.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.451+0000 7fdb53400640 1 -- 172.21.15.167:0/57114354 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c007740 con 0x7fdb541619d0 2024-05-11T12:10:39.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.451+0000 7fdb53400640 1 -- 172.21.15.167:0/57114354 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb4c0078e0 con 0x7fdb541619d0 2024-05-11T12:10:39.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.451+0000 7fdb53400640 1 -- 172.21.15.167:0/57114354 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c007cb0 con 0x7fdb541619d0 2024-05-11T12:10:39.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.451+0000 7fdb5aa00640 1 -- 172.21.15.167:0/57114354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb541619d0 msgr2=0x7fdb5406d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:39.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.451+0000 7fdb5aa00640 1 --2- 172.21.15.167:0/57114354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb541619d0 0x7fdb5406d460 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdb540702f0 tx=0x7fdb4c02a680 comp rx=0 tx=0).stop 2024-05-11T12:10:39.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.452+0000 7fdb5aa00640 1 -- 172.21.15.167:0/57114354 shutdown_connections 2024-05-11T12:10:39.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.452+0000 7fdb5aa00640 1 --2- 172.21.15.167:0/57114354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb541619d0 0x7fdb5406d460 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:39.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.452+0000 7fdb5aa00640 1 --2- 172.21.15.167:0/57114354 >> v2:172.21.15.204:3300/0 conn(0x7fdb540b3ea0 0x7fdb54161490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:39.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.452+0000 7fdb5aa00640 1 -- 172.21.15.167:0/57114354 >> 172.21.15.167:0/57114354 conn(0x7fdb540cd8f0 msgr2=0x7fdb540cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:39.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.452+0000 7fdb5aa00640 1 -- 172.21.15.167:0/57114354 shutdown_connections 2024-05-11T12:10:39.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.452+0000 7fdb5aa00640 1 -- 172.21.15.167:0/57114354 wait complete. 2024-05-11T12:10:39.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.453+0000 7fdb5aa00640 1 Processor -- start 2024-05-11T12:10:39.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.454+0000 7fdb5aa00640 1 -- start start 2024-05-11T12:10:39.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.454+0000 7fdb5aa00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb540b3ea0 0x7fdb541f28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:39.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.454+0000 7fdb5aa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb54157750 0x7fdb54159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:39.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.454+0000 7fdb5aa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb541619d0 0x7fdb54141420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:39.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.454+0000 7fdb5aa00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5406db20 con 0x7fdb54157750 2024-05-11T12:10:39.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.454+0000 7fdb5aa00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5406d9a0 con 0x7fdb541619d0 2024-05-11T12:10:39.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.454+0000 7fdb5aa00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fdb5406dca0 con 0x7fdb540b3ea0 2024-05-11T12:10:39.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.455+0000 7fdb4b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb54157750 0x7fdb54159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:39.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.455+0000 7fdb4b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb54157750 0x7fdb54159cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58880/0 (socket says 172.21.15.167:58880) 2024-05-11T12:10:39.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.455+0000 7fdb4b400640 1 -- 172.21.15.167:0/2218913250 learned_addr learned my addr 172.21.15.167:0/2218913250 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:10:39.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.455+0000 7fdb58a00640 1 --2- 172.21.15.167:0/2218913250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb541619d0 0x7fdb54141420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:39.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.455+0000 7fdb53e00640 1 --2- 172.21.15.167:0/2218913250 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb540b3ea0 0x7fdb541f28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:39.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.455+0000 7fdb4b400640 1 -- 172.21.15.167:0/2218913250 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb540b3ea0 msgr2=0x7fdb541f28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:39.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.455+0000 7fdb4b400640 1 --2- 172.21.15.167:0/2218913250 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb540b3ea0 0x7fdb541f28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:39.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.455+0000 7fdb4b400640 1 -- 172.21.15.167:0/2218913250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb541619d0 msgr2=0x7fdb54141420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:39.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.455+0000 7fdb4b400640 1 --2- 172.21.15.167:0/2218913250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb541619d0 0x7fdb54141420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:39.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.455+0000 7fdb4b400640 1 -- 172.21.15.167:0/2218913250 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb541f6e60 con 0x7fdb54157750 2024-05-11T12:10:39.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.455+0000 7fdb58a00640 1 --2- 172.21.15.167:0/2218913250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb541619d0 0x7fdb54141420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:10:39.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.455+0000 7fdb53e00640 1 --2- 172.21.15.167:0/2218913250 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb540b3ea0 0x7fdb541f28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:10:39.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.455+0000 7fdb4b400640 1 --2- 172.21.15.167:0/2218913250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb54157750 0x7fdb54159cf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c00ebe0 tx=0x7fdb3c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:39.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.456+0000 7fdb50c00640 1 -- 172.21.15.167:0/2218913250 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c017070 con 0x7fdb54157750 2024-05-11T12:10:39.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.456+0000 7fdb50c00640 1 -- 172.21.15.167:0/2218913250 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb3c00cc30 con 0x7fdb54157750 2024-05-11T12:10:39.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.456+0000 7fdb50c00640 1 -- 172.21.15.167:0/2218913250 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c013470 con 0x7fdb54157750 2024-05-11T12:10:39.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.456+0000 7fdb5aa00640 1 -- 172.21.15.167:0/2218913250 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb541f2e30 con 0x7fdb54157750 2024-05-11T12:10:39.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.456+0000 7fdb5aa00640 1 -- 172.21.15.167:0/2218913250 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb54141960 con 0x7fdb54157750 2024-05-11T12:10:39.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.457+0000 7fdb5aa00640 1 -- 172.21.15.167:0/2218913250 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb541f6340 con 0x7fdb54157750 2024-05-11T12:10:39.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.458+0000 7fdb50c00640 1 -- 172.21.15.167:0/2218913250 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fdb3c01a070 con 0x7fdb54157750 2024-05-11T12:10:39.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.459+0000 7fdb50c00640 1 --2- 172.21.15.167:0/2218913250 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fdb300762d0 0x7fdb30078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:39.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.459+0000 7fdb50c00640 1 -- 172.21.15.167:0/2218913250 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7fdb3c097210 con 0x7fdb54157750 2024-05-11T12:10:39.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.460+0000 7fdb53e00640 1 --2- 172.21.15.167:0/2218913250 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fdb300762d0 0x7fdb30078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:39.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.460+0000 7fdb53e00640 1 --2- 172.21.15.167:0/2218913250 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fdb300762d0 0x7fdb30078790 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdb44001220 tx=0x7fdb44009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:39.463 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.464+0000 7fdb50c00640 1 -- 172.21.15.167:0/2218913250 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fdb541f6340 con 0x7fdb54157750 2024-05-11T12:10:39.698 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.699+0000 7fdb5aa00640 1 -- 172.21.15.167:0/2218913250 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdb54069590 con 0x7fdb54157750 2024-05-11T12:10:39.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.700+0000 7fdb50c00640 1 -- 172.21.15.167:0/2218913250 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1395 (secure 0 0 0) 0x7fdb3c060e20 con 0x7fdb54157750 2024-05-11T12:10:39.699 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:10:39.699 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3127277486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3127277486},{"type":"v1","addr":"172.21.15.167:6835","nonce":3127277486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-11T12:10:39.699 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 2 2024-05-11T12:10:39.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.702+0000 7fdb5aa00640 1 -- 172.21.15.167:0/2218913250 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fdb300762d0 msgr2=0x7fdb30078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:39.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.702+0000 7fdb5aa00640 1 --2- 172.21.15.167:0/2218913250 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fdb300762d0 0x7fdb30078790 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdb44001220 tx=0x7fdb44009040 comp rx=0 tx=0).stop 2024-05-11T12:10:39.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.702+0000 7fdb5aa00640 1 -- 172.21.15.167:0/2218913250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb54157750 msgr2=0x7fdb54159cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:39.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.702+0000 7fdb5aa00640 1 --2- 172.21.15.167:0/2218913250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb54157750 0x7fdb54159cf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c00ebe0 tx=0x7fdb3c00c520 comp rx=0 tx=0).stop 2024-05-11T12:10:39.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.702+0000 7fdb5aa00640 1 -- 172.21.15.167:0/2218913250 shutdown_connections 2024-05-11T12:10:39.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.702+0000 7fdb5aa00640 1 --2- 172.21.15.167:0/2218913250 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fdb300762d0 0x7fdb30078790 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:39.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.702+0000 7fdb5aa00640 1 --2- 172.21.15.167:0/2218913250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb541619d0 0x7fdb54141420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:39.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.702+0000 7fdb5aa00640 1 --2- 172.21.15.167:0/2218913250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb54157750 0x7fdb54159cf0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:39.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.703+0000 7fdb5aa00640 1 --2- 172.21.15.167:0/2218913250 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb540b3ea0 0x7fdb541f28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:39.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.703+0000 7fdb5aa00640 1 -- 172.21.15.167:0/2218913250 >> 172.21.15.167:0/2218913250 conn(0x7fdb540cd8f0 msgr2=0x7fdb54160140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:39.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.703+0000 7fdb5aa00640 1 -- 172.21.15.167:0/2218913250 shutdown_connections 2024-05-11T12:10:39.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:39.703+0000 7fdb5aa00640 1 -- 172.21.15.167:0/2218913250 wait complete. 2024-05-11T12:10:41.720 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:10:41.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.924+0000 7f27a4c00640 1 Processor -- start 2024-05-11T12:10:41.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.924+0000 7f27a4c00640 1 -- start start 2024-05-11T12:10:41.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.926+0000 7f27a4c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f27a013eeb0 0x7f27a006f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:41.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.926+0000 7f27a4c00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f27a013f290 0x7f27a006f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:41.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.926+0000 7f27a4c00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27a015fb90 con 0x7f27a00b50a0 2024-05-11T12:10:41.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.926+0000 7f27a4c00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f27a015d5b0 con 0x7f27a013f290 2024-05-11T12:10:41.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.926+0000 7f27a4c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f27a015fe70 con 0x7f27a013eeb0 2024-05-11T12:10:41.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.926+0000 7f279d400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f27a013f290 0x7f27a006f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:41.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.926+0000 7f279e800640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f27a00b50a0 0x7f27a00735c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:51320/0 (socket says 172.21.15.167:51320) 2024-05-11T12:10:41.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.926+0000 7f279e800640 1 -- 172.21.15.167:0/3761879383 learned_addr learned my addr 172.21.15.167:0/3761879383 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:10:41.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.926+0000 7f279de00640 1 --2- 172.21.15.167:0/3761879383 >> v2:172.21.15.204:3300/0 conn(0x7f27a013eeb0 0x7f27a006f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:41.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.927+0000 7f279d400640 1 -- 172.21.15.167:0/3761879383 >> v2:172.21.15.204:3300/0 conn(0x7f27a013eeb0 msgr2=0x7f27a006f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:41.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.927+0000 7f279d400640 1 --2- 172.21.15.167:0/3761879383 >> v2:172.21.15.204:3300/0 conn(0x7f27a013eeb0 0x7f27a006f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:41.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.927+0000 7f279d400640 1 -- 172.21.15.167:0/3761879383 >> v1:172.21.15.167:6789/0 conn(0x7f27a00b50a0 legacy=0x7f27a00735c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:41.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.927+0000 7f279ca00640 1 -- 172.21.15.167:0/3761879383 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1327096034 0 0) 0x7f27a015fb90 con 0x7f27a00b50a0 2024-05-11T12:10:41.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.927+0000 7f279d400640 1 -- 172.21.15.167:0/3761879383 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27a0160150 con 0x7f27a013f290 2024-05-11T12:10:41.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.928+0000 7f279d400640 1 --2- 172.21.15.167:0/3761879383 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27a013f290 0x7f27a006f9c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f27a00b6590 tx=0x7f278c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f5613229aacd2ea server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:41.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.928+0000 7f279ca00640 1 -- 172.21.15.167:0/3761879383 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c037070 con 0x7f27a013f290 2024-05-11T12:10:41.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.928+0000 7f279ca00640 1 -- 172.21.15.167:0/3761879383 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f278c032440 con 0x7f27a013f290 2024-05-11T12:10:41.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.928+0000 7f279ca00640 1 -- 172.21.15.167:0/3761879383 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c02ae60 con 0x7f27a013f290 2024-05-11T12:10:41.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.929+0000 7f27a4c00640 1 -- 172.21.15.167:0/3761879383 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27a013f290 msgr2=0x7f27a006f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:41.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.929+0000 7f27a4c00640 1 --2- 172.21.15.167:0/3761879383 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27a013f290 0x7f27a006f9c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f27a00b6590 tx=0x7f278c02a110 comp rx=0 tx=0).stop 2024-05-11T12:10:41.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.929+0000 7f27a4c00640 1 -- 172.21.15.167:0/3761879383 shutdown_connections 2024-05-11T12:10:41.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.929+0000 7f27a4c00640 1 --2- 172.21.15.167:0/3761879383 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27a013f290 0x7f27a006f9c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:41.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.929+0000 7f27a4c00640 1 --2- 172.21.15.167:0/3761879383 >> v2:172.21.15.204:3300/0 conn(0x7f27a013eeb0 0x7f27a006f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:41.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.929+0000 7f27a4c00640 1 -- 172.21.15.167:0/3761879383 >> 172.21.15.167:0/3761879383 conn(0x7f27a00cd8f0 msgr2=0x7f27a00cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:41.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.929+0000 7f27a4c00640 1 -- 172.21.15.167:0/3761879383 shutdown_connections 2024-05-11T12:10:41.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.929+0000 7f27a4c00640 1 -- 172.21.15.167:0/3761879383 wait complete. 2024-05-11T12:10:41.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.930+0000 7f27a4c00640 1 Processor -- start 2024-05-11T12:10:41.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.931+0000 7f27a4c00640 1 -- start start 2024-05-11T12:10:41.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.931+0000 7f27a4c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27a00b50a0 0x7f27a014efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:41.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.931+0000 7f27a4c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27a013eeb0 0x7f27a013ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:41.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.932+0000 7f27a4c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27a013f290 0x7f27a013d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:41.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.932+0000 7f27a4c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f27a0073cd0 con 0x7f27a013f290 2024-05-11T12:10:41.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.932+0000 7f27a4c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f27a0160710 con 0x7f27a013eeb0 2024-05-11T12:10:41.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.932+0000 7f27a4c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f27a0073e50 con 0x7f27a00b50a0 2024-05-11T12:10:41.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.932+0000 7f279e800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27a013f290 0x7f27a013d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:41.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.932+0000 7f279e800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27a013f290 0x7f27a013d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58906/0 (socket says 172.21.15.167:58906) 2024-05-11T12:10:41.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.932+0000 7f279e800640 1 -- 172.21.15.167:0/4115626590 learned_addr learned my addr 172.21.15.167:0/4115626590 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:10:41.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.932+0000 7f279de00640 1 --2- 172.21.15.167:0/4115626590 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27a00b50a0 0x7f27a014efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:41.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.932+0000 7f279d400640 1 --2- 172.21.15.167:0/4115626590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27a013eeb0 0x7f27a013ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:41.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.932+0000 7f279e800640 1 -- 172.21.15.167:0/4115626590 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27a00b50a0 msgr2=0x7f27a014efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:41.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.932+0000 7f279e800640 1 --2- 172.21.15.167:0/4115626590 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27a00b50a0 0x7f27a014efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:41.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.932+0000 7f279e800640 1 -- 172.21.15.167:0/4115626590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27a013eeb0 msgr2=0x7f27a013ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:41.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.932+0000 7f279e800640 1 --2- 172.21.15.167:0/4115626590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27a013eeb0 0x7f27a013ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:41.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.932+0000 7f279e800640 1 -- 172.21.15.167:0/4115626590 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27a0154030 con 0x7f27a013f290 2024-05-11T12:10:41.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.933+0000 7f279e800640 1 --2- 172.21.15.167:0/4115626590 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27a013f290 0x7f27a013d360 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f279800c7d0 tx=0x7f279800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:41.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.933+0000 7f2796000640 1 -- 172.21.15.167:0/4115626590 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2798018070 con 0x7f27a013f290 2024-05-11T12:10:41.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.933+0000 7f2796000640 1 -- 172.21.15.167:0/4115626590 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f279800a5e0 con 0x7f27a013f290 2024-05-11T12:10:41.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.933+0000 7f2796000640 1 -- 172.21.15.167:0/4115626590 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279800aa10 con 0x7f27a013f290 2024-05-11T12:10:41.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.933+0000 7f27a4c00640 1 -- 172.21.15.167:0/4115626590 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27a0154310 con 0x7f27a013f290 2024-05-11T12:10:41.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.933+0000 7f27a4c00640 1 -- 172.21.15.167:0/4115626590 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27a00cef00 con 0x7f27a013f290 2024-05-11T12:10:41.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.934+0000 7f27a4c00640 1 -- 172.21.15.167:0/4115626590 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27a00cecf0 con 0x7f27a013f290 2024-05-11T12:10:41.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.935+0000 7f2796000640 1 -- 172.21.15.167:0/4115626590 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f279801b070 con 0x7f27a013f290 2024-05-11T12:10:41.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.936+0000 7f2796000640 1 --2- 172.21.15.167:0/4115626590 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f27800762d0 0x7f2780078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:41.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.937+0000 7f2796000640 1 -- 172.21.15.167:0/4115626590 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f2798096950 con 0x7f27a013f290 2024-05-11T12:10:41.936 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.937+0000 7f279de00640 1 --2- 172.21.15.167:0/4115626590 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f27800762d0 0x7f2780078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:41.936 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.937+0000 7f279de00640 1 --2- 172.21.15.167:0/4115626590 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f27800762d0 0x7f2780078790 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2790006fd0 tx=0x7f2790008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:41.940 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:41.941+0000 7f2796000640 1 -- 172.21.15.167:0/4115626590 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2798060560 con 0x7f27a013f290 2024-05-11T12:10:42.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.190+0000 7f27a4c00640 1 -- 172.21.15.167:0/4115626590 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27a0069590 con 0x7f27a013f290 2024-05-11T12:10:42.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.190+0000 7f2796000640 1 -- 172.21.15.167:0/4115626590 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3876 (secure 0 0 0) 0x7f279805fe60 con 0x7f27a013f290 2024-05-11T12:10:42.190 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:10:42.190 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3127277486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3127277486},{"type":"v1","addr":"172.21.15.167:6835","nonce":3127277486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2547133600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2547133600},{"type":"v1","addr":"172.21.15.167:6837","nonce":2547133600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3200960100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3200960100},{"type":"v1","addr":"172.21.15.204:6834","nonce":3200960100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1889842375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":1889842375},{"type":"v1","addr":"172.21.15.204:6835","nonce":1889842375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-11T12:10:42.190 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 3 2024-05-11T12:10:42.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.193+0000 7f27a4c00640 1 -- 172.21.15.167:0/4115626590 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f27800762d0 msgr2=0x7f2780078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:42.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.193+0000 7f27a4c00640 1 --2- 172.21.15.167:0/4115626590 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f27800762d0 0x7f2780078790 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2790006fd0 tx=0x7f2790008040 comp rx=0 tx=0).stop 2024-05-11T12:10:42.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.193+0000 7f27a4c00640 1 -- 172.21.15.167:0/4115626590 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27a013f290 msgr2=0x7f27a013d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:42.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.193+0000 7f27a4c00640 1 --2- 172.21.15.167:0/4115626590 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27a013f290 0x7f27a013d360 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f279800c7d0 tx=0x7f279800cc90 comp rx=0 tx=0).stop 2024-05-11T12:10:42.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.193+0000 7f27a4c00640 1 -- 172.21.15.167:0/4115626590 shutdown_connections 2024-05-11T12:10:42.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.193+0000 7f27a4c00640 1 --2- 172.21.15.167:0/4115626590 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27a013f290 0x7f27a013d360 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:42.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.193+0000 7f27a4c00640 1 --2- 172.21.15.167:0/4115626590 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f27800762d0 0x7f2780078790 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:42.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.193+0000 7f27a4c00640 1 --2- 172.21.15.167:0/4115626590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27a013eeb0 0x7f27a013ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:42.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.194+0000 7f27a4c00640 1 --2- 172.21.15.167:0/4115626590 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27a00b50a0 0x7f27a014efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:42.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.194+0000 7f27a4c00640 1 -- 172.21.15.167:0/4115626590 >> 172.21.15.167:0/4115626590 conn(0x7f27a00cd8f0 msgr2=0x7f27a00b5d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:42.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.194+0000 7f27a4c00640 1 -- 172.21.15.167:0/4115626590 shutdown_connections 2024-05-11T12:10:42.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.194+0000 7f27a4c00640 1 -- 172.21.15.167:0/4115626590 wait complete. 2024-05-11T12:10:42.211 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-11T12:10:42.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.405+0000 7ff091200640 1 Processor -- start 2024-05-11T12:10:42.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.405+0000 7ff091200640 1 -- start start 2024-05-11T12:10:42.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.406+0000 7ff091200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff08c13eeb0 0x7ff08c06f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:42.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.407+0000 7ff091200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff08c0b50a0 0x7ff08c073cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:42.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.407+0000 7ff091200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff08c15fb90 con 0x7ff08c13f290 2024-05-11T12:10:42.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.407+0000 7ff091200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7ff08c15d5b0 con 0x7ff08c13eeb0 2024-05-11T12:10:42.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.407+0000 7ff091200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff08c15fe70 con 0x7ff08c0b50a0 2024-05-11T12:10:42.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.407+0000 7ff08a800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff08c0b50a0 0x7ff08c073cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:42.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.407+0000 7ff089e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff08c13eeb0 0x7ff08c06f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:42.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.407+0000 7ff089400640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7ff08c13f290 0x7ff08c0735c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.167:48184/0 (socket says 172.21.15.167:48184) 2024-05-11T12:10:42.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.407+0000 7ff089400640 1 -- 172.21.15.167:0/3162238392 learned_addr learned my addr 172.21.15.167:0/3162238392 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:10:42.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.408+0000 7ff089e00640 1 -- 172.21.15.167:0/3162238392 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff08c0b50a0 msgr2=0x7ff08c073cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:42.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.408+0000 7ff089e00640 1 --2- 172.21.15.167:0/3162238392 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff08c0b50a0 0x7ff08c073cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:42.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.408+0000 7ff089e00640 1 -- 172.21.15.167:0/3162238392 >> v1:172.21.15.204:6789/0 conn(0x7ff08c13f290 legacy=0x7ff08c0735c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:42.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.408+0000 7ff089e00640 1 -- 172.21.15.167:0/3162238392 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff08c160150 con 0x7ff08c13eeb0 2024-05-11T12:10:42.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.409+0000 7ff089e00640 1 --2- 172.21.15.167:0/3162238392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff08c13eeb0 0x7ff08c06f480 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff07c0097a0 tx=0x7ff07c02c310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bb8d082f620a323 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:42.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.410+0000 7ff088a00640 1 -- 172.21.15.167:0/3162238392 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c039070 con 0x7ff08c13eeb0 2024-05-11T12:10:42.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.410+0000 7ff088a00640 1 -- 172.21.15.167:0/3162238392 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff07c034640 con 0x7ff08c13eeb0 2024-05-11T12:10:42.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.410+0000 7ff088a00640 1 -- 172.21.15.167:0/3162238392 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c02ce60 con 0x7ff08c13eeb0 2024-05-11T12:10:42.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.411+0000 7ff091200640 1 -- 172.21.15.167:0/3162238392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff08c13eeb0 msgr2=0x7ff08c06f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:42.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.411+0000 7ff091200640 1 --2- 172.21.15.167:0/3162238392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff08c13eeb0 0x7ff08c06f480 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff07c0097a0 tx=0x7ff07c02c310 comp rx=0 tx=0).stop 2024-05-11T12:10:42.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.411+0000 7ff091200640 1 -- 172.21.15.167:0/3162238392 shutdown_connections 2024-05-11T12:10:42.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.411+0000 7ff091200640 1 --2- 172.21.15.167:0/3162238392 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff08c0b50a0 0x7ff08c073cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:42.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.411+0000 7ff091200640 1 --2- 172.21.15.167:0/3162238392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff08c13eeb0 0x7ff08c06f480 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:42.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.411+0000 7ff091200640 1 -- 172.21.15.167:0/3162238392 >> 172.21.15.167:0/3162238392 conn(0x7ff08c0cd8f0 msgr2=0x7ff08c0cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:42.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.411+0000 7ff091200640 1 -- 172.21.15.167:0/3162238392 shutdown_connections 2024-05-11T12:10:42.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.411+0000 7ff091200640 1 -- 172.21.15.167:0/3162238392 wait complete. 2024-05-11T12:10:42.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.413+0000 7ff091200640 1 Processor -- start 2024-05-11T12:10:42.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.413+0000 7ff091200640 1 -- start start 2024-05-11T12:10:42.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.414+0000 7ff091200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff08c0b50a0 0x7ff08c13bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:42.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.414+0000 7ff091200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff08c13eeb0 0x7ff08c1f8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:42.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.414+0000 7ff091200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff08c13f290 0x7ff08c0b1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:42.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.414+0000 7ff091200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff08c06f9c0 con 0x7ff08c13eeb0 2024-05-11T12:10:42.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.414+0000 7ff091200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff08c160710 con 0x7ff08c13f290 2024-05-11T12:10:42.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.414+0000 7ff091200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff08c06fb40 con 0x7ff08c0b50a0 2024-05-11T12:10:42.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.414+0000 7ff089400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff08c13eeb0 0x7ff08c1f8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:42.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.414+0000 7ff089400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff08c13eeb0 0x7ff08c1f8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58920/0 (socket says 172.21.15.167:58920) 2024-05-11T12:10:42.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.414+0000 7ff089400640 1 -- 172.21.15.167:0/1622266120 learned_addr learned my addr 172.21.15.167:0/1622266120 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:10:42.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.414+0000 7ff08a800640 1 --2- 172.21.15.167:0/1622266120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff08c13f290 0x7ff08c0b1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:42.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.414+0000 7ff089e00640 1 --2- 172.21.15.167:0/1622266120 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff08c0b50a0 0x7ff08c13bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:42.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.415+0000 7ff089400640 1 -- 172.21.15.167:0/1622266120 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff08c0b50a0 msgr2=0x7ff08c13bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:42.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.415+0000 7ff089400640 1 --2- 172.21.15.167:0/1622266120 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff08c0b50a0 0x7ff08c13bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:42.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.415+0000 7ff089400640 1 -- 172.21.15.167:0/1622266120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff08c13f290 msgr2=0x7ff08c0b1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:42.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.415+0000 7ff089400640 1 --2- 172.21.15.167:0/1622266120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff08c13f290 0x7ff08c0b1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:42.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.415+0000 7ff089400640 1 -- 172.21.15.167:0/1622266120 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff08c0d04c0 con 0x7ff08c13eeb0 2024-05-11T12:10:42.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.415+0000 7ff089400640 1 --2- 172.21.15.167:0/1622266120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff08c13eeb0 0x7ff08c1f8fa0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff07400cae0 tx=0x7ff07400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:42.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.415+0000 7ff082000640 1 -- 172.21.15.167:0/1622266120 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff074017070 con 0x7ff08c13eeb0 2024-05-11T12:10:42.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.415+0000 7ff082000640 1 -- 172.21.15.167:0/1622266120 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff07400ac60 con 0x7ff08c13eeb0 2024-05-11T12:10:42.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.415+0000 7ff082000640 1 -- 172.21.15.167:0/1622266120 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff074012640 con 0x7ff08c13eeb0 2024-05-11T12:10:42.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.415+0000 7ff091200640 1 -- 172.21.15.167:0/1622266120 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff08c0d0a10 con 0x7ff08c13eeb0 2024-05-11T12:10:42.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.415+0000 7ff091200640 1 -- 172.21.15.167:0/1622266120 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff08c0b0b30 con 0x7ff08c13eeb0 2024-05-11T12:10:42.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.416+0000 7ff091200640 1 -- 172.21.15.167:0/1622266120 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff08c0b28a0 con 0x7ff08c13eeb0 2024-05-11T12:10:42.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.417+0000 7ff082000640 1 -- 172.21.15.167:0/1622266120 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff07401a070 con 0x7ff08c13eeb0 2024-05-11T12:10:42.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.419+0000 7ff082000640 1 --2- 172.21.15.167:0/1622266120 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff0680762d0 0x7ff068078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:42.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.419+0000 7ff082000640 1 -- 172.21.15.167:0/1622266120 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7ff074096fe0 con 0x7ff08c13eeb0 2024-05-11T12:10:42.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.419+0000 7ff089e00640 1 --2- 172.21.15.167:0/1622266120 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff0680762d0 0x7ff068078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:42.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.420+0000 7ff089e00640 1 --2- 172.21.15.167:0/1622266120 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff0680762d0 0x7ff068078790 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff07c036000 tx=0x7ff07c030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:42.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.424+0000 7ff082000640 1 -- 172.21.15.167:0/1622266120 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff074060ce0 con 0x7ff08c13eeb0 2024-05-11T12:10:42.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.632+0000 7ff091200640 1 -- 172.21.15.167:0/1622266120 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ff08c154260 con 0x7ff08c13eeb0 2024-05-11T12:10:42.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.633+0000 7ff082000640 1 -- 172.21.15.167:0/1622266120 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) ==== 84+0+65 (secure 0 0 0) 0x7ff0740605e0 con 0x7ff08c13eeb0 2024-05-11T12:10:42.632 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:10:42.633 INFO:teuthology.orchestra.run.smithi167.stdout:[ 2024-05-11T12:10:42.633 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:10:42.633 INFO:teuthology.orchestra.run.smithi167.stdout: "poolnum": 1, 2024-05-11T12:10:42.633 INFO:teuthology.orchestra.run.smithi167.stdout: "poolname": ".mgr" 2024-05-11T12:10:42.633 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:10:42.633 INFO:teuthology.orchestra.run.smithi167.stdout:] 2024-05-11T12:10:42.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.636+0000 7ff091200640 1 -- 172.21.15.167:0/1622266120 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff0680762d0 msgr2=0x7ff068078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:42.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.636+0000 7ff091200640 1 --2- 172.21.15.167:0/1622266120 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff0680762d0 0x7ff068078790 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff07c036000 tx=0x7ff07c030000 comp rx=0 tx=0).stop 2024-05-11T12:10:42.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.636+0000 7ff091200640 1 -- 172.21.15.167:0/1622266120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff08c13eeb0 msgr2=0x7ff08c1f8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:42.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.636+0000 7ff091200640 1 --2- 172.21.15.167:0/1622266120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff08c13eeb0 0x7ff08c1f8fa0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff07400cae0 tx=0x7ff07400a060 comp rx=0 tx=0).stop 2024-05-11T12:10:42.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.636+0000 7ff091200640 1 -- 172.21.15.167:0/1622266120 shutdown_connections 2024-05-11T12:10:42.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.636+0000 7ff091200640 1 --2- 172.21.15.167:0/1622266120 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff0680762d0 0x7ff068078790 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:42.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.636+0000 7ff091200640 1 --2- 172.21.15.167:0/1622266120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff08c13f290 0x7ff08c0b1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:42.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.636+0000 7ff091200640 1 --2- 172.21.15.167:0/1622266120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff08c13eeb0 0x7ff08c1f8fa0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:42.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.636+0000 7ff091200640 1 --2- 172.21.15.167:0/1622266120 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff08c0b50a0 0x7ff08c13bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:42.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.636+0000 7ff091200640 1 -- 172.21.15.167:0/1622266120 >> 172.21.15.167:0/1622266120 conn(0x7ff08c0cd8f0 msgr2=0x7ff08c06eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:42.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.636+0000 7ff091200640 1 -- 172.21.15.167:0/1622266120 shutdown_connections 2024-05-11T12:10:42.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.636+0000 7ff091200640 1 -- 172.21.15.167:0/1622266120 wait complete. 2024-05-11T12:10:42.654 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-11T12:10:42.654 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-11T12:10:42.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.847+0000 7fe49e400640 1 Processor -- start 2024-05-11T12:10:42.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.847+0000 7fe49e400640 1 -- start start 2024-05-11T12:10:42.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.849+0000 7fe49e400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe49815cda0 0x7fe4980b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:42.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.849+0000 7fe49e400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe498070140 0x7fe4980b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:42.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.849+0000 7fe49e400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe49815f970 con 0x7fe4980b5eb0 2024-05-11T12:10:42.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.849+0000 7fe49e400640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fe49815fc50 con 0x7fe498070140 2024-05-11T12:10:42.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.849+0000 7fe49e400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe49815ff30 con 0x7fe49815cda0 2024-05-11T12:10:42.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.849+0000 7fe497e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe498070140 0x7fe4980b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:42.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.849+0000 7fe497e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe498070140 0x7fe4980b3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58936/0 (socket says 172.21.15.167:58936) 2024-05-11T12:10:42.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.849+0000 7fe497e00640 1 -- 172.21.15.167:0/1238396221 learned_addr learned my addr 172.21.15.167:0/1238396221 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:10:42.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.850+0000 7fe497e00640 1 -- 172.21.15.167:0/1238396221 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe49815cda0 msgr2=0x7fe4980b5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:42.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.850+0000 7fe497e00640 1 --2- 172.21.15.167:0/1238396221 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe49815cda0 0x7fe4980b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:42.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.850+0000 7fe497e00640 1 -- 172.21.15.167:0/1238396221 >> v1:172.21.15.204:6789/0 conn(0x7fe4980b5eb0 legacy=0x7fe49806fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:42.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.850+0000 7fe497e00640 1 -- 172.21.15.167:0/1238396221 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe498160210 con 0x7fe498070140 2024-05-11T12:10:42.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.851+0000 7fe497e00640 1 --2- 172.21.15.167:0/1238396221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe498070140 0x7fe4980b3de0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe490009e50 tx=0x7fe49002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=509e6f0c60a0d868 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:42.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.851+0000 7fe496a00640 1 -- 172.21.15.167:0/1238396221 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe490036070 con 0x7fe498070140 2024-05-11T12:10:42.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.851+0000 7fe496a00640 1 -- 172.21.15.167:0/1238396221 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe490031600 con 0x7fe498070140 2024-05-11T12:10:42.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.851+0000 7fe496a00640 1 -- 172.21.15.167:0/1238396221 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4900319d0 con 0x7fe498070140 2024-05-11T12:10:42.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.852+0000 7fe49e400640 1 -- 172.21.15.167:0/1238396221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe498070140 msgr2=0x7fe4980b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:42.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.852+0000 7fe49e400640 1 --2- 172.21.15.167:0/1238396221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe498070140 0x7fe4980b3de0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe490009e50 tx=0x7fe49002a680 comp rx=0 tx=0).stop 2024-05-11T12:10:42.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.852+0000 7fe49e400640 1 -- 172.21.15.167:0/1238396221 shutdown_connections 2024-05-11T12:10:42.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.852+0000 7fe49e400640 1 --2- 172.21.15.167:0/1238396221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe498070140 0x7fe4980b3de0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:42.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.852+0000 7fe49e400640 1 --2- 172.21.15.167:0/1238396221 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe49815cda0 0x7fe4980b5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:42.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.852+0000 7fe49e400640 1 -- 172.21.15.167:0/1238396221 >> 172.21.15.167:0/1238396221 conn(0x7fe4980d0c70 msgr2=0x7fe4980d1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:42.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.852+0000 7fe49e400640 1 -- 172.21.15.167:0/1238396221 shutdown_connections 2024-05-11T12:10:42.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.852+0000 7fe49e400640 1 -- 172.21.15.167:0/1238396221 wait complete. 2024-05-11T12:10:42.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.854+0000 7fe49e400640 1 Processor -- start 2024-05-11T12:10:42.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.854+0000 7fe49e400640 1 -- start start 2024-05-11T12:10:42.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.855+0000 7fe49e400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe498070140 0x7fe49814ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:42.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.855+0000 7fe49e400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4980b5eb0 0x7fe49815e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:42.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.855+0000 7fe49e400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe49815cda0 0x7fe49806cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:42.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.855+0000 7fe49e400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe4980af6c0 con 0x7fe4980b5eb0 2024-05-11T12:10:42.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.855+0000 7fe49e400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe49815d180 con 0x7fe49815cda0 2024-05-11T12:10:42.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.855+0000 7fe49e400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe4980b4320 con 0x7fe498070140 2024-05-11T12:10:42.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.856+0000 7fe48fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4980b5eb0 0x7fe49815e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:42.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.856+0000 7fe48fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4980b5eb0 0x7fe49815e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58944/0 (socket says 172.21.15.167:58944) 2024-05-11T12:10:42.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.856+0000 7fe48fe00640 1 -- 172.21.15.167:0/3808280231 learned_addr learned my addr 172.21.15.167:0/3808280231 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:10:42.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.856+0000 7fe497400640 1 --2- 172.21.15.167:0/3808280231 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe498070140 0x7fe49814ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:42.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.856+0000 7fe497e00640 1 --2- 172.21.15.167:0/3808280231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe49815cda0 0x7fe49806cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:42.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.856+0000 7fe48fe00640 1 -- 172.21.15.167:0/3808280231 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe498070140 msgr2=0x7fe49814ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:42.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.856+0000 7fe48fe00640 1 --2- 172.21.15.167:0/3808280231 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe498070140 0x7fe49814ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:42.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.856+0000 7fe48fe00640 1 -- 172.21.15.167:0/3808280231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe49815cda0 msgr2=0x7fe49806cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:42.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.856+0000 7fe48fe00640 1 --2- 172.21.15.167:0/3808280231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe49815cda0 0x7fe49806cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:42.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.856+0000 7fe48fe00640 1 -- 172.21.15.167:0/3808280231 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4981f8a80 con 0x7fe4980b5eb0 2024-05-11T12:10:42.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.856+0000 7fe497e00640 1 --2- 172.21.15.167:0/3808280231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe49815cda0 0x7fe49806cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:10:42.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.856+0000 7fe48fe00640 1 --2- 172.21.15.167:0/3808280231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4980b5eb0 0x7fe49815e9f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe48000c430 tx=0x7fe48000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:42.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.857+0000 7fe48f400640 1 -- 172.21.15.167:0/3808280231 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe480011790 con 0x7fe4980b5eb0 2024-05-11T12:10:42.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.857+0000 7fe48f400640 1 -- 172.21.15.167:0/3808280231 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe480017070 con 0x7fe4980b5eb0 2024-05-11T12:10:42.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.857+0000 7fe49e400640 1 -- 172.21.15.167:0/3808280231 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4981f8d60 con 0x7fe4980b5eb0 2024-05-11T12:10:42.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.857+0000 7fe48f400640 1 -- 172.21.15.167:0/3808280231 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe480011b70 con 0x7fe4980b5eb0 2024-05-11T12:10:42.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.857+0000 7fe49e400640 1 -- 172.21.15.167:0/3808280231 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4981f9320 con 0x7fe4980b5eb0 2024-05-11T12:10:42.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.858+0000 7fe49e400640 1 -- 172.21.15.167:0/3808280231 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4981f9520 con 0x7fe4980b5eb0 2024-05-11T12:10:42.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.859+0000 7fe48f400640 1 -- 172.21.15.167:0/3808280231 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe480011d10 con 0x7fe4980b5eb0 2024-05-11T12:10:42.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.860+0000 7fe48f400640 1 --2- 172.21.15.167:0/3808280231 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe4740762d0 0x7fe474078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:42.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.860+0000 7fe48f400640 1 -- 172.21.15.167:0/3808280231 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7fe480097090 con 0x7fe4980b5eb0 2024-05-11T12:10:42.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.860+0000 7fe497400640 1 --2- 172.21.15.167:0/3808280231 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe4740762d0 0x7fe474078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:42.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.861+0000 7fe497400640 1 --2- 172.21.15.167:0/3808280231 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe4740762d0 0x7fe474078790 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe48800b810 tx=0x7fe488008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:42.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:42.865+0000 7fe48f400640 1 -- 172.21.15.167:0/3808280231 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe4981f9520 con 0x7fe4980b5eb0 2024-05-11T12:10:43.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.070+0000 7fe49e400640 1 -- 172.21.15.167:0/3808280231 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fe498069590 con 0x7fe4980b5eb0 2024-05-11T12:10:43.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.717+0000 7fe48f400640 1 -- 172.21.15.167:0/3808280231 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v20) ==== 138+0+0 (secure 0 0 0) 0x7fe480060dd0 con 0x7fe4980b5eb0 2024-05-11T12:10:43.717 INFO:teuthology.orchestra.run.smithi167.stderr:pool 'cephfs_metadata' created 2024-05-11T12:10:43.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.721+0000 7fe49e400640 1 -- 172.21.15.167:0/3808280231 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe4740762d0 msgr2=0x7fe474078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:43.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.721+0000 7fe49e400640 1 --2- 172.21.15.167:0/3808280231 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe4740762d0 0x7fe474078790 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe48800b810 tx=0x7fe488008040 comp rx=0 tx=0).stop 2024-05-11T12:10:43.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.721+0000 7fe49e400640 1 -- 172.21.15.167:0/3808280231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4980b5eb0 msgr2=0x7fe49815e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:43.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.721+0000 7fe49e400640 1 --2- 172.21.15.167:0/3808280231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4980b5eb0 0x7fe49815e9f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe48000c430 tx=0x7fe48000c8f0 comp rx=0 tx=0).stop 2024-05-11T12:10:43.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.722+0000 7fe49e400640 1 -- 172.21.15.167:0/3808280231 shutdown_connections 2024-05-11T12:10:43.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.722+0000 7fe49e400640 1 --2- 172.21.15.167:0/3808280231 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe4740762d0 0x7fe474078790 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:43.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.722+0000 7fe49e400640 1 --2- 172.21.15.167:0/3808280231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe49815cda0 0x7fe49806cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:43.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.722+0000 7fe49e400640 1 --2- 172.21.15.167:0/3808280231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4980b5eb0 0x7fe49815e9f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:43.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.722+0000 7fe49e400640 1 --2- 172.21.15.167:0/3808280231 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe498070140 0x7fe49814ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:43.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.722+0000 7fe49e400640 1 -- 172.21.15.167:0/3808280231 >> 172.21.15.167:0/3808280231 conn(0x7fe4980d0c70 msgr2=0x7fe4980cdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:43.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.723+0000 7fe49e400640 1 -- 172.21.15.167:0/3808280231 shutdown_connections 2024-05-11T12:10:43.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.723+0000 7fe49e400640 1 -- 172.21.15.167:0/3808280231 wait complete. 2024-05-11T12:10:43.748 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-05-11T12:10:43.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.921+0000 7f3d66a00640 1 Processor -- start 2024-05-11T12:10:43.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.921+0000 7f3d66a00640 1 -- start start 2024-05-11T12:10:43.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.923+0000 7f3d66a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3d6015f230 0x7f3d6006c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:43.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.923+0000 7f3d66a00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3d6006d6c0 0x7f3d600d2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:43.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.923+0000 7f3d66a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d600d0c40 con 0x7f3d6015f610 2024-05-11T12:10:43.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.923+0000 7f3d66a00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f3d600d1850 con 0x7f3d6006d6c0 2024-05-11T12:10:43.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.923+0000 7f3d66a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f3d600d0f20 con 0x7f3d6015f230 2024-05-11T12:10:43.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.923+0000 7f3d64a00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3d6006d6c0 0x7f3d600d2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:43.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.923+0000 7f3d64a00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3d6006d6c0 0x7f3d600d2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58950/0 (socket says 172.21.15.167:58950) 2024-05-11T12:10:43.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.923+0000 7f3d64a00640 1 -- 172.21.15.167:0/2425575350 learned_addr learned my addr 172.21.15.167:0/2425575350 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:10:43.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.924+0000 7f3d64a00640 1 -- 172.21.15.167:0/2425575350 >> v2:172.21.15.204:3300/0 conn(0x7f3d6015f230 msgr2=0x7f3d6006c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:43.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.924+0000 7f3d64a00640 1 --2- 172.21.15.167:0/2425575350 >> v2:172.21.15.204:3300/0 conn(0x7f3d6015f230 0x7f3d6006c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:43.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.924+0000 7f3d64a00640 1 -- 172.21.15.167:0/2425575350 >> v1:172.21.15.204:6789/0 conn(0x7f3d6015f610 legacy=0x7f3d6006cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:43.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.924+0000 7f3d64a00640 1 -- 172.21.15.167:0/2425575350 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d600d1200 con 0x7f3d6006d6c0 2024-05-11T12:10:43.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.925+0000 7f3d64a00640 1 --2- 172.21.15.167:0/2425575350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d6006d6c0 0x7f3d600d2f50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3d600d5020 tx=0x7f3d5802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=542c05f9dbca0534 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:43.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.925+0000 7f3d5ea00640 1 -- 172.21.15.167:0/2425575350 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d58037070 con 0x7f3d6006d6c0 2024-05-11T12:10:43.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.925+0000 7f3d5ea00640 1 -- 172.21.15.167:0/2425575350 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d58032440 con 0x7f3d6006d6c0 2024-05-11T12:10:43.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.925+0000 7f3d5ea00640 1 -- 172.21.15.167:0/2425575350 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d58032810 con 0x7f3d6006d6c0 2024-05-11T12:10:43.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.925+0000 7f3d66a00640 1 -- 172.21.15.167:0/2425575350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d6006d6c0 msgr2=0x7f3d600d2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:43.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.925+0000 7f3d66a00640 1 --2- 172.21.15.167:0/2425575350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d6006d6c0 0x7f3d600d2f50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3d600d5020 tx=0x7f3d5802a1e0 comp rx=0 tx=0).stop 2024-05-11T12:10:43.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.926+0000 7f3d66a00640 1 -- 172.21.15.167:0/2425575350 shutdown_connections 2024-05-11T12:10:43.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.926+0000 7f3d66a00640 1 --2- 172.21.15.167:0/2425575350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d6006d6c0 0x7f3d600d2f50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:43.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.926+0000 7f3d66a00640 1 --2- 172.21.15.167:0/2425575350 >> v2:172.21.15.204:3300/0 conn(0x7f3d6015f230 0x7f3d6006c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:43.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.926+0000 7f3d66a00640 1 -- 172.21.15.167:0/2425575350 >> 172.21.15.167:0/2425575350 conn(0x7f3d600d5140 msgr2=0x7f3d600d5550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:43.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.926+0000 7f3d66a00640 1 -- 172.21.15.167:0/2425575350 shutdown_connections 2024-05-11T12:10:43.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.926+0000 7f3d66a00640 1 -- 172.21.15.167:0/2425575350 wait complete. 2024-05-11T12:10:43.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.927+0000 7f3d66a00640 1 Processor -- start 2024-05-11T12:10:43.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.927+0000 7f3d66a00640 1 -- start start 2024-05-11T12:10:43.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.928+0000 7f3d66a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d6006d6c0 0x7f3d600c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:43.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.928+0000 7f3d66a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d6015f230 0x7f3d600cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:43.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.928+0000 7f3d66a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d6015f610 0x7f3d600c9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:43.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.928+0000 7f3d66a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6006dcb0 con 0x7f3d6015f230 2024-05-11T12:10:43.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.928+0000 7f3d66a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6006db30 con 0x7f3d6006d6c0 2024-05-11T12:10:43.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.928+0000 7f3d66a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3d6006cba0 con 0x7f3d6015f610 2024-05-11T12:10:43.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.929+0000 7f3d5f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d6015f230 0x7f3d600cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:43.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.929+0000 7f3d5f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d6015f230 0x7f3d600cb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58960/0 (socket says 172.21.15.167:58960) 2024-05-11T12:10:43.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.929+0000 7f3d5f400640 1 -- 172.21.15.167:0/2768164982 learned_addr learned my addr 172.21.15.167:0/2768164982 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:10:43.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.929+0000 7f3d5fe00640 1 --2- 172.21.15.167:0/2768164982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d6006d6c0 0x7f3d600c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:43.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.929+0000 7f3d5f400640 1 -- 172.21.15.167:0/2768164982 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d6015f610 msgr2=0x7f3d600c9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:43.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.929+0000 7f3d5f400640 1 --2- 172.21.15.167:0/2768164982 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d6015f610 0x7f3d600c9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:43.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.929+0000 7f3d5f400640 1 -- 172.21.15.167:0/2768164982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d6006d6c0 msgr2=0x7f3d600c8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:43.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.929+0000 7f3d5f400640 1 --2- 172.21.15.167:0/2768164982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d6006d6c0 0x7f3d600c8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:43.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.929+0000 7f3d5f400640 1 -- 172.21.15.167:0/2768164982 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d600c9250 con 0x7f3d6015f230 2024-05-11T12:10:43.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.929+0000 7f3d5f400640 1 --2- 172.21.15.167:0/2768164982 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d6015f230 0x7f3d600cb9e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c00c420 tx=0x7f3d4c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:43.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.929+0000 7f3d57e00640 1 -- 172.21.15.167:0/2768164982 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c017070 con 0x7f3d6015f230 2024-05-11T12:10:43.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.930+0000 7f3d57e00640 1 -- 172.21.15.167:0/2768164982 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d4c003640 con 0x7f3d6015f230 2024-05-11T12:10:43.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.930+0000 7f3d57e00640 1 -- 172.21.15.167:0/2768164982 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c003a50 con 0x7f3d6015f230 2024-05-11T12:10:43.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.930+0000 7f3d66a00640 1 -- 172.21.15.167:0/2768164982 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d601f8e30 con 0x7f3d6015f230 2024-05-11T12:10:43.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.930+0000 7f3d66a00640 1 -- 172.21.15.167:0/2768164982 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d601f93f0 con 0x7f3d6015f230 2024-05-11T12:10:43.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.931+0000 7f3d66a00640 1 -- 172.21.15.167:0/2768164982 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d60069630 con 0x7f3d6015f230 2024-05-11T12:10:43.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.932+0000 7f3d57e00640 1 -- 172.21.15.167:0/2768164982 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3d4c003bf0 con 0x7f3d6015f230 2024-05-11T12:10:43.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.933+0000 7f3d57e00640 1 --2- 172.21.15.167:0/2768164982 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3d400762d0 0x7f3d40078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:43.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.933+0000 7f3d57e00640 1 -- 172.21.15.167:0/2768164982 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5227+0+0 (secure 0 0 0) 0x7f3d4c0966a0 con 0x7f3d6015f230 2024-05-11T12:10:43.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.933+0000 7f3d5fe00640 1 --2- 172.21.15.167:0/2768164982 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3d400762d0 0x7f3d40078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:43.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.934+0000 7f3d5fe00640 1 --2- 172.21.15.167:0/2768164982 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3d400762d0 0x7f3d40078790 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3d50007dc0 tx=0x7f3d500072f0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:43.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:43.938+0000 7f3d57e00640 1 -- 172.21.15.167:0/2768164982 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3d4c05faf0 con 0x7f3d6015f230 2024-05-11T12:10:44.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.142+0000 7f3d66a00640 1 -- 172.21.15.167:0/2768164982 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f3d600bb010 con 0x7f3d6015f230 2024-05-11T12:10:44.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.735+0000 7f3d57e00640 1 -- 172.21.15.167:0/2768164982 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v21) ==== 171+0+0 (secure 0 0 0) 0x7f3d4c063df0 con 0x7f3d6015f230 2024-05-11T12:10:44.734 INFO:teuthology.orchestra.run.smithi167.stderr:pool 'cephfs_data' created 2024-05-11T12:10:44.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.738+0000 7f3d66a00640 1 -- 172.21.15.167:0/2768164982 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3d400762d0 msgr2=0x7f3d40078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:44.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.738+0000 7f3d66a00640 1 --2- 172.21.15.167:0/2768164982 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3d400762d0 0x7f3d40078790 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3d50007dc0 tx=0x7f3d500072f0 comp rx=0 tx=0).stop 2024-05-11T12:10:44.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.738+0000 7f3d66a00640 1 -- 172.21.15.167:0/2768164982 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d6015f230 msgr2=0x7f3d600cb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:44.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.739+0000 7f3d66a00640 1 --2- 172.21.15.167:0/2768164982 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d6015f230 0x7f3d600cb9e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c00c420 tx=0x7f3d4c00c8e0 comp rx=0 tx=0).stop 2024-05-11T12:10:44.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.740+0000 7f3d66a00640 1 -- 172.21.15.167:0/2768164982 shutdown_connections 2024-05-11T12:10:44.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.740+0000 7f3d66a00640 1 --2- 172.21.15.167:0/2768164982 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d6015f610 0x7f3d600c9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:44.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.740+0000 7f3d66a00640 1 --2- 172.21.15.167:0/2768164982 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d6015f230 0x7f3d600cb9e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:44.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.740+0000 7f3d66a00640 1 --2- 172.21.15.167:0/2768164982 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3d400762d0 0x7f3d40078790 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:44.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.740+0000 7f3d66a00640 1 --2- 172.21.15.167:0/2768164982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d6006d6c0 0x7f3d600c8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:44.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.740+0000 7f3d66a00640 1 -- 172.21.15.167:0/2768164982 >> 172.21.15.167:0/2768164982 conn(0x7f3d600d5140 msgr2=0x7f3d600ba810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:44.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.740+0000 7f3d66a00640 1 -- 172.21.15.167:0/2768164982 shutdown_connections 2024-05-11T12:10:44.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.740+0000 7f3d66a00640 1 -- 172.21.15.167:0/2768164982 wait complete. 2024-05-11T12:10:44.783 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-11T12:10:44.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.958+0000 7f26cd200640 1 Processor -- start 2024-05-11T12:10:44.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.958+0000 7f26cd200640 1 -- start start 2024-05-11T12:10:44.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.960+0000 7f26cd200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26c815ac80 0x7f26c8074260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:44.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.960+0000 7f26cd200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f26c80b3d50 0x7f26c8160c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:44.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.960+0000 7f26cd200640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26c80b65e0 con 0x7f26c81611b0 2024-05-11T12:10:44.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.960+0000 7f26cd200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f26c806e3c0 con 0x7f26c80b3d50 2024-05-11T12:10:44.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.960+0000 7f26cd200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f26c815b060 con 0x7f26c815ac80 2024-05-11T12:10:44.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.960+0000 7f26c6800640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f26c81611b0 0x7f26c815d680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:51332/0 (socket says 172.21.15.167:51332) 2024-05-11T12:10:44.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.960+0000 7f26c6800640 1 -- 172.21.15.167:0/3955212776 learned_addr learned my addr 172.21.15.167:0/3955212776 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:10:44.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.961+0000 7f26c5e00640 1 --2- 172.21.15.167:0/3955212776 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26c815ac80 0x7f26c8074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:44.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.961+0000 7f26c5400640 1 --2- 172.21.15.167:0/3955212776 >> v2:172.21.15.204:3300/0 conn(0x7f26c80b3d50 0x7f26c8160c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:44.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.961+0000 7f26c4a00640 1 -- 172.21.15.167:0/3955212776 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3909056947 0 0) 0x7f26c80b65e0 con 0x7f26c81611b0 2024-05-11T12:10:44.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.961+0000 7f26c4a00640 1 -- 172.21.15.167:0/3955212776 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f26ac003520 con 0x7f26c81611b0 2024-05-11T12:10:44.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.961+0000 7f26c4a00640 1 -- 172.21.15.167:0/3955212776 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 831068665 0 0) 0x7f26ac003520 con 0x7f26c81611b0 2024-05-11T12:10:44.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.961+0000 7f26c4a00640 1 -- 172.21.15.167:0/3955212776 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26c815ac80 msgr2=0x7f26c8074260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:44.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.961+0000 7f26c4a00640 1 --2- 172.21.15.167:0/3955212776 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26c815ac80 0x7f26c8074260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:44.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.961+0000 7f26c4a00640 1 -- 172.21.15.167:0/3955212776 >> v2:172.21.15.204:3300/0 conn(0x7f26c80b3d50 msgr2=0x7f26c8160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:44.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.961+0000 7f26c4a00640 1 --2- 172.21.15.167:0/3955212776 >> v2:172.21.15.204:3300/0 conn(0x7f26c80b3d50 0x7f26c8160c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:44.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.961+0000 7f26c4a00640 1 -- 172.21.15.167:0/3955212776 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26c80b68c0 con 0x7f26c81611b0 2024-05-11T12:10:44.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.961+0000 7f26c4a00640 1 -- 172.21.15.167:0/3955212776 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f26c0001a70 con 0x7f26c81611b0 2024-05-11T12:10:44.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.961+0000 7f26c4a00640 1 -- 172.21.15.167:0/3955212776 >> v1:172.21.15.167:6789/0 conn(0x7f26c81611b0 legacy=0x7f26c815d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:44.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.961+0000 7f26c4a00640 1 --2- 172.21.15.167:0/3955212776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26ac003a00 0x7f26ac003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:44.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.961+0000 7f26c4a00640 1 --2- 172.21.15.167:0/3955212776 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26c815ac80 0x7f26ac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:44.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.961+0000 7f26c4a00640 1 --2- 172.21.15.167:0/3955212776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26ac004d40 0x7f26ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:44.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.961+0000 7f26c4a00640 1 -- 172.21.15.167:0/3955212776 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f26c815b060 con 0x7f26ac004d40 2024-05-11T12:10:44.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.961+0000 7f26c4a00640 1 -- 172.21.15.167:0/3955212776 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f26ac00b9c0 con 0x7f26ac003a00 2024-05-11T12:10:44.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.961+0000 7f26c4a00640 1 -- 172.21.15.167:0/3955212776 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f26ac00bb70 con 0x7f26c815ac80 2024-05-11T12:10:44.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.962+0000 7f26c6800640 1 --2- 172.21.15.167:0/3955212776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26ac004d40 0x7f26ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:44.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.962+0000 7f26c5400640 1 --2- 172.21.15.167:0/3955212776 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26c815ac80 0x7f26ac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:44.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.962+0000 7f26c5e00640 1 --2- 172.21.15.167:0/3955212776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26ac003a00 0x7f26ac003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:44.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.962+0000 7f26c6800640 1 -- 172.21.15.167:0/3955212776 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26c815ac80 msgr2=0x7f26ac004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:44.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.962+0000 7f26c6800640 1 --2- 172.21.15.167:0/3955212776 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26c815ac80 0x7f26ac004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:44.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.962+0000 7f26c6800640 1 -- 172.21.15.167:0/3955212776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26ac003a00 msgr2=0x7f26ac003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:44.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.962+0000 7f26c6800640 1 --2- 172.21.15.167:0/3955212776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26ac003a00 0x7f26ac003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:44.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.962+0000 7f26c6800640 1 -- 172.21.15.167:0/3955212776 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26ac003520 con 0x7f26ac004d40 2024-05-11T12:10:44.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.962+0000 7f26c5e00640 1 --2- 172.21.15.167:0/3955212776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26ac003a00 0x7f26ac003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:10:44.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.963+0000 7f26c6800640 1 --2- 172.21.15.167:0/3955212776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26ac004d40 0x7f26ac0051c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f26c000a850 tx=0x7f26ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baeb07e64cfe090d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:44.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.963+0000 7f26c4a00640 1 -- 172.21.15.167:0/3955212776 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c000a4a0 con 0x7f26ac004d40 2024-05-11T12:10:44.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.963+0000 7f26c4a00640 1 -- 172.21.15.167:0/3955212776 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26c002f070 con 0x7f26ac004d40 2024-05-11T12:10:44.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.963+0000 7f26c4a00640 1 -- 172.21.15.167:0/3955212776 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c002bc30 con 0x7f26ac004d40 2024-05-11T12:10:44.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.964+0000 7f26cd200640 1 -- 172.21.15.167:0/3955212776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26ac004d40 msgr2=0x7f26ac0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:44.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.964+0000 7f26cd200640 1 --2- 172.21.15.167:0/3955212776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26ac004d40 0x7f26ac0051c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f26c000a850 tx=0x7f26ac005830 comp rx=0 tx=0).stop 2024-05-11T12:10:44.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.964+0000 7f26c6800640 1 -- 172.21.15.167:0/3955212776 reap_dead start 2024-05-11T12:10:44.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.964+0000 7f26cd200640 1 -- 172.21.15.167:0/3955212776 shutdown_connections 2024-05-11T12:10:44.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.964+0000 7f26cd200640 1 -- 172.21.15.167:0/3955212776 >> 172.21.15.167:0/3955212776 conn(0x7f26c806c450 msgr2=0x7f26c806c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:44.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.964+0000 7f26cd200640 1 -- 172.21.15.167:0/3955212776 shutdown_connections 2024-05-11T12:10:44.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.964+0000 7f26cd200640 1 -- 172.21.15.167:0/3955212776 wait complete. 2024-05-11T12:10:44.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.966+0000 7f26cd200640 1 Processor -- start 2024-05-11T12:10:44.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.966+0000 7f26cd200640 1 -- start start 2024-05-11T12:10:44.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.967+0000 7f26cd200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26c8159780 0x7f26c80c66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:44.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.967+0000 7f26cd200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26c80c6e80 0x7f26c80cc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:44.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.967+0000 7f26cd200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26c80ccf20 0x7f26c81f8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:44.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.967+0000 7f26cd200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f26c80b44f0 con 0x7f26c80c6e80 2024-05-11T12:10:44.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.967+0000 7f26cd200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f26c80b41b0 con 0x7f26c80ccf20 2024-05-11T12:10:44.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.967+0000 7f26cd200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f26c8073960 con 0x7f26c8159780 2024-05-11T12:10:44.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.967+0000 7f26c5400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26c80c6e80 0x7f26c80cc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:44.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.967+0000 7f26c5400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26c80c6e80 0x7f26c80cc9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58984/0 (socket says 172.21.15.167:58984) 2024-05-11T12:10:44.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.967+0000 7f26c5400640 1 -- 172.21.15.167:0/1487295256 learned_addr learned my addr 172.21.15.167:0/1487295256 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:10:44.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.968+0000 7f26c5400640 1 -- 172.21.15.167:0/1487295256 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26c8159780 msgr2=0x7f26c80c66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:44.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.968+0000 7f26c5400640 1 --2- 172.21.15.167:0/1487295256 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26c8159780 0x7f26c80c66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:44.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.968+0000 7f26c5400640 1 -- 172.21.15.167:0/1487295256 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26c80ccf20 msgr2=0x7f26c81f8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:44.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.968+0000 7f26c5400640 1 --2- 172.21.15.167:0/1487295256 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26c80ccf20 0x7f26c81f8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:44.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.968+0000 7f26c5400640 1 -- 172.21.15.167:0/1487295256 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26c80c7ec0 con 0x7f26c80c6e80 2024-05-11T12:10:44.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.968+0000 7f26c5400640 1 --2- 172.21.15.167:0/1487295256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26c80c6e80 0x7f26c80cc9e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f26b00116d0 tx=0x7f26b0011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:44.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.968+0000 7f26be000640 1 -- 172.21.15.167:0/1487295256 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b000f940 con 0x7f26c80c6e80 2024-05-11T12:10:44.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.968+0000 7f26be000640 1 -- 172.21.15.167:0/1487295256 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26b000fae0 con 0x7f26c80c6e80 2024-05-11T12:10:44.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.968+0000 7f26be000640 1 -- 172.21.15.167:0/1487295256 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b000fe90 con 0x7f26c80c6e80 2024-05-11T12:10:44.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.968+0000 7f26cd200640 1 -- 172.21.15.167:0/1487295256 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26c80c7340 con 0x7f26c80c6e80 2024-05-11T12:10:44.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.968+0000 7f26cd200640 1 -- 172.21.15.167:0/1487295256 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26c80c7900 con 0x7f26c80c6e80 2024-05-11T12:10:44.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.969+0000 7f26cd200640 1 -- 172.21.15.167:0/1487295256 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26c8069520 con 0x7f26c80c6e80 2024-05-11T12:10:44.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.971+0000 7f26be000640 1 -- 172.21.15.167:0/1487295256 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f26b0015460 con 0x7f26c80c6e80 2024-05-11T12:10:44.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.972+0000 7f26be000640 1 --2- 172.21.15.167:0/1487295256 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f26a4076470 0x7f26a4078930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:44.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.972+0000 7f26be000640 1 -- 172.21.15.167:0/1487295256 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5609+0+0 (secure 0 0 0) 0x7f26b0097fa0 con 0x7f26c80c6e80 2024-05-11T12:10:44.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.972+0000 7f26c5e00640 1 --2- 172.21.15.167:0/1487295256 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f26a4076470 0x7f26a4078930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:44.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.973+0000 7f26c5e00640 1 --2- 172.21.15.167:0/1487295256 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f26a4076470 0x7f26a4078930 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f26b8009a00 tx=0x7f26b8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:44.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:44.976+0000 7f26be000640 1 -- 172.21.15.167:0/1487295256 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f26b0019030 con 0x7f26c80c6e80 2024-05-11T12:10:45.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.224+0000 7f26cd200640 1 -- 172.21.15.167:0/1487295256 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f26c8070120 con 0x7f26c80c6e80 2024-05-11T12:10:45.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.764+0000 7f26be000640 1 -- 172.21.15.167:0/1487295256 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f26b0061940 con 0x7f26c80c6e80 2024-05-11T12:10:45.770 INFO:teuthology.orchestra.run.smithi167.stderr:new fs with metadata pool 2 and data pool 3 2024-05-11T12:10:45.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.776+0000 7f26cd200640 1 -- 172.21.15.167:0/1487295256 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f26a4076470 msgr2=0x7f26a4078930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:45.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.776+0000 7f26cd200640 1 --2- 172.21.15.167:0/1487295256 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f26a4076470 0x7f26a4078930 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f26b8009a00 tx=0x7f26b8008040 comp rx=0 tx=0).stop 2024-05-11T12:10:45.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.776+0000 7f26cd200640 1 -- 172.21.15.167:0/1487295256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26c80c6e80 msgr2=0x7f26c80cc9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:45.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.776+0000 7f26cd200640 1 --2- 172.21.15.167:0/1487295256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26c80c6e80 0x7f26c80cc9e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f26b00116d0 tx=0x7f26b0011b90 comp rx=0 tx=0).stop 2024-05-11T12:10:45.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.777+0000 7f26cd200640 1 -- 172.21.15.167:0/1487295256 shutdown_connections 2024-05-11T12:10:45.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.777+0000 7f26cd200640 1 --2- 172.21.15.167:0/1487295256 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f26a4076470 0x7f26a4078930 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:45.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.777+0000 7f26cd200640 1 --2- 172.21.15.167:0/1487295256 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26c80ccf20 0x7f26c81f8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:45.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.777+0000 7f26cd200640 1 --2- 172.21.15.167:0/1487295256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26c80c6e80 0x7f26c80cc9e0 secure :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f26b00116d0 tx=0x7f26b0011b90 comp rx=0 tx=0).stop 2024-05-11T12:10:45.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.777+0000 7f26cd200640 1 --2- 172.21.15.167:0/1487295256 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26c8159780 0x7f26c80c66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:45.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.777+0000 7f26cd200640 1 -- 172.21.15.167:0/1487295256 >> 172.21.15.167:0/1487295256 conn(0x7f26c806c450 msgr2=0x7f26c80af630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:45.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.779+0000 7f26cd200640 1 -- 172.21.15.167:0/1487295256 shutdown_connections 2024-05-11T12:10:45.780 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.782+0000 7f26cd200640 1 -- 172.21.15.167:0/1487295256 wait complete. 2024-05-11T12:10:45.808 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-05-11T12:10:45.808 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-05-11T12:10:45.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.988+0000 7ff6ea400640 1 Processor -- start 2024-05-11T12:10:45.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.988+0000 7ff6ea400640 1 -- start start 2024-05-11T12:10:45.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.990+0000 7ff6ea400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6e415f260 0x7ff6e41637e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:45.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.990+0000 7ff6ea400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff6e406d060 0x7ff6e406d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:45.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.990+0000 7ff6ea400640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6e4163d20 con 0x7ff6e40cfd90 2024-05-11T12:10:45.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.990+0000 7ff6ea400640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7ff6e40d3940 con 0x7ff6e406d060 2024-05-11T12:10:45.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.990+0000 7ff6ea400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff6e415f640 con 0x7ff6e415f260 2024-05-11T12:10:45.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.990+0000 7ff6e3e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7ff6e40cfd90 0x7ff6e40d3dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:51348/0 (socket says 172.21.15.167:51348) 2024-05-11T12:10:45.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.990+0000 7ff6e3e00640 1 -- 172.21.15.167:0/1707388038 learned_addr learned my addr 172.21.15.167:0/1707388038 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:10:45.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.990+0000 7ff6e9a00640 1 --2- 172.21.15.167:0/1707388038 >> v2:172.21.15.167:3300/0 conn(0x7ff6e406d060 0x7ff6e406d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:45.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.991+0000 7ff6e9000640 1 --2- 172.21.15.167:0/1707388038 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6e415f260 0x7ff6e41637e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:45.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.991+0000 7ff6e3400640 1 -- 172.21.15.167:0/1707388038 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2523156186 0 0) 0x7ff6cc001110 con 0x7ff6e40cfd90 2024-05-11T12:10:45.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.991+0000 7ff6e3400640 1 -- 172.21.15.167:0/1707388038 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff6c8003520 con 0x7ff6e40cfd90 2024-05-11T12:10:45.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.991+0000 7ff6e9a00640 1 -- 172.21.15.167:0/1707388038 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6e415f260 msgr2=0x7ff6e41637e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:45.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.991+0000 7ff6e9a00640 1 --2- 172.21.15.167:0/1707388038 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6e415f260 0x7ff6e41637e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:45.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.991+0000 7ff6e9a00640 1 -- 172.21.15.167:0/1707388038 >> v1:172.21.15.167:6789/0 conn(0x7ff6e40cfd90 legacy=0x7ff6e40d3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:45.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.991+0000 7ff6e9a00640 1 -- 172.21.15.167:0/1707388038 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6e40d2d30 con 0x7ff6e406d060 2024-05-11T12:10:45.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.992+0000 7ff6e9a00640 1 --2- 172.21.15.167:0/1707388038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6e406d060 0x7ff6e406d4f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc0097a0 tx=0x7ff6dc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a392646331b26297 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:45.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.992+0000 7ff6e3400640 1 -- 172.21.15.167:0/1707388038 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc037070 con 0x7ff6e406d060 2024-05-11T12:10:45.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.992+0000 7ff6e3400640 1 -- 172.21.15.167:0/1707388038 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6dc032440 con 0x7ff6e406d060 2024-05-11T12:10:45.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.992+0000 7ff6e3400640 1 -- 172.21.15.167:0/1707388038 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc032810 con 0x7ff6e406d060 2024-05-11T12:10:45.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.993+0000 7ff6ea400640 1 -- 172.21.15.167:0/1707388038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6e406d060 msgr2=0x7ff6e406d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:45.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.993+0000 7ff6ea400640 1 --2- 172.21.15.167:0/1707388038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6e406d060 0x7ff6e406d4f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc0097a0 tx=0x7ff6dc02a180 comp rx=0 tx=0).stop 2024-05-11T12:10:45.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.993+0000 7ff6ea400640 1 -- 172.21.15.167:0/1707388038 shutdown_connections 2024-05-11T12:10:45.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.993+0000 7ff6ea400640 1 --2- 172.21.15.167:0/1707388038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6e406d060 0x7ff6e406d4f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:45.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.993+0000 7ff6ea400640 1 --2- 172.21.15.167:0/1707388038 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6e415f260 0x7ff6e41637e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:45.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.993+0000 7ff6ea400640 1 -- 172.21.15.167:0/1707388038 >> 172.21.15.167:0/1707388038 conn(0x7ff6e40b6cd0 msgr2=0x7ff6e40b70e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:45.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.993+0000 7ff6ea400640 1 -- 172.21.15.167:0/1707388038 shutdown_connections 2024-05-11T12:10:45.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.993+0000 7ff6ea400640 1 -- 172.21.15.167:0/1707388038 wait complete. 2024-05-11T12:10:45.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.995+0000 7ff6ea400640 1 Processor -- start 2024-05-11T12:10:45.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.995+0000 7ff6ea400640 1 -- start start 2024-05-11T12:10:45.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.996+0000 7ff6ea400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6e406d060 0x7ff6e414f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:45.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.996+0000 7ff6ea400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6e40cfd90 0x7ff6e40cc930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:45.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.996+0000 7ff6ea400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6e415f260 0x7ff6e40cce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:45.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.996+0000 7ff6ea400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e40d35d0 con 0x7ff6e406d060 2024-05-11T12:10:45.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.996+0000 7ff6ea400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e40d32f0 con 0x7ff6e40cfd90 2024-05-11T12:10:45.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.996+0000 7ff6ea400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff6e40d44e0 con 0x7ff6e415f260 2024-05-11T12:10:45.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.996+0000 7ff6e9000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6e406d060 0x7ff6e414f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:45.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.996+0000 7ff6e9000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6e406d060 0x7ff6e414f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58998/0 (socket says 172.21.15.167:58998) 2024-05-11T12:10:45.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.996+0000 7ff6e9000640 1 -- 172.21.15.167:0/2262857189 learned_addr learned my addr 172.21.15.167:0/2262857189 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:10:45.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.997+0000 7ff6e9000640 1 -- 172.21.15.167:0/2262857189 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6e415f260 msgr2=0x7ff6e40cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:45.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.997+0000 7ff6e9000640 1 --2- 172.21.15.167:0/2262857189 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6e415f260 0x7ff6e40cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:45.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.997+0000 7ff6e9000640 1 -- 172.21.15.167:0/2262857189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6e40cfd90 msgr2=0x7ff6e40cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:45.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.997+0000 7ff6e9000640 1 --2- 172.21.15.167:0/2262857189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6e40cfd90 0x7ff6e40cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:45.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.997+0000 7ff6e9000640 1 -- 172.21.15.167:0/2262857189 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6e40ca020 con 0x7ff6e406d060 2024-05-11T12:10:45.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.997+0000 7ff6e9000640 1 --2- 172.21.15.167:0/2262857189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6e406d060 0x7ff6e414f1d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff6d400efc0 tx=0x7ff6d400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:45.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.997+0000 7ff6e0c00640 1 -- 172.21.15.167:0/2262857189 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d4018020 con 0x7ff6e406d060 2024-05-11T12:10:45.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.997+0000 7ff6e0c00640 1 -- 172.21.15.167:0/2262857189 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6d4014480 con 0x7ff6e406d060 2024-05-11T12:10:45.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.997+0000 7ff6e0c00640 1 -- 172.21.15.167:0/2262857189 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d4014850 con 0x7ff6e406d060 2024-05-11T12:10:45.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.997+0000 7ff6ea400640 1 -- 172.21.15.167:0/2262857189 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6e40ca300 con 0x7ff6e406d060 2024-05-11T12:10:45.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.998+0000 7ff6ea400640 1 -- 172.21.15.167:0/2262857189 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6e40ca8c0 con 0x7ff6e406d060 2024-05-11T12:10:45.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:45.999+0000 7ff6ea400640 1 -- 172.21.15.167:0/2262857189 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6e40694b0 con 0x7ff6e406d060 2024-05-11T12:10:45.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:46.000+0000 7ff6e0c00640 1 -- 172.21.15.167:0/2262857189 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff6d4005d10 con 0x7ff6e406d060 2024-05-11T12:10:46.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:46.001+0000 7ff6e0c00640 1 --2- 172.21.15.167:0/2262857189 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff6c00762d0 0x7ff6c0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:46.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:46.001+0000 7ff6e0c00640 1 -- 172.21.15.167:0/2262857189 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7ff6d4098530 con 0x7ff6e406d060 2024-05-11T12:10:46.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:46.002+0000 7ff6e3e00640 1 --2- 172.21.15.167:0/2262857189 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff6c00762d0 0x7ff6c0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:46.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:46.002+0000 7ff6e3e00640 1 --2- 172.21.15.167:0/2262857189 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff6c00762d0 0x7ff6c0078790 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc0013f0 tx=0x7ff6cc001240 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:46.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:46.006+0000 7ff6e0c00640 1 -- 172.21.15.167:0/2262857189 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff6d4022030 con 0x7ff6e406d060 2024-05-11T12:10:46.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:46.226+0000 7ff6ea400640 1 -- 172.21.15.167:0/2262857189 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7ff6e4163870 con 0x7ff6e406d060 2024-05-11T12:10:46.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:46.777+0000 7ff6e0c00640 1 -- 172.21.15.167:0/2262857189 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v23) ==== 157+0+0 (secure 0 0 0) 0x7ff6d4010040 con 0x7ff6e406d060 2024-05-11T12:10:46.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:46.781+0000 7ff6ea400640 1 -- 172.21.15.167:0/2262857189 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff6c00762d0 msgr2=0x7ff6c0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:46.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:46.781+0000 7ff6ea400640 1 --2- 172.21.15.167:0/2262857189 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff6c00762d0 0x7ff6c0078790 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc0013f0 tx=0x7ff6cc001240 comp rx=0 tx=0).stop 2024-05-11T12:10:46.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:46.781+0000 7ff6ea400640 1 -- 172.21.15.167:0/2262857189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6e406d060 msgr2=0x7ff6e414f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:46.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:46.781+0000 7ff6ea400640 1 --2- 172.21.15.167:0/2262857189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6e406d060 0x7ff6e414f1d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff6d400efc0 tx=0x7ff6d400c490 comp rx=0 tx=0).stop 2024-05-11T12:10:46.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:46.781+0000 7ff6ea400640 1 -- 172.21.15.167:0/2262857189 shutdown_connections 2024-05-11T12:10:46.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:46.781+0000 7ff6ea400640 1 --2- 172.21.15.167:0/2262857189 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6e415f260 0x7ff6e40cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:46.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:46.781+0000 7ff6ea400640 1 --2- 172.21.15.167:0/2262857189 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff6c00762d0 0x7ff6c0078790 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:46.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:46.781+0000 7ff6ea400640 1 --2- 172.21.15.167:0/2262857189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6e40cfd90 0x7ff6e40cc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:46.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:46.781+0000 7ff6ea400640 1 --2- 172.21.15.167:0/2262857189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6e406d060 0x7ff6e414f1d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:46.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:46.781+0000 7ff6ea400640 1 -- 172.21.15.167:0/2262857189 >> 172.21.15.167:0/2262857189 conn(0x7ff6e40b6cd0 msgr2=0x7ff6e406d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:46.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:46.782+0000 7ff6ea400640 1 -- 172.21.15.167:0/2262857189 shutdown_connections 2024-05-11T12:10:46.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:46.782+0000 7ff6ea400640 1 -- 172.21.15.167:0/2262857189 wait complete. 2024-05-11T12:10:46.809 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-05-11T12:10:47.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.015+0000 7f9cdf200640 1 Processor -- start 2024-05-11T12:10:47.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.016+0000 7f9cdf200640 1 -- start start 2024-05-11T12:10:47.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.018+0000 7f9cdf200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9ce00cf720 0x7f9ce00723c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:47.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.018+0000 7f9cdf200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ce00d0810 0x7f9ce00cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:47.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.018+0000 7f9cdf200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ce0146310 con 0x7f9ce00d0040 2024-05-11T12:10:47.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.018+0000 7f9cdf200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f9ce0072d10 con 0x7f9ce00cf720 2024-05-11T12:10:47.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.018+0000 7f9cdf200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f9ce0072ff0 con 0x7f9ce00d0810 2024-05-11T12:10:47.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.018+0000 7f9cdd400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ce00d0810 0x7f9ce00cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:47.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.018+0000 7f9cde800640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f9ce00d0040 0x7f9ce006c8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.167:48210/0 (socket says 172.21.15.167:48210) 2024-05-11T12:10:47.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.018+0000 7f9cde800640 1 -- 172.21.15.167:0/437580859 learned_addr learned my addr 172.21.15.167:0/437580859 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:10:47.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.018+0000 7f9cdde00640 1 --2- 172.21.15.167:0/437580859 >> v2:172.21.15.204:3300/0 conn(0x7f9ce00cf720 0x7f9ce00723c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:47.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.019+0000 7f9cdca00640 1 -- 172.21.15.167:0/437580859 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2399805710 0 0) 0x7f9ce0146310 con 0x7f9ce00d0040 2024-05-11T12:10:47.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.019+0000 7f9cdca00640 1 -- 172.21.15.167:0/437580859 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9cc0003520 con 0x7f9ce00d0040 2024-05-11T12:10:47.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.019+0000 7f9cdde00640 1 -- 172.21.15.167:0/437580859 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ce00d0810 msgr2=0x7f9ce00cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:47.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.019+0000 7f9cdde00640 1 --2- 172.21.15.167:0/437580859 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ce00d0810 0x7f9ce00cfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:47.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.019+0000 7f9cdde00640 1 -- 172.21.15.167:0/437580859 >> v1:172.21.15.204:6789/0 conn(0x7f9ce00d0040 legacy=0x7f9ce006c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:47.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.019+0000 7f9cdde00640 1 -- 172.21.15.167:0/437580859 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ce00732d0 con 0x7f9ce00cf720 2024-05-11T12:10:47.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.019+0000 7f9cdd400640 1 --2- 172.21.15.167:0/437580859 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ce00d0810 0x7f9ce00cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:10:47.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.021+0000 7f9cdde00640 1 --2- 172.21.15.167:0/437580859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ce00cf720 0x7f9ce00723c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9cd00097a0 tx=0x7f9cd002a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60c1f7c6b9ddde5f server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:47.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.021+0000 7f9cdca00640 1 -- 172.21.15.167:0/437580859 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd0037070 con 0x7f9ce00cf720 2024-05-11T12:10:47.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.021+0000 7f9cdca00640 1 -- 172.21.15.167:0/437580859 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cd002ae50 con 0x7f9ce00cf720 2024-05-11T12:10:47.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.021+0000 7f9cdca00640 1 -- 172.21.15.167:0/437580859 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd0032660 con 0x7f9ce00cf720 2024-05-11T12:10:47.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.022+0000 7f9cdf200640 1 -- 172.21.15.167:0/437580859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ce00cf720 msgr2=0x7f9ce00723c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:47.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.022+0000 7f9cdf200640 1 --2- 172.21.15.167:0/437580859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ce00cf720 0x7f9ce00723c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9cd00097a0 tx=0x7f9cd002a100 comp rx=0 tx=0).stop 2024-05-11T12:10:47.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.022+0000 7f9cdf200640 1 -- 172.21.15.167:0/437580859 shutdown_connections 2024-05-11T12:10:47.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.022+0000 7f9cdf200640 1 --2- 172.21.15.167:0/437580859 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ce00d0810 0x7f9ce00cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:47.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.022+0000 7f9cdf200640 1 --2- 172.21.15.167:0/437580859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ce00cf720 0x7f9ce00723c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:47.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.022+0000 7f9cdf200640 1 -- 172.21.15.167:0/437580859 >> 172.21.15.167:0/437580859 conn(0x7f9ce00d0f30 msgr2=0x7f9ce00d1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:47.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.022+0000 7f9cdf200640 1 -- 172.21.15.167:0/437580859 shutdown_connections 2024-05-11T12:10:47.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.022+0000 7f9cdf200640 1 -- 172.21.15.167:0/437580859 wait complete. 2024-05-11T12:10:47.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.024+0000 7f9cdf200640 1 Processor -- start 2024-05-11T12:10:47.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.024+0000 7f9cdf200640 1 -- start start 2024-05-11T12:10:47.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.025+0000 7f9cdf200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ce00cf720 0x7f9ce015fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:47.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.025+0000 7f9cdf200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ce00d0040 0x7f9ce015c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:47.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.025+0000 7f9cdf200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ce00d0810 0x7f9ce015c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:47.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.025+0000 7f9cdf200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce0072900 con 0x7f9ce00d0040 2024-05-11T12:10:47.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.025+0000 7f9cdf200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce00d0490 con 0x7f9ce00cf720 2024-05-11T12:10:47.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.025+0000 7f9cdf200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f9ce0072a80 con 0x7f9ce00d0810 2024-05-11T12:10:47.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.025+0000 7f9cdd400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ce00d0040 0x7f9ce015c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:47.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.025+0000 7f9cdd400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ce00d0040 0x7f9ce015c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59014/0 (socket says 172.21.15.167:59014) 2024-05-11T12:10:47.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.025+0000 7f9cdd400640 1 -- 172.21.15.167:0/848301540 learned_addr learned my addr 172.21.15.167:0/848301540 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:10:47.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.025+0000 7f9cdde00640 1 --2- 172.21.15.167:0/848301540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ce00cf720 0x7f9ce015fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:47.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.025+0000 7f9cde800640 1 --2- 172.21.15.167:0/848301540 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ce00d0810 0x7f9ce015c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:47.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.026+0000 7f9cdd400640 1 -- 172.21.15.167:0/848301540 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ce00d0810 msgr2=0x7f9ce015c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:47.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.026+0000 7f9cdd400640 1 --2- 172.21.15.167:0/848301540 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ce00d0810 0x7f9ce015c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:47.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.026+0000 7f9cdd400640 1 -- 172.21.15.167:0/848301540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ce00cf720 msgr2=0x7f9ce015fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:47.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.026+0000 7f9cdd400640 1 --2- 172.21.15.167:0/848301540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ce00cf720 0x7f9ce015fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:47.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.026+0000 7f9cdd400640 1 -- 172.21.15.167:0/848301540 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ce00b4e50 con 0x7f9ce00d0040 2024-05-11T12:10:47.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.026+0000 7f9cdd400640 1 --2- 172.21.15.167:0/848301540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ce00d0040 0x7f9ce015c480 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8009f60 tx=0x7f9cc8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:47.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.026+0000 7f9cd6000640 1 -- 172.21.15.167:0/848301540 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc8018020 con 0x7f9ce00d0040 2024-05-11T12:10:47.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.026+0000 7f9cd6000640 1 -- 172.21.15.167:0/848301540 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cc8014680 con 0x7f9ce00d0040 2024-05-11T12:10:47.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.026+0000 7f9cd6000640 1 -- 172.21.15.167:0/848301540 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc8014a50 con 0x7f9ce00d0040 2024-05-11T12:10:47.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.026+0000 7f9cdf200640 1 -- 172.21.15.167:0/848301540 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ce00b5130 con 0x7f9ce00d0040 2024-05-11T12:10:47.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.026+0000 7f9cdf200640 1 -- 172.21.15.167:0/848301540 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ce00b56f0 con 0x7f9ce00d0040 2024-05-11T12:10:47.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.028+0000 7f9cdf200640 1 -- 172.21.15.167:0/848301540 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ce00b22f0 con 0x7f9ce00d0040 2024-05-11T12:10:47.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.029+0000 7f9cd6000640 1 -- 172.21.15.167:0/848301540 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9cc8005d10 con 0x7f9ce00d0040 2024-05-11T12:10:47.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.030+0000 7f9cd6000640 1 --2- 172.21.15.167:0/848301540 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f9cbc0762d0 0x7f9cbc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:47.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.030+0000 7f9cd6000640 1 -- 172.21.15.167:0/848301540 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6007+0+0 (secure 0 0 0) 0x7f9cc8098510 con 0x7f9ce00d0040 2024-05-11T12:10:47.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.030+0000 7f9cdde00640 1 --2- 172.21.15.167:0/848301540 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f9cbc0762d0 0x7f9cbc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:47.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.031+0000 7f9cdde00640 1 --2- 172.21.15.167:0/848301540 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f9cbc0762d0 0x7f9cbc078790 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0034000 tx=0x7f9cd002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:47.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.034+0000 7f9cd6000640 1 -- 172.21.15.167:0/848301540 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9ce00b22f0 con 0x7f9ce00d0040 2024-05-11T12:10:47.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:47.241+0000 7f9cdf200640 1 -- 172.21.15.167:0/848301540 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f9ce0069bc0 con 0x7f9ce00d0040 2024-05-11T12:10:48.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:48.795+0000 7f9cd6000640 1 -- 172.21.15.167:0/848301540 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v25) ==== 228+0+0 (secure 0 0 0) 0x7f9cc8022030 con 0x7f9ce00d0040 2024-05-11T12:10:48.796 INFO:teuthology.orchestra.run.smithi167.stderr:pool 'cephfs_data_ec' created 2024-05-11T12:10:48.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:48.799+0000 7f9cdf200640 1 -- 172.21.15.167:0/848301540 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f9cbc0762d0 msgr2=0x7f9cbc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:48.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:48.799+0000 7f9cdf200640 1 --2- 172.21.15.167:0/848301540 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f9cbc0762d0 0x7f9cbc078790 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0034000 tx=0x7f9cd002e000 comp rx=0 tx=0).stop 2024-05-11T12:10:48.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:48.799+0000 7f9cdf200640 1 -- 172.21.15.167:0/848301540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ce00d0040 msgr2=0x7f9ce015c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:48.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:48.799+0000 7f9cdf200640 1 --2- 172.21.15.167:0/848301540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ce00d0040 0x7f9ce015c480 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8009f60 tx=0x7f9cc8009f90 comp rx=0 tx=0).stop 2024-05-11T12:10:48.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:48.799+0000 7f9cdf200640 1 -- 172.21.15.167:0/848301540 shutdown_connections 2024-05-11T12:10:48.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:48.799+0000 7f9cdf200640 1 --2- 172.21.15.167:0/848301540 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ce00d0810 0x7f9ce015c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:48.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:48.799+0000 7f9cdf200640 1 --2- 172.21.15.167:0/848301540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ce00d0040 0x7f9ce015c480 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:48.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:48.799+0000 7f9cdf200640 1 --2- 172.21.15.167:0/848301540 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f9cbc0762d0 0x7f9cbc078790 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:48.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:48.799+0000 7f9cdf200640 1 --2- 172.21.15.167:0/848301540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ce00cf720 0x7f9ce015fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:48.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:48.799+0000 7f9cdf200640 1 -- 172.21.15.167:0/848301540 >> 172.21.15.167:0/848301540 conn(0x7f9ce00d0f30 msgr2=0x7f9ce015e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:48.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:48.800+0000 7f9cdf200640 1 -- 172.21.15.167:0/848301540 shutdown_connections 2024-05-11T12:10:48.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:48.801+0000 7f9cdf200640 1 -- 172.21.15.167:0/848301540 wait complete. 2024-05-11T12:10:48.830 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-05-11T12:10:49.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.009+0000 7f65a9e00640 1 Processor -- start 2024-05-11T12:10:49.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.010+0000 7f65a9e00640 1 -- start start 2024-05-11T12:10:49.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.011+0000 7f65a9e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65a415d2a0 0x7f65a415e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:49.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.011+0000 7f65a9e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f65a40730f0 0x7f65a415f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:49.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.011+0000 7f65a9e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65a415e660 con 0x7f65a40b3040 2024-05-11T12:10:49.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.011+0000 7f65a9e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f65a415ef50 con 0x7f65a40730f0 2024-05-11T12:10:49.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.011+0000 7f65a9e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f65a4073cc0 con 0x7f65a415d2a0 2024-05-11T12:10:49.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.012+0000 7f65a2a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65a415d2a0 0x7f65a415e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:49.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.012+0000 7f65a2a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65a415d2a0 0x7f65a415e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:38408/0 (socket says 172.21.15.167:38408) 2024-05-11T12:10:49.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.012+0000 7f65a2a00640 1 -- 172.21.15.167:0/1660852496 learned_addr learned my addr 172.21.15.167:0/1660852496 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:10:49.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.012+0000 7f65a3400640 1 --2- 172.21.15.167:0/1660852496 >> v2:172.21.15.204:3300/0 conn(0x7f65a40730f0 0x7f65a415f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:49.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.013+0000 7f65a2a00640 1 -- 172.21.15.167:0/1660852496 >> v2:172.21.15.204:3300/0 conn(0x7f65a40730f0 msgr2=0x7f65a415f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:49.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.013+0000 7f65a2a00640 1 --2- 172.21.15.167:0/1660852496 >> v2:172.21.15.204:3300/0 conn(0x7f65a40730f0 0x7f65a415f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:49.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.013+0000 7f65a2a00640 1 -- 172.21.15.167:0/1660852496 >> v1:172.21.15.204:6789/0 conn(0x7f65a40b3040 legacy=0x7f65a40729e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:49.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.013+0000 7f65a2a00640 1 -- 172.21.15.167:0/1660852496 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65a415e900 con 0x7f65a415d2a0 2024-05-11T12:10:49.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.013+0000 7f65a1600640 1 -- 172.21.15.167:0/1660852496 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3101354583 0 0) 0x7f658c001110 con 0x7f65a40b3040 2024-05-11T12:10:49.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.014+0000 7f65a2a00640 1 --2- 172.21.15.167:0/1660852496 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65a415d2a0 0x7f65a415e120 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6594009e50 tx=0x7f659402a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1163d8c69eca905e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:49.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.015+0000 7f65a1600640 1 -- 172.21.15.167:0/1660852496 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6594036070 con 0x7f65a415d2a0 2024-05-11T12:10:49.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.015+0000 7f65a1600640 1 -- 172.21.15.167:0/1660852496 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6594031600 con 0x7f65a415d2a0 2024-05-11T12:10:49.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.015+0000 7f65a1600640 1 -- 172.21.15.167:0/1660852496 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65940319d0 con 0x7f65a415d2a0 2024-05-11T12:10:49.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.016+0000 7f65a9e00640 1 -- 172.21.15.167:0/1660852496 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65a415d2a0 msgr2=0x7f65a415e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:49.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.016+0000 7f65a9e00640 1 --2- 172.21.15.167:0/1660852496 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65a415d2a0 0x7f65a415e120 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6594009e50 tx=0x7f659402a5f0 comp rx=0 tx=0).stop 2024-05-11T12:10:49.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.016+0000 7f65a9e00640 1 -- 172.21.15.167:0/1660852496 shutdown_connections 2024-05-11T12:10:49.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.016+0000 7f65a9e00640 1 --2- 172.21.15.167:0/1660852496 >> v2:172.21.15.204:3300/0 conn(0x7f65a40730f0 0x7f65a415f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:49.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.016+0000 7f65a9e00640 1 --2- 172.21.15.167:0/1660852496 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65a415d2a0 0x7f65a415e120 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:49.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.016+0000 7f65a9e00640 1 -- 172.21.15.167:0/1660852496 >> 172.21.15.167:0/1660852496 conn(0x7f65a4070190 msgr2=0x7f65a40705a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:49.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.016+0000 7f65a9e00640 1 -- 172.21.15.167:0/1660852496 shutdown_connections 2024-05-11T12:10:49.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.016+0000 7f65a9e00640 1 -- 172.21.15.167:0/1660852496 wait complete. 2024-05-11T12:10:49.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.018+0000 7f65a9e00640 1 Processor -- start 2024-05-11T12:10:49.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.018+0000 7f65a9e00640 1 -- start start 2024-05-11T12:10:49.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.020+0000 7f65a9e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65a40730f0 0x7f65a40b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:49.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.020+0000 7f65a9e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65a40b3040 0x7f65a4157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:49.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.020+0000 7f65a9e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65a415d2a0 0x7f65a4157bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:49.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.020+0000 7f65a9e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f65a4073950 con 0x7f65a415d2a0 2024-05-11T12:10:49.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.020+0000 7f65a9e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f65a4073fa0 con 0x7f65a40730f0 2024-05-11T12:10:49.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.020+0000 7f65a9e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f65a415f920 con 0x7f65a40b3040 2024-05-11T12:10:49.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.020+0000 7f65a3400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65a415d2a0 0x7f65a4157bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:49.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.020+0000 7f65a3400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65a415d2a0 0x7f65a4157bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59026/0 (socket says 172.21.15.167:59026) 2024-05-11T12:10:49.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.020+0000 7f65a3400640 1 -- 172.21.15.167:0/1817678282 learned_addr learned my addr 172.21.15.167:0/1817678282 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:10:49.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.020+0000 7f65a2a00640 1 --2- 172.21.15.167:0/1817678282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65a40730f0 0x7f65a40b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:49.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.020+0000 7f65a2000640 1 --2- 172.21.15.167:0/1817678282 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65a40b3040 0x7f65a4157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:49.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.021+0000 7f65a3400640 1 -- 172.21.15.167:0/1817678282 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65a40b3040 msgr2=0x7f65a4157680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:49.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.021+0000 7f65a3400640 1 --2- 172.21.15.167:0/1817678282 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65a40b3040 0x7f65a4157680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:49.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.021+0000 7f65a3400640 1 -- 172.21.15.167:0/1817678282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65a40730f0 msgr2=0x7f65a40b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:49.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.021+0000 7f65a3400640 1 --2- 172.21.15.167:0/1817678282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65a40730f0 0x7f65a40b1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:49.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.021+0000 7f65a3400640 1 -- 172.21.15.167:0/1817678282 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65a414c160 con 0x7f65a415d2a0 2024-05-11T12:10:49.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.021+0000 7f65a2000640 1 --2- 172.21.15.167:0/1817678282 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65a40b3040 0x7f65a4157680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:10:49.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.021+0000 7f65a3400640 1 --2- 172.21.15.167:0/1817678282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65a415d2a0 0x7f65a4157bc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f659c00e850 tx=0x7f659c00ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:49.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.021+0000 7f659b400640 1 -- 172.21.15.167:0/1817678282 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c017070 con 0x7f65a415d2a0 2024-05-11T12:10:49.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.021+0000 7f659b400640 1 -- 172.21.15.167:0/1817678282 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f659c00c940 con 0x7f65a415d2a0 2024-05-11T12:10:49.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.021+0000 7f659b400640 1 -- 172.21.15.167:0/1817678282 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c00cd30 con 0x7f65a415d2a0 2024-05-11T12:10:49.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.021+0000 7f65a9e00640 1 -- 172.21.15.167:0/1817678282 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65a414c440 con 0x7f65a415d2a0 2024-05-11T12:10:49.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.022+0000 7f65a9e00640 1 -- 172.21.15.167:0/1817678282 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65a414ca00 con 0x7f65a415d2a0 2024-05-11T12:10:49.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.023+0000 7f65a9e00640 1 -- 172.21.15.167:0/1817678282 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65a4069bc0 con 0x7f65a415d2a0 2024-05-11T12:10:49.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.024+0000 7f659b400640 1 -- 172.21.15.167:0/1817678282 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f659c01a070 con 0x7f65a415d2a0 2024-05-11T12:10:49.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.026+0000 7f659b400640 1 --2- 172.21.15.167:0/1817678282 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f65800762d0 0x7f6580078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:49.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.026+0000 7f659b400640 1 -- 172.21.15.167:0/1817678282 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6500+0+0 (secure 0 0 0) 0x7f659c0980c0 con 0x7f65a415d2a0 2024-05-11T12:10:49.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.027+0000 7f65a2a00640 1 --2- 172.21.15.167:0/1817678282 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f65800762d0 0x7f6580078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:49.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.027+0000 7f65a2a00640 1 --2- 172.21.15.167:0/1817678282 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f65800762d0 0x7f6580078790 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6594033000 tx=0x7f6594009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:49.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.032+0000 7f659b400640 1 -- 172.21.15.167:0/1817678282 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f659c061090 con 0x7f65a415d2a0 2024-05-11T12:10:49.239 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.239+0000 7f65a9e00640 1 -- 172.21.15.167:0/1817678282 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f65a4070ed0 con 0x7f65a415d2a0 2024-05-11T12:10:49.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.801+0000 7f659b400640 1 -- 172.21.15.167:0/1817678282 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v26) ==== 169+0+0 (secure 0 0 0) 0x7f659c065390 con 0x7f65a415d2a0 2024-05-11T12:10:49.802 INFO:teuthology.orchestra.run.smithi167.stderr:set pool 4 allow_ec_overwrites to true 2024-05-11T12:10:49.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.806+0000 7f65a9e00640 1 -- 172.21.15.167:0/1817678282 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f65800762d0 msgr2=0x7f6580078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:49.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.806+0000 7f65a9e00640 1 --2- 172.21.15.167:0/1817678282 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f65800762d0 0x7f6580078790 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6594033000 tx=0x7f6594009000 comp rx=0 tx=0).stop 2024-05-11T12:10:49.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.806+0000 7f65a9e00640 1 -- 172.21.15.167:0/1817678282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65a415d2a0 msgr2=0x7f65a4157bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:49.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.806+0000 7f65a9e00640 1 --2- 172.21.15.167:0/1817678282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65a415d2a0 0x7f65a4157bc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f659c00e850 tx=0x7f659c00ed10 comp rx=0 tx=0).stop 2024-05-11T12:10:49.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.807+0000 7f65a9e00640 1 -- 172.21.15.167:0/1817678282 shutdown_connections 2024-05-11T12:10:49.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.807+0000 7f65a9e00640 1 --2- 172.21.15.167:0/1817678282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65a415d2a0 0x7f65a4157bc0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:49.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.807+0000 7f65a9e00640 1 --2- 172.21.15.167:0/1817678282 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65a40b3040 0x7f65a4157680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:49.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.807+0000 7f65a9e00640 1 --2- 172.21.15.167:0/1817678282 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f65800762d0 0x7f6580078790 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:49.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.807+0000 7f65a9e00640 1 --2- 172.21.15.167:0/1817678282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65a40730f0 0x7f65a40b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:49.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.807+0000 7f65a9e00640 1 -- 172.21.15.167:0/1817678282 >> 172.21.15.167:0/1817678282 conn(0x7f65a4070190 msgr2=0x7f65a415bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:49.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.807+0000 7f65a9e00640 1 -- 172.21.15.167:0/1817678282 shutdown_connections 2024-05-11T12:10:49.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:49.807+0000 7f65a9e00640 1 -- 172.21.15.167:0/1817678282 wait complete. 2024-05-11T12:10:49.842 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-05-11T12:10:50.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.018+0000 7fd8afe00640 1 Processor -- start 2024-05-11T12:10:50.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.019+0000 7fd8afe00640 1 -- start start 2024-05-11T12:10:50.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.020+0000 7fd8afe00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd8b0072830 0x7fd8b0072cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:50.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.020+0000 7fd8afe00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd8b015efb0 0x7fd8b015f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:50.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.020+0000 7fd8afe00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8b015af00 con 0x7fd8b015ab20 2024-05-11T12:10:50.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.020+0000 7fd8afe00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fd8b015e4b0 con 0x7fd8b0072830 2024-05-11T12:10:50.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.020+0000 7fd8afe00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fd8b015def0 con 0x7fd8b015efb0 2024-05-11T12:10:50.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.020+0000 7fd8a6000640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd8b0072830 0x7fd8b0072cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:50.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.020+0000 7fd8a6000640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd8b0072830 0x7fd8b0072cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59036/0 (socket says 172.21.15.167:59036) 2024-05-11T12:10:50.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.020+0000 7fd8a6000640 1 -- 172.21.15.167:0/1230953226 learned_addr learned my addr 172.21.15.167:0/1230953226 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:10:50.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.020+0000 7fd8a6000640 1 -- 172.21.15.167:0/1230953226 >> v2:172.21.15.204:3300/0 conn(0x7fd8b015efb0 msgr2=0x7fd8b015f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:50.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.020+0000 7fd8a6000640 1 --2- 172.21.15.167:0/1230953226 >> v2:172.21.15.204:3300/0 conn(0x7fd8b015efb0 0x7fd8b015f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:50.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.020+0000 7fd8a6000640 1 -- 172.21.15.167:0/1230953226 >> v1:172.21.15.204:6789/0 conn(0x7fd8b015ab20 legacy=0x7fd8b015d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:50.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.020+0000 7fd8a6000640 1 -- 172.21.15.167:0/1230953226 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8b015e1d0 con 0x7fd8b0072830 2024-05-11T12:10:50.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.021+0000 7fd8a6000640 1 --2- 172.21.15.167:0/1230953226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8b0072830 0x7fd8b0072cc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd8b015e630 tx=0x7fd89802a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c57ef2ec0d04574a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:50.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.022+0000 7fd8ae000640 1 -- 172.21.15.167:0/1230953226 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd898037070 con 0x7fd8b0072830 2024-05-11T12:10:50.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.022+0000 7fd8ae000640 1 -- 172.21.15.167:0/1230953226 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd89802adf0 con 0x7fd8b0072830 2024-05-11T12:10:50.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.022+0000 7fd8ae000640 1 -- 172.21.15.167:0/1230953226 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd898032660 con 0x7fd8b0072830 2024-05-11T12:10:50.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.022+0000 7fd8afe00640 1 -- 172.21.15.167:0/1230953226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8b0072830 msgr2=0x7fd8b0072cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:50.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.022+0000 7fd8afe00640 1 --2- 172.21.15.167:0/1230953226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8b0072830 0x7fd8b0072cc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd8b015e630 tx=0x7fd89802a0a0 comp rx=0 tx=0).stop 2024-05-11T12:10:50.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.022+0000 7fd8afe00640 1 -- 172.21.15.167:0/1230953226 shutdown_connections 2024-05-11T12:10:50.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.022+0000 7fd8afe00640 1 --2- 172.21.15.167:0/1230953226 >> v2:172.21.15.204:3300/0 conn(0x7fd8b015efb0 0x7fd8b015f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:50.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.023+0000 7fd8afe00640 1 --2- 172.21.15.167:0/1230953226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8b0072830 0x7fd8b0072cc0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:50.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.023+0000 7fd8afe00640 1 -- 172.21.15.167:0/1230953226 >> 172.21.15.167:0/1230953226 conn(0x7fd8b006ffe0 msgr2=0x7fd8b00703c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:50.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.023+0000 7fd8afe00640 1 -- 172.21.15.167:0/1230953226 shutdown_connections 2024-05-11T12:10:50.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.023+0000 7fd8afe00640 1 -- 172.21.15.167:0/1230953226 wait complete. 2024-05-11T12:10:50.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.024+0000 7fd8afe00640 1 Processor -- start 2024-05-11T12:10:50.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.024+0000 7fd8afe00640 1 -- start start 2024-05-11T12:10:50.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.025+0000 7fd8afe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8b0072830 0x7fd8b0145f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:50.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.025+0000 7fd8afe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8b015ab20 0x7fd8b00ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:50.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.025+0000 7fd8afe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd8b015efb0 0x7fd8b00cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:50.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.025+0000 7fd8afe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b00c6560 con 0x7fd8b0072830 2024-05-11T12:10:50.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.025+0000 7fd8afe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b015dbd0 con 0x7fd8b015ab20 2024-05-11T12:10:50.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.025+0000 7fd8afe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd8b00c66e0 con 0x7fd8b015efb0 2024-05-11T12:10:50.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.026+0000 7fd8aea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8b0072830 0x7fd8b0145f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:50.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.026+0000 7fd8aea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8b0072830 0x7fd8b0145f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59038/0 (socket says 172.21.15.167:59038) 2024-05-11T12:10:50.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.026+0000 7fd8aea00640 1 -- 172.21.15.167:0/821825535 learned_addr learned my addr 172.21.15.167:0/821825535 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:10:50.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.026+0000 7fd8aea00640 1 -- 172.21.15.167:0/821825535 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd8b015efb0 msgr2=0x7fd8b00cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:50.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.026+0000 7fd8aea00640 1 --2- 172.21.15.167:0/821825535 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd8b015efb0 0x7fd8b00cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:50.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.026+0000 7fd8aea00640 1 -- 172.21.15.167:0/821825535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8b015ab20 msgr2=0x7fd8b00ccc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:50.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.026+0000 7fd8aea00640 1 --2- 172.21.15.167:0/821825535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8b015ab20 0x7fd8b00ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:50.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.026+0000 7fd8aea00640 1 -- 172.21.15.167:0/821825535 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8b00b6ba0 con 0x7fd8b0072830 2024-05-11T12:10:50.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.026+0000 7fd8aea00640 1 --2- 172.21.15.167:0/821825535 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8b0072830 0x7fd8b0145f40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd8a000c410 tx=0x7fd8a000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:50.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.026+0000 7fd8a7400640 1 -- 172.21.15.167:0/821825535 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0017070 con 0x7fd8b0072830 2024-05-11T12:10:50.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.027+0000 7fd8a7400640 1 -- 172.21.15.167:0/821825535 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8a00035f0 con 0x7fd8b0072830 2024-05-11T12:10:50.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.027+0000 7fd8afe00640 1 -- 172.21.15.167:0/821825535 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8b00cdb10 con 0x7fd8b0072830 2024-05-11T12:10:50.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.027+0000 7fd8a7400640 1 -- 172.21.15.167:0/821825535 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0003a20 con 0x7fd8b0072830 2024-05-11T12:10:50.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.027+0000 7fd8afe00640 1 -- 172.21.15.167:0/821825535 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8b00b4620 con 0x7fd8b0072830 2024-05-11T12:10:50.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.028+0000 7fd8afe00640 1 -- 172.21.15.167:0/821825535 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8b00694b0 con 0x7fd8b0072830 2024-05-11T12:10:50.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.029+0000 7fd8a7400640 1 -- 172.21.15.167:0/821825535 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd8a001a070 con 0x7fd8b0072830 2024-05-11T12:10:50.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.030+0000 7fd8a7400640 1 --2- 172.21.15.167:0/821825535 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd88c076300 0x7fd88c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:50.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.030+0000 7fd8a7400640 1 -- 172.21.15.167:0/821825535 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6500+0+0 (secure 0 0 0) 0x7fd8a0096e30 con 0x7fd8b0072830 2024-05-11T12:10:50.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.030+0000 7fd8a6000640 1 --2- 172.21.15.167:0/821825535 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd88c076300 0x7fd88c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:50.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.031+0000 7fd8a6000640 1 --2- 172.21.15.167:0/821825535 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd88c076300 0x7fd88c0787c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd898034000 tx=0x7fd89802c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:50.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.035+0000 7fd8a7400640 1 -- 172.21.15.167:0/821825535 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd8a00603d0 con 0x7fd8b0072830 2024-05-11T12:10:50.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.282+0000 7fd8afe00640 1 -- 172.21.15.167:0/821825535 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7fd8b0073860 con 0x7fd8b0072830 2024-05-11T12:10:50.421 INFO:tasks.ceph.mds.b.smithi204.stderr:2024-05-11T12:10:50.421+0000 7f8b5e200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-11T12:10:50.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.837+0000 7fd8a7400640 1 -- 172.21.15.167:0/821825535 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) ==== 137+0+0 (secure 0 0 0) 0x7fd8a00010c0 con 0x7fd8b0072830 2024-05-11T12:10:50.838 INFO:teuthology.orchestra.run.smithi167.stderr:added data pool 4 to fsmap 2024-05-11T12:10:50.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.840+0000 7fd8afe00640 1 -- 172.21.15.167:0/821825535 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd88c076300 msgr2=0x7fd88c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:50.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.840+0000 7fd8afe00640 1 --2- 172.21.15.167:0/821825535 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd88c076300 0x7fd88c0787c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd898034000 tx=0x7fd89802c000 comp rx=0 tx=0).stop 2024-05-11T12:10:50.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.841+0000 7fd8afe00640 1 -- 172.21.15.167:0/821825535 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8b0072830 msgr2=0x7fd8b0145f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:50.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.841+0000 7fd8afe00640 1 --2- 172.21.15.167:0/821825535 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8b0072830 0x7fd8b0145f40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd8a000c410 tx=0x7fd8a000c8d0 comp rx=0 tx=0).stop 2024-05-11T12:10:50.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.841+0000 7fd8afe00640 1 -- 172.21.15.167:0/821825535 shutdown_connections 2024-05-11T12:10:50.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.841+0000 7fd8afe00640 1 --2- 172.21.15.167:0/821825535 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd8b015efb0 0x7fd8b00cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:50.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.841+0000 7fd8afe00640 1 --2- 172.21.15.167:0/821825535 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd88c076300 0x7fd88c0787c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:50.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.841+0000 7fd8afe00640 1 --2- 172.21.15.167:0/821825535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8b015ab20 0x7fd8b00ccc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:50.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.841+0000 7fd8afe00640 1 --2- 172.21.15.167:0/821825535 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8b0072830 0x7fd8b0145f40 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:50.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.841+0000 7fd8afe00640 1 -- 172.21.15.167:0/821825535 >> 172.21.15.167:0/821825535 conn(0x7fd8b006ffe0 msgr2=0x7fd8b006c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:50.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.841+0000 7fd8afe00640 1 -- 172.21.15.167:0/821825535 shutdown_connections 2024-05-11T12:10:50.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:50.841+0000 7fd8afe00640 1 -- 172.21.15.167:0/821825535 wait complete. 2024-05-11T12:10:50.862 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:10:51.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.062+0000 7fc00a800640 1 Processor -- start 2024-05-11T12:10:51.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.062+0000 7fc00a800640 1 -- start start 2024-05-11T12:10:51.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.063+0000 7fc00a800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc0040b3ea0 0x7fc004161490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:51.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.063+0000 7fc00a800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0041619d0 0x7fc00406d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:51.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.063+0000 7fc00a800640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc00415df30 con 0x7fc004157750 2024-05-11T12:10:51.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.063+0000 7fc00a800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fc004161000 con 0x7fc0040b3ea0 2024-05-11T12:10:51.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.063+0000 7fc00a800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc00415e210 con 0x7fc0041619d0 2024-05-11T12:10:51.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.064+0000 7fc003400640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fc004157750 0x7fc00415d740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:57180/0 (socket says 172.21.15.167:57180) 2024-05-11T12:10:51.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.064+0000 7fc003400640 1 -- 172.21.15.167:0/3523075116 learned_addr learned my addr 172.21.15.167:0/3523075116 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:10:51.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.064+0000 7fbffbe00640 1 --2- 172.21.15.167:0/3523075116 >> v2:172.21.15.204:3300/0 conn(0x7fc0040b3ea0 0x7fc004161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:51.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.064+0000 7fc003e00640 1 --2- 172.21.15.167:0/3523075116 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0041619d0 0x7fc00406d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:51.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.064+0000 7fc002a00640 1 -- 172.21.15.167:0/3523075116 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2465912215 0 0) 0x7fc00415df30 con 0x7fc004157750 2024-05-11T12:10:51.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.064+0000 7fc002a00640 1 -- 172.21.15.167:0/3523075116 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbfe8003520 con 0x7fc004157750 2024-05-11T12:10:51.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.065+0000 7fc002a00640 1 -- 172.21.15.167:0/3523075116 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3216986492 0 0) 0x7fbfe8003520 con 0x7fc004157750 2024-05-11T12:10:51.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.065+0000 7fc002a00640 1 -- 172.21.15.167:0/3523075116 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0041619d0 msgr2=0x7fc00406d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:51.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.065+0000 7fc002a00640 1 --2- 172.21.15.167:0/3523075116 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0041619d0 0x7fc00406d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:51.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.065+0000 7fc002a00640 1 -- 172.21.15.167:0/3523075116 >> v2:172.21.15.204:3300/0 conn(0x7fc0040b3ea0 msgr2=0x7fc004161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:51.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.065+0000 7fc002a00640 1 --2- 172.21.15.167:0/3523075116 >> v2:172.21.15.204:3300/0 conn(0x7fc0040b3ea0 0x7fc004161490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:51.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.065+0000 7fc002a00640 1 -- 172.21.15.167:0/3523075116 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc00415e4f0 con 0x7fc004157750 2024-05-11T12:10:51.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.065+0000 7fc002a00640 1 -- 172.21.15.167:0/3523075116 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7fbff40016f0 con 0x7fc004157750 2024-05-11T12:10:51.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.065+0000 7fc002a00640 1 -- 172.21.15.167:0/3523075116 >> v1:172.21.15.167:6789/0 conn(0x7fc004157750 legacy=0x7fc00415d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:51.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.065+0000 7fc002a00640 1 --2- 172.21.15.167:0/3523075116 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0041619d0 0x7fbfe8003a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:51.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.065+0000 7fc002a00640 1 --2- 172.21.15.167:0/3523075116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbfe8004020 0x7fbfe8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:51.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.065+0000 7fc002a00640 1 --2- 172.21.15.167:0/3523075116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbfe8004d40 0x7fbfe80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:51.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.065+0000 7fc002a00640 1 -- 172.21.15.167:0/3523075116 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc004161000 con 0x7fbfe8004d40 2024-05-11T12:10:51.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.065+0000 7fc002a00640 1 -- 172.21.15.167:0/3523075116 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc00415e210 con 0x7fbfe8004020 2024-05-11T12:10:51.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.065+0000 7fc002a00640 1 -- 172.21.15.167:0/3523075116 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbfe800ba70 con 0x7fc0041619d0 2024-05-11T12:10:51.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.065+0000 7fc003e00640 1 --2- 172.21.15.167:0/3523075116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbfe8004d40 0x7fbfe80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:51.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.065+0000 7fc003e00640 1 -- 172.21.15.167:0/3523075116 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0041619d0 msgr2=0x7fbfe8003a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:51.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.065+0000 7fc003400640 1 --2- 172.21.15.167:0/3523075116 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0041619d0 0x7fbfe8003a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:51.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.066+0000 7fbffbe00640 1 --2- 172.21.15.167:0/3523075116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbfe8004020 0x7fbfe8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:51.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.066+0000 7fc003e00640 1 --2- 172.21.15.167:0/3523075116 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0041619d0 0x7fbfe8003a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:51.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.066+0000 7fc003e00640 1 -- 172.21.15.167:0/3523075116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbfe8004020 msgr2=0x7fbfe8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:51.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.066+0000 7fc003e00640 1 --2- 172.21.15.167:0/3523075116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbfe8004020 0x7fbfe8004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:51.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.066+0000 7fc003e00640 1 -- 172.21.15.167:0/3523075116 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfe8003520 con 0x7fbfe8004d40 2024-05-11T12:10:51.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.066+0000 7fc003400640 1 --2- 172.21.15.167:0/3523075116 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0041619d0 0x7fbfe8003a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:10:51.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.067+0000 7fc003e00640 1 --2- 172.21.15.167:0/3523075116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbfe8004d40 0x7fbfe80051c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbffc009f90 tx=0x7fbfe8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db365211a9b6608d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:51.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.067+0000 7fc002a00640 1 -- 172.21.15.167:0/3523075116 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc009c10 con 0x7fbfe8004d40 2024-05-11T12:10:51.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.067+0000 7fc002a00640 1 -- 172.21.15.167:0/3523075116 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbffc036070 con 0x7fbfe8004d40 2024-05-11T12:10:51.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.067+0000 7fc002a00640 1 -- 172.21.15.167:0/3523075116 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc031850 con 0x7fbfe8004d40 2024-05-11T12:10:51.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.067+0000 7fc00a800640 1 -- 172.21.15.167:0/3523075116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbfe8004d40 msgr2=0x7fbfe80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:51.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.067+0000 7fc00a800640 1 --2- 172.21.15.167:0/3523075116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbfe8004d40 0x7fbfe80051c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbffc009f90 tx=0x7fbfe8005830 comp rx=0 tx=0).stop 2024-05-11T12:10:51.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.067+0000 7fc003e00640 1 -- 172.21.15.167:0/3523075116 reap_dead start 2024-05-11T12:10:51.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.068+0000 7fc00a800640 1 -- 172.21.15.167:0/3523075116 shutdown_connections 2024-05-11T12:10:51.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.068+0000 7fc00a800640 1 -- 172.21.15.167:0/3523075116 >> 172.21.15.167:0/3523075116 conn(0x7fc0040cd8f0 msgr2=0x7fc0040cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:51.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.068+0000 7fc00a800640 1 -- 172.21.15.167:0/3523075116 shutdown_connections 2024-05-11T12:10:51.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.068+0000 7fc00a800640 1 -- 172.21.15.167:0/3523075116 wait complete. 2024-05-11T12:10:51.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.069+0000 7fc00a800640 1 Processor -- start 2024-05-11T12:10:51.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.070+0000 7fc00a800640 1 -- start start 2024-05-11T12:10:51.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.070+0000 7fc00a800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbfe8004d40 0x7fc00415bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:51.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.070+0000 7fc00a800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0041f29f0 0x7fc00415a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:51.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.070+0000 7fc00a800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc00415a660 0x7fc0041450d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:51.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.070+0000 7fc00a800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc00406d5e0 con 0x7fc00415a660 2024-05-11T12:10:51.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.071+0000 7fc00a800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc00406d460 con 0x7fbfe8004d40 2024-05-11T12:10:51.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.071+0000 7fc00a800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc00406d760 con 0x7fc0041f29f0 2024-05-11T12:10:51.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.071+0000 7fc003e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc00415a660 0x7fc0041450d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:51.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.071+0000 7fc003e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc00415a660 0x7fc0041450d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59064/0 (socket says 172.21.15.167:59064) 2024-05-11T12:10:51.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.071+0000 7fc003e00640 1 -- 172.21.15.167:0/3979817214 learned_addr learned my addr 172.21.15.167:0/3979817214 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:10:51.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.071+0000 7fbffbe00640 1 --2- 172.21.15.167:0/3979817214 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0041f29f0 0x7fc00415a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:51.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.071+0000 7fc003400640 1 --2- 172.21.15.167:0/3979817214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbfe8004d40 0x7fc00415bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:51.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.071+0000 7fc003e00640 1 -- 172.21.15.167:0/3979817214 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0041f29f0 msgr2=0x7fc00415a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:51.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.071+0000 7fc003e00640 1 --2- 172.21.15.167:0/3979817214 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0041f29f0 0x7fc00415a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:51.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.071+0000 7fc003e00640 1 -- 172.21.15.167:0/3979817214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbfe8004d40 msgr2=0x7fc00415bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:51.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.071+0000 7fc003e00640 1 --2- 172.21.15.167:0/3979817214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbfe8004d40 0x7fc00415bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:51.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.071+0000 7fc003e00640 1 -- 172.21.15.167:0/3979817214 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0041f39c0 con 0x7fc00415a660 2024-05-11T12:10:51.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.071+0000 7fc003400640 1 --2- 172.21.15.167:0/3979817214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbfe8004d40 0x7fc00415bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:10:51.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.072+0000 7fc003e00640 1 --2- 172.21.15.167:0/3979817214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc00415a660 0x7fc0041450d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbffc033040 tx=0x7fbffc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:51.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.072+0000 7fbffb400640 1 -- 172.21.15.167:0/3979817214 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc009c10 con 0x7fc00415a660 2024-05-11T12:10:51.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.072+0000 7fbffb400640 1 -- 172.21.15.167:0/3979817214 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbffc003990 con 0x7fc00415a660 2024-05-11T12:10:51.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.072+0000 7fbffb400640 1 -- 172.21.15.167:0/3979817214 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc003d80 con 0x7fc00415a660 2024-05-11T12:10:51.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.072+0000 7fc00a800640 1 -- 172.21.15.167:0/3979817214 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0041f3400 con 0x7fc00415a660 2024-05-11T12:10:51.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.072+0000 7fc00a800640 1 -- 172.21.15.167:0/3979817214 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc00415b330 con 0x7fc00415a660 2024-05-11T12:10:51.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.073+0000 7fc00a800640 1 -- 172.21.15.167:0/3979817214 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc004069590 con 0x7fc00415a660 2024-05-11T12:10:51.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.074+0000 7fbffb400640 1 -- 172.21.15.167:0/3979817214 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbffc005d10 con 0x7fc00415a660 2024-05-11T12:10:51.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.076+0000 7fbffb400640 1 --2- 172.21.15.167:0/3979817214 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbfe0076300 0x7fbfe00787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:51.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.076+0000 7fbffb400640 1 -- 172.21.15.167:0/3979817214 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fbffc0b87e0 con 0x7fc00415a660 2024-05-11T12:10:51.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.076+0000 7fc003400640 1 --2- 172.21.15.167:0/3979817214 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbfe0076300 0x7fbfe00787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:51.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.077+0000 7fc003400640 1 --2- 172.21.15.167:0/3979817214 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbfe0076300 0x7fbfe00787c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbff40079b0 tx=0x7fbff4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:51.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.081+0000 7fbffb400640 1 -- 172.21.15.167:0/3979817214 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fbffc02e050 con 0x7fc00415a660 2024-05-11T12:10:51.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.326+0000 7fc00a800640 1 -- 172.21.15.167:0/3979817214 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc004075b40 con 0x7fc00415a660 2024-05-11T12:10:51.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.327+0000 7fbffb400640 1 -- 172.21.15.167:0/3979817214 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5189 (secure 0 0 0) 0x7fbffc081e90 con 0x7fc00415a660 2024-05-11T12:10:51.327 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:10:51.327 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3127277486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3127277486},{"type":"v1","addr":"172.21.15.167:6835","nonce":3127277486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2547133600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2547133600},{"type":"v1","addr":"172.21.15.167:6837","nonce":2547133600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3200960100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3200960100},{"type":"v1","addr":"172.21.15.204:6834","nonce":3200960100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:10:45.225745+0000","modified":"2024-05-11T12:10:50.284984+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/1889842375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":1889842375},{"type":"v1","addr":"172.21.15.204:6835","nonce":1889842375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-05-11T12:10:51.328 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 7 2024-05-11T12:10:51.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.329+0000 7fc00a800640 1 -- 172.21.15.167:0/3979817214 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbfe0076300 msgr2=0x7fbfe00787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:51.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.330+0000 7fc00a800640 1 --2- 172.21.15.167:0/3979817214 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbfe0076300 0x7fbfe00787c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbff40079b0 tx=0x7fbff4009040 comp rx=0 tx=0).stop 2024-05-11T12:10:51.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.330+0000 7fc00a800640 1 -- 172.21.15.167:0/3979817214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc00415a660 msgr2=0x7fc0041450d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:51.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.330+0000 7fc00a800640 1 --2- 172.21.15.167:0/3979817214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc00415a660 0x7fc0041450d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbffc033040 tx=0x7fbffc009750 comp rx=0 tx=0).stop 2024-05-11T12:10:51.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.330+0000 7fc00a800640 1 -- 172.21.15.167:0/3979817214 shutdown_connections 2024-05-11T12:10:51.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.330+0000 7fc00a800640 1 --2- 172.21.15.167:0/3979817214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc00415a660 0x7fc0041450d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:51.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.330+0000 7fc00a800640 1 --2- 172.21.15.167:0/3979817214 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0041f29f0 0x7fc00415a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:51.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.330+0000 7fc00a800640 1 --2- 172.21.15.167:0/3979817214 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbfe0076300 0x7fbfe00787c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:51.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.330+0000 7fc00a800640 1 --2- 172.21.15.167:0/3979817214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbfe8004d40 0x7fc00415bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:51.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.330+0000 7fc00a800640 1 -- 172.21.15.167:0/3979817214 >> 172.21.15.167:0/3979817214 conn(0x7fc0040cd8f0 msgr2=0x7fc0040cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:51.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.330+0000 7fc00a800640 1 -- 172.21.15.167:0/3979817214 shutdown_connections 2024-05-11T12:10:51.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.330+0000 7fc00a800640 1 -- 172.21.15.167:0/3979817214 wait complete. 2024-05-11T12:10:51.349 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:10:51.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.547+0000 7f568ac00640 1 Processor -- start 2024-05-11T12:10:51.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.547+0000 7f568ac00640 1 -- start start 2024-05-11T12:10:51.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.548+0000 7f568ac00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f56840afcd0 0x7f56841601f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:51.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.548+0000 7f568ac00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f56840d04a0 0x7f56840b55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:51.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.549+0000 7f568ac00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f56840cf260 0x7f56840cf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:51.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.549+0000 7f568ac00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f56840b5130 con 0x7f56840cf260 2024-05-11T12:10:51.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.549+0000 7f568ac00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f56840b4560 con 0x7f56840afcd0 2024-05-11T12:10:51.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.549+0000 7f568ac00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f56840b4800 con 0x7f56840d04a0 2024-05-11T12:10:51.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.549+0000 7f568a200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f56840cf260 0x7f56840cf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:51.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.549+0000 7f568a200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f56840cf260 0x7f56840cf6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59074/0 (socket says 172.21.15.167:59074) 2024-05-11T12:10:51.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.549+0000 7f568a200640 1 -- 172.21.15.167:0/1501027830 learned_addr learned my addr 172.21.15.167:0/1501027830 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:10:51.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.549+0000 7f5688e00640 1 --2- 172.21.15.167:0/1501027830 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f56840d04a0 0x7f56840b55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:51.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.549+0000 7f5689800640 1 --2- 172.21.15.167:0/1501027830 >> v2:172.21.15.204:3300/0 conn(0x7f56840afcd0 0x7f56841601f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:51.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.549+0000 7f568a200640 1 -- 172.21.15.167:0/1501027830 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f56840d04a0 msgr2=0x7f56840b55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:51.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.549+0000 7f568a200640 1 --2- 172.21.15.167:0/1501027830 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f56840d04a0 0x7f56840b55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:51.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.549+0000 7f568a200640 1 -- 172.21.15.167:0/1501027830 >> v2:172.21.15.204:3300/0 conn(0x7f56840afcd0 msgr2=0x7f56841601f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:51.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.549+0000 7f568a200640 1 --2- 172.21.15.167:0/1501027830 >> v2:172.21.15.204:3300/0 conn(0x7f56840afcd0 0x7f56841601f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:51.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.549+0000 7f568a200640 1 -- 172.21.15.167:0/1501027830 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56840b4ae0 con 0x7f56840cf260 2024-05-11T12:10:51.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.550+0000 7f568a200640 1 --2- 172.21.15.167:0/1501027830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56840cf260 0x7f56840cf6f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f56840ce830 tx=0x7f567c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcc5de0591d2b2af server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:51.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.551+0000 7f5683e00640 1 -- 172.21.15.167:0/1501027830 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c036070 con 0x7f56840cf260 2024-05-11T12:10:51.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.551+0000 7f5683e00640 1 -- 172.21.15.167:0/1501027830 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f567c031600 con 0x7f56840cf260 2024-05-11T12:10:51.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.551+0000 7f5683e00640 1 -- 172.21.15.167:0/1501027830 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c0319d0 con 0x7f56840cf260 2024-05-11T12:10:51.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.551+0000 7f568ac00640 1 -- 172.21.15.167:0/1501027830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56840cf260 msgr2=0x7f56840cf6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:51.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.551+0000 7f568ac00640 1 --2- 172.21.15.167:0/1501027830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56840cf260 0x7f56840cf6f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f56840ce830 tx=0x7f567c02a6a0 comp rx=0 tx=0).stop 2024-05-11T12:10:51.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.551+0000 7f568ac00640 1 -- 172.21.15.167:0/1501027830 shutdown_connections 2024-05-11T12:10:51.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.551+0000 7f568ac00640 1 --2- 172.21.15.167:0/1501027830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56840cf260 0x7f56840cf6f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:51.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.551+0000 7f568ac00640 1 --2- 172.21.15.167:0/1501027830 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f56840d04a0 0x7f56840b55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:51.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.551+0000 7f568ac00640 1 --2- 172.21.15.167:0/1501027830 >> v2:172.21.15.204:3300/0 conn(0x7f56840afcd0 0x7f56841601f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:51.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.551+0000 7f568ac00640 1 -- 172.21.15.167:0/1501027830 >> 172.21.15.167:0/1501027830 conn(0x7f56840b3460 msgr2=0x7f56840d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:51.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.552+0000 7f568ac00640 1 -- 172.21.15.167:0/1501027830 shutdown_connections 2024-05-11T12:10:51.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.552+0000 7f568ac00640 1 -- 172.21.15.167:0/1501027830 wait complete. 2024-05-11T12:10:51.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.553+0000 7f568ac00640 1 Processor -- start 2024-05-11T12:10:51.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.553+0000 7f568ac00640 1 -- start start 2024-05-11T12:10:51.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.554+0000 7f568ac00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56840afcd0 0x7f568406f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:51.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.554+0000 7f568ac00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56840cf260 0x7f5684154930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:51.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.554+0000 7f568ac00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f56840d04a0 0x7f568406d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:51.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.554+0000 7f568ac00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f56840cff40 con 0x7f56840cf260 2024-05-11T12:10:51.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.554+0000 7f568ac00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f56840b00b0 con 0x7f56840afcd0 2024-05-11T12:10:51.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.554+0000 7f568ac00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f56840d00c0 con 0x7f56840d04a0 2024-05-11T12:10:51.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.555+0000 7f5688e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56840cf260 0x7f5684154930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:51.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.555+0000 7f5688e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56840cf260 0x7f5684154930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59086/0 (socket says 172.21.15.167:59086) 2024-05-11T12:10:51.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.555+0000 7f5688e00640 1 -- 172.21.15.167:0/2019087335 learned_addr learned my addr 172.21.15.167:0/2019087335 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:10:51.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.555+0000 7f568a200640 1 --2- 172.21.15.167:0/2019087335 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f56840d04a0 0x7f568406d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:51.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.555+0000 7f5689800640 1 --2- 172.21.15.167:0/2019087335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56840afcd0 0x7f568406f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:51.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.555+0000 7f5688e00640 1 -- 172.21.15.167:0/2019087335 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f56840d04a0 msgr2=0x7f568406d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:51.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.555+0000 7f5688e00640 1 --2- 172.21.15.167:0/2019087335 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f56840d04a0 0x7f568406d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:51.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.555+0000 7f5688e00640 1 -- 172.21.15.167:0/2019087335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56840afcd0 msgr2=0x7f568406f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:51.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.555+0000 7f5688e00640 1 --2- 172.21.15.167:0/2019087335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56840afcd0 0x7f568406f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:51.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.555+0000 7f5688e00640 1 -- 172.21.15.167:0/2019087335 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f568406e200 con 0x7f56840cf260 2024-05-11T12:10:51.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.555+0000 7f568a200640 1 --2- 172.21.15.167:0/2019087335 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f56840d04a0 0x7f568406d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:10:51.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.555+0000 7f5688e00640 1 --2- 172.21.15.167:0/2019087335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56840cf260 0x7f5684154930 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f567000d5f0 tx=0x7f567000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:51.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.556+0000 7f5681600640 1 -- 172.21.15.167:0/2019087335 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56700125f0 con 0x7f56840cf260 2024-05-11T12:10:51.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.556+0000 7f5681600640 1 -- 172.21.15.167:0/2019087335 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5670019070 con 0x7f56840cf260 2024-05-11T12:10:51.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.556+0000 7f5681600640 1 -- 172.21.15.167:0/2019087335 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56700129d0 con 0x7f56840cf260 2024-05-11T12:10:51.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.556+0000 7f568ac00640 1 -- 172.21.15.167:0/2019087335 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f568406e4e0 con 0x7f56840cf260 2024-05-11T12:10:51.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.556+0000 7f568ac00640 1 -- 172.21.15.167:0/2019087335 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f568406eaa0 con 0x7f56840cf260 2024-05-11T12:10:51.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.557+0000 7f568ac00640 1 -- 172.21.15.167:0/2019087335 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56840694b0 con 0x7f56840cf260 2024-05-11T12:10:51.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.558+0000 7f5681600640 1 -- 172.21.15.167:0/2019087335 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5670012b70 con 0x7f56840cf260 2024-05-11T12:10:51.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.559+0000 7f5681600640 1 --2- 172.21.15.167:0/2019087335 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f56640762d0 0x7f5664078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:51.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.559+0000 7f5681600640 1 -- 172.21.15.167:0/2019087335 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f5670097c20 con 0x7f56840cf260 2024-05-11T12:10:51.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.559+0000 7f5689800640 1 --2- 172.21.15.167:0/2019087335 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f56640762d0 0x7f5664078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:51.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.560+0000 7f5689800640 1 --2- 172.21.15.167:0/2019087335 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f56640762d0 0x7f5664078790 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5678006fd0 tx=0x7f5678008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:51.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.563+0000 7f5681600640 1 -- 172.21.15.167:0/2019087335 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f56700612d0 con 0x7f56840cf260 2024-05-11T12:10:51.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.769+0000 7f568ac00640 1 -- 172.21.15.167:0/2019087335 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f56840cf040 con 0x7f56840cf260 2024-05-11T12:10:51.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.772+0000 7f5681600640 1 -- 172.21.15.167:0/2019087335 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18562 (secure 0 0 0) 0x7f5670060bd0 con 0x7f56840cf260 2024-05-11T12:10:51.773 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:10:51.773 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":27,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:10:50.284769+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:10:43.084511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T12:10:44.157782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-11T12:10:47.814674+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:10:51.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.775+0000 7f568ac00640 1 -- 172.21.15.167:0/2019087335 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f56640762d0 msgr2=0x7f5664078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:51.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.775+0000 7f568ac00640 1 --2- 172.21.15.167:0/2019087335 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f56640762d0 0x7f5664078790 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5678006fd0 tx=0x7f5678008040 comp rx=0 tx=0).stop 2024-05-11T12:10:51.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.775+0000 7f568ac00640 1 -- 172.21.15.167:0/2019087335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56840cf260 msgr2=0x7f5684154930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:51.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.775+0000 7f568ac00640 1 --2- 172.21.15.167:0/2019087335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56840cf260 0x7f5684154930 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f567000d5f0 tx=0x7f567000dab0 comp rx=0 tx=0).stop 2024-05-11T12:10:51.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.775+0000 7f568ac00640 1 -- 172.21.15.167:0/2019087335 shutdown_connections 2024-05-11T12:10:51.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.775+0000 7f568ac00640 1 --2- 172.21.15.167:0/2019087335 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f56840d04a0 0x7f568406d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:51.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.775+0000 7f568ac00640 1 --2- 172.21.15.167:0/2019087335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56840cf260 0x7f5684154930 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:51.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.775+0000 7f568ac00640 1 --2- 172.21.15.167:0/2019087335 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f56640762d0 0x7f5664078790 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:51.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.775+0000 7f568ac00640 1 --2- 172.21.15.167:0/2019087335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56840afcd0 0x7f568406f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:51.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.775+0000 7f568ac00640 1 -- 172.21.15.167:0/2019087335 >> 172.21.15.167:0/2019087335 conn(0x7f56840b3460 msgr2=0x7f568415fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:51.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.775+0000 7f568ac00640 1 -- 172.21.15.167:0/2019087335 shutdown_connections 2024-05-11T12:10:51.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.775+0000 7f568ac00640 1 -- 172.21.15.167:0/2019087335 wait complete. 2024-05-11T12:10:51.794 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:10:51.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.990+0000 7f6303200640 1 Processor -- start 2024-05-11T12:10:51.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.990+0000 7f6303200640 1 -- start start 2024-05-11T12:10:51.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.992+0000 7f6303200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f630415f460 0x7f63040b83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:51.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.992+0000 7f6303200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f63040d42c0 0x7f63040d0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:51.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.992+0000 7f6303200640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63040cf990 con 0x7f63040b88e0 2024-05-11T12:10:51.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.992+0000 7f6303200640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f63040cfc70 con 0x7f63040d42c0 2024-05-11T12:10:51.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.992+0000 7f6303200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f63040cff50 con 0x7f630415f460 2024-05-11T12:10:51.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.992+0000 7f6302800640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f63040d42c0 0x7f63040d0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:51.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.992+0000 7f6302800640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f63040d42c0 0x7f63040d0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59094/0 (socket says 172.21.15.167:59094) 2024-05-11T12:10:51.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.992+0000 7f6302800640 1 -- 172.21.15.167:0/2447070458 learned_addr learned my addr 172.21.15.167:0/2447070458 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:10:51.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.992+0000 7f62fbe00640 1 --1- 172.21.15.167:0/2447070458 >> v1:172.21.15.167:6789/0 conn(0x7f63040b88e0 0x7f63040d3bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:57186/0 (socket says 172.21.15.167:57186) 2024-05-11T12:10:51.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.993+0000 7f6301e00640 1 --2- 172.21.15.167:0/2447070458 >> v2:172.21.15.204:3300/0 conn(0x7f630415f460 0x7f63040b83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:51.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.993+0000 7f6302800640 1 -- 172.21.15.167:0/2447070458 >> v2:172.21.15.204:3300/0 conn(0x7f630415f460 msgr2=0x7f63040b83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:51.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.993+0000 7f6302800640 1 --2- 172.21.15.167:0/2447070458 >> v2:172.21.15.204:3300/0 conn(0x7f630415f460 0x7f63040b83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:51.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.993+0000 7f6302800640 1 -- 172.21.15.167:0/2447070458 >> v1:172.21.15.167:6789/0 conn(0x7f63040b88e0 legacy=0x7f63040d3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:51.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.993+0000 7f6302800640 1 -- 172.21.15.167:0/2447070458 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63040d0230 con 0x7f63040d42c0 2024-05-11T12:10:51.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.993+0000 7f6301400640 1 -- 172.21.15.167:0/2447070458 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1740772720 0 0) 0x7f63040cf990 con 0x7f63040b88e0 2024-05-11T12:10:51.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.994+0000 7f6302800640 1 --2- 172.21.15.167:0/2447070458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63040d42c0 0x7f63040d0a30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f63040d2b50 tx=0x7f62fc02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=843242376e6b9c0a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:51.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.994+0000 7f6301400640 1 -- 172.21.15.167:0/2447070458 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62fc036070 con 0x7f63040d42c0 2024-05-11T12:10:51.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.994+0000 7f6301400640 1 -- 172.21.15.167:0/2447070458 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62fc031600 con 0x7f63040d42c0 2024-05-11T12:10:51.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.994+0000 7f6301400640 1 -- 172.21.15.167:0/2447070458 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62fc0319d0 con 0x7f63040d42c0 2024-05-11T12:10:51.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.995+0000 7f6303200640 1 -- 172.21.15.167:0/2447070458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63040d42c0 msgr2=0x7f63040d0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:51.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.995+0000 7f6303200640 1 --2- 172.21.15.167:0/2447070458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63040d42c0 0x7f63040d0a30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f63040d2b50 tx=0x7f62fc02a730 comp rx=0 tx=0).stop 2024-05-11T12:10:51.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.995+0000 7f6303200640 1 -- 172.21.15.167:0/2447070458 shutdown_connections 2024-05-11T12:10:51.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.995+0000 7f6303200640 1 --2- 172.21.15.167:0/2447070458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63040d42c0 0x7f63040d0a30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:51.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.995+0000 7f6303200640 1 --2- 172.21.15.167:0/2447070458 >> v2:172.21.15.204:3300/0 conn(0x7f630415f460 0x7f63040b83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:51.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.995+0000 7f6303200640 1 -- 172.21.15.167:0/2447070458 >> 172.21.15.167:0/2447070458 conn(0x7f63040d1a50 msgr2=0x7f63040d1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:51.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.995+0000 7f6303200640 1 -- 172.21.15.167:0/2447070458 shutdown_connections 2024-05-11T12:10:51.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.995+0000 7f6303200640 1 -- 172.21.15.167:0/2447070458 wait complete. 2024-05-11T12:10:51.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.997+0000 7f6303200640 1 Processor -- start 2024-05-11T12:10:51.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.997+0000 7f6303200640 1 -- start start 2024-05-11T12:10:51.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.998+0000 7f6303200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63040b88e0 0x7f63041f46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:51.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.998+0000 7f6303200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63040d42c0 0x7f6304155f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:52.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.998+0000 7f6303200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f630415f460 0x7f6304157b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:52.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.998+0000 7f6303200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f630406cc40 con 0x7f63040d42c0 2024-05-11T12:10:52.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.998+0000 7f6303200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f63040cf620 con 0x7f63040b88e0 2024-05-11T12:10:52.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.998+0000 7f6303200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f630406cdc0 con 0x7f630415f460 2024-05-11T12:10:52.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.998+0000 7f62fbe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63040d42c0 0x7f6304155f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:52.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.998+0000 7f62fbe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63040d42c0 0x7f6304155f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59100/0 (socket says 172.21.15.167:59100) 2024-05-11T12:10:52.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.998+0000 7f62fbe00640 1 -- 172.21.15.167:0/1206456898 learned_addr learned my addr 172.21.15.167:0/1206456898 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:10:52.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.998+0000 7f6301e00640 1 --2- 172.21.15.167:0/1206456898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63040b88e0 0x7f63041f46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:52.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.998+0000 7f6302800640 1 --2- 172.21.15.167:0/1206456898 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f630415f460 0x7f6304157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:52.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.998+0000 7f62fbe00640 1 -- 172.21.15.167:0/1206456898 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f630415f460 msgr2=0x7f6304157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:52.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.998+0000 7f62fbe00640 1 --2- 172.21.15.167:0/1206456898 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f630415f460 0x7f6304157b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:52.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.998+0000 7f62fbe00640 1 -- 172.21.15.167:0/1206456898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63040b88e0 msgr2=0x7f63041f46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:52.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.998+0000 7f62fbe00640 1 --2- 172.21.15.167:0/1206456898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63040b88e0 0x7f63041f46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:52.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.998+0000 7f62fbe00640 1 -- 172.21.15.167:0/1206456898 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63040cddc0 con 0x7f63040d42c0 2024-05-11T12:10:52.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.999+0000 7f62fbe00640 1 --2- 172.21.15.167:0/1206456898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63040d42c0 0x7f6304155f50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f62ec00c480 tx=0x7f62ec00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:52.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.999+0000 7f62fa000640 1 -- 172.21.15.167:0/1206456898 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ec018070 con 0x7f63040d42c0 2024-05-11T12:10:52.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.999+0000 7f62fa000640 1 -- 172.21.15.167:0/1206456898 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62ec007500 con 0x7f63040d42c0 2024-05-11T12:10:52.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.999+0000 7f6303200640 1 -- 172.21.15.167:0/1206456898 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6304158820 con 0x7f63040d42c0 2024-05-11T12:10:52.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.999+0000 7f62fa000640 1 -- 172.21.15.167:0/1206456898 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ec004a20 con 0x7f63040d42c0 2024-05-11T12:10:52.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:51.999+0000 7f6303200640 1 -- 172.21.15.167:0/1206456898 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63040cc310 con 0x7f63040d42c0 2024-05-11T12:10:52.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.000+0000 7f6303200640 1 -- 172.21.15.167:0/1206456898 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63040d32b0 con 0x7f63040d42c0 2024-05-11T12:10:52.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.001+0000 7f62fa000640 1 -- 172.21.15.167:0/1206456898 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f62ec01b070 con 0x7f63040d42c0 2024-05-11T12:10:52.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.002+0000 7f62fa000640 1 --2- 172.21.15.167:0/1206456898 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f62e00762d0 0x7f62e0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:52.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.002+0000 7f62fa000640 1 -- 172.21.15.167:0/1206456898 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f62ec096c60 con 0x7f63040d42c0 2024-05-11T12:10:52.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.002+0000 7f6301e00640 1 --2- 172.21.15.167:0/1206456898 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f62e00762d0 0x7f62e0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:52.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.003+0000 7f6301e00640 1 --2- 172.21.15.167:0/1206456898 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f62e00762d0 0x7f62e0078790 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f62f4006fd0 tx=0x7f62f4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:52.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.007+0000 7f62fa000640 1 -- 172.21.15.167:0/1206456898 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f63040d32b0 con 0x7f63040d42c0 2024-05-11T12:10:52.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.200+0000 7f6303200640 1 -- 172.21.15.167:0/1206456898 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f63040d4900 con 0x7f63040d42c0 2024-05-11T12:10:52.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.203+0000 7f62fa000640 1 -- 172.21.15.167:0/1206456898 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18562 (secure 0 0 0) 0x7f62ec060270 con 0x7f63040d42c0 2024-05-11T12:10:52.204 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:10:52.204 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":27,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:10:50.284769+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:10:43.084511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T12:10:44.157782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-11T12:10:47.814674+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:10:52.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.206+0000 7f6303200640 1 -- 172.21.15.167:0/1206456898 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f62e00762d0 msgr2=0x7f62e0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:52.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.206+0000 7f6303200640 1 --2- 172.21.15.167:0/1206456898 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f62e00762d0 0x7f62e0078790 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f62f4006fd0 tx=0x7f62f4008040 comp rx=0 tx=0).stop 2024-05-11T12:10:52.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.206+0000 7f6303200640 1 -- 172.21.15.167:0/1206456898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63040d42c0 msgr2=0x7f6304155f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:52.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.206+0000 7f6303200640 1 --2- 172.21.15.167:0/1206456898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63040d42c0 0x7f6304155f50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f62ec00c480 tx=0x7f62ec00c940 comp rx=0 tx=0).stop 2024-05-11T12:10:52.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.206+0000 7f6303200640 1 -- 172.21.15.167:0/1206456898 shutdown_connections 2024-05-11T12:10:52.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.206+0000 7f6303200640 1 --2- 172.21.15.167:0/1206456898 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f630415f460 0x7f6304157b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:52.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.206+0000 7f6303200640 1 --2- 172.21.15.167:0/1206456898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63040d42c0 0x7f6304155f50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:52.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.206+0000 7f6303200640 1 --2- 172.21.15.167:0/1206456898 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f62e00762d0 0x7f62e0078790 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:52.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.206+0000 7f6303200640 1 --2- 172.21.15.167:0/1206456898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63040b88e0 0x7f63041f46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:52.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.206+0000 7f6303200640 1 -- 172.21.15.167:0/1206456898 >> 172.21.15.167:0/1206456898 conn(0x7f63040d1a50 msgr2=0x7f63040d2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:52.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.206+0000 7f6303200640 1 -- 172.21.15.167:0/1206456898 shutdown_connections 2024-05-11T12:10:52.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.206+0000 7f6303200640 1 -- 172.21.15.167:0/1206456898 wait complete. 2024-05-11T12:10:52.238 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:10:52.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.416+0000 7f0051a00640 1 Processor -- start 2024-05-11T12:10:52.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.416+0000 7f0051a00640 1 -- start start 2024-05-11T12:10:52.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.418+0000 7f0051a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f004c0b50a0 0x7f004c073cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:52.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.418+0000 7f0051a00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f004c15fb90 con 0x7f004c13f290 2024-05-11T12:10:52.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.418+0000 7f0051a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f004c15fe70 con 0x7f004c13eeb0 2024-05-11T12:10:52.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.418+0000 7f0051a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f004c15d5b0 con 0x7f004c0b50a0 2024-05-11T12:10:52.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.419+0000 7f0042000640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f004c13f290 0x7f004c0735c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:57194/0 (socket says 172.21.15.167:57194) 2024-05-11T12:10:52.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.419+0000 7f0042000640 1 -- 172.21.15.167:0/1848463858 learned_addr learned my addr 172.21.15.167:0/1848463858 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:10:52.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.419+0000 7f004a000640 1 -- 172.21.15.167:0/1848463858 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 249865369 0 0) 0x7f004c15fb90 con 0x7f004c13f290 2024-05-11T12:10:52.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.419+0000 7f004a000640 1 -- 172.21.15.167:0/1848463858 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0030003520 con 0x7f004c13f290 2024-05-11T12:10:52.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.419+0000 7f004b400640 1 --2- 172.21.15.167:0/1848463858 >> v2:172.21.15.204:3300/0 conn(0x7f004c0b50a0 0x7f004c073cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:52.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.419+0000 7f004a000640 1 -- 172.21.15.167:0/1848463858 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2427753918 0 0) 0x7f0030003520 con 0x7f004c13f290 2024-05-11T12:10:52.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.419+0000 7f004a000640 1 -- 172.21.15.167:0/1848463858 >> v2:172.21.15.204:3300/0 conn(0x7f004c0b50a0 msgr2=0x7f004c073cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:52.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.419+0000 7f004a000640 1 --2- 172.21.15.167:0/1848463858 >> v2:172.21.15.204:3300/0 conn(0x7f004c0b50a0 0x7f004c073cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:52.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.419+0000 7f004a000640 1 -- 172.21.15.167:0/1848463858 >> v1:172.21.15.204:6789/0 conn(0x7f004c13eeb0 legacy=0x7f004c06f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:52.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.419+0000 7f004a000640 1 -- 172.21.15.167:0/1848463858 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f004c160150 con 0x7f004c13f290 2024-05-11T12:10:52.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.419+0000 7f004a000640 1 -- 172.21.15.167:0/1848463858 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f00340016f0 con 0x7f004c13f290 2024-05-11T12:10:52.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.419+0000 7f004a000640 1 -- 172.21.15.167:0/1848463858 >> v1:172.21.15.167:6789/0 conn(0x7f004c13f290 legacy=0x7f004c0735c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:52.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.419+0000 7f004a000640 1 --2- 172.21.15.167:0/1848463858 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0030003aa0 0x7f0030003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:52.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.419+0000 7f004a000640 1 --2- 172.21.15.167:0/1848463858 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0030004800 0x7f0030004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:52.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.420+0000 7f004a000640 1 --2- 172.21.15.167:0/1848463858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00300053c0 0x7f003000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:52.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.420+0000 7f004a000640 1 -- 172.21.15.167:0/1848463858 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f003000c270 con 0x7f0030003aa0 2024-05-11T12:10:52.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.420+0000 7f004a000640 1 -- 172.21.15.167:0/1848463858 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f003000c420 con 0x7f00300053c0 2024-05-11T12:10:52.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.420+0000 7f004a000640 1 -- 172.21.15.167:0/1848463858 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f003000e830 con 0x7f0030004800 2024-05-11T12:10:52.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.420+0000 7f004aa00640 1 --2- 172.21.15.167:0/1848463858 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0030003aa0 0x7f0030003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:52.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.420+0000 7f004b400640 1 --2- 172.21.15.167:0/1848463858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00300053c0 0x7f003000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:52.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.420+0000 7f004aa00640 1 -- 172.21.15.167:0/1848463858 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0030004800 msgr2=0x7f0030004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:52.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.420+0000 7f004aa00640 1 --2- 172.21.15.167:0/1848463858 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0030004800 0x7f0030004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:52.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.420+0000 7f004aa00640 1 -- 172.21.15.167:0/1848463858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00300053c0 msgr2=0x7f003000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:52.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.420+0000 7f004aa00640 1 --2- 172.21.15.167:0/1848463858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00300053c0 0x7f003000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:52.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.420+0000 7f004aa00640 1 -- 172.21.15.167:0/1848463858 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0030003520 con 0x7f0030003aa0 2024-05-11T12:10:52.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.420+0000 7f004b400640 1 --2- 172.21.15.167:0/1848463858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00300053c0 0x7f003000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:10:52.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.420+0000 7f004b400640 1 -- 172.21.15.167:0/1848463858 reap_dead start 2024-05-11T12:10:52.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.421+0000 7f004aa00640 1 --2- 172.21.15.167:0/1848463858 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0030003aa0 0x7f0030003ec0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f003c001790 tx=0x7f0030005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f22d8bcccfb81acf server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:52.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.421+0000 7f004a000640 1 -- 172.21.15.167:0/1848463858 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003c0013b0 con 0x7f0030003aa0 2024-05-11T12:10:52.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.421+0000 7f004a000640 1 -- 172.21.15.167:0/1848463858 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f003c0052d0 con 0x7f0030003aa0 2024-05-11T12:10:52.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.421+0000 7f004a000640 1 -- 172.21.15.167:0/1848463858 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003c02aa80 con 0x7f0030003aa0 2024-05-11T12:10:52.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.422+0000 7f0051a00640 1 -- 172.21.15.167:0/1848463858 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0030003aa0 msgr2=0x7f0030003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:52.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.422+0000 7f0051a00640 1 --2- 172.21.15.167:0/1848463858 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0030003aa0 0x7f0030003ec0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f003c001790 tx=0x7f0030005830 comp rx=0 tx=0).stop 2024-05-11T12:10:52.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.422+0000 7f0051a00640 1 -- 172.21.15.167:0/1848463858 shutdown_connections 2024-05-11T12:10:52.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.422+0000 7f0051a00640 1 --2- 172.21.15.167:0/1848463858 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0030003aa0 0x7f0030003ec0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:52.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.422+0000 7f0051a00640 1 -- 172.21.15.167:0/1848463858 >> 172.21.15.167:0/1848463858 conn(0x7f004c0cd8f0 msgr2=0x7f004c0cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:52.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.422+0000 7f0051a00640 1 -- 172.21.15.167:0/1848463858 shutdown_connections 2024-05-11T12:10:52.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.422+0000 7f0051a00640 1 -- 172.21.15.167:0/1848463858 wait complete. 2024-05-11T12:10:52.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.424+0000 7f0051a00640 1 Processor -- start 2024-05-11T12:10:52.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.424+0000 7f0051a00640 1 -- start start 2024-05-11T12:10:52.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.425+0000 7f0051a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0030003aa0 0x7f004c0aa9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:52.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.425+0000 7f0051a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f004c15ebb0 0x7f004c15b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:52.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.425+0000 7f0051a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f004c15f030 0x7f004c15b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:52.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.425+0000 7f0051a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f004c0735c0 con 0x7f004c15ebb0 2024-05-11T12:10:52.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.425+0000 7f0051a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f004c160710 con 0x7f004c15f030 2024-05-11T12:10:52.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.425+0000 7f0051a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f004c073740 con 0x7f0030003aa0 2024-05-11T12:10:52.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.425+0000 7f0042000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f004c15ebb0 0x7f004c15b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:52.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.425+0000 7f0042000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f004c15ebb0 0x7f004c15b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59106/0 (socket says 172.21.15.167:59106) 2024-05-11T12:10:52.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.425+0000 7f0042000640 1 -- 172.21.15.167:0/3700432552 learned_addr learned my addr 172.21.15.167:0/3700432552 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:10:52.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.425+0000 7f004aa00640 1 --2- 172.21.15.167:0/3700432552 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0030003aa0 0x7f004c0aa9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:52.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.425+0000 7f004b400640 1 --2- 172.21.15.167:0/3700432552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f004c15f030 0x7f004c15b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:52.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.425+0000 7f0042000640 1 -- 172.21.15.167:0/3700432552 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0030003aa0 msgr2=0x7f004c0aa9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:52.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.425+0000 7f0042000640 1 --2- 172.21.15.167:0/3700432552 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0030003aa0 0x7f004c0aa9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:52.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.425+0000 7f0042000640 1 -- 172.21.15.167:0/3700432552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f004c15f030 msgr2=0x7f004c15b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:52.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.425+0000 7f0042000640 1 --2- 172.21.15.167:0/3700432552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f004c15f030 0x7f004c15b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:52.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.425+0000 7f0042000640 1 -- 172.21.15.167:0/3700432552 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f004c15db10 con 0x7f004c15ebb0 2024-05-11T12:10:52.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.425+0000 7f004aa00640 1 --2- 172.21.15.167:0/3700432552 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0030003aa0 0x7f004c0aa9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:10:52.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.425+0000 7f004b400640 1 --2- 172.21.15.167:0/3700432552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f004c15f030 0x7f004c15b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:10:52.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.425+0000 7f0042000640 1 --2- 172.21.15.167:0/3700432552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f004c15ebb0 0x7f004c15b050 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f003400c870 tx=0x7f003400cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:52.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.426+0000 7f0043400640 1 -- 172.21.15.167:0/3700432552 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00340074e0 con 0x7f004c15ebb0 2024-05-11T12:10:52.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.426+0000 7f0043400640 1 -- 172.21.15.167:0/3700432552 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00340047e0 con 0x7f004c15ebb0 2024-05-11T12:10:52.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.426+0000 7f0051a00640 1 -- 172.21.15.167:0/3700432552 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f004c15ddf0 con 0x7f004c15ebb0 2024-05-11T12:10:52.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.426+0000 7f0043400640 1 -- 172.21.15.167:0/3700432552 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0034004bd0 con 0x7f004c15ebb0 2024-05-11T12:10:52.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.426+0000 7f0051a00640 1 -- 172.21.15.167:0/3700432552 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f004c15e3b0 con 0x7f004c15ebb0 2024-05-11T12:10:52.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.427+0000 7f0051a00640 1 -- 172.21.15.167:0/3700432552 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f004c069590 con 0x7f004c15ebb0 2024-05-11T12:10:52.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.428+0000 7f0043400640 1 -- 172.21.15.167:0/3700432552 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0034011070 con 0x7f004c15ebb0 2024-05-11T12:10:52.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.429+0000 7f0043400640 1 --2- 172.21.15.167:0/3700432552 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0028076300 0x7f00280787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:52.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.429+0000 7f0043400640 1 -- 172.21.15.167:0/3700432552 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f0034097760 con 0x7f004c15ebb0 2024-05-11T12:10:52.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.429+0000 7f004aa00640 1 --2- 172.21.15.167:0/3700432552 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0028076300 0x7f00280787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:52.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.430+0000 7f004aa00640 1 --2- 172.21.15.167:0/3700432552 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0028076300 0x7f00280787c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f003c032000 tx=0x7f003c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:52.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.434+0000 7f0043400640 1 -- 172.21.15.167:0/3700432552 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0034060ce0 con 0x7f004c15ebb0 2024-05-11T12:10:52.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.684+0000 7f0051a00640 1 -- 172.21.15.167:0/3700432552 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f004c075b40 con 0x7f004c15ebb0 2024-05-11T12:10:52.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.685+0000 7f0043400640 1 -- 172.21.15.167:0/3700432552 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5189 (secure 0 0 0) 0x7f0034001cc0 con 0x7f004c15ebb0 2024-05-11T12:10:52.686 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:10:52.686 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3127277486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3127277486},{"type":"v1","addr":"172.21.15.167:6835","nonce":3127277486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2547133600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2547133600},{"type":"v1","addr":"172.21.15.167:6837","nonce":2547133600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3200960100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3200960100},{"type":"v1","addr":"172.21.15.204:6834","nonce":3200960100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:10:45.225745+0000","modified":"2024-05-11T12:10:50.284984+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/1889842375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":1889842375},{"type":"v1","addr":"172.21.15.204:6835","nonce":1889842375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-05-11T12:10:52.686 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 7 2024-05-11T12:10:52.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.688+0000 7f0051a00640 1 -- 172.21.15.167:0/3700432552 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0028076300 msgr2=0x7f00280787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:52.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.688+0000 7f0051a00640 1 --2- 172.21.15.167:0/3700432552 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0028076300 0x7f00280787c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f003c032000 tx=0x7f003c02e000 comp rx=0 tx=0).stop 2024-05-11T12:10:52.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.689+0000 7f0051a00640 1 -- 172.21.15.167:0/3700432552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f004c15ebb0 msgr2=0x7f004c15b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:52.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.689+0000 7f0051a00640 1 --2- 172.21.15.167:0/3700432552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f004c15ebb0 0x7f004c15b050 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f003400c870 tx=0x7f003400cd30 comp rx=0 tx=0).stop 2024-05-11T12:10:52.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.689+0000 7f0051a00640 1 -- 172.21.15.167:0/3700432552 shutdown_connections 2024-05-11T12:10:52.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.689+0000 7f0051a00640 1 --2- 172.21.15.167:0/3700432552 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0028076300 0x7f00280787c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:52.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.689+0000 7f0051a00640 1 --2- 172.21.15.167:0/3700432552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f004c15f030 0x7f004c15b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:52.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.689+0000 7f0051a00640 1 --2- 172.21.15.167:0/3700432552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f004c15ebb0 0x7f004c15b050 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:52.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.689+0000 7f0051a00640 1 --2- 172.21.15.167:0/3700432552 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0030003aa0 0x7f004c0aa9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:52.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.689+0000 7f0051a00640 1 -- 172.21.15.167:0/3700432552 >> 172.21.15.167:0/3700432552 conn(0x7f004c0cd8f0 msgr2=0x7f004c071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:52.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.689+0000 7f0051a00640 1 -- 172.21.15.167:0/3700432552 shutdown_connections 2024-05-11T12:10:52.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.689+0000 7f0051a00640 1 -- 172.21.15.167:0/3700432552 wait complete. 2024-05-11T12:10:52.708 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:10:52.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.898+0000 7f36f7e00640 1 Processor -- start 2024-05-11T12:10:52.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.898+0000 7f36f7e00640 1 -- start start 2024-05-11T12:10:52.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.899+0000 7f36f7e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f36f0157750 0x7f36f015d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:52.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.899+0000 7f36f7e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f36f00b3ea0 0x7f36f0161490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:52.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.899+0000 7f36f7e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36f015dc80 con 0x7f36f01619d0 2024-05-11T12:10:52.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.899+0000 7f36f7e00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f36f0161000 con 0x7f36f00b3ea0 2024-05-11T12:10:52.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.899+0000 7f36f7e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f36f015df30 con 0x7f36f0157750 2024-05-11T12:10:52.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.900+0000 7f36f4a00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f36f00b3ea0 0x7f36f0161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:52.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.900+0000 7f36f4a00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f36f00b3ea0 0x7f36f0161490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59122/0 (socket says 172.21.15.167:59122) 2024-05-11T12:10:52.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.900+0000 7f36f4a00640 1 -- 172.21.15.167:0/1008298667 learned_addr learned my addr 172.21.15.167:0/1008298667 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:10:52.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.900+0000 7f36f5400640 1 --2- 172.21.15.167:0/1008298667 >> v2:172.21.15.204:3300/0 conn(0x7f36f0157750 0x7f36f015d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:52.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.900+0000 7f36f4a00640 1 -- 172.21.15.167:0/1008298667 >> v2:172.21.15.204:3300/0 conn(0x7f36f0157750 msgr2=0x7f36f015d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:52.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.900+0000 7f36f4a00640 1 --2- 172.21.15.167:0/1008298667 >> v2:172.21.15.204:3300/0 conn(0x7f36f0157750 0x7f36f015d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:52.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.900+0000 7f36f4a00640 1 -- 172.21.15.167:0/1008298667 >> v1:172.21.15.204:6789/0 conn(0x7f36f01619d0 legacy=0x7f36f006d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:52.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.900+0000 7f36f4a00640 1 -- 172.21.15.167:0/1008298667 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36f015e210 con 0x7f36f00b3ea0 2024-05-11T12:10:52.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.901+0000 7f36f4a00640 1 --2- 172.21.15.167:0/1008298667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36f00b3ea0 0x7f36f0161490 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f36f00ce840 tx=0x7f36dc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7ded6400e106e7a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:52.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.901+0000 7f36efe00640 1 -- 172.21.15.167:0/1008298667 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36dc037070 con 0x7f36f00b3ea0 2024-05-11T12:10:52.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.901+0000 7f36efe00640 1 -- 172.21.15.167:0/1008298667 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36dc032440 con 0x7f36f00b3ea0 2024-05-11T12:10:52.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.901+0000 7f36efe00640 1 -- 172.21.15.167:0/1008298667 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36dc032810 con 0x7f36f00b3ea0 2024-05-11T12:10:52.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.902+0000 7f36f7e00640 1 -- 172.21.15.167:0/1008298667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36f00b3ea0 msgr2=0x7f36f0161490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:52.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.902+0000 7f36f7e00640 1 --2- 172.21.15.167:0/1008298667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36f00b3ea0 0x7f36f0161490 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f36f00ce840 tx=0x7f36dc02a1e0 comp rx=0 tx=0).stop 2024-05-11T12:10:52.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.902+0000 7f36f7e00640 1 -- 172.21.15.167:0/1008298667 shutdown_connections 2024-05-11T12:10:52.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.902+0000 7f36f7e00640 1 --2- 172.21.15.167:0/1008298667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36f00b3ea0 0x7f36f0161490 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:52.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.902+0000 7f36f7e00640 1 --2- 172.21.15.167:0/1008298667 >> v2:172.21.15.204:3300/0 conn(0x7f36f0157750 0x7f36f015d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:52.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.902+0000 7f36f7e00640 1 -- 172.21.15.167:0/1008298667 >> 172.21.15.167:0/1008298667 conn(0x7f36f00cd8f0 msgr2=0x7f36f00cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:52.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.902+0000 7f36f7e00640 1 -- 172.21.15.167:0/1008298667 shutdown_connections 2024-05-11T12:10:52.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.902+0000 7f36f7e00640 1 -- 172.21.15.167:0/1008298667 wait complete. 2024-05-11T12:10:52.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.904+0000 7f36f7e00640 1 Processor -- start 2024-05-11T12:10:52.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.904+0000 7f36f7e00640 1 -- start start 2024-05-11T12:10:52.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.905+0000 7f36f7e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36f00b3ea0 0x7f36f00af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:52.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.905+0000 7f36f7e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36f0157750 0x7f36f0143c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:52.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.905+0000 7f36f7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36f01619d0 0x7f36f00aa410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:52.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.905+0000 7f36f7e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f36f006db70 con 0x7f36f01619d0 2024-05-11T12:10:52.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.905+0000 7f36f7e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f36f0157b30 con 0x7f36f0157750 2024-05-11T12:10:52.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.905+0000 7f36f7e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f36f006dcf0 con 0x7f36f00b3ea0 2024-05-11T12:10:52.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.906+0000 7f36f5e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36f01619d0 0x7f36f00aa410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:52.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.906+0000 7f36f5e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36f01619d0 0x7f36f00aa410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59138/0 (socket says 172.21.15.167:59138) 2024-05-11T12:10:52.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.906+0000 7f36f5e00640 1 -- 172.21.15.167:0/3893984185 learned_addr learned my addr 172.21.15.167:0/3893984185 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:10:52.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.906+0000 7f36f4a00640 1 --2- 172.21.15.167:0/3893984185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36f0157750 0x7f36f0143c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:52.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.906+0000 7f36f5400640 1 --2- 172.21.15.167:0/3893984185 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36f00b3ea0 0x7f36f00af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:52.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.906+0000 7f36f5e00640 1 -- 172.21.15.167:0/3893984185 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36f00b3ea0 msgr2=0x7f36f00af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:52.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.906+0000 7f36f5e00640 1 --2- 172.21.15.167:0/3893984185 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36f00b3ea0 0x7f36f00af6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:52.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.906+0000 7f36f5e00640 1 -- 172.21.15.167:0/3893984185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36f0157750 msgr2=0x7f36f0143c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:52.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.906+0000 7f36f5e00640 1 --2- 172.21.15.167:0/3893984185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36f0157750 0x7f36f0143c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:52.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.906+0000 7f36f5e00640 1 -- 172.21.15.167:0/3893984185 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36f00afc00 con 0x7f36f01619d0 2024-05-11T12:10:52.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.906+0000 7f36f5400640 1 --2- 172.21.15.167:0/3893984185 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36f00b3ea0 0x7f36f00af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:10:52.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.906+0000 7f36f5e00640 1 --2- 172.21.15.167:0/3893984185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36f01619d0 0x7f36f00aa410 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f36e800b990 tx=0x7f36e800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:52.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.907+0000 7f36ed600640 1 -- 172.21.15.167:0/3893984185 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e8019070 con 0x7f36f01619d0 2024-05-11T12:10:52.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.907+0000 7f36ed600640 1 -- 172.21.15.167:0/3893984185 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36e800cba0 con 0x7f36f01619d0 2024-05-11T12:10:52.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.907+0000 7f36ed600640 1 -- 172.21.15.167:0/3893984185 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e8014440 con 0x7f36f01619d0 2024-05-11T12:10:52.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.907+0000 7f36f7e00640 1 -- 172.21.15.167:0/3893984185 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36f0142b80 con 0x7f36f01619d0 2024-05-11T12:10:52.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.907+0000 7f36f7e00640 1 -- 172.21.15.167:0/3893984185 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36f0143140 con 0x7f36f01619d0 2024-05-11T12:10:52.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.908+0000 7f36f7e00640 1 -- 172.21.15.167:0/3893984185 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36f0076420 con 0x7f36f01619d0 2024-05-11T12:10:52.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.909+0000 7f36ed600640 1 -- 172.21.15.167:0/3893984185 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f36e801c070 con 0x7f36f01619d0 2024-05-11T12:10:52.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.910+0000 7f36ed600640 1 --2- 172.21.15.167:0/3893984185 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f36d00762d0 0x7f36d0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:52.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.910+0000 7f36ed600640 1 -- 172.21.15.167:0/3893984185 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f36e80975f0 con 0x7f36f01619d0 2024-05-11T12:10:52.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.911+0000 7f36f5400640 1 --2- 172.21.15.167:0/3893984185 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f36d00762d0 0x7f36d0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:52.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.911+0000 7f36f5400640 1 --2- 172.21.15.167:0/3893984185 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f36d00762d0 0x7f36d0078790 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f36e4008890 tx=0x7f36e4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:52.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:52.915+0000 7f36ed600640 1 -- 172.21.15.167:0/3893984185 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f36f0076420 con 0x7f36f01619d0 2024-05-11T12:10:53.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.129+0000 7f36f7e00640 1 -- 172.21.15.167:0/3893984185 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f36f00b4570 con 0x7f36f01619d0 2024-05-11T12:10:53.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.132+0000 7f36ed600640 1 -- 172.21.15.167:0/3893984185 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18562 (secure 0 0 0) 0x7f36e8060ca0 con 0x7f36f01619d0 2024-05-11T12:10:53.132 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:10:53.132 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":27,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:10:50.284769+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:10:43.084511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T12:10:44.157782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-11T12:10:47.814674+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:10:53.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.135+0000 7f36f7e00640 1 -- 172.21.15.167:0/3893984185 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f36d00762d0 msgr2=0x7f36d0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:53.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.135+0000 7f36f7e00640 1 --2- 172.21.15.167:0/3893984185 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f36d00762d0 0x7f36d0078790 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f36e4008890 tx=0x7f36e4008040 comp rx=0 tx=0).stop 2024-05-11T12:10:53.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.135+0000 7f36f7e00640 1 -- 172.21.15.167:0/3893984185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36f01619d0 msgr2=0x7f36f00aa410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:53.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.135+0000 7f36f7e00640 1 --2- 172.21.15.167:0/3893984185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36f01619d0 0x7f36f00aa410 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f36e800b990 tx=0x7f36e800be50 comp rx=0 tx=0).stop 2024-05-11T12:10:53.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.135+0000 7f36f7e00640 1 -- 172.21.15.167:0/3893984185 shutdown_connections 2024-05-11T12:10:53.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.135+0000 7f36f7e00640 1 --2- 172.21.15.167:0/3893984185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36f01619d0 0x7f36f00aa410 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:53.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.135+0000 7f36f7e00640 1 --2- 172.21.15.167:0/3893984185 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f36d00762d0 0x7f36d0078790 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:53.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.135+0000 7f36f7e00640 1 --2- 172.21.15.167:0/3893984185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36f0157750 0x7f36f0143c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:53.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.135+0000 7f36f7e00640 1 --2- 172.21.15.167:0/3893984185 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36f00b3ea0 0x7f36f00af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:53.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.135+0000 7f36f7e00640 1 -- 172.21.15.167:0/3893984185 >> 172.21.15.167:0/3893984185 conn(0x7f36f00cd8f0 msgr2=0x7f36f006c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:53.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.135+0000 7f36f7e00640 1 -- 172.21.15.167:0/3893984185 shutdown_connections 2024-05-11T12:10:53.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.135+0000 7f36f7e00640 1 -- 172.21.15.167:0/3893984185 wait complete. 2024-05-11T12:10:53.557 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-11T12:10:45.225745+0000', 'modified': '2024-05-11T12:10:50.284984+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4285}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4285': {'gid': 4285, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.204:6835/1889842375', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6833', 'nonce': 1889842375}, {'type': 'v1', 'addr': '172.21.15.204:6835', 'nonce': 1889842375}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4285, 'qdb_cluster': [4285]} 2024-05-11T12:10:53.558 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-11T12:10:53.558 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-11T12:10:53.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.770+0000 7f6a96200640 1 Processor -- start 2024-05-11T12:10:53.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.770+0000 7f6a96200640 1 -- start start 2024-05-11T12:10:53.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.772+0000 7f6a96200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a9006f420 0x7f6a900b6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:53.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.772+0000 7f6a96200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6a90070050 0x7f6a90072a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:53.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.772+0000 7f6a96200640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a90156140 con 0x7f6a90072fa0 2024-05-11T12:10:53.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.772+0000 7f6a96200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f6a900b6e80 con 0x7f6a90070050 2024-05-11T12:10:53.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.772+0000 7f6a96200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6a9015bc60 con 0x7f6a9006f420 2024-05-11T12:10:53.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.772+0000 7f6a8fe00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f6a90072fa0 0x7f6a9015f250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:57204/0 (socket says 172.21.15.167:57204) 2024-05-11T12:10:53.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.773+0000 7f6a8fe00640 1 -- 172.21.15.167:0/1425756418 learned_addr learned my addr 172.21.15.167:0/1425756418 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:10:53.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.773+0000 7f6a8f400640 1 --2- 172.21.15.167:0/1425756418 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a9006f420 0x7f6a900b6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:53.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.773+0000 7f6a87e00640 1 --2- 172.21.15.167:0/1425756418 >> v2:172.21.15.204:3300/0 conn(0x7f6a90070050 0x7f6a90072a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:53.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.773+0000 7f6a8ea00640 1 -- 172.21.15.167:0/1425756418 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1241815037 0 0) 0x7f6a88001490 con 0x7f6a90072fa0 2024-05-11T12:10:53.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.773+0000 7f6a8ea00640 1 -- 172.21.15.167:0/1425756418 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a74003520 con 0x7f6a90072fa0 2024-05-11T12:10:53.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.773+0000 7f6a8ea00640 1 -- 172.21.15.167:0/1425756418 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3607525635 0 0) 0x7f6a74003520 con 0x7f6a90072fa0 2024-05-11T12:10:53.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.773+0000 7f6a8ea00640 1 -- 172.21.15.167:0/1425756418 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a9006f420 msgr2=0x7f6a900b6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:53.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.773+0000 7f6a8ea00640 1 --2- 172.21.15.167:0/1425756418 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a9006f420 0x7f6a900b6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:53.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.773+0000 7f6a8ea00640 1 -- 172.21.15.167:0/1425756418 >> v2:172.21.15.204:3300/0 conn(0x7f6a90070050 msgr2=0x7f6a90072a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:53.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.773+0000 7f6a8ea00640 1 --2- 172.21.15.167:0/1425756418 >> v2:172.21.15.204:3300/0 conn(0x7f6a90070050 0x7f6a90072a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:53.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.773+0000 7f6a8ea00640 1 -- 172.21.15.167:0/1425756418 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a9015bf20 con 0x7f6a90072fa0 2024-05-11T12:10:53.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.773+0000 7f6a8ea00640 1 -- 172.21.15.167:0/1425756418 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f6a88001c40 con 0x7f6a90072fa0 2024-05-11T12:10:53.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.773+0000 7f6a8ea00640 1 -- 172.21.15.167:0/1425756418 >> v1:172.21.15.167:6789/0 conn(0x7f6a90072fa0 legacy=0x7f6a9015f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:53.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.773+0000 7f6a8ea00640 1 --2- 172.21.15.167:0/1425756418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a74003a00 0x7f6a74003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:53.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.773+0000 7f6a8ea00640 1 --2- 172.21.15.167:0/1425756418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a74004800 0x7f6a74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:53.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.774+0000 7f6a8ea00640 1 --2- 172.21.15.167:0/1425756418 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a9006f420 0x7f6a740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:53.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.774+0000 7f6a8ea00640 1 -- 172.21.15.167:0/1425756418 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6a900b6e80 con 0x7f6a74003a00 2024-05-11T12:10:53.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.774+0000 7f6a8ea00640 1 -- 172.21.15.167:0/1425756418 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9015bc60 con 0x7f6a74004800 2024-05-11T12:10:53.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.774+0000 7f6a8ea00640 1 -- 172.21.15.167:0/1425756418 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6a7400b9c0 con 0x7f6a9006f420 2024-05-11T12:10:53.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.774+0000 7f6a8f400640 1 --2- 172.21.15.167:0/1425756418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a74003a00 0x7f6a74003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:53.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.774+0000 7f6a8f400640 1 -- 172.21.15.167:0/1425756418 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a9006f420 msgr2=0x7f6a740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:53.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.774+0000 7f6a8fe00640 1 --2- 172.21.15.167:0/1425756418 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a9006f420 0x7f6a740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:53.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.774+0000 7f6a8f400640 1 --2- 172.21.15.167:0/1425756418 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a9006f420 0x7f6a740051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:53.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.774+0000 7f6a87e00640 1 --2- 172.21.15.167:0/1425756418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a74004800 0x7f6a74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:53.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.774+0000 7f6a8f400640 1 -- 172.21.15.167:0/1425756418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a74004800 msgr2=0x7f6a74004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:53.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.774+0000 7f6a8f400640 1 --2- 172.21.15.167:0/1425756418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a74004800 0x7f6a74004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:53.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.774+0000 7f6a8f400640 1 -- 172.21.15.167:0/1425756418 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a74003520 con 0x7f6a74003a00 2024-05-11T12:10:53.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.775+0000 7f6a8f400640 1 --2- 172.21.15.167:0/1425756418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a74003a00 0x7f6a74003e20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6a8000aaf0 tx=0x7f6a74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca93a9999e7bf327 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:53.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.776+0000 7f6a8ea00640 1 -- 172.21.15.167:0/1425756418 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a80039070 con 0x7f6a74003a00 2024-05-11T12:10:53.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.776+0000 7f6a8ea00640 1 -- 172.21.15.167:0/1425756418 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a8002c780 con 0x7f6a74003a00 2024-05-11T12:10:53.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.776+0000 7f6a8ea00640 1 -- 172.21.15.167:0/1425756418 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8000b040 con 0x7f6a74003a00 2024-05-11T12:10:53.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.776+0000 7f6a96200640 1 -- 172.21.15.167:0/1425756418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a74003a00 msgr2=0x7f6a74003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:53.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.776+0000 7f6a96200640 1 --2- 172.21.15.167:0/1425756418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a74003a00 0x7f6a74003e20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6a8000aaf0 tx=0x7f6a74005830 comp rx=0 tx=0).stop 2024-05-11T12:10:53.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.776+0000 7f6a8fe00640 1 -- 172.21.15.167:0/1425756418 reap_dead start 2024-05-11T12:10:53.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.776+0000 7f6a96200640 1 -- 172.21.15.167:0/1425756418 shutdown_connections 2024-05-11T12:10:53.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.776+0000 7f6a96200640 1 -- 172.21.15.167:0/1425756418 >> 172.21.15.167:0/1425756418 conn(0x7f6a9006c450 msgr2=0x7f6a9006c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:53.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.777+0000 7f6a96200640 1 -- 172.21.15.167:0/1425756418 shutdown_connections 2024-05-11T12:10:53.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.777+0000 7f6a96200640 1 -- 172.21.15.167:0/1425756418 wait complete. 2024-05-11T12:10:53.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.779+0000 7f6a96200640 1 Processor -- start 2024-05-11T12:10:53.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.779+0000 7f6a96200640 1 -- start start 2024-05-11T12:10:53.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.780+0000 7f6a96200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a74003a00 0x7f6a901f1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:53.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.780+0000 7f6a96200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a901f4600 0x7f6a901f3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:53.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.780+0000 7f6a96200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a90157460 0x7f6a901f26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:53.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.780+0000 7f6a96200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9015c7c0 con 0x7f6a90157460 2024-05-11T12:10:53.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.780+0000 7f6a96200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9015c4e0 con 0x7f6a901f4600 2024-05-11T12:10:53.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.780+0000 7f6a96200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6a900cfab0 con 0x7f6a74003a00 2024-05-11T12:10:53.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.780+0000 7f6a8fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a90157460 0x7f6a901f26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:53.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.780+0000 7f6a8fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a90157460 0x7f6a901f26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59162/0 (socket says 172.21.15.167:59162) 2024-05-11T12:10:53.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.780+0000 7f6a8fe00640 1 -- 172.21.15.167:0/3237895835 learned_addr learned my addr 172.21.15.167:0/3237895835 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:10:53.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.780+0000 7f6a8f400640 1 --2- 172.21.15.167:0/3237895835 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a74003a00 0x7f6a901f1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:53.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.780+0000 7f6a87e00640 1 --2- 172.21.15.167:0/3237895835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a901f4600 0x7f6a901f3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:53.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.780+0000 7f6a8fe00640 1 -- 172.21.15.167:0/3237895835 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a74003a00 msgr2=0x7f6a901f1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:53.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.780+0000 7f6a8fe00640 1 --2- 172.21.15.167:0/3237895835 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a74003a00 0x7f6a901f1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:53.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.780+0000 7f6a8fe00640 1 -- 172.21.15.167:0/3237895835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a901f4600 msgr2=0x7f6a901f3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:53.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.780+0000 7f6a8fe00640 1 --2- 172.21.15.167:0/3237895835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a901f4600 0x7f6a901f3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:53.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.781+0000 7f6a8fe00640 1 -- 172.21.15.167:0/3237895835 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a901f4400 con 0x7f6a90157460 2024-05-11T12:10:53.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.781+0000 7f6a8fe00640 1 --2- 172.21.15.167:0/3237895835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a90157460 0x7f6a901f26f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6a90073380 tx=0x7f6a8800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:53.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.781+0000 7f6a87400640 1 -- 172.21.15.167:0/3237895835 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a88001c20 con 0x7f6a90157460 2024-05-11T12:10:53.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.781+0000 7f6a87400640 1 -- 172.21.15.167:0/3237895835 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a8800bb80 con 0x7f6a90157460 2024-05-11T12:10:53.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.781+0000 7f6a87400640 1 -- 172.21.15.167:0/3237895835 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a88007070 con 0x7f6a90157460 2024-05-11T12:10:53.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.781+0000 7f6a96200640 1 -- 172.21.15.167:0/3237895835 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a901f4ac0 con 0x7f6a90157460 2024-05-11T12:10:53.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.781+0000 7f6a96200640 1 -- 172.21.15.167:0/3237895835 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a901f5080 con 0x7f6a90157460 2024-05-11T12:10:53.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.785+0000 7f6a87400640 1 -- 172.21.15.167:0/3237895835 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6a8801b020 con 0x7f6a90157460 2024-05-11T12:10:53.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.786+0000 7f6a87400640 1 --2- 172.21.15.167:0/3237895835 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f6a6c0762d0 0x7f6a6c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:53.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.786+0000 7f6a87400640 1 -- 172.21.15.167:0/3237895835 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f6a88097c00 con 0x7f6a90157460 2024-05-11T12:10:53.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.786+0000 7f6a8f400640 1 --2- 172.21.15.167:0/3237895835 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f6a6c0762d0 0x7f6a6c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:53.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.786+0000 7f6a99020740 1 Processor -- start 2024-05-11T12:10:53.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.787+0000 7f6a99020740 1 -- start start 2024-05-11T12:10:53.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.787+0000 7f6a8f400640 1 --2- 172.21.15.167:0/3237895835 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f6a6c0762d0 0x7f6a6c078790 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6a80036000 tx=0x7f6a80030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:53.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.788+0000 7f6a99020740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc7fa91840 0x55dc7fa91c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:53.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.788+0000 7f6a99020740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc7fa8c6d0 0x55dc7fa90440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:53.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.788+0000 7f6a99020740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55dc7fa90980 0x55dc7fa90dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:53.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.788+0000 7f6a99020740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55dc7f9f5790 con 0x55dc7fa91840 2024-05-11T12:10:53.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.788+0000 7f6a99020740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55dc7f99af20 con 0x55dc7fa8c6d0 2024-05-11T12:10:53.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.788+0000 7f6a99020740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55dc7f9f1750 con 0x55dc7fa90980 2024-05-11T12:10:53.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.788+0000 7f6a8f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc7fa91840 0x55dc7fa91c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:53.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.788+0000 7f6a8f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc7fa91840 0x55dc7fa91c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59166/0 (socket says 172.21.15.167:59166) 2024-05-11T12:10:53.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.788+0000 7f6a8f400640 1 -- 172.21.15.167:0/957775613 learned_addr learned my addr 172.21.15.167:0/957775613 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:10:53.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.788+0000 7f6a87e00640 1 --2- 172.21.15.167:0/957775613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc7fa8c6d0 0x55dc7fa90440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:53.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.788+0000 7f6a8fe00640 1 --2- 172.21.15.167:0/957775613 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55dc7fa90980 0x55dc7fa90dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:53.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.788+0000 7f6a8f400640 1 -- 172.21.15.167:0/957775613 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55dc7fa90980 msgr2=0x55dc7fa90dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:53.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.788+0000 7f6a8f400640 1 --2- 172.21.15.167:0/957775613 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55dc7fa90980 0x55dc7fa90dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:53.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.788+0000 7f6a8f400640 1 -- 172.21.15.167:0/957775613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc7fa8c6d0 msgr2=0x55dc7fa90440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:53.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.788+0000 7f6a8f400640 1 --2- 172.21.15.167:0/957775613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc7fa8c6d0 0x55dc7fa90440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:53.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.788+0000 7f6a8f400640 1 -- 172.21.15.167:0/957775613 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc7f7072d0 con 0x55dc7fa91840 2024-05-11T12:10:53.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.788+0000 7f6a8fe00640 1 --2- 172.21.15.167:0/957775613 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55dc7fa90980 0x55dc7fa90dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:10:53.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.789+0000 7f6a8f400640 1 --2- 172.21.15.167:0/957775613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc7fa91840 0x55dc7fa91c20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6a8000abe0 tx=0x7f6a8002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c70fb30a95e750 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:53.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.789+0000 7f6a7f400640 1 -- 172.21.15.167:0/957775613 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a80039040 con 0x55dc7fa91840 2024-05-11T12:10:53.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.789+0000 7f6a7f400640 1 -- 172.21.15.167:0/957775613 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a80003810 con 0x55dc7fa91840 2024-05-11T12:10:53.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.789+0000 7f6a7f400640 1 -- 172.21.15.167:0/957775613 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a80003be0 con 0x55dc7fa91840 2024-05-11T12:10:53.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.790+0000 7f6a99020740 1 -- 172.21.15.167:0/957775613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc7fa91840 msgr2=0x55dc7fa91c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:53.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.790+0000 7f6a99020740 1 --2- 172.21.15.167:0/957775613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc7fa91840 0x55dc7fa91c20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6a8000abe0 tx=0x7f6a8002e000 comp rx=0 tx=0).stop 2024-05-11T12:10:53.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.790+0000 7f6a99020740 1 -- 172.21.15.167:0/957775613 shutdown_connections 2024-05-11T12:10:53.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.790+0000 7f6a99020740 1 --2- 172.21.15.167:0/957775613 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55dc7fa90980 0x55dc7fa90dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:53.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.790+0000 7f6a99020740 1 --2- 172.21.15.167:0/957775613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc7fa8c6d0 0x55dc7fa90440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:53.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.790+0000 7f6a99020740 1 --2- 172.21.15.167:0/957775613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc7fa91840 0x55dc7fa91c20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:53.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.790+0000 7f6a99020740 1 -- 172.21.15.167:0/957775613 >> 172.21.15.167:0/957775613 conn(0x55dc7fa8e940 msgr2=0x55dc7fad7400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:53.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.790+0000 7f6a99020740 1 -- 172.21.15.167:0/957775613 shutdown_connections 2024-05-11T12:10:53.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.790+0000 7f6a99020740 1 -- 172.21.15.167:0/957775613 wait complete. 2024-05-11T12:10:53.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.791+0000 7f6a99020740 1 -- start start 2024-05-11T12:10:53.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.791+0000 7f6a99020740 1 Processor -- start 2024-05-11T12:10:53.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.792+0000 7f6a99020740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc7fa90980 0x55dc7fadf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:53.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.792+0000 7f6a99020740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55dc7fa91840 0x55dc7fadac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:53.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.792+0000 7f6a99020740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc7fab4600 0x55dc7fab4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:53.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.792+0000 7f6a99020740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55dc7fabd3d0 con 0x55dc7fab4600 2024-05-11T12:10:53.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.792+0000 7f6a99020740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55dc7f9ae9c0 con 0x55dc7fa90980 2024-05-11T12:10:53.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.792+0000 7f6a99020740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55dc7fabc850 con 0x55dc7fa91840 2024-05-11T12:10:53.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.792+0000 7f6a99020740 10 client.0 _ll_register_callbacks cb 0x55dc7f97b120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:10:53.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.792+0000 7f6a8fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc7fab4600 0x55dc7fab4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:53.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.792+0000 7f6a8fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc7fab4600 0x55dc7fab4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59178/0 (socket says 172.21.15.167:59178) 2024-05-11T12:10:53.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.792+0000 7f6a8fe00640 1 -- 172.21.15.167:0/2448560462 learned_addr learned my addr 172.21.15.167:0/2448560462 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:10:53.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.793+0000 7f6a87e00640 1 --2- 172.21.15.167:0/2448560462 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55dc7fa91840 0x55dc7fadac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:53.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.793+0000 7f6a8f400640 1 --2- 172.21.15.167:0/2448560462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc7fa90980 0x55dc7fadf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:53.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.793+0000 7f6a8fe00640 1 -- 172.21.15.167:0/2448560462 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55dc7fa91840 msgr2=0x55dc7fadac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:53.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.793+0000 7f6a8fe00640 1 --2- 172.21.15.167:0/2448560462 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55dc7fa91840 0x55dc7fadac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:53.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.793+0000 7f6a8fe00640 1 -- 172.21.15.167:0/2448560462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc7fa90980 msgr2=0x55dc7fadf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:53.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.793+0000 7f6a8fe00640 1 --2- 172.21.15.167:0/2448560462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc7fa90980 0x55dc7fadf6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:53.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.793+0000 7f6a8fe00640 1 -- 172.21.15.167:0/2448560462 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc7f7d16b0 con 0x55dc7fab4600 2024-05-11T12:10:53.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.793+0000 7f6a87e00640 1 --2- 172.21.15.167:0/2448560462 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55dc7fa91840 0x55dc7fadac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:10:53.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.793+0000 7f6a8f400640 1 --2- 172.21.15.167:0/2448560462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc7fa90980 0x55dc7fadf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:10:53.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.793+0000 7f6a8fe00640 1 --2- 172.21.15.167:0/2448560462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc7fab4600 0x55dc7fab4a80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6a88002230 tx=0x7f6a88061240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=368eef2be9b561fb server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:53.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.793+0000 7f6a73e00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-11T12:10:53.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.793+0000 7f6a73e00640 1 -- 172.21.15.167:0/2448560462 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a88069850 con 0x55dc7fab4600 2024-05-11T12:10:53.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.793+0000 7f6a73e00640 1 -- 172.21.15.167:0/2448560462 <== mon.0 v2:172.21.15.167:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f6a880670f0 con 0x55dc7fab4600 2024-05-11T12:10:53.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.793+0000 7f6a99020740 1 -- 172.21.15.167:0/2448560462 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a80003810 con 0x55dc7fab4600 2024-05-11T12:10:53.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.793+0000 7f6a73e00640 1 -- 172.21.15.167:0/2448560462 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f6a88095050 con 0x55dc7fab4600 2024-05-11T12:10:53.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.793+0000 7f6a99020740 10 client.4386 fetch_fsmap learned FSMap version 7 2024-05-11T12:10:53.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.793+0000 7f6a99020740 1 -- 172.21.15.167:0/2448560462 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55dc7fadc650 con 0x55dc7fab4600 2024-05-11T12:10:53.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.794+0000 7f6a73e00640 1 -- 172.21.15.167:0/2448560462 <== mon.0 v2:172.21.15.167:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a80003810 con 0x55dc7fab4600 2024-05-11T12:10:53.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.794+0000 7f6a73e00640 1 -- 172.21.15.167:0/2448560462 <== mon.0 v2:172.21.15.167:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7f6a8805fce0 con 0x55dc7fab4600 2024-05-11T12:10:53.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.794+0000 7f6a99020740 10 client.4386 fetch_fsmap finished waiting for FSMap version 7 2024-05-11T12:10:53.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.794+0000 7f6a99020740 10 client.4386 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-11T12:10:53.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.794+0000 7f6a99020740 20 client.4386 populate_metadata read hostname 'smithi167' 2024-05-11T12:10:53.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.794+0000 7f6a99020740 1 --2- 172.21.15.167:0/2448560462 >> [v2:172.21.15.204:6833/1889842375,v1:172.21.15.204:6835/1889842375] conn(0x55dc7fadd4b0 0x55dc7fadd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:53.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.794+0000 7f6a99020740 4 client.4386 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:10:53.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.794+0000 7f6a99020740 1 -- 172.21.15.167:0/2448560462 --> [v2:172.21.15.204:6833/1889842375,v1:172.21.15.204:6835/1889842375] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc7fa033a0 con 0x55dc7fadd4b0 2024-05-11T12:10:53.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.794+0000 7f6a8f400640 1 --2- 172.21.15.167:0/2448560462 >> [v2:172.21.15.204:6833/1889842375,v1:172.21.15.204:6835/1889842375] conn(0x55dc7fadd4b0 0x55dc7fadd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:53.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.795+0000 7f6a8f400640 1 --2- 172.21.15.167:0/2448560462 >> [v2:172.21.15.204:6833/1889842375,v1:172.21.15.204:6835/1889842375] conn(0x55dc7fadd4b0 0x55dc7fadd890 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c321be8eac60394 server_cookie=d084daa0424c029 in_seq=0 out_seq=0 2024-05-11T12:10:53.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.795+0000 7f6a73e00640 10 client.4386 ms_handle_connect on v2:172.21.15.204:6833/1889842375 2024-05-11T12:10:53.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.797+0000 7f6a73e00640 1 -- 172.21.15.167:0/2448560462 <== mds.0 v2:172.21.15.204:6833/1889842375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dc7fa033a0 con 0x55dc7fadd4b0 2024-05-11T12:10:53.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.797+0000 7f6a73e00640 10 client.4386 handle_command_reply: tid=0 2024-05-11T12:10:53.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.797+0000 7f6a99020740 1 client.4386 shutdown 2024-05-11T12:10:53.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.797+0000 7f6a99020740 1 -- 172.21.15.167:0/2448560462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc7fab4600 msgr2=0x55dc7fab4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:53.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.797+0000 7f6a99020740 1 --2- 172.21.15.167:0/2448560462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc7fab4600 0x55dc7fab4a80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6a88002230 tx=0x7f6a88061240 comp rx=0 tx=0).stop 2024-05-11T12:10:53.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.797+0000 7f6a99020740 1 -- 172.21.15.167:0/2448560462 shutdown_connections 2024-05-11T12:10:53.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.797+0000 7f6a99020740 1 --2- 172.21.15.167:0/2448560462 >> [v2:172.21.15.204:6833/1889842375,v1:172.21.15.204:6835/1889842375] conn(0x55dc7fadd4b0 0x55dc7fadd890 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:53.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.797+0000 7f6a99020740 1 --2- 172.21.15.167:0/2448560462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc7fab4600 0x55dc7fab4a80 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:53.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.797+0000 7f6a99020740 1 --2- 172.21.15.167:0/2448560462 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55dc7fa91840 0x55dc7fadac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:53.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.797+0000 7f6a99020740 1 --2- 172.21.15.167:0/2448560462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc7fa90980 0x55dc7fadf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:53.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.797+0000 7f6a73e00640 0 client.4386 ms_handle_reset on v2:172.21.15.204:6833/1889842375 2024-05-11T12:10:53.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.797+0000 7f6a99020740 1 -- 172.21.15.167:0/2448560462 >> 172.21.15.167:0/2448560462 conn(0x55dc7fa8c6d0 msgr2=0x55dc7fa8bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:53.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.797+0000 7f6a99020740 1 -- 172.21.15.167:0/2448560462 shutdown_connections 2024-05-11T12:10:53.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.797+0000 7f6a99020740 1 -- 172.21.15.167:0/2448560462 wait complete. 2024-05-11T12:10:53.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.798+0000 7f6a99020740 20 client.4386 trim_cache size 0 max 16384 2024-05-11T12:10:53.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.818+0000 7f6a99020740 1 Processor -- start 2024-05-11T12:10:53.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.818+0000 7f6a99020740 1 -- start start 2024-05-11T12:10:53.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.819+0000 7f6a99020740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc7fa90980 0x55dc7fad7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:53.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.819+0000 7f6a99020740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55dc7fa91840 0x55dc7fadeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:53.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.819+0000 7f6a99020740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc7fab4600 0x55dc7fadf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:53.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.819+0000 7f6a99020740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6a90201340 con 0x55dc7fab4600 2024-05-11T12:10:53.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.819+0000 7f6a99020740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6a90200360 con 0x55dc7fa90980 2024-05-11T12:10:53.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.819+0000 7f6a99020740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6a901f7340 con 0x55dc7fa91840 2024-05-11T12:10:53.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.819+0000 7f6a8fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc7fab4600 0x55dc7fadf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:53.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.819+0000 7f6a8fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc7fab4600 0x55dc7fadf3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59182/0 (socket says 172.21.15.167:59182) 2024-05-11T12:10:53.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.819+0000 7f6a8fe00640 1 -- 172.21.15.167:0/3155618951 learned_addr learned my addr 172.21.15.167:0/3155618951 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:10:53.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.820+0000 7f6a87e00640 1 --2- 172.21.15.167:0/3155618951 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55dc7fa91840 0x55dc7fadeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:53.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.820+0000 7f6a8fe00640 1 -- 172.21.15.167:0/3155618951 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55dc7fa91840 msgr2=0x55dc7fadeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:53.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.820+0000 7f6a8fe00640 1 --2- 172.21.15.167:0/3155618951 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55dc7fa91840 0x55dc7fadeeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:53.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.820+0000 7f6a8fe00640 1 -- 172.21.15.167:0/3155618951 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc7fa90980 msgr2=0x55dc7fad7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:53.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.820+0000 7f6a8fe00640 1 --2- 172.21.15.167:0/3155618951 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc7fa90980 0x55dc7fad7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:53.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.820+0000 7f6a8fe00640 1 -- 172.21.15.167:0/3155618951 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc7fa8e770 con 0x55dc7fab4600 2024-05-11T12:10:53.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.820+0000 7f6a8fe00640 1 --2- 172.21.15.167:0/3155618951 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc7fab4600 0x55dc7fadf3f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6a88060780 tx=0x7f6a88067000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e848c4fa61ee6ad server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:53.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.820+0000 7f6a7ea00640 1 -- 172.21.15.167:0/3155618951 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8805d0d0 con 0x55dc7fab4600 2024-05-11T12:10:53.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.820+0000 7f6a7ea00640 1 -- 172.21.15.167:0/3155618951 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a88082120 con 0x55dc7fab4600 2024-05-11T12:10:53.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.820+0000 7f6a7ea00640 1 -- 172.21.15.167:0/3155618951 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a88067070 con 0x55dc7fab4600 2024-05-11T12:10:53.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.821+0000 7f6a99020740 1 -- 172.21.15.167:0/3155618951 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc7fab4600 msgr2=0x55dc7fadf3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:53.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.821+0000 7f6a99020740 1 --2- 172.21.15.167:0/3155618951 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc7fab4600 0x55dc7fadf3f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6a88060780 tx=0x7f6a88067000 comp rx=0 tx=0).stop 2024-05-11T12:10:53.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.821+0000 7f6a99020740 1 -- 172.21.15.167:0/3155618951 shutdown_connections 2024-05-11T12:10:53.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.821+0000 7f6a99020740 1 --2- 172.21.15.167:0/3155618951 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc7fab4600 0x55dc7fadf3f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:53.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.821+0000 7f6a99020740 1 --2- 172.21.15.167:0/3155618951 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55dc7fa91840 0x55dc7fadeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:53.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.821+0000 7f6a99020740 1 --2- 172.21.15.167:0/3155618951 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc7fa90980 0x55dc7fad7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:53.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.821+0000 7f6a99020740 1 -- 172.21.15.167:0/3155618951 >> 172.21.15.167:0/3155618951 conn(0x55dc7fa8c6d0 msgr2=0x55dc7fa8d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:53.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.821+0000 7f6a99020740 1 -- 172.21.15.167:0/3155618951 shutdown_connections 2024-05-11T12:10:53.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.821+0000 7f6a99020740 1 -- 172.21.15.167:0/3155618951 wait complete. 2024-05-11T12:10:53.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.822+0000 7f6a99020740 1 -- start start 2024-05-11T12:10:53.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.822+0000 7f6a99020740 1 Processor -- start 2024-05-11T12:10:53.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.823+0000 7f6a99020740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55dc7fa90980 0x55dc7fae9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:53.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.823+0000 7f6a99020740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc7fa91840 0x55dc7fa91c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:53.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.823+0000 7f6a99020740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc7fab4600 0x55dc7fa92160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:53.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.823+0000 7f6a99020740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55dc7fade9d0 con 0x55dc7fa91840 2024-05-11T12:10:53.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.823+0000 7f6a99020740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6a901ffdb0 con 0x55dc7fab4600 2024-05-11T12:10:53.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.823+0000 7f6a99020740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55dc7faeb1e0 con 0x55dc7fa90980 2024-05-11T12:10:53.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.823+0000 7f6a99020740 10 client.0 _ll_register_callbacks cb 0x7f6a8805c940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:10:53.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.823+0000 7f6a87e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc7fa91840 0x55dc7fa91c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:53.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.823+0000 7f6a87e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc7fa91840 0x55dc7fa91c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59196/0 (socket says 172.21.15.167:59196) 2024-05-11T12:10:53.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.823+0000 7f6a87e00640 1 -- 172.21.15.167:0/364705316 learned_addr learned my addr 172.21.15.167:0/364705316 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:10:53.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.823+0000 7f6a8fe00640 1 --2- 172.21.15.167:0/364705316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc7fab4600 0x55dc7fa92160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:53.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.823+0000 7f6a87e00640 1 -- 172.21.15.167:0/364705316 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55dc7fa90980 msgr2=0x55dc7fae9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:53.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.823+0000 7f6a8f400640 1 --2- 172.21.15.167:0/364705316 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55dc7fa90980 0x55dc7fae9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:53.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.823+0000 7f6a87e00640 1 --2- 172.21.15.167:0/364705316 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55dc7fa90980 0x55dc7fae9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:53.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.823+0000 7f6a87e00640 1 -- 172.21.15.167:0/364705316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc7fab4600 msgr2=0x55dc7fa92160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:53.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.823+0000 7f6a87e00640 1 --2- 172.21.15.167:0/364705316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc7fab4600 0x55dc7fa92160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:53.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.823+0000 7f6a87e00640 1 -- 172.21.15.167:0/364705316 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc7faea110 con 0x55dc7fa91840 2024-05-11T12:10:53.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.823+0000 7f6a8fe00640 1 --2- 172.21.15.167:0/364705316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc7fab4600 0x55dc7fa92160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:10:53.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.824+0000 7f6a87e00640 1 --2- 172.21.15.167:0/364705316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc7fa91840 0x55dc7fa91c20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6a90070470 tx=0x7f6a78014840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d806c8f26acd9bad server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:53.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.824+0000 7f6a73e00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-11T12:10:53.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.824+0000 7f6a73e00640 1 -- 172.21.15.167:0/364705316 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a780158f0 con 0x55dc7fa91840 2024-05-11T12:10:53.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.824+0000 7f6a73e00640 1 -- 172.21.15.167:0/364705316 <== mon.0 v2:172.21.15.167:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f6a78015a90 con 0x55dc7fa91840 2024-05-11T12:10:53.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.824+0000 7f6a99020740 1 -- 172.21.15.167:0/364705316 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a88082120 con 0x55dc7fa91840 2024-05-11T12:10:53.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.824+0000 7f6a73e00640 1 -- 172.21.15.167:0/364705316 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f6a78015c90 con 0x55dc7fa91840 2024-05-11T12:10:53.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.824+0000 7f6a99020740 10 client.4392 fetch_fsmap learned FSMap version 7 2024-05-11T12:10:53.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.824+0000 7f6a99020740 1 -- 172.21.15.167:0/364705316 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55dc7fa8bd70 con 0x55dc7fa91840 2024-05-11T12:10:53.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.825+0000 7f6a73e00640 1 -- 172.21.15.167:0/364705316 <== mon.0 v2:172.21.15.167:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a88082120 con 0x55dc7fa91840 2024-05-11T12:10:53.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.825+0000 7f6a73e00640 1 -- 172.21.15.167:0/364705316 <== mon.0 v2:172.21.15.167:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7f6a7802a080 con 0x55dc7fa91840 2024-05-11T12:10:53.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.825+0000 7f6a99020740 10 client.4392 fetch_fsmap finished waiting for FSMap version 7 2024-05-11T12:10:53.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.825+0000 7f6a99020740 10 client.4392 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-11T12:10:53.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.825+0000 7f6a99020740 20 client.4392 populate_metadata read hostname 'smithi167' 2024-05-11T12:10:53.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.825+0000 7f6a99020740 1 --2- 172.21.15.167:0/364705316 >> [v2:172.21.15.204:6833/1889842375,v1:172.21.15.204:6835/1889842375] conn(0x55dc7fb5ca40 0x55dc7faf9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:53.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.825+0000 7f6a99020740 4 client.4392 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-11T12:10:53.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.825+0000 7f6a99020740 1 -- 172.21.15.167:0/364705316 --> [v2:172.21.15.204:6833/1889842375,v1:172.21.15.204:6835/1889842375] -- command(tid 0: {"prefix": "status"}) -- 0x55dc7f9fd210 con 0x55dc7fb5ca40 2024-05-11T12:10:53.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.825+0000 7f6a8f400640 1 --2- 172.21.15.167:0/364705316 >> [v2:172.21.15.204:6833/1889842375,v1:172.21.15.204:6835/1889842375] conn(0x55dc7fb5ca40 0x55dc7faf9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:53.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.826+0000 7f6a8f400640 1 --2- 172.21.15.167:0/364705316 >> [v2:172.21.15.204:6833/1889842375,v1:172.21.15.204:6835/1889842375] conn(0x55dc7fb5ca40 0x55dc7faf9df0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78553b4be5fccba5 server_cookie=7c504b3105b196ec in_seq=0 out_seq=0 2024-05-11T12:10:53.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.826+0000 7f6a73e00640 10 client.4392 ms_handle_connect on v2:172.21.15.204:6833/1889842375 2024-05-11T12:10:53.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.827+0000 7f6a73e00640 1 -- 172.21.15.167:0/364705316 <== mds.0 v2:172.21.15.204:6833/1889842375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55dc7f9fd210 con 0x55dc7fb5ca40 2024-05-11T12:10:53.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.827+0000 7f6a73e00640 10 client.4392 handle_command_reply: tid=0 2024-05-11T12:10:53.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.827+0000 7f6a99020740 1 client.4392 shutdown 2024-05-11T12:10:53.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.827+0000 7f6a99020740 1 -- 172.21.15.167:0/364705316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc7fa91840 msgr2=0x55dc7fa91c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:53.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.827+0000 7f6a99020740 1 --2- 172.21.15.167:0/364705316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc7fa91840 0x55dc7fa91c20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6a90070470 tx=0x7f6a78014840 comp rx=0 tx=0).stop 2024-05-11T12:10:53.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.827+0000 7f6a99020740 1 -- 172.21.15.167:0/364705316 shutdown_connections 2024-05-11T12:10:53.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.827+0000 7f6a99020740 1 --2- 172.21.15.167:0/364705316 >> [v2:172.21.15.204:6833/1889842375,v1:172.21.15.204:6835/1889842375] conn(0x55dc7fb5ca40 0x55dc7faf9df0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:53.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.827+0000 7f6a99020740 1 --2- 172.21.15.167:0/364705316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc7fab4600 0x55dc7fa92160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:53.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.827+0000 7f6a99020740 1 --2- 172.21.15.167:0/364705316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc7fa91840 0x55dc7fa91c20 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:53.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.827+0000 7f6a99020740 1 --2- 172.21.15.167:0/364705316 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55dc7fa90980 0x55dc7fae9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:53.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.827+0000 7f6a99020740 1 -- 172.21.15.167:0/364705316 >> 172.21.15.167:0/364705316 conn(0x55dc7fa8c6d0 msgr2=0x55dc7fadb6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:53.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.827+0000 7f6a73e00640 0 client.4392 ms_handle_reset on v2:172.21.15.204:6833/1889842375 2024-05-11T12:10:53.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.827+0000 7f6a99020740 1 -- 172.21.15.167:0/364705316 shutdown_connections 2024-05-11T12:10:53.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.827+0000 7f6a99020740 1 -- 172.21.15.167:0/364705316 wait complete. 2024-05-11T12:10:53.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.827+0000 7f6a99020740 20 client.4392 trim_cache size 0 max 16384 2024-05-11T12:10:53.831 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T12:10:53.831 INFO:teuthology.orchestra.run.smithi167.stdout: "cluster_fsid": "5f32d344-7289-4c27-881b-1533419ab678", 2024-05-11T12:10:53.831 INFO:teuthology.orchestra.run.smithi167.stdout: "whoami": 0, 2024-05-11T12:10:53.832 INFO:teuthology.orchestra.run.smithi167.stdout: "id": 4285, 2024-05-11T12:10:53.832 INFO:teuthology.orchestra.run.smithi167.stdout: "want_state": "up:active", 2024-05-11T12:10:53.832 INFO:teuthology.orchestra.run.smithi167.stdout: "state": "up:active", 2024-05-11T12:10:53.832 INFO:teuthology.orchestra.run.smithi167.stdout: "fs_name": "cephfs", 2024-05-11T12:10:53.832 INFO:teuthology.orchestra.run.smithi167.stdout: "rank_uptime": 8.0587717889999997, 2024-05-11T12:10:53.832 INFO:teuthology.orchestra.run.smithi167.stdout: "mdsmap_epoch": 7, 2024-05-11T12:10:53.832 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_epoch": 27, 2024-05-11T12:10:53.832 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_epoch_barrier": 27, 2024-05-11T12:10:53.832 INFO:teuthology.orchestra.run.smithi167.stdout: "uptime": 14.404186279999999 2024-05-11T12:10:53.832 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T12:10:53.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.828+0000 7f6a96200640 1 -- 172.21.15.167:0/3237895835 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f6a6c0762d0 msgr2=0x7f6a6c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:53.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.828+0000 7f6a96200640 1 --2- 172.21.15.167:0/3237895835 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f6a6c0762d0 0x7f6a6c078790 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6a80036000 tx=0x7f6a80030000 comp rx=0 tx=0).stop 2024-05-11T12:10:53.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.828+0000 7f6a96200640 1 -- 172.21.15.167:0/3237895835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a90157460 msgr2=0x7f6a901f26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:53.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.828+0000 7f6a96200640 1 --2- 172.21.15.167:0/3237895835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a90157460 0x7f6a901f26f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6a90073380 tx=0x7f6a8800b4a0 comp rx=0 tx=0).stop 2024-05-11T12:10:53.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.828+0000 7f6a96200640 1 -- 172.21.15.167:0/3237895835 shutdown_connections 2024-05-11T12:10:53.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.828+0000 7f6a96200640 1 --2- 172.21.15.167:0/3237895835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a90157460 0x7f6a901f26f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:53.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.828+0000 7f6a96200640 1 --2- 172.21.15.167:0/3237895835 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f6a6c0762d0 0x7f6a6c078790 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:53.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.828+0000 7f6a96200640 1 --2- 172.21.15.167:0/3237895835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a901f4600 0x7f6a901f3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:53.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.828+0000 7f6a96200640 1 --2- 172.21.15.167:0/3237895835 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a74003a00 0x7f6a901f1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:53.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.828+0000 7f6a96200640 1 -- 172.21.15.167:0/3237895835 >> 172.21.15.167:0/3237895835 conn(0x7f6a9006c450 msgr2=0x7f6a90072760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:53.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.828+0000 7f6a96200640 1 -- 172.21.15.167:0/3237895835 shutdown_connections 2024-05-11T12:10:53.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:53.828+0000 7f6a96200640 1 -- 172.21.15.167:0/3237895835 wait complete. 2024-05-11T12:10:53.847 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-11T12:10:53.847 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-11T12:10:53.848 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-11T12:10:54.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.034+0000 7fadb4800640 1 Processor -- start 2024-05-11T12:10:54.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.034+0000 7fadb4800640 1 -- start start 2024-05-11T12:10:54.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.036+0000 7fadb4800640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fadac13eeb0 0x7fadac06f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:54.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.036+0000 7fadb4800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fadac13f290 0x7fadac06f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:54.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.036+0000 7fadb4800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fadac0b50a0 0x7fadac0735c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:54.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.036+0000 7fadb4800640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fadac15fb90 con 0x7fadac13eeb0 2024-05-11T12:10:54.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.036+0000 7fadb4800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fadac15fe70 con 0x7fadac0b50a0 2024-05-11T12:10:54.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.036+0000 7fadb4800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fadac160150 con 0x7fadac13f290 2024-05-11T12:10:54.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.036+0000 7fadb1e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fadac13eeb0 0x7fadac06f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:54.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.036+0000 7fadb1e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fadac13eeb0 0x7fadac06f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59206/0 (socket says 172.21.15.167:59206) 2024-05-11T12:10:54.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.036+0000 7fadb1e00640 1 -- 172.21.15.167:0/195866396 learned_addr learned my addr 172.21.15.167:0/195866396 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:10:54.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.037+0000 7fadb1400640 1 --2- 172.21.15.167:0/195866396 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fadac13f290 0x7fadac06f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:54.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.037+0000 7fadb1e00640 1 -- 172.21.15.167:0/195866396 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fadac13f290 msgr2=0x7fadac06f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:54.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.037+0000 7fadb1e00640 1 --2- 172.21.15.167:0/195866396 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fadac13f290 0x7fadac06f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:54.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.037+0000 7fadb2800640 1 --2- 172.21.15.167:0/195866396 >> v2:172.21.15.204:3300/0 conn(0x7fadac0b50a0 0x7fadac0735c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:54.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.037+0000 7fadb1e00640 1 -- 172.21.15.167:0/195866396 >> v2:172.21.15.204:3300/0 conn(0x7fadac0b50a0 msgr2=0x7fadac0735c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:10:54.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.037+0000 7fadb1e00640 1 --2- 172.21.15.167:0/195866396 >> v2:172.21.15.204:3300/0 conn(0x7fadac0b50a0 0x7fadac0735c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:54.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.037+0000 7fadb1e00640 1 -- 172.21.15.167:0/195866396 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadac160430 con 0x7fadac13eeb0 2024-05-11T12:10:54.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.038+0000 7fadb1e00640 1 --2- 172.21.15.167:0/195866396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadac13eeb0 0x7fadac06f480 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fadac156c10 tx=0x7fada002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe92245a5f140d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:54.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.038+0000 7fadb0a00640 1 -- 172.21.15.167:0/195866396 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada0037070 con 0x7fadac13eeb0 2024-05-11T12:10:54.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.038+0000 7fadb0a00640 1 -- 172.21.15.167:0/195866396 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fada0032440 con 0x7fadac13eeb0 2024-05-11T12:10:54.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.038+0000 7fadb0a00640 1 -- 172.21.15.167:0/195866396 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada0032810 con 0x7fadac13eeb0 2024-05-11T12:10:54.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.039+0000 7fadb4800640 1 -- 172.21.15.167:0/195866396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadac13eeb0 msgr2=0x7fadac06f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:54.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.039+0000 7fadb4800640 1 --2- 172.21.15.167:0/195866396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadac13eeb0 0x7fadac06f480 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fadac156c10 tx=0x7fada002a140 comp rx=0 tx=0).stop 2024-05-11T12:10:54.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.039+0000 7fadb4800640 1 -- 172.21.15.167:0/195866396 shutdown_connections 2024-05-11T12:10:54.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.039+0000 7fadb4800640 1 --2- 172.21.15.167:0/195866396 >> v2:172.21.15.204:3300/0 conn(0x7fadac0b50a0 0x7fadac0735c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:54.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.039+0000 7fadb4800640 1 --2- 172.21.15.167:0/195866396 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fadac13f290 0x7fadac06f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:54.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.039+0000 7fadb4800640 1 --2- 172.21.15.167:0/195866396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadac13eeb0 0x7fadac06f480 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:54.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.039+0000 7fadb4800640 1 -- 172.21.15.167:0/195866396 >> 172.21.15.167:0/195866396 conn(0x7fadac0cd8f0 msgr2=0x7fadac0cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:54.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.039+0000 7fadb4800640 1 -- 172.21.15.167:0/195866396 shutdown_connections 2024-05-11T12:10:54.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.039+0000 7fadb4800640 1 -- 172.21.15.167:0/195866396 wait complete. 2024-05-11T12:10:54.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.041+0000 7fadb4800640 1 Processor -- start 2024-05-11T12:10:54.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.041+0000 7fadb4800640 1 -- start start 2024-05-11T12:10:54.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.042+0000 7fadb4800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fadac0b50a0 0x7fadac133700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:54.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.042+0000 7fadb4800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fadac13eeb0 0x7fadac152780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:54.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.042+0000 7fadb4800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadac13f290 0x7fadac154df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:54.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.042+0000 7fadb4800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fadac073c80 con 0x7fadac13f290 2024-05-11T12:10:54.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.042+0000 7fadb4800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fadac073b00 con 0x7fadac0b50a0 2024-05-11T12:10:54.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.042+0000 7fadb4800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fadac073e00 con 0x7fadac13eeb0 2024-05-11T12:10:54.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.042+0000 7fadb2800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadac13f290 0x7fadac154df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:54.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.042+0000 7fadb2800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadac13f290 0x7fadac154df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59218/0 (socket says 172.21.15.167:59218) 2024-05-11T12:10:54.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.042+0000 7fadb2800640 1 -- 172.21.15.167:0/3807110271 learned_addr learned my addr 172.21.15.167:0/3807110271 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:10:54.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.043+0000 7fadb1400640 1 --2- 172.21.15.167:0/3807110271 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fadac13eeb0 0x7fadac152780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:54.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.043+0000 7fadb1e00640 1 --2- 172.21.15.167:0/3807110271 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fadac0b50a0 0x7fadac133700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:54.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.043+0000 7fadb2800640 1 -- 172.21.15.167:0/3807110271 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fadac13eeb0 msgr2=0x7fadac152780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:54.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.043+0000 7fadb2800640 1 --2- 172.21.15.167:0/3807110271 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fadac13eeb0 0x7fadac152780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:54.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.043+0000 7fadb2800640 1 -- 172.21.15.167:0/3807110271 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fadac0b50a0 msgr2=0x7fadac133700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:54.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.043+0000 7fadb2800640 1 --2- 172.21.15.167:0/3807110271 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fadac0b50a0 0x7fadac133700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:54.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.043+0000 7fadb2800640 1 -- 172.21.15.167:0/3807110271 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadac15c9a0 con 0x7fadac13f290 2024-05-11T12:10:54.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.043+0000 7fadb2800640 1 --2- 172.21.15.167:0/3807110271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadac13f290 0x7fadac154df0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fada800cda0 tx=0x7fada800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:54.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.043+0000 7fada6000640 1 -- 172.21.15.167:0/3807110271 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada8003b90 con 0x7fadac13f290 2024-05-11T12:10:54.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.043+0000 7fada6000640 1 -- 172.21.15.167:0/3807110271 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fada8003d30 con 0x7fadac13f290 2024-05-11T12:10:54.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.043+0000 7fadb4800640 1 -- 172.21.15.167:0/3807110271 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadac15cc80 con 0x7fadac13f290 2024-05-11T12:10:54.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.043+0000 7fada6000640 1 -- 172.21.15.167:0/3807110271 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada8004240 con 0x7fadac13f290 2024-05-11T12:10:54.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.043+0000 7fadb4800640 1 -- 172.21.15.167:0/3807110271 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadac15d240 con 0x7fadac13f290 2024-05-11T12:10:54.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.045+0000 7fadb4800640 1 -- 172.21.15.167:0/3807110271 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadac069590 con 0x7fadac13f290 2024-05-11T12:10:54.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.046+0000 7fada6000640 1 -- 172.21.15.167:0/3807110271 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fada80043e0 con 0x7fadac13f290 2024-05-11T12:10:54.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.047+0000 7fada6000640 1 --2- 172.21.15.167:0/3807110271 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fad900762d0 0x7fad90078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:10:54.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.047+0000 7fada6000640 1 -- 172.21.15.167:0/3807110271 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fada80983a0 con 0x7fadac13f290 2024-05-11T12:10:54.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.047+0000 7fadb1e00640 1 --2- 172.21.15.167:0/3807110271 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fad900762d0 0x7fad90078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:10:54.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.048+0000 7fadb1e00640 1 --2- 172.21.15.167:0/3807110271 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fad900762d0 0x7fad90078790 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fada0034000 tx=0x7fada002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:10:54.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.052+0000 7fada6000640 1 -- 172.21.15.167:0/3807110271 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fada8061a50 con 0x7fadac13f290 2024-05-11T12:10:54.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.297+0000 7fadb4800640 1 -- 172.21.15.167:0/3807110271 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fadac156320 con 0x7fadac13f290 2024-05-11T12:10:54.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.298+0000 7fada6000640 1 -- 172.21.15.167:0/3807110271 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+88 (secure 0 0 0) 0x7fada8061350 con 0x7fadac13f290 2024-05-11T12:10:54.298 INFO:teuthology.orchestra.run.smithi167.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-11T12:10:54.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.300+0000 7fadb4800640 1 -- 172.21.15.167:0/3807110271 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fad900762d0 msgr2=0x7fad90078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:54.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.300+0000 7fadb4800640 1 --2- 172.21.15.167:0/3807110271 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fad900762d0 0x7fad90078790 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fada0034000 tx=0x7fada002e000 comp rx=0 tx=0).stop 2024-05-11T12:10:54.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.300+0000 7fadb4800640 1 -- 172.21.15.167:0/3807110271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadac13f290 msgr2=0x7fadac154df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:10:54.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.300+0000 7fadb4800640 1 --2- 172.21.15.167:0/3807110271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadac13f290 0x7fadac154df0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fada800cda0 tx=0x7fada800a660 comp rx=0 tx=0).stop 2024-05-11T12:10:54.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.301+0000 7fadb4800640 1 -- 172.21.15.167:0/3807110271 shutdown_connections 2024-05-11T12:10:54.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.301+0000 7fadb4800640 1 --2- 172.21.15.167:0/3807110271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadac13f290 0x7fadac154df0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:54.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.301+0000 7fadb4800640 1 --2- 172.21.15.167:0/3807110271 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fadac13eeb0 0x7fadac152780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:54.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.301+0000 7fadb4800640 1 --2- 172.21.15.167:0/3807110271 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fad900762d0 0x7fad90078790 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:54.302 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.301+0000 7fadb4800640 1 --2- 172.21.15.167:0/3807110271 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fadac0b50a0 0x7fadac133700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:10:54.302 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.301+0000 7fadb4800640 1 -- 172.21.15.167:0/3807110271 >> 172.21.15.167:0/3807110271 conn(0x7fadac0cd8f0 msgr2=0x7fadac0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:10:54.302 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.301+0000 7fadb4800640 1 -- 172.21.15.167:0/3807110271 shutdown_connections 2024-05-11T12:10:54.302 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54.301+0000 7fadb4800640 1 -- 172.21.15.167:0/3807110271 wait complete. 2024-05-11T12:10:54.320 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-11T12:10:54.320 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-11T12:10:54.320 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi167.front.sepia.ceph.com 2024-05-11T12:10:54.321 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-05-11T12:10:54.321 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-11T12:10:54.321 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-11T12:10:54.321 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-11T12:10:54.321 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-11T12:10:54.321 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-05-11T12:10:54.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:10:54.321 DEBUG:teuthology.orchestra.run.smithi167:> ip addr 2024-05-11T12:10:54.368 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-11T12:10:54.369 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-11T12:10:54.369 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2024-05-11T12:10:54.369 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:10:54.369 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2024-05-11T12:10:54.369 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:10:54.369 INFO:teuthology.orchestra.run.smithi167.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:10:54.369 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e6 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:10:54.369 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f0 2024-05-11T12:10:54.369 INFO:teuthology.orchestra.run.smithi167.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:10:54.369 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e7 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:10:54.370 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f1 2024-05-11T12:10:54.370 INFO:teuthology.orchestra.run.smithi167.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:10:54.370 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:08 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:10:54.370 INFO:teuthology.orchestra.run.smithi167.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-11T12:10:54.370 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:09 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:10:54.370 INFO:teuthology.orchestra.run.smithi167.stdout: inet 172.21.15.167/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-11T12:10:54.370 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft 42615sec preferred_lft 42615sec 2024-05-11T12:10:54.370 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::ec4:7aff:fe88:7309/64 scope link 2024-05-11T12:10:54.370 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:10:54.372 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-11T12:10:54.372 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T12:10:54.372 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-05-11T12:10:54.372 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link add name ceph-brx type bridge 2024-05-11T12:10:54.372 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip addr flush dev ceph-brx 2024-05-11T12:10:54.373 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set ceph-brx up 2024-05-11T12:10:54.373 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-11T12:10:54.373 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-05-11T12:10:54.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T12:10:54.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:10:54.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:10:54.901 DEBUG:teuthology.orchestra.run.smithi167:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-11T12:10:54.968 INFO:teuthology.orchestra.run.smithi167.stdout:1 2024-05-11T12:10:54.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:10:54.972 DEBUG:teuthology.orchestra.run.smithi167:> route 2024-05-11T12:10:55.030 INFO:teuthology.orchestra.run.smithi167.stdout:Kernel IP routing table 2024-05-11T12:10:55.030 INFO:teuthology.orchestra.run.smithi167.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-11T12:10:55.030 INFO:teuthology.orchestra.run.smithi167.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-11T12:10:55.030 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-11T12:10:55.030 INFO:teuthology.orchestra.run.smithi167.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-11T12:10:55.031 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T12:10:55.031 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-05-11T12:10:55.031 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-11T12:10:55.031 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-11T12:10:55.031 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-11T12:10:55.031 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-05-11T12:10:55.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T12:10:55.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:10:55.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:10:55.768 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2024-05-11T12:10:55.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:10:55.812 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list-id 2024-05-11T12:10:55.895 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T12:10:55.896 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-05-11T12:10:55.896 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-11T12:10:55.896 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-05-11T12:10:55.896 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-05-11T12:10:56.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T12:10:56.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:10:56.102 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-05-11T12:10:56.102 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T12:10:56.102 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-05-11T12:10:56.102 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-05-11T12:10:56.102 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-11T12:10:56.102 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-05-11T12:10:56.102 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-05-11T12:10:56.102 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-05-11T12:10:56.102 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-05-11T12:10:56.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T12:10:57.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:10:57.208 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T12:10:57.208 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-05-11T12:10:57.208 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set brx.0 up 2024-05-11T12:10:57.208 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set dev brx.0 master ceph-brx 2024-05-11T12:10:57.208 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-05-11T12:10:57.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T12:10:57.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:10:57.393 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-05-11T12:10:57.393 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T12:10:57.393 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-05-11T12:10:57.437 INFO:teuthology.orchestra.run.smithi167.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-05-11T12:10:57.437 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T12:10:57.437 DEBUG:teuthology.orchestra.run.smithi167:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-05-11T12:10:57.527 DEBUG:teuthology.orchestra.run.smithi167:> sudo modprobe fuse 2024-05-11T12:10:57.605 DEBUG:teuthology.orchestra.run.smithi167:> cat /proc/self/mounts | awk '{print $2}' 2024-05-11T12:10:57.657 INFO:teuthology.orchestra.run.smithi167.stdout:/proc 2024-05-11T12:10:57.658 INFO:teuthology.orchestra.run.smithi167.stdout:/sys 2024-05-11T12:10:57.658 INFO:teuthology.orchestra.run.smithi167.stdout:/dev 2024-05-11T12:10:57.658 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/kernel/security 2024-05-11T12:10:57.658 INFO:teuthology.orchestra.run.smithi167.stdout:/dev/shm 2024-05-11T12:10:57.658 INFO:teuthology.orchestra.run.smithi167.stdout:/dev/pts 2024-05-11T12:10:57.658 INFO:teuthology.orchestra.run.smithi167.stdout:/run 2024-05-11T12:10:57.658 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/cgroup 2024-05-11T12:10:57.658 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/pstore 2024-05-11T12:10:57.658 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/bpf 2024-05-11T12:10:57.658 INFO:teuthology.orchestra.run.smithi167.stdout:/ 2024-05-11T12:10:57.658 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/selinux 2024-05-11T12:10:57.658 INFO:teuthology.orchestra.run.smithi167.stdout:/proc/sys/fs/binfmt_misc 2024-05-11T12:10:57.658 INFO:teuthology.orchestra.run.smithi167.stdout:/dev/mqueue 2024-05-11T12:10:57.658 INFO:teuthology.orchestra.run.smithi167.stdout:/dev/hugepages 2024-05-11T12:10:57.659 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/kernel/debug 2024-05-11T12:10:57.659 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/kernel/tracing 2024-05-11T12:10:57.659 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/fuse/connections 2024-05-11T12:10:57.659 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/kernel/config 2024-05-11T12:10:57.659 INFO:teuthology.orchestra.run.smithi167.stdout:/run/credentials/systemd-sysctl.service 2024-05-11T12:10:57.659 INFO:teuthology.orchestra.run.smithi167.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-11T12:10:57.659 INFO:teuthology.orchestra.run.smithi167.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-11T12:10:57.659 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/nfs/rpc_pipefs 2024-05-11T12:10:57.659 INFO:teuthology.orchestra.run.smithi167.stdout:/run/user/1000 2024-05-11T12:10:57.659 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph 2024-05-11T12:10:57.659 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph/osd/ceph-0 2024-05-11T12:10:57.659 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph/osd/ceph-1 2024-05-11T12:10:57.659 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph/osd/ceph-2 2024-05-11T12:10:57.659 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph/osd/ceph-3 2024-05-11T12:10:57.659 INFO:teuthology.orchestra.run.smithi167.stdout:/run/netns 2024-05-11T12:10:57.660 INFO:teuthology.orchestra.run.smithi167.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-11T12:10:57.660 INFO:teuthology.orchestra.run.smithi167.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-11T12:10:57.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:10:57.661 DEBUG:teuthology.orchestra.run.smithi167:> ls /sys/fs/fuse/connections 2024-05-11T12:10:57.711 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-11T12:10:57.712 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-05-11T12:10:57.719 DEBUG:teuthology.orchestra.run.smithi167:> sudo modprobe fuse 2024-05-11T12:10:57.793 DEBUG:teuthology.orchestra.run.smithi167:> cat /proc/self/mounts | awk '{print $2}' 2024-05-11T12:10:57.835 INFO:teuthology.orchestra.run.smithi167.stdout:/proc 2024-05-11T12:10:57.836 INFO:teuthology.orchestra.run.smithi167.stdout:/sys 2024-05-11T12:10:57.836 INFO:teuthology.orchestra.run.smithi167.stdout:/dev 2024-05-11T12:10:57.836 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/kernel/security 2024-05-11T12:10:57.836 INFO:teuthology.orchestra.run.smithi167.stdout:/dev/shm 2024-05-11T12:10:57.836 INFO:teuthology.orchestra.run.smithi167.stdout:/dev/pts 2024-05-11T12:10:57.836 INFO:teuthology.orchestra.run.smithi167.stdout:/run 2024-05-11T12:10:57.836 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/cgroup 2024-05-11T12:10:57.836 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/pstore 2024-05-11T12:10:57.836 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/bpf 2024-05-11T12:10:57.836 INFO:teuthology.orchestra.run.smithi167.stdout:/ 2024-05-11T12:10:57.836 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/selinux 2024-05-11T12:10:57.836 INFO:teuthology.orchestra.run.smithi167.stdout:/proc/sys/fs/binfmt_misc 2024-05-11T12:10:57.836 INFO:teuthology.orchestra.run.smithi167.stdout:/dev/mqueue 2024-05-11T12:10:57.837 INFO:teuthology.orchestra.run.smithi167.stdout:/dev/hugepages 2024-05-11T12:10:57.837 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/kernel/debug 2024-05-11T12:10:57.837 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/kernel/tracing 2024-05-11T12:10:57.837 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/fuse/connections 2024-05-11T12:10:57.837 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/kernel/config 2024-05-11T12:10:57.837 INFO:teuthology.orchestra.run.smithi167.stdout:/run/credentials/systemd-sysctl.service 2024-05-11T12:10:57.837 INFO:teuthology.orchestra.run.smithi167.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-11T12:10:57.837 INFO:teuthology.orchestra.run.smithi167.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-11T12:10:57.837 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/nfs/rpc_pipefs 2024-05-11T12:10:57.837 INFO:teuthology.orchestra.run.smithi167.stdout:/run/user/1000 2024-05-11T12:10:57.837 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph 2024-05-11T12:10:57.837 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph/osd/ceph-0 2024-05-11T12:10:57.837 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph/osd/ceph-1 2024-05-11T12:10:57.837 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph/osd/ceph-2 2024-05-11T12:10:57.838 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph/osd/ceph-3 2024-05-11T12:10:57.838 INFO:teuthology.orchestra.run.smithi167.stdout:/run/netns 2024-05-11T12:10:57.838 INFO:teuthology.orchestra.run.smithi167.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-11T12:10:57.838 INFO:teuthology.orchestra.run.smithi167.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-11T12:10:57.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:10:57.839 DEBUG:teuthology.orchestra.run.smithi167:> ls /sys/fs/fuse/connections 2024-05-11T12:10:57.905 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi167.stderr:2024-05-11T12:10:57.905+0000 7facbc734480 -1 init, newargv = 0x55aefedbbe10 newargc=15 2024-05-11T12:10:57.905 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi167.stderr:ceph-fuse[41137]: starting ceph client 2024-05-11T12:10:57.920 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi167.stderr:ceph-fuse[41137]: starting fuse 2024-05-11T12:10:59.021 DEBUG:teuthology.orchestra.run.smithi167:> sudo modprobe fuse 2024-05-11T12:10:59.117 DEBUG:teuthology.orchestra.run.smithi167:> cat /proc/self/mounts | awk '{print $2}' 2024-05-11T12:10:59.173 INFO:teuthology.orchestra.run.smithi167.stdout:/proc 2024-05-11T12:10:59.174 INFO:teuthology.orchestra.run.smithi167.stdout:/sys 2024-05-11T12:10:59.174 INFO:teuthology.orchestra.run.smithi167.stdout:/dev 2024-05-11T12:10:59.174 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/kernel/security 2024-05-11T12:10:59.174 INFO:teuthology.orchestra.run.smithi167.stdout:/dev/shm 2024-05-11T12:10:59.174 INFO:teuthology.orchestra.run.smithi167.stdout:/dev/pts 2024-05-11T12:10:59.174 INFO:teuthology.orchestra.run.smithi167.stdout:/run 2024-05-11T12:10:59.174 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/cgroup 2024-05-11T12:10:59.174 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/pstore 2024-05-11T12:10:59.174 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/bpf 2024-05-11T12:10:59.174 INFO:teuthology.orchestra.run.smithi167.stdout:/ 2024-05-11T12:10:59.174 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/selinux 2024-05-11T12:10:59.174 INFO:teuthology.orchestra.run.smithi167.stdout:/proc/sys/fs/binfmt_misc 2024-05-11T12:10:59.174 INFO:teuthology.orchestra.run.smithi167.stdout:/dev/mqueue 2024-05-11T12:10:59.174 INFO:teuthology.orchestra.run.smithi167.stdout:/dev/hugepages 2024-05-11T12:10:59.175 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/kernel/debug 2024-05-11T12:10:59.175 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/kernel/tracing 2024-05-11T12:10:59.175 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/fuse/connections 2024-05-11T12:10:59.175 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/kernel/config 2024-05-11T12:10:59.175 INFO:teuthology.orchestra.run.smithi167.stdout:/run/credentials/systemd-sysctl.service 2024-05-11T12:10:59.175 INFO:teuthology.orchestra.run.smithi167.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-11T12:10:59.175 INFO:teuthology.orchestra.run.smithi167.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-11T12:10:59.175 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/nfs/rpc_pipefs 2024-05-11T12:10:59.175 INFO:teuthology.orchestra.run.smithi167.stdout:/run/user/1000 2024-05-11T12:10:59.175 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph 2024-05-11T12:10:59.175 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph/osd/ceph-0 2024-05-11T12:10:59.175 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph/osd/ceph-1 2024-05-11T12:10:59.175 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph/osd/ceph-2 2024-05-11T12:10:59.175 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph/osd/ceph-3 2024-05-11T12:10:59.175 INFO:teuthology.orchestra.run.smithi167.stdout:/run/netns 2024-05-11T12:10:59.175 INFO:teuthology.orchestra.run.smithi167.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-11T12:10:59.176 INFO:teuthology.orchestra.run.smithi167.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-11T12:10:59.176 INFO:teuthology.orchestra.run.smithi167.stdout:/home/ubuntu/cephtest/mnt.admin 2024-05-11T12:10:59.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:10:59.177 DEBUG:teuthology.orchestra.run.smithi167:> ls /sys/fs/fuse/connections 2024-05-11T12:10:59.227 INFO:teuthology.orchestra.run.smithi167.stdout:41 2024-05-11T12:10:59.228 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-05-11T12:10:59.228 DEBUG:teuthology.orchestra.run.smithi167:> sudo stdin-killer -- python3 -c ' 2024-05-11T12:10:59.228 DEBUG:teuthology.orchestra.run.smithi167:> import glob 2024-05-11T12:10:59.228 DEBUG:teuthology.orchestra.run.smithi167:> import re 2024-05-11T12:10:59.228 DEBUG:teuthology.orchestra.run.smithi167:> import os 2024-05-11T12:10:59.228 DEBUG:teuthology.orchestra.run.smithi167:> import subprocess 2024-05-11T12:10:59.228 DEBUG:teuthology.orchestra.run.smithi167:> 2024-05-11T12:10:59.228 DEBUG:teuthology.orchestra.run.smithi167:> def _find_admin_socket(client_name): 2024-05-11T12:10:59.228 DEBUG:teuthology.orchestra.run.smithi167:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-05-11T12:10:59.228 DEBUG:teuthology.orchestra.run.smithi167:> files = glob.glob(asok_path) 2024-05-11T12:10:59.228 DEBUG:teuthology.orchestra.run.smithi167:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-05-11T12:10:59.228 DEBUG:teuthology.orchestra.run.smithi167:> 2024-05-11T12:10:59.229 DEBUG:teuthology.orchestra.run.smithi167:> # Given a non-glob path, it better be there 2024-05-11T12:10:59.229 DEBUG:teuthology.orchestra.run.smithi167:> if "*" not in asok_path: 2024-05-11T12:10:59.229 DEBUG:teuthology.orchestra.run.smithi167:> assert(len(files) == 1) 2024-05-11T12:10:59.229 DEBUG:teuthology.orchestra.run.smithi167:> return files[0] 2024-05-11T12:10:59.229 DEBUG:teuthology.orchestra.run.smithi167:> 2024-05-11T12:10:59.229 DEBUG:teuthology.orchestra.run.smithi167:> for f in files: 2024-05-11T12:10:59.229 DEBUG:teuthology.orchestra.run.smithi167:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-11T12:10:59.229 DEBUG:teuthology.orchestra.run.smithi167:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-11T12:10:59.229 DEBUG:teuthology.orchestra.run.smithi167:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-11T12:10:59.229 DEBUG:teuthology.orchestra.run.smithi167:> contents = proc_f.read() 2024-05-11T12:10:59.229 DEBUG:teuthology.orchestra.run.smithi167:> if mountpoint in contents: 2024-05-11T12:10:59.229 DEBUG:teuthology.orchestra.run.smithi167:> return f 2024-05-11T12:10:59.229 DEBUG:teuthology.orchestra.run.smithi167:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-11T12:10:59.229 DEBUG:teuthology.orchestra.run.smithi167:> 2024-05-11T12:10:59.229 DEBUG:teuthology.orchestra.run.smithi167:> print(_find_admin_socket("client.admin")) 2024-05-11T12:10:59.229 DEBUG:teuthology.orchestra.run.smithi167:> ' 2024-05-11T12:10:59.394 INFO:teuthology.orchestra.run.smithi167.stdout:/var/run/ceph/ceph-client.admin.41137.asok 2024-05-11T12:10:59.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:10:59.412 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.41137.asok 2024-05-11T12:10:59.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:10:59.412 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.41137.asok status 2024-05-11T12:10:59.640 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T12:10:59.640 INFO:teuthology.orchestra.run.smithi167.stdout: "metadata": { 2024-05-11T12:10:59.640 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_sha1": "308506802b0bdea4df7050cfc1fcbdc04fea6a6e", 2024-05-11T12:10:59.641 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_version": "ceph version 19.0.0-3601-g30850680 (308506802b0bdea4df7050cfc1fcbdc04fea6a6e) squid (dev)", 2024-05-11T12:10:59.641 INFO:teuthology.orchestra.run.smithi167.stdout: "entity_id": "admin", 2024-05-11T12:10:59.641 INFO:teuthology.orchestra.run.smithi167.stdout: "hostname": "smithi167", 2024-05-11T12:10:59.641 INFO:teuthology.orchestra.run.smithi167.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-05-11T12:10:59.641 INFO:teuthology.orchestra.run.smithi167.stdout: "pid": "41137", 2024-05-11T12:10:59.641 INFO:teuthology.orchestra.run.smithi167.stdout: "root": "/" 2024-05-11T12:10:59.641 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:10:59.641 INFO:teuthology.orchestra.run.smithi167.stdout: "dentry_count": 0, 2024-05-11T12:10:59.641 INFO:teuthology.orchestra.run.smithi167.stdout: "dentry_pinned_count": 0, 2024-05-11T12:10:59.641 INFO:teuthology.orchestra.run.smithi167.stdout: "id": 4404, 2024-05-11T12:10:59.641 INFO:teuthology.orchestra.run.smithi167.stdout: "inst": { 2024-05-11T12:10:59.641 INFO:teuthology.orchestra.run.smithi167.stdout: "name": { 2024-05-11T12:10:59.641 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "client", 2024-05-11T12:10:59.641 INFO:teuthology.orchestra.run.smithi167.stdout: "num": 4404 2024-05-11T12:10:59.642 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:10:59.642 INFO:teuthology.orchestra.run.smithi167.stdout: "addr": { 2024-05-11T12:10:59.642 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "v1", 2024-05-11T12:10:59.642 INFO:teuthology.orchestra.run.smithi167.stdout: "addr": "192.168.0.1:0", 2024-05-11T12:10:59.642 INFO:teuthology.orchestra.run.smithi167.stdout: "nonce": 1785413050 2024-05-11T12:10:59.642 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:10:59.642 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:10:59.642 INFO:teuthology.orchestra.run.smithi167.stdout: "addr": { 2024-05-11T12:10:59.642 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "v1", 2024-05-11T12:10:59.642 INFO:teuthology.orchestra.run.smithi167.stdout: "addr": "192.168.0.1:0", 2024-05-11T12:10:59.642 INFO:teuthology.orchestra.run.smithi167.stdout: "nonce": 1785413050 2024-05-11T12:10:59.642 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:10:59.642 INFO:teuthology.orchestra.run.smithi167.stdout: "inst_str": "client.4404 192.168.0.1:0/1785413050", 2024-05-11T12:10:59.642 INFO:teuthology.orchestra.run.smithi167.stdout: "addr_str": "192.168.0.1:0/1785413050", 2024-05-11T12:10:59.643 INFO:teuthology.orchestra.run.smithi167.stdout: "inode_count": 1, 2024-05-11T12:10:59.643 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_epoch": 7, 2024-05-11T12:10:59.643 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_epoch": 27, 2024-05-11T12:10:59.643 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_epoch_barrier": 0, 2024-05-11T12:10:59.643 INFO:teuthology.orchestra.run.smithi167.stdout: "blocklisted": false, 2024-05-11T12:10:59.643 INFO:teuthology.orchestra.run.smithi167.stdout: "fs_name": "cephfs" 2024-05-11T12:10:59.643 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T12:10:59.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:10:59.655 DEBUG:teuthology.orchestra.run.smithi167:> stat --file-system '--printf=%T 2024-05-11T12:10:59.655 DEBUG:teuthology.orchestra.run.smithi167:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-05-11T12:10:59.711 INFO:teuthology.orchestra.run.smithi167.stdout:fuseblk 2024-05-11T12:10:59.711 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-05-11T12:10:59.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:10:59.712 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-05-11T12:10:59.798 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-05-11T12:10:59.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-11T12:10:59.947 INFO:teuthology.orchestra.run.smithi167.stdout:# file: . 2024-05-11T12:10:59.947 INFO:teuthology.orchestra.run.smithi167.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-05-11T12:10:59.947 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:10:59.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:10:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:10:59.968 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:10:59.968 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:11:00.020 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-11T12:11:00.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.222+0000 7fe3a0000640 1 Processor -- start 2024-05-11T12:11:00.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.223+0000 7fe3a0000640 1 -- start start 2024-05-11T12:11:00.224 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.224+0000 7fe3a0000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe3980cf210 0x7fe3980afe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:00.224 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.224+0000 7fe3a0000640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3980aed90 con 0x7fe3980ae820 2024-05-11T12:11:00.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.224+0000 7fe3a0000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3980af070 con 0x7fe398145a00 2024-05-11T12:11:00.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.224+0000 7fe3a0000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fe3980af9a0 con 0x7fe3980cf210 2024-05-11T12:11:00.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.225+0000 7fe39cc00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fe3980ae820 0x7fe3980ceb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:45070/0 (socket says 172.21.15.167:45070) 2024-05-11T12:11:00.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.225+0000 7fe39cc00640 1 -- 172.21.15.167:0/3760195003 learned_addr learned my addr 172.21.15.167:0/3760195003 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:00.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.225+0000 7fe39e000640 1 --2- 172.21.15.167:0/3760195003 >> v2:172.21.15.204:3300/0 conn(0x7fe3980cf210 0x7fe3980afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:00.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.225+0000 7fe397e00640 1 -- 172.21.15.167:0/3760195003 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 641787475 0 0) 0x7fe3980aed90 con 0x7fe3980ae820 2024-05-11T12:11:00.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.225+0000 7fe397e00640 1 -- 172.21.15.167:0/3760195003 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe380003520 con 0x7fe3980ae820 2024-05-11T12:11:00.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.225+0000 7fe397e00640 1 -- 172.21.15.167:0/3760195003 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 496013051 0 0) 0x7fe380003520 con 0x7fe3980ae820 2024-05-11T12:11:00.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.225+0000 7fe397e00640 1 -- 172.21.15.167:0/3760195003 >> v2:172.21.15.204:3300/0 conn(0x7fe3980cf210 msgr2=0x7fe3980afe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:00.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.225+0000 7fe397e00640 1 --2- 172.21.15.167:0/3760195003 >> v2:172.21.15.204:3300/0 conn(0x7fe3980cf210 0x7fe3980afe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:00.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.225+0000 7fe397e00640 1 -- 172.21.15.167:0/3760195003 >> v1:172.21.15.204:6789/0 conn(0x7fe398145a00 legacy=0x7fe39806f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:00.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.225+0000 7fe397e00640 1 -- 172.21.15.167:0/3760195003 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3980af350 con 0x7fe3980ae820 2024-05-11T12:11:00.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.225+0000 7fe397e00640 1 -- 172.21.15.167:0/3760195003 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7fe3840016f0 con 0x7fe3980ae820 2024-05-11T12:11:00.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.225+0000 7fe397e00640 1 -- 172.21.15.167:0/3760195003 >> v1:172.21.15.167:6789/0 conn(0x7fe3980ae820 legacy=0x7fe3980ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:00.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.226+0000 7fe397e00640 1 --2- 172.21.15.167:0/3760195003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe380003aa0 0x7fe380003ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:00.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.226+0000 7fe397e00640 1 --2- 172.21.15.167:0/3760195003 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe380004800 0x7fe380004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:00.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.226+0000 7fe397e00640 1 --2- 172.21.15.167:0/3760195003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3800052c0 0x7fe38000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:00.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.226+0000 7fe397e00640 1 -- 172.21.15.167:0/3760195003 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe3980af9a0 con 0x7fe3800052c0 2024-05-11T12:11:00.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.226+0000 7fe397e00640 1 -- 172.21.15.167:0/3760195003 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe38000c200 con 0x7fe380003aa0 2024-05-11T12:11:00.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.226+0000 7fe397e00640 1 -- 172.21.15.167:0/3760195003 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe38000c400 con 0x7fe380004800 2024-05-11T12:11:00.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.226+0000 7fe39e000640 1 --2- 172.21.15.167:0/3760195003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3800052c0 0x7fe38000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:00.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.226+0000 7fe39d600640 1 --2- 172.21.15.167:0/3760195003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe380003aa0 0x7fe380003ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:00.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.226+0000 7fe39cc00640 1 --2- 172.21.15.167:0/3760195003 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe380004800 0x7fe380004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:00.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.226+0000 7fe39e000640 1 -- 172.21.15.167:0/3760195003 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe380004800 msgr2=0x7fe380004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:00.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.226+0000 7fe39e000640 1 --2- 172.21.15.167:0/3760195003 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe380004800 0x7fe380004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:00.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.226+0000 7fe39e000640 1 -- 172.21.15.167:0/3760195003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe380003aa0 msgr2=0x7fe380003ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:00.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.226+0000 7fe39e000640 1 --2- 172.21.15.167:0/3760195003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe380003aa0 0x7fe380003ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:00.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.226+0000 7fe39e000640 1 -- 172.21.15.167:0/3760195003 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe380003520 con 0x7fe3800052c0 2024-05-11T12:11:00.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.226+0000 7fe39cc00640 1 --2- 172.21.15.167:0/3760195003 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe380004800 0x7fe380004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:00.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.227+0000 7fe39e000640 1 -- 172.21.15.167:0/3760195003 reap_dead start 2024-05-11T12:11:00.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.227+0000 7fe39e000640 1 --2- 172.21.15.167:0/3760195003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3800052c0 0x7fe38000b9c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe390009f90 tx=0x7fe380005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6819ba30c1b8568b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:00.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.228+0000 7fe397e00640 1 -- 172.21.15.167:0/3760195003 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe390035020 con 0x7fe3800052c0 2024-05-11T12:11:00.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.228+0000 7fe397e00640 1 -- 172.21.15.167:0/3760195003 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe390003b90 con 0x7fe3800052c0 2024-05-11T12:11:00.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.228+0000 7fe397e00640 1 -- 172.21.15.167:0/3760195003 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe390009c10 con 0x7fe3800052c0 2024-05-11T12:11:00.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.228+0000 7fe3a0000640 1 -- 172.21.15.167:0/3760195003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3800052c0 msgr2=0x7fe38000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:00.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.228+0000 7fe3a0000640 1 --2- 172.21.15.167:0/3760195003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3800052c0 0x7fe38000b9c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe390009f90 tx=0x7fe380005830 comp rx=0 tx=0).stop 2024-05-11T12:11:00.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.228+0000 7fe3a0000640 1 -- 172.21.15.167:0/3760195003 shutdown_connections 2024-05-11T12:11:00.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.228+0000 7fe3a0000640 1 --2- 172.21.15.167:0/3760195003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3800052c0 0x7fe38000b9c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:00.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.228+0000 7fe3a0000640 1 -- 172.21.15.167:0/3760195003 >> 172.21.15.167:0/3760195003 conn(0x7fe39806c450 msgr2=0x7fe39806c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:00.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.229+0000 7fe3a0000640 1 -- 172.21.15.167:0/3760195003 shutdown_connections 2024-05-11T12:11:00.232 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.229+0000 7fe3a0000640 1 -- 172.21.15.167:0/3760195003 wait complete. 2024-05-11T12:11:00.232 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.230+0000 7fe3a0000640 1 Processor -- start 2024-05-11T12:11:00.232 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.231+0000 7fe3a0000640 1 -- start start 2024-05-11T12:11:00.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.231+0000 7fe3a0000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3800052c0 0x7fe398142cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:00.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.231+0000 7fe3a0000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3981d5250 0x7fe3981d86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:00.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.231+0000 7fe3a0000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe398140840 0x7fe398143300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:00.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.231+0000 7fe3a0000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe3980affb0 con 0x7fe398140840 2024-05-11T12:11:00.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.231+0000 7fe3a0000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe3980afe30 con 0x7fe3981d5250 2024-05-11T12:11:00.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.231+0000 7fe3a0000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe3980b0130 con 0x7fe3800052c0 2024-05-11T12:11:00.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.232+0000 7fe39e000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe398140840 0x7fe398143300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:00.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.232+0000 7fe39e000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe398140840 0x7fe398143300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:42130/0 (socket says 172.21.15.167:42130) 2024-05-11T12:11:00.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.232+0000 7fe39e000640 1 -- 172.21.15.167:0/529909397 learned_addr learned my addr 172.21.15.167:0/529909397 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:00.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.232+0000 7fe39d600640 1 --2- 172.21.15.167:0/529909397 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3800052c0 0x7fe398142cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:00.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.232+0000 7fe39cc00640 1 --2- 172.21.15.167:0/529909397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3981d5250 0x7fe3981d86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:00.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.232+0000 7fe39e000640 1 -- 172.21.15.167:0/529909397 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3800052c0 msgr2=0x7fe398142cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:00.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.232+0000 7fe39e000640 1 --2- 172.21.15.167:0/529909397 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3800052c0 0x7fe398142cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:00.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.232+0000 7fe39e000640 1 -- 172.21.15.167:0/529909397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3981d5250 msgr2=0x7fe3981d86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:00.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.232+0000 7fe39e000640 1 --2- 172.21.15.167:0/529909397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3981d5250 0x7fe3981d86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:00.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.232+0000 7fe39e000640 1 -- 172.21.15.167:0/529909397 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3981d8f00 con 0x7fe398140840 2024-05-11T12:11:00.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.232+0000 7fe39d600640 1 --2- 172.21.15.167:0/529909397 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3800052c0 0x7fe398142cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:00.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.233+0000 7fe39e000640 1 --2- 172.21.15.167:0/529909397 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe398140840 0x7fe398143300 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe39002aee0 tx=0x7fe390009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:00.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.233+0000 7fe395600640 1 -- 172.21.15.167:0/529909397 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe390035070 con 0x7fe398140840 2024-05-11T12:11:00.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.233+0000 7fe395600640 1 -- 172.21.15.167:0/529909397 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe390041070 con 0x7fe398140840 2024-05-11T12:11:00.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.233+0000 7fe395600640 1 -- 172.21.15.167:0/529909397 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe390009c10 con 0x7fe398140840 2024-05-11T12:11:00.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.233+0000 7fe3a0000640 1 -- 172.21.15.167:0/529909397 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3981d91e0 con 0x7fe398140840 2024-05-11T12:11:00.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.233+0000 7fe3a0000640 1 -- 172.21.15.167:0/529909397 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3981d97a0 con 0x7fe398140840 2024-05-11T12:11:00.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.234+0000 7fe3a0000640 1 -- 172.21.15.167:0/529909397 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3980694b0 con 0x7fe398140840 2024-05-11T12:11:00.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.236+0000 7fe395600640 1 -- 172.21.15.167:0/529909397 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe390005d10 con 0x7fe398140840 2024-05-11T12:11:00.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.236+0000 7fe395600640 1 --2- 172.21.15.167:0/529909397 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe378076300 0x7fe3780787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:00.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.237+0000 7fe395600640 1 -- 172.21.15.167:0/529909397 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fe3900b84c0 con 0x7fe398140840 2024-05-11T12:11:00.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.237+0000 7fe39d600640 1 --2- 172.21.15.167:0/529909397 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe378076300 0x7fe3780787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:00.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.238+0000 7fe39d600640 1 --2- 172.21.15.167:0/529909397 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe378076300 0x7fe3780787c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe38c001670 tx=0x7fe38c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:00.242 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.242+0000 7fe395600640 1 -- 172.21.15.167:0/529909397 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe390081b90 con 0x7fe398140840 2024-05-11T12:11:00.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.462+0000 7fe3a0000640 1 -- 172.21.15.167:0/529909397 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe398073b80 con 0x7fe398140840 2024-05-11T12:11:00.463 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.463+0000 7fe395600640 1 -- 172.21.15.167:0/529909397 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7fe390081490 con 0x7fe398140840 2024-05-11T12:11:00.463 INFO:teuthology.orchestra.run.smithi167.stderr:listed 0 entries 2024-05-11T12:11:00.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.466+0000 7fe3a0000640 1 -- 172.21.15.167:0/529909397 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe378076300 msgr2=0x7fe3780787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:00.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.466+0000 7fe3a0000640 1 --2- 172.21.15.167:0/529909397 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe378076300 0x7fe3780787c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe38c001670 tx=0x7fe38c009040 comp rx=0 tx=0).stop 2024-05-11T12:11:00.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.466+0000 7fe3a0000640 1 -- 172.21.15.167:0/529909397 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe398140840 msgr2=0x7fe398143300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:00.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.466+0000 7fe3a0000640 1 --2- 172.21.15.167:0/529909397 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe398140840 0x7fe398143300 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe39002aee0 tx=0x7fe390009750 comp rx=0 tx=0).stop 2024-05-11T12:11:00.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.466+0000 7fe3a0000640 1 -- 172.21.15.167:0/529909397 shutdown_connections 2024-05-11T12:11:00.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.466+0000 7fe3a0000640 1 --2- 172.21.15.167:0/529909397 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe398140840 0x7fe398143300 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:00.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.466+0000 7fe3a0000640 1 --2- 172.21.15.167:0/529909397 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe378076300 0x7fe3780787c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:00.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.466+0000 7fe3a0000640 1 --2- 172.21.15.167:0/529909397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3981d5250 0x7fe3981d86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:00.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.466+0000 7fe3a0000640 1 --2- 172.21.15.167:0/529909397 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3800052c0 0x7fe398142cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:00.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.466+0000 7fe3a0000640 1 -- 172.21.15.167:0/529909397 >> 172.21.15.167:0/529909397 conn(0x7fe39806c450 msgr2=0x7fe3980b3a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:00.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.466+0000 7fe3a0000640 1 -- 172.21.15.167:0/529909397 shutdown_connections 2024-05-11T12:11:00.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.466+0000 7fe3a0000640 1 -- 172.21.15.167:0/529909397 wait complete. 2024-05-11T12:11:00.484 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:11:00.484 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:11:00.533 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-11T12:11:00.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.785+0000 7fe696600640 1 Processor -- start 2024-05-11T12:11:00.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.786+0000 7fe696600640 1 -- start start 2024-05-11T12:11:00.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.787+0000 7fe696600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe6900ce1a0 0x7fe6900ce580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:00.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.787+0000 7fe696600640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe69006df10 0x7fe69015c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:00.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.787+0000 7fe696600640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe69006e350 con 0x7fe69015fb30 2024-05-11T12:11:00.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.787+0000 7fe696600640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fe6900b60a0 con 0x7fe69006df10 2024-05-11T12:11:00.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.787+0000 7fe696600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fe6900b6380 con 0x7fe6900ce1a0 2024-05-11T12:11:00.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.787+0000 7fe68ea00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe69006df10 0x7fe69015c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:00.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.788+0000 7fe68ea00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe69006df10 0x7fe69015c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:42138/0 (socket says 172.21.15.167:42138) 2024-05-11T12:11:00.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.788+0000 7fe68ea00640 1 -- 172.21.15.167:0/944923352 learned_addr learned my addr 172.21.15.167:0/944923352 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:00.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.788+0000 7fe68fe00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fe69015fb30 0x7fe69015ffa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:45082/0 (socket says 172.21.15.167:45082) 2024-05-11T12:11:00.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.788+0000 7fe68f400640 1 --2- 172.21.15.167:0/944923352 >> v2:172.21.15.204:3300/0 conn(0x7fe6900ce1a0 0x7fe6900ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:00.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.788+0000 7fe68ea00640 1 -- 172.21.15.167:0/944923352 >> v2:172.21.15.204:3300/0 conn(0x7fe6900ce1a0 msgr2=0x7fe6900ce580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:00.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.788+0000 7fe68ea00640 1 --2- 172.21.15.167:0/944923352 >> v2:172.21.15.204:3300/0 conn(0x7fe6900ce1a0 0x7fe6900ce580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:00.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.788+0000 7fe68ea00640 1 -- 172.21.15.167:0/944923352 >> v1:172.21.15.167:6789/0 conn(0x7fe69015fb30 legacy=0x7fe69015ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:00.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.788+0000 7fe68ea00640 1 -- 172.21.15.167:0/944923352 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe69015cdc0 con 0x7fe69006df10 2024-05-11T12:11:00.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.788+0000 7fe68f400640 1 --2- 172.21.15.167:0/944923352 >> v2:172.21.15.204:3300/0 conn(0x7fe6900ce1a0 0x7fe6900ce580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:11:00.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.788+0000 7fe68e000640 1 -- 172.21.15.167:0/944923352 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3718454373 0 0) 0x7fe69006e350 con 0x7fe69015fb30 2024-05-11T12:11:00.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.789+0000 7fe68ea00640 1 --2- 172.21.15.167:0/944923352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe69006df10 0x7fe69015c540 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe6900cecf0 tx=0x7fe67802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecaa5d949725cf52 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:00.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.789+0000 7fe68e000640 1 -- 172.21.15.167:0/944923352 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe678036070 con 0x7fe69006df10 2024-05-11T12:11:00.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.790+0000 7fe68e000640 1 -- 172.21.15.167:0/944923352 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe678031600 con 0x7fe69006df10 2024-05-11T12:11:00.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.790+0000 7fe68e000640 1 -- 172.21.15.167:0/944923352 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6780319d0 con 0x7fe69006df10 2024-05-11T12:11:00.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.790+0000 7fe696600640 1 -- 172.21.15.167:0/944923352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe69006df10 msgr2=0x7fe69015c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:00.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.790+0000 7fe696600640 1 --2- 172.21.15.167:0/944923352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe69006df10 0x7fe69015c540 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe6900cecf0 tx=0x7fe67802a730 comp rx=0 tx=0).stop 2024-05-11T12:11:00.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.790+0000 7fe696600640 1 -- 172.21.15.167:0/944923352 shutdown_connections 2024-05-11T12:11:00.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.790+0000 7fe696600640 1 --2- 172.21.15.167:0/944923352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe69006df10 0x7fe69015c540 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:00.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.790+0000 7fe696600640 1 --2- 172.21.15.167:0/944923352 >> v2:172.21.15.204:3300/0 conn(0x7fe6900ce1a0 0x7fe6900ce580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:00.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.790+0000 7fe696600640 1 -- 172.21.15.167:0/944923352 >> 172.21.15.167:0/944923352 conn(0x7fe6900732a0 msgr2=0x7fe6900736b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:00.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.790+0000 7fe696600640 1 -- 172.21.15.167:0/944923352 shutdown_connections 2024-05-11T12:11:00.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.790+0000 7fe696600640 1 -- 172.21.15.167:0/944923352 wait complete. 2024-05-11T12:11:00.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.792+0000 7fe696600640 1 Processor -- start 2024-05-11T12:11:00.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.792+0000 7fe696600640 1 -- start start 2024-05-11T12:11:00.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.793+0000 7fe696600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe69006df10 0x7fe6900d0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:00.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.793+0000 7fe696600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6900ce1a0 0x7fe69015d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:00.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.793+0000 7fe696600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe69015fb30 0x7fe690151350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:00.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.793+0000 7fe696600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe69015ca80 con 0x7fe69015fb30 2024-05-11T12:11:00.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.793+0000 7fe696600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe69015d380 con 0x7fe6900ce1a0 2024-05-11T12:11:00.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.793+0000 7fe696600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe6900b7820 con 0x7fe69006df10 2024-05-11T12:11:00.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.793+0000 7fe68fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe69015fb30 0x7fe690151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:00.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.793+0000 7fe68fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe69015fb30 0x7fe690151350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:42146/0 (socket says 172.21.15.167:42146) 2024-05-11T12:11:00.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.793+0000 7fe68fe00640 1 -- 172.21.15.167:0/2227204467 learned_addr learned my addr 172.21.15.167:0/2227204467 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:00.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.793+0000 7fe68f400640 1 --2- 172.21.15.167:0/2227204467 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe69006df10 0x7fe6900d0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:00.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.794+0000 7fe68ea00640 1 --2- 172.21.15.167:0/2227204467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6900ce1a0 0x7fe69015d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:00.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.794+0000 7fe68fe00640 1 -- 172.21.15.167:0/2227204467 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe69006df10 msgr2=0x7fe6900d0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:00.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.794+0000 7fe68fe00640 1 --2- 172.21.15.167:0/2227204467 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe69006df10 0x7fe6900d0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:00.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.794+0000 7fe68fe00640 1 -- 172.21.15.167:0/2227204467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6900ce1a0 msgr2=0x7fe69015d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:00.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.794+0000 7fe68fe00640 1 --2- 172.21.15.167:0/2227204467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6900ce1a0 0x7fe69015d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:00.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.794+0000 7fe68fe00640 1 -- 172.21.15.167:0/2227204467 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6900d0f60 con 0x7fe69015fb30 2024-05-11T12:11:00.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.794+0000 7fe68fe00640 1 --2- 172.21.15.167:0/2227204467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe69015fb30 0x7fe690151350 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe68800c440 tx=0x7fe68800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:00.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.794+0000 7fe687400640 1 -- 172.21.15.167:0/2227204467 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe688018070 con 0x7fe69015fb30 2024-05-11T12:11:00.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.794+0000 7fe687400640 1 -- 172.21.15.167:0/2227204467 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6880074e0 con 0x7fe69015fb30 2024-05-11T12:11:00.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.794+0000 7fe687400640 1 -- 172.21.15.167:0/2227204467 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe688004a20 con 0x7fe69015fb30 2024-05-11T12:11:00.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.794+0000 7fe696600640 1 -- 172.21.15.167:0/2227204467 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe69013d1c0 con 0x7fe69015fb30 2024-05-11T12:11:00.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.795+0000 7fe696600640 1 -- 172.21.15.167:0/2227204467 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6900d03b0 con 0x7fe69015fb30 2024-05-11T12:11:00.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.796+0000 7fe696600640 1 -- 172.21.15.167:0/2227204467 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe690069bc0 con 0x7fe69015fb30 2024-05-11T12:11:00.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.797+0000 7fe687400640 1 -- 172.21.15.167:0/2227204467 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe68801b070 con 0x7fe69015fb30 2024-05-11T12:11:00.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.798+0000 7fe687400640 1 --2- 172.21.15.167:0/2227204467 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe66c0762d0 0x7fe66c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:00.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.798+0000 7fe687400640 1 -- 172.21.15.167:0/2227204467 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fe688096b70 con 0x7fe69015fb30 2024-05-11T12:11:00.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.798+0000 7fe68f400640 1 --2- 172.21.15.167:0/2227204467 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe66c0762d0 0x7fe66c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:00.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.799+0000 7fe68f400640 1 --2- 172.21.15.167:0/2227204467 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe66c0762d0 0x7fe66c078790 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe680006fd0 tx=0x7fe680008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:00.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:00.802+0000 7fe687400640 1 -- 172.21.15.167:0/2227204467 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe688060180 con 0x7fe69015fb30 2024-05-11T12:11:01.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:01.018+0000 7fe696600640 1 -- 172.21.15.167:0/2227204467 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe6900b81e0 con 0x7fe69015fb30 2024-05-11T12:11:01.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:01.019+0000 7fe687400640 1 -- 172.21.15.167:0/2227204467 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7fe68805fa80 con 0x7fe69015fb30 2024-05-11T12:11:01.020 INFO:teuthology.orchestra.run.smithi167.stderr:listed 0 entries 2024-05-11T12:11:01.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:01.022+0000 7fe696600640 1 -- 172.21.15.167:0/2227204467 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe66c0762d0 msgr2=0x7fe66c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:01.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:01.022+0000 7fe696600640 1 --2- 172.21.15.167:0/2227204467 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe66c0762d0 0x7fe66c078790 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe680006fd0 tx=0x7fe680008040 comp rx=0 tx=0).stop 2024-05-11T12:11:01.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:01.022+0000 7fe696600640 1 -- 172.21.15.167:0/2227204467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe69015fb30 msgr2=0x7fe690151350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:01.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:01.022+0000 7fe696600640 1 --2- 172.21.15.167:0/2227204467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe69015fb30 0x7fe690151350 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe68800c440 tx=0x7fe68800c900 comp rx=0 tx=0).stop 2024-05-11T12:11:01.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:01.022+0000 7fe696600640 1 -- 172.21.15.167:0/2227204467 shutdown_connections 2024-05-11T12:11:01.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:01.022+0000 7fe696600640 1 --2- 172.21.15.167:0/2227204467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe69015fb30 0x7fe690151350 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:01.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:01.022+0000 7fe696600640 1 --2- 172.21.15.167:0/2227204467 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe66c0762d0 0x7fe66c078790 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:01.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:01.022+0000 7fe696600640 1 --2- 172.21.15.167:0/2227204467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6900ce1a0 0x7fe69015d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:01.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:01.022+0000 7fe696600640 1 --2- 172.21.15.167:0/2227204467 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe69006df10 0x7fe6900d0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:01.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:01.022+0000 7fe696600640 1 -- 172.21.15.167:0/2227204467 >> 172.21.15.167:0/2227204467 conn(0x7fe6900732a0 msgr2=0x7fe6900d1730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:01.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:01.023+0000 7fe696600640 1 -- 172.21.15.167:0/2227204467 shutdown_connections 2024-05-11T12:11:01.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:01.023+0000 7fe696600640 1 -- 172.21.15.167:0/2227204467 wait complete. 2024-05-11T12:11:01.040 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi167.front.sepia.ceph.com... 2024-05-11T12:11:01.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:01.040 DEBUG:teuthology.orchestra.run.smithi167:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-05-11T12:11:01.117 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi167.stderr:ceph-fuse[41137]: fuse finished with error 0 and tester_r 0 2024-05-11T12:11:01.123 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T12:11:06.655 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T12:11:06.662 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T12:11:07.224 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2024-05-11T12:11:07.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:07.225 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-05-11T12:11:07.289 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:11:07.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.491+0000 7f0e63200640 1 Processor -- start 2024-05-11T12:11:07.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.491+0000 7f0e63200640 1 -- start start 2024-05-11T12:11:07.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.493+0000 7f0e63200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0e5c157750 0x7f0e5c15d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:07.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.493+0000 7f0e63200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e5c0b3ea0 0x7f0e5c161490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:07.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.493+0000 7f0e63200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e5c15dc80 con 0x7f0e5c1619d0 2024-05-11T12:11:07.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.493+0000 7f0e63200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f0e5c161000 con 0x7f0e5c157750 2024-05-11T12:11:07.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.493+0000 7f0e63200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0e5c15df30 con 0x7f0e5c0b3ea0 2024-05-11T12:11:07.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.494+0000 7f0e5b400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e5c0b3ea0 0x7f0e5c161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:07.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.494+0000 7f0e61200640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f0e5c1619d0 0x7f0e5c06d460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.167:55612/0 (socket says 172.21.15.167:55612) 2024-05-11T12:11:07.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.494+0000 7f0e61200640 1 -- 172.21.15.167:0/3112725824 learned_addr learned my addr 172.21.15.167:0/3112725824 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:07.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.494+0000 7f0e5be00640 1 --2- 172.21.15.167:0/3112725824 >> v2:172.21.15.204:3300/0 conn(0x7f0e5c157750 0x7f0e5c15d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:07.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.494+0000 7f0e5b400640 1 -- 172.21.15.167:0/3112725824 >> v2:172.21.15.204:3300/0 conn(0x7f0e5c157750 msgr2=0x7f0e5c15d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:07.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.494+0000 7f0e5b400640 1 --2- 172.21.15.167:0/3112725824 >> v2:172.21.15.204:3300/0 conn(0x7f0e5c157750 0x7f0e5c15d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:07.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.495+0000 7f0e5b400640 1 -- 172.21.15.167:0/3112725824 >> v1:172.21.15.204:6789/0 conn(0x7f0e5c1619d0 legacy=0x7f0e5c06d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:07.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.495+0000 7f0e5b400640 1 -- 172.21.15.167:0/3112725824 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e5c15e210 con 0x7f0e5c0b3ea0 2024-05-11T12:11:07.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.494+0000 7f0e5aa00640 1 -- 172.21.15.167:0/3112725824 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 259832303 0 0) 0x7f0e5c15dc80 con 0x7f0e5c1619d0 2024-05-11T12:11:07.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.496+0000 7f0e5b400640 1 --2- 172.21.15.167:0/3112725824 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e5c0b3ea0 0x7f0e5c161490 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0e480097a0 tx=0x7f0e4802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=656a420fb086b740 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:07.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.496+0000 7f0e5aa00640 1 -- 172.21.15.167:0/3112725824 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e48037070 con 0x7f0e5c0b3ea0 2024-05-11T12:11:07.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.496+0000 7f0e5aa00640 1 -- 172.21.15.167:0/3112725824 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e48032440 con 0x7f0e5c0b3ea0 2024-05-11T12:11:07.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.496+0000 7f0e5aa00640 1 -- 172.21.15.167:0/3112725824 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e48032810 con 0x7f0e5c0b3ea0 2024-05-11T12:11:07.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.497+0000 7f0e63200640 1 -- 172.21.15.167:0/3112725824 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e5c0b3ea0 msgr2=0x7f0e5c161490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:07.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.497+0000 7f0e63200640 1 --2- 172.21.15.167:0/3112725824 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e5c0b3ea0 0x7f0e5c161490 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0e480097a0 tx=0x7f0e4802a180 comp rx=0 tx=0).stop 2024-05-11T12:11:07.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.497+0000 7f0e63200640 1 -- 172.21.15.167:0/3112725824 shutdown_connections 2024-05-11T12:11:07.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.497+0000 7f0e63200640 1 --2- 172.21.15.167:0/3112725824 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e5c0b3ea0 0x7f0e5c161490 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:07.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.497+0000 7f0e63200640 1 --2- 172.21.15.167:0/3112725824 >> v2:172.21.15.204:3300/0 conn(0x7f0e5c157750 0x7f0e5c15d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:07.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.497+0000 7f0e63200640 1 -- 172.21.15.167:0/3112725824 >> 172.21.15.167:0/3112725824 conn(0x7f0e5c0cd8f0 msgr2=0x7f0e5c0cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:07.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.497+0000 7f0e63200640 1 -- 172.21.15.167:0/3112725824 shutdown_connections 2024-05-11T12:11:07.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.497+0000 7f0e63200640 1 -- 172.21.15.167:0/3112725824 wait complete. 2024-05-11T12:11:07.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.499+0000 7f0e63200640 1 Processor -- start 2024-05-11T12:11:07.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.499+0000 7f0e63200640 1 -- start start 2024-05-11T12:11:07.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.500+0000 7f0e63200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e5c0b3ea0 0x7f0e5c15c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:07.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.500+0000 7f0e63200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e5c157750 0x7f0e5c141560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:07.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.500+0000 7f0e63200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e5c1619d0 0x7f0e5c1f3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:07.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.500+0000 7f0e63200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5c06db70 con 0x7f0e5c1619d0 2024-05-11T12:11:07.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.500+0000 7f0e63200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5c157b30 con 0x7f0e5c0b3ea0 2024-05-11T12:11:07.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.500+0000 7f0e63200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0e5c06dcf0 con 0x7f0e5c157750 2024-05-11T12:11:07.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.500+0000 7f0e61200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e5c1619d0 0x7f0e5c1f3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:07.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.500+0000 7f0e61200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e5c1619d0 0x7f0e5c1f3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:42158/0 (socket says 172.21.15.167:42158) 2024-05-11T12:11:07.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.500+0000 7f0e61200640 1 -- 172.21.15.167:0/2369700348 learned_addr learned my addr 172.21.15.167:0/2369700348 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:07.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.500+0000 7f0e5b400640 1 --2- 172.21.15.167:0/2369700348 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e5c157750 0x7f0e5c141560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:07.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.501+0000 7f0e61200640 1 -- 172.21.15.167:0/2369700348 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e5c157750 msgr2=0x7f0e5c141560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:07.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.501+0000 7f0e61200640 1 --2- 172.21.15.167:0/2369700348 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e5c157750 0x7f0e5c141560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:07.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.501+0000 7f0e61200640 1 -- 172.21.15.167:0/2369700348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e5c0b3ea0 msgr2=0x7f0e5c15c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:07.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.501+0000 7f0e61200640 1 --2- 172.21.15.167:0/2369700348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e5c0b3ea0 0x7f0e5c15c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:07.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.501+0000 7f0e61200640 1 -- 172.21.15.167:0/2369700348 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e5c15b7a0 con 0x7f0e5c1619d0 2024-05-11T12:11:07.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.501+0000 7f0e61200640 1 --2- 172.21.15.167:0/2369700348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e5c1619d0 0x7f0e5c1f3ff0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0e5400c390 tx=0x7f0e5400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:07.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.501+0000 7f0e53e00640 1 -- 172.21.15.167:0/2369700348 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e540074e0 con 0x7f0e5c1619d0 2024-05-11T12:11:07.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.501+0000 7f0e53e00640 1 -- 172.21.15.167:0/2369700348 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e540047e0 con 0x7f0e5c1619d0 2024-05-11T12:11:07.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.501+0000 7f0e63200640 1 -- 172.21.15.167:0/2369700348 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e5c15ba80 con 0x7f0e5c1619d0 2024-05-11T12:11:07.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.501+0000 7f0e53e00640 1 -- 172.21.15.167:0/2369700348 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e54004bd0 con 0x7f0e5c1619d0 2024-05-11T12:11:07.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.501+0000 7f0e63200640 1 -- 172.21.15.167:0/2369700348 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e5c15c040 con 0x7f0e5c1619d0 2024-05-11T12:11:07.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.503+0000 7f0e63200640 1 -- 172.21.15.167:0/2369700348 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e5c069590 con 0x7f0e5c1619d0 2024-05-11T12:11:07.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.504+0000 7f0e53e00640 1 -- 172.21.15.167:0/2369700348 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0e54011070 con 0x7f0e5c1619d0 2024-05-11T12:11:07.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.505+0000 7f0e53e00640 1 --2- 172.21.15.167:0/2369700348 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0e3c0762d0 0x7f0e3c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:07.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.505+0000 7f0e53e00640 1 -- 172.21.15.167:0/2369700348 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f0e54097550 con 0x7f0e5c1619d0 2024-05-11T12:11:07.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.505+0000 7f0e5be00640 1 --2- 172.21.15.167:0/2369700348 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0e3c0762d0 0x7f0e3c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:07.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.506+0000 7f0e5be00640 1 --2- 172.21.15.167:0/2369700348 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0e3c0762d0 0x7f0e3c078790 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c15b170 tx=0x7f0e4c00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:07.510 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.510+0000 7f0e53e00640 1 -- 172.21.15.167:0/2369700348 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0e54060ad0 con 0x7f0e5c1619d0 2024-05-11T12:11:07.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.718+0000 7f0e63200640 1 -- 172.21.15.167:0/2369700348 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0e5c06d930 con 0x7f0e5c1619d0 2024-05-11T12:11:07.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.720+0000 7f0e53e00640 1 -- 172.21.15.167:0/2369700348 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18562 (secure 0 0 0) 0x7f0e540603d0 con 0x7f0e5c1619d0 2024-05-11T12:11:07.721 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:11:07.721 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":27,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:10:50.284769+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:10:43.084511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T12:10:44.157782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-11T12:10:47.814674+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:11:07.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.722+0000 7f0e63200640 1 -- 172.21.15.167:0/2369700348 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0e3c0762d0 msgr2=0x7f0e3c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:07.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.722+0000 7f0e63200640 1 --2- 172.21.15.167:0/2369700348 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0e3c0762d0 0x7f0e3c078790 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c15b170 tx=0x7f0e4c00b040 comp rx=0 tx=0).stop 2024-05-11T12:11:07.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.723+0000 7f0e63200640 1 -- 172.21.15.167:0/2369700348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e5c1619d0 msgr2=0x7f0e5c1f3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:07.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.723+0000 7f0e63200640 1 --2- 172.21.15.167:0/2369700348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e5c1619d0 0x7f0e5c1f3ff0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0e5400c390 tx=0x7f0e5400c850 comp rx=0 tx=0).stop 2024-05-11T12:11:07.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.723+0000 7f0e63200640 1 -- 172.21.15.167:0/2369700348 shutdown_connections 2024-05-11T12:11:07.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.723+0000 7f0e63200640 1 --2- 172.21.15.167:0/2369700348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e5c1619d0 0x7f0e5c1f3ff0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:07.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.723+0000 7f0e63200640 1 --2- 172.21.15.167:0/2369700348 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e5c157750 0x7f0e5c141560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:07.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.723+0000 7f0e63200640 1 --2- 172.21.15.167:0/2369700348 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0e3c0762d0 0x7f0e3c078790 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:07.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.723+0000 7f0e63200640 1 --2- 172.21.15.167:0/2369700348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e5c0b3ea0 0x7f0e5c15c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:07.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.723+0000 7f0e63200640 1 -- 172.21.15.167:0/2369700348 >> 172.21.15.167:0/2369700348 conn(0x7f0e5c0cd8f0 msgr2=0x7f0e5c15e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:07.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.723+0000 7f0e63200640 1 -- 172.21.15.167:0/2369700348 shutdown_connections 2024-05-11T12:11:07.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.723+0000 7f0e63200640 1 -- 172.21.15.167:0/2369700348 wait complete. 2024-05-11T12:11:07.743 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:11:07.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.944+0000 7fb799600640 1 Processor -- start 2024-05-11T12:11:07.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.944+0000 7fb799600640 1 -- start start 2024-05-11T12:11:07.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.945+0000 7fb799600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb79414c8a0 0x7fb7940cf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:07.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.945+0000 7fb799600640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb7940cf700 0x7fb79406d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:07.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.946+0000 7fb799600640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7940cfc10 con 0x7fb79406d9a0 2024-05-11T12:11:07.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.946+0000 7fb799600640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fb79406ddf0 con 0x7fb7940cf700 2024-05-11T12:11:07.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.946+0000 7fb799600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fb7940cfef0 con 0x7fb79414c8a0 2024-05-11T12:11:07.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.946+0000 7fb78be00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb7940cf700 0x7fb79406d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:07.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.946+0000 7fb78be00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb7940cf700 0x7fb79406d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:42170/0 (socket says 172.21.15.167:42170) 2024-05-11T12:11:07.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.946+0000 7fb78be00640 1 -- 172.21.15.167:0/2197237435 learned_addr learned my addr 172.21.15.167:0/2197237435 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:07.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.946+0000 7fb793400640 1 --1- 172.21.15.167:0/2197237435 >> v1:172.21.15.167:6789/0 conn(0x7fb79406d9a0 0x7fb79406e370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:45090/0 (socket says 172.21.15.167:45090) 2024-05-11T12:11:07.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.946+0000 7fb792a00640 1 --2- 172.21.15.167:0/2197237435 >> v2:172.21.15.204:3300/0 conn(0x7fb79414c8a0 0x7fb7940cf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:07.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.946+0000 7fb78be00640 1 -- 172.21.15.167:0/2197237435 >> v2:172.21.15.204:3300/0 conn(0x7fb79414c8a0 msgr2=0x7fb7940cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:07.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.946+0000 7fb78be00640 1 --2- 172.21.15.167:0/2197237435 >> v2:172.21.15.204:3300/0 conn(0x7fb79414c8a0 0x7fb7940cf1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:07.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.946+0000 7fb78be00640 1 -- 172.21.15.167:0/2197237435 >> v1:172.21.15.167:6789/0 conn(0x7fb79406d9a0 legacy=0x7fb79406e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:07.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.946+0000 7fb78be00640 1 -- 172.21.15.167:0/2197237435 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7940d01d0 con 0x7fb7940cf700 2024-05-11T12:11:07.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.946+0000 7fb792a00640 1 --2- 172.21.15.167:0/2197237435 >> v2:172.21.15.204:3300/0 conn(0x7fb79414c8a0 0x7fb7940cf1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:07.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.947+0000 7fb78be00640 1 --2- 172.21.15.167:0/2197237435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7940cf700 0x7fb79406d460 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb7940cebc0 tx=0x7fb77c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5be7d5a4335853ec server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:07.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.948+0000 7fb792000640 1 -- 172.21.15.167:0/2197237435 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c037070 con 0x7fb7940cf700 2024-05-11T12:11:07.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.948+0000 7fb792000640 1 -- 172.21.15.167:0/2197237435 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb77c032440 con 0x7fb7940cf700 2024-05-11T12:11:07.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.948+0000 7fb792000640 1 -- 172.21.15.167:0/2197237435 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c032810 con 0x7fb7940cf700 2024-05-11T12:11:07.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.948+0000 7fb799600640 1 -- 172.21.15.167:0/2197237435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7940cf700 msgr2=0x7fb79406d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:07.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.948+0000 7fb799600640 1 --2- 172.21.15.167:0/2197237435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7940cf700 0x7fb79406d460 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb7940cebc0 tx=0x7fb77c02a270 comp rx=0 tx=0).stop 2024-05-11T12:11:07.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.949+0000 7fb799600640 1 -- 172.21.15.167:0/2197237435 shutdown_connections 2024-05-11T12:11:07.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.949+0000 7fb799600640 1 --2- 172.21.15.167:0/2197237435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7940cf700 0x7fb79406d460 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:07.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.949+0000 7fb799600640 1 --2- 172.21.15.167:0/2197237435 >> v2:172.21.15.204:3300/0 conn(0x7fb79414c8a0 0x7fb7940cf1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:07.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.949+0000 7fb799600640 1 -- 172.21.15.167:0/2197237435 >> 172.21.15.167:0/2197237435 conn(0x7fb7940cd8f0 msgr2=0x7fb7940cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:07.950 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.949+0000 7fb799600640 1 -- 172.21.15.167:0/2197237435 shutdown_connections 2024-05-11T12:11:07.950 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.949+0000 7fb799600640 1 -- 172.21.15.167:0/2197237435 wait complete. 2024-05-11T12:11:07.950 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.950+0000 7fb799600640 1 Processor -- start 2024-05-11T12:11:07.950 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.951+0000 7fb799600640 1 -- start start 2024-05-11T12:11:07.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.952+0000 7fb799600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb79406d9a0 0x7fb7940afc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:07.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.952+0000 7fb799600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7940cf700 0x7fb7941625c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:07.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.952+0000 7fb799600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb79414c8a0 0x7fb7941fc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:07.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.952+0000 7fb799600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb79406fd30 con 0x7fb79406d9a0 2024-05-11T12:11:07.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.952+0000 7fb799600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb7940b3fa0 con 0x7fb7940cf700 2024-05-11T12:11:07.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.952+0000 7fb799600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb79406feb0 con 0x7fb79414c8a0 2024-05-11T12:11:07.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.952+0000 7fb792a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb79406d9a0 0x7fb7940afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:07.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.952+0000 7fb792a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb79406d9a0 0x7fb7940afc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:42178/0 (socket says 172.21.15.167:42178) 2024-05-11T12:11:07.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.952+0000 7fb792a00640 1 -- 172.21.15.167:0/3651806329 learned_addr learned my addr 172.21.15.167:0/3651806329 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:07.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.952+0000 7fb792a00640 1 -- 172.21.15.167:0/3651806329 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb79414c8a0 msgr2=0x7fb7941fc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:07.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.952+0000 7fb792a00640 1 --2- 172.21.15.167:0/3651806329 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb79414c8a0 0x7fb7941fc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:07.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.952+0000 7fb792a00640 1 -- 172.21.15.167:0/3651806329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7940cf700 msgr2=0x7fb7941625c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:07.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.952+0000 7fb792a00640 1 --2- 172.21.15.167:0/3651806329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7940cf700 0x7fb7941625c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:07.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.952+0000 7fb792a00640 1 -- 172.21.15.167:0/3651806329 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7940aebe0 con 0x7fb79406d9a0 2024-05-11T12:11:07.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.953+0000 7fb792a00640 1 --2- 172.21.15.167:0/3651806329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb79406d9a0 0x7fb7940afc80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb78400c7f0 tx=0x7fb78400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:07.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.953+0000 7fb78aa00640 1 -- 172.21.15.167:0/3651806329 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb784015070 con 0x7fb79406d9a0 2024-05-11T12:11:07.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.953+0000 7fb78aa00640 1 -- 172.21.15.167:0/3651806329 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7840077c0 con 0x7fb79406d9a0 2024-05-11T12:11:07.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.953+0000 7fb78aa00640 1 -- 172.21.15.167:0/3651806329 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb784007b70 con 0x7fb79406d9a0 2024-05-11T12:11:07.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.953+0000 7fb799600640 1 -- 172.21.15.167:0/3651806329 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7940aeec0 con 0x7fb79406d9a0 2024-05-11T12:11:07.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.953+0000 7fb799600640 1 -- 172.21.15.167:0/3651806329 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7940af480 con 0x7fb79406d9a0 2024-05-11T12:11:07.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.954+0000 7fb799600640 1 -- 172.21.15.167:0/3651806329 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb794069590 con 0x7fb79406d9a0 2024-05-11T12:11:07.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.956+0000 7fb78aa00640 1 -- 172.21.15.167:0/3651806329 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb784018070 con 0x7fb79406d9a0 2024-05-11T12:11:07.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.956+0000 7fb78aa00640 1 --2- 172.21.15.167:0/3651806329 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb770076300 0x7fb7700787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:07.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.957+0000 7fb78aa00640 1 -- 172.21.15.167:0/3651806329 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fb7840979c0 con 0x7fb79406d9a0 2024-05-11T12:11:07.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.957+0000 7fb78be00640 1 --2- 172.21.15.167:0/3651806329 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb770076300 0x7fb7700787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:07.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.958+0000 7fb78be00640 1 --2- 172.21.15.167:0/3651806329 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb770076300 0x7fb7700787c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb77c034000 tx=0x7fb77c02c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:07.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:07.961+0000 7fb78aa00640 1 -- 172.21.15.167:0/3651806329 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb784060fd0 con 0x7fb79406d9a0 2024-05-11T12:11:08.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.170+0000 7fb799600640 1 -- 172.21.15.167:0/3651806329 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb794075b40 con 0x7fb79406d9a0 2024-05-11T12:11:08.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.173+0000 7fb78aa00640 1 -- 172.21.15.167:0/3651806329 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18562 (secure 0 0 0) 0x7fb7840608d0 con 0x7fb79406d9a0 2024-05-11T12:11:08.173 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:11:08.173 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":27,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:10:50.284769+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:10:43.084511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T12:10:44.157782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-11T12:10:47.814674+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:11:08.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.175+0000 7fb799600640 1 -- 172.21.15.167:0/3651806329 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb770076300 msgr2=0x7fb7700787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:08.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.175+0000 7fb799600640 1 --2- 172.21.15.167:0/3651806329 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb770076300 0x7fb7700787c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb77c034000 tx=0x7fb77c02c000 comp rx=0 tx=0).stop 2024-05-11T12:11:08.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.175+0000 7fb799600640 1 -- 172.21.15.167:0/3651806329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb79406d9a0 msgr2=0x7fb7940afc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:08.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.175+0000 7fb799600640 1 --2- 172.21.15.167:0/3651806329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb79406d9a0 0x7fb7940afc80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb78400c7f0 tx=0x7fb78400ccb0 comp rx=0 tx=0).stop 2024-05-11T12:11:08.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.175+0000 7fb799600640 1 -- 172.21.15.167:0/3651806329 shutdown_connections 2024-05-11T12:11:08.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.175+0000 7fb799600640 1 --2- 172.21.15.167:0/3651806329 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb79414c8a0 0x7fb7941fc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:08.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.175+0000 7fb799600640 1 --2- 172.21.15.167:0/3651806329 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb770076300 0x7fb7700787c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:08.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.175+0000 7fb799600640 1 --2- 172.21.15.167:0/3651806329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7940cf700 0x7fb7941625c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:08.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.175+0000 7fb799600640 1 --2- 172.21.15.167:0/3651806329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb79406d9a0 0x7fb7940afc80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:08.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.175+0000 7fb799600640 1 -- 172.21.15.167:0/3651806329 >> 172.21.15.167:0/3651806329 conn(0x7fb7940cd8f0 msgr2=0x7fb7940b1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:08.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.175+0000 7fb799600640 1 -- 172.21.15.167:0/3651806329 shutdown_connections 2024-05-11T12:11:08.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.176+0000 7fb799600640 1 -- 172.21.15.167:0/3651806329 wait complete. 2024-05-11T12:11:08.193 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-11T12:11:08.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.399+0000 7f2eea800640 1 Processor -- start 2024-05-11T12:11:08.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.400+0000 7f2eea800640 1 -- start start 2024-05-11T12:11:08.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.401+0000 7f2eea800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2ee415f0d0 0x7f2ee40d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:08.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.401+0000 7f2eea800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2ee406cef0 0x7f2ee406d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:08.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.401+0000 7f2eea800640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ee40d2c50 con 0x7f2ee40cfda0 2024-05-11T12:11:08.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.401+0000 7f2eea800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f2ee40d2f30 con 0x7f2ee415f0d0 2024-05-11T12:11:08.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.401+0000 7f2eea800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2ee40d3210 con 0x7f2ee406cef0 2024-05-11T12:11:08.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.401+0000 7f2ee8a00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f2ee40cfda0 0x7f2ee40d3cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:45100/0 (socket says 172.21.15.167:45100) 2024-05-11T12:11:08.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.401+0000 7f2ee8a00640 1 -- 172.21.15.167:0/4127547864 learned_addr learned my addr 172.21.15.167:0/4127547864 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:08.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.402+0000 7f2ee9e00640 1 --2- 172.21.15.167:0/4127547864 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2ee406cef0 0x7f2ee406d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:08.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.402+0000 7f2ee9400640 1 --2- 172.21.15.167:0/4127547864 >> v2:172.21.15.204:3300/0 conn(0x7f2ee415f0d0 0x7f2ee40d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:08.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.402+0000 7f2ee3e00640 1 -- 172.21.15.167:0/4127547864 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 454882886 0 0) 0x7f2ee40d2c50 con 0x7f2ee40cfda0 2024-05-11T12:11:08.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.402+0000 7f2ee3e00640 1 -- 172.21.15.167:0/4127547864 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ecc003520 con 0x7f2ee40cfda0 2024-05-11T12:11:08.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.402+0000 7f2ee3e00640 1 -- 172.21.15.167:0/4127547864 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4131260893 0 0) 0x7f2ecc003520 con 0x7f2ee40cfda0 2024-05-11T12:11:08.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.402+0000 7f2ee3e00640 1 -- 172.21.15.167:0/4127547864 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2ee406cef0 msgr2=0x7f2ee406d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:08.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.402+0000 7f2ee3e00640 1 --2- 172.21.15.167:0/4127547864 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2ee406cef0 0x7f2ee406d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:08.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.402+0000 7f2ee3e00640 1 -- 172.21.15.167:0/4127547864 >> v2:172.21.15.204:3300/0 conn(0x7f2ee415f0d0 msgr2=0x7f2ee40d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:08.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.402+0000 7f2ee3e00640 1 --2- 172.21.15.167:0/4127547864 >> v2:172.21.15.204:3300/0 conn(0x7f2ee415f0d0 0x7f2ee40d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:08.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.402+0000 7f2ee3e00640 1 -- 172.21.15.167:0/4127547864 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee40d34f0 con 0x7f2ee40cfda0 2024-05-11T12:11:08.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.402+0000 7f2ee3e00640 1 -- 172.21.15.167:0/4127547864 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f2ed00016f0 con 0x7f2ee40cfda0 2024-05-11T12:11:08.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.402+0000 7f2ee9400640 1 --2- 172.21.15.167:0/4127547864 >> v2:172.21.15.204:3300/0 conn(0x7f2ee415f0d0 0x7f2ee40d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:11:08.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.402+0000 7f2ee3e00640 1 -- 172.21.15.167:0/4127547864 >> v1:172.21.15.167:6789/0 conn(0x7f2ee40cfda0 legacy=0x7f2ee40d3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:08.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.402+0000 7f2ee3e00640 1 --2- 172.21.15.167:0/4127547864 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2ee406cef0 0x7f2ecc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:08.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.402+0000 7f2ee3e00640 1 --2- 172.21.15.167:0/4127547864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ecc003f30 0x7f2ecc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:08.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.402+0000 7f2ee3e00640 1 --2- 172.21.15.167:0/4127547864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ecc004d40 0x7f2ecc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:08.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.402+0000 7f2ee3e00640 1 -- 172.21.15.167:0/4127547864 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee40d2f30 con 0x7f2ecc003f30 2024-05-11T12:11:08.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.402+0000 7f2ee3e00640 1 -- 172.21.15.167:0/4127547864 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee40d3210 con 0x7f2ecc004d40 2024-05-11T12:11:08.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.402+0000 7f2ee3e00640 1 -- 172.21.15.167:0/4127547864 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2ecc00ba10 con 0x7f2ee406cef0 2024-05-11T12:11:08.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.403+0000 7f2ee8a00640 1 --2- 172.21.15.167:0/4127547864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ecc003f30 0x7f2ecc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:08.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.403+0000 7f2ee9400640 1 --2- 172.21.15.167:0/4127547864 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2ee406cef0 0x7f2ecc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:08.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.403+0000 7f2ee9e00640 1 --2- 172.21.15.167:0/4127547864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ecc004d40 0x7f2ecc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:08.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.403+0000 7f2ee8a00640 1 -- 172.21.15.167:0/4127547864 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2ee406cef0 msgr2=0x7f2ecc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:08.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.403+0000 7f2ee8a00640 1 --2- 172.21.15.167:0/4127547864 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2ee406cef0 0x7f2ecc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:08.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.403+0000 7f2ee8a00640 1 -- 172.21.15.167:0/4127547864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ecc004d40 msgr2=0x7f2ecc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:08.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.403+0000 7f2ee8a00640 1 --2- 172.21.15.167:0/4127547864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ecc004d40 0x7f2ecc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:08.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.403+0000 7f2ee8a00640 1 -- 172.21.15.167:0/4127547864 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee40d34f0 con 0x7f2ecc003f30 2024-05-11T12:11:08.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.403+0000 7f2ee9e00640 1 --2- 172.21.15.167:0/4127547864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ecc004d40 0x7f2ecc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:08.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.404+0000 7f2ee8a00640 1 --2- 172.21.15.167:0/4127547864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ecc003f30 0x7f2ecc004800 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2ed000c2f0 tx=0x7f2ecc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83a329d88b9634a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:08.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.405+0000 7f2ee3e00640 1 -- 172.21.15.167:0/4127547864 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0036070 con 0x7f2ecc003f30 2024-05-11T12:11:08.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.405+0000 7f2ee3e00640 1 -- 172.21.15.167:0/4127547864 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ed002d070 con 0x7f2ecc003f30 2024-05-11T12:11:08.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.405+0000 7f2ee3e00640 1 -- 172.21.15.167:0/4127547864 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0007e00 con 0x7f2ecc003f30 2024-05-11T12:11:08.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.405+0000 7f2eea800640 1 -- 172.21.15.167:0/4127547864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ecc003f30 msgr2=0x7f2ecc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:08.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.405+0000 7f2eea800640 1 --2- 172.21.15.167:0/4127547864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ecc003f30 0x7f2ecc004800 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2ed000c2f0 tx=0x7f2ecc005830 comp rx=0 tx=0).stop 2024-05-11T12:11:08.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.405+0000 7f2ee9e00640 1 -- 172.21.15.167:0/4127547864 reap_dead start 2024-05-11T12:11:08.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.405+0000 7f2eea800640 1 -- 172.21.15.167:0/4127547864 shutdown_connections 2024-05-11T12:11:08.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.405+0000 7f2eea800640 1 -- 172.21.15.167:0/4127547864 >> 172.21.15.167:0/4127547864 conn(0x7f2ee40b6af0 msgr2=0x7f2ee40b6f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:08.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.405+0000 7f2eea800640 1 -- 172.21.15.167:0/4127547864 shutdown_connections 2024-05-11T12:11:08.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.405+0000 7f2eea800640 1 -- 172.21.15.167:0/4127547864 wait complete. 2024-05-11T12:11:08.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.407+0000 7f2eea800640 1 Processor -- start 2024-05-11T12:11:08.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.407+0000 7f2eea800640 1 -- start start 2024-05-11T12:11:08.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.408+0000 7f2eea800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2ecc003f30 0x7f2ee4152d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:08.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.408+0000 7f2eea800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ee40c8af0 0x7f2ee41f83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:08.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.408+0000 7f2eea800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ee41f88f0 0x7f2ee41f4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:08.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.408+0000 7f2eea800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee415f4b0 con 0x7f2ee40c8af0 2024-05-11T12:11:08.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.408+0000 7f2eea800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee40d13e0 con 0x7f2ee41f88f0 2024-05-11T12:11:08.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.408+0000 7f2eea800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2ee40d4400 con 0x7f2ecc003f30 2024-05-11T12:11:08.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.408+0000 7f2ee8a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ee40c8af0 0x7f2ee41f83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:08.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.409+0000 7f2ee8a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ee40c8af0 0x7f2ee41f83b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:42184/0 (socket says 172.21.15.167:42184) 2024-05-11T12:11:08.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.409+0000 7f2ee8a00640 1 -- 172.21.15.167:0/2105872962 learned_addr learned my addr 172.21.15.167:0/2105872962 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:08.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.409+0000 7f2ee9400640 1 --2- 172.21.15.167:0/2105872962 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2ecc003f30 0x7f2ee4152d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:08.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.409+0000 7f2ee9e00640 1 --2- 172.21.15.167:0/2105872962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ee41f88f0 0x7f2ee41f4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:08.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.409+0000 7f2ee8a00640 1 -- 172.21.15.167:0/2105872962 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2ecc003f30 msgr2=0x7f2ee4152d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:08.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.409+0000 7f2ee8a00640 1 --2- 172.21.15.167:0/2105872962 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2ecc003f30 0x7f2ee4152d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:08.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.409+0000 7f2ee8a00640 1 -- 172.21.15.167:0/2105872962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ee41f88f0 msgr2=0x7f2ee41f4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:08.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.409+0000 7f2ee8a00640 1 --2- 172.21.15.167:0/2105872962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ee41f88f0 0x7f2ee41f4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:08.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.409+0000 7f2ee8a00640 1 -- 172.21.15.167:0/2105872962 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee40c8ff0 con 0x7f2ee40c8af0 2024-05-11T12:11:08.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.409+0000 7f2ee9e00640 1 --2- 172.21.15.167:0/2105872962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ee41f88f0 0x7f2ee41f4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:08.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.409+0000 7f2ee8a00640 1 --2- 172.21.15.167:0/2105872962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ee40c8af0 0x7f2ee41f83b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2ed002aee0 tx=0x7f2ed0000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:08.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.410+0000 7f2ee1600640 1 -- 172.21.15.167:0/2105872962 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0041070 con 0x7f2ee40c8af0 2024-05-11T12:11:08.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.410+0000 7f2ee1600640 1 -- 172.21.15.167:0/2105872962 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ed0007270 con 0x7f2ee40c8af0 2024-05-11T12:11:08.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.410+0000 7f2ee1600640 1 -- 172.21.15.167:0/2105872962 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0036040 con 0x7f2ee40c8af0 2024-05-11T12:11:08.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.410+0000 7f2eea800640 1 -- 172.21.15.167:0/2105872962 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ee40c92d0 con 0x7f2ee40c8af0 2024-05-11T12:11:08.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.410+0000 7f2eea800640 1 -- 172.21.15.167:0/2105872962 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ee40c9890 con 0x7f2ee40c8af0 2024-05-11T12:11:08.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.411+0000 7f2eea800640 1 -- 172.21.15.167:0/2105872962 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ee41f8d00 con 0x7f2ee40c8af0 2024-05-11T12:11:08.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.412+0000 7f2ee1600640 1 -- 172.21.15.167:0/2105872962 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2ed0044070 con 0x7f2ee40c8af0 2024-05-11T12:11:08.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.413+0000 7f2ee1600640 1 --2- 172.21.15.167:0/2105872962 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2ec40762d0 0x7f2ec4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:08.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.413+0000 7f2ee1600640 1 -- 172.21.15.167:0/2105872962 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f2ed00b7f20 con 0x7f2ee40c8af0 2024-05-11T12:11:08.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.413+0000 7f2ee9400640 1 --2- 172.21.15.167:0/2105872962 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2ec40762d0 0x7f2ec4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:08.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.414+0000 7f2ee9400640 1 --2- 172.21.15.167:0/2105872962 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2ec40762d0 0x7f2ec4078790 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2ee40d07b0 tx=0x7f2ed8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:08.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.418+0000 7f2ee1600640 1 -- 172.21.15.167:0/2105872962 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2ee41f8d00 con 0x7f2ee40c8af0 2024-05-11T12:11:08.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.664+0000 7f2eea800640 1 -- 172.21.15.167:0/2105872962 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f2ee40694b0 con 0x7f2ee40c8af0 2024-05-11T12:11:08.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.899+0000 7f2ee1600640 1 -- 172.21.15.167:0/2105872962 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) ==== 118+0+0 (secure 0 0 0) 0x7f2ed004b030 con 0x7f2ee40c8af0 2024-05-11T12:11:08.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.903+0000 7f2eea800640 1 -- 172.21.15.167:0/2105872962 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2ec40762d0 msgr2=0x7f2ec4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:08.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.903+0000 7f2eea800640 1 --2- 172.21.15.167:0/2105872962 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2ec40762d0 0x7f2ec4078790 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2ee40d07b0 tx=0x7f2ed8008040 comp rx=0 tx=0).stop 2024-05-11T12:11:08.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.903+0000 7f2eea800640 1 -- 172.21.15.167:0/2105872962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ee40c8af0 msgr2=0x7f2ee41f83b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:08.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.903+0000 7f2eea800640 1 --2- 172.21.15.167:0/2105872962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ee40c8af0 0x7f2ee41f83b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2ed002aee0 tx=0x7f2ed0000fe0 comp rx=0 tx=0).stop 2024-05-11T12:11:08.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.903+0000 7f2eea800640 1 -- 172.21.15.167:0/2105872962 shutdown_connections 2024-05-11T12:11:08.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.903+0000 7f2eea800640 1 --2- 172.21.15.167:0/2105872962 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2ec40762d0 0x7f2ec4078790 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:08.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.903+0000 7f2eea800640 1 --2- 172.21.15.167:0/2105872962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ee41f88f0 0x7f2ee41f4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:08.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.903+0000 7f2eea800640 1 --2- 172.21.15.167:0/2105872962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ee40c8af0 0x7f2ee41f83b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:08.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.903+0000 7f2eea800640 1 --2- 172.21.15.167:0/2105872962 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2ecc003f30 0x7f2ee4152d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:08.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.903+0000 7f2eea800640 1 -- 172.21.15.167:0/2105872962 >> 172.21.15.167:0/2105872962 conn(0x7f2ee40b6af0 msgr2=0x7f2ee406dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:08.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.903+0000 7f2eea800640 1 -- 172.21.15.167:0/2105872962 shutdown_connections 2024-05-11T12:11:08.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:08.903+0000 7f2eea800640 1 -- 172.21.15.167:0/2105872962 wait complete. 2024-05-11T12:11:08.925 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-05-11T12:11:08.925 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-11T12:11:09.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.130+0000 7f7883e00640 1 Processor -- start 2024-05-11T12:11:09.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.130+0000 7f7883e00640 1 -- start start 2024-05-11T12:11:09.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.132+0000 7f7883e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f787c15ac80 0x7f787c074260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:09.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.132+0000 7f7883e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f787c161380 0x7f787c072950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:09.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.132+0000 7f7883e00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f787c0b65e0 con 0x7f787c0b3d50 2024-05-11T12:11:09.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.132+0000 7f7883e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f787c06e3c0 con 0x7f787c161380 2024-05-11T12:11:09.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.132+0000 7f7883e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f787c15b060 con 0x7f787c15ac80 2024-05-11T12:11:09.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.133+0000 7f7880a00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f787c0b3d50 0x7f787c160c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49330/0 (socket says 172.21.15.167:49330) 2024-05-11T12:11:09.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.133+0000 7f7880a00640 1 -- 172.21.15.167:0/1734030648 learned_addr learned my addr 172.21.15.167:0/1734030648 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:09.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.133+0000 7f7881400640 1 --2- 172.21.15.167:0/1734030648 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f787c15ac80 0x7f787c074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:09.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.133+0000 7f7881e00640 1 --2- 172.21.15.167:0/1734030648 >> v2:172.21.15.204:3300/0 conn(0x7f787c161380 0x7f787c072950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:09.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.133+0000 7f787be00640 1 -- 172.21.15.167:0/1734030648 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1337434979 0 0) 0x7f787c0b65e0 con 0x7f787c0b3d50 2024-05-11T12:11:09.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.133+0000 7f787be00640 1 -- 172.21.15.167:0/1734030648 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7864003520 con 0x7f787c0b3d50 2024-05-11T12:11:09.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.133+0000 7f787be00640 1 -- 172.21.15.167:0/1734030648 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2995568414 0 0) 0x7f7864003520 con 0x7f787c0b3d50 2024-05-11T12:11:09.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.133+0000 7f787be00640 1 -- 172.21.15.167:0/1734030648 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f787c15ac80 msgr2=0x7f787c074260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:09.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.133+0000 7f787be00640 1 --2- 172.21.15.167:0/1734030648 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f787c15ac80 0x7f787c074260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:09.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.133+0000 7f787be00640 1 -- 172.21.15.167:0/1734030648 >> v2:172.21.15.204:3300/0 conn(0x7f787c161380 msgr2=0x7f787c072950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:09.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.133+0000 7f787be00640 1 --2- 172.21.15.167:0/1734030648 >> v2:172.21.15.204:3300/0 conn(0x7f787c161380 0x7f787c072950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:09.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.133+0000 7f787be00640 1 -- 172.21.15.167:0/1734030648 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f787c0b68c0 con 0x7f787c0b3d50 2024-05-11T12:11:09.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.133+0000 7f787be00640 1 -- 172.21.15.167:0/1734030648 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f786c0016f0 con 0x7f787c0b3d50 2024-05-11T12:11:09.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.134+0000 7f787be00640 1 -- 172.21.15.167:0/1734030648 >> v1:172.21.15.167:6789/0 conn(0x7f787c0b3d50 legacy=0x7f787c160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:09.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.134+0000 7f787be00640 1 --2- 172.21.15.167:0/1734030648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7864003a00 0x7f7864003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:09.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.134+0000 7f787be00640 1 --2- 172.21.15.167:0/1734030648 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f787c15ac80 0x7f7864004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:09.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.134+0000 7f787be00640 1 --2- 172.21.15.167:0/1734030648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7864004d40 0x7f78640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:09.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.134+0000 7f787be00640 1 -- 172.21.15.167:0/1734030648 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f787c15b060 con 0x7f7864003a00 2024-05-11T12:11:09.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.134+0000 7f787be00640 1 -- 172.21.15.167:0/1734030648 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f786400b9c0 con 0x7f7864004d40 2024-05-11T12:11:09.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.134+0000 7f787be00640 1 -- 172.21.15.167:0/1734030648 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f786400bb70 con 0x7f787c15ac80 2024-05-11T12:11:09.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.134+0000 7f7881400640 1 --2- 172.21.15.167:0/1734030648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7864003a00 0x7f7864003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:09.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.134+0000 7f7880a00640 1 --2- 172.21.15.167:0/1734030648 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f787c15ac80 0x7f7864004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:09.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.134+0000 7f7881e00640 1 --2- 172.21.15.167:0/1734030648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7864004d40 0x7f78640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:09.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.134+0000 7f7881400640 1 -- 172.21.15.167:0/1734030648 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f787c15ac80 msgr2=0x7f7864004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:09.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.134+0000 7f7881400640 1 --2- 172.21.15.167:0/1734030648 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f787c15ac80 0x7f7864004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:09.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.134+0000 7f7881400640 1 -- 172.21.15.167:0/1734030648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7864004d40 msgr2=0x7f78640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:09.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.134+0000 7f7881400640 1 --2- 172.21.15.167:0/1734030648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7864004d40 0x7f78640051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:09.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.134+0000 7f7881400640 1 -- 172.21.15.167:0/1734030648 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7864003520 con 0x7f7864003a00 2024-05-11T12:11:09.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.134+0000 7f7881e00640 1 --2- 172.21.15.167:0/1734030648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7864004d40 0x7f78640051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:09.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.135+0000 7f7881400640 1 --2- 172.21.15.167:0/1734030648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7864003a00 0x7f7864003e20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7870009ac0 tx=0x7f7864005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=681e9f62446aeb5f server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:09.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.136+0000 7f787be00640 1 -- 172.21.15.167:0/1734030648 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7870036070 con 0x7f7864003a00 2024-05-11T12:11:09.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.136+0000 7f787be00640 1 -- 172.21.15.167:0/1734030648 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7870031400 con 0x7f7864003a00 2024-05-11T12:11:09.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.136+0000 7f787be00640 1 -- 172.21.15.167:0/1734030648 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7870009690 con 0x7f7864003a00 2024-05-11T12:11:09.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.136+0000 7f7883e00640 1 -- 172.21.15.167:0/1734030648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7864003a00 msgr2=0x7f7864003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:09.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.136+0000 7f7883e00640 1 --2- 172.21.15.167:0/1734030648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7864003a00 0x7f7864003e20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7870009ac0 tx=0x7f7864005830 comp rx=0 tx=0).stop 2024-05-11T12:11:09.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.136+0000 7f7881e00640 1 -- 172.21.15.167:0/1734030648 reap_dead start 2024-05-11T12:11:09.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.136+0000 7f7883e00640 1 -- 172.21.15.167:0/1734030648 shutdown_connections 2024-05-11T12:11:09.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.136+0000 7f7883e00640 1 -- 172.21.15.167:0/1734030648 >> 172.21.15.167:0/1734030648 conn(0x7f787c06c450 msgr2=0x7f787c06c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:09.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.136+0000 7f7883e00640 1 -- 172.21.15.167:0/1734030648 shutdown_connections 2024-05-11T12:11:09.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.136+0000 7f7883e00640 1 -- 172.21.15.167:0/1734030648 wait complete. 2024-05-11T12:11:09.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.138+0000 7f7883e00640 1 Processor -- start 2024-05-11T12:11:09.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.138+0000 7f7883e00640 1 -- start start 2024-05-11T12:11:09.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.139+0000 7f7883e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7864003a00 0x7f787c1455b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:09.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.139+0000 7f7883e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f787c0cea80 0x7f787c15c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:09.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.139+0000 7f7883e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f787c0cef00 0x7f787c15cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:09.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.139+0000 7f7883e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f787c0b44f0 con 0x7f787c0cef00 2024-05-11T12:11:09.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.139+0000 7f7883e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f787c0b41b0 con 0x7f787c0cea80 2024-05-11T12:11:09.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.139+0000 7f7883e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f787c074260 con 0x7f7864003a00 2024-05-11T12:11:09.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.139+0000 7f7881e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f787c0cef00 0x7f787c15cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:09.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.140+0000 7f7881e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f787c0cef00 0x7f787c15cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41366/0 (socket says 172.21.15.167:41366) 2024-05-11T12:11:09.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.140+0000 7f7881e00640 1 -- 172.21.15.167:0/2456769441 learned_addr learned my addr 172.21.15.167:0/2456769441 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:09.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.140+0000 7f7881400640 1 --2- 172.21.15.167:0/2456769441 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7864003a00 0x7f787c1455b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:09.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.140+0000 7f7881e00640 1 -- 172.21.15.167:0/2456769441 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7864003a00 msgr2=0x7f787c1455b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:09.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.140+0000 7f7881e00640 1 --2- 172.21.15.167:0/2456769441 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7864003a00 0x7f787c1455b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:09.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.140+0000 7f7881e00640 1 -- 172.21.15.167:0/2456769441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f787c0cea80 msgr2=0x7f787c15c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:09.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.140+0000 7f7881e00640 1 --2- 172.21.15.167:0/2456769441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f787c0cea80 0x7f787c15c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:09.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.140+0000 7f7881e00640 1 -- 172.21.15.167:0/2456769441 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f787c1f5080 con 0x7f787c0cef00 2024-05-11T12:11:09.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.140+0000 7f7881e00640 1 --2- 172.21.15.167:0/2456769441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f787c0cef00 0x7f787c15cb60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f787c161d50 tx=0x7f787400f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:09.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.140+0000 7f7879600640 1 -- 172.21.15.167:0/2456769441 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7874016620 con 0x7f787c0cef00 2024-05-11T12:11:09.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.140+0000 7f7879600640 1 -- 172.21.15.167:0/2456769441 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f787401b070 con 0x7f787c0cef00 2024-05-11T12:11:09.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.140+0000 7f7879600640 1 -- 172.21.15.167:0/2456769441 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78740169e0 con 0x7f787c0cef00 2024-05-11T12:11:09.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.141+0000 7f7883e00640 1 -- 172.21.15.167:0/2456769441 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f787c1f5360 con 0x7f787c0cef00 2024-05-11T12:11:09.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.141+0000 7f7883e00640 1 -- 172.21.15.167:0/2456769441 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f787c1f5920 con 0x7f787c0cef00 2024-05-11T12:11:09.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.142+0000 7f7883e00640 1 -- 172.21.15.167:0/2456769441 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f787c069520 con 0x7f787c0cef00 2024-05-11T12:11:09.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.143+0000 7f7879600640 1 -- 172.21.15.167:0/2456769441 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7874005d10 con 0x7f787c0cef00 2024-05-11T12:11:09.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.144+0000 7f7879600640 1 --2- 172.21.15.167:0/2456769441 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f78600762d0 0x7f7860078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:09.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.144+0000 7f7879600640 1 -- 172.21.15.167:0/2456769441 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f78740982e0 con 0x7f787c0cef00 2024-05-11T12:11:09.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.144+0000 7f7881400640 1 --2- 172.21.15.167:0/2456769441 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f78600762d0 0x7f7860078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:09.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.145+0000 7f7881400640 1 --2- 172.21.15.167:0/2456769441 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f78600762d0 0x7f7860078790 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7870033000 tx=0x7f7870008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:09.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.148+0000 7f7879600640 1 -- 172.21.15.167:0/2456769441 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f78740618e0 con 0x7f787c0cef00 2024-05-11T12:11:09.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.403+0000 7f7883e00640 1 -- 172.21.15.167:0/2456769441 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f787c070120 con 0x7f787c0cef00 2024-05-11T12:11:09.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.919+0000 7f7879600640 1 -- 172.21.15.167:0/2456769441 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) ==== 122+0+0 (secure 0 0 0) 0x7f78740611e0 con 0x7f787c0cef00 2024-05-11T12:11:09.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.923+0000 7f7883e00640 1 -- 172.21.15.167:0/2456769441 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f78600762d0 msgr2=0x7f7860078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:09.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.923+0000 7f7883e00640 1 --2- 172.21.15.167:0/2456769441 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f78600762d0 0x7f7860078790 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7870033000 tx=0x7f7870008030 comp rx=0 tx=0).stop 2024-05-11T12:11:09.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.923+0000 7f7883e00640 1 -- 172.21.15.167:0/2456769441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f787c0cef00 msgr2=0x7f787c15cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:09.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.923+0000 7f7883e00640 1 --2- 172.21.15.167:0/2456769441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f787c0cef00 0x7f787c15cb60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f787c161d50 tx=0x7f787400f520 comp rx=0 tx=0).stop 2024-05-11T12:11:09.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.923+0000 7f7883e00640 1 -- 172.21.15.167:0/2456769441 shutdown_connections 2024-05-11T12:11:09.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.923+0000 7f7883e00640 1 --2- 172.21.15.167:0/2456769441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f787c0cef00 0x7f787c15cb60 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:09.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.923+0000 7f7883e00640 1 --2- 172.21.15.167:0/2456769441 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f78600762d0 0x7f7860078790 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:09.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.923+0000 7f7883e00640 1 --2- 172.21.15.167:0/2456769441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f787c0cea80 0x7f787c15c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:09.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.923+0000 7f7883e00640 1 --2- 172.21.15.167:0/2456769441 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7864003a00 0x7f787c1455b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:09.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.923+0000 7f7883e00640 1 -- 172.21.15.167:0/2456769441 >> 172.21.15.167:0/2456769441 conn(0x7f787c06c450 msgr2=0x7f787c06ebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:09.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.924+0000 7f7883e00640 1 -- 172.21.15.167:0/2456769441 shutdown_connections 2024-05-11T12:11:09.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:09.924+0000 7f7883e00640 1 -- 172.21.15.167:0/2456769441 wait complete. 2024-05-11T12:11:09.944 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-11T12:11:09.944 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:11:10.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.146+0000 7fa797e00640 1 Processor -- start 2024-05-11T12:11:10.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.146+0000 7fa797e00640 1 -- start start 2024-05-11T12:11:10.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.148+0000 7fa797e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa7980d04a0 0x7fa7981601f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:10.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.148+0000 7fa797e00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7980b4560 con 0x7fa7980cf970 2024-05-11T12:11:10.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.148+0000 7fa797e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7980b4800 con 0x7fa7980afcd0 2024-05-11T12:11:10.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.148+0000 7fa797e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fa7980b5130 con 0x7fa7980d04a0 2024-05-11T12:11:10.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.148+0000 7fa78fe00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fa7980cf970 0x7fa7980721d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49344/0 (socket says 172.21.15.167:49344) 2024-05-11T12:11:10.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.148+0000 7fa78fe00640 1 -- 172.21.15.167:0/2439756025 learned_addr learned my addr 172.21.15.167:0/2439756025 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:10.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.148+0000 7fa797400640 1 --2- 172.21.15.167:0/2439756025 >> v2:172.21.15.204:3300/0 conn(0x7fa7980d04a0 0x7fa7981601f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:10.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.148+0000 7fa796000640 1 -- 172.21.15.167:0/2439756025 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3172246615 0 0) 0x7fa7980b4560 con 0x7fa7980cf970 2024-05-11T12:11:10.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.148+0000 7fa796000640 1 -- 172.21.15.167:0/2439756025 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa77c003520 con 0x7fa7980cf970 2024-05-11T12:11:10.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.149+0000 7fa796000640 1 -- 172.21.15.167:0/2439756025 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1657313715 0 0) 0x7fa77c003520 con 0x7fa7980cf970 2024-05-11T12:11:10.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.149+0000 7fa796000640 1 -- 172.21.15.167:0/2439756025 >> v2:172.21.15.204:3300/0 conn(0x7fa7980d04a0 msgr2=0x7fa7981601f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:10.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.149+0000 7fa796000640 1 --2- 172.21.15.167:0/2439756025 >> v2:172.21.15.204:3300/0 conn(0x7fa7980d04a0 0x7fa7981601f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:10.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.149+0000 7fa796000640 1 -- 172.21.15.167:0/2439756025 >> v1:172.21.15.204:6789/0 conn(0x7fa7980afcd0 legacy=0x7fa7980cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:10.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.149+0000 7fa796000640 1 -- 172.21.15.167:0/2439756025 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7980b4ae0 con 0x7fa7980cf970 2024-05-11T12:11:10.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.149+0000 7fa796000640 1 -- 172.21.15.167:0/2439756025 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7fa780001710 con 0x7fa7980cf970 2024-05-11T12:11:10.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.149+0000 7fa796000640 1 -- 172.21.15.167:0/2439756025 >> v1:172.21.15.167:6789/0 conn(0x7fa7980cf970 legacy=0x7fa7980721d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:10.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.149+0000 7fa796000640 1 --2- 172.21.15.167:0/2439756025 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa77c003a50 0x7fa77c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:10.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.149+0000 7fa796000640 1 --2- 172.21.15.167:0/2439756025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa77c004800 0x7fa77c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:10.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.149+0000 7fa796000640 1 --2- 172.21.15.167:0/2439756025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa77c0051c0 0x7fa77c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:10.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.149+0000 7fa796000640 1 -- 172.21.15.167:0/2439756025 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa7980b5130 con 0x7fa77c004800 2024-05-11T12:11:10.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.149+0000 7fa796000640 1 -- 172.21.15.167:0/2439756025 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa77c00c160 con 0x7fa77c0051c0 2024-05-11T12:11:10.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.149+0000 7fa796000640 1 -- 172.21.15.167:0/2439756025 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa77c00c360 con 0x7fa77c003a50 2024-05-11T12:11:10.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.149+0000 7fa78fe00640 1 --2- 172.21.15.167:0/2439756025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa77c004800 0x7fa77c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:10.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.150+0000 7fa797400640 1 --2- 172.21.15.167:0/2439756025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa77c0051c0 0x7fa77c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:10.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.150+0000 7fa796a00640 1 --2- 172.21.15.167:0/2439756025 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa77c003a50 0x7fa77c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:10.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.150+0000 7fa78fe00640 1 -- 172.21.15.167:0/2439756025 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa77c003a50 msgr2=0x7fa77c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:10.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.150+0000 7fa78fe00640 1 --2- 172.21.15.167:0/2439756025 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa77c003a50 0x7fa77c003e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:10.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.150+0000 7fa78fe00640 1 -- 172.21.15.167:0/2439756025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa77c0051c0 msgr2=0x7fa77c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:10.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.150+0000 7fa78fe00640 1 --2- 172.21.15.167:0/2439756025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa77c0051c0 0x7fa77c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:10.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.150+0000 7fa78fe00640 1 -- 172.21.15.167:0/2439756025 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa77c003520 con 0x7fa77c004800 2024-05-11T12:11:10.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.150+0000 7fa797400640 1 -- 172.21.15.167:0/2439756025 reap_dead start 2024-05-11T12:11:10.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.151+0000 7fa78fe00640 1 --2- 172.21.15.167:0/2439756025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa77c004800 0x7fa77c004c80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa78000c2f0 tx=0x7fa77c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436a0b7c40891197 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:10.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.151+0000 7fa796000640 1 -- 172.21.15.167:0/2439756025 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa780007d90 con 0x7fa77c004800 2024-05-11T12:11:10.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.151+0000 7fa796000640 1 -- 172.21.15.167:0/2439756025 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa78002e070 con 0x7fa77c004800 2024-05-11T12:11:10.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.151+0000 7fa796000640 1 -- 172.21.15.167:0/2439756025 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa780005d10 con 0x7fa77c004800 2024-05-11T12:11:10.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.152+0000 7fa797e00640 1 -- 172.21.15.167:0/2439756025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa77c004800 msgr2=0x7fa77c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:10.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.152+0000 7fa797e00640 1 --2- 172.21.15.167:0/2439756025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa77c004800 0x7fa77c004c80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa78000c2f0 tx=0x7fa77c005830 comp rx=0 tx=0).stop 2024-05-11T12:11:10.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.152+0000 7fa797e00640 1 -- 172.21.15.167:0/2439756025 shutdown_connections 2024-05-11T12:11:10.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.152+0000 7fa797e00640 1 --2- 172.21.15.167:0/2439756025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa77c004800 0x7fa77c004c80 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:10.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.152+0000 7fa797e00640 1 -- 172.21.15.167:0/2439756025 >> 172.21.15.167:0/2439756025 conn(0x7fa7980b3460 msgr2=0x7fa7980d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:10.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.152+0000 7fa797e00640 1 -- 172.21.15.167:0/2439756025 shutdown_connections 2024-05-11T12:11:10.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.152+0000 7fa797e00640 1 -- 172.21.15.167:0/2439756025 wait complete. 2024-05-11T12:11:10.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.154+0000 7fa797e00640 1 Processor -- start 2024-05-11T12:11:10.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.154+0000 7fa797e00640 1 -- start start 2024-05-11T12:11:10.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.155+0000 7fa797e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa77c004800 0x7fa79814d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:10.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.155+0000 7fa797e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa79815bbd0 0x7fa798159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:10.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.155+0000 7fa797e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa79815a500 0x7fa79815c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:10.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.155+0000 7fa797e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa7980cff40 con 0x7fa79815a500 2024-05-11T12:11:10.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.155+0000 7fa797e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa7980b00b0 con 0x7fa79815bbd0 2024-05-11T12:11:10.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.155+0000 7fa797e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa7980d00c0 con 0x7fa77c004800 2024-05-11T12:11:10.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.155+0000 7fa797400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa79815a500 0x7fa79815c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:10.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.155+0000 7fa797400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa79815a500 0x7fa79815c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41386/0 (socket says 172.21.15.167:41386) 2024-05-11T12:11:10.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.155+0000 7fa797400640 1 -- 172.21.15.167:0/821754071 learned_addr learned my addr 172.21.15.167:0/821754071 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:10.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.155+0000 7fa796a00640 1 --2- 172.21.15.167:0/821754071 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa77c004800 0x7fa79814d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:10.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.155+0000 7fa78fe00640 1 --2- 172.21.15.167:0/821754071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa79815bbd0 0x7fa798159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:10.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.156+0000 7fa797400640 1 -- 172.21.15.167:0/821754071 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa77c004800 msgr2=0x7fa79814d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:10.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.156+0000 7fa797400640 1 --2- 172.21.15.167:0/821754071 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa77c004800 0x7fa79814d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:10.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.156+0000 7fa797400640 1 -- 172.21.15.167:0/821754071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa79815bbd0 msgr2=0x7fa798159fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:10.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.156+0000 7fa797400640 1 --2- 172.21.15.167:0/821754071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa79815bbd0 0x7fa798159fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:10.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.156+0000 7fa797400640 1 -- 172.21.15.167:0/821754071 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa79815d3b0 con 0x7fa79815a500 2024-05-11T12:11:10.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.156+0000 7fa797400640 1 --2- 172.21.15.167:0/821754071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa79815a500 0x7fa79815c5c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa7980b0580 tx=0x7fa7900115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:10.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.156+0000 7fa78ea00640 1 -- 172.21.15.167:0/821754071 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa790011de0 con 0x7fa79815a500 2024-05-11T12:11:10.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.156+0000 7fa78ea00640 1 -- 172.21.15.167:0/821754071 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa79001b070 con 0x7fa79815a500 2024-05-11T12:11:10.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.156+0000 7fa797e00640 1 -- 172.21.15.167:0/821754071 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa79815ab30 con 0x7fa79815a500 2024-05-11T12:11:10.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.156+0000 7fa78ea00640 1 -- 172.21.15.167:0/821754071 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa790003990 con 0x7fa79815a500 2024-05-11T12:11:10.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.156+0000 7fa797e00640 1 -- 172.21.15.167:0/821754071 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa79815b0f0 con 0x7fa79815a500 2024-05-11T12:11:10.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.157+0000 7fa797e00640 1 -- 172.21.15.167:0/821754071 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7980694b0 con 0x7fa79815a500 2024-05-11T12:11:10.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.159+0000 7fa78ea00640 1 -- 172.21.15.167:0/821754071 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa790003b30 con 0x7fa79815a500 2024-05-11T12:11:10.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.160+0000 7fa78ea00640 1 --2- 172.21.15.167:0/821754071 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa7740762d0 0x7fa774078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:10.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.160+0000 7fa78ea00640 1 -- 172.21.15.167:0/821754071 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fa7900982e0 con 0x7fa79815a500 2024-05-11T12:11:10.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.160+0000 7fa796a00640 1 --2- 172.21.15.167:0/821754071 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa7740762d0 0x7fa774078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:10.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.161+0000 7fa796a00640 1 --2- 172.21.15.167:0/821754071 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa7740762d0 0x7fa774078790 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa79815ee50 tx=0x7fa788009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:10.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.165+0000 7fa78ea00640 1 -- 172.21.15.167:0/821754071 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa790003e20 con 0x7fa79815a500 2024-05-11T12:11:10.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.421+0000 7fa797e00640 1 -- 172.21.15.167:0/821754071 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa7981600a0 con 0x7fa79815a500 2024-05-11T12:11:10.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.422+0000 7fa78ea00640 1 -- 172.21.15.167:0/821754071 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5189 (secure 0 0 0) 0x7fa790061860 con 0x7fa79815a500 2024-05-11T12:11:10.423 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:11:10.423 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3127277486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3127277486},{"type":"v1","addr":"172.21.15.167:6835","nonce":3127277486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2547133600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2547133600},{"type":"v1","addr":"172.21.15.167:6837","nonce":2547133600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3200960100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3200960100},{"type":"v1","addr":"172.21.15.204:6834","nonce":3200960100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:10:45.225745+0000","modified":"2024-05-11T12:11:09.405446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/1889842375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":1889842375},{"type":"v1","addr":"172.21.15.204:6835","nonce":1889842375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-05-11T12:11:10.423 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 9 2024-05-11T12:11:10.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.425+0000 7fa797e00640 1 -- 172.21.15.167:0/821754071 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa7740762d0 msgr2=0x7fa774078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:10.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.425+0000 7fa797e00640 1 --2- 172.21.15.167:0/821754071 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa7740762d0 0x7fa774078790 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa79815ee50 tx=0x7fa788009040 comp rx=0 tx=0).stop 2024-05-11T12:11:10.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.425+0000 7fa797e00640 1 -- 172.21.15.167:0/821754071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa79815a500 msgr2=0x7fa79815c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:10.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.426+0000 7fa797e00640 1 --2- 172.21.15.167:0/821754071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa79815a500 0x7fa79815c5c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa7980b0580 tx=0x7fa7900115a0 comp rx=0 tx=0).stop 2024-05-11T12:11:10.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.426+0000 7fa797e00640 1 -- 172.21.15.167:0/821754071 shutdown_connections 2024-05-11T12:11:10.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.426+0000 7fa797e00640 1 --2- 172.21.15.167:0/821754071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa79815a500 0x7fa79815c5c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:10.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.426+0000 7fa797e00640 1 --2- 172.21.15.167:0/821754071 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa7740762d0 0x7fa774078790 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:10.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.426+0000 7fa797e00640 1 --2- 172.21.15.167:0/821754071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa79815bbd0 0x7fa798159fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:10.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.426+0000 7fa797e00640 1 --2- 172.21.15.167:0/821754071 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa77c004800 0x7fa79814d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:10.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.426+0000 7fa797e00640 1 -- 172.21.15.167:0/821754071 >> 172.21.15.167:0/821754071 conn(0x7fa7980b3460 msgr2=0x7fa7980b1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:10.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.426+0000 7fa797e00640 1 -- 172.21.15.167:0/821754071 shutdown_connections 2024-05-11T12:11:10.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.426+0000 7fa797e00640 1 -- 172.21.15.167:0/821754071 wait complete. 2024-05-11T12:11:10.447 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:11:10.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.651+0000 7f3417200640 1 Processor -- start 2024-05-11T12:11:10.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.651+0000 7f3417200640 1 -- start start 2024-05-11T12:11:10.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.652+0000 7f3417200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34180b3a80 0x7f341806d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:10.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.652+0000 7f3417200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f34180b5220 0x7f34180b56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:10.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.652+0000 7f3417200640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3418071f30 con 0x7f34181458f0 2024-05-11T12:11:10.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.652+0000 7f3417200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f34180b5bf0 con 0x7f34180b5220 2024-05-11T12:11:10.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.652+0000 7f3417200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f341806db80 con 0x7f34180b3a80 2024-05-11T12:11:10.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.653+0000 7f3415e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f34181458f0 0x7f34180b4970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49356/0 (socket says 172.21.15.167:49356) 2024-05-11T12:11:10.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.653+0000 7f3415e00640 1 -- 172.21.15.167:0/2561724615 learned_addr learned my addr 172.21.15.167:0/2561724615 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:10.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.653+0000 7f340d400640 1 --2- 172.21.15.167:0/2561724615 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34180b3a80 0x7f341806d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:10.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.653+0000 7f3416800640 1 --2- 172.21.15.167:0/2561724615 >> v2:172.21.15.204:3300/0 conn(0x7f34180b5220 0x7f34180b56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:10.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.653+0000 7f3415400640 1 -- 172.21.15.167:0/2561724615 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2869565866 0 0) 0x7f3418071f30 con 0x7f34181458f0 2024-05-11T12:11:10.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.653+0000 7f3415400640 1 -- 172.21.15.167:0/2561724615 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f33fc003520 con 0x7f34181458f0 2024-05-11T12:11:10.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.654+0000 7f3415400640 1 -- 172.21.15.167:0/2561724615 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3855431302 0 0) 0x7f33fc003520 con 0x7f34181458f0 2024-05-11T12:11:10.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.654+0000 7f3415400640 1 -- 172.21.15.167:0/2561724615 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34180b3a80 msgr2=0x7f341806d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:10.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.654+0000 7f3415400640 1 --2- 172.21.15.167:0/2561724615 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34180b3a80 0x7f341806d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:10.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.654+0000 7f3415400640 1 -- 172.21.15.167:0/2561724615 >> v2:172.21.15.204:3300/0 conn(0x7f34180b5220 msgr2=0x7f34180b56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:10.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.654+0000 7f3415400640 1 --2- 172.21.15.167:0/2561724615 >> v2:172.21.15.204:3300/0 conn(0x7f34180b5220 0x7f34180b56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:10.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.654+0000 7f3415400640 1 -- 172.21.15.167:0/2561724615 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3418072210 con 0x7f34181458f0 2024-05-11T12:11:10.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.654+0000 7f3415400640 1 -- 172.21.15.167:0/2561724615 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f34080016f0 con 0x7f34181458f0 2024-05-11T12:11:10.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.654+0000 7f3415400640 1 -- 172.21.15.167:0/2561724615 >> v1:172.21.15.167:6789/0 conn(0x7f34181458f0 legacy=0x7f34180b4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:10.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.654+0000 7f3415400640 1 --2- 172.21.15.167:0/2561724615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33fc003a00 0x7f33fc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:10.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.654+0000 7f3415400640 1 --2- 172.21.15.167:0/2561724615 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34180b3a80 0x7f33fc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:10.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.654+0000 7f3415400640 1 --2- 172.21.15.167:0/2561724615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33fc004d40 0x7f33fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:10.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.654+0000 7f3415400640 1 -- 172.21.15.167:0/2561724615 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f34180b5bf0 con 0x7f33fc003a00 2024-05-11T12:11:10.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.654+0000 7f3415400640 1 -- 172.21.15.167:0/2561724615 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc00b9c0 con 0x7f33fc004d40 2024-05-11T12:11:10.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.654+0000 7f3415400640 1 -- 172.21.15.167:0/2561724615 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f33fc00bb70 con 0x7f34180b3a80 2024-05-11T12:11:10.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.654+0000 7f3415e00640 1 --2- 172.21.15.167:0/2561724615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33fc003a00 0x7f33fc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:10.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.654+0000 7f340d400640 1 --2- 172.21.15.167:0/2561724615 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34180b3a80 0x7f33fc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:10.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.655+0000 7f3415e00640 1 -- 172.21.15.167:0/2561724615 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34180b3a80 msgr2=0x7f33fc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:10.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.655+0000 7f3415e00640 1 --2- 172.21.15.167:0/2561724615 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34180b3a80 0x7f33fc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:10.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.655+0000 7f3416800640 1 --2- 172.21.15.167:0/2561724615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33fc004d40 0x7f33fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:10.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.655+0000 7f3415e00640 1 -- 172.21.15.167:0/2561724615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33fc004d40 msgr2=0x7f33fc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:10.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.655+0000 7f3415e00640 1 --2- 172.21.15.167:0/2561724615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33fc004d40 0x7f33fc0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:10.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.655+0000 7f3415e00640 1 -- 172.21.15.167:0/2561724615 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33fc003520 con 0x7f33fc003a00 2024-05-11T12:11:10.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.655+0000 7f340d400640 1 --2- 172.21.15.167:0/2561724615 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34180b3a80 0x7f33fc004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:10.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.656+0000 7f3415e00640 1 --2- 172.21.15.167:0/2561724615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33fc003a00 0x7f33fc003e20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f340800c290 tx=0x7f33fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53b8c5b3c0bd9fb server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:10.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.656+0000 7f3415400640 1 -- 172.21.15.167:0/2561724615 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3408007d50 con 0x7f33fc003a00 2024-05-11T12:11:10.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.656+0000 7f3415400640 1 -- 172.21.15.167:0/2561724615 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3408005af0 con 0x7f33fc003a00 2024-05-11T12:11:10.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.656+0000 7f3415400640 1 -- 172.21.15.167:0/2561724615 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3408007070 con 0x7f33fc003a00 2024-05-11T12:11:10.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.656+0000 7f3417200640 1 -- 172.21.15.167:0/2561724615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33fc003a00 msgr2=0x7f33fc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:10.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.656+0000 7f3417200640 1 --2- 172.21.15.167:0/2561724615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33fc003a00 0x7f33fc003e20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f340800c290 tx=0x7f33fc005830 comp rx=0 tx=0).stop 2024-05-11T12:11:10.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.656+0000 7f3416800640 1 -- 172.21.15.167:0/2561724615 reap_dead start 2024-05-11T12:11:10.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.657+0000 7f3417200640 1 -- 172.21.15.167:0/2561724615 shutdown_connections 2024-05-11T12:11:10.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.657+0000 7f3417200640 1 -- 172.21.15.167:0/2561724615 >> 172.21.15.167:0/2561724615 conn(0x7f34180b3460 msgr2=0x7f34180d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:10.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.657+0000 7f3417200640 1 -- 172.21.15.167:0/2561724615 shutdown_connections 2024-05-11T12:11:10.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.657+0000 7f3417200640 1 -- 172.21.15.167:0/2561724615 wait complete. 2024-05-11T12:11:10.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.658+0000 7f3417200640 1 Processor -- start 2024-05-11T12:11:10.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.659+0000 7f3417200640 1 -- start start 2024-05-11T12:11:10.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.659+0000 7f3417200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33fc003a00 0x7f34181383f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:10.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.660+0000 7f3417200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3418141940 0x7f3418144570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:10.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.660+0000 7f3417200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f341815fa30 0x7f341815fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:10.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.660+0000 7f3417200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3418072ab0 con 0x7f341815fa30 2024-05-11T12:11:10.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.660+0000 7f3417200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f34180727d0 con 0x7f3418141940 2024-05-11T12:11:10.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.660+0000 7f3417200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f341806d640 con 0x7f33fc003a00 2024-05-11T12:11:10.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.660+0000 7f3416800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f341815fa30 0x7f341815fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:10.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.660+0000 7f3416800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f341815fa30 0x7f341815fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41406/0 (socket says 172.21.15.167:41406) 2024-05-11T12:11:10.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.660+0000 7f3416800640 1 -- 172.21.15.167:0/3823302379 learned_addr learned my addr 172.21.15.167:0/3823302379 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:10.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.660+0000 7f3415e00640 1 --2- 172.21.15.167:0/3823302379 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33fc003a00 0x7f34181383f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:10.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.660+0000 7f340d400640 1 --2- 172.21.15.167:0/3823302379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3418141940 0x7f3418144570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:10.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.660+0000 7f3416800640 1 -- 172.21.15.167:0/3823302379 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33fc003a00 msgr2=0x7f34181383f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:10.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.660+0000 7f3416800640 1 --2- 172.21.15.167:0/3823302379 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33fc003a00 0x7f34181383f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:10.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.660+0000 7f3416800640 1 -- 172.21.15.167:0/3823302379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3418141940 msgr2=0x7f3418144570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:10.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.660+0000 7f3416800640 1 --2- 172.21.15.167:0/3823302379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3418141940 0x7f3418144570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:10.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.660+0000 7f3416800640 1 -- 172.21.15.167:0/3823302379 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3418136920 con 0x7f341815fa30 2024-05-11T12:11:10.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.660+0000 7f340d400640 1 --2- 172.21.15.167:0/3823302379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3418141940 0x7f3418144570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:10.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.661+0000 7f3416800640 1 --2- 172.21.15.167:0/3823302379 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f341815fa30 0x7f341815fee0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f34180702e0 tx=0x7f341000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:10.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.661+0000 7f340ea00640 1 -- 172.21.15.167:0/3823302379 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3410016620 con 0x7f341815fa30 2024-05-11T12:11:10.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.661+0000 7f340ea00640 1 -- 172.21.15.167:0/3823302379 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f341001b070 con 0x7f341815fa30 2024-05-11T12:11:10.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.661+0000 7f340ea00640 1 -- 172.21.15.167:0/3823302379 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3410016a20 con 0x7f341815fa30 2024-05-11T12:11:10.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.661+0000 7f3417200640 1 -- 172.21.15.167:0/3823302379 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3418138930 con 0x7f341815fa30 2024-05-11T12:11:10.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.661+0000 7f3417200640 1 -- 172.21.15.167:0/3823302379 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34180aa780 con 0x7f341815fa30 2024-05-11T12:11:10.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.662+0000 7f3417200640 1 -- 172.21.15.167:0/3823302379 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34180694b0 con 0x7f341815fa30 2024-05-11T12:11:10.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.664+0000 7f340ea00640 1 -- 172.21.15.167:0/3823302379 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3410005d10 con 0x7f341815fa30 2024-05-11T12:11:10.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.665+0000 7f340ea00640 1 --2- 172.21.15.167:0/3823302379 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f33f40762d0 0x7f33f4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:10.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.665+0000 7f340ea00640 1 -- 172.21.15.167:0/3823302379 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f3410098080 con 0x7f341815fa30 2024-05-11T12:11:10.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.665+0000 7f3415e00640 1 --2- 172.21.15.167:0/3823302379 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f33f40762d0 0x7f33f4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:10.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.666+0000 7f3415e00640 1 --2- 172.21.15.167:0/3823302379 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f33f40762d0 0x7f33f4078790 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3408007fd0 tx=0x7f340802d000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:10.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.669+0000 7f340ea00640 1 -- 172.21.15.167:0/3823302379 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3410024020 con 0x7f341815fa30 2024-05-11T12:11:10.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.884+0000 7f3417200640 1 -- 172.21.15.167:0/3823302379 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f34180ceb80 con 0x7f341815fa30 2024-05-11T12:11:10.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.887+0000 7f340ea00640 1 -- 172.21.15.167:0/3823302379 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18562 (secure 0 0 0) 0x7f3410060fb0 con 0x7f341815fa30 2024-05-11T12:11:10.888 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:11:10.888 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":27,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:10:50.284769+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:10:43.084511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T12:10:44.157782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-11T12:10:47.814674+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:11:10.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.890+0000 7f3417200640 1 -- 172.21.15.167:0/3823302379 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f33f40762d0 msgr2=0x7f33f4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:10.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.890+0000 7f3417200640 1 --2- 172.21.15.167:0/3823302379 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f33f40762d0 0x7f33f4078790 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3408007fd0 tx=0x7f340802d000 comp rx=0 tx=0).stop 2024-05-11T12:11:10.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.890+0000 7f3417200640 1 -- 172.21.15.167:0/3823302379 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f341815fa30 msgr2=0x7f341815fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:10.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.890+0000 7f3417200640 1 --2- 172.21.15.167:0/3823302379 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f341815fa30 0x7f341815fee0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f34180702e0 tx=0x7f341000f520 comp rx=0 tx=0).stop 2024-05-11T12:11:10.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.890+0000 7f3417200640 1 -- 172.21.15.167:0/3823302379 shutdown_connections 2024-05-11T12:11:10.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.890+0000 7f3417200640 1 --2- 172.21.15.167:0/3823302379 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f341815fa30 0x7f341815fee0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:10.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.890+0000 7f3417200640 1 --2- 172.21.15.167:0/3823302379 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f33f40762d0 0x7f33f4078790 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:10.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.890+0000 7f3417200640 1 --2- 172.21.15.167:0/3823302379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3418141940 0x7f3418144570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:10.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.890+0000 7f3417200640 1 --2- 172.21.15.167:0/3823302379 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33fc003a00 0x7f34181383f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:10.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.890+0000 7f3417200640 1 -- 172.21.15.167:0/3823302379 >> 172.21.15.167:0/3823302379 conn(0x7f34180b3460 msgr2=0x7f34180b49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:10.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.890+0000 7f3417200640 1 -- 172.21.15.167:0/3823302379 shutdown_connections 2024-05-11T12:11:10.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:10.891+0000 7f3417200640 1 -- 172.21.15.167:0/3823302379 wait complete. 2024-05-11T12:11:10.910 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-11T12:11:10.910 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T12:11:11.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.112+0000 7f2cc8c00640 1 Processor -- start 2024-05-11T12:11:11.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.112+0000 7f2cc8c00640 1 -- start start 2024-05-11T12:11:11.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.114+0000 7f2cc8c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2cc4145b00 0x7f2cc406f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:11.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.114+0000 7f2cc8c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2cc4146770 0x7f2cc40ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:11.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.114+0000 7f2cc8c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cc406fbc0 con 0x7f2cc40cef10 2024-05-11T12:11:11.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.114+0000 7f2cc8c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f2cc40b4520 con 0x7f2cc4146770 2024-05-11T12:11:11.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.114+0000 7f2cc8c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2cc40b4800 con 0x7f2cc4145b00 2024-05-11T12:11:11.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.114+0000 7f2cc3400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2cc4145b00 0x7f2cc406f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:11.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.114+0000 7f2cc3400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2cc4145b00 0x7f2cc406f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:59518/0 (socket says 172.21.15.167:59518) 2024-05-11T12:11:11.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.114+0000 7f2cc3400640 1 -- 172.21.15.167:0/742760122 learned_addr learned my addr 172.21.15.167:0/742760122 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:11.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.115+0000 7f2cc2a00640 1 --2- 172.21.15.167:0/742760122 >> v2:172.21.15.204:3300/0 conn(0x7f2cc4146770 0x7f2cc40ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:11.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.115+0000 7f2cc3400640 1 -- 172.21.15.167:0/742760122 >> v2:172.21.15.204:3300/0 conn(0x7f2cc4146770 msgr2=0x7f2cc40ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:11.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.115+0000 7f2cc3400640 1 --2- 172.21.15.167:0/742760122 >> v2:172.21.15.204:3300/0 conn(0x7f2cc4146770 0x7f2cc40ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:11.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.115+0000 7f2cc3400640 1 -- 172.21.15.167:0/742760122 >> v1:172.21.15.204:6789/0 conn(0x7f2cc40cef10 legacy=0x7f2cc406c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:11.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.115+0000 7f2cc3400640 1 -- 172.21.15.167:0/742760122 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cc40b4ae0 con 0x7f2cc4145b00 2024-05-11T12:11:11.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.117+0000 7f2cc3400640 1 --2- 172.21.15.167:0/742760122 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2cc4145b00 0x7f2cc406f2c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4009870 tx=0x7f2cb402a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea383344f67ee355 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:11.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.117+0000 7f2cc2000640 1 -- 172.21.15.167:0/742760122 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb4037070 con 0x7f2cc4145b00 2024-05-11T12:11:11.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.117+0000 7f2cc2000640 1 -- 172.21.15.167:0/742760122 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cb4032440 con 0x7f2cc4145b00 2024-05-11T12:11:11.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.117+0000 7f2cc2000640 1 -- 172.21.15.167:0/742760122 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb4032810 con 0x7f2cc4145b00 2024-05-11T12:11:11.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.118+0000 7f2cc8c00640 1 -- 172.21.15.167:0/742760122 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2cc4145b00 msgr2=0x7f2cc406f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:11.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.118+0000 7f2cc8c00640 1 --2- 172.21.15.167:0/742760122 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2cc4145b00 0x7f2cc406f2c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4009870 tx=0x7f2cb402a1c0 comp rx=0 tx=0).stop 2024-05-11T12:11:11.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.118+0000 7f2cc8c00640 1 -- 172.21.15.167:0/742760122 shutdown_connections 2024-05-11T12:11:11.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.118+0000 7f2cc8c00640 1 --2- 172.21.15.167:0/742760122 >> v2:172.21.15.204:3300/0 conn(0x7f2cc4146770 0x7f2cc40ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:11.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.118+0000 7f2cc8c00640 1 --2- 172.21.15.167:0/742760122 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2cc4145b00 0x7f2cc406f2c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:11.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.118+0000 7f2cc8c00640 1 -- 172.21.15.167:0/742760122 >> 172.21.15.167:0/742760122 conn(0x7f2cc40b3460 msgr2=0x7f2cc40d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:11.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.118+0000 7f2cc8c00640 1 -- 172.21.15.167:0/742760122 shutdown_connections 2024-05-11T12:11:11.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.118+0000 7f2cc8c00640 1 -- 172.21.15.167:0/742760122 wait complete. 2024-05-11T12:11:11.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.119+0000 7f2cc8c00640 1 Processor -- start 2024-05-11T12:11:11.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.120+0000 7f2cc8c00640 1 -- start start 2024-05-11T12:11:11.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.121+0000 7f2cc8c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2cc40cef10 0x7f2cc4141b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:11.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.121+0000 7f2cc8c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2cc4145b00 0x7f2cc41f2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:11.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.121+0000 7f2cc8c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2cc4146770 0x7f2cc40b0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:11.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.121+0000 7f2cc8c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc406f980 con 0x7f2cc40cef10 2024-05-11T12:11:11.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.121+0000 7f2cc8c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc406f800 con 0x7f2cc4146770 2024-05-11T12:11:11.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.121+0000 7f2cc8c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2cc406cd70 con 0x7f2cc4145b00 2024-05-11T12:11:11.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.121+0000 7f2cc3400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2cc40cef10 0x7f2cc4141b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:11.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.121+0000 7f2cc3400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2cc40cef10 0x7f2cc4141b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41408/0 (socket says 172.21.15.167:41408) 2024-05-11T12:11:11.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.121+0000 7f2cc3400640 1 -- 172.21.15.167:0/3691433503 learned_addr learned my addr 172.21.15.167:0/3691433503 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:11.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.121+0000 7f2cc2a00640 1 --2- 172.21.15.167:0/3691433503 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2cc4145b00 0x7f2cc41f2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:11.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.121+0000 7f2cc3e00640 1 --2- 172.21.15.167:0/3691433503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2cc4146770 0x7f2cc40b0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:11.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.121+0000 7f2cc3400640 1 -- 172.21.15.167:0/3691433503 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2cc4145b00 msgr2=0x7f2cc41f2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:11.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.121+0000 7f2cc3400640 1 --2- 172.21.15.167:0/3691433503 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2cc4145b00 0x7f2cc41f2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:11.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.121+0000 7f2cc3400640 1 -- 172.21.15.167:0/3691433503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2cc4146770 msgr2=0x7f2cc40b0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:11.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.121+0000 7f2cc3400640 1 --2- 172.21.15.167:0/3691433503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2cc4146770 0x7f2cc40b0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:11.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.121+0000 7f2cc3400640 1 -- 172.21.15.167:0/3691433503 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cc4144720 con 0x7f2cc40cef10 2024-05-11T12:11:11.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.122+0000 7f2cc3400640 1 --2- 172.21.15.167:0/3691433503 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2cc40cef10 0x7f2cc4141b50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4000c00 tx=0x7f2cb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:11.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.122+0000 7f2cbb400640 1 -- 172.21.15.167:0/3691433503 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb4041020 con 0x7f2cc40cef10 2024-05-11T12:11:11.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.122+0000 7f2cbb400640 1 -- 172.21.15.167:0/3691433503 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cb4030780 con 0x7f2cc40cef10 2024-05-11T12:11:11.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.122+0000 7f2cbb400640 1 -- 172.21.15.167:0/3691433503 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb4037040 con 0x7f2cc40cef10 2024-05-11T12:11:11.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.122+0000 7f2cc8c00640 1 -- 172.21.15.167:0/3691433503 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cc4144a00 con 0x7f2cc40cef10 2024-05-11T12:11:11.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.122+0000 7f2cc8c00640 1 -- 172.21.15.167:0/3691433503 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cc4144fc0 con 0x7f2cc40cef10 2024-05-11T12:11:11.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.123+0000 7f2cc8c00640 1 -- 172.21.15.167:0/3691433503 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cc40694b0 con 0x7f2cc40cef10 2024-05-11T12:11:11.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.125+0000 7f2cbb400640 1 -- 172.21.15.167:0/3691433503 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2cb4044020 con 0x7f2cc40cef10 2024-05-11T12:11:11.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.126+0000 7f2cbb400640 1 --2- 172.21.15.167:0/3691433503 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2ca40762d0 0x7f2ca4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:11.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.126+0000 7f2cbb400640 1 -- 172.21.15.167:0/3691433503 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f2cb40b8110 con 0x7f2cc40cef10 2024-05-11T12:11:11.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.126+0000 7f2cc2a00640 1 --2- 172.21.15.167:0/3691433503 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2ca40762d0 0x7f2ca4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:11.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.127+0000 7f2cc2a00640 1 --2- 172.21.15.167:0/3691433503 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2ca40762d0 0x7f2ca4078790 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0006fd0 tx=0x7f2cb0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:11.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.130+0000 7f2cbb400640 1 -- 172.21.15.167:0/3691433503 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2cb404b030 con 0x7f2cc40cef10 2024-05-11T12:11:11.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.335+0000 7f2cc8c00640 1 -- 172.21.15.167:0/3691433503 --> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2cc40759b0 con 0x7f2ca40762d0 2024-05-11T12:11:11.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.347+0000 7f2cbb400640 1 -- 172.21.15.167:0/3691433503 <== mgr.4106 v2:172.21.15.167:6832/578352559 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332527 (secure 0 0 0) 0x7f2cc40759b0 con 0x7f2ca40762d0 2024-05-11T12:11:11.352 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:11:11.354 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2024-05-11T12:11:11.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.354+0000 7f2cc8c00640 1 -- 172.21.15.167:0/3691433503 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2ca40762d0 msgr2=0x7f2ca4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:11.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.354+0000 7f2cc8c00640 1 --2- 172.21.15.167:0/3691433503 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2ca40762d0 0x7f2ca4078790 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0006fd0 tx=0x7f2cb0008040 comp rx=0 tx=0).stop 2024-05-11T12:11:11.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.354+0000 7f2cc8c00640 1 -- 172.21.15.167:0/3691433503 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2cc40cef10 msgr2=0x7f2cc4141b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:11.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.354+0000 7f2cc8c00640 1 --2- 172.21.15.167:0/3691433503 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2cc40cef10 0x7f2cc4141b50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4000c00 tx=0x7f2cb40046c0 comp rx=0 tx=0).stop 2024-05-11T12:11:11.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.354+0000 7f2cc8c00640 1 -- 172.21.15.167:0/3691433503 shutdown_connections 2024-05-11T12:11:11.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.354+0000 7f2cc8c00640 1 --2- 172.21.15.167:0/3691433503 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2ca40762d0 0x7f2ca4078790 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:11.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.354+0000 7f2cc8c00640 1 --2- 172.21.15.167:0/3691433503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2cc4146770 0x7f2cc40b0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:11.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.354+0000 7f2cc8c00640 1 --2- 172.21.15.167:0/3691433503 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2cc4145b00 0x7f2cc41f2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:11.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.354+0000 7f2cc8c00640 1 --2- 172.21.15.167:0/3691433503 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2cc40cef10 0x7f2cc4141b50 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:11.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.354+0000 7f2cc8c00640 1 -- 172.21.15.167:0/3691433503 >> 172.21.15.167:0/3691433503 conn(0x7f2cc40b3460 msgr2=0x7f2cc40b4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:11.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.354+0000 7f2cc8c00640 1 -- 172.21.15.167:0/3691433503 shutdown_connections 2024-05-11T12:11:11.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.354+0000 7f2cc8c00640 1 -- 172.21.15.167:0/3691433503 wait complete. 2024-05-11T12:11:11.373 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2024-05-11T12:11:11.239330+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":150,"num_write_kb":1647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219560,"kb_used_data":3400,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755160,"statfs":{"total":773094113280,"available":772869283840,"internally_reserved":0,"allocated":3481600,"data_stored":2063272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005612"},"pg_stats":[{"pgid":"2.1f","version":"22'4","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.057257+0000","last_change":"2024-05-11T12:10:44.771802+0000","last_active":"2024-05-11T12:10:53.057257+0000","last_peered":"2024-05-11T12:10:53.057257+0000","last_clean":"2024-05-11T12:10:53.057257+0000","last_became_active":"2024-05-11T12:10:44.771511+0000","last_became_peered":"2024-05-11T12:10:44.771511+0000","last_unstale":"2024-05-11T12:10:53.057257+0000","last_undegraded":"2024-05-11T12:10:53.057257+0000","last_fullsized":"2024-05-11T12:10:53.057257+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:38:50.526921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.832716+0000","last_change":"2024-05-11T12:10:45.765221+0000","last_active":"2024-05-11T12:10:50.832716+0000","last_peered":"2024-05-11T12:10:50.832716+0000","last_clean":"2024-05-11T12:10:50.832716+0000","last_became_active":"2024-05-11T12:10:45.764595+0000","last_became_peered":"2024-05-11T12:10:45.764595+0000","last_unstale":"2024-05-11T12:10:50.832716+0000","last_undegraded":"2024-05-11T12:10:50.832716+0000","last_fullsized":"2024-05-11T12:10:50.832716+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:01:30.356378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.830235+0000","last_change":"2024-05-11T12:10:49.979783+0000","last_active":"2024-05-11T12:10:50.830235+0000","last_peered":"2024-05-11T12:10:50.830235+0000","last_clean":"2024-05-11T12:10:50.830235+0000","last_became_active":"2024-05-11T12:10:49.977927+0000","last_became_peered":"2024-05-11T12:10:49.977927+0000","last_unstale":"2024-05-11T12:10:50.830235+0000","last_undegraded":"2024-05-11T12:10:50.830235+0000","last_fullsized":"2024-05-11T12:10:50.830235+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:31:59.631048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.833109+0000","last_change":"2024-05-11T12:10:44.769768+0000","last_active":"2024-05-11T12:10:50.833109+0000","last_peered":"2024-05-11T12:10:50.833109+0000","last_clean":"2024-05-11T12:10:50.833109+0000","last_became_active":"2024-05-11T12:10:44.769498+0000","last_became_peered":"2024-05-11T12:10:44.769498+0000","last_unstale":"2024-05-11T12:10:50.833109+0000","last_undegraded":"2024-05-11T12:10:50.833109+0000","last_fullsized":"2024-05-11T12:10:50.833109+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:14:58.421726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.056999+0000","last_change":"2024-05-11T12:10:45.791108+0000","last_active":"2024-05-11T12:10:53.056999+0000","last_peered":"2024-05-11T12:10:53.056999+0000","last_clean":"2024-05-11T12:10:53.056999+0000","last_became_active":"2024-05-11T12:10:45.790807+0000","last_became_peered":"2024-05-11T12:10:45.790807+0000","last_unstale":"2024-05-11T12:10:53.056999+0000","last_undegraded":"2024-05-11T12:10:53.056999+0000","last_fullsized":"2024-05-11T12:10:53.056999+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:52:36.650847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.177575+0000","last_change":"2024-05-11T12:10:49.843347+0000","last_active":"2024-05-11T12:10:51.177575+0000","last_peered":"2024-05-11T12:10:51.177575+0000","last_clean":"2024-05-11T12:10:51.177575+0000","last_became_active":"2024-05-11T12:10:49.840522+0000","last_became_peered":"2024-05-11T12:10:49.840522+0000","last_unstale":"2024-05-11T12:10:51.177575+0000","last_undegraded":"2024-05-11T12:10:51.177575+0000","last_fullsized":"2024-05-11T12:10:51.177575+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:09:52.792492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.986041+0000","last_change":"2024-05-11T12:10:44.788150+0000","last_active":"2024-05-11T12:10:52.986041+0000","last_peered":"2024-05-11T12:10:52.986041+0000","last_clean":"2024-05-11T12:10:52.986041+0000","last_became_active":"2024-05-11T12:10:44.787827+0000","last_became_peered":"2024-05-11T12:10:44.787827+0000","last_unstale":"2024-05-11T12:10:52.986041+0000","last_undegraded":"2024-05-11T12:10:52.986041+0000","last_fullsized":"2024-05-11T12:10:52.986041+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:50:29.143515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.834938+0000","last_change":"2024-05-11T12:10:45.769675+0000","last_active":"2024-05-11T12:10:50.834938+0000","last_peered":"2024-05-11T12:10:50.834938+0000","last_clean":"2024-05-11T12:10:50.834938+0000","last_became_active":"2024-05-11T12:10:45.766444+0000","last_became_peered":"2024-05-11T12:10:45.766444+0000","last_unstale":"2024-05-11T12:10:50.834938+0000","last_undegraded":"2024-05-11T12:10:50.834938+0000","last_fullsized":"2024-05-11T12:10:50.834938+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:02:47.506141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.177937+0000","last_change":"2024-05-11T12:10:49.843249+0000","last_active":"2024-05-11T12:10:51.177937+0000","last_peered":"2024-05-11T12:10:51.177937+0000","last_clean":"2024-05-11T12:10:51.177937+0000","last_became_active":"2024-05-11T12:10:49.841714+0000","last_became_peered":"2024-05-11T12:10:49.841714+0000","last_unstale":"2024-05-11T12:10:51.177937+0000","last_undegraded":"2024-05-11T12:10:51.177937+0000","last_fullsized":"2024-05-11T12:10:51.177937+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:15:59.709719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.986035+0000","last_change":"2024-05-11T12:10:44.783078+0000","last_active":"2024-05-11T12:10:52.986035+0000","last_peered":"2024-05-11T12:10:52.986035+0000","last_clean":"2024-05-11T12:10:52.986035+0000","last_became_active":"2024-05-11T12:10:44.782750+0000","last_became_peered":"2024-05-11T12:10:44.782750+0000","last_unstale":"2024-05-11T12:10:52.986035+0000","last_undegraded":"2024-05-11T12:10:52.986035+0000","last_fullsized":"2024-05-11T12:10:52.986035+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:32:31.312066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.836129+0000","last_change":"2024-05-11T12:10:45.771583+0000","last_active":"2024-05-11T12:10:50.836129+0000","last_peered":"2024-05-11T12:10:50.836129+0000","last_clean":"2024-05-11T12:10:50.836129+0000","last_became_active":"2024-05-11T12:10:45.766864+0000","last_became_peered":"2024-05-11T12:10:45.766864+0000","last_unstale":"2024-05-11T12:10:50.836129+0000","last_undegraded":"2024-05-11T12:10:50.836129+0000","last_fullsized":"2024-05-11T12:10:50.836129+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:29:58.902743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.177649+0000","last_change":"2024-05-11T12:10:49.833489+0000","last_active":"2024-05-11T12:10:51.177649+0000","last_peered":"2024-05-11T12:10:51.177649+0000","last_clean":"2024-05-11T12:10:51.177649+0000","last_became_active":"2024-05-11T12:10:49.833030+0000","last_became_peered":"2024-05-11T12:10:49.833030+0000","last_unstale":"2024-05-11T12:10:51.177649+0000","last_undegraded":"2024-05-11T12:10:51.177649+0000","last_fullsized":"2024-05-11T12:10:51.177649+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:40:37.671338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.833888+0000","last_change":"2024-05-11T12:10:45.777692+0000","last_active":"2024-05-11T12:10:50.833888+0000","last_peered":"2024-05-11T12:10:50.833888+0000","last_clean":"2024-05-11T12:10:50.833888+0000","last_became_active":"2024-05-11T12:10:45.777388+0000","last_became_peered":"2024-05-11T12:10:45.777388+0000","last_unstale":"2024-05-11T12:10:50.833888+0000","last_undegraded":"2024-05-11T12:10:50.833888+0000","last_fullsized":"2024-05-11T12:10:50.833888+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:30:11.665446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.986035+0000","last_change":"2024-05-11T12:10:44.780645+0000","last_active":"2024-05-11T12:10:52.986035+0000","last_peered":"2024-05-11T12:10:52.986035+0000","last_clean":"2024-05-11T12:10:52.986035+0000","last_became_active":"2024-05-11T12:10:44.780371+0000","last_became_peered":"2024-05-11T12:10:44.780371+0000","last_unstale":"2024-05-11T12:10:52.986035+0000","last_undegraded":"2024-05-11T12:10:52.986035+0000","last_fullsized":"2024-05-11T12:10:52.986035+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:46:19.349691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.178802+0000","last_change":"2024-05-11T12:10:49.842333+0000","last_active":"2024-05-11T12:10:51.178802+0000","last_peered":"2024-05-11T12:10:51.178802+0000","last_clean":"2024-05-11T12:10:51.178802+0000","last_became_active":"2024-05-11T12:10:49.841805+0000","last_became_peered":"2024-05-11T12:10:49.841805+0000","last_unstale":"2024-05-11T12:10:51.178802+0000","last_undegraded":"2024-05-11T12:10:51.178802+0000","last_fullsized":"2024-05-11T12:10:51.178802+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:30:02.975329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.831265+0000","last_change":"2024-05-11T12:10:45.785101+0000","last_active":"2024-05-11T12:10:50.831265+0000","last_peered":"2024-05-11T12:10:50.831265+0000","last_clean":"2024-05-11T12:10:50.831265+0000","last_became_active":"2024-05-11T12:10:45.784830+0000","last_became_peered":"2024-05-11T12:10:45.784830+0000","last_unstale":"2024-05-11T12:10:50.831265+0000","last_undegraded":"2024-05-11T12:10:50.831265+0000","last_fullsized":"2024-05-11T12:10:50.831265+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:15:55.637133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.084641+0000","last_change":"2024-05-11T12:10:44.762965+0000","last_active":"2024-05-11T12:10:53.084641+0000","last_peered":"2024-05-11T12:10:53.084641+0000","last_clean":"2024-05-11T12:10:53.084641+0000","last_became_active":"2024-05-11T12:10:44.761074+0000","last_became_peered":"2024-05-11T12:10:44.761074+0000","last_unstale":"2024-05-11T12:10:53.084641+0000","last_undegraded":"2024-05-11T12:10:53.084641+0000","last_fullsized":"2024-05-11T12:10:53.084641+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:11:44.935841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.178724+0000","last_change":"2024-05-11T12:10:49.842014+0000","last_active":"2024-05-11T12:10:51.178724+0000","last_peered":"2024-05-11T12:10:51.178724+0000","last_clean":"2024-05-11T12:10:51.178724+0000","last_became_active":"2024-05-11T12:10:49.840640+0000","last_became_peered":"2024-05-11T12:10:49.840640+0000","last_unstale":"2024-05-11T12:10:51.178724+0000","last_undegraded":"2024-05-11T12:10:51.178724+0000","last_fullsized":"2024-05-11T12:10:51.178724+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:31:28.750933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.830617+0000","last_change":"2024-05-11T12:10:45.767414+0000","last_active":"2024-05-11T12:10:50.830617+0000","last_peered":"2024-05-11T12:10:50.830617+0000","last_clean":"2024-05-11T12:10:50.830617+0000","last_became_active":"2024-05-11T12:10:45.765560+0000","last_became_peered":"2024-05-11T12:10:45.765560+0000","last_unstale":"2024-05-11T12:10:50.830617+0000","last_undegraded":"2024-05-11T12:10:50.830617+0000","last_fullsized":"2024-05-11T12:10:50.830617+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:25:39.934222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.085197+0000","last_change":"2024-05-11T12:10:44.764974+0000","last_active":"2024-05-11T12:10:53.085197+0000","last_peered":"2024-05-11T12:10:53.085197+0000","last_clean":"2024-05-11T12:10:53.085197+0000","last_became_active":"2024-05-11T12:10:44.764692+0000","last_became_peered":"2024-05-11T12:10:44.764692+0000","last_unstale":"2024-05-11T12:10:53.085197+0000","last_undegraded":"2024-05-11T12:10:53.085197+0000","last_fullsized":"2024-05-11T12:10:53.085197+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:27:48.475653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.086845+0000","last_change":"2024-05-11T12:10:49.982086+0000","last_active":"2024-05-11T12:10:53.086845+0000","last_peered":"2024-05-11T12:10:53.086845+0000","last_clean":"2024-05-11T12:10:53.086845+0000","last_became_active":"2024-05-11T12:10:49.981499+0000","last_became_peered":"2024-05-11T12:10:49.981499+0000","last_unstale":"2024-05-11T12:10:53.086845+0000","last_undegraded":"2024-05-11T12:10:53.086845+0000","last_fullsized":"2024-05-11T12:10:53.086845+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:36:11.661440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.178118+0000","last_change":"2024-05-11T12:10:45.780179+0000","last_active":"2024-05-11T12:10:51.178118+0000","last_peered":"2024-05-11T12:10:51.178118+0000","last_clean":"2024-05-11T12:10:51.178118+0000","last_became_active":"2024-05-11T12:10:45.777747+0000","last_became_peered":"2024-05-11T12:10:45.777747+0000","last_unstale":"2024-05-11T12:10:51.178118+0000","last_undegraded":"2024-05-11T12:10:51.178118+0000","last_fullsized":"2024-05-11T12:10:51.178118+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:39:51.243581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.986553+0000","last_change":"2024-05-11T12:10:44.775537+0000","last_active":"2024-05-11T12:10:52.986553+0000","last_peered":"2024-05-11T12:10:52.986553+0000","last_clean":"2024-05-11T12:10:52.986553+0000","last_became_active":"2024-05-11T12:10:44.775159+0000","last_became_peered":"2024-05-11T12:10:44.775159+0000","last_unstale":"2024-05-11T12:10:52.986553+0000","last_undegraded":"2024-05-11T12:10:52.986553+0000","last_fullsized":"2024-05-11T12:10:52.986553+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:39:50.229692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.179022+0000","last_change":"2024-05-11T12:10:49.847192+0000","last_active":"2024-05-11T12:10:51.179022+0000","last_peered":"2024-05-11T12:10:51.179022+0000","last_clean":"2024-05-11T12:10:51.179022+0000","last_became_active":"2024-05-11T12:10:49.846655+0000","last_became_peered":"2024-05-11T12:10:49.846655+0000","last_unstale":"2024-05-11T12:10:51.179022+0000","last_undegraded":"2024-05-11T12:10:51.179022+0000","last_fullsized":"2024-05-11T12:10:51.179022+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:52:55.215724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.057239+0000","last_change":"2024-05-11T12:10:45.788180+0000","last_active":"2024-05-11T12:10:53.057239+0000","last_peered":"2024-05-11T12:10:53.057239+0000","last_clean":"2024-05-11T12:10:53.057239+0000","last_became_active":"2024-05-11T12:10:45.787883+0000","last_became_peered":"2024-05-11T12:10:45.787883+0000","last_unstale":"2024-05-11T12:10:53.057239+0000","last_undegraded":"2024-05-11T12:10:53.057239+0000","last_fullsized":"2024-05-11T12:10:53.057239+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:00:49.028293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.837705+0000","last_change":"2024-05-11T12:10:44.783756+0000","last_active":"2024-05-11T12:10:50.837705+0000","last_peered":"2024-05-11T12:10:50.837705+0000","last_clean":"2024-05-11T12:10:50.837705+0000","last_became_active":"2024-05-11T12:10:44.779442+0000","last_became_peered":"2024-05-11T12:10:44.779442+0000","last_unstale":"2024-05-11T12:10:50.837705+0000","last_undegraded":"2024-05-11T12:10:50.837705+0000","last_fullsized":"2024-05-11T12:10:50.837705+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:12:58.847001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.177959+0000","last_change":"2024-05-11T12:10:49.989122+0000","last_active":"2024-05-11T12:10:51.177959+0000","last_peered":"2024-05-11T12:10:51.177959+0000","last_clean":"2024-05-11T12:10:51.177959+0000","last_became_active":"2024-05-11T12:10:49.988059+0000","last_became_peered":"2024-05-11T12:10:49.988059+0000","last_unstale":"2024-05-11T12:10:51.177959+0000","last_undegraded":"2024-05-11T12:10:51.177959+0000","last_fullsized":"2024-05-11T12:10:51.177959+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:11:21.324557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.832784+0000","last_change":"2024-05-11T12:10:45.780311+0000","last_active":"2024-05-11T12:10:50.832784+0000","last_peered":"2024-05-11T12:10:50.832784+0000","last_clean":"2024-05-11T12:10:50.832784+0000","last_became_active":"2024-05-11T12:10:45.779912+0000","last_became_peered":"2024-05-11T12:10:45.779912+0000","last_unstale":"2024-05-11T12:10:50.832784+0000","last_undegraded":"2024-05-11T12:10:50.832784+0000","last_fullsized":"2024-05-11T12:10:50.832784+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:09:48.719906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.084558+0000","last_change":"2024-05-11T12:10:44.776437+0000","last_active":"2024-05-11T12:10:53.084558+0000","last_peered":"2024-05-11T12:10:53.084558+0000","last_clean":"2024-05-11T12:10:53.084558+0000","last_became_active":"2024-05-11T12:10:44.775585+0000","last_became_peered":"2024-05-11T12:10:44.775585+0000","last_unstale":"2024-05-11T12:10:53.084558+0000","last_undegraded":"2024-05-11T12:10:53.084558+0000","last_fullsized":"2024-05-11T12:10:53.084558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:32:31.312066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.830010+0000","last_change":"2024-05-11T12:10:49.980738+0000","last_active":"2024-05-11T12:10:50.830010+0000","last_peered":"2024-05-11T12:10:50.830010+0000","last_clean":"2024-05-11T12:10:50.830010+0000","last_became_active":"2024-05-11T12:10:49.977869+0000","last_became_peered":"2024-05-11T12:10:49.977869+0000","last_unstale":"2024-05-11T12:10:50.830010+0000","last_undegraded":"2024-05-11T12:10:50.830010+0000","last_fullsized":"2024-05-11T12:10:50.830010+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:30:17.395939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.086118+0000","last_change":"2024-05-11T12:10:45.773559+0000","last_active":"2024-05-11T12:10:53.086118+0000","last_peered":"2024-05-11T12:10:53.086118+0000","last_clean":"2024-05-11T12:10:53.086118+0000","last_became_active":"2024-05-11T12:10:45.773152+0000","last_became_peered":"2024-05-11T12:10:45.773152+0000","last_unstale":"2024-05-11T12:10:53.086118+0000","last_undegraded":"2024-05-11T12:10:53.086118+0000","last_fullsized":"2024-05-11T12:10:53.086118+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:27:46.323505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.988658+0000","last_change":"2024-05-11T12:10:44.779798+0000","last_active":"2024-05-11T12:10:52.988658+0000","last_peered":"2024-05-11T12:10:52.988658+0000","last_clean":"2024-05-11T12:10:52.988658+0000","last_became_active":"2024-05-11T12:10:44.779517+0000","last_became_peered":"2024-05-11T12:10:44.779517+0000","last_unstale":"2024-05-11T12:10:52.988658+0000","last_undegraded":"2024-05-11T12:10:52.988658+0000","last_fullsized":"2024-05-11T12:10:52.988658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:27:48.475653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.055959+0000","last_change":"2024-05-11T12:10:49.977964+0000","last_active":"2024-05-11T12:10:53.055959+0000","last_peered":"2024-05-11T12:10:53.055959+0000","last_clean":"2024-05-11T12:10:53.055959+0000","last_became_active":"2024-05-11T12:10:49.977533+0000","last_became_peered":"2024-05-11T12:10:49.977533+0000","last_unstale":"2024-05-11T12:10:53.055959+0000","last_undegraded":"2024-05-11T12:10:53.055959+0000","last_fullsized":"2024-05-11T12:10:53.055959+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:02:51.578727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.836384+0000","last_change":"2024-05-11T12:10:45.788527+0000","last_active":"2024-05-11T12:10:50.836384+0000","last_peered":"2024-05-11T12:10:50.836384+0000","last_clean":"2024-05-11T12:10:50.836384+0000","last_became_active":"2024-05-11T12:10:45.783632+0000","last_became_peered":"2024-05-11T12:10:45.783632+0000","last_unstale":"2024-05-11T12:10:50.836384+0000","last_undegraded":"2024-05-11T12:10:50.836384+0000","last_fullsized":"2024-05-11T12:10:50.836384+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:52:51.143138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.086299+0000","last_change":"2024-05-11T12:10:44.764150+0000","last_active":"2024-05-11T12:10:53.086299+0000","last_peered":"2024-05-11T12:10:53.086299+0000","last_clean":"2024-05-11T12:10:53.086299+0000","last_became_active":"2024-05-11T12:10:44.763836+0000","last_became_peered":"2024-05-11T12:10:44.763836+0000","last_unstale":"2024-05-11T12:10:53.086299+0000","last_undegraded":"2024-05-11T12:10:53.086299+0000","last_fullsized":"2024-05-11T12:10:53.086299+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:39:50.229692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.085357+0000","last_change":"2024-05-11T12:10:49.841029+0000","last_active":"2024-05-11T12:10:53.085357+0000","last_peered":"2024-05-11T12:10:53.085357+0000","last_clean":"2024-05-11T12:10:53.085357+0000","last_became_active":"2024-05-11T12:10:49.840067+0000","last_became_peered":"2024-05-11T12:10:49.840067+0000","last_unstale":"2024-05-11T12:10:53.085357+0000","last_undegraded":"2024-05-11T12:10:53.085357+0000","last_fullsized":"2024-05-11T12:10:53.085357+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:02:51.578727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.178524+0000","last_change":"2024-05-11T12:10:45.785688+0000","last_active":"2024-05-11T12:10:51.178524+0000","last_peered":"2024-05-11T12:10:51.178524+0000","last_clean":"2024-05-11T12:10:51.178524+0000","last_became_active":"2024-05-11T12:10:45.785267+0000","last_became_peered":"2024-05-11T12:10:45.785267+0000","last_unstale":"2024-05-11T12:10:51.178524+0000","last_undegraded":"2024-05-11T12:10:51.178524+0000","last_fullsized":"2024-05-11T12:10:51.178524+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:50:30.157404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.833626+0000","last_change":"2024-05-11T12:10:44.777030+0000","last_active":"2024-05-11T12:10:50.833626+0000","last_peered":"2024-05-11T12:10:50.833626+0000","last_clean":"2024-05-11T12:10:50.833626+0000","last_became_active":"2024-05-11T12:10:44.776600+0000","last_became_peered":"2024-05-11T12:10:44.776600+0000","last_unstale":"2024-05-11T12:10:50.833626+0000","last_undegraded":"2024-05-11T12:10:50.833626+0000","last_fullsized":"2024-05-11T12:10:50.833626+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:32:31.312066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.086293+0000","last_change":"2024-05-11T12:10:49.843811+0000","last_active":"2024-05-11T12:10:53.086293+0000","last_peered":"2024-05-11T12:10:53.086293+0000","last_clean":"2024-05-11T12:10:53.086293+0000","last_became_active":"2024-05-11T12:10:49.843361+0000","last_became_peered":"2024-05-11T12:10:49.843361+0000","last_unstale":"2024-05-11T12:10:53.086293+0000","last_undegraded":"2024-05-11T12:10:53.086293+0000","last_fullsized":"2024-05-11T12:10:53.086293+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:03:19.587244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.085874+0000","last_change":"2024-05-11T12:10:45.773922+0000","last_active":"2024-05-11T12:10:53.085874+0000","last_peered":"2024-05-11T12:10:53.085874+0000","last_clean":"2024-05-11T12:10:53.085874+0000","last_became_active":"2024-05-11T12:10:45.773584+0000","last_became_peered":"2024-05-11T12:10:45.773584+0000","last_unstale":"2024-05-11T12:10:53.085874+0000","last_undegraded":"2024-05-11T12:10:53.085874+0000","last_fullsized":"2024-05-11T12:10:53.085874+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:25:39.934222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.956061+0000","last_change":"2024-05-11T12:10:44.767869+0000","last_active":"2024-05-11T12:10:50.956061+0000","last_peered":"2024-05-11T12:10:50.956061+0000","last_clean":"2024-05-11T12:10:50.956061+0000","last_became_active":"2024-05-11T12:10:44.766486+0000","last_became_peered":"2024-05-11T12:10:44.766486+0000","last_unstale":"2024-05-11T12:10:50.956061+0000","last_undegraded":"2024-05-11T12:10:50.956061+0000","last_fullsized":"2024-05-11T12:10:50.956061+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:49:38.234192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.087005+0000","last_change":"2024-05-11T12:10:49.837338+0000","last_active":"2024-05-11T12:10:53.087005+0000","last_peered":"2024-05-11T12:10:53.087005+0000","last_clean":"2024-05-11T12:10:53.087005+0000","last_became_active":"2024-05-11T12:10:49.836880+0000","last_became_peered":"2024-05-11T12:10:49.836880+0000","last_unstale":"2024-05-11T12:10:53.087005+0000","last_undegraded":"2024-05-11T12:10:53.087005+0000","last_fullsized":"2024-05-11T12:10:53.087005+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:30:58.400447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.055566+0000","last_change":"2024-05-11T12:10:45.784532+0000","last_active":"2024-05-11T12:10:53.055566+0000","last_peered":"2024-05-11T12:10:53.055566+0000","last_clean":"2024-05-11T12:10:53.055566+0000","last_became_active":"2024-05-11T12:10:45.783747+0000","last_became_peered":"2024-05-11T12:10:45.783747+0000","last_unstale":"2024-05-11T12:10:53.055566+0000","last_undegraded":"2024-05-11T12:10:53.055566+0000","last_fullsized":"2024-05-11T12:10:53.055566+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:33:50.642754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.986822+0000","last_change":"2024-05-11T12:10:44.778140+0000","last_active":"2024-05-11T12:10:52.986822+0000","last_peered":"2024-05-11T12:10:52.986822+0000","last_clean":"2024-05-11T12:10:52.986822+0000","last_became_active":"2024-05-11T12:10:44.777780+0000","last_became_peered":"2024-05-11T12:10:44.777780+0000","last_unstale":"2024-05-11T12:10:52.986822+0000","last_undegraded":"2024-05-11T12:10:52.986822+0000","last_fullsized":"2024-05-11T12:10:52.986822+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:25:35.246177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.057469+0000","last_change":"2024-05-11T12:10:49.984316+0000","last_active":"2024-05-11T12:10:53.057469+0000","last_peered":"2024-05-11T12:10:53.057469+0000","last_clean":"2024-05-11T12:10:53.057469+0000","last_became_active":"2024-05-11T12:10:49.981788+0000","last_became_peered":"2024-05-11T12:10:49.981788+0000","last_unstale":"2024-05-11T12:10:53.057469+0000","last_undegraded":"2024-05-11T12:10:53.057469+0000","last_fullsized":"2024-05-11T12:10:53.057469+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:07:34.303729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.830946+0000","last_change":"2024-05-11T12:10:45.781491+0000","last_active":"2024-05-11T12:10:50.830946+0000","last_peered":"2024-05-11T12:10:50.830946+0000","last_clean":"2024-05-11T12:10:50.830946+0000","last_became_active":"2024-05-11T12:10:45.779112+0000","last_became_peered":"2024-05-11T12:10:45.779112+0000","last_unstale":"2024-05-11T12:10:50.830946+0000","last_undegraded":"2024-05-11T12:10:50.830946+0000","last_fullsized":"2024-05-11T12:10:50.830946+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:00:49.028293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.833659+0000","last_change":"2024-05-11T12:10:44.765494+0000","last_active":"2024-05-11T12:10:50.833659+0000","last_peered":"2024-05-11T12:10:50.833659+0000","last_clean":"2024-05-11T12:10:50.833659+0000","last_became_active":"2024-05-11T12:10:44.764921+0000","last_became_peered":"2024-05-11T12:10:44.764921+0000","last_unstale":"2024-05-11T12:10:50.833659+0000","last_undegraded":"2024-05-11T12:10:50.833659+0000","last_fullsized":"2024-05-11T12:10:50.833659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:11:44.935841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":117,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.986840+0000","last_change":"2024-05-11T12:10:35.066673+0000","last_active":"2024-05-11T12:10:52.986840+0000","last_peered":"2024-05-11T12:10:52.986840+0000","last_clean":"2024-05-11T12:10:52.986840+0000","last_became_active":"2024-05-11T12:10:35.065942+0000","last_became_peered":"2024-05-11T12:10:35.065942+0000","last_unstale":"2024-05-11T12:10:52.986840+0000","last_undegraded":"2024-05-11T12:10:52.986840+0000","last_fullsized":"2024-05-11T12:10:52.986840+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_clean_scrub_stamp":"2024-05-11T12:10:33.601136+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:12:02.133182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.836369+0000","last_change":"2024-05-11T12:10:49.983023+0000","last_active":"2024-05-11T12:10:50.836369+0000","last_peered":"2024-05-11T12:10:50.836369+0000","last_clean":"2024-05-11T12:10:50.836369+0000","last_became_active":"2024-05-11T12:10:49.981575+0000","last_became_peered":"2024-05-11T12:10:49.981575+0000","last_unstale":"2024-05-11T12:10:50.836369+0000","last_undegraded":"2024-05-11T12:10:50.836369+0000","last_fullsized":"2024-05-11T12:10:50.836369+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:52:55.215724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.988305+0000","last_change":"2024-05-11T12:10:45.787709+0000","last_active":"2024-05-11T12:10:52.988305+0000","last_peered":"2024-05-11T12:10:52.988305+0000","last_clean":"2024-05-11T12:10:52.988305+0000","last_became_active":"2024-05-11T12:10:45.787205+0000","last_became_peered":"2024-05-11T12:10:45.787205+0000","last_unstale":"2024-05-11T12:10:52.988305+0000","last_undegraded":"2024-05-11T12:10:52.988305+0000","last_fullsized":"2024-05-11T12:10:52.988305+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:29:58.902743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.956475+0000","last_change":"2024-05-11T12:10:44.777012+0000","last_active":"2024-05-11T12:10:50.956475+0000","last_peered":"2024-05-11T12:10:50.956475+0000","last_clean":"2024-05-11T12:10:50.956475+0000","last_became_active":"2024-05-11T12:10:44.776709+0000","last_became_peered":"2024-05-11T12:10:44.776709+0000","last_unstale":"2024-05-11T12:10:50.956475+0000","last_undegraded":"2024-05-11T12:10:50.956475+0000","last_fullsized":"2024-05-11T12:10:50.956475+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:32:06.207257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.833701+0000","last_change":"2024-05-11T12:10:49.980334+0000","last_active":"2024-05-11T12:10:50.833701+0000","last_peered":"2024-05-11T12:10:50.833701+0000","last_clean":"2024-05-11T12:10:50.833701+0000","last_became_active":"2024-05-11T12:10:49.977819+0000","last_became_peered":"2024-05-11T12:10:49.977819+0000","last_unstale":"2024-05-11T12:10:50.833701+0000","last_undegraded":"2024-05-11T12:10:50.833701+0000","last_fullsized":"2024-05-11T12:10:50.833701+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:30:02.975329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.835272+0000","last_change":"2024-05-11T12:10:45.780131+0000","last_active":"2024-05-11T12:10:50.835272+0000","last_peered":"2024-05-11T12:10:50.835272+0000","last_clean":"2024-05-11T12:10:50.835272+0000","last_became_active":"2024-05-11T12:10:45.779867+0000","last_became_peered":"2024-05-11T12:10:45.779867+0000","last_unstale":"2024-05-11T12:10:50.835272+0000","last_undegraded":"2024-05-11T12:10:50.835272+0000","last_fullsized":"2024-05-11T12:10:50.835272+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:30:54.327861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.084659+0000","last_change":"2024-05-11T12:10:44.775148+0000","last_active":"2024-05-11T12:10:53.084659+0000","last_peered":"2024-05-11T12:10:53.084659+0000","last_clean":"2024-05-11T12:10:53.084659+0000","last_became_active":"2024-05-11T12:10:44.774850+0000","last_became_peered":"2024-05-11T12:10:44.774850+0000","last_unstale":"2024-05-11T12:10:53.084659+0000","last_undegraded":"2024-05-11T12:10:53.084659+0000","last_fullsized":"2024-05-11T12:10:53.084659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:25:35.246177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.056241+0000","last_change":"2024-05-11T12:10:49.842656+0000","last_active":"2024-05-11T12:10:53.056241+0000","last_peered":"2024-05-11T12:10:53.056241+0000","last_clean":"2024-05-11T12:10:53.056241+0000","last_became_active":"2024-05-11T12:10:49.841982+0000","last_became_peered":"2024-05-11T12:10:49.841982+0000","last_unstale":"2024-05-11T12:10:53.056241+0000","last_undegraded":"2024-05-11T12:10:53.056241+0000","last_fullsized":"2024-05-11T12:10:53.056241+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:03:19.587244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.988912+0000","last_change":"2024-05-11T12:10:45.794103+0000","last_active":"2024-05-11T12:10:52.988912+0000","last_peered":"2024-05-11T12:10:52.988912+0000","last_clean":"2024-05-11T12:10:52.988912+0000","last_became_active":"2024-05-11T12:10:45.792080+0000","last_became_peered":"2024-05-11T12:10:45.792080+0000","last_unstale":"2024-05-11T12:10:52.988912+0000","last_undegraded":"2024-05-11T12:10:52.988912+0000","last_fullsized":"2024-05-11T12:10:52.988912+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:30:54.327861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.956469+0000","last_change":"2024-05-11T12:10:44.768108+0000","last_active":"2024-05-11T12:10:50.956469+0000","last_peered":"2024-05-11T12:10:50.956469+0000","last_clean":"2024-05-11T12:10:50.956469+0000","last_became_active":"2024-05-11T12:10:44.766606+0000","last_became_peered":"2024-05-11T12:10:44.766606+0000","last_unstale":"2024-05-11T12:10:50.956469+0000","last_undegraded":"2024-05-11T12:10:50.956469+0000","last_fullsized":"2024-05-11T12:10:50.956469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:02:04.596576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.832239+0000","last_change":"2024-05-11T12:10:49.982446+0000","last_active":"2024-05-11T12:10:50.832239+0000","last_peered":"2024-05-11T12:10:50.832239+0000","last_clean":"2024-05-11T12:10:50.832239+0000","last_became_active":"2024-05-11T12:10:49.981272+0000","last_became_peered":"2024-05-11T12:10:49.981272+0000","last_unstale":"2024-05-11T12:10:50.832239+0000","last_undegraded":"2024-05-11T12:10:50.832239+0000","last_fullsized":"2024-05-11T12:10:50.832239+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:36:40.828180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.988575+0000","last_change":"2024-05-11T12:10:45.786373+0000","last_active":"2024-05-11T12:10:52.988575+0000","last_peered":"2024-05-11T12:10:52.988575+0000","last_clean":"2024-05-11T12:10:52.988575+0000","last_became_active":"2024-05-11T12:10:45.786050+0000","last_became_peered":"2024-05-11T12:10:45.786050+0000","last_unstale":"2024-05-11T12:10:52.988575+0000","last_undegraded":"2024-05-11T12:10:52.988575+0000","last_fullsized":"2024-05-11T12:10:52.988575+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:31:24.678347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.178969+0000","last_change":"2024-05-11T12:10:44.775124+0000","last_active":"2024-05-11T12:10:51.178969+0000","last_peered":"2024-05-11T12:10:51.178969+0000","last_clean":"2024-05-11T12:10:51.178969+0000","last_became_active":"2024-05-11T12:10:44.774810+0000","last_became_peered":"2024-05-11T12:10:44.774810+0000","last_unstale":"2024-05-11T12:10:51.178969+0000","last_undegraded":"2024-05-11T12:10:51.178969+0000","last_fullsized":"2024-05-11T12:10:51.178969+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:44:00.317135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.177431+0000","last_change":"2024-05-11T12:10:49.845241+0000","last_active":"2024-05-11T12:10:51.177431+0000","last_peered":"2024-05-11T12:10:51.177431+0000","last_clean":"2024-05-11T12:10:51.177431+0000","last_became_active":"2024-05-11T12:10:49.844564+0000","last_became_peered":"2024-05-11T12:10:49.844564+0000","last_unstale":"2024-05-11T12:10:51.177431+0000","last_undegraded":"2024-05-11T12:10:51.177431+0000","last_fullsized":"2024-05-11T12:10:51.177431+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:27:50.396091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.987029+0000","last_change":"2024-05-11T12:10:45.789312+0000","last_active":"2024-05-11T12:10:52.987029+0000","last_peered":"2024-05-11T12:10:52.987029+0000","last_clean":"2024-05-11T12:10:52.987029+0000","last_became_active":"2024-05-11T12:10:45.789069+0000","last_became_peered":"2024-05-11T12:10:45.789069+0000","last_unstale":"2024-05-11T12:10:52.987029+0000","last_undegraded":"2024-05-11T12:10:52.987029+0000","last_fullsized":"2024-05-11T12:10:52.987029+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:15:55.637133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.955873+0000","last_change":"2024-05-11T12:10:44.767205+0000","last_active":"2024-05-11T12:10:50.955873+0000","last_peered":"2024-05-11T12:10:50.955873+0000","last_clean":"2024-05-11T12:10:50.955873+0000","last_became_active":"2024-05-11T12:10:44.766855+0000","last_became_peered":"2024-05-11T12:10:44.766855+0000","last_unstale":"2024-05-11T12:10:50.955873+0000","last_undegraded":"2024-05-11T12:10:50.955873+0000","last_fullsized":"2024-05-11T12:10:50.955873+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:10:35.839599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.056975+0000","last_change":"2024-05-11T12:10:49.835689+0000","last_active":"2024-05-11T12:10:53.056975+0000","last_peered":"2024-05-11T12:10:53.056975+0000","last_clean":"2024-05-11T12:10:53.056975+0000","last_became_active":"2024-05-11T12:10:49.835305+0000","last_became_peered":"2024-05-11T12:10:49.835305+0000","last_unstale":"2024-05-11T12:10:53.056975+0000","last_undegraded":"2024-05-11T12:10:53.056975+0000","last_fullsized":"2024-05-11T12:10:53.056975+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:30:58.400447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.833870+0000","last_change":"2024-05-11T12:10:45.788030+0000","last_active":"2024-05-11T12:10:50.833870+0000","last_peered":"2024-05-11T12:10:50.833870+0000","last_clean":"2024-05-11T12:10:50.833870+0000","last_became_active":"2024-05-11T12:10:45.787098+0000","last_became_peered":"2024-05-11T12:10:45.787098+0000","last_unstale":"2024-05-11T12:10:50.833870+0000","last_undegraded":"2024-05-11T12:10:50.833870+0000","last_fullsized":"2024-05-11T12:10:50.833870+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:00:49.028293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.834966+0000","last_change":"2024-05-11T12:10:44.784319+0000","last_active":"2024-05-11T12:10:50.834966+0000","last_peered":"2024-05-11T12:10:50.834966+0000","last_clean":"2024-05-11T12:10:50.834966+0000","last_became_active":"2024-05-11T12:10:44.784027+0000","last_became_peered":"2024-05-11T12:10:44.784027+0000","last_unstale":"2024-05-11T12:10:50.834966+0000","last_undegraded":"2024-05-11T12:10:50.834966+0000","last_fullsized":"2024-05-11T12:10:50.834966+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:50:29.143515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.832654+0000","last_change":"2024-05-11T12:10:49.985187+0000","last_active":"2024-05-11T12:10:50.832654+0000","last_peered":"2024-05-11T12:10:50.832654+0000","last_clean":"2024-05-11T12:10:50.832654+0000","last_became_active":"2024-05-11T12:10:49.983450+0000","last_became_peered":"2024-05-11T12:10:49.983450+0000","last_unstale":"2024-05-11T12:10:50.832654+0000","last_undegraded":"2024-05-11T12:10:50.832654+0000","last_fullsized":"2024-05-11T12:10:50.832654+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:02:13.707329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.987787+0000","last_change":"2024-05-11T12:10:45.789328+0000","last_active":"2024-05-11T12:10:52.987787+0000","last_peered":"2024-05-11T12:10:52.987787+0000","last_clean":"2024-05-11T12:10:52.987787+0000","last_became_active":"2024-05-11T12:10:45.788778+0000","last_became_peered":"2024-05-11T12:10:45.788778+0000","last_unstale":"2024-05-11T12:10:52.987787+0000","last_undegraded":"2024-05-11T12:10:52.987787+0000","last_fullsized":"2024-05-11T12:10:52.987787+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:01:30.356378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.833719+0000","last_change":"2024-05-11T12:10:44.779353+0000","last_active":"2024-05-11T12:10:50.833719+0000","last_peered":"2024-05-11T12:10:50.833719+0000","last_clean":"2024-05-11T12:10:50.833719+0000","last_became_active":"2024-05-11T12:10:44.779078+0000","last_became_peered":"2024-05-11T12:10:44.779078+0000","last_unstale":"2024-05-11T12:10:50.833719+0000","last_undegraded":"2024-05-11T12:10:50.833719+0000","last_fullsized":"2024-05-11T12:10:50.833719+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:25:35.246177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.831951+0000","last_change":"2024-05-11T12:10:49.979416+0000","last_active":"2024-05-11T12:10:50.831951+0000","last_peered":"2024-05-11T12:10:50.831951+0000","last_clean":"2024-05-11T12:10:50.831951+0000","last_became_active":"2024-05-11T12:10:49.978262+0000","last_became_peered":"2024-05-11T12:10:49.978262+0000","last_unstale":"2024-05-11T12:10:50.831951+0000","last_undegraded":"2024-05-11T12:10:50.831951+0000","last_fullsized":"2024-05-11T12:10:50.831951+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:53:47.237894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.056303+0000","last_change":"2024-05-11T12:10:45.784434+0000","last_active":"2024-05-11T12:10:53.056303+0000","last_peered":"2024-05-11T12:10:53.056303+0000","last_clean":"2024-05-11T12:10:53.056303+0000","last_became_active":"2024-05-11T12:10:45.784143+0000","last_became_peered":"2024-05-11T12:10:45.784143+0000","last_unstale":"2024-05-11T12:10:53.056303+0000","last_undegraded":"2024-05-11T12:10:53.056303+0000","last_fullsized":"2024-05-11T12:10:53.056303+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:25:39.934222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.054871+0000","last_change":"2024-05-11T12:10:44.783217+0000","last_active":"2024-05-11T12:10:53.054871+0000","last_peered":"2024-05-11T12:10:53.054871+0000","last_clean":"2024-05-11T12:10:53.054871+0000","last_became_active":"2024-05-11T12:10:44.782942+0000","last_became_peered":"2024-05-11T12:10:44.782942+0000","last_unstale":"2024-05-11T12:10:53.054871+0000","last_undegraded":"2024-05-11T12:10:53.054871+0000","last_fullsized":"2024-05-11T12:10:53.054871+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:14:58.421726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.836406+0000","last_change":"2024-05-11T12:10:49.981126+0000","last_active":"2024-05-11T12:10:50.836406+0000","last_peered":"2024-05-11T12:10:50.836406+0000","last_clean":"2024-05-11T12:10:50.836406+0000","last_became_active":"2024-05-11T12:10:49.978149+0000","last_became_peered":"2024-05-11T12:10:49.978149+0000","last_unstale":"2024-05-11T12:10:50.836406+0000","last_undegraded":"2024-05-11T12:10:50.836406+0000","last_fullsized":"2024-05-11T12:10:50.836406+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:45:08.936896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.988035+0000","last_change":"2024-05-11T12:10:45.791440+0000","last_active":"2024-05-11T12:10:52.988035+0000","last_peered":"2024-05-11T12:10:52.988035+0000","last_clean":"2024-05-11T12:10:52.988035+0000","last_became_active":"2024-05-11T12:10:45.791154+0000","last_became_peered":"2024-05-11T12:10:45.791154+0000","last_unstale":"2024-05-11T12:10:52.988035+0000","last_undegraded":"2024-05-11T12:10:52.988035+0000","last_fullsized":"2024-05-11T12:10:52.988035+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:52:51.143138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'10","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:11:01.126783+0000","last_change":"2024-05-11T12:10:44.773468+0000","last_active":"2024-05-11T12:11:01.126783+0000","last_peered":"2024-05-11T12:11:01.126783+0000","last_clean":"2024-05-11T12:11:01.126783+0000","last_became_active":"2024-05-11T12:10:44.769134+0000","last_became_peered":"2024-05-11T12:10:44.769134+0000","last_unstale":"2024-05-11T12:11:01.126783+0000","last_undegraded":"2024-05-11T12:11:01.126783+0000","last_fullsized":"2024-05-11T12:11:01.126783+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:08:58.049386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.085619+0000","last_change":"2024-05-11T12:10:49.847456+0000","last_active":"2024-05-11T12:10:53.085619+0000","last_peered":"2024-05-11T12:10:53.085619+0000","last_clean":"2024-05-11T12:10:53.085619+0000","last_became_active":"2024-05-11T12:10:49.847009+0000","last_became_peered":"2024-05-11T12:10:49.847009+0000","last_unstale":"2024-05-11T12:10:53.085619+0000","last_undegraded":"2024-05-11T12:10:53.085619+0000","last_fullsized":"2024-05-11T12:10:53.085619+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:52:55.215724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.178813+0000","last_change":"2024-05-11T12:10:45.781361+0000","last_active":"2024-05-11T12:10:51.178813+0000","last_peered":"2024-05-11T12:10:51.178813+0000","last_clean":"2024-05-11T12:10:51.178813+0000","last_became_active":"2024-05-11T12:10:45.781039+0000","last_became_peered":"2024-05-11T12:10:45.781039+0000","last_unstale":"2024-05-11T12:10:51.178813+0000","last_undegraded":"2024-05-11T12:10:51.178813+0000","last_fullsized":"2024-05-11T12:10:51.178813+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:27:49.489542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.084600+0000","last_change":"2024-05-11T12:10:44.770795+0000","last_active":"2024-05-11T12:10:53.084600+0000","last_peered":"2024-05-11T12:10:53.084600+0000","last_clean":"2024-05-11T12:10:53.084600+0000","last_became_active":"2024-05-11T12:10:44.770438+0000","last_became_peered":"2024-05-11T12:10:44.770438+0000","last_unstale":"2024-05-11T12:10:53.084600+0000","last_undegraded":"2024-05-11T12:10:53.084600+0000","last_fullsized":"2024-05-11T12:10:53.084600+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:50:29.143515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.178367+0000","last_change":"2024-05-11T12:10:49.841798+0000","last_active":"2024-05-11T12:10:51.178367+0000","last_peered":"2024-05-11T12:10:51.178367+0000","last_clean":"2024-05-11T12:10:51.178367+0000","last_became_active":"2024-05-11T12:10:49.840773+0000","last_became_peered":"2024-05-11T12:10:49.840773+0000","last_unstale":"2024-05-11T12:10:51.178367+0000","last_undegraded":"2024-05-11T12:10:51.178367+0000","last_fullsized":"2024-05-11T12:10:51.178367+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:01:34.428964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.057219+0000","last_change":"2024-05-11T12:10:45.779106+0000","last_active":"2024-05-11T12:10:53.057219+0000","last_peered":"2024-05-11T12:10:53.057219+0000","last_clean":"2024-05-11T12:10:53.057219+0000","last_became_active":"2024-05-11T12:10:45.778616+0000","last_became_peered":"2024-05-11T12:10:45.778616+0000","last_unstale":"2024-05-11T12:10:53.057219+0000","last_undegraded":"2024-05-11T12:10:53.057219+0000","last_fullsized":"2024-05-11T12:10:53.057219+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:40:33.598752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.837351+0000","last_change":"2024-05-11T12:10:44.781127+0000","last_active":"2024-05-11T12:10:50.837351+0000","last_peered":"2024-05-11T12:10:50.837351+0000","last_clean":"2024-05-11T12:10:50.837351+0000","last_became_active":"2024-05-11T12:10:44.780840+0000","last_became_peered":"2024-05-11T12:10:44.780840+0000","last_unstale":"2024-05-11T12:10:50.837351+0000","last_undegraded":"2024-05-11T12:10:50.837351+0000","last_fullsized":"2024-05-11T12:10:50.837351+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:50:29.143515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.830538+0000","last_change":"2024-05-11T12:10:49.839762+0000","last_active":"2024-05-11T12:10:50.830538+0000","last_peered":"2024-05-11T12:10:50.830538+0000","last_clean":"2024-05-11T12:10:50.830538+0000","last_became_active":"2024-05-11T12:10:49.838073+0000","last_became_peered":"2024-05-11T12:10:49.838073+0000","last_unstale":"2024-05-11T12:10:50.830538+0000","last_undegraded":"2024-05-11T12:10:50.830538+0000","last_fullsized":"2024-05-11T12:10:50.830538+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:36:40.828180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.837108+0000","last_change":"2024-05-11T12:10:45.778187+0000","last_active":"2024-05-11T12:10:50.837108+0000","last_peered":"2024-05-11T12:10:50.837108+0000","last_clean":"2024-05-11T12:10:50.837108+0000","last_became_active":"2024-05-11T12:10:45.776905+0000","last_became_peered":"2024-05-11T12:10:45.776905+0000","last_unstale":"2024-05-11T12:10:50.837108+0000","last_undegraded":"2024-05-11T12:10:50.837108+0000","last_fullsized":"2024-05-11T12:10:50.837108+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:36:07.588854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.837368+0000","last_change":"2024-05-11T12:10:44.778851+0000","last_active":"2024-05-11T12:10:50.837368+0000","last_peered":"2024-05-11T12:10:50.837368+0000","last_clean":"2024-05-11T12:10:50.837368+0000","last_became_active":"2024-05-11T12:10:44.776695+0000","last_became_peered":"2024-05-11T12:10:44.776695+0000","last_unstale":"2024-05-11T12:10:50.837368+0000","last_undegraded":"2024-05-11T12:10:50.837368+0000","last_fullsized":"2024-05-11T12:10:50.837368+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:49:09.103744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.831100+0000","last_change":"2024-05-11T12:10:49.982849+0000","last_active":"2024-05-11T12:10:50.831100+0000","last_peered":"2024-05-11T12:10:50.831100+0000","last_clean":"2024-05-11T12:10:50.831100+0000","last_became_active":"2024-05-11T12:10:49.981894+0000","last_became_peered":"2024-05-11T12:10:49.981894+0000","last_unstale":"2024-05-11T12:10:50.831100+0000","last_undegraded":"2024-05-11T12:10:50.831100+0000","last_fullsized":"2024-05-11T12:10:50.831100+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:45:08.936896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.056935+0000","last_change":"2024-05-11T12:10:49.982163+0000","last_active":"2024-05-11T12:10:53.056935+0000","last_peered":"2024-05-11T12:10:53.056935+0000","last_clean":"2024-05-11T12:10:53.056935+0000","last_became_active":"2024-05-11T12:10:49.981202+0000","last_became_peered":"2024-05-11T12:10:49.981202+0000","last_unstale":"2024-05-11T12:10:53.056935+0000","last_undegraded":"2024-05-11T12:10:53.056935+0000","last_fullsized":"2024-05-11T12:10:53.056935+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:36:11.661440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.085018+0000","last_change":"2024-05-11T12:10:45.772477+0000","last_active":"2024-05-11T12:10:53.085018+0000","last_peered":"2024-05-11T12:10:53.085018+0000","last_clean":"2024-05-11T12:10:53.085018+0000","last_became_active":"2024-05-11T12:10:45.772053+0000","last_became_peered":"2024-05-11T12:10:45.772053+0000","last_unstale":"2024-05-11T12:10:53.085018+0000","last_undegraded":"2024-05-11T12:10:53.085018+0000","last_fullsized":"2024-05-11T12:10:53.085018+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:00:49.028293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.832812+0000","last_change":"2024-05-11T12:10:44.782828+0000","last_active":"2024-05-11T12:10:50.832812+0000","last_peered":"2024-05-11T12:10:50.832812+0000","last_clean":"2024-05-11T12:10:50.832812+0000","last_became_active":"2024-05-11T12:10:44.782412+0000","last_became_peered":"2024-05-11T12:10:44.782412+0000","last_unstale":"2024-05-11T12:10:50.832812+0000","last_undegraded":"2024-05-11T12:10:50.832812+0000","last_fullsized":"2024-05-11T12:10:50.832812+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:39:50.229692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.836608+0000","last_change":"2024-05-11T12:10:49.849632+0000","last_active":"2024-05-11T12:10:50.836608+0000","last_peered":"2024-05-11T12:10:50.836608+0000","last_clean":"2024-05-11T12:10:50.836608+0000","last_became_active":"2024-05-11T12:10:49.849221+0000","last_became_peered":"2024-05-11T12:10:49.849221+0000","last_unstale":"2024-05-11T12:10:50.836608+0000","last_undegraded":"2024-05-11T12:10:50.836608+0000","last_fullsized":"2024-05-11T12:10:50.836608+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:02:51.578727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.830326+0000","last_change":"2024-05-11T12:10:45.784432+0000","last_active":"2024-05-11T12:10:50.830326+0000","last_peered":"2024-05-11T12:10:50.830326+0000","last_clean":"2024-05-11T12:10:50.830326+0000","last_became_active":"2024-05-11T12:10:45.784152+0000","last_became_peered":"2024-05-11T12:10:45.784152+0000","last_unstale":"2024-05-11T12:10:50.830326+0000","last_undegraded":"2024-05-11T12:10:50.830326+0000","last_fullsized":"2024-05-11T12:10:50.830326+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:21:38.211953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.831004+0000","last_change":"2024-05-11T12:10:44.775112+0000","last_active":"2024-05-11T12:10:50.831004+0000","last_peered":"2024-05-11T12:10:50.831004+0000","last_clean":"2024-05-11T12:10:50.831004+0000","last_became_active":"2024-05-11T12:10:44.774758+0000","last_became_peered":"2024-05-11T12:10:44.774758+0000","last_unstale":"2024-05-11T12:10:50.831004+0000","last_undegraded":"2024-05-11T12:10:50.831004+0000","last_fullsized":"2024-05-11T12:10:50.831004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:38:50.526921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.835462+0000","last_change":"2024-05-11T12:10:44.778249+0000","last_active":"2024-05-11T12:10:50.835462+0000","last_peered":"2024-05-11T12:10:50.835462+0000","last_clean":"2024-05-11T12:10:50.835462+0000","last_became_active":"2024-05-11T12:10:44.775251+0000","last_became_peered":"2024-05-11T12:10:44.775251+0000","last_unstale":"2024-05-11T12:10:50.835462+0000","last_undegraded":"2024-05-11T12:10:50.835462+0000","last_fullsized":"2024-05-11T12:10:50.835462+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:11:44.935841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.056459+0000","last_change":"2024-05-11T12:10:45.783894+0000","last_active":"2024-05-11T12:10:53.056459+0000","last_peered":"2024-05-11T12:10:53.056459+0000","last_clean":"2024-05-11T12:10:53.056459+0000","last_became_active":"2024-05-11T12:10:45.783452+0000","last_became_peered":"2024-05-11T12:10:45.783452+0000","last_unstale":"2024-05-11T12:10:53.056459+0000","last_undegraded":"2024-05-11T12:10:53.056459+0000","last_fullsized":"2024-05-11T12:10:53.056459+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:09:48.719906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.956355+0000","last_change":"2024-05-11T12:10:49.841006+0000","last_active":"2024-05-11T12:10:50.956355+0000","last_peered":"2024-05-11T12:10:50.956355+0000","last_clean":"2024-05-11T12:10:50.956355+0000","last_became_active":"2024-05-11T12:10:49.840609+0000","last_became_peered":"2024-05-11T12:10:49.840609+0000","last_unstale":"2024-05-11T12:10:50.956355+0000","last_undegraded":"2024-05-11T12:10:50.956355+0000","last_fullsized":"2024-05-11T12:10:50.956355+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:11:33.092215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.831304+0000","last_change":"2024-05-11T12:10:44.769720+0000","last_active":"2024-05-11T12:10:50.831304+0000","last_peered":"2024-05-11T12:10:50.831304+0000","last_clean":"2024-05-11T12:10:50.831304+0000","last_became_active":"2024-05-11T12:10:44.768923+0000","last_became_peered":"2024-05-11T12:10:44.768923+0000","last_unstale":"2024-05-11T12:10:50.831304+0000","last_undegraded":"2024-05-11T12:10:50.831304+0000","last_fullsized":"2024-05-11T12:10:50.831304+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:04:49.439739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.177838+0000","last_change":"2024-05-11T12:10:45.776362+0000","last_active":"2024-05-11T12:10:51.177838+0000","last_peered":"2024-05-11T12:10:51.177838+0000","last_clean":"2024-05-11T12:10:51.177838+0000","last_became_active":"2024-05-11T12:10:45.773081+0000","last_became_peered":"2024-05-11T12:10:45.773081+0000","last_unstale":"2024-05-11T12:10:51.177838+0000","last_undegraded":"2024-05-11T12:10:51.177838+0000","last_fullsized":"2024-05-11T12:10:51.177838+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:25:36.260066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.832413+0000","last_change":"2024-05-11T12:10:49.837438+0000","last_active":"2024-05-11T12:10:50.832413+0000","last_peered":"2024-05-11T12:10:50.832413+0000","last_clean":"2024-05-11T12:10:50.832413+0000","last_became_active":"2024-05-11T12:10:49.836966+0000","last_became_peered":"2024-05-11T12:10:49.836966+0000","last_unstale":"2024-05-11T12:10:50.832413+0000","last_undegraded":"2024-05-11T12:10:50.832413+0000","last_fullsized":"2024-05-11T12:10:50.832413+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:16:26.266475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.832119+0000","last_change":"2024-05-11T12:10:45.792181+0000","last_active":"2024-05-11T12:10:50.832119+0000","last_peered":"2024-05-11T12:10:50.832119+0000","last_clean":"2024-05-11T12:10:50.832119+0000","last_became_active":"2024-05-11T12:10:45.791827+0000","last_became_peered":"2024-05-11T12:10:45.791827+0000","last_unstale":"2024-05-11T12:10:50.832119+0000","last_undegraded":"2024-05-11T12:10:50.832119+0000","last_fullsized":"2024-05-11T12:10:50.832119+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:25:39.934222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.956054+0000","last_change":"2024-05-11T12:10:45.786514+0000","last_active":"2024-05-11T12:10:50.956054+0000","last_peered":"2024-05-11T12:10:50.956054+0000","last_clean":"2024-05-11T12:10:50.956054+0000","last_became_active":"2024-05-11T12:10:45.786170+0000","last_became_peered":"2024-05-11T12:10:45.786170+0000","last_unstale":"2024-05-11T12:10:50.956054+0000","last_undegraded":"2024-05-11T12:10:50.956054+0000","last_fullsized":"2024-05-11T12:10:50.956054+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:07:42.553060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.833366+0000","last_change":"2024-05-11T12:10:45.764626+0000","last_active":"2024-05-11T12:10:50.833366+0000","last_peered":"2024-05-11T12:10:50.833366+0000","last_clean":"2024-05-11T12:10:50.833366+0000","last_became_active":"2024-05-11T12:10:45.764234+0000","last_became_peered":"2024-05-11T12:10:45.764234+0000","last_unstale":"2024-05-11T12:10:50.833366+0000","last_undegraded":"2024-05-11T12:10:50.833366+0000","last_fullsized":"2024-05-11T12:10:50.833366+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:31:24.678347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.056344+0000","last_change":"2024-05-11T12:10:45.787822+0000","last_active":"2024-05-11T12:10:53.056344+0000","last_peered":"2024-05-11T12:10:53.056344+0000","last_clean":"2024-05-11T12:10:53.056344+0000","last_became_active":"2024-05-11T12:10:45.787503+0000","last_became_peered":"2024-05-11T12:10:45.787503+0000","last_unstale":"2024-05-11T12:10:53.056344+0000","last_undegraded":"2024-05-11T12:10:53.056344+0000","last_fullsized":"2024-05-11T12:10:53.056344+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:30:11.665446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.832309+0000","last_change":"2024-05-11T12:10:45.769648+0000","last_active":"2024-05-11T12:10:50.832309+0000","last_peered":"2024-05-11T12:10:50.832309+0000","last_clean":"2024-05-11T12:10:50.832309+0000","last_became_active":"2024-05-11T12:10:45.769168+0000","last_became_peered":"2024-05-11T12:10:45.769168+0000","last_unstale":"2024-05-11T12:10:50.832309+0000","last_undegraded":"2024-05-11T12:10:50.832309+0000","last_fullsized":"2024-05-11T12:10:50.832309+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:40:33.598752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.086161+0000","last_change":"2024-05-11T12:10:45.773070+0000","last_active":"2024-05-11T12:10:53.086161+0000","last_peered":"2024-05-11T12:10:53.086161+0000","last_clean":"2024-05-11T12:10:53.086161+0000","last_became_active":"2024-05-11T12:10:45.772734+0000","last_became_peered":"2024-05-11T12:10:45.772734+0000","last_unstale":"2024-05-11T12:10:53.086161+0000","last_undegraded":"2024-05-11T12:10:53.086161+0000","last_fullsized":"2024-05-11T12:10:53.086161+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:52:36.650847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.836911+0000","last_change":"2024-05-11T12:10:45.776117+0000","last_active":"2024-05-11T12:10:50.836911+0000","last_peered":"2024-05-11T12:10:50.836911+0000","last_clean":"2024-05-11T12:10:50.836911+0000","last_became_active":"2024-05-11T12:10:45.775042+0000","last_became_peered":"2024-05-11T12:10:45.775042+0000","last_unstale":"2024-05-11T12:10:50.836911+0000","last_undegraded":"2024-05-11T12:10:50.836911+0000","last_fullsized":"2024-05-11T12:10:50.836911+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:40:33.598752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.832667+0000","last_change":"2024-05-11T12:10:45.767897+0000","last_active":"2024-05-11T12:10:50.832667+0000","last_peered":"2024-05-11T12:10:50.832667+0000","last_clean":"2024-05-11T12:10:50.832667+0000","last_became_active":"2024-05-11T12:10:45.767619+0000","last_became_peered":"2024-05-11T12:10:45.767619+0000","last_unstale":"2024-05-11T12:10:50.832667+0000","last_undegraded":"2024-05-11T12:10:50.832667+0000","last_fullsized":"2024-05-11T12:10:50.832667+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:27:46.323505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.832683+0000","last_change":"2024-05-11T12:10:45.763701+0000","last_active":"2024-05-11T12:10:50.832683+0000","last_peered":"2024-05-11T12:10:50.832683+0000","last_clean":"2024-05-11T12:10:50.832683+0000","last_became_active":"2024-05-11T12:10:45.763260+0000","last_became_peered":"2024-05-11T12:10:45.763260+0000","last_unstale":"2024-05-11T12:10:50.832683+0000","last_undegraded":"2024-05-11T12:10:50.832683+0000","last_fullsized":"2024-05-11T12:10:50.832683+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:11:17.251971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.830765+0000","last_change":"2024-05-11T12:10:45.767500+0000","last_active":"2024-05-11T12:10:50.830765+0000","last_peered":"2024-05-11T12:10:50.830765+0000","last_clean":"2024-05-11T12:10:50.830765+0000","last_became_active":"2024-05-11T12:10:45.767137+0000","last_became_peered":"2024-05-11T12:10:45.767137+0000","last_unstale":"2024-05-11T12:10:50.830765+0000","last_undegraded":"2024-05-11T12:10:50.830765+0000","last_fullsized":"2024-05-11T12:10:50.830765+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:52:36.650847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.833200+0000","last_change":"2024-05-11T12:10:45.779266+0000","last_active":"2024-05-11T12:10:50.833200+0000","last_peered":"2024-05-11T12:10:50.833200+0000","last_clean":"2024-05-11T12:10:50.833200+0000","last_became_active":"2024-05-11T12:10:45.778968+0000","last_became_peered":"2024-05-11T12:10:45.778968+0000","last_unstale":"2024-05-11T12:10:50.833200+0000","last_undegraded":"2024-05-11T12:10:50.833200+0000","last_fullsized":"2024-05-11T12:10:50.833200+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:40:33.598752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.056067+0000","last_change":"2024-05-11T12:10:45.793850+0000","last_active":"2024-05-11T12:10:53.056067+0000","last_peered":"2024-05-11T12:10:53.056067+0000","last_clean":"2024-05-11T12:10:53.056067+0000","last_became_active":"2024-05-11T12:10:45.793503+0000","last_became_peered":"2024-05-11T12:10:45.793503+0000","last_unstale":"2024-05-11T12:10:53.056067+0000","last_undegraded":"2024-05-11T12:10:53.056067+0000","last_fullsized":"2024-05-11T12:10:53.056067+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:21:38.211953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.836150+0000","last_change":"2024-05-11T12:10:45.776958+0000","last_active":"2024-05-11T12:10:50.836150+0000","last_peered":"2024-05-11T12:10:50.836150+0000","last_clean":"2024-05-11T12:10:50.836150+0000","last_became_active":"2024-05-11T12:10:45.770839+0000","last_became_peered":"2024-05-11T12:10:45.770839+0000","last_unstale":"2024-05-11T12:10:50.836150+0000","last_undegraded":"2024-05-11T12:10:50.836150+0000","last_fullsized":"2024-05-11T12:10:50.836150+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:52:36.650847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.988652+0000","last_change":"2024-05-11T12:10:45.784456+0000","last_active":"2024-05-11T12:10:52.988652+0000","last_peered":"2024-05-11T12:10:52.988652+0000","last_clean":"2024-05-11T12:10:52.988652+0000","last_became_active":"2024-05-11T12:10:45.784149+0000","last_became_peered":"2024-05-11T12:10:45.784149+0000","last_unstale":"2024-05-11T12:10:52.988652+0000","last_undegraded":"2024-05-11T12:10:52.988652+0000","last_fullsized":"2024-05-11T12:10:52.988652+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:36:07.588854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.956434+0000","last_change":"2024-05-11T12:10:45.778083+0000","last_active":"2024-05-11T12:10:50.956434+0000","last_peered":"2024-05-11T12:10:50.956434+0000","last_clean":"2024-05-11T12:10:50.956434+0000","last_became_active":"2024-05-11T12:10:45.777758+0000","last_became_peered":"2024-05-11T12:10:45.777758+0000","last_unstale":"2024-05-11T12:10:50.956434+0000","last_undegraded":"2024-05-11T12:10:50.956434+0000","last_fullsized":"2024-05-11T12:10:50.956434+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:38:01.288328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.838005+0000","last_change":"2024-05-11T12:10:45.780997+0000","last_active":"2024-05-11T12:10:50.838005+0000","last_peered":"2024-05-11T12:10:50.838005+0000","last_clean":"2024-05-11T12:10:50.838005+0000","last_became_active":"2024-05-11T12:10:45.780744+0000","last_became_peered":"2024-05-11T12:10:45.780744+0000","last_unstale":"2024-05-11T12:10:50.838005+0000","last_undegraded":"2024-05-11T12:10:50.838005+0000","last_fullsized":"2024-05-11T12:10:50.838005+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:03:15.514658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.832468+0000","last_change":"2024-05-11T12:10:45.790012+0000","last_active":"2024-05-11T12:10:50.832468+0000","last_peered":"2024-05-11T12:10:50.832468+0000","last_clean":"2024-05-11T12:10:50.832468+0000","last_became_active":"2024-05-11T12:10:45.786296+0000","last_became_peered":"2024-05-11T12:10:45.786296+0000","last_unstale":"2024-05-11T12:10:50.832468+0000","last_undegraded":"2024-05-11T12:10:50.832468+0000","last_fullsized":"2024-05-11T12:10:50.832468+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:09:48.719906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.085092+0000","last_change":"2024-05-11T12:10:45.766294+0000","last_active":"2024-05-11T12:10:53.085092+0000","last_peered":"2024-05-11T12:10:53.085092+0000","last_clean":"2024-05-11T12:10:53.085092+0000","last_became_active":"2024-05-11T12:10:45.765946+0000","last_became_peered":"2024-05-11T12:10:45.765946+0000","last_unstale":"2024-05-11T12:10:53.085092+0000","last_undegraded":"2024-05-11T12:10:53.085092+0000","last_fullsized":"2024-05-11T12:10:53.085092+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:30:11.665446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.056206+0000","last_change":"2024-05-11T12:10:45.777473+0000","last_active":"2024-05-11T12:10:53.056206+0000","last_peered":"2024-05-11T12:10:53.056206+0000","last_clean":"2024-05-11T12:10:53.056206+0000","last_became_active":"2024-05-11T12:10:45.777140+0000","last_became_peered":"2024-05-11T12:10:45.777140+0000","last_unstale":"2024-05-11T12:10:53.056206+0000","last_undegraded":"2024-05-11T12:10:53.056206+0000","last_fullsized":"2024-05-11T12:10:53.056206+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:12:59.860890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.830966+0000","last_change":"2024-05-11T12:10:45.779099+0000","last_active":"2024-05-11T12:10:50.830966+0000","last_peered":"2024-05-11T12:10:50.830966+0000","last_clean":"2024-05-11T12:10:50.830966+0000","last_became_active":"2024-05-11T12:10:45.767177+0000","last_became_peered":"2024-05-11T12:10:45.767177+0000","last_unstale":"2024-05-11T12:10:50.830966+0000","last_undegraded":"2024-05-11T12:10:50.830966+0000","last_fullsized":"2024-05-11T12:10:50.830966+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:30:11.665446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.178521+0000","last_change":"2024-05-11T12:10:45.781140+0000","last_active":"2024-05-11T12:10:51.178521+0000","last_peered":"2024-05-11T12:10:51.178521+0000","last_clean":"2024-05-11T12:10:51.178521+0000","last_became_active":"2024-05-11T12:10:45.780390+0000","last_became_peered":"2024-05-11T12:10:45.780390+0000","last_unstale":"2024-05-11T12:10:51.178521+0000","last_undegraded":"2024-05-11T12:10:51.178521+0000","last_fullsized":"2024-05-11T12:10:51.178521+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:32:32.325955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.836702+0000","last_change":"2024-05-11T12:10:45.772180+0000","last_active":"2024-05-11T12:10:50.836702+0000","last_peered":"2024-05-11T12:10:50.836702+0000","last_clean":"2024-05-11T12:10:50.836702+0000","last_became_active":"2024-05-11T12:10:45.771785+0000","last_became_peered":"2024-05-11T12:10:45.771785+0000","last_unstale":"2024-05-11T12:10:50.836702+0000","last_undegraded":"2024-05-11T12:10:50.836702+0000","last_fullsized":"2024-05-11T12:10:50.836702+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:27:46.323505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.836913+0000","last_change":"2024-05-11T12:10:45.773903+0000","last_active":"2024-05-11T12:10:50.836913+0000","last_peered":"2024-05-11T12:10:50.836913+0000","last_clean":"2024-05-11T12:10:50.836913+0000","last_became_active":"2024-05-11T12:10:45.773639+0000","last_became_peered":"2024-05-11T12:10:45.773639+0000","last_unstale":"2024-05-11T12:10:50.836913+0000","last_undegraded":"2024-05-11T12:10:50.836913+0000","last_fullsized":"2024-05-11T12:10:50.836913+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:09:48.719906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.988774+0000","last_change":"2024-05-11T12:10:45.794802+0000","last_active":"2024-05-11T12:10:52.988774+0000","last_peered":"2024-05-11T12:10:52.988774+0000","last_clean":"2024-05-11T12:10:52.988774+0000","last_became_active":"2024-05-11T12:10:45.794505+0000","last_became_peered":"2024-05-11T12:10:45.794505+0000","last_unstale":"2024-05-11T12:10:52.988774+0000","last_undegraded":"2024-05-11T12:10:52.988774+0000","last_fullsized":"2024-05-11T12:10:52.988774+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:03:15.514658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.830033+0000","last_change":"2024-05-11T12:10:45.764551+0000","last_active":"2024-05-11T12:10:50.830033+0000","last_peered":"2024-05-11T12:10:50.830033+0000","last_clean":"2024-05-11T12:10:50.830033+0000","last_became_active":"2024-05-11T12:10:45.763049+0000","last_became_peered":"2024-05-11T12:10:45.763049+0000","last_unstale":"2024-05-11T12:10:50.830033+0000","last_undegraded":"2024-05-11T12:10:50.830033+0000","last_fullsized":"2024-05-11T12:10:50.830033+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:11:17.251971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.986814+0000","last_change":"2024-05-11T12:10:45.785129+0000","last_active":"2024-05-11T12:10:52.986814+0000","last_peered":"2024-05-11T12:10:52.986814+0000","last_clean":"2024-05-11T12:10:52.986814+0000","last_became_active":"2024-05-11T12:10:45.784827+0000","last_became_peered":"2024-05-11T12:10:45.784827+0000","last_unstale":"2024-05-11T12:10:52.986814+0000","last_undegraded":"2024-05-11T12:10:52.986814+0000","last_fullsized":"2024-05-11T12:10:52.986814+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:40:33.598752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.833132+0000","last_change":"2024-05-11T12:10:45.785406+0000","last_active":"2024-05-11T12:10:50.833132+0000","last_peered":"2024-05-11T12:10:50.833132+0000","last_clean":"2024-05-11T12:10:50.833132+0000","last_became_active":"2024-05-11T12:10:45.785083+0000","last_became_peered":"2024-05-11T12:10:45.785083+0000","last_unstale":"2024-05-11T12:10:50.833132+0000","last_undegraded":"2024-05-11T12:10:50.833132+0000","last_fullsized":"2024-05-11T12:10:50.833132+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:15:55.637133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.987595+0000","last_change":"2024-05-11T12:10:45.779636+0000","last_active":"2024-05-11T12:10:52.987595+0000","last_peered":"2024-05-11T12:10:52.987595+0000","last_clean":"2024-05-11T12:10:52.987595+0000","last_became_active":"2024-05-11T12:10:45.779327+0000","last_became_peered":"2024-05-11T12:10:45.779327+0000","last_unstale":"2024-05-11T12:10:52.987595+0000","last_undegraded":"2024-05-11T12:10:52.987595+0000","last_fullsized":"2024-05-11T12:10:52.987595+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:11:17.251971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.834940+0000","last_change":"2024-05-11T12:10:45.782057+0000","last_active":"2024-05-11T12:10:50.834940+0000","last_peered":"2024-05-11T12:10:50.834940+0000","last_clean":"2024-05-11T12:10:50.834940+0000","last_became_active":"2024-05-11T12:10:45.781008+0000","last_became_peered":"2024-05-11T12:10:45.781008+0000","last_unstale":"2024-05-11T12:10:50.834940+0000","last_undegraded":"2024-05-11T12:10:50.834940+0000","last_fullsized":"2024-05-11T12:10:50.834940+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:01:30.356378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.056602+0000","last_change":"2024-05-11T12:10:45.783937+0000","last_active":"2024-05-11T12:10:53.056602+0000","last_peered":"2024-05-11T12:10:53.056602+0000","last_clean":"2024-05-11T12:10:53.056602+0000","last_became_active":"2024-05-11T12:10:45.783553+0000","last_became_peered":"2024-05-11T12:10:45.783553+0000","last_unstale":"2024-05-11T12:10:53.056602+0000","last_undegraded":"2024-05-11T12:10:53.056602+0000","last_fullsized":"2024-05-11T12:10:53.056602+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:27:46.323505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.086881+0000","last_change":"2024-05-11T12:10:45.786860+0000","last_active":"2024-05-11T12:10:53.086881+0000","last_peered":"2024-05-11T12:10:53.086881+0000","last_clean":"2024-05-11T12:10:53.086881+0000","last_became_active":"2024-05-11T12:10:45.785741+0000","last_became_peered":"2024-05-11T12:10:45.785741+0000","last_unstale":"2024-05-11T12:10:53.086881+0000","last_undegraded":"2024-05-11T12:10:53.086881+0000","last_fullsized":"2024-05-11T12:10:53.086881+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:09:48.719906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.988637+0000","last_change":"2024-05-11T12:10:45.794362+0000","last_active":"2024-05-11T12:10:52.988637+0000","last_peered":"2024-05-11T12:10:52.988637+0000","last_clean":"2024-05-11T12:10:52.988637+0000","last_became_active":"2024-05-11T12:10:45.793845+0000","last_became_peered":"2024-05-11T12:10:45.793845+0000","last_unstale":"2024-05-11T12:10:52.988637+0000","last_undegraded":"2024-05-11T12:10:52.988637+0000","last_fullsized":"2024-05-11T12:10:52.988637+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:02:47.506141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476745,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":876544,"data_stored":700107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476746,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":112999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476745,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":282624,"data_stored":109247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476747,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":290816,"data_stored":109283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509450,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":112999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509451,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":109317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509450,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":884736,"data_stored":700163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542154,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":109157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T12:11:11.378 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T12:11:11.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.618+0000 7f7921c00640 1 Processor -- start 2024-05-11T12:11:11.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.619+0000 7f7921c00640 1 -- start start 2024-05-11T12:11:11.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.620+0000 7f7921c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f791c157750 0x7f791c15d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:11.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.620+0000 7f7921c00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f791c0b3ea0 0x7f791c161490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:11.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.620+0000 7f7921c00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f791c15dc80 con 0x7f791c1619d0 2024-05-11T12:11:11.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.620+0000 7f7921c00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f791c161000 con 0x7f791c0b3ea0 2024-05-11T12:11:11.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.620+0000 7f7921c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f791c15df30 con 0x7f791c157750 2024-05-11T12:11:11.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.620+0000 7f791a000640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f791c0b3ea0 0x7f791c161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:11.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.621+0000 7f791a000640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f791c0b3ea0 0x7f791c161490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41418/0 (socket says 172.21.15.167:41418) 2024-05-11T12:11:11.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.621+0000 7f791a000640 1 -- 172.21.15.167:0/2267938599 learned_addr learned my addr 172.21.15.167:0/2267938599 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:11.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.621+0000 7f791b400640 1 --1- 172.21.15.167:0/2267938599 >> v1:172.21.15.167:6789/0 conn(0x7f791c1619d0 0x7f791c06d460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49370/0 (socket says 172.21.15.167:49370) 2024-05-11T12:11:11.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.621+0000 7f791aa00640 1 --2- 172.21.15.167:0/2267938599 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f791c157750 0x7f791c15d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:11.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.621+0000 7f791a000640 1 -- 172.21.15.167:0/2267938599 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f791c157750 msgr2=0x7f791c15d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:11.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.621+0000 7f791a000640 1 --2- 172.21.15.167:0/2267938599 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f791c157750 0x7f791c15d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:11.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.621+0000 7f791a000640 1 -- 172.21.15.167:0/2267938599 >> v1:172.21.15.167:6789/0 conn(0x7f791c1619d0 legacy=0x7f791c06d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:11.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.621+0000 7f791a000640 1 -- 172.21.15.167:0/2267938599 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f791c15e210 con 0x7f791c0b3ea0 2024-05-11T12:11:11.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.621+0000 7f7919600640 1 -- 172.21.15.167:0/2267938599 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2572592769 0 0) 0x7f791c15dc80 con 0x7f791c1619d0 2024-05-11T12:11:11.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.622+0000 7f791a000640 1 --2- 172.21.15.167:0/2267938599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f791c0b3ea0 0x7f791c161490 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7908009870 tx=0x7f790802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdf209a462e95aca server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:11.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.622+0000 7f7919600640 1 -- 172.21.15.167:0/2267938599 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7908037070 con 0x7f791c0b3ea0 2024-05-11T12:11:11.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.622+0000 7f7919600640 1 -- 172.21.15.167:0/2267938599 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7908032440 con 0x7f791c0b3ea0 2024-05-11T12:11:11.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.622+0000 7f7919600640 1 -- 172.21.15.167:0/2267938599 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7908032810 con 0x7f791c0b3ea0 2024-05-11T12:11:11.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.623+0000 7f7921c00640 1 -- 172.21.15.167:0/2267938599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f791c0b3ea0 msgr2=0x7f791c161490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:11.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.623+0000 7f7921c00640 1 --2- 172.21.15.167:0/2267938599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f791c0b3ea0 0x7f791c161490 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7908009870 tx=0x7f790802a270 comp rx=0 tx=0).stop 2024-05-11T12:11:11.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.623+0000 7f7921c00640 1 -- 172.21.15.167:0/2267938599 shutdown_connections 2024-05-11T12:11:11.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.623+0000 7f7921c00640 1 --2- 172.21.15.167:0/2267938599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f791c0b3ea0 0x7f791c161490 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:11.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.623+0000 7f7921c00640 1 --2- 172.21.15.167:0/2267938599 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f791c157750 0x7f791c15d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:11.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.623+0000 7f7921c00640 1 -- 172.21.15.167:0/2267938599 >> 172.21.15.167:0/2267938599 conn(0x7f791c0cd8f0 msgr2=0x7f791c0cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:11.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.623+0000 7f7921c00640 1 -- 172.21.15.167:0/2267938599 shutdown_connections 2024-05-11T12:11:11.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.623+0000 7f7921c00640 1 -- 172.21.15.167:0/2267938599 wait complete. 2024-05-11T12:11:11.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.625+0000 7f7921c00640 1 Processor -- start 2024-05-11T12:11:11.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.625+0000 7f7921c00640 1 -- start start 2024-05-11T12:11:11.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.626+0000 7f7921c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f791c0b3ea0 0x7f791c1f7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:11.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.626+0000 7f7921c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f791c157750 0x7f791c0af870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:11.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.626+0000 7f7921c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f791c1619d0 0x7f791c0afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:11.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.626+0000 7f7921c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f791c06db70 con 0x7f791c157750 2024-05-11T12:11:11.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.626+0000 7f7921c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f791c157b30 con 0x7f791c0b3ea0 2024-05-11T12:11:11.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.626+0000 7f7921c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f791c06dcf0 con 0x7f791c1619d0 2024-05-11T12:11:11.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.626+0000 7f791a000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f791c157750 0x7f791c0af870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:11.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.626+0000 7f791a000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f791c157750 0x7f791c0af870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41420/0 (socket says 172.21.15.167:41420) 2024-05-11T12:11:11.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.626+0000 7f791a000640 1 -- 172.21.15.167:0/2753751209 learned_addr learned my addr 172.21.15.167:0/2753751209 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:11.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.627+0000 7f791aa00640 1 --2- 172.21.15.167:0/2753751209 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f791c0b3ea0 0x7f791c1f7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:11.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.627+0000 7f791b400640 1 --2- 172.21.15.167:0/2753751209 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f791c1619d0 0x7f791c0afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:11.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.627+0000 7f791a000640 1 -- 172.21.15.167:0/2753751209 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f791c1619d0 msgr2=0x7f791c0afdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:11.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.627+0000 7f791a000640 1 --2- 172.21.15.167:0/2753751209 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f791c1619d0 0x7f791c0afdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:11.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.627+0000 7f791a000640 1 -- 172.21.15.167:0/2753751209 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f791c0b3ea0 msgr2=0x7f791c1f7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:11.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.627+0000 7f791a000640 1 --2- 172.21.15.167:0/2753751209 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f791c0b3ea0 0x7f791c1f7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:11.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.627+0000 7f791a000640 1 -- 172.21.15.167:0/2753751209 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f791c15a9e0 con 0x7f791c157750 2024-05-11T12:11:11.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.627+0000 7f791aa00640 1 --2- 172.21.15.167:0/2753751209 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f791c0b3ea0 0x7f791c1f7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:11.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.627+0000 7f791a000640 1 --2- 172.21.15.167:0/2753751209 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f791c157750 0x7f791c0af870 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7908009870 tx=0x7f79080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:11.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.627+0000 7f7912a00640 1 -- 172.21.15.167:0/2753751209 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7908041020 con 0x7f791c157750 2024-05-11T12:11:11.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.628+0000 7f7912a00640 1 -- 172.21.15.167:0/2753751209 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79080306c0 con 0x7f791c157750 2024-05-11T12:11:11.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.628+0000 7f7921c00640 1 -- 172.21.15.167:0/2753751209 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f791c15acc0 con 0x7f791c157750 2024-05-11T12:11:11.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.628+0000 7f7912a00640 1 -- 172.21.15.167:0/2753751209 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7908037040 con 0x7f791c157750 2024-05-11T12:11:11.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.628+0000 7f7921c00640 1 -- 172.21.15.167:0/2753751209 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f791c15b4e0 con 0x7f791c157750 2024-05-11T12:11:11.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.629+0000 7f7921c00640 1 -- 172.21.15.167:0/2753751209 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f791c069590 con 0x7f791c157750 2024-05-11T12:11:11.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.630+0000 7f7912a00640 1 -- 172.21.15.167:0/2753751209 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7908044020 con 0x7f791c157750 2024-05-11T12:11:11.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.631+0000 7f7912a00640 1 --2- 172.21.15.167:0/2753751209 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f78fc0762d0 0x7f78fc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:11.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.632+0000 7f7912a00640 1 -- 172.21.15.167:0/2753751209 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f79080b8240 con 0x7f791c157750 2024-05-11T12:11:11.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.632+0000 7f791aa00640 1 --2- 172.21.15.167:0/2753751209 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f78fc0762d0 0x7f78fc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:11.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.633+0000 7f791aa00640 1 --2- 172.21.15.167:0/2753751209 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f78fc0762d0 0x7f78fc078790 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f790c006fd0 tx=0x7f790c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:11.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.636+0000 7f7912a00640 1 -- 172.21.15.167:0/2753751209 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f790804b030 con 0x7f791c157750 2024-05-11T12:11:11.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.843+0000 7f7921c00640 1 -- 172.21.15.167:0/2753751209 --> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f791c1604c0 con 0x7f78fc0762d0 2024-05-11T12:11:11.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.855+0000 7f7912a00640 1 -- 172.21.15.167:0/2753751209 <== mgr.4106 v2:172.21.15.167:6832/578352559 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332527 (secure 0 0 0) 0x7f791c1604c0 con 0x7f78fc0762d0 2024-05-11T12:11:11.859 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:11:11.862 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2024-05-11T12:11:11.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.863+0000 7f7921c00640 1 -- 172.21.15.167:0/2753751209 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f78fc0762d0 msgr2=0x7f78fc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:11.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.863+0000 7f7921c00640 1 --2- 172.21.15.167:0/2753751209 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f78fc0762d0 0x7f78fc078790 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f790c006fd0 tx=0x7f790c008040 comp rx=0 tx=0).stop 2024-05-11T12:11:11.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.863+0000 7f7921c00640 1 -- 172.21.15.167:0/2753751209 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f791c157750 msgr2=0x7f791c0af870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:11.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.863+0000 7f7921c00640 1 --2- 172.21.15.167:0/2753751209 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f791c157750 0x7f791c0af870 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7908009870 tx=0x7f79080046c0 comp rx=0 tx=0).stop 2024-05-11T12:11:11.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.863+0000 7f7921c00640 1 -- 172.21.15.167:0/2753751209 shutdown_connections 2024-05-11T12:11:11.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.863+0000 7f7921c00640 1 --2- 172.21.15.167:0/2753751209 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f791c1619d0 0x7f791c0afdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:11.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.863+0000 7f7921c00640 1 --2- 172.21.15.167:0/2753751209 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f791c157750 0x7f791c0af870 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:11.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.863+0000 7f7921c00640 1 --2- 172.21.15.167:0/2753751209 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f78fc0762d0 0x7f78fc078790 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:11.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.863+0000 7f7921c00640 1 --2- 172.21.15.167:0/2753751209 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f791c0b3ea0 0x7f791c1f7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:11.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.863+0000 7f7921c00640 1 -- 172.21.15.167:0/2753751209 >> 172.21.15.167:0/2753751209 conn(0x7f791c0cd8f0 msgr2=0x7f791c0cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:11.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.863+0000 7f7921c00640 1 -- 172.21.15.167:0/2753751209 shutdown_connections 2024-05-11T12:11:11.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:11.863+0000 7f7921c00640 1 -- 172.21.15.167:0/2753751209 wait complete. 2024-05-11T12:11:11.883 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2024-05-11T12:11:11.239330+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":150,"num_write_kb":1647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219560,"kb_used_data":3400,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755160,"statfs":{"total":773094113280,"available":772869283840,"internally_reserved":0,"allocated":3481600,"data_stored":2063272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005612"},"pg_stats":[{"pgid":"2.1f","version":"22'4","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.057257+0000","last_change":"2024-05-11T12:10:44.771802+0000","last_active":"2024-05-11T12:10:53.057257+0000","last_peered":"2024-05-11T12:10:53.057257+0000","last_clean":"2024-05-11T12:10:53.057257+0000","last_became_active":"2024-05-11T12:10:44.771511+0000","last_became_peered":"2024-05-11T12:10:44.771511+0000","last_unstale":"2024-05-11T12:10:53.057257+0000","last_undegraded":"2024-05-11T12:10:53.057257+0000","last_fullsized":"2024-05-11T12:10:53.057257+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:38:50.526921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.832716+0000","last_change":"2024-05-11T12:10:45.765221+0000","last_active":"2024-05-11T12:10:50.832716+0000","last_peered":"2024-05-11T12:10:50.832716+0000","last_clean":"2024-05-11T12:10:50.832716+0000","last_became_active":"2024-05-11T12:10:45.764595+0000","last_became_peered":"2024-05-11T12:10:45.764595+0000","last_unstale":"2024-05-11T12:10:50.832716+0000","last_undegraded":"2024-05-11T12:10:50.832716+0000","last_fullsized":"2024-05-11T12:10:50.832716+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:01:30.356378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.830235+0000","last_change":"2024-05-11T12:10:49.979783+0000","last_active":"2024-05-11T12:10:50.830235+0000","last_peered":"2024-05-11T12:10:50.830235+0000","last_clean":"2024-05-11T12:10:50.830235+0000","last_became_active":"2024-05-11T12:10:49.977927+0000","last_became_peered":"2024-05-11T12:10:49.977927+0000","last_unstale":"2024-05-11T12:10:50.830235+0000","last_undegraded":"2024-05-11T12:10:50.830235+0000","last_fullsized":"2024-05-11T12:10:50.830235+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:31:59.631048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.833109+0000","last_change":"2024-05-11T12:10:44.769768+0000","last_active":"2024-05-11T12:10:50.833109+0000","last_peered":"2024-05-11T12:10:50.833109+0000","last_clean":"2024-05-11T12:10:50.833109+0000","last_became_active":"2024-05-11T12:10:44.769498+0000","last_became_peered":"2024-05-11T12:10:44.769498+0000","last_unstale":"2024-05-11T12:10:50.833109+0000","last_undegraded":"2024-05-11T12:10:50.833109+0000","last_fullsized":"2024-05-11T12:10:50.833109+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:14:58.421726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.056999+0000","last_change":"2024-05-11T12:10:45.791108+0000","last_active":"2024-05-11T12:10:53.056999+0000","last_peered":"2024-05-11T12:10:53.056999+0000","last_clean":"2024-05-11T12:10:53.056999+0000","last_became_active":"2024-05-11T12:10:45.790807+0000","last_became_peered":"2024-05-11T12:10:45.790807+0000","last_unstale":"2024-05-11T12:10:53.056999+0000","last_undegraded":"2024-05-11T12:10:53.056999+0000","last_fullsized":"2024-05-11T12:10:53.056999+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:52:36.650847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.177575+0000","last_change":"2024-05-11T12:10:49.843347+0000","last_active":"2024-05-11T12:10:51.177575+0000","last_peered":"2024-05-11T12:10:51.177575+0000","last_clean":"2024-05-11T12:10:51.177575+0000","last_became_active":"2024-05-11T12:10:49.840522+0000","last_became_peered":"2024-05-11T12:10:49.840522+0000","last_unstale":"2024-05-11T12:10:51.177575+0000","last_undegraded":"2024-05-11T12:10:51.177575+0000","last_fullsized":"2024-05-11T12:10:51.177575+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:09:52.792492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.986041+0000","last_change":"2024-05-11T12:10:44.788150+0000","last_active":"2024-05-11T12:10:52.986041+0000","last_peered":"2024-05-11T12:10:52.986041+0000","last_clean":"2024-05-11T12:10:52.986041+0000","last_became_active":"2024-05-11T12:10:44.787827+0000","last_became_peered":"2024-05-11T12:10:44.787827+0000","last_unstale":"2024-05-11T12:10:52.986041+0000","last_undegraded":"2024-05-11T12:10:52.986041+0000","last_fullsized":"2024-05-11T12:10:52.986041+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:50:29.143515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.834938+0000","last_change":"2024-05-11T12:10:45.769675+0000","last_active":"2024-05-11T12:10:50.834938+0000","last_peered":"2024-05-11T12:10:50.834938+0000","last_clean":"2024-05-11T12:10:50.834938+0000","last_became_active":"2024-05-11T12:10:45.766444+0000","last_became_peered":"2024-05-11T12:10:45.766444+0000","last_unstale":"2024-05-11T12:10:50.834938+0000","last_undegraded":"2024-05-11T12:10:50.834938+0000","last_fullsized":"2024-05-11T12:10:50.834938+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:02:47.506141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.177937+0000","last_change":"2024-05-11T12:10:49.843249+0000","last_active":"2024-05-11T12:10:51.177937+0000","last_peered":"2024-05-11T12:10:51.177937+0000","last_clean":"2024-05-11T12:10:51.177937+0000","last_became_active":"2024-05-11T12:10:49.841714+0000","last_became_peered":"2024-05-11T12:10:49.841714+0000","last_unstale":"2024-05-11T12:10:51.177937+0000","last_undegraded":"2024-05-11T12:10:51.177937+0000","last_fullsized":"2024-05-11T12:10:51.177937+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:15:59.709719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.986035+0000","last_change":"2024-05-11T12:10:44.783078+0000","last_active":"2024-05-11T12:10:52.986035+0000","last_peered":"2024-05-11T12:10:52.986035+0000","last_clean":"2024-05-11T12:10:52.986035+0000","last_became_active":"2024-05-11T12:10:44.782750+0000","last_became_peered":"2024-05-11T12:10:44.782750+0000","last_unstale":"2024-05-11T12:10:52.986035+0000","last_undegraded":"2024-05-11T12:10:52.986035+0000","last_fullsized":"2024-05-11T12:10:52.986035+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:32:31.312066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.836129+0000","last_change":"2024-05-11T12:10:45.771583+0000","last_active":"2024-05-11T12:10:50.836129+0000","last_peered":"2024-05-11T12:10:50.836129+0000","last_clean":"2024-05-11T12:10:50.836129+0000","last_became_active":"2024-05-11T12:10:45.766864+0000","last_became_peered":"2024-05-11T12:10:45.766864+0000","last_unstale":"2024-05-11T12:10:50.836129+0000","last_undegraded":"2024-05-11T12:10:50.836129+0000","last_fullsized":"2024-05-11T12:10:50.836129+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:29:58.902743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.177649+0000","last_change":"2024-05-11T12:10:49.833489+0000","last_active":"2024-05-11T12:10:51.177649+0000","last_peered":"2024-05-11T12:10:51.177649+0000","last_clean":"2024-05-11T12:10:51.177649+0000","last_became_active":"2024-05-11T12:10:49.833030+0000","last_became_peered":"2024-05-11T12:10:49.833030+0000","last_unstale":"2024-05-11T12:10:51.177649+0000","last_undegraded":"2024-05-11T12:10:51.177649+0000","last_fullsized":"2024-05-11T12:10:51.177649+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:40:37.671338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.833888+0000","last_change":"2024-05-11T12:10:45.777692+0000","last_active":"2024-05-11T12:10:50.833888+0000","last_peered":"2024-05-11T12:10:50.833888+0000","last_clean":"2024-05-11T12:10:50.833888+0000","last_became_active":"2024-05-11T12:10:45.777388+0000","last_became_peered":"2024-05-11T12:10:45.777388+0000","last_unstale":"2024-05-11T12:10:50.833888+0000","last_undegraded":"2024-05-11T12:10:50.833888+0000","last_fullsized":"2024-05-11T12:10:50.833888+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:30:11.665446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.986035+0000","last_change":"2024-05-11T12:10:44.780645+0000","last_active":"2024-05-11T12:10:52.986035+0000","last_peered":"2024-05-11T12:10:52.986035+0000","last_clean":"2024-05-11T12:10:52.986035+0000","last_became_active":"2024-05-11T12:10:44.780371+0000","last_became_peered":"2024-05-11T12:10:44.780371+0000","last_unstale":"2024-05-11T12:10:52.986035+0000","last_undegraded":"2024-05-11T12:10:52.986035+0000","last_fullsized":"2024-05-11T12:10:52.986035+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:46:19.349691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.178802+0000","last_change":"2024-05-11T12:10:49.842333+0000","last_active":"2024-05-11T12:10:51.178802+0000","last_peered":"2024-05-11T12:10:51.178802+0000","last_clean":"2024-05-11T12:10:51.178802+0000","last_became_active":"2024-05-11T12:10:49.841805+0000","last_became_peered":"2024-05-11T12:10:49.841805+0000","last_unstale":"2024-05-11T12:10:51.178802+0000","last_undegraded":"2024-05-11T12:10:51.178802+0000","last_fullsized":"2024-05-11T12:10:51.178802+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:30:02.975329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.831265+0000","last_change":"2024-05-11T12:10:45.785101+0000","last_active":"2024-05-11T12:10:50.831265+0000","last_peered":"2024-05-11T12:10:50.831265+0000","last_clean":"2024-05-11T12:10:50.831265+0000","last_became_active":"2024-05-11T12:10:45.784830+0000","last_became_peered":"2024-05-11T12:10:45.784830+0000","last_unstale":"2024-05-11T12:10:50.831265+0000","last_undegraded":"2024-05-11T12:10:50.831265+0000","last_fullsized":"2024-05-11T12:10:50.831265+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:15:55.637133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.084641+0000","last_change":"2024-05-11T12:10:44.762965+0000","last_active":"2024-05-11T12:10:53.084641+0000","last_peered":"2024-05-11T12:10:53.084641+0000","last_clean":"2024-05-11T12:10:53.084641+0000","last_became_active":"2024-05-11T12:10:44.761074+0000","last_became_peered":"2024-05-11T12:10:44.761074+0000","last_unstale":"2024-05-11T12:10:53.084641+0000","last_undegraded":"2024-05-11T12:10:53.084641+0000","last_fullsized":"2024-05-11T12:10:53.084641+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:11:44.935841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.178724+0000","last_change":"2024-05-11T12:10:49.842014+0000","last_active":"2024-05-11T12:10:51.178724+0000","last_peered":"2024-05-11T12:10:51.178724+0000","last_clean":"2024-05-11T12:10:51.178724+0000","last_became_active":"2024-05-11T12:10:49.840640+0000","last_became_peered":"2024-05-11T12:10:49.840640+0000","last_unstale":"2024-05-11T12:10:51.178724+0000","last_undegraded":"2024-05-11T12:10:51.178724+0000","last_fullsized":"2024-05-11T12:10:51.178724+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:31:28.750933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.830617+0000","last_change":"2024-05-11T12:10:45.767414+0000","last_active":"2024-05-11T12:10:50.830617+0000","last_peered":"2024-05-11T12:10:50.830617+0000","last_clean":"2024-05-11T12:10:50.830617+0000","last_became_active":"2024-05-11T12:10:45.765560+0000","last_became_peered":"2024-05-11T12:10:45.765560+0000","last_unstale":"2024-05-11T12:10:50.830617+0000","last_undegraded":"2024-05-11T12:10:50.830617+0000","last_fullsized":"2024-05-11T12:10:50.830617+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:25:39.934222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.085197+0000","last_change":"2024-05-11T12:10:44.764974+0000","last_active":"2024-05-11T12:10:53.085197+0000","last_peered":"2024-05-11T12:10:53.085197+0000","last_clean":"2024-05-11T12:10:53.085197+0000","last_became_active":"2024-05-11T12:10:44.764692+0000","last_became_peered":"2024-05-11T12:10:44.764692+0000","last_unstale":"2024-05-11T12:10:53.085197+0000","last_undegraded":"2024-05-11T12:10:53.085197+0000","last_fullsized":"2024-05-11T12:10:53.085197+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:27:48.475653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.086845+0000","last_change":"2024-05-11T12:10:49.982086+0000","last_active":"2024-05-11T12:10:53.086845+0000","last_peered":"2024-05-11T12:10:53.086845+0000","last_clean":"2024-05-11T12:10:53.086845+0000","last_became_active":"2024-05-11T12:10:49.981499+0000","last_became_peered":"2024-05-11T12:10:49.981499+0000","last_unstale":"2024-05-11T12:10:53.086845+0000","last_undegraded":"2024-05-11T12:10:53.086845+0000","last_fullsized":"2024-05-11T12:10:53.086845+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:36:11.661440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.178118+0000","last_change":"2024-05-11T12:10:45.780179+0000","last_active":"2024-05-11T12:10:51.178118+0000","last_peered":"2024-05-11T12:10:51.178118+0000","last_clean":"2024-05-11T12:10:51.178118+0000","last_became_active":"2024-05-11T12:10:45.777747+0000","last_became_peered":"2024-05-11T12:10:45.777747+0000","last_unstale":"2024-05-11T12:10:51.178118+0000","last_undegraded":"2024-05-11T12:10:51.178118+0000","last_fullsized":"2024-05-11T12:10:51.178118+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:39:51.243581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.986553+0000","last_change":"2024-05-11T12:10:44.775537+0000","last_active":"2024-05-11T12:10:52.986553+0000","last_peered":"2024-05-11T12:10:52.986553+0000","last_clean":"2024-05-11T12:10:52.986553+0000","last_became_active":"2024-05-11T12:10:44.775159+0000","last_became_peered":"2024-05-11T12:10:44.775159+0000","last_unstale":"2024-05-11T12:10:52.986553+0000","last_undegraded":"2024-05-11T12:10:52.986553+0000","last_fullsized":"2024-05-11T12:10:52.986553+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:39:50.229692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.179022+0000","last_change":"2024-05-11T12:10:49.847192+0000","last_active":"2024-05-11T12:10:51.179022+0000","last_peered":"2024-05-11T12:10:51.179022+0000","last_clean":"2024-05-11T12:10:51.179022+0000","last_became_active":"2024-05-11T12:10:49.846655+0000","last_became_peered":"2024-05-11T12:10:49.846655+0000","last_unstale":"2024-05-11T12:10:51.179022+0000","last_undegraded":"2024-05-11T12:10:51.179022+0000","last_fullsized":"2024-05-11T12:10:51.179022+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:52:55.215724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.057239+0000","last_change":"2024-05-11T12:10:45.788180+0000","last_active":"2024-05-11T12:10:53.057239+0000","last_peered":"2024-05-11T12:10:53.057239+0000","last_clean":"2024-05-11T12:10:53.057239+0000","last_became_active":"2024-05-11T12:10:45.787883+0000","last_became_peered":"2024-05-11T12:10:45.787883+0000","last_unstale":"2024-05-11T12:10:53.057239+0000","last_undegraded":"2024-05-11T12:10:53.057239+0000","last_fullsized":"2024-05-11T12:10:53.057239+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:00:49.028293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.837705+0000","last_change":"2024-05-11T12:10:44.783756+0000","last_active":"2024-05-11T12:10:50.837705+0000","last_peered":"2024-05-11T12:10:50.837705+0000","last_clean":"2024-05-11T12:10:50.837705+0000","last_became_active":"2024-05-11T12:10:44.779442+0000","last_became_peered":"2024-05-11T12:10:44.779442+0000","last_unstale":"2024-05-11T12:10:50.837705+0000","last_undegraded":"2024-05-11T12:10:50.837705+0000","last_fullsized":"2024-05-11T12:10:50.837705+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:12:58.847001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.177959+0000","last_change":"2024-05-11T12:10:49.989122+0000","last_active":"2024-05-11T12:10:51.177959+0000","last_peered":"2024-05-11T12:10:51.177959+0000","last_clean":"2024-05-11T12:10:51.177959+0000","last_became_active":"2024-05-11T12:10:49.988059+0000","last_became_peered":"2024-05-11T12:10:49.988059+0000","last_unstale":"2024-05-11T12:10:51.177959+0000","last_undegraded":"2024-05-11T12:10:51.177959+0000","last_fullsized":"2024-05-11T12:10:51.177959+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:11:21.324557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.832784+0000","last_change":"2024-05-11T12:10:45.780311+0000","last_active":"2024-05-11T12:10:50.832784+0000","last_peered":"2024-05-11T12:10:50.832784+0000","last_clean":"2024-05-11T12:10:50.832784+0000","last_became_active":"2024-05-11T12:10:45.779912+0000","last_became_peered":"2024-05-11T12:10:45.779912+0000","last_unstale":"2024-05-11T12:10:50.832784+0000","last_undegraded":"2024-05-11T12:10:50.832784+0000","last_fullsized":"2024-05-11T12:10:50.832784+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:09:48.719906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.084558+0000","last_change":"2024-05-11T12:10:44.776437+0000","last_active":"2024-05-11T12:10:53.084558+0000","last_peered":"2024-05-11T12:10:53.084558+0000","last_clean":"2024-05-11T12:10:53.084558+0000","last_became_active":"2024-05-11T12:10:44.775585+0000","last_became_peered":"2024-05-11T12:10:44.775585+0000","last_unstale":"2024-05-11T12:10:53.084558+0000","last_undegraded":"2024-05-11T12:10:53.084558+0000","last_fullsized":"2024-05-11T12:10:53.084558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:32:31.312066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.830010+0000","last_change":"2024-05-11T12:10:49.980738+0000","last_active":"2024-05-11T12:10:50.830010+0000","last_peered":"2024-05-11T12:10:50.830010+0000","last_clean":"2024-05-11T12:10:50.830010+0000","last_became_active":"2024-05-11T12:10:49.977869+0000","last_became_peered":"2024-05-11T12:10:49.977869+0000","last_unstale":"2024-05-11T12:10:50.830010+0000","last_undegraded":"2024-05-11T12:10:50.830010+0000","last_fullsized":"2024-05-11T12:10:50.830010+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:30:17.395939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.086118+0000","last_change":"2024-05-11T12:10:45.773559+0000","last_active":"2024-05-11T12:10:53.086118+0000","last_peered":"2024-05-11T12:10:53.086118+0000","last_clean":"2024-05-11T12:10:53.086118+0000","last_became_active":"2024-05-11T12:10:45.773152+0000","last_became_peered":"2024-05-11T12:10:45.773152+0000","last_unstale":"2024-05-11T12:10:53.086118+0000","last_undegraded":"2024-05-11T12:10:53.086118+0000","last_fullsized":"2024-05-11T12:10:53.086118+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:27:46.323505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.988658+0000","last_change":"2024-05-11T12:10:44.779798+0000","last_active":"2024-05-11T12:10:52.988658+0000","last_peered":"2024-05-11T12:10:52.988658+0000","last_clean":"2024-05-11T12:10:52.988658+0000","last_became_active":"2024-05-11T12:10:44.779517+0000","last_became_peered":"2024-05-11T12:10:44.779517+0000","last_unstale":"2024-05-11T12:10:52.988658+0000","last_undegraded":"2024-05-11T12:10:52.988658+0000","last_fullsized":"2024-05-11T12:10:52.988658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:27:48.475653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.055959+0000","last_change":"2024-05-11T12:10:49.977964+0000","last_active":"2024-05-11T12:10:53.055959+0000","last_peered":"2024-05-11T12:10:53.055959+0000","last_clean":"2024-05-11T12:10:53.055959+0000","last_became_active":"2024-05-11T12:10:49.977533+0000","last_became_peered":"2024-05-11T12:10:49.977533+0000","last_unstale":"2024-05-11T12:10:53.055959+0000","last_undegraded":"2024-05-11T12:10:53.055959+0000","last_fullsized":"2024-05-11T12:10:53.055959+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:02:51.578727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.836384+0000","last_change":"2024-05-11T12:10:45.788527+0000","last_active":"2024-05-11T12:10:50.836384+0000","last_peered":"2024-05-11T12:10:50.836384+0000","last_clean":"2024-05-11T12:10:50.836384+0000","last_became_active":"2024-05-11T12:10:45.783632+0000","last_became_peered":"2024-05-11T12:10:45.783632+0000","last_unstale":"2024-05-11T12:10:50.836384+0000","last_undegraded":"2024-05-11T12:10:50.836384+0000","last_fullsized":"2024-05-11T12:10:50.836384+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:52:51.143138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.086299+0000","last_change":"2024-05-11T12:10:44.764150+0000","last_active":"2024-05-11T12:10:53.086299+0000","last_peered":"2024-05-11T12:10:53.086299+0000","last_clean":"2024-05-11T12:10:53.086299+0000","last_became_active":"2024-05-11T12:10:44.763836+0000","last_became_peered":"2024-05-11T12:10:44.763836+0000","last_unstale":"2024-05-11T12:10:53.086299+0000","last_undegraded":"2024-05-11T12:10:53.086299+0000","last_fullsized":"2024-05-11T12:10:53.086299+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:39:50.229692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.085357+0000","last_change":"2024-05-11T12:10:49.841029+0000","last_active":"2024-05-11T12:10:53.085357+0000","last_peered":"2024-05-11T12:10:53.085357+0000","last_clean":"2024-05-11T12:10:53.085357+0000","last_became_active":"2024-05-11T12:10:49.840067+0000","last_became_peered":"2024-05-11T12:10:49.840067+0000","last_unstale":"2024-05-11T12:10:53.085357+0000","last_undegraded":"2024-05-11T12:10:53.085357+0000","last_fullsized":"2024-05-11T12:10:53.085357+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:02:51.578727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.178524+0000","last_change":"2024-05-11T12:10:45.785688+0000","last_active":"2024-05-11T12:10:51.178524+0000","last_peered":"2024-05-11T12:10:51.178524+0000","last_clean":"2024-05-11T12:10:51.178524+0000","last_became_active":"2024-05-11T12:10:45.785267+0000","last_became_peered":"2024-05-11T12:10:45.785267+0000","last_unstale":"2024-05-11T12:10:51.178524+0000","last_undegraded":"2024-05-11T12:10:51.178524+0000","last_fullsized":"2024-05-11T12:10:51.178524+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:50:30.157404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.833626+0000","last_change":"2024-05-11T12:10:44.777030+0000","last_active":"2024-05-11T12:10:50.833626+0000","last_peered":"2024-05-11T12:10:50.833626+0000","last_clean":"2024-05-11T12:10:50.833626+0000","last_became_active":"2024-05-11T12:10:44.776600+0000","last_became_peered":"2024-05-11T12:10:44.776600+0000","last_unstale":"2024-05-11T12:10:50.833626+0000","last_undegraded":"2024-05-11T12:10:50.833626+0000","last_fullsized":"2024-05-11T12:10:50.833626+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:32:31.312066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.086293+0000","last_change":"2024-05-11T12:10:49.843811+0000","last_active":"2024-05-11T12:10:53.086293+0000","last_peered":"2024-05-11T12:10:53.086293+0000","last_clean":"2024-05-11T12:10:53.086293+0000","last_became_active":"2024-05-11T12:10:49.843361+0000","last_became_peered":"2024-05-11T12:10:49.843361+0000","last_unstale":"2024-05-11T12:10:53.086293+0000","last_undegraded":"2024-05-11T12:10:53.086293+0000","last_fullsized":"2024-05-11T12:10:53.086293+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:03:19.587244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.085874+0000","last_change":"2024-05-11T12:10:45.773922+0000","last_active":"2024-05-11T12:10:53.085874+0000","last_peered":"2024-05-11T12:10:53.085874+0000","last_clean":"2024-05-11T12:10:53.085874+0000","last_became_active":"2024-05-11T12:10:45.773584+0000","last_became_peered":"2024-05-11T12:10:45.773584+0000","last_unstale":"2024-05-11T12:10:53.085874+0000","last_undegraded":"2024-05-11T12:10:53.085874+0000","last_fullsized":"2024-05-11T12:10:53.085874+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:25:39.934222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.956061+0000","last_change":"2024-05-11T12:10:44.767869+0000","last_active":"2024-05-11T12:10:50.956061+0000","last_peered":"2024-05-11T12:10:50.956061+0000","last_clean":"2024-05-11T12:10:50.956061+0000","last_became_active":"2024-05-11T12:10:44.766486+0000","last_became_peered":"2024-05-11T12:10:44.766486+0000","last_unstale":"2024-05-11T12:10:50.956061+0000","last_undegraded":"2024-05-11T12:10:50.956061+0000","last_fullsized":"2024-05-11T12:10:50.956061+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:49:38.234192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.087005+0000","last_change":"2024-05-11T12:10:49.837338+0000","last_active":"2024-05-11T12:10:53.087005+0000","last_peered":"2024-05-11T12:10:53.087005+0000","last_clean":"2024-05-11T12:10:53.087005+0000","last_became_active":"2024-05-11T12:10:49.836880+0000","last_became_peered":"2024-05-11T12:10:49.836880+0000","last_unstale":"2024-05-11T12:10:53.087005+0000","last_undegraded":"2024-05-11T12:10:53.087005+0000","last_fullsized":"2024-05-11T12:10:53.087005+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:30:58.400447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.055566+0000","last_change":"2024-05-11T12:10:45.784532+0000","last_active":"2024-05-11T12:10:53.055566+0000","last_peered":"2024-05-11T12:10:53.055566+0000","last_clean":"2024-05-11T12:10:53.055566+0000","last_became_active":"2024-05-11T12:10:45.783747+0000","last_became_peered":"2024-05-11T12:10:45.783747+0000","last_unstale":"2024-05-11T12:10:53.055566+0000","last_undegraded":"2024-05-11T12:10:53.055566+0000","last_fullsized":"2024-05-11T12:10:53.055566+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:33:50.642754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.986822+0000","last_change":"2024-05-11T12:10:44.778140+0000","last_active":"2024-05-11T12:10:52.986822+0000","last_peered":"2024-05-11T12:10:52.986822+0000","last_clean":"2024-05-11T12:10:52.986822+0000","last_became_active":"2024-05-11T12:10:44.777780+0000","last_became_peered":"2024-05-11T12:10:44.777780+0000","last_unstale":"2024-05-11T12:10:52.986822+0000","last_undegraded":"2024-05-11T12:10:52.986822+0000","last_fullsized":"2024-05-11T12:10:52.986822+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:25:35.246177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.057469+0000","last_change":"2024-05-11T12:10:49.984316+0000","last_active":"2024-05-11T12:10:53.057469+0000","last_peered":"2024-05-11T12:10:53.057469+0000","last_clean":"2024-05-11T12:10:53.057469+0000","last_became_active":"2024-05-11T12:10:49.981788+0000","last_became_peered":"2024-05-11T12:10:49.981788+0000","last_unstale":"2024-05-11T12:10:53.057469+0000","last_undegraded":"2024-05-11T12:10:53.057469+0000","last_fullsized":"2024-05-11T12:10:53.057469+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:07:34.303729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.830946+0000","last_change":"2024-05-11T12:10:45.781491+0000","last_active":"2024-05-11T12:10:50.830946+0000","last_peered":"2024-05-11T12:10:50.830946+0000","last_clean":"2024-05-11T12:10:50.830946+0000","last_became_active":"2024-05-11T12:10:45.779112+0000","last_became_peered":"2024-05-11T12:10:45.779112+0000","last_unstale":"2024-05-11T12:10:50.830946+0000","last_undegraded":"2024-05-11T12:10:50.830946+0000","last_fullsized":"2024-05-11T12:10:50.830946+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:00:49.028293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.833659+0000","last_change":"2024-05-11T12:10:44.765494+0000","last_active":"2024-05-11T12:10:50.833659+0000","last_peered":"2024-05-11T12:10:50.833659+0000","last_clean":"2024-05-11T12:10:50.833659+0000","last_became_active":"2024-05-11T12:10:44.764921+0000","last_became_peered":"2024-05-11T12:10:44.764921+0000","last_unstale":"2024-05-11T12:10:50.833659+0000","last_undegraded":"2024-05-11T12:10:50.833659+0000","last_fullsized":"2024-05-11T12:10:50.833659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:11:44.935841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":117,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.986840+0000","last_change":"2024-05-11T12:10:35.066673+0000","last_active":"2024-05-11T12:10:52.986840+0000","last_peered":"2024-05-11T12:10:52.986840+0000","last_clean":"2024-05-11T12:10:52.986840+0000","last_became_active":"2024-05-11T12:10:35.065942+0000","last_became_peered":"2024-05-11T12:10:35.065942+0000","last_unstale":"2024-05-11T12:10:52.986840+0000","last_undegraded":"2024-05-11T12:10:52.986840+0000","last_fullsized":"2024-05-11T12:10:52.986840+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_clean_scrub_stamp":"2024-05-11T12:10:33.601136+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:12:02.133182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.836369+0000","last_change":"2024-05-11T12:10:49.983023+0000","last_active":"2024-05-11T12:10:50.836369+0000","last_peered":"2024-05-11T12:10:50.836369+0000","last_clean":"2024-05-11T12:10:50.836369+0000","last_became_active":"2024-05-11T12:10:49.981575+0000","last_became_peered":"2024-05-11T12:10:49.981575+0000","last_unstale":"2024-05-11T12:10:50.836369+0000","last_undegraded":"2024-05-11T12:10:50.836369+0000","last_fullsized":"2024-05-11T12:10:50.836369+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:52:55.215724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.988305+0000","last_change":"2024-05-11T12:10:45.787709+0000","last_active":"2024-05-11T12:10:52.988305+0000","last_peered":"2024-05-11T12:10:52.988305+0000","last_clean":"2024-05-11T12:10:52.988305+0000","last_became_active":"2024-05-11T12:10:45.787205+0000","last_became_peered":"2024-05-11T12:10:45.787205+0000","last_unstale":"2024-05-11T12:10:52.988305+0000","last_undegraded":"2024-05-11T12:10:52.988305+0000","last_fullsized":"2024-05-11T12:10:52.988305+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:29:58.902743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.956475+0000","last_change":"2024-05-11T12:10:44.777012+0000","last_active":"2024-05-11T12:10:50.956475+0000","last_peered":"2024-05-11T12:10:50.956475+0000","last_clean":"2024-05-11T12:10:50.956475+0000","last_became_active":"2024-05-11T12:10:44.776709+0000","last_became_peered":"2024-05-11T12:10:44.776709+0000","last_unstale":"2024-05-11T12:10:50.956475+0000","last_undegraded":"2024-05-11T12:10:50.956475+0000","last_fullsized":"2024-05-11T12:10:50.956475+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:32:06.207257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.833701+0000","last_change":"2024-05-11T12:10:49.980334+0000","last_active":"2024-05-11T12:10:50.833701+0000","last_peered":"2024-05-11T12:10:50.833701+0000","last_clean":"2024-05-11T12:10:50.833701+0000","last_became_active":"2024-05-11T12:10:49.977819+0000","last_became_peered":"2024-05-11T12:10:49.977819+0000","last_unstale":"2024-05-11T12:10:50.833701+0000","last_undegraded":"2024-05-11T12:10:50.833701+0000","last_fullsized":"2024-05-11T12:10:50.833701+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:30:02.975329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.835272+0000","last_change":"2024-05-11T12:10:45.780131+0000","last_active":"2024-05-11T12:10:50.835272+0000","last_peered":"2024-05-11T12:10:50.835272+0000","last_clean":"2024-05-11T12:10:50.835272+0000","last_became_active":"2024-05-11T12:10:45.779867+0000","last_became_peered":"2024-05-11T12:10:45.779867+0000","last_unstale":"2024-05-11T12:10:50.835272+0000","last_undegraded":"2024-05-11T12:10:50.835272+0000","last_fullsized":"2024-05-11T12:10:50.835272+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:30:54.327861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.084659+0000","last_change":"2024-05-11T12:10:44.775148+0000","last_active":"2024-05-11T12:10:53.084659+0000","last_peered":"2024-05-11T12:10:53.084659+0000","last_clean":"2024-05-11T12:10:53.084659+0000","last_became_active":"2024-05-11T12:10:44.774850+0000","last_became_peered":"2024-05-11T12:10:44.774850+0000","last_unstale":"2024-05-11T12:10:53.084659+0000","last_undegraded":"2024-05-11T12:10:53.084659+0000","last_fullsized":"2024-05-11T12:10:53.084659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:25:35.246177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.056241+0000","last_change":"2024-05-11T12:10:49.842656+0000","last_active":"2024-05-11T12:10:53.056241+0000","last_peered":"2024-05-11T12:10:53.056241+0000","last_clean":"2024-05-11T12:10:53.056241+0000","last_became_active":"2024-05-11T12:10:49.841982+0000","last_became_peered":"2024-05-11T12:10:49.841982+0000","last_unstale":"2024-05-11T12:10:53.056241+0000","last_undegraded":"2024-05-11T12:10:53.056241+0000","last_fullsized":"2024-05-11T12:10:53.056241+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:03:19.587244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.988912+0000","last_change":"2024-05-11T12:10:45.794103+0000","last_active":"2024-05-11T12:10:52.988912+0000","last_peered":"2024-05-11T12:10:52.988912+0000","last_clean":"2024-05-11T12:10:52.988912+0000","last_became_active":"2024-05-11T12:10:45.792080+0000","last_became_peered":"2024-05-11T12:10:45.792080+0000","last_unstale":"2024-05-11T12:10:52.988912+0000","last_undegraded":"2024-05-11T12:10:52.988912+0000","last_fullsized":"2024-05-11T12:10:52.988912+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:30:54.327861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.956469+0000","last_change":"2024-05-11T12:10:44.768108+0000","last_active":"2024-05-11T12:10:50.956469+0000","last_peered":"2024-05-11T12:10:50.956469+0000","last_clean":"2024-05-11T12:10:50.956469+0000","last_became_active":"2024-05-11T12:10:44.766606+0000","last_became_peered":"2024-05-11T12:10:44.766606+0000","last_unstale":"2024-05-11T12:10:50.956469+0000","last_undegraded":"2024-05-11T12:10:50.956469+0000","last_fullsized":"2024-05-11T12:10:50.956469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:02:04.596576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.832239+0000","last_change":"2024-05-11T12:10:49.982446+0000","last_active":"2024-05-11T12:10:50.832239+0000","last_peered":"2024-05-11T12:10:50.832239+0000","last_clean":"2024-05-11T12:10:50.832239+0000","last_became_active":"2024-05-11T12:10:49.981272+0000","last_became_peered":"2024-05-11T12:10:49.981272+0000","last_unstale":"2024-05-11T12:10:50.832239+0000","last_undegraded":"2024-05-11T12:10:50.832239+0000","last_fullsized":"2024-05-11T12:10:50.832239+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:36:40.828180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.988575+0000","last_change":"2024-05-11T12:10:45.786373+0000","last_active":"2024-05-11T12:10:52.988575+0000","last_peered":"2024-05-11T12:10:52.988575+0000","last_clean":"2024-05-11T12:10:52.988575+0000","last_became_active":"2024-05-11T12:10:45.786050+0000","last_became_peered":"2024-05-11T12:10:45.786050+0000","last_unstale":"2024-05-11T12:10:52.988575+0000","last_undegraded":"2024-05-11T12:10:52.988575+0000","last_fullsized":"2024-05-11T12:10:52.988575+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:31:24.678347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.178969+0000","last_change":"2024-05-11T12:10:44.775124+0000","last_active":"2024-05-11T12:10:51.178969+0000","last_peered":"2024-05-11T12:10:51.178969+0000","last_clean":"2024-05-11T12:10:51.178969+0000","last_became_active":"2024-05-11T12:10:44.774810+0000","last_became_peered":"2024-05-11T12:10:44.774810+0000","last_unstale":"2024-05-11T12:10:51.178969+0000","last_undegraded":"2024-05-11T12:10:51.178969+0000","last_fullsized":"2024-05-11T12:10:51.178969+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:44:00.317135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.177431+0000","last_change":"2024-05-11T12:10:49.845241+0000","last_active":"2024-05-11T12:10:51.177431+0000","last_peered":"2024-05-11T12:10:51.177431+0000","last_clean":"2024-05-11T12:10:51.177431+0000","last_became_active":"2024-05-11T12:10:49.844564+0000","last_became_peered":"2024-05-11T12:10:49.844564+0000","last_unstale":"2024-05-11T12:10:51.177431+0000","last_undegraded":"2024-05-11T12:10:51.177431+0000","last_fullsized":"2024-05-11T12:10:51.177431+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:27:50.396091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.987029+0000","last_change":"2024-05-11T12:10:45.789312+0000","last_active":"2024-05-11T12:10:52.987029+0000","last_peered":"2024-05-11T12:10:52.987029+0000","last_clean":"2024-05-11T12:10:52.987029+0000","last_became_active":"2024-05-11T12:10:45.789069+0000","last_became_peered":"2024-05-11T12:10:45.789069+0000","last_unstale":"2024-05-11T12:10:52.987029+0000","last_undegraded":"2024-05-11T12:10:52.987029+0000","last_fullsized":"2024-05-11T12:10:52.987029+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:15:55.637133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.955873+0000","last_change":"2024-05-11T12:10:44.767205+0000","last_active":"2024-05-11T12:10:50.955873+0000","last_peered":"2024-05-11T12:10:50.955873+0000","last_clean":"2024-05-11T12:10:50.955873+0000","last_became_active":"2024-05-11T12:10:44.766855+0000","last_became_peered":"2024-05-11T12:10:44.766855+0000","last_unstale":"2024-05-11T12:10:50.955873+0000","last_undegraded":"2024-05-11T12:10:50.955873+0000","last_fullsized":"2024-05-11T12:10:50.955873+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:10:35.839599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.056975+0000","last_change":"2024-05-11T12:10:49.835689+0000","last_active":"2024-05-11T12:10:53.056975+0000","last_peered":"2024-05-11T12:10:53.056975+0000","last_clean":"2024-05-11T12:10:53.056975+0000","last_became_active":"2024-05-11T12:10:49.835305+0000","last_became_peered":"2024-05-11T12:10:49.835305+0000","last_unstale":"2024-05-11T12:10:53.056975+0000","last_undegraded":"2024-05-11T12:10:53.056975+0000","last_fullsized":"2024-05-11T12:10:53.056975+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:30:58.400447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.833870+0000","last_change":"2024-05-11T12:10:45.788030+0000","last_active":"2024-05-11T12:10:50.833870+0000","last_peered":"2024-05-11T12:10:50.833870+0000","last_clean":"2024-05-11T12:10:50.833870+0000","last_became_active":"2024-05-11T12:10:45.787098+0000","last_became_peered":"2024-05-11T12:10:45.787098+0000","last_unstale":"2024-05-11T12:10:50.833870+0000","last_undegraded":"2024-05-11T12:10:50.833870+0000","last_fullsized":"2024-05-11T12:10:50.833870+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:00:49.028293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.834966+0000","last_change":"2024-05-11T12:10:44.784319+0000","last_active":"2024-05-11T12:10:50.834966+0000","last_peered":"2024-05-11T12:10:50.834966+0000","last_clean":"2024-05-11T12:10:50.834966+0000","last_became_active":"2024-05-11T12:10:44.784027+0000","last_became_peered":"2024-05-11T12:10:44.784027+0000","last_unstale":"2024-05-11T12:10:50.834966+0000","last_undegraded":"2024-05-11T12:10:50.834966+0000","last_fullsized":"2024-05-11T12:10:50.834966+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:50:29.143515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.832654+0000","last_change":"2024-05-11T12:10:49.985187+0000","last_active":"2024-05-11T12:10:50.832654+0000","last_peered":"2024-05-11T12:10:50.832654+0000","last_clean":"2024-05-11T12:10:50.832654+0000","last_became_active":"2024-05-11T12:10:49.983450+0000","last_became_peered":"2024-05-11T12:10:49.983450+0000","last_unstale":"2024-05-11T12:10:50.832654+0000","last_undegraded":"2024-05-11T12:10:50.832654+0000","last_fullsized":"2024-05-11T12:10:50.832654+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:02:13.707329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.987787+0000","last_change":"2024-05-11T12:10:45.789328+0000","last_active":"2024-05-11T12:10:52.987787+0000","last_peered":"2024-05-11T12:10:52.987787+0000","last_clean":"2024-05-11T12:10:52.987787+0000","last_became_active":"2024-05-11T12:10:45.788778+0000","last_became_peered":"2024-05-11T12:10:45.788778+0000","last_unstale":"2024-05-11T12:10:52.987787+0000","last_undegraded":"2024-05-11T12:10:52.987787+0000","last_fullsized":"2024-05-11T12:10:52.987787+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:01:30.356378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.833719+0000","last_change":"2024-05-11T12:10:44.779353+0000","last_active":"2024-05-11T12:10:50.833719+0000","last_peered":"2024-05-11T12:10:50.833719+0000","last_clean":"2024-05-11T12:10:50.833719+0000","last_became_active":"2024-05-11T12:10:44.779078+0000","last_became_peered":"2024-05-11T12:10:44.779078+0000","last_unstale":"2024-05-11T12:10:50.833719+0000","last_undegraded":"2024-05-11T12:10:50.833719+0000","last_fullsized":"2024-05-11T12:10:50.833719+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:25:35.246177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.831951+0000","last_change":"2024-05-11T12:10:49.979416+0000","last_active":"2024-05-11T12:10:50.831951+0000","last_peered":"2024-05-11T12:10:50.831951+0000","last_clean":"2024-05-11T12:10:50.831951+0000","last_became_active":"2024-05-11T12:10:49.978262+0000","last_became_peered":"2024-05-11T12:10:49.978262+0000","last_unstale":"2024-05-11T12:10:50.831951+0000","last_undegraded":"2024-05-11T12:10:50.831951+0000","last_fullsized":"2024-05-11T12:10:50.831951+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:53:47.237894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.056303+0000","last_change":"2024-05-11T12:10:45.784434+0000","last_active":"2024-05-11T12:10:53.056303+0000","last_peered":"2024-05-11T12:10:53.056303+0000","last_clean":"2024-05-11T12:10:53.056303+0000","last_became_active":"2024-05-11T12:10:45.784143+0000","last_became_peered":"2024-05-11T12:10:45.784143+0000","last_unstale":"2024-05-11T12:10:53.056303+0000","last_undegraded":"2024-05-11T12:10:53.056303+0000","last_fullsized":"2024-05-11T12:10:53.056303+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:25:39.934222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.054871+0000","last_change":"2024-05-11T12:10:44.783217+0000","last_active":"2024-05-11T12:10:53.054871+0000","last_peered":"2024-05-11T12:10:53.054871+0000","last_clean":"2024-05-11T12:10:53.054871+0000","last_became_active":"2024-05-11T12:10:44.782942+0000","last_became_peered":"2024-05-11T12:10:44.782942+0000","last_unstale":"2024-05-11T12:10:53.054871+0000","last_undegraded":"2024-05-11T12:10:53.054871+0000","last_fullsized":"2024-05-11T12:10:53.054871+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:14:58.421726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.836406+0000","last_change":"2024-05-11T12:10:49.981126+0000","last_active":"2024-05-11T12:10:50.836406+0000","last_peered":"2024-05-11T12:10:50.836406+0000","last_clean":"2024-05-11T12:10:50.836406+0000","last_became_active":"2024-05-11T12:10:49.978149+0000","last_became_peered":"2024-05-11T12:10:49.978149+0000","last_unstale":"2024-05-11T12:10:50.836406+0000","last_undegraded":"2024-05-11T12:10:50.836406+0000","last_fullsized":"2024-05-11T12:10:50.836406+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:45:08.936896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.988035+0000","last_change":"2024-05-11T12:10:45.791440+0000","last_active":"2024-05-11T12:10:52.988035+0000","last_peered":"2024-05-11T12:10:52.988035+0000","last_clean":"2024-05-11T12:10:52.988035+0000","last_became_active":"2024-05-11T12:10:45.791154+0000","last_became_peered":"2024-05-11T12:10:45.791154+0000","last_unstale":"2024-05-11T12:10:52.988035+0000","last_undegraded":"2024-05-11T12:10:52.988035+0000","last_fullsized":"2024-05-11T12:10:52.988035+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:52:51.143138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'10","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:11:01.126783+0000","last_change":"2024-05-11T12:10:44.773468+0000","last_active":"2024-05-11T12:11:01.126783+0000","last_peered":"2024-05-11T12:11:01.126783+0000","last_clean":"2024-05-11T12:11:01.126783+0000","last_became_active":"2024-05-11T12:10:44.769134+0000","last_became_peered":"2024-05-11T12:10:44.769134+0000","last_unstale":"2024-05-11T12:11:01.126783+0000","last_undegraded":"2024-05-11T12:11:01.126783+0000","last_fullsized":"2024-05-11T12:11:01.126783+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:08:58.049386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.085619+0000","last_change":"2024-05-11T12:10:49.847456+0000","last_active":"2024-05-11T12:10:53.085619+0000","last_peered":"2024-05-11T12:10:53.085619+0000","last_clean":"2024-05-11T12:10:53.085619+0000","last_became_active":"2024-05-11T12:10:49.847009+0000","last_became_peered":"2024-05-11T12:10:49.847009+0000","last_unstale":"2024-05-11T12:10:53.085619+0000","last_undegraded":"2024-05-11T12:10:53.085619+0000","last_fullsized":"2024-05-11T12:10:53.085619+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:52:55.215724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.178813+0000","last_change":"2024-05-11T12:10:45.781361+0000","last_active":"2024-05-11T12:10:51.178813+0000","last_peered":"2024-05-11T12:10:51.178813+0000","last_clean":"2024-05-11T12:10:51.178813+0000","last_became_active":"2024-05-11T12:10:45.781039+0000","last_became_peered":"2024-05-11T12:10:45.781039+0000","last_unstale":"2024-05-11T12:10:51.178813+0000","last_undegraded":"2024-05-11T12:10:51.178813+0000","last_fullsized":"2024-05-11T12:10:51.178813+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:27:49.489542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.084600+0000","last_change":"2024-05-11T12:10:44.770795+0000","last_active":"2024-05-11T12:10:53.084600+0000","last_peered":"2024-05-11T12:10:53.084600+0000","last_clean":"2024-05-11T12:10:53.084600+0000","last_became_active":"2024-05-11T12:10:44.770438+0000","last_became_peered":"2024-05-11T12:10:44.770438+0000","last_unstale":"2024-05-11T12:10:53.084600+0000","last_undegraded":"2024-05-11T12:10:53.084600+0000","last_fullsized":"2024-05-11T12:10:53.084600+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:50:29.143515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.178367+0000","last_change":"2024-05-11T12:10:49.841798+0000","last_active":"2024-05-11T12:10:51.178367+0000","last_peered":"2024-05-11T12:10:51.178367+0000","last_clean":"2024-05-11T12:10:51.178367+0000","last_became_active":"2024-05-11T12:10:49.840773+0000","last_became_peered":"2024-05-11T12:10:49.840773+0000","last_unstale":"2024-05-11T12:10:51.178367+0000","last_undegraded":"2024-05-11T12:10:51.178367+0000","last_fullsized":"2024-05-11T12:10:51.178367+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:01:34.428964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.057219+0000","last_change":"2024-05-11T12:10:45.779106+0000","last_active":"2024-05-11T12:10:53.057219+0000","last_peered":"2024-05-11T12:10:53.057219+0000","last_clean":"2024-05-11T12:10:53.057219+0000","last_became_active":"2024-05-11T12:10:45.778616+0000","last_became_peered":"2024-05-11T12:10:45.778616+0000","last_unstale":"2024-05-11T12:10:53.057219+0000","last_undegraded":"2024-05-11T12:10:53.057219+0000","last_fullsized":"2024-05-11T12:10:53.057219+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:40:33.598752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.837351+0000","last_change":"2024-05-11T12:10:44.781127+0000","last_active":"2024-05-11T12:10:50.837351+0000","last_peered":"2024-05-11T12:10:50.837351+0000","last_clean":"2024-05-11T12:10:50.837351+0000","last_became_active":"2024-05-11T12:10:44.780840+0000","last_became_peered":"2024-05-11T12:10:44.780840+0000","last_unstale":"2024-05-11T12:10:50.837351+0000","last_undegraded":"2024-05-11T12:10:50.837351+0000","last_fullsized":"2024-05-11T12:10:50.837351+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:50:29.143515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.830538+0000","last_change":"2024-05-11T12:10:49.839762+0000","last_active":"2024-05-11T12:10:50.830538+0000","last_peered":"2024-05-11T12:10:50.830538+0000","last_clean":"2024-05-11T12:10:50.830538+0000","last_became_active":"2024-05-11T12:10:49.838073+0000","last_became_peered":"2024-05-11T12:10:49.838073+0000","last_unstale":"2024-05-11T12:10:50.830538+0000","last_undegraded":"2024-05-11T12:10:50.830538+0000","last_fullsized":"2024-05-11T12:10:50.830538+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:36:40.828180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.837108+0000","last_change":"2024-05-11T12:10:45.778187+0000","last_active":"2024-05-11T12:10:50.837108+0000","last_peered":"2024-05-11T12:10:50.837108+0000","last_clean":"2024-05-11T12:10:50.837108+0000","last_became_active":"2024-05-11T12:10:45.776905+0000","last_became_peered":"2024-05-11T12:10:45.776905+0000","last_unstale":"2024-05-11T12:10:50.837108+0000","last_undegraded":"2024-05-11T12:10:50.837108+0000","last_fullsized":"2024-05-11T12:10:50.837108+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:36:07.588854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.837368+0000","last_change":"2024-05-11T12:10:44.778851+0000","last_active":"2024-05-11T12:10:50.837368+0000","last_peered":"2024-05-11T12:10:50.837368+0000","last_clean":"2024-05-11T12:10:50.837368+0000","last_became_active":"2024-05-11T12:10:44.776695+0000","last_became_peered":"2024-05-11T12:10:44.776695+0000","last_unstale":"2024-05-11T12:10:50.837368+0000","last_undegraded":"2024-05-11T12:10:50.837368+0000","last_fullsized":"2024-05-11T12:10:50.837368+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:49:09.103744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.831100+0000","last_change":"2024-05-11T12:10:49.982849+0000","last_active":"2024-05-11T12:10:50.831100+0000","last_peered":"2024-05-11T12:10:50.831100+0000","last_clean":"2024-05-11T12:10:50.831100+0000","last_became_active":"2024-05-11T12:10:49.981894+0000","last_became_peered":"2024-05-11T12:10:49.981894+0000","last_unstale":"2024-05-11T12:10:50.831100+0000","last_undegraded":"2024-05-11T12:10:50.831100+0000","last_fullsized":"2024-05-11T12:10:50.831100+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:45:08.936896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.056935+0000","last_change":"2024-05-11T12:10:49.982163+0000","last_active":"2024-05-11T12:10:53.056935+0000","last_peered":"2024-05-11T12:10:53.056935+0000","last_clean":"2024-05-11T12:10:53.056935+0000","last_became_active":"2024-05-11T12:10:49.981202+0000","last_became_peered":"2024-05-11T12:10:49.981202+0000","last_unstale":"2024-05-11T12:10:53.056935+0000","last_undegraded":"2024-05-11T12:10:53.056935+0000","last_fullsized":"2024-05-11T12:10:53.056935+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:36:11.661440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.085018+0000","last_change":"2024-05-11T12:10:45.772477+0000","last_active":"2024-05-11T12:10:53.085018+0000","last_peered":"2024-05-11T12:10:53.085018+0000","last_clean":"2024-05-11T12:10:53.085018+0000","last_became_active":"2024-05-11T12:10:45.772053+0000","last_became_peered":"2024-05-11T12:10:45.772053+0000","last_unstale":"2024-05-11T12:10:53.085018+0000","last_undegraded":"2024-05-11T12:10:53.085018+0000","last_fullsized":"2024-05-11T12:10:53.085018+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:00:49.028293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.832812+0000","last_change":"2024-05-11T12:10:44.782828+0000","last_active":"2024-05-11T12:10:50.832812+0000","last_peered":"2024-05-11T12:10:50.832812+0000","last_clean":"2024-05-11T12:10:50.832812+0000","last_became_active":"2024-05-11T12:10:44.782412+0000","last_became_peered":"2024-05-11T12:10:44.782412+0000","last_unstale":"2024-05-11T12:10:50.832812+0000","last_undegraded":"2024-05-11T12:10:50.832812+0000","last_fullsized":"2024-05-11T12:10:50.832812+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:39:50.229692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.836608+0000","last_change":"2024-05-11T12:10:49.849632+0000","last_active":"2024-05-11T12:10:50.836608+0000","last_peered":"2024-05-11T12:10:50.836608+0000","last_clean":"2024-05-11T12:10:50.836608+0000","last_became_active":"2024-05-11T12:10:49.849221+0000","last_became_peered":"2024-05-11T12:10:49.849221+0000","last_unstale":"2024-05-11T12:10:50.836608+0000","last_undegraded":"2024-05-11T12:10:50.836608+0000","last_fullsized":"2024-05-11T12:10:50.836608+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:02:51.578727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.830326+0000","last_change":"2024-05-11T12:10:45.784432+0000","last_active":"2024-05-11T12:10:50.830326+0000","last_peered":"2024-05-11T12:10:50.830326+0000","last_clean":"2024-05-11T12:10:50.830326+0000","last_became_active":"2024-05-11T12:10:45.784152+0000","last_became_peered":"2024-05-11T12:10:45.784152+0000","last_unstale":"2024-05-11T12:10:50.830326+0000","last_undegraded":"2024-05-11T12:10:50.830326+0000","last_fullsized":"2024-05-11T12:10:50.830326+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:21:38.211953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.831004+0000","last_change":"2024-05-11T12:10:44.775112+0000","last_active":"2024-05-11T12:10:50.831004+0000","last_peered":"2024-05-11T12:10:50.831004+0000","last_clean":"2024-05-11T12:10:50.831004+0000","last_became_active":"2024-05-11T12:10:44.774758+0000","last_became_peered":"2024-05-11T12:10:44.774758+0000","last_unstale":"2024-05-11T12:10:50.831004+0000","last_undegraded":"2024-05-11T12:10:50.831004+0000","last_fullsized":"2024-05-11T12:10:50.831004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:38:50.526921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.835462+0000","last_change":"2024-05-11T12:10:44.778249+0000","last_active":"2024-05-11T12:10:50.835462+0000","last_peered":"2024-05-11T12:10:50.835462+0000","last_clean":"2024-05-11T12:10:50.835462+0000","last_became_active":"2024-05-11T12:10:44.775251+0000","last_became_peered":"2024-05-11T12:10:44.775251+0000","last_unstale":"2024-05-11T12:10:50.835462+0000","last_undegraded":"2024-05-11T12:10:50.835462+0000","last_fullsized":"2024-05-11T12:10:50.835462+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:11:44.935841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.056459+0000","last_change":"2024-05-11T12:10:45.783894+0000","last_active":"2024-05-11T12:10:53.056459+0000","last_peered":"2024-05-11T12:10:53.056459+0000","last_clean":"2024-05-11T12:10:53.056459+0000","last_became_active":"2024-05-11T12:10:45.783452+0000","last_became_peered":"2024-05-11T12:10:45.783452+0000","last_unstale":"2024-05-11T12:10:53.056459+0000","last_undegraded":"2024-05-11T12:10:53.056459+0000","last_fullsized":"2024-05-11T12:10:53.056459+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:09:48.719906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.956355+0000","last_change":"2024-05-11T12:10:49.841006+0000","last_active":"2024-05-11T12:10:50.956355+0000","last_peered":"2024-05-11T12:10:50.956355+0000","last_clean":"2024-05-11T12:10:50.956355+0000","last_became_active":"2024-05-11T12:10:49.840609+0000","last_became_peered":"2024-05-11T12:10:49.840609+0000","last_unstale":"2024-05-11T12:10:50.956355+0000","last_undegraded":"2024-05-11T12:10:50.956355+0000","last_fullsized":"2024-05-11T12:10:50.956355+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:11:33.092215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.831304+0000","last_change":"2024-05-11T12:10:44.769720+0000","last_active":"2024-05-11T12:10:50.831304+0000","last_peered":"2024-05-11T12:10:50.831304+0000","last_clean":"2024-05-11T12:10:50.831304+0000","last_became_active":"2024-05-11T12:10:44.768923+0000","last_became_peered":"2024-05-11T12:10:44.768923+0000","last_unstale":"2024-05-11T12:10:50.831304+0000","last_undegraded":"2024-05-11T12:10:50.831304+0000","last_fullsized":"2024-05-11T12:10:50.831304+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:04:49.439739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.177838+0000","last_change":"2024-05-11T12:10:45.776362+0000","last_active":"2024-05-11T12:10:51.177838+0000","last_peered":"2024-05-11T12:10:51.177838+0000","last_clean":"2024-05-11T12:10:51.177838+0000","last_became_active":"2024-05-11T12:10:45.773081+0000","last_became_peered":"2024-05-11T12:10:45.773081+0000","last_unstale":"2024-05-11T12:10:51.177838+0000","last_undegraded":"2024-05-11T12:10:51.177838+0000","last_fullsized":"2024-05-11T12:10:51.177838+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:25:36.260066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.832413+0000","last_change":"2024-05-11T12:10:49.837438+0000","last_active":"2024-05-11T12:10:50.832413+0000","last_peered":"2024-05-11T12:10:50.832413+0000","last_clean":"2024-05-11T12:10:50.832413+0000","last_became_active":"2024-05-11T12:10:49.836966+0000","last_became_peered":"2024-05-11T12:10:49.836966+0000","last_unstale":"2024-05-11T12:10:50.832413+0000","last_undegraded":"2024-05-11T12:10:50.832413+0000","last_fullsized":"2024-05-11T12:10:50.832413+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:16:26.266475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.832119+0000","last_change":"2024-05-11T12:10:45.792181+0000","last_active":"2024-05-11T12:10:50.832119+0000","last_peered":"2024-05-11T12:10:50.832119+0000","last_clean":"2024-05-11T12:10:50.832119+0000","last_became_active":"2024-05-11T12:10:45.791827+0000","last_became_peered":"2024-05-11T12:10:45.791827+0000","last_unstale":"2024-05-11T12:10:50.832119+0000","last_undegraded":"2024-05-11T12:10:50.832119+0000","last_fullsized":"2024-05-11T12:10:50.832119+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:25:39.934222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.956054+0000","last_change":"2024-05-11T12:10:45.786514+0000","last_active":"2024-05-11T12:10:50.956054+0000","last_peered":"2024-05-11T12:10:50.956054+0000","last_clean":"2024-05-11T12:10:50.956054+0000","last_became_active":"2024-05-11T12:10:45.786170+0000","last_became_peered":"2024-05-11T12:10:45.786170+0000","last_unstale":"2024-05-11T12:10:50.956054+0000","last_undegraded":"2024-05-11T12:10:50.956054+0000","last_fullsized":"2024-05-11T12:10:50.956054+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:07:42.553060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.833366+0000","last_change":"2024-05-11T12:10:45.764626+0000","last_active":"2024-05-11T12:10:50.833366+0000","last_peered":"2024-05-11T12:10:50.833366+0000","last_clean":"2024-05-11T12:10:50.833366+0000","last_became_active":"2024-05-11T12:10:45.764234+0000","last_became_peered":"2024-05-11T12:10:45.764234+0000","last_unstale":"2024-05-11T12:10:50.833366+0000","last_undegraded":"2024-05-11T12:10:50.833366+0000","last_fullsized":"2024-05-11T12:10:50.833366+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:31:24.678347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.056344+0000","last_change":"2024-05-11T12:10:45.787822+0000","last_active":"2024-05-11T12:10:53.056344+0000","last_peered":"2024-05-11T12:10:53.056344+0000","last_clean":"2024-05-11T12:10:53.056344+0000","last_became_active":"2024-05-11T12:10:45.787503+0000","last_became_peered":"2024-05-11T12:10:45.787503+0000","last_unstale":"2024-05-11T12:10:53.056344+0000","last_undegraded":"2024-05-11T12:10:53.056344+0000","last_fullsized":"2024-05-11T12:10:53.056344+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:30:11.665446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.832309+0000","last_change":"2024-05-11T12:10:45.769648+0000","last_active":"2024-05-11T12:10:50.832309+0000","last_peered":"2024-05-11T12:10:50.832309+0000","last_clean":"2024-05-11T12:10:50.832309+0000","last_became_active":"2024-05-11T12:10:45.769168+0000","last_became_peered":"2024-05-11T12:10:45.769168+0000","last_unstale":"2024-05-11T12:10:50.832309+0000","last_undegraded":"2024-05-11T12:10:50.832309+0000","last_fullsized":"2024-05-11T12:10:50.832309+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:40:33.598752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.086161+0000","last_change":"2024-05-11T12:10:45.773070+0000","last_active":"2024-05-11T12:10:53.086161+0000","last_peered":"2024-05-11T12:10:53.086161+0000","last_clean":"2024-05-11T12:10:53.086161+0000","last_became_active":"2024-05-11T12:10:45.772734+0000","last_became_peered":"2024-05-11T12:10:45.772734+0000","last_unstale":"2024-05-11T12:10:53.086161+0000","last_undegraded":"2024-05-11T12:10:53.086161+0000","last_fullsized":"2024-05-11T12:10:53.086161+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:52:36.650847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.836911+0000","last_change":"2024-05-11T12:10:45.776117+0000","last_active":"2024-05-11T12:10:50.836911+0000","last_peered":"2024-05-11T12:10:50.836911+0000","last_clean":"2024-05-11T12:10:50.836911+0000","last_became_active":"2024-05-11T12:10:45.775042+0000","last_became_peered":"2024-05-11T12:10:45.775042+0000","last_unstale":"2024-05-11T12:10:50.836911+0000","last_undegraded":"2024-05-11T12:10:50.836911+0000","last_fullsized":"2024-05-11T12:10:50.836911+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:40:33.598752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.832667+0000","last_change":"2024-05-11T12:10:45.767897+0000","last_active":"2024-05-11T12:10:50.832667+0000","last_peered":"2024-05-11T12:10:50.832667+0000","last_clean":"2024-05-11T12:10:50.832667+0000","last_became_active":"2024-05-11T12:10:45.767619+0000","last_became_peered":"2024-05-11T12:10:45.767619+0000","last_unstale":"2024-05-11T12:10:50.832667+0000","last_undegraded":"2024-05-11T12:10:50.832667+0000","last_fullsized":"2024-05-11T12:10:50.832667+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:27:46.323505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.832683+0000","last_change":"2024-05-11T12:10:45.763701+0000","last_active":"2024-05-11T12:10:50.832683+0000","last_peered":"2024-05-11T12:10:50.832683+0000","last_clean":"2024-05-11T12:10:50.832683+0000","last_became_active":"2024-05-11T12:10:45.763260+0000","last_became_peered":"2024-05-11T12:10:45.763260+0000","last_unstale":"2024-05-11T12:10:50.832683+0000","last_undegraded":"2024-05-11T12:10:50.832683+0000","last_fullsized":"2024-05-11T12:10:50.832683+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:11:17.251971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.830765+0000","last_change":"2024-05-11T12:10:45.767500+0000","last_active":"2024-05-11T12:10:50.830765+0000","last_peered":"2024-05-11T12:10:50.830765+0000","last_clean":"2024-05-11T12:10:50.830765+0000","last_became_active":"2024-05-11T12:10:45.767137+0000","last_became_peered":"2024-05-11T12:10:45.767137+0000","last_unstale":"2024-05-11T12:10:50.830765+0000","last_undegraded":"2024-05-11T12:10:50.830765+0000","last_fullsized":"2024-05-11T12:10:50.830765+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:52:36.650847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.833200+0000","last_change":"2024-05-11T12:10:45.779266+0000","last_active":"2024-05-11T12:10:50.833200+0000","last_peered":"2024-05-11T12:10:50.833200+0000","last_clean":"2024-05-11T12:10:50.833200+0000","last_became_active":"2024-05-11T12:10:45.778968+0000","last_became_peered":"2024-05-11T12:10:45.778968+0000","last_unstale":"2024-05-11T12:10:50.833200+0000","last_undegraded":"2024-05-11T12:10:50.833200+0000","last_fullsized":"2024-05-11T12:10:50.833200+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:40:33.598752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.056067+0000","last_change":"2024-05-11T12:10:45.793850+0000","last_active":"2024-05-11T12:10:53.056067+0000","last_peered":"2024-05-11T12:10:53.056067+0000","last_clean":"2024-05-11T12:10:53.056067+0000","last_became_active":"2024-05-11T12:10:45.793503+0000","last_became_peered":"2024-05-11T12:10:45.793503+0000","last_unstale":"2024-05-11T12:10:53.056067+0000","last_undegraded":"2024-05-11T12:10:53.056067+0000","last_fullsized":"2024-05-11T12:10:53.056067+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:21:38.211953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.836150+0000","last_change":"2024-05-11T12:10:45.776958+0000","last_active":"2024-05-11T12:10:50.836150+0000","last_peered":"2024-05-11T12:10:50.836150+0000","last_clean":"2024-05-11T12:10:50.836150+0000","last_became_active":"2024-05-11T12:10:45.770839+0000","last_became_peered":"2024-05-11T12:10:45.770839+0000","last_unstale":"2024-05-11T12:10:50.836150+0000","last_undegraded":"2024-05-11T12:10:50.836150+0000","last_fullsized":"2024-05-11T12:10:50.836150+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:52:36.650847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.988652+0000","last_change":"2024-05-11T12:10:45.784456+0000","last_active":"2024-05-11T12:10:52.988652+0000","last_peered":"2024-05-11T12:10:52.988652+0000","last_clean":"2024-05-11T12:10:52.988652+0000","last_became_active":"2024-05-11T12:10:45.784149+0000","last_became_peered":"2024-05-11T12:10:45.784149+0000","last_unstale":"2024-05-11T12:10:52.988652+0000","last_undegraded":"2024-05-11T12:10:52.988652+0000","last_fullsized":"2024-05-11T12:10:52.988652+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:36:07.588854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.956434+0000","last_change":"2024-05-11T12:10:45.778083+0000","last_active":"2024-05-11T12:10:50.956434+0000","last_peered":"2024-05-11T12:10:50.956434+0000","last_clean":"2024-05-11T12:10:50.956434+0000","last_became_active":"2024-05-11T12:10:45.777758+0000","last_became_peered":"2024-05-11T12:10:45.777758+0000","last_unstale":"2024-05-11T12:10:50.956434+0000","last_undegraded":"2024-05-11T12:10:50.956434+0000","last_fullsized":"2024-05-11T12:10:50.956434+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:38:01.288328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.838005+0000","last_change":"2024-05-11T12:10:45.780997+0000","last_active":"2024-05-11T12:10:50.838005+0000","last_peered":"2024-05-11T12:10:50.838005+0000","last_clean":"2024-05-11T12:10:50.838005+0000","last_became_active":"2024-05-11T12:10:45.780744+0000","last_became_peered":"2024-05-11T12:10:45.780744+0000","last_unstale":"2024-05-11T12:10:50.838005+0000","last_undegraded":"2024-05-11T12:10:50.838005+0000","last_fullsized":"2024-05-11T12:10:50.838005+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:03:15.514658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.832468+0000","last_change":"2024-05-11T12:10:45.790012+0000","last_active":"2024-05-11T12:10:50.832468+0000","last_peered":"2024-05-11T12:10:50.832468+0000","last_clean":"2024-05-11T12:10:50.832468+0000","last_became_active":"2024-05-11T12:10:45.786296+0000","last_became_peered":"2024-05-11T12:10:45.786296+0000","last_unstale":"2024-05-11T12:10:50.832468+0000","last_undegraded":"2024-05-11T12:10:50.832468+0000","last_fullsized":"2024-05-11T12:10:50.832468+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:09:48.719906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.085092+0000","last_change":"2024-05-11T12:10:45.766294+0000","last_active":"2024-05-11T12:10:53.085092+0000","last_peered":"2024-05-11T12:10:53.085092+0000","last_clean":"2024-05-11T12:10:53.085092+0000","last_became_active":"2024-05-11T12:10:45.765946+0000","last_became_peered":"2024-05-11T12:10:45.765946+0000","last_unstale":"2024-05-11T12:10:53.085092+0000","last_undegraded":"2024-05-11T12:10:53.085092+0000","last_fullsized":"2024-05-11T12:10:53.085092+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:30:11.665446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.056206+0000","last_change":"2024-05-11T12:10:45.777473+0000","last_active":"2024-05-11T12:10:53.056206+0000","last_peered":"2024-05-11T12:10:53.056206+0000","last_clean":"2024-05-11T12:10:53.056206+0000","last_became_active":"2024-05-11T12:10:45.777140+0000","last_became_peered":"2024-05-11T12:10:45.777140+0000","last_unstale":"2024-05-11T12:10:53.056206+0000","last_undegraded":"2024-05-11T12:10:53.056206+0000","last_fullsized":"2024-05-11T12:10:53.056206+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:12:59.860890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.830966+0000","last_change":"2024-05-11T12:10:45.779099+0000","last_active":"2024-05-11T12:10:50.830966+0000","last_peered":"2024-05-11T12:10:50.830966+0000","last_clean":"2024-05-11T12:10:50.830966+0000","last_became_active":"2024-05-11T12:10:45.767177+0000","last_became_peered":"2024-05-11T12:10:45.767177+0000","last_unstale":"2024-05-11T12:10:50.830966+0000","last_undegraded":"2024-05-11T12:10:50.830966+0000","last_fullsized":"2024-05-11T12:10:50.830966+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:30:11.665446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.178521+0000","last_change":"2024-05-11T12:10:45.781140+0000","last_active":"2024-05-11T12:10:51.178521+0000","last_peered":"2024-05-11T12:10:51.178521+0000","last_clean":"2024-05-11T12:10:51.178521+0000","last_became_active":"2024-05-11T12:10:45.780390+0000","last_became_peered":"2024-05-11T12:10:45.780390+0000","last_unstale":"2024-05-11T12:10:51.178521+0000","last_undegraded":"2024-05-11T12:10:51.178521+0000","last_fullsized":"2024-05-11T12:10:51.178521+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:32:32.325955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.836702+0000","last_change":"2024-05-11T12:10:45.772180+0000","last_active":"2024-05-11T12:10:50.836702+0000","last_peered":"2024-05-11T12:10:50.836702+0000","last_clean":"2024-05-11T12:10:50.836702+0000","last_became_active":"2024-05-11T12:10:45.771785+0000","last_became_peered":"2024-05-11T12:10:45.771785+0000","last_unstale":"2024-05-11T12:10:50.836702+0000","last_undegraded":"2024-05-11T12:10:50.836702+0000","last_fullsized":"2024-05-11T12:10:50.836702+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:27:46.323505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.836913+0000","last_change":"2024-05-11T12:10:45.773903+0000","last_active":"2024-05-11T12:10:50.836913+0000","last_peered":"2024-05-11T12:10:50.836913+0000","last_clean":"2024-05-11T12:10:50.836913+0000","last_became_active":"2024-05-11T12:10:45.773639+0000","last_became_peered":"2024-05-11T12:10:45.773639+0000","last_unstale":"2024-05-11T12:10:50.836913+0000","last_undegraded":"2024-05-11T12:10:50.836913+0000","last_fullsized":"2024-05-11T12:10:50.836913+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:09:48.719906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.988774+0000","last_change":"2024-05-11T12:10:45.794802+0000","last_active":"2024-05-11T12:10:52.988774+0000","last_peered":"2024-05-11T12:10:52.988774+0000","last_clean":"2024-05-11T12:10:52.988774+0000","last_became_active":"2024-05-11T12:10:45.794505+0000","last_became_peered":"2024-05-11T12:10:45.794505+0000","last_unstale":"2024-05-11T12:10:52.988774+0000","last_undegraded":"2024-05-11T12:10:52.988774+0000","last_fullsized":"2024-05-11T12:10:52.988774+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:03:15.514658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.830033+0000","last_change":"2024-05-11T12:10:45.764551+0000","last_active":"2024-05-11T12:10:50.830033+0000","last_peered":"2024-05-11T12:10:50.830033+0000","last_clean":"2024-05-11T12:10:50.830033+0000","last_became_active":"2024-05-11T12:10:45.763049+0000","last_became_peered":"2024-05-11T12:10:45.763049+0000","last_unstale":"2024-05-11T12:10:50.830033+0000","last_undegraded":"2024-05-11T12:10:50.830033+0000","last_fullsized":"2024-05-11T12:10:50.830033+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:11:17.251971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.986814+0000","last_change":"2024-05-11T12:10:45.785129+0000","last_active":"2024-05-11T12:10:52.986814+0000","last_peered":"2024-05-11T12:10:52.986814+0000","last_clean":"2024-05-11T12:10:52.986814+0000","last_became_active":"2024-05-11T12:10:45.784827+0000","last_became_peered":"2024-05-11T12:10:45.784827+0000","last_unstale":"2024-05-11T12:10:52.986814+0000","last_undegraded":"2024-05-11T12:10:52.986814+0000","last_fullsized":"2024-05-11T12:10:52.986814+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:40:33.598752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.833132+0000","last_change":"2024-05-11T12:10:45.785406+0000","last_active":"2024-05-11T12:10:50.833132+0000","last_peered":"2024-05-11T12:10:50.833132+0000","last_clean":"2024-05-11T12:10:50.833132+0000","last_became_active":"2024-05-11T12:10:45.785083+0000","last_became_peered":"2024-05-11T12:10:45.785083+0000","last_unstale":"2024-05-11T12:10:50.833132+0000","last_undegraded":"2024-05-11T12:10:50.833132+0000","last_fullsized":"2024-05-11T12:10:50.833132+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:15:55.637133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.987595+0000","last_change":"2024-05-11T12:10:45.779636+0000","last_active":"2024-05-11T12:10:52.987595+0000","last_peered":"2024-05-11T12:10:52.987595+0000","last_clean":"2024-05-11T12:10:52.987595+0000","last_became_active":"2024-05-11T12:10:45.779327+0000","last_became_peered":"2024-05-11T12:10:45.779327+0000","last_unstale":"2024-05-11T12:10:52.987595+0000","last_undegraded":"2024-05-11T12:10:52.987595+0000","last_fullsized":"2024-05-11T12:10:52.987595+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:11:17.251971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.834940+0000","last_change":"2024-05-11T12:10:45.782057+0000","last_active":"2024-05-11T12:10:50.834940+0000","last_peered":"2024-05-11T12:10:50.834940+0000","last_clean":"2024-05-11T12:10:50.834940+0000","last_became_active":"2024-05-11T12:10:45.781008+0000","last_became_peered":"2024-05-11T12:10:45.781008+0000","last_unstale":"2024-05-11T12:10:50.834940+0000","last_undegraded":"2024-05-11T12:10:50.834940+0000","last_fullsized":"2024-05-11T12:10:50.834940+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:01:30.356378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.056602+0000","last_change":"2024-05-11T12:10:45.783937+0000","last_active":"2024-05-11T12:10:53.056602+0000","last_peered":"2024-05-11T12:10:53.056602+0000","last_clean":"2024-05-11T12:10:53.056602+0000","last_became_active":"2024-05-11T12:10:45.783553+0000","last_became_peered":"2024-05-11T12:10:45.783553+0000","last_unstale":"2024-05-11T12:10:53.056602+0000","last_undegraded":"2024-05-11T12:10:53.056602+0000","last_fullsized":"2024-05-11T12:10:53.056602+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:27:46.323505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.086881+0000","last_change":"2024-05-11T12:10:45.786860+0000","last_active":"2024-05-11T12:10:53.086881+0000","last_peered":"2024-05-11T12:10:53.086881+0000","last_clean":"2024-05-11T12:10:53.086881+0000","last_became_active":"2024-05-11T12:10:45.785741+0000","last_became_peered":"2024-05-11T12:10:45.785741+0000","last_unstale":"2024-05-11T12:10:53.086881+0000","last_undegraded":"2024-05-11T12:10:53.086881+0000","last_fullsized":"2024-05-11T12:10:53.086881+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:09:48.719906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.988637+0000","last_change":"2024-05-11T12:10:45.794362+0000","last_active":"2024-05-11T12:10:52.988637+0000","last_peered":"2024-05-11T12:10:52.988637+0000","last_clean":"2024-05-11T12:10:52.988637+0000","last_became_active":"2024-05-11T12:10:45.793845+0000","last_became_peered":"2024-05-11T12:10:45.793845+0000","last_unstale":"2024-05-11T12:10:52.988637+0000","last_undegraded":"2024-05-11T12:10:52.988637+0000","last_fullsized":"2024-05-11T12:10:52.988637+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:02:47.506141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476745,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":876544,"data_stored":700107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476746,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":112999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476745,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":282624,"data_stored":109247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476747,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":290816,"data_stored":109283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509450,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":112999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509451,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":109317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509450,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":884736,"data_stored":700163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542154,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":109157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T12:11:11.888 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-11T12:11:11.888 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-11T12:11:11.888 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-05-11T12:11:11.889 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-05-11T12:11:11.889 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-05-11T12:11:12.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.131+0000 7fe68a000640 1 Processor -- start 2024-05-11T12:11:12.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.131+0000 7fe68a000640 1 -- start start 2024-05-11T12:11:12.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.133+0000 7fe68a000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe68406d460 0x7fe68406d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:12.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.133+0000 7fe68a000640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe68406fe10 con 0x7fe6840af5d0 2024-05-11T12:11:12.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.133+0000 7fe68a000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe68415fcc0 con 0x7fe6840af1f0 2024-05-11T12:11:12.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.133+0000 7fe68a000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fe6841608d0 con 0x7fe68406d460 2024-05-11T12:11:12.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.133+0000 7fe682a00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fe6840af5d0 0x7fe684073ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49378/0 (socket says 172.21.15.167:49378) 2024-05-11T12:11:12.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.133+0000 7fe682a00640 1 -- 172.21.15.167:0/3708692647 learned_addr learned my addr 172.21.15.167:0/3708692647 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:12.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.134+0000 7fe683e00640 1 --2- 172.21.15.167:0/3708692647 >> v2:172.21.15.204:3300/0 conn(0x7fe68406d460 0x7fe68406d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:12.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.134+0000 7fe682000640 1 -- 172.21.15.167:0/3708692647 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1220074984 0 0) 0x7fe68406fe10 con 0x7fe6840af5d0 2024-05-11T12:11:12.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.134+0000 7fe682000640 1 -- 172.21.15.167:0/3708692647 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe668003520 con 0x7fe6840af5d0 2024-05-11T12:11:12.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.134+0000 7fe682000640 1 -- 172.21.15.167:0/3708692647 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1160508530 0 0) 0x7fe668003520 con 0x7fe6840af5d0 2024-05-11T12:11:12.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.134+0000 7fe682000640 1 -- 172.21.15.167:0/3708692647 >> v2:172.21.15.204:3300/0 conn(0x7fe68406d460 msgr2=0x7fe68406d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:12.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.134+0000 7fe682000640 1 --2- 172.21.15.167:0/3708692647 >> v2:172.21.15.204:3300/0 conn(0x7fe68406d460 0x7fe68406d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:12.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.134+0000 7fe682000640 1 -- 172.21.15.167:0/3708692647 >> v1:172.21.15.204:6789/0 conn(0x7fe6840af1f0 legacy=0x7fe6840cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:12.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.134+0000 7fe682000640 1 -- 172.21.15.167:0/3708692647 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe68415ffa0 con 0x7fe6840af5d0 2024-05-11T12:11:12.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.134+0000 7fe682000640 1 -- 172.21.15.167:0/3708692647 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7fe6700016f0 con 0x7fe6840af5d0 2024-05-11T12:11:12.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.134+0000 7fe682000640 1 -- 172.21.15.167:0/3708692647 >> v1:172.21.15.167:6789/0 conn(0x7fe6840af5d0 legacy=0x7fe684073ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:12.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.134+0000 7fe682000640 1 --2- 172.21.15.167:0/3708692647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe668003aa0 0x7fe668003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:12.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.134+0000 7fe682000640 1 --2- 172.21.15.167:0/3708692647 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe668004800 0x7fe668004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:12.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.134+0000 7fe682000640 1 --2- 172.21.15.167:0/3708692647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6680052e0 0x7fe66800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:12.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.134+0000 7fe682000640 1 -- 172.21.15.167:0/3708692647 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe6841608d0 con 0x7fe668003aa0 2024-05-11T12:11:12.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.134+0000 7fe682000640 1 -- 172.21.15.167:0/3708692647 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe66800c1e0 con 0x7fe6680052e0 2024-05-11T12:11:12.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.134+0000 7fe682000640 1 -- 172.21.15.167:0/3708692647 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe66800c3e0 con 0x7fe668004800 2024-05-11T12:11:12.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.134+0000 7fe683400640 1 --2- 172.21.15.167:0/3708692647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe668003aa0 0x7fe668003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:12.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.135+0000 7fe683e00640 1 --2- 172.21.15.167:0/3708692647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6680052e0 0x7fe66800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:12.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.135+0000 7fe682a00640 1 --2- 172.21.15.167:0/3708692647 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe668004800 0x7fe668004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:12.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.135+0000 7fe683400640 1 -- 172.21.15.167:0/3708692647 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe668004800 msgr2=0x7fe668004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:12.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.135+0000 7fe683400640 1 --2- 172.21.15.167:0/3708692647 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe668004800 0x7fe668004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:12.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.135+0000 7fe683400640 1 -- 172.21.15.167:0/3708692647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6680052e0 msgr2=0x7fe66800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:12.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.135+0000 7fe683400640 1 --2- 172.21.15.167:0/3708692647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6680052e0 0x7fe66800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:12.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.135+0000 7fe683400640 1 -- 172.21.15.167:0/3708692647 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe668003520 con 0x7fe668003aa0 2024-05-11T12:11:12.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.135+0000 7fe683e00640 1 -- 172.21.15.167:0/3708692647 reap_dead start 2024-05-11T12:11:12.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.136+0000 7fe683400640 1 --2- 172.21.15.167:0/3708692647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe668003aa0 0x7fe668003ec0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe674001790 tx=0x7fe668005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed03b8b2ea9ac1ff server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:12.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.137+0000 7fe682000640 1 -- 172.21.15.167:0/3708692647 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6740013b0 con 0x7fe668003aa0 2024-05-11T12:11:12.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.137+0000 7fe682000640 1 -- 172.21.15.167:0/3708692647 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6740052d0 con 0x7fe668003aa0 2024-05-11T12:11:12.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.137+0000 7fe682000640 1 -- 172.21.15.167:0/3708692647 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67402aa90 con 0x7fe668003aa0 2024-05-11T12:11:12.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.137+0000 7fe68a000640 1 -- 172.21.15.167:0/3708692647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe668003aa0 msgr2=0x7fe668003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:12.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.137+0000 7fe68a000640 1 --2- 172.21.15.167:0/3708692647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe668003aa0 0x7fe668003ec0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe674001790 tx=0x7fe668005830 comp rx=0 tx=0).stop 2024-05-11T12:11:12.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.137+0000 7fe68a000640 1 -- 172.21.15.167:0/3708692647 shutdown_connections 2024-05-11T12:11:12.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.137+0000 7fe68a000640 1 --2- 172.21.15.167:0/3708692647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe668003aa0 0x7fe668003ec0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:12.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.137+0000 7fe68a000640 1 -- 172.21.15.167:0/3708692647 >> 172.21.15.167:0/3708692647 conn(0x7fe6840cd8f0 msgr2=0x7fe6840cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:12.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.138+0000 7fe68a000640 1 -- 172.21.15.167:0/3708692647 shutdown_connections 2024-05-11T12:11:12.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.138+0000 7fe68a000640 1 -- 172.21.15.167:0/3708692647 wait complete. 2024-05-11T12:11:12.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.139+0000 7fe68a000640 1 Processor -- start 2024-05-11T12:11:12.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.140+0000 7fe68a000640 1 -- start start 2024-05-11T12:11:12.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.140+0000 7fe68a000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe668003aa0 0x7fe6840732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:12.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.140+0000 7fe68a000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6840737e0 0x7fe6840b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:12.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.140+0000 7fe68a000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6840b5280 0x7fe6840b5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:12.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.140+0000 7fe68a000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe684160560 con 0x7fe668003aa0 2024-05-11T12:11:12.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.140+0000 7fe68a000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe68406de10 con 0x7fe6840b5280 2024-05-11T12:11:12.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.140+0000 7fe68a000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe68415f920 con 0x7fe6840737e0 2024-05-11T12:11:12.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.141+0000 7fe683400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe668003aa0 0x7fe6840732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:12.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.141+0000 7fe683400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe668003aa0 0x7fe6840732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41444/0 (socket says 172.21.15.167:41444) 2024-05-11T12:11:12.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.141+0000 7fe683400640 1 -- 172.21.15.167:0/3689615059 learned_addr learned my addr 172.21.15.167:0/3689615059 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:12.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.141+0000 7fe683400640 1 -- 172.21.15.167:0/3689615059 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6840737e0 msgr2=0x7fe6840b4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:12.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.141+0000 7fe683400640 1 --2- 172.21.15.167:0/3689615059 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6840737e0 0x7fe6840b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:12.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.141+0000 7fe683400640 1 -- 172.21.15.167:0/3689615059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6840b5280 msgr2=0x7fe6840b5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:12.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.141+0000 7fe683400640 1 --2- 172.21.15.167:0/3689615059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6840b5280 0x7fe6840b5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:12.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.141+0000 7fe683400640 1 -- 172.21.15.167:0/3689615059 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6840742b0 con 0x7fe668003aa0 2024-05-11T12:11:12.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.141+0000 7fe683400640 1 --2- 172.21.15.167:0/3689615059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe668003aa0 0x7fe6840732a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe674005040 tx=0x7fe674001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:12.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.142+0000 7fe67b400640 1 -- 172.21.15.167:0/3689615059 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe674042070 con 0x7fe668003aa0 2024-05-11T12:11:12.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.142+0000 7fe67b400640 1 -- 172.21.15.167:0/3689615059 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe674005480 con 0x7fe668003aa0 2024-05-11T12:11:12.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.142+0000 7fe67b400640 1 -- 172.21.15.167:0/3689615059 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe674037040 con 0x7fe668003aa0 2024-05-11T12:11:12.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.142+0000 7fe68a000640 1 -- 172.21.15.167:0/3689615059 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe684073cf0 con 0x7fe668003aa0 2024-05-11T12:11:12.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.142+0000 7fe68a000640 1 -- 172.21.15.167:0/3689615059 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe684154710 con 0x7fe668003aa0 2024-05-11T12:11:12.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.143+0000 7fe68a000640 1 -- 172.21.15.167:0/3689615059 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe68406dbd0 con 0x7fe668003aa0 2024-05-11T12:11:12.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.144+0000 7fe67b400640 1 -- 172.21.15.167:0/3689615059 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe674045070 con 0x7fe668003aa0 2024-05-11T12:11:12.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.145+0000 7fe67b400640 1 --2- 172.21.15.167:0/3689615059 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe6640762d0 0x7fe664078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:12.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.145+0000 7fe67b400640 1 -- 172.21.15.167:0/3689615059 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fe6740b6ff0 con 0x7fe668003aa0 2024-05-11T12:11:12.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.145+0000 7fe682a00640 1 --2- 172.21.15.167:0/3689615059 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe6640762d0 0x7fe664078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:12.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.146+0000 7fe682a00640 1 --2- 172.21.15.167:0/3689615059 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe6640762d0 0x7fe664078790 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe670001eb0 tx=0x7fe670000ed0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:12.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.150+0000 7fe67b400640 1 -- 172.21.15.167:0/3689615059 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe68406dbd0 con 0x7fe668003aa0 2024-05-11T12:11:12.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.408+0000 7fe68a000640 1 -- 172.21.15.167:0/3689615059 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fe684163980 con 0x7fe668003aa0 2024-05-11T12:11:12.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.416+0000 7fe67b400640 1 -- 172.21.15.167:0/3689615059 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+189523 (secure 0 0 0) 0x7fe674080620 con 0x7fe668003aa0 2024-05-11T12:11:12.419 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:11:12.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.420+0000 7fe68a000640 1 -- 172.21.15.167:0/3689615059 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe6640762d0 msgr2=0x7fe664078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:12.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.420+0000 7fe68a000640 1 --2- 172.21.15.167:0/3689615059 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe6640762d0 0x7fe664078790 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe670001eb0 tx=0x7fe670000ed0 comp rx=0 tx=0).stop 2024-05-11T12:11:12.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.420+0000 7fe68a000640 1 -- 172.21.15.167:0/3689615059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe668003aa0 msgr2=0x7fe6840732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:12.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.420+0000 7fe68a000640 1 --2- 172.21.15.167:0/3689615059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe668003aa0 0x7fe6840732a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe674005040 tx=0x7fe674001120 comp rx=0 tx=0).stop 2024-05-11T12:11:12.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.420+0000 7fe68a000640 1 -- 172.21.15.167:0/3689615059 shutdown_connections 2024-05-11T12:11:12.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.420+0000 7fe68a000640 1 --2- 172.21.15.167:0/3689615059 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe6640762d0 0x7fe664078790 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:12.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.420+0000 7fe68a000640 1 --2- 172.21.15.167:0/3689615059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6840b5280 0x7fe6840b5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:12.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.420+0000 7fe68a000640 1 --2- 172.21.15.167:0/3689615059 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6840737e0 0x7fe6840b4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:12.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.420+0000 7fe68a000640 1 --2- 172.21.15.167:0/3689615059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe668003aa0 0x7fe6840732a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:12.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.420+0000 7fe68a000640 1 -- 172.21.15.167:0/3689615059 >> 172.21.15.167:0/3689615059 conn(0x7fe6840cd8f0 msgr2=0x7fe684162160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:12.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.420+0000 7fe68a000640 1 -- 172.21.15.167:0/3689615059 shutdown_connections 2024-05-11T12:11:12.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.420+0000 7fe68a000640 1 -- 172.21.15.167:0/3689615059 wait complete. 2024-05-11T12:11:12.439 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":5,"flags":0,"active_gid":4106,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":578352559},{"type":"v1","addr":"172.21.15.167:6833","nonce":578352559}]},"active_addr":"172.21.15.167:6833/578352559","active_change":"2024-05-11T12:10:25.211934+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4108,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.167:0","nonce":1552233244}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.167:0","nonce":892609630}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.167:0","nonce":300615192}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.167:0","nonce":1126199566}]}]} 2024-05-11T12:11:12.444 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-05-11T12:11:12.444 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-11T12:11:12.444 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:11:12.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.642+0000 7f6dbd600640 1 Processor -- start 2024-05-11T12:11:12.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.642+0000 7f6dbd600640 1 -- start start 2024-05-11T12:11:12.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.643+0000 7f6dbd600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6db8145b00 0x7f6db806f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:12.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.643+0000 7f6dbd600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6db8146770 0x7f6db80ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:12.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.643+0000 7f6dbd600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6db806fbc0 con 0x7f6db80cef10 2024-05-11T12:11:12.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.643+0000 7f6dbd600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f6db80b4520 con 0x7f6db8145b00 2024-05-11T12:11:12.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.643+0000 7f6dbd600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6db80b4800 con 0x7f6db8146770 2024-05-11T12:11:12.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.644+0000 7f6db7400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6db8146770 0x7f6db80ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:12.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.644+0000 7f6db7400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6db8146770 0x7f6db80ce9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:59580/0 (socket says 172.21.15.167:59580) 2024-05-11T12:11:12.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.644+0000 7f6db7400640 1 -- 172.21.15.167:0/2036492132 learned_addr learned my addr 172.21.15.167:0/2036492132 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:12.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.644+0000 7f6dbcc00640 1 --1- 172.21.15.167:0/2036492132 >> v1:172.21.15.204:6789/0 conn(0x7f6db80cef10 0x7f6db806c660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.167:55182/0 (socket says 172.21.15.167:55182) 2024-05-11T12:11:12.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.644+0000 7f6db7e00640 1 --2- 172.21.15.167:0/2036492132 >> v2:172.21.15.204:3300/0 conn(0x7f6db8145b00 0x7f6db806f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:12.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.645+0000 7f6db7400640 1 -- 172.21.15.167:0/2036492132 >> v2:172.21.15.204:3300/0 conn(0x7f6db8145b00 msgr2=0x7f6db806f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:12.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.645+0000 7f6db7400640 1 --2- 172.21.15.167:0/2036492132 >> v2:172.21.15.204:3300/0 conn(0x7f6db8145b00 0x7f6db806f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:12.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.645+0000 7f6db7400640 1 -- 172.21.15.167:0/2036492132 >> v1:172.21.15.204:6789/0 conn(0x7f6db80cef10 legacy=0x7f6db806c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:12.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.645+0000 7f6db7400640 1 -- 172.21.15.167:0/2036492132 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6db80b4ae0 con 0x7f6db8146770 2024-05-11T12:11:12.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.646+0000 7f6db7400640 1 --2- 172.21.15.167:0/2036492132 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6db8146770 0x7f6db80ce9d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6da4009870 tx=0x7f6da402a0d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba108a393f862d2b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:12.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.647+0000 7f6db6a00640 1 -- 172.21.15.167:0/2036492132 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da4037070 con 0x7f6db8146770 2024-05-11T12:11:12.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.647+0000 7f6db6a00640 1 -- 172.21.15.167:0/2036492132 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6da402ad60 con 0x7f6db8146770 2024-05-11T12:11:12.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.647+0000 7f6db6a00640 1 -- 172.21.15.167:0/2036492132 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da4032660 con 0x7f6db8146770 2024-05-11T12:11:12.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.647+0000 7f6dbd600640 1 -- 172.21.15.167:0/2036492132 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6db8146770 msgr2=0x7f6db80ce9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:12.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.647+0000 7f6dbd600640 1 --2- 172.21.15.167:0/2036492132 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6db8146770 0x7f6db80ce9d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6da4009870 tx=0x7f6da402a0d0 comp rx=0 tx=0).stop 2024-05-11T12:11:12.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.648+0000 7f6dbd600640 1 -- 172.21.15.167:0/2036492132 shutdown_connections 2024-05-11T12:11:12.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.648+0000 7f6dbd600640 1 --2- 172.21.15.167:0/2036492132 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6db8146770 0x7f6db80ce9d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:12.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.648+0000 7f6dbd600640 1 --2- 172.21.15.167:0/2036492132 >> v2:172.21.15.204:3300/0 conn(0x7f6db8145b00 0x7f6db806f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:12.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.648+0000 7f6dbd600640 1 -- 172.21.15.167:0/2036492132 >> 172.21.15.167:0/2036492132 conn(0x7f6db80b3460 msgr2=0x7f6db80d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:12.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.648+0000 7f6dbd600640 1 -- 172.21.15.167:0/2036492132 shutdown_connections 2024-05-11T12:11:12.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.648+0000 7f6dbd600640 1 -- 172.21.15.167:0/2036492132 wait complete. 2024-05-11T12:11:12.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.649+0000 7f6dbd600640 1 Processor -- start 2024-05-11T12:11:12.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.649+0000 7f6dbd600640 1 -- start start 2024-05-11T12:11:12.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.650+0000 7f6dbd600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6db80cef10 0x7f6db8073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:12.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.650+0000 7f6dbd600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6db8145b00 0x7f6db8132d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:12.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.650+0000 7f6dbd600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6db8146770 0x7f6db815eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:12.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.650+0000 7f6dbd600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6db806f980 con 0x7f6db80cef10 2024-05-11T12:11:12.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.650+0000 7f6dbd600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6db806f800 con 0x7f6db8145b00 2024-05-11T12:11:12.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.650+0000 7f6dbd600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6db806cd70 con 0x7f6db8146770 2024-05-11T12:11:12.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.651+0000 7f6db7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6db80cef10 0x7f6db8073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:12.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.651+0000 7f6db7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6db80cef10 0x7f6db8073b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41448/0 (socket says 172.21.15.167:41448) 2024-05-11T12:11:12.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.651+0000 7f6db7e00640 1 -- 172.21.15.167:0/554636790 learned_addr learned my addr 172.21.15.167:0/554636790 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:12.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.651+0000 7f6dbcc00640 1 --2- 172.21.15.167:0/554636790 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6db8146770 0x7f6db815eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:12.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.651+0000 7f6db7400640 1 --2- 172.21.15.167:0/554636790 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6db8145b00 0x7f6db8132d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:12.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.651+0000 7f6db7e00640 1 -- 172.21.15.167:0/554636790 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6db8146770 msgr2=0x7f6db815eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:12.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.651+0000 7f6db7e00640 1 --2- 172.21.15.167:0/554636790 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6db8146770 0x7f6db815eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:12.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.651+0000 7f6db7e00640 1 -- 172.21.15.167:0/554636790 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6db8145b00 msgr2=0x7f6db8132d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:12.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.651+0000 7f6db7e00640 1 --2- 172.21.15.167:0/554636790 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6db8145b00 0x7f6db8132d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:12.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.651+0000 7f6db7e00640 1 -- 172.21.15.167:0/554636790 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6db81332b0 con 0x7f6db80cef10 2024-05-11T12:11:12.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.651+0000 7f6dbcc00640 1 --2- 172.21.15.167:0/554636790 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6db8146770 0x7f6db815eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:12.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.651+0000 7f6db7400640 1 --2- 172.21.15.167:0/554636790 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6db8145b00 0x7f6db8132d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:12.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.652+0000 7f6db7e00640 1 --2- 172.21.15.167:0/554636790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6db80cef10 0x7f6db8073b40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6da800e9b0 tx=0x7f6da800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:12.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.652+0000 7f6dafe00640 1 -- 172.21.15.167:0/554636790 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da8017070 con 0x7f6db80cef10 2024-05-11T12:11:12.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.652+0000 7f6dafe00640 1 -- 172.21.15.167:0/554636790 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6da800c9a0 con 0x7f6db80cef10 2024-05-11T12:11:12.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.652+0000 7f6dafe00640 1 -- 172.21.15.167:0/554636790 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da800cd90 con 0x7f6db80cef10 2024-05-11T12:11:12.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.652+0000 7f6dbd600640 1 -- 172.21.15.167:0/554636790 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6db8133540 con 0x7f6db80cef10 2024-05-11T12:11:12.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.652+0000 7f6dbd600640 1 -- 172.21.15.167:0/554636790 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6db8133b00 con 0x7f6db80cef10 2024-05-11T12:11:12.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.653+0000 7f6dbd600640 1 -- 172.21.15.167:0/554636790 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6db80cd040 con 0x7f6db80cef10 2024-05-11T12:11:12.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.654+0000 7f6dafe00640 1 -- 172.21.15.167:0/554636790 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6da801a070 con 0x7f6db80cef10 2024-05-11T12:11:12.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.655+0000 7f6dafe00640 1 --2- 172.21.15.167:0/554636790 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f6d980762d0 0x7f6d98078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:12.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.655+0000 7f6dafe00640 1 -- 172.21.15.167:0/554636790 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f6da80979c0 con 0x7f6db80cef10 2024-05-11T12:11:12.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.655+0000 7f6db7400640 1 --2- 172.21.15.167:0/554636790 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f6d980762d0 0x7f6d98078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:12.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.656+0000 7f6db7400640 1 --2- 172.21.15.167:0/554636790 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f6d980762d0 0x7f6d98078790 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6da4034000 tx=0x7f6da402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:12.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.659+0000 7f6dafe00640 1 -- 172.21.15.167:0/554636790 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6db80cd040 con 0x7f6db80cef10 2024-05-11T12:11:12.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.868+0000 7f6dbd600640 1 -- 172.21.15.167:0/554636790 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6db80694b0 con 0x7f6db80cef10 2024-05-11T12:11:12.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.870+0000 7f6dafe00640 1 -- 172.21.15.167:0/554636790 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18562 (secure 0 0 0) 0x7f6da8060f40 con 0x7f6db80cef10 2024-05-11T12:11:12.870 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:11:12.870 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":27,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:10:50.284769+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:10:43.084511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T12:10:44.157782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-11T12:10:47.814674+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:11:12.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.872+0000 7f6dbd600640 1 -- 172.21.15.167:0/554636790 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f6d980762d0 msgr2=0x7f6d98078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:12.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.872+0000 7f6dbd600640 1 --2- 172.21.15.167:0/554636790 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f6d980762d0 0x7f6d98078790 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6da4034000 tx=0x7f6da402e000 comp rx=0 tx=0).stop 2024-05-11T12:11:12.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.872+0000 7f6dbd600640 1 -- 172.21.15.167:0/554636790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6db80cef10 msgr2=0x7f6db8073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:12.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.872+0000 7f6dbd600640 1 --2- 172.21.15.167:0/554636790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6db80cef10 0x7f6db8073b40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6da800e9b0 tx=0x7f6da800ee70 comp rx=0 tx=0).stop 2024-05-11T12:11:12.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.873+0000 7f6dbd600640 1 -- 172.21.15.167:0/554636790 shutdown_connections 2024-05-11T12:11:12.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.873+0000 7f6dbd600640 1 --2- 172.21.15.167:0/554636790 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6db8146770 0x7f6db815eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:12.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.873+0000 7f6dbd600640 1 --2- 172.21.15.167:0/554636790 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f6d980762d0 0x7f6d98078790 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:12.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.873+0000 7f6dbd600640 1 --2- 172.21.15.167:0/554636790 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6db8145b00 0x7f6db8132d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:12.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.873+0000 7f6dbd600640 1 --2- 172.21.15.167:0/554636790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6db80cef10 0x7f6db8073b40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:12.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.873+0000 7f6dbd600640 1 -- 172.21.15.167:0/554636790 >> 172.21.15.167:0/554636790 conn(0x7f6db80b3460 msgr2=0x7f6db8160900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:12.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.873+0000 7f6dbd600640 1 -- 172.21.15.167:0/554636790 shutdown_connections 2024-05-11T12:11:12.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:12.873+0000 7f6dbd600640 1 -- 172.21.15.167:0/554636790 wait complete. 2024-05-11T12:11:12.891 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-11T12:11:12.891 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:11:13.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.087+0000 7fb799e00640 1 Processor -- start 2024-05-11T12:11:13.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.087+0000 7fb799e00640 1 -- start start 2024-05-11T12:11:13.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.089+0000 7fb799e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb794157750 0x7fb79415d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.089+0000 7fb799e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb7940b3ea0 0x7fb794161490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.089+0000 7fb799e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb79415dc80 con 0x7fb7941619d0 2024-05-11T12:11:13.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.089+0000 7fb799e00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fb794161000 con 0x7fb794157750 2024-05-11T12:11:13.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.089+0000 7fb799e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fb79415df30 con 0x7fb7940b3ea0 2024-05-11T12:11:13.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.089+0000 7fb792a00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb794157750 0x7fb79415d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.089+0000 7fb792a00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb794157750 0x7fb79415d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41464/0 (socket says 172.21.15.167:41464) 2024-05-11T12:11:13.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.089+0000 7fb792a00640 1 -- 172.21.15.167:0/2583203958 learned_addr learned my addr 172.21.15.167:0/2583203958 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:13.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.090+0000 7fb792000640 1 --2- 172.21.15.167:0/2583203958 >> v2:172.21.15.204:3300/0 conn(0x7fb7940b3ea0 0x7fb794161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.090+0000 7fb792a00640 1 -- 172.21.15.167:0/2583203958 >> v2:172.21.15.204:3300/0 conn(0x7fb7940b3ea0 msgr2=0x7fb794161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:13.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.090+0000 7fb792a00640 1 --2- 172.21.15.167:0/2583203958 >> v2:172.21.15.204:3300/0 conn(0x7fb7940b3ea0 0x7fb794161490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.090+0000 7fb792a00640 1 -- 172.21.15.167:0/2583203958 >> v1:172.21.15.204:6789/0 conn(0x7fb7941619d0 legacy=0x7fb79406d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:13.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.090+0000 7fb792a00640 1 -- 172.21.15.167:0/2583203958 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb79415e210 con 0x7fb794157750 2024-05-11T12:11:13.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.090+0000 7fb792000640 1 --2- 172.21.15.167:0/2583203958 >> v2:172.21.15.204:3300/0 conn(0x7fb7940b3ea0 0x7fb794161490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:13.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.091+0000 7fb792a00640 1 --2- 172.21.15.167:0/2583203958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb794157750 0x7fb79415d740 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb7940702f0 tx=0x7fb78402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=739cc0f1a20d1924 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:13.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.091+0000 7fb791600640 1 -- 172.21.15.167:0/2583203958 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb784037070 con 0x7fb794157750 2024-05-11T12:11:13.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.091+0000 7fb791600640 1 -- 172.21.15.167:0/2583203958 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb784032440 con 0x7fb794157750 2024-05-11T12:11:13.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.091+0000 7fb791600640 1 -- 172.21.15.167:0/2583203958 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb784032810 con 0x7fb794157750 2024-05-11T12:11:13.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.092+0000 7fb799e00640 1 -- 172.21.15.167:0/2583203958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb794157750 msgr2=0x7fb79415d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.092+0000 7fb799e00640 1 --2- 172.21.15.167:0/2583203958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb794157750 0x7fb79415d740 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb7940702f0 tx=0x7fb78402a1e0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.092+0000 7fb799e00640 1 -- 172.21.15.167:0/2583203958 shutdown_connections 2024-05-11T12:11:13.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.092+0000 7fb799e00640 1 --2- 172.21.15.167:0/2583203958 >> v2:172.21.15.204:3300/0 conn(0x7fb7940b3ea0 0x7fb794161490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.092+0000 7fb799e00640 1 --2- 172.21.15.167:0/2583203958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb794157750 0x7fb79415d740 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.092+0000 7fb799e00640 1 -- 172.21.15.167:0/2583203958 >> 172.21.15.167:0/2583203958 conn(0x7fb7940cd8f0 msgr2=0x7fb7940cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:13.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.092+0000 7fb799e00640 1 -- 172.21.15.167:0/2583203958 shutdown_connections 2024-05-11T12:11:13.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.092+0000 7fb799e00640 1 -- 172.21.15.167:0/2583203958 wait complete. 2024-05-11T12:11:13.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.094+0000 7fb799e00640 1 Processor -- start 2024-05-11T12:11:13.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.094+0000 7fb799e00640 1 -- start start 2024-05-11T12:11:13.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.095+0000 7fb799e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7940b3ea0 0x7fb79415cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.095+0000 7fb799e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb794157750 0x7fb794072280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.095+0000 7fb799e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7941619d0 0x7fb794145e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.095+0000 7fb799e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb79406db70 con 0x7fb7940b3ea0 2024-05-11T12:11:13.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.095+0000 7fb799e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb794157b30 con 0x7fb7941619d0 2024-05-11T12:11:13.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.095+0000 7fb799e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb79406dcf0 con 0x7fb794157750 2024-05-11T12:11:13.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.095+0000 7fb792a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7940b3ea0 0x7fb79415cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.095+0000 7fb792a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7940b3ea0 0x7fb79415cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41468/0 (socket says 172.21.15.167:41468) 2024-05-11T12:11:13.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.095+0000 7fb792a00640 1 -- 172.21.15.167:0/3009914306 learned_addr learned my addr 172.21.15.167:0/3009914306 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:13.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.096+0000 7fb792a00640 1 -- 172.21.15.167:0/3009914306 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb794157750 msgr2=0x7fb794072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.096+0000 7fb792000640 1 --2- 172.21.15.167:0/3009914306 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb794157750 0x7fb794072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.096+0000 7fb792a00640 1 --2- 172.21.15.167:0/3009914306 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb794157750 0x7fb794072280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.096+0000 7fb792a00640 1 -- 172.21.15.167:0/3009914306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7941619d0 msgr2=0x7fb794145e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.096+0000 7fb792a00640 1 --2- 172.21.15.167:0/3009914306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7941619d0 0x7fb794145e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.096+0000 7fb792a00640 1 -- 172.21.15.167:0/3009914306 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb79415be90 con 0x7fb7940b3ea0 2024-05-11T12:11:13.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.096+0000 7fb792a00640 1 --2- 172.21.15.167:0/3009914306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7940b3ea0 0x7fb79415cc30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb784000c00 tx=0x7fb7840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:13.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.096+0000 7fb78b400640 1 -- 172.21.15.167:0/3009914306 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb784037040 con 0x7fb7940b3ea0 2024-05-11T12:11:13.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.096+0000 7fb78b400640 1 -- 172.21.15.167:0/3009914306 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb784042070 con 0x7fb7940b3ea0 2024-05-11T12:11:13.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.096+0000 7fb799e00640 1 -- 172.21.15.167:0/3009914306 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb794072cb0 con 0x7fb7940b3ea0 2024-05-11T12:11:13.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.096+0000 7fb78b400640 1 -- 172.21.15.167:0/3009914306 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb784030bb0 con 0x7fb7940b3ea0 2024-05-11T12:11:13.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.097+0000 7fb799e00640 1 -- 172.21.15.167:0/3009914306 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb79415c430 con 0x7fb7940b3ea0 2024-05-11T12:11:13.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.098+0000 7fb799e00640 1 -- 172.21.15.167:0/3009914306 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb794069590 con 0x7fb7940b3ea0 2024-05-11T12:11:13.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.099+0000 7fb78b400640 1 -- 172.21.15.167:0/3009914306 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb784047460 con 0x7fb7940b3ea0 2024-05-11T12:11:13.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.100+0000 7fb78b400640 1 --2- 172.21.15.167:0/3009914306 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb7740762d0 0x7fb774078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.100+0000 7fb78b400640 1 -- 172.21.15.167:0/3009914306 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fb7840b8560 con 0x7fb7940b3ea0 2024-05-11T12:11:13.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.100+0000 7fb792000640 1 --2- 172.21.15.167:0/3009914306 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb7740762d0 0x7fb774078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.101 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.101+0000 7fb792000640 1 --2- 172.21.15.167:0/3009914306 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb7740762d0 0x7fb774078790 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb79406dae0 tx=0x7fb780008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:13.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.105+0000 7fb78b400640 1 -- 172.21.15.167:0/3009914306 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb784081ae0 con 0x7fb7940b3ea0 2024-05-11T12:11:13.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.310+0000 7fb799e00640 1 -- 172.21.15.167:0/3009914306 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb79415c830 con 0x7fb7940b3ea0 2024-05-11T12:11:13.313 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.313+0000 7fb78b400640 1 -- 172.21.15.167:0/3009914306 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18562 (secure 0 0 0) 0x7fb7840813e0 con 0x7fb7940b3ea0 2024-05-11T12:11:13.314 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:11:13.314 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":27,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:10:50.284769+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:10:43.084511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T12:10:44.157782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-11T12:10:47.814674+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:11:13.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.316+0000 7fb799e00640 1 -- 172.21.15.167:0/3009914306 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb7740762d0 msgr2=0x7fb774078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.316+0000 7fb799e00640 1 --2- 172.21.15.167:0/3009914306 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb7740762d0 0x7fb774078790 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb79406dae0 tx=0x7fb780008040 comp rx=0 tx=0).stop 2024-05-11T12:11:13.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.316+0000 7fb799e00640 1 -- 172.21.15.167:0/3009914306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7940b3ea0 msgr2=0x7fb79415cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.316+0000 7fb799e00640 1 --2- 172.21.15.167:0/3009914306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7940b3ea0 0x7fb79415cc30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb784000c00 tx=0x7fb7840046c0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.316+0000 7fb799e00640 1 -- 172.21.15.167:0/3009914306 shutdown_connections 2024-05-11T12:11:13.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.316+0000 7fb799e00640 1 --2- 172.21.15.167:0/3009914306 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb7740762d0 0x7fb774078790 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.316+0000 7fb799e00640 1 --2- 172.21.15.167:0/3009914306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7941619d0 0x7fb794145e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.316+0000 7fb799e00640 1 --2- 172.21.15.167:0/3009914306 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb794157750 0x7fb794072280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.316+0000 7fb799e00640 1 --2- 172.21.15.167:0/3009914306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7940b3ea0 0x7fb79415cc30 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.316+0000 7fb799e00640 1 -- 172.21.15.167:0/3009914306 >> 172.21.15.167:0/3009914306 conn(0x7fb7940cd8f0 msgr2=0x7fb79406c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:13.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.316+0000 7fb799e00640 1 -- 172.21.15.167:0/3009914306 shutdown_connections 2024-05-11T12:11:13.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.316+0000 7fb799e00640 1 -- 172.21.15.167:0/3009914306 wait complete. 2024-05-11T12:11:13.334 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-11T12:11:13.335 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-11T12:11:13.335 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-11T12:11:13.336 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-11T12:11:13.336 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-05-11T12:11:13.336 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-05-11T12:11:13.337 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-05-11T12:11:13.337 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-05-11T12:11:13.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.616+0000 7f65f9800640 1 Processor -- start 2024-05-11T12:11:13.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.617+0000 7f65f9800640 1 -- start start 2024-05-11T12:11:13.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.618+0000 7fe75b600640 1 Processor -- start 2024-05-11T12:11:13.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.618+0000 7fe75b600640 1 -- start start 2024-05-11T12:11:13.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.619+0000 7f65f9800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65ec0548a0 0x7f65ec054ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.620+0000 7f65f3e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f65ec0e7a50 0x7f65ec049b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49390/0 (socket says 172.21.15.167:49390) 2024-05-11T12:11:13.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.620+0000 7fe75b600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe74c0eacf0 0x7fe74c04fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.620+0000 7f65f3400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65ec0548a0 0x7f65ec054ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.620+0000 7f65f3e00640 1 -- 172.21.15.167:0/2860184983 learned_addr learned my addr 172.21.15.167:0/2860184983 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:13.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.621+0000 7f65f9800640 1 -- 172.21.15.167:0/2860184983 --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65ec0e7e30 con 0x7f65ec0e7a50 2024-05-11T12:11:13.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.621+0000 7f65f9800640 1 -- 172.21.15.167:0/2860184983 --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65ec03b5a0 con 0x7f65ec03c360 2024-05-11T12:11:13.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.621+0000 7f65f9800640 1 -- 172.21.15.167:0/2860184983 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f65ec03b880 con 0x7f65ec0548a0 2024-05-11T12:11:13.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.622+0000 7f65f2a00640 1 -- 172.21.15.167:0/2860184983 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 124650621 0 0) 0x7f65ec0e7e30 con 0x7f65ec0e7a50 2024-05-11T12:11:13.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.622+0000 7f65f2a00640 1 -- 172.21.15.167:0/2860184983 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f65dc003520 con 0x7f65ec0e7a50 2024-05-11T12:11:13.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.622+0000 7f65f2a00640 1 -- 172.21.15.167:0/2860184983 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1040485683 0 0) 0x7f65ec03b5a0 con 0x7f65ec03c360 2024-05-11T12:11:13.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.623+0000 7f65f2a00640 1 -- 172.21.15.167:0/2860184983 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f65ec0e7e30 con 0x7f65ec03c360 2024-05-11T12:11:13.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.623+0000 7f65f2a00640 1 -- 172.21.15.167:0/2860184983 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3584621011 0 0) 0x7f65dc003520 con 0x7f65ec0e7a50 2024-05-11T12:11:13.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.623+0000 7f65f2a00640 1 -- 172.21.15.167:0/2860184983 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65ec0548a0 msgr2=0x7f65ec054ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:13.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.623+0000 7f65f2a00640 1 --2- 172.21.15.167:0/2860184983 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65ec0548a0 0x7f65ec054ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.623+0000 7f65f2a00640 1 -- 172.21.15.167:0/2860184983 >> v1:172.21.15.204:6789/0 conn(0x7f65ec03c360 legacy=0x7f65ec051320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.623+0000 7f65f2a00640 1 -- 172.21.15.167:0/2860184983 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65ec03bb60 con 0x7f65ec0e7a50 2024-05-11T12:11:13.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.623+0000 7f65f2a00640 1 -- 172.21.15.167:0/2860184983 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f65e80016f0 con 0x7f65ec0e7a50 2024-05-11T12:11:13.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.623+0000 7f65f2a00640 1 -- 172.21.15.167:0/2860184983 >> v1:172.21.15.167:6789/0 conn(0x7f65ec0e7a50 legacy=0x7f65ec049b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.623+0000 7f65f2a00640 1 --2- 172.21.15.167:0/2860184983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65dc003b70 0x7f65dc003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.623+0000 7f65f2a00640 1 --2- 172.21.15.167:0/2860184983 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65ec0548a0 0x7f65dc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.623+0000 7f65f2a00640 1 --2- 172.21.15.167:0/2860184983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65dc004d40 0x7f65dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.623+0000 7f65f2a00640 1 -- 172.21.15.167:0/2860184983 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec03b880 con 0x7f65dc004d40 2024-05-11T12:11:13.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.623+0000 7f65f2a00640 1 -- 172.21.15.167:0/2860184983 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f65dc00bb20 con 0x7f65dc003b70 2024-05-11T12:11:13.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.623+0000 7f65f2a00640 1 -- 172.21.15.167:0/2860184983 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f65dc00bd20 con 0x7f65ec0548a0 2024-05-11T12:11:13.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.624+0000 7f65f8e00640 1 --2- 172.21.15.167:0/2860184983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65dc004d40 0x7f65dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.624+0000 7f65f3e00640 1 --2- 172.21.15.167:0/2860184983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65dc003b70 0x7f65dc003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.620+0000 7fe75b600640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe74c053710 con 0x7fe74c006e90 2024-05-11T12:11:13.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.620+0000 7fe75b600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe74c0539f0 con 0x7fe74c04ffc0 2024-05-11T12:11:13.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.620+0000 7fe75b600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe74c054320 con 0x7fe74c0eacf0 2024-05-11T12:11:13.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.621+0000 7fe753e00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fe74c04ffc0 0x7fe74c037bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.167:55214/0 (socket says 172.21.15.167:55214) 2024-05-11T12:11:13.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.621+0000 7fe753e00640 1 -- 172.21.15.167:0/3692710662 learned_addr learned my addr 172.21.15.167:0/3692710662 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:13.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.621+0000 7fe758c00640 1 --2- 172.21.15.167:0/3692710662 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe74c0eacf0 0x7fe74c04fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.622+0000 7fe753400640 1 -- 172.21.15.167:0/3692710662 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 979387100 0 0) 0x7fe74c053710 con 0x7fe74c006e90 2024-05-11T12:11:13.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.622+0000 7fe753400640 1 -- 172.21.15.167:0/3692710662 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe73c003520 con 0x7fe74c006e90 2024-05-11T12:11:13.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.622+0000 7fe753400640 1 -- 172.21.15.167:0/3692710662 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3304322903 0 0) 0x7fe74c0539f0 con 0x7fe74c04ffc0 2024-05-11T12:11:13.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.622+0000 7fe753400640 1 -- 172.21.15.167:0/3692710662 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe74c053710 con 0x7fe74c04ffc0 2024-05-11T12:11:13.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.622+0000 7fe758c00640 1 -- 172.21.15.167:0/3692710662 >> v1:172.21.15.204:6789/0 conn(0x7fe74c04ffc0 legacy=0x7fe74c037bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.622+0000 7fe758c00640 1 -- 172.21.15.167:0/3692710662 >> v1:172.21.15.167:6789/0 conn(0x7fe74c006e90 legacy=0x7fe74c0382d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.622+0000 7fe758c00640 1 -- 172.21.15.167:0/3692710662 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe74c053cd0 con 0x7fe74c0eacf0 2024-05-11T12:11:13.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.624+0000 7fe758c00640 1 --2- 172.21.15.167:0/3692710662 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe74c0eacf0 0x7fe74c04fa80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe75402e140 tx=0x7fe75406a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64c436f688d9b65c server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:13.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.624+0000 7fe753400640 1 -- 172.21.15.167:0/3692710662 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7540b2070 con 0x7fe74c0eacf0 2024-05-11T12:11:13.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.624+0000 7fe753400640 1 -- 172.21.15.167:0/3692710662 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7540ab7d0 con 0x7fe74c0eacf0 2024-05-11T12:11:13.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.624+0000 7fe753400640 1 -- 172.21.15.167:0/3692710662 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7540abc40 con 0x7fe74c0eacf0 2024-05-11T12:11:13.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.624+0000 7f65f3400640 1 --2- 172.21.15.167:0/2860184983 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65ec0548a0 0x7f65dc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.625+0000 7f65f8e00640 1 -- 172.21.15.167:0/2860184983 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65ec0548a0 msgr2=0x7f65dc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:13.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.625+0000 7f65f8e00640 1 --2- 172.21.15.167:0/2860184983 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65ec0548a0 0x7f65dc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.625+0000 7f65f8e00640 1 -- 172.21.15.167:0/2860184983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65dc003b70 msgr2=0x7f65dc003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:13.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.625+0000 7f65f8e00640 1 --2- 172.21.15.167:0/2860184983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65dc003b70 0x7f65dc003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.625+0000 7f65f8e00640 1 -- 172.21.15.167:0/2860184983 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65ec03bb60 con 0x7f65dc004d40 2024-05-11T12:11:13.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.625+0000 7fe75b600640 1 -- 172.21.15.167:0/3692710662 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe74c0eacf0 msgr2=0x7fe74c04fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.625+0000 7fe75b600640 1 --2- 172.21.15.167:0/3692710662 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe74c0eacf0 0x7fe74c04fa80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe75402e140 tx=0x7fe75406a230 comp rx=0 tx=0).stop 2024-05-11T12:11:13.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.626+0000 7fe75b600640 1 -- 172.21.15.167:0/3692710662 shutdown_connections 2024-05-11T12:11:13.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.626+0000 7fe75b600640 1 --2- 172.21.15.167:0/3692710662 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe74c0eacf0 0x7fe74c04fa80 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.626+0000 7fe75b600640 1 -- 172.21.15.167:0/3692710662 >> 172.21.15.167:0/3692710662 conn(0x7fe74c009fa0 msgr2=0x7fe74c00a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:13.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.626+0000 7fe75b600640 1 -- 172.21.15.167:0/3692710662 shutdown_connections 2024-05-11T12:11:13.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.626+0000 7fe75b600640 1 -- 172.21.15.167:0/3692710662 wait complete. 2024-05-11T12:11:13.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.626+0000 7f8a20e00640 1 Processor -- start 2024-05-11T12:11:13.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.626+0000 7f8a20e00640 1 -- start start 2024-05-11T12:11:13.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.624+0000 7f27a3a00640 1 Processor -- start 2024-05-11T12:11:13.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.624+0000 7f27a3a00640 1 -- start start 2024-05-11T12:11:13.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.626+0000 7f27a3a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f279c0b6550 0x7f279c0cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.626+0000 7f27a3a00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f279c06ebf0 0x7f279c06f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.626+0000 7f27a3a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f279c06e460 con 0x7f279c15b410 2024-05-11T12:11:13.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.626+0000 7f27a3a00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f279c072b00 con 0x7f279c06ebf0 2024-05-11T12:11:13.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.626+0000 7f27a3a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f279c072da0 con 0x7f279c0b6550 2024-05-11T12:11:13.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.627+0000 7f27a1000640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f279c15b410 0x7f279c0cf0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.167:55230/0 (socket says 172.21.15.167:55230) 2024-05-11T12:11:13.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.627+0000 7f27a1000640 1 -- 172.21.15.167:0/4292902804 learned_addr learned my addr 172.21.15.167:0/4292902804 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:13.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.627+0000 7f27a1a00640 1 --2- 172.21.15.167:0/4292902804 >> v2:172.21.15.167:3300/0 conn(0x7f279c06ebf0 0x7f279c06f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.627+0000 7f279be00640 1 --2- 172.21.15.167:0/4292902804 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f279c0b6550 0x7f279c0cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.628+0000 7f279be00640 1 -- 172.21.15.167:0/4292902804 >> v2:172.21.15.167:3300/0 conn(0x7f279c06ebf0 msgr2=0x7f279c06f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:13.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.628+0000 7f279be00640 1 --2- 172.21.15.167:0/4292902804 >> v2:172.21.15.167:3300/0 conn(0x7f279c06ebf0 0x7f279c06f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.628+0000 7f279be00640 1 -- 172.21.15.167:0/4292902804 >> v1:172.21.15.204:6789/0 conn(0x7f279c15b410 legacy=0x7f279c0cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.628+0000 7f279b400640 1 -- 172.21.15.167:0/4292902804 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3000148434 0 0) 0x7f279c06e460 con 0x7f279c15b410 2024-05-11T12:11:13.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.628+0000 7f8a20e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f8a140e30c0 0x7f8a14057d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.628+0000 7fe75b600640 1 Processor -- start 2024-05-11T12:11:13.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.628+0000 7f8a20e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a1403e840 0x7f8a14053bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.628+0000 7f65f8e00640 1 --2- 172.21.15.167:0/2860184983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65dc004d40 0x7f65dc0051c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f65f40520f0 tx=0x7f65dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a47b898ec36705 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:13.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.629+0000 7fe75b600640 1 -- start start 2024-05-11T12:11:13.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.628+0000 7f8a20e00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a140582d0 con 0x7f8a14054130 2024-05-11T12:11:13.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.628+0000 7f8a20e00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f8a14058570 con 0x7f8a140e30c0 2024-05-11T12:11:13.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.628+0000 7f8a20e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f8a14058850 con 0x7f8a1403e840 2024-05-11T12:11:13.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.629+0000 7f65f2a00640 1 -- 172.21.15.167:0/2860184983 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f4050d60 con 0x7f65dc004d40 2024-05-11T12:11:13.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.629+0000 7f279be00640 1 -- 172.21.15.167:0/4292902804 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f279c06e6d0 con 0x7f279c0b6550 2024-05-11T12:11:13.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.629+0000 7f27a1a00640 1 --2- 172.21.15.167:0/4292902804 >> v2:172.21.15.167:3300/0 conn(0x7f279c06ebf0 0x7f279c06f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:11:13.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.629+0000 7f8a1be00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f8a14054130 0x7f8a140e6c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49410/0 (socket says 172.21.15.167:49410) 2024-05-11T12:11:13.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.629+0000 7f8a1be00640 1 -- 172.21.15.167:0/2171943315 learned_addr learned my addr 172.21.15.167:0/2171943315 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:13.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.630+0000 7f8a1aa00640 1 --2- 172.21.15.167:0/2171943315 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a1403e840 0x7f8a14053bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.630+0000 7f8a1b400640 1 --2- 172.21.15.167:0/2171943315 >> v2:172.21.15.167:3300/0 conn(0x7f8a140e30c0 0x7f8a14057d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.629+0000 7f65f2a00640 1 -- 172.21.15.167:0/2860184983 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65f40ac420 con 0x7f65dc004d40 2024-05-11T12:11:13.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.629+0000 7f65f2a00640 1 -- 172.21.15.167:0/2860184983 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f40ac870 con 0x7f65dc004d40 2024-05-11T12:11:13.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.630+0000 7f65f9800640 1 -- 172.21.15.167:0/2860184983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65dc004d40 msgr2=0x7f65dc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.630+0000 7f65f9800640 1 --2- 172.21.15.167:0/2860184983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65dc004d40 0x7f65dc0051c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f65f40520f0 tx=0x7f65dc005830 comp rx=0 tx=0).stop 2024-05-11T12:11:13.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.631+0000 7f65f8e00640 1 -- 172.21.15.167:0/2860184983 reap_dead start 2024-05-11T12:11:13.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.631+0000 7f279be00640 1 --2- 172.21.15.167:0/4292902804 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f279c0b6550 0x7f279c0cf7e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f278c0090e0 tx=0x7f278c02b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da9cdc6d762730e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:13.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.631+0000 7f65f9800640 1 -- 172.21.15.167:0/2860184983 shutdown_connections 2024-05-11T12:11:13.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.631+0000 7f65f9800640 1 -- 172.21.15.167:0/2860184983 >> 172.21.15.167:0/2860184983 conn(0x7f65ec056cd0 msgr2=0x7f65ec0570e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:13.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.631+0000 7f8a1a000640 1 -- 172.21.15.167:0/2171943315 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3055905726 0 0) 0x7f8a140582d0 con 0x7f8a14054130 2024-05-11T12:11:13.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.631+0000 7f8a1aa00640 1 -- 172.21.15.167:0/2171943315 >> v2:172.21.15.167:3300/0 conn(0x7f8a140e30c0 msgr2=0x7f8a14057d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:13.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.631+0000 7f65f9800640 1 -- 172.21.15.167:0/2860184983 shutdown_connections 2024-05-11T12:11:13.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.631+0000 7f8a1aa00640 1 --2- 172.21.15.167:0/2171943315 >> v2:172.21.15.167:3300/0 conn(0x7f8a140e30c0 0x7f8a14057d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.631+0000 7f8a1aa00640 1 -- 172.21.15.167:0/2171943315 >> v1:172.21.15.167:6789/0 conn(0x7f8a14054130 legacy=0x7f8a140e6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.632+0000 7fe75b600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe74c006e90 0x7fe74c0def60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.632+0000 7fe75b600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe74c04ffc0 0x7fe74c038a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.632+0000 7f8a1aa00640 1 -- 172.21.15.167:0/2171943315 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a14058b30 con 0x7f8a1403e840 2024-05-11T12:11:13.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.632+0000 7f279b400640 1 -- 172.21.15.167:0/4292902804 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c038070 con 0x7f279c0b6550 2024-05-11T12:11:13.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.632+0000 7f279b400640 1 -- 172.21.15.167:0/4292902804 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f278c033640 con 0x7f279c0b6550 2024-05-11T12:11:13.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.632+0000 7f279b400640 1 -- 172.21.15.167:0/4292902804 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c033a10 con 0x7f279c0b6550 2024-05-11T12:11:13.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.632+0000 7fe75b600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe74c0eacf0 0x7fe74c038fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.632+0000 7fe75b600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe74c054930 con 0x7fe74c006e90 2024-05-11T12:11:13.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.632+0000 7fe75b600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe74c0547b0 con 0x7fe74c04ffc0 2024-05-11T12:11:13.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.632+0000 7fe75b600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe74c04f540 con 0x7fe74c0eacf0 2024-05-11T12:11:13.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.633+0000 7f8a1b400640 1 --2- 172.21.15.167:0/2171943315 >> v2:172.21.15.167:3300/0 conn(0x7f8a140e30c0 0x7f8a14057d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:11:13.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.632+0000 7f65f9800640 1 -- 172.21.15.167:0/2860184983 wait complete. 2024-05-11T12:11:13.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.633+0000 7fe759600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe74c0eacf0 0x7fe74c038fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.634+0000 7fe753e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe74c04ffc0 0x7fe74c038a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.634+0000 7fe759600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe74c0eacf0 0x7fe74c038fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:59658/0 (socket says 172.21.15.167:59658) 2024-05-11T12:11:13.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.634+0000 7fe759600640 1 -- 172.21.15.167:0/2634601466 learned_addr learned my addr 172.21.15.167:0/2634601466 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:13.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.634+0000 7fe758c00640 1 --2- 172.21.15.167:0/2634601466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe74c006e90 0x7fe74c0def60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.634+0000 7f27a3a00640 1 -- 172.21.15.167:0/4292902804 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f279c0b6550 msgr2=0x7f279c0cf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.634+0000 7f8a1aa00640 1 --2- 172.21.15.167:0/2171943315 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a1403e840 0x7f8a14053bf0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8a080097a0 tx=0x7f8a0802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e7221c2b446f970 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:13.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.634+0000 7f27a3a00640 1 --2- 172.21.15.167:0/4292902804 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f279c0b6550 0x7f279c0cf7e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f278c0090e0 tx=0x7f278c02b3b0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.635+0000 7fe753e00640 1 -- 172.21.15.167:0/2634601466 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe74c0eacf0 msgr2=0x7fe74c038fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.635+0000 7fe753e00640 1 --2- 172.21.15.167:0/2634601466 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe74c0eacf0 0x7fe74c038fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.635+0000 7fe753e00640 1 -- 172.21.15.167:0/2634601466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe74c006e90 msgr2=0x7fe74c0def60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.634+0000 7f8a1a000640 1 -- 172.21.15.167:0/2171943315 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a08037070 con 0x7f8a1403e840 2024-05-11T12:11:13.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.634+0000 7f27a3a00640 1 -- 172.21.15.167:0/4292902804 shutdown_connections 2024-05-11T12:11:13.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.634+0000 7f27a3a00640 1 --2- 172.21.15.167:0/4292902804 >> v2:172.21.15.167:3300/0 conn(0x7f279c06ebf0 0x7f279c06f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.635+0000 7f27a3a00640 1 --2- 172.21.15.167:0/4292902804 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f279c0b6550 0x7f279c0cf7e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.635+0000 7f27a3a00640 1 -- 172.21.15.167:0/4292902804 >> 172.21.15.167:0/4292902804 conn(0x7f279c06c450 msgr2=0x7f279c06c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:13.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.635+0000 7f27a3a00640 1 -- 172.21.15.167:0/4292902804 shutdown_connections 2024-05-11T12:11:13.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.635+0000 7f27a3a00640 1 -- 172.21.15.167:0/4292902804 wait complete. 2024-05-11T12:11:13.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.634+0000 7fb749000640 1 Processor -- start 2024-05-11T12:11:13.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.635+0000 7f65f9800640 1 Processor -- start 2024-05-11T12:11:13.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.635+0000 7fe753e00640 1 --2- 172.21.15.167:0/2634601466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe74c006e90 0x7fe74c0def60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.635+0000 7fe753e00640 1 -- 172.21.15.167:0/2634601466 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe74c0362e0 con 0x7fe74c04ffc0 2024-05-11T12:11:13.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.635+0000 7fe758c00640 1 --2- 172.21.15.167:0/2634601466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe74c006e90 0x7fe74c0def60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:11:13.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.635+0000 7fe753e00640 1 --2- 172.21.15.167:0/2634601466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe74c04ffc0 0x7fe74c038a70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe74000c3e0 tx=0x7fe74000c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:13.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.635+0000 7f8a1a000640 1 -- 172.21.15.167:0/2171943315 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a08032440 con 0x7f8a1403e840 2024-05-11T12:11:13.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.635+0000 7f8a1a000640 1 -- 172.21.15.167:0/2171943315 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a08032810 con 0x7f8a1403e840 2024-05-11T12:11:13.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.635+0000 7f65f9800640 1 -- start start 2024-05-11T12:11:13.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.635+0000 7fb749000640 1 -- start start 2024-05-11T12:11:13.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.636+0000 7fe750c00640 1 -- 172.21.15.167:0/2634601466 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7400074e0 con 0x7fe74c04ffc0 2024-05-11T12:11:13.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.636+0000 7fe750c00640 1 -- 172.21.15.167:0/2634601466 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7400047e0 con 0x7fe74c04ffc0 2024-05-11T12:11:13.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.636+0000 7fe750c00640 1 -- 172.21.15.167:0/2634601466 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe740004c10 con 0x7fe74c04ffc0 2024-05-11T12:11:13.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.636+0000 7f8a20e00640 1 -- 172.21.15.167:0/2171943315 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a1403e840 msgr2=0x7f8a14053bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.636+0000 7fc789600640 1 Processor -- start 2024-05-11T12:11:13.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.636+0000 7f8a20e00640 1 --2- 172.21.15.167:0/2171943315 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a1403e840 0x7f8a14053bf0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8a080097a0 tx=0x7f8a0802a210 comp rx=0 tx=0).stop 2024-05-11T12:11:13.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.637+0000 7fe75b600640 1 -- 172.21.15.167:0/2634601466 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe74c0365c0 con 0x7fe74c04ffc0 2024-05-11T12:11:13.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.637+0000 7fe75b600640 1 -- 172.21.15.167:0/2634601466 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe74c036b80 con 0x7fe74c04ffc0 2024-05-11T12:11:13.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.637+0000 7fb749000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb74415f330 0x7fb7440d1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.637+0000 7fb749000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb7440d1670 0x7fb7440b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.637+0000 7fb749000640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb7440b8800 0x7fb7440d3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.637+0000 7fb749000640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fb74406c450 con 0x7fb7440b8800 2024-05-11T12:11:13.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.637+0000 7fb749000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fb74406d2b0 con 0x7fb7440d1670 2024-05-11T12:11:13.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.637+0000 7fb749000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb74406c6a0 con 0x7fb74415f330 2024-05-11T12:11:13.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.638+0000 7f65f9800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65dc004d40 0x7f65ec0e8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.638+0000 7f65f9800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65ec00a760 0x7f65ec04f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.638+0000 7f65f9800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65ec04ff30 0x7f65ec0ea1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.638+0000 7f65f9800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec04a270 con 0x7f65ec00a760 2024-05-11T12:11:13.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.638+0000 7f65f9800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec04a5b0 con 0x7f65dc004d40 2024-05-11T12:11:13.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.638+0000 7f65f9800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f65ec039cb0 con 0x7f65ec04ff30 2024-05-11T12:11:13.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.638+0000 7fe75b600640 1 -- 172.21.15.167:0/2634601466 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe74c005380 con 0x7fe74c04ffc0 2024-05-11T12:11:13.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.638+0000 7f8a20e00640 1 -- 172.21.15.167:0/2171943315 shutdown_connections 2024-05-11T12:11:13.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.638+0000 7f8a20e00640 1 --2- 172.21.15.167:0/2171943315 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a1403e840 0x7f8a14053bf0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.638+0000 7f8a20e00640 1 --2- 172.21.15.167:0/2171943315 >> v2:172.21.15.167:3300/0 conn(0x7f8a140e30c0 0x7f8a14057d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.638+0000 7f8a20e00640 1 -- 172.21.15.167:0/2171943315 >> 172.21.15.167:0/2171943315 conn(0x7f8a140559e0 msgr2=0x7f8a14055df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:13.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.638+0000 7f27a3a00640 1 Processor -- start 2024-05-11T12:11:13.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.638+0000 7fb742800640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb7440b8800 0x7fb7440d3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.638+0000 7fb742800640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb7440b8800 0x7fb7440d3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41500/0 (socket says 172.21.15.167:41500) 2024-05-11T12:11:13.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.638+0000 7fb742800640 1 -- 172.21.15.167:0/3922657903 learned_addr learned my addr 172.21.15.167:0/3922657903 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:13.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.638+0000 7fb741400640 1 --2- 172.21.15.167:0/3922657903 >> v2:172.21.15.204:3300/0 conn(0x7fb7440d1670 0x7fb7440b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.638+0000 7fb742800640 1 -- 172.21.15.167:0/3922657903 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb74415f330 msgr2=0x7fb7440d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:13.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.638+0000 7fb742800640 1 --2- 172.21.15.167:0/3922657903 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb74415f330 0x7fb7440d1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.638+0000 7fb742800640 1 -- 172.21.15.167:0/3922657903 >> v2:172.21.15.204:3300/0 conn(0x7fb7440d1670 msgr2=0x7fb7440b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:13.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.638+0000 7fb742800640 1 --2- 172.21.15.167:0/3922657903 >> v2:172.21.15.204:3300/0 conn(0x7fb7440d1670 0x7fb7440b82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.638+0000 7fb742800640 1 -- 172.21.15.167:0/3922657903 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb74406c980 con 0x7fb7440b8800 2024-05-11T12:11:13.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.638+0000 7fb741400640 1 --2- 172.21.15.167:0/3922657903 >> v2:172.21.15.204:3300/0 conn(0x7fb7440d1670 0x7fb7440b82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:13.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.638+0000 7fc789600640 1 -- start start 2024-05-11T12:11:13.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.638+0000 7f27a3a00640 1 -- start start 2024-05-11T12:11:13.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.639+0000 7f8a20e00640 1 -- 172.21.15.167:0/2171943315 shutdown_connections 2024-05-11T12:11:13.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.639+0000 7f65f3400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65ec00a760 0x7f65ec04f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.639+0000 7f65f3400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65ec00a760 0x7f65ec04f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41502/0 (socket says 172.21.15.167:41502) 2024-05-11T12:11:13.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.639+0000 7f65f3400640 1 -- 172.21.15.167:0/2326752806 learned_addr learned my addr 172.21.15.167:0/2326752806 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:13.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.639+0000 7f65f8e00640 1 --2- 172.21.15.167:0/2326752806 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65ec04ff30 0x7f65ec0ea1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.639+0000 7f096ce00640 1 Processor -- start 2024-05-11T12:11:13.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.639+0000 7f096ce00640 1 -- start start 2024-05-11T12:11:13.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.640+0000 7f65f3400640 1 -- 172.21.15.167:0/2326752806 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65ec04ff30 msgr2=0x7f65ec0ea1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.640+0000 7f65f3400640 1 --2- 172.21.15.167:0/2326752806 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65ec04ff30 0x7f65ec0ea1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.640+0000 7f65f3400640 1 -- 172.21.15.167:0/2326752806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65dc004d40 msgr2=0x7f65ec0e8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.640+0000 7f65f3400640 1 --2- 172.21.15.167:0/2326752806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65dc004d40 0x7f65ec0e8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.640+0000 7f65f3400640 1 -- 172.21.15.167:0/2326752806 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65ec007ce0 con 0x7f65ec00a760 2024-05-11T12:11:13.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.640+0000 7f27a3a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f279c06ebf0 0x7f279c1523f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.640+0000 7f27a3a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f279c0b6550 0x7f279c155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.640+0000 7fc789600640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc784163c20 0x7fc78406c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.640+0000 7fc789600640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc78406dd80 con 0x7fc784163840 2024-05-11T12:11:13.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.640+0000 7fc789600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7840b6c90 con 0x7fc78406d080 2024-05-11T12:11:13.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.640+0000 7fc789600640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fc7840b0b00 con 0x7fc784163c20 2024-05-11T12:11:13.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.640+0000 7fc782a00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fc784163840 0x7fc78406d670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49422/0 (socket says 172.21.15.167:49422) 2024-05-11T12:11:13.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.640+0000 7fc782a00640 1 -- 172.21.15.167:0/185926213 learned_addr learned my addr 172.21.15.167:0/185926213 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:13.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.640+0000 7f27a3a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f279c15b410 0x7f279c157900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.639+0000 7f8a20e00640 1 -- 172.21.15.167:0/2171943315 wait complete. 2024-05-11T12:11:13.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.641+0000 7f27a1000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f279c06ebf0 0x7f279c1523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.641+0000 7f27a1000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f279c06ebf0 0x7f279c1523f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.167:47090/0 (socket says 172.21.15.167:47090) 2024-05-11T12:11:13.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.640+0000 7fb742800640 1 --2- 172.21.15.167:0/3922657903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7440b8800 0x7fb7440d3c70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb74408d780 tx=0x7fb73c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4d98881f7bfb875 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:13.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.641+0000 7f27a1000640 1 -- 172.21.15.167:0/2842278370 learned_addr learned my addr 172.21.15.167:0/2842278370 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:13.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.641+0000 7f27a3a00640 1 -- 172.21.15.167:0/2842278370 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f279c0b6e20 con 0x7f279c15b410 2024-05-11T12:11:13.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.641+0000 7fc783400640 1 --2- 172.21.15.167:0/185926213 >> v2:172.21.15.167:3300/0 conn(0x7fc784163c20 0x7fc78406c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.641+0000 7f096ce00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f09600e30c0 0x7f0960057d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.641+0000 7f096ce00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f096003e840 0x7f0960053bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.641+0000 7f096ce00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09600582d0 con 0x7f0960054130 2024-05-11T12:11:13.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.641+0000 7f096ce00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f0960058570 con 0x7f096003e840 2024-05-11T12:11:13.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.641+0000 7f096ce00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0960058850 con 0x7f09600e30c0 2024-05-11T12:11:13.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.641+0000 7f27a3a00640 1 -- 172.21.15.167:0/2842278370 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f279c0b6ca0 con 0x7f279c06ebf0 2024-05-11T12:11:13.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.641+0000 7fb740a00640 1 -- 172.21.15.167:0/3922657903 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73c036070 con 0x7fb7440b8800 2024-05-11T12:11:13.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.642+0000 7fb740a00640 1 -- 172.21.15.167:0/3922657903 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb73c031600 con 0x7fb7440b8800 2024-05-11T12:11:13.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.642+0000 7fb740a00640 1 -- 172.21.15.167:0/3922657903 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73c0319d0 con 0x7fb7440b8800 2024-05-11T12:11:13.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.642+0000 7fd26a200640 1 Processor -- start 2024-05-11T12:11:13.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.641+0000 7f27a3a00640 1 -- 172.21.15.167:0/2842278370 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f279c0cfd20 con 0x7f279c0b6550 2024-05-11T12:11:13.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.641+0000 7f279be00640 1 --2- 172.21.15.167:0/2842278370 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f279c0b6550 0x7f279c155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.642+0000 7fd26a200640 1 -- start start 2024-05-11T12:11:13.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.642+0000 7f0966a00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f096003e840 0x7f0960053bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.642+0000 7f0966a00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f096003e840 0x7f0960053bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41524/0 (socket says 172.21.15.167:41524) 2024-05-11T12:11:13.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.642+0000 7f8a20e00640 1 Processor -- start 2024-05-11T12:11:13.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.642+0000 7f0966a00640 1 -- 172.21.15.167:0/2462581640 learned_addr learned my addr 172.21.15.167:0/2462581640 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:13.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.642+0000 7f27a1a00640 1 --2- 172.21.15.167:0/2842278370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f279c15b410 0x7f279c157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.643+0000 7fb749000640 1 -- 172.21.15.167:0/3922657903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7440b8800 msgr2=0x7fb7440d3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.643+0000 7fb749000640 1 --2- 172.21.15.167:0/3922657903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7440b8800 0x7fb7440d3c70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb74408d780 tx=0x7fb73c02a6a0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.642+0000 7f0967400640 1 --2- 172.21.15.167:0/2462581640 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f09600e30c0 0x7f0960057d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.642+0000 7f65f3400640 1 --2- 172.21.15.167:0/2326752806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65ec00a760 0x7f65ec04f9f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f65e000f580 tx=0x7f65e000fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:13.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.643+0000 7f8a20e00640 1 -- start start 2024-05-11T12:11:13.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.642+0000 7fc781600640 1 -- 172.21.15.167:0/185926213 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3699371224 0 0) 0x7fc7840b6c90 con 0x7fc78406d080 2024-05-11T12:11:13.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.642+0000 7fc781600640 1 -- 172.21.15.167:0/185926213 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc768003520 con 0x7fc78406d080 2024-05-11T12:11:13.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.642+0000 7fc783400640 1 -- 172.21.15.167:0/185926213 >> v1:172.21.15.204:6789/0 conn(0x7fc78406d080 legacy=0x7fc7840ae9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.642+0000 7fc783400640 1 -- 172.21.15.167:0/185926213 >> v1:172.21.15.167:6789/0 conn(0x7fc784163840 legacy=0x7fc78406d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.642+0000 7fc783400640 1 -- 172.21.15.167:0/185926213 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7840b6f30 con 0x7fc784163c20 2024-05-11T12:11:13.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.643+0000 7f279be00640 1 -- 172.21.15.167:0/2842278370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f279c06ebf0 msgr2=0x7f279c1523f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.643+0000 7f279be00640 1 --2- 172.21.15.167:0/2842278370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f279c06ebf0 0x7f279c1523f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.643+0000 7f279be00640 1 -- 172.21.15.167:0/2842278370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f279c15b410 msgr2=0x7f279c157900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.643+0000 7f279be00640 1 --2- 172.21.15.167:0/2842278370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f279c15b410 0x7f279c157900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.643+0000 7f279be00640 1 -- 172.21.15.167:0/2842278370 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f279c155ed0 con 0x7f279c0b6550 2024-05-11T12:11:13.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.642+0000 7fe750c00640 1 -- 172.21.15.167:0/2634601466 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe740011070 con 0x7fe74c04ffc0 2024-05-11T12:11:13.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.643+0000 7fb749000640 1 -- 172.21.15.167:0/3922657903 shutdown_connections 2024-05-11T12:11:13.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.643+0000 7fb749000640 1 --2- 172.21.15.167:0/3922657903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7440b8800 0x7fb7440d3c70 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.643+0000 7fb749000640 1 --2- 172.21.15.167:0/3922657903 >> v2:172.21.15.204:3300/0 conn(0x7fb7440d1670 0x7fb7440b82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.643+0000 7fb749000640 1 --2- 172.21.15.167:0/3922657903 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb74415f330 0x7fb7440d1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.643+0000 7fb749000640 1 -- 172.21.15.167:0/3922657903 >> 172.21.15.167:0/3922657903 conn(0x7fb7440b6b80 msgr2=0x7fb7440b6f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:13.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.644+0000 7fc783400640 1 --2- 172.21.15.167:0/185926213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc784163c20 0x7fc78406c450 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc77c009d90 tx=0x7fc77c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32bd2817763254a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:13.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.644+0000 7fb749000640 1 -- 172.21.15.167:0/3922657903 shutdown_connections 2024-05-11T12:11:13.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.644+0000 7fb749000640 1 -- 172.21.15.167:0/3922657903 wait complete. 2024-05-11T12:11:13.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.644+0000 7f8a20e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a1403e840 0x7f8a140d66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.644+0000 7f8a20e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a14054130 0x7f8a140d7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.644+0000 7f8a20e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a140e30c0 0x7f8a140de720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.644+0000 7fe750c00640 1 --2- 172.21.15.167:0/2634601466 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe7340762d0 0x7fe734078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.644+0000 7f279be00640 1 --2- 172.21.15.167:0/2842278370 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f279c0b6550 0x7f279c155480 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f279c072f20 tx=0x7f278c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:13.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.644+0000 7f0966a00640 1 -- 172.21.15.167:0/2462581640 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f09600e30c0 msgr2=0x7f0960057d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:13.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.644+0000 7f0966a00640 1 --2- 172.21.15.167:0/2462581640 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f09600e30c0 0x7f0960057d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.644+0000 7f0966a00640 1 -- 172.21.15.167:0/2462581640 >> v1:172.21.15.167:6789/0 conn(0x7f0960054130 legacy=0x7f09600e6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.644+0000 7f0966a00640 1 -- 172.21.15.167:0/2462581640 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0960058b30 con 0x7f096003e840 2024-05-11T12:11:13.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.644+0000 7fc781600640 1 -- 172.21.15.167:0/185926213 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77c031600 con 0x7fc784163c20 2024-05-11T12:11:13.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.644+0000 7fc781600640 1 -- 172.21.15.167:0/185926213 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc77c0317a0 con 0x7fc784163c20 2024-05-11T12:11:13.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.644+0000 7fc781600640 1 -- 172.21.15.167:0/185926213 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77c031b70 con 0x7fc784163c20 2024-05-11T12:11:13.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.645+0000 7f8a20e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8a140e34a0 con 0x7f8a14054130 2024-05-11T12:11:13.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.645+0000 7f8a1aa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a14054130 0x7f8a140d7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.645+0000 7f8a20e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8a1403bfc0 con 0x7f8a1403e840 2024-05-11T12:11:13.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.645+0000 7fd26a200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd26415f330 0x7fd2640d1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.645+0000 7fd26a200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd2640d1670 0x7fd2640b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.645+0000 7fd26a200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd2640b8800 0x7fd2640d3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.645+0000 7f8a20e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f8a140078e0 con 0x7f8a140e30c0 2024-05-11T12:11:13.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.645+0000 7fd26a200640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fd26406c450 con 0x7fd2640b8800 2024-05-11T12:11:13.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.645+0000 7fd26a200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fd26406d2b0 con 0x7fd2640d1670 2024-05-11T12:11:13.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.645+0000 7fd26a200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd26406c6a0 con 0x7fd26415f330 2024-05-11T12:11:13.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.646+0000 7fd263400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd26415f330 0x7fd2640d1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.646+0000 7fc789600640 1 -- 172.21.15.167:0/185926213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc784163c20 msgr2=0x7fc78406c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.646+0000 7fc789600640 1 --2- 172.21.15.167:0/185926213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc784163c20 0x7fc78406c450 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc77c009d90 tx=0x7fc77c02a5d0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.646+0000 7f65e7e00640 1 -- 172.21.15.167:0/2326752806 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e0019020 con 0x7f65ec00a760 2024-05-11T12:11:13.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.646+0000 7f65e7e00640 1 -- 172.21.15.167:0/2326752806 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65e0014870 con 0x7f65ec00a760 2024-05-11T12:11:13.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.646+0000 7f65e7e00640 1 -- 172.21.15.167:0/2326752806 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e0014c60 con 0x7f65ec00a760 2024-05-11T12:11:13.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.646+0000 7fe750c00640 1 -- 172.21.15.167:0/2634601466 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fe740097cc0 con 0x7fe74c04ffc0 2024-05-11T12:11:13.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.646+0000 7fd263e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd2640b8800 0x7fd2640d3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.646+0000 7fd263400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd26415f330 0x7fd2640d1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:59720/0 (socket says 172.21.15.167:59720) 2024-05-11T12:11:13.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.646+0000 7fd263400640 1 -- 172.21.15.167:0/1349479720 learned_addr learned my addr 172.21.15.167:0/1349479720 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:13.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.646+0000 7fc789600640 1 -- 172.21.15.167:0/185926213 shutdown_connections 2024-05-11T12:11:13.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.646+0000 7fd262a00640 1 --2- 172.21.15.167:0/1349479720 >> v2:172.21.15.204:3300/0 conn(0x7fd2640d1670 0x7fd2640b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.646+0000 7f2798c00640 1 -- 172.21.15.167:0/2842278370 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c038040 con 0x7f279c0b6550 2024-05-11T12:11:13.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.646+0000 7fc789600640 1 --2- 172.21.15.167:0/185926213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc784163c20 0x7fc78406c450 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.646+0000 7f2798c00640 1 -- 172.21.15.167:0/2842278370 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f278c03ca00 con 0x7f279c0b6550 2024-05-11T12:11:13.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.646+0000 7fc789600640 1 -- 172.21.15.167:0/185926213 >> 172.21.15.167:0/185926213 conn(0x7fc7840b5e20 msgr2=0x7fc7840b64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:13.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.646+0000 7f2798c00640 1 -- 172.21.15.167:0/2842278370 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c03cdf0 con 0x7f279c0b6550 2024-05-11T12:11:13.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.646+0000 7f27a3a00640 1 -- 172.21.15.167:0/2842278370 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f279c1561b0 con 0x7f279c0b6550 2024-05-11T12:11:13.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.646+0000 7fb749000640 1 Processor -- start 2024-05-11T12:11:13.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.646+0000 7fe750c00640 1 --2- 172.21.15.167:0/2634601466 >> [v2:172.21.15.167:6800/1480182821,v1:172.21.15.167:6801/1480182821] conn(0x7fe73407fa50 0x7fe734081e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.646+0000 7f27a3a00640 1 -- 172.21.15.167:0/2842278370 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f279c156770 con 0x7f279c0b6550 2024-05-11T12:11:13.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.646+0000 7fd263400640 1 -- 172.21.15.167:0/1349479720 >> v2:172.21.15.204:3300/0 conn(0x7fd2640d1670 msgr2=0x7fd2640b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:13.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.646+0000 7fd263400640 1 --2- 172.21.15.167:0/1349479720 >> v2:172.21.15.204:3300/0 conn(0x7fd2640d1670 0x7fd2640b82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.646+0000 7fd263400640 1 -- 172.21.15.167:0/1349479720 >> v2:172.21.15.167:3300/0 conn(0x7fd2640b8800 msgr2=0x7fd2640d3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:13.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.646+0000 7fd263400640 1 --2- 172.21.15.167:0/1349479720 >> v2:172.21.15.167:3300/0 conn(0x7fd2640b8800 0x7fd2640d3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.646+0000 7fd263400640 1 -- 172.21.15.167:0/1349479720 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd26406c980 con 0x7fd26415f330 2024-05-11T12:11:13.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.646+0000 7fc789600640 1 -- 172.21.15.167:0/185926213 shutdown_connections 2024-05-11T12:11:13.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.646+0000 7f65f9800640 1 -- 172.21.15.167:0/2326752806 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65ec007fc0 con 0x7f65ec00a760 2024-05-11T12:11:13.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.647+0000 7f65f9800640 1 -- 172.21.15.167:0/2326752806 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65ec0e8540 con 0x7f65ec00a760 2024-05-11T12:11:13.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.646+0000 7fe758c00640 1 --2- 172.21.15.167:0/2634601466 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe7340762d0 0x7fe734078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.647+0000 7fe750c00640 1 -- 172.21.15.167:0/2634601466 --> [v2:172.21.15.167:6800/1480182821,v1:172.21.15.167:6801/1480182821] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe740005e50 con 0x7fe73407fa50 2024-05-11T12:11:13.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.647+0000 7fe750c00640 1 -- 172.21.15.167:0/2634601466 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe7400980b0 con 0x7fe74c04ffc0 2024-05-11T12:11:13.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.648+0000 7fe758c00640 1 --2- 172.21.15.167:0/2634601466 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe7340762d0 0x7fe734078790 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe74c00b500 tx=0x7fe754070030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:13.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.645+0000 7f8a1aa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a14054130 0x7f8a140d7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41530/0 (socket says 172.21.15.167:41530) 2024-05-11T12:11:13.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.645+0000 7f8a1aa00640 1 -- 172.21.15.167:0/1736697742 learned_addr learned my addr 172.21.15.167:0/1736697742 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:13.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.645+0000 7f8a1b400640 1 --2- 172.21.15.167:0/1736697742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a1403e840 0x7f8a140d66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.645+0000 7f8a1be00640 1 --2- 172.21.15.167:0/1736697742 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a140e30c0 0x7f8a140de720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.645+0000 7f8a1aa00640 1 -- 172.21.15.167:0/1736697742 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a140e30c0 msgr2=0x7f8a140de720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.647+0000 7f8a1aa00640 1 --2- 172.21.15.167:0/1736697742 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a140e30c0 0x7f8a140de720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.647+0000 7f8a1aa00640 1 -- 172.21.15.167:0/1736697742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a1403e840 msgr2=0x7f8a140d66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.647+0000 7f8a1aa00640 1 --2- 172.21.15.167:0/1736697742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a1403e840 0x7f8a140d66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.647+0000 7f8a1aa00640 1 -- 172.21.15.167:0/1736697742 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a140da350 con 0x7f8a14054130 2024-05-11T12:11:13.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.647+0000 7f8a1b400640 1 --2- 172.21.15.167:0/1736697742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a1403e840 0x7f8a140d66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:11:13.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.648+0000 7f8a1be00640 1 --2- 172.21.15.167:0/1736697742 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a140e30c0 0x7f8a140de720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:11:13.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.646+0000 7fb749000640 1 -- start start 2024-05-11T12:11:13.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.647+0000 7fb749000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb7440b8800 0x7fb7440cc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.648+0000 7fb749000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb74415f330 0x7fb744154130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.648+0000 7fb749000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb744154670 0x7fb7440c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.648+0000 7fc789600640 1 -- 172.21.15.167:0/185926213 wait complete. 2024-05-11T12:11:13.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.648+0000 7fb749000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb74415f710 con 0x7fb74415f330 2024-05-11T12:11:13.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.648+0000 7fb749000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb74406cf40 con 0x7fb744154670 2024-05-11T12:11:13.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.648+0000 7fd263400640 1 --2- 172.21.15.167:0/1349479720 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd26415f330 0x7fd2640d1130 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd26408d780 tx=0x7fd25c02a6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b859cc373d80af4 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:13.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.648+0000 7fb749000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb7440d06b0 con 0x7fb7440b8800 2024-05-11T12:11:13.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.648+0000 7fe759600640 1 --2- 172.21.15.167:0/2634601466 >> [v2:172.21.15.167:6800/1480182821,v1:172.21.15.167:6801/1480182821] conn(0x7fe73407fa50 0x7fe734081e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.648+0000 7f65f9800640 1 -- 172.21.15.167:0/2326752806 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f65ec0057e0 con 0x7f65ec00a760 2024-05-11T12:11:13.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.648+0000 7f0966a00640 1 --2- 172.21.15.167:0/2462581640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f096003e840 0x7f0960053bf0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0954009870 tx=0x7f095402a2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a44319ae518c2bfa server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:13.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.649+0000 7f27a3a00640 1 -- 172.21.15.167:0/2842278370 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2774001030 con 0x7f279c0b6550 2024-05-11T12:11:13.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.649+0000 7fd262000640 1 -- 172.21.15.167:0/1349479720 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd25c036070 con 0x7fd26415f330 2024-05-11T12:11:13.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.649+0000 7fb742800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb744154670 0x7fb7440c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.649+0000 7fd262000640 1 -- 172.21.15.167:0/1349479720 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd25c031600 con 0x7fd26415f330 2024-05-11T12:11:13.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.649+0000 7fd262000640 1 -- 172.21.15.167:0/1349479720 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd25c0319d0 con 0x7fd26415f330 2024-05-11T12:11:13.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.649+0000 7f0966000640 1 -- 172.21.15.167:0/2462581640 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0954037070 con 0x7f096003e840 2024-05-11T12:11:13.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.649+0000 7f0966000640 1 -- 172.21.15.167:0/2462581640 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0954032640 con 0x7f096003e840 2024-05-11T12:11:13.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.649+0000 7f0966000640 1 -- 172.21.15.167:0/2462581640 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0954032a10 con 0x7f096003e840 2024-05-11T12:11:13.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.649+0000 7fe759600640 1 --2- 172.21.15.167:0/2634601466 >> [v2:172.21.15.167:6800/1480182821,v1:172.21.15.167:6801/1480182821] conn(0x7fe73407fa50 0x7fe734081e90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:13.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.649+0000 7fb741400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb74415f330 0x7fb744154130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.649+0000 7fb742800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb744154670 0x7fb7440c8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.167:47102/0 (socket says 172.21.15.167:47102) 2024-05-11T12:11:13.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.649+0000 7f8a1aa00640 1 --2- 172.21.15.167:0/1736697742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a14054130 0x7f8a140d7820 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8a080098c0 tx=0x7f8a080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:13.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.650+0000 7fb741400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb74415f330 0x7fb744154130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41558/0 (socket says 172.21.15.167:41558) 2024-05-11T12:11:13.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.650+0000 7fb741400640 1 -- 172.21.15.167:0/838361361 learned_addr learned my addr 172.21.15.167:0/838361361 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:13.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.650+0000 7f096ce00640 1 -- 172.21.15.167:0/2462581640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f096003e840 msgr2=0x7f0960053bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.650+0000 7f096ce00640 1 --2- 172.21.15.167:0/2462581640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f096003e840 0x7f0960053bf0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0954009870 tx=0x7f095402a2f0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.650+0000 7f8a0f400640 1 -- 172.21.15.167:0/1736697742 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a08037040 con 0x7f8a14054130 2024-05-11T12:11:13.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.650+0000 7fd26a200640 1 -- 172.21.15.167:0/1349479720 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd26415f330 msgr2=0x7fd2640d1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.650+0000 7fd26a200640 1 --2- 172.21.15.167:0/1349479720 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd26415f330 0x7fd2640d1130 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd26408d780 tx=0x7fd25c02a6a0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.650+0000 7f096ce00640 1 -- 172.21.15.167:0/2462581640 shutdown_connections 2024-05-11T12:11:13.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.650+0000 7f096ce00640 1 --2- 172.21.15.167:0/2462581640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f096003e840 0x7f0960053bf0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.650+0000 7f096ce00640 1 --2- 172.21.15.167:0/2462581640 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f09600e30c0 0x7f0960057d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.650+0000 7f096ce00640 1 -- 172.21.15.167:0/2462581640 >> 172.21.15.167:0/2462581640 conn(0x7f09600559e0 msgr2=0x7f0960055df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:13.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.650+0000 7f8a0f400640 1 -- 172.21.15.167:0/1736697742 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a08042070 con 0x7f8a14054130 2024-05-11T12:11:13.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.650+0000 7fb741400640 1 -- 172.21.15.167:0/838361361 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb7440b8800 msgr2=0x7fb7440cc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.650+0000 7fb741400640 1 --2- 172.21.15.167:0/838361361 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb7440b8800 0x7fb7440cc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.650+0000 7fb741400640 1 -- 172.21.15.167:0/838361361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb744154670 msgr2=0x7fb7440c8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.651+0000 7fb741400640 1 --2- 172.21.15.167:0/838361361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb744154670 0x7fb7440c8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.650+0000 7fd26a200640 1 -- 172.21.15.167:0/1349479720 shutdown_connections 2024-05-11T12:11:13.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.650+0000 7fd26a200640 1 --2- 172.21.15.167:0/1349479720 >> v2:172.21.15.167:3300/0 conn(0x7fd2640b8800 0x7fd2640d3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.651+0000 7fd26a200640 1 --2- 172.21.15.167:0/1349479720 >> v2:172.21.15.204:3300/0 conn(0x7fd2640d1670 0x7fd2640b82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.651+0000 7fd26a200640 1 --2- 172.21.15.167:0/1349479720 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd26415f330 0x7fd2640d1130 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.651+0000 7fd26a200640 1 -- 172.21.15.167:0/1349479720 >> 172.21.15.167:0/1349479720 conn(0x7fd2640b6b80 msgr2=0x7fd2640b6f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:13.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.651+0000 7fb741400640 1 -- 172.21.15.167:0/838361361 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7440c9e60 con 0x7fb74415f330 2024-05-11T12:11:13.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.650+0000 7f8a0f400640 1 -- 172.21.15.167:0/1736697742 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a08030940 con 0x7f8a14054130 2024-05-11T12:11:13.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.651+0000 7f8a20e00640 1 -- 172.21.15.167:0/1736697742 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a140d7d60 con 0x7f8a14054130 2024-05-11T12:11:13.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.651+0000 7fb742800640 1 --2- 172.21.15.167:0/838361361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb744154670 0x7fb7440c8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:13.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.651+0000 7fd26a200640 1 -- 172.21.15.167:0/1349479720 shutdown_connections 2024-05-11T12:11:13.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.651+0000 7fc789600640 1 Processor -- start 2024-05-11T12:11:13.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.651+0000 7fb741400640 1 --2- 172.21.15.167:0/838361361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb74415f330 0x7fb744154130 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb73400c850 tx=0x7fb73400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:13.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.651+0000 7fc789600640 1 -- start start 2024-05-11T12:11:13.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.651+0000 7fb73a000640 1 -- 172.21.15.167:0/838361361 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb734015070 con 0x7fb74415f330 2024-05-11T12:11:13.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.652+0000 7fe750c00640 1 -- 172.21.15.167:0/2634601466 <== osd.1 v2:172.21.15.167:6800/1480182821 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe740005e50 con 0x7fe73407fa50 2024-05-11T12:11:13.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.651+0000 7fd26a200640 1 -- 172.21.15.167:0/1349479720 wait complete. 2024-05-11T12:11:13.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.653+0000 7f8a20e00640 1 -- 172.21.15.167:0/1736697742 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a14050320 con 0x7f8a14054130 2024-05-11T12:11:13.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.651+0000 7fb749000640 1 -- 172.21.15.167:0/838361361 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7440ca140 con 0x7fb74415f330 2024-05-11T12:11:13.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.652+0000 7fb749000640 1 -- 172.21.15.167:0/838361361 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7440ca700 con 0x7fb74415f330 2024-05-11T12:11:13.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.652+0000 7fb73a000640 1 -- 172.21.15.167:0/838361361 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb734007a30 con 0x7fb74415f330 2024-05-11T12:11:13.687 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.652+0000 7fb73a000640 1 -- 172.21.15.167:0/838361361 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73401f060 con 0x7fb74415f330 2024-05-11T12:11:13.687 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.652+0000 7fb749000640 1 -- 172.21.15.167:0/838361361 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb71c001030 con 0x7fb74415f330 2024-05-11T12:11:13.687 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.653+0000 7fc789600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc78406d080 0x7fc784159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.687 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.653+0000 7f65e7e00640 1 -- 172.21.15.167:0/2326752806 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f65e001c020 con 0x7f65ec00a760 2024-05-11T12:11:13.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.653+0000 7fc789600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc784163840 0x7fc7840992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.653+0000 7fc789600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc784163c20 0x7fc7840997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.654+0000 7f096ce00640 1 -- 172.21.15.167:0/2462581640 shutdown_connections 2024-05-11T12:11:13.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.654+0000 7f8a20e00640 1 -- 172.21.15.167:0/1736697742 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8a140057e0 con 0x7f8a14054130 2024-05-11T12:11:13.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.654+0000 7fc782a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc78406d080 0x7fc784159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.654+0000 7fc789600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc7840b1000 con 0x7fc784163840 2024-05-11T12:11:13.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.654+0000 7fc789600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc7840af0c0 con 0x7fc78406d080 2024-05-11T12:11:13.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.654+0000 7fc789600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc784159b70 con 0x7fc784163c20 2024-05-11T12:11:13.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.654+0000 7fc782a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc78406d080 0x7fc784159630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.167:47110/0 (socket says 172.21.15.167:47110) 2024-05-11T12:11:13.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.654+0000 7fc783400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc784163c20 0x7fc7840997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.655+0000 7fc783400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc784163c20 0x7fc7840997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:59742/0 (socket says 172.21.15.167:59742) 2024-05-11T12:11:13.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.655+0000 7fc783400640 1 -- 172.21.15.167:0/3317294111 learned_addr learned my addr 172.21.15.167:0/3317294111 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:13.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.655+0000 7fc782000640 1 --2- 172.21.15.167:0/3317294111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc784163840 0x7fc7840992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.655+0000 7fc783400640 1 -- 172.21.15.167:0/3317294111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc78406d080 msgr2=0x7fc784159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.654+0000 7f096ce00640 1 -- 172.21.15.167:0/2462581640 wait complete. 2024-05-11T12:11:13.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.656+0000 7f2798c00640 1 -- 172.21.15.167:0/2842278370 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f278c031570 con 0x7f279c0b6550 2024-05-11T12:11:13.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.656+0000 7fc783400640 1 --2- 172.21.15.167:0/3317294111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc78406d080 0x7fc784159630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.653+0000 7fd26a200640 1 Processor -- start 2024-05-11T12:11:13.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.654+0000 7fd26a200640 1 -- start start 2024-05-11T12:11:13.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.656+0000 7fc783400640 1 -- 172.21.15.167:0/3317294111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc784163840 msgr2=0x7fc7840992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.655+0000 7fd26a200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2640b8800 0x7fd2641e9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.655+0000 7fd26a200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd26415f330 0x7fd264150300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.655+0000 7fd26a200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2640ce2c0 0x7fd2641e79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.655+0000 7fd26a200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd26415f710 con 0x7fd2640b8800 2024-05-11T12:11:13.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.655+0000 7fd26a200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd26406cf40 con 0x7fd2640ce2c0 2024-05-11T12:11:13.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.655+0000 7fd26a200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd2640d06b0 con 0x7fd26415f330 2024-05-11T12:11:13.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.656+0000 7fc783400640 1 --2- 172.21.15.167:0/3317294111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc784163840 0x7fc7840992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.656+0000 7fc783400640 1 -- 172.21.15.167:0/3317294111 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc784097860 con 0x7fc784163c20 2024-05-11T12:11:13.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.656+0000 7fd263400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2640b8800 0x7fd2641e9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.656+0000 7fb73a000640 1 -- 172.21.15.167:0/838361361 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb734018070 con 0x7fb74415f330 2024-05-11T12:11:13.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.656+0000 7fd263400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2640b8800 0x7fd2641e9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41572/0 (socket says 172.21.15.167:41572) 2024-05-11T12:11:13.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.656+0000 7fd263400640 1 -- 172.21.15.167:0/102625492 learned_addr learned my addr 172.21.15.167:0/102625492 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:13.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.656+0000 7f65e7e00640 1 --2- 172.21.15.167:0/2326752806 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f65d40763d0 0x7f65d4078890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.656+0000 7fd262a00640 1 --2- 172.21.15.167:0/102625492 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd26415f330 0x7fd264150300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.656+0000 7fd263e00640 1 --2- 172.21.15.167:0/102625492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2640ce2c0 0x7fd2641e79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.657+0000 7f65e7e00640 1 -- 172.21.15.167:0/2326752806 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f65e0098140 con 0x7f65ec00a760 2024-05-11T12:11:13.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.657+0000 7f096ce00640 1 Processor -- start 2024-05-11T12:11:13.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.657+0000 7f65e7e00640 1 --2- 172.21.15.167:0/2326752806 >> [v2:172.21.15.167:6808/1777085638,v1:172.21.15.167:6809/1777085638] conn(0x7f65d407fad0 0x7f65d4081f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.657+0000 7f096ce00640 1 -- start start 2024-05-11T12:11:13.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.657+0000 7f8a0f400640 1 -- 172.21.15.167:0/1736697742 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8a08030ae0 con 0x7f8a14054130 2024-05-11T12:11:13.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.657+0000 7f65e7e00640 1 -- 172.21.15.167:0/2326752806 --> [v2:172.21.15.167:6808/1777085638,v1:172.21.15.167:6809/1777085638] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f65e0008e50 con 0x7f65d407fad0 2024-05-11T12:11:13.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.657+0000 7fd262a00640 1 -- 172.21.15.167:0/102625492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2640ce2c0 msgr2=0x7fd2641e79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.657+0000 7fd262a00640 1 --2- 172.21.15.167:0/102625492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2640ce2c0 0x7fd2641e79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.657+0000 7fd262a00640 1 -- 172.21.15.167:0/102625492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2640b8800 msgr2=0x7fd2641e9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.658+0000 7f65f8e00640 1 --2- 172.21.15.167:0/2326752806 >> [v2:172.21.15.167:6808/1777085638,v1:172.21.15.167:6809/1777085638] conn(0x7f65d407fad0 0x7f65d4081f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.658+0000 7f65f3e00640 1 --2- 172.21.15.167:0/2326752806 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f65d40763d0 0x7f65d4078890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.658+0000 7fd262a00640 1 --2- 172.21.15.167:0/102625492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2640b8800 0x7fd2641e9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.658+0000 7fd262a00640 1 -- 172.21.15.167:0/102625492 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2641e83c0 con 0x7fd26415f330 2024-05-11T12:11:13.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.658+0000 7fd263e00640 1 --2- 172.21.15.167:0/102625492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2640ce2c0 0x7fd2641e79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:11:13.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.658+0000 7fd263400640 1 --2- 172.21.15.167:0/102625492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2640b8800 0x7fd2641e9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:11:13.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.657+0000 7fc782000640 1 --2- 172.21.15.167:0/3317294111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc784163840 0x7fc7840992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:13.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.657+0000 7fc783400640 1 --2- 172.21.15.167:0/3317294111 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc784163c20 0x7fc7840997f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc77c009d90 tx=0x7fc77c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:13.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.656+0000 7fc782a00640 1 --2- 172.21.15.167:0/3317294111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc78406d080 0x7fc784159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:11:13.698 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.658+0000 7fd262a00640 1 --2- 172.21.15.167:0/102625492 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd26415f330 0x7fd264150300 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd25000d5f0 tx=0x7fd25000dab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:13.698 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.659+0000 7f65f8e00640 1 --2- 172.21.15.167:0/2326752806 >> [v2:172.21.15.167:6808/1777085638,v1:172.21.15.167:6809/1777085638] conn(0x7f65d407fad0 0x7f65d4081f10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:13.698 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.658+0000 7fb73a000640 1 --2- 172.21.15.167:0/838361361 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb7240762d0 0x7fb724078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.698 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.658+0000 7fb73a000640 1 -- 172.21.15.167:0/838361361 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fb734097d40 con 0x7fb74415f330 2024-05-11T12:11:13.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.659+0000 7fb73a000640 1 --2- 172.21.15.167:0/838361361 >> [v2:172.21.15.204:6815/2030699881,v1:172.21.15.204:6817/2030699881] conn(0x7fb72407fa50 0x7fb724081e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.659+0000 7fb73a000640 1 -- 172.21.15.167:0/838361361 --> [v2:172.21.15.204:6815/2030699881,v1:172.21.15.204:6817/2030699881] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb734001d30 con 0x7fb72407fa50 2024-05-11T12:11:13.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.659+0000 7fb73a000640 1 -- 172.21.15.167:0/838361361 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb734098130 con 0x7fb74415f330 2024-05-11T12:11:13.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.659+0000 7fc77aa00640 1 -- 172.21.15.167:0/3317294111 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77c03f070 con 0x7fc784163c20 2024-05-11T12:11:13.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.659+0000 7f096ce00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f096003e840 0x7f09600dc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.659+0000 7f096ce00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0960054130 0x7f09600d70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.659+0000 7f096ce00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09600e30c0 0x7f09600dd130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.659+0000 7fc77aa00640 1 -- 172.21.15.167:0/3317294111 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc77c030ae0 con 0x7fc784163c20 2024-05-11T12:11:13.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.659+0000 7fc789600640 1 -- 172.21.15.167:0/3317294111 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc784097b40 con 0x7fc784163c20 2024-05-11T12:11:13.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.659+0000 7fc77aa00640 1 -- 172.21.15.167:0/3317294111 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77c043410 con 0x7fc784163c20 2024-05-11T12:11:13.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.659+0000 7fd25b400640 1 -- 172.21.15.167:0/102625492 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2500125f0 con 0x7fd26415f330 2024-05-11T12:11:13.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.659+0000 7fd25b400640 1 -- 172.21.15.167:0/102625492 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd250019070 con 0x7fd26415f330 2024-05-11T12:11:13.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.659+0000 7fd26a200640 1 -- 172.21.15.167:0/102625492 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2641e86a0 con 0x7fd26415f330 2024-05-11T12:11:13.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.660+0000 7fd25b400640 1 -- 172.21.15.167:0/102625492 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2500129d0 con 0x7fd26415f330 2024-05-11T12:11:13.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.659+0000 7f096ce00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f09600e34a0 con 0x7f0960054130 2024-05-11T12:11:13.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.659+0000 7f096ce00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f096003bfc0 con 0x7f09600e30c0 2024-05-11T12:11:13.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.659+0000 7f096ce00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f09600078e0 con 0x7f096003e840 2024-05-11T12:11:13.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.660+0000 7f0967400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f096003e840 0x7f09600dc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.660+0000 7f0966a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0960054130 0x7f09600d70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.660+0000 7fb742800640 1 --2- 172.21.15.167:0/838361361 >> [v2:172.21.15.204:6815/2030699881,v1:172.21.15.204:6817/2030699881] conn(0x7fb72407fa50 0x7fb724081e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.660+0000 7f0966a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0960054130 0x7f09600d70b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41582/0 (socket says 172.21.15.167:41582) 2024-05-11T12:11:13.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.660+0000 7f0966a00640 1 -- 172.21.15.167:0/2554847325 learned_addr learned my addr 172.21.15.167:0/2554847325 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:13.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.660+0000 7fb741e00640 1 --2- 172.21.15.167:0/838361361 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb7240762d0 0x7fb724078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.661+0000 7f65e7e00640 1 -- 172.21.15.167:0/2326752806 <== osd.0 v2:172.21.15.167:6808/1777085638 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f65e0008e50 con 0x7f65d407fad0 2024-05-11T12:11:13.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.661+0000 7f65e7e00640 1 -- 172.21.15.167:0/2326752806 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f65e009f050 con 0x7f65ec00a760 2024-05-11T12:11:13.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.661+0000 7f65f3e00640 1 --2- 172.21.15.167:0/2326752806 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f65d40763d0 0x7f65d4078890 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f65e8000ff0 tx=0x7f65e8007cd0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:13.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.657+0000 7f2798c00640 1 --2- 172.21.15.167:0/2842278370 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f277c0763a0 0x7f277c078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.657+0000 7f2798c00640 1 -- 172.21.15.167:0/2842278370 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f278c0bbef0 con 0x7f279c0b6550 2024-05-11T12:11:13.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.657+0000 7f2798c00640 1 --2- 172.21.15.167:0/2842278370 >> [v2:172.21.15.167:6816/1903389836,v1:172.21.15.167:6817/1903389836] conn(0x7f277c07fb20 0x7f277c081f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.657+0000 7f2798c00640 1 -- 172.21.15.167:0/2842278370 --> [v2:172.21.15.167:6816/1903389836,v1:172.21.15.167:6817/1903389836] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f278c004490 con 0x7f277c07fb20 2024-05-11T12:11:13.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.657+0000 7f2798c00640 1 -- 172.21.15.167:0/2842278370 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f278c0bc2e0 con 0x7f279c0b6550 2024-05-11T12:11:13.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.659+0000 7f27a1000640 1 --2- 172.21.15.167:0/2842278370 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f277c0763a0 0x7f277c078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.662+0000 7f27a1a00640 1 --2- 172.21.15.167:0/2842278370 >> [v2:172.21.15.167:6816/1903389836,v1:172.21.15.167:6817/1903389836] conn(0x7f277c07fb20 0x7f277c081f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.660+0000 7fd26a200640 1 -- 172.21.15.167:0/102625492 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2641e8c60 con 0x7fd26415f330 2024-05-11T12:11:13.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.660+0000 7fc789600640 1 -- 172.21.15.167:0/3317294111 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc784098100 con 0x7fc784163c20 2024-05-11T12:11:13.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.661+0000 7fc789600640 1 -- 172.21.15.167:0/3317294111 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc784069bc0 con 0x7fc784163c20 2024-05-11T12:11:13.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.660+0000 7f0967e00640 1 --2- 172.21.15.167:0/2554847325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09600e30c0 0x7f09600dd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.661+0000 7f0966a00640 1 -- 172.21.15.167:0/2554847325 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f096003e840 msgr2=0x7f09600dc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.661+0000 7f0966a00640 1 --2- 172.21.15.167:0/2554847325 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f096003e840 0x7f09600dc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.661+0000 7f0966a00640 1 -- 172.21.15.167:0/2554847325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09600e30c0 msgr2=0x7f09600dd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.661+0000 7f0966a00640 1 --2- 172.21.15.167:0/2554847325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09600e30c0 0x7f09600dd130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.661+0000 7f0966a00640 1 -- 172.21.15.167:0/2554847325 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09600d75f0 con 0x7f0960054130 2024-05-11T12:11:13.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.662+0000 7f0966a00640 1 --2- 172.21.15.167:0/2554847325 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0960054130 0x7f09600d70b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0954009870 tx=0x7f09540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:13.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.662+0000 7f27a1000640 1 --2- 172.21.15.167:0/2842278370 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f277c0763a0 0x7f277c078860 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2788000f60 tx=0x7f2788007db0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:13.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.662+0000 7f27a1a00640 1 --2- 172.21.15.167:0/2842278370 >> [v2:172.21.15.167:6816/1903389836,v1:172.21.15.167:6817/1903389836] conn(0x7f277c07fb20 0x7f277c081f60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:13.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.662+0000 7fb742800640 1 --2- 172.21.15.167:0/838361361 >> [v2:172.21.15.204:6815/2030699881,v1:172.21.15.204:6817/2030699881] conn(0x7fb72407fa50 0x7fb724081e90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:13.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.662+0000 7fd26a200640 1 -- 172.21.15.167:0/102625492 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd2640694b0 con 0x7fd26415f330 2024-05-11T12:11:13.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.663+0000 7fb741e00640 1 --2- 172.21.15.167:0/838361361 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb7240762d0 0x7fb724078790 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb730009670 tx=0x7fb730006bf0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:13.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.663+0000 7f095b400640 1 -- 172.21.15.167:0/2554847325 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0954041020 con 0x7f0960054130 2024-05-11T12:11:13.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.663+0000 7f095b400640 1 -- 172.21.15.167:0/2554847325 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0954030770 con 0x7f0960054130 2024-05-11T12:11:13.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.663+0000 7f095b400640 1 -- 172.21.15.167:0/2554847325 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0954037040 con 0x7f0960054130 2024-05-11T12:11:13.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.663+0000 7f096ce00640 1 -- 172.21.15.167:0/2554847325 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f096016e910 con 0x7f0960054130 2024-05-11T12:11:13.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.663+0000 7f2798c00640 1 -- 172.21.15.167:0/2842278370 <== osd.3 v2:172.21.15.167:6816/1903389836 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f278c004490 con 0x7f277c07fb20 2024-05-11T12:11:13.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.663+0000 7f096ce00640 1 -- 172.21.15.167:0/2554847325 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f096016eed0 con 0x7f0960054130 2024-05-11T12:11:13.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.663+0000 7f8a0f400640 1 --2- 172.21.15.167:0/1736697742 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f89fc076350 0x7f89fc078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.663+0000 7f8a0f400640 1 -- 172.21.15.167:0/1736697742 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f8a080b8ba0 con 0x7f8a14054130 2024-05-11T12:11:13.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.664+0000 7f8a0f400640 1 --2- 172.21.15.167:0/1736697742 >> [v2:172.21.15.167:6818/1902208221,v1:172.21.15.167:6820/1902208221] conn(0x7f89fc07fad0 0x7f89fc081f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.664+0000 7f8a0f400640 1 -- 172.21.15.167:0/1736697742 --> [v2:172.21.15.167:6818/1902208221,v1:172.21.15.167:6820/1902208221] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8a080067f0 con 0x7f89fc07fad0 2024-05-11T12:11:13.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.664+0000 7f8a0f400640 1 -- 172.21.15.167:0/1736697742 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8a080b8f50 con 0x7f8a14054130 2024-05-11T12:11:13.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.665+0000 7f8a1b400640 1 --2- 172.21.15.167:0/1736697742 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f89fc076350 0x7f89fc078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.665+0000 7fc77aa00640 1 -- 172.21.15.167:0/3317294111 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc77c03b410 con 0x7fc784163c20 2024-05-11T12:11:13.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.666+0000 7fc77aa00640 1 --2- 172.21.15.167:0/3317294111 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc7640762d0 0x7fc764078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.666+0000 7fc77aa00640 1 -- 172.21.15.167:0/3317294111 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fc77c0b8cd0 con 0x7fc784163c20 2024-05-11T12:11:13.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.667+0000 7fc77aa00640 1 --2- 172.21.15.167:0/3317294111 >> [v2:172.21.15.204:6800/3396850806,v1:172.21.15.204:6801/3396850806] conn(0x7fc76407f9d0 0x7fc764081e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.667+0000 7fc77aa00640 1 -- 172.21.15.167:0/3317294111 --> [v2:172.21.15.204:6800/3396850806,v1:172.21.15.204:6801/3396850806] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc77c03de50 con 0x7fc76407f9d0 2024-05-11T12:11:13.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.667+0000 7fc77aa00640 1 -- 172.21.15.167:0/3317294111 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc77c0b9160 con 0x7fc784163c20 2024-05-11T12:11:13.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.667+0000 7fc782a00640 1 --2- 172.21.15.167:0/3317294111 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc7640762d0 0x7fc764078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.668+0000 7f8a1b400640 1 --2- 172.21.15.167:0/1736697742 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f89fc076350 0x7f89fc078810 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8a10009560 tx=0x7f8a10009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:13.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.668+0000 7fb73a000640 1 -- 172.21.15.167:0/838361361 <== osd.4 v2:172.21.15.204:6815/2030699881 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb734001d30 con 0x7fb72407fa50 2024-05-11T12:11:13.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.668+0000 7fd25b400640 1 -- 172.21.15.167:0/102625492 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd250012b70 con 0x7fd26415f330 2024-05-11T12:11:13.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.668+0000 7f096ce00640 1 -- 172.21.15.167:0/2554847325 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0944001030 con 0x7f0960054130 2024-05-11T12:11:13.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.669+0000 7fc782a00640 1 --2- 172.21.15.167:0/3317294111 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc7640762d0 0x7fc764078790 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc774000f60 tx=0x7fc774009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:13.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.669+0000 7fc782000640 1 --2- 172.21.15.167:0/3317294111 >> [v2:172.21.15.204:6800/3396850806,v1:172.21.15.204:6801/3396850806] conn(0x7fc76407f9d0 0x7fc764081e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.669+0000 7f8a1be00640 1 --2- 172.21.15.167:0/1736697742 >> [v2:172.21.15.167:6818/1902208221,v1:172.21.15.167:6820/1902208221] conn(0x7f89fc07fad0 0x7f89fc081f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.670+0000 7fd25b400640 1 --2- 172.21.15.167:0/102625492 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd2440763a0 0x7fd244078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.670+0000 7fd25b400640 1 -- 172.21.15.167:0/102625492 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fd250098720 con 0x7fd26415f330 2024-05-11T12:11:13.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.670+0000 7f8a1be00640 1 --2- 172.21.15.167:0/1736697742 >> [v2:172.21.15.167:6818/1902208221,v1:172.21.15.167:6820/1902208221] conn(0x7f89fc07fad0 0x7f89fc081f10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:13.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.670+0000 7fd25b400640 1 --2- 172.21.15.167:0/102625492 >> [v2:172.21.15.204:6824/1178824319,v1:172.21.15.204:6825/1178824319] conn(0x7fd24407fb20 0x7fd244081f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.670+0000 7fd25b400640 1 -- 172.21.15.167:0/102625492 --> [v2:172.21.15.204:6824/1178824319,v1:172.21.15.204:6825/1178824319] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd250004470 con 0x7fd24407fb20 2024-05-11T12:11:13.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.670+0000 7fd25b400640 1 -- 172.21.15.167:0/102625492 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd250098b10 con 0x7fd26415f330 2024-05-11T12:11:13.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.671+0000 7fd263400640 1 --2- 172.21.15.167:0/102625492 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd2440763a0 0x7fd244078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.671+0000 7fd263e00640 1 --2- 172.21.15.167:0/102625492 >> [v2:172.21.15.204:6824/1178824319,v1:172.21.15.204:6825/1178824319] conn(0x7fd24407fb20 0x7fd244081f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.672+0000 7fc782000640 1 --2- 172.21.15.167:0/3317294111 >> [v2:172.21.15.204:6800/3396850806,v1:172.21.15.204:6801/3396850806] conn(0x7fc76407f9d0 0x7fc764081e10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:13.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.671+0000 7f095b400640 1 -- 172.21.15.167:0/2554847325 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0954044020 con 0x7f0960054130 2024-05-11T12:11:13.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.672+0000 7f8a0f400640 1 -- 172.21.15.167:0/1736697742 <== osd.2 v2:172.21.15.167:6818/1902208221 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8a080067f0 con 0x7f89fc07fad0 2024-05-11T12:11:13.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.672+0000 7fd263400640 1 --2- 172.21.15.167:0/102625492 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd2440763a0 0x7fd244078860 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd25c000f80 tx=0x7fd25c009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:13.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.673+0000 7f095b400640 1 --2- 172.21.15.167:0/2554847325 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f09480762d0 0x7f0948078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.673+0000 7f095b400640 1 -- 172.21.15.167:0/2554847325 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f09540b8040 con 0x7f0960054130 2024-05-11T12:11:13.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.674+0000 7f0967400640 1 --2- 172.21.15.167:0/2554847325 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f09480762d0 0x7f0948078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.674+0000 7f095b400640 1 --2- 172.21.15.167:0/2554847325 >> [v2:172.21.15.204:6808/223313265,v1:172.21.15.204:6809/223313265] conn(0x7f094807fa40 0x7f0948081e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:13.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.674+0000 7f095b400640 1 -- 172.21.15.167:0/2554847325 --> [v2:172.21.15.204:6808/223313265,v1:172.21.15.204:6809/223313265] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0954045e70 con 0x7f094807fa40 2024-05-11T12:11:13.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.674+0000 7f095b400640 1 -- 172.21.15.167:0/2554847325 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f09540bf050 con 0x7f0960054130 2024-05-11T12:11:13.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.674+0000 7fd263e00640 1 --2- 172.21.15.167:0/102625492 >> [v2:172.21.15.204:6824/1178824319,v1:172.21.15.204:6825/1178824319] conn(0x7fd24407fb20 0x7fd244081f60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:13.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.675+0000 7fc77aa00640 1 -- 172.21.15.167:0/3317294111 <== osd.6 v2:172.21.15.204:6800/3396850806 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc77c03de50 con 0x7fc76407f9d0 2024-05-11T12:11:13.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.675+0000 7f0967400640 1 --2- 172.21.15.167:0/2554847325 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f09480762d0 0x7f0948078790 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f095c006fd0 tx=0x7f095c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:13.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.676+0000 7fd25b400640 1 -- 172.21.15.167:0/102625492 <== osd.5 v2:172.21.15.204:6824/1178824319 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd250004470 con 0x7fd24407fb20 2024-05-11T12:11:13.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.678+0000 7f0967e00640 1 --2- 172.21.15.167:0/2554847325 >> [v2:172.21.15.204:6808/223313265,v1:172.21.15.204:6809/223313265] conn(0x7f094807fa40 0x7f0948081e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:13.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.679+0000 7f0967e00640 1 --2- 172.21.15.167:0/2554847325 >> [v2:172.21.15.204:6808/223313265,v1:172.21.15.204:6809/223313265] conn(0x7f094807fa40 0x7f0948081e80 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:13.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.681+0000 7f095b400640 1 -- 172.21.15.167:0/2554847325 <== osd.7 v2:172.21.15.204:6808/223313265 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0954045e70 con 0x7f094807fa40 2024-05-11T12:11:13.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.683+0000 7fe75b600640 1 -- 172.21.15.167:0/2634601466 --> [v2:172.21.15.167:6800/1480182821,v1:172.21.15.167:6801/1480182821] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe74c036ed0 con 0x7fe73407fa50 2024-05-11T12:11:13.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.684+0000 7fe750c00640 1 -- 172.21.15.167:0/2634601466 <== osd.1 v2:172.21.15.167:6800/1480182821 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe74c036ed0 con 0x7fe73407fa50 2024-05-11T12:11:13.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.686+0000 7fe75b600640 1 -- 172.21.15.167:0/2634601466 >> [v2:172.21.15.167:6800/1480182821,v1:172.21.15.167:6801/1480182821] conn(0x7fe73407fa50 msgr2=0x7fe734081e90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.686+0000 7fe75b600640 1 --2- 172.21.15.167:0/2634601466 >> [v2:172.21.15.167:6800/1480182821,v1:172.21.15.167:6801/1480182821] conn(0x7fe73407fa50 0x7fe734081e90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.686+0000 7fe75b600640 1 -- 172.21.15.167:0/2634601466 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe7340762d0 msgr2=0x7fe734078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.686+0000 7fe75b600640 1 --2- 172.21.15.167:0/2634601466 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe7340762d0 0x7fe734078790 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe74c00b500 tx=0x7fe754070030 comp rx=0 tx=0).stop 2024-05-11T12:11:13.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.686+0000 7fe75b600640 1 -- 172.21.15.167:0/2634601466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe74c04ffc0 msgr2=0x7fe74c038a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.686+0000 7fe75b600640 1 --2- 172.21.15.167:0/2634601466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe74c04ffc0 0x7fe74c038a70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe74000c3e0 tx=0x7fe74000c8a0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.687+0000 7fe759600640 1 -- 172.21.15.167:0/2634601466 reap_dead start 2024-05-11T12:11:13.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.687+0000 7fe75b600640 1 -- 172.21.15.167:0/2634601466 shutdown_connections 2024-05-11T12:11:13.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.687+0000 7fe75b600640 1 -- 172.21.15.167:0/2634601466 >> 172.21.15.167:0/2634601466 conn(0x7fe74c009fa0 msgr2=0x7fe74c004920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:13.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.689+0000 7fe75b600640 1 -- 172.21.15.167:0/2634601466 shutdown_connections 2024-05-11T12:11:13.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.689+0000 7fe75b600640 1 -- 172.21.15.167:0/2634601466 wait complete. 2024-05-11T12:11:13.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.690+0000 7f65f9800640 1 -- 172.21.15.167:0/2326752806 --> [v2:172.21.15.167:6808/1777085638,v1:172.21.15.167:6809/1777085638] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f65ec0578f0 con 0x7f65d407fad0 2024-05-11T12:11:13.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.693+0000 7f65e7e00640 1 -- 172.21.15.167:0/2326752806 <== osd.0 v2:172.21.15.167:6808/1777085638 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f65ec0578f0 con 0x7f65d407fad0 2024-05-11T12:11:13.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.694+0000 7f8a20e00640 1 -- 172.21.15.167:0/1736697742 --> [v2:172.21.15.167:6818/1902208221,v1:172.21.15.167:6820/1902208221] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8a14056380 con 0x7f89fc07fad0 2024-05-11T12:11:13.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.694+0000 7f8a0f400640 1 -- 172.21.15.167:0/1736697742 <== osd.2 v2:172.21.15.167:6818/1902208221 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8a14056380 con 0x7f89fc07fad0 2024-05-11T12:11:13.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.695+0000 7f65e5600640 1 -- 172.21.15.167:0/2326752806 >> [v2:172.21.15.167:6808/1777085638,v1:172.21.15.167:6809/1777085638] conn(0x7f65d407fad0 msgr2=0x7f65d4081f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.695+0000 7f65e5600640 1 --2- 172.21.15.167:0/2326752806 >> [v2:172.21.15.167:6808/1777085638,v1:172.21.15.167:6809/1777085638] conn(0x7f65d407fad0 0x7f65d4081f10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.695+0000 7f65e5600640 1 -- 172.21.15.167:0/2326752806 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f65d40763d0 msgr2=0x7f65d4078890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.695+0000 7f65e5600640 1 --2- 172.21.15.167:0/2326752806 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f65d40763d0 0x7f65d4078890 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f65e8000ff0 tx=0x7f65e8007cd0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.695+0000 7f65e5600640 1 -- 172.21.15.167:0/2326752806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65ec00a760 msgr2=0x7f65ec04f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.695+0000 7f65e5600640 1 --2- 172.21.15.167:0/2326752806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65ec00a760 0x7f65ec04f9f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f65e000f580 tx=0x7f65e000fa40 comp rx=0 tx=0).stop 2024-05-11T12:11:13.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.696+0000 7f65f8e00640 1 -- 172.21.15.167:0/2326752806 reap_dead start 2024-05-11T12:11:13.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.696+0000 7f65e5600640 1 -- 172.21.15.167:0/2326752806 shutdown_connections 2024-05-11T12:11:13.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.696+0000 7f65e5600640 1 -- 172.21.15.167:0/2326752806 >> 172.21.15.167:0/2326752806 conn(0x7f65ec056cd0 msgr2=0x7f65ec0563e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:13.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.695+0000 7f8a20e00640 1 -- 172.21.15.167:0/1736697742 >> [v2:172.21.15.167:6818/1902208221,v1:172.21.15.167:6820/1902208221] conn(0x7f89fc07fad0 msgr2=0x7f89fc081f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.695+0000 7f8a20e00640 1 --2- 172.21.15.167:0/1736697742 >> [v2:172.21.15.167:6818/1902208221,v1:172.21.15.167:6820/1902208221] conn(0x7f89fc07fad0 0x7f89fc081f10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.696+0000 7f65e5600640 1 -- 172.21.15.167:0/2326752806 shutdown_connections 2024-05-11T12:11:13.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.696+0000 7f65e5600640 1 -- 172.21.15.167:0/2326752806 wait complete. 2024-05-11T12:11:13.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.696+0000 7f8a20e00640 1 -- 172.21.15.167:0/1736697742 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f89fc076350 msgr2=0x7f89fc078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.696+0000 7f8a20e00640 1 --2- 172.21.15.167:0/1736697742 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f89fc076350 0x7f89fc078810 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8a10009560 tx=0x7f8a10009190 comp rx=0 tx=0).stop 2024-05-11T12:11:13.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.696+0000 7f8a20e00640 1 -- 172.21.15.167:0/1736697742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a14054130 msgr2=0x7f8a140d7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.696+0000 7f8a20e00640 1 --2- 172.21.15.167:0/1736697742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a14054130 0x7f8a140d7820 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8a080098c0 tx=0x7f8a080046c0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.697+0000 7f8a1be00640 1 -- 172.21.15.167:0/1736697742 reap_dead start 2024-05-11T12:11:13.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.698+0000 7f8a20e00640 1 -- 172.21.15.167:0/1736697742 shutdown_connections 2024-05-11T12:11:13.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.698+0000 7f8a20e00640 1 -- 172.21.15.167:0/1736697742 >> 172.21.15.167:0/1736697742 conn(0x7f8a140559e0 msgr2=0x7f8a14039f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:13.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.698+0000 7f8a20e00640 1 -- 172.21.15.167:0/1736697742 shutdown_connections 2024-05-11T12:11:13.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.698+0000 7f8a20e00640 1 -- 172.21.15.167:0/1736697742 wait complete. 2024-05-11T12:11:13.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.700+0000 7f27a3a00640 1 -- 172.21.15.167:0/2842278370 --> [v2:172.21.15.167:6816/1903389836,v1:172.21.15.167:6817/1903389836] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2774002dd0 con 0x7f277c07fb20 2024-05-11T12:11:13.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.700+0000 7f2798c00640 1 -- 172.21.15.167:0/2842278370 <== osd.3 v2:172.21.15.167:6816/1903389836 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2774002dd0 con 0x7f277c07fb20 2024-05-11T12:11:13.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.700+0000 7fc789600640 1 -- 172.21.15.167:0/3317294111 --> [v2:172.21.15.204:6800/3396850806,v1:172.21.15.204:6801/3396850806] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc78406d5e0 con 0x7fc76407f9d0 2024-05-11T12:11:13.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.701+0000 7f2792000640 1 -- 172.21.15.167:0/2842278370 >> [v2:172.21.15.167:6816/1903389836,v1:172.21.15.167:6817/1903389836] conn(0x7f277c07fb20 msgr2=0x7f277c081f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.701+0000 7f2792000640 1 --2- 172.21.15.167:0/2842278370 >> [v2:172.21.15.167:6816/1903389836,v1:172.21.15.167:6817/1903389836] conn(0x7f277c07fb20 0x7f277c081f60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.701+0000 7fc77aa00640 1 -- 172.21.15.167:0/3317294111 <== osd.6 v2:172.21.15.204:6800/3396850806 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc78406d5e0 con 0x7fc76407f9d0 2024-05-11T12:11:13.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.702+0000 7f2792000640 1 -- 172.21.15.167:0/2842278370 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f277c0763a0 msgr2=0x7f277c078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.702+0000 7f2792000640 1 --2- 172.21.15.167:0/2842278370 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f277c0763a0 0x7f277c078860 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2788000f60 tx=0x7f2788007db0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.702+0000 7f2792000640 1 -- 172.21.15.167:0/2842278370 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f279c0b6550 msgr2=0x7f279c155480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.702+0000 7f2792000640 1 --2- 172.21.15.167:0/2842278370 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f279c0b6550 0x7f279c155480 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f279c072f20 tx=0x7f278c0046c0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.702+0000 7f27a1a00640 1 -- 172.21.15.167:0/2842278370 reap_dead start 2024-05-11T12:11:13.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.702+0000 7f2792000640 1 -- 172.21.15.167:0/2842278370 shutdown_connections 2024-05-11T12:11:13.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.702+0000 7f2792000640 1 -- 172.21.15.167:0/2842278370 >> 172.21.15.167:0/2842278370 conn(0x7f279c06c450 msgr2=0x7f279c15ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:13.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.702+0000 7fb749000640 1 -- 172.21.15.167:0/838361361 --> [v2:172.21.15.204:6815/2030699881,v1:172.21.15.204:6817/2030699881] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb71c002dd0 con 0x7fb72407fa50 2024-05-11T12:11:13.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.703+0000 7fb73a000640 1 -- 172.21.15.167:0/838361361 <== osd.4 v2:172.21.15.204:6815/2030699881 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb71c002dd0 con 0x7fb72407fa50 2024-05-11T12:11:13.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.704+0000 7fb749000640 1 -- 172.21.15.167:0/838361361 >> [v2:172.21.15.204:6815/2030699881,v1:172.21.15.204:6817/2030699881] conn(0x7fb72407fa50 msgr2=0x7fb724081e90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.704+0000 7fb749000640 1 --2- 172.21.15.167:0/838361361 >> [v2:172.21.15.204:6815/2030699881,v1:172.21.15.204:6817/2030699881] conn(0x7fb72407fa50 0x7fb724081e90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.704+0000 7fb749000640 1 -- 172.21.15.167:0/838361361 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb7240762d0 msgr2=0x7fb724078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.704+0000 7fb749000640 1 --2- 172.21.15.167:0/838361361 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb7240762d0 0x7fb724078790 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb730009670 tx=0x7fb730006bf0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.704+0000 7fb749000640 1 -- 172.21.15.167:0/838361361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb74415f330 msgr2=0x7fb744154130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.704+0000 7fb749000640 1 --2- 172.21.15.167:0/838361361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb74415f330 0x7fb744154130 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb73400c850 tx=0x7fb73400cc50 comp rx=0 tx=0).stop 2024-05-11T12:11:13.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.705+0000 7fb742800640 1 -- 172.21.15.167:0/838361361 reap_dead start 2024-05-11T12:11:13.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.705+0000 7fb749000640 1 -- 172.21.15.167:0/838361361 shutdown_connections 2024-05-11T12:11:13.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.705+0000 7fb749000640 1 -- 172.21.15.167:0/838361361 >> 172.21.15.167:0/838361361 conn(0x7fb7440b6b80 msgr2=0x7fb74406d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:13.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.705+0000 7fc789600640 1 -- 172.21.15.167:0/3317294111 >> [v2:172.21.15.204:6800/3396850806,v1:172.21.15.204:6801/3396850806] conn(0x7fc76407f9d0 msgr2=0x7fc764081e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.705+0000 7fc789600640 1 --2- 172.21.15.167:0/3317294111 >> [v2:172.21.15.204:6800/3396850806,v1:172.21.15.204:6801/3396850806] conn(0x7fc76407f9d0 0x7fc764081e10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.705+0000 7fc789600640 1 -- 172.21.15.167:0/3317294111 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc7640762d0 msgr2=0x7fc764078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.705+0000 7fc789600640 1 --2- 172.21.15.167:0/3317294111 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc7640762d0 0x7fc764078790 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc774000f60 tx=0x7fc774009040 comp rx=0 tx=0).stop 2024-05-11T12:11:13.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.705+0000 7fc789600640 1 -- 172.21.15.167:0/3317294111 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc784163c20 msgr2=0x7fc7840997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.705+0000 7fc789600640 1 --2- 172.21.15.167:0/3317294111 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc784163c20 0x7fc7840997f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc77c009d90 tx=0x7fc77c004a40 comp rx=0 tx=0).stop 2024-05-11T12:11:13.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.706+0000 7fc783400640 1 -- 172.21.15.167:0/3317294111 reap_dead start 2024-05-11T12:11:13.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.706+0000 7fc789600640 1 -- 172.21.15.167:0/3317294111 shutdown_connections 2024-05-11T12:11:13.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.706+0000 7fc789600640 1 -- 172.21.15.167:0/3317294111 >> 172.21.15.167:0/3317294111 conn(0x7fc7840b5e20 msgr2=0x7fc78406f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:13.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.706+0000 7fc789600640 1 -- 172.21.15.167:0/3317294111 shutdown_connections 2024-05-11T12:11:13.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.706+0000 7fc789600640 1 -- 172.21.15.167:0/3317294111 wait complete. 2024-05-11T12:11:13.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.706+0000 7fb749000640 1 -- 172.21.15.167:0/838361361 shutdown_connections 2024-05-11T12:11:13.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.706+0000 7fb749000640 1 -- 172.21.15.167:0/838361361 wait complete. 2024-05-11T12:11:13.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.707+0000 7f2792000640 1 -- 172.21.15.167:0/2842278370 shutdown_connections 2024-05-11T12:11:13.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.707+0000 7f2792000640 1 -- 172.21.15.167:0/2842278370 wait complete. 2024-05-11T12:11:13.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.708+0000 7fd26a200640 1 -- 172.21.15.167:0/102625492 --> [v2:172.21.15.204:6824/1178824319,v1:172.21.15.204:6825/1178824319] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd2641e91f0 con 0x7fd24407fb20 2024-05-11T12:11:13.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.712+0000 7fd25b400640 1 -- 172.21.15.167:0/102625492 <== osd.5 v2:172.21.15.204:6824/1178824319 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd2641e91f0 con 0x7fd24407fb20 2024-05-11T12:11:13.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.715+0000 7fd26a200640 1 -- 172.21.15.167:0/102625492 >> [v2:172.21.15.204:6824/1178824319,v1:172.21.15.204:6825/1178824319] conn(0x7fd24407fb20 msgr2=0x7fd244081f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.715+0000 7fd26a200640 1 --2- 172.21.15.167:0/102625492 >> [v2:172.21.15.204:6824/1178824319,v1:172.21.15.204:6825/1178824319] conn(0x7fd24407fb20 0x7fd244081f60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.732 INFO:teuthology.orchestra.run.smithi167.stdout:64424509453 2024-05-11T12:11:13.732 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-11T12:11:13.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.717+0000 7f096ce00640 1 -- 172.21.15.167:0/2554847325 --> [v2:172.21.15.204:6808/223313265,v1:172.21.15.204:6809/223313265] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0944002dd0 con 0x7f094807fa40 2024-05-11T12:11:13.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.717+0000 7fd26a200640 1 -- 172.21.15.167:0/102625492 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd2440763a0 msgr2=0x7fd244078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.717+0000 7fd26a200640 1 --2- 172.21.15.167:0/102625492 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd2440763a0 0x7fd244078860 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd25c000f80 tx=0x7fd25c009000 comp rx=0 tx=0).stop 2024-05-11T12:11:13.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.717+0000 7fd26a200640 1 -- 172.21.15.167:0/102625492 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd26415f330 msgr2=0x7fd264150300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.717+0000 7fd26a200640 1 --2- 172.21.15.167:0/102625492 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd26415f330 0x7fd264150300 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd25000d5f0 tx=0x7fd25000dab0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.718+0000 7fd263e00640 1 -- 172.21.15.167:0/102625492 reap_dead start 2024-05-11T12:11:13.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.718+0000 7fd26a200640 1 -- 172.21.15.167:0/102625492 shutdown_connections 2024-05-11T12:11:13.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.718+0000 7fd26a200640 1 -- 172.21.15.167:0/102625492 >> 172.21.15.167:0/102625492 conn(0x7fd2640b6b80 msgr2=0x7fd2640d3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:13.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.719+0000 7f095b400640 1 -- 172.21.15.167:0/2554847325 <== osd.7 v2:172.21.15.204:6808/223313265 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0944002dd0 con 0x7f094807fa40 2024-05-11T12:11:13.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.718+0000 7fd26a200640 1 -- 172.21.15.167:0/102625492 shutdown_connections 2024-05-11T12:11:13.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.718+0000 7fd26a200640 1 -- 172.21.15.167:0/102625492 wait complete. 2024-05-11T12:11:13.735 INFO:teuthology.orchestra.run.smithi167.stdout:64424509452 2024-05-11T12:11:13.735 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-11T12:11:13.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.720+0000 7f096ce00640 1 -- 172.21.15.167:0/2554847325 >> [v2:172.21.15.204:6808/223313265,v1:172.21.15.204:6809/223313265] conn(0x7f094807fa40 msgr2=0x7f0948081e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.720+0000 7f096ce00640 1 --2- 172.21.15.167:0/2554847325 >> [v2:172.21.15.204:6808/223313265,v1:172.21.15.204:6809/223313265] conn(0x7f094807fa40 0x7f0948081e80 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.720+0000 7f096ce00640 1 -- 172.21.15.167:0/2554847325 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f09480762d0 msgr2=0x7f0948078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.720+0000 7f096ce00640 1 --2- 172.21.15.167:0/2554847325 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f09480762d0 0x7f0948078790 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f095c006fd0 tx=0x7f095c008040 comp rx=0 tx=0).stop 2024-05-11T12:11:13.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.720+0000 7f096ce00640 1 -- 172.21.15.167:0/2554847325 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0960054130 msgr2=0x7f09600d70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:13.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.720+0000 7f096ce00640 1 --2- 172.21.15.167:0/2554847325 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0960054130 0x7f09600d70b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0954009870 tx=0x7f09540046c0 comp rx=0 tx=0).stop 2024-05-11T12:11:13.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.721+0000 7f0967e00640 1 -- 172.21.15.167:0/2554847325 reap_dead start 2024-05-11T12:11:13.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.721+0000 7f096ce00640 1 -- 172.21.15.167:0/2554847325 shutdown_connections 2024-05-11T12:11:13.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.721+0000 7f096ce00640 1 -- 172.21.15.167:0/2554847325 >> 172.21.15.167:0/2554847325 conn(0x7f09600559e0 msgr2=0x7f0960039c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:13.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.721+0000 7f096ce00640 1 -- 172.21.15.167:0/2554847325 shutdown_connections 2024-05-11T12:11:13.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:13.721+0000 7f096ce00640 1 -- 172.21.15.167:0/2554847325 wait complete. 2024-05-11T12:11:13.737 INFO:teuthology.orchestra.run.smithi167.stdout:60129542156 2024-05-11T12:11:13.737 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-11T12:11:13.738 INFO:teuthology.orchestra.run.smithi167.stdout:68719476749 2024-05-11T12:11:13.739 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-11T12:11:13.739 INFO:teuthology.orchestra.run.smithi167.stdout:68719476748 2024-05-11T12:11:13.740 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-11T12:11:13.741 INFO:teuthology.orchestra.run.smithi167.stdout:64424509452 2024-05-11T12:11:13.741 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-11T12:11:13.742 INFO:teuthology.orchestra.run.smithi167.stdout:68719476747 2024-05-11T12:11:13.742 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-11T12:11:13.743 INFO:teuthology.orchestra.run.smithi167.stdout:68719476747 2024-05-11T12:11:13.743 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-11T12:11:14.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.023+0000 7f3409400640 1 Processor -- start 2024-05-11T12:11:14.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.023+0000 7f2e64c00640 1 Processor -- start 2024-05-11T12:11:14.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.023+0000 7f1dabe00640 1 Processor -- start 2024-05-11T12:11:14.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.024+0000 7f3409400640 1 -- start start 2024-05-11T12:11:14.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.024+0000 7f1dabe00640 1 -- start start 2024-05-11T12:11:14.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.024+0000 7f2e64c00640 1 -- start start 2024-05-11T12:11:14.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.025+0000 7f3409400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f34040b82c0 0x7f34040b8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.025+0000 7f3409400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34040d3c70 0x7f34040d4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.025+0000 7f3409400640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34040d1840 con 0x7f340415f330 2024-05-11T12:11:14.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.025+0000 7f3409400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f340406c450 con 0x7f34040b82c0 2024-05-11T12:11:14.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.025+0000 7f3409400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f340406d2b0 con 0x7f34040d3c70 2024-05-11T12:11:14.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.026+0000 7f3402800640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f340415f330 0x7f34040d1130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49438/0 (socket says 172.21.15.167:49438) 2024-05-11T12:11:14.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.026+0000 7f3402800640 1 -- 172.21.15.167:0/3353112313 learned_addr learned my addr 172.21.15.167:0/3353112313 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:14.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.026+0000 7f1dabe00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1dac073b10 0x7f1dac072890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.027+0000 7f1dabe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1dac15b240 0x7f1dac06e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.027+0000 7f3403200640 1 --2- 172.21.15.167:0/3353112313 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34040d3c70 0x7f34040d4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:14.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.026+0000 7f2e64c00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2e60159970 0x7f2e600d1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.026+0000 7f2e64c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2e600d2db0 0x7f2e600d3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.026+0000 7f2e64c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e60159d50 con 0x7f2e600d17b0 2024-05-11T12:11:14.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.026+0000 7f2e64c00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f2e600d04b0 con 0x7f2e60159970 2024-05-11T12:11:14.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.026+0000 7f2e64c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f2e600d0790 con 0x7f2e600d2db0 2024-05-11T12:11:14.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.027+0000 7f1dabe00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dac0b3c80 con 0x7f1dac15ae60 2024-05-11T12:11:14.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.027+0000 7f1dabe00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f1dac0b4830 con 0x7f1dac073b10 2024-05-11T12:11:14.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.028+0000 7f1dabe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1dac0b3f00 con 0x7f1dac15b240 2024-05-11T12:11:14.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.028+0000 7f1daa000640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1dac073b10 0x7f1dac072890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:14.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.028+0000 7f1daaa00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f1dac15ae60 0x7f1dac0cd5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49454/0 (socket says 172.21.15.167:49454) 2024-05-11T12:11:14.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.028+0000 7f1dab400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1dac15b240 0x7f1dac06e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:14.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.028+0000 7f1daaa00640 1 -- 172.21.15.167:0/474024002 learned_addr learned my addr 172.21.15.167:0/474024002 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:14.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.027+0000 7f2e5de00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2e60159970 0x7f2e600d1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:14.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.027+0000 7f2e5de00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2e60159970 0x7f2e600d1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41598/0 (socket says 172.21.15.167:41598) 2024-05-11T12:11:14.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.027+0000 7f2e5de00640 1 -- 172.21.15.167:0/2318034010 learned_addr learned my addr 172.21.15.167:0/2318034010 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:14.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.029+0000 7f2e5e800640 1 --2- 172.21.15.167:0/2318034010 >> v2:172.21.15.204:3300/0 conn(0x7f2e600d2db0 0x7f2e600d3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:14.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.028+0000 7f3403200640 1 -- 172.21.15.167:0/3353112313 >> v2:172.21.15.204:3300/0 conn(0x7f34040b82c0 msgr2=0x7f34040b8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:14.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.028+0000 7f3403200640 1 --2- 172.21.15.167:0/3353112313 >> v2:172.21.15.204:3300/0 conn(0x7f34040b82c0 0x7f34040b8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.028+0000 7f3403200640 1 -- 172.21.15.167:0/3353112313 >> v1:172.21.15.167:6789/0 conn(0x7f340415f330 legacy=0x7f34040d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:14.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.028+0000 7f3403200640 1 -- 172.21.15.167:0/3353112313 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f340406c6a0 con 0x7f34040d3c70 2024-05-11T12:11:14.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.029+0000 7f2e5de00640 1 -- 172.21.15.167:0/2318034010 >> v2:172.21.15.204:3300/0 conn(0x7f2e600d2db0 msgr2=0x7f2e600d3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:14.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.029+0000 7f2e5de00640 1 --2- 172.21.15.167:0/2318034010 >> v2:172.21.15.204:3300/0 conn(0x7f2e600d2db0 0x7f2e600d3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.029+0000 7f2e5de00640 1 -- 172.21.15.167:0/2318034010 >> v1:172.21.15.204:6789/0 conn(0x7f2e600d17b0 legacy=0x7f2e6006d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:14.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.029+0000 7f2e5de00640 1 -- 172.21.15.167:0/2318034010 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e600d0a70 con 0x7f2e60159970 2024-05-11T12:11:14.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.029+0000 7f3403200640 1 --2- 172.21.15.167:0/3353112313 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34040d3c70 0x7f34040d4100 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f33fc009d90 tx=0x7f33fc02a6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3716603fdfb94e4e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:14.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.030+0000 7f3401400640 1 -- 172.21.15.167:0/3353112313 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33fc007740 con 0x7f34040d3c70 2024-05-11T12:11:14.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.030+0000 7f3401400640 1 -- 172.21.15.167:0/3353112313 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33fc0078e0 con 0x7f34040d3c70 2024-05-11T12:11:14.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.030+0000 7f3401400640 1 -- 172.21.15.167:0/3353112313 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33fc007cf0 con 0x7f34040d3c70 2024-05-11T12:11:14.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.030+0000 7f1da9600640 1 -- 172.21.15.167:0/474024002 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1593570872 0 0) 0x7f1d9c001110 con 0x7f1dac15ae60 2024-05-11T12:11:14.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.030+0000 7f1da9600640 1 -- 172.21.15.167:0/474024002 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d90003520 con 0x7f1dac15ae60 2024-05-11T12:11:14.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.030+0000 7f1dab400640 1 -- 172.21.15.167:0/474024002 >> v2:172.21.15.167:3300/0 conn(0x7f1dac073b10 msgr2=0x7f1dac072890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:14.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.030+0000 7f1dab400640 1 --2- 172.21.15.167:0/474024002 >> v2:172.21.15.167:3300/0 conn(0x7f1dac073b10 0x7f1dac072890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.030+0000 7f1dab400640 1 -- 172.21.15.167:0/474024002 >> v1:172.21.15.167:6789/0 conn(0x7f1dac15ae60 legacy=0x7f1dac0cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:14.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.030+0000 7f1dab400640 1 -- 172.21.15.167:0/474024002 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dac0b41e0 con 0x7f1dac15b240 2024-05-11T12:11:14.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.031+0000 7f5cb3c00640 1 Processor -- start 2024-05-11T12:11:14.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.032+0000 7f5cb3c00640 1 -- start start 2024-05-11T12:11:14.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.032+0000 7f3409400640 1 -- 172.21.15.167:0/3353112313 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34040d3c70 msgr2=0x7f34040d4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:14.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.032+0000 7f3409400640 1 --2- 172.21.15.167:0/3353112313 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34040d3c70 0x7f34040d4100 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f33fc009d90 tx=0x7f33fc02a6a0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.032+0000 7f3409400640 1 -- 172.21.15.167:0/3353112313 shutdown_connections 2024-05-11T12:11:14.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.032+0000 7f3409400640 1 --2- 172.21.15.167:0/3353112313 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34040d3c70 0x7f34040d4100 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.032+0000 7f3409400640 1 --2- 172.21.15.167:0/3353112313 >> v2:172.21.15.204:3300/0 conn(0x7f34040b82c0 0x7f34040b8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.032+0000 7f3409400640 1 -- 172.21.15.167:0/3353112313 >> 172.21.15.167:0/3353112313 conn(0x7f34040b6b80 msgr2=0x7f34040b6f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:14.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.031+0000 7f2e5de00640 1 --2- 172.21.15.167:0/2318034010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e60159970 0x7f2e600d1270 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2e600d1c10 tx=0x7f2e4c02a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b04910645a0033 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:14.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.032+0000 7f2e5ca00640 1 -- 172.21.15.167:0/2318034010 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c037070 con 0x7f2e60159970 2024-05-11T12:11:14.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.032+0000 7f2e5ca00640 1 -- 172.21.15.167:0/2318034010 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e4c032640 con 0x7f2e60159970 2024-05-11T12:11:14.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.032+0000 7f2e5ca00640 1 -- 172.21.15.167:0/2318034010 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c032a50 con 0x7f2e60159970 2024-05-11T12:11:14.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.033+0000 7f1dab400640 1 --2- 172.21.15.167:0/474024002 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1dac15b240 0x7f1dac06e870 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1da400b250 tx=0x7f1da4008740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb3496208608d41c server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:14.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.033+0000 7f2e64c00640 1 -- 172.21.15.167:0/2318034010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e60159970 msgr2=0x7f2e600d1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:14.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.033+0000 7f2e64c00640 1 --2- 172.21.15.167:0/2318034010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e60159970 0x7f2e600d1270 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2e600d1c10 tx=0x7f2e4c02a2c0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.033+0000 7f3409400640 1 -- 172.21.15.167:0/3353112313 shutdown_connections 2024-05-11T12:11:14.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.033+0000 7f2e64c00640 1 -- 172.21.15.167:0/2318034010 shutdown_connections 2024-05-11T12:11:14.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.033+0000 7f2e64c00640 1 --2- 172.21.15.167:0/2318034010 >> v2:172.21.15.204:3300/0 conn(0x7f2e600d2db0 0x7f2e600d3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.033+0000 7f2e64c00640 1 --2- 172.21.15.167:0/2318034010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e60159970 0x7f2e600d1270 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.033+0000 7f2e64c00640 1 -- 172.21.15.167:0/2318034010 >> 172.21.15.167:0/2318034010 conn(0x7f2e600d51e0 msgr2=0x7f2e600d55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:14.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.033+0000 7f3409400640 1 -- 172.21.15.167:0/3353112313 wait complete. 2024-05-11T12:11:14.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.033+0000 7f1da9600640 1 -- 172.21.15.167:0/474024002 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da4032070 con 0x7f1dac15b240 2024-05-11T12:11:14.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.034+0000 7f1da9600640 1 -- 172.21.15.167:0/474024002 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1da400a4e0 con 0x7f1dac15b240 2024-05-11T12:11:14.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.034+0000 7f1da9600640 1 -- 172.21.15.167:0/474024002 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da4032070 con 0x7f1dac15b240 2024-05-11T12:11:14.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.035+0000 7f1dabe00640 1 -- 172.21.15.167:0/474024002 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1dac15b240 msgr2=0x7f1dac06e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:14.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.035+0000 7f1dabe00640 1 --2- 172.21.15.167:0/474024002 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1dac15b240 0x7f1dac06e870 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1da400b250 tx=0x7f1da4008740 comp rx=0 tx=0).stop 2024-05-11T12:11:14.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.035+0000 7f1dabe00640 1 -- 172.21.15.167:0/474024002 shutdown_connections 2024-05-11T12:11:14.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.034+0000 7f5cb3c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5cac15b410 0x7f5cac0cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.034+0000 7f5cb3c00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5cac0b6550 0x7f5cac0cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.034+0000 7f5cb3c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cac0cfb50 0x7f5cac15ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.034+0000 7f5cb3c00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f5cac06e460 con 0x7f5cac0b6550 2024-05-11T12:11:14.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.034+0000 7f5cb3c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f5cac072b00 con 0x7f5cac15b410 2024-05-11T12:11:14.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.034+0000 7f5cb3c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5cac072da0 con 0x7f5cac0cfb50 2024-05-11T12:11:14.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.035+0000 7f5cb1c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cac0cfb50 0x7f5cac15ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:14.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.035+0000 7f5cabe00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5cac0b6550 0x7f5cac0cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:14.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.035+0000 7f5cb1c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cac0cfb50 0x7f5cac15ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:59772/0 (socket says 172.21.15.167:59772) 2024-05-11T12:11:14.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.035+0000 7f5cb1c00640 1 -- 172.21.15.167:0/1618114231 learned_addr learned my addr 172.21.15.167:0/1618114231 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:14.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.033+0000 7f2e64c00640 1 -- 172.21.15.167:0/2318034010 shutdown_connections 2024-05-11T12:11:14.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.033+0000 7f2e64c00640 1 -- 172.21.15.167:0/2318034010 wait complete. 2024-05-11T12:11:14.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.036+0000 7f3409400640 1 Processor -- start 2024-05-11T12:11:14.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.036+0000 7f1dabe00640 1 --2- 172.21.15.167:0/474024002 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1dac15b240 0x7f1dac06e870 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.036+0000 7f1dabe00640 1 --2- 172.21.15.167:0/474024002 >> v2:172.21.15.167:3300/0 conn(0x7f1dac073b10 0x7f1dac072890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.036+0000 7f1dabe00640 1 -- 172.21.15.167:0/474024002 >> 172.21.15.167:0/474024002 conn(0x7f1dac06c450 msgr2=0x7f1dac06c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:14.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.035+0000 7f5cb1200640 1 --2- 172.21.15.167:0/1618114231 >> v2:172.21.15.204:3300/0 conn(0x7f5cac15b410 0x7f5cac0cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:14.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.036+0000 7f3409400640 1 -- start start 2024-05-11T12:11:14.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.036+0000 7f1dabe00640 1 -- 172.21.15.167:0/474024002 shutdown_connections 2024-05-11T12:11:14.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.036+0000 7f1dabe00640 1 -- 172.21.15.167:0/474024002 wait complete. 2024-05-11T12:11:14.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.037+0000 7f3409400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34040b82c0 0x7f34041612c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.037+0000 7f3409400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34040d3c70 0x7f34040ce330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.037+0000 7f3409400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f340415f330 0x7f34040ce870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.037+0000 7f3409400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f340406cf40 con 0x7f340415f330 2024-05-11T12:11:14.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.037+0000 7f3409400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f340406cc60 con 0x7f34040d3c70 2024-05-11T12:11:14.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.037+0000 7f3409400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f340415f710 con 0x7f34040b82c0 2024-05-11T12:11:14.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.038+0000 7f3401e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34040d3c70 0x7f34040ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:14.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.036+0000 7f5cb1c00640 1 -- 172.21.15.167:0/1618114231 >> v2:172.21.15.204:3300/0 conn(0x7f5cac15b410 msgr2=0x7f5cac0cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:14.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.036+0000 7f5cb1c00640 1 --2- 172.21.15.167:0/1618114231 >> v2:172.21.15.204:3300/0 conn(0x7f5cac15b410 0x7f5cac0cf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.036+0000 7f5cb1c00640 1 -- 172.21.15.167:0/1618114231 >> v2:172.21.15.167:3300/0 conn(0x7f5cac0b6550 msgr2=0x7f5cac0cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:14.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.036+0000 7f5cb1c00640 1 --2- 172.21.15.167:0/1618114231 >> v2:172.21.15.167:3300/0 conn(0x7f5cac0b6550 0x7f5cac0cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.036+0000 7f5cb1c00640 1 -- 172.21.15.167:0/1618114231 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cac06e6d0 con 0x7f5cac0cfb50 2024-05-11T12:11:14.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.037+0000 7f8ed2a00640 1 Processor -- start 2024-05-11T12:11:14.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.036+0000 7f2e64c00640 1 Processor -- start 2024-05-11T12:11:14.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.037+0000 7f2e64c00640 1 -- start start 2024-05-11T12:11:14.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.038+0000 7f3401e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34040d3c70 0x7f34040ce330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.167:47174/0 (socket says 172.21.15.167:47174) 2024-05-11T12:11:14.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.038+0000 7f3401e00640 1 -- 172.21.15.167:0/3335545124 learned_addr learned my addr 172.21.15.167:0/3335545124 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:14.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.038+0000 7f3402800640 1 --2- 172.21.15.167:0/3335545124 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34040b82c0 0x7f34041612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:14.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.038+0000 7f8ed2a00640 1 -- start start 2024-05-11T12:11:14.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.038+0000 7f5cb1c00640 1 --2- 172.21.15.167:0/1618114231 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cac0cfb50 0x7f5cac15ee30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5cac06dff0 tx=0x7f5ca402eac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb9fb13f13dfda91 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:14.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.039+0000 7f3403200640 1 --2- 172.21.15.167:0/3335545124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f340415f330 0x7f34040ce870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:14.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.039+0000 7f1dabe00640 1 Processor -- start 2024-05-11T12:11:14.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.039+0000 7f3401e00640 1 -- 172.21.15.167:0/3335545124 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34040b82c0 msgr2=0x7f34041612c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:14.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.039+0000 7f5cab400640 1 -- 172.21.15.167:0/1618114231 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca4036070 con 0x7f5cac0cfb50 2024-05-11T12:11:14.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.039+0000 7f5cab400640 1 -- 172.21.15.167:0/1618114231 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ca4030610 con 0x7f5cac0cfb50 2024-05-11T12:11:14.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.039+0000 7f5cab400640 1 -- 172.21.15.167:0/1618114231 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca4030a20 con 0x7f5cac0cfb50 2024-05-11T12:11:14.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.039+0000 7f1dabe00640 1 -- start start 2024-05-11T12:11:14.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.039+0000 7f93c0800640 1 Processor -- start 2024-05-11T12:11:14.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.040+0000 7f3401e00640 1 --2- 172.21.15.167:0/3335545124 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34040b82c0 0x7f34041612c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.040+0000 7f3401e00640 1 -- 172.21.15.167:0/3335545124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f340415f330 msgr2=0x7f34040ce870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:14.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.040+0000 7f3401e00640 1 --2- 172.21.15.167:0/3335545124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f340415f330 0x7f34040ce870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.040+0000 7f3401e00640 1 -- 172.21.15.167:0/3335545124 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f340415e7d0 con 0x7f34040d3c70 2024-05-11T12:11:14.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.040+0000 7f3403200640 1 --2- 172.21.15.167:0/3335545124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f340415f330 0x7f34040ce870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:11:14.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.040+0000 7f3402800640 1 --2- 172.21.15.167:0/3335545124 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34040b82c0 0x7f34041612c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:11:14.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.040+0000 7f2e64c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e600d17b0 0x7f2e600caff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.040+0000 7f2e64c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e600d2db0 0x7f2e601fc740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.040+0000 7f2e64c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e60159970 0x7f2e6014af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.040+0000 7f3401e00640 1 --2- 172.21.15.167:0/3335545124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34040d3c70 0x7f34040ce330 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f33f000b4a0 tx=0x7f33f000b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:14.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.040+0000 7f5cb3c00640 1 -- 172.21.15.167:0/1618114231 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cac0cfb50 msgr2=0x7f5cac15ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:14.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.040+0000 7f5cb3c00640 1 --2- 172.21.15.167:0/1618114231 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cac0cfb50 0x7f5cac15ee30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5cac06dff0 tx=0x7f5ca402eac0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.041+0000 7f5cb3c00640 1 -- 172.21.15.167:0/1618114231 shutdown_connections 2024-05-11T12:11:14.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.041+0000 7f5cb3c00640 1 --2- 172.21.15.167:0/1618114231 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cac0cfb50 0x7f5cac15ee30 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.041+0000 7f5cb3c00640 1 --2- 172.21.15.167:0/1618114231 >> v2:172.21.15.167:3300/0 conn(0x7f5cac0b6550 0x7f5cac0cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.041+0000 7f5cb3c00640 1 --2- 172.21.15.167:0/1618114231 >> v2:172.21.15.204:3300/0 conn(0x7f5cac15b410 0x7f5cac0cf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.041+0000 7f5cb3c00640 1 -- 172.21.15.167:0/1618114231 >> 172.21.15.167:0/1618114231 conn(0x7f5cac06c450 msgr2=0x7f5cac06c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:14.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.041+0000 7f33faa00640 1 -- 172.21.15.167:0/3335545124 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f0014070 con 0x7f34040d3c70 2024-05-11T12:11:14.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.041+0000 7f33faa00640 1 -- 172.21.15.167:0/3335545124 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33f0010610 con 0x7f34040d3c70 2024-05-11T12:11:14.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.041+0000 7f33faa00640 1 -- 172.21.15.167:0/3335545124 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f0010a00 con 0x7f34040d3c70 2024-05-11T12:11:14.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.041+0000 7f2e5d400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e600d2db0 0x7f2e601fc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:14.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.041+0000 7f2e5d400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e600d2db0 0x7f2e601fc740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41638/0 (socket says 172.21.15.167:41638) 2024-05-11T12:11:14.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.041+0000 7f2e5de00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e600d17b0 0x7f2e600caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:14.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.041+0000 7f3409400640 1 -- 172.21.15.167:0/3335545124 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f340415ea90 con 0x7f34040d3c70 2024-05-11T12:11:14.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.041+0000 7f3409400640 1 -- 172.21.15.167:0/3335545124 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3404071340 con 0x7f34040d3c70 2024-05-11T12:11:14.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.041+0000 7f2e64c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6006d910 con 0x7f2e600d2db0 2024-05-11T12:11:14.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.041+0000 7f2e64c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2e60163d00 con 0x7f2e60159970 2024-05-11T12:11:14.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.041+0000 7f2e64c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2e6006da90 con 0x7f2e600d17b0 2024-05-11T12:11:14.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.041+0000 7f2e5d400640 1 -- 172.21.15.167:0/193936246 learned_addr learned my addr 172.21.15.167:0/193936246 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:14.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.040+0000 7f93c0800640 1 -- start start 2024-05-11T12:11:14.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.042+0000 7f93c0800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f93b81636b0 0x7f93b80d0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.042+0000 7f93c0800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93b80b4ae0 0x7f93b80b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.041+0000 7f8ed2a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8ecc15f330 0x7f8ecc0d1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.041+0000 7f8ed2a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8ecc0d3c70 0x7f8ecc0d4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.041+0000 7f8ed2a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ecc06c6a0 con 0x7f8ecc0d1670 2024-05-11T12:11:14.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.041+0000 7f8ed2a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f8ecc06c450 con 0x7f8ecc15f330 2024-05-11T12:11:14.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.041+0000 7f8ed2a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f8ecc06d2b0 con 0x7f8ecc0d3c70 2024-05-11T12:11:14.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.042+0000 7f93c0800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93b80b6b10 con 0x7f93b80b5e50 2024-05-11T12:11:14.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.042+0000 7f93c0800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f93b80b5480 con 0x7f93b81636b0 2024-05-11T12:11:14.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.042+0000 7f93c0800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f93b80cf9b0 con 0x7f93b80b4ae0 2024-05-11T12:11:14.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.042+0000 7f5cb3c00640 1 -- 172.21.15.167:0/1618114231 shutdown_connections 2024-05-11T12:11:14.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.042+0000 7f2e5d400640 1 -- 172.21.15.167:0/193936246 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e600d17b0 msgr2=0x7f2e600caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:14.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.042+0000 7f1dabe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dac073b10 0x7f1dac1ea290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.042+0000 7f8ecbe00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8ecc15f330 0x7f8ecc0d1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:14.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.042+0000 7f8ecbe00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8ecc15f330 0x7f8ecc0d1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.167:47192/0 (socket says 172.21.15.167:47192) 2024-05-11T12:11:14.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.042+0000 7f8ecbe00640 1 -- 172.21.15.167:0/630077051 learned_addr learned my addr 172.21.15.167:0/630077051 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:14.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.042+0000 7f5cb3c00640 1 -- 172.21.15.167:0/1618114231 wait complete. 2024-05-11T12:11:14.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.043+0000 7f93bd400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93b80b4ae0 0x7f93b80b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:14.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.043+0000 7f93bd400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93b80b4ae0 0x7f93b80b4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:59788/0 (socket says 172.21.15.167:59788) 2024-05-11T12:11:14.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.043+0000 7f93bd400640 1 -- 172.21.15.167:0/3345083388 learned_addr learned my addr 172.21.15.167:0/3345083388 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:14.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.042+0000 7f1dabe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1dac15ae60 0x7f1dac14dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.043+0000 7f1dabe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dac15b240 0x7f1dac14fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.043+0000 7f1dabe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac072f50 con 0x7f1dac073b10 2024-05-11T12:11:14.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.043+0000 7f8ecbe00640 1 -- 172.21.15.167:0/630077051 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8ecc0d3c70 msgr2=0x7f8ecc0d4100 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T12:11:14.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.043+0000 7f1dabe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac072dd0 con 0x7f1dac15b240 2024-05-11T12:11:14.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.043+0000 7f8ecbe00640 1 --2- 172.21.15.167:0/630077051 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8ecc0d3c70 0x7f8ecc0d4100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.043+0000 7f8ecbe00640 1 -- 172.21.15.167:0/630077051 >> v1:172.21.15.204:6789/0 conn(0x7f8ecc0d1670 legacy=0x7f8ecc0b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:14.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.043+0000 7f2e5d400640 1 --2- 172.21.15.167:0/193936246 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e600d17b0 0x7f2e600caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.043+0000 7f2e5d400640 1 -- 172.21.15.167:0/193936246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e60159970 msgr2=0x7f2e6014af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:14.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.043+0000 7f2e5d400640 1 --2- 172.21.15.167:0/193936246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e60159970 0x7f2e6014af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.043+0000 7f2e5d400640 1 -- 172.21.15.167:0/193936246 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e601f7260 con 0x7f2e600d2db0 2024-05-11T12:11:14.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.043+0000 7f1dabe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1dac06e420 con 0x7f1dac15ae60 2024-05-11T12:11:14.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.043+0000 7f2e5de00640 1 --2- 172.21.15.167:0/193936246 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e600d17b0 0x7f2e600caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:14.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.044+0000 7f93bd400640 1 -- 172.21.15.167:0/3345083388 >> v2:172.21.15.204:3300/0 conn(0x7f93b81636b0 msgr2=0x7f93b80d0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:14.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.044+0000 7f93bd400640 1 --2- 172.21.15.167:0/3345083388 >> v2:172.21.15.204:3300/0 conn(0x7f93b81636b0 0x7f93b80d0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.044+0000 7f93bd400640 1 -- 172.21.15.167:0/3345083388 >> v1:172.21.15.204:6789/0 conn(0x7f93b80b5e50 legacy=0x7f93b80b62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:14.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.043+0000 7f1daaa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dac073b10 0x7f1dac1ea290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:14.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.044+0000 7f1dab400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dac15b240 0x7f1dac14fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:14.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.044+0000 7f1dab400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dac15b240 0x7f1dac14fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.167:47206/0 (socket says 172.21.15.167:47206) 2024-05-11T12:11:14.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.044+0000 7f1dab400640 1 -- 172.21.15.167:0/2680442783 learned_addr learned my addr 172.21.15.167:0/2680442783 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:14.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.044+0000 7f93bd400640 1 -- 172.21.15.167:0/3345083388 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93b80cfc90 con 0x7f93b80b4ae0 2024-05-11T12:11:14.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.044+0000 7f1daa000640 1 --2- 172.21.15.167:0/2680442783 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1dac15ae60 0x7f1dac14dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:14.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.043+0000 7f8ecbe00640 1 -- 172.21.15.167:0/630077051 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ecc06c980 con 0x7f8ecc15f330 2024-05-11T12:11:14.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.045+0000 7f5cb3c00640 1 Processor -- start 2024-05-11T12:11:14.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.045+0000 7f8ecbe00640 1 --2- 172.21.15.167:0/630077051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ecc15f330 0x7f8ecc0d1130 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4009e50 tx=0x7f8ec402a730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a2e555f67164144 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:14.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.045+0000 7f1dab400640 1 -- 172.21.15.167:0/2680442783 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1dac15ae60 msgr2=0x7f1dac14dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:14.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.045+0000 7f1dab400640 1 --2- 172.21.15.167:0/2680442783 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1dac15ae60 0x7f1dac14dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.045+0000 7f1dab400640 1 -- 172.21.15.167:0/2680442783 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dac073b10 msgr2=0x7f1dac1ea290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:14.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.045+0000 7f1dab400640 1 --2- 172.21.15.167:0/2680442783 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dac073b10 0x7f1dac1ea290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.045+0000 7f1dab400640 1 -- 172.21.15.167:0/2680442783 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dac14ea20 con 0x7f1dac15b240 2024-05-11T12:11:14.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.046+0000 7f8ecaa00640 1 -- 172.21.15.167:0/630077051 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4036070 con 0x7f8ecc15f330 2024-05-11T12:11:14.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.046+0000 7f1daa000640 1 --2- 172.21.15.167:0/2680442783 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1dac15ae60 0x7f1dac14dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:11:14.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.046+0000 7f1dab400640 1 --2- 172.21.15.167:0/2680442783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dac15b240 0x7f1dac14fac0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1da400b370 tx=0x7f1da4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:14.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.046+0000 7f8ecaa00640 1 -- 172.21.15.167:0/630077051 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ec4031600 con 0x7f8ecc15f330 2024-05-11T12:11:14.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.046+0000 7f8ecaa00640 1 -- 172.21.15.167:0/630077051 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec40319d0 con 0x7f8ecc15f330 2024-05-11T12:11:14.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.046+0000 7f2e5d400640 1 --2- 172.21.15.167:0/193936246 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e600d2db0 0x7f2e601fc740 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2e5000c3a0 tx=0x7f2e5000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:14.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.046+0000 7f3409400640 1 -- 172.21.15.167:0/3335545124 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34040694b0 con 0x7f34040d3c70 2024-05-11T12:11:14.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.046+0000 7f2e56000640 1 -- 172.21.15.167:0/193936246 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e50017070 con 0x7f2e600d2db0 2024-05-11T12:11:14.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.046+0000 7f2e56000640 1 -- 172.21.15.167:0/193936246 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e500052d0 con 0x7f2e600d2db0 2024-05-11T12:11:14.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.045+0000 7f5cb3c00640 1 -- start start 2024-05-11T12:11:14.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.046+0000 7f33faa00640 1 -- 172.21.15.167:0/3335545124 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f33f0017070 con 0x7f34040d3c70 2024-05-11T12:11:14.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.046+0000 7f1da2a00640 1 -- 172.21.15.167:0/2680442783 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da4032070 con 0x7f1dac15b240 2024-05-11T12:11:14.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.047+0000 7f1dabe00640 1 -- 172.21.15.167:0/2680442783 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dac14ed00 con 0x7f1dac15b240 2024-05-11T12:11:14.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.047+0000 7f1dabe00640 1 -- 172.21.15.167:0/2680442783 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dac14f5a0 con 0x7f1dac15b240 2024-05-11T12:11:14.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.047+0000 7f1da2a00640 1 -- 172.21.15.167:0/2680442783 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1da400a2e0 con 0x7f1dac15b240 2024-05-11T12:11:14.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.047+0000 7f1da2a00640 1 -- 172.21.15.167:0/2680442783 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da40043d0 con 0x7f1dac15b240 2024-05-11T12:11:14.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.047+0000 7f8ed2a00640 1 -- 172.21.15.167:0/630077051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ecc15f330 msgr2=0x7f8ecc0d1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:14.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.047+0000 7f8ed2a00640 1 --2- 172.21.15.167:0/630077051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ecc15f330 0x7f8ecc0d1130 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4009e50 tx=0x7f8ec402a730 comp rx=0 tx=0).stop 2024-05-11T12:11:14.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.047+0000 7f8ed2a00640 1 -- 172.21.15.167:0/630077051 shutdown_connections 2024-05-11T12:11:14.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.047+0000 7f8ed2a00640 1 --2- 172.21.15.167:0/630077051 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8ecc0d3c70 0x7f8ecc0d4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.047+0000 7f8ed2a00640 1 --2- 172.21.15.167:0/630077051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ecc15f330 0x7f8ecc0d1130 secure :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4009e50 tx=0x7f8ec402a730 comp rx=0 tx=0).stop 2024-05-11T12:11:14.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.047+0000 7f8ed2a00640 1 -- 172.21.15.167:0/630077051 >> 172.21.15.167:0/630077051 conn(0x7f8ecc0b6b80 msgr2=0x7f8ecc0b6f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:14.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.048+0000 7f93bd400640 1 --2- 172.21.15.167:0/3345083388 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93b80b4ae0 0x7f93b80b4f40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f93ac009870 tx=0x7f93ac02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b267a17719975596 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:14.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.048+0000 7f8ed2a00640 1 -- 172.21.15.167:0/630077051 shutdown_connections 2024-05-11T12:11:14.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.047+0000 7f2e56000640 1 -- 172.21.15.167:0/193936246 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e50003830 con 0x7f2e600d2db0 2024-05-11T12:11:14.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.048+0000 7f8ed2a00640 1 -- 172.21.15.167:0/630077051 wait complete. 2024-05-11T12:11:14.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.049+0000 7f1dabe00640 1 -- 172.21.15.167:0/2680442783 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d84005290 con 0x7f1dac15b240 2024-05-11T12:11:14.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.047+0000 7f33faa00640 1 --2- 172.21.15.167:0/3335545124 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f33e40763a0 0x7f33e4078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.048+0000 7f33faa00640 1 -- 172.21.15.167:0/3335545124 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f33f0097290 con 0x7f34040d3c70 2024-05-11T12:11:14.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.048+0000 7f5cb3c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cac0b6550 0x7f5cac152ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.048+0000 7f5cb3c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cac0cfb50 0x7f5cac0aa840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.048+0000 7f5cb3c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cac15b410 0x7f5cac154220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.048+0000 7f5cb3c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5cac0b6e20 con 0x7f5cac0b6550 2024-05-11T12:11:14.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.048+0000 7f5cb3c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5cac0b6ca0 con 0x7f5cac0cfb50 2024-05-11T12:11:14.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.048+0000 7f5cb3c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5cac070240 con 0x7f5cac15b410 2024-05-11T12:11:14.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.049+0000 7f5cabe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cac0cfb50 0x7f5cac0aa840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:14.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.048+0000 7f93bca00640 1 -- 172.21.15.167:0/3345083388 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ac037070 con 0x7f93b80b4ae0 2024-05-11T12:11:14.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.048+0000 7f2e64c00640 1 -- 172.21.15.167:0/193936246 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e600c9040 con 0x7f2e600d2db0 2024-05-11T12:11:14.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.048+0000 7f2e64c00640 1 -- 172.21.15.167:0/193936246 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e600cb9a0 con 0x7f2e600d2db0 2024-05-11T12:11:14.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.049+0000 7f5cb1200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cac0b6550 0x7f5cac152ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:14.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.049+0000 7f5cabe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cac0cfb50 0x7f5cac0aa840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.167:47214/0 (socket says 172.21.15.167:47214) 2024-05-11T12:11:14.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.049+0000 7f5cabe00640 1 -- 172.21.15.167:0/147929241 learned_addr learned my addr 172.21.15.167:0/147929241 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:14.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.050+0000 7f5cb1c00640 1 --2- 172.21.15.167:0/147929241 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cac15b410 0x7f5cac154220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:14.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.050+0000 7f3402800640 1 --2- 172.21.15.167:0/3335545124 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f33e40763a0 0x7f33e4078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:14.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.048+0000 7f93bca00640 1 -- 172.21.15.167:0/3345083388 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93ac032440 con 0x7f93b80b4ae0 2024-05-11T12:11:14.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.049+0000 7f93c0800640 1 -- 172.21.15.167:0/3345083388 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93b80b4ae0 msgr2=0x7f93b80b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:14.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.050+0000 7f93bca00640 1 -- 172.21.15.167:0/3345083388 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ac037070 con 0x7f93b80b4ae0 2024-05-11T12:11:14.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.050+0000 7f93c0800640 1 --2- 172.21.15.167:0/3345083388 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93b80b4ae0 0x7f93b80b4f40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f93ac009870 tx=0x7f93ac02a210 comp rx=0 tx=0).stop 2024-05-11T12:11:14.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.050+0000 7f93c0800640 1 -- 172.21.15.167:0/3345083388 shutdown_connections 2024-05-11T12:11:14.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.050+0000 7f93c0800640 1 --2- 172.21.15.167:0/3345083388 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93b80b4ae0 0x7f93b80b4f40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.050+0000 7f93c0800640 1 --2- 172.21.15.167:0/3345083388 >> v2:172.21.15.204:3300/0 conn(0x7f93b81636b0 0x7f93b80d0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.050+0000 7f93c0800640 1 -- 172.21.15.167:0/3345083388 >> 172.21.15.167:0/3345083388 conn(0x7f93b80b8370 msgr2=0x7f93b80b8750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:14.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.050+0000 7f93c0800640 1 -- 172.21.15.167:0/3345083388 shutdown_connections 2024-05-11T12:11:14.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.050+0000 7f5cabe00640 1 -- 172.21.15.167:0/147929241 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cac15b410 msgr2=0x7f5cac154220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:14.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.050+0000 7f5cabe00640 1 --2- 172.21.15.167:0/147929241 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cac15b410 0x7f5cac154220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.050+0000 7f5cabe00640 1 -- 172.21.15.167:0/147929241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cac0b6550 msgr2=0x7f5cac152ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:14.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.050+0000 7f5cabe00640 1 --2- 172.21.15.167:0/147929241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cac0b6550 0x7f5cac152ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.050+0000 7f5cabe00640 1 -- 172.21.15.167:0/147929241 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cac0aad80 con 0x7f5cac0cfb50 2024-05-11T12:11:14.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.050+0000 7f5cb1200640 1 --2- 172.21.15.167:0/147929241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cac0b6550 0x7f5cac152ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:11:14.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.051+0000 7f8ed2a00640 1 Processor -- start 2024-05-11T12:11:14.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.049+0000 7f2e64c00640 1 -- 172.21.15.167:0/193936246 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e600694b0 con 0x7f2e600d2db0 2024-05-11T12:11:14.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.051+0000 7f5cabe00640 1 --2- 172.21.15.167:0/147929241 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cac0cfb50 0x7f5cac0aa840 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5c9800b950 tx=0x7f5c9800be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:14.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.052+0000 7f8aa1a00640 1 Processor -- start 2024-05-11T12:11:14.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.052+0000 7f3402800640 1 --2- 172.21.15.167:0/3335545124 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f33e40763a0 0x7f33e4078860 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f33f4000f60 tx=0x7f33f4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:14.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.050+0000 7f93c0800640 1 -- 172.21.15.167:0/3345083388 wait complete. 2024-05-11T12:11:14.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.052+0000 7f93c0800640 1 Processor -- start 2024-05-11T12:11:14.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.051+0000 7f8ed2a00640 1 -- start start 2024-05-11T12:11:14.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.052+0000 7f8ed2a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ecc0d3c70 0x7f8ecc15db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.052+0000 7f8ed2a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ecc1e9590 0x7f8ecc1e7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.052+0000 7f8ed2a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8ecc1e8050 0x7f8ecc0ce970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.052+0000 7f8ed2a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc15f710 con 0x7f8ecc1e9590 2024-05-11T12:11:14.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.052+0000 7f8ed2a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc06cf40 con 0x7f8ecc0d3c70 2024-05-11T12:11:14.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.052+0000 7f8ed2a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f8ecc0d06b0 con 0x7f8ecc1e8050 2024-05-11T12:11:14.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.052+0000 7f8aa1a00640 1 -- start start 2024-05-11T12:11:14.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.053+0000 7f93c0800640 1 -- start start 2024-05-11T12:11:14.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.053+0000 7f8ecb400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ecc1e9590 0x7f8ecc1e7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:14.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.053+0000 7f8ecb400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ecc1e9590 0x7f8ecc1e7b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41664/0 (socket says 172.21.15.167:41664) 2024-05-11T12:11:14.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.053+0000 7f8ed0a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8ecc1e8050 0x7f8ecc0ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:14.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.053+0000 7f8ecb400640 1 -- 172.21.15.167:0/49760027 learned_addr learned my addr 172.21.15.167:0/49760027 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:14.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.053+0000 7f8ecbe00640 1 --2- 172.21.15.167:0/49760027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ecc0d3c70 0x7f8ecc15db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:14.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.054+0000 7f93c0800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93b80b4ae0 0x7f93b815b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.054+0000 7f93c0800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93b80b5e50 0x7f93b815f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.052+0000 7f2e56000640 1 -- 172.21.15.167:0/193936246 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2e5001a070 con 0x7f2e600d2db0 2024-05-11T12:11:14.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.054+0000 7f2e56000640 1 --2- 172.21.15.167:0/193936246 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2e400763d0 0x7f2e40078890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.054+0000 7f2e56000640 1 -- 172.21.15.167:0/193936246 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f2e50096bd0 con 0x7f2e600d2db0 2024-05-11T12:11:14.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.054+0000 7f2e5de00640 1 --2- 172.21.15.167:0/193936246 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2e400763d0 0x7f2e40078890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:14.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.054+0000 7f8aa1a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8a9c0b5c20 0x7f8a9c0afc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.054+0000 7f8ed0a00640 1 -- 172.21.15.167:0/49760027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ecc0d3c70 msgr2=0x7f8ecc15db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:14.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.054+0000 7f8ed0a00640 1 --2- 172.21.15.167:0/49760027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ecc0d3c70 0x7f8ecc15db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.054+0000 7f8ed0a00640 1 -- 172.21.15.167:0/49760027 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ecc1e9590 msgr2=0x7f8ecc1e7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:14.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.054+0000 7f8ed0a00640 1 --2- 172.21.15.167:0/49760027 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ecc1e9590 0x7f8ecc1e7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.054+0000 7f8aa1a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a9c0b01b0 0x7f8a9c15f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.055+0000 7f8ed0a00640 1 -- 172.21.15.167:0/49760027 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ecc1e84f0 con 0x7f8ecc1e8050 2024-05-11T12:11:14.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.055+0000 7f8ecb400640 1 --2- 172.21.15.167:0/49760027 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ecc1e9590 0x7f8ecc1e7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:11:14.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.055+0000 7f8aa1a00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a9c0743d0 con 0x7f8a9c15ff10 2024-05-11T12:11:14.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.055+0000 7f8aa1a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f8a9c0b4d60 con 0x7f8a9c0b5c20 2024-05-11T12:11:14.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.055+0000 7f8aa1a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f8a9c073330 con 0x7f8a9c0b01b0 2024-05-11T12:11:14.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.055+0000 7f93bde00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93b80b4ae0 0x7f93b815b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:14.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.055+0000 7f93bde00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93b80b4ae0 0x7f93b815b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:59826/0 (socket says 172.21.15.167:59826) 2024-05-11T12:11:14.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.055+0000 7f8a9be00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8a9c0b5c20 0x7f8a9c0afc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:14.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.055+0000 7f8aa1000640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f8a9c15ff10 0x7f8a9c162c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49466/0 (socket says 172.21.15.167:49466) 2024-05-11T12:11:14.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.055+0000 7f8a9be00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8a9c0b5c20 0x7f8a9c0afc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.167:47230/0 (socket says 172.21.15.167:47230) 2024-05-11T12:11:14.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.055+0000 7f8aa1000640 1 -- 172.21.15.167:0/1734843709 learned_addr learned my addr 172.21.15.167:0/1734843709 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:14.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.055+0000 7f93bd400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93b80b5e50 0x7f93b815f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:14.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.055+0000 7f93bde00640 1 -- 172.21.15.167:0/3918971926 learned_addr learned my addr 172.21.15.167:0/3918971926 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:14.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.055+0000 7f8a9b400640 1 --2- 172.21.15.167:0/1734843709 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a9c0b01b0 0x7f8a9c15f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:14.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.055+0000 7f8ed0a00640 1 --2- 172.21.15.167:0/49760027 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8ecc1e8050 0x7f8ecc0ce970 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc00b440 tx=0x7f8ebc00b900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:14.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.055+0000 7f93c0800640 1 --2- 172.21.15.167:0/3918971926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93b81636b0 0x7f93b815f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.056+0000 7f93c0800640 1 -- 172.21.15.167:0/3918971926 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f93b80d0b50 con 0x7f93b81636b0 2024-05-11T12:11:14.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.056+0000 7f8a9be00640 1 -- 172.21.15.167:0/1734843709 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a9c0b01b0 msgr2=0x7f8a9c15f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:14.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.056+0000 7f8a9be00640 1 --2- 172.21.15.167:0/1734843709 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a9c0b01b0 0x7f8a9c15f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.056+0000 7f8a9be00640 1 -- 172.21.15.167:0/1734843709 >> v1:172.21.15.167:6789/0 conn(0x7f8a9c15ff10 legacy=0x7f8a9c162c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:14.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.056+0000 7f8a9be00640 1 -- 172.21.15.167:0/1734843709 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a9c073610 con 0x7f8a9c0b5c20 2024-05-11T12:11:14.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.056+0000 7f93c0800640 1 -- 172.21.15.167:0/3918971926 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f93b80d09d0 con 0x7f93b80b5e50 2024-05-11T12:11:14.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.056+0000 7f93c0800640 1 -- 172.21.15.167:0/3918971926 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f93b8163a90 con 0x7f93b80b4ae0 2024-05-11T12:11:14.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.056+0000 7f2e5de00640 1 --2- 172.21.15.167:0/193936246 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2e400763d0 0x7f2e40078890 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c034000 tx=0x7f2e4c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:14.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.056+0000 7f93be800640 1 --2- 172.21.15.167:0/3918971926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93b81636b0 0x7f93b815f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:14.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.056+0000 7f8ec3e00640 1 -- 172.21.15.167:0/49760027 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc015020 con 0x7f8ecc1e8050 2024-05-11T12:11:14.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.056+0000 7f8ec3e00640 1 -- 172.21.15.167:0/49760027 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ebc011830 con 0x7f8ecc1e8050 2024-05-11T12:11:14.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.056+0000 7f8ec3e00640 1 -- 172.21.15.167:0/49760027 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc011c20 con 0x7f8ecc1e8050 2024-05-11T12:11:14.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.056+0000 7f8ed2a00640 1 -- 172.21.15.167:0/49760027 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ecc1e87d0 con 0x7f8ecc1e8050 2024-05-11T12:11:14.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.057+0000 7f93bd400640 1 -- 172.21.15.167:0/3918971926 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93b80b4ae0 msgr2=0x7f93b815b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:14.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.057+0000 7f93bd400640 1 --2- 172.21.15.167:0/3918971926 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93b80b4ae0 0x7f93b815b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.057+0000 7f93bd400640 1 -- 172.21.15.167:0/3918971926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93b81636b0 msgr2=0x7f93b815f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:14.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.057+0000 7f93bd400640 1 --2- 172.21.15.167:0/3918971926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93b81636b0 0x7f93b815f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.057+0000 7f93bd400640 1 -- 172.21.15.167:0/3918971926 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93b815be70 con 0x7f93b80b5e50 2024-05-11T12:11:14.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.057+0000 7f93be800640 1 --2- 172.21.15.167:0/3918971926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93b81636b0 0x7f93b815f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:11:14.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.057+0000 7f8ed2a00640 1 -- 172.21.15.167:0/49760027 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ecc1e8d90 con 0x7f8ecc1e8050 2024-05-11T12:11:14.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.057+0000 7f8a9be00640 1 --2- 172.21.15.167:0/1734843709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a9c0b5c20 0x7f8a9c0afc70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8a840097a0 tx=0x7f8a8402a0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa1a61becba8d0e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:14.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.058+0000 7f93bd400640 1 --2- 172.21.15.167:0/3918971926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93b80b5e50 0x7f93b815f2b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f93ac031000 tx=0x7f93ac0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:14.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.058+0000 7f8ed2a00640 1 -- 172.21.15.167:0/49760027 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ecc0694b0 con 0x7f8ecc1e8050 2024-05-11T12:11:14.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.059+0000 7fa067a00640 1 Processor -- start 2024-05-11T12:11:14.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.059+0000 7fa067a00640 1 -- start start 2024-05-11T12:11:14.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.060+0000 7f2e56000640 1 -- 172.21.15.167:0/193936246 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2e600c9040 con 0x7f2e600d2db0 2024-05-11T12:11:14.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.060+0000 7f8ec3e00640 1 -- 172.21.15.167:0/49760027 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8ebc018020 con 0x7f8ecc1e8050 2024-05-11T12:11:14.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.060+0000 7fa067a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa06015f330 0x7fa0600d1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.060+0000 7fa067a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa0600d3c70 0x7fa0600d40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.060+0000 7fa067a00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa06006c6a0 con 0x7fa0600d1670 2024-05-11T12:11:14.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.060+0000 7fa067a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fa0600d4610 con 0x7fa0600d3c70 2024-05-11T12:11:14.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.060+0000 7fa067a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa06006c450 con 0x7fa06015f330 2024-05-11T12:11:14.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.061+0000 7fa05fe00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fa0600d1670 0x7fa0600b82c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49470/0 (socket says 172.21.15.167:49470) 2024-05-11T12:11:14.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.061+0000 7fa05fe00640 1 -- 172.21.15.167:0/2494254235 learned_addr learned my addr 172.21.15.167:0/2494254235 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:14.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.061+0000 7fa065a00640 1 --2- 172.21.15.167:0/2494254235 >> v2:172.21.15.204:3300/0 conn(0x7fa0600d3c70 0x7fa0600d40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:14.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.061+0000 7fa065000640 1 --2- 172.21.15.167:0/2494254235 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa06015f330 0x7fa0600d1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:14.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.061+0000 7fa05f400640 1 -- 172.21.15.167:0/2494254235 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2622998074 0 0) 0x7fa06006c6a0 con 0x7fa0600d1670 2024-05-11T12:11:14.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.061+0000 7fa05f400640 1 -- 172.21.15.167:0/2494254235 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa048003520 con 0x7fa0600d1670 2024-05-11T12:11:14.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.062+0000 7fa05f400640 1 -- 172.21.15.167:0/2494254235 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 41040435 0 0) 0x7fa048003520 con 0x7fa0600d1670 2024-05-11T12:11:14.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.062+0000 7fa05f400640 1 -- 172.21.15.167:0/2494254235 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa06015f330 msgr2=0x7fa0600d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:14.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.062+0000 7fa05f400640 1 --2- 172.21.15.167:0/2494254235 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa06015f330 0x7fa0600d1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.062+0000 7fa05f400640 1 -- 172.21.15.167:0/2494254235 >> v2:172.21.15.204:3300/0 conn(0x7fa0600d3c70 msgr2=0x7fa0600d40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:14.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.062+0000 7fa05f400640 1 --2- 172.21.15.167:0/2494254235 >> v2:172.21.15.204:3300/0 conn(0x7fa0600d3c70 0x7fa0600d40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.062+0000 7fa05f400640 1 -- 172.21.15.167:0/2494254235 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa06006c980 con 0x7fa0600d1670 2024-05-11T12:11:14.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.062+0000 7fa05f400640 1 -- 172.21.15.167:0/2494254235 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7fa04c0016f0 con 0x7fa0600d1670 2024-05-11T12:11:14.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.061+0000 7f8ec3e00640 1 --2- 172.21.15.167:0/49760027 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f8eac076320 0x7f8eac0787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.062+0000 7f8ec3e00640 1 -- 172.21.15.167:0/49760027 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f8ebc097c90 con 0x7f8ecc1e8050 2024-05-11T12:11:14.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.062+0000 7f8ecbe00640 1 --2- 172.21.15.167:0/49760027 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f8eac076320 0x7f8eac0787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:14.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.062+0000 7fa05f400640 1 -- 172.21.15.167:0/2494254235 >> v1:172.21.15.167:6789/0 conn(0x7fa0600d1670 legacy=0x7fa0600b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:14.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.062+0000 7fa05f400640 1 --2- 172.21.15.167:0/2494254235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0480039b0 0x7fa048003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.062+0000 7fa05f400640 1 --2- 172.21.15.167:0/2494254235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa048004800 0x7fa048004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.062+0000 7fa05f400640 1 --2- 172.21.15.167:0/2494254235 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa06015f330 0x7fa0480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.062+0000 7fa05f400640 1 -- 172.21.15.167:0/2494254235 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa06006c450 con 0x7fa0480039b0 2024-05-11T12:11:14.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.062+0000 7fa05f400640 1 -- 172.21.15.167:0/2494254235 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa04800b9c0 con 0x7fa048004800 2024-05-11T12:11:14.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.062+0000 7fa05f400640 1 -- 172.21.15.167:0/2494254235 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa04800bb40 con 0x7fa06015f330 2024-05-11T12:11:14.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.062+0000 7fa065000640 1 --2- 172.21.15.167:0/2494254235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0480039b0 0x7fa048003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:14.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.063+0000 7fa065a00640 1 --2- 172.21.15.167:0/2494254235 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa06015f330 0x7fa0480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:14.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.063+0000 7f8ecbe00640 1 --2- 172.21.15.167:0/49760027 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f8eac076320 0x7f8eac0787e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4033000 tx=0x7f8ec4009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:14.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.063+0000 7fa065000640 1 -- 172.21.15.167:0/2494254235 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa06015f330 msgr2=0x7fa0480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:14.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.063+0000 7fa065000640 1 --2- 172.21.15.167:0/2494254235 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa06015f330 0x7fa0480051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.063+0000 7fa05fe00640 1 --2- 172.21.15.167:0/2494254235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa048004800 0x7fa048004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:14.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.063+0000 7fa065000640 1 -- 172.21.15.167:0/2494254235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa048004800 msgr2=0x7fa048004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:14.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.063+0000 7fa065000640 1 --2- 172.21.15.167:0/2494254235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa048004800 0x7fa048004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.063+0000 7fa065000640 1 -- 172.21.15.167:0/2494254235 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa06006c980 con 0x7fa0480039b0 2024-05-11T12:11:14.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.064+0000 7fa065000640 1 --2- 172.21.15.167:0/2494254235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0480039b0 0x7fa048003dd0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa054009ac0 tx=0x7fa048005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84540af5c1e91ebe server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:14.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.064+0000 7fa05f400640 1 -- 172.21.15.167:0/2494254235 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa054036070 con 0x7fa0480039b0 2024-05-11T12:11:14.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.064+0000 7fa05f400640 1 -- 172.21.15.167:0/2494254235 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa054031400 con 0x7fa0480039b0 2024-05-11T12:11:14.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.064+0000 7fa05f400640 1 -- 172.21.15.167:0/2494254235 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa054009690 con 0x7fa0480039b0 2024-05-11T12:11:14.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.065+0000 7fa067a00640 1 -- 172.21.15.167:0/2494254235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0480039b0 msgr2=0x7fa048003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:14.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.065+0000 7fa067a00640 1 --2- 172.21.15.167:0/2494254235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0480039b0 0x7fa048003dd0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa054009ac0 tx=0x7fa048005830 comp rx=0 tx=0).stop 2024-05-11T12:11:14.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.065+0000 7fa065a00640 1 -- 172.21.15.167:0/2494254235 reap_dead start 2024-05-11T12:11:14.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.065+0000 7fa067a00640 1 -- 172.21.15.167:0/2494254235 shutdown_connections 2024-05-11T12:11:14.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.065+0000 7fa067a00640 1 -- 172.21.15.167:0/2494254235 >> 172.21.15.167:0/2494254235 conn(0x7fa0600b6b80 msgr2=0x7fa0600b6f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:14.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.065+0000 7fa067a00640 1 -- 172.21.15.167:0/2494254235 shutdown_connections 2024-05-11T12:11:14.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.065+0000 7fa067a00640 1 -- 172.21.15.167:0/2494254235 wait complete. 2024-05-11T12:11:14.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.067+0000 7fa067a00640 1 Processor -- start 2024-05-11T12:11:14.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.067+0000 7f33faa00640 1 -- 172.21.15.167:0/3335545124 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f33f0060240 con 0x7f34040d3c70 2024-05-11T12:11:14.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.067+0000 7fa067a00640 1 -- start start 2024-05-11T12:11:14.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.068+0000 7f5ca8c00640 1 -- 172.21.15.167:0/147929241 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9801b070 con 0x7f5cac0cfb50 2024-05-11T12:11:14.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.068+0000 7fa067a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa0480039b0 0x7fa0600ccb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.068+0000 7fa067a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0600cfcd0 0x7fa0600cad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.068+0000 7fa067a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa0600ce430 0x7fa0601572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.068+0000 7fa067a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa06006cf40 con 0x7fa0600cfcd0 2024-05-11T12:11:14.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.068+0000 7fa067a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa06006d2b0 con 0x7fa0600ce430 2024-05-11T12:11:14.101 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.068+0000 7f5cb3c00640 1 -- 172.21.15.167:0/147929241 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cac153180 con 0x7f5cac0cfb50 2024-05-11T12:11:14.101 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.068+0000 7f5ca8c00640 1 -- 172.21.15.167:0/147929241 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c9800ab70 con 0x7f5cac0cfb50 2024-05-11T12:11:14.101 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.068+0000 7f5ca8c00640 1 -- 172.21.15.167:0/147929241 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c98016410 con 0x7f5cac0cfb50 2024-05-11T12:11:14.101 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.068+0000 7f5cb3c00640 1 -- 172.21.15.167:0/147929241 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cac153740 con 0x7f5cac0cfb50 2024-05-11T12:11:14.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.068+0000 7f8a9aa00640 1 -- 172.21.15.167:0/1734843709 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a84037070 con 0x7f8a9c0b5c20 2024-05-11T12:11:14.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.068+0000 7f8a9aa00640 1 -- 172.21.15.167:0/1734843709 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a8402ae40 con 0x7f8a9c0b5c20 2024-05-11T12:11:14.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.068+0000 7f8a9aa00640 1 -- 172.21.15.167:0/1734843709 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a84032660 con 0x7f8a9c0b5c20 2024-05-11T12:11:14.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.068+0000 7fa067a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa06015f710 con 0x7fa0480039b0 2024-05-11T12:11:14.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.069+0000 7fa05fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0600cfcd0 0x7fa0600cad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:14.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.069+0000 7fa05fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0600cfcd0 0x7fa0600cad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41692/0 (socket says 172.21.15.167:41692) 2024-05-11T12:11:14.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.069+0000 7fa05fe00640 1 -- 172.21.15.167:0/1254697110 learned_addr learned my addr 172.21.15.167:0/1254697110 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:14.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.069+0000 7fa065a00640 1 --2- 172.21.15.167:0/1254697110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa0600ce430 0x7fa0601572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:14.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.069+0000 7fa065000640 1 --2- 172.21.15.167:0/1254697110 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa0480039b0 0x7fa0600ccb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:14.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.069+0000 7f8aa1a00640 1 -- 172.21.15.167:0/1734843709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a9c0b5c20 msgr2=0x7f8a9c0afc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:14.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.069+0000 7f8aa1a00640 1 --2- 172.21.15.167:0/1734843709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a9c0b5c20 0x7f8a9c0afc70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8a840097a0 tx=0x7f8a8402a0f0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.070+0000 7f8aa1a00640 1 -- 172.21.15.167:0/1734843709 shutdown_connections 2024-05-11T12:11:14.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.070+0000 7f8aa1a00640 1 --2- 172.21.15.167:0/1734843709 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a9c0b01b0 0x7f8a9c15f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.070+0000 7f8aa1a00640 1 --2- 172.21.15.167:0/1734843709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a9c0b5c20 0x7f8a9c0afc70 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.070+0000 7f8aa1a00640 1 -- 172.21.15.167:0/1734843709 >> 172.21.15.167:0/1734843709 conn(0x7f8a9c06c450 msgr2=0x7f8a9c06c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:14.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.069+0000 7f93b2000640 1 -- 172.21.15.167:0/3918971926 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ac037040 con 0x7f93b80b5e50 2024-05-11T12:11:14.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.069+0000 7f93b2000640 1 -- 172.21.15.167:0/3918971926 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93ac0325f0 con 0x7f93b80b5e50 2024-05-11T12:11:14.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.069+0000 7f93b2000640 1 -- 172.21.15.167:0/3918971926 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ac0329c0 con 0x7f93b80b5e50 2024-05-11T12:11:14.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.070+0000 7f93c0800640 1 -- 172.21.15.167:0/3918971926 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93b815c150 con 0x7f93b80b5e50 2024-05-11T12:11:14.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.070+0000 7f93c0800640 1 -- 172.21.15.167:0/3918971926 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93b815c710 con 0x7f93b80b5e50 2024-05-11T12:11:14.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.070+0000 7f5cb3c00640 1 -- 172.21.15.167:0/147929241 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cac0694b0 con 0x7f5cac0cfb50 2024-05-11T12:11:14.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.071+0000 7f1da2a00640 1 -- 172.21.15.167:0/2680442783 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1da4035070 con 0x7f1dac15b240 2024-05-11T12:11:14.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.071+0000 7fa065a00640 1 -- 172.21.15.167:0/1254697110 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa0480039b0 msgr2=0x7fa0600ccb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:14.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.071+0000 7f8aa1a00640 1 -- 172.21.15.167:0/1734843709 shutdown_connections 2024-05-11T12:11:14.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.071+0000 7f8aa1a00640 1 -- 172.21.15.167:0/1734843709 wait complete. 2024-05-11T12:11:14.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.071+0000 7fa065a00640 1 --2- 172.21.15.167:0/1254697110 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa0480039b0 0x7fa0600ccb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.071+0000 7fa065a00640 1 -- 172.21.15.167:0/1254697110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0600cfcd0 msgr2=0x7fa0600cad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:14.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.071+0000 7fa065a00640 1 --2- 172.21.15.167:0/1254697110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0600cfcd0 0x7fa0600cad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.071+0000 7fa065a00640 1 -- 172.21.15.167:0/1254697110 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0600cb2d0 con 0x7fa0600ce430 2024-05-11T12:11:14.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.072+0000 7fa065a00640 1 --2- 172.21.15.167:0/1254697110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa0600ce430 0x7fa0601572d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa048004750 tx=0x7fa0580115a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:14.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.072+0000 7f93c0800640 1 -- 172.21.15.167:0/3918971926 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9394005290 con 0x7f93b80b5e50 2024-05-11T12:11:14.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.072+0000 7f1da2a00640 1 --2- 172.21.15.167:0/2680442783 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f1d8c0762d0 0x7f1d8c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.072+0000 7f1da2a00640 1 -- 172.21.15.167:0/2680442783 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f1da40b8c20 con 0x7f1dac15b240 2024-05-11T12:11:14.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.073+0000 7f8aa1a00640 1 Processor -- start 2024-05-11T12:11:14.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.073+0000 7f1daaa00640 1 --2- 172.21.15.167:0/2680442783 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f1d8c0762d0 0x7f1d8c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:14.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.073+0000 7f8aa1a00640 1 -- start start 2024-05-11T12:11:14.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.074+0000 7f1daaa00640 1 --2- 172.21.15.167:0/2680442783 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f1d8c0762d0 0x7f1d8c078790 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c007fd0 tx=0x7f1d9c007d10 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:14.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.075+0000 7f8aa1a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a9c0b01b0 0x7f8a9c0c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.075+0000 7f8aa1a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a9c0b5c20 0x7f8a9c15c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.075+0000 7f8aa1a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a9c15ff10 0x7f8a9c1f8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.075+0000 7f8aa1a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9c0b6000 con 0x7f8a9c0b01b0 2024-05-11T12:11:14.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.075+0000 7f8aa1a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9c073bd0 con 0x7f8a9c0b5c20 2024-05-11T12:11:14.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.075+0000 7f8aa1a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f8a9c0b6180 con 0x7f8a9c15ff10 2024-05-11T12:11:14.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.075+0000 7f8a9be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a9c0b01b0 0x7f8a9c0c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:14.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.075+0000 7f8a9be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a9c0b01b0 0x7f8a9c0c5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41702/0 (socket says 172.21.15.167:41702) 2024-05-11T12:11:14.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.075+0000 7f8a9be00640 1 -- 172.21.15.167:0/4024604000 learned_addr learned my addr 172.21.15.167:0/4024604000 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:14.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.075+0000 7f8a9be00640 1 -- 172.21.15.167:0/4024604000 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a9c15ff10 msgr2=0x7f8a9c1f8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:14.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.075+0000 7f8a9be00640 1 --2- 172.21.15.167:0/4024604000 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a9c15ff10 0x7f8a9c1f8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.075+0000 7f8a9be00640 1 -- 172.21.15.167:0/4024604000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a9c0b5c20 msgr2=0x7f8a9c15c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:14.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.075+0000 7f8a9b400640 1 --2- 172.21.15.167:0/4024604000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a9c0b5c20 0x7f8a9c15c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:14.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.075+0000 7f8a9be00640 1 --2- 172.21.15.167:0/4024604000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a9c0b5c20 0x7f8a9c15c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.075+0000 7f8a9be00640 1 -- 172.21.15.167:0/4024604000 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a9c15c620 con 0x7f8a9c0b01b0 2024-05-11T12:11:14.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.076+0000 7f8a9be00640 1 --2- 172.21.15.167:0/4024604000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a9c0b01b0 0x7f8a9c0c5cc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8a8402a130 tx=0x7f8a840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:14.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.076+0000 7f8a93e00640 1 -- 172.21.15.167:0/4024604000 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a84041070 con 0x7f8a9c0b01b0 2024-05-11T12:11:14.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.076+0000 7f8a93e00640 1 -- 172.21.15.167:0/4024604000 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a8403c670 con 0x7f8a9c0b01b0 2024-05-11T12:11:14.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.076+0000 7f8a93e00640 1 -- 172.21.15.167:0/4024604000 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a84037040 con 0x7f8a9c0b01b0 2024-05-11T12:11:14.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.076+0000 7f8aa1a00640 1 -- 172.21.15.167:0/4024604000 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a9c0c6a60 con 0x7f8a9c0b01b0 2024-05-11T12:11:14.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.076+0000 7f8aa1a00640 1 -- 172.21.15.167:0/4024604000 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a9c15b8e0 con 0x7f8a9c0b01b0 2024-05-11T12:11:14.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.077+0000 7f8aa1a00640 1 -- 172.21.15.167:0/4024604000 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a9c0694b0 con 0x7f8a9c0b01b0 2024-05-11T12:11:14.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.079+0000 7f8a93e00640 1 -- 172.21.15.167:0/4024604000 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8a84044070 con 0x7f8a9c0b01b0 2024-05-11T12:11:14.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.080+0000 7f8ec3e00640 1 -- 172.21.15.167:0/49760027 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8ebc01f020 con 0x7f8ecc1e8050 2024-05-11T12:11:14.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.080+0000 7f8a93e00640 1 --2- 172.21.15.167:0/4024604000 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f8a780762d0 0x7f8a78078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.080+0000 7f8a93e00640 1 -- 172.21.15.167:0/4024604000 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f8a840b7fa0 con 0x7f8a9c0b01b0 2024-05-11T12:11:14.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.080+0000 7f8a9b400640 1 --2- 172.21.15.167:0/4024604000 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f8a780762d0 0x7f8a78078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:14.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.081+0000 7f8a9b400640 1 --2- 172.21.15.167:0/4024604000 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f8a780762d0 0x7f8a78078790 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c159ab0 tx=0x7f8a8c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:14.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.084+0000 7f1da2a00640 1 -- 172.21.15.167:0/2680442783 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1da4081c60 con 0x7f1dac15b240 2024-05-11T12:11:14.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.085+0000 7fa05cc00640 1 -- 172.21.15.167:0/1254697110 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa058011e10 con 0x7fa0600ce430 2024-05-11T12:11:14.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.085+0000 7fa067a00640 1 -- 172.21.15.167:0/1254697110 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0600cec30 con 0x7fa0600ce430 2024-05-11T12:11:14.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.085+0000 7fa067a00640 1 -- 172.21.15.167:0/1254697110 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0600cf1f0 con 0x7fa0600ce430 2024-05-11T12:11:14.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.085+0000 7fa05cc00640 1 -- 172.21.15.167:0/1254697110 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa05801b070 con 0x7fa0600ce430 2024-05-11T12:11:14.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.085+0000 7fa05cc00640 1 -- 172.21.15.167:0/1254697110 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa058003990 con 0x7fa0600ce430 2024-05-11T12:11:14.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.087+0000 7fa067a00640 1 -- 172.21.15.167:0/1254697110 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa03c005290 con 0x7fa0600ce430 2024-05-11T12:11:14.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.087+0000 7f5ca8c00640 1 -- 172.21.15.167:0/147929241 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5c9801e070 con 0x7f5cac0cfb50 2024-05-11T12:11:14.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.086+0000 7f8a93e00640 1 -- 172.21.15.167:0/4024604000 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8a840815b0 con 0x7f8a9c0b01b0 2024-05-11T12:11:14.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.088+0000 7f5ca8c00640 1 --2- 172.21.15.167:0/147929241 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5c8c0762d0 0x7f5c8c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.088+0000 7f5ca8c00640 1 -- 172.21.15.167:0/147929241 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f5c98097cf0 con 0x7f5cac0cfb50 2024-05-11T12:11:14.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.088+0000 7f93b2000640 1 -- 172.21.15.167:0/3918971926 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f93ac044020 con 0x7f93b80b5e50 2024-05-11T12:11:14.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.089+0000 7f5cb1200640 1 --2- 172.21.15.167:0/147929241 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5c8c0762d0 0x7f5c8c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:14.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.089+0000 7f93b2000640 1 --2- 172.21.15.167:0/3918971926 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f939c0763a0 0x7f939c078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.089+0000 7f93b2000640 1 -- 172.21.15.167:0/3918971926 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f93ac0b85b0 con 0x7f93b80b5e50 2024-05-11T12:11:14.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.090+0000 7f93bde00640 1 --2- 172.21.15.167:0/3918971926 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f939c0763a0 0x7f939c078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:14.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.091+0000 7f93bde00640 1 --2- 172.21.15.167:0/3918971926 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f939c0763a0 0x7f939c078860 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f93a80099a0 tx=0x7f93a8006f90 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:14.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.092+0000 7f5cb1200640 1 --2- 172.21.15.167:0/147929241 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5c8c0762d0 0x7f5c8c078790 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c009ee0 tx=0x7f5c9c009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:14.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.098+0000 7f5ca8c00640 1 -- 172.21.15.167:0/147929241 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5c98025070 con 0x7f5cac0cfb50 2024-05-11T12:11:14.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.107+0000 7f93b2000640 1 -- 172.21.15.167:0/3918971926 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f93ac04b020 con 0x7f93b80b5e50 2024-05-11T12:11:14.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.110+0000 7fa05cc00640 1 -- 172.21.15.167:0/1254697110 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa058005d10 con 0x7fa0600ce430 2024-05-11T12:11:14.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.112+0000 7fa05cc00640 1 --2- 172.21.15.167:0/1254697110 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa0400762d0 0x7fa040078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:14.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.112+0000 7fa05cc00640 1 -- 172.21.15.167:0/1254697110 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fa058098a40 con 0x7fa0600ce430 2024-05-11T12:11:14.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.113+0000 7fa065000640 1 --2- 172.21.15.167:0/1254697110 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa0400762d0 0x7fa040078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:14.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.114+0000 7fa065000640 1 --2- 172.21.15.167:0/1254697110 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa0400762d0 0x7fa040078790 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa054033000 tx=0x7fa054008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:14.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.116+0000 7fa05cc00640 1 -- 172.21.15.167:0/1254697110 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa0600cb2d0 con 0x7fa0600ce430 2024-05-11T12:11:14.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.482+0000 7f2e64c00640 1 -- 172.21.15.167:0/193936246 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f2e600d3880 con 0x7f2e600d2db0 2024-05-11T12:11:14.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.483+0000 7f2e56000640 1 -- 172.21.15.167:0/193936246 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2e50060260 con 0x7f2e600d2db0 2024-05-11T12:11:14.487 INFO:teuthology.orchestra.run.smithi167.stdout:60129542155 2024-05-11T12:11:14.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.487+0000 7f2e64c00640 1 -- 172.21.15.167:0/193936246 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2e400763d0 msgr2=0x7f2e40078890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:14.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.487+0000 7f2e64c00640 1 --2- 172.21.15.167:0/193936246 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2e400763d0 0x7f2e40078890 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c034000 tx=0x7f2e4c02e000 comp rx=0 tx=0).stop 2024-05-11T12:11:14.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.487+0000 7f2e64c00640 1 -- 172.21.15.167:0/193936246 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e600d2db0 msgr2=0x7f2e601fc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:14.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.487+0000 7f2e64c00640 1 --2- 172.21.15.167:0/193936246 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e600d2db0 0x7f2e601fc740 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2e5000c3a0 tx=0x7f2e5000c860 comp rx=0 tx=0).stop 2024-05-11T12:11:14.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.488+0000 7f2e64c00640 1 -- 172.21.15.167:0/193936246 shutdown_connections 2024-05-11T12:11:14.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.488+0000 7f2e64c00640 1 --2- 172.21.15.167:0/193936246 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2e400763d0 0x7f2e40078890 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.488+0000 7f2e64c00640 1 --2- 172.21.15.167:0/193936246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e60159970 0x7f2e6014af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.488+0000 7f2e64c00640 1 --2- 172.21.15.167:0/193936246 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e600d2db0 0x7f2e601fc740 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.489+0000 7f2e64c00640 1 --2- 172.21.15.167:0/193936246 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e600d17b0 0x7f2e600caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.489+0000 7f2e64c00640 1 -- 172.21.15.167:0/193936246 >> 172.21.15.167:0/193936246 conn(0x7f2e600d51e0 msgr2=0x7f2e60161ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:14.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.489+0000 7f2e64c00640 1 -- 172.21.15.167:0/193936246 shutdown_connections 2024-05-11T12:11:14.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.489+0000 7f2e64c00640 1 -- 172.21.15.167:0/193936246 wait complete. 2024-05-11T12:11:14.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.509+0000 7f3409400640 1 -- 172.21.15.167:0/3335545124 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f34040d4c10 con 0x7f34040d3c70 2024-05-11T12:11:14.510 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.510+0000 7f33faa00640 1 -- 172.21.15.167:0/3335545124 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f33f0064540 con 0x7f34040d3c70 2024-05-11T12:11:14.513 INFO:teuthology.orchestra.run.smithi167.stdout:64424509452 2024-05-11T12:11:14.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.514+0000 7f3409400640 1 -- 172.21.15.167:0/3335545124 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f33e40763a0 msgr2=0x7f33e4078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:14.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.514+0000 7f3409400640 1 --2- 172.21.15.167:0/3335545124 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f33e40763a0 0x7f33e4078860 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f33f4000f60 tx=0x7f33f4009040 comp rx=0 tx=0).stop 2024-05-11T12:11:14.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.514+0000 7f3409400640 1 -- 172.21.15.167:0/3335545124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34040d3c70 msgr2=0x7f34040ce330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:14.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.514+0000 7f3409400640 1 --2- 172.21.15.167:0/3335545124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34040d3c70 0x7f34040ce330 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f33f000b4a0 tx=0x7f33f000b960 comp rx=0 tx=0).stop 2024-05-11T12:11:14.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.514+0000 7f3409400640 1 -- 172.21.15.167:0/3335545124 shutdown_connections 2024-05-11T12:11:14.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.514+0000 7f3409400640 1 --2- 172.21.15.167:0/3335545124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f340415f330 0x7f34040ce870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.514+0000 7f3409400640 1 --2- 172.21.15.167:0/3335545124 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f33e40763a0 0x7f33e4078860 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.514+0000 7f3409400640 1 --2- 172.21.15.167:0/3335545124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34040d3c70 0x7f34040ce330 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.514+0000 7f3409400640 1 --2- 172.21.15.167:0/3335545124 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34040b82c0 0x7f34041612c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.514+0000 7f3409400640 1 -- 172.21.15.167:0/3335545124 >> 172.21.15.167:0/3335545124 conn(0x7f34040b6b80 msgr2=0x7f34040d0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:14.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.514+0000 7f3409400640 1 -- 172.21.15.167:0/3335545124 shutdown_connections 2024-05-11T12:11:14.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.514+0000 7f3409400640 1 -- 172.21.15.167:0/3335545124 wait complete. 2024-05-11T12:11:14.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.515+0000 7f8ed2a00640 1 -- 172.21.15.167:0/49760027 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f8ecc162380 con 0x7f8ecc1e8050 2024-05-11T12:11:14.516 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542156 got 60129542155 for osd.2 2024-05-11T12:11:14.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.516+0000 7f8ec3e00640 1 -- 172.21.15.167:0/49760027 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8ebc060c40 con 0x7f8ecc1e8050 2024-05-11T12:11:14.517 INFO:teuthology.orchestra.run.smithi167.stdout:64424509451 2024-05-11T12:11:14.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.520+0000 7f8ed2a00640 1 -- 172.21.15.167:0/49760027 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f8eac076320 msgr2=0x7f8eac0787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:14.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.520+0000 7f8ed2a00640 1 --2- 172.21.15.167:0/49760027 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f8eac076320 0x7f8eac0787e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4033000 tx=0x7f8ec4009000 comp rx=0 tx=0).stop 2024-05-11T12:11:14.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.520+0000 7f8ed2a00640 1 -- 172.21.15.167:0/49760027 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8ecc1e8050 msgr2=0x7f8ecc0ce970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:14.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.520+0000 7f8ed2a00640 1 --2- 172.21.15.167:0/49760027 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8ecc1e8050 0x7f8ecc0ce970 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc00b440 tx=0x7f8ebc00b900 comp rx=0 tx=0).stop 2024-05-11T12:11:14.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.520+0000 7f8ed2a00640 1 -- 172.21.15.167:0/49760027 shutdown_connections 2024-05-11T12:11:14.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.521+0000 7f8ed2a00640 1 --2- 172.21.15.167:0/49760027 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8ecc1e8050 0x7f8ecc0ce970 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.521+0000 7f8ed2a00640 1 --2- 172.21.15.167:0/49760027 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ecc1e9590 0x7f8ecc1e7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.521+0000 7f8ed2a00640 1 --2- 172.21.15.167:0/49760027 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f8eac076320 0x7f8eac0787e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.521+0000 7f8ed2a00640 1 --2- 172.21.15.167:0/49760027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ecc0d3c70 0x7f8ecc15db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.521+0000 7f8ed2a00640 1 -- 172.21.15.167:0/49760027 >> 172.21.15.167:0/49760027 conn(0x7f8ecc0b6b80 msgr2=0x7f8ecc0ba080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:14.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.521+0000 7f8ed2a00640 1 -- 172.21.15.167:0/49760027 shutdown_connections 2024-05-11T12:11:14.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.521+0000 7f8ed2a00640 1 -- 172.21.15.167:0/49760027 wait complete. 2024-05-11T12:11:14.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.532+0000 7f1dabe00640 1 -- 172.21.15.167:0/2680442783 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f1d84005850 con 0x7f1dac15b240 2024-05-11T12:11:14.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.533+0000 7f93c0800640 1 -- 172.21.15.167:0/3918971926 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f9394005850 con 0x7f93b80b5e50 2024-05-11T12:11:14.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.533+0000 7f1da2a00640 1 -- 172.21.15.167:0/2680442783 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1da4085f60 con 0x7f1dac15b240 2024-05-11T12:11:14.534 INFO:teuthology.orchestra.run.smithi167.stdout:64424509451 2024-05-11T12:11:14.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.535+0000 7f93b2000640 1 -- 172.21.15.167:0/3918971926 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f93ac081c40 con 0x7f93b80b5e50 2024-05-11T12:11:14.535 INFO:teuthology.orchestra.run.smithi167.stdout:68719476746 2024-05-11T12:11:14.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.537+0000 7f1dabe00640 1 -- 172.21.15.167:0/2680442783 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f1d8c0762d0 msgr2=0x7f1d8c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:14.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.537+0000 7f1dabe00640 1 --2- 172.21.15.167:0/2680442783 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f1d8c0762d0 0x7f1d8c078790 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c007fd0 tx=0x7f1d9c007d10 comp rx=0 tx=0).stop 2024-05-11T12:11:14.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.537+0000 7f1dabe00640 1 -- 172.21.15.167:0/2680442783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dac15b240 msgr2=0x7f1dac14fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:14.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.537+0000 7f1dabe00640 1 --2- 172.21.15.167:0/2680442783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dac15b240 0x7f1dac14fac0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1da400b370 tx=0x7f1da4004a40 comp rx=0 tx=0).stop 2024-05-11T12:11:14.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.538+0000 7f1dabe00640 1 -- 172.21.15.167:0/2680442783 shutdown_connections 2024-05-11T12:11:14.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.538+0000 7f1dabe00640 1 --2- 172.21.15.167:0/2680442783 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f1d8c0762d0 0x7f1d8c078790 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.538+0000 7f1dabe00640 1 --2- 172.21.15.167:0/2680442783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dac15b240 0x7f1dac14fac0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.538+0000 7f1dabe00640 1 --2- 172.21.15.167:0/2680442783 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1dac15ae60 0x7f1dac14dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.538+0000 7f1dabe00640 1 --2- 172.21.15.167:0/2680442783 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dac073b10 0x7f1dac1ea290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.538+0000 7f1dabe00640 1 -- 172.21.15.167:0/2680442783 >> 172.21.15.167:0/2680442783 conn(0x7f1dac06c450 msgr2=0x7f1dac0b4cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:14.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.538+0000 7f1dabe00640 1 -- 172.21.15.167:0/2680442783 shutdown_connections 2024-05-11T12:11:14.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.538+0000 7f1dabe00640 1 -- 172.21.15.167:0/2680442783 wait complete. 2024-05-11T12:11:14.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.540+0000 7f93c0800640 1 -- 172.21.15.167:0/3918971926 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f939c0763a0 msgr2=0x7f939c078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:14.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.540+0000 7f93c0800640 1 --2- 172.21.15.167:0/3918971926 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f939c0763a0 0x7f939c078860 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f93a80099a0 tx=0x7f93a8006f90 comp rx=0 tx=0).stop 2024-05-11T12:11:14.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.540+0000 7f93c0800640 1 -- 172.21.15.167:0/3918971926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93b80b5e50 msgr2=0x7f93b815f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:14.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.540+0000 7f93c0800640 1 --2- 172.21.15.167:0/3918971926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93b80b5e50 0x7f93b815f2b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f93ac031000 tx=0x7f93ac0046c0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.540+0000 7f93c0800640 1 -- 172.21.15.167:0/3918971926 shutdown_connections 2024-05-11T12:11:14.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.540+0000 7f93c0800640 1 --2- 172.21.15.167:0/3918971926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93b81636b0 0x7f93b815f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.540+0000 7f93c0800640 1 --2- 172.21.15.167:0/3918971926 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f939c0763a0 0x7f939c078860 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.540+0000 7f93c0800640 1 --2- 172.21.15.167:0/3918971926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93b80b5e50 0x7f93b815f2b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.540+0000 7f93c0800640 1 --2- 172.21.15.167:0/3918971926 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93b80b4ae0 0x7f93b815b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.540+0000 7f93c0800640 1 -- 172.21.15.167:0/3918971926 >> 172.21.15.167:0/3918971926 conn(0x7f93b80b8370 msgr2=0x7f93b80b62e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:14.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.541+0000 7f8aa1a00640 1 -- 172.21.15.167:0/4024604000 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f8a9c148990 con 0x7f8a9c0b01b0 2024-05-11T12:11:14.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.541+0000 7f93c0800640 1 -- 172.21.15.167:0/3918971926 shutdown_connections 2024-05-11T12:11:14.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.541+0000 7f93c0800640 1 -- 172.21.15.167:0/3918971926 wait complete. 2024-05-11T12:11:14.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.541+0000 7f8a93e00640 1 -- 172.21.15.167:0/4024604000 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8a84080eb0 con 0x7f8a9c0b01b0 2024-05-11T12:11:14.542 INFO:teuthology.orchestra.run.smithi167.stdout:68719476745 2024-05-11T12:11:14.543 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509453 got 64424509452 for osd.1 2024-05-11T12:11:14.545 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509452 got 64424509451 for osd.3 2024-05-11T12:11:14.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.545+0000 7f8aa1a00640 1 -- 172.21.15.167:0/4024604000 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f8a780762d0 msgr2=0x7f8a78078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:14.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.545+0000 7f8aa1a00640 1 --2- 172.21.15.167:0/4024604000 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f8a780762d0 0x7f8a78078790 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c159ab0 tx=0x7f8a8c008040 comp rx=0 tx=0).stop 2024-05-11T12:11:14.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.545+0000 7f8aa1a00640 1 -- 172.21.15.167:0/4024604000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a9c0b01b0 msgr2=0x7f8a9c0c5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:14.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.545+0000 7f8aa1a00640 1 --2- 172.21.15.167:0/4024604000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a9c0b01b0 0x7f8a9c0c5cc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8a8402a130 tx=0x7f8a840046c0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.545+0000 7f8aa1a00640 1 -- 172.21.15.167:0/4024604000 shutdown_connections 2024-05-11T12:11:14.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.545+0000 7f8aa1a00640 1 --2- 172.21.15.167:0/4024604000 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a9c15ff10 0x7f8a9c1f8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.545+0000 7f8aa1a00640 1 --2- 172.21.15.167:0/4024604000 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f8a780762d0 0x7f8a78078790 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.545+0000 7f8aa1a00640 1 --2- 172.21.15.167:0/4024604000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a9c0b5c20 0x7f8a9c15c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.545+0000 7f8aa1a00640 1 --2- 172.21.15.167:0/4024604000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a9c0b01b0 0x7f8a9c0c5cc0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.545+0000 7f8aa1a00640 1 -- 172.21.15.167:0/4024604000 >> 172.21.15.167:0/4024604000 conn(0x7f8a9c06c450 msgr2=0x7f8a9c160990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:14.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.547+0000 7f8aa1a00640 1 -- 172.21.15.167:0/4024604000 shutdown_connections 2024-05-11T12:11:14.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.547+0000 7f8aa1a00640 1 -- 172.21.15.167:0/4024604000 wait complete. 2024-05-11T12:11:14.548 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.547+0000 7f5cb3c00640 1 -- 172.21.15.167:0/147929241 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f5cac06f860 con 0x7f5cac0cfb50 2024-05-11T12:11:14.548 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.548+0000 7f5ca8c00640 1 -- 172.21.15.167:0/147929241 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5c980693c0 con 0x7f5cac0cfb50 2024-05-11T12:11:14.548 INFO:teuthology.orchestra.run.smithi167.stdout:68719476747 2024-05-11T12:11:14.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.551+0000 7f5cb3c00640 1 -- 172.21.15.167:0/147929241 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5c8c0762d0 msgr2=0x7f5c8c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:14.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.551+0000 7f5cb3c00640 1 --2- 172.21.15.167:0/147929241 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5c8c0762d0 0x7f5c8c078790 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c009ee0 tx=0x7f5c9c009190 comp rx=0 tx=0).stop 2024-05-11T12:11:14.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.552+0000 7f5cb3c00640 1 -- 172.21.15.167:0/147929241 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cac0cfb50 msgr2=0x7f5cac0aa840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:14.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.552+0000 7f5cb3c00640 1 --2- 172.21.15.167:0/147929241 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cac0cfb50 0x7f5cac0aa840 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5c9800b950 tx=0x7f5c9800be10 comp rx=0 tx=0).stop 2024-05-11T12:11:14.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.552+0000 7f5cb3c00640 1 -- 172.21.15.167:0/147929241 shutdown_connections 2024-05-11T12:11:14.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.552+0000 7f5cb3c00640 1 --2- 172.21.15.167:0/147929241 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cac15b410 0x7f5cac154220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.552+0000 7f5cb3c00640 1 --2- 172.21.15.167:0/147929241 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5c8c0762d0 0x7f5c8c078790 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.552+0000 7f5cb3c00640 1 --2- 172.21.15.167:0/147929241 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cac0cfb50 0x7f5cac0aa840 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.552+0000 7f5cb3c00640 1 --2- 172.21.15.167:0/147929241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cac0b6550 0x7f5cac152ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.552+0000 7f5cb3c00640 1 -- 172.21.15.167:0/147929241 >> 172.21.15.167:0/147929241 conn(0x7f5cac06c450 msgr2=0x7f5cac0b0780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:14.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.552+0000 7f5cb3c00640 1 -- 172.21.15.167:0/147929241 shutdown_connections 2024-05-11T12:11:14.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.552+0000 7f5cb3c00640 1 -- 172.21.15.167:0/147929241 wait complete. 2024-05-11T12:11:14.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.565+0000 7fa067a00640 1 -- 172.21.15.167:0/1254697110 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fa03c005850 con 0x7fa0600ce430 2024-05-11T12:11:14.566 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509452 got 64424509451 for osd.0 2024-05-11T12:11:14.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.566+0000 7fa05cc00640 1 -- 172.21.15.167:0/1254697110 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa0580620d0 con 0x7fa0600ce430 2024-05-11T12:11:14.567 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476748 got 68719476746 for osd.6 2024-05-11T12:11:14.567 INFO:teuthology.orchestra.run.smithi167.stdout:68719476745 2024-05-11T12:11:14.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.570+0000 7fa067a00640 1 -- 172.21.15.167:0/1254697110 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa0400762d0 msgr2=0x7fa040078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:14.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.570+0000 7fa067a00640 1 --2- 172.21.15.167:0/1254697110 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa0400762d0 0x7fa040078790 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa054033000 tx=0x7fa054008030 comp rx=0 tx=0).stop 2024-05-11T12:11:14.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.570+0000 7fa067a00640 1 -- 172.21.15.167:0/1254697110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa0600ce430 msgr2=0x7fa0601572d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:14.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.570+0000 7fa067a00640 1 --2- 172.21.15.167:0/1254697110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa0600ce430 0x7fa0601572d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa048004750 tx=0x7fa0580115a0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.570+0000 7fa067a00640 1 -- 172.21.15.167:0/1254697110 shutdown_connections 2024-05-11T12:11:14.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.570+0000 7fa067a00640 1 --2- 172.21.15.167:0/1254697110 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa0400762d0 0x7fa040078790 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.570+0000 7fa067a00640 1 --2- 172.21.15.167:0/1254697110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa0600ce430 0x7fa0601572d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.570+0000 7fa067a00640 1 --2- 172.21.15.167:0/1254697110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0600cfcd0 0x7fa0600cad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.570+0000 7fa067a00640 1 --2- 172.21.15.167:0/1254697110 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa0480039b0 0x7fa0600ccb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:14.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.570+0000 7fa067a00640 1 -- 172.21.15.167:0/1254697110 >> 172.21.15.167:0/1254697110 conn(0x7fa0600b6b80 msgr2=0x7fa060161510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:14.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.571+0000 7fa067a00640 1 -- 172.21.15.167:0/1254697110 shutdown_connections 2024-05-11T12:11:14.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:14.571+0000 7fa067a00640 1 -- 172.21.15.167:0/1254697110 wait complete. 2024-05-11T12:11:14.572 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476749 got 68719476747 for osd.4 2024-05-11T12:11:14.573 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476747 got 68719476745 for osd.7 2024-05-11T12:11:14.589 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476747 got 68719476745 for osd.5 2024-05-11T12:11:15.518 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-11T12:11:15.544 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-11T12:11:15.546 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-11T12:11:15.566 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-11T12:11:15.567 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-11T12:11:15.573 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-11T12:11:15.575 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-11T12:11:15.590 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-11T12:11:15.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.786+0000 7fa3fbc00640 1 Processor -- start 2024-05-11T12:11:15.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.786+0000 7fa3fbc00640 1 -- start start 2024-05-11T12:11:15.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.788+0000 7fa3fbc00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa3ec0e7a40 0x7fa3ec049b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.788+0000 7fa3fbc00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3ec0e7e20 con 0x7fa3ec04a090 2024-05-11T12:11:15.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.788+0000 7fa3fbc00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3ec03b590 con 0x7fa3ec03c350 2024-05-11T12:11:15.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.788+0000 7fa3fbc00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fa3ec03b870 con 0x7fa3ec0e7a40 2024-05-11T12:11:15.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.790+0000 7fa3f3e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fa3ec04a090 0x7fa3ec054890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49476/0 (socket says 172.21.15.167:49476) 2024-05-11T12:11:15.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.790+0000 7fa3f9c00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fa3ec03c350 0x7fa3ec051310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.167:55270/0 (socket says 172.21.15.167:55270) 2024-05-11T12:11:15.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.790+0000 7fa3f9c00640 1 -- 172.21.15.167:0/597142406 learned_addr learned my addr 172.21.15.167:0/597142406 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:15.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.791+0000 7fa3f9200640 1 --2- 172.21.15.167:0/597142406 >> v2:172.21.15.167:3300/0 conn(0x7fa3ec0e7a40 0x7fa3ec049b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:15.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.791+0000 7fa3f3400640 1 -- 172.21.15.167:0/597142406 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1768280258 0 0) 0x7fa3ec03b590 con 0x7fa3ec03c350 2024-05-11T12:11:15.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.791+0000 7fa3f3400640 1 -- 172.21.15.167:0/597142406 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa3dc003520 con 0x7fa3ec03c350 2024-05-11T12:11:15.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.793+0000 7fa3f9200640 1 -- 172.21.15.167:0/597142406 >> v1:172.21.15.204:6789/0 conn(0x7fa3ec03c350 legacy=0x7fa3ec051310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:15.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.793+0000 7fa3f9200640 1 -- 172.21.15.167:0/597142406 >> v1:172.21.15.167:6789/0 conn(0x7fa3ec04a090 legacy=0x7fa3ec054890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:15.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.793+0000 7fa3f9200640 1 -- 172.21.15.167:0/597142406 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3ec03bb50 con 0x7fa3ec0e7a40 2024-05-11T12:11:15.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.795+0000 7fa3f9200640 1 --2- 172.21.15.167:0/597142406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3ec0e7a40 0x7fa3ec049b50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8009720 tx=0x7fa3e8029fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41989ca80e488a72 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:15.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.796+0000 7fa3f3400640 1 -- 172.21.15.167:0/597142406 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8037070 con 0x7fa3ec0e7a40 2024-05-11T12:11:15.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.796+0000 7fa3f3400640 1 -- 172.21.15.167:0/597142406 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3e802aa70 con 0x7fa3ec0e7a40 2024-05-11T12:11:15.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.796+0000 7fa3f3400640 1 -- 172.21.15.167:0/597142406 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e00016f0 con 0x7fa3ec0e7a40 2024-05-11T12:11:15.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.797+0000 7fa3fbc00640 1 -- 172.21.15.167:0/597142406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3ec0e7a40 msgr2=0x7fa3ec049b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:15.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.797+0000 7fa3fbc00640 1 --2- 172.21.15.167:0/597142406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3ec0e7a40 0x7fa3ec049b50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8009720 tx=0x7fa3e8029fe0 comp rx=0 tx=0).stop 2024-05-11T12:11:15.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.798+0000 7fa3fbc00640 1 -- 172.21.15.167:0/597142406 shutdown_connections 2024-05-11T12:11:15.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.798+0000 7fa3fbc00640 1 --2- 172.21.15.167:0/597142406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3ec0e7a40 0x7fa3ec049b50 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:15.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.798+0000 7fa3fbc00640 1 -- 172.21.15.167:0/597142406 >> 172.21.15.167:0/597142406 conn(0x7fa3ec056cc0 msgr2=0x7fa3ec0570d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:15.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.799+0000 7fa3fbc00640 1 -- 172.21.15.167:0/597142406 shutdown_connections 2024-05-11T12:11:15.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.799+0000 7fa3fbc00640 1 -- 172.21.15.167:0/597142406 wait complete. 2024-05-11T12:11:15.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.801+0000 7fa3fbc00640 1 Processor -- start 2024-05-11T12:11:15.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.802+0000 7fa3fbc00640 1 -- start start 2024-05-11T12:11:15.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.803+0000 7fa3fbc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3ec03c350 0x7fa3ec17de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.803+0000 7fa3fbc00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3ec04a090 0x7fa3ec0da120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.803+0000 7fa3fbc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa3ec0e7a40 0x7fa3ec0558f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.805+0000 7fa3f9200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3ec03c350 0x7fa3ec17de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:15.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.805+0000 7fa3fbc00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ec054fa0 con 0x7fa3ec04a090 2024-05-11T12:11:15.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.805+0000 7fa3f9200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3ec03c350 0x7fa3ec17de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.167:47278/0 (socket says 172.21.15.167:47278) 2024-05-11T12:11:15.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.805+0000 7fa3f9200640 1 -- 172.21.15.167:0/3373131380 learned_addr learned my addr 172.21.15.167:0/3373131380 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:15.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.805+0000 7fa3fbc00640 1 -- 172.21.15.167:0/3373131380 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ec04a5a0 con 0x7fa3ec03c350 2024-05-11T12:11:15.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.805+0000 7fa3fbc00640 1 -- 172.21.15.167:0/3373131380 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa3ec17e360 con 0x7fa3ec0e7a40 2024-05-11T12:11:15.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.805+0000 7fa3f9c00640 1 --2- 172.21.15.167:0/3373131380 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa3ec0e7a40 0x7fa3ec0558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:15.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.806+0000 7fa3f3e00640 1 --2- 172.21.15.167:0/3373131380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3ec04a090 0x7fa3ec0da120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:15.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.806+0000 7fa3f9c00640 1 -- 172.21.15.167:0/3373131380 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3ec03c350 msgr2=0x7fa3ec17de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:15.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.806+0000 7fa3f9c00640 1 --2- 172.21.15.167:0/3373131380 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3ec03c350 0x7fa3ec17de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:15.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.806+0000 7fa3f9c00640 1 -- 172.21.15.167:0/3373131380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3ec04a090 msgr2=0x7fa3ec0da120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:15.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.807+0000 7fa3f9c00640 1 --2- 172.21.15.167:0/3373131380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3ec04a090 0x7fa3ec0da120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:15.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.807+0000 7fa3f9c00640 1 -- 172.21.15.167:0/3373131380 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3ec04df10 con 0x7fa3ec0e7a40 2024-05-11T12:11:15.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.807+0000 7fa3f3e00640 1 --2- 172.21.15.167:0/3373131380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3ec04a090 0x7fa3ec0da120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:15.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.808+0000 7fa3f9c00640 1 --2- 172.21.15.167:0/3373131380 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa3ec0e7a40 0x7fa3ec0558f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa3e000c6b0 tx=0x7fa3e000cb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:15.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.809+0000 7fa3f0c00640 1 -- 172.21.15.167:0/3373131380 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0018070 con 0x7fa3ec0e7a40 2024-05-11T12:11:15.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.809+0000 7fa3fbc00640 1 -- 172.21.15.167:0/3373131380 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3ec04e1d0 con 0x7fa3ec0e7a40 2024-05-11T12:11:15.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.809+0000 7fa3f0c00640 1 -- 172.21.15.167:0/3373131380 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3e00072d0 con 0x7fa3ec0e7a40 2024-05-11T12:11:15.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.810+0000 7fa3fbc00640 1 -- 172.21.15.167:0/3373131380 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3ec04e790 con 0x7fa3ec0e7a40 2024-05-11T12:11:15.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.810+0000 7fa3f0c00640 1 -- 172.21.15.167:0/3373131380 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e00047e0 con 0x7fa3ec0e7a40 2024-05-11T12:11:15.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.813+0000 7fa3fbc00640 1 -- 172.21.15.167:0/3373131380 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3ec0057d0 con 0x7fa3ec0e7a40 2024-05-11T12:11:15.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.818+0000 7fa3f0c00640 1 -- 172.21.15.167:0/3373131380 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa3e0004980 con 0x7fa3ec0e7a40 2024-05-11T12:11:15.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.820+0000 7fa3f0c00640 1 --2- 172.21.15.167:0/3373131380 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa3d4076300 0x7fa3d40787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.821+0000 7fa3f9200640 1 --2- 172.21.15.167:0/3373131380 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa3d4076300 0x7fa3d40787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:15.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.822+0000 7fa3f9200640 1 --2- 172.21.15.167:0/3373131380 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa3d4076300 0x7fa3d40787c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa3e802a500 tx=0x7fa3e802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:15.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.823+0000 7fa3f0c00640 1 -- 172.21.15.167:0/3373131380 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fa3e0097100 con 0x7fa3ec0e7a40 2024-05-11T12:11:15.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.830+0000 7f0e26800640 1 Processor -- start 2024-05-11T12:11:15.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.831+0000 7f0e26800640 1 -- start start 2024-05-11T12:11:15.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.832+0000 7fa3f0c00640 1 -- 172.21.15.167:0/3373131380 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa3e003a080 con 0x7fa3ec0e7a40 2024-05-11T12:11:15.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.833+0000 7f0e26800640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0e200d4380 0x7f0e200b8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.834+0000 7f0e26800640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e200d1840 con 0x7f0e200b82c0 2024-05-11T12:11:15.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.834+0000 7f0e26800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e2006c6a0 con 0x7f0e2015f330 2024-05-11T12:11:15.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.834+0000 7f0e26800640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f0e2006c450 con 0x7f0e200d4380 2024-05-11T12:11:15.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.834+0000 7f0e1f400640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f0e2015f330 0x7f0e200d1130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.167:55278/0 (socket says 172.21.15.167:55278) 2024-05-11T12:11:15.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.834+0000 7f0e1ea00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f0e200b82c0 0x7f0e200d3c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49486/0 (socket says 172.21.15.167:49486) 2024-05-11T12:11:15.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.834+0000 7f0e1ea00640 1 -- 172.21.15.167:0/2349501077 learned_addr learned my addr 172.21.15.167:0/2349501077 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:15.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.835+0000 7f0e1fe00640 1 --2- 172.21.15.167:0/2349501077 >> v2:172.21.15.167:3300/0 conn(0x7f0e200d4380 0x7f0e200b8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:15.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.836+0000 7f0e1e000640 1 -- 172.21.15.167:0/2349501077 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3282909451 0 0) 0x7f0e200d1840 con 0x7f0e200b82c0 2024-05-11T12:11:15.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.836+0000 7fbd86000640 1 Processor -- start 2024-05-11T12:11:15.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.837+0000 7fbd86000640 1 -- start start 2024-05-11T12:11:15.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.836+0000 7f0e1fe00640 1 -- 172.21.15.167:0/2349501077 >> v1:172.21.15.204:6789/0 conn(0x7f0e2015f330 legacy=0x7f0e200d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:15.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.836+0000 7f0e1fe00640 1 -- 172.21.15.167:0/2349501077 >> v1:172.21.15.167:6789/0 conn(0x7f0e200b82c0 legacy=0x7f0e200d3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:15.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.836+0000 7f0e1fe00640 1 -- 172.21.15.167:0/2349501077 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e2006c980 con 0x7f0e200d4380 2024-05-11T12:11:15.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.838+0000 7fae07200640 1 Processor -- start 2024-05-11T12:11:15.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.839+0000 7fbd86000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd8015f330 0x7fbd800d1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.839+0000 7fbd86000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fbd800d1670 0x7fbd800b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.839+0000 7fbd86000640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd8006c6a0 con 0x7fbd800b8800 2024-05-11T12:11:15.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.839+0000 7fbd86000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fbd8006c450 con 0x7fbd800d1670 2024-05-11T12:11:15.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.839+0000 7fbd86000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbd8006d2b0 con 0x7fbd8015f330 2024-05-11T12:11:15.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.839+0000 7f0e1fe00640 1 --2- 172.21.15.167:0/2349501077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e200d4380 0x7f0e200b8720 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0e18009d90 tx=0x7f0e1802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3f76e8ea31cad13 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:15.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.839+0000 7fae07200640 1 -- start start 2024-05-11T12:11:15.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.840+0000 7f0e1e000640 1 -- 172.21.15.167:0/2349501077 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e18007740 con 0x7f0e200d4380 2024-05-11T12:11:15.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.840+0000 7fbd7f400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd8015f330 0x7fbd800d1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:15.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.840+0000 7fbd7f400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd8015f330 0x7fbd800d1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:59876/0 (socket says 172.21.15.167:59876) 2024-05-11T12:11:15.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.840+0000 7fbd7f400640 1 -- 172.21.15.167:0/2775750503 learned_addr learned my addr 172.21.15.167:0/2775750503 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:15.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.840+0000 7fbd7ea00640 1 --2- 172.21.15.167:0/2775750503 >> v2:172.21.15.204:3300/0 conn(0x7fbd800d1670 0x7fbd800b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:15.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.840+0000 7f0e1e000640 1 -- 172.21.15.167:0/2349501077 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e180078e0 con 0x7f0e200d4380 2024-05-11T12:11:15.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.840+0000 7f0e1e000640 1 -- 172.21.15.167:0/2349501077 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e18007740 con 0x7f0e200d4380 2024-05-11T12:11:15.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.841+0000 7fbd7f400640 1 -- 172.21.15.167:0/2775750503 >> v2:172.21.15.204:3300/0 conn(0x7fbd800d1670 msgr2=0x7fbd800b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:15.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.841+0000 7fbd7f400640 1 --2- 172.21.15.167:0/2775750503 >> v2:172.21.15.204:3300/0 conn(0x7fbd800d1670 0x7fbd800b82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:15.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.841+0000 7fbd7f400640 1 -- 172.21.15.167:0/2775750503 >> v1:172.21.15.167:6789/0 conn(0x7fbd800b8800 legacy=0x7fbd800d3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:15.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.841+0000 7fbd7f400640 1 -- 172.21.15.167:0/2775750503 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd8006c980 con 0x7fbd8015f330 2024-05-11T12:11:15.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.842+0000 7f0e26800640 1 -- 172.21.15.167:0/2349501077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e200d4380 msgr2=0x7f0e200b8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:15.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.842+0000 7f0e26800640 1 --2- 172.21.15.167:0/2349501077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e200d4380 0x7f0e200b8720 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0e18009d90 tx=0x7f0e1802a6f0 comp rx=0 tx=0).stop 2024-05-11T12:11:15.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.842+0000 7f0e26800640 1 -- 172.21.15.167:0/2349501077 shutdown_connections 2024-05-11T12:11:15.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.843+0000 7f0e26800640 1 --2- 172.21.15.167:0/2349501077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e200d4380 0x7f0e200b8720 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:15.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.843+0000 7f0e26800640 1 -- 172.21.15.167:0/2349501077 >> 172.21.15.167:0/2349501077 conn(0x7f0e200b6b80 msgr2=0x7f0e200b6f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:15.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.843+0000 7fbd7f400640 1 --2- 172.21.15.167:0/2775750503 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd8015f330 0x7fbd800d1130 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbd70009870 tx=0x7fbd7002a160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfce909ca5b8f652 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:15.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.843+0000 7f0e26800640 1 -- 172.21.15.167:0/2349501077 shutdown_connections 2024-05-11T12:11:15.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.843+0000 7fae07200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fae000b82c0 0x7fae000b8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.843+0000 7f0e26800640 1 -- 172.21.15.167:0/2349501077 wait complete. 2024-05-11T12:11:15.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.843+0000 7fae07200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fae000d3c70 0x7fae000d4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.843+0000 7fae07200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae000d1840 con 0x7fae0015f330 2024-05-11T12:11:15.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.843+0000 7fae07200640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fae0006c450 con 0x7fae000d3c70 2024-05-11T12:11:15.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.844+0000 7fae07200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fae0006d2b0 con 0x7fae000b82c0 2024-05-11T12:11:15.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.844+0000 7fbd7e000640 1 -- 172.21.15.167:0/2775750503 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd70037070 con 0x7fbd8015f330 2024-05-11T12:11:15.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.844+0000 7fadffe00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fae0015f330 0x7fae000d1130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.167:55290/0 (socket says 172.21.15.167:55290) 2024-05-11T12:11:15.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.844+0000 7fadffe00640 1 -- 172.21.15.167:0/1726489215 learned_addr learned my addr 172.21.15.167:0/1726489215 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:15.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.844+0000 7fbd7e000640 1 -- 172.21.15.167:0/2775750503 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd7002adf0 con 0x7fbd8015f330 2024-05-11T12:11:15.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.844+0000 7fbd7e000640 1 -- 172.21.15.167:0/2775750503 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd70032660 con 0x7fbd8015f330 2024-05-11T12:11:15.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.845+0000 7fae05200640 1 --2- 172.21.15.167:0/1726489215 >> v2:172.21.15.167:3300/0 conn(0x7fae000d3c70 0x7fae000d4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:15.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.845+0000 7fadfea00640 1 -- 172.21.15.167:0/1726489215 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2071637014 0 0) 0x7fae000d1840 con 0x7fae0015f330 2024-05-11T12:11:15.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.845+0000 7fbd86000640 1 -- 172.21.15.167:0/2775750503 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd8015f330 msgr2=0x7fbd800d1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:15.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.845+0000 7fbd86000640 1 --2- 172.21.15.167:0/2775750503 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd8015f330 0x7fbd800d1130 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbd70009870 tx=0x7fbd7002a160 comp rx=0 tx=0).stop 2024-05-11T12:11:15.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.845+0000 7fadff400640 1 --2- 172.21.15.167:0/1726489215 >> v2:172.21.15.204:3300/0 conn(0x7fae000b82c0 0x7fae000b8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:15.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.846+0000 7fbd86000640 1 -- 172.21.15.167:0/2775750503 shutdown_connections 2024-05-11T12:11:15.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.846+0000 7fbd86000640 1 --2- 172.21.15.167:0/2775750503 >> v2:172.21.15.204:3300/0 conn(0x7fbd800d1670 0x7fbd800b82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:15.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.846+0000 7fbd86000640 1 --2- 172.21.15.167:0/2775750503 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd8015f330 0x7fbd800d1130 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:15.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.846+0000 7fbd86000640 1 -- 172.21.15.167:0/2775750503 >> 172.21.15.167:0/2775750503 conn(0x7fbd800b6b80 msgr2=0x7fbd800b6f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:15.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.846+0000 7faac7400640 1 Processor -- start 2024-05-11T12:11:15.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.845+0000 7fadfea00640 1 -- 172.21.15.167:0/1726489215 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fade4003520 con 0x7fae0015f330 2024-05-11T12:11:15.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.846+0000 7fbd86000640 1 -- 172.21.15.167:0/2775750503 shutdown_connections 2024-05-11T12:11:15.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.846+0000 7fbd86000640 1 -- 172.21.15.167:0/2775750503 wait complete. 2024-05-11T12:11:15.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.846+0000 7faac7400640 1 -- start start 2024-05-11T12:11:15.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.846+0000 7fadff400640 1 -- 172.21.15.167:0/1726489215 >> v2:172.21.15.167:3300/0 conn(0x7fae000d3c70 msgr2=0x7fae000d4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:15.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.846+0000 7fadff400640 1 --2- 172.21.15.167:0/1726489215 >> v2:172.21.15.167:3300/0 conn(0x7fae000d3c70 0x7fae000d4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:15.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.846+0000 7fadff400640 1 -- 172.21.15.167:0/1726489215 >> v1:172.21.15.204:6789/0 conn(0x7fae0015f330 legacy=0x7fae000d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:15.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.846+0000 7fadfea00640 1 -- 172.21.15.167:0/1726489215 <== mon.1 v1:172.21.15.204:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2613810340 0 0) 0x7fade4003520 con 0x7fae0015f330 2024-05-11T12:11:15.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.847+0000 7fadff400640 1 -- 172.21.15.167:0/1726489215 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae0006c6a0 con 0x7fae000b82c0 2024-05-11T12:11:15.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.848+0000 7f0e26800640 1 Processor -- start 2024-05-11T12:11:15.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.848+0000 7faac7400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7faac015f330 0x7faac00d1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.849+0000 7fbd86000640 1 Processor -- start 2024-05-11T12:11:15.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.848+0000 7faac7400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faac00d3c70 0x7faac00d4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.848+0000 7faac7400640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faac006c6a0 con 0x7faac00d1670 2024-05-11T12:11:15.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.848+0000 7faac7400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7faac006c450 con 0x7faac015f330 2024-05-11T12:11:15.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.848+0000 7faac7400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7faac006d2b0 con 0x7faac00d3c70 2024-05-11T12:11:15.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.848+0000 7f0e26800640 1 -- start start 2024-05-11T12:11:15.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.849+0000 7fadff400640 1 --2- 172.21.15.167:0/1726489215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae000b82c0 0x7fae000b8720 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fae0006eb00 tx=0x7fadec02a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3285abe1019b9d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:15.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.849+0000 7fbd86000640 1 -- start start 2024-05-11T12:11:15.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.849+0000 7fadfea00640 1 -- 172.21.15.167:0/1726489215 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadec037070 con 0x7fae000b82c0 2024-05-11T12:11:15.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.850+0000 7fadfea00640 1 -- 172.21.15.167:0/1726489215 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadec02abc0 con 0x7fae000b82c0 2024-05-11T12:11:15.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.849+0000 7faac4a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7faac015f330 0x7faac00d1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:15.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.850+0000 7fadfea00640 1 -- 172.21.15.167:0/1726489215 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadec037070 con 0x7fae000b82c0 2024-05-11T12:11:15.856 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.849+0000 7faabfe00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7faac00d1670 0x7faac00b82c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49502/0 (socket says 172.21.15.167:49502) 2024-05-11T12:11:15.856 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.849+0000 7faabfe00640 1 -- 172.21.15.167:0/616323555 learned_addr learned my addr 172.21.15.167:0/616323555 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:15.856 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.849+0000 7faac4a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7faac015f330 0x7faac00d1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.167:47322/0 (socket says 172.21.15.167:47322) 2024-05-11T12:11:15.856 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.850+0000 7faac5400640 1 --2- 172.21.15.167:0/616323555 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faac00d3c70 0x7faac00d4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:15.856 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.850+0000 7faabf400640 1 -- 172.21.15.167:0/616323555 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 959324586 0 0) 0x7faac006c6a0 con 0x7faac00d1670 2024-05-11T12:11:15.856 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.850+0000 7faabf400640 1 -- 172.21.15.167:0/616323555 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faaa4003520 con 0x7faac00d1670 2024-05-11T12:11:15.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.850+0000 7fbd86000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd800b8800 0x7fbd80151660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.850+0000 7fbd86000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd8015f330 0x7fbd800ced30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.850+0000 7fbd86000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd801f5c20 0x7fbd800b59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.851+0000 7fbd86000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8015f710 con 0x7fbd8015f330 2024-05-11T12:11:15.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.851+0000 7fbd86000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8006cf40 con 0x7fbd801f5c20 2024-05-11T12:11:15.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.851+0000 7fbd86000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbd800d06b0 con 0x7fbd800b8800 2024-05-11T12:11:15.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.851+0000 7faabf400640 1 -- 172.21.15.167:0/616323555 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2580694905 0 0) 0x7faaa4003520 con 0x7faac00d1670 2024-05-11T12:11:15.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.851+0000 7faabf400640 1 -- 172.21.15.167:0/616323555 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faac00d3c70 msgr2=0x7faac00d4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:15.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.851+0000 7faabf400640 1 --2- 172.21.15.167:0/616323555 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faac00d3c70 0x7faac00d4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:15.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.851+0000 7faabf400640 1 -- 172.21.15.167:0/616323555 >> v2:172.21.15.204:3300/0 conn(0x7faac015f330 msgr2=0x7faac00d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:15.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.851+0000 7faabf400640 1 --2- 172.21.15.167:0/616323555 >> v2:172.21.15.204:3300/0 conn(0x7faac015f330 0x7faac00d1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:15.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.851+0000 7faabf400640 1 -- 172.21.15.167:0/616323555 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faac006c980 con 0x7faac00d1670 2024-05-11T12:11:15.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.851+0000 7fae07200640 1 -- 172.21.15.167:0/1726489215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae000b82c0 msgr2=0x7fae000b8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:15.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.851+0000 7fae07200640 1 --2- 172.21.15.167:0/1726489215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae000b82c0 0x7fae000b8720 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fae0006eb00 tx=0x7fadec02a130 comp rx=0 tx=0).stop 2024-05-11T12:11:15.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.851+0000 7faabf400640 1 -- 172.21.15.167:0/616323555 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7faab8001a70 con 0x7faac00d1670 2024-05-11T12:11:15.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.851+0000 7faabf400640 1 -- 172.21.15.167:0/616323555 >> v1:172.21.15.167:6789/0 conn(0x7faac00d1670 legacy=0x7faac00b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:15.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.851+0000 7faabf400640 1 --2- 172.21.15.167:0/616323555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faaa40039b0 0x7faaa4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.851+0000 7faabf400640 1 --2- 172.21.15.167:0/616323555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faaa4004800 0x7faaa4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.851+0000 7faabf400640 1 --2- 172.21.15.167:0/616323555 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faaa40051c0 0x7faaa400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.851+0000 7faabf400640 1 -- 172.21.15.167:0/616323555 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faac006c450 con 0x7faaa4004800 2024-05-11T12:11:15.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.851+0000 7faabf400640 1 -- 172.21.15.167:0/616323555 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7faac006d2b0 con 0x7faaa40039b0 2024-05-11T12:11:15.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.851+0000 7faabf400640 1 -- 172.21.15.167:0/616323555 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7faaa400bf30 con 0x7faaa40051c0 2024-05-11T12:11:15.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.851+0000 7f0e26800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e200b82c0 0x7f0e2015be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.851+0000 7fae07200640 1 -- 172.21.15.167:0/1726489215 shutdown_connections 2024-05-11T12:11:15.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.851+0000 7fae07200640 1 --2- 172.21.15.167:0/1726489215 >> v2:172.21.15.167:3300/0 conn(0x7fae000d3c70 0x7fae000d4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:15.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.851+0000 7fae07200640 1 --2- 172.21.15.167:0/1726489215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae000b82c0 0x7fae000b8720 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:15.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.852+0000 7fbd7ea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd8015f330 0x7fbd800ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:15.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.852+0000 7fbd7ea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd8015f330 0x7fbd800ced30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41754/0 (socket says 172.21.15.167:41754) 2024-05-11T12:11:15.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.852+0000 7fbd7ea00640 1 -- 172.21.15.167:0/1280517288 learned_addr learned my addr 172.21.15.167:0/1280517288 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:15.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.852+0000 7fbd7f400640 1 --2- 172.21.15.167:0/1280517288 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd800b8800 0x7fbd80151660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:15.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.851+0000 7f0e26800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e200d4380 0x7f0e200cea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.852+0000 7faac4a00640 1 --2- 172.21.15.167:0/616323555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faaa40039b0 0x7faaa4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:15.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.852+0000 7faac5400640 1 --2- 172.21.15.167:0/616323555 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faaa40051c0 0x7faaa400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:15.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.852+0000 7fae07200640 1 -- 172.21.15.167:0/1726489215 >> 172.21.15.167:0/1726489215 conn(0x7fae000b6b80 msgr2=0x7fae000b6f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:15.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.852+0000 7fbd7fe00640 1 --2- 172.21.15.167:0/1280517288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd801f5c20 0x7fbd800b59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:15.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.852+0000 7f0e1f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e200b82c0 0x7f0e2015be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:15.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.852+0000 7f0e26800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e2015f330 0x7f0e200b5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.852+0000 7faabfe00640 1 --2- 172.21.15.167:0/616323555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faaa4004800 0x7faaa4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:15.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.852+0000 7f0e26800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2006cf40 con 0x7f0e200d4380 2024-05-11T12:11:15.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.852+0000 7f0e26800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2006d2b0 con 0x7f0e200b82c0 2024-05-11T12:11:15.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.852+0000 7f0e26800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0e2015f710 con 0x7f0e2015f330 2024-05-11T12:11:15.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.853+0000 7f0e1f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e200b82c0 0x7f0e2015be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.167:47356/0 (socket says 172.21.15.167:47356) 2024-05-11T12:11:15.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.853+0000 7f0e1f400640 1 -- 172.21.15.167:0/691173991 learned_addr learned my addr 172.21.15.167:0/691173991 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:15.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.853+0000 7f0e1ea00640 1 --2- 172.21.15.167:0/691173991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e200d4380 0x7f0e200cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:15.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.853+0000 7fbd7ea00640 1 -- 172.21.15.167:0/1280517288 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd800b8800 msgr2=0x7fbd80151660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:15.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.853+0000 7fbd7ea00640 1 --2- 172.21.15.167:0/1280517288 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd800b8800 0x7fbd80151660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:15.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.853+0000 7fbd7ea00640 1 -- 172.21.15.167:0/1280517288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd801f5c20 msgr2=0x7fbd800b59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:15.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.853+0000 7fbd7ea00640 1 --2- 172.21.15.167:0/1280517288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd801f5c20 0x7fbd800b59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:15.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.853+0000 7fbd7ea00640 1 -- 172.21.15.167:0/1280517288 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd801f63c0 con 0x7fbd8015f330 2024-05-11T12:11:15.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.853+0000 7fae07200640 1 -- 172.21.15.167:0/1726489215 shutdown_connections 2024-05-11T12:11:15.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.853+0000 7f0e1fe00640 1 --2- 172.21.15.167:0/691173991 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e2015f330 0x7f0e200b5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:15.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.854+0000 7faabfe00640 1 -- 172.21.15.167:0/616323555 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faaa40051c0 msgr2=0x7faaa400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:15.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.853+0000 7fae07200640 1 -- 172.21.15.167:0/1726489215 wait complete. 2024-05-11T12:11:15.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.854+0000 7fbd7ea00640 1 --2- 172.21.15.167:0/1280517288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd8015f330 0x7fbd800ced30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00e9b0 tx=0x7fbd6c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:15.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.854+0000 7f0e1ea00640 1 -- 172.21.15.167:0/691173991 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e2015f330 msgr2=0x7f0e200b5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:15.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.854+0000 7f0e1ea00640 1 --2- 172.21.15.167:0/691173991 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e2015f330 0x7f0e200b5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:15.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.854+0000 7f0e1ea00640 1 -- 172.21.15.167:0/691173991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e200b82c0 msgr2=0x7f0e2015be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:15.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.854+0000 7f0e1ea00640 1 --2- 172.21.15.167:0/691173991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e200b82c0 0x7f0e2015be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:15.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.854+0000 7f0e1ea00640 1 -- 172.21.15.167:0/691173991 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e2015c370 con 0x7f0e200d4380 2024-05-11T12:11:15.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.855+0000 7f0e1f400640 1 --2- 172.21.15.167:0/691173991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e200b82c0 0x7f0e2015be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:11:15.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.854+0000 7faabfe00640 1 --2- 172.21.15.167:0/616323555 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faaa40051c0 0x7faaa400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:15.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.854+0000 7faabfe00640 1 -- 172.21.15.167:0/616323555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faaa40039b0 msgr2=0x7faaa4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:15.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.855+0000 7fbd77400640 1 -- 172.21.15.167:0/1280517288 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c017070 con 0x7fbd8015f330 2024-05-11T12:11:15.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.855+0000 7fbd77400640 1 -- 172.21.15.167:0/1280517288 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd6c00c9a0 con 0x7fbd8015f330 2024-05-11T12:11:15.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.855+0000 7fbd86000640 1 -- 172.21.15.167:0/1280517288 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd801f60a0 con 0x7fbd8015f330 2024-05-11T12:11:15.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.854+0000 7faabfe00640 1 --2- 172.21.15.167:0/616323555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faaa40039b0 0x7faaa4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:15.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.854+0000 7faabfe00640 1 -- 172.21.15.167:0/616323555 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faac006c980 con 0x7faaa4004800 2024-05-11T12:11:15.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.854+0000 7faac5400640 1 --2- 172.21.15.167:0/616323555 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faaa40051c0 0x7faaa400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:11:15.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.855+0000 7f0e1ea00640 1 --2- 172.21.15.167:0/691173991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e200d4380 0x7f0e200cea20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c00c3e0 tx=0x7f0e0c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:15.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.855+0000 7fbd86000640 1 -- 172.21.15.167:0/1280517288 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd801f6960 con 0x7fbd8015f330 2024-05-11T12:11:15.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.856+0000 7fbd77400640 1 -- 172.21.15.167:0/1280517288 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c00cd90 con 0x7fbd8015f330 2024-05-11T12:11:15.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.856+0000 7faac4a00640 1 --2- 172.21.15.167:0/616323555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faaa40039b0 0x7faaa4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:11:15.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.856+0000 7faabfe00640 1 --2- 172.21.15.167:0/616323555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faaa4004800 0x7faaa4004c80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7faab800a920 tx=0x7faaa4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ad17ea0f45b5ab3 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:15.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.857+0000 7fae07200640 1 Processor -- start 2024-05-11T12:11:15.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.857+0000 7fae07200640 1 -- start start 2024-05-11T12:11:15.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.857+0000 7f0e17400640 1 -- 172.21.15.167:0/691173991 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c0074e0 con 0x7f0e200d4380 2024-05-11T12:11:15.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.857+0000 7f0e17400640 1 -- 172.21.15.167:0/691173991 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e0c0047e0 con 0x7f0e200d4380 2024-05-11T12:11:15.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.857+0000 7f0e17400640 1 -- 172.21.15.167:0/691173991 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c004bd0 con 0x7f0e200d4380 2024-05-11T12:11:15.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.857+0000 7f0e26800640 1 -- 172.21.15.167:0/691173991 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e20159d20 con 0x7f0e200d4380 2024-05-11T12:11:15.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.857+0000 7f0e26800640 1 -- 172.21.15.167:0/691173991 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e2015a2e0 con 0x7f0e200d4380 2024-05-11T12:11:15.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.857+0000 7fbd86000640 1 -- 172.21.15.167:0/1280517288 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd58005290 con 0x7fbd8015f330 2024-05-11T12:11:15.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.858+0000 7faabf400640 1 -- 172.21.15.167:0/616323555 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab800a570 con 0x7faaa4004800 2024-05-11T12:11:15.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.858+0000 7faabf400640 1 -- 172.21.15.167:0/616323555 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faab802f070 con 0x7faaa4004800 2024-05-11T12:11:15.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.858+0000 7faabf400640 1 -- 172.21.15.167:0/616323555 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab802bdc0 con 0x7faaa4004800 2024-05-11T12:11:15.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.859+0000 7f0e26800640 1 -- 172.21.15.167:0/691173991 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e200694b0 con 0x7f0e200d4380 2024-05-11T12:11:15.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.859+0000 7fae07200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae000b82c0 0x7fae0015a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.859+0000 7fae07200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae000d3c70 0x7fae00158610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.859+0000 7fae07200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fae0015f330 0x7fae00157200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.859+0000 7fae07200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fae0006cf40 con 0x7fae000b82c0 2024-05-11T12:11:15.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.859+0000 7fae07200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fae0006cc60 con 0x7fae000d3c70 2024-05-11T12:11:15.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.859+0000 7fae07200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fae0015f710 con 0x7fae0015f330 2024-05-11T12:11:15.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.860+0000 7faac7400640 1 -- 172.21.15.167:0/616323555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faaa4004800 msgr2=0x7faaa4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:15.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.860+0000 7faac7400640 1 --2- 172.21.15.167:0/616323555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faaa4004800 0x7faaa4004c80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7faab800a920 tx=0x7faaa4005830 comp rx=0 tx=0).stop 2024-05-11T12:11:15.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.860+0000 7faac5400640 1 -- 172.21.15.167:0/616323555 reap_dead start 2024-05-11T12:11:15.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.860+0000 7fadffe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae000b82c0 0x7fae0015a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:15.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.860+0000 7fadffe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae000b82c0 0x7fae0015a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41790/0 (socket says 172.21.15.167:41790) 2024-05-11T12:11:15.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.860+0000 7faac7400640 1 -- 172.21.15.167:0/616323555 shutdown_connections 2024-05-11T12:11:15.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.860+0000 7faac7400640 1 -- 172.21.15.167:0/616323555 >> 172.21.15.167:0/616323555 conn(0x7faac00b6b80 msgr2=0x7faac00b6f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:15.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.860+0000 7fadff400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae000d3c70 0x7fae00158610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:15.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.860+0000 7fadff400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae000d3c70 0x7fae00158610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.167:47368/0 (socket says 172.21.15.167:47368) 2024-05-11T12:11:15.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.860+0000 7fadff400640 1 -- 172.21.15.167:0/70207814 learned_addr learned my addr 172.21.15.167:0/70207814 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:15.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.860+0000 7fae05200640 1 --2- 172.21.15.167:0/70207814 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fae0015f330 0x7fae00157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:15.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.862+0000 7fadff400640 1 -- 172.21.15.167:0/70207814 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fae0015f330 msgr2=0x7fae00157200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:15.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.862+0000 7fadff400640 1 --2- 172.21.15.167:0/70207814 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fae0015f330 0x7fae00157200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:15.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.862+0000 7fadff400640 1 -- 172.21.15.167:0/70207814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae000b82c0 msgr2=0x7fae0015a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:15.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.862+0000 7fadff400640 1 --2- 172.21.15.167:0/70207814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae000b82c0 0x7fae0015a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:15.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.862+0000 7fadff400640 1 -- 172.21.15.167:0/70207814 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae00158f60 con 0x7fae000d3c70 2024-05-11T12:11:15.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.862+0000 7fadffe00640 1 --2- 172.21.15.167:0/70207814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae000b82c0 0x7fae0015a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:15.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.863+0000 7faac7400640 1 -- 172.21.15.167:0/616323555 shutdown_connections 2024-05-11T12:11:15.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.863+0000 7fadff400640 1 --2- 172.21.15.167:0/70207814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae000d3c70 0x7fae00158610 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fadec035040 tx=0x7fadec0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:15.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.864+0000 7fadf7e00640 1 -- 172.21.15.167:0/70207814 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadec042020 con 0x7fae000d3c70 2024-05-11T12:11:15.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.863+0000 7fbd77400640 1 -- 172.21.15.167:0/1280517288 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbd6c01a070 con 0x7fbd8015f330 2024-05-11T12:11:15.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.864+0000 7fadf7e00640 1 -- 172.21.15.167:0/70207814 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadec03e660 con 0x7fae000d3c70 2024-05-11T12:11:15.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.864+0000 7fadf7e00640 1 -- 172.21.15.167:0/70207814 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadec037070 con 0x7fae000d3c70 2024-05-11T12:11:15.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.865+0000 7fbd77400640 1 --2- 172.21.15.167:0/1280517288 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbd60076320 0x7fbd600787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.865+0000 7fae07200640 1 -- 172.21.15.167:0/70207814 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae001592a0 con 0x7fae000d3c70 2024-05-11T12:11:15.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.865+0000 7fae07200640 1 -- 172.21.15.167:0/70207814 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae00159ae0 con 0x7fae000d3c70 2024-05-11T12:11:15.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.863+0000 7faac7400640 1 -- 172.21.15.167:0/616323555 wait complete. 2024-05-11T12:11:15.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.865+0000 7faac7400640 1 Processor -- start 2024-05-11T12:11:15.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.865+0000 7fbd77400640 1 -- 172.21.15.167:0/1280517288 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fbd6c097b70 con 0x7fbd8015f330 2024-05-11T12:11:15.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.866+0000 7faac7400640 1 -- start start 2024-05-11T12:11:15.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.866+0000 7fbd7f400640 1 --2- 172.21.15.167:0/1280517288 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbd60076320 0x7fbd600787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:15.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.867+0000 7fae07200640 1 -- 172.21.15.167:0/70207814 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae000694b0 con 0x7fae000d3c70 2024-05-11T12:11:15.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.867+0000 7faac7400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faaa4004800 0x7faac0156500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.867+0000 7faac7400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faac0155090 0x7faac00cc280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.868+0000 7faac7400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faac00cd130 0x7faac015a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.868+0000 7fbd7f400640 1 --2- 172.21.15.167:0/1280517288 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbd60076320 0x7fbd600787e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbd70034000 tx=0x7fbd7002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:15.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.868+0000 7faac7400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faac015f710 con 0x7faac0155090 2024-05-11T12:11:15.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.868+0000 7faac7400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7faac006cf40 con 0x7faac00cd130 2024-05-11T12:11:15.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.868+0000 7faac7400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7faac00d06b0 con 0x7faaa4004800 2024-05-11T12:11:15.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.868+0000 7faac4a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faaa4004800 0x7faac0156500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:15.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.868+0000 7faac4a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faaa4004800 0x7faac0156500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:59934/0 (socket says 172.21.15.167:59934) 2024-05-11T12:11:15.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.868+0000 7faac4a00640 1 -- 172.21.15.167:0/1757782696 learned_addr learned my addr 172.21.15.167:0/1757782696 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:15.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.868+0000 7faabfe00640 1 --2- 172.21.15.167:0/1757782696 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faac0155090 0x7faac00cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:15.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.869+0000 7faac5400640 1 --2- 172.21.15.167:0/1757782696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faac00cd130 0x7faac015a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:15.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.869+0000 7faac4a00640 1 -- 172.21.15.167:0/1757782696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faac00cd130 msgr2=0x7faac015a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:15.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.869+0000 7faac4a00640 1 --2- 172.21.15.167:0/1757782696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faac00cd130 0x7faac015a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:15.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.869+0000 7faac4a00640 1 -- 172.21.15.167:0/1757782696 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faac0155090 msgr2=0x7faac00cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:15.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.869+0000 7faac4a00640 1 --2- 172.21.15.167:0/1757782696 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faac0155090 0x7faac00cc280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:15.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.869+0000 7faac4a00640 1 -- 172.21.15.167:0/1757782696 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faac0155510 con 0x7faaa4004800 2024-05-11T12:11:15.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.870+0000 7faac4a00640 1 --2- 172.21.15.167:0/1757782696 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faaa4004800 0x7faac0156500 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7faab0003940 tx=0x7faab0003e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:15.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.870+0000 7fadf7e00640 1 -- 172.21.15.167:0/70207814 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fadec045020 con 0x7fae000d3c70 2024-05-11T12:11:15.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.870+0000 7faabcc00640 1 -- 172.21.15.167:0/1757782696 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab0018070 con 0x7faaa4004800 2024-05-11T12:11:15.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.870+0000 7faabcc00640 1 -- 172.21.15.167:0/1757782696 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faab0010b00 con 0x7faaa4004800 2024-05-11T12:11:15.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.870+0000 7faabcc00640 1 -- 172.21.15.167:0/1757782696 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab0014430 con 0x7faaa4004800 2024-05-11T12:11:15.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.870+0000 7faac7400640 1 -- 172.21.15.167:0/1757782696 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faac0155790 con 0x7faaa4004800 2024-05-11T12:11:15.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.871+0000 7faac7400640 1 -- 172.21.15.167:0/1757782696 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7faac0155d50 con 0x7faaa4004800 2024-05-11T12:11:15.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.871+0000 7fadf7e00640 1 --2- 172.21.15.167:0/70207814 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fade0076320 0x7fade00787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.872+0000 7fadf7e00640 1 -- 172.21.15.167:0/70207814 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fadec0b8140 con 0x7fae000d3c70 2024-05-11T12:11:15.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.874+0000 7fbd77400640 1 -- 172.21.15.167:0/1280517288 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fbd6c060cb0 con 0x7fbd8015f330 2024-05-11T12:11:15.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.873+0000 7faac7400640 1 -- 172.21.15.167:0/1757782696 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faac00694b0 con 0x7faaa4004800 2024-05-11T12:11:15.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.873+0000 7fadffe00640 1 --2- 172.21.15.167:0/70207814 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fade0076320 0x7fade00787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:15.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.874+0000 7fadffe00640 1 --2- 172.21.15.167:0/70207814 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fade0076320 0x7fade00787e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fadf0001860 tx=0x7fadf0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:15.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.875+0000 7f0e17400640 1 -- 172.21.15.167:0/691173991 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0e0c011070 con 0x7f0e200d4380 2024-05-11T12:11:15.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.876+0000 7faabcc00640 1 -- 172.21.15.167:0/1757782696 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7faab001b070 con 0x7faaa4004800 2024-05-11T12:11:15.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.877+0000 7faabcc00640 1 --2- 172.21.15.167:0/1757782696 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7faaa0076320 0x7faaa00787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.877+0000 7faabcc00640 1 -- 172.21.15.167:0/1757782696 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7faab00982f0 con 0x7faaa4004800 2024-05-11T12:11:15.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.879+0000 7faabfe00640 1 --2- 172.21.15.167:0/1757782696 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7faaa0076320 0x7faaa00787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:15.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.880+0000 7faabfe00640 1 --2- 172.21.15.167:0/1757782696 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7faaa0076320 0x7faaa00787e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7faab8033000 tx=0x7faab8009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:15.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.878+0000 7f0e17400640 1 --2- 172.21.15.167:0/691173991 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0e000763a0 0x7f0e00078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.879+0000 7f0e17400640 1 -- 172.21.15.167:0/691173991 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f0e0c097950 con 0x7f0e200d4380 2024-05-11T12:11:15.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.879+0000 7f0e1f400640 1 --2- 172.21.15.167:0/691173991 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0e000763a0 0x7f0e00078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:15.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.880+0000 7f0e1f400640 1 --2- 172.21.15.167:0/691173991 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0e000763a0 0x7f0e00078860 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0e10000f60 tx=0x7f0e10007f00 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:15.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.882+0000 7f0e17400640 1 -- 172.21.15.167:0/691173991 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0e0c007e20 con 0x7f0e200d4380 2024-05-11T12:11:15.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.886+0000 7fadf7e00640 1 -- 172.21.15.167:0/70207814 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fadec04c030 con 0x7fae000d3c70 2024-05-11T12:11:15.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.890+0000 7faabcc00640 1 -- 172.21.15.167:0/1757782696 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7faac0155510 con 0x7faaa4004800 2024-05-11T12:11:15.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.901+0000 7f036a400640 1 Processor -- start 2024-05-11T12:11:15.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.901+0000 7f036a400640 1 -- start start 2024-05-11T12:11:15.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.903+0000 7f036a400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f036406ed80 0x7f0364073be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.903+0000 7f036a400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0364074120 0x7f036415ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.904+0000 7f036a400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f036406f440 con 0x7f0364158f10 2024-05-11T12:11:15.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.904+0000 7f036a400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f03640cf450 con 0x7f036406ed80 2024-05-11T12:11:15.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.904+0000 7f0362a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f036406ed80 0x7f0364073be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:15.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.904+0000 7f036a400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f03641592f0 con 0x7f0364074120 2024-05-11T12:11:15.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.904+0000 7f0362a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f036406ed80 0x7f0364073be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.167:47396/0 (socket says 172.21.15.167:47396) 2024-05-11T12:11:15.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.904+0000 7f0363400640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f0364158f10 0x7f03640ced40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.167:55298/0 (socket says 172.21.15.167:55298) 2024-05-11T12:11:15.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.904+0000 7f0363400640 1 -- 172.21.15.167:0/3231507500 learned_addr learned my addr 172.21.15.167:0/3231507500 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:15.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.904+0000 7f0363e00640 1 --2- 172.21.15.167:0/3231507500 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0364074120 0x7f036415ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:15.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.905+0000 7f0362a00640 1 -- 172.21.15.167:0/3231507500 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0364074120 msgr2=0x7f036415ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:15.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.905+0000 7f0362a00640 1 --2- 172.21.15.167:0/3231507500 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0364074120 0x7f036415ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:15.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.905+0000 7f0362a00640 1 -- 172.21.15.167:0/3231507500 >> v1:172.21.15.204:6789/0 conn(0x7f0364158f10 legacy=0x7f03640ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:15.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.905+0000 7f0362a00640 1 -- 172.21.15.167:0/3231507500 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0364072e20 con 0x7f036406ed80 2024-05-11T12:11:15.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.907+0000 7f0362a00640 1 --2- 172.21.15.167:0/3231507500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f036406ed80 0x7f0364073be0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f035c009c80 tx=0x7f035c02b6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71d5809228839d36 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:15.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.908+0000 7f0362000640 1 -- 172.21.15.167:0/3231507500 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f035c007740 con 0x7f036406ed80 2024-05-11T12:11:15.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.908+0000 7f0362000640 1 -- 172.21.15.167:0/3231507500 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f035c0078e0 con 0x7f036406ed80 2024-05-11T12:11:15.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.908+0000 7f0362000640 1 -- 172.21.15.167:0/3231507500 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f035c007cb0 con 0x7f036406ed80 2024-05-11T12:11:15.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.908+0000 7fd7d0c00640 1 Processor -- start 2024-05-11T12:11:15.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.909+0000 7f036a400640 1 -- 172.21.15.167:0/3231507500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f036406ed80 msgr2=0x7f0364073be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:15.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.909+0000 7f036a400640 1 --2- 172.21.15.167:0/3231507500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f036406ed80 0x7f0364073be0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f035c009c80 tx=0x7f035c02b6e0 comp rx=0 tx=0).stop 2024-05-11T12:11:15.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.909+0000 7f036a400640 1 -- 172.21.15.167:0/3231507500 shutdown_connections 2024-05-11T12:11:15.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.909+0000 7f036a400640 1 --2- 172.21.15.167:0/3231507500 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0364074120 0x7f036415ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:15.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.909+0000 7f036a400640 1 --2- 172.21.15.167:0/3231507500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f036406ed80 0x7f0364073be0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:15.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.909+0000 7f036a400640 1 -- 172.21.15.167:0/3231507500 >> 172.21.15.167:0/3231507500 conn(0x7f036406c450 msgr2=0x7f036406c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:15.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.910+0000 7f036a400640 1 -- 172.21.15.167:0/3231507500 shutdown_connections 2024-05-11T12:11:15.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.910+0000 7f036a400640 1 -- 172.21.15.167:0/3231507500 wait complete. 2024-05-11T12:11:15.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.912+0000 7f036a400640 1 Processor -- start 2024-05-11T12:11:15.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.913+0000 7f036a400640 1 -- start start 2024-05-11T12:11:15.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.911+0000 7fd7d0c00640 1 -- start start 2024-05-11T12:11:15.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.913+0000 7fd7d0c00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd7cc158f10 0x7fd7cc06ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.913+0000 7fd7d0c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7cc0ced40 0x7fd7cc073be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.913+0000 7fd7d0c00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7cc06f440 con 0x7fd7cc0cf1d0 2024-05-11T12:11:15.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.913+0000 7fd7d0c00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fd7cc1592f0 con 0x7fd7cc158f10 2024-05-11T12:11:15.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.913+0000 7fd7d0c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd7cc072e20 con 0x7fd7cc0ced40 2024-05-11T12:11:15.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.913+0000 7fd7c3e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7cc0ced40 0x7fd7cc073be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:15.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.913+0000 7fd7c3e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7cc0ced40 0x7fd7cc073be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:59952/0 (socket says 172.21.15.167:59952) 2024-05-11T12:11:15.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.914+0000 7fd7c3e00640 1 -- 172.21.15.167:0/1198198893 learned_addr learned my addr 172.21.15.167:0/1198198893 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:15.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.914+0000 7fd7c9e00640 1 --2- 172.21.15.167:0/1198198893 >> v2:172.21.15.167:3300/0 conn(0x7fd7cc158f10 0x7fd7cc06ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:15.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.914+0000 7f036a400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f036406ed80 0x7f03641490a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.914+0000 7f036a400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0364074120 0x7f036414d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.914+0000 7fd7c3e00640 1 -- 172.21.15.167:0/1198198893 >> v2:172.21.15.167:3300/0 conn(0x7fd7cc158f10 msgr2=0x7fd7cc06ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:15.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.914+0000 7fd7c3e00640 1 --2- 172.21.15.167:0/1198198893 >> v2:172.21.15.167:3300/0 conn(0x7fd7cc158f10 0x7fd7cc06ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:15.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.914+0000 7fd7c3e00640 1 -- 172.21.15.167:0/1198198893 >> v1:172.21.15.167:6789/0 conn(0x7fd7cc0cf1d0 legacy=0x7fd7cc15b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:15.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.914+0000 7fd7c3e00640 1 -- 172.21.15.167:0/1198198893 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7cc073100 con 0x7fd7cc0ced40 2024-05-11T12:11:15.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.915+0000 7f036a400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0364158f10 0x7f036414dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.915+0000 7f036a400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f03640733e0 con 0x7f0364074120 2024-05-11T12:11:15.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.915+0000 7f036a400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f036406f1f0 con 0x7f036406ed80 2024-05-11T12:11:15.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.915+0000 7f036a400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f03640736c0 con 0x7f0364158f10 2024-05-11T12:11:15.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.915+0000 7f0363400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f036406ed80 0x7f03641490a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:15.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.915+0000 7f0363400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f036406ed80 0x7f03641490a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.167:47408/0 (socket says 172.21.15.167:47408) 2024-05-11T12:11:15.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.915+0000 7f0363400640 1 -- 172.21.15.167:0/3698685759 learned_addr learned my addr 172.21.15.167:0/3698685759 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:15.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.915+0000 7f0363e00640 1 --2- 172.21.15.167:0/3698685759 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0364158f10 0x7f036414dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:15.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.916+0000 7f0363400640 1 -- 172.21.15.167:0/3698685759 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0364158f10 msgr2=0x7f036414dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:15.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.916+0000 7f0363400640 1 --2- 172.21.15.167:0/3698685759 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0364158f10 0x7f036414dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:15.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.916+0000 7f0363400640 1 -- 172.21.15.167:0/3698685759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0364074120 msgr2=0x7f036414d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:15.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.916+0000 7f0363400640 1 --2- 172.21.15.167:0/3698685759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0364074120 0x7f036414d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:15.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.916+0000 7f0363400640 1 -- 172.21.15.167:0/3698685759 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03641495e0 con 0x7f036406ed80 2024-05-11T12:11:15.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.916+0000 7f0363400640 1 --2- 172.21.15.167:0/3698685759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f036406ed80 0x7f03641490a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f035400c430 tx=0x7f035400c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:15.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.917+0000 7f035b400640 1 -- 172.21.15.167:0/3698685759 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0354017070 con 0x7f036406ed80 2024-05-11T12:11:15.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.917+0000 7f035b400640 1 -- 172.21.15.167:0/3698685759 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0354005ce0 con 0x7f036406ed80 2024-05-11T12:11:15.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.917+0000 7f036a400640 1 -- 172.21.15.167:0/3698685759 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f036414e2b0 con 0x7f036406ed80 2024-05-11T12:11:15.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.917+0000 7f035b400640 1 -- 172.21.15.167:0/3698685759 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03540072d0 con 0x7f036406ed80 2024-05-11T12:11:15.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.917+0000 7f036a400640 1 -- 172.21.15.167:0/3698685759 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f036414e870 con 0x7f036406ed80 2024-05-11T12:11:15.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.919+0000 7f036a400640 1 -- 172.21.15.167:0/3698685759 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03640694b0 con 0x7f036406ed80 2024-05-11T12:11:15.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.919+0000 7fd7c3e00640 1 --2- 172.21.15.167:0/1198198893 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7cc0ced40 0x7fd7cc073be0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4008870 tx=0x7fd7b402c620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c342bad696e91e48 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:15.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.922+0000 7f035b400640 1 -- 172.21.15.167:0/3698685759 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f035401a070 con 0x7f036406ed80 2024-05-11T12:11:15.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.923+0000 7fd7c9400640 1 -- 172.21.15.167:0/1198198893 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b403a070 con 0x7fd7cc0ced40 2024-05-11T12:11:15.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.923+0000 7fd7c9400640 1 -- 172.21.15.167:0/1198198893 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7b4032480 con 0x7fd7cc0ced40 2024-05-11T12:11:15.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.923+0000 7fd7c9400640 1 -- 172.21.15.167:0/1198198893 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b4032850 con 0x7fd7cc0ced40 2024-05-11T12:11:15.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.924+0000 7fd7d0c00640 1 -- 172.21.15.167:0/1198198893 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7cc0ced40 msgr2=0x7fd7cc073be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:15.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.924+0000 7fd7d0c00640 1 --2- 172.21.15.167:0/1198198893 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7cc0ced40 0x7fd7cc073be0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4008870 tx=0x7fd7b402c620 comp rx=0 tx=0).stop 2024-05-11T12:11:15.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.924+0000 7f035b400640 1 --2- 172.21.15.167:0/3698685759 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f03440762d0 0x7f0344078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.924+0000 7f035b400640 1 -- 172.21.15.167:0/3698685759 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f0354096ff0 con 0x7f036406ed80 2024-05-11T12:11:15.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.925+0000 7fd7d0c00640 1 -- 172.21.15.167:0/1198198893 shutdown_connections 2024-05-11T12:11:15.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.925+0000 7fd7d0c00640 1 --2- 172.21.15.167:0/1198198893 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7cc0ced40 0x7fd7cc073be0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:15.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.925+0000 7fd7d0c00640 1 --2- 172.21.15.167:0/1198198893 >> v2:172.21.15.167:3300/0 conn(0x7fd7cc158f10 0x7fd7cc06ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:15.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.925+0000 7fd7d0c00640 1 -- 172.21.15.167:0/1198198893 >> 172.21.15.167:0/1198198893 conn(0x7fd7cc06c450 msgr2=0x7fd7cc06c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:15.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.925+0000 7f0362a00640 1 --2- 172.21.15.167:0/3698685759 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f03440762d0 0x7f0344078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:15.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.926+0000 7fd7d0c00640 1 -- 172.21.15.167:0/1198198893 shutdown_connections 2024-05-11T12:11:15.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.926+0000 7f0362a00640 1 --2- 172.21.15.167:0/3698685759 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f03440762d0 0x7f0344078790 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f035c035000 tx=0x7f035c02f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:15.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.926+0000 7fd7d0c00640 1 -- 172.21.15.167:0/1198198893 wait complete. 2024-05-11T12:11:15.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.929+0000 7fd7d0c00640 1 Processor -- start 2024-05-11T12:11:15.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.930+0000 7fd7d0c00640 1 -- start start 2024-05-11T12:11:15.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.932+0000 7fd7d0c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7cc0ced40 0x7fd7cc14f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.932+0000 7fd7d0c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7cc0cf1d0 0x7fd7cc151b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.932+0000 7fd7c9e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7cc0ced40 0x7fd7cc14f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:15.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.933+0000 7fd7c9e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7cc0ced40 0x7fd7cc14f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41826/0 (socket says 172.21.15.167:41826) 2024-05-11T12:11:15.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.933+0000 7fd7c9e00640 1 -- 172.21.15.167:0/1666722092 learned_addr learned my addr 172.21.15.167:0/1666722092 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:15.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.933+0000 7fd7d0c00640 1 --2- 172.21.15.167:0/1666722092 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7cc158f10 0x7fd7cc1f41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.933+0000 7fd7c3e00640 1 --2- 172.21.15.167:0/1666722092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7cc0cf1d0 0x7fd7cc151b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:15.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.933+0000 7fd7d0c00640 1 -- 172.21.15.167:0/1666722092 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd7cc0ce8f0 con 0x7fd7cc0ced40 2024-05-11T12:11:15.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.933+0000 7fd7d0c00640 1 -- 172.21.15.167:0/1666722092 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd7cc0736c0 con 0x7fd7cc0cf1d0 2024-05-11T12:11:15.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.934+0000 7fd7d0c00640 1 -- 172.21.15.167:0/1666722092 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd7cc0cea70 con 0x7fd7cc158f10 2024-05-11T12:11:15.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.934+0000 7fd7ca800640 1 --2- 172.21.15.167:0/1666722092 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7cc158f10 0x7fd7cc1f41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:15.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.935+0000 7fd7c9e00640 1 -- 172.21.15.167:0/1666722092 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7cc158f10 msgr2=0x7fd7cc1f41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:15.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.935+0000 7fd7c9e00640 1 --2- 172.21.15.167:0/1666722092 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7cc158f10 0x7fd7cc1f41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:15.936 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.936+0000 7fd7c9e00640 1 -- 172.21.15.167:0/1666722092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7cc0cf1d0 msgr2=0x7fd7cc151b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:15.936 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.936+0000 7fd7c9e00640 1 --2- 172.21.15.167:0/1666722092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7cc0cf1d0 0x7fd7cc151b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:15.936 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.936+0000 7fd7c9e00640 1 -- 172.21.15.167:0/1666722092 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7cc152400 con 0x7fd7cc0ced40 2024-05-11T12:11:15.936 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.936+0000 7f035b400640 1 -- 172.21.15.167:0/3698685759 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f03540600b0 con 0x7f036406ed80 2024-05-11T12:11:15.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.936+0000 7fd7c3e00640 1 --2- 172.21.15.167:0/1666722092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7cc0cf1d0 0x7fd7cc151b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:11:15.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.937+0000 7fd7c9e00640 1 --2- 172.21.15.167:0/1666722092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7cc0ced40 0x7fd7cc14f6e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc00c820 tx=0x7fd7bc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:15.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.937+0000 7fd7c2000640 1 -- 172.21.15.167:0/1666722092 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc015070 con 0x7fd7cc0ced40 2024-05-11T12:11:15.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.937+0000 7fd7c2000640 1 -- 172.21.15.167:0/1666722092 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7bc004020 con 0x7fd7cc0ced40 2024-05-11T12:11:15.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.937+0000 7fd7d0c00640 1 -- 172.21.15.167:0/1666722092 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7cc0b3d90 con 0x7fd7cc0ced40 2024-05-11T12:11:15.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.937+0000 7fd7d0c00640 1 -- 172.21.15.167:0/1666722092 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7cc14fc20 con 0x7fd7cc0ced40 2024-05-11T12:11:15.939 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.938+0000 7fd7c2000640 1 -- 172.21.15.167:0/1666722092 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc0043f0 con 0x7fd7cc0ced40 2024-05-11T12:11:15.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.942+0000 7fd7d0c00640 1 -- 172.21.15.167:0/1666722092 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7a4005290 con 0x7fd7cc0ced40 2024-05-11T12:11:15.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.948+0000 7fd7c2000640 1 -- 172.21.15.167:0/1666722092 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd7bc007830 con 0x7fd7cc0ced40 2024-05-11T12:11:15.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.949+0000 7fd7c2000640 1 --2- 172.21.15.167:0/1666722092 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd7a8076300 0x7fd7a80787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.957+0000 7fd7c3e00640 1 --2- 172.21.15.167:0/1666722092 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd7a8076300 0x7fd7a80787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:15.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.957+0000 7fd7c2000640 1 -- 172.21.15.167:0/1666722092 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fd7bc097690 con 0x7fd7cc0ced40 2024-05-11T12:11:15.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.958+0000 7fd7c3e00640 1 --2- 172.21.15.167:0/1666722092 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd7a8076300 0x7fd7a80787c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4037000 tx=0x7fd7b4030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:15.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.958+0000 7fd7c2000640 1 -- 172.21.15.167:0/1666722092 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd7bc0605c0 con 0x7fd7cc0ced40 2024-05-11T12:11:15.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.959+0000 7f8041000640 1 Processor -- start 2024-05-11T12:11:15.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.959+0000 7f8041000640 1 -- start start 2024-05-11T12:11:15.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.964+0000 7f8041000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f803c06cb20 0x7f803c0d3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.964+0000 7f8041000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f803c162f20 0x7f803c0d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.964+0000 7f8041000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f803c15f810 con 0x7f803c15f430 2024-05-11T12:11:15.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.964+0000 7f8041000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f803c0d0610 con 0x7f803c06cb20 2024-05-11T12:11:15.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.964+0000 7f8041000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f803c0cfa00 con 0x7f803c162f20 2024-05-11T12:11:15.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.966+0000 7f803be00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f803c162f20 0x7f803c0d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:15.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.966+0000 7f803be00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f803c162f20 0x7f803c0d4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:59974/0 (socket says 172.21.15.167:59974) 2024-05-11T12:11:15.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.966+0000 7f803be00640 1 -- 172.21.15.167:0/371909378 learned_addr learned my addr 172.21.15.167:0/371909378 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:15.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.967+0000 7f803aa00640 1 --2- 172.21.15.167:0/371909378 >> v2:172.21.15.204:3300/0 conn(0x7f803c06cb20 0x7f803c0d3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:15.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.967+0000 7f803a000640 1 -- 172.21.15.167:0/371909378 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 146932426 0 0) 0x7f803c15f810 con 0x7f803c15f430 2024-05-11T12:11:15.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.968+0000 7f803be00640 1 -- 172.21.15.167:0/371909378 >> v2:172.21.15.204:3300/0 conn(0x7f803c06cb20 msgr2=0x7f803c0d3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:15.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.968+0000 7f803be00640 1 --2- 172.21.15.167:0/371909378 >> v2:172.21.15.204:3300/0 conn(0x7f803c06cb20 0x7f803c0d3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:15.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.968+0000 7f803be00640 1 -- 172.21.15.167:0/371909378 >> v1:172.21.15.204:6789/0 conn(0x7f803c15f430 legacy=0x7f803c0b8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:15.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.968+0000 7f803be00640 1 -- 172.21.15.167:0/371909378 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f803c0cfce0 con 0x7f803c162f20 2024-05-11T12:11:15.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.970+0000 7f803be00640 1 --2- 172.21.15.167:0/371909378 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f803c162f20 0x7f803c0d4160 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8034009e50 tx=0x7f803402a730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a71e27f4bd3c8ab server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:15.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.971+0000 7f803a000640 1 -- 172.21.15.167:0/371909378 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8034036070 con 0x7f803c162f20 2024-05-11T12:11:15.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.971+0000 7f803a000640 1 -- 172.21.15.167:0/371909378 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8034031600 con 0x7f803c162f20 2024-05-11T12:11:15.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.971+0000 7f803a000640 1 -- 172.21.15.167:0/371909378 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80340319b0 con 0x7f803c162f20 2024-05-11T12:11:15.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.972+0000 7f8041000640 1 -- 172.21.15.167:0/371909378 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f803c162f20 msgr2=0x7f803c0d4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:15.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.972+0000 7f8041000640 1 --2- 172.21.15.167:0/371909378 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f803c162f20 0x7f803c0d4160 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8034009e50 tx=0x7f803402a730 comp rx=0 tx=0).stop 2024-05-11T12:11:15.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.975+0000 7f8041000640 1 -- 172.21.15.167:0/371909378 shutdown_connections 2024-05-11T12:11:15.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.975+0000 7f8041000640 1 --2- 172.21.15.167:0/371909378 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f803c162f20 0x7f803c0d4160 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:15.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.975+0000 7f8041000640 1 --2- 172.21.15.167:0/371909378 >> v2:172.21.15.204:3300/0 conn(0x7f803c06cb20 0x7f803c0d3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:15.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.975+0000 7f8041000640 1 -- 172.21.15.167:0/371909378 >> 172.21.15.167:0/371909378 conn(0x7f803c0d1ac0 msgr2=0x7f803c0d1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:15.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.975+0000 7f8041000640 1 -- 172.21.15.167:0/371909378 shutdown_connections 2024-05-11T12:11:15.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.975+0000 7f8041000640 1 -- 172.21.15.167:0/371909378 wait complete. 2024-05-11T12:11:15.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.978+0000 7f8041000640 1 Processor -- start 2024-05-11T12:11:15.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.978+0000 7f8041000640 1 -- start start 2024-05-11T12:11:15.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.979+0000 7f8041000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f803c06cb20 0x7f803c16aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.980+0000 7f8041000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f803c15f430 0x7f803c0cb380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.980+0000 7f8041000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f803c162f20 0x7f803c0b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.980+0000 7f8041000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f803c0b8a30 con 0x7f803c162f20 2024-05-11T12:11:15.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.980+0000 7f8041000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f803c0d02a0 con 0x7f803c06cb20 2024-05-11T12:11:15.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.980+0000 7f8041000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f803c0cf690 con 0x7f803c15f430 2024-05-11T12:11:15.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.980+0000 7f803aa00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f803c15f430 0x7f803c0cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:15.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.980+0000 7f803aa00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f803c15f430 0x7f803c0cb380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:59988/0 (socket says 172.21.15.167:59988) 2024-05-11T12:11:15.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.980+0000 7f803b400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f803c06cb20 0x7f803c16aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:15.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.980+0000 7f803aa00640 1 -- 172.21.15.167:0/1841279258 learned_addr learned my addr 172.21.15.167:0/1841279258 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:15.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.981+0000 7f803be00640 1 --2- 172.21.15.167:0/1841279258 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f803c162f20 0x7f803c0b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:15.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.981+0000 7f803aa00640 1 -- 172.21.15.167:0/1841279258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f803c06cb20 msgr2=0x7f803c16aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:15.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.981+0000 7f803aa00640 1 --2- 172.21.15.167:0/1841279258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f803c06cb20 0x7f803c16aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:15.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.981+0000 7f803aa00640 1 -- 172.21.15.167:0/1841279258 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f803c162f20 msgr2=0x7f803c0b8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:15.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.982+0000 7f803aa00640 1 --2- 172.21.15.167:0/1841279258 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f803c162f20 0x7f803c0b8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:15.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.982+0000 7f803aa00640 1 -- 172.21.15.167:0/1841279258 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f803c150d30 con 0x7f803c15f430 2024-05-11T12:11:15.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.982+0000 7f803be00640 1 --2- 172.21.15.167:0/1841279258 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f803c162f20 0x7f803c0b8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:11:15.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.983+0000 7f803aa00640 1 --2- 172.21.15.167:0/1841279258 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f803c15f430 0x7f803c0cb380 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f802800e8a0 tx=0x7f802800ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:15.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.984+0000 7f8033400640 1 -- 172.21.15.167:0/1841279258 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8028017070 con 0x7f803c15f430 2024-05-11T12:11:15.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.984+0000 7f8033400640 1 -- 172.21.15.167:0/1841279258 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f802800c940 con 0x7f803c15f430 2024-05-11T12:11:15.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.984+0000 7f8033400640 1 -- 172.21.15.167:0/1841279258 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802800cd30 con 0x7f803c15f430 2024-05-11T12:11:15.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.984+0000 7f8041000640 1 -- 172.21.15.167:0/1841279258 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f803c151010 con 0x7f803c15f430 2024-05-11T12:11:15.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.984+0000 7f8041000640 1 -- 172.21.15.167:0/1841279258 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f803c0cb8c0 con 0x7f803c15f430 2024-05-11T12:11:15.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.987+0000 7f8041000640 1 -- 172.21.15.167:0/1841279258 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8014005290 con 0x7f803c15f430 2024-05-11T12:11:15.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.991+0000 7f8033400640 1 -- 172.21.15.167:0/1841279258 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f802801a070 con 0x7f803c15f430 2024-05-11T12:11:15.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.993+0000 7f8033400640 1 --2- 172.21.15.167:0/1841279258 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f801c0763a0 0x7f801c078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:15.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.993+0000 7f8033400640 1 -- 172.21.15.167:0/1841279258 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f8028097eb0 con 0x7f803c15f430 2024-05-11T12:11:15.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.994+0000 7f803b400640 1 --2- 172.21.15.167:0/1841279258 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f801c0763a0 0x7f801c078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:15.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:15.996+0000 7f803b400640 1 --2- 172.21.15.167:0/1841279258 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f801c0763a0 0x7f801c078860 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f802c007fd0 tx=0x7f802c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:16.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.002+0000 7f8033400640 1 -- 172.21.15.167:0/1841279258 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8028060ef0 con 0x7f803c15f430 2024-05-11T12:11:16.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.277+0000 7fa3fbc00640 1 -- 172.21.15.167:0/3373131380 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fa3ec0578e0 con 0x7fa3ec0e7a40 2024-05-11T12:11:16.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.279+0000 7fa3f0c00640 1 -- 172.21.15.167:0/3373131380 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa3e0060140 con 0x7fa3ec0e7a40 2024-05-11T12:11:16.279 INFO:teuthology.orchestra.run.smithi167.stdout:60129542156 2024-05-11T12:11:16.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.282+0000 7fa3fbc00640 1 -- 172.21.15.167:0/3373131380 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa3d4076300 msgr2=0x7fa3d40787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:16.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.282+0000 7fa3fbc00640 1 --2- 172.21.15.167:0/3373131380 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa3d4076300 0x7fa3d40787c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa3e802a500 tx=0x7fa3e802e000 comp rx=0 tx=0).stop 2024-05-11T12:11:16.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.283+0000 7fa3fbc00640 1 -- 172.21.15.167:0/3373131380 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa3ec0e7a40 msgr2=0x7fa3ec0558f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:16.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.283+0000 7fa3fbc00640 1 --2- 172.21.15.167:0/3373131380 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa3ec0e7a40 0x7fa3ec0558f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa3e000c6b0 tx=0x7fa3e000cb70 comp rx=0 tx=0).stop 2024-05-11T12:11:16.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.283+0000 7fa3fbc00640 1 -- 172.21.15.167:0/3373131380 shutdown_connections 2024-05-11T12:11:16.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.283+0000 7fa3fbc00640 1 --2- 172.21.15.167:0/3373131380 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa3ec0e7a40 0x7fa3ec0558f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:16.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.283+0000 7fa3fbc00640 1 --2- 172.21.15.167:0/3373131380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3ec04a090 0x7fa3ec0da120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:16.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.283+0000 7fa3fbc00640 1 --2- 172.21.15.167:0/3373131380 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa3d4076300 0x7fa3d40787c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:16.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.283+0000 7fa3fbc00640 1 --2- 172.21.15.167:0/3373131380 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3ec03c350 0x7fa3ec17de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:16.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.284+0000 7fa3fbc00640 1 -- 172.21.15.167:0/3373131380 >> 172.21.15.167:0/3373131380 conn(0x7fa3ec056cc0 msgr2=0x7fa3ec0512f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:16.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.284+0000 7fa3fbc00640 1 -- 172.21.15.167:0/3373131380 shutdown_connections 2024-05-11T12:11:16.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.284+0000 7fa3fbc00640 1 -- 172.21.15.167:0/3373131380 wait complete. 2024-05-11T12:11:16.302 INFO:teuthology.orchestra.run.smithi167.stdout:64424509453 2024-05-11T12:11:16.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.301+0000 7fbd86000640 1 -- 172.21.15.167:0/1280517288 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fbd58005850 con 0x7fbd8015f330 2024-05-11T12:11:16.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.302+0000 7fbd77400640 1 -- 172.21.15.167:0/1280517288 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbd6c064fb0 con 0x7fbd8015f330 2024-05-11T12:11:16.306 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.306+0000 7fbd86000640 1 -- 172.21.15.167:0/1280517288 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbd60076320 msgr2=0x7fbd600787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:16.306 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.306+0000 7fbd86000640 1 --2- 172.21.15.167:0/1280517288 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbd60076320 0x7fbd600787e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbd70034000 tx=0x7fbd7002e000 comp rx=0 tx=0).stop 2024-05-11T12:11:16.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.306+0000 7fbd86000640 1 -- 172.21.15.167:0/1280517288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd8015f330 msgr2=0x7fbd800ced30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:16.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.306+0000 7fbd86000640 1 --2- 172.21.15.167:0/1280517288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd8015f330 0x7fbd800ced30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00e9b0 tx=0x7fbd6c00ee70 comp rx=0 tx=0).stop 2024-05-11T12:11:16.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.307+0000 7fbd86000640 1 -- 172.21.15.167:0/1280517288 shutdown_connections 2024-05-11T12:11:16.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.307+0000 7fbd86000640 1 --2- 172.21.15.167:0/1280517288 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbd60076320 0x7fbd600787e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:16.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.307+0000 7fbd86000640 1 --2- 172.21.15.167:0/1280517288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd801f5c20 0x7fbd800b59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:16.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.307+0000 7fbd86000640 1 --2- 172.21.15.167:0/1280517288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd8015f330 0x7fbd800ced30 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:16.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.307+0000 7fbd86000640 1 --2- 172.21.15.167:0/1280517288 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd800b8800 0x7fbd80151660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:16.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.307+0000 7fbd86000640 1 -- 172.21.15.167:0/1280517288 >> 172.21.15.167:0/1280517288 conn(0x7fbd800b6b80 msgr2=0x7fbd80160d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:16.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.307+0000 7fbd86000640 1 -- 172.21.15.167:0/1280517288 shutdown_connections 2024-05-11T12:11:16.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.307+0000 7fbd86000640 1 -- 172.21.15.167:0/1280517288 wait complete. 2024-05-11T12:11:16.313 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542156 got 60129542156 for osd.2 2024-05-11T12:11:16.313 DEBUG:teuthology.parallel:result is None 2024-05-11T12:11:16.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.319+0000 7fae07200640 1 -- 172.21.15.167:0/70207814 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fae000d0230 con 0x7fae000d3c70 2024-05-11T12:11:16.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.320+0000 7fadf7e00640 1 -- 172.21.15.167:0/70207814 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fae001592a0 con 0x7fae000d3c70 2024-05-11T12:11:16.320 INFO:teuthology.orchestra.run.smithi167.stdout:68719476748 2024-05-11T12:11:16.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.324+0000 7fae07200640 1 -- 172.21.15.167:0/70207814 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fade0076320 msgr2=0x7fade00787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:16.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.324+0000 7fae07200640 1 --2- 172.21.15.167:0/70207814 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fade0076320 0x7fade00787e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fadf0001860 tx=0x7fadf0009040 comp rx=0 tx=0).stop 2024-05-11T12:11:16.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.324+0000 7fae07200640 1 -- 172.21.15.167:0/70207814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae000d3c70 msgr2=0x7fae00158610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:16.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.324+0000 7fae07200640 1 --2- 172.21.15.167:0/70207814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae000d3c70 0x7fae00158610 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fadec035040 tx=0x7fadec0046c0 comp rx=0 tx=0).stop 2024-05-11T12:11:16.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.324+0000 7fae07200640 1 -- 172.21.15.167:0/70207814 shutdown_connections 2024-05-11T12:11:16.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.324+0000 7fae07200640 1 --2- 172.21.15.167:0/70207814 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fae0015f330 0x7fae00157200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:16.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.324+0000 7fae07200640 1 --2- 172.21.15.167:0/70207814 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fade0076320 0x7fade00787e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:16.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.324+0000 7fae07200640 1 --2- 172.21.15.167:0/70207814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae000d3c70 0x7fae00158610 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:16.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.324+0000 7fae07200640 1 --2- 172.21.15.167:0/70207814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae000b82c0 0x7fae0015a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:16.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.324+0000 7fae07200640 1 -- 172.21.15.167:0/70207814 >> 172.21.15.167:0/70207814 conn(0x7fae000b6b80 msgr2=0x7fae000c9750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:16.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.325+0000 7fae07200640 1 -- 172.21.15.167:0/70207814 shutdown_connections 2024-05-11T12:11:16.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.325+0000 7fae07200640 1 -- 172.21.15.167:0/70207814 wait complete. 2024-05-11T12:11:16.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.328+0000 7faac7400640 1 -- 172.21.15.167:0/1757782696 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7faac00d37f0 con 0x7faaa4004800 2024-05-11T12:11:16.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.329+0000 7faabcc00640 1 -- 172.21.15.167:0/1757782696 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7faab00619f0 con 0x7faaa4004800 2024-05-11T12:11:16.329 INFO:teuthology.orchestra.run.smithi167.stdout:64424509452 2024-05-11T12:11:16.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.331+0000 7faac7400640 1 -- 172.21.15.167:0/1757782696 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7faaa0076320 msgr2=0x7faaa00787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:16.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.331+0000 7faac7400640 1 --2- 172.21.15.167:0/1757782696 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7faaa0076320 0x7faaa00787e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7faab8033000 tx=0x7faab8009030 comp rx=0 tx=0).stop 2024-05-11T12:11:16.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.331+0000 7faac7400640 1 -- 172.21.15.167:0/1757782696 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faaa4004800 msgr2=0x7faac0156500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:16.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.331+0000 7faac7400640 1 --2- 172.21.15.167:0/1757782696 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faaa4004800 0x7faac0156500 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7faab0003940 tx=0x7faab0003e00 comp rx=0 tx=0).stop 2024-05-11T12:11:16.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.331+0000 7faac7400640 1 -- 172.21.15.167:0/1757782696 shutdown_connections 2024-05-11T12:11:16.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.331+0000 7faac7400640 1 --2- 172.21.15.167:0/1757782696 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7faaa0076320 0x7faaa00787e0 secure :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7faab8033000 tx=0x7faab8009030 comp rx=0 tx=0).stop 2024-05-11T12:11:16.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.331+0000 7faac7400640 1 --2- 172.21.15.167:0/1757782696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faac00cd130 0x7faac015a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:16.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.331+0000 7faac7400640 1 --2- 172.21.15.167:0/1757782696 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faac0155090 0x7faac00cc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:16.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.331+0000 7faac7400640 1 --2- 172.21.15.167:0/1757782696 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faaa4004800 0x7faac0156500 secure :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7faab0003940 tx=0x7faab0003e00 comp rx=0 tx=0).stop 2024-05-11T12:11:16.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.331+0000 7faac7400640 1 -- 172.21.15.167:0/1757782696 >> 172.21.15.167:0/1757782696 conn(0x7faac00b6b80 msgr2=0x7faac01613a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:16.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.332+0000 7faac7400640 1 -- 172.21.15.167:0/1757782696 shutdown_connections 2024-05-11T12:11:16.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.332+0000 7faac7400640 1 -- 172.21.15.167:0/1757782696 wait complete. 2024-05-11T12:11:16.334 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509453 got 64424509453 for osd.1 2024-05-11T12:11:16.334 DEBUG:teuthology.parallel:result is None 2024-05-11T12:11:16.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.339+0000 7f0e26800640 1 -- 172.21.15.167:0/691173991 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f0e200d4c10 con 0x7f0e200d4380 2024-05-11T12:11:16.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.340+0000 7f0e17400640 1 -- 172.21.15.167:0/691173991 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0e0c065e20 con 0x7f0e200d4380 2024-05-11T12:11:16.340 INFO:teuthology.orchestra.run.smithi167.stdout:64424509452 2024-05-11T12:11:16.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.344+0000 7f0e26800640 1 -- 172.21.15.167:0/691173991 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0e000763a0 msgr2=0x7f0e00078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:16.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.344+0000 7f0e26800640 1 --2- 172.21.15.167:0/691173991 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0e000763a0 0x7f0e00078860 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0e10000f60 tx=0x7f0e10007f00 comp rx=0 tx=0).stop 2024-05-11T12:11:16.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.344+0000 7f0e26800640 1 -- 172.21.15.167:0/691173991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e200d4380 msgr2=0x7f0e200cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:16.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.344+0000 7f0e26800640 1 --2- 172.21.15.167:0/691173991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e200d4380 0x7f0e200cea20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c00c3e0 tx=0x7f0e0c00c8a0 comp rx=0 tx=0).stop 2024-05-11T12:11:16.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.344+0000 7f0e26800640 1 -- 172.21.15.167:0/691173991 shutdown_connections 2024-05-11T12:11:16.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.344+0000 7f0e26800640 1 --2- 172.21.15.167:0/691173991 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e2015f330 0x7f0e200b5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:16.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.344+0000 7f0e26800640 1 --2- 172.21.15.167:0/691173991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e200d4380 0x7f0e200cea20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:16.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.344+0000 7f0e26800640 1 --2- 172.21.15.167:0/691173991 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0e000763a0 0x7f0e00078860 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:16.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.344+0000 7f0e26800640 1 --2- 172.21.15.167:0/691173991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e200b82c0 0x7f0e2015be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:16.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.344+0000 7f0e26800640 1 -- 172.21.15.167:0/691173991 >> 172.21.15.167:0/691173991 conn(0x7f0e200b6b80 msgr2=0x7f0e200d3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:16.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.344+0000 7f0e26800640 1 -- 172.21.15.167:0/691173991 shutdown_connections 2024-05-11T12:11:16.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.345+0000 7f0e26800640 1 -- 172.21.15.167:0/691173991 wait complete. 2024-05-11T12:11:16.352 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509452 got 64424509452 for osd.0 2024-05-11T12:11:16.352 DEBUG:teuthology.parallel:result is None 2024-05-11T12:11:16.353 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476748 got 68719476748 for osd.6 2024-05-11T12:11:16.353 DEBUG:teuthology.parallel:result is None 2024-05-11T12:11:16.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.368+0000 7fd7d0c00640 1 -- 172.21.15.167:0/1666722092 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fd7a4005580 con 0x7fd7cc0ced40 2024-05-11T12:11:16.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.368+0000 7fd7c2000640 1 -- 172.21.15.167:0/1666722092 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd7bc0648c0 con 0x7fd7cc0ced40 2024-05-11T12:11:16.368 INFO:teuthology.orchestra.run.smithi167.stdout:68719476747 2024-05-11T12:11:16.370 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509452 got 64424509452 for osd.3 2024-05-11T12:11:16.370 DEBUG:teuthology.parallel:result is None 2024-05-11T12:11:16.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.370+0000 7fd7d0c00640 1 -- 172.21.15.167:0/1666722092 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd7a8076300 msgr2=0x7fd7a80787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:16.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.370+0000 7fd7d0c00640 1 --2- 172.21.15.167:0/1666722092 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd7a8076300 0x7fd7a80787c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4037000 tx=0x7fd7b4030000 comp rx=0 tx=0).stop 2024-05-11T12:11:16.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.370+0000 7fd7d0c00640 1 -- 172.21.15.167:0/1666722092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7cc0ced40 msgr2=0x7fd7cc14f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:16.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.370+0000 7fd7d0c00640 1 --2- 172.21.15.167:0/1666722092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7cc0ced40 0x7fd7cc14f6e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc00c820 tx=0x7fd7bc00cce0 comp rx=0 tx=0).stop 2024-05-11T12:11:16.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.371+0000 7fd7d0c00640 1 -- 172.21.15.167:0/1666722092 shutdown_connections 2024-05-11T12:11:16.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.371+0000 7fd7d0c00640 1 --2- 172.21.15.167:0/1666722092 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7cc158f10 0x7fd7cc1f41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:16.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.371+0000 7fd7d0c00640 1 --2- 172.21.15.167:0/1666722092 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd7a8076300 0x7fd7a80787c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:16.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.371+0000 7fd7d0c00640 1 --2- 172.21.15.167:0/1666722092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7cc0cf1d0 0x7fd7cc151b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:16.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.371+0000 7fd7d0c00640 1 --2- 172.21.15.167:0/1666722092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7cc0ced40 0x7fd7cc14f6e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:16.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.371+0000 7fd7d0c00640 1 -- 172.21.15.167:0/1666722092 >> 172.21.15.167:0/1666722092 conn(0x7fd7cc06c450 msgr2=0x7fd7cc15adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:16.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.371+0000 7fd7d0c00640 1 -- 172.21.15.167:0/1666722092 shutdown_connections 2024-05-11T12:11:16.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.371+0000 7fd7d0c00640 1 -- 172.21.15.167:0/1666722092 wait complete. 2024-05-11T12:11:16.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.375+0000 7f036a400640 1 -- 172.21.15.167:0/3698685759 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f03640d0310 con 0x7f036406ed80 2024-05-11T12:11:16.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.376+0000 7f035b400640 1 -- 172.21.15.167:0/3698685759 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f03540643b0 con 0x7f036406ed80 2024-05-11T12:11:16.376 INFO:teuthology.orchestra.run.smithi167.stdout:68719476749 2024-05-11T12:11:16.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.378+0000 7f036a400640 1 -- 172.21.15.167:0/3698685759 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f03440762d0 msgr2=0x7f0344078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:16.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.378+0000 7f036a400640 1 --2- 172.21.15.167:0/3698685759 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f03440762d0 0x7f0344078790 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f035c035000 tx=0x7f035c02f000 comp rx=0 tx=0).stop 2024-05-11T12:11:16.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.378+0000 7f036a400640 1 -- 172.21.15.167:0/3698685759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f036406ed80 msgr2=0x7f03641490a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:16.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.378+0000 7f036a400640 1 --2- 172.21.15.167:0/3698685759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f036406ed80 0x7f03641490a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f035400c430 tx=0x7f035400c8f0 comp rx=0 tx=0).stop 2024-05-11T12:11:16.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.378+0000 7f036a400640 1 -- 172.21.15.167:0/3698685759 shutdown_connections 2024-05-11T12:11:16.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.378+0000 7f036a400640 1 --2- 172.21.15.167:0/3698685759 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0364158f10 0x7f036414dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:16.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.378+0000 7f036a400640 1 --2- 172.21.15.167:0/3698685759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0364074120 0x7f036414d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:16.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.378+0000 7f036a400640 1 --2- 172.21.15.167:0/3698685759 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f03440762d0 0x7f0344078790 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:16.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.378+0000 7f036a400640 1 --2- 172.21.15.167:0/3698685759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f036406ed80 0x7f03641490a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:16.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.378+0000 7f036a400640 1 -- 172.21.15.167:0/3698685759 >> 172.21.15.167:0/3698685759 conn(0x7f036406c450 msgr2=0x7f03640b6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:16.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.378+0000 7f036a400640 1 -- 172.21.15.167:0/3698685759 shutdown_connections 2024-05-11T12:11:16.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.378+0000 7f036a400640 1 -- 172.21.15.167:0/3698685759 wait complete. 2024-05-11T12:11:16.388 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476747 got 68719476747 for osd.7 2024-05-11T12:11:16.388 DEBUG:teuthology.parallel:result is None 2024-05-11T12:11:16.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.394+0000 7f8041000640 1 -- 172.21.15.167:0/1841279258 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f8014005850 con 0x7f803c15f430 2024-05-11T12:11:16.396 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476749 got 68719476749 for osd.4 2024-05-11T12:11:16.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.395+0000 7f8033400640 1 -- 172.21.15.167:0/1841279258 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f80280651f0 con 0x7f803c15f430 2024-05-11T12:11:16.396 INFO:teuthology.orchestra.run.smithi167.stdout:68719476747 2024-05-11T12:11:16.396 DEBUG:teuthology.parallel:result is None 2024-05-11T12:11:16.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.398+0000 7f8041000640 1 -- 172.21.15.167:0/1841279258 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f801c0763a0 msgr2=0x7f801c078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:16.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.398+0000 7f8041000640 1 --2- 172.21.15.167:0/1841279258 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f801c0763a0 0x7f801c078860 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f802c007fd0 tx=0x7f802c009040 comp rx=0 tx=0).stop 2024-05-11T12:11:16.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.398+0000 7f8041000640 1 -- 172.21.15.167:0/1841279258 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f803c15f430 msgr2=0x7f803c0cb380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:16.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.398+0000 7f8041000640 1 --2- 172.21.15.167:0/1841279258 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f803c15f430 0x7f803c0cb380 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f802800e8a0 tx=0x7f802800ed60 comp rx=0 tx=0).stop 2024-05-11T12:11:16.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.398+0000 7f8041000640 1 -- 172.21.15.167:0/1841279258 shutdown_connections 2024-05-11T12:11:16.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.398+0000 7f8041000640 1 --2- 172.21.15.167:0/1841279258 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f803c162f20 0x7f803c0b8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:16.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.398+0000 7f8041000640 1 --2- 172.21.15.167:0/1841279258 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f803c15f430 0x7f803c0cb380 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:16.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.398+0000 7f8041000640 1 --2- 172.21.15.167:0/1841279258 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f801c0763a0 0x7f801c078860 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:16.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.398+0000 7f8041000640 1 --2- 172.21.15.167:0/1841279258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f803c06cb20 0x7f803c16aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:16.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.398+0000 7f8041000640 1 -- 172.21.15.167:0/1841279258 >> 172.21.15.167:0/1841279258 conn(0x7f803c0d1ac0 msgr2=0x7f803c0b4ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:16.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.398+0000 7f8041000640 1 -- 172.21.15.167:0/1841279258 shutdown_connections 2024-05-11T12:11:16.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.398+0000 7f8041000640 1 -- 172.21.15.167:0/1841279258 wait complete. 2024-05-11T12:11:16.416 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476747 got 68719476747 for osd.5 2024-05-11T12:11:16.416 DEBUG:teuthology.parallel:result is None 2024-05-11T12:11:16.416 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-11T12:11:16.416 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T12:11:16.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.646+0000 7f8952c00640 1 Processor -- start 2024-05-11T12:11:16.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.646+0000 7f8952c00640 1 -- start start 2024-05-11T12:11:16.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.647+0000 7f8952c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f894c06d460 0x7f894c06d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:16.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.647+0000 7f8952c00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f894c15df30 con 0x7f894c157750 2024-05-11T12:11:16.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.647+0000 7f8952c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f894c15e210 con 0x7f894c0b3ea0 2024-05-11T12:11:16.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.647+0000 7f8952c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f894c06ddf0 con 0x7f894c06d460 2024-05-11T12:11:16.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.647+0000 7f894be00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f894c157750 0x7f894c15d740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49512/0 (socket says 172.21.15.167:49512) 2024-05-11T12:11:16.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.647+0000 7f894be00640 1 -- 172.21.15.167:0/269654449 learned_addr learned my addr 172.21.15.167:0/269654449 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:16.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.648+0000 7f894aa00640 1 -- 172.21.15.167:0/269654449 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2712445849 0 0) 0x7f894c15df30 con 0x7f894c157750 2024-05-11T12:11:16.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.648+0000 7f894aa00640 1 -- 172.21.15.167:0/269654449 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8930003520 con 0x7f894c157750 2024-05-11T12:11:16.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.648+0000 7f8950c00640 1 --2- 172.21.15.167:0/269654449 >> v2:172.21.15.204:3300/0 conn(0x7f894c06d460 0x7f894c06d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:16.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.648+0000 7f894aa00640 1 -- 172.21.15.167:0/269654449 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1416206215 0 0) 0x7f8930003520 con 0x7f894c157750 2024-05-11T12:11:16.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.648+0000 7f894aa00640 1 -- 172.21.15.167:0/269654449 >> v2:172.21.15.204:3300/0 conn(0x7f894c06d460 msgr2=0x7f894c06d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:16.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.648+0000 7f894aa00640 1 --2- 172.21.15.167:0/269654449 >> v2:172.21.15.204:3300/0 conn(0x7f894c06d460 0x7f894c06d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:16.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.648+0000 7f894aa00640 1 -- 172.21.15.167:0/269654449 >> v1:172.21.15.204:6789/0 conn(0x7f894c0b3ea0 legacy=0x7f894c161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:16.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.648+0000 7f894aa00640 1 -- 172.21.15.167:0/269654449 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f894c15e4f0 con 0x7f894c157750 2024-05-11T12:11:16.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.648+0000 7f894aa00640 1 -- 172.21.15.167:0/269654449 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f893c0016f0 con 0x7f894c157750 2024-05-11T12:11:16.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.648+0000 7f894aa00640 1 -- 172.21.15.167:0/269654449 >> v1:172.21.15.167:6789/0 conn(0x7f894c157750 legacy=0x7f894c15d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:16.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.648+0000 7f894aa00640 1 --2- 172.21.15.167:0/269654449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8930003a50 0x7f8930003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:16.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.648+0000 7f894aa00640 1 --2- 172.21.15.167:0/269654449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8930004800 0x7f8930004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:16.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.648+0000 7f894aa00640 1 --2- 172.21.15.167:0/269654449 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f89300052e0 0x7f893000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:16.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.648+0000 7f894aa00640 1 -- 172.21.15.167:0/269654449 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f894c06ddf0 con 0x7f8930004800 2024-05-11T12:11:16.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.648+0000 7f894aa00640 1 -- 172.21.15.167:0/269654449 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f893000c1b0 con 0x7f8930003a50 2024-05-11T12:11:16.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.648+0000 7f894aa00640 1 -- 172.21.15.167:0/269654449 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f893000c3b0 con 0x7f89300052e0 2024-05-11T12:11:16.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.648+0000 7f8950c00640 1 --2- 172.21.15.167:0/269654449 >> v2:172.21.15.204:3300/0 conn(0x7f894c06d460 0x7f894c06d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:11:16.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.648+0000 7f894b400640 1 --2- 172.21.15.167:0/269654449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8930004800 0x7f8930004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:16.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.649+0000 7f894b400640 1 -- 172.21.15.167:0/269654449 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f89300052e0 msgr2=0x7f893000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:16.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.649+0000 7f894b400640 1 --2- 172.21.15.167:0/269654449 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f89300052e0 0x7f893000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:16.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.649+0000 7f894b400640 1 -- 172.21.15.167:0/269654449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8930003a50 msgr2=0x7f8930003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:16.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.649+0000 7f894be00640 1 --2- 172.21.15.167:0/269654449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8930003a50 0x7f8930003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:16.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.649+0000 7f894b400640 1 --2- 172.21.15.167:0/269654449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8930003a50 0x7f8930003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:16.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.649+0000 7f894b400640 1 -- 172.21.15.167:0/269654449 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8930003520 con 0x7f8930004800 2024-05-11T12:11:16.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.649+0000 7f8950c00640 1 -- 172.21.15.167:0/269654449 reap_dead start 2024-05-11T12:11:16.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.649+0000 7f894be00640 1 --2- 172.21.15.167:0/269654449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8930003a50 0x7f8930003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:16.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.650+0000 7f894b400640 1 --2- 172.21.15.167:0/269654449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8930004800 0x7f8930004c80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8934001760 tx=0x7f8930005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20cae479cd90dc8e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:16.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.650+0000 7f894aa00640 1 -- 172.21.15.167:0/269654449 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89340013b0 con 0x7f8930004800 2024-05-11T12:11:16.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.650+0000 7f894aa00640 1 -- 172.21.15.167:0/269654449 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89340052d0 con 0x7f8930004800 2024-05-11T12:11:16.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.650+0000 7f894aa00640 1 -- 172.21.15.167:0/269654449 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893402aa70 con 0x7f8930004800 2024-05-11T12:11:16.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.651+0000 7f8952c00640 1 -- 172.21.15.167:0/269654449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8930004800 msgr2=0x7f8930004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:16.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.651+0000 7f8952c00640 1 --2- 172.21.15.167:0/269654449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8930004800 0x7f8930004c80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8934001760 tx=0x7f8930005830 comp rx=0 tx=0).stop 2024-05-11T12:11:16.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.651+0000 7f8952c00640 1 -- 172.21.15.167:0/269654449 shutdown_connections 2024-05-11T12:11:16.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.651+0000 7f8952c00640 1 --2- 172.21.15.167:0/269654449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8930004800 0x7f8930004c80 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:16.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.651+0000 7f8952c00640 1 -- 172.21.15.167:0/269654449 >> 172.21.15.167:0/269654449 conn(0x7f894c0cd8f0 msgr2=0x7f894c0cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:16.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.651+0000 7f8952c00640 1 -- 172.21.15.167:0/269654449 shutdown_connections 2024-05-11T12:11:16.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.651+0000 7f8952c00640 1 -- 172.21.15.167:0/269654449 wait complete. 2024-05-11T12:11:16.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.652+0000 7f8952c00640 1 Processor -- start 2024-05-11T12:11:16.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.653+0000 7f8952c00640 1 -- start start 2024-05-11T12:11:16.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.653+0000 7f8952c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8930004800 0x7f894c15c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:16.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.654+0000 7f8952c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f894c1e7d20 0x7f894c1eaa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:16.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.654+0000 7f8952c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f894c1e6db0 0x7f894c1e7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:16.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.654+0000 7f8952c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f894c06d8b0 con 0x7f8930004800 2024-05-11T12:11:16.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.654+0000 7f8952c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f894c161000 con 0x7f894c1e7d20 2024-05-11T12:11:16.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.654+0000 7f8952c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f894c06da30 con 0x7f894c1e6db0 2024-05-11T12:11:16.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.654+0000 7f894be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8930004800 0x7f894c15c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:16.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.654+0000 7f894be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8930004800 0x7f894c15c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41846/0 (socket says 172.21.15.167:41846) 2024-05-11T12:11:16.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.654+0000 7f894be00640 1 -- 172.21.15.167:0/610609541 learned_addr learned my addr 172.21.15.167:0/610609541 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:16.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.654+0000 7f8950c00640 1 --2- 172.21.15.167:0/610609541 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f894c1e6db0 0x7f894c1e7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:16.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.654+0000 7f894b400640 1 --2- 172.21.15.167:0/610609541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f894c1e7d20 0x7f894c1eaa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:16.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.654+0000 7f894be00640 1 -- 172.21.15.167:0/610609541 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f894c1e6db0 msgr2=0x7f894c1e7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:16.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.654+0000 7f894be00640 1 --2- 172.21.15.167:0/610609541 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f894c1e6db0 0x7f894c1e7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:16.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.654+0000 7f894be00640 1 -- 172.21.15.167:0/610609541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f894c1e7d20 msgr2=0x7f894c1eaa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:16.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.654+0000 7f894be00640 1 --2- 172.21.15.167:0/610609541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f894c1e7d20 0x7f894c1eaa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:16.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.654+0000 7f894be00640 1 -- 172.21.15.167:0/610609541 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f894c131490 con 0x7f8930004800 2024-05-11T12:11:16.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.654+0000 7f8950c00640 1 --2- 172.21.15.167:0/610609541 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f894c1e6db0 0x7f894c1e7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:16.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.654+0000 7f894be00640 1 --2- 172.21.15.167:0/610609541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8930004800 0x7f894c15c420 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f893c00d7a0 tx=0x7f893c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:16.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.655+0000 7f8943e00640 1 -- 172.21.15.167:0/610609541 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893c0074e0 con 0x7f8930004800 2024-05-11T12:11:16.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.655+0000 7f8943e00640 1 -- 172.21.15.167:0/610609541 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f893c0047e0 con 0x7f8930004800 2024-05-11T12:11:16.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.655+0000 7f8952c00640 1 -- 172.21.15.167:0/610609541 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f894c131730 con 0x7f8930004800 2024-05-11T12:11:16.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.655+0000 7f8943e00640 1 -- 172.21.15.167:0/610609541 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893c004b90 con 0x7f8930004800 2024-05-11T12:11:16.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.655+0000 7f8952c00640 1 -- 172.21.15.167:0/610609541 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f894c131cf0 con 0x7f8930004800 2024-05-11T12:11:16.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.656+0000 7f8952c00640 1 -- 172.21.15.167:0/610609541 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f894c069590 con 0x7f8930004800 2024-05-11T12:11:16.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.657+0000 7f8943e00640 1 -- 172.21.15.167:0/610609541 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f893c01b020 con 0x7f8930004800 2024-05-11T12:11:16.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.658+0000 7f8943e00640 1 --2- 172.21.15.167:0/610609541 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f8928076300 0x7f89280787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:16.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.658+0000 7f8943e00640 1 -- 172.21.15.167:0/610609541 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f893c097780 con 0x7f8930004800 2024-05-11T12:11:16.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.659+0000 7f894b400640 1 --2- 172.21.15.167:0/610609541 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f8928076300 0x7f89280787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:16.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.659+0000 7f894b400640 1 --2- 172.21.15.167:0/610609541 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f8928076300 0x7f89280787c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8934032000 tx=0x7f893402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:16.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.663+0000 7f8943e00640 1 -- 172.21.15.167:0/610609541 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f893c060c70 con 0x7f8930004800 2024-05-11T12:11:16.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.855+0000 7f8952c00640 1 -- 172.21.15.167:0/610609541 --> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f894c15da20 con 0x7f8928076300 2024-05-11T12:11:16.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.882+0000 7f8943e00640 1 -- 172.21.15.167:0/610609541 <== mgr.4106 v2:172.21.15.167:6832/578352559 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332527 (secure 0 0 0) 0x7f894c15da20 con 0x7f8928076300 2024-05-11T12:11:16.888 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:11:16.890 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2024-05-11T12:11:16.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.891+0000 7f8952c00640 1 -- 172.21.15.167:0/610609541 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f8928076300 msgr2=0x7f89280787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:16.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.891+0000 7f8952c00640 1 --2- 172.21.15.167:0/610609541 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f8928076300 0x7f89280787c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8934032000 tx=0x7f893402e000 comp rx=0 tx=0).stop 2024-05-11T12:11:16.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.891+0000 7f8952c00640 1 -- 172.21.15.167:0/610609541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8930004800 msgr2=0x7f894c15c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:16.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.891+0000 7f8952c00640 1 --2- 172.21.15.167:0/610609541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8930004800 0x7f894c15c420 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f893c00d7a0 tx=0x7f893c00dc60 comp rx=0 tx=0).stop 2024-05-11T12:11:16.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.891+0000 7f8952c00640 1 -- 172.21.15.167:0/610609541 shutdown_connections 2024-05-11T12:11:16.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.891+0000 7f8952c00640 1 --2- 172.21.15.167:0/610609541 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f894c1e6db0 0x7f894c1e7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:16.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.891+0000 7f8952c00640 1 --2- 172.21.15.167:0/610609541 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f8928076300 0x7f89280787c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:16.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.891+0000 7f8952c00640 1 --2- 172.21.15.167:0/610609541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f894c1e7d20 0x7f894c1eaa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:16.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.891+0000 7f8952c00640 1 --2- 172.21.15.167:0/610609541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8930004800 0x7f894c15c420 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:16.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.891+0000 7f8952c00640 1 -- 172.21.15.167:0/610609541 >> 172.21.15.167:0/610609541 conn(0x7f894c0cd8f0 msgr2=0x7f894c0b4360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:16.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.891+0000 7f8952c00640 1 -- 172.21.15.167:0/610609541 shutdown_connections 2024-05-11T12:11:16.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:16.891+0000 7f8952c00640 1 -- 172.21.15.167:0/610609541 wait complete. 2024-05-11T12:11:16.911 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2024-05-11T12:11:15.240783+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":150,"num_write_kb":1647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219560,"kb_used_data":3400,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755160,"statfs":{"total":773094113280,"available":772869283840,"internally_reserved":0,"allocated":3481600,"data_stored":2063272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005512"},"pg_stats":[{"pgid":"2.1f","version":"22'4","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.057257+0000","last_change":"2024-05-11T12:10:44.771802+0000","last_active":"2024-05-11T12:10:53.057257+0000","last_peered":"2024-05-11T12:10:53.057257+0000","last_clean":"2024-05-11T12:10:53.057257+0000","last_became_active":"2024-05-11T12:10:44.771511+0000","last_became_peered":"2024-05-11T12:10:44.771511+0000","last_unstale":"2024-05-11T12:10:53.057257+0000","last_undegraded":"2024-05-11T12:10:53.057257+0000","last_fullsized":"2024-05-11T12:10:53.057257+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:38:50.526921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.832716+0000","last_change":"2024-05-11T12:10:45.765221+0000","last_active":"2024-05-11T12:10:50.832716+0000","last_peered":"2024-05-11T12:10:50.832716+0000","last_clean":"2024-05-11T12:10:50.832716+0000","last_became_active":"2024-05-11T12:10:45.764595+0000","last_became_peered":"2024-05-11T12:10:45.764595+0000","last_unstale":"2024-05-11T12:10:50.832716+0000","last_undegraded":"2024-05-11T12:10:50.832716+0000","last_fullsized":"2024-05-11T12:10:50.832716+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:01:30.356378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.830235+0000","last_change":"2024-05-11T12:10:49.979783+0000","last_active":"2024-05-11T12:10:50.830235+0000","last_peered":"2024-05-11T12:10:50.830235+0000","last_clean":"2024-05-11T12:10:50.830235+0000","last_became_active":"2024-05-11T12:10:49.977927+0000","last_became_peered":"2024-05-11T12:10:49.977927+0000","last_unstale":"2024-05-11T12:10:50.830235+0000","last_undegraded":"2024-05-11T12:10:50.830235+0000","last_fullsized":"2024-05-11T12:10:50.830235+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:31:59.631048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.833109+0000","last_change":"2024-05-11T12:10:44.769768+0000","last_active":"2024-05-11T12:10:50.833109+0000","last_peered":"2024-05-11T12:10:50.833109+0000","last_clean":"2024-05-11T12:10:50.833109+0000","last_became_active":"2024-05-11T12:10:44.769498+0000","last_became_peered":"2024-05-11T12:10:44.769498+0000","last_unstale":"2024-05-11T12:10:50.833109+0000","last_undegraded":"2024-05-11T12:10:50.833109+0000","last_fullsized":"2024-05-11T12:10:50.833109+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:14:58.421726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.056999+0000","last_change":"2024-05-11T12:10:45.791108+0000","last_active":"2024-05-11T12:10:53.056999+0000","last_peered":"2024-05-11T12:10:53.056999+0000","last_clean":"2024-05-11T12:10:53.056999+0000","last_became_active":"2024-05-11T12:10:45.790807+0000","last_became_peered":"2024-05-11T12:10:45.790807+0000","last_unstale":"2024-05-11T12:10:53.056999+0000","last_undegraded":"2024-05-11T12:10:53.056999+0000","last_fullsized":"2024-05-11T12:10:53.056999+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:52:36.650847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.177575+0000","last_change":"2024-05-11T12:10:49.843347+0000","last_active":"2024-05-11T12:10:51.177575+0000","last_peered":"2024-05-11T12:10:51.177575+0000","last_clean":"2024-05-11T12:10:51.177575+0000","last_became_active":"2024-05-11T12:10:49.840522+0000","last_became_peered":"2024-05-11T12:10:49.840522+0000","last_unstale":"2024-05-11T12:10:51.177575+0000","last_undegraded":"2024-05-11T12:10:51.177575+0000","last_fullsized":"2024-05-11T12:10:51.177575+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:09:52.792492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.986041+0000","last_change":"2024-05-11T12:10:44.788150+0000","last_active":"2024-05-11T12:10:52.986041+0000","last_peered":"2024-05-11T12:10:52.986041+0000","last_clean":"2024-05-11T12:10:52.986041+0000","last_became_active":"2024-05-11T12:10:44.787827+0000","last_became_peered":"2024-05-11T12:10:44.787827+0000","last_unstale":"2024-05-11T12:10:52.986041+0000","last_undegraded":"2024-05-11T12:10:52.986041+0000","last_fullsized":"2024-05-11T12:10:52.986041+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:50:29.143515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.834938+0000","last_change":"2024-05-11T12:10:45.769675+0000","last_active":"2024-05-11T12:10:50.834938+0000","last_peered":"2024-05-11T12:10:50.834938+0000","last_clean":"2024-05-11T12:10:50.834938+0000","last_became_active":"2024-05-11T12:10:45.766444+0000","last_became_peered":"2024-05-11T12:10:45.766444+0000","last_unstale":"2024-05-11T12:10:50.834938+0000","last_undegraded":"2024-05-11T12:10:50.834938+0000","last_fullsized":"2024-05-11T12:10:50.834938+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:02:47.506141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.177937+0000","last_change":"2024-05-11T12:10:49.843249+0000","last_active":"2024-05-11T12:10:51.177937+0000","last_peered":"2024-05-11T12:10:51.177937+0000","last_clean":"2024-05-11T12:10:51.177937+0000","last_became_active":"2024-05-11T12:10:49.841714+0000","last_became_peered":"2024-05-11T12:10:49.841714+0000","last_unstale":"2024-05-11T12:10:51.177937+0000","last_undegraded":"2024-05-11T12:10:51.177937+0000","last_fullsized":"2024-05-11T12:10:51.177937+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:15:59.709719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.986035+0000","last_change":"2024-05-11T12:10:44.783078+0000","last_active":"2024-05-11T12:10:52.986035+0000","last_peered":"2024-05-11T12:10:52.986035+0000","last_clean":"2024-05-11T12:10:52.986035+0000","last_became_active":"2024-05-11T12:10:44.782750+0000","last_became_peered":"2024-05-11T12:10:44.782750+0000","last_unstale":"2024-05-11T12:10:52.986035+0000","last_undegraded":"2024-05-11T12:10:52.986035+0000","last_fullsized":"2024-05-11T12:10:52.986035+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:32:31.312066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.836129+0000","last_change":"2024-05-11T12:10:45.771583+0000","last_active":"2024-05-11T12:10:50.836129+0000","last_peered":"2024-05-11T12:10:50.836129+0000","last_clean":"2024-05-11T12:10:50.836129+0000","last_became_active":"2024-05-11T12:10:45.766864+0000","last_became_peered":"2024-05-11T12:10:45.766864+0000","last_unstale":"2024-05-11T12:10:50.836129+0000","last_undegraded":"2024-05-11T12:10:50.836129+0000","last_fullsized":"2024-05-11T12:10:50.836129+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:29:58.902743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.177649+0000","last_change":"2024-05-11T12:10:49.833489+0000","last_active":"2024-05-11T12:10:51.177649+0000","last_peered":"2024-05-11T12:10:51.177649+0000","last_clean":"2024-05-11T12:10:51.177649+0000","last_became_active":"2024-05-11T12:10:49.833030+0000","last_became_peered":"2024-05-11T12:10:49.833030+0000","last_unstale":"2024-05-11T12:10:51.177649+0000","last_undegraded":"2024-05-11T12:10:51.177649+0000","last_fullsized":"2024-05-11T12:10:51.177649+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:40:37.671338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.833888+0000","last_change":"2024-05-11T12:10:45.777692+0000","last_active":"2024-05-11T12:10:50.833888+0000","last_peered":"2024-05-11T12:10:50.833888+0000","last_clean":"2024-05-11T12:10:50.833888+0000","last_became_active":"2024-05-11T12:10:45.777388+0000","last_became_peered":"2024-05-11T12:10:45.777388+0000","last_unstale":"2024-05-11T12:10:50.833888+0000","last_undegraded":"2024-05-11T12:10:50.833888+0000","last_fullsized":"2024-05-11T12:10:50.833888+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:30:11.665446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.986035+0000","last_change":"2024-05-11T12:10:44.780645+0000","last_active":"2024-05-11T12:10:52.986035+0000","last_peered":"2024-05-11T12:10:52.986035+0000","last_clean":"2024-05-11T12:10:52.986035+0000","last_became_active":"2024-05-11T12:10:44.780371+0000","last_became_peered":"2024-05-11T12:10:44.780371+0000","last_unstale":"2024-05-11T12:10:52.986035+0000","last_undegraded":"2024-05-11T12:10:52.986035+0000","last_fullsized":"2024-05-11T12:10:52.986035+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:46:19.349691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.178802+0000","last_change":"2024-05-11T12:10:49.842333+0000","last_active":"2024-05-11T12:10:51.178802+0000","last_peered":"2024-05-11T12:10:51.178802+0000","last_clean":"2024-05-11T12:10:51.178802+0000","last_became_active":"2024-05-11T12:10:49.841805+0000","last_became_peered":"2024-05-11T12:10:49.841805+0000","last_unstale":"2024-05-11T12:10:51.178802+0000","last_undegraded":"2024-05-11T12:10:51.178802+0000","last_fullsized":"2024-05-11T12:10:51.178802+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:30:02.975329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.831265+0000","last_change":"2024-05-11T12:10:45.785101+0000","last_active":"2024-05-11T12:10:50.831265+0000","last_peered":"2024-05-11T12:10:50.831265+0000","last_clean":"2024-05-11T12:10:50.831265+0000","last_became_active":"2024-05-11T12:10:45.784830+0000","last_became_peered":"2024-05-11T12:10:45.784830+0000","last_unstale":"2024-05-11T12:10:50.831265+0000","last_undegraded":"2024-05-11T12:10:50.831265+0000","last_fullsized":"2024-05-11T12:10:50.831265+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:15:55.637133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.084641+0000","last_change":"2024-05-11T12:10:44.762965+0000","last_active":"2024-05-11T12:10:53.084641+0000","last_peered":"2024-05-11T12:10:53.084641+0000","last_clean":"2024-05-11T12:10:53.084641+0000","last_became_active":"2024-05-11T12:10:44.761074+0000","last_became_peered":"2024-05-11T12:10:44.761074+0000","last_unstale":"2024-05-11T12:10:53.084641+0000","last_undegraded":"2024-05-11T12:10:53.084641+0000","last_fullsized":"2024-05-11T12:10:53.084641+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:11:44.935841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.178724+0000","last_change":"2024-05-11T12:10:49.842014+0000","last_active":"2024-05-11T12:10:51.178724+0000","last_peered":"2024-05-11T12:10:51.178724+0000","last_clean":"2024-05-11T12:10:51.178724+0000","last_became_active":"2024-05-11T12:10:49.840640+0000","last_became_peered":"2024-05-11T12:10:49.840640+0000","last_unstale":"2024-05-11T12:10:51.178724+0000","last_undegraded":"2024-05-11T12:10:51.178724+0000","last_fullsized":"2024-05-11T12:10:51.178724+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:31:28.750933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.830617+0000","last_change":"2024-05-11T12:10:45.767414+0000","last_active":"2024-05-11T12:10:50.830617+0000","last_peered":"2024-05-11T12:10:50.830617+0000","last_clean":"2024-05-11T12:10:50.830617+0000","last_became_active":"2024-05-11T12:10:45.765560+0000","last_became_peered":"2024-05-11T12:10:45.765560+0000","last_unstale":"2024-05-11T12:10:50.830617+0000","last_undegraded":"2024-05-11T12:10:50.830617+0000","last_fullsized":"2024-05-11T12:10:50.830617+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:25:39.934222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.085197+0000","last_change":"2024-05-11T12:10:44.764974+0000","last_active":"2024-05-11T12:10:53.085197+0000","last_peered":"2024-05-11T12:10:53.085197+0000","last_clean":"2024-05-11T12:10:53.085197+0000","last_became_active":"2024-05-11T12:10:44.764692+0000","last_became_peered":"2024-05-11T12:10:44.764692+0000","last_unstale":"2024-05-11T12:10:53.085197+0000","last_undegraded":"2024-05-11T12:10:53.085197+0000","last_fullsized":"2024-05-11T12:10:53.085197+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:27:48.475653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.086845+0000","last_change":"2024-05-11T12:10:49.982086+0000","last_active":"2024-05-11T12:10:53.086845+0000","last_peered":"2024-05-11T12:10:53.086845+0000","last_clean":"2024-05-11T12:10:53.086845+0000","last_became_active":"2024-05-11T12:10:49.981499+0000","last_became_peered":"2024-05-11T12:10:49.981499+0000","last_unstale":"2024-05-11T12:10:53.086845+0000","last_undegraded":"2024-05-11T12:10:53.086845+0000","last_fullsized":"2024-05-11T12:10:53.086845+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:36:11.661440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.178118+0000","last_change":"2024-05-11T12:10:45.780179+0000","last_active":"2024-05-11T12:10:51.178118+0000","last_peered":"2024-05-11T12:10:51.178118+0000","last_clean":"2024-05-11T12:10:51.178118+0000","last_became_active":"2024-05-11T12:10:45.777747+0000","last_became_peered":"2024-05-11T12:10:45.777747+0000","last_unstale":"2024-05-11T12:10:51.178118+0000","last_undegraded":"2024-05-11T12:10:51.178118+0000","last_fullsized":"2024-05-11T12:10:51.178118+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:39:51.243581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.986553+0000","last_change":"2024-05-11T12:10:44.775537+0000","last_active":"2024-05-11T12:10:52.986553+0000","last_peered":"2024-05-11T12:10:52.986553+0000","last_clean":"2024-05-11T12:10:52.986553+0000","last_became_active":"2024-05-11T12:10:44.775159+0000","last_became_peered":"2024-05-11T12:10:44.775159+0000","last_unstale":"2024-05-11T12:10:52.986553+0000","last_undegraded":"2024-05-11T12:10:52.986553+0000","last_fullsized":"2024-05-11T12:10:52.986553+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:39:50.229692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.179022+0000","last_change":"2024-05-11T12:10:49.847192+0000","last_active":"2024-05-11T12:10:51.179022+0000","last_peered":"2024-05-11T12:10:51.179022+0000","last_clean":"2024-05-11T12:10:51.179022+0000","last_became_active":"2024-05-11T12:10:49.846655+0000","last_became_peered":"2024-05-11T12:10:49.846655+0000","last_unstale":"2024-05-11T12:10:51.179022+0000","last_undegraded":"2024-05-11T12:10:51.179022+0000","last_fullsized":"2024-05-11T12:10:51.179022+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:52:55.215724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.057239+0000","last_change":"2024-05-11T12:10:45.788180+0000","last_active":"2024-05-11T12:10:53.057239+0000","last_peered":"2024-05-11T12:10:53.057239+0000","last_clean":"2024-05-11T12:10:53.057239+0000","last_became_active":"2024-05-11T12:10:45.787883+0000","last_became_peered":"2024-05-11T12:10:45.787883+0000","last_unstale":"2024-05-11T12:10:53.057239+0000","last_undegraded":"2024-05-11T12:10:53.057239+0000","last_fullsized":"2024-05-11T12:10:53.057239+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:00:49.028293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.837705+0000","last_change":"2024-05-11T12:10:44.783756+0000","last_active":"2024-05-11T12:10:50.837705+0000","last_peered":"2024-05-11T12:10:50.837705+0000","last_clean":"2024-05-11T12:10:50.837705+0000","last_became_active":"2024-05-11T12:10:44.779442+0000","last_became_peered":"2024-05-11T12:10:44.779442+0000","last_unstale":"2024-05-11T12:10:50.837705+0000","last_undegraded":"2024-05-11T12:10:50.837705+0000","last_fullsized":"2024-05-11T12:10:50.837705+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:12:58.847001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.177959+0000","last_change":"2024-05-11T12:10:49.989122+0000","last_active":"2024-05-11T12:10:51.177959+0000","last_peered":"2024-05-11T12:10:51.177959+0000","last_clean":"2024-05-11T12:10:51.177959+0000","last_became_active":"2024-05-11T12:10:49.988059+0000","last_became_peered":"2024-05-11T12:10:49.988059+0000","last_unstale":"2024-05-11T12:10:51.177959+0000","last_undegraded":"2024-05-11T12:10:51.177959+0000","last_fullsized":"2024-05-11T12:10:51.177959+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:11:21.324557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.832784+0000","last_change":"2024-05-11T12:10:45.780311+0000","last_active":"2024-05-11T12:10:50.832784+0000","last_peered":"2024-05-11T12:10:50.832784+0000","last_clean":"2024-05-11T12:10:50.832784+0000","last_became_active":"2024-05-11T12:10:45.779912+0000","last_became_peered":"2024-05-11T12:10:45.779912+0000","last_unstale":"2024-05-11T12:10:50.832784+0000","last_undegraded":"2024-05-11T12:10:50.832784+0000","last_fullsized":"2024-05-11T12:10:50.832784+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:09:48.719906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.084558+0000","last_change":"2024-05-11T12:10:44.776437+0000","last_active":"2024-05-11T12:10:53.084558+0000","last_peered":"2024-05-11T12:10:53.084558+0000","last_clean":"2024-05-11T12:10:53.084558+0000","last_became_active":"2024-05-11T12:10:44.775585+0000","last_became_peered":"2024-05-11T12:10:44.775585+0000","last_unstale":"2024-05-11T12:10:53.084558+0000","last_undegraded":"2024-05-11T12:10:53.084558+0000","last_fullsized":"2024-05-11T12:10:53.084558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:32:31.312066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.830010+0000","last_change":"2024-05-11T12:10:49.980738+0000","last_active":"2024-05-11T12:10:50.830010+0000","last_peered":"2024-05-11T12:10:50.830010+0000","last_clean":"2024-05-11T12:10:50.830010+0000","last_became_active":"2024-05-11T12:10:49.977869+0000","last_became_peered":"2024-05-11T12:10:49.977869+0000","last_unstale":"2024-05-11T12:10:50.830010+0000","last_undegraded":"2024-05-11T12:10:50.830010+0000","last_fullsized":"2024-05-11T12:10:50.830010+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:30:17.395939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.086118+0000","last_change":"2024-05-11T12:10:45.773559+0000","last_active":"2024-05-11T12:10:53.086118+0000","last_peered":"2024-05-11T12:10:53.086118+0000","last_clean":"2024-05-11T12:10:53.086118+0000","last_became_active":"2024-05-11T12:10:45.773152+0000","last_became_peered":"2024-05-11T12:10:45.773152+0000","last_unstale":"2024-05-11T12:10:53.086118+0000","last_undegraded":"2024-05-11T12:10:53.086118+0000","last_fullsized":"2024-05-11T12:10:53.086118+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:27:46.323505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.988658+0000","last_change":"2024-05-11T12:10:44.779798+0000","last_active":"2024-05-11T12:10:52.988658+0000","last_peered":"2024-05-11T12:10:52.988658+0000","last_clean":"2024-05-11T12:10:52.988658+0000","last_became_active":"2024-05-11T12:10:44.779517+0000","last_became_peered":"2024-05-11T12:10:44.779517+0000","last_unstale":"2024-05-11T12:10:52.988658+0000","last_undegraded":"2024-05-11T12:10:52.988658+0000","last_fullsized":"2024-05-11T12:10:52.988658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:27:48.475653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.055959+0000","last_change":"2024-05-11T12:10:49.977964+0000","last_active":"2024-05-11T12:10:53.055959+0000","last_peered":"2024-05-11T12:10:53.055959+0000","last_clean":"2024-05-11T12:10:53.055959+0000","last_became_active":"2024-05-11T12:10:49.977533+0000","last_became_peered":"2024-05-11T12:10:49.977533+0000","last_unstale":"2024-05-11T12:10:53.055959+0000","last_undegraded":"2024-05-11T12:10:53.055959+0000","last_fullsized":"2024-05-11T12:10:53.055959+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:02:51.578727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.836384+0000","last_change":"2024-05-11T12:10:45.788527+0000","last_active":"2024-05-11T12:10:50.836384+0000","last_peered":"2024-05-11T12:10:50.836384+0000","last_clean":"2024-05-11T12:10:50.836384+0000","last_became_active":"2024-05-11T12:10:45.783632+0000","last_became_peered":"2024-05-11T12:10:45.783632+0000","last_unstale":"2024-05-11T12:10:50.836384+0000","last_undegraded":"2024-05-11T12:10:50.836384+0000","last_fullsized":"2024-05-11T12:10:50.836384+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:52:51.143138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.086299+0000","last_change":"2024-05-11T12:10:44.764150+0000","last_active":"2024-05-11T12:10:53.086299+0000","last_peered":"2024-05-11T12:10:53.086299+0000","last_clean":"2024-05-11T12:10:53.086299+0000","last_became_active":"2024-05-11T12:10:44.763836+0000","last_became_peered":"2024-05-11T12:10:44.763836+0000","last_unstale":"2024-05-11T12:10:53.086299+0000","last_undegraded":"2024-05-11T12:10:53.086299+0000","last_fullsized":"2024-05-11T12:10:53.086299+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:39:50.229692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.085357+0000","last_change":"2024-05-11T12:10:49.841029+0000","last_active":"2024-05-11T12:10:53.085357+0000","last_peered":"2024-05-11T12:10:53.085357+0000","last_clean":"2024-05-11T12:10:53.085357+0000","last_became_active":"2024-05-11T12:10:49.840067+0000","last_became_peered":"2024-05-11T12:10:49.840067+0000","last_unstale":"2024-05-11T12:10:53.085357+0000","last_undegraded":"2024-05-11T12:10:53.085357+0000","last_fullsized":"2024-05-11T12:10:53.085357+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:02:51.578727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.178524+0000","last_change":"2024-05-11T12:10:45.785688+0000","last_active":"2024-05-11T12:10:51.178524+0000","last_peered":"2024-05-11T12:10:51.178524+0000","last_clean":"2024-05-11T12:10:51.178524+0000","last_became_active":"2024-05-11T12:10:45.785267+0000","last_became_peered":"2024-05-11T12:10:45.785267+0000","last_unstale":"2024-05-11T12:10:51.178524+0000","last_undegraded":"2024-05-11T12:10:51.178524+0000","last_fullsized":"2024-05-11T12:10:51.178524+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:50:30.157404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.833626+0000","last_change":"2024-05-11T12:10:44.777030+0000","last_active":"2024-05-11T12:10:50.833626+0000","last_peered":"2024-05-11T12:10:50.833626+0000","last_clean":"2024-05-11T12:10:50.833626+0000","last_became_active":"2024-05-11T12:10:44.776600+0000","last_became_peered":"2024-05-11T12:10:44.776600+0000","last_unstale":"2024-05-11T12:10:50.833626+0000","last_undegraded":"2024-05-11T12:10:50.833626+0000","last_fullsized":"2024-05-11T12:10:50.833626+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:32:31.312066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.086293+0000","last_change":"2024-05-11T12:10:49.843811+0000","last_active":"2024-05-11T12:10:53.086293+0000","last_peered":"2024-05-11T12:10:53.086293+0000","last_clean":"2024-05-11T12:10:53.086293+0000","last_became_active":"2024-05-11T12:10:49.843361+0000","last_became_peered":"2024-05-11T12:10:49.843361+0000","last_unstale":"2024-05-11T12:10:53.086293+0000","last_undegraded":"2024-05-11T12:10:53.086293+0000","last_fullsized":"2024-05-11T12:10:53.086293+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:03:19.587244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.085874+0000","last_change":"2024-05-11T12:10:45.773922+0000","last_active":"2024-05-11T12:10:53.085874+0000","last_peered":"2024-05-11T12:10:53.085874+0000","last_clean":"2024-05-11T12:10:53.085874+0000","last_became_active":"2024-05-11T12:10:45.773584+0000","last_became_peered":"2024-05-11T12:10:45.773584+0000","last_unstale":"2024-05-11T12:10:53.085874+0000","last_undegraded":"2024-05-11T12:10:53.085874+0000","last_fullsized":"2024-05-11T12:10:53.085874+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:25:39.934222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.956061+0000","last_change":"2024-05-11T12:10:44.767869+0000","last_active":"2024-05-11T12:10:50.956061+0000","last_peered":"2024-05-11T12:10:50.956061+0000","last_clean":"2024-05-11T12:10:50.956061+0000","last_became_active":"2024-05-11T12:10:44.766486+0000","last_became_peered":"2024-05-11T12:10:44.766486+0000","last_unstale":"2024-05-11T12:10:50.956061+0000","last_undegraded":"2024-05-11T12:10:50.956061+0000","last_fullsized":"2024-05-11T12:10:50.956061+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:49:38.234192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.087005+0000","last_change":"2024-05-11T12:10:49.837338+0000","last_active":"2024-05-11T12:10:53.087005+0000","last_peered":"2024-05-11T12:10:53.087005+0000","last_clean":"2024-05-11T12:10:53.087005+0000","last_became_active":"2024-05-11T12:10:49.836880+0000","last_became_peered":"2024-05-11T12:10:49.836880+0000","last_unstale":"2024-05-11T12:10:53.087005+0000","last_undegraded":"2024-05-11T12:10:53.087005+0000","last_fullsized":"2024-05-11T12:10:53.087005+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:30:58.400447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.055566+0000","last_change":"2024-05-11T12:10:45.784532+0000","last_active":"2024-05-11T12:10:53.055566+0000","last_peered":"2024-05-11T12:10:53.055566+0000","last_clean":"2024-05-11T12:10:53.055566+0000","last_became_active":"2024-05-11T12:10:45.783747+0000","last_became_peered":"2024-05-11T12:10:45.783747+0000","last_unstale":"2024-05-11T12:10:53.055566+0000","last_undegraded":"2024-05-11T12:10:53.055566+0000","last_fullsized":"2024-05-11T12:10:53.055566+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:33:50.642754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.986822+0000","last_change":"2024-05-11T12:10:44.778140+0000","last_active":"2024-05-11T12:10:52.986822+0000","last_peered":"2024-05-11T12:10:52.986822+0000","last_clean":"2024-05-11T12:10:52.986822+0000","last_became_active":"2024-05-11T12:10:44.777780+0000","last_became_peered":"2024-05-11T12:10:44.777780+0000","last_unstale":"2024-05-11T12:10:52.986822+0000","last_undegraded":"2024-05-11T12:10:52.986822+0000","last_fullsized":"2024-05-11T12:10:52.986822+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:25:35.246177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.057469+0000","last_change":"2024-05-11T12:10:49.984316+0000","last_active":"2024-05-11T12:10:53.057469+0000","last_peered":"2024-05-11T12:10:53.057469+0000","last_clean":"2024-05-11T12:10:53.057469+0000","last_became_active":"2024-05-11T12:10:49.981788+0000","last_became_peered":"2024-05-11T12:10:49.981788+0000","last_unstale":"2024-05-11T12:10:53.057469+0000","last_undegraded":"2024-05-11T12:10:53.057469+0000","last_fullsized":"2024-05-11T12:10:53.057469+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:07:34.303729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.830946+0000","last_change":"2024-05-11T12:10:45.781491+0000","last_active":"2024-05-11T12:10:50.830946+0000","last_peered":"2024-05-11T12:10:50.830946+0000","last_clean":"2024-05-11T12:10:50.830946+0000","last_became_active":"2024-05-11T12:10:45.779112+0000","last_became_peered":"2024-05-11T12:10:45.779112+0000","last_unstale":"2024-05-11T12:10:50.830946+0000","last_undegraded":"2024-05-11T12:10:50.830946+0000","last_fullsized":"2024-05-11T12:10:50.830946+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:00:49.028293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.833659+0000","last_change":"2024-05-11T12:10:44.765494+0000","last_active":"2024-05-11T12:10:50.833659+0000","last_peered":"2024-05-11T12:10:50.833659+0000","last_clean":"2024-05-11T12:10:50.833659+0000","last_became_active":"2024-05-11T12:10:44.764921+0000","last_became_peered":"2024-05-11T12:10:44.764921+0000","last_unstale":"2024-05-11T12:10:50.833659+0000","last_undegraded":"2024-05-11T12:10:50.833659+0000","last_fullsized":"2024-05-11T12:10:50.833659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:11:44.935841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":117,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.986840+0000","last_change":"2024-05-11T12:10:35.066673+0000","last_active":"2024-05-11T12:10:52.986840+0000","last_peered":"2024-05-11T12:10:52.986840+0000","last_clean":"2024-05-11T12:10:52.986840+0000","last_became_active":"2024-05-11T12:10:35.065942+0000","last_became_peered":"2024-05-11T12:10:35.065942+0000","last_unstale":"2024-05-11T12:10:52.986840+0000","last_undegraded":"2024-05-11T12:10:52.986840+0000","last_fullsized":"2024-05-11T12:10:52.986840+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_clean_scrub_stamp":"2024-05-11T12:10:33.601136+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:12:02.133182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.836369+0000","last_change":"2024-05-11T12:10:49.983023+0000","last_active":"2024-05-11T12:10:50.836369+0000","last_peered":"2024-05-11T12:10:50.836369+0000","last_clean":"2024-05-11T12:10:50.836369+0000","last_became_active":"2024-05-11T12:10:49.981575+0000","last_became_peered":"2024-05-11T12:10:49.981575+0000","last_unstale":"2024-05-11T12:10:50.836369+0000","last_undegraded":"2024-05-11T12:10:50.836369+0000","last_fullsized":"2024-05-11T12:10:50.836369+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:52:55.215724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.988305+0000","last_change":"2024-05-11T12:10:45.787709+0000","last_active":"2024-05-11T12:10:52.988305+0000","last_peered":"2024-05-11T12:10:52.988305+0000","last_clean":"2024-05-11T12:10:52.988305+0000","last_became_active":"2024-05-11T12:10:45.787205+0000","last_became_peered":"2024-05-11T12:10:45.787205+0000","last_unstale":"2024-05-11T12:10:52.988305+0000","last_undegraded":"2024-05-11T12:10:52.988305+0000","last_fullsized":"2024-05-11T12:10:52.988305+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:29:58.902743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.956475+0000","last_change":"2024-05-11T12:10:44.777012+0000","last_active":"2024-05-11T12:10:50.956475+0000","last_peered":"2024-05-11T12:10:50.956475+0000","last_clean":"2024-05-11T12:10:50.956475+0000","last_became_active":"2024-05-11T12:10:44.776709+0000","last_became_peered":"2024-05-11T12:10:44.776709+0000","last_unstale":"2024-05-11T12:10:50.956475+0000","last_undegraded":"2024-05-11T12:10:50.956475+0000","last_fullsized":"2024-05-11T12:10:50.956475+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:32:06.207257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.833701+0000","last_change":"2024-05-11T12:10:49.980334+0000","last_active":"2024-05-11T12:10:50.833701+0000","last_peered":"2024-05-11T12:10:50.833701+0000","last_clean":"2024-05-11T12:10:50.833701+0000","last_became_active":"2024-05-11T12:10:49.977819+0000","last_became_peered":"2024-05-11T12:10:49.977819+0000","last_unstale":"2024-05-11T12:10:50.833701+0000","last_undegraded":"2024-05-11T12:10:50.833701+0000","last_fullsized":"2024-05-11T12:10:50.833701+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:30:02.975329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.835272+0000","last_change":"2024-05-11T12:10:45.780131+0000","last_active":"2024-05-11T12:10:50.835272+0000","last_peered":"2024-05-11T12:10:50.835272+0000","last_clean":"2024-05-11T12:10:50.835272+0000","last_became_active":"2024-05-11T12:10:45.779867+0000","last_became_peered":"2024-05-11T12:10:45.779867+0000","last_unstale":"2024-05-11T12:10:50.835272+0000","last_undegraded":"2024-05-11T12:10:50.835272+0000","last_fullsized":"2024-05-11T12:10:50.835272+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:30:54.327861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.084659+0000","last_change":"2024-05-11T12:10:44.775148+0000","last_active":"2024-05-11T12:10:53.084659+0000","last_peered":"2024-05-11T12:10:53.084659+0000","last_clean":"2024-05-11T12:10:53.084659+0000","last_became_active":"2024-05-11T12:10:44.774850+0000","last_became_peered":"2024-05-11T12:10:44.774850+0000","last_unstale":"2024-05-11T12:10:53.084659+0000","last_undegraded":"2024-05-11T12:10:53.084659+0000","last_fullsized":"2024-05-11T12:10:53.084659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:25:35.246177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.056241+0000","last_change":"2024-05-11T12:10:49.842656+0000","last_active":"2024-05-11T12:10:53.056241+0000","last_peered":"2024-05-11T12:10:53.056241+0000","last_clean":"2024-05-11T12:10:53.056241+0000","last_became_active":"2024-05-11T12:10:49.841982+0000","last_became_peered":"2024-05-11T12:10:49.841982+0000","last_unstale":"2024-05-11T12:10:53.056241+0000","last_undegraded":"2024-05-11T12:10:53.056241+0000","last_fullsized":"2024-05-11T12:10:53.056241+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:03:19.587244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.988912+0000","last_change":"2024-05-11T12:10:45.794103+0000","last_active":"2024-05-11T12:10:52.988912+0000","last_peered":"2024-05-11T12:10:52.988912+0000","last_clean":"2024-05-11T12:10:52.988912+0000","last_became_active":"2024-05-11T12:10:45.792080+0000","last_became_peered":"2024-05-11T12:10:45.792080+0000","last_unstale":"2024-05-11T12:10:52.988912+0000","last_undegraded":"2024-05-11T12:10:52.988912+0000","last_fullsized":"2024-05-11T12:10:52.988912+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:30:54.327861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.956469+0000","last_change":"2024-05-11T12:10:44.768108+0000","last_active":"2024-05-11T12:10:50.956469+0000","last_peered":"2024-05-11T12:10:50.956469+0000","last_clean":"2024-05-11T12:10:50.956469+0000","last_became_active":"2024-05-11T12:10:44.766606+0000","last_became_peered":"2024-05-11T12:10:44.766606+0000","last_unstale":"2024-05-11T12:10:50.956469+0000","last_undegraded":"2024-05-11T12:10:50.956469+0000","last_fullsized":"2024-05-11T12:10:50.956469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:02:04.596576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.832239+0000","last_change":"2024-05-11T12:10:49.982446+0000","last_active":"2024-05-11T12:10:50.832239+0000","last_peered":"2024-05-11T12:10:50.832239+0000","last_clean":"2024-05-11T12:10:50.832239+0000","last_became_active":"2024-05-11T12:10:49.981272+0000","last_became_peered":"2024-05-11T12:10:49.981272+0000","last_unstale":"2024-05-11T12:10:50.832239+0000","last_undegraded":"2024-05-11T12:10:50.832239+0000","last_fullsized":"2024-05-11T12:10:50.832239+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:36:40.828180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.988575+0000","last_change":"2024-05-11T12:10:45.786373+0000","last_active":"2024-05-11T12:10:52.988575+0000","last_peered":"2024-05-11T12:10:52.988575+0000","last_clean":"2024-05-11T12:10:52.988575+0000","last_became_active":"2024-05-11T12:10:45.786050+0000","last_became_peered":"2024-05-11T12:10:45.786050+0000","last_unstale":"2024-05-11T12:10:52.988575+0000","last_undegraded":"2024-05-11T12:10:52.988575+0000","last_fullsized":"2024-05-11T12:10:52.988575+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:31:24.678347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.178969+0000","last_change":"2024-05-11T12:10:44.775124+0000","last_active":"2024-05-11T12:10:51.178969+0000","last_peered":"2024-05-11T12:10:51.178969+0000","last_clean":"2024-05-11T12:10:51.178969+0000","last_became_active":"2024-05-11T12:10:44.774810+0000","last_became_peered":"2024-05-11T12:10:44.774810+0000","last_unstale":"2024-05-11T12:10:51.178969+0000","last_undegraded":"2024-05-11T12:10:51.178969+0000","last_fullsized":"2024-05-11T12:10:51.178969+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:44:00.317135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.177431+0000","last_change":"2024-05-11T12:10:49.845241+0000","last_active":"2024-05-11T12:10:51.177431+0000","last_peered":"2024-05-11T12:10:51.177431+0000","last_clean":"2024-05-11T12:10:51.177431+0000","last_became_active":"2024-05-11T12:10:49.844564+0000","last_became_peered":"2024-05-11T12:10:49.844564+0000","last_unstale":"2024-05-11T12:10:51.177431+0000","last_undegraded":"2024-05-11T12:10:51.177431+0000","last_fullsized":"2024-05-11T12:10:51.177431+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:27:50.396091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.987029+0000","last_change":"2024-05-11T12:10:45.789312+0000","last_active":"2024-05-11T12:10:52.987029+0000","last_peered":"2024-05-11T12:10:52.987029+0000","last_clean":"2024-05-11T12:10:52.987029+0000","last_became_active":"2024-05-11T12:10:45.789069+0000","last_became_peered":"2024-05-11T12:10:45.789069+0000","last_unstale":"2024-05-11T12:10:52.987029+0000","last_undegraded":"2024-05-11T12:10:52.987029+0000","last_fullsized":"2024-05-11T12:10:52.987029+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:15:55.637133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.955873+0000","last_change":"2024-05-11T12:10:44.767205+0000","last_active":"2024-05-11T12:10:50.955873+0000","last_peered":"2024-05-11T12:10:50.955873+0000","last_clean":"2024-05-11T12:10:50.955873+0000","last_became_active":"2024-05-11T12:10:44.766855+0000","last_became_peered":"2024-05-11T12:10:44.766855+0000","last_unstale":"2024-05-11T12:10:50.955873+0000","last_undegraded":"2024-05-11T12:10:50.955873+0000","last_fullsized":"2024-05-11T12:10:50.955873+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:10:35.839599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.056975+0000","last_change":"2024-05-11T12:10:49.835689+0000","last_active":"2024-05-11T12:10:53.056975+0000","last_peered":"2024-05-11T12:10:53.056975+0000","last_clean":"2024-05-11T12:10:53.056975+0000","last_became_active":"2024-05-11T12:10:49.835305+0000","last_became_peered":"2024-05-11T12:10:49.835305+0000","last_unstale":"2024-05-11T12:10:53.056975+0000","last_undegraded":"2024-05-11T12:10:53.056975+0000","last_fullsized":"2024-05-11T12:10:53.056975+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:30:58.400447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.833870+0000","last_change":"2024-05-11T12:10:45.788030+0000","last_active":"2024-05-11T12:10:50.833870+0000","last_peered":"2024-05-11T12:10:50.833870+0000","last_clean":"2024-05-11T12:10:50.833870+0000","last_became_active":"2024-05-11T12:10:45.787098+0000","last_became_peered":"2024-05-11T12:10:45.787098+0000","last_unstale":"2024-05-11T12:10:50.833870+0000","last_undegraded":"2024-05-11T12:10:50.833870+0000","last_fullsized":"2024-05-11T12:10:50.833870+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:00:49.028293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.834966+0000","last_change":"2024-05-11T12:10:44.784319+0000","last_active":"2024-05-11T12:10:50.834966+0000","last_peered":"2024-05-11T12:10:50.834966+0000","last_clean":"2024-05-11T12:10:50.834966+0000","last_became_active":"2024-05-11T12:10:44.784027+0000","last_became_peered":"2024-05-11T12:10:44.784027+0000","last_unstale":"2024-05-11T12:10:50.834966+0000","last_undegraded":"2024-05-11T12:10:50.834966+0000","last_fullsized":"2024-05-11T12:10:50.834966+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:50:29.143515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.832654+0000","last_change":"2024-05-11T12:10:49.985187+0000","last_active":"2024-05-11T12:10:50.832654+0000","last_peered":"2024-05-11T12:10:50.832654+0000","last_clean":"2024-05-11T12:10:50.832654+0000","last_became_active":"2024-05-11T12:10:49.983450+0000","last_became_peered":"2024-05-11T12:10:49.983450+0000","last_unstale":"2024-05-11T12:10:50.832654+0000","last_undegraded":"2024-05-11T12:10:50.832654+0000","last_fullsized":"2024-05-11T12:10:50.832654+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:02:13.707329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.987787+0000","last_change":"2024-05-11T12:10:45.789328+0000","last_active":"2024-05-11T12:10:52.987787+0000","last_peered":"2024-05-11T12:10:52.987787+0000","last_clean":"2024-05-11T12:10:52.987787+0000","last_became_active":"2024-05-11T12:10:45.788778+0000","last_became_peered":"2024-05-11T12:10:45.788778+0000","last_unstale":"2024-05-11T12:10:52.987787+0000","last_undegraded":"2024-05-11T12:10:52.987787+0000","last_fullsized":"2024-05-11T12:10:52.987787+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:01:30.356378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.833719+0000","last_change":"2024-05-11T12:10:44.779353+0000","last_active":"2024-05-11T12:10:50.833719+0000","last_peered":"2024-05-11T12:10:50.833719+0000","last_clean":"2024-05-11T12:10:50.833719+0000","last_became_active":"2024-05-11T12:10:44.779078+0000","last_became_peered":"2024-05-11T12:10:44.779078+0000","last_unstale":"2024-05-11T12:10:50.833719+0000","last_undegraded":"2024-05-11T12:10:50.833719+0000","last_fullsized":"2024-05-11T12:10:50.833719+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:25:35.246177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.831951+0000","last_change":"2024-05-11T12:10:49.979416+0000","last_active":"2024-05-11T12:10:50.831951+0000","last_peered":"2024-05-11T12:10:50.831951+0000","last_clean":"2024-05-11T12:10:50.831951+0000","last_became_active":"2024-05-11T12:10:49.978262+0000","last_became_peered":"2024-05-11T12:10:49.978262+0000","last_unstale":"2024-05-11T12:10:50.831951+0000","last_undegraded":"2024-05-11T12:10:50.831951+0000","last_fullsized":"2024-05-11T12:10:50.831951+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:53:47.237894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.056303+0000","last_change":"2024-05-11T12:10:45.784434+0000","last_active":"2024-05-11T12:10:53.056303+0000","last_peered":"2024-05-11T12:10:53.056303+0000","last_clean":"2024-05-11T12:10:53.056303+0000","last_became_active":"2024-05-11T12:10:45.784143+0000","last_became_peered":"2024-05-11T12:10:45.784143+0000","last_unstale":"2024-05-11T12:10:53.056303+0000","last_undegraded":"2024-05-11T12:10:53.056303+0000","last_fullsized":"2024-05-11T12:10:53.056303+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:25:39.934222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.054871+0000","last_change":"2024-05-11T12:10:44.783217+0000","last_active":"2024-05-11T12:10:53.054871+0000","last_peered":"2024-05-11T12:10:53.054871+0000","last_clean":"2024-05-11T12:10:53.054871+0000","last_became_active":"2024-05-11T12:10:44.782942+0000","last_became_peered":"2024-05-11T12:10:44.782942+0000","last_unstale":"2024-05-11T12:10:53.054871+0000","last_undegraded":"2024-05-11T12:10:53.054871+0000","last_fullsized":"2024-05-11T12:10:53.054871+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:14:58.421726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.836406+0000","last_change":"2024-05-11T12:10:49.981126+0000","last_active":"2024-05-11T12:10:50.836406+0000","last_peered":"2024-05-11T12:10:50.836406+0000","last_clean":"2024-05-11T12:10:50.836406+0000","last_became_active":"2024-05-11T12:10:49.978149+0000","last_became_peered":"2024-05-11T12:10:49.978149+0000","last_unstale":"2024-05-11T12:10:50.836406+0000","last_undegraded":"2024-05-11T12:10:50.836406+0000","last_fullsized":"2024-05-11T12:10:50.836406+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:45:08.936896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.988035+0000","last_change":"2024-05-11T12:10:45.791440+0000","last_active":"2024-05-11T12:10:52.988035+0000","last_peered":"2024-05-11T12:10:52.988035+0000","last_clean":"2024-05-11T12:10:52.988035+0000","last_became_active":"2024-05-11T12:10:45.791154+0000","last_became_peered":"2024-05-11T12:10:45.791154+0000","last_unstale":"2024-05-11T12:10:52.988035+0000","last_undegraded":"2024-05-11T12:10:52.988035+0000","last_fullsized":"2024-05-11T12:10:52.988035+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:52:51.143138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'10","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:11:01.126783+0000","last_change":"2024-05-11T12:10:44.773468+0000","last_active":"2024-05-11T12:11:01.126783+0000","last_peered":"2024-05-11T12:11:01.126783+0000","last_clean":"2024-05-11T12:11:01.126783+0000","last_became_active":"2024-05-11T12:10:44.769134+0000","last_became_peered":"2024-05-11T12:10:44.769134+0000","last_unstale":"2024-05-11T12:11:01.126783+0000","last_undegraded":"2024-05-11T12:11:01.126783+0000","last_fullsized":"2024-05-11T12:11:01.126783+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:08:58.049386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.085619+0000","last_change":"2024-05-11T12:10:49.847456+0000","last_active":"2024-05-11T12:10:53.085619+0000","last_peered":"2024-05-11T12:10:53.085619+0000","last_clean":"2024-05-11T12:10:53.085619+0000","last_became_active":"2024-05-11T12:10:49.847009+0000","last_became_peered":"2024-05-11T12:10:49.847009+0000","last_unstale":"2024-05-11T12:10:53.085619+0000","last_undegraded":"2024-05-11T12:10:53.085619+0000","last_fullsized":"2024-05-11T12:10:53.085619+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:52:55.215724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.178813+0000","last_change":"2024-05-11T12:10:45.781361+0000","last_active":"2024-05-11T12:10:51.178813+0000","last_peered":"2024-05-11T12:10:51.178813+0000","last_clean":"2024-05-11T12:10:51.178813+0000","last_became_active":"2024-05-11T12:10:45.781039+0000","last_became_peered":"2024-05-11T12:10:45.781039+0000","last_unstale":"2024-05-11T12:10:51.178813+0000","last_undegraded":"2024-05-11T12:10:51.178813+0000","last_fullsized":"2024-05-11T12:10:51.178813+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:27:49.489542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.084600+0000","last_change":"2024-05-11T12:10:44.770795+0000","last_active":"2024-05-11T12:10:53.084600+0000","last_peered":"2024-05-11T12:10:53.084600+0000","last_clean":"2024-05-11T12:10:53.084600+0000","last_became_active":"2024-05-11T12:10:44.770438+0000","last_became_peered":"2024-05-11T12:10:44.770438+0000","last_unstale":"2024-05-11T12:10:53.084600+0000","last_undegraded":"2024-05-11T12:10:53.084600+0000","last_fullsized":"2024-05-11T12:10:53.084600+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:50:29.143515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.178367+0000","last_change":"2024-05-11T12:10:49.841798+0000","last_active":"2024-05-11T12:10:51.178367+0000","last_peered":"2024-05-11T12:10:51.178367+0000","last_clean":"2024-05-11T12:10:51.178367+0000","last_became_active":"2024-05-11T12:10:49.840773+0000","last_became_peered":"2024-05-11T12:10:49.840773+0000","last_unstale":"2024-05-11T12:10:51.178367+0000","last_undegraded":"2024-05-11T12:10:51.178367+0000","last_fullsized":"2024-05-11T12:10:51.178367+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:01:34.428964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.057219+0000","last_change":"2024-05-11T12:10:45.779106+0000","last_active":"2024-05-11T12:10:53.057219+0000","last_peered":"2024-05-11T12:10:53.057219+0000","last_clean":"2024-05-11T12:10:53.057219+0000","last_became_active":"2024-05-11T12:10:45.778616+0000","last_became_peered":"2024-05-11T12:10:45.778616+0000","last_unstale":"2024-05-11T12:10:53.057219+0000","last_undegraded":"2024-05-11T12:10:53.057219+0000","last_fullsized":"2024-05-11T12:10:53.057219+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:40:33.598752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.837351+0000","last_change":"2024-05-11T12:10:44.781127+0000","last_active":"2024-05-11T12:10:50.837351+0000","last_peered":"2024-05-11T12:10:50.837351+0000","last_clean":"2024-05-11T12:10:50.837351+0000","last_became_active":"2024-05-11T12:10:44.780840+0000","last_became_peered":"2024-05-11T12:10:44.780840+0000","last_unstale":"2024-05-11T12:10:50.837351+0000","last_undegraded":"2024-05-11T12:10:50.837351+0000","last_fullsized":"2024-05-11T12:10:50.837351+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:50:29.143515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.830538+0000","last_change":"2024-05-11T12:10:49.839762+0000","last_active":"2024-05-11T12:10:50.830538+0000","last_peered":"2024-05-11T12:10:50.830538+0000","last_clean":"2024-05-11T12:10:50.830538+0000","last_became_active":"2024-05-11T12:10:49.838073+0000","last_became_peered":"2024-05-11T12:10:49.838073+0000","last_unstale":"2024-05-11T12:10:50.830538+0000","last_undegraded":"2024-05-11T12:10:50.830538+0000","last_fullsized":"2024-05-11T12:10:50.830538+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:36:40.828180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.837108+0000","last_change":"2024-05-11T12:10:45.778187+0000","last_active":"2024-05-11T12:10:50.837108+0000","last_peered":"2024-05-11T12:10:50.837108+0000","last_clean":"2024-05-11T12:10:50.837108+0000","last_became_active":"2024-05-11T12:10:45.776905+0000","last_became_peered":"2024-05-11T12:10:45.776905+0000","last_unstale":"2024-05-11T12:10:50.837108+0000","last_undegraded":"2024-05-11T12:10:50.837108+0000","last_fullsized":"2024-05-11T12:10:50.837108+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:36:07.588854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.837368+0000","last_change":"2024-05-11T12:10:44.778851+0000","last_active":"2024-05-11T12:10:50.837368+0000","last_peered":"2024-05-11T12:10:50.837368+0000","last_clean":"2024-05-11T12:10:50.837368+0000","last_became_active":"2024-05-11T12:10:44.776695+0000","last_became_peered":"2024-05-11T12:10:44.776695+0000","last_unstale":"2024-05-11T12:10:50.837368+0000","last_undegraded":"2024-05-11T12:10:50.837368+0000","last_fullsized":"2024-05-11T12:10:50.837368+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:49:09.103744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.831100+0000","last_change":"2024-05-11T12:10:49.982849+0000","last_active":"2024-05-11T12:10:50.831100+0000","last_peered":"2024-05-11T12:10:50.831100+0000","last_clean":"2024-05-11T12:10:50.831100+0000","last_became_active":"2024-05-11T12:10:49.981894+0000","last_became_peered":"2024-05-11T12:10:49.981894+0000","last_unstale":"2024-05-11T12:10:50.831100+0000","last_undegraded":"2024-05-11T12:10:50.831100+0000","last_fullsized":"2024-05-11T12:10:50.831100+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:45:08.936896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.056935+0000","last_change":"2024-05-11T12:10:49.982163+0000","last_active":"2024-05-11T12:10:53.056935+0000","last_peered":"2024-05-11T12:10:53.056935+0000","last_clean":"2024-05-11T12:10:53.056935+0000","last_became_active":"2024-05-11T12:10:49.981202+0000","last_became_peered":"2024-05-11T12:10:49.981202+0000","last_unstale":"2024-05-11T12:10:53.056935+0000","last_undegraded":"2024-05-11T12:10:53.056935+0000","last_fullsized":"2024-05-11T12:10:53.056935+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:36:11.661440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.085018+0000","last_change":"2024-05-11T12:10:45.772477+0000","last_active":"2024-05-11T12:10:53.085018+0000","last_peered":"2024-05-11T12:10:53.085018+0000","last_clean":"2024-05-11T12:10:53.085018+0000","last_became_active":"2024-05-11T12:10:45.772053+0000","last_became_peered":"2024-05-11T12:10:45.772053+0000","last_unstale":"2024-05-11T12:10:53.085018+0000","last_undegraded":"2024-05-11T12:10:53.085018+0000","last_fullsized":"2024-05-11T12:10:53.085018+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:00:49.028293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.832812+0000","last_change":"2024-05-11T12:10:44.782828+0000","last_active":"2024-05-11T12:10:50.832812+0000","last_peered":"2024-05-11T12:10:50.832812+0000","last_clean":"2024-05-11T12:10:50.832812+0000","last_became_active":"2024-05-11T12:10:44.782412+0000","last_became_peered":"2024-05-11T12:10:44.782412+0000","last_unstale":"2024-05-11T12:10:50.832812+0000","last_undegraded":"2024-05-11T12:10:50.832812+0000","last_fullsized":"2024-05-11T12:10:50.832812+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:39:50.229692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.836608+0000","last_change":"2024-05-11T12:10:49.849632+0000","last_active":"2024-05-11T12:10:50.836608+0000","last_peered":"2024-05-11T12:10:50.836608+0000","last_clean":"2024-05-11T12:10:50.836608+0000","last_became_active":"2024-05-11T12:10:49.849221+0000","last_became_peered":"2024-05-11T12:10:49.849221+0000","last_unstale":"2024-05-11T12:10:50.836608+0000","last_undegraded":"2024-05-11T12:10:50.836608+0000","last_fullsized":"2024-05-11T12:10:50.836608+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:02:51.578727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.830326+0000","last_change":"2024-05-11T12:10:45.784432+0000","last_active":"2024-05-11T12:10:50.830326+0000","last_peered":"2024-05-11T12:10:50.830326+0000","last_clean":"2024-05-11T12:10:50.830326+0000","last_became_active":"2024-05-11T12:10:45.784152+0000","last_became_peered":"2024-05-11T12:10:45.784152+0000","last_unstale":"2024-05-11T12:10:50.830326+0000","last_undegraded":"2024-05-11T12:10:50.830326+0000","last_fullsized":"2024-05-11T12:10:50.830326+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:21:38.211953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.831004+0000","last_change":"2024-05-11T12:10:44.775112+0000","last_active":"2024-05-11T12:10:50.831004+0000","last_peered":"2024-05-11T12:10:50.831004+0000","last_clean":"2024-05-11T12:10:50.831004+0000","last_became_active":"2024-05-11T12:10:44.774758+0000","last_became_peered":"2024-05-11T12:10:44.774758+0000","last_unstale":"2024-05-11T12:10:50.831004+0000","last_undegraded":"2024-05-11T12:10:50.831004+0000","last_fullsized":"2024-05-11T12:10:50.831004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:38:50.526921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.835462+0000","last_change":"2024-05-11T12:10:44.778249+0000","last_active":"2024-05-11T12:10:50.835462+0000","last_peered":"2024-05-11T12:10:50.835462+0000","last_clean":"2024-05-11T12:10:50.835462+0000","last_became_active":"2024-05-11T12:10:44.775251+0000","last_became_peered":"2024-05-11T12:10:44.775251+0000","last_unstale":"2024-05-11T12:10:50.835462+0000","last_undegraded":"2024-05-11T12:10:50.835462+0000","last_fullsized":"2024-05-11T12:10:50.835462+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:11:44.935841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.056459+0000","last_change":"2024-05-11T12:10:45.783894+0000","last_active":"2024-05-11T12:10:53.056459+0000","last_peered":"2024-05-11T12:10:53.056459+0000","last_clean":"2024-05-11T12:10:53.056459+0000","last_became_active":"2024-05-11T12:10:45.783452+0000","last_became_peered":"2024-05-11T12:10:45.783452+0000","last_unstale":"2024-05-11T12:10:53.056459+0000","last_undegraded":"2024-05-11T12:10:53.056459+0000","last_fullsized":"2024-05-11T12:10:53.056459+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:09:48.719906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.956355+0000","last_change":"2024-05-11T12:10:49.841006+0000","last_active":"2024-05-11T12:10:50.956355+0000","last_peered":"2024-05-11T12:10:50.956355+0000","last_clean":"2024-05-11T12:10:50.956355+0000","last_became_active":"2024-05-11T12:10:49.840609+0000","last_became_peered":"2024-05-11T12:10:49.840609+0000","last_unstale":"2024-05-11T12:10:50.956355+0000","last_undegraded":"2024-05-11T12:10:50.956355+0000","last_fullsized":"2024-05-11T12:10:50.956355+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:11:33.092215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.831304+0000","last_change":"2024-05-11T12:10:44.769720+0000","last_active":"2024-05-11T12:10:50.831304+0000","last_peered":"2024-05-11T12:10:50.831304+0000","last_clean":"2024-05-11T12:10:50.831304+0000","last_became_active":"2024-05-11T12:10:44.768923+0000","last_became_peered":"2024-05-11T12:10:44.768923+0000","last_unstale":"2024-05-11T12:10:50.831304+0000","last_undegraded":"2024-05-11T12:10:50.831304+0000","last_fullsized":"2024-05-11T12:10:50.831304+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:04:49.439739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.177838+0000","last_change":"2024-05-11T12:10:45.776362+0000","last_active":"2024-05-11T12:10:51.177838+0000","last_peered":"2024-05-11T12:10:51.177838+0000","last_clean":"2024-05-11T12:10:51.177838+0000","last_became_active":"2024-05-11T12:10:45.773081+0000","last_became_peered":"2024-05-11T12:10:45.773081+0000","last_unstale":"2024-05-11T12:10:51.177838+0000","last_undegraded":"2024-05-11T12:10:51.177838+0000","last_fullsized":"2024-05-11T12:10:51.177838+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:25:36.260066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.832413+0000","last_change":"2024-05-11T12:10:49.837438+0000","last_active":"2024-05-11T12:10:50.832413+0000","last_peered":"2024-05-11T12:10:50.832413+0000","last_clean":"2024-05-11T12:10:50.832413+0000","last_became_active":"2024-05-11T12:10:49.836966+0000","last_became_peered":"2024-05-11T12:10:49.836966+0000","last_unstale":"2024-05-11T12:10:50.832413+0000","last_undegraded":"2024-05-11T12:10:50.832413+0000","last_fullsized":"2024-05-11T12:10:50.832413+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:16:26.266475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.832119+0000","last_change":"2024-05-11T12:10:45.792181+0000","last_active":"2024-05-11T12:10:50.832119+0000","last_peered":"2024-05-11T12:10:50.832119+0000","last_clean":"2024-05-11T12:10:50.832119+0000","last_became_active":"2024-05-11T12:10:45.791827+0000","last_became_peered":"2024-05-11T12:10:45.791827+0000","last_unstale":"2024-05-11T12:10:50.832119+0000","last_undegraded":"2024-05-11T12:10:50.832119+0000","last_fullsized":"2024-05-11T12:10:50.832119+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:25:39.934222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.956054+0000","last_change":"2024-05-11T12:10:45.786514+0000","last_active":"2024-05-11T12:10:50.956054+0000","last_peered":"2024-05-11T12:10:50.956054+0000","last_clean":"2024-05-11T12:10:50.956054+0000","last_became_active":"2024-05-11T12:10:45.786170+0000","last_became_peered":"2024-05-11T12:10:45.786170+0000","last_unstale":"2024-05-11T12:10:50.956054+0000","last_undegraded":"2024-05-11T12:10:50.956054+0000","last_fullsized":"2024-05-11T12:10:50.956054+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:07:42.553060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.833366+0000","last_change":"2024-05-11T12:10:45.764626+0000","last_active":"2024-05-11T12:10:50.833366+0000","last_peered":"2024-05-11T12:10:50.833366+0000","last_clean":"2024-05-11T12:10:50.833366+0000","last_became_active":"2024-05-11T12:10:45.764234+0000","last_became_peered":"2024-05-11T12:10:45.764234+0000","last_unstale":"2024-05-11T12:10:50.833366+0000","last_undegraded":"2024-05-11T12:10:50.833366+0000","last_fullsized":"2024-05-11T12:10:50.833366+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:31:24.678347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.056344+0000","last_change":"2024-05-11T12:10:45.787822+0000","last_active":"2024-05-11T12:10:53.056344+0000","last_peered":"2024-05-11T12:10:53.056344+0000","last_clean":"2024-05-11T12:10:53.056344+0000","last_became_active":"2024-05-11T12:10:45.787503+0000","last_became_peered":"2024-05-11T12:10:45.787503+0000","last_unstale":"2024-05-11T12:10:53.056344+0000","last_undegraded":"2024-05-11T12:10:53.056344+0000","last_fullsized":"2024-05-11T12:10:53.056344+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:30:11.665446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.832309+0000","last_change":"2024-05-11T12:10:45.769648+0000","last_active":"2024-05-11T12:10:50.832309+0000","last_peered":"2024-05-11T12:10:50.832309+0000","last_clean":"2024-05-11T12:10:50.832309+0000","last_became_active":"2024-05-11T12:10:45.769168+0000","last_became_peered":"2024-05-11T12:10:45.769168+0000","last_unstale":"2024-05-11T12:10:50.832309+0000","last_undegraded":"2024-05-11T12:10:50.832309+0000","last_fullsized":"2024-05-11T12:10:50.832309+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:40:33.598752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.086161+0000","last_change":"2024-05-11T12:10:45.773070+0000","last_active":"2024-05-11T12:10:53.086161+0000","last_peered":"2024-05-11T12:10:53.086161+0000","last_clean":"2024-05-11T12:10:53.086161+0000","last_became_active":"2024-05-11T12:10:45.772734+0000","last_became_peered":"2024-05-11T12:10:45.772734+0000","last_unstale":"2024-05-11T12:10:53.086161+0000","last_undegraded":"2024-05-11T12:10:53.086161+0000","last_fullsized":"2024-05-11T12:10:53.086161+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:52:36.650847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.836911+0000","last_change":"2024-05-11T12:10:45.776117+0000","last_active":"2024-05-11T12:10:50.836911+0000","last_peered":"2024-05-11T12:10:50.836911+0000","last_clean":"2024-05-11T12:10:50.836911+0000","last_became_active":"2024-05-11T12:10:45.775042+0000","last_became_peered":"2024-05-11T12:10:45.775042+0000","last_unstale":"2024-05-11T12:10:50.836911+0000","last_undegraded":"2024-05-11T12:10:50.836911+0000","last_fullsized":"2024-05-11T12:10:50.836911+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:40:33.598752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.832667+0000","last_change":"2024-05-11T12:10:45.767897+0000","last_active":"2024-05-11T12:10:50.832667+0000","last_peered":"2024-05-11T12:10:50.832667+0000","last_clean":"2024-05-11T12:10:50.832667+0000","last_became_active":"2024-05-11T12:10:45.767619+0000","last_became_peered":"2024-05-11T12:10:45.767619+0000","last_unstale":"2024-05-11T12:10:50.832667+0000","last_undegraded":"2024-05-11T12:10:50.832667+0000","last_fullsized":"2024-05-11T12:10:50.832667+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:27:46.323505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.832683+0000","last_change":"2024-05-11T12:10:45.763701+0000","last_active":"2024-05-11T12:10:50.832683+0000","last_peered":"2024-05-11T12:10:50.832683+0000","last_clean":"2024-05-11T12:10:50.832683+0000","last_became_active":"2024-05-11T12:10:45.763260+0000","last_became_peered":"2024-05-11T12:10:45.763260+0000","last_unstale":"2024-05-11T12:10:50.832683+0000","last_undegraded":"2024-05-11T12:10:50.832683+0000","last_fullsized":"2024-05-11T12:10:50.832683+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:11:17.251971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.830765+0000","last_change":"2024-05-11T12:10:45.767500+0000","last_active":"2024-05-11T12:10:50.830765+0000","last_peered":"2024-05-11T12:10:50.830765+0000","last_clean":"2024-05-11T12:10:50.830765+0000","last_became_active":"2024-05-11T12:10:45.767137+0000","last_became_peered":"2024-05-11T12:10:45.767137+0000","last_unstale":"2024-05-11T12:10:50.830765+0000","last_undegraded":"2024-05-11T12:10:50.830765+0000","last_fullsized":"2024-05-11T12:10:50.830765+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:52:36.650847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.833200+0000","last_change":"2024-05-11T12:10:45.779266+0000","last_active":"2024-05-11T12:10:50.833200+0000","last_peered":"2024-05-11T12:10:50.833200+0000","last_clean":"2024-05-11T12:10:50.833200+0000","last_became_active":"2024-05-11T12:10:45.778968+0000","last_became_peered":"2024-05-11T12:10:45.778968+0000","last_unstale":"2024-05-11T12:10:50.833200+0000","last_undegraded":"2024-05-11T12:10:50.833200+0000","last_fullsized":"2024-05-11T12:10:50.833200+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:40:33.598752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.056067+0000","last_change":"2024-05-11T12:10:45.793850+0000","last_active":"2024-05-11T12:10:53.056067+0000","last_peered":"2024-05-11T12:10:53.056067+0000","last_clean":"2024-05-11T12:10:53.056067+0000","last_became_active":"2024-05-11T12:10:45.793503+0000","last_became_peered":"2024-05-11T12:10:45.793503+0000","last_unstale":"2024-05-11T12:10:53.056067+0000","last_undegraded":"2024-05-11T12:10:53.056067+0000","last_fullsized":"2024-05-11T12:10:53.056067+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:21:38.211953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.836150+0000","last_change":"2024-05-11T12:10:45.776958+0000","last_active":"2024-05-11T12:10:50.836150+0000","last_peered":"2024-05-11T12:10:50.836150+0000","last_clean":"2024-05-11T12:10:50.836150+0000","last_became_active":"2024-05-11T12:10:45.770839+0000","last_became_peered":"2024-05-11T12:10:45.770839+0000","last_unstale":"2024-05-11T12:10:50.836150+0000","last_undegraded":"2024-05-11T12:10:50.836150+0000","last_fullsized":"2024-05-11T12:10:50.836150+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:52:36.650847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.988652+0000","last_change":"2024-05-11T12:10:45.784456+0000","last_active":"2024-05-11T12:10:52.988652+0000","last_peered":"2024-05-11T12:10:52.988652+0000","last_clean":"2024-05-11T12:10:52.988652+0000","last_became_active":"2024-05-11T12:10:45.784149+0000","last_became_peered":"2024-05-11T12:10:45.784149+0000","last_unstale":"2024-05-11T12:10:52.988652+0000","last_undegraded":"2024-05-11T12:10:52.988652+0000","last_fullsized":"2024-05-11T12:10:52.988652+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:36:07.588854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.956434+0000","last_change":"2024-05-11T12:10:45.778083+0000","last_active":"2024-05-11T12:10:50.956434+0000","last_peered":"2024-05-11T12:10:50.956434+0000","last_clean":"2024-05-11T12:10:50.956434+0000","last_became_active":"2024-05-11T12:10:45.777758+0000","last_became_peered":"2024-05-11T12:10:45.777758+0000","last_unstale":"2024-05-11T12:10:50.956434+0000","last_undegraded":"2024-05-11T12:10:50.956434+0000","last_fullsized":"2024-05-11T12:10:50.956434+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:38:01.288328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.838005+0000","last_change":"2024-05-11T12:10:45.780997+0000","last_active":"2024-05-11T12:10:50.838005+0000","last_peered":"2024-05-11T12:10:50.838005+0000","last_clean":"2024-05-11T12:10:50.838005+0000","last_became_active":"2024-05-11T12:10:45.780744+0000","last_became_peered":"2024-05-11T12:10:45.780744+0000","last_unstale":"2024-05-11T12:10:50.838005+0000","last_undegraded":"2024-05-11T12:10:50.838005+0000","last_fullsized":"2024-05-11T12:10:50.838005+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:03:15.514658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.832468+0000","last_change":"2024-05-11T12:10:45.790012+0000","last_active":"2024-05-11T12:10:50.832468+0000","last_peered":"2024-05-11T12:10:50.832468+0000","last_clean":"2024-05-11T12:10:50.832468+0000","last_became_active":"2024-05-11T12:10:45.786296+0000","last_became_peered":"2024-05-11T12:10:45.786296+0000","last_unstale":"2024-05-11T12:10:50.832468+0000","last_undegraded":"2024-05-11T12:10:50.832468+0000","last_fullsized":"2024-05-11T12:10:50.832468+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:09:48.719906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.085092+0000","last_change":"2024-05-11T12:10:45.766294+0000","last_active":"2024-05-11T12:10:53.085092+0000","last_peered":"2024-05-11T12:10:53.085092+0000","last_clean":"2024-05-11T12:10:53.085092+0000","last_became_active":"2024-05-11T12:10:45.765946+0000","last_became_peered":"2024-05-11T12:10:45.765946+0000","last_unstale":"2024-05-11T12:10:53.085092+0000","last_undegraded":"2024-05-11T12:10:53.085092+0000","last_fullsized":"2024-05-11T12:10:53.085092+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:30:11.665446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.056206+0000","last_change":"2024-05-11T12:10:45.777473+0000","last_active":"2024-05-11T12:10:53.056206+0000","last_peered":"2024-05-11T12:10:53.056206+0000","last_clean":"2024-05-11T12:10:53.056206+0000","last_became_active":"2024-05-11T12:10:45.777140+0000","last_became_peered":"2024-05-11T12:10:45.777140+0000","last_unstale":"2024-05-11T12:10:53.056206+0000","last_undegraded":"2024-05-11T12:10:53.056206+0000","last_fullsized":"2024-05-11T12:10:53.056206+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:12:59.860890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.830966+0000","last_change":"2024-05-11T12:10:45.779099+0000","last_active":"2024-05-11T12:10:50.830966+0000","last_peered":"2024-05-11T12:10:50.830966+0000","last_clean":"2024-05-11T12:10:50.830966+0000","last_became_active":"2024-05-11T12:10:45.767177+0000","last_became_peered":"2024-05-11T12:10:45.767177+0000","last_unstale":"2024-05-11T12:10:50.830966+0000","last_undegraded":"2024-05-11T12:10:50.830966+0000","last_fullsized":"2024-05-11T12:10:50.830966+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:30:11.665446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.178521+0000","last_change":"2024-05-11T12:10:45.781140+0000","last_active":"2024-05-11T12:10:51.178521+0000","last_peered":"2024-05-11T12:10:51.178521+0000","last_clean":"2024-05-11T12:10:51.178521+0000","last_became_active":"2024-05-11T12:10:45.780390+0000","last_became_peered":"2024-05-11T12:10:45.780390+0000","last_unstale":"2024-05-11T12:10:51.178521+0000","last_undegraded":"2024-05-11T12:10:51.178521+0000","last_fullsized":"2024-05-11T12:10:51.178521+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:32:32.325955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.836702+0000","last_change":"2024-05-11T12:10:45.772180+0000","last_active":"2024-05-11T12:10:50.836702+0000","last_peered":"2024-05-11T12:10:50.836702+0000","last_clean":"2024-05-11T12:10:50.836702+0000","last_became_active":"2024-05-11T12:10:45.771785+0000","last_became_peered":"2024-05-11T12:10:45.771785+0000","last_unstale":"2024-05-11T12:10:50.836702+0000","last_undegraded":"2024-05-11T12:10:50.836702+0000","last_fullsized":"2024-05-11T12:10:50.836702+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:27:46.323505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.836913+0000","last_change":"2024-05-11T12:10:45.773903+0000","last_active":"2024-05-11T12:10:50.836913+0000","last_peered":"2024-05-11T12:10:50.836913+0000","last_clean":"2024-05-11T12:10:50.836913+0000","last_became_active":"2024-05-11T12:10:45.773639+0000","last_became_peered":"2024-05-11T12:10:45.773639+0000","last_unstale":"2024-05-11T12:10:50.836913+0000","last_undegraded":"2024-05-11T12:10:50.836913+0000","last_fullsized":"2024-05-11T12:10:50.836913+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:09:48.719906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.988774+0000","last_change":"2024-05-11T12:10:45.794802+0000","last_active":"2024-05-11T12:10:52.988774+0000","last_peered":"2024-05-11T12:10:52.988774+0000","last_clean":"2024-05-11T12:10:52.988774+0000","last_became_active":"2024-05-11T12:10:45.794505+0000","last_became_peered":"2024-05-11T12:10:45.794505+0000","last_unstale":"2024-05-11T12:10:52.988774+0000","last_undegraded":"2024-05-11T12:10:52.988774+0000","last_fullsized":"2024-05-11T12:10:52.988774+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:03:15.514658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.830033+0000","last_change":"2024-05-11T12:10:45.764551+0000","last_active":"2024-05-11T12:10:50.830033+0000","last_peered":"2024-05-11T12:10:50.830033+0000","last_clean":"2024-05-11T12:10:50.830033+0000","last_became_active":"2024-05-11T12:10:45.763049+0000","last_became_peered":"2024-05-11T12:10:45.763049+0000","last_unstale":"2024-05-11T12:10:50.830033+0000","last_undegraded":"2024-05-11T12:10:50.830033+0000","last_fullsized":"2024-05-11T12:10:50.830033+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:11:17.251971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.986814+0000","last_change":"2024-05-11T12:10:45.785129+0000","last_active":"2024-05-11T12:10:52.986814+0000","last_peered":"2024-05-11T12:10:52.986814+0000","last_clean":"2024-05-11T12:10:52.986814+0000","last_became_active":"2024-05-11T12:10:45.784827+0000","last_became_peered":"2024-05-11T12:10:45.784827+0000","last_unstale":"2024-05-11T12:10:52.986814+0000","last_undegraded":"2024-05-11T12:10:52.986814+0000","last_fullsized":"2024-05-11T12:10:52.986814+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:40:33.598752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.833132+0000","last_change":"2024-05-11T12:10:45.785406+0000","last_active":"2024-05-11T12:10:50.833132+0000","last_peered":"2024-05-11T12:10:50.833132+0000","last_clean":"2024-05-11T12:10:50.833132+0000","last_became_active":"2024-05-11T12:10:45.785083+0000","last_became_peered":"2024-05-11T12:10:45.785083+0000","last_unstale":"2024-05-11T12:10:50.833132+0000","last_undegraded":"2024-05-11T12:10:50.833132+0000","last_fullsized":"2024-05-11T12:10:50.833132+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:15:55.637133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.987595+0000","last_change":"2024-05-11T12:10:45.779636+0000","last_active":"2024-05-11T12:10:52.987595+0000","last_peered":"2024-05-11T12:10:52.987595+0000","last_clean":"2024-05-11T12:10:52.987595+0000","last_became_active":"2024-05-11T12:10:45.779327+0000","last_became_peered":"2024-05-11T12:10:45.779327+0000","last_unstale":"2024-05-11T12:10:52.987595+0000","last_undegraded":"2024-05-11T12:10:52.987595+0000","last_fullsized":"2024-05-11T12:10:52.987595+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:11:17.251971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.834940+0000","last_change":"2024-05-11T12:10:45.782057+0000","last_active":"2024-05-11T12:10:50.834940+0000","last_peered":"2024-05-11T12:10:50.834940+0000","last_clean":"2024-05-11T12:10:50.834940+0000","last_became_active":"2024-05-11T12:10:45.781008+0000","last_became_peered":"2024-05-11T12:10:45.781008+0000","last_unstale":"2024-05-11T12:10:50.834940+0000","last_undegraded":"2024-05-11T12:10:50.834940+0000","last_fullsized":"2024-05-11T12:10:50.834940+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:01:30.356378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.056602+0000","last_change":"2024-05-11T12:10:45.783937+0000","last_active":"2024-05-11T12:10:53.056602+0000","last_peered":"2024-05-11T12:10:53.056602+0000","last_clean":"2024-05-11T12:10:53.056602+0000","last_became_active":"2024-05-11T12:10:45.783553+0000","last_became_peered":"2024-05-11T12:10:45.783553+0000","last_unstale":"2024-05-11T12:10:53.056602+0000","last_undegraded":"2024-05-11T12:10:53.056602+0000","last_fullsized":"2024-05-11T12:10:53.056602+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:27:46.323505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.086881+0000","last_change":"2024-05-11T12:10:45.786860+0000","last_active":"2024-05-11T12:10:53.086881+0000","last_peered":"2024-05-11T12:10:53.086881+0000","last_clean":"2024-05-11T12:10:53.086881+0000","last_became_active":"2024-05-11T12:10:45.785741+0000","last_became_peered":"2024-05-11T12:10:45.785741+0000","last_unstale":"2024-05-11T12:10:53.086881+0000","last_undegraded":"2024-05-11T12:10:53.086881+0000","last_fullsized":"2024-05-11T12:10:53.086881+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:09:48.719906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.988637+0000","last_change":"2024-05-11T12:10:45.794362+0000","last_active":"2024-05-11T12:10:52.988637+0000","last_peered":"2024-05-11T12:10:52.988637+0000","last_clean":"2024-05-11T12:10:52.988637+0000","last_became_active":"2024-05-11T12:10:45.793845+0000","last_became_peered":"2024-05-11T12:10:45.793845+0000","last_unstale":"2024-05-11T12:10:52.988637+0000","last_undegraded":"2024-05-11T12:10:52.988637+0000","last_fullsized":"2024-05-11T12:10:52.988637+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:02:47.506141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476747,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":876544,"data_stored":700107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476748,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":112999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476747,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":282624,"data_stored":109247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476749,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":290816,"data_stored":109283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509452,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":112999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509453,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":109317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509452,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":884736,"data_stored":700163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542156,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":109157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T12:11:16.916 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T12:11:17.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.145+0000 7fca7fe00640 1 Processor -- start 2024-05-11T12:11:17.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.145+0000 7fca7fe00640 1 -- start start 2024-05-11T12:11:17.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.147+0000 7fca7fe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fca80145b00 0x7fca8006f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:17.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.147+0000 7fca7fe00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fca8006c660 0x7fca8006ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:17.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.147+0000 7fca7fe00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca8006fbc0 con 0x7fca80146770 2024-05-11T12:11:17.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.147+0000 7fca7fe00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fca8006cfd0 con 0x7fca8006c660 2024-05-11T12:11:17.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.147+0000 7fca7fe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fca800b4520 con 0x7fca80145b00 2024-05-11T12:11:17.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.147+0000 7fca7f400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fca8006c660 0x7fca8006ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:17.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.147+0000 7fca7f400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fca8006c660 0x7fca8006ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41848/0 (socket says 172.21.15.167:41848) 2024-05-11T12:11:17.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.147+0000 7fca7f400640 1 -- 172.21.15.167:0/196369186 learned_addr learned my addr 172.21.15.167:0/196369186 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:17.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.147+0000 7fca7ea00640 1 --2- 172.21.15.167:0/196369186 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fca80145b00 0x7fca8006f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:17.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.147+0000 7fca7f400640 1 -- 172.21.15.167:0/196369186 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fca80145b00 msgr2=0x7fca8006f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:17.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.147+0000 7fca7f400640 1 --2- 172.21.15.167:0/196369186 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fca80145b00 0x7fca8006f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:17.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.147+0000 7fca7f400640 1 -- 172.21.15.167:0/196369186 >> v1:172.21.15.204:6789/0 conn(0x7fca80146770 legacy=0x7fca800ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:17.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.148+0000 7fca7f400640 1 -- 172.21.15.167:0/196369186 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca800b4800 con 0x7fca8006c660 2024-05-11T12:11:17.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.149+0000 7fca7f400640 1 --2- 172.21.15.167:0/196369186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca8006c660 0x7fca8006ca90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fca78009e50 tx=0x7fca7802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d01bbcb55998d2ec server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:17.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.149+0000 7fca7d600640 1 -- 172.21.15.167:0/196369186 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca78036070 con 0x7fca8006c660 2024-05-11T12:11:17.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.149+0000 7fca7d600640 1 -- 172.21.15.167:0/196369186 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca78031600 con 0x7fca8006c660 2024-05-11T12:11:17.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.149+0000 7fca7d600640 1 -- 172.21.15.167:0/196369186 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca780319d0 con 0x7fca8006c660 2024-05-11T12:11:17.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.149+0000 7fca7fe00640 1 -- 172.21.15.167:0/196369186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca8006c660 msgr2=0x7fca8006ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:17.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.149+0000 7fca7fe00640 1 --2- 172.21.15.167:0/196369186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca8006c660 0x7fca8006ca90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fca78009e50 tx=0x7fca7802a680 comp rx=0 tx=0).stop 2024-05-11T12:11:17.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.150+0000 7fca7fe00640 1 -- 172.21.15.167:0/196369186 shutdown_connections 2024-05-11T12:11:17.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.150+0000 7fca7fe00640 1 --2- 172.21.15.167:0/196369186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca8006c660 0x7fca8006ca90 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:17.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.150+0000 7fca7fe00640 1 --2- 172.21.15.167:0/196369186 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fca80145b00 0x7fca8006f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:17.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.150+0000 7fca7fe00640 1 -- 172.21.15.167:0/196369186 >> 172.21.15.167:0/196369186 conn(0x7fca800b3460 msgr2=0x7fca800d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:17.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.150+0000 7fca7fe00640 1 -- 172.21.15.167:0/196369186 shutdown_connections 2024-05-11T12:11:17.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.150+0000 7fca7fe00640 1 -- 172.21.15.167:0/196369186 wait complete. 2024-05-11T12:11:17.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.151+0000 7fca7fe00640 1 Processor -- start 2024-05-11T12:11:17.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.152+0000 7fca7fe00640 1 -- start start 2024-05-11T12:11:17.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.152+0000 7fca7fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca8006c660 0x7fca8015d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:17.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.152+0000 7fca7fe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fca80145b00 0x7fca80144940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:17.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.153+0000 7fca7fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca80146770 0x7fca800731f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:17.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.153+0000 7fca7fe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fca80145ee0 con 0x7fca80146770 2024-05-11T12:11:17.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.153+0000 7fca7fe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fca800cf0e0 con 0x7fca8006c660 2024-05-11T12:11:17.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.153+0000 7fca7fe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fca8006f800 con 0x7fca80145b00 2024-05-11T12:11:17.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.153+0000 7fca7f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca80146770 0x7fca800731f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:17.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.153+0000 7fca7f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca80146770 0x7fca800731f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41858/0 (socket says 172.21.15.167:41858) 2024-05-11T12:11:17.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.153+0000 7fca7f400640 1 -- 172.21.15.167:0/2794093308 learned_addr learned my addr 172.21.15.167:0/2794093308 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:17.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.153+0000 7fca7f400640 1 -- 172.21.15.167:0/2794093308 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fca80145b00 msgr2=0x7fca80144940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:17.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.153+0000 7fca7e000640 1 --2- 172.21.15.167:0/2794093308 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fca80145b00 0x7fca80144940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:17.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.153+0000 7fca7f400640 1 --2- 172.21.15.167:0/2794093308 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fca80145b00 0x7fca80144940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:17.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.153+0000 7fca7f400640 1 -- 172.21.15.167:0/2794093308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca8006c660 msgr2=0x7fca8015d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:17.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.153+0000 7fca7f400640 1 --2- 172.21.15.167:0/2794093308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca8006c660 0x7fca8015d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:17.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.153+0000 7fca7f400640 1 -- 172.21.15.167:0/2794093308 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca801438a0 con 0x7fca80146770 2024-05-11T12:11:17.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.154+0000 7fca7f400640 1 --2- 172.21.15.167:0/2794093308 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca80146770 0x7fca800731f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fca78009f70 tx=0x7fca78004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:17.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.154+0000 7fca76a00640 1 -- 172.21.15.167:0/2794093308 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7803f020 con 0x7fca80146770 2024-05-11T12:11:17.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.154+0000 7fca76a00640 1 -- 172.21.15.167:0/2794093308 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca78037040 con 0x7fca80146770 2024-05-11T12:11:17.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.154+0000 7fca76a00640 1 -- 172.21.15.167:0/2794093308 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca78036070 con 0x7fca80146770 2024-05-11T12:11:17.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.154+0000 7fca7fe00640 1 -- 172.21.15.167:0/2794093308 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca80143b80 con 0x7fca80146770 2024-05-11T12:11:17.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.154+0000 7fca7fe00640 1 -- 172.21.15.167:0/2794093308 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca80144140 con 0x7fca80146770 2024-05-11T12:11:17.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.155+0000 7fca7fe00640 1 -- 172.21.15.167:0/2794093308 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca800694b0 con 0x7fca80146770 2024-05-11T12:11:17.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.156+0000 7fca76a00640 1 -- 172.21.15.167:0/2794093308 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fca78042020 con 0x7fca80146770 2024-05-11T12:11:17.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.157+0000 7fca76a00640 1 --2- 172.21.15.167:0/2794093308 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fca600762d0 0x7fca60078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:17.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.157+0000 7fca76a00640 1 -- 172.21.15.167:0/2794093308 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fca780b7e70 con 0x7fca80146770 2024-05-11T12:11:17.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.157+0000 7fca7ea00640 1 --2- 172.21.15.167:0/2794093308 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fca600762d0 0x7fca60078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:17.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.158+0000 7fca7ea00640 1 --2- 172.21.15.167:0/2794093308 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fca600762d0 0x7fca60078790 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fca700098f0 tx=0x7fca70008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:17.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.162+0000 7fca76a00640 1 -- 172.21.15.167:0/2794093308 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fca78049030 con 0x7fca80146770 2024-05-11T12:11:17.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.352+0000 7fca7fe00640 1 -- 172.21.15.167:0/2794093308 --> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fca800b01e0 con 0x7fca600762d0 2024-05-11T12:11:17.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.375+0000 7fca76a00640 1 -- 172.21.15.167:0/2794093308 <== mgr.4106 v2:172.21.15.167:6832/578352559 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332524 (secure 0 0 0) 0x7fca800b01e0 con 0x7fca600762d0 2024-05-11T12:11:17.379 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:11:17.383 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2024-05-11T12:11:17.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.384+0000 7fca7fe00640 1 -- 172.21.15.167:0/2794093308 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fca600762d0 msgr2=0x7fca60078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:17.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.384+0000 7fca7fe00640 1 --2- 172.21.15.167:0/2794093308 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fca600762d0 0x7fca60078790 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fca700098f0 tx=0x7fca70008040 comp rx=0 tx=0).stop 2024-05-11T12:11:17.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.384+0000 7fca7fe00640 1 -- 172.21.15.167:0/2794093308 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca80146770 msgr2=0x7fca800731f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:17.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.384+0000 7fca7fe00640 1 --2- 172.21.15.167:0/2794093308 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca80146770 0x7fca800731f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fca78009f70 tx=0x7fca78004a40 comp rx=0 tx=0).stop 2024-05-11T12:11:17.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.384+0000 7fca7fe00640 1 -- 172.21.15.167:0/2794093308 shutdown_connections 2024-05-11T12:11:17.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.384+0000 7fca7fe00640 1 --2- 172.21.15.167:0/2794093308 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca80146770 0x7fca800731f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:17.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.384+0000 7fca7fe00640 1 --2- 172.21.15.167:0/2794093308 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fca80145b00 0x7fca80144940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:17.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.384+0000 7fca7fe00640 1 --2- 172.21.15.167:0/2794093308 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fca600762d0 0x7fca60078790 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:17.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.384+0000 7fca7fe00640 1 --2- 172.21.15.167:0/2794093308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca8006c660 0x7fca8015d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:17.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.384+0000 7fca7fe00640 1 -- 172.21.15.167:0/2794093308 >> 172.21.15.167:0/2794093308 conn(0x7fca800b3460 msgr2=0x7fca80146060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:17.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.384+0000 7fca7fe00640 1 -- 172.21.15.167:0/2794093308 shutdown_connections 2024-05-11T12:11:17.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.385+0000 7fca7fe00640 1 -- 172.21.15.167:0/2794093308 wait complete. 2024-05-11T12:11:17.404 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":43,"stamp":"2024-05-11T12:11:17.241890+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":150,"num_write_kb":1647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219560,"kb_used_data":3400,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755160,"statfs":{"total":773094113280,"available":772869283840,"internally_reserved":0,"allocated":3481600,"data_stored":2063272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005660"},"pg_stats":[{"pgid":"2.1f","version":"22'4","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.057257+0000","last_change":"2024-05-11T12:10:44.771802+0000","last_active":"2024-05-11T12:10:53.057257+0000","last_peered":"2024-05-11T12:10:53.057257+0000","last_clean":"2024-05-11T12:10:53.057257+0000","last_became_active":"2024-05-11T12:10:44.771511+0000","last_became_peered":"2024-05-11T12:10:44.771511+0000","last_unstale":"2024-05-11T12:10:53.057257+0000","last_undegraded":"2024-05-11T12:10:53.057257+0000","last_fullsized":"2024-05-11T12:10:53.057257+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:38:50.526921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.832716+0000","last_change":"2024-05-11T12:10:45.765221+0000","last_active":"2024-05-11T12:10:50.832716+0000","last_peered":"2024-05-11T12:10:50.832716+0000","last_clean":"2024-05-11T12:10:50.832716+0000","last_became_active":"2024-05-11T12:10:45.764595+0000","last_became_peered":"2024-05-11T12:10:45.764595+0000","last_unstale":"2024-05-11T12:10:50.832716+0000","last_undegraded":"2024-05-11T12:10:50.832716+0000","last_fullsized":"2024-05-11T12:10:50.832716+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:01:30.356378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.830235+0000","last_change":"2024-05-11T12:10:49.979783+0000","last_active":"2024-05-11T12:10:50.830235+0000","last_peered":"2024-05-11T12:10:50.830235+0000","last_clean":"2024-05-11T12:10:50.830235+0000","last_became_active":"2024-05-11T12:10:49.977927+0000","last_became_peered":"2024-05-11T12:10:49.977927+0000","last_unstale":"2024-05-11T12:10:50.830235+0000","last_undegraded":"2024-05-11T12:10:50.830235+0000","last_fullsized":"2024-05-11T12:10:50.830235+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:31:59.631048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.833109+0000","last_change":"2024-05-11T12:10:44.769768+0000","last_active":"2024-05-11T12:10:50.833109+0000","last_peered":"2024-05-11T12:10:50.833109+0000","last_clean":"2024-05-11T12:10:50.833109+0000","last_became_active":"2024-05-11T12:10:44.769498+0000","last_became_peered":"2024-05-11T12:10:44.769498+0000","last_unstale":"2024-05-11T12:10:50.833109+0000","last_undegraded":"2024-05-11T12:10:50.833109+0000","last_fullsized":"2024-05-11T12:10:50.833109+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:14:58.421726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.056999+0000","last_change":"2024-05-11T12:10:45.791108+0000","last_active":"2024-05-11T12:10:53.056999+0000","last_peered":"2024-05-11T12:10:53.056999+0000","last_clean":"2024-05-11T12:10:53.056999+0000","last_became_active":"2024-05-11T12:10:45.790807+0000","last_became_peered":"2024-05-11T12:10:45.790807+0000","last_unstale":"2024-05-11T12:10:53.056999+0000","last_undegraded":"2024-05-11T12:10:53.056999+0000","last_fullsized":"2024-05-11T12:10:53.056999+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:52:36.650847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.177575+0000","last_change":"2024-05-11T12:10:49.843347+0000","last_active":"2024-05-11T12:10:51.177575+0000","last_peered":"2024-05-11T12:10:51.177575+0000","last_clean":"2024-05-11T12:10:51.177575+0000","last_became_active":"2024-05-11T12:10:49.840522+0000","last_became_peered":"2024-05-11T12:10:49.840522+0000","last_unstale":"2024-05-11T12:10:51.177575+0000","last_undegraded":"2024-05-11T12:10:51.177575+0000","last_fullsized":"2024-05-11T12:10:51.177575+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:09:52.792492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.986041+0000","last_change":"2024-05-11T12:10:44.788150+0000","last_active":"2024-05-11T12:10:52.986041+0000","last_peered":"2024-05-11T12:10:52.986041+0000","last_clean":"2024-05-11T12:10:52.986041+0000","last_became_active":"2024-05-11T12:10:44.787827+0000","last_became_peered":"2024-05-11T12:10:44.787827+0000","last_unstale":"2024-05-11T12:10:52.986041+0000","last_undegraded":"2024-05-11T12:10:52.986041+0000","last_fullsized":"2024-05-11T12:10:52.986041+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:50:29.143515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.834938+0000","last_change":"2024-05-11T12:10:45.769675+0000","last_active":"2024-05-11T12:10:50.834938+0000","last_peered":"2024-05-11T12:10:50.834938+0000","last_clean":"2024-05-11T12:10:50.834938+0000","last_became_active":"2024-05-11T12:10:45.766444+0000","last_became_peered":"2024-05-11T12:10:45.766444+0000","last_unstale":"2024-05-11T12:10:50.834938+0000","last_undegraded":"2024-05-11T12:10:50.834938+0000","last_fullsized":"2024-05-11T12:10:50.834938+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:02:47.506141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.177937+0000","last_change":"2024-05-11T12:10:49.843249+0000","last_active":"2024-05-11T12:10:51.177937+0000","last_peered":"2024-05-11T12:10:51.177937+0000","last_clean":"2024-05-11T12:10:51.177937+0000","last_became_active":"2024-05-11T12:10:49.841714+0000","last_became_peered":"2024-05-11T12:10:49.841714+0000","last_unstale":"2024-05-11T12:10:51.177937+0000","last_undegraded":"2024-05-11T12:10:51.177937+0000","last_fullsized":"2024-05-11T12:10:51.177937+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:15:59.709719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.986035+0000","last_change":"2024-05-11T12:10:44.783078+0000","last_active":"2024-05-11T12:10:52.986035+0000","last_peered":"2024-05-11T12:10:52.986035+0000","last_clean":"2024-05-11T12:10:52.986035+0000","last_became_active":"2024-05-11T12:10:44.782750+0000","last_became_peered":"2024-05-11T12:10:44.782750+0000","last_unstale":"2024-05-11T12:10:52.986035+0000","last_undegraded":"2024-05-11T12:10:52.986035+0000","last_fullsized":"2024-05-11T12:10:52.986035+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:32:31.312066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.836129+0000","last_change":"2024-05-11T12:10:45.771583+0000","last_active":"2024-05-11T12:10:50.836129+0000","last_peered":"2024-05-11T12:10:50.836129+0000","last_clean":"2024-05-11T12:10:50.836129+0000","last_became_active":"2024-05-11T12:10:45.766864+0000","last_became_peered":"2024-05-11T12:10:45.766864+0000","last_unstale":"2024-05-11T12:10:50.836129+0000","last_undegraded":"2024-05-11T12:10:50.836129+0000","last_fullsized":"2024-05-11T12:10:50.836129+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:29:58.902743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.177649+0000","last_change":"2024-05-11T12:10:49.833489+0000","last_active":"2024-05-11T12:10:51.177649+0000","last_peered":"2024-05-11T12:10:51.177649+0000","last_clean":"2024-05-11T12:10:51.177649+0000","last_became_active":"2024-05-11T12:10:49.833030+0000","last_became_peered":"2024-05-11T12:10:49.833030+0000","last_unstale":"2024-05-11T12:10:51.177649+0000","last_undegraded":"2024-05-11T12:10:51.177649+0000","last_fullsized":"2024-05-11T12:10:51.177649+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:40:37.671338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.833888+0000","last_change":"2024-05-11T12:10:45.777692+0000","last_active":"2024-05-11T12:10:50.833888+0000","last_peered":"2024-05-11T12:10:50.833888+0000","last_clean":"2024-05-11T12:10:50.833888+0000","last_became_active":"2024-05-11T12:10:45.777388+0000","last_became_peered":"2024-05-11T12:10:45.777388+0000","last_unstale":"2024-05-11T12:10:50.833888+0000","last_undegraded":"2024-05-11T12:10:50.833888+0000","last_fullsized":"2024-05-11T12:10:50.833888+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:30:11.665446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.986035+0000","last_change":"2024-05-11T12:10:44.780645+0000","last_active":"2024-05-11T12:10:52.986035+0000","last_peered":"2024-05-11T12:10:52.986035+0000","last_clean":"2024-05-11T12:10:52.986035+0000","last_became_active":"2024-05-11T12:10:44.780371+0000","last_became_peered":"2024-05-11T12:10:44.780371+0000","last_unstale":"2024-05-11T12:10:52.986035+0000","last_undegraded":"2024-05-11T12:10:52.986035+0000","last_fullsized":"2024-05-11T12:10:52.986035+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:46:19.349691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.178802+0000","last_change":"2024-05-11T12:10:49.842333+0000","last_active":"2024-05-11T12:10:51.178802+0000","last_peered":"2024-05-11T12:10:51.178802+0000","last_clean":"2024-05-11T12:10:51.178802+0000","last_became_active":"2024-05-11T12:10:49.841805+0000","last_became_peered":"2024-05-11T12:10:49.841805+0000","last_unstale":"2024-05-11T12:10:51.178802+0000","last_undegraded":"2024-05-11T12:10:51.178802+0000","last_fullsized":"2024-05-11T12:10:51.178802+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:30:02.975329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.831265+0000","last_change":"2024-05-11T12:10:45.785101+0000","last_active":"2024-05-11T12:10:50.831265+0000","last_peered":"2024-05-11T12:10:50.831265+0000","last_clean":"2024-05-11T12:10:50.831265+0000","last_became_active":"2024-05-11T12:10:45.784830+0000","last_became_peered":"2024-05-11T12:10:45.784830+0000","last_unstale":"2024-05-11T12:10:50.831265+0000","last_undegraded":"2024-05-11T12:10:50.831265+0000","last_fullsized":"2024-05-11T12:10:50.831265+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:15:55.637133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.084641+0000","last_change":"2024-05-11T12:10:44.762965+0000","last_active":"2024-05-11T12:10:53.084641+0000","last_peered":"2024-05-11T12:10:53.084641+0000","last_clean":"2024-05-11T12:10:53.084641+0000","last_became_active":"2024-05-11T12:10:44.761074+0000","last_became_peered":"2024-05-11T12:10:44.761074+0000","last_unstale":"2024-05-11T12:10:53.084641+0000","last_undegraded":"2024-05-11T12:10:53.084641+0000","last_fullsized":"2024-05-11T12:10:53.084641+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:11:44.935841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.178724+0000","last_change":"2024-05-11T12:10:49.842014+0000","last_active":"2024-05-11T12:10:51.178724+0000","last_peered":"2024-05-11T12:10:51.178724+0000","last_clean":"2024-05-11T12:10:51.178724+0000","last_became_active":"2024-05-11T12:10:49.840640+0000","last_became_peered":"2024-05-11T12:10:49.840640+0000","last_unstale":"2024-05-11T12:10:51.178724+0000","last_undegraded":"2024-05-11T12:10:51.178724+0000","last_fullsized":"2024-05-11T12:10:51.178724+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:31:28.750933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.830617+0000","last_change":"2024-05-11T12:10:45.767414+0000","last_active":"2024-05-11T12:10:50.830617+0000","last_peered":"2024-05-11T12:10:50.830617+0000","last_clean":"2024-05-11T12:10:50.830617+0000","last_became_active":"2024-05-11T12:10:45.765560+0000","last_became_peered":"2024-05-11T12:10:45.765560+0000","last_unstale":"2024-05-11T12:10:50.830617+0000","last_undegraded":"2024-05-11T12:10:50.830617+0000","last_fullsized":"2024-05-11T12:10:50.830617+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:25:39.934222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.085197+0000","last_change":"2024-05-11T12:10:44.764974+0000","last_active":"2024-05-11T12:10:53.085197+0000","last_peered":"2024-05-11T12:10:53.085197+0000","last_clean":"2024-05-11T12:10:53.085197+0000","last_became_active":"2024-05-11T12:10:44.764692+0000","last_became_peered":"2024-05-11T12:10:44.764692+0000","last_unstale":"2024-05-11T12:10:53.085197+0000","last_undegraded":"2024-05-11T12:10:53.085197+0000","last_fullsized":"2024-05-11T12:10:53.085197+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:27:48.475653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.086845+0000","last_change":"2024-05-11T12:10:49.982086+0000","last_active":"2024-05-11T12:10:53.086845+0000","last_peered":"2024-05-11T12:10:53.086845+0000","last_clean":"2024-05-11T12:10:53.086845+0000","last_became_active":"2024-05-11T12:10:49.981499+0000","last_became_peered":"2024-05-11T12:10:49.981499+0000","last_unstale":"2024-05-11T12:10:53.086845+0000","last_undegraded":"2024-05-11T12:10:53.086845+0000","last_fullsized":"2024-05-11T12:10:53.086845+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:36:11.661440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.178118+0000","last_change":"2024-05-11T12:10:45.780179+0000","last_active":"2024-05-11T12:10:51.178118+0000","last_peered":"2024-05-11T12:10:51.178118+0000","last_clean":"2024-05-11T12:10:51.178118+0000","last_became_active":"2024-05-11T12:10:45.777747+0000","last_became_peered":"2024-05-11T12:10:45.777747+0000","last_unstale":"2024-05-11T12:10:51.178118+0000","last_undegraded":"2024-05-11T12:10:51.178118+0000","last_fullsized":"2024-05-11T12:10:51.178118+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:39:51.243581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.986553+0000","last_change":"2024-05-11T12:10:44.775537+0000","last_active":"2024-05-11T12:10:52.986553+0000","last_peered":"2024-05-11T12:10:52.986553+0000","last_clean":"2024-05-11T12:10:52.986553+0000","last_became_active":"2024-05-11T12:10:44.775159+0000","last_became_peered":"2024-05-11T12:10:44.775159+0000","last_unstale":"2024-05-11T12:10:52.986553+0000","last_undegraded":"2024-05-11T12:10:52.986553+0000","last_fullsized":"2024-05-11T12:10:52.986553+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:39:50.229692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.179022+0000","last_change":"2024-05-11T12:10:49.847192+0000","last_active":"2024-05-11T12:10:51.179022+0000","last_peered":"2024-05-11T12:10:51.179022+0000","last_clean":"2024-05-11T12:10:51.179022+0000","last_became_active":"2024-05-11T12:10:49.846655+0000","last_became_peered":"2024-05-11T12:10:49.846655+0000","last_unstale":"2024-05-11T12:10:51.179022+0000","last_undegraded":"2024-05-11T12:10:51.179022+0000","last_fullsized":"2024-05-11T12:10:51.179022+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:52:55.215724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.057239+0000","last_change":"2024-05-11T12:10:45.788180+0000","last_active":"2024-05-11T12:10:53.057239+0000","last_peered":"2024-05-11T12:10:53.057239+0000","last_clean":"2024-05-11T12:10:53.057239+0000","last_became_active":"2024-05-11T12:10:45.787883+0000","last_became_peered":"2024-05-11T12:10:45.787883+0000","last_unstale":"2024-05-11T12:10:53.057239+0000","last_undegraded":"2024-05-11T12:10:53.057239+0000","last_fullsized":"2024-05-11T12:10:53.057239+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:00:49.028293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.837705+0000","last_change":"2024-05-11T12:10:44.783756+0000","last_active":"2024-05-11T12:10:50.837705+0000","last_peered":"2024-05-11T12:10:50.837705+0000","last_clean":"2024-05-11T12:10:50.837705+0000","last_became_active":"2024-05-11T12:10:44.779442+0000","last_became_peered":"2024-05-11T12:10:44.779442+0000","last_unstale":"2024-05-11T12:10:50.837705+0000","last_undegraded":"2024-05-11T12:10:50.837705+0000","last_fullsized":"2024-05-11T12:10:50.837705+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:12:58.847001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.177959+0000","last_change":"2024-05-11T12:10:49.989122+0000","last_active":"2024-05-11T12:10:51.177959+0000","last_peered":"2024-05-11T12:10:51.177959+0000","last_clean":"2024-05-11T12:10:51.177959+0000","last_became_active":"2024-05-11T12:10:49.988059+0000","last_became_peered":"2024-05-11T12:10:49.988059+0000","last_unstale":"2024-05-11T12:10:51.177959+0000","last_undegraded":"2024-05-11T12:10:51.177959+0000","last_fullsized":"2024-05-11T12:10:51.177959+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:11:21.324557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.832784+0000","last_change":"2024-05-11T12:10:45.780311+0000","last_active":"2024-05-11T12:10:50.832784+0000","last_peered":"2024-05-11T12:10:50.832784+0000","last_clean":"2024-05-11T12:10:50.832784+0000","last_became_active":"2024-05-11T12:10:45.779912+0000","last_became_peered":"2024-05-11T12:10:45.779912+0000","last_unstale":"2024-05-11T12:10:50.832784+0000","last_undegraded":"2024-05-11T12:10:50.832784+0000","last_fullsized":"2024-05-11T12:10:50.832784+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:09:48.719906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.084558+0000","last_change":"2024-05-11T12:10:44.776437+0000","last_active":"2024-05-11T12:10:53.084558+0000","last_peered":"2024-05-11T12:10:53.084558+0000","last_clean":"2024-05-11T12:10:53.084558+0000","last_became_active":"2024-05-11T12:10:44.775585+0000","last_became_peered":"2024-05-11T12:10:44.775585+0000","last_unstale":"2024-05-11T12:10:53.084558+0000","last_undegraded":"2024-05-11T12:10:53.084558+0000","last_fullsized":"2024-05-11T12:10:53.084558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:32:31.312066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.830010+0000","last_change":"2024-05-11T12:10:49.980738+0000","last_active":"2024-05-11T12:10:50.830010+0000","last_peered":"2024-05-11T12:10:50.830010+0000","last_clean":"2024-05-11T12:10:50.830010+0000","last_became_active":"2024-05-11T12:10:49.977869+0000","last_became_peered":"2024-05-11T12:10:49.977869+0000","last_unstale":"2024-05-11T12:10:50.830010+0000","last_undegraded":"2024-05-11T12:10:50.830010+0000","last_fullsized":"2024-05-11T12:10:50.830010+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:30:17.395939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.086118+0000","last_change":"2024-05-11T12:10:45.773559+0000","last_active":"2024-05-11T12:10:53.086118+0000","last_peered":"2024-05-11T12:10:53.086118+0000","last_clean":"2024-05-11T12:10:53.086118+0000","last_became_active":"2024-05-11T12:10:45.773152+0000","last_became_peered":"2024-05-11T12:10:45.773152+0000","last_unstale":"2024-05-11T12:10:53.086118+0000","last_undegraded":"2024-05-11T12:10:53.086118+0000","last_fullsized":"2024-05-11T12:10:53.086118+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:27:46.323505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.988658+0000","last_change":"2024-05-11T12:10:44.779798+0000","last_active":"2024-05-11T12:10:52.988658+0000","last_peered":"2024-05-11T12:10:52.988658+0000","last_clean":"2024-05-11T12:10:52.988658+0000","last_became_active":"2024-05-11T12:10:44.779517+0000","last_became_peered":"2024-05-11T12:10:44.779517+0000","last_unstale":"2024-05-11T12:10:52.988658+0000","last_undegraded":"2024-05-11T12:10:52.988658+0000","last_fullsized":"2024-05-11T12:10:52.988658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:27:48.475653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.055959+0000","last_change":"2024-05-11T12:10:49.977964+0000","last_active":"2024-05-11T12:10:53.055959+0000","last_peered":"2024-05-11T12:10:53.055959+0000","last_clean":"2024-05-11T12:10:53.055959+0000","last_became_active":"2024-05-11T12:10:49.977533+0000","last_became_peered":"2024-05-11T12:10:49.977533+0000","last_unstale":"2024-05-11T12:10:53.055959+0000","last_undegraded":"2024-05-11T12:10:53.055959+0000","last_fullsized":"2024-05-11T12:10:53.055959+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:02:51.578727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.836384+0000","last_change":"2024-05-11T12:10:45.788527+0000","last_active":"2024-05-11T12:10:50.836384+0000","last_peered":"2024-05-11T12:10:50.836384+0000","last_clean":"2024-05-11T12:10:50.836384+0000","last_became_active":"2024-05-11T12:10:45.783632+0000","last_became_peered":"2024-05-11T12:10:45.783632+0000","last_unstale":"2024-05-11T12:10:50.836384+0000","last_undegraded":"2024-05-11T12:10:50.836384+0000","last_fullsized":"2024-05-11T12:10:50.836384+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:52:51.143138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.086299+0000","last_change":"2024-05-11T12:10:44.764150+0000","last_active":"2024-05-11T12:10:53.086299+0000","last_peered":"2024-05-11T12:10:53.086299+0000","last_clean":"2024-05-11T12:10:53.086299+0000","last_became_active":"2024-05-11T12:10:44.763836+0000","last_became_peered":"2024-05-11T12:10:44.763836+0000","last_unstale":"2024-05-11T12:10:53.086299+0000","last_undegraded":"2024-05-11T12:10:53.086299+0000","last_fullsized":"2024-05-11T12:10:53.086299+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:39:50.229692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.085357+0000","last_change":"2024-05-11T12:10:49.841029+0000","last_active":"2024-05-11T12:10:53.085357+0000","last_peered":"2024-05-11T12:10:53.085357+0000","last_clean":"2024-05-11T12:10:53.085357+0000","last_became_active":"2024-05-11T12:10:49.840067+0000","last_became_peered":"2024-05-11T12:10:49.840067+0000","last_unstale":"2024-05-11T12:10:53.085357+0000","last_undegraded":"2024-05-11T12:10:53.085357+0000","last_fullsized":"2024-05-11T12:10:53.085357+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:02:51.578727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.178524+0000","last_change":"2024-05-11T12:10:45.785688+0000","last_active":"2024-05-11T12:10:51.178524+0000","last_peered":"2024-05-11T12:10:51.178524+0000","last_clean":"2024-05-11T12:10:51.178524+0000","last_became_active":"2024-05-11T12:10:45.785267+0000","last_became_peered":"2024-05-11T12:10:45.785267+0000","last_unstale":"2024-05-11T12:10:51.178524+0000","last_undegraded":"2024-05-11T12:10:51.178524+0000","last_fullsized":"2024-05-11T12:10:51.178524+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:50:30.157404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.833626+0000","last_change":"2024-05-11T12:10:44.777030+0000","last_active":"2024-05-11T12:10:50.833626+0000","last_peered":"2024-05-11T12:10:50.833626+0000","last_clean":"2024-05-11T12:10:50.833626+0000","last_became_active":"2024-05-11T12:10:44.776600+0000","last_became_peered":"2024-05-11T12:10:44.776600+0000","last_unstale":"2024-05-11T12:10:50.833626+0000","last_undegraded":"2024-05-11T12:10:50.833626+0000","last_fullsized":"2024-05-11T12:10:50.833626+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:32:31.312066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.086293+0000","last_change":"2024-05-11T12:10:49.843811+0000","last_active":"2024-05-11T12:10:53.086293+0000","last_peered":"2024-05-11T12:10:53.086293+0000","last_clean":"2024-05-11T12:10:53.086293+0000","last_became_active":"2024-05-11T12:10:49.843361+0000","last_became_peered":"2024-05-11T12:10:49.843361+0000","last_unstale":"2024-05-11T12:10:53.086293+0000","last_undegraded":"2024-05-11T12:10:53.086293+0000","last_fullsized":"2024-05-11T12:10:53.086293+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:03:19.587244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.085874+0000","last_change":"2024-05-11T12:10:45.773922+0000","last_active":"2024-05-11T12:10:53.085874+0000","last_peered":"2024-05-11T12:10:53.085874+0000","last_clean":"2024-05-11T12:10:53.085874+0000","last_became_active":"2024-05-11T12:10:45.773584+0000","last_became_peered":"2024-05-11T12:10:45.773584+0000","last_unstale":"2024-05-11T12:10:53.085874+0000","last_undegraded":"2024-05-11T12:10:53.085874+0000","last_fullsized":"2024-05-11T12:10:53.085874+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:25:39.934222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.956061+0000","last_change":"2024-05-11T12:10:44.767869+0000","last_active":"2024-05-11T12:10:50.956061+0000","last_peered":"2024-05-11T12:10:50.956061+0000","last_clean":"2024-05-11T12:10:50.956061+0000","last_became_active":"2024-05-11T12:10:44.766486+0000","last_became_peered":"2024-05-11T12:10:44.766486+0000","last_unstale":"2024-05-11T12:10:50.956061+0000","last_undegraded":"2024-05-11T12:10:50.956061+0000","last_fullsized":"2024-05-11T12:10:50.956061+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:49:38.234192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.087005+0000","last_change":"2024-05-11T12:10:49.837338+0000","last_active":"2024-05-11T12:10:53.087005+0000","last_peered":"2024-05-11T12:10:53.087005+0000","last_clean":"2024-05-11T12:10:53.087005+0000","last_became_active":"2024-05-11T12:10:49.836880+0000","last_became_peered":"2024-05-11T12:10:49.836880+0000","last_unstale":"2024-05-11T12:10:53.087005+0000","last_undegraded":"2024-05-11T12:10:53.087005+0000","last_fullsized":"2024-05-11T12:10:53.087005+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:30:58.400447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.055566+0000","last_change":"2024-05-11T12:10:45.784532+0000","last_active":"2024-05-11T12:10:53.055566+0000","last_peered":"2024-05-11T12:10:53.055566+0000","last_clean":"2024-05-11T12:10:53.055566+0000","last_became_active":"2024-05-11T12:10:45.783747+0000","last_became_peered":"2024-05-11T12:10:45.783747+0000","last_unstale":"2024-05-11T12:10:53.055566+0000","last_undegraded":"2024-05-11T12:10:53.055566+0000","last_fullsized":"2024-05-11T12:10:53.055566+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:33:50.642754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.986822+0000","last_change":"2024-05-11T12:10:44.778140+0000","last_active":"2024-05-11T12:10:52.986822+0000","last_peered":"2024-05-11T12:10:52.986822+0000","last_clean":"2024-05-11T12:10:52.986822+0000","last_became_active":"2024-05-11T12:10:44.777780+0000","last_became_peered":"2024-05-11T12:10:44.777780+0000","last_unstale":"2024-05-11T12:10:52.986822+0000","last_undegraded":"2024-05-11T12:10:52.986822+0000","last_fullsized":"2024-05-11T12:10:52.986822+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:25:35.246177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.057469+0000","last_change":"2024-05-11T12:10:49.984316+0000","last_active":"2024-05-11T12:10:53.057469+0000","last_peered":"2024-05-11T12:10:53.057469+0000","last_clean":"2024-05-11T12:10:53.057469+0000","last_became_active":"2024-05-11T12:10:49.981788+0000","last_became_peered":"2024-05-11T12:10:49.981788+0000","last_unstale":"2024-05-11T12:10:53.057469+0000","last_undegraded":"2024-05-11T12:10:53.057469+0000","last_fullsized":"2024-05-11T12:10:53.057469+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:07:34.303729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.830946+0000","last_change":"2024-05-11T12:10:45.781491+0000","last_active":"2024-05-11T12:10:50.830946+0000","last_peered":"2024-05-11T12:10:50.830946+0000","last_clean":"2024-05-11T12:10:50.830946+0000","last_became_active":"2024-05-11T12:10:45.779112+0000","last_became_peered":"2024-05-11T12:10:45.779112+0000","last_unstale":"2024-05-11T12:10:50.830946+0000","last_undegraded":"2024-05-11T12:10:50.830946+0000","last_fullsized":"2024-05-11T12:10:50.830946+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:00:49.028293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.833659+0000","last_change":"2024-05-11T12:10:44.765494+0000","last_active":"2024-05-11T12:10:50.833659+0000","last_peered":"2024-05-11T12:10:50.833659+0000","last_clean":"2024-05-11T12:10:50.833659+0000","last_became_active":"2024-05-11T12:10:44.764921+0000","last_became_peered":"2024-05-11T12:10:44.764921+0000","last_unstale":"2024-05-11T12:10:50.833659+0000","last_undegraded":"2024-05-11T12:10:50.833659+0000","last_fullsized":"2024-05-11T12:10:50.833659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:11:44.935841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":117,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.986840+0000","last_change":"2024-05-11T12:10:35.066673+0000","last_active":"2024-05-11T12:10:52.986840+0000","last_peered":"2024-05-11T12:10:52.986840+0000","last_clean":"2024-05-11T12:10:52.986840+0000","last_became_active":"2024-05-11T12:10:35.065942+0000","last_became_peered":"2024-05-11T12:10:35.065942+0000","last_unstale":"2024-05-11T12:10:52.986840+0000","last_undegraded":"2024-05-11T12:10:52.986840+0000","last_fullsized":"2024-05-11T12:10:52.986840+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_clean_scrub_stamp":"2024-05-11T12:10:33.601136+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:12:02.133182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.836369+0000","last_change":"2024-05-11T12:10:49.983023+0000","last_active":"2024-05-11T12:10:50.836369+0000","last_peered":"2024-05-11T12:10:50.836369+0000","last_clean":"2024-05-11T12:10:50.836369+0000","last_became_active":"2024-05-11T12:10:49.981575+0000","last_became_peered":"2024-05-11T12:10:49.981575+0000","last_unstale":"2024-05-11T12:10:50.836369+0000","last_undegraded":"2024-05-11T12:10:50.836369+0000","last_fullsized":"2024-05-11T12:10:50.836369+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:52:55.215724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.988305+0000","last_change":"2024-05-11T12:10:45.787709+0000","last_active":"2024-05-11T12:10:52.988305+0000","last_peered":"2024-05-11T12:10:52.988305+0000","last_clean":"2024-05-11T12:10:52.988305+0000","last_became_active":"2024-05-11T12:10:45.787205+0000","last_became_peered":"2024-05-11T12:10:45.787205+0000","last_unstale":"2024-05-11T12:10:52.988305+0000","last_undegraded":"2024-05-11T12:10:52.988305+0000","last_fullsized":"2024-05-11T12:10:52.988305+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:29:58.902743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.956475+0000","last_change":"2024-05-11T12:10:44.777012+0000","last_active":"2024-05-11T12:10:50.956475+0000","last_peered":"2024-05-11T12:10:50.956475+0000","last_clean":"2024-05-11T12:10:50.956475+0000","last_became_active":"2024-05-11T12:10:44.776709+0000","last_became_peered":"2024-05-11T12:10:44.776709+0000","last_unstale":"2024-05-11T12:10:50.956475+0000","last_undegraded":"2024-05-11T12:10:50.956475+0000","last_fullsized":"2024-05-11T12:10:50.956475+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:32:06.207257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.833701+0000","last_change":"2024-05-11T12:10:49.980334+0000","last_active":"2024-05-11T12:10:50.833701+0000","last_peered":"2024-05-11T12:10:50.833701+0000","last_clean":"2024-05-11T12:10:50.833701+0000","last_became_active":"2024-05-11T12:10:49.977819+0000","last_became_peered":"2024-05-11T12:10:49.977819+0000","last_unstale":"2024-05-11T12:10:50.833701+0000","last_undegraded":"2024-05-11T12:10:50.833701+0000","last_fullsized":"2024-05-11T12:10:50.833701+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:30:02.975329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.835272+0000","last_change":"2024-05-11T12:10:45.780131+0000","last_active":"2024-05-11T12:10:50.835272+0000","last_peered":"2024-05-11T12:10:50.835272+0000","last_clean":"2024-05-11T12:10:50.835272+0000","last_became_active":"2024-05-11T12:10:45.779867+0000","last_became_peered":"2024-05-11T12:10:45.779867+0000","last_unstale":"2024-05-11T12:10:50.835272+0000","last_undegraded":"2024-05-11T12:10:50.835272+0000","last_fullsized":"2024-05-11T12:10:50.835272+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:30:54.327861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.084659+0000","last_change":"2024-05-11T12:10:44.775148+0000","last_active":"2024-05-11T12:10:53.084659+0000","last_peered":"2024-05-11T12:10:53.084659+0000","last_clean":"2024-05-11T12:10:53.084659+0000","last_became_active":"2024-05-11T12:10:44.774850+0000","last_became_peered":"2024-05-11T12:10:44.774850+0000","last_unstale":"2024-05-11T12:10:53.084659+0000","last_undegraded":"2024-05-11T12:10:53.084659+0000","last_fullsized":"2024-05-11T12:10:53.084659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:25:35.246177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.056241+0000","last_change":"2024-05-11T12:10:49.842656+0000","last_active":"2024-05-11T12:10:53.056241+0000","last_peered":"2024-05-11T12:10:53.056241+0000","last_clean":"2024-05-11T12:10:53.056241+0000","last_became_active":"2024-05-11T12:10:49.841982+0000","last_became_peered":"2024-05-11T12:10:49.841982+0000","last_unstale":"2024-05-11T12:10:53.056241+0000","last_undegraded":"2024-05-11T12:10:53.056241+0000","last_fullsized":"2024-05-11T12:10:53.056241+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:03:19.587244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.988912+0000","last_change":"2024-05-11T12:10:45.794103+0000","last_active":"2024-05-11T12:10:52.988912+0000","last_peered":"2024-05-11T12:10:52.988912+0000","last_clean":"2024-05-11T12:10:52.988912+0000","last_became_active":"2024-05-11T12:10:45.792080+0000","last_became_peered":"2024-05-11T12:10:45.792080+0000","last_unstale":"2024-05-11T12:10:52.988912+0000","last_undegraded":"2024-05-11T12:10:52.988912+0000","last_fullsized":"2024-05-11T12:10:52.988912+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:30:54.327861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.956469+0000","last_change":"2024-05-11T12:10:44.768108+0000","last_active":"2024-05-11T12:10:50.956469+0000","last_peered":"2024-05-11T12:10:50.956469+0000","last_clean":"2024-05-11T12:10:50.956469+0000","last_became_active":"2024-05-11T12:10:44.766606+0000","last_became_peered":"2024-05-11T12:10:44.766606+0000","last_unstale":"2024-05-11T12:10:50.956469+0000","last_undegraded":"2024-05-11T12:10:50.956469+0000","last_fullsized":"2024-05-11T12:10:50.956469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:02:04.596576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.832239+0000","last_change":"2024-05-11T12:10:49.982446+0000","last_active":"2024-05-11T12:10:50.832239+0000","last_peered":"2024-05-11T12:10:50.832239+0000","last_clean":"2024-05-11T12:10:50.832239+0000","last_became_active":"2024-05-11T12:10:49.981272+0000","last_became_peered":"2024-05-11T12:10:49.981272+0000","last_unstale":"2024-05-11T12:10:50.832239+0000","last_undegraded":"2024-05-11T12:10:50.832239+0000","last_fullsized":"2024-05-11T12:10:50.832239+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:36:40.828180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.988575+0000","last_change":"2024-05-11T12:10:45.786373+0000","last_active":"2024-05-11T12:10:52.988575+0000","last_peered":"2024-05-11T12:10:52.988575+0000","last_clean":"2024-05-11T12:10:52.988575+0000","last_became_active":"2024-05-11T12:10:45.786050+0000","last_became_peered":"2024-05-11T12:10:45.786050+0000","last_unstale":"2024-05-11T12:10:52.988575+0000","last_undegraded":"2024-05-11T12:10:52.988575+0000","last_fullsized":"2024-05-11T12:10:52.988575+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:31:24.678347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.178969+0000","last_change":"2024-05-11T12:10:44.775124+0000","last_active":"2024-05-11T12:10:51.178969+0000","last_peered":"2024-05-11T12:10:51.178969+0000","last_clean":"2024-05-11T12:10:51.178969+0000","last_became_active":"2024-05-11T12:10:44.774810+0000","last_became_peered":"2024-05-11T12:10:44.774810+0000","last_unstale":"2024-05-11T12:10:51.178969+0000","last_undegraded":"2024-05-11T12:10:51.178969+0000","last_fullsized":"2024-05-11T12:10:51.178969+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:44:00.317135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.177431+0000","last_change":"2024-05-11T12:10:49.845241+0000","last_active":"2024-05-11T12:10:51.177431+0000","last_peered":"2024-05-11T12:10:51.177431+0000","last_clean":"2024-05-11T12:10:51.177431+0000","last_became_active":"2024-05-11T12:10:49.844564+0000","last_became_peered":"2024-05-11T12:10:49.844564+0000","last_unstale":"2024-05-11T12:10:51.177431+0000","last_undegraded":"2024-05-11T12:10:51.177431+0000","last_fullsized":"2024-05-11T12:10:51.177431+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:27:50.396091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.987029+0000","last_change":"2024-05-11T12:10:45.789312+0000","last_active":"2024-05-11T12:10:52.987029+0000","last_peered":"2024-05-11T12:10:52.987029+0000","last_clean":"2024-05-11T12:10:52.987029+0000","last_became_active":"2024-05-11T12:10:45.789069+0000","last_became_peered":"2024-05-11T12:10:45.789069+0000","last_unstale":"2024-05-11T12:10:52.987029+0000","last_undegraded":"2024-05-11T12:10:52.987029+0000","last_fullsized":"2024-05-11T12:10:52.987029+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:15:55.637133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.955873+0000","last_change":"2024-05-11T12:10:44.767205+0000","last_active":"2024-05-11T12:10:50.955873+0000","last_peered":"2024-05-11T12:10:50.955873+0000","last_clean":"2024-05-11T12:10:50.955873+0000","last_became_active":"2024-05-11T12:10:44.766855+0000","last_became_peered":"2024-05-11T12:10:44.766855+0000","last_unstale":"2024-05-11T12:10:50.955873+0000","last_undegraded":"2024-05-11T12:10:50.955873+0000","last_fullsized":"2024-05-11T12:10:50.955873+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:10:35.839599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.056975+0000","last_change":"2024-05-11T12:10:49.835689+0000","last_active":"2024-05-11T12:10:53.056975+0000","last_peered":"2024-05-11T12:10:53.056975+0000","last_clean":"2024-05-11T12:10:53.056975+0000","last_became_active":"2024-05-11T12:10:49.835305+0000","last_became_peered":"2024-05-11T12:10:49.835305+0000","last_unstale":"2024-05-11T12:10:53.056975+0000","last_undegraded":"2024-05-11T12:10:53.056975+0000","last_fullsized":"2024-05-11T12:10:53.056975+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:30:58.400447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.833870+0000","last_change":"2024-05-11T12:10:45.788030+0000","last_active":"2024-05-11T12:10:50.833870+0000","last_peered":"2024-05-11T12:10:50.833870+0000","last_clean":"2024-05-11T12:10:50.833870+0000","last_became_active":"2024-05-11T12:10:45.787098+0000","last_became_peered":"2024-05-11T12:10:45.787098+0000","last_unstale":"2024-05-11T12:10:50.833870+0000","last_undegraded":"2024-05-11T12:10:50.833870+0000","last_fullsized":"2024-05-11T12:10:50.833870+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:00:49.028293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.834966+0000","last_change":"2024-05-11T12:10:44.784319+0000","last_active":"2024-05-11T12:10:50.834966+0000","last_peered":"2024-05-11T12:10:50.834966+0000","last_clean":"2024-05-11T12:10:50.834966+0000","last_became_active":"2024-05-11T12:10:44.784027+0000","last_became_peered":"2024-05-11T12:10:44.784027+0000","last_unstale":"2024-05-11T12:10:50.834966+0000","last_undegraded":"2024-05-11T12:10:50.834966+0000","last_fullsized":"2024-05-11T12:10:50.834966+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:50:29.143515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.832654+0000","last_change":"2024-05-11T12:10:49.985187+0000","last_active":"2024-05-11T12:10:50.832654+0000","last_peered":"2024-05-11T12:10:50.832654+0000","last_clean":"2024-05-11T12:10:50.832654+0000","last_became_active":"2024-05-11T12:10:49.983450+0000","last_became_peered":"2024-05-11T12:10:49.983450+0000","last_unstale":"2024-05-11T12:10:50.832654+0000","last_undegraded":"2024-05-11T12:10:50.832654+0000","last_fullsized":"2024-05-11T12:10:50.832654+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:02:13.707329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.987787+0000","last_change":"2024-05-11T12:10:45.789328+0000","last_active":"2024-05-11T12:10:52.987787+0000","last_peered":"2024-05-11T12:10:52.987787+0000","last_clean":"2024-05-11T12:10:52.987787+0000","last_became_active":"2024-05-11T12:10:45.788778+0000","last_became_peered":"2024-05-11T12:10:45.788778+0000","last_unstale":"2024-05-11T12:10:52.987787+0000","last_undegraded":"2024-05-11T12:10:52.987787+0000","last_fullsized":"2024-05-11T12:10:52.987787+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:01:30.356378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.833719+0000","last_change":"2024-05-11T12:10:44.779353+0000","last_active":"2024-05-11T12:10:50.833719+0000","last_peered":"2024-05-11T12:10:50.833719+0000","last_clean":"2024-05-11T12:10:50.833719+0000","last_became_active":"2024-05-11T12:10:44.779078+0000","last_became_peered":"2024-05-11T12:10:44.779078+0000","last_unstale":"2024-05-11T12:10:50.833719+0000","last_undegraded":"2024-05-11T12:10:50.833719+0000","last_fullsized":"2024-05-11T12:10:50.833719+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:25:35.246177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.831951+0000","last_change":"2024-05-11T12:10:49.979416+0000","last_active":"2024-05-11T12:10:50.831951+0000","last_peered":"2024-05-11T12:10:50.831951+0000","last_clean":"2024-05-11T12:10:50.831951+0000","last_became_active":"2024-05-11T12:10:49.978262+0000","last_became_peered":"2024-05-11T12:10:49.978262+0000","last_unstale":"2024-05-11T12:10:50.831951+0000","last_undegraded":"2024-05-11T12:10:50.831951+0000","last_fullsized":"2024-05-11T12:10:50.831951+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:53:47.237894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.056303+0000","last_change":"2024-05-11T12:10:45.784434+0000","last_active":"2024-05-11T12:10:53.056303+0000","last_peered":"2024-05-11T12:10:53.056303+0000","last_clean":"2024-05-11T12:10:53.056303+0000","last_became_active":"2024-05-11T12:10:45.784143+0000","last_became_peered":"2024-05-11T12:10:45.784143+0000","last_unstale":"2024-05-11T12:10:53.056303+0000","last_undegraded":"2024-05-11T12:10:53.056303+0000","last_fullsized":"2024-05-11T12:10:53.056303+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:25:39.934222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.054871+0000","last_change":"2024-05-11T12:10:44.783217+0000","last_active":"2024-05-11T12:10:53.054871+0000","last_peered":"2024-05-11T12:10:53.054871+0000","last_clean":"2024-05-11T12:10:53.054871+0000","last_became_active":"2024-05-11T12:10:44.782942+0000","last_became_peered":"2024-05-11T12:10:44.782942+0000","last_unstale":"2024-05-11T12:10:53.054871+0000","last_undegraded":"2024-05-11T12:10:53.054871+0000","last_fullsized":"2024-05-11T12:10:53.054871+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:14:58.421726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.836406+0000","last_change":"2024-05-11T12:10:49.981126+0000","last_active":"2024-05-11T12:10:50.836406+0000","last_peered":"2024-05-11T12:10:50.836406+0000","last_clean":"2024-05-11T12:10:50.836406+0000","last_became_active":"2024-05-11T12:10:49.978149+0000","last_became_peered":"2024-05-11T12:10:49.978149+0000","last_unstale":"2024-05-11T12:10:50.836406+0000","last_undegraded":"2024-05-11T12:10:50.836406+0000","last_fullsized":"2024-05-11T12:10:50.836406+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:45:08.936896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.988035+0000","last_change":"2024-05-11T12:10:45.791440+0000","last_active":"2024-05-11T12:10:52.988035+0000","last_peered":"2024-05-11T12:10:52.988035+0000","last_clean":"2024-05-11T12:10:52.988035+0000","last_became_active":"2024-05-11T12:10:45.791154+0000","last_became_peered":"2024-05-11T12:10:45.791154+0000","last_unstale":"2024-05-11T12:10:52.988035+0000","last_undegraded":"2024-05-11T12:10:52.988035+0000","last_fullsized":"2024-05-11T12:10:52.988035+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:52:51.143138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'10","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:11:01.126783+0000","last_change":"2024-05-11T12:10:44.773468+0000","last_active":"2024-05-11T12:11:01.126783+0000","last_peered":"2024-05-11T12:11:01.126783+0000","last_clean":"2024-05-11T12:11:01.126783+0000","last_became_active":"2024-05-11T12:10:44.769134+0000","last_became_peered":"2024-05-11T12:10:44.769134+0000","last_unstale":"2024-05-11T12:11:01.126783+0000","last_undegraded":"2024-05-11T12:11:01.126783+0000","last_fullsized":"2024-05-11T12:11:01.126783+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:08:58.049386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.085619+0000","last_change":"2024-05-11T12:10:49.847456+0000","last_active":"2024-05-11T12:10:53.085619+0000","last_peered":"2024-05-11T12:10:53.085619+0000","last_clean":"2024-05-11T12:10:53.085619+0000","last_became_active":"2024-05-11T12:10:49.847009+0000","last_became_peered":"2024-05-11T12:10:49.847009+0000","last_unstale":"2024-05-11T12:10:53.085619+0000","last_undegraded":"2024-05-11T12:10:53.085619+0000","last_fullsized":"2024-05-11T12:10:53.085619+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:52:55.215724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.178813+0000","last_change":"2024-05-11T12:10:45.781361+0000","last_active":"2024-05-11T12:10:51.178813+0000","last_peered":"2024-05-11T12:10:51.178813+0000","last_clean":"2024-05-11T12:10:51.178813+0000","last_became_active":"2024-05-11T12:10:45.781039+0000","last_became_peered":"2024-05-11T12:10:45.781039+0000","last_unstale":"2024-05-11T12:10:51.178813+0000","last_undegraded":"2024-05-11T12:10:51.178813+0000","last_fullsized":"2024-05-11T12:10:51.178813+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:27:49.489542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.084600+0000","last_change":"2024-05-11T12:10:44.770795+0000","last_active":"2024-05-11T12:10:53.084600+0000","last_peered":"2024-05-11T12:10:53.084600+0000","last_clean":"2024-05-11T12:10:53.084600+0000","last_became_active":"2024-05-11T12:10:44.770438+0000","last_became_peered":"2024-05-11T12:10:44.770438+0000","last_unstale":"2024-05-11T12:10:53.084600+0000","last_undegraded":"2024-05-11T12:10:53.084600+0000","last_fullsized":"2024-05-11T12:10:53.084600+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:50:29.143515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.178367+0000","last_change":"2024-05-11T12:10:49.841798+0000","last_active":"2024-05-11T12:10:51.178367+0000","last_peered":"2024-05-11T12:10:51.178367+0000","last_clean":"2024-05-11T12:10:51.178367+0000","last_became_active":"2024-05-11T12:10:49.840773+0000","last_became_peered":"2024-05-11T12:10:49.840773+0000","last_unstale":"2024-05-11T12:10:51.178367+0000","last_undegraded":"2024-05-11T12:10:51.178367+0000","last_fullsized":"2024-05-11T12:10:51.178367+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:01:34.428964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.057219+0000","last_change":"2024-05-11T12:10:45.779106+0000","last_active":"2024-05-11T12:10:53.057219+0000","last_peered":"2024-05-11T12:10:53.057219+0000","last_clean":"2024-05-11T12:10:53.057219+0000","last_became_active":"2024-05-11T12:10:45.778616+0000","last_became_peered":"2024-05-11T12:10:45.778616+0000","last_unstale":"2024-05-11T12:10:53.057219+0000","last_undegraded":"2024-05-11T12:10:53.057219+0000","last_fullsized":"2024-05-11T12:10:53.057219+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:40:33.598752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.837351+0000","last_change":"2024-05-11T12:10:44.781127+0000","last_active":"2024-05-11T12:10:50.837351+0000","last_peered":"2024-05-11T12:10:50.837351+0000","last_clean":"2024-05-11T12:10:50.837351+0000","last_became_active":"2024-05-11T12:10:44.780840+0000","last_became_peered":"2024-05-11T12:10:44.780840+0000","last_unstale":"2024-05-11T12:10:50.837351+0000","last_undegraded":"2024-05-11T12:10:50.837351+0000","last_fullsized":"2024-05-11T12:10:50.837351+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:50:29.143515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.830538+0000","last_change":"2024-05-11T12:10:49.839762+0000","last_active":"2024-05-11T12:10:50.830538+0000","last_peered":"2024-05-11T12:10:50.830538+0000","last_clean":"2024-05-11T12:10:50.830538+0000","last_became_active":"2024-05-11T12:10:49.838073+0000","last_became_peered":"2024-05-11T12:10:49.838073+0000","last_unstale":"2024-05-11T12:10:50.830538+0000","last_undegraded":"2024-05-11T12:10:50.830538+0000","last_fullsized":"2024-05-11T12:10:50.830538+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:36:40.828180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.837108+0000","last_change":"2024-05-11T12:10:45.778187+0000","last_active":"2024-05-11T12:10:50.837108+0000","last_peered":"2024-05-11T12:10:50.837108+0000","last_clean":"2024-05-11T12:10:50.837108+0000","last_became_active":"2024-05-11T12:10:45.776905+0000","last_became_peered":"2024-05-11T12:10:45.776905+0000","last_unstale":"2024-05-11T12:10:50.837108+0000","last_undegraded":"2024-05-11T12:10:50.837108+0000","last_fullsized":"2024-05-11T12:10:50.837108+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:36:07.588854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.837368+0000","last_change":"2024-05-11T12:10:44.778851+0000","last_active":"2024-05-11T12:10:50.837368+0000","last_peered":"2024-05-11T12:10:50.837368+0000","last_clean":"2024-05-11T12:10:50.837368+0000","last_became_active":"2024-05-11T12:10:44.776695+0000","last_became_peered":"2024-05-11T12:10:44.776695+0000","last_unstale":"2024-05-11T12:10:50.837368+0000","last_undegraded":"2024-05-11T12:10:50.837368+0000","last_fullsized":"2024-05-11T12:10:50.837368+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:49:09.103744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.831100+0000","last_change":"2024-05-11T12:10:49.982849+0000","last_active":"2024-05-11T12:10:50.831100+0000","last_peered":"2024-05-11T12:10:50.831100+0000","last_clean":"2024-05-11T12:10:50.831100+0000","last_became_active":"2024-05-11T12:10:49.981894+0000","last_became_peered":"2024-05-11T12:10:49.981894+0000","last_unstale":"2024-05-11T12:10:50.831100+0000","last_undegraded":"2024-05-11T12:10:50.831100+0000","last_fullsized":"2024-05-11T12:10:50.831100+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:45:08.936896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.056935+0000","last_change":"2024-05-11T12:10:49.982163+0000","last_active":"2024-05-11T12:10:53.056935+0000","last_peered":"2024-05-11T12:10:53.056935+0000","last_clean":"2024-05-11T12:10:53.056935+0000","last_became_active":"2024-05-11T12:10:49.981202+0000","last_became_peered":"2024-05-11T12:10:49.981202+0000","last_unstale":"2024-05-11T12:10:53.056935+0000","last_undegraded":"2024-05-11T12:10:53.056935+0000","last_fullsized":"2024-05-11T12:10:53.056935+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:36:11.661440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.085018+0000","last_change":"2024-05-11T12:10:45.772477+0000","last_active":"2024-05-11T12:10:53.085018+0000","last_peered":"2024-05-11T12:10:53.085018+0000","last_clean":"2024-05-11T12:10:53.085018+0000","last_became_active":"2024-05-11T12:10:45.772053+0000","last_became_peered":"2024-05-11T12:10:45.772053+0000","last_unstale":"2024-05-11T12:10:53.085018+0000","last_undegraded":"2024-05-11T12:10:53.085018+0000","last_fullsized":"2024-05-11T12:10:53.085018+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:00:49.028293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.832812+0000","last_change":"2024-05-11T12:10:44.782828+0000","last_active":"2024-05-11T12:10:50.832812+0000","last_peered":"2024-05-11T12:10:50.832812+0000","last_clean":"2024-05-11T12:10:50.832812+0000","last_became_active":"2024-05-11T12:10:44.782412+0000","last_became_peered":"2024-05-11T12:10:44.782412+0000","last_unstale":"2024-05-11T12:10:50.832812+0000","last_undegraded":"2024-05-11T12:10:50.832812+0000","last_fullsized":"2024-05-11T12:10:50.832812+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:39:50.229692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.836608+0000","last_change":"2024-05-11T12:10:49.849632+0000","last_active":"2024-05-11T12:10:50.836608+0000","last_peered":"2024-05-11T12:10:50.836608+0000","last_clean":"2024-05-11T12:10:50.836608+0000","last_became_active":"2024-05-11T12:10:49.849221+0000","last_became_peered":"2024-05-11T12:10:49.849221+0000","last_unstale":"2024-05-11T12:10:50.836608+0000","last_undegraded":"2024-05-11T12:10:50.836608+0000","last_fullsized":"2024-05-11T12:10:50.836608+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:02:51.578727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.830326+0000","last_change":"2024-05-11T12:10:45.784432+0000","last_active":"2024-05-11T12:10:50.830326+0000","last_peered":"2024-05-11T12:10:50.830326+0000","last_clean":"2024-05-11T12:10:50.830326+0000","last_became_active":"2024-05-11T12:10:45.784152+0000","last_became_peered":"2024-05-11T12:10:45.784152+0000","last_unstale":"2024-05-11T12:10:50.830326+0000","last_undegraded":"2024-05-11T12:10:50.830326+0000","last_fullsized":"2024-05-11T12:10:50.830326+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:21:38.211953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.831004+0000","last_change":"2024-05-11T12:10:44.775112+0000","last_active":"2024-05-11T12:10:50.831004+0000","last_peered":"2024-05-11T12:10:50.831004+0000","last_clean":"2024-05-11T12:10:50.831004+0000","last_became_active":"2024-05-11T12:10:44.774758+0000","last_became_peered":"2024-05-11T12:10:44.774758+0000","last_unstale":"2024-05-11T12:10:50.831004+0000","last_undegraded":"2024-05-11T12:10:50.831004+0000","last_fullsized":"2024-05-11T12:10:50.831004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:38:50.526921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.835462+0000","last_change":"2024-05-11T12:10:44.778249+0000","last_active":"2024-05-11T12:10:50.835462+0000","last_peered":"2024-05-11T12:10:50.835462+0000","last_clean":"2024-05-11T12:10:50.835462+0000","last_became_active":"2024-05-11T12:10:44.775251+0000","last_became_peered":"2024-05-11T12:10:44.775251+0000","last_unstale":"2024-05-11T12:10:50.835462+0000","last_undegraded":"2024-05-11T12:10:50.835462+0000","last_fullsized":"2024-05-11T12:10:50.835462+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:11:44.935841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.056459+0000","last_change":"2024-05-11T12:10:45.783894+0000","last_active":"2024-05-11T12:10:53.056459+0000","last_peered":"2024-05-11T12:10:53.056459+0000","last_clean":"2024-05-11T12:10:53.056459+0000","last_became_active":"2024-05-11T12:10:45.783452+0000","last_became_peered":"2024-05-11T12:10:45.783452+0000","last_unstale":"2024-05-11T12:10:53.056459+0000","last_undegraded":"2024-05-11T12:10:53.056459+0000","last_fullsized":"2024-05-11T12:10:53.056459+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:09:48.719906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.956355+0000","last_change":"2024-05-11T12:10:49.841006+0000","last_active":"2024-05-11T12:10:50.956355+0000","last_peered":"2024-05-11T12:10:50.956355+0000","last_clean":"2024-05-11T12:10:50.956355+0000","last_became_active":"2024-05-11T12:10:49.840609+0000","last_became_peered":"2024-05-11T12:10:49.840609+0000","last_unstale":"2024-05-11T12:10:50.956355+0000","last_undegraded":"2024-05-11T12:10:50.956355+0000","last_fullsized":"2024-05-11T12:10:50.956355+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:11:33.092215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.831304+0000","last_change":"2024-05-11T12:10:44.769720+0000","last_active":"2024-05-11T12:10:50.831304+0000","last_peered":"2024-05-11T12:10:50.831304+0000","last_clean":"2024-05-11T12:10:50.831304+0000","last_became_active":"2024-05-11T12:10:44.768923+0000","last_became_peered":"2024-05-11T12:10:44.768923+0000","last_unstale":"2024-05-11T12:10:50.831304+0000","last_undegraded":"2024-05-11T12:10:50.831304+0000","last_fullsized":"2024-05-11T12:10:50.831304+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:43.705221+0000","last_clean_scrub_stamp":"2024-05-11T12:10:43.705221+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:04:49.439739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.177838+0000","last_change":"2024-05-11T12:10:45.776362+0000","last_active":"2024-05-11T12:10:51.177838+0000","last_peered":"2024-05-11T12:10:51.177838+0000","last_clean":"2024-05-11T12:10:51.177838+0000","last_became_active":"2024-05-11T12:10:45.773081+0000","last_became_peered":"2024-05-11T12:10:45.773081+0000","last_unstale":"2024-05-11T12:10:51.177838+0000","last_undegraded":"2024-05-11T12:10:51.177838+0000","last_fullsized":"2024-05-11T12:10:51.177838+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:25:36.260066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.832413+0000","last_change":"2024-05-11T12:10:49.837438+0000","last_active":"2024-05-11T12:10:50.832413+0000","last_peered":"2024-05-11T12:10:50.832413+0000","last_clean":"2024-05-11T12:10:50.832413+0000","last_became_active":"2024-05-11T12:10:49.836966+0000","last_became_peered":"2024-05-11T12:10:49.836966+0000","last_unstale":"2024-05-11T12:10:50.832413+0000","last_undegraded":"2024-05-11T12:10:50.832413+0000","last_fullsized":"2024-05-11T12:10:50.832413+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:48.791696+0000","last_clean_scrub_stamp":"2024-05-11T12:10:48.791696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:16:26.266475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.832119+0000","last_change":"2024-05-11T12:10:45.792181+0000","last_active":"2024-05-11T12:10:50.832119+0000","last_peered":"2024-05-11T12:10:50.832119+0000","last_clean":"2024-05-11T12:10:50.832119+0000","last_became_active":"2024-05-11T12:10:45.791827+0000","last_became_peered":"2024-05-11T12:10:45.791827+0000","last_unstale":"2024-05-11T12:10:50.832119+0000","last_undegraded":"2024-05-11T12:10:50.832119+0000","last_fullsized":"2024-05-11T12:10:50.832119+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:25:39.934222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.956054+0000","last_change":"2024-05-11T12:10:45.786514+0000","last_active":"2024-05-11T12:10:50.956054+0000","last_peered":"2024-05-11T12:10:50.956054+0000","last_clean":"2024-05-11T12:10:50.956054+0000","last_became_active":"2024-05-11T12:10:45.786170+0000","last_became_peered":"2024-05-11T12:10:45.786170+0000","last_unstale":"2024-05-11T12:10:50.956054+0000","last_undegraded":"2024-05-11T12:10:50.956054+0000","last_fullsized":"2024-05-11T12:10:50.956054+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:07:42.553060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.833366+0000","last_change":"2024-05-11T12:10:45.764626+0000","last_active":"2024-05-11T12:10:50.833366+0000","last_peered":"2024-05-11T12:10:50.833366+0000","last_clean":"2024-05-11T12:10:50.833366+0000","last_became_active":"2024-05-11T12:10:45.764234+0000","last_became_peered":"2024-05-11T12:10:45.764234+0000","last_unstale":"2024-05-11T12:10:50.833366+0000","last_undegraded":"2024-05-11T12:10:50.833366+0000","last_fullsized":"2024-05-11T12:10:50.833366+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:31:24.678347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.056344+0000","last_change":"2024-05-11T12:10:45.787822+0000","last_active":"2024-05-11T12:10:53.056344+0000","last_peered":"2024-05-11T12:10:53.056344+0000","last_clean":"2024-05-11T12:10:53.056344+0000","last_became_active":"2024-05-11T12:10:45.787503+0000","last_became_peered":"2024-05-11T12:10:45.787503+0000","last_unstale":"2024-05-11T12:10:53.056344+0000","last_undegraded":"2024-05-11T12:10:53.056344+0000","last_fullsized":"2024-05-11T12:10:53.056344+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:30:11.665446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.832309+0000","last_change":"2024-05-11T12:10:45.769648+0000","last_active":"2024-05-11T12:10:50.832309+0000","last_peered":"2024-05-11T12:10:50.832309+0000","last_clean":"2024-05-11T12:10:50.832309+0000","last_became_active":"2024-05-11T12:10:45.769168+0000","last_became_peered":"2024-05-11T12:10:45.769168+0000","last_unstale":"2024-05-11T12:10:50.832309+0000","last_undegraded":"2024-05-11T12:10:50.832309+0000","last_fullsized":"2024-05-11T12:10:50.832309+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:40:33.598752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.086161+0000","last_change":"2024-05-11T12:10:45.773070+0000","last_active":"2024-05-11T12:10:53.086161+0000","last_peered":"2024-05-11T12:10:53.086161+0000","last_clean":"2024-05-11T12:10:53.086161+0000","last_became_active":"2024-05-11T12:10:45.772734+0000","last_became_peered":"2024-05-11T12:10:45.772734+0000","last_unstale":"2024-05-11T12:10:53.086161+0000","last_undegraded":"2024-05-11T12:10:53.086161+0000","last_fullsized":"2024-05-11T12:10:53.086161+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:52:36.650847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.836911+0000","last_change":"2024-05-11T12:10:45.776117+0000","last_active":"2024-05-11T12:10:50.836911+0000","last_peered":"2024-05-11T12:10:50.836911+0000","last_clean":"2024-05-11T12:10:50.836911+0000","last_became_active":"2024-05-11T12:10:45.775042+0000","last_became_peered":"2024-05-11T12:10:45.775042+0000","last_unstale":"2024-05-11T12:10:50.836911+0000","last_undegraded":"2024-05-11T12:10:50.836911+0000","last_fullsized":"2024-05-11T12:10:50.836911+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:40:33.598752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.832667+0000","last_change":"2024-05-11T12:10:45.767897+0000","last_active":"2024-05-11T12:10:50.832667+0000","last_peered":"2024-05-11T12:10:50.832667+0000","last_clean":"2024-05-11T12:10:50.832667+0000","last_became_active":"2024-05-11T12:10:45.767619+0000","last_became_peered":"2024-05-11T12:10:45.767619+0000","last_unstale":"2024-05-11T12:10:50.832667+0000","last_undegraded":"2024-05-11T12:10:50.832667+0000","last_fullsized":"2024-05-11T12:10:50.832667+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:27:46.323505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.832683+0000","last_change":"2024-05-11T12:10:45.763701+0000","last_active":"2024-05-11T12:10:50.832683+0000","last_peered":"2024-05-11T12:10:50.832683+0000","last_clean":"2024-05-11T12:10:50.832683+0000","last_became_active":"2024-05-11T12:10:45.763260+0000","last_became_peered":"2024-05-11T12:10:45.763260+0000","last_unstale":"2024-05-11T12:10:50.832683+0000","last_undegraded":"2024-05-11T12:10:50.832683+0000","last_fullsized":"2024-05-11T12:10:50.832683+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:11:17.251971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.830765+0000","last_change":"2024-05-11T12:10:45.767500+0000","last_active":"2024-05-11T12:10:50.830765+0000","last_peered":"2024-05-11T12:10:50.830765+0000","last_clean":"2024-05-11T12:10:50.830765+0000","last_became_active":"2024-05-11T12:10:45.767137+0000","last_became_peered":"2024-05-11T12:10:45.767137+0000","last_unstale":"2024-05-11T12:10:50.830765+0000","last_undegraded":"2024-05-11T12:10:50.830765+0000","last_fullsized":"2024-05-11T12:10:50.830765+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:52:36.650847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.833200+0000","last_change":"2024-05-11T12:10:45.779266+0000","last_active":"2024-05-11T12:10:50.833200+0000","last_peered":"2024-05-11T12:10:50.833200+0000","last_clean":"2024-05-11T12:10:50.833200+0000","last_became_active":"2024-05-11T12:10:45.778968+0000","last_became_peered":"2024-05-11T12:10:45.778968+0000","last_unstale":"2024-05-11T12:10:50.833200+0000","last_undegraded":"2024-05-11T12:10:50.833200+0000","last_fullsized":"2024-05-11T12:10:50.833200+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:40:33.598752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.056067+0000","last_change":"2024-05-11T12:10:45.793850+0000","last_active":"2024-05-11T12:10:53.056067+0000","last_peered":"2024-05-11T12:10:53.056067+0000","last_clean":"2024-05-11T12:10:53.056067+0000","last_became_active":"2024-05-11T12:10:45.793503+0000","last_became_peered":"2024-05-11T12:10:45.793503+0000","last_unstale":"2024-05-11T12:10:53.056067+0000","last_undegraded":"2024-05-11T12:10:53.056067+0000","last_fullsized":"2024-05-11T12:10:53.056067+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:21:38.211953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.836150+0000","last_change":"2024-05-11T12:10:45.776958+0000","last_active":"2024-05-11T12:10:50.836150+0000","last_peered":"2024-05-11T12:10:50.836150+0000","last_clean":"2024-05-11T12:10:50.836150+0000","last_became_active":"2024-05-11T12:10:45.770839+0000","last_became_peered":"2024-05-11T12:10:45.770839+0000","last_unstale":"2024-05-11T12:10:50.836150+0000","last_undegraded":"2024-05-11T12:10:50.836150+0000","last_fullsized":"2024-05-11T12:10:50.836150+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:52:36.650847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.988652+0000","last_change":"2024-05-11T12:10:45.784456+0000","last_active":"2024-05-11T12:10:52.988652+0000","last_peered":"2024-05-11T12:10:52.988652+0000","last_clean":"2024-05-11T12:10:52.988652+0000","last_became_active":"2024-05-11T12:10:45.784149+0000","last_became_peered":"2024-05-11T12:10:45.784149+0000","last_unstale":"2024-05-11T12:10:52.988652+0000","last_undegraded":"2024-05-11T12:10:52.988652+0000","last_fullsized":"2024-05-11T12:10:52.988652+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:36:07.588854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.956434+0000","last_change":"2024-05-11T12:10:45.778083+0000","last_active":"2024-05-11T12:10:50.956434+0000","last_peered":"2024-05-11T12:10:50.956434+0000","last_clean":"2024-05-11T12:10:50.956434+0000","last_became_active":"2024-05-11T12:10:45.777758+0000","last_became_peered":"2024-05-11T12:10:45.777758+0000","last_unstale":"2024-05-11T12:10:50.956434+0000","last_undegraded":"2024-05-11T12:10:50.956434+0000","last_fullsized":"2024-05-11T12:10:50.956434+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:38:01.288328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.838005+0000","last_change":"2024-05-11T12:10:45.780997+0000","last_active":"2024-05-11T12:10:50.838005+0000","last_peered":"2024-05-11T12:10:50.838005+0000","last_clean":"2024-05-11T12:10:50.838005+0000","last_became_active":"2024-05-11T12:10:45.780744+0000","last_became_peered":"2024-05-11T12:10:45.780744+0000","last_unstale":"2024-05-11T12:10:50.838005+0000","last_undegraded":"2024-05-11T12:10:50.838005+0000","last_fullsized":"2024-05-11T12:10:50.838005+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:03:15.514658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.832468+0000","last_change":"2024-05-11T12:10:45.790012+0000","last_active":"2024-05-11T12:10:50.832468+0000","last_peered":"2024-05-11T12:10:50.832468+0000","last_clean":"2024-05-11T12:10:50.832468+0000","last_became_active":"2024-05-11T12:10:45.786296+0000","last_became_peered":"2024-05-11T12:10:45.786296+0000","last_unstale":"2024-05-11T12:10:50.832468+0000","last_undegraded":"2024-05-11T12:10:50.832468+0000","last_fullsized":"2024-05-11T12:10:50.832468+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:09:48.719906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.085092+0000","last_change":"2024-05-11T12:10:45.766294+0000","last_active":"2024-05-11T12:10:53.085092+0000","last_peered":"2024-05-11T12:10:53.085092+0000","last_clean":"2024-05-11T12:10:53.085092+0000","last_became_active":"2024-05-11T12:10:45.765946+0000","last_became_peered":"2024-05-11T12:10:45.765946+0000","last_unstale":"2024-05-11T12:10:53.085092+0000","last_undegraded":"2024-05-11T12:10:53.085092+0000","last_fullsized":"2024-05-11T12:10:53.085092+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:30:11.665446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.056206+0000","last_change":"2024-05-11T12:10:45.777473+0000","last_active":"2024-05-11T12:10:53.056206+0000","last_peered":"2024-05-11T12:10:53.056206+0000","last_clean":"2024-05-11T12:10:53.056206+0000","last_became_active":"2024-05-11T12:10:45.777140+0000","last_became_peered":"2024-05-11T12:10:45.777140+0000","last_unstale":"2024-05-11T12:10:53.056206+0000","last_undegraded":"2024-05-11T12:10:53.056206+0000","last_fullsized":"2024-05-11T12:10:53.056206+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:12:59.860890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.830966+0000","last_change":"2024-05-11T12:10:45.779099+0000","last_active":"2024-05-11T12:10:50.830966+0000","last_peered":"2024-05-11T12:10:50.830966+0000","last_clean":"2024-05-11T12:10:50.830966+0000","last_became_active":"2024-05-11T12:10:45.767177+0000","last_became_peered":"2024-05-11T12:10:45.767177+0000","last_unstale":"2024-05-11T12:10:50.830966+0000","last_undegraded":"2024-05-11T12:10:50.830966+0000","last_fullsized":"2024-05-11T12:10:50.830966+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:30:11.665446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:51.178521+0000","last_change":"2024-05-11T12:10:45.781140+0000","last_active":"2024-05-11T12:10:51.178521+0000","last_peered":"2024-05-11T12:10:51.178521+0000","last_clean":"2024-05-11T12:10:51.178521+0000","last_became_active":"2024-05-11T12:10:45.780390+0000","last_became_peered":"2024-05-11T12:10:45.780390+0000","last_unstale":"2024-05-11T12:10:51.178521+0000","last_undegraded":"2024-05-11T12:10:51.178521+0000","last_fullsized":"2024-05-11T12:10:51.178521+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:32:32.325955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.836702+0000","last_change":"2024-05-11T12:10:45.772180+0000","last_active":"2024-05-11T12:10:50.836702+0000","last_peered":"2024-05-11T12:10:50.836702+0000","last_clean":"2024-05-11T12:10:50.836702+0000","last_became_active":"2024-05-11T12:10:45.771785+0000","last_became_peered":"2024-05-11T12:10:45.771785+0000","last_unstale":"2024-05-11T12:10:50.836702+0000","last_undegraded":"2024-05-11T12:10:50.836702+0000","last_fullsized":"2024-05-11T12:10:50.836702+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:27:46.323505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.836913+0000","last_change":"2024-05-11T12:10:45.773903+0000","last_active":"2024-05-11T12:10:50.836913+0000","last_peered":"2024-05-11T12:10:50.836913+0000","last_clean":"2024-05-11T12:10:50.836913+0000","last_became_active":"2024-05-11T12:10:45.773639+0000","last_became_peered":"2024-05-11T12:10:45.773639+0000","last_unstale":"2024-05-11T12:10:50.836913+0000","last_undegraded":"2024-05-11T12:10:50.836913+0000","last_fullsized":"2024-05-11T12:10:50.836913+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:09:48.719906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.988774+0000","last_change":"2024-05-11T12:10:45.794802+0000","last_active":"2024-05-11T12:10:52.988774+0000","last_peered":"2024-05-11T12:10:52.988774+0000","last_clean":"2024-05-11T12:10:52.988774+0000","last_became_active":"2024-05-11T12:10:45.794505+0000","last_became_peered":"2024-05-11T12:10:45.794505+0000","last_unstale":"2024-05-11T12:10:52.988774+0000","last_undegraded":"2024-05-11T12:10:52.988774+0000","last_fullsized":"2024-05-11T12:10:52.988774+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:03:15.514658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.830033+0000","last_change":"2024-05-11T12:10:45.764551+0000","last_active":"2024-05-11T12:10:50.830033+0000","last_peered":"2024-05-11T12:10:50.830033+0000","last_clean":"2024-05-11T12:10:50.830033+0000","last_became_active":"2024-05-11T12:10:45.763049+0000","last_became_peered":"2024-05-11T12:10:45.763049+0000","last_unstale":"2024-05-11T12:10:50.830033+0000","last_undegraded":"2024-05-11T12:10:50.830033+0000","last_fullsized":"2024-05-11T12:10:50.830033+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:11:17.251971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.986814+0000","last_change":"2024-05-11T12:10:45.785129+0000","last_active":"2024-05-11T12:10:52.986814+0000","last_peered":"2024-05-11T12:10:52.986814+0000","last_clean":"2024-05-11T12:10:52.986814+0000","last_became_active":"2024-05-11T12:10:45.784827+0000","last_became_peered":"2024-05-11T12:10:45.784827+0000","last_unstale":"2024-05-11T12:10:52.986814+0000","last_undegraded":"2024-05-11T12:10:52.986814+0000","last_fullsized":"2024-05-11T12:10:52.986814+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:40:33.598752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.833132+0000","last_change":"2024-05-11T12:10:45.785406+0000","last_active":"2024-05-11T12:10:50.833132+0000","last_peered":"2024-05-11T12:10:50.833132+0000","last_clean":"2024-05-11T12:10:50.833132+0000","last_became_active":"2024-05-11T12:10:45.785083+0000","last_became_peered":"2024-05-11T12:10:45.785083+0000","last_unstale":"2024-05-11T12:10:50.833132+0000","last_undegraded":"2024-05-11T12:10:50.833132+0000","last_fullsized":"2024-05-11T12:10:50.833132+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:15:55.637133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.987595+0000","last_change":"2024-05-11T12:10:45.779636+0000","last_active":"2024-05-11T12:10:52.987595+0000","last_peered":"2024-05-11T12:10:52.987595+0000","last_clean":"2024-05-11T12:10:52.987595+0000","last_became_active":"2024-05-11T12:10:45.779327+0000","last_became_peered":"2024-05-11T12:10:45.779327+0000","last_unstale":"2024-05-11T12:10:52.987595+0000","last_undegraded":"2024-05-11T12:10:52.987595+0000","last_fullsized":"2024-05-11T12:10:52.987595+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:11:17.251971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:50.834940+0000","last_change":"2024-05-11T12:10:45.782057+0000","last_active":"2024-05-11T12:10:50.834940+0000","last_peered":"2024-05-11T12:10:50.834940+0000","last_clean":"2024-05-11T12:10:50.834940+0000","last_became_active":"2024-05-11T12:10:45.781008+0000","last_became_peered":"2024-05-11T12:10:45.781008+0000","last_unstale":"2024-05-11T12:10:50.834940+0000","last_undegraded":"2024-05-11T12:10:50.834940+0000","last_fullsized":"2024-05-11T12:10:50.834940+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:01:30.356378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.056602+0000","last_change":"2024-05-11T12:10:45.783937+0000","last_active":"2024-05-11T12:10:53.056602+0000","last_peered":"2024-05-11T12:10:53.056602+0000","last_clean":"2024-05-11T12:10:53.056602+0000","last_became_active":"2024-05-11T12:10:45.783553+0000","last_became_peered":"2024-05-11T12:10:45.783553+0000","last_unstale":"2024-05-11T12:10:53.056602+0000","last_undegraded":"2024-05-11T12:10:53.056602+0000","last_fullsized":"2024-05-11T12:10:53.056602+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:27:46.323505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:53.086881+0000","last_change":"2024-05-11T12:10:45.786860+0000","last_active":"2024-05-11T12:10:53.086881+0000","last_peered":"2024-05-11T12:10:53.086881+0000","last_clean":"2024-05-11T12:10:53.086881+0000","last_became_active":"2024-05-11T12:10:45.785741+0000","last_became_peered":"2024-05-11T12:10:45.785741+0000","last_unstale":"2024-05-11T12:10:53.086881+0000","last_undegraded":"2024-05-11T12:10:53.086881+0000","last_fullsized":"2024-05-11T12:10:53.086881+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:09:48.719906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T12:10:52.988637+0000","last_change":"2024-05-11T12:10:45.794362+0000","last_active":"2024-05-11T12:10:52.988637+0000","last_peered":"2024-05-11T12:10:52.988637+0000","last_clean":"2024-05-11T12:10:52.988637+0000","last_became_active":"2024-05-11T12:10:45.793845+0000","last_became_peered":"2024-05-11T12:10:45.793845+0000","last_unstale":"2024-05-11T12:10:52.988637+0000","last_undegraded":"2024-05-11T12:10:52.988637+0000","last_fullsized":"2024-05-11T12:10:52.988637+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:44.719109+0000","last_clean_scrub_stamp":"2024-05-11T12:10:44.719109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:02:47.506141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476747,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":876544,"data_stored":700107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476748,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":112999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476747,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":282624,"data_stored":109247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476749,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":290816,"data_stored":109283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509452,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":112999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509453,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":109317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509452,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":884736,"data_stored":700163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542157,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":109157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T12:11:17.409 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-11T12:11:17.409 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-11T12:11:17.409 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-05-11T12:11:17.409 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-11T12:11:17.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.641+0000 7faf81e00640 1 Processor -- start 2024-05-11T12:11:17.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.641+0000 7faf81e00640 1 -- start start 2024-05-11T12:11:17.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.643+0000 7faf81e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7faf7c146770 0x7faf7c0ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:17.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.643+0000 7faf81e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7faf7c0cef10 0x7faf7c06c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:17.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.643+0000 7faf81e00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf7c06fbc0 con 0x7faf7c145b00 2024-05-11T12:11:17.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.643+0000 7faf81e00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7faf7c0b4520 con 0x7faf7c0cef10 2024-05-11T12:11:17.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.643+0000 7faf81e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7faf7c0b4800 con 0x7faf7c146770 2024-05-11T12:11:17.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.643+0000 7faf80a00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7faf7c145b00 0x7faf7c06f2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49516/0 (socket says 172.21.15.167:49516) 2024-05-11T12:11:17.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.643+0000 7faf80a00640 1 -- 172.21.15.167:0/1379364439 learned_addr learned my addr 172.21.15.167:0/1379364439 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:17.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.643+0000 7faf81400640 1 --2- 172.21.15.167:0/1379364439 >> v2:172.21.15.167:3300/0 conn(0x7faf7c0cef10 0x7faf7c06c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:17.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.643+0000 7faf7be00640 1 --2- 172.21.15.167:0/1379364439 >> v2:172.21.15.204:3300/0 conn(0x7faf7c146770 0x7faf7c0ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:17.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.643+0000 7faf81400640 1 -- 172.21.15.167:0/1379364439 >> v2:172.21.15.204:3300/0 conn(0x7faf7c146770 msgr2=0x7faf7c0ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:17.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.643+0000 7faf81400640 1 --2- 172.21.15.167:0/1379364439 >> v2:172.21.15.204:3300/0 conn(0x7faf7c146770 0x7faf7c0ce9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:17.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.643+0000 7faf81400640 1 -- 172.21.15.167:0/1379364439 >> v1:172.21.15.167:6789/0 conn(0x7faf7c145b00 legacy=0x7faf7c06f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:17.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.644+0000 7faf7b400640 1 -- 172.21.15.167:0/1379364439 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2794678943 0 0) 0x7faf7c06fbc0 con 0x7faf7c145b00 2024-05-11T12:11:17.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.644+0000 7faf81400640 1 -- 172.21.15.167:0/1379364439 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf7c0b4ae0 con 0x7faf7c0cef10 2024-05-11T12:11:17.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.644+0000 7faf81400640 1 --2- 172.21.15.167:0/1379364439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf7c0cef10 0x7faf7c06c660 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7faf7c146d70 tx=0x7faf7402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f3a8a3ffbd5eb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:17.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.645+0000 7faf7b400640 1 -- 172.21.15.167:0/1379364439 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf74007740 con 0x7faf7c0cef10 2024-05-11T12:11:17.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.645+0000 7faf7b400640 1 -- 172.21.15.167:0/1379364439 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf740078e0 con 0x7faf7c0cef10 2024-05-11T12:11:17.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.645+0000 7faf7b400640 1 -- 172.21.15.167:0/1379364439 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf74007cb0 con 0x7faf7c0cef10 2024-05-11T12:11:17.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.645+0000 7faf81e00640 1 -- 172.21.15.167:0/1379364439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf7c0cef10 msgr2=0x7faf7c06c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:17.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.645+0000 7faf81e00640 1 --2- 172.21.15.167:0/1379364439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf7c0cef10 0x7faf7c06c660 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7faf7c146d70 tx=0x7faf7402a5f0 comp rx=0 tx=0).stop 2024-05-11T12:11:17.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.645+0000 7faf81e00640 1 -- 172.21.15.167:0/1379364439 shutdown_connections 2024-05-11T12:11:17.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.645+0000 7faf81e00640 1 --2- 172.21.15.167:0/1379364439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf7c0cef10 0x7faf7c06c660 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:17.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.645+0000 7faf81e00640 1 --2- 172.21.15.167:0/1379364439 >> v2:172.21.15.204:3300/0 conn(0x7faf7c146770 0x7faf7c0ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:17.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.645+0000 7faf81e00640 1 -- 172.21.15.167:0/1379364439 >> 172.21.15.167:0/1379364439 conn(0x7faf7c0b3460 msgr2=0x7faf7c0d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:17.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.645+0000 7faf81e00640 1 -- 172.21.15.167:0/1379364439 shutdown_connections 2024-05-11T12:11:17.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.645+0000 7faf81e00640 1 -- 172.21.15.167:0/1379364439 wait complete. 2024-05-11T12:11:17.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.647+0000 7faf81e00640 1 Processor -- start 2024-05-11T12:11:17.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.647+0000 7faf81e00640 1 -- start start 2024-05-11T12:11:17.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.648+0000 7faf81e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faf7c0cef10 0x7faf7c0ae820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:17.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.648+0000 7faf81e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf7c145b00 0x7faf7c13d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:17.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.648+0000 7faf81e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf7c146770 0x7faf7c13de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:17.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.648+0000 7faf81e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faf7c06cd20 con 0x7faf7c145b00 2024-05-11T12:11:17.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.648+0000 7faf81e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7faf7c06cba0 con 0x7faf7c146770 2024-05-11T12:11:17.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.648+0000 7faf81e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7faf7c06cea0 con 0x7faf7c0cef10 2024-05-11T12:11:17.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.648+0000 7faf7be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf7c145b00 0x7faf7c13d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:17.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.648+0000 7faf7be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf7c145b00 0x7faf7c13d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41870/0 (socket says 172.21.15.167:41870) 2024-05-11T12:11:17.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.648+0000 7faf7be00640 1 -- 172.21.15.167:0/189301306 learned_addr learned my addr 172.21.15.167:0/189301306 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:17.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.649+0000 7faf7be00640 1 -- 172.21.15.167:0/189301306 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faf7c0cef10 msgr2=0x7faf7c0ae820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:17.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.649+0000 7faf7be00640 1 --2- 172.21.15.167:0/189301306 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faf7c0cef10 0x7faf7c0ae820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:17.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.649+0000 7faf7be00640 1 -- 172.21.15.167:0/189301306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf7c146770 msgr2=0x7faf7c13de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:17.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.649+0000 7faf7be00640 1 --2- 172.21.15.167:0/189301306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf7c146770 0x7faf7c13de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:17.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.649+0000 7faf7be00640 1 -- 172.21.15.167:0/189301306 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf7c0aed60 con 0x7faf7c145b00 2024-05-11T12:11:17.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.649+0000 7faf7be00640 1 --2- 172.21.15.167:0/189301306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf7c145b00 0x7faf7c13d550 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7faf6800c7f0 tx=0x7faf6800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:17.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.649+0000 7faf78c00640 1 -- 172.21.15.167:0/189301306 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf68015070 con 0x7faf7c145b00 2024-05-11T12:11:17.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.649+0000 7faf78c00640 1 -- 172.21.15.167:0/189301306 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf68004220 con 0x7faf7c145b00 2024-05-11T12:11:17.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.649+0000 7faf78c00640 1 -- 172.21.15.167:0/189301306 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf68007620 con 0x7faf7c145b00 2024-05-11T12:11:17.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.649+0000 7faf81e00640 1 -- 172.21.15.167:0/189301306 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf7c0aefe0 con 0x7faf7c145b00 2024-05-11T12:11:17.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.649+0000 7faf81e00640 1 -- 172.21.15.167:0/189301306 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf7c0af5a0 con 0x7faf7c145b00 2024-05-11T12:11:17.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.650+0000 7faf81e00640 1 -- 172.21.15.167:0/189301306 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf7c0694b0 con 0x7faf7c145b00 2024-05-11T12:11:17.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.652+0000 7faf78c00640 1 -- 172.21.15.167:0/189301306 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7faf680079d0 con 0x7faf7c145b00 2024-05-11T12:11:17.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.653+0000 7faf78c00640 1 --2- 172.21.15.167:0/189301306 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7faf5c0762d0 0x7faf5c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:17.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.653+0000 7faf78c00640 1 -- 172.21.15.167:0/189301306 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7faf68097990 con 0x7faf7c145b00 2024-05-11T12:11:17.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.653+0000 7faf80a00640 1 --2- 172.21.15.167:0/189301306 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7faf5c0762d0 0x7faf5c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:17.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.654+0000 7faf80a00640 1 --2- 172.21.15.167:0/189301306 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7faf5c0762d0 0x7faf5c078790 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7faf700015d0 tx=0x7faf700078d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:17.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.657+0000 7faf78c00640 1 -- 172.21.15.167:0/189301306 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7faf7c0aed60 con 0x7faf7c145b00 2024-05-11T12:11:17.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.901+0000 7faf81e00640 1 -- 172.21.15.167:0/189301306 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7faf7c0d1810 con 0x7faf7c145b00 2024-05-11T12:11:17.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.901+0000 7faf78c00640 1 -- 172.21.15.167:0/189301306 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7faf68060f10 con 0x7faf7c145b00 2024-05-11T12:11:17.902 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:11:17.902 INFO:teuthology.orchestra.run.smithi167.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-11T12:11:17.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.904+0000 7faf81e00640 1 -- 172.21.15.167:0/189301306 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7faf5c0762d0 msgr2=0x7faf5c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:17.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.904+0000 7faf81e00640 1 --2- 172.21.15.167:0/189301306 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7faf5c0762d0 0x7faf5c078790 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7faf700015d0 tx=0x7faf700078d0 comp rx=0 tx=0).stop 2024-05-11T12:11:17.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.904+0000 7faf81e00640 1 -- 172.21.15.167:0/189301306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf7c145b00 msgr2=0x7faf7c13d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:17.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.904+0000 7faf81e00640 1 --2- 172.21.15.167:0/189301306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf7c145b00 0x7faf7c13d550 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7faf6800c7f0 tx=0x7faf6800ccb0 comp rx=0 tx=0).stop 2024-05-11T12:11:17.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.904+0000 7faf81e00640 1 -- 172.21.15.167:0/189301306 shutdown_connections 2024-05-11T12:11:17.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.904+0000 7faf81e00640 1 --2- 172.21.15.167:0/189301306 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7faf5c0762d0 0x7faf5c078790 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:17.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.904+0000 7faf81e00640 1 --2- 172.21.15.167:0/189301306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf7c146770 0x7faf7c13de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:17.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.904+0000 7faf81e00640 1 --2- 172.21.15.167:0/189301306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf7c145b00 0x7faf7c13d550 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:17.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.904+0000 7faf81e00640 1 --2- 172.21.15.167:0/189301306 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faf7c0cef10 0x7faf7c0ae820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:17.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.904+0000 7faf81e00640 1 -- 172.21.15.167:0/189301306 >> 172.21.15.167:0/189301306 conn(0x7faf7c0b3460 msgr2=0x7faf7c0d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:17.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.905+0000 7faf81e00640 1 -- 172.21.15.167:0/189301306 shutdown_connections 2024-05-11T12:11:17.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:17.905+0000 7faf81e00640 1 -- 172.21.15.167:0/189301306 wait complete. 2024-05-11T12:11:17.924 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-05-11T12:11:17.925 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:11:18.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.129+0000 7f79e3e00640 1 Processor -- start 2024-05-11T12:11:18.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.129+0000 7f79e3e00640 1 -- start start 2024-05-11T12:11:18.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.130+0000 7f79e3e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f79e415f460 0x7f79e40b83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:18.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.131+0000 7f79e3e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f79e40d42c0 0x7f79e40d0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:18.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.131+0000 7f79e3e00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79e40cf990 con 0x7f79e40b88e0 2024-05-11T12:11:18.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.131+0000 7f79e3e00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f79e40cfc70 con 0x7f79e40d42c0 2024-05-11T12:11:18.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.131+0000 7f79e3e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f79e40cff50 con 0x7f79e415f460 2024-05-11T12:11:18.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.131+0000 7f79e2000640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f79e40b88e0 0x7f79e40d3bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49522/0 (socket says 172.21.15.167:49522) 2024-05-11T12:11:18.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.131+0000 7f79e2000640 1 -- 172.21.15.167:0/3350228606 learned_addr learned my addr 172.21.15.167:0/3350228606 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:18.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.131+0000 7f79e3400640 1 --2- 172.21.15.167:0/3350228606 >> v2:172.21.15.167:3300/0 conn(0x7f79e40d42c0 0x7f79e40d0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:18.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.131+0000 7f79e2a00640 1 --2- 172.21.15.167:0/3350228606 >> v2:172.21.15.204:3300/0 conn(0x7f79e415f460 0x7f79e40b83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:18.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.131+0000 7f79e3400640 1 -- 172.21.15.167:0/3350228606 >> v2:172.21.15.204:3300/0 conn(0x7f79e415f460 msgr2=0x7f79e40b83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:18.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.131+0000 7f79e3400640 1 --2- 172.21.15.167:0/3350228606 >> v2:172.21.15.204:3300/0 conn(0x7f79e415f460 0x7f79e40b83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:18.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.131+0000 7f79e3400640 1 -- 172.21.15.167:0/3350228606 >> v1:172.21.15.167:6789/0 conn(0x7f79e40b88e0 legacy=0x7f79e40d3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:18.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.132+0000 7f79e3400640 1 -- 172.21.15.167:0/3350228606 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79e40d0230 con 0x7f79e40d42c0 2024-05-11T12:11:18.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.132+0000 7f79e1600640 1 -- 172.21.15.167:0/3350228606 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 280748358 0 0) 0x7f79e40cf990 con 0x7f79e40b88e0 2024-05-11T12:11:18.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.133+0000 7f79e3400640 1 --2- 172.21.15.167:0/3350228606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79e40d42c0 0x7f79e40d0a30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f79e40d2b50 tx=0x7f79dc02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=682cea4f98e41c95 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:18.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.133+0000 7f79e1600640 1 -- 172.21.15.167:0/3350228606 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc007740 con 0x7f79e40d42c0 2024-05-11T12:11:18.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.133+0000 7f79e1600640 1 -- 172.21.15.167:0/3350228606 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79dc0078e0 con 0x7f79e40d42c0 2024-05-11T12:11:18.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.133+0000 7f79e1600640 1 -- 172.21.15.167:0/3350228606 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc007cb0 con 0x7f79e40d42c0 2024-05-11T12:11:18.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.133+0000 7f79e3e00640 1 -- 172.21.15.167:0/3350228606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79e40d42c0 msgr2=0x7f79e40d0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:18.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.133+0000 7f79e3e00640 1 --2- 172.21.15.167:0/3350228606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79e40d42c0 0x7f79e40d0a30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f79e40d2b50 tx=0x7f79dc02a680 comp rx=0 tx=0).stop 2024-05-11T12:11:18.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.134+0000 7f79e3e00640 1 -- 172.21.15.167:0/3350228606 shutdown_connections 2024-05-11T12:11:18.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.134+0000 7f79e3e00640 1 --2- 172.21.15.167:0/3350228606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79e40d42c0 0x7f79e40d0a30 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:18.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.134+0000 7f79e3e00640 1 --2- 172.21.15.167:0/3350228606 >> v2:172.21.15.204:3300/0 conn(0x7f79e415f460 0x7f79e40b83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:18.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.134+0000 7f79e3e00640 1 -- 172.21.15.167:0/3350228606 >> 172.21.15.167:0/3350228606 conn(0x7f79e40d1a50 msgr2=0x7f79e40d1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:18.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.134+0000 7f79e3e00640 1 -- 172.21.15.167:0/3350228606 shutdown_connections 2024-05-11T12:11:18.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.134+0000 7f79e3e00640 1 -- 172.21.15.167:0/3350228606 wait complete. 2024-05-11T12:11:18.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.135+0000 7f79e3e00640 1 Processor -- start 2024-05-11T12:11:18.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.136+0000 7f79e3e00640 1 -- start start 2024-05-11T12:11:18.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.136+0000 7f79e3e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f79e40b88e0 0x7f79e41f5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:18.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.136+0000 7f79e3e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79e40d42c0 0x7f79e4157890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:18.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.137+0000 7f79e3e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79e415f460 0x7f79e40ca3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:18.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.137+0000 7f79e3e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f79e406cc40 con 0x7f79e40d42c0 2024-05-11T12:11:18.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.137+0000 7f79e3e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f79e40cf620 con 0x7f79e40b88e0 2024-05-11T12:11:18.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.137+0000 7f79e3e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f79e406cdc0 con 0x7f79e415f460 2024-05-11T12:11:18.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.137+0000 7f79e2000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79e40d42c0 0x7f79e4157890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:18.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.137+0000 7f79e2000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79e40d42c0 0x7f79e4157890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41890/0 (socket says 172.21.15.167:41890) 2024-05-11T12:11:18.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.137+0000 7f79e2000640 1 -- 172.21.15.167:0/4191863361 learned_addr learned my addr 172.21.15.167:0/4191863361 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:18.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.137+0000 7f79e2000640 1 -- 172.21.15.167:0/4191863361 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79e415f460 msgr2=0x7f79e40ca3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:18.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.137+0000 7f79e2000640 1 --2- 172.21.15.167:0/4191863361 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79e415f460 0x7f79e40ca3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:18.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.137+0000 7f79e2000640 1 -- 172.21.15.167:0/4191863361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f79e40b88e0 msgr2=0x7f79e41f5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:18.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.137+0000 7f79e2000640 1 --2- 172.21.15.167:0/4191863361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f79e40b88e0 0x7f79e41f5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:18.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.137+0000 7f79e2000640 1 -- 172.21.15.167:0/4191863361 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79e40c8c40 con 0x7f79e40d42c0 2024-05-11T12:11:18.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.137+0000 7f79e2000640 1 --2- 172.21.15.167:0/4191863361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79e40d42c0 0x7f79e4157890 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f79d000c450 tx=0x7f79d000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:18.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.138+0000 7f79daa00640 1 -- 172.21.15.167:0/4191863361 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d0018070 con 0x7f79e40d42c0 2024-05-11T12:11:18.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.138+0000 7f79daa00640 1 -- 172.21.15.167:0/4191863361 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79d0007500 con 0x7f79e40d42c0 2024-05-11T12:11:18.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.138+0000 7f79daa00640 1 -- 172.21.15.167:0/4191863361 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d0004a20 con 0x7f79e40d42c0 2024-05-11T12:11:18.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.138+0000 7f79e3e00640 1 -- 172.21.15.167:0/4191863361 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79e40c8f20 con 0x7f79e40d42c0 2024-05-11T12:11:18.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.138+0000 7f79e3e00640 1 -- 172.21.15.167:0/4191863361 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79e40c94e0 con 0x7f79e40d42c0 2024-05-11T12:11:18.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.139+0000 7f79e3e00640 1 -- 172.21.15.167:0/4191863361 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79e40694b0 con 0x7f79e40d42c0 2024-05-11T12:11:18.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.140+0000 7f79daa00640 1 -- 172.21.15.167:0/4191863361 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f79d0004bc0 con 0x7f79e40d42c0 2024-05-11T12:11:18.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.142+0000 7f79daa00640 1 --2- 172.21.15.167:0/4191863361 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f79c4076300 0x7f79c40787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:18.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.142+0000 7f79daa00640 1 -- 172.21.15.167:0/4191863361 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f79d0096c30 con 0x7f79e40d42c0 2024-05-11T12:11:18.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.142+0000 7f79e2a00640 1 --2- 172.21.15.167:0/4191863361 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f79c4076300 0x7f79c40787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:18.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.143+0000 7f79e2a00640 1 --2- 172.21.15.167:0/4191863361 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f79c4076300 0x7f79c40787c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f79d4006fd0 tx=0x7f79d4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:18.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.146+0000 7f79daa00640 1 -- 172.21.15.167:0/4191863361 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f79d00601b0 con 0x7f79e40d42c0 2024-05-11T12:11:18.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.400+0000 7f79e3e00640 1 -- 172.21.15.167:0/4191863361 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79e40d4900 con 0x7f79e40d42c0 2024-05-11T12:11:18.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.401+0000 7f79daa00640 1 -- 172.21.15.167:0/4191863361 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5189 (secure 0 0 0) 0x7f79d005fab0 con 0x7f79e40d42c0 2024-05-11T12:11:18.402 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:11:18.402 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3127277486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3127277486},{"type":"v1","addr":"172.21.15.167:6835","nonce":3127277486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2547133600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2547133600},{"type":"v1","addr":"172.21.15.167:6837","nonce":2547133600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3200960100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3200960100},{"type":"v1","addr":"172.21.15.204:6834","nonce":3200960100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:10:45.225745+0000","modified":"2024-05-11T12:11:09.405446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/1889842375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":1889842375},{"type":"v1","addr":"172.21.15.204:6835","nonce":1889842375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-05-11T12:11:18.402 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 9 2024-05-11T12:11:18.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.404+0000 7f79e3e00640 1 -- 172.21.15.167:0/4191863361 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f79c4076300 msgr2=0x7f79c40787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:18.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.404+0000 7f79e3e00640 1 --2- 172.21.15.167:0/4191863361 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f79c4076300 0x7f79c40787c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f79d4006fd0 tx=0x7f79d4008040 comp rx=0 tx=0).stop 2024-05-11T12:11:18.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.404+0000 7f79e3e00640 1 -- 172.21.15.167:0/4191863361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79e40d42c0 msgr2=0x7f79e4157890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:18.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.404+0000 7f79e3e00640 1 --2- 172.21.15.167:0/4191863361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79e40d42c0 0x7f79e4157890 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f79d000c450 tx=0x7f79d000c910 comp rx=0 tx=0).stop 2024-05-11T12:11:18.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.404+0000 7f79e3e00640 1 -- 172.21.15.167:0/4191863361 shutdown_connections 2024-05-11T12:11:18.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.404+0000 7f79e3e00640 1 --2- 172.21.15.167:0/4191863361 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79e415f460 0x7f79e40ca3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:18.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.404+0000 7f79e3e00640 1 --2- 172.21.15.167:0/4191863361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79e40d42c0 0x7f79e4157890 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:18.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.404+0000 7f79e3e00640 1 --2- 172.21.15.167:0/4191863361 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f79c4076300 0x7f79c40787c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:18.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.404+0000 7f79e3e00640 1 --2- 172.21.15.167:0/4191863361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f79e40b88e0 0x7f79e41f5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:18.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.404+0000 7f79e3e00640 1 -- 172.21.15.167:0/4191863361 >> 172.21.15.167:0/4191863361 conn(0x7f79e40d1a50 msgr2=0x7f79e40b5940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:18.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.405+0000 7f79e3e00640 1 -- 172.21.15.167:0/4191863361 shutdown_connections 2024-05-11T12:11:18.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.405+0000 7f79e3e00640 1 -- 172.21.15.167:0/4191863361 wait complete. 2024-05-11T12:11:18.425 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:11:18.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.624+0000 7f9753200640 1 Processor -- start 2024-05-11T12:11:18.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.624+0000 7f9753200640 1 -- start start 2024-05-11T12:11:18.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.626+0000 7f9753200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f97541458f0 0x7f9754145ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:18.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.626+0000 7f9753200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f97540d0bb0 0x7f97540d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:18.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.626+0000 7f9753200640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97540b4fb0 con 0x7f9754074100 2024-05-11T12:11:18.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.626+0000 7f9753200640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f97540d1550 con 0x7f97541458f0 2024-05-11T12:11:18.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.626+0000 7f9753200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f97540b4d60 con 0x7f97540d0bb0 2024-05-11T12:11:18.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.626+0000 7f9751e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f97541458f0 0x7f9754145ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:18.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.626+0000 7f9751e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f97541458f0 0x7f9754145ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41896/0 (socket says 172.21.15.167:41896) 2024-05-11T12:11:18.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.626+0000 7f9751e00640 1 -- 172.21.15.167:0/3672613807 learned_addr learned my addr 172.21.15.167:0/3672613807 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:18.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.626+0000 7f9752800640 1 --1- 172.21.15.167:0/3672613807 >> v1:172.21.15.167:6789/0 conn(0x7f9754074100 0x7f97540b6050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49524/0 (socket says 172.21.15.167:49524) 2024-05-11T12:11:18.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.626+0000 7f9751400640 1 --2- 172.21.15.167:0/3672613807 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f97540d0bb0 0x7f97540d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:18.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.627+0000 7f9751e00640 1 -- 172.21.15.167:0/3672613807 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f97540d0bb0 msgr2=0x7f97540d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:18.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.627+0000 7f9751e00640 1 --2- 172.21.15.167:0/3672613807 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f97540d0bb0 0x7f97540d1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:18.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.627+0000 7f9751e00640 1 -- 172.21.15.167:0/3672613807 >> v1:172.21.15.167:6789/0 conn(0x7f9754074100 legacy=0x7f97540b6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:18.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.627+0000 7f9751e00640 1 -- 172.21.15.167:0/3672613807 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97540b5290 con 0x7f97541458f0 2024-05-11T12:11:18.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.627+0000 7f9750a00640 1 -- 172.21.15.167:0/3672613807 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 30182544 0 0) 0x7f97540b4fb0 con 0x7f9754074100 2024-05-11T12:11:18.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.628+0000 7f9751e00640 1 --2- 172.21.15.167:0/3672613807 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97541458f0 0x7f9754145ef0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9744009870 tx=0x7f974402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32bba12eadb81a47 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:18.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.628+0000 7f9750a00640 1 -- 172.21.15.167:0/3672613807 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9744037070 con 0x7f97541458f0 2024-05-11T12:11:18.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.628+0000 7f9750a00640 1 -- 172.21.15.167:0/3672613807 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9744032440 con 0x7f97541458f0 2024-05-11T12:11:18.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.628+0000 7f9750a00640 1 -- 172.21.15.167:0/3672613807 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9744032810 con 0x7f97541458f0 2024-05-11T12:11:18.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.629+0000 7f9753200640 1 -- 172.21.15.167:0/3672613807 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97541458f0 msgr2=0x7f9754145ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:18.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.629+0000 7f9753200640 1 --2- 172.21.15.167:0/3672613807 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97541458f0 0x7f9754145ef0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9744009870 tx=0x7f974402a1c0 comp rx=0 tx=0).stop 2024-05-11T12:11:18.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.629+0000 7f9753200640 1 -- 172.21.15.167:0/3672613807 shutdown_connections 2024-05-11T12:11:18.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.629+0000 7f9753200640 1 --2- 172.21.15.167:0/3672613807 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f97540d0bb0 0x7f97540d1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:18.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.629+0000 7f9753200640 1 --2- 172.21.15.167:0/3672613807 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97541458f0 0x7f9754145ef0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:18.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.629+0000 7f9753200640 1 -- 172.21.15.167:0/3672613807 >> 172.21.15.167:0/3672613807 conn(0x7f97540ce7b0 msgr2=0x7f97540cebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:18.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.629+0000 7f9753200640 1 -- 172.21.15.167:0/3672613807 shutdown_connections 2024-05-11T12:11:18.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.629+0000 7f9753200640 1 -- 172.21.15.167:0/3672613807 wait complete. 2024-05-11T12:11:18.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.631+0000 7f9753200640 1 Processor -- start 2024-05-11T12:11:18.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.631+0000 7f9753200640 1 -- start start 2024-05-11T12:11:18.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.632+0000 7f9753200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9754074100 0x7f9754140110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:18.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.632+0000 7f9753200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97540d0bb0 0x7f9754142de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:18.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.632+0000 7f9753200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f97541458f0 0x7f97540b2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:18.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.632+0000 7f9753200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9754146430 con 0x7f9754074100 2024-05-11T12:11:18.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.632+0000 7f9753200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f97540b5850 con 0x7f97540d0bb0 2024-05-11T12:11:18.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.632+0000 7f9753200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f97541465b0 con 0x7f97541458f0 2024-05-11T12:11:18.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.632+0000 7f9751e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9754074100 0x7f9754140110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:18.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.632+0000 7f9751e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9754074100 0x7f9754140110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41906/0 (socket says 172.21.15.167:41906) 2024-05-11T12:11:18.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.632+0000 7f9751e00640 1 -- 172.21.15.167:0/1914095137 learned_addr learned my addr 172.21.15.167:0/1914095137 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:18.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.632+0000 7f9751e00640 1 -- 172.21.15.167:0/1914095137 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f97541458f0 msgr2=0x7f97540b2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:18.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.632+0000 7f9751e00640 1 --2- 172.21.15.167:0/1914095137 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f97541458f0 0x7f97540b2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:18.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.632+0000 7f9751e00640 1 -- 172.21.15.167:0/1914095137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97540d0bb0 msgr2=0x7f9754142de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:18.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.632+0000 7f9751e00640 1 --2- 172.21.15.167:0/1914095137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97540d0bb0 0x7f9754142de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:18.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.632+0000 7f9751e00640 1 -- 172.21.15.167:0/1914095137 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97541f5a40 con 0x7f9754074100 2024-05-11T12:11:18.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.633+0000 7f9751e00640 1 --2- 172.21.15.167:0/1914095137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9754074100 0x7f9754140110 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9744035040 tx=0x7f97440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:18.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.633+0000 7f974a000640 1 -- 172.21.15.167:0/1914095137 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9744037040 con 0x7f9754074100 2024-05-11T12:11:18.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.633+0000 7f974a000640 1 -- 172.21.15.167:0/1914095137 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9744004230 con 0x7f9754074100 2024-05-11T12:11:18.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.633+0000 7f9753200640 1 -- 172.21.15.167:0/1914095137 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97541f5d20 con 0x7f9754074100 2024-05-11T12:11:18.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.633+0000 7f974a000640 1 -- 172.21.15.167:0/1914095137 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9744003610 con 0x7f9754074100 2024-05-11T12:11:18.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.633+0000 7f9753200640 1 -- 172.21.15.167:0/1914095137 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97541f6430 con 0x7f9754074100 2024-05-11T12:11:18.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.634+0000 7f9753200640 1 -- 172.21.15.167:0/1914095137 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97540694b0 con 0x7f9754074100 2024-05-11T12:11:18.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.635+0000 7f974a000640 1 -- 172.21.15.167:0/1914095137 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f974402ad10 con 0x7f9754074100 2024-05-11T12:11:18.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.636+0000 7f974a000640 1 --2- 172.21.15.167:0/1914095137 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f97340762d0 0x7f9734078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:18.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.636+0000 7f974a000640 1 -- 172.21.15.167:0/1914095137 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f97440b8420 con 0x7f9754074100 2024-05-11T12:11:18.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.637+0000 7f9751400640 1 --2- 172.21.15.167:0/1914095137 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f97340762d0 0x7f9734078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:18.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.637+0000 7f9751400640 1 --2- 172.21.15.167:0/1914095137 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f97340762d0 0x7f9734078790 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f97540cdfd0 tx=0x7f9740008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:18.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.641+0000 7f974a000640 1 -- 172.21.15.167:0/1914095137 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9744081ab0 con 0x7f9754074100 2024-05-11T12:11:18.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.890+0000 7f9753200640 1 -- 172.21.15.167:0/1914095137 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97540cf410 con 0x7f9754074100 2024-05-11T12:11:18.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.891+0000 7f974a000640 1 -- 172.21.15.167:0/1914095137 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5189 (secure 0 0 0) 0x7f97440813b0 con 0x7f9754074100 2024-05-11T12:11:18.892 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:11:18.892 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3127277486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3127277486},{"type":"v1","addr":"172.21.15.167:6835","nonce":3127277486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2547133600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2547133600},{"type":"v1","addr":"172.21.15.167:6837","nonce":2547133600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3200960100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3200960100},{"type":"v1","addr":"172.21.15.204:6834","nonce":3200960100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:10:45.225745+0000","modified":"2024-05-11T12:11:09.405446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/1889842375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":1889842375},{"type":"v1","addr":"172.21.15.204:6835","nonce":1889842375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-05-11T12:11:18.892 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 9 2024-05-11T12:11:18.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.894+0000 7f9753200640 1 -- 172.21.15.167:0/1914095137 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f97340762d0 msgr2=0x7f9734078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:18.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.894+0000 7f9753200640 1 --2- 172.21.15.167:0/1914095137 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f97340762d0 0x7f9734078790 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f97540cdfd0 tx=0x7f9740008040 comp rx=0 tx=0).stop 2024-05-11T12:11:18.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.894+0000 7f9753200640 1 -- 172.21.15.167:0/1914095137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9754074100 msgr2=0x7f9754140110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:18.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.894+0000 7f9753200640 1 --2- 172.21.15.167:0/1914095137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9754074100 0x7f9754140110 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9744035040 tx=0x7f97440046c0 comp rx=0 tx=0).stop 2024-05-11T12:11:18.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.894+0000 7f9753200640 1 -- 172.21.15.167:0/1914095137 shutdown_connections 2024-05-11T12:11:18.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.894+0000 7f9753200640 1 --2- 172.21.15.167:0/1914095137 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f97541458f0 0x7f97540b2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:18.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.894+0000 7f9753200640 1 --2- 172.21.15.167:0/1914095137 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f97340762d0 0x7f9734078790 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:18.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.894+0000 7f9753200640 1 --2- 172.21.15.167:0/1914095137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97540d0bb0 0x7f9754142de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:18.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.894+0000 7f9753200640 1 --2- 172.21.15.167:0/1914095137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9754074100 0x7f9754140110 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:18.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.894+0000 7f9753200640 1 -- 172.21.15.167:0/1914095137 >> 172.21.15.167:0/1914095137 conn(0x7f97540ce7b0 msgr2=0x7f9754162650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:18.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.895+0000 7f9753200640 1 -- 172.21.15.167:0/1914095137 shutdown_connections 2024-05-11T12:11:18.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:18.895+0000 7f9753200640 1 -- 172.21.15.167:0/1914095137 wait complete. 2024-05-11T12:11:18.914 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:11:19.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.116+0000 7f240fe00640 1 Processor -- start 2024-05-11T12:11:19.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.116+0000 7f240fe00640 1 -- start start 2024-05-11T12:11:19.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.117+0000 7f240fe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f241006c660 0x7f241006ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:19.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.117+0000 7f240fe00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f241006cfb0 con 0x7f2410145b00 2024-05-11T12:11:19.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.117+0000 7f240fe00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f241006fbc0 con 0x7f2410146770 2024-05-11T12:11:19.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.117+0000 7f240fe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f24100b4520 con 0x7f241006c660 2024-05-11T12:11:19.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.118+0000 7f240ea00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f2410145b00 0x7f241006f2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:40858/0 (socket says 172.21.15.167:40858) 2024-05-11T12:11:19.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.118+0000 7f240ea00640 1 -- 172.21.15.167:0/2112535585 learned_addr learned my addr 172.21.15.167:0/2112535585 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:19.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.118+0000 7f240f400640 1 --2- 172.21.15.167:0/2112535585 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f241006c660 0x7f241006ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:19.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.118+0000 7f240d600640 1 -- 172.21.15.167:0/2112535585 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1717655844 0 0) 0x7f2400001490 con 0x7f2410145b00 2024-05-11T12:11:19.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.118+0000 7f240d600640 1 -- 172.21.15.167:0/2112535585 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f23f4003520 con 0x7f2410145b00 2024-05-11T12:11:19.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.118+0000 7f240d600640 1 -- 172.21.15.167:0/2112535585 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1524689235 0 0) 0x7f23f4003520 con 0x7f2410145b00 2024-05-11T12:11:19.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.118+0000 7f240d600640 1 -- 172.21.15.167:0/2112535585 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f241006c660 msgr2=0x7f241006ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:19.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.118+0000 7f240d600640 1 --2- 172.21.15.167:0/2112535585 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f241006c660 0x7f241006ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:19.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.118+0000 7f240d600640 1 -- 172.21.15.167:0/2112535585 >> v1:172.21.15.204:6789/0 conn(0x7f2410146770 legacy=0x7f24100ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:19.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.118+0000 7f240d600640 1 -- 172.21.15.167:0/2112535585 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24100b4800 con 0x7f2410145b00 2024-05-11T12:11:19.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.118+0000 7f240d600640 1 -- 172.21.15.167:0/2112535585 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f2400001c40 con 0x7f2410145b00 2024-05-11T12:11:19.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.118+0000 7f240d600640 1 -- 172.21.15.167:0/2112535585 >> v1:172.21.15.167:6789/0 conn(0x7f2410145b00 legacy=0x7f241006f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:19.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.118+0000 7f240d600640 1 --2- 172.21.15.167:0/2112535585 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f241006c660 0x7f23f4003a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:19.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.119+0000 7f240d600640 1 --2- 172.21.15.167:0/2112535585 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23f4004050 0x7f23f4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:19.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.119+0000 7f240d600640 1 --2- 172.21.15.167:0/2112535585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23f4004d40 0x7f23f40051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:19.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.119+0000 7f240d600640 1 -- 172.21.15.167:0/2112535585 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f24100b4520 con 0x7f23f4004050 2024-05-11T12:11:19.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.119+0000 7f240d600640 1 -- 172.21.15.167:0/2112535585 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f23f400bc50 con 0x7f23f4004d40 2024-05-11T12:11:19.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.119+0000 7f240d600640 1 -- 172.21.15.167:0/2112535585 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f23f400be50 con 0x7f241006c660 2024-05-11T12:11:19.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.119+0000 7f240e000640 1 --2- 172.21.15.167:0/2112535585 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23f4004050 0x7f23f4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:19.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.119+0000 7f240ea00640 1 --2- 172.21.15.167:0/2112535585 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f241006c660 0x7f23f4003a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:19.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.119+0000 7f240f400640 1 --2- 172.21.15.167:0/2112535585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23f4004d40 0x7f23f40051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:19.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.119+0000 7f240e000640 1 -- 172.21.15.167:0/2112535585 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f241006c660 msgr2=0x7f23f4003a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:19.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.119+0000 7f240e000640 1 --2- 172.21.15.167:0/2112535585 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f241006c660 0x7f23f4003a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:19.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.119+0000 7f240e000640 1 -- 172.21.15.167:0/2112535585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23f4004d40 msgr2=0x7f23f40051a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:19.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.119+0000 7f240e000640 1 --2- 172.21.15.167:0/2112535585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23f4004d40 0x7f23f40051a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:19.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.119+0000 7f240e000640 1 -- 172.21.15.167:0/2112535585 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23f4003520 con 0x7f23f4004050 2024-05-11T12:11:19.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.119+0000 7f240f400640 1 --2- 172.21.15.167:0/2112535585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23f4004d40 0x7f23f40051a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:19.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.120+0000 7f240e000640 1 --2- 172.21.15.167:0/2112535585 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23f4004050 0x7f23f4004800 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f23fc001790 tx=0x7f23f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e03b9e15533e930 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:19.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.121+0000 7f240d600640 1 -- 172.21.15.167:0/2112535585 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23fc0013b0 con 0x7f23f4004050 2024-05-11T12:11:19.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.121+0000 7f240d600640 1 -- 172.21.15.167:0/2112535585 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23fc0052d0 con 0x7f23f4004050 2024-05-11T12:11:19.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.121+0000 7f240d600640 1 -- 172.21.15.167:0/2112535585 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23fc02aae0 con 0x7f23f4004050 2024-05-11T12:11:19.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.121+0000 7f240fe00640 1 -- 172.21.15.167:0/2112535585 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23f4004050 msgr2=0x7f23f4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:19.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.121+0000 7f240fe00640 1 --2- 172.21.15.167:0/2112535585 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23f4004050 0x7f23f4004800 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f23fc001790 tx=0x7f23f4005830 comp rx=0 tx=0).stop 2024-05-11T12:11:19.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.121+0000 7f240f400640 1 -- 172.21.15.167:0/2112535585 reap_dead start 2024-05-11T12:11:19.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.121+0000 7f240fe00640 1 -- 172.21.15.167:0/2112535585 shutdown_connections 2024-05-11T12:11:19.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.121+0000 7f240fe00640 1 -- 172.21.15.167:0/2112535585 >> 172.21.15.167:0/2112535585 conn(0x7f24100b3460 msgr2=0x7f24100d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:19.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.121+0000 7f240fe00640 1 -- 172.21.15.167:0/2112535585 shutdown_connections 2024-05-11T12:11:19.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.121+0000 7f240fe00640 1 -- 172.21.15.167:0/2112535585 wait complete. 2024-05-11T12:11:19.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.123+0000 7f240fe00640 1 Processor -- start 2024-05-11T12:11:19.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.123+0000 7f240fe00640 1 -- start start 2024-05-11T12:11:19.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.124+0000 7f240fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23f4004050 0x7f2410159da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:19.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.124+0000 7f240fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f241015f010 0x7f24101407a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:19.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.124+0000 7f240fe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f241015f490 0x7f2410140ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:19.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.124+0000 7f240fe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2410145ee0 con 0x7f23f4004050 2024-05-11T12:11:19.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.124+0000 7f240fe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f24100cf0e0 con 0x7f241015f010 2024-05-11T12:11:19.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.124+0000 7f240fe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f241006ca70 con 0x7f241015f490 2024-05-11T12:11:19.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.124+0000 7f240ea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23f4004050 0x7f2410159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:19.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.124+0000 7f240ea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23f4004050 0x7f2410159da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:53654/0 (socket says 172.21.15.167:53654) 2024-05-11T12:11:19.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.124+0000 7f240ea00640 1 -- 172.21.15.167:0/3953919753 learned_addr learned my addr 172.21.15.167:0/3953919753 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:19.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.125+0000 7f240f400640 1 --2- 172.21.15.167:0/3953919753 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f241015f490 0x7f2410140ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:19.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.125+0000 7f240ea00640 1 -- 172.21.15.167:0/3953919753 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f241015f490 msgr2=0x7f2410140ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:19.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.125+0000 7f240e000640 1 --2- 172.21.15.167:0/3953919753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f241015f010 0x7f24101407a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:19.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.125+0000 7f240ea00640 1 --2- 172.21.15.167:0/3953919753 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f241015f490 0x7f2410140ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:19.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.125+0000 7f240ea00640 1 -- 172.21.15.167:0/3953919753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f241015f010 msgr2=0x7f24101407a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:19.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.125+0000 7f240ea00640 1 --2- 172.21.15.167:0/3953919753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f241015f010 0x7f24101407a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:19.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.125+0000 7f240ea00640 1 -- 172.21.15.167:0/3953919753 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f241015a2e0 con 0x7f23f4004050 2024-05-11T12:11:19.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.125+0000 7f240ea00640 1 --2- 172.21.15.167:0/3953919753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23f4004050 0x7f2410159da0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2400007a80 tx=0x7f240000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:19.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.125+0000 7f2406a00640 1 -- 172.21.15.167:0/3953919753 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2400014430 con 0x7f23f4004050 2024-05-11T12:11:19.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.125+0000 7f2406a00640 1 -- 172.21.15.167:0/3953919753 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24000145f0 con 0x7f23f4004050 2024-05-11T12:11:19.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.125+0000 7f240fe00640 1 -- 172.21.15.167:0/3953919753 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f241015e250 con 0x7f23f4004050 2024-05-11T12:11:19.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.125+0000 7f2406a00640 1 -- 172.21.15.167:0/3953919753 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24000149e0 con 0x7f23f4004050 2024-05-11T12:11:19.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.125+0000 7f240fe00640 1 -- 172.21.15.167:0/3953919753 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f241015e810 con 0x7f23f4004050 2024-05-11T12:11:19.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.127+0000 7f240fe00640 1 -- 172.21.15.167:0/3953919753 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24100694b0 con 0x7f23f4004050 2024-05-11T12:11:19.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.128+0000 7f2406a00640 1 -- 172.21.15.167:0/3953919753 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2400014b80 con 0x7f23f4004050 2024-05-11T12:11:19.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.129+0000 7f2406a00640 1 --2- 172.21.15.167:0/3953919753 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f23f00762d0 0x7f23f0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:19.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.129+0000 7f2406a00640 1 -- 172.21.15.167:0/3953919753 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f2400097a20 con 0x7f23f4004050 2024-05-11T12:11:19.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.129+0000 7f240e000640 1 --2- 172.21.15.167:0/3953919753 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f23f00762d0 0x7f23f0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:19.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.130+0000 7f240e000640 1 --2- 172.21.15.167:0/3953919753 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f23f00762d0 0x7f23f0078790 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f23fc032000 tx=0x7f23fc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:19.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.133+0000 7f2406a00640 1 -- 172.21.15.167:0/3953919753 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f241006cfb0 con 0x7f23f4004050 2024-05-11T12:11:19.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.341+0000 7f240fe00640 1 -- 172.21.15.167:0/3953919753 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f24100cff40 con 0x7f23f4004050 2024-05-11T12:11:19.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.344+0000 7f2406a00640 1 -- 172.21.15.167:0/3953919753 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18562 (secure 0 0 0) 0x7f2400061040 con 0x7f23f4004050 2024-05-11T12:11:19.345 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:11:19.345 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":27,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:10:50.284769+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:10:43.084511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T12:10:44.157782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-11T12:10:47.814674+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:11:19.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.347+0000 7f240fe00640 1 -- 172.21.15.167:0/3953919753 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f23f00762d0 msgr2=0x7f23f0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:19.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.347+0000 7f240fe00640 1 --2- 172.21.15.167:0/3953919753 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f23f00762d0 0x7f23f0078790 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f23fc032000 tx=0x7f23fc02e000 comp rx=0 tx=0).stop 2024-05-11T12:11:19.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.347+0000 7f240fe00640 1 -- 172.21.15.167:0/3953919753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23f4004050 msgr2=0x7f2410159da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:19.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.347+0000 7f240fe00640 1 --2- 172.21.15.167:0/3953919753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23f4004050 0x7f2410159da0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2400007a80 tx=0x7f240000e530 comp rx=0 tx=0).stop 2024-05-11T12:11:19.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.347+0000 7f240fe00640 1 -- 172.21.15.167:0/3953919753 shutdown_connections 2024-05-11T12:11:19.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.347+0000 7f240fe00640 1 --2- 172.21.15.167:0/3953919753 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f241015f490 0x7f2410140ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:19.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.347+0000 7f240fe00640 1 --2- 172.21.15.167:0/3953919753 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f23f00762d0 0x7f23f0078790 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:19.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.347+0000 7f240fe00640 1 --2- 172.21.15.167:0/3953919753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f241015f010 0x7f24101407a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:19.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.347+0000 7f240fe00640 1 --2- 172.21.15.167:0/3953919753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23f4004050 0x7f2410159da0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:19.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.347+0000 7f240fe00640 1 -- 172.21.15.167:0/3953919753 >> 172.21.15.167:0/3953919753 conn(0x7f24100b3460 msgr2=0x7f24100cf830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:19.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.347+0000 7f240fe00640 1 -- 172.21.15.167:0/3953919753 shutdown_connections 2024-05-11T12:11:19.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.347+0000 7f240fe00640 1 -- 172.21.15.167:0/3953919753 wait complete. 2024-05-11T12:11:19.367 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:11:19.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.574+0000 7f0be8600640 1 Processor -- start 2024-05-11T12:11:19.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.574+0000 7f0be8600640 1 -- start start 2024-05-11T12:11:19.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.575+0000 7f0be8600640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0be013f290 0x7f0be00735c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:19.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.575+0000 7f0be8600640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0be015fb90 con 0x7f0be013eeb0 2024-05-11T12:11:19.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.576+0000 7f0be8600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0be015fe70 con 0x7f0be00b50a0 2024-05-11T12:11:19.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.576+0000 7f0be8600640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f0be015d5b0 con 0x7f0be013f290 2024-05-11T12:11:19.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.576+0000 7f0be5200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0be013f290 0x7f0be00735c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:19.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.576+0000 7f0be5200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0be013f290 0x7f0be00735c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:53660/0 (socket says 172.21.15.167:53660) 2024-05-11T12:11:19.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.576+0000 7f0be5200640 1 -- 172.21.15.167:0/3058614387 learned_addr learned my addr 172.21.15.167:0/3058614387 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:19.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.576+0000 7f0be5c00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f0be013eeb0 0x7f0be006f480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:40860/0 (socket says 172.21.15.167:40860) 2024-05-11T12:11:19.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.576+0000 7f0be5200640 1 -- 172.21.15.167:0/3058614387 >> v1:172.21.15.204:6789/0 conn(0x7f0be00b50a0 legacy=0x7f0be0073b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:19.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.576+0000 7f0be5200640 1 -- 172.21.15.167:0/3058614387 >> v1:172.21.15.167:6789/0 conn(0x7f0be013eeb0 legacy=0x7f0be006f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:19.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.576+0000 7f0bdfe00640 1 -- 172.21.15.167:0/3058614387 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2036513434 0 0) 0x7f0be015fb90 con 0x7f0be013eeb0 2024-05-11T12:11:19.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.576+0000 7f0be5200640 1 -- 172.21.15.167:0/3058614387 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be0160150 con 0x7f0be013f290 2024-05-11T12:11:19.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.577+0000 7f0be5200640 1 --2- 172.21.15.167:0/3058614387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0be013f290 0x7f0be00735c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc009870 tx=0x7f0bcc02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24174e9bdbf6f3c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:19.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.578+0000 7f0bdfe00640 1 -- 172.21.15.167:0/3058614387 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc037070 con 0x7f0be013f290 2024-05-11T12:11:19.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.578+0000 7f0bdfe00640 1 -- 172.21.15.167:0/3058614387 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bcc032440 con 0x7f0be013f290 2024-05-11T12:11:19.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.578+0000 7f0bdfe00640 1 -- 172.21.15.167:0/3058614387 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc032810 con 0x7f0be013f290 2024-05-11T12:11:19.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.578+0000 7f0be8600640 1 -- 172.21.15.167:0/3058614387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0be013f290 msgr2=0x7f0be00735c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:19.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.578+0000 7f0be8600640 1 --2- 172.21.15.167:0/3058614387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0be013f290 0x7f0be00735c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc009870 tx=0x7f0bcc02a1c0 comp rx=0 tx=0).stop 2024-05-11T12:11:19.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.578+0000 7f0be8600640 1 -- 172.21.15.167:0/3058614387 shutdown_connections 2024-05-11T12:11:19.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.578+0000 7f0be8600640 1 --2- 172.21.15.167:0/3058614387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0be013f290 0x7f0be00735c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:19.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.578+0000 7f0be8600640 1 -- 172.21.15.167:0/3058614387 >> 172.21.15.167:0/3058614387 conn(0x7f0be00cd8f0 msgr2=0x7f0be00cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:19.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.579+0000 7f0be8600640 1 -- 172.21.15.167:0/3058614387 shutdown_connections 2024-05-11T12:11:19.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.579+0000 7f0be8600640 1 -- 172.21.15.167:0/3058614387 wait complete. 2024-05-11T12:11:19.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.580+0000 7f0be8600640 1 Processor -- start 2024-05-11T12:11:19.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.580+0000 7f0be8600640 1 -- start start 2024-05-11T12:11:19.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.581+0000 7f0be8600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0be00b50a0 0x7f0be0137fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:19.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.581+0000 7f0be8600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0be013eeb0 0x7f0be0138f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:19.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.581+0000 7f0be8600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0be013f290 0x7f0be013c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:19.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.581+0000 7f0be8600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0be006fb90 con 0x7f0be00b50a0 2024-05-11T12:11:19.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.581+0000 7f0be8600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0be0160710 con 0x7f0be013eeb0 2024-05-11T12:11:19.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.581+0000 7f0be8600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0be006fd10 con 0x7f0be013f290 2024-05-11T12:11:19.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.582+0000 7f0be5c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0be00b50a0 0x7f0be0137fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:19.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.582+0000 7f0be5c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0be00b50a0 0x7f0be0137fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:53674/0 (socket says 172.21.15.167:53674) 2024-05-11T12:11:19.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.582+0000 7f0be5c00640 1 -- 172.21.15.167:0/3969169138 learned_addr learned my addr 172.21.15.167:0/3969169138 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:19.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.582+0000 7f0be5200640 1 --2- 172.21.15.167:0/3969169138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0be013eeb0 0x7f0be0138f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:19.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.582+0000 7f0be6600640 1 --2- 172.21.15.167:0/3969169138 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0be013f290 0x7f0be013c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:19.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.582+0000 7f0be5c00640 1 -- 172.21.15.167:0/3969169138 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0be013f290 msgr2=0x7f0be013c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:19.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.582+0000 7f0be5c00640 1 --2- 172.21.15.167:0/3969169138 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0be013f290 0x7f0be013c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:19.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.582+0000 7f0be5c00640 1 -- 172.21.15.167:0/3969169138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0be013eeb0 msgr2=0x7f0be0138f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:19.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.582+0000 7f0be5c00640 1 --2- 172.21.15.167:0/3969169138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0be013eeb0 0x7f0be0138f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:19.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.582+0000 7f0be5c00640 1 -- 172.21.15.167:0/3969169138 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be015f640 con 0x7f0be00b50a0 2024-05-11T12:11:19.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.582+0000 7f0be5200640 1 --2- 172.21.15.167:0/3969169138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0be013eeb0 0x7f0be0138f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:19.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.582+0000 7f0be6600640 1 --2- 172.21.15.167:0/3969169138 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0be013f290 0x7f0be013c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:19.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.582+0000 7f0be5c00640 1 --2- 172.21.15.167:0/3969169138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0be00b50a0 0x7f0be0137fd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0bd400c4c0 tx=0x7f0bd400c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:19.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.583+0000 7f0bdd600640 1 -- 172.21.15.167:0/3969169138 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd4018070 con 0x7f0be00b50a0 2024-05-11T12:11:19.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.583+0000 7f0bdd600640 1 -- 172.21.15.167:0/3969169138 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bd4007500 con 0x7f0be00b50a0 2024-05-11T12:11:19.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.583+0000 7f0bdd600640 1 -- 172.21.15.167:0/3969169138 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd4004a20 con 0x7f0be00b50a0 2024-05-11T12:11:19.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.583+0000 7f0be8600640 1 -- 172.21.15.167:0/3969169138 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0be0139620 con 0x7f0be00b50a0 2024-05-11T12:11:19.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.583+0000 7f0be8600640 1 -- 172.21.15.167:0/3969169138 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0be0139be0 con 0x7f0be00b50a0 2024-05-11T12:11:19.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.584+0000 7f0be8600640 1 -- 172.21.15.167:0/3969169138 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0be0069590 con 0x7f0be00b50a0 2024-05-11T12:11:19.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.585+0000 7f0bdd600640 1 -- 172.21.15.167:0/3969169138 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0bd401b070 con 0x7f0be00b50a0 2024-05-11T12:11:19.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.586+0000 7f0bdd600640 1 --2- 172.21.15.167:0/3969169138 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0bc00762d0 0x7f0bc0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:19.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.586+0000 7f0bdd600640 1 -- 172.21.15.167:0/3969169138 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f0bd4096cc0 con 0x7f0be00b50a0 2024-05-11T12:11:19.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.586+0000 7f0be5200640 1 --2- 172.21.15.167:0/3969169138 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0bc00762d0 0x7f0bc0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:19.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.587+0000 7f0be5200640 1 --2- 172.21.15.167:0/3969169138 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0bc00762d0 0x7f0bc0078790 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc034000 tx=0x7f0bcc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:19.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.591+0000 7f0bdd600640 1 -- 172.21.15.167:0/3969169138 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0bd409d080 con 0x7f0be00b50a0 2024-05-11T12:11:19.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.835+0000 7f0be8600640 1 -- 172.21.15.167:0/3969169138 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0be0073e10 con 0x7f0be00b50a0 2024-05-11T12:11:19.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.836+0000 7f0bdd600640 1 -- 172.21.15.167:0/3969169138 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5189 (secure 0 0 0) 0x7f0bd4060240 con 0x7f0be00b50a0 2024-05-11T12:11:19.837 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:11:19.837 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3127277486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3127277486},{"type":"v1","addr":"172.21.15.167:6835","nonce":3127277486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2547133600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2547133600},{"type":"v1","addr":"172.21.15.167:6837","nonce":2547133600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3200960100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3200960100},{"type":"v1","addr":"172.21.15.204:6834","nonce":3200960100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:10:45.225745+0000","modified":"2024-05-11T12:11:09.405446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/1889842375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":1889842375},{"type":"v1","addr":"172.21.15.204:6835","nonce":1889842375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-05-11T12:11:19.837 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 9 2024-05-11T12:11:19.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.839+0000 7f0be8600640 1 -- 172.21.15.167:0/3969169138 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0bc00762d0 msgr2=0x7f0bc0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:19.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.839+0000 7f0be8600640 1 --2- 172.21.15.167:0/3969169138 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0bc00762d0 0x7f0bc0078790 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc034000 tx=0x7f0bcc02e000 comp rx=0 tx=0).stop 2024-05-11T12:11:19.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.839+0000 7f0be8600640 1 -- 172.21.15.167:0/3969169138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0be00b50a0 msgr2=0x7f0be0137fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:19.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.839+0000 7f0be8600640 1 --2- 172.21.15.167:0/3969169138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0be00b50a0 0x7f0be0137fd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0bd400c4c0 tx=0x7f0bd400c980 comp rx=0 tx=0).stop 2024-05-11T12:11:19.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.840+0000 7f0be8600640 1 -- 172.21.15.167:0/3969169138 shutdown_connections 2024-05-11T12:11:19.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.840+0000 7f0be8600640 1 --2- 172.21.15.167:0/3969169138 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0be013f290 0x7f0be013c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:19.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.840+0000 7f0be8600640 1 --2- 172.21.15.167:0/3969169138 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0bc00762d0 0x7f0bc0078790 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:19.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.840+0000 7f0be8600640 1 --2- 172.21.15.167:0/3969169138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0be013eeb0 0x7f0be0138f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:19.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.840+0000 7f0be8600640 1 --2- 172.21.15.167:0/3969169138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0be00b50a0 0x7f0be0137fd0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:19.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.840+0000 7f0be8600640 1 -- 172.21.15.167:0/3969169138 >> 172.21.15.167:0/3969169138 conn(0x7f0be00cd8f0 msgr2=0x7f0be00b5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:19.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.840+0000 7f0be8600640 1 -- 172.21.15.167:0/3969169138 shutdown_connections 2024-05-11T12:11:19.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:19.840+0000 7f0be8600640 1 -- 172.21.15.167:0/3969169138 wait complete. 2024-05-11T12:11:20.263 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-11T12:10:45.225745+0000', 'modified': '2024-05-11T12:11:09.405446+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4285}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4285': {'gid': 4285, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.204:6835/1889842375', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6833', 'nonce': 1889842375}, {'type': 'v1', 'addr': '172.21.15.204:6835', 'nonce': 1889842375}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4285, 'qdb_cluster': [4285]} 2024-05-11T12:11:20.263 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-11T12:11:20.263 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-11T12:11:20.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.470+0000 7fb60a400640 1 Processor -- start 2024-05-11T12:11:20.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.471+0000 7fb60a400640 1 -- start start 2024-05-11T12:11:20.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.472+0000 7fb60a400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb60415ac80 0x7fb604074260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:20.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.472+0000 7fb60a400640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6040b65e0 con 0x7fb604161380 2024-05-11T12:11:20.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.472+0000 7fb60a400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6040b68c0 con 0x7fb6040b3d50 2024-05-11T12:11:20.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.472+0000 7fb60a400640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fb60406e3c0 con 0x7fb60415ac80 2024-05-11T12:11:20.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.473+0000 7fb603400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb60415ac80 0x7fb604074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:20.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.473+0000 7fb603400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb60415ac80 0x7fb604074260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:53682/0 (socket says 172.21.15.167:53682) 2024-05-11T12:11:20.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.473+0000 7fb603400640 1 -- 172.21.15.167:0/587117771 learned_addr learned my addr 172.21.15.167:0/587117771 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:20.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.473+0000 7fb603400640 1 -- 172.21.15.167:0/587117771 >> v1:172.21.15.204:6789/0 conn(0x7fb6040b3d50 legacy=0x7fb604160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:20.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.473+0000 7fb603400640 1 -- 172.21.15.167:0/587117771 >> v1:172.21.15.167:6789/0 conn(0x7fb604161380 legacy=0x7fb604072950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:20.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.473+0000 7fb603400640 1 -- 172.21.15.167:0/587117771 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6040b6ba0 con 0x7fb60415ac80 2024-05-11T12:11:20.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.474+0000 7fb603400640 1 --2- 172.21.15.167:0/587117771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb60415ac80 0x7fb604074260 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb5f40098a0 tx=0x7fb5f402a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88f10320ea47228d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:20.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.474+0000 7fb602000640 1 -- 172.21.15.167:0/587117771 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4037070 con 0x7fb60415ac80 2024-05-11T12:11:20.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.475+0000 7fb602000640 1 -- 172.21.15.167:0/587117771 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5f4032440 con 0x7fb60415ac80 2024-05-11T12:11:20.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.475+0000 7fb602000640 1 -- 172.21.15.167:0/587117771 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4032810 con 0x7fb60415ac80 2024-05-11T12:11:20.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.475+0000 7fb60a400640 1 -- 172.21.15.167:0/587117771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb60415ac80 msgr2=0x7fb604074260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:20.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.475+0000 7fb60a400640 1 --2- 172.21.15.167:0/587117771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb60415ac80 0x7fb604074260 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb5f40098a0 tx=0x7fb5f402a290 comp rx=0 tx=0).stop 2024-05-11T12:11:20.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.475+0000 7fb60a400640 1 -- 172.21.15.167:0/587117771 shutdown_connections 2024-05-11T12:11:20.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.475+0000 7fb60a400640 1 --2- 172.21.15.167:0/587117771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb60415ac80 0x7fb604074260 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:20.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.475+0000 7fb60a400640 1 -- 172.21.15.167:0/587117771 >> 172.21.15.167:0/587117771 conn(0x7fb60406c450 msgr2=0x7fb60406c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:20.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.475+0000 7fb60a400640 1 -- 172.21.15.167:0/587117771 shutdown_connections 2024-05-11T12:11:20.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.475+0000 7fb60a400640 1 -- 172.21.15.167:0/587117771 wait complete. 2024-05-11T12:11:20.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.477+0000 7fb60a400640 1 Processor -- start 2024-05-11T12:11:20.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.477+0000 7fb60a400640 1 -- start start 2024-05-11T12:11:20.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.478+0000 7fb60a400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6040b3d50 0x7fb6040aff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:20.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.478+0000 7fb60a400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb60415ac80 0x7fb60415b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:20.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.478+0000 7fb60a400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb604161380 0x7fb6040ae820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:20.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.478+0000 7fb60a400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb6040b44f0 con 0x7fb6040b3d50 2024-05-11T12:11:20.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.478+0000 7fb60a400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb60415b060 con 0x7fb60415ac80 2024-05-11T12:11:20.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.478+0000 7fb60a400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb604073060 con 0x7fb604161380 2024-05-11T12:11:20.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.478+0000 7fb603400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6040b3d50 0x7fb6040aff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:20.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.478+0000 7fb603400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6040b3d50 0x7fb6040aff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:53686/0 (socket says 172.21.15.167:53686) 2024-05-11T12:11:20.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.478+0000 7fb603400640 1 -- 172.21.15.167:0/4175589172 learned_addr learned my addr 172.21.15.167:0/4175589172 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:20.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.479+0000 7fb602a00640 1 --2- 172.21.15.167:0/4175589172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb60415ac80 0x7fb60415b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:20.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.479+0000 7fb603400640 1 -- 172.21.15.167:0/4175589172 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb604161380 msgr2=0x7fb6040ae820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:20.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.479+0000 7fb603400640 1 --2- 172.21.15.167:0/4175589172 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb604161380 0x7fb6040ae820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:20.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.479+0000 7fb603400640 1 -- 172.21.15.167:0/4175589172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb60415ac80 msgr2=0x7fb60415b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:20.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.479+0000 7fb603400640 1 --2- 172.21.15.167:0/4175589172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb60415ac80 0x7fb60415b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:20.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.479+0000 7fb603400640 1 -- 172.21.15.167:0/4175589172 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6040aed60 con 0x7fb6040b3d50 2024-05-11T12:11:20.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.479+0000 7fb602a00640 1 --2- 172.21.15.167:0/4175589172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb60415ac80 0x7fb60415b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:20.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.479+0000 7fb603400640 1 --2- 172.21.15.167:0/4175589172 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6040b3d50 0x7fb6040aff90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4000c00 tx=0x7fb5f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:20.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.479+0000 7fb5fb400640 1 -- 172.21.15.167:0/4175589172 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4041070 con 0x7fb6040b3d50 2024-05-11T12:11:20.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.479+0000 7fb5fb400640 1 -- 172.21.15.167:0/4175589172 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5f403c950 con 0x7fb6040b3d50 2024-05-11T12:11:20.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.479+0000 7fb60a400640 1 -- 172.21.15.167:0/4175589172 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6040aefe0 con 0x7fb6040b3d50 2024-05-11T12:11:20.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.480+0000 7fb5fb400640 1 -- 172.21.15.167:0/4175589172 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4037040 con 0x7fb6040b3d50 2024-05-11T12:11:20.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.480+0000 7fb60a400640 1 -- 172.21.15.167:0/4175589172 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6040af5a0 con 0x7fb6040b3d50 2024-05-11T12:11:20.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.482+0000 7fb5fb400640 1 -- 172.21.15.167:0/4175589172 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb5f4044070 con 0x7fb6040b3d50 2024-05-11T12:11:20.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.483+0000 7fb5fb400640 1 --2- 172.21.15.167:0/4175589172 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb5e4076300 0x7fb5e40787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:20.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.483+0000 7fb5fb400640 1 -- 172.21.15.167:0/4175589172 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fb5f40b8580 con 0x7fb6040b3d50 2024-05-11T12:11:20.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.483+0000 7fb602a00640 1 --2- 172.21.15.167:0/4175589172 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb5e4076300 0x7fb5e40787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:20.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.483+0000 7fb60d244740 1 Processor -- start 2024-05-11T12:11:20.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.484+0000 7fb60d244740 1 -- start start 2024-05-11T12:11:20.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.484+0000 7fb602a00640 1 --2- 172.21.15.167:0/4175589172 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb5e4076300 0x7fb5e40787c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb6040b2630 tx=0x7fb5f0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:20.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.484+0000 7fb60d244740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55a26438d840 0x55a26438dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:20.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.484+0000 7fb60d244740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a2643886d0 0x55a26438c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:20.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.484+0000 7fb60d244740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a26438c980 0x55a26438cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:20.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.484+0000 7fb60d244740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a2642f1790 con 0x55a26438c980 2024-05-11T12:11:20.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.484+0000 7fb60d244740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a264296f20 con 0x55a2643886d0 2024-05-11T12:11:20.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.484+0000 7fb60d244740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55a2642ed750 con 0x55a26438d840 2024-05-11T12:11:20.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.485+0000 7fb603e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a26438c980 0x55a26438cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:20.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.485+0000 7fb603e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a26438c980 0x55a26438cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:53696/0 (socket says 172.21.15.167:53696) 2024-05-11T12:11:20.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.485+0000 7fb603e00640 1 -- 172.21.15.167:0/398230248 learned_addr learned my addr 172.21.15.167:0/398230248 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:20.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.485+0000 7fb603400640 1 --2- 172.21.15.167:0/398230248 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55a26438d840 0x55a26438dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:20.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.485+0000 7fb602a00640 1 --2- 172.21.15.167:0/398230248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a2643886d0 0x55a26438c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:20.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.485+0000 7fb603e00640 1 -- 172.21.15.167:0/398230248 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55a26438d840 msgr2=0x55a26438dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:20.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.485+0000 7fb603e00640 1 --2- 172.21.15.167:0/398230248 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55a26438d840 0x55a26438dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:20.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.485+0000 7fb603e00640 1 -- 172.21.15.167:0/398230248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a2643886d0 msgr2=0x55a26438c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:20.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.485+0000 7fb603e00640 1 --2- 172.21.15.167:0/398230248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a2643886d0 0x55a26438c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:20.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.485+0000 7fb603e00640 1 -- 172.21.15.167:0/398230248 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2640032d0 con 0x55a26438c980 2024-05-11T12:11:20.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.486+0000 7fb603e00640 1 --2- 172.21.15.167:0/398230248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a26438c980 0x55a26438cdc0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc001c80 tx=0x7fb5fc002140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae8827ad1fabbf39 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:20.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.486+0000 7fb5efe00640 1 -- 172.21.15.167:0/398230248 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc017070 con 0x55a26438c980 2024-05-11T12:11:20.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.486+0000 7fb5efe00640 1 -- 172.21.15.167:0/398230248 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5fc00cc70 con 0x55a26438c980 2024-05-11T12:11:20.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.486+0000 7fb5efe00640 1 -- 172.21.15.167:0/398230248 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc003b70 con 0x55a26438c980 2024-05-11T12:11:20.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.486+0000 7fb60d244740 1 -- 172.21.15.167:0/398230248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a26438c980 msgr2=0x55a26438cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:20.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.486+0000 7fb60d244740 1 --2- 172.21.15.167:0/398230248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a26438c980 0x55a26438cdc0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc001c80 tx=0x7fb5fc002140 comp rx=0 tx=0).stop 2024-05-11T12:11:20.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.487+0000 7fb60d244740 1 -- 172.21.15.167:0/398230248 shutdown_connections 2024-05-11T12:11:20.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.487+0000 7fb60d244740 1 --2- 172.21.15.167:0/398230248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a26438c980 0x55a26438cdc0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:20.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.487+0000 7fb60d244740 1 --2- 172.21.15.167:0/398230248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a2643886d0 0x55a26438c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:20.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.487+0000 7fb60d244740 1 --2- 172.21.15.167:0/398230248 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55a26438d840 0x55a26438dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:20.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.487+0000 7fb60d244740 1 -- 172.21.15.167:0/398230248 >> 172.21.15.167:0/398230248 conn(0x55a26438a940 msgr2=0x55a2643d3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:20.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.487+0000 7fb60d244740 1 -- 172.21.15.167:0/398230248 shutdown_connections 2024-05-11T12:11:20.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.487+0000 7fb60d244740 1 -- 172.21.15.167:0/398230248 wait complete. 2024-05-11T12:11:20.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.487+0000 7fb60d244740 1 -- start start 2024-05-11T12:11:20.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.488+0000 7fb60d244740 1 Processor -- start 2024-05-11T12:11:20.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.489+0000 7fb60d244740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a26438c980 0x55a2643ae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:20.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.489+0000 7fb60d244740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55a26438d840 0x55a2643a8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:20.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.489+0000 7fb60d244740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a2643a8680 0x55a2643a9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:20.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.489+0000 7fb60d244740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a2643b93d0 con 0x55a26438c980 2024-05-11T12:11:20.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.489+0000 7fb60d244740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a2642aa9c0 con 0x55a2643a8680 2024-05-11T12:11:20.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.489+0000 7fb60d244740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55a2643b8850 con 0x55a26438d840 2024-05-11T12:11:20.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.489+0000 7fb60d244740 10 client.0 _ll_register_callbacks cb 0x55a264277120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:11:20.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.489+0000 7fb603400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a26438c980 0x55a2643ae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:20.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.489+0000 7fb603400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a26438c980 0x55a2643ae1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:53712/0 (socket says 172.21.15.167:53712) 2024-05-11T12:11:20.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.489+0000 7fb603400640 1 -- 172.21.15.167:0/2201125213 learned_addr learned my addr 172.21.15.167:0/2201125213 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:20.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.489+0000 7fb603e00640 1 --2- 172.21.15.167:0/2201125213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a2643a8680 0x55a2643a9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:20.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.489+0000 7fb602a00640 1 --2- 172.21.15.167:0/2201125213 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55a26438d840 0x55a2643a8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:20.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.489+0000 7fb603400640 1 -- 172.21.15.167:0/2201125213 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55a26438d840 msgr2=0x55a2643a8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:20.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.489+0000 7fb603400640 1 --2- 172.21.15.167:0/2201125213 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55a26438d840 0x55a2643a8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:20.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.489+0000 7fb603400640 1 -- 172.21.15.167:0/2201125213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a2643a8680 msgr2=0x55a2643a9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:20.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.489+0000 7fb603400640 1 --2- 172.21.15.167:0/2201125213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a2643a8680 0x55a2643a9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:20.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.489+0000 7fb603400640 1 -- 172.21.15.167:0/2201125213 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2640cd6b0 con 0x55a26438c980 2024-05-11T12:11:20.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.490+0000 7fb603400640 1 --2- 172.21.15.167:0/2201125213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a26438c980 0x55a2643ae1f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4081390 tx=0x7fb5f4080c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32532045958b9b82 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:20.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.490+0000 7fb5ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-11T12:11:20.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.490+0000 7fb5ecc00640 1 -- 172.21.15.167:0/2201125213 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f404b040 con 0x55a26438c980 2024-05-11T12:11:20.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.490+0000 7fb5ecc00640 1 -- 172.21.15.167:0/2201125213 <== mon.0 v2:172.21.15.167:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fb5f40b82f0 con 0x55a26438c980 2024-05-11T12:11:20.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.490+0000 7fb60d244740 1 -- 172.21.15.167:0/2201125213 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5fc00cc70 con 0x55a26438c980 2024-05-11T12:11:20.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.490+0000 7fb5ecc00640 1 -- 172.21.15.167:0/2201125213 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7fb5f4081400 con 0x55a26438c980 2024-05-11T12:11:20.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.490+0000 7fb60d244740 10 client.4683 fetch_fsmap learned FSMap version 9 2024-05-11T12:11:20.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.490+0000 7fb60d244740 1 -- 172.21.15.167:0/2201125213 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x55a2643a6c00 con 0x55a26438c980 2024-05-11T12:11:20.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.490+0000 7fb5ecc00640 1 -- 172.21.15.167:0/2201125213 <== mon.0 v2:172.21.15.167:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5fc00cc70 con 0x55a26438c980 2024-05-11T12:11:20.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.490+0000 7fb5ecc00640 1 -- 172.21.15.167:0/2201125213 <== mon.0 v2:172.21.15.167:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7fb5f4089d40 con 0x55a26438c980 2024-05-11T12:11:20.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.491+0000 7fb60d244740 10 client.4683 fetch_fsmap finished waiting for FSMap version 9 2024-05-11T12:11:20.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.491+0000 7fb60d244740 10 client.4683 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-11T12:11:20.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.491+0000 7fb60d244740 20 client.4683 populate_metadata read hostname 'smithi167' 2024-05-11T12:11:20.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.491+0000 7fb60d244740 1 --2- 172.21.15.167:0/2201125213 >> [v2:172.21.15.204:6833/1889842375,v1:172.21.15.204:6835/1889842375] conn(0x55a2643e71c0 0x55a2643a71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:20.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.491+0000 7fb60d244740 4 client.4683 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:11:20.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.491+0000 7fb60d244740 1 -- 172.21.15.167:0/2201125213 --> [v2:172.21.15.204:6833/1889842375,v1:172.21.15.204:6835/1889842375] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2642ff3a0 con 0x55a2643e71c0 2024-05-11T12:11:20.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.491+0000 7fb602a00640 1 --2- 172.21.15.167:0/2201125213 >> [v2:172.21.15.204:6833/1889842375,v1:172.21.15.204:6835/1889842375] conn(0x55a2643e71c0 0x55a2643a71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:20.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.492+0000 7fb602a00640 1 --2- 172.21.15.167:0/2201125213 >> [v2:172.21.15.204:6833/1889842375,v1:172.21.15.204:6835/1889842375] conn(0x55a2643e71c0 0x55a2643a71b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f380e68f594a97f server_cookie=bf31d3e843da0c04 in_seq=0 out_seq=0 2024-05-11T12:11:20.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.492+0000 7fb5ecc00640 10 client.4683 ms_handle_connect on v2:172.21.15.204:6833/1889842375 2024-05-11T12:11:20.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.494+0000 7fb5ecc00640 1 -- 172.21.15.167:0/2201125213 <== mds.0 v2:172.21.15.204:6833/1889842375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a2642ff3a0 con 0x55a2643e71c0 2024-05-11T12:11:20.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.494+0000 7fb5ecc00640 10 client.4683 handle_command_reply: tid=0 2024-05-11T12:11:20.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.494+0000 7fb60d244740 1 client.4683 shutdown 2024-05-11T12:11:20.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.494+0000 7fb60d244740 1 -- 172.21.15.167:0/2201125213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a26438c980 msgr2=0x55a2643ae1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:20.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.494+0000 7fb60d244740 1 --2- 172.21.15.167:0/2201125213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a26438c980 0x55a2643ae1f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4081390 tx=0x7fb5f4080c30 comp rx=0 tx=0).stop 2024-05-11T12:11:20.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.494+0000 7fb60d244740 1 -- 172.21.15.167:0/2201125213 shutdown_connections 2024-05-11T12:11:20.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.494+0000 7fb60d244740 1 --2- 172.21.15.167:0/2201125213 >> [v2:172.21.15.204:6833/1889842375,v1:172.21.15.204:6835/1889842375] conn(0x55a2643e71c0 0x55a2643a71b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:20.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.494+0000 7fb60d244740 1 --2- 172.21.15.167:0/2201125213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a2643a8680 0x55a2643a9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:20.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.494+0000 7fb60d244740 1 --2- 172.21.15.167:0/2201125213 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55a26438d840 0x55a2643a8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:20.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.494+0000 7fb60d244740 1 --2- 172.21.15.167:0/2201125213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a26438c980 0x55a2643ae1f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:20.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.494+0000 7fb5ecc00640 0 client.4683 ms_handle_reset on v2:172.21.15.204:6833/1889842375 2024-05-11T12:11:20.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.494+0000 7fb60d244740 1 -- 172.21.15.167:0/2201125213 >> 172.21.15.167:0/2201125213 conn(0x55a2643886d0 msgr2=0x55a264387df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:20.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.494+0000 7fb60d244740 1 -- 172.21.15.167:0/2201125213 shutdown_connections 2024-05-11T12:11:20.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.494+0000 7fb60d244740 1 -- 172.21.15.167:0/2201125213 wait complete. 2024-05-11T12:11:20.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.495+0000 7fb60d244740 20 client.4683 trim_cache size 0 max 16384 2024-05-11T12:11:20.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.515+0000 7fb60d244740 1 Processor -- start 2024-05-11T12:11:20.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.515+0000 7fb60d244740 1 -- start start 2024-05-11T12:11:20.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.515+0000 7fb60d244740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55a26438c980 0x55a2643a9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:20.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.515+0000 7fb60d244740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a26438d840 0x55a264383d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:20.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.515+0000 7fb60d244740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a2643a8680 0x55a2643b76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:20.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.515+0000 7fb60d244740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb604202150 con 0x55a26438d840 2024-05-11T12:11:20.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.515+0000 7fb60d244740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb6042015d0 con 0x55a2643a8680 2024-05-11T12:11:20.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.515+0000 7fb60d244740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb604201b00 con 0x55a26438c980 2024-05-11T12:11:20.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.516+0000 7fb602a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a26438d840 0x55a264383d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:20.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.516+0000 7fb602a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a26438d840 0x55a264383d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:53722/0 (socket says 172.21.15.167:53722) 2024-05-11T12:11:20.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.516+0000 7fb602a00640 1 -- 172.21.15.167:0/1571268947 learned_addr learned my addr 172.21.15.167:0/1571268947 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:20.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.516+0000 7fb603400640 1 --2- 172.21.15.167:0/1571268947 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55a26438c980 0x55a2643a9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:20.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.516+0000 7fb602a00640 1 -- 172.21.15.167:0/1571268947 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55a26438c980 msgr2=0x55a2643a9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:20.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.516+0000 7fb602a00640 1 --2- 172.21.15.167:0/1571268947 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55a26438c980 0x55a2643a9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:20.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.516+0000 7fb602a00640 1 -- 172.21.15.167:0/1571268947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a2643a8680 msgr2=0x55a2643b76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:20.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.516+0000 7fb602a00640 1 --2- 172.21.15.167:0/1571268947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a2643a8680 0x55a2643b76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:20.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.516+0000 7fb602a00640 1 -- 172.21.15.167:0/1571268947 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2643a6060 con 0x55a26438d840 2024-05-11T12:11:20.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.516+0000 7fb603400640 1 --2- 172.21.15.167:0/1571268947 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55a26438c980 0x55a2643a9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:20.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.516+0000 7fb602a00640 1 --2- 172.21.15.167:0/1571268947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a26438d840 0x55a264383d00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb6041f89d0 tx=0x7fb5f0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21151d3a5e0fc5ff server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:20.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.517+0000 7fb5ef400640 1 -- 172.21.15.167:0/1571268947 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f0029070 con 0x55a26438d840 2024-05-11T12:11:20.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.517+0000 7fb5ef400640 1 -- 172.21.15.167:0/1571268947 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5f0014af0 con 0x55a26438d840 2024-05-11T12:11:20.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.517+0000 7fb5ef400640 1 -- 172.21.15.167:0/1571268947 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f0007050 con 0x55a26438d840 2024-05-11T12:11:20.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.517+0000 7fb60d244740 1 -- 172.21.15.167:0/1571268947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a26438d840 msgr2=0x55a264383d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:20.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.517+0000 7fb60d244740 1 --2- 172.21.15.167:0/1571268947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a26438d840 0x55a264383d00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb6041f89d0 tx=0x7fb5f0013000 comp rx=0 tx=0).stop 2024-05-11T12:11:20.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.517+0000 7fb60d244740 1 -- 172.21.15.167:0/1571268947 shutdown_connections 2024-05-11T12:11:20.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.517+0000 7fb60d244740 1 --2- 172.21.15.167:0/1571268947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a2643a8680 0x55a2643b76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:20.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.517+0000 7fb60d244740 1 --2- 172.21.15.167:0/1571268947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a26438d840 0x55a264383d00 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:20.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.517+0000 7fb60d244740 1 --2- 172.21.15.167:0/1571268947 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55a26438c980 0x55a2643a9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:20.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.517+0000 7fb60d244740 1 -- 172.21.15.167:0/1571268947 >> 172.21.15.167:0/1571268947 conn(0x55a2643886d0 msgr2=0x55a26438b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:20.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.517+0000 7fb60d244740 1 -- 172.21.15.167:0/1571268947 shutdown_connections 2024-05-11T12:11:20.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.517+0000 7fb60d244740 1 -- 172.21.15.167:0/1571268947 wait complete. 2024-05-11T12:11:20.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.518+0000 7fb60d244740 1 -- start start 2024-05-11T12:11:20.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.518+0000 7fb60d244740 1 Processor -- start 2024-05-11T12:11:20.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.519+0000 7fb60d244740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a26438c980 0x55a264445b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:20.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.519+0000 7fb60d244740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a26438d840 0x55a2644460a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:20.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.519+0000 7fb60d244740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55a2643a8680 0x55a2643f6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:20.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.519+0000 7fb60d244740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a2643e7040 con 0x55a26438d840 2024-05-11T12:11:20.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.519+0000 7fb60d244740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a2643aa1a0 con 0x55a26438c980 2024-05-11T12:11:20.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.519+0000 7fb60d244740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55a2643e7660 con 0x55a2643a8680 2024-05-11T12:11:20.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.519+0000 7fb60d244740 10 client.0 _ll_register_callbacks cb 0x7fb5f40094d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:11:20.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.520+0000 7fb602a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a26438d840 0x55a2644460a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:20.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.520+0000 7fb602a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a26438d840 0x55a2644460a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:53734/0 (socket says 172.21.15.167:53734) 2024-05-11T12:11:20.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.520+0000 7fb602a00640 1 -- 172.21.15.167:0/3096188117 learned_addr learned my addr 172.21.15.167:0/3096188117 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:20.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.520+0000 7fb602a00640 1 -- 172.21.15.167:0/3096188117 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55a2643a8680 msgr2=0x55a2643f6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:20.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.520+0000 7fb603400640 1 --2- 172.21.15.167:0/3096188117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a26438c980 0x55a264445b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:20.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.520+0000 7fb602a00640 1 --2- 172.21.15.167:0/3096188117 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55a2643a8680 0x55a2643f6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:20.525 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.520+0000 7fb602a00640 1 -- 172.21.15.167:0/3096188117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a26438c980 msgr2=0x55a264445b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:20.525 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.520+0000 7fb602a00640 1 --2- 172.21.15.167:0/3096188117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a26438c980 0x55a264445b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:20.525 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.520+0000 7fb602a00640 1 -- 172.21.15.167:0/3096188117 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2643e5be0 con 0x55a26438d840 2024-05-11T12:11:20.525 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.520+0000 7fb602a00640 1 --2- 172.21.15.167:0/3096188117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a26438d840 0x55a2644460a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb6041605d0 tx=0x7fb5f00036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f36758663e334a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:20.525 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.520+0000 7fb5ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-11T12:11:20.525 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.521+0000 7fb5ecc00640 1 -- 172.21.15.167:0/3096188117 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f0029040 con 0x55a26438d840 2024-05-11T12:11:20.525 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.521+0000 7fb5ecc00640 1 -- 172.21.15.167:0/3096188117 <== mon.0 v2:172.21.15.167:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fb5f0005240 con 0x55a26438d840 2024-05-11T12:11:20.525 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.521+0000 7fb60d244740 1 -- 172.21.15.167:0/3096188117 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5f0014af0 con 0x55a26438d840 2024-05-11T12:11:20.525 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.521+0000 7fb5ecc00640 1 -- 172.21.15.167:0/3096188117 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7fb5f00067d0 con 0x55a26438d840 2024-05-11T12:11:20.525 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.521+0000 7fb60d244740 10 client.4689 fetch_fsmap learned FSMap version 9 2024-05-11T12:11:20.525 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.521+0000 7fb60d244740 1 -- 172.21.15.167:0/3096188117 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x55a2644451c0 con 0x55a26438d840 2024-05-11T12:11:20.525 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.521+0000 7fb5ecc00640 1 -- 172.21.15.167:0/3096188117 <== mon.0 v2:172.21.15.167:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5f0014af0 con 0x55a26438d840 2024-05-11T12:11:20.525 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.521+0000 7fb5ecc00640 1 -- 172.21.15.167:0/3096188117 <== mon.0 v2:172.21.15.167:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7fb5f002c030 con 0x55a26438d840 2024-05-11T12:11:20.525 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.521+0000 7fb60d244740 10 client.4689 fetch_fsmap finished waiting for FSMap version 9 2024-05-11T12:11:20.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.521+0000 7fb60d244740 10 client.4689 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-11T12:11:20.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.521+0000 7fb60d244740 20 client.4689 populate_metadata read hostname 'smithi167' 2024-05-11T12:11:20.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.521+0000 7fb60d244740 1 --2- 172.21.15.167:0/3096188117 >> [v2:172.21.15.204:6833/1889842375,v1:172.21.15.204:6835/1889842375] conn(0x55a2643f7210 0x55a264444140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:20.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.521+0000 7fb60d244740 4 client.4689 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-11T12:11:20.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.521+0000 7fb60d244740 1 -- 172.21.15.167:0/3096188117 --> [v2:172.21.15.204:6833/1889842375,v1:172.21.15.204:6835/1889842375] -- command(tid 0: {"prefix": "status"}) -- 0x55a2642f9210 con 0x55a2643f7210 2024-05-11T12:11:20.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.522+0000 7fb603400640 1 --2- 172.21.15.167:0/3096188117 >> [v2:172.21.15.204:6833/1889842375,v1:172.21.15.204:6835/1889842375] conn(0x55a2643f7210 0x55a264444140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:20.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.523+0000 7fb603400640 1 --2- 172.21.15.167:0/3096188117 >> [v2:172.21.15.204:6833/1889842375,v1:172.21.15.204:6835/1889842375] conn(0x55a2643f7210 0x55a264444140 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa91c8bf67e0c202 server_cookie=4a26c3d4c56bab95 in_seq=0 out_seq=0 2024-05-11T12:11:20.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.523+0000 7fb5ecc00640 10 client.4689 ms_handle_connect on v2:172.21.15.204:6833/1889842375 2024-05-11T12:11:20.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.524+0000 7fb5ecc00640 1 -- 172.21.15.167:0/3096188117 <== mds.0 v2:172.21.15.204:6833/1889842375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55a2642f9210 con 0x55a2643f7210 2024-05-11T12:11:20.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.524+0000 7fb5ecc00640 10 client.4689 handle_command_reply: tid=0 2024-05-11T12:11:20.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.524+0000 7fb60d244740 1 client.4689 shutdown 2024-05-11T12:11:20.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.524+0000 7fb60d244740 1 -- 172.21.15.167:0/3096188117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a26438d840 msgr2=0x55a2644460a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:20.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.524+0000 7fb60d244740 1 --2- 172.21.15.167:0/3096188117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a26438d840 0x55a2644460a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb6041605d0 tx=0x7fb5f00036f0 comp rx=0 tx=0).stop 2024-05-11T12:11:20.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.524+0000 7fb60d244740 1 -- 172.21.15.167:0/3096188117 shutdown_connections 2024-05-11T12:11:20.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.524+0000 7fb60d244740 1 --2- 172.21.15.167:0/3096188117 >> [v2:172.21.15.204:6833/1889842375,v1:172.21.15.204:6835/1889842375] conn(0x55a2643f7210 0x55a264444140 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:20.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.524+0000 7fb60d244740 1 --2- 172.21.15.167:0/3096188117 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55a2643a8680 0x55a2643f6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:20.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.524+0000 7fb60d244740 1 --2- 172.21.15.167:0/3096188117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a26438d840 0x55a2644460a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:20.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.524+0000 7fb60d244740 1 --2- 172.21.15.167:0/3096188117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a26438c980 0x55a264445b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:20.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.524+0000 7fb5ecc00640 0 client.4689 ms_handle_reset on v2:172.21.15.204:6833/1889842375 2024-05-11T12:11:20.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.524+0000 7fb60d244740 1 -- 172.21.15.167:0/3096188117 >> 172.21.15.167:0/3096188117 conn(0x55a2643886d0 msgr2=0x55a264446c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:20.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.524+0000 7fb60d244740 1 -- 172.21.15.167:0/3096188117 shutdown_connections 2024-05-11T12:11:20.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.524+0000 7fb60d244740 1 -- 172.21.15.167:0/3096188117 wait complete. 2024-05-11T12:11:20.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.524+0000 7fb60d244740 20 client.4689 trim_cache size 0 max 16384 2024-05-11T12:11:20.529 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T12:11:20.530 INFO:teuthology.orchestra.run.smithi167.stdout: "cluster_fsid": "5f32d344-7289-4c27-881b-1533419ab678", 2024-05-11T12:11:20.530 INFO:teuthology.orchestra.run.smithi167.stdout: "whoami": 0, 2024-05-11T12:11:20.530 INFO:teuthology.orchestra.run.smithi167.stdout: "id": 4285, 2024-05-11T12:11:20.530 INFO:teuthology.orchestra.run.smithi167.stdout: "want_state": "up:active", 2024-05-11T12:11:20.530 INFO:teuthology.orchestra.run.smithi167.stdout: "state": "up:active", 2024-05-11T12:11:20.530 INFO:teuthology.orchestra.run.smithi167.stdout: "fs_name": "cephfs", 2024-05-11T12:11:20.530 INFO:teuthology.orchestra.run.smithi167.stdout: "rank_uptime": 34.755620671000003, 2024-05-11T12:11:20.530 INFO:teuthology.orchestra.run.smithi167.stdout: "mdsmap_epoch": 9, 2024-05-11T12:11:20.530 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_epoch": 27, 2024-05-11T12:11:20.530 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_epoch_barrier": 27, 2024-05-11T12:11:20.531 INFO:teuthology.orchestra.run.smithi167.stdout: "uptime": 41.101037319 2024-05-11T12:11:20.531 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T12:11:20.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.525+0000 7fb60a400640 1 -- 172.21.15.167:0/4175589172 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb5e4076300 msgr2=0x7fb5e40787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:20.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.525+0000 7fb60a400640 1 --2- 172.21.15.167:0/4175589172 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb5e4076300 0x7fb5e40787c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb6040b2630 tx=0x7fb5f0009040 comp rx=0 tx=0).stop 2024-05-11T12:11:20.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.525+0000 7fb60a400640 1 -- 172.21.15.167:0/4175589172 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6040b3d50 msgr2=0x7fb6040aff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:20.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.525+0000 7fb60a400640 1 --2- 172.21.15.167:0/4175589172 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6040b3d50 0x7fb6040aff90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4000c00 tx=0x7fb5f40046c0 comp rx=0 tx=0).stop 2024-05-11T12:11:20.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.525+0000 7fb60a400640 1 -- 172.21.15.167:0/4175589172 shutdown_connections 2024-05-11T12:11:20.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.525+0000 7fb60a400640 1 --2- 172.21.15.167:0/4175589172 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb604161380 0x7fb6040ae820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:20.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.525+0000 7fb60a400640 1 --2- 172.21.15.167:0/4175589172 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb5e4076300 0x7fb5e40787c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:20.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.525+0000 7fb60a400640 1 --2- 172.21.15.167:0/4175589172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb60415ac80 0x7fb60415b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:20.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.525+0000 7fb60a400640 1 --2- 172.21.15.167:0/4175589172 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6040b3d50 0x7fb6040aff90 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:20.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.525+0000 7fb60a400640 1 -- 172.21.15.167:0/4175589172 >> 172.21.15.167:0/4175589172 conn(0x7fb60406c450 msgr2=0x7fb60415e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:20.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.525+0000 7fb60a400640 1 -- 172.21.15.167:0/4175589172 shutdown_connections 2024-05-11T12:11:20.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:20.525+0000 7fb60a400640 1 -- 172.21.15.167:0/4175589172 wait complete. 2024-05-11T12:11:20.543 INFO:teuthology.run_tasks:Running task sequential... 2024-05-11T12:11:20.552 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-11T12:11:20.552 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-11T12:11:20.553 INFO:teuthology.task.print:Enabling mgr modules 2024-05-11T12:11:20.553 INFO:teuthology.run_tasks:Running task kclient... 2024-05-11T12:11:20.602 INFO:tasks.kclient:Mounting kernel clients... 2024-05-11T12:11:20.602 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-05-11T12:11:20.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:20.603 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2024-05-11T12:11:20.656 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-05-11T12:11:20.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:20.658 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-11T12:11:20.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:20.741 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link delete ceph-brx 2024-05-11T12:11:20.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:20.857 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2024-05-11T12:11:20.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:20.899 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link delete ceph-brx 2024-05-11T12:11:21.022 INFO:teuthology.orchestra.run.smithi167.stderr:Cannot find device "ceph-brx" 2024-05-11T12:11:21.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T12:11:21.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:21.026 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list 2024-05-11T12:11:21.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:21.082 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link delete ceph-brx 2024-05-11T12:11:21.168 INFO:teuthology.orchestra.run.smithi204.stderr:Cannot find device "ceph-brx" 2024-05-11T12:11:21.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T12:11:21.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:21.173 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list 2024-05-11T12:11:21.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:21.228 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link delete ceph-brx 2024-05-11T12:11:21.310 INFO:teuthology.orchestra.run.smithi204.stderr:Cannot find device "ceph-brx" 2024-05-11T12:11:21.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T12:11:21.315 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-05-11T12:11:21.315 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-11T12:11:21.315 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-11T12:11:21.315 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-11T12:11:21.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.524+0000 7f7f3f200640 1 Processor -- start 2024-05-11T12:11:21.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.524+0000 7f7f3f200640 1 -- start start 2024-05-11T12:11:21.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.526+0000 7f7f3f200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7f400b5930 0x7f7f4006d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:21.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.526+0000 7f7f3f200640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f40071f30 con 0x7f7f400b3a80 2024-05-11T12:11:21.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.526+0000 7f7f3f200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f40072210 con 0x7f7f401458f0 2024-05-11T12:11:21.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.526+0000 7f7f3f200640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f7f4006db80 con 0x7f7f400b5930 2024-05-11T12:11:21.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.526+0000 7f7f3e800640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7f400b5930 0x7f7f4006d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:21.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.526+0000 7f7f3e800640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7f400b5930 0x7f7f4006d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:53742/0 (socket says 172.21.15.167:53742) 2024-05-11T12:11:21.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.526+0000 7f7f3e800640 1 -- 172.21.15.167:0/2201230062 learned_addr learned my addr 172.21.15.167:0/2201230062 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:21.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.526+0000 7f7f37e00640 1 --1- 172.21.15.167:0/2201230062 >> v1:172.21.15.167:6789/0 conn(0x7f7f400b3a80 0x7f7f400b5220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:40880/0 (socket says 172.21.15.167:40880) 2024-05-11T12:11:21.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.527+0000 7f7f3e800640 1 -- 172.21.15.167:0/2201230062 >> v1:172.21.15.204:6789/0 conn(0x7f7f401458f0 legacy=0x7f7f400b4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:21.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.527+0000 7f7f3e800640 1 -- 172.21.15.167:0/2201230062 >> v1:172.21.15.167:6789/0 conn(0x7f7f400b3a80 legacy=0x7f7f400b5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:21.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.527+0000 7f7f3e800640 1 -- 172.21.15.167:0/2201230062 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f400724f0 con 0x7f7f400b5930 2024-05-11T12:11:21.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.528+0000 7f7f3e800640 1 --2- 172.21.15.167:0/2201230062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f400b5930 0x7f7f4006d640 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7f3800d150 tx=0x7f7f3802c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=384eada8099be79e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:21.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.528+0000 7f7f3d400640 1 -- 172.21.15.167:0/2201230062 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f38009600 con 0x7f7f400b5930 2024-05-11T12:11:21.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.528+0000 7f7f3d400640 1 -- 172.21.15.167:0/2201230062 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f38037070 con 0x7f7f400b5930 2024-05-11T12:11:21.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.528+0000 7f7f3d400640 1 -- 172.21.15.167:0/2201230062 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f380099c0 con 0x7f7f400b5930 2024-05-11T12:11:21.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.529+0000 7f7f3f200640 1 -- 172.21.15.167:0/2201230062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f400b5930 msgr2=0x7f7f4006d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:21.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.529+0000 7f7f3f200640 1 --2- 172.21.15.167:0/2201230062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f400b5930 0x7f7f4006d640 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7f3800d150 tx=0x7f7f3802c930 comp rx=0 tx=0).stop 2024-05-11T12:11:21.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.529+0000 7f7f3f200640 1 -- 172.21.15.167:0/2201230062 shutdown_connections 2024-05-11T12:11:21.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.529+0000 7f7f3f200640 1 --2- 172.21.15.167:0/2201230062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f400b5930 0x7f7f4006d640 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:21.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.529+0000 7f7f3f200640 1 -- 172.21.15.167:0/2201230062 >> 172.21.15.167:0/2201230062 conn(0x7f7f400b3460 msgr2=0x7f7f400d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:21.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.529+0000 7f7f3f200640 1 -- 172.21.15.167:0/2201230062 shutdown_connections 2024-05-11T12:11:21.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.529+0000 7f7f3f200640 1 -- 172.21.15.167:0/2201230062 wait complete. 2024-05-11T12:11:21.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.531+0000 7f7f3f200640 1 Processor -- start 2024-05-11T12:11:21.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.531+0000 7f7f3f200640 1 -- start start 2024-05-11T12:11:21.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.532+0000 7f7f3f200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f400b3a80 0x7f7f4015ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:21.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.532+0000 7f7f3f200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7f400b5930 0x7f7f4013c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:21.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.532+0000 7f7f3f200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f401458f0 0x7f7f40140270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:21.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.532+0000 7f7f3f200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7f40072fd0 con 0x7f7f400b3a80 2024-05-11T12:11:21.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.532+0000 7f7f3f200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7f40072ab0 con 0x7f7f401458f0 2024-05-11T12:11:21.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.532+0000 7f7f3f200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f7f40073150 con 0x7f7f400b5930 2024-05-11T12:11:21.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.532+0000 7f7f3de00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f400b3a80 0x7f7f4015ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:21.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.532+0000 7f7f3de00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f400b3a80 0x7f7f4015ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:53754/0 (socket says 172.21.15.167:53754) 2024-05-11T12:11:21.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.532+0000 7f7f3de00640 1 -- 172.21.15.167:0/1432048937 learned_addr learned my addr 172.21.15.167:0/1432048937 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:21.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.532+0000 7f7f37e00640 1 --2- 172.21.15.167:0/1432048937 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7f400b5930 0x7f7f4013c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:21.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.532+0000 7f7f3e800640 1 --2- 172.21.15.167:0/1432048937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f401458f0 0x7f7f40140270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:21.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.532+0000 7f7f3de00640 1 -- 172.21.15.167:0/1432048937 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7f400b5930 msgr2=0x7f7f4013c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:21.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.532+0000 7f7f3de00640 1 --2- 172.21.15.167:0/1432048937 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7f400b5930 0x7f7f4013c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:21.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.532+0000 7f7f3de00640 1 -- 172.21.15.167:0/1432048937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f401458f0 msgr2=0x7f7f40140270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:21.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.532+0000 7f7f3de00640 1 --2- 172.21.15.167:0/1432048937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f401458f0 0x7f7f40140270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:21.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.532+0000 7f7f3de00640 1 -- 172.21.15.167:0/1432048937 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4015f420 con 0x7f7f400b3a80 2024-05-11T12:11:21.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.533+0000 7f7f3de00640 1 --2- 172.21.15.167:0/1432048937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f400b3a80 0x7f7f4015ebd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7f3000b420 tx=0x7f7f3000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:21.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.533+0000 7f7f36000640 1 -- 172.21.15.167:0/1432048937 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f30011830 con 0x7f7f400b3a80 2024-05-11T12:11:21.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.533+0000 7f7f36000640 1 -- 172.21.15.167:0/1432048937 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f30016070 con 0x7f7f400b3a80 2024-05-11T12:11:21.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.533+0000 7f7f36000640 1 -- 172.21.15.167:0/1432048937 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f30011c10 con 0x7f7f400b3a80 2024-05-11T12:11:21.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.533+0000 7f7f3f200640 1 -- 172.21.15.167:0/1432048937 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f4015f700 con 0x7f7f400b3a80 2024-05-11T12:11:21.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.533+0000 7f7f3f200640 1 -- 172.21.15.167:0/1432048937 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f4013d160 con 0x7f7f400b3a80 2024-05-11T12:11:21.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.534+0000 7f7f3f200640 1 -- 172.21.15.167:0/1432048937 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f400694b0 con 0x7f7f400b3a80 2024-05-11T12:11:21.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.535+0000 7f7f36000640 1 -- 172.21.15.167:0/1432048937 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7f3001b460 con 0x7f7f400b3a80 2024-05-11T12:11:21.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.537+0000 7f7f36000640 1 --2- 172.21.15.167:0/1432048937 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f7f1c0762d0 0x7f7f1c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:21.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.537+0000 7f7f36000640 1 -- 172.21.15.167:0/1432048937 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f7f300974c0 con 0x7f7f400b3a80 2024-05-11T12:11:21.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.537+0000 7f7f37e00640 1 --2- 172.21.15.167:0/1432048937 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f7f1c0762d0 0x7f7f1c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:21.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.538+0000 7f7f37e00640 1 --2- 172.21.15.167:0/1432048937 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f7f1c0762d0 0x7f7f1c078790 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7f28000f60 tx=0x7f7f28007c80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:21.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.542+0000 7f7f36000640 1 -- 172.21.15.167:0/1432048937 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7f30020020 con 0x7f7f400b3a80 2024-05-11T12:11:21.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.788+0000 7f7f3f200640 1 -- 172.21.15.167:0/1432048937 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7f400ceb80 con 0x7f7f400b3a80 2024-05-11T12:11:21.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.789+0000 7f7f36000640 1 -- 172.21.15.167:0/1432048937 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f7f30060a40 con 0x7f7f400b3a80 2024-05-11T12:11:21.789 INFO:teuthology.orchestra.run.smithi167.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-11T12:11:21.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.792+0000 7f7f3f200640 1 -- 172.21.15.167:0/1432048937 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f7f1c0762d0 msgr2=0x7f7f1c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:21.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.792+0000 7f7f3f200640 1 --2- 172.21.15.167:0/1432048937 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f7f1c0762d0 0x7f7f1c078790 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7f28000f60 tx=0x7f7f28007c80 comp rx=0 tx=0).stop 2024-05-11T12:11:21.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.792+0000 7f7f3f200640 1 -- 172.21.15.167:0/1432048937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f400b3a80 msgr2=0x7f7f4015ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:21.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.792+0000 7f7f3f200640 1 --2- 172.21.15.167:0/1432048937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f400b3a80 0x7f7f4015ebd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7f3000b420 tx=0x7f7f3000b8e0 comp rx=0 tx=0).stop 2024-05-11T12:11:21.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.792+0000 7f7f3f200640 1 -- 172.21.15.167:0/1432048937 shutdown_connections 2024-05-11T12:11:21.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.792+0000 7f7f3f200640 1 --2- 172.21.15.167:0/1432048937 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f7f1c0762d0 0x7f7f1c078790 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:21.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.792+0000 7f7f3f200640 1 --2- 172.21.15.167:0/1432048937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f401458f0 0x7f7f40140270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:21.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.792+0000 7f7f3f200640 1 --2- 172.21.15.167:0/1432048937 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7f400b5930 0x7f7f4013c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:21.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.792+0000 7f7f3f200640 1 --2- 172.21.15.167:0/1432048937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f400b3a80 0x7f7f4015ebd0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:21.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.792+0000 7f7f3f200640 1 -- 172.21.15.167:0/1432048937 >> 172.21.15.167:0/1432048937 conn(0x7f7f400b3460 msgr2=0x7f7f400cf210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:21.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.792+0000 7f7f3f200640 1 -- 172.21.15.167:0/1432048937 shutdown_connections 2024-05-11T12:11:21.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21.792+0000 7f7f3f200640 1 -- 172.21.15.167:0/1432048937 wait complete. 2024-05-11T12:11:21.812 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-11T12:11:21.812 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-11T12:11:21.812 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi167.front.sepia.ceph.com 2024-05-11T12:11:21.812 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-11T12:11:21.812 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-11T12:11:21.812 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-11T12:11:21.812 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-11T12:11:21.812 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-11T12:11:21.812 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-11T12:11:21.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:21.813 DEBUG:teuthology.orchestra.run.smithi167:> ip addr 2024-05-11T12:11:21.862 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-11T12:11:21.863 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-11T12:11:21.863 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2024-05-11T12:11:21.863 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:11:21.863 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2024-05-11T12:11:21.863 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:11:21.863 INFO:teuthology.orchestra.run.smithi167.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:11:21.863 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e6 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:11:21.863 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f0 2024-05-11T12:11:21.863 INFO:teuthology.orchestra.run.smithi167.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:11:21.863 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e7 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:11:21.863 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f1 2024-05-11T12:11:21.863 INFO:teuthology.orchestra.run.smithi167.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:11:21.863 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:08 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:11:21.863 INFO:teuthology.orchestra.run.smithi167.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-11T12:11:21.864 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:09 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:11:21.864 INFO:teuthology.orchestra.run.smithi167.stdout: inet 172.21.15.167/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-11T12:11:21.864 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft 42587sec preferred_lft 42587sec 2024-05-11T12:11:21.864 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::ec4:7aff:fe88:7309/64 scope link 2024-05-11T12:11:21.864 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:11:21.865 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-11T12:11:21.865 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T12:11:21.865 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-05-11T12:11:21.865 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link add name ceph-brx type bridge 2024-05-11T12:11:21.865 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip addr flush dev ceph-brx 2024-05-11T12:11:21.865 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set ceph-brx up 2024-05-11T12:11:21.865 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-11T12:11:21.865 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-05-11T12:11:21.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T12:11:22.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:11:22.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:22.089 DEBUG:teuthology.orchestra.run.smithi167:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-11T12:11:22.149 INFO:teuthology.orchestra.run.smithi167.stdout:1 2024-05-11T12:11:22.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:22.153 DEBUG:teuthology.orchestra.run.smithi167:> route 2024-05-11T12:11:22.197 INFO:teuthology.orchestra.run.smithi167.stdout:Kernel IP routing table 2024-05-11T12:11:22.197 INFO:teuthology.orchestra.run.smithi167.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-11T12:11:22.197 INFO:teuthology.orchestra.run.smithi167.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-11T12:11:22.197 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-11T12:11:22.197 INFO:teuthology.orchestra.run.smithi167.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-11T12:11:22.198 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T12:11:22.198 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-05-11T12:11:22.198 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-11T12:11:22.198 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-11T12:11:22.198 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-11T12:11:22.198 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-05-11T12:11:22.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T12:11:22.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:11:22.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:22.444 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2024-05-11T12:11:22.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:22.495 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list-id 2024-05-11T12:11:22.542 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T12:11:22.542 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-05-11T12:11:22.543 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-11T12:11:22.543 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-11T12:11:22.543 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-05-11T12:11:22.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T12:11:22.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:11:22.721 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-11T12:11:22.722 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T12:11:22.722 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-05-11T12:11:22.722 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-11T12:11:22.722 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-11T12:11:22.722 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-11T12:11:22.722 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-11T12:11:22.722 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-11T12:11:22.722 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-05-11T12:11:22.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T12:11:22.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:11:22.983 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T12:11:22.983 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-05-11T12:11:22.983 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set brx.0 up 2024-05-11T12:11:22.983 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set dev brx.0 master ceph-brx 2024-05-11T12:11:22.983 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-05-11T12:11:23.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T12:11:23.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:11:23.186 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T12:11:23.186 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-11T12:11:23.232 INFO:teuthology.orchestra.run.smithi167.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-11T12:11:23.232 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T12:11:23.232 DEBUG:teuthology.orchestra.run.smithi167:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-11T12:11:23.328 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-05-11T12:11:23.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:23.328 DEBUG:teuthology.orchestra.run.smithi167:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:11:23.461 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:11:23.461 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-11T12:11:24.196 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:11:24.196 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-11T12:11:24.196 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.167:3300,172.21.15.204:3300,172.21.15.204:3301" 2024-05-11T12:11:24.196 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: trying mount with new device syntax: 0@5f32d344-7289-4c27-881b-1533419ab678.cephfs=/ 2024-05-11T12:11:24.196 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.167:3300/172.21.15.204:3300/172.21.15.204:3301" will pass to kernel 2024-05-11T12:11:24.199 INFO:teuthology.orchestra.run.smithi167.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-11T12:11:24.199 INFO:teuthology.orchestra.run.smithi167.stdout: You just mounted a file system that supports labels which does not 2024-05-11T12:11:24.200 INFO:teuthology.orchestra.run.smithi167.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-11T12:11:24.200 INFO:teuthology.orchestra.run.smithi167.stdout: applications will generate AVC messages and not be allowed access to 2024-05-11T12:11:24.200 INFO:teuthology.orchestra.run.smithi167.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-11T12:11:24.204 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-11T12:11:24.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:24.204 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-11T12:11:24.274 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-11T12:11:24.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-11T12:11:24.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:11:24.387 INFO:teuthology.orchestra.run.smithi167.stdout:client4701 2024-05-11T12:11:24.387 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:11:24.387 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:11:24.430 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-11T12:11:24.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-11T12:11:24.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:11:24.577 INFO:teuthology.orchestra.run.smithi167.stdout:client4701 2024-05-11T12:11:24.578 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-11T12:11:24.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-11T12:11:24.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:11:24.690 INFO:teuthology.orchestra.run.smithi167.stdout:client4701 2024-05-11T12:11:24.691 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/5f32d344-7289-4c27-881b-1533419ab678.client4701/status') 2024-05-11T12:11:24.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T12:11:24.809 INFO:teuthology.orchestra.run.smithi167.stdout:instance: client.4701 (3)192.168.0.1:0/3608080478 2024-05-11T12:11:24.809 INFO:teuthology.orchestra.run.smithi167.stdout:blocklisted: false 2024-05-11T12:11:24.809 INFO:teuthology.orchestra.run.smithi167.stderr:0+1 records in 2024-05-11T12:11:24.809 INFO:teuthology.orchestra.run.smithi167.stderr:0+1 records out 2024-05-11T12:11:24.810 INFO:teuthology.orchestra.run.smithi167.stderr:69 bytes copied, 7.8776e-05 s, 876 kB/s 2024-05-11T12:11:24.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:11:24.821 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2024-05-11T12:11:24.821 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-11T12:11:24.821 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-11T12:11:24.822 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-11T12:11:25.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.002+0000 7f6693e00640 1 Processor -- start 2024-05-11T12:11:25.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.002+0000 7f6693e00640 1 -- start start 2024-05-11T12:11:25.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.003+0000 7f6693e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6694145b00 0x7f669406f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:25.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.003+0000 7f6693e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6694146770 0x7f66940ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:25.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.004+0000 7f6693e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f669406fbc0 con 0x7f66940cef10 2024-05-11T12:11:25.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.004+0000 7f6693e00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f66940b4520 con 0x7f6694145b00 2024-05-11T12:11:25.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.004+0000 7f6693e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f66940b4800 con 0x7f6694146770 2024-05-11T12:11:25.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.004+0000 7f6692a00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6694145b00 0x7f669406f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:25.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.004+0000 7f6692a00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6694145b00 0x7f669406f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:53770/0 (socket says 172.21.15.167:53770) 2024-05-11T12:11:25.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.004+0000 7f6692a00640 1 -- 172.21.15.167:0/945693429 learned_addr learned my addr 172.21.15.167:0/945693429 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:25.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.004+0000 7f6692a00640 1 -- 172.21.15.167:0/945693429 >> v2:172.21.15.204:3300/0 conn(0x7f6694146770 msgr2=0x7f66940ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:25.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.004+0000 7f6692a00640 1 --2- 172.21.15.167:0/945693429 >> v2:172.21.15.204:3300/0 conn(0x7f6694146770 0x7f66940ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:25.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.004+0000 7f6692a00640 1 -- 172.21.15.167:0/945693429 >> v1:172.21.15.204:6789/0 conn(0x7f66940cef10 legacy=0x7f669406c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:25.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.004+0000 7f6692a00640 1 -- 172.21.15.167:0/945693429 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66940b4ae0 con 0x7f6694145b00 2024-05-11T12:11:25.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.005+0000 7f6692a00640 1 --2- 172.21.15.167:0/945693429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6694145b00 0x7f669406f2c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6694146d70 tx=0x7f668402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a9ea1333e5a0b79 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:25.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.006+0000 7f6692000640 1 -- 172.21.15.167:0/945693429 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6684037070 con 0x7f6694145b00 2024-05-11T12:11:25.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.006+0000 7f6692000640 1 -- 172.21.15.167:0/945693429 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6684032440 con 0x7f6694145b00 2024-05-11T12:11:25.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.006+0000 7f6692000640 1 -- 172.21.15.167:0/945693429 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6684032810 con 0x7f6694145b00 2024-05-11T12:11:25.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.007+0000 7f6693e00640 1 -- 172.21.15.167:0/945693429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6694145b00 msgr2=0x7f669406f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:25.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.007+0000 7f6693e00640 1 --2- 172.21.15.167:0/945693429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6694145b00 0x7f669406f2c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6694146d70 tx=0x7f668402a140 comp rx=0 tx=0).stop 2024-05-11T12:11:25.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.007+0000 7f6693e00640 1 -- 172.21.15.167:0/945693429 shutdown_connections 2024-05-11T12:11:25.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.007+0000 7f6693e00640 1 --2- 172.21.15.167:0/945693429 >> v2:172.21.15.204:3300/0 conn(0x7f6694146770 0x7f66940ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:25.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.007+0000 7f6693e00640 1 --2- 172.21.15.167:0/945693429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6694145b00 0x7f669406f2c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:25.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.007+0000 7f6693e00640 1 -- 172.21.15.167:0/945693429 >> 172.21.15.167:0/945693429 conn(0x7f66940b3460 msgr2=0x7f66940d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:25.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.007+0000 7f6693e00640 1 -- 172.21.15.167:0/945693429 shutdown_connections 2024-05-11T12:11:25.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.007+0000 7f6693e00640 1 -- 172.21.15.167:0/945693429 wait complete. 2024-05-11T12:11:25.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.009+0000 7f6693e00640 1 Processor -- start 2024-05-11T12:11:25.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.009+0000 7f6693e00640 1 -- start start 2024-05-11T12:11:25.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.010+0000 7f6693e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66940cef10 0x7f66941f86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:25.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.010+0000 7f6693e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6694145b00 0x7f6694072f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:25.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.010+0000 7f6693e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6694146770 0x7f66940734c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:25.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.010+0000 7f6693e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f669406f980 con 0x7f6694146770 2024-05-11T12:11:25.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.010+0000 7f6693e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f669406f800 con 0x7f6694145b00 2024-05-11T12:11:25.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.010+0000 7f6693e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f669406cd70 con 0x7f66940cef10 2024-05-11T12:11:25.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.010+0000 7f6693400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6694146770 0x7f66940734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:25.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.010+0000 7f6693400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6694146770 0x7f66940734c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:53778/0 (socket says 172.21.15.167:53778) 2024-05-11T12:11:25.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.010+0000 7f6693400640 1 -- 172.21.15.167:0/19047504 learned_addr learned my addr 172.21.15.167:0/19047504 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:25.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.010+0000 7f668be00640 1 --2- 172.21.15.167:0/19047504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6694145b00 0x7f6694072f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:25.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.011+0000 7f6692a00640 1 --2- 172.21.15.167:0/19047504 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66940cef10 0x7f66941f86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:25.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.011+0000 7f6693400640 1 -- 172.21.15.167:0/19047504 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66940cef10 msgr2=0x7f66941f86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:25.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.011+0000 7f6693400640 1 --2- 172.21.15.167:0/19047504 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66940cef10 0x7f66941f86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:25.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.011+0000 7f6693400640 1 -- 172.21.15.167:0/19047504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6694145b00 msgr2=0x7f6694072f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:25.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.011+0000 7f6693400640 1 --2- 172.21.15.167:0/19047504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6694145b00 0x7f6694072f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:25.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.011+0000 7f6693400640 1 -- 172.21.15.167:0/19047504 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66941f8c30 con 0x7f6694146770 2024-05-11T12:11:25.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.011+0000 7f668be00640 1 --2- 172.21.15.167:0/19047504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6694145b00 0x7f6694072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:11:25.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.011+0000 7f6693400640 1 --2- 172.21.15.167:0/19047504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6694146770 0x7f66940734c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f668c00c7b0 tx=0x7f668c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:25.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.012+0000 7f668aa00640 1 -- 172.21.15.167:0/19047504 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668c017070 con 0x7f6694146770 2024-05-11T12:11:25.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.012+0000 7f668aa00640 1 -- 172.21.15.167:0/19047504 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f668c003da0 con 0x7f6694146770 2024-05-11T12:11:25.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.012+0000 7f668aa00640 1 -- 172.21.15.167:0/19047504 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668c0052b0 con 0x7f6694146770 2024-05-11T12:11:25.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.012+0000 7f6693e00640 1 -- 172.21.15.167:0/19047504 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f669415f670 con 0x7f6694146770 2024-05-11T12:11:25.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.012+0000 7f6693e00640 1 -- 172.21.15.167:0/19047504 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f669415fbf0 con 0x7f6694146770 2024-05-11T12:11:25.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.013+0000 7f6693e00640 1 -- 172.21.15.167:0/19047504 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66940694b0 con 0x7f6694146770 2024-05-11T12:11:25.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.018+0000 7f668aa00640 1 -- 172.21.15.167:0/19047504 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f668c005450 con 0x7f6694146770 2024-05-11T12:11:25.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.018+0000 7f668aa00640 1 --2- 172.21.15.167:0/19047504 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f66700762d0 0x7f6670078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:25.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.018+0000 7f668aa00640 1 -- 172.21.15.167:0/19047504 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f668c096fa0 con 0x7f6694146770 2024-05-11T12:11:25.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.018+0000 7f6692a00640 1 --2- 172.21.15.167:0/19047504 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f66700762d0 0x7f6670078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:25.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.019+0000 7f6692a00640 1 --2- 172.21.15.167:0/19047504 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f66700762d0 0x7f6670078790 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6684034000 tx=0x7f668402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:25.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.023+0000 7f668aa00640 1 -- 172.21.15.167:0/19047504 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f668c060520 con 0x7f6694146770 2024-05-11T12:11:25.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.275+0000 7f6693e00640 1 -- 172.21.15.167:0/19047504 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f66941409b0 con 0x7f6694146770 2024-05-11T12:11:25.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.276+0000 7f668aa00640 1 -- 172.21.15.167:0/19047504 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f668c05fe20 con 0x7f6694146770 2024-05-11T12:11:25.276 INFO:teuthology.orchestra.run.smithi167.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-11T12:11:25.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.278+0000 7f6693e00640 1 -- 172.21.15.167:0/19047504 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f66700762d0 msgr2=0x7f6670078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:25.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.278+0000 7f6693e00640 1 --2- 172.21.15.167:0/19047504 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f66700762d0 0x7f6670078790 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6684034000 tx=0x7f668402e000 comp rx=0 tx=0).stop 2024-05-11T12:11:25.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.278+0000 7f6693e00640 1 -- 172.21.15.167:0/19047504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6694146770 msgr2=0x7f66940734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:25.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.278+0000 7f6693e00640 1 --2- 172.21.15.167:0/19047504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6694146770 0x7f66940734c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f668c00c7b0 tx=0x7f668c00cc70 comp rx=0 tx=0).stop 2024-05-11T12:11:25.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.279+0000 7f6693e00640 1 -- 172.21.15.167:0/19047504 shutdown_connections 2024-05-11T12:11:25.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.279+0000 7f6693e00640 1 --2- 172.21.15.167:0/19047504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6694146770 0x7f66940734c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:25.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.279+0000 7f6693e00640 1 --2- 172.21.15.167:0/19047504 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f66700762d0 0x7f6670078790 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:25.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.279+0000 7f6693e00640 1 --2- 172.21.15.167:0/19047504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6694145b00 0x7f6694072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:25.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.279+0000 7f6693e00640 1 --2- 172.21.15.167:0/19047504 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66940cef10 0x7f66941f86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:25.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.279+0000 7f6693e00640 1 -- 172.21.15.167:0/19047504 >> 172.21.15.167:0/19047504 conn(0x7f66940b3460 msgr2=0x7f66940d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:25.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.279+0000 7f6693e00640 1 -- 172.21.15.167:0/19047504 shutdown_connections 2024-05-11T12:11:25.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25.279+0000 7f6693e00640 1 -- 172.21.15.167:0/19047504 wait complete. 2024-05-11T12:11:25.297 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-11T12:11:25.297 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-11T12:11:25.297 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi167.front.sepia.ceph.com 2024-05-11T12:11:25.297 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-05-11T12:11:25.297 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-11T12:11:25.297 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-11T12:11:25.297 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-11T12:11:25.297 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-11T12:11:25.297 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-05-11T12:11:25.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:25.298 DEBUG:teuthology.orchestra.run.smithi167:> ip addr 2024-05-11T12:11:25.345 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-11T12:11:25.411 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-11T12:11:25.411 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2024-05-11T12:11:25.411 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:11:25.411 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2024-05-11T12:11:25.411 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:11:25.411 INFO:teuthology.orchestra.run.smithi167.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:11:25.411 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e6 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:11:25.411 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f0 2024-05-11T12:11:25.411 INFO:teuthology.orchestra.run.smithi167.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:11:25.412 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e7 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:11:25.412 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f1 2024-05-11T12:11:25.412 INFO:teuthology.orchestra.run.smithi167.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:11:25.412 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:08 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:11:25.412 INFO:teuthology.orchestra.run.smithi167.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-11T12:11:25.412 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:09 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:11:25.412 INFO:teuthology.orchestra.run.smithi167.stdout: inet 172.21.15.167/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-11T12:11:25.412 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft 42584sec preferred_lft 42584sec 2024-05-11T12:11:25.412 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::ec4:7aff:fe88:7309/64 scope link 2024-05-11T12:11:25.412 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:11:25.412 INFO:teuthology.orchestra.run.smithi167.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-11T12:11:25.412 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 1a:a8:8d:b3:f4:32 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:11:25.412 INFO:teuthology.orchestra.run.smithi167.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-11T12:11:25.412 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:11:25.413 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::b017:48ff:fea0:7183/64 scope link 2024-05-11T12:11:25.413 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:11:25.413 INFO:teuthology.orchestra.run.smithi167.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-11T12:11:25.413 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 1a:a8:8d:b3:f4:32 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-11T12:11:25.413 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::18a8:8dff:feb3:f432/64 scope link 2024-05-11T12:11:25.413 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:11:25.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:25.413 DEBUG:teuthology.orchestra.run.smithi167:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-11T12:11:25.494 INFO:teuthology.orchestra.run.smithi167.stdout:1 2024-05-11T12:11:25.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:25.498 DEBUG:teuthology.orchestra.run.smithi167:> route 2024-05-11T12:11:25.552 INFO:teuthology.orchestra.run.smithi167.stdout:Kernel IP routing table 2024-05-11T12:11:25.552 INFO:teuthology.orchestra.run.smithi167.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-11T12:11:25.552 INFO:teuthology.orchestra.run.smithi167.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-11T12:11:25.552 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-11T12:11:25.552 INFO:teuthology.orchestra.run.smithi167.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-11T12:11:25.553 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T12:11:25.553 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-05-11T12:11:25.553 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-11T12:11:25.554 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-11T12:11:25.554 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-11T12:11:25.554 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-05-11T12:11:25.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T12:11:25.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:11:25.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:25.766 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2024-05-11T12:11:25.818 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-11T12:11:25.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:25.819 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list-id 2024-05-11T12:11:25.865 INFO:teuthology.orchestra.run.smithi167.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-05-11T12:11:25.866 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T12:11:25.866 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-05-11T12:11:25.866 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-11T12:11:25.867 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-05-11T12:11:25.867 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-05-11T12:11:26.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T12:11:26.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:11:26.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:26.099 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-11T12:11:26.164 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-11T12:11:26.164 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-11T12:11:26.164 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2024-05-11T12:11:26.165 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:11:26.165 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2024-05-11T12:11:26.165 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:11:26.165 INFO:teuthology.orchestra.run.smithi167.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-11T12:11:26.165 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether aa:5a:9f:30:cc:1e brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-11T12:11:26.165 INFO:teuthology.orchestra.run.smithi167.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-11T12:11:26.165 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:11:26.165 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::a85a:9fff:fe30:cc1e/64 scope link 2024-05-11T12:11:26.165 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:11:26.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:26.168 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-11T12:11:26.245 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-11T12:11:26.245 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-11T12:11:26.245 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2024-05-11T12:11:26.245 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:11:26.245 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2024-05-11T12:11:26.245 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:11:26.245 INFO:teuthology.orchestra.run.smithi167.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-11T12:11:26.245 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether aa:5a:9f:30:cc:1e brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-11T12:11:26.245 INFO:teuthology.orchestra.run.smithi167.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-11T12:11:26.246 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:11:26.246 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::a85a:9fff:fe30:cc1e/64 scope link 2024-05-11T12:11:26.246 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:11:26.249 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-05-11T12:11:26.249 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T12:11:26.249 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-05-11T12:11:26.249 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-05-11T12:11:26.249 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-11T12:11:26.250 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-05-11T12:11:26.250 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-05-11T12:11:26.250 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-05-11T12:11:26.250 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-05-11T12:11:26.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T12:11:26.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:11:26.526 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T12:11:26.526 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-05-11T12:11:26.526 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set brx.1 up 2024-05-11T12:11:26.526 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set dev brx.1 master ceph-brx 2024-05-11T12:11:26.526 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-05-11T12:11:26.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T12:11:26.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:11:26.730 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T12:11:26.730 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-05-11T12:11:26.775 INFO:teuthology.orchestra.run.smithi167.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-05-11T12:11:26.775 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T12:11:26.775 DEBUG:teuthology.orchestra.run.smithi167:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-05-11T12:11:26.870 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2024-05-11T12:11:26.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:26.870 DEBUG:teuthology.orchestra.run.smithi167:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:11:26.997 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:11:26.998 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-11T12:11:27.032 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:11:27.033 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-11T12:11:27.033 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.167:3300,172.21.15.204:3300,172.21.15.204:3301" 2024-05-11T12:11:27.033 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: trying mount with new device syntax: 2@5f32d344-7289-4c27-881b-1533419ab678.cephfs=/ 2024-05-11T12:11:27.033 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.167:3300/172.21.15.204:3300/172.21.15.204:3301" will pass to kernel 2024-05-11T12:11:27.036 INFO:teuthology.orchestra.run.smithi167.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-05-11T12:11:27.036 INFO:teuthology.orchestra.run.smithi167.stdout: You just mounted a file system that supports labels which does not 2024-05-11T12:11:27.036 INFO:teuthology.orchestra.run.smithi167.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-11T12:11:27.036 INFO:teuthology.orchestra.run.smithi167.stdout: applications will generate AVC messages and not be allowed access to 2024-05-11T12:11:27.036 INFO:teuthology.orchestra.run.smithi167.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-11T12:11:27.040 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-11T12:11:27.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:27.041 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-05-11T12:11:27.110 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-11T12:11:27.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-11T12:11:27.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:11:27.221 INFO:teuthology.orchestra.run.smithi167.stdout:client4710 2024-05-11T12:11:27.221 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:11:27.221 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:11:27.272 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-11T12:11:27.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-11T12:11:27.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:11:27.439 INFO:teuthology.orchestra.run.smithi167.stdout:client4710 2024-05-11T12:11:27.439 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-11T12:11:27.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-11T12:11:27.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:11:27.551 INFO:teuthology.orchestra.run.smithi167.stdout:client4710 2024-05-11T12:11:27.551 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/5f32d344-7289-4c27-881b-1533419ab678.client4710/status') 2024-05-11T12:11:27.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T12:11:27.681 INFO:teuthology.orchestra.run.smithi167.stdout:instance: client.4710 (3)192.168.0.2:0/450030953 2024-05-11T12:11:27.681 INFO:teuthology.orchestra.run.smithi167.stdout:blocklisted: false 2024-05-11T12:11:27.681 INFO:teuthology.orchestra.run.smithi167.stderr:0+1 records in 2024-05-11T12:11:27.681 INFO:teuthology.orchestra.run.smithi167.stderr:0+1 records out 2024-05-11T12:11:27.681 INFO:teuthology.orchestra.run.smithi167.stderr:68 bytes copied, 0.000132177 s, 514 kB/s 2024-05-11T12:11:27.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:11:27.693 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-05-11T12:11:27.693 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-11T12:11:27.694 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-11T12:11:27.694 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-11T12:11:27.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.893+0000 7f1f97c00640 1 Processor -- start 2024-05-11T12:11:27.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.893+0000 7f1f97c00640 1 -- start start 2024-05-11T12:11:27.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.894+0000 7f1f97c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1f9014c8a0 0x7f1f900cf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:27.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.894+0000 7f1f97c00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f900cfc10 con 0x7f1f900cf700 2024-05-11T12:11:27.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.895+0000 7f1f97c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f900cfef0 con 0x7f1f9006db70 2024-05-11T12:11:27.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.895+0000 7f1f97c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f1f900d01d0 con 0x7f1f9014c8a0 2024-05-11T12:11:27.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.895+0000 7f1f87e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f1f900cf700 0x7f1f9006d460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:40896/0 (socket says 172.21.15.167:40896) 2024-05-11T12:11:27.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.895+0000 7f1f87e00640 1 -- 172.21.15.167:0/1073880108 learned_addr learned my addr 172.21.15.167:0/1073880108 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:27.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.895+0000 7f1f95200640 1 --2- 172.21.15.167:0/1073880108 >> v2:172.21.15.204:3300/0 conn(0x7f1f9014c8a0 0x7f1f900cf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:27.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.895+0000 7f1f8fe00640 1 -- 172.21.15.167:0/1073880108 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 702723600 0 0) 0x7f1f900cfc10 con 0x7f1f900cf700 2024-05-11T12:11:27.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.895+0000 7f1f8fe00640 1 -- 172.21.15.167:0/1073880108 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f74003520 con 0x7f1f900cf700 2024-05-11T12:11:27.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.896+0000 7f1f8fe00640 1 -- 172.21.15.167:0/1073880108 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2818029403 0 0) 0x7f1f74003520 con 0x7f1f900cf700 2024-05-11T12:11:27.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.896+0000 7f1f8fe00640 1 -- 172.21.15.167:0/1073880108 >> v2:172.21.15.204:3300/0 conn(0x7f1f9014c8a0 msgr2=0x7f1f900cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:27.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.896+0000 7f1f8fe00640 1 --2- 172.21.15.167:0/1073880108 >> v2:172.21.15.204:3300/0 conn(0x7f1f9014c8a0 0x7f1f900cf1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:27.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.896+0000 7f1f8fe00640 1 -- 172.21.15.167:0/1073880108 >> v1:172.21.15.204:6789/0 conn(0x7f1f9006db70 legacy=0x7f1f900b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:27.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.896+0000 7f1f8fe00640 1 -- 172.21.15.167:0/1073880108 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f900b42b0 con 0x7f1f900cf700 2024-05-11T12:11:27.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.896+0000 7f1f8fe00640 1 -- 172.21.15.167:0/1073880108 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f1f780016f0 con 0x7f1f900cf700 2024-05-11T12:11:27.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.896+0000 7f1f8fe00640 1 -- 172.21.15.167:0/1073880108 >> v1:172.21.15.167:6789/0 conn(0x7f1f900cf700 legacy=0x7f1f9006d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:27.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.896+0000 7f1f8fe00640 1 --2- 172.21.15.167:0/1073880108 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f74003aa0 0x7f1f74003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:27.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.896+0000 7f1f8fe00640 1 --2- 172.21.15.167:0/1073880108 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f74004800 0x7f1f74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:27.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.896+0000 7f1f8fe00640 1 --2- 172.21.15.167:0/1073880108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f740052e0 0x7f1f7400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:27.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.896+0000 7f1f8fe00640 1 -- 172.21.15.167:0/1073880108 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1f900d01d0 con 0x7f1f74004800 2024-05-11T12:11:27.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.896+0000 7f1f8fe00640 1 -- 172.21.15.167:0/1073880108 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7400c200 con 0x7f1f740052e0 2024-05-11T12:11:27.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.896+0000 7f1f8fe00640 1 -- 172.21.15.167:0/1073880108 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1f7400c400 con 0x7f1f74003aa0 2024-05-11T12:11:27.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.896+0000 7f1f87e00640 1 --2- 172.21.15.167:0/1073880108 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f74004800 0x7f1f74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:27.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.896+0000 7f1f95200640 1 --2- 172.21.15.167:0/1073880108 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f74003aa0 0x7f1f74003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:27.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.897+0000 7f1f95c00640 1 --2- 172.21.15.167:0/1073880108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f740052e0 0x7f1f7400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:27.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.897+0000 7f1f87e00640 1 -- 172.21.15.167:0/1073880108 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f74003aa0 msgr2=0x7f1f74003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:27.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.897+0000 7f1f87e00640 1 --2- 172.21.15.167:0/1073880108 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f74003aa0 0x7f1f74003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:27.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.897+0000 7f1f87e00640 1 -- 172.21.15.167:0/1073880108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f740052e0 msgr2=0x7f1f7400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:27.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.897+0000 7f1f87e00640 1 --2- 172.21.15.167:0/1073880108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f740052e0 0x7f1f7400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:27.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.897+0000 7f1f87e00640 1 -- 172.21.15.167:0/1073880108 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f74003520 con 0x7f1f74004800 2024-05-11T12:11:27.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.897+0000 7f1f95c00640 1 --2- 172.21.15.167:0/1073880108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f740052e0 0x7f1f7400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:27.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.897+0000 7f1f95c00640 1 -- 172.21.15.167:0/1073880108 reap_dead start 2024-05-11T12:11:27.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.898+0000 7f1f87e00640 1 --2- 172.21.15.167:0/1073880108 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f74004800 0x7f1f74004c80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1f7800c160 tx=0x7f1f74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fcfd2450aacbae0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:27.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.898+0000 7f1f8fe00640 1 -- 172.21.15.167:0/1073880108 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f78007c80 con 0x7f1f74004800 2024-05-11T12:11:27.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.898+0000 7f1f8fe00640 1 -- 172.21.15.167:0/1073880108 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f78005ad0 con 0x7f1f74004800 2024-05-11T12:11:27.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.898+0000 7f1f8fe00640 1 -- 172.21.15.167:0/1073880108 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f78007070 con 0x7f1f74004800 2024-05-11T12:11:27.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.899+0000 7f1f97c00640 1 -- 172.21.15.167:0/1073880108 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f74004800 msgr2=0x7f1f74004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:27.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.899+0000 7f1f97c00640 1 --2- 172.21.15.167:0/1073880108 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f74004800 0x7f1f74004c80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1f7800c160 tx=0x7f1f74005830 comp rx=0 tx=0).stop 2024-05-11T12:11:27.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.899+0000 7f1f97c00640 1 -- 172.21.15.167:0/1073880108 shutdown_connections 2024-05-11T12:11:27.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.899+0000 7f1f97c00640 1 --2- 172.21.15.167:0/1073880108 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f74004800 0x7f1f74004c80 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:27.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.899+0000 7f1f97c00640 1 -- 172.21.15.167:0/1073880108 >> 172.21.15.167:0/1073880108 conn(0x7f1f900cd8f0 msgr2=0x7f1f900cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:27.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.899+0000 7f1f97c00640 1 -- 172.21.15.167:0/1073880108 shutdown_connections 2024-05-11T12:11:27.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.899+0000 7f1f97c00640 1 -- 172.21.15.167:0/1073880108 wait complete. 2024-05-11T12:11:27.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.901+0000 7f1f97c00640 1 Processor -- start 2024-05-11T12:11:27.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.901+0000 7f1f97c00640 1 -- start start 2024-05-11T12:11:27.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.902+0000 7f1f97c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f74004800 0x7f1f90162af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:27.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.902+0000 7f1f97c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f900ccb20 0x7f1f900c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:27.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.902+0000 7f1f97c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f901489c0 0x7f1f90148e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:27.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.902+0000 7f1f97c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9006feb0 con 0x7f1f900ccb20 2024-05-11T12:11:27.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.902+0000 7f1f97c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9006fd30 con 0x7f1f901489c0 2024-05-11T12:11:27.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.902+0000 7f1f97c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1f9014cc80 con 0x7f1f74004800 2024-05-11T12:11:27.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.903+0000 7f1f87e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f900ccb20 0x7f1f900c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:27.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.903+0000 7f1f87e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f900ccb20 0x7f1f900c97a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:53784/0 (socket says 172.21.15.167:53784) 2024-05-11T12:11:27.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.903+0000 7f1f87e00640 1 -- 172.21.15.167:0/3730954585 learned_addr learned my addr 172.21.15.167:0/3730954585 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:27.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.903+0000 7f1f95200640 1 --2- 172.21.15.167:0/3730954585 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f74004800 0x7f1f90162af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:27.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.903+0000 7f1f95c00640 1 --2- 172.21.15.167:0/3730954585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f901489c0 0x7f1f90148e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:27.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.903+0000 7f1f87e00640 1 -- 172.21.15.167:0/3730954585 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f74004800 msgr2=0x7f1f90162af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:27.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.903+0000 7f1f87e00640 1 --2- 172.21.15.167:0/3730954585 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f74004800 0x7f1f90162af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:27.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.903+0000 7f1f87e00640 1 -- 172.21.15.167:0/3730954585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f901489c0 msgr2=0x7f1f90148e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:27.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.903+0000 7f1f87e00640 1 --2- 172.21.15.167:0/3730954585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f901489c0 0x7f1f90148e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:27.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.903+0000 7f1f87e00640 1 -- 172.21.15.167:0/3730954585 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f900ca110 con 0x7f1f900ccb20 2024-05-11T12:11:27.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.903+0000 7f1f95c00640 1 --2- 172.21.15.167:0/3730954585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f901489c0 0x7f1f90148e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:27.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.904+0000 7f1f87e00640 1 --2- 172.21.15.167:0/3730954585 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f900ccb20 0x7f1f900c97a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1f7800c280 tx=0x7f1f900cfde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:27.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.904+0000 7f1f8d600640 1 -- 172.21.15.167:0/3730954585 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7803f070 con 0x7f1f900ccb20 2024-05-11T12:11:27.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.904+0000 7f1f8d600640 1 -- 172.21.15.167:0/3730954585 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f780014b0 con 0x7f1f900ccb20 2024-05-11T12:11:27.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.904+0000 7f1f8d600640 1 -- 172.21.15.167:0/3730954585 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f78005930 con 0x7f1f900ccb20 2024-05-11T12:11:27.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.904+0000 7f1f97c00640 1 -- 172.21.15.167:0/3730954585 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f900ca3f0 con 0x7f1f900ccb20 2024-05-11T12:11:27.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.904+0000 7f1f97c00640 1 -- 172.21.15.167:0/3730954585 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f900c7330 con 0x7f1f900ccb20 2024-05-11T12:11:27.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.905+0000 7f1f97c00640 1 -- 172.21.15.167:0/3730954585 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f90069590 con 0x7f1f900ccb20 2024-05-11T12:11:27.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.906+0000 7f1f8d600640 1 -- 172.21.15.167:0/3730954585 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1f78042070 con 0x7f1f900ccb20 2024-05-11T12:11:27.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.907+0000 7f1f8d600640 1 --2- 172.21.15.167:0/3730954585 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f1f6c0762d0 0x7f1f6c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:27.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.907+0000 7f1f8d600640 1 -- 172.21.15.167:0/3730954585 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6735+0+0 (secure 0 0 0) 0x7f1f780b7860 con 0x7f1f900ccb20 2024-05-11T12:11:27.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.907+0000 7f1f95200640 1 --2- 172.21.15.167:0/3730954585 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f1f6c0762d0 0x7f1f6c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:27.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.908+0000 7f1f95200640 1 --2- 172.21.15.167:0/3730954585 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f1f6c0762d0 0x7f1f6c078790 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1f80006fd0 tx=0x7f1f80008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:27.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:27.912+0000 7f1f8d600640 1 -- 172.21.15.167:0/3730954585 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1f78049030 con 0x7f1f900ccb20 2024-05-11T12:11:28.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:28.165+0000 7f1f97c00640 1 -- 172.21.15.167:0/3730954585 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1f900728d0 con 0x7f1f900ccb20 2024-05-11T12:11:28.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:28.166+0000 7f1f8d600640 1 -- 172.21.15.167:0/3730954585 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f1f78080c90 con 0x7f1f900ccb20 2024-05-11T12:11:28.165 INFO:teuthology.orchestra.run.smithi167.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-11T12:11:28.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:28.168+0000 7f1f97c00640 1 -- 172.21.15.167:0/3730954585 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f1f6c0762d0 msgr2=0x7f1f6c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:28.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:28.168+0000 7f1f97c00640 1 --2- 172.21.15.167:0/3730954585 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f1f6c0762d0 0x7f1f6c078790 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1f80006fd0 tx=0x7f1f80008040 comp rx=0 tx=0).stop 2024-05-11T12:11:28.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:28.168+0000 7f1f97c00640 1 -- 172.21.15.167:0/3730954585 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f900ccb20 msgr2=0x7f1f900c97a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:28.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:28.168+0000 7f1f97c00640 1 --2- 172.21.15.167:0/3730954585 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f900ccb20 0x7f1f900c97a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1f7800c280 tx=0x7f1f900cfde0 comp rx=0 tx=0).stop 2024-05-11T12:11:28.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:28.169+0000 7f1f97c00640 1 -- 172.21.15.167:0/3730954585 shutdown_connections 2024-05-11T12:11:28.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:28.169+0000 7f1f97c00640 1 --2- 172.21.15.167:0/3730954585 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f1f6c0762d0 0x7f1f6c078790 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:28.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:28.169+0000 7f1f97c00640 1 --2- 172.21.15.167:0/3730954585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f901489c0 0x7f1f90148e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:28.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:28.169+0000 7f1f97c00640 1 --2- 172.21.15.167:0/3730954585 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f900ccb20 0x7f1f900c97a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:28.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:28.169+0000 7f1f97c00640 1 --2- 172.21.15.167:0/3730954585 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f74004800 0x7f1f90162af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:28.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:28.169+0000 7f1f97c00640 1 -- 172.21.15.167:0/3730954585 >> 172.21.15.167:0/3730954585 conn(0x7f1f900cd8f0 msgr2=0x7f1f900b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:28.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:28.169+0000 7f1f97c00640 1 -- 172.21.15.167:0/3730954585 shutdown_connections 2024-05-11T12:11:28.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:28.169+0000 7f1f97c00640 1 -- 172.21.15.167:0/3730954585 wait complete. 2024-05-11T12:11:28.187 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-11T12:11:28.187 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-11T12:11:28.187 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi204.front.sepia.ceph.com 2024-05-11T12:11:28.187 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-11T12:11:28.187 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-11T12:11:28.187 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-11T12:11:28.187 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-11T12:11:28.187 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-11T12:11:28.187 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-11T12:11:28.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:28.188 DEBUG:teuthology.orchestra.run.smithi204:> ip addr 2024-05-11T12:11:28.245 INFO:teuthology.orchestra.run.smithi204.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-11T12:11:28.245 INFO:teuthology.orchestra.run.smithi204.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-11T12:11:28.245 INFO:teuthology.orchestra.run.smithi204.stdout: inet 127.0.0.1/8 scope host lo 2024-05-11T12:11:28.246 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:11:28.246 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 ::1/128 scope host 2024-05-11T12:11:28.246 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:11:28.246 INFO:teuthology.orchestra.run.smithi204.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:11:28.246 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2a brd ff:ff:ff:ff:ff:ff 2024-05-11T12:11:28.246 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f0 2024-05-11T12:11:28.246 INFO:teuthology.orchestra.run.smithi204.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:11:28.246 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2b brd ff:ff:ff:ff:ff:ff 2024-05-11T12:11:28.246 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f1 2024-05-11T12:11:28.246 INFO:teuthology.orchestra.run.smithi204.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:11:28.246 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:aa brd ff:ff:ff:ff:ff:ff 2024-05-11T12:11:28.247 INFO:teuthology.orchestra.run.smithi204.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-11T12:11:28.247 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:ab brd ff:ff:ff:ff:ff:ff 2024-05-11T12:11:28.247 INFO:teuthology.orchestra.run.smithi204.stdout: inet 172.21.15.204/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-11T12:11:28.247 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft 42576sec preferred_lft 42576sec 2024-05-11T12:11:28.247 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::ec4:7aff:fe88:77ab/64 scope link 2024-05-11T12:11:28.247 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:11:28.248 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-11T12:11:28.248 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T12:11:28.248 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-05-11T12:11:28.248 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link add name ceph-brx type bridge 2024-05-11T12:11:28.248 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip addr flush dev ceph-brx 2024-05-11T12:11:28.248 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link set ceph-brx up 2024-05-11T12:11:28.248 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-11T12:11:28.248 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-05-11T12:11:28.350 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:11:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T12:11:28.726 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:11:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:11:28.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:28.736 DEBUG:teuthology.orchestra.run.smithi204:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-11T12:11:28.807 INFO:teuthology.orchestra.run.smithi204.stdout:1 2024-05-11T12:11:28.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:28.811 DEBUG:teuthology.orchestra.run.smithi204:> route 2024-05-11T12:11:28.863 INFO:teuthology.orchestra.run.smithi204.stdout:Kernel IP routing table 2024-05-11T12:11:28.864 INFO:teuthology.orchestra.run.smithi204.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-11T12:11:28.864 INFO:teuthology.orchestra.run.smithi204.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-11T12:11:28.864 INFO:teuthology.orchestra.run.smithi204.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-11T12:11:28.864 INFO:teuthology.orchestra.run.smithi204.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-11T12:11:28.864 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T12:11:28.864 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-05-11T12:11:28.864 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-11T12:11:28.864 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-11T12:11:28.864 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-11T12:11:28.864 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-05-11T12:11:28.960 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:11:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T12:11:29.612 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:11:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:11:29.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:29.622 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list 2024-05-11T12:11:29.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:29.674 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list-id 2024-05-11T12:11:29.724 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T12:11:29.725 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-05-11T12:11:29.725 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-11T12:11:29.725 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-05-11T12:11:29.725 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-05-11T12:11:29.829 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:11:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T12:11:29.902 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:11:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:11:29.911 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-05-11T12:11:29.912 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T12:11:29.912 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-05-11T12:11:29.912 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-05-11T12:11:29.912 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-11T12:11:29.912 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-05-11T12:11:29.912 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-05-11T12:11:29.912 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-05-11T12:11:29.912 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-05-11T12:11:30.011 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:11:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T12:11:30.199 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:11:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:11:30.208 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T12:11:30.209 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-05-11T12:11:30.209 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link set brx.0 up 2024-05-11T12:11:30.209 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link set dev brx.0 master ceph-brx 2024-05-11T12:11:30.209 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-05-11T12:11:30.297 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:11:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T12:11:30.396 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:11:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:11:30.406 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T12:11:30.407 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-11T12:11:30.453 INFO:teuthology.orchestra.run.smithi204.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-11T12:11:30.453 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T12:11:30.453 DEBUG:teuthology.orchestra.run.smithi204:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-11T12:11:30.501 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-05-11T12:11:30.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:30.502 DEBUG:teuthology.orchestra.run.smithi204:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:11:30.649 INFO:teuthology.orchestra.run.smithi204.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:11:30.649 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-11T12:11:31.398 INFO:teuthology.orchestra.run.smithi204.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:11:31.398 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-11T12:11:31.398 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: resolved to: "172.21.15.167:3300,172.21.15.204:3300,172.21.15.204:3301" 2024-05-11T12:11:31.398 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: trying mount with new device syntax: 1@5f32d344-7289-4c27-881b-1533419ab678.cephfs=/ 2024-05-11T12:11:31.398 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.167:3300/172.21.15.204:3300/172.21.15.204:3301" will pass to kernel 2024-05-11T12:11:31.401 INFO:teuthology.orchestra.run.smithi204.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-05-11T12:11:31.401 INFO:teuthology.orchestra.run.smithi204.stdout: You just mounted a file system that supports labels which does not 2024-05-11T12:11:31.402 INFO:teuthology.orchestra.run.smithi204.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-11T12:11:31.402 INFO:teuthology.orchestra.run.smithi204.stdout: applications will generate AVC messages and not be allowed access to 2024-05-11T12:11:31.402 INFO:teuthology.orchestra.run.smithi204.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-11T12:11:31.406 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-11T12:11:31.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:31.406 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-11T12:11:31.491 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-11T12:11:31.597 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:11:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-11T12:11:31.602 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:11:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:11:31.611 INFO:teuthology.orchestra.run.smithi204.stdout:client4457 2024-05-11T12:11:31.612 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:11:31.612 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:11:31.662 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-11T12:11:31.760 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:11:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-11T12:11:31.765 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:11:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:11:31.773 INFO:teuthology.orchestra.run.smithi204.stdout:client4457 2024-05-11T12:11:31.774 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-11T12:11:31.879 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:11:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-11T12:11:31.884 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:11:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:11:31.893 INFO:teuthology.orchestra.run.smithi204.stdout:client4457 2024-05-11T12:11:31.893 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/5f32d344-7289-4c27-881b-1533419ab678.client4457/status') 2024-05-11T12:11:31.996 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:11:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T12:11:32.025 INFO:teuthology.orchestra.run.smithi204.stdout:instance: client.4457 (3)192.168.0.1:0/4047159809 2024-05-11T12:11:32.026 INFO:teuthology.orchestra.run.smithi204.stdout:blocklisted: false 2024-05-11T12:11:32.026 INFO:teuthology.orchestra.run.smithi204.stderr:0+1 records in 2024-05-11T12:11:32.026 INFO:teuthology.orchestra.run.smithi204.stderr:0+1 records out 2024-05-11T12:11:32.026 INFO:teuthology.orchestra.run.smithi204.stderr:69 bytes copied, 7.4375e-05 s, 928 kB/s 2024-05-11T12:11:32.029 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:11:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:11:32.038 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2024-05-11T12:11:32.038 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-11T12:11:32.038 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-11T12:11:32.039 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-11T12:11:32.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.253+0000 7ff5e1800640 1 Processor -- start 2024-05-11T12:11:32.253 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.253+0000 7ff5e1800640 1 -- start start 2024-05-11T12:11:32.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.255+0000 7ff5e1800640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff5dc0b50a0 0x7ff5dc073cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:32.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.255+0000 7ff5e1800640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5dc15fb90 con 0x7ff5dc13eeb0 2024-05-11T12:11:32.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.255+0000 7ff5e1800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5dc15fe70 con 0x7ff5dc13f290 2024-05-11T12:11:32.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.255+0000 7ff5e1800640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7ff5dc15d5b0 con 0x7ff5dc0b50a0 2024-05-11T12:11:32.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.256+0000 7ff5daa00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7ff5dc13eeb0 0x7ff5dc06f480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:54846/0 (socket says 172.21.15.167:54846) 2024-05-11T12:11:32.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.256+0000 7ff5daa00640 1 -- 172.21.15.167:0/998406549 learned_addr learned my addr 172.21.15.167:0/998406549 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:32.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.256+0000 7ff5db400640 1 --2- 172.21.15.167:0/998406549 >> v2:172.21.15.167:3300/0 conn(0x7ff5dc0b50a0 0x7ff5dc073cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:32.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.256+0000 7ff5da000640 1 -- 172.21.15.167:0/998406549 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3607830543 0 0) 0x7ff5dc15fb90 con 0x7ff5dc13eeb0 2024-05-11T12:11:32.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.256+0000 7ff5da000640 1 -- 172.21.15.167:0/998406549 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5c0003520 con 0x7ff5dc13eeb0 2024-05-11T12:11:32.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.256+0000 7ff5db400640 1 -- 172.21.15.167:0/998406549 >> v1:172.21.15.204:6789/0 conn(0x7ff5dc13f290 legacy=0x7ff5dc0735c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:32.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.256+0000 7ff5db400640 1 -- 172.21.15.167:0/998406549 >> v1:172.21.15.167:6789/0 conn(0x7ff5dc13eeb0 legacy=0x7ff5dc06f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:32.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.256+0000 7ff5db400640 1 -- 172.21.15.167:0/998406549 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5dc160150 con 0x7ff5dc0b50a0 2024-05-11T12:11:32.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.257+0000 7ff5db400640 1 --2- 172.21.15.167:0/998406549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5dc0b50a0 0x7ff5dc073cd0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff5d40097a0 tx=0x7ff5d402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3149dba18abdd70a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:32.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.258+0000 7ff5da000640 1 -- 172.21.15.167:0/998406549 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d4037070 con 0x7ff5dc0b50a0 2024-05-11T12:11:32.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.258+0000 7ff5da000640 1 -- 172.21.15.167:0/998406549 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5d4032440 con 0x7ff5dc0b50a0 2024-05-11T12:11:32.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.258+0000 7ff5da000640 1 -- 172.21.15.167:0/998406549 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d40327f0 con 0x7ff5dc0b50a0 2024-05-11T12:11:32.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.258+0000 7ff5e1800640 1 -- 172.21.15.167:0/998406549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5dc0b50a0 msgr2=0x7ff5dc073cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:32.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.258+0000 7ff5e1800640 1 --2- 172.21.15.167:0/998406549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5dc0b50a0 0x7ff5dc073cd0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff5d40097a0 tx=0x7ff5d402a1a0 comp rx=0 tx=0).stop 2024-05-11T12:11:32.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.258+0000 7ff5e1800640 1 -- 172.21.15.167:0/998406549 shutdown_connections 2024-05-11T12:11:32.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.258+0000 7ff5e1800640 1 --2- 172.21.15.167:0/998406549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5dc0b50a0 0x7ff5dc073cd0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:32.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.258+0000 7ff5e1800640 1 -- 172.21.15.167:0/998406549 >> 172.21.15.167:0/998406549 conn(0x7ff5dc0cd8f0 msgr2=0x7ff5dc0cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:32.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.258+0000 7ff5e1800640 1 -- 172.21.15.167:0/998406549 shutdown_connections 2024-05-11T12:11:32.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.258+0000 7ff5e1800640 1 -- 172.21.15.167:0/998406549 wait complete. 2024-05-11T12:11:32.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.260+0000 7ff5e1800640 1 Processor -- start 2024-05-11T12:11:32.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.260+0000 7ff5e1800640 1 -- start start 2024-05-11T12:11:32.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.261+0000 7ff5e1800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5dc0b50a0 0x7ff5dc138650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:32.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.261+0000 7ff5e1800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5dc13eeb0 0x7ff5dc14bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:32.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.261+0000 7ff5e1800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff5dc13f290 0x7ff5dc0b2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:32.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.261+0000 7ff5e1800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff5dc06fb90 con 0x7ff5dc0b50a0 2024-05-11T12:11:32.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.261+0000 7ff5e1800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff5dc160710 con 0x7ff5dc13eeb0 2024-05-11T12:11:32.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.261+0000 7ff5e1800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff5dc06fd10 con 0x7ff5dc13f290 2024-05-11T12:11:32.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.261+0000 7ff5daa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5dc0b50a0 0x7ff5dc138650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:32.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.262+0000 7ff5daa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5dc0b50a0 0x7ff5dc138650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:51260/0 (socket says 172.21.15.167:51260) 2024-05-11T12:11:32.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.262+0000 7ff5daa00640 1 -- 172.21.15.167:0/138529778 learned_addr learned my addr 172.21.15.167:0/138529778 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:32.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.262+0000 7ff5daa00640 1 -- 172.21.15.167:0/138529778 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff5dc13f290 msgr2=0x7ff5dc0b2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:32.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.262+0000 7ff5daa00640 1 --2- 172.21.15.167:0/138529778 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff5dc13f290 0x7ff5dc0b2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:32.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.262+0000 7ff5daa00640 1 -- 172.21.15.167:0/138529778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5dc13eeb0 msgr2=0x7ff5dc14bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:32.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.262+0000 7ff5daa00640 1 --2- 172.21.15.167:0/138529778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5dc13eeb0 0x7ff5dc14bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:32.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.262+0000 7ff5daa00640 1 -- 172.21.15.167:0/138529778 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5dc138b90 con 0x7ff5dc0b50a0 2024-05-11T12:11:32.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.262+0000 7ff5daa00640 1 --2- 172.21.15.167:0/138529778 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5dc0b50a0 0x7ff5dc138650 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc00cbf0 tx=0x7ff5cc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:32.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.263+0000 7ff5d3400640 1 -- 172.21.15.167:0/138529778 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc00ad50 con 0x7ff5dc0b50a0 2024-05-11T12:11:32.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.263+0000 7ff5d3400640 1 -- 172.21.15.167:0/138529778 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5cc012400 con 0x7ff5dc0b50a0 2024-05-11T12:11:32.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.263+0000 7ff5d3400640 1 -- 172.21.15.167:0/138529778 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc0127b0 con 0x7ff5dc0b50a0 2024-05-11T12:11:32.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.263+0000 7ff5e1800640 1 -- 172.21.15.167:0/138529778 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5dc0b0ca0 con 0x7ff5dc0b50a0 2024-05-11T12:11:32.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.263+0000 7ff5e1800640 1 -- 172.21.15.167:0/138529778 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5dc0b1540 con 0x7ff5dc0b50a0 2024-05-11T12:11:32.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.264+0000 7ff5e1800640 1 -- 172.21.15.167:0/138529778 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5dc069590 con 0x7ff5dc0b50a0 2024-05-11T12:11:32.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.265+0000 7ff5d3400640 1 -- 172.21.15.167:0/138529778 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff5cc012950 con 0x7ff5dc0b50a0 2024-05-11T12:11:32.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.266+0000 7ff5d3400640 1 --2- 172.21.15.167:0/138529778 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff5b80762d0 0x7ff5b8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:32.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.266+0000 7ff5d3400640 1 -- 172.21.15.167:0/138529778 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6735+0+0 (secure 0 0 0) 0x7ff5cc0981d0 con 0x7ff5dc0b50a0 2024-05-11T12:11:32.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.266+0000 7ff5d2000640 1 --2- 172.21.15.167:0/138529778 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff5b80762d0 0x7ff5b8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:32.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.267+0000 7ff5d2000640 1 --2- 172.21.15.167:0/138529778 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff5b80762d0 0x7ff5b8078790 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc072cd0 tx=0x7ff5c4007970 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:32.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.271+0000 7ff5d3400640 1 -- 172.21.15.167:0/138529778 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff5dc138b90 con 0x7ff5dc0b50a0 2024-05-11T12:11:32.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.516+0000 7ff5e1800640 1 -- 172.21.15.167:0/138529778 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff5dc06dee0 con 0x7ff5dc0b50a0 2024-05-11T12:11:32.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.516+0000 7ff5d3400640 1 -- 172.21.15.167:0/138529778 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7ff5cc061600 con 0x7ff5dc0b50a0 2024-05-11T12:11:32.516 INFO:teuthology.orchestra.run.smithi167.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-11T12:11:32.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.519+0000 7ff5e1800640 1 -- 172.21.15.167:0/138529778 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff5b80762d0 msgr2=0x7ff5b8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:32.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.519+0000 7ff5e1800640 1 --2- 172.21.15.167:0/138529778 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff5b80762d0 0x7ff5b8078790 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc072cd0 tx=0x7ff5c4007970 comp rx=0 tx=0).stop 2024-05-11T12:11:32.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.519+0000 7ff5e1800640 1 -- 172.21.15.167:0/138529778 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5dc0b50a0 msgr2=0x7ff5dc138650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:32.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.519+0000 7ff5e1800640 1 --2- 172.21.15.167:0/138529778 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5dc0b50a0 0x7ff5dc138650 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc00cbf0 tx=0x7ff5cc00a430 comp rx=0 tx=0).stop 2024-05-11T12:11:32.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.520+0000 7ff5e1800640 1 -- 172.21.15.167:0/138529778 shutdown_connections 2024-05-11T12:11:32.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.520+0000 7ff5e1800640 1 --2- 172.21.15.167:0/138529778 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff5dc13f290 0x7ff5dc0b2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:32.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.520+0000 7ff5e1800640 1 --2- 172.21.15.167:0/138529778 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff5b80762d0 0x7ff5b8078790 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:32.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.520+0000 7ff5e1800640 1 --2- 172.21.15.167:0/138529778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5dc13eeb0 0x7ff5dc14bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:32.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.520+0000 7ff5e1800640 1 --2- 172.21.15.167:0/138529778 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5dc0b50a0 0x7ff5dc138650 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:32.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.520+0000 7ff5e1800640 1 -- 172.21.15.167:0/138529778 >> 172.21.15.167:0/138529778 conn(0x7ff5dc0cd8f0 msgr2=0x7ff5dc06ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:32.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.520+0000 7ff5e1800640 1 -- 172.21.15.167:0/138529778 shutdown_connections 2024-05-11T12:11:32.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:32.520+0000 7ff5e1800640 1 -- 172.21.15.167:0/138529778 wait complete. 2024-05-11T12:11:32.536 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-11T12:11:32.536 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-11T12:11:32.536 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi204.front.sepia.ceph.com 2024-05-11T12:11:32.536 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-05-11T12:11:32.536 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-11T12:11:32.536 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-11T12:11:32.536 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-11T12:11:32.536 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-11T12:11:32.536 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-05-11T12:11:32.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:32.537 DEBUG:teuthology.orchestra.run.smithi204:> ip addr 2024-05-11T12:11:32.595 INFO:teuthology.orchestra.run.smithi204.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-11T12:11:32.596 INFO:teuthology.orchestra.run.smithi204.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-11T12:11:32.596 INFO:teuthology.orchestra.run.smithi204.stdout: inet 127.0.0.1/8 scope host lo 2024-05-11T12:11:32.596 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:11:32.596 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 ::1/128 scope host 2024-05-11T12:11:32.596 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:11:32.596 INFO:teuthology.orchestra.run.smithi204.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:11:32.596 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2a brd ff:ff:ff:ff:ff:ff 2024-05-11T12:11:32.596 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f0 2024-05-11T12:11:32.596 INFO:teuthology.orchestra.run.smithi204.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:11:32.596 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2b brd ff:ff:ff:ff:ff:ff 2024-05-11T12:11:32.596 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f1 2024-05-11T12:11:32.597 INFO:teuthology.orchestra.run.smithi204.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:11:32.597 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:aa brd ff:ff:ff:ff:ff:ff 2024-05-11T12:11:32.597 INFO:teuthology.orchestra.run.smithi204.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-11T12:11:32.597 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:ab brd ff:ff:ff:ff:ff:ff 2024-05-11T12:11:32.597 INFO:teuthology.orchestra.run.smithi204.stdout: inet 172.21.15.204/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-11T12:11:32.597 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft 42572sec preferred_lft 42572sec 2024-05-11T12:11:32.597 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::ec4:7aff:fe88:77ab/64 scope link 2024-05-11T12:11:32.597 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:11:32.597 INFO:teuthology.orchestra.run.smithi204.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-11T12:11:32.597 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether fa:0a:7f:e4:30:35 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:11:32.597 INFO:teuthology.orchestra.run.smithi204.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-11T12:11:32.597 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:11:32.597 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::a498:7aff:fe50:48c3/64 scope link 2024-05-11T12:11:32.597 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:11:32.597 INFO:teuthology.orchestra.run.smithi204.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-11T12:11:32.598 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether fa:0a:7f:e4:30:35 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-11T12:11:32.598 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::f80a:7fff:fee4:3035/64 scope link 2024-05-11T12:11:32.598 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:11:32.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:32.599 DEBUG:teuthology.orchestra.run.smithi204:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-11T12:11:32.682 INFO:teuthology.orchestra.run.smithi204.stdout:1 2024-05-11T12:11:32.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:32.686 DEBUG:teuthology.orchestra.run.smithi204:> route 2024-05-11T12:11:32.735 INFO:teuthology.orchestra.run.smithi204.stdout:Kernel IP routing table 2024-05-11T12:11:32.735 INFO:teuthology.orchestra.run.smithi204.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-11T12:11:32.735 INFO:teuthology.orchestra.run.smithi204.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-11T12:11:32.736 INFO:teuthology.orchestra.run.smithi204.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-11T12:11:32.736 INFO:teuthology.orchestra.run.smithi204.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-11T12:11:32.736 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T12:11:32.736 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-05-11T12:11:32.736 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-11T12:11:32.736 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-11T12:11:32.736 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-11T12:11:32.736 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-05-11T12:11:32.836 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:11:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T12:11:32.932 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:11:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:11:32.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:32.942 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list 2024-05-11T12:11:32.990 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-11T12:11:32.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:32.992 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list-id 2024-05-11T12:11:33.040 INFO:teuthology.orchestra.run.smithi204.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-05-11T12:11:33.041 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T12:11:33.041 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-05-11T12:11:33.041 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-11T12:11:33.041 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-05-11T12:11:33.041 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-05-11T12:11:33.138 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:11:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T12:11:33.203 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:11:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:11:33.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:33.213 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-11T12:11:33.280 INFO:teuthology.orchestra.run.smithi204.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-11T12:11:33.280 INFO:teuthology.orchestra.run.smithi204.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-11T12:11:33.280 INFO:teuthology.orchestra.run.smithi204.stdout: inet 127.0.0.1/8 scope host lo 2024-05-11T12:11:33.281 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:11:33.281 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 ::1/128 scope host 2024-05-11T12:11:33.281 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:11:33.281 INFO:teuthology.orchestra.run.smithi204.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-11T12:11:33.281 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 5a:2d:8b:ad:c6:61 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-11T12:11:33.281 INFO:teuthology.orchestra.run.smithi204.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-11T12:11:33.281 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:11:33.281 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::582d:8bff:fead:c661/64 scope link 2024-05-11T12:11:33.281 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:11:33.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:33.284 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-11T12:11:33.364 INFO:teuthology.orchestra.run.smithi204.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-11T12:11:33.364 INFO:teuthology.orchestra.run.smithi204.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-11T12:11:33.364 INFO:teuthology.orchestra.run.smithi204.stdout: inet 127.0.0.1/8 scope host lo 2024-05-11T12:11:33.364 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:11:33.364 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 ::1/128 scope host 2024-05-11T12:11:33.364 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:11:33.364 INFO:teuthology.orchestra.run.smithi204.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-11T12:11:33.364 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 5a:2d:8b:ad:c6:61 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-11T12:11:33.365 INFO:teuthology.orchestra.run.smithi204.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-11T12:11:33.365 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:11:33.365 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::582d:8bff:fead:c661/64 scope link 2024-05-11T12:11:33.365 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:11:33.368 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-05-11T12:11:33.368 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T12:11:33.368 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-05-11T12:11:33.368 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-05-11T12:11:33.368 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-11T12:11:33.369 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-05-11T12:11:33.369 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-05-11T12:11:33.369 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-05-11T12:11:33.369 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-05-11T12:11:33.472 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:11:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T12:11:33.641 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:11:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:11:33.651 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T12:11:33.651 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-05-11T12:11:33.651 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link set brx.1 up 2024-05-11T12:11:33.651 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link set dev brx.1 master ceph-brx 2024-05-11T12:11:33.651 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-05-11T12:11:33.753 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:11:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T12:11:33.852 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:11:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:11:33.862 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T12:11:33.862 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-05-11T12:11:33.910 INFO:teuthology.orchestra.run.smithi204.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-05-11T12:11:33.910 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T12:11:33.910 DEBUG:teuthology.orchestra.run.smithi204:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-05-11T12:11:34.000 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2024-05-11T12:11:34.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:34.001 DEBUG:teuthology.orchestra.run.smithi204:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:11:34.129 INFO:teuthology.orchestra.run.smithi204.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:11:34.129 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-11T12:11:34.161 INFO:teuthology.orchestra.run.smithi204.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:11:34.161 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-11T12:11:34.162 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: resolved to: "172.21.15.167:3300,172.21.15.204:3300,172.21.15.204:3301" 2024-05-11T12:11:34.162 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: trying mount with new device syntax: 3@5f32d344-7289-4c27-881b-1533419ab678.cephfs=/ 2024-05-11T12:11:34.162 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.167:3300/172.21.15.204:3300/172.21.15.204:3301" will pass to kernel 2024-05-11T12:11:34.164 INFO:teuthology.orchestra.run.smithi204.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-05-11T12:11:34.164 INFO:teuthology.orchestra.run.smithi204.stdout: You just mounted a file system that supports labels which does not 2024-05-11T12:11:34.164 INFO:teuthology.orchestra.run.smithi204.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-11T12:11:34.164 INFO:teuthology.orchestra.run.smithi204.stdout: applications will generate AVC messages and not be allowed access to 2024-05-11T12:11:34.164 INFO:teuthology.orchestra.run.smithi204.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-11T12:11:34.170 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-11T12:11:34.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:34.170 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-05-11T12:11:34.238 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-11T12:11:34.335 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:11:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-11T12:11:34.340 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:11:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:11:34.349 INFO:teuthology.orchestra.run.smithi204.stdout:client4567 2024-05-11T12:11:34.349 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:11:34.349 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:11:34.398 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-11T12:11:34.506 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:11:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-11T12:11:34.511 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:11:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:11:34.521 INFO:teuthology.orchestra.run.smithi204.stdout:client4567 2024-05-11T12:11:34.521 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-11T12:11:34.621 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:11:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-11T12:11:34.625 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:11:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:11:34.635 INFO:teuthology.orchestra.run.smithi204.stdout:client4567 2024-05-11T12:11:34.635 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/5f32d344-7289-4c27-881b-1533419ab678.client4567/status') 2024-05-11T12:11:34.729 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:11:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T12:11:34.759 INFO:teuthology.orchestra.run.smithi204.stdout:instance: client.4567 (3)192.168.0.2:0/2314978615 2024-05-11T12:11:34.759 INFO:teuthology.orchestra.run.smithi204.stdout:blocklisted: false 2024-05-11T12:11:34.759 INFO:teuthology.orchestra.run.smithi204.stderr:0+1 records in 2024-05-11T12:11:34.760 INFO:teuthology.orchestra.run.smithi204.stderr:0+1 records out 2024-05-11T12:11:34.760 INFO:teuthology.orchestra.run.smithi204.stderr:69 bytes copied, 6.669e-05 s, 1.0 MB/s 2024-05-11T12:11:34.762 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:11:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:11:34.773 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-05-11T12:11:34.794 INFO:tasks.cephfs_test_runner:Starting test: test_mount_conn_close (tasks.cephfs.test_sessionmap.TestSessionMap) That when a client unmounts, the thread count on the MDS goes back 2024-05-11T12:11:34.794 INFO:tasks.cephfs_test_runner:test_mount_conn_close (tasks.cephfs.test_sessionmap.TestSessionMap) 2024-05-11T12:11:34.794 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:11:35.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.009+0000 7fea87a00640 1 Processor -- start 2024-05-11T12:11:35.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.009+0000 7fea87a00640 1 -- start start 2024-05-11T12:11:35.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.011+0000 7fea87a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea800af5d0 0x7fea80073ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:35.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.011+0000 7fea87a00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea8006fe10 con 0x7fea800af1f0 2024-05-11T12:11:35.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.011+0000 7fea87a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea8015fcc0 con 0x7fea8006d460 2024-05-11T12:11:35.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.011+0000 7fea87a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fea801608d0 con 0x7fea800af5d0 2024-05-11T12:11:35.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.011+0000 7fea85000640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fea800af1f0 0x7fea800cf1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:54848/0 (socket says 172.21.15.167:54848) 2024-05-11T12:11:35.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.011+0000 7fea85000640 1 -- 172.21.15.167:0/1935311813 learned_addr learned my addr 172.21.15.167:0/1935311813 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:35.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.012+0000 7fea7fe00640 1 -- 172.21.15.167:0/1935311813 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3732345427 0 0) 0x7fea8006fe10 con 0x7fea800af1f0 2024-05-11T12:11:35.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.012+0000 7fea7fe00640 1 -- 172.21.15.167:0/1935311813 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fea64003520 con 0x7fea800af1f0 2024-05-11T12:11:35.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.012+0000 7fea77e00640 1 --2- 172.21.15.167:0/1935311813 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea800af5d0 0x7fea80073ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:35.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.012+0000 7fea7fe00640 1 -- 172.21.15.167:0/1935311813 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3319885524 0 0) 0x7fea64003520 con 0x7fea800af1f0 2024-05-11T12:11:35.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.012+0000 7fea7fe00640 1 -- 172.21.15.167:0/1935311813 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea800af5d0 msgr2=0x7fea80073ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:35.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.012+0000 7fea7fe00640 1 --2- 172.21.15.167:0/1935311813 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea800af5d0 0x7fea80073ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:35.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.012+0000 7fea7fe00640 1 -- 172.21.15.167:0/1935311813 >> v1:172.21.15.204:6789/0 conn(0x7fea8006d460 legacy=0x7fea8006e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:35.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.012+0000 7fea7fe00640 1 -- 172.21.15.167:0/1935311813 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea8015ffa0 con 0x7fea800af1f0 2024-05-11T12:11:35.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.012+0000 7fea7fe00640 1 -- 172.21.15.167:0/1935311813 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7fea700016f0 con 0x7fea800af1f0 2024-05-11T12:11:35.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.012+0000 7fea7fe00640 1 -- 172.21.15.167:0/1935311813 >> v1:172.21.15.167:6789/0 conn(0x7fea800af1f0 legacy=0x7fea800cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:35.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.012+0000 7fea7fe00640 1 --2- 172.21.15.167:0/1935311813 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea800af5d0 0x7fea64003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:35.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.012+0000 7fea7fe00640 1 --2- 172.21.15.167:0/1935311813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea640040b0 0x7fea64004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:35.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.012+0000 7fea7fe00640 1 --2- 172.21.15.167:0/1935311813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea64004d90 0x7fea64005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:35.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.012+0000 7fea7fe00640 1 -- 172.21.15.167:0/1935311813 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fea801608d0 con 0x7fea64004d90 2024-05-11T12:11:35.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.012+0000 7fea7fe00640 1 -- 172.21.15.167:0/1935311813 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fea6400bca0 con 0x7fea640040b0 2024-05-11T12:11:35.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.013+0000 7fea7fe00640 1 -- 172.21.15.167:0/1935311813 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fea6400bea0 con 0x7fea800af5d0 2024-05-11T12:11:35.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.013+0000 7fea85a00640 1 --2- 172.21.15.167:0/1935311813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea64004d90 0x7fea64005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:35.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.013+0000 7fea85000640 1 --2- 172.21.15.167:0/1935311813 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea800af5d0 0x7fea64003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:35.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.013+0000 7fea85a00640 1 -- 172.21.15.167:0/1935311813 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea800af5d0 msgr2=0x7fea64003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:35.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.013+0000 7fea85a00640 1 --2- 172.21.15.167:0/1935311813 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea800af5d0 0x7fea64003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:35.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.013+0000 7fea77e00640 1 --2- 172.21.15.167:0/1935311813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea640040b0 0x7fea64004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:35.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.013+0000 7fea85a00640 1 -- 172.21.15.167:0/1935311813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea640040b0 msgr2=0x7fea64004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:35.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.013+0000 7fea85a00640 1 --2- 172.21.15.167:0/1935311813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea640040b0 0x7fea64004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:35.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.013+0000 7fea85a00640 1 -- 172.21.15.167:0/1935311813 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea64003520 con 0x7fea64004d90 2024-05-11T12:11:35.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.014+0000 7fea85a00640 1 --2- 172.21.15.167:0/1935311813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea64004d90 0x7fea64005210 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fea78001e60 tx=0x7fea64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb4eed97e8693971 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:35.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.015+0000 7fea7fe00640 1 -- 172.21.15.167:0/1935311813 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea78001ab0 con 0x7fea64004d90 2024-05-11T12:11:35.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.015+0000 7fea7fe00640 1 -- 172.21.15.167:0/1935311813 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea7802ae00 con 0x7fea64004d90 2024-05-11T12:11:35.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.015+0000 7fea7fe00640 1 -- 172.21.15.167:0/1935311813 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea78003b70 con 0x7fea64004d90 2024-05-11T12:11:35.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.015+0000 7fea87a00640 1 -- 172.21.15.167:0/1935311813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea64004d90 msgr2=0x7fea64005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:35.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.015+0000 7fea87a00640 1 --2- 172.21.15.167:0/1935311813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea64004d90 0x7fea64005210 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fea78001e60 tx=0x7fea64005830 comp rx=0 tx=0).stop 2024-05-11T12:11:35.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.015+0000 7fea85a00640 1 -- 172.21.15.167:0/1935311813 reap_dead start 2024-05-11T12:11:35.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.015+0000 7fea87a00640 1 -- 172.21.15.167:0/1935311813 shutdown_connections 2024-05-11T12:11:35.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.015+0000 7fea87a00640 1 -- 172.21.15.167:0/1935311813 >> 172.21.15.167:0/1935311813 conn(0x7fea800cd8f0 msgr2=0x7fea800cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:35.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.015+0000 7fea87a00640 1 -- 172.21.15.167:0/1935311813 shutdown_connections 2024-05-11T12:11:35.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.015+0000 7fea87a00640 1 -- 172.21.15.167:0/1935311813 wait complete. 2024-05-11T12:11:35.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.017+0000 7fea87a00640 1 Processor -- start 2024-05-11T12:11:35.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.017+0000 7fea87a00640 1 -- start start 2024-05-11T12:11:35.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.018+0000 7fea87a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea64004d90 0x7fea800c8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:35.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.018+0000 7fea87a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea801512d0 0x7fea80151750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:35.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.018+0000 7fea87a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea801e9b50 0x7fea801e9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:35.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.018+0000 7fea87a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fea8015f920 con 0x7fea801512d0 2024-05-11T12:11:35.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.018+0000 7fea87a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fea80160560 con 0x7fea801e9b50 2024-05-11T12:11:35.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.018+0000 7fea87a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fea8015faa0 con 0x7fea64004d90 2024-05-11T12:11:35.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.018+0000 7fea77e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea801512d0 0x7fea80151750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:35.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.019+0000 7fea77e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea801512d0 0x7fea80151750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:51270/0 (socket says 172.21.15.167:51270) 2024-05-11T12:11:35.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.019+0000 7fea77e00640 1 -- 172.21.15.167:0/2693188395 learned_addr learned my addr 172.21.15.167:0/2693188395 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:35.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.019+0000 7fea85a00640 1 --2- 172.21.15.167:0/2693188395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea801e9b50 0x7fea801e9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:35.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.019+0000 7fea77e00640 1 -- 172.21.15.167:0/2693188395 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea64004d90 msgr2=0x7fea800c8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:35.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.019+0000 7fea77e00640 1 --2- 172.21.15.167:0/2693188395 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea64004d90 0x7fea800c8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:35.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.019+0000 7fea77e00640 1 -- 172.21.15.167:0/2693188395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea801e9b50 msgr2=0x7fea801e9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:35.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.019+0000 7fea77e00640 1 --2- 172.21.15.167:0/2693188395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea801e9b50 0x7fea801e9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:35.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.019+0000 7fea77e00640 1 -- 172.21.15.167:0/2693188395 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea800c5f60 con 0x7fea801512d0 2024-05-11T12:11:35.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.019+0000 7fea77e00640 1 --2- 172.21.15.167:0/2693188395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea801512d0 0x7fea80151750 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fea6800e7e0 tx=0x7fea6800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:35.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.019+0000 7fea7d600640 1 -- 172.21.15.167:0/2693188395 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea68017070 con 0x7fea801512d0 2024-05-11T12:11:35.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.020+0000 7fea7d600640 1 -- 172.21.15.167:0/2693188395 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea6800c6d0 con 0x7fea801512d0 2024-05-11T12:11:35.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.020+0000 7fea7d600640 1 -- 172.21.15.167:0/2693188395 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6800f040 con 0x7fea801512d0 2024-05-11T12:11:35.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.020+0000 7fea87a00640 1 -- 172.21.15.167:0/2693188395 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea800c6200 con 0x7fea801512d0 2024-05-11T12:11:35.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.020+0000 7fea87a00640 1 -- 172.21.15.167:0/2693188395 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea800ca860 con 0x7fea801512d0 2024-05-11T12:11:35.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.021+0000 7fea87a00640 1 -- 172.21.15.167:0/2693188395 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea80069590 con 0x7fea801512d0 2024-05-11T12:11:35.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.022+0000 7fea7d600640 1 -- 172.21.15.167:0/2693188395 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fea6801a070 con 0x7fea801512d0 2024-05-11T12:11:35.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.023+0000 7fea7d600640 1 --2- 172.21.15.167:0/2693188395 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fea5c076300 0x7fea5c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:35.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.023+0000 7fea7d600640 1 -- 172.21.15.167:0/2693188395 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6735+0+0 (secure 0 0 0) 0x7fea68097560 con 0x7fea801512d0 2024-05-11T12:11:35.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.023+0000 7fea85000640 1 --2- 172.21.15.167:0/2693188395 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fea5c076300 0x7fea5c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:35.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.024+0000 7fea85000640 1 --2- 172.21.15.167:0/2693188395 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fea5c076300 0x7fea5c0787c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fea800b5b60 tx=0x7fea70000ed0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:35.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.028+0000 7fea7d600640 1 -- 172.21.15.167:0/2693188395 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fea68060b50 con 0x7fea801512d0 2024-05-11T12:11:35.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.239+0000 7fea87a00640 1 -- 172.21.15.167:0/2693188395 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fea800cf9b0 con 0x7fea801512d0 2024-05-11T12:11:35.241 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.242+0000 7fea7d600640 1 -- 172.21.15.167:0/2693188395 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18950 (secure 0 0 0) 0x7fea68060450 con 0x7fea801512d0 2024-05-11T12:11:35.242 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:11:35.242 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":29,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:11:27.038079+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:10:43.084511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T12:10:44.157782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-11T12:10:47.814674+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:11:35.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.244+0000 7fea87a00640 1 -- 172.21.15.167:0/2693188395 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fea5c076300 msgr2=0x7fea5c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:35.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.245+0000 7fea87a00640 1 --2- 172.21.15.167:0/2693188395 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fea5c076300 0x7fea5c0787c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fea800b5b60 tx=0x7fea70000ed0 comp rx=0 tx=0).stop 2024-05-11T12:11:35.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.245+0000 7fea87a00640 1 -- 172.21.15.167:0/2693188395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea801512d0 msgr2=0x7fea80151750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:35.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.245+0000 7fea87a00640 1 --2- 172.21.15.167:0/2693188395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea801512d0 0x7fea80151750 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fea6800e7e0 tx=0x7fea6800eca0 comp rx=0 tx=0).stop 2024-05-11T12:11:35.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.245+0000 7fea87a00640 1 -- 172.21.15.167:0/2693188395 shutdown_connections 2024-05-11T12:11:35.245 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.245+0000 7fea87a00640 1 --2- 172.21.15.167:0/2693188395 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fea5c076300 0x7fea5c0787c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:35.245 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.245+0000 7fea87a00640 1 --2- 172.21.15.167:0/2693188395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea801e9b50 0x7fea801e9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:35.245 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.245+0000 7fea87a00640 1 --2- 172.21.15.167:0/2693188395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea801512d0 0x7fea80151750 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:35.245 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.245+0000 7fea87a00640 1 --2- 172.21.15.167:0/2693188395 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea64004d90 0x7fea800c8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:35.245 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.245+0000 7fea87a00640 1 -- 172.21.15.167:0/2693188395 >> 172.21.15.167:0/2693188395 conn(0x7fea800cd8f0 msgr2=0x7fea800b1930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:35.245 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.245+0000 7fea87a00640 1 -- 172.21.15.167:0/2693188395 shutdown_connections 2024-05-11T12:11:35.245 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.245+0000 7fea87a00640 1 -- 172.21.15.167:0/2693188395 wait complete. 2024-05-11T12:11:35.264 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-11T12:10:33.260270+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-11T12:10:43.084511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-11T12:10:44.157782+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-11T12:10:47.814674+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-11T12:11:35.264 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-11T12:11:35.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.471+0000 7f62b7200640 1 Processor -- start 2024-05-11T12:11:35.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.472+0000 7f62b7200640 1 -- start start 2024-05-11T12:11:35.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.473+0000 7f62b7200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f62b8160290 0x7f62b806cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:35.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.473+0000 7f62b7200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f62b806d190 0x7f62b8168d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:35.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.473+0000 7f62b7200640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62b806d980 con 0x7f62b815f7a0 2024-05-11T12:11:35.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.473+0000 7f62b7200640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f62b81608f0 con 0x7f62b806d190 2024-05-11T12:11:35.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.473+0000 7f62b7200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f62b806df40 con 0x7f62b8160290 2024-05-11T12:11:35.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.473+0000 7f62b5e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f62b815f7a0 0x7f62b815fb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:54856/0 (socket says 172.21.15.167:54856) 2024-05-11T12:11:35.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.473+0000 7f62b5e00640 1 -- 172.21.15.167:0/3437116802 learned_addr learned my addr 172.21.15.167:0/3437116802 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:35.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.473+0000 7f62b6800640 1 --2- 172.21.15.167:0/3437116802 >> v2:172.21.15.167:3300/0 conn(0x7f62b806d190 0x7f62b8168d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:35.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.474+0000 7f62b5400640 1 --2- 172.21.15.167:0/3437116802 >> v2:172.21.15.204:3300/0 conn(0x7f62b8160290 0x7f62b806cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:35.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.474+0000 7f62b4a00640 1 -- 172.21.15.167:0/3437116802 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3771807463 0 0) 0x7f62b806d980 con 0x7f62b815f7a0 2024-05-11T12:11:35.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.474+0000 7f62b4a00640 1 -- 172.21.15.167:0/3437116802 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f629c003520 con 0x7f62b815f7a0 2024-05-11T12:11:35.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.474+0000 7f62b6800640 1 -- 172.21.15.167:0/3437116802 >> v2:172.21.15.204:3300/0 conn(0x7f62b8160290 msgr2=0x7f62b806cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:35.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.474+0000 7f62b6800640 1 --2- 172.21.15.167:0/3437116802 >> v2:172.21.15.204:3300/0 conn(0x7f62b8160290 0x7f62b806cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:35.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.474+0000 7f62b4a00640 1 -- 172.21.15.167:0/3437116802 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3656817329 0 0) 0x7f629c003520 con 0x7f62b815f7a0 2024-05-11T12:11:35.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.474+0000 7f62b6800640 1 -- 172.21.15.167:0/3437116802 >> v1:172.21.15.167:6789/0 conn(0x7f62b815f7a0 legacy=0x7f62b815fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:35.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.474+0000 7f62b6800640 1 -- 172.21.15.167:0/3437116802 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b806dc60 con 0x7f62b806d190 2024-05-11T12:11:35.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.475+0000 7f62b6800640 1 --2- 172.21.15.167:0/3437116802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62b806d190 0x7f62b8168d50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f62b80d1740 tx=0x7f62b002a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4679cff8b4cd62f server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:35.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.476+0000 7f62b4a00640 1 -- 172.21.15.167:0/3437116802 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a80016f0 con 0x7f62b806d190 2024-05-11T12:11:35.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.476+0000 7f62b4a00640 1 -- 172.21.15.167:0/3437116802 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62b0007540 con 0x7f62b806d190 2024-05-11T12:11:35.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.476+0000 7f62b4a00640 1 -- 172.21.15.167:0/3437116802 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b0007910 con 0x7f62b806d190 2024-05-11T12:11:35.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.476+0000 7f62b7200640 1 -- 172.21.15.167:0/3437116802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62b806d190 msgr2=0x7f62b8168d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:35.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.476+0000 7f62b7200640 1 --2- 172.21.15.167:0/3437116802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62b806d190 0x7f62b8168d50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f62b80d1740 tx=0x7f62b002a570 comp rx=0 tx=0).stop 2024-05-11T12:11:35.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.476+0000 7f62b7200640 1 -- 172.21.15.167:0/3437116802 shutdown_connections 2024-05-11T12:11:35.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.476+0000 7f62b7200640 1 --2- 172.21.15.167:0/3437116802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62b806d190 0x7f62b8168d50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:35.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.476+0000 7f62b7200640 1 --2- 172.21.15.167:0/3437116802 >> v2:172.21.15.204:3300/0 conn(0x7f62b8160290 0x7f62b806cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:35.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.476+0000 7f62b7200640 1 -- 172.21.15.167:0/3437116802 >> 172.21.15.167:0/3437116802 conn(0x7f62b80b6ee0 msgr2=0x7f62b80b72f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:35.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.476+0000 7f62b7200640 1 -- 172.21.15.167:0/3437116802 shutdown_connections 2024-05-11T12:11:35.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.476+0000 7f62b7200640 1 -- 172.21.15.167:0/3437116802 wait complete. 2024-05-11T12:11:35.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.478+0000 7f62b7200640 1 Processor -- start 2024-05-11T12:11:35.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.478+0000 7f62b7200640 1 -- start start 2024-05-11T12:11:35.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.479+0000 7f62b7200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62b806d190 0x7f62b80cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:35.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.479+0000 7f62b7200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62b815f7a0 0x7f62b81f4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:35.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.479+0000 7f62b7200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62b8160290 0x7f62b80c9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:35.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.479+0000 7f62b7200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f62b80d3450 con 0x7f62b8160290 2024-05-11T12:11:35.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.479+0000 7f62b7200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f62b80d2f20 con 0x7f62b806d190 2024-05-11T12:11:35.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.479+0000 7f62b7200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f62b806d5c0 con 0x7f62b815f7a0 2024-05-11T12:11:35.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.479+0000 7f62b6800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62b8160290 0x7f62b80c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:35.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.480+0000 7f62b6800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62b8160290 0x7f62b80c9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:51294/0 (socket says 172.21.15.167:51294) 2024-05-11T12:11:35.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.480+0000 7f62b6800640 1 -- 172.21.15.167:0/2760619613 learned_addr learned my addr 172.21.15.167:0/2760619613 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:35.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.480+0000 7f62b5e00640 1 --2- 172.21.15.167:0/2760619613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62b806d190 0x7f62b80cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:35.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.480+0000 7f62b6800640 1 -- 172.21.15.167:0/2760619613 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62b815f7a0 msgr2=0x7f62b81f4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:35.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.480+0000 7f62b6800640 1 --2- 172.21.15.167:0/2760619613 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62b815f7a0 0x7f62b81f4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:35.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.480+0000 7f62b6800640 1 -- 172.21.15.167:0/2760619613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62b806d190 msgr2=0x7f62b80cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:35.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.480+0000 7f62b6800640 1 --2- 172.21.15.167:0/2760619613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62b806d190 0x7f62b80cc0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:35.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.480+0000 7f62b6800640 1 -- 172.21.15.167:0/2760619613 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b80b8c90 con 0x7f62b8160290 2024-05-11T12:11:35.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.480+0000 7f62b6800640 1 --2- 172.21.15.167:0/2760619613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62b8160290 0x7f62b80c9420 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f62b0009e80 tx=0x7f62b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:35.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.481+0000 7f62ae000640 1 -- 172.21.15.167:0/2760619613 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b0040070 con 0x7f62b8160290 2024-05-11T12:11:35.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.481+0000 7f62ae000640 1 -- 172.21.15.167:0/2760619613 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62b0031bc0 con 0x7f62b8160290 2024-05-11T12:11:35.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.481+0000 7f62b7200640 1 -- 172.21.15.167:0/2760619613 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62b80cb790 con 0x7f62b8160290 2024-05-11T12:11:35.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.481+0000 7f62ae000640 1 -- 172.21.15.167:0/2760619613 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b0009450 con 0x7f62b8160290 2024-05-11T12:11:35.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.481+0000 7f62b7200640 1 -- 172.21.15.167:0/2760619613 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62b80b91e0 con 0x7f62b8160290 2024-05-11T12:11:35.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.482+0000 7f62b7200640 1 -- 172.21.15.167:0/2760619613 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62b8069bc0 con 0x7f62b8160290 2024-05-11T12:11:35.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.483+0000 7f62ae000640 1 -- 172.21.15.167:0/2760619613 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f62b0039040 con 0x7f62b8160290 2024-05-11T12:11:35.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.484+0000 7f62ae000640 1 --2- 172.21.15.167:0/2760619613 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f62980762d0 0x7f6298078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:35.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.484+0000 7f62ae000640 1 -- 172.21.15.167:0/2760619613 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6735+0+0 (secure 0 0 0) 0x7f62b00b91f0 con 0x7f62b8160290 2024-05-11T12:11:35.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.484+0000 7f62b5e00640 1 --2- 172.21.15.167:0/2760619613 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f62980762d0 0x7f6298078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:35.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.485+0000 7f62b5e00640 1 --2- 172.21.15.167:0/2760619613 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f62980762d0 0x7f6298078790 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f62a8005880 tx=0x7f62a8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:35.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.488+0000 7f62ae000640 1 -- 172.21.15.167:0/2760619613 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f62b00826b0 con 0x7f62b8160290 2024-05-11T12:11:35.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.695+0000 7f62b7200640 1 -- 172.21.15.167:0/2760619613 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f62b80cf410 con 0x7f62b8160290 2024-05-11T12:11:35.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.696+0000 7f62ae000640 1 -- 172.21.15.167:0/2760619613 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f62b0081fb0 con 0x7f62b8160290 2024-05-11T12:11:35.695 INFO:teuthology.orchestra.run.smithi167.stdout:pg_num: 1 2024-05-11T12:11:35.698 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.699+0000 7f62b7200640 1 -- 172.21.15.167:0/2760619613 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f62980762d0 msgr2=0x7f6298078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:35.698 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.699+0000 7f62b7200640 1 --2- 172.21.15.167:0/2760619613 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f62980762d0 0x7f6298078790 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f62a8005880 tx=0x7f62a8009040 comp rx=0 tx=0).stop 2024-05-11T12:11:35.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.699+0000 7f62b7200640 1 -- 172.21.15.167:0/2760619613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62b8160290 msgr2=0x7f62b80c9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:35.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.699+0000 7f62b7200640 1 --2- 172.21.15.167:0/2760619613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62b8160290 0x7f62b80c9420 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f62b0009e80 tx=0x7f62b0004a40 comp rx=0 tx=0).stop 2024-05-11T12:11:35.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.699+0000 7f62b7200640 1 -- 172.21.15.167:0/2760619613 shutdown_connections 2024-05-11T12:11:35.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.699+0000 7f62b7200640 1 --2- 172.21.15.167:0/2760619613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62b8160290 0x7f62b80c9420 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:35.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.699+0000 7f62b7200640 1 --2- 172.21.15.167:0/2760619613 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62b815f7a0 0x7f62b81f4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:35.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.699+0000 7f62b7200640 1 --2- 172.21.15.167:0/2760619613 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f62980762d0 0x7f6298078790 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:35.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.699+0000 7f62b7200640 1 --2- 172.21.15.167:0/2760619613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62b806d190 0x7f62b80cc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:35.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.699+0000 7f62b7200640 1 -- 172.21.15.167:0/2760619613 >> 172.21.15.167:0/2760619613 conn(0x7f62b80b6ee0 msgr2=0x7f62b815bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:35.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.699+0000 7f62b7200640 1 -- 172.21.15.167:0/2760619613 shutdown_connections 2024-05-11T12:11:35.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.699+0000 7f62b7200640 1 -- 172.21.15.167:0/2760619613 wait complete. 2024-05-11T12:11:35.716 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-11T12:11:35.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.912+0000 7fada3e00640 1 Processor -- start 2024-05-11T12:11:35.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.912+0000 7fada3e00640 1 -- start start 2024-05-11T12:11:35.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.914+0000 7fada3e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fada40d0660 0x7fada41609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:35.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.914+0000 7fada3e00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fada40cf5c0 con 0x7fada4160ef0 2024-05-11T12:11:35.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.914+0000 7fada3e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fada40cf8a0 con 0x7fada413fe30 2024-05-11T12:11:35.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.914+0000 7fada3e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fada4161340 con 0x7fada40d0660 2024-05-11T12:11:35.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.914+0000 7fada3400640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fada4160ef0 0x7fada406c450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:54872/0 (socket says 172.21.15.167:54872) 2024-05-11T12:11:35.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.914+0000 7fada3400640 1 -- 172.21.15.167:0/1712350111 learned_addr learned my addr 172.21.15.167:0/1712350111 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:35.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.914+0000 7fad9be00640 1 --2- 172.21.15.167:0/1712350111 >> v2:172.21.15.204:3300/0 conn(0x7fada40d0660 0x7fada41609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:35.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.914+0000 7fada2000640 1 -- 172.21.15.167:0/1712350111 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3571247056 0 0) 0x7fada40cf5c0 con 0x7fada4160ef0 2024-05-11T12:11:35.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.914+0000 7fada2000640 1 -- 172.21.15.167:0/1712350111 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad88003520 con 0x7fada4160ef0 2024-05-11T12:11:35.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.915+0000 7fada2000640 1 -- 172.21.15.167:0/1712350111 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3276812761 0 0) 0x7fad88003520 con 0x7fada4160ef0 2024-05-11T12:11:35.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.915+0000 7fada2000640 1 -- 172.21.15.167:0/1712350111 >> v2:172.21.15.204:3300/0 conn(0x7fada40d0660 msgr2=0x7fada41609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:35.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.915+0000 7fada2000640 1 --2- 172.21.15.167:0/1712350111 >> v2:172.21.15.204:3300/0 conn(0x7fada40d0660 0x7fada41609b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:35.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.915+0000 7fada2000640 1 -- 172.21.15.167:0/1712350111 >> v1:172.21.15.204:6789/0 conn(0x7fada413fe30 legacy=0x7fada415de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:35.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.915+0000 7fada2000640 1 -- 172.21.15.167:0/1712350111 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada40cfb80 con 0x7fada4160ef0 2024-05-11T12:11:35.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.915+0000 7fada2000640 1 -- 172.21.15.167:0/1712350111 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7fad9c001a70 con 0x7fada4160ef0 2024-05-11T12:11:35.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.915+0000 7fada2000640 1 -- 172.21.15.167:0/1712350111 >> v1:172.21.15.167:6789/0 conn(0x7fada4160ef0 legacy=0x7fada406c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:35.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.915+0000 7fada2000640 1 --2- 172.21.15.167:0/1712350111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad88003a00 0x7fad88003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:35.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.915+0000 7fada2000640 1 --2- 172.21.15.167:0/1712350111 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad88004800 0x7fad88004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:35.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.915+0000 7fada2000640 1 --2- 172.21.15.167:0/1712350111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad88005210 0x7fad8800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:35.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.915+0000 7fada2000640 1 -- 172.21.15.167:0/1712350111 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fada4161340 con 0x7fad88005210 2024-05-11T12:11:35.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.915+0000 7fada2000640 1 -- 172.21.15.167:0/1712350111 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fad8800c140 con 0x7fad88003a00 2024-05-11T12:11:35.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.915+0000 7fada2000640 1 -- 172.21.15.167:0/1712350111 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fad8800c340 con 0x7fad88004800 2024-05-11T12:11:35.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.915+0000 7fada3400640 1 --2- 172.21.15.167:0/1712350111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad88005210 0x7fad8800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:35.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.915+0000 7fada2a00640 1 --2- 172.21.15.167:0/1712350111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad88003a00 0x7fad88003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:35.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.915+0000 7fada3400640 1 -- 172.21.15.167:0/1712350111 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad88004800 msgr2=0x7fad88004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:35.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.915+0000 7fada3400640 1 --2- 172.21.15.167:0/1712350111 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad88004800 0x7fad88004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:35.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.915+0000 7fada3400640 1 -- 172.21.15.167:0/1712350111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad88003a00 msgr2=0x7fad88003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:35.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.916+0000 7fada3400640 1 --2- 172.21.15.167:0/1712350111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad88003a00 0x7fad88003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:35.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.916+0000 7fada3400640 1 -- 172.21.15.167:0/1712350111 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad88003520 con 0x7fad88005210 2024-05-11T12:11:35.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.916+0000 7fada2a00640 1 --2- 172.21.15.167:0/1712350111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad88003a00 0x7fad88003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:35.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.916+0000 7fada3400640 1 -- 172.21.15.167:0/1712350111 reap_dead start 2024-05-11T12:11:35.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.916+0000 7fada3400640 1 --2- 172.21.15.167:0/1712350111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad88005210 0x7fad8800b9c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fad9c00a8f0 tx=0x7fad88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c111827dc4d9474e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:35.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.917+0000 7fada2000640 1 -- 172.21.15.167:0/1712350111 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c00a4c0 con 0x7fad88005210 2024-05-11T12:11:35.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.917+0000 7fada2000640 1 -- 172.21.15.167:0/1712350111 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad9c02f070 con 0x7fad88005210 2024-05-11T12:11:35.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.917+0000 7fada2000640 1 -- 172.21.15.167:0/1712350111 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c02bbf0 con 0x7fad88005210 2024-05-11T12:11:35.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.917+0000 7fada3e00640 1 -- 172.21.15.167:0/1712350111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad88005210 msgr2=0x7fad8800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:35.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.917+0000 7fada3e00640 1 --2- 172.21.15.167:0/1712350111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad88005210 0x7fad8800b9c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fad9c00a8f0 tx=0x7fad88005830 comp rx=0 tx=0).stop 2024-05-11T12:11:35.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.917+0000 7fada3e00640 1 -- 172.21.15.167:0/1712350111 shutdown_connections 2024-05-11T12:11:35.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.917+0000 7fada3e00640 1 --2- 172.21.15.167:0/1712350111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad88005210 0x7fad8800b9c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:35.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.917+0000 7fada3e00640 1 -- 172.21.15.167:0/1712350111 >> 172.21.15.167:0/1712350111 conn(0x7fada40b2a20 msgr2=0x7fada40b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:35.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.917+0000 7fada3e00640 1 -- 172.21.15.167:0/1712350111 shutdown_connections 2024-05-11T12:11:35.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.917+0000 7fada3e00640 1 -- 172.21.15.167:0/1712350111 wait complete. 2024-05-11T12:11:35.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.919+0000 7fada3e00640 1 Processor -- start 2024-05-11T12:11:35.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.919+0000 7fada3e00640 1 -- start start 2024-05-11T12:11:35.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.920+0000 7fada3e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad88005210 0x7fada40b1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:35.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.920+0000 7fada3e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fada4139310 0x7fada413c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:35.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.920+0000 7fada3e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fada41f3620 0x7fada413cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:35.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.920+0000 7fada3e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fada41616a0 con 0x7fada4139310 2024-05-11T12:11:35.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.920+0000 7fada3e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fada40d0140 con 0x7fada41f3620 2024-05-11T12:11:35.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.920+0000 7fada3e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fada4161820 con 0x7fad88005210 2024-05-11T12:11:35.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.920+0000 7fad9be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fada4139310 0x7fada413c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:35.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.920+0000 7fad9be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fada4139310 0x7fada413c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:51314/0 (socket says 172.21.15.167:51314) 2024-05-11T12:11:35.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.920+0000 7fad9be00640 1 -- 172.21.15.167:0/3863804072 learned_addr learned my addr 172.21.15.167:0/3863804072 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:35.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.921+0000 7fada3400640 1 --2- 172.21.15.167:0/3863804072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fada41f3620 0x7fada413cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:35.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.921+0000 7fada2a00640 1 --2- 172.21.15.167:0/3863804072 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad88005210 0x7fada40b1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:35.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.921+0000 7fad9be00640 1 -- 172.21.15.167:0/3863804072 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad88005210 msgr2=0x7fada40b1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:35.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.921+0000 7fad9be00640 1 --2- 172.21.15.167:0/3863804072 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad88005210 0x7fada40b1ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:35.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.921+0000 7fad9be00640 1 -- 172.21.15.167:0/3863804072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fada41f3620 msgr2=0x7fada413cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:35.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.921+0000 7fad9be00640 1 --2- 172.21.15.167:0/3863804072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fada41f3620 0x7fada413cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:35.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.921+0000 7fad9be00640 1 -- 172.21.15.167:0/3863804072 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada41f3e90 con 0x7fada4139310 2024-05-11T12:11:35.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.921+0000 7fad9be00640 1 --2- 172.21.15.167:0/3863804072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fada4139310 0x7fada413c610 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fad8c00d7d0 tx=0x7fad8c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:35.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.921+0000 7fad9aa00640 1 -- 172.21.15.167:0/3863804072 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c018020 con 0x7fada4139310 2024-05-11T12:11:35.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.921+0000 7fad9aa00640 1 -- 172.21.15.167:0/3863804072 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad8c00b6c0 con 0x7fada4139310 2024-05-11T12:11:35.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.921+0000 7fad9aa00640 1 -- 172.21.15.167:0/3863804072 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c00bad0 con 0x7fada4139310 2024-05-11T12:11:35.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.921+0000 7fada3e00640 1 -- 172.21.15.167:0/3863804072 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fada41f4170 con 0x7fada4139310 2024-05-11T12:11:35.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.922+0000 7fada3e00640 1 -- 172.21.15.167:0/3863804072 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fada41f4730 con 0x7fada4139310 2024-05-11T12:11:35.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.923+0000 7fada3e00640 1 -- 172.21.15.167:0/3863804072 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fada40694b0 con 0x7fada4139310 2024-05-11T12:11:35.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.924+0000 7fad9aa00640 1 -- 172.21.15.167:0/3863804072 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fad8c01b020 con 0x7fada4139310 2024-05-11T12:11:35.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.925+0000 7fad9aa00640 1 --2- 172.21.15.167:0/3863804072 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fad800762d0 0x7fad80078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:35.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.925+0000 7fad9aa00640 1 -- 172.21.15.167:0/3863804072 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6735+0+0 (secure 0 0 0) 0x7fad8c097bf0 con 0x7fada4139310 2024-05-11T12:11:35.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.925+0000 7fada2a00640 1 --2- 172.21.15.167:0/3863804072 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fad800762d0 0x7fad80078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:35.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.926+0000 7fada2a00640 1 --2- 172.21.15.167:0/3863804072 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fad800762d0 0x7fad80078790 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fad94007fd0 tx=0x7fad94009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:35.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:35.929+0000 7fad9aa00640 1 -- 172.21.15.167:0/3863804072 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fad8c022030 con 0x7fada4139310 2024-05-11T12:11:36.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.136+0000 7fada3e00640 1 -- 172.21.15.167:0/3863804072 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fada40b68d0 con 0x7fada4139310 2024-05-11T12:11:36.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.137+0000 7fad9aa00640 1 -- 172.21.15.167:0/3863804072 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7fad8c061140 con 0x7fada4139310 2024-05-11T12:11:36.136 INFO:teuthology.orchestra.run.smithi167.stdout:pg_num: 32 2024-05-11T12:11:36.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.139+0000 7fada3e00640 1 -- 172.21.15.167:0/3863804072 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fad800762d0 msgr2=0x7fad80078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:36.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.139+0000 7fada3e00640 1 --2- 172.21.15.167:0/3863804072 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fad800762d0 0x7fad80078790 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fad94007fd0 tx=0x7fad94009040 comp rx=0 tx=0).stop 2024-05-11T12:11:36.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.139+0000 7fada3e00640 1 -- 172.21.15.167:0/3863804072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fada4139310 msgr2=0x7fada413c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:36.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.139+0000 7fada3e00640 1 --2- 172.21.15.167:0/3863804072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fada4139310 0x7fada413c610 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fad8c00d7d0 tx=0x7fad8c00dc90 comp rx=0 tx=0).stop 2024-05-11T12:11:36.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.139+0000 7fada3e00640 1 -- 172.21.15.167:0/3863804072 shutdown_connections 2024-05-11T12:11:36.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.140+0000 7fada3e00640 1 --2- 172.21.15.167:0/3863804072 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fad800762d0 0x7fad80078790 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:36.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.140+0000 7fada3e00640 1 --2- 172.21.15.167:0/3863804072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fada41f3620 0x7fada413cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:36.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.140+0000 7fada3e00640 1 --2- 172.21.15.167:0/3863804072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fada4139310 0x7fada413c610 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:36.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.140+0000 7fada3e00640 1 --2- 172.21.15.167:0/3863804072 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad88005210 0x7fada40b1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:36.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.140+0000 7fada3e00640 1 -- 172.21.15.167:0/3863804072 >> 172.21.15.167:0/3863804072 conn(0x7fada40b2a20 msgr2=0x7fada40b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:36.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.140+0000 7fada3e00640 1 -- 172.21.15.167:0/3863804072 shutdown_connections 2024-05-11T12:11:36.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.140+0000 7fada3e00640 1 -- 172.21.15.167:0/3863804072 wait complete. 2024-05-11T12:11:36.157 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-11T12:11:36.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.364+0000 7fc945400640 1 Processor -- start 2024-05-11T12:11:36.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.365+0000 7fc945400640 1 -- start start 2024-05-11T12:11:36.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.366+0000 7fc945400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc940160400 0x7fc940169ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:36.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.366+0000 7fc945400640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc94006dcc0 con 0x7fc9400d40a0 2024-05-11T12:11:36.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.366+0000 7fc945400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc94006d140 con 0x7fc94015f910 2024-05-11T12:11:36.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.366+0000 7fc945400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc940156820 con 0x7fc940160400 2024-05-11T12:11:36.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.366+0000 7fc93de00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fc9400d40a0 0x7fc94015fcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:54888/0 (socket says 172.21.15.167:54888) 2024-05-11T12:11:36.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.366+0000 7fc93de00640 1 -- 172.21.15.167:0/1634177477 learned_addr learned my addr 172.21.15.167:0/1634177477 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:36.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.367+0000 7fc93d400640 1 -- 172.21.15.167:0/1634177477 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1743650171 0 0) 0x7fc94006dcc0 con 0x7fc9400d40a0 2024-05-11T12:11:36.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.367+0000 7fc93f200640 1 --2- 172.21.15.167:0/1634177477 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc940160400 0x7fc940169ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:36.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.367+0000 7fc93d400640 1 -- 172.21.15.167:0/1634177477 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc920003520 con 0x7fc9400d40a0 2024-05-11T12:11:36.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.367+0000 7fc93d400640 1 -- 172.21.15.167:0/1634177477 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 289056383 0 0) 0x7fc920003520 con 0x7fc9400d40a0 2024-05-11T12:11:36.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.367+0000 7fc93d400640 1 -- 172.21.15.167:0/1634177477 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc940160400 msgr2=0x7fc940169ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:36.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.367+0000 7fc93d400640 1 --2- 172.21.15.167:0/1634177477 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc940160400 0x7fc940169ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:36.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.367+0000 7fc93d400640 1 -- 172.21.15.167:0/1634177477 >> v1:172.21.15.204:6789/0 conn(0x7fc94015f910 legacy=0x7fc94006c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:36.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.367+0000 7fc93d400640 1 -- 172.21.15.167:0/1634177477 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc94006d420 con 0x7fc9400d40a0 2024-05-11T12:11:36.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.367+0000 7fc93d400640 1 -- 172.21.15.167:0/1634177477 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7fc928001a70 con 0x7fc9400d40a0 2024-05-11T12:11:36.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.367+0000 7fc93d400640 1 -- 172.21.15.167:0/1634177477 >> v1:172.21.15.167:6789/0 conn(0x7fc9400d40a0 legacy=0x7fc94015fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:36.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.367+0000 7fc93d400640 1 --2- 172.21.15.167:0/1634177477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc920003a80 0x7fc920003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:36.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.367+0000 7fc93d400640 1 --2- 172.21.15.167:0/1634177477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc920004800 0x7fc920004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:36.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.367+0000 7fc93d400640 1 --2- 172.21.15.167:0/1634177477 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc940160400 0x7fc920005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:36.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.367+0000 7fc93d400640 1 -- 172.21.15.167:0/1634177477 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc940156820 con 0x7fc920004800 2024-05-11T12:11:36.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.367+0000 7fc93d400640 1 -- 172.21.15.167:0/1634177477 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc92000bb80 con 0x7fc920003a80 2024-05-11T12:11:36.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.367+0000 7fc93d400640 1 -- 172.21.15.167:0/1634177477 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc92000bd80 con 0x7fc940160400 2024-05-11T12:11:36.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.368+0000 7fc93de00640 1 --2- 172.21.15.167:0/1634177477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc920004800 0x7fc920004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:36.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.368+0000 7fc93e800640 1 --2- 172.21.15.167:0/1634177477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc920003a80 0x7fc920003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:36.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.368+0000 7fc93de00640 1 -- 172.21.15.167:0/1634177477 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc940160400 msgr2=0x7fc920005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:36.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.368+0000 7fc93de00640 1 --2- 172.21.15.167:0/1634177477 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc940160400 0x7fc920005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:36.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.368+0000 7fc93de00640 1 -- 172.21.15.167:0/1634177477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc920003a80 msgr2=0x7fc920003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:36.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.368+0000 7fc93de00640 1 --2- 172.21.15.167:0/1634177477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc920003a80 0x7fc920003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:36.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.368+0000 7fc93de00640 1 -- 172.21.15.167:0/1634177477 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc920003520 con 0x7fc920004800 2024-05-11T12:11:36.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.369+0000 7fc93de00640 1 --2- 172.21.15.167:0/1634177477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc920004800 0x7fc920004c80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc92800a800 tx=0x7fc920005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b1adf0955e81bc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:36.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.369+0000 7fc93d400640 1 -- 172.21.15.167:0/1634177477 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92800a450 con 0x7fc920004800 2024-05-11T12:11:36.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.369+0000 7fc93d400640 1 -- 172.21.15.167:0/1634177477 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc92802f070 con 0x7fc920004800 2024-05-11T12:11:36.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.369+0000 7fc93d400640 1 -- 172.21.15.167:0/1634177477 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92802bba0 con 0x7fc920004800 2024-05-11T12:11:36.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.370+0000 7fc945400640 1 -- 172.21.15.167:0/1634177477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc920004800 msgr2=0x7fc920004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:36.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.370+0000 7fc945400640 1 --2- 172.21.15.167:0/1634177477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc920004800 0x7fc920004c80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc92800a800 tx=0x7fc920005830 comp rx=0 tx=0).stop 2024-05-11T12:11:36.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.370+0000 7fc93f200640 1 -- 172.21.15.167:0/1634177477 reap_dead start 2024-05-11T12:11:36.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.370+0000 7fc945400640 1 -- 172.21.15.167:0/1634177477 shutdown_connections 2024-05-11T12:11:36.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.370+0000 7fc945400640 1 -- 172.21.15.167:0/1634177477 >> 172.21.15.167:0/1634177477 conn(0x7fc9400d51e0 msgr2=0x7fc9400d55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:36.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.370+0000 7fc945400640 1 -- 172.21.15.167:0/1634177477 shutdown_connections 2024-05-11T12:11:36.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.370+0000 7fc945400640 1 -- 172.21.15.167:0/1634177477 wait complete. 2024-05-11T12:11:36.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.372+0000 7fc945400640 1 Processor -- start 2024-05-11T12:11:36.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.372+0000 7fc945400640 1 -- start start 2024-05-11T12:11:36.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.373+0000 7fc945400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc920004800 0x7fc940154750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:36.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.373+0000 7fc945400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc9400cc2b0 0x7fc9400cc710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:36.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.373+0000 7fc945400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9400cd940 0x7fc9400cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:36.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.373+0000 7fc945400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc94006ce00 con 0x7fc9400cc2b0 2024-05-11T12:11:36.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.373+0000 7fc945400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc94006d9e0 con 0x7fc9400cd940 2024-05-11T12:11:36.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.373+0000 7fc945400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc9400d0060 con 0x7fc920004800 2024-05-11T12:11:36.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.373+0000 7fc93de00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc9400cc2b0 0x7fc9400cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:36.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.373+0000 7fc93de00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc9400cc2b0 0x7fc9400cc710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:51342/0 (socket says 172.21.15.167:51342) 2024-05-11T12:11:36.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.373+0000 7fc93de00640 1 -- 172.21.15.167:0/1499741748 learned_addr learned my addr 172.21.15.167:0/1499741748 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:36.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.374+0000 7fc93de00640 1 -- 172.21.15.167:0/1499741748 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc920004800 msgr2=0x7fc940154750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:36.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.374+0000 7fc93de00640 1 --2- 172.21.15.167:0/1499741748 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc920004800 0x7fc940154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:36.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.374+0000 7fc93de00640 1 -- 172.21.15.167:0/1499741748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9400cd940 msgr2=0x7fc9400cddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:36.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.374+0000 7fc93de00640 1 --2- 172.21.15.167:0/1499741748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9400cd940 0x7fc9400cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:36.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.374+0000 7fc93de00640 1 -- 172.21.15.167:0/1499741748 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9400ce300 con 0x7fc9400cc2b0 2024-05-11T12:11:36.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.374+0000 7fc93de00640 1 --2- 172.21.15.167:0/1499741748 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc9400cc2b0 0x7fc9400cc710 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc92800a8f0 tx=0x7fc94006d5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:36.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.374+0000 7fc936a00640 1 -- 172.21.15.167:0/1499741748 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc928001f70 con 0x7fc9400cc2b0 2024-05-11T12:11:36.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.374+0000 7fc936a00640 1 -- 172.21.15.167:0/1499741748 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc928034470 con 0x7fc9400cc2b0 2024-05-11T12:11:36.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.374+0000 7fc936a00640 1 -- 172.21.15.167:0/1499741748 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc928034860 con 0x7fc9400cc2b0 2024-05-11T12:11:36.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.374+0000 7fc945400640 1 -- 172.21.15.167:0/1499741748 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9400ccd30 con 0x7fc9400cc2b0 2024-05-11T12:11:36.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.375+0000 7fc945400640 1 -- 172.21.15.167:0/1499741748 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9400cd2f0 con 0x7fc9400cc2b0 2024-05-11T12:11:36.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.376+0000 7fc945400640 1 -- 172.21.15.167:0/1499741748 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc940069520 con 0x7fc9400cc2b0 2024-05-11T12:11:36.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.377+0000 7fc936a00640 1 -- 172.21.15.167:0/1499741748 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc928034a00 con 0x7fc9400cc2b0 2024-05-11T12:11:36.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.378+0000 7fc936a00640 1 --2- 172.21.15.167:0/1499741748 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc91c0762d0 0x7fc91c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:36.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.378+0000 7fc936a00640 1 -- 172.21.15.167:0/1499741748 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6735+0+0 (secure 0 0 0) 0x7fc9280b8370 con 0x7fc9400cc2b0 2024-05-11T12:11:36.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.378+0000 7fc93e800640 1 --2- 172.21.15.167:0/1499741748 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc91c0762d0 0x7fc91c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:36.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.379+0000 7fc93e800640 1 --2- 172.21.15.167:0/1499741748 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc91c0762d0 0x7fc91c078790 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc930007fd0 tx=0x7fc930009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:36.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.383+0000 7fc936a00640 1 -- 172.21.15.167:0/1499741748 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc928081830 con 0x7fc9400cc2b0 2024-05-11T12:11:36.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.585+0000 7fc945400640 1 -- 172.21.15.167:0/1499741748 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc940075bb0 con 0x7fc9400cc2b0 2024-05-11T12:11:36.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.586+0000 7fc936a00640 1 -- 172.21.15.167:0/1499741748 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7fc928081130 con 0x7fc9400cc2b0 2024-05-11T12:11:36.585 INFO:teuthology.orchestra.run.smithi167.stdout:pg_num: 64 2024-05-11T12:11:36.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.588+0000 7fc945400640 1 -- 172.21.15.167:0/1499741748 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc91c0762d0 msgr2=0x7fc91c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:36.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.588+0000 7fc945400640 1 --2- 172.21.15.167:0/1499741748 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc91c0762d0 0x7fc91c078790 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc930007fd0 tx=0x7fc930009040 comp rx=0 tx=0).stop 2024-05-11T12:11:36.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.588+0000 7fc945400640 1 -- 172.21.15.167:0/1499741748 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc9400cc2b0 msgr2=0x7fc9400cc710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:36.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.588+0000 7fc945400640 1 --2- 172.21.15.167:0/1499741748 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc9400cc2b0 0x7fc9400cc710 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc92800a8f0 tx=0x7fc94006d5f0 comp rx=0 tx=0).stop 2024-05-11T12:11:36.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.588+0000 7fc945400640 1 -- 172.21.15.167:0/1499741748 shutdown_connections 2024-05-11T12:11:36.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.588+0000 7fc945400640 1 --2- 172.21.15.167:0/1499741748 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc91c0762d0 0x7fc91c078790 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:36.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.588+0000 7fc945400640 1 --2- 172.21.15.167:0/1499741748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9400cd940 0x7fc9400cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:36.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.588+0000 7fc945400640 1 --2- 172.21.15.167:0/1499741748 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc9400cc2b0 0x7fc9400cc710 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:36.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.588+0000 7fc945400640 1 --2- 172.21.15.167:0/1499741748 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc920004800 0x7fc940154750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:36.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.588+0000 7fc945400640 1 -- 172.21.15.167:0/1499741748 >> 172.21.15.167:0/1499741748 conn(0x7fc9400d51e0 msgr2=0x7fc9401615c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:36.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.589+0000 7fc945400640 1 -- 172.21.15.167:0/1499741748 shutdown_connections 2024-05-11T12:11:36.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.589+0000 7fc945400640 1 -- 172.21.15.167:0/1499741748 wait complete. 2024-05-11T12:11:36.606 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-11T12:11:36.788 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T12:11:36.795 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T12:11:36.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.810+0000 7f5794e00640 1 Processor -- start 2024-05-11T12:11:36.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.810+0000 7f5794e00640 1 -- start start 2024-05-11T12:11:36.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.811+0000 7f5794e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57900cfd90 0x7f57900d3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:36.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.811+0000 7f5794e00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f579015f640 con 0x7f57900d4310 2024-05-11T12:11:36.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.811+0000 7f5794e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57900d2d30 con 0x7f579015f260 2024-05-11T12:11:36.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.811+0000 7f5794e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f57900d3940 con 0x7f57900cfd90 2024-05-11T12:11:36.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.811+0000 7f578fe00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f57900d4310 0x7f579006d060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:54894/0 (socket says 172.21.15.167:54894) 2024-05-11T12:11:36.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.812+0000 7f578fe00640 1 -- 172.21.15.167:0/337344999 learned_addr learned my addr 172.21.15.167:0/337344999 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:36.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.812+0000 7f578ea00640 1 --2- 172.21.15.167:0/337344999 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57900cfd90 0x7f57900d3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:36.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.812+0000 7f578e000640 1 -- 172.21.15.167:0/337344999 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1718850736 0 0) 0x7f579015f640 con 0x7f57900d4310 2024-05-11T12:11:36.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.812+0000 7f578e000640 1 -- 172.21.15.167:0/337344999 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5774003520 con 0x7f57900d4310 2024-05-11T12:11:36.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.812+0000 7f578ea00640 1 -- 172.21.15.167:0/337344999 >> v1:172.21.15.204:6789/0 conn(0x7f579015f260 legacy=0x7f57901637e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:36.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.812+0000 7f578ea00640 1 -- 172.21.15.167:0/337344999 >> v1:172.21.15.167:6789/0 conn(0x7f57900d4310 legacy=0x7f579006d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:36.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.812+0000 7f578ea00640 1 -- 172.21.15.167:0/337344999 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57900d3010 con 0x7f57900cfd90 2024-05-11T12:11:36.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.814+0000 7f578ea00640 1 --2- 172.21.15.167:0/337344999 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57900cfd90 0x7f57900d3dd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f577c0097d0 tx=0x7f577c02a200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b33dd0a6f1a68a52 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:36.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.814+0000 7f578e000640 1 -- 172.21.15.167:0/337344999 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c037070 con 0x7f57900cfd90 2024-05-11T12:11:36.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.814+0000 7f578e000640 1 -- 172.21.15.167:0/337344999 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f577c032440 con 0x7f57900cfd90 2024-05-11T12:11:36.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.814+0000 7f578e000640 1 -- 172.21.15.167:0/337344999 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c0327f0 con 0x7f57900cfd90 2024-05-11T12:11:36.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.815+0000 7f5794e00640 1 -- 172.21.15.167:0/337344999 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57900cfd90 msgr2=0x7f57900d3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:36.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.815+0000 7f5794e00640 1 --2- 172.21.15.167:0/337344999 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57900cfd90 0x7f57900d3dd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f577c0097d0 tx=0x7f577c02a200 comp rx=0 tx=0).stop 2024-05-11T12:11:36.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.815+0000 7f5794e00640 1 -- 172.21.15.167:0/337344999 shutdown_connections 2024-05-11T12:11:36.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.815+0000 7f5794e00640 1 --2- 172.21.15.167:0/337344999 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57900cfd90 0x7f57900d3dd0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:36.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.815+0000 7f5794e00640 1 -- 172.21.15.167:0/337344999 >> 172.21.15.167:0/337344999 conn(0x7f57900b6cd0 msgr2=0x7f57900b70e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:36.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.815+0000 7f5794e00640 1 -- 172.21.15.167:0/337344999 shutdown_connections 2024-05-11T12:11:36.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.815+0000 7f5794e00640 1 -- 172.21.15.167:0/337344999 wait complete. 2024-05-11T12:11:36.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.816+0000 7f5794e00640 1 Processor -- start 2024-05-11T12:11:36.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.817+0000 7f5794e00640 1 -- start start 2024-05-11T12:11:36.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.817+0000 7f5794e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57900cfd90 0x7f579014af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:36.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.817+0000 7f5794e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57900d4310 0x7f57900b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:36.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.817+0000 7f5794e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f579015f260 0x7f57901ea230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:36.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.818+0000 7f5794e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f579006d770 con 0x7f579015f260 2024-05-11T12:11:36.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.818+0000 7f5794e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f57900d35d0 con 0x7f57900d4310 2024-05-11T12:11:36.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.818+0000 7f5794e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f579006d8f0 con 0x7f57900cfd90 2024-05-11T12:11:36.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.818+0000 7f578fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f579015f260 0x7f57901ea230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:36.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.818+0000 7f578fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f579015f260 0x7f57901ea230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:51350/0 (socket says 172.21.15.167:51350) 2024-05-11T12:11:36.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.818+0000 7f578fe00640 1 -- 172.21.15.167:0/320120066 learned_addr learned my addr 172.21.15.167:0/320120066 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:36.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.818+0000 7f578f400640 1 --2- 172.21.15.167:0/320120066 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57900cfd90 0x7f579014af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:36.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.818+0000 7f578ea00640 1 --2- 172.21.15.167:0/320120066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57900d4310 0x7f57900b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:36.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.818+0000 7f578fe00640 1 -- 172.21.15.167:0/320120066 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57900cfd90 msgr2=0x7f579014af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:36.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.818+0000 7f578fe00640 1 --2- 172.21.15.167:0/320120066 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57900cfd90 0x7f579014af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:36.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.818+0000 7f578fe00640 1 -- 172.21.15.167:0/320120066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57900d4310 msgr2=0x7f57900b6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:36.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.818+0000 7f578fe00640 1 --2- 172.21.15.167:0/320120066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57900d4310 0x7f57900b6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:36.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.818+0000 7f578fe00640 1 -- 172.21.15.167:0/320120066 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57900aba60 con 0x7f579015f260 2024-05-11T12:11:36.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.818+0000 7f578f400640 1 --2- 172.21.15.167:0/320120066 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57900cfd90 0x7f579014af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:11:36.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.819+0000 7f578fe00640 1 --2- 172.21.15.167:0/320120066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f579015f260 0x7f57901ea230 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f578800cbb0 tx=0x7f578800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:36.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.819+0000 7f5787400640 1 -- 172.21.15.167:0/320120066 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578800ad40 con 0x7f579015f260 2024-05-11T12:11:36.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.819+0000 7f5787400640 1 -- 172.21.15.167:0/320120066 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5788012400 con 0x7f579015f260 2024-05-11T12:11:36.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.819+0000 7f5787400640 1 -- 172.21.15.167:0/320120066 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5788012810 con 0x7f579015f260 2024-05-11T12:11:36.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.819+0000 7f5794e00640 1 -- 172.21.15.167:0/320120066 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57900abd40 con 0x7f579015f260 2024-05-11T12:11:36.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.819+0000 7f5794e00640 1 -- 172.21.15.167:0/320120066 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57900ac820 con 0x7f579015f260 2024-05-11T12:11:36.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.821+0000 7f5794e00640 1 -- 172.21.15.167:0/320120066 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57900694b0 con 0x7f579015f260 2024-05-11T12:11:36.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.822+0000 7f5787400640 1 -- 172.21.15.167:0/320120066 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f57880129b0 con 0x7f579015f260 2024-05-11T12:11:36.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.823+0000 7f5787400640 1 --2- 172.21.15.167:0/320120066 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f57700762d0 0x7f5770078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:36.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.823+0000 7f5787400640 1 -- 172.21.15.167:0/320120066 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6735+0+0 (secure 0 0 0) 0x7f5788098030 con 0x7f579015f260 2024-05-11T12:11:36.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.823+0000 7f578f400640 1 --2- 172.21.15.167:0/320120066 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f57700762d0 0x7f5770078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:36.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.824+0000 7f578f400640 1 --2- 172.21.15.167:0/320120066 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f57700762d0 0x7f5770078790 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f57900b9d60 tx=0x7f5780009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:36.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:36.827+0000 7f5787400640 1 -- 172.21.15.167:0/320120066 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5788061620 con 0x7f579015f260 2024-05-11T12:11:37.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.027+0000 7f5794e00640 1 -- 172.21.15.167:0/320120066 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f57900cf390 con 0x7f579015f260 2024-05-11T12:11:37.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.027+0000 7f5787400640 1 -- 172.21.15.167:0/320120066 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) ==== 103+0+11 (secure 0 0 0) 0x7f5788060f20 con 0x7f579015f260 2024-05-11T12:11:37.027 INFO:teuthology.orchestra.run.smithi167.stdout:pg_num: 32 2024-05-11T12:11:37.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.030+0000 7f5794e00640 1 -- 172.21.15.167:0/320120066 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f57700762d0 msgr2=0x7f5770078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:37.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.030+0000 7f5794e00640 1 --2- 172.21.15.167:0/320120066 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f57700762d0 0x7f5770078790 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f57900b9d60 tx=0x7f5780009040 comp rx=0 tx=0).stop 2024-05-11T12:11:37.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.030+0000 7f5794e00640 1 -- 172.21.15.167:0/320120066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f579015f260 msgr2=0x7f57901ea230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:37.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.030+0000 7f5794e00640 1 --2- 172.21.15.167:0/320120066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f579015f260 0x7f57901ea230 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f578800cbb0 tx=0x7f578800a430 comp rx=0 tx=0).stop 2024-05-11T12:11:37.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.030+0000 7f5794e00640 1 -- 172.21.15.167:0/320120066 shutdown_connections 2024-05-11T12:11:37.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.030+0000 7f5794e00640 1 --2- 172.21.15.167:0/320120066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f579015f260 0x7f57901ea230 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:37.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.030+0000 7f5794e00640 1 --2- 172.21.15.167:0/320120066 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f57700762d0 0x7f5770078790 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:37.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.030+0000 7f5794e00640 1 --2- 172.21.15.167:0/320120066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57900d4310 0x7f57900b6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:37.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.030+0000 7f5794e00640 1 --2- 172.21.15.167:0/320120066 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57900cfd90 0x7f579014af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:37.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.030+0000 7f5794e00640 1 -- 172.21.15.167:0/320120066 >> 172.21.15.167:0/320120066 conn(0x7f57900b6cd0 msgr2=0x7f57900b9fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:37.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.030+0000 7f5794e00640 1 -- 172.21.15.167:0/320120066 shutdown_connections 2024-05-11T12:11:37.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.030+0000 7f5794e00640 1 -- 172.21.15.167:0/320120066 wait complete. 2024-05-11T12:11:37.049 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_mount_conn_close' 2024-05-11T12:11:37.241 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.241+0000 7f226f200640 1 Processor -- start 2024-05-11T12:11:37.241 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.242+0000 7f226f200640 1 -- start start 2024-05-11T12:11:37.242 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.243+0000 7f226f200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2270162780 0x7f22700d2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:37.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.243+0000 7f226f200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f22700d4340 0x7f227006cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:37.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.243+0000 7f226f200640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22700b9480 con 0x7f227006d310 2024-05-11T12:11:37.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.243+0000 7f226f200640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f22700d3070 con 0x7f22700d4340 2024-05-11T12:11:37.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.243+0000 7f226f200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f22700d32a0 con 0x7f2270162780 2024-05-11T12:11:37.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.244+0000 7f226d400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f22700d4340 0x7f227006cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:37.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.244+0000 7f226e800640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f227006d310 0x7f22700cff30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:54900/0 (socket says 172.21.15.167:54900) 2024-05-11T12:11:37.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.244+0000 7f226e800640 1 -- 172.21.15.167:0/633205636 learned_addr learned my addr 172.21.15.167:0/633205636 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:37.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.244+0000 7f226de00640 1 --2- 172.21.15.167:0/633205636 >> v2:172.21.15.204:3300/0 conn(0x7f2270162780 0x7f22700d2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:37.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.244+0000 7f226d400640 1 -- 172.21.15.167:0/633205636 >> v2:172.21.15.204:3300/0 conn(0x7f2270162780 msgr2=0x7f22700d2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:37.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.244+0000 7f226d400640 1 --2- 172.21.15.167:0/633205636 >> v2:172.21.15.204:3300/0 conn(0x7f2270162780 0x7f22700d2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:37.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.244+0000 7f226d400640 1 -- 172.21.15.167:0/633205636 >> v1:172.21.15.167:6789/0 conn(0x7f227006d310 legacy=0x7f22700cff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:37.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.244+0000 7f226d400640 1 -- 172.21.15.167:0/633205636 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22700d3580 con 0x7f22700d4340 2024-05-11T12:11:37.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.244+0000 7f226ca00640 1 -- 172.21.15.167:0/633205636 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4069788856 0 0) 0x7f22700b9480 con 0x7f227006d310 2024-05-11T12:11:37.245 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.245+0000 7f226d400640 1 --2- 172.21.15.167:0/633205636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22700d4340 0x7f227006cdd0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f22700d48e0 tx=0x7f225802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66ab917d790cf856 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:37.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.245+0000 7f226ca00640 1 -- 172.21.15.167:0/633205636 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2258007740 con 0x7f22700d4340 2024-05-11T12:11:37.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.245+0000 7f226ca00640 1 -- 172.21.15.167:0/633205636 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22580078e0 con 0x7f22700d4340 2024-05-11T12:11:37.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.245+0000 7f226ca00640 1 -- 172.21.15.167:0/633205636 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2258007cb0 con 0x7f22700d4340 2024-05-11T12:11:37.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.246+0000 7f226f200640 1 -- 172.21.15.167:0/633205636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22700d4340 msgr2=0x7f227006cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:37.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.246+0000 7f226f200640 1 --2- 172.21.15.167:0/633205636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22700d4340 0x7f227006cdd0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f22700d48e0 tx=0x7f225802a5f0 comp rx=0 tx=0).stop 2024-05-11T12:11:37.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.246+0000 7f226f200640 1 -- 172.21.15.167:0/633205636 shutdown_connections 2024-05-11T12:11:37.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.246+0000 7f226f200640 1 --2- 172.21.15.167:0/633205636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22700d4340 0x7f227006cdd0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:37.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.246+0000 7f226f200640 1 --2- 172.21.15.167:0/633205636 >> v2:172.21.15.204:3300/0 conn(0x7f2270162780 0x7f22700d2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:37.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.246+0000 7f226f200640 1 -- 172.21.15.167:0/633205636 >> 172.21.15.167:0/633205636 conn(0x7f22700d1bc0 msgr2=0x7f22700d1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:37.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.246+0000 7f226f200640 1 -- 172.21.15.167:0/633205636 shutdown_connections 2024-05-11T12:11:37.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.246+0000 7f226f200640 1 -- 172.21.15.167:0/633205636 wait complete. 2024-05-11T12:11:37.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.248+0000 7f226f200640 1 Processor -- start 2024-05-11T12:11:37.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.248+0000 7f226f200640 1 -- start start 2024-05-11T12:11:37.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.249+0000 7f226f200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f227006d310 0x7f22701e7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:37.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.249+0000 7f226f200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2270162780 0x7f22700cda90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:37.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.249+0000 7f226f200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22701f5390 0x7f22701f5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:37.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.249+0000 7f226f200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f22700d3e20 con 0x7f22701f5390 2024-05-11T12:11:37.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.249+0000 7f226f200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f22700d3b40 con 0x7f2270162780 2024-05-11T12:11:37.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.249+0000 7f226f200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2270162b60 con 0x7f227006d310 2024-05-11T12:11:37.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.249+0000 7f226e800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22701f5390 0x7f22701f5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:37.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.249+0000 7f226e800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22701f5390 0x7f22701f5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:51370/0 (socket says 172.21.15.167:51370) 2024-05-11T12:11:37.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.249+0000 7f226e800640 1 -- 172.21.15.167:0/3972599109 learned_addr learned my addr 172.21.15.167:0/3972599109 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:37.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.250+0000 7f226d400640 1 --2- 172.21.15.167:0/3972599109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2270162780 0x7f22700cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:37.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.250+0000 7f226de00640 1 --2- 172.21.15.167:0/3972599109 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f227006d310 0x7f22701e7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:37.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.250+0000 7f226e800640 1 -- 172.21.15.167:0/3972599109 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f227006d310 msgr2=0x7f22701e7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:37.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.250+0000 7f226e800640 1 --2- 172.21.15.167:0/3972599109 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f227006d310 0x7f22701e7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:37.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.250+0000 7f226e800640 1 -- 172.21.15.167:0/3972599109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2270162780 msgr2=0x7f22700cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:37.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.250+0000 7f226e800640 1 --2- 172.21.15.167:0/3972599109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2270162780 0x7f22700cda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:37.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.250+0000 7f226e800640 1 -- 172.21.15.167:0/3972599109 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22701f2dc0 con 0x7f22701f5390 2024-05-11T12:11:37.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.250+0000 7f226d400640 1 --2- 172.21.15.167:0/3972599109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2270162780 0x7f22700cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:37.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.250+0000 7f226de00640 1 --2- 172.21.15.167:0/3972599109 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f227006d310 0x7f22701e7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:37.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.250+0000 7f226e800640 1 --2- 172.21.15.167:0/3972599109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22701f5390 0x7f22701f5840 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f226800c410 tx=0x7f226800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:37.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.250+0000 7f2266000640 1 -- 172.21.15.167:0/3972599109 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22680074e0 con 0x7f22701f5390 2024-05-11T12:11:37.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.250+0000 7f2266000640 1 -- 172.21.15.167:0/3972599109 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22680047e0 con 0x7f22701f5390 2024-05-11T12:11:37.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.250+0000 7f2266000640 1 -- 172.21.15.167:0/3972599109 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2268004bb0 con 0x7f22701f5390 2024-05-11T12:11:37.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.250+0000 7f226f200640 1 -- 172.21.15.167:0/3972599109 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22701f6700 con 0x7f22701f5390 2024-05-11T12:11:37.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.250+0000 7f226f200640 1 -- 172.21.15.167:0/3972599109 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22700ce4e0 con 0x7f22701f5390 2024-05-11T12:11:37.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.251+0000 7f226f200640 1 -- 172.21.15.167:0/3972599109 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22700694b0 con 0x7f22701f5390 2024-05-11T12:11:37.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.253+0000 7f2266000640 1 -- 172.21.15.167:0/3972599109 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2268011070 con 0x7f22701f5390 2024-05-11T12:11:37.253 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.254+0000 7f2266000640 1 --2- 172.21.15.167:0/3972599109 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f224c0762d0 0x7f224c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:37.253 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.254+0000 7f2266000640 1 -- 172.21.15.167:0/3972599109 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6735+0+0 (secure 0 0 0) 0x7f2268097650 con 0x7f22701f5390 2024-05-11T12:11:37.253 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.254+0000 7f226de00640 1 --2- 172.21.15.167:0/3972599109 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f224c0762d0 0x7f224c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:37.253 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.254+0000 7f226de00640 1 --2- 172.21.15.167:0/3972599109 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f224c0762d0 0x7f224c078790 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2260006fd0 tx=0x7f2260008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:37.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.258+0000 7f2266000640 1 -- 172.21.15.167:0/3972599109 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2268060b10 con 0x7f22701f5390 2024-05-11T12:11:37.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:37.540+0000 7f226f200640 1 -- 172.21.15.167:0/3972599109 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_mount_conn_close"]} v 0) -- 0x7f22700cf370 con 0x7f22701f5390 2024-05-11T12:11:38.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.109+0000 7f2266000640 1 -- 172.21.15.167:0/3972599109 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_mount_conn_close"]}]=0 v67) ==== 147+0+0 (secure 0 0 0) 0x7f2268060410 con 0x7f22701f5390 2024-05-11T12:11:38.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.112+0000 7f226f200640 1 -- 172.21.15.167:0/3972599109 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f224c0762d0 msgr2=0x7f224c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:38.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.112+0000 7f226f200640 1 --2- 172.21.15.167:0/3972599109 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f224c0762d0 0x7f224c078790 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2260006fd0 tx=0x7f2260008040 comp rx=0 tx=0).stop 2024-05-11T12:11:38.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.112+0000 7f226f200640 1 -- 172.21.15.167:0/3972599109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22701f5390 msgr2=0x7f22701f5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:38.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.112+0000 7f226f200640 1 --2- 172.21.15.167:0/3972599109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22701f5390 0x7f22701f5840 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f226800c410 tx=0x7f226800c8d0 comp rx=0 tx=0).stop 2024-05-11T12:11:38.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.112+0000 7f226f200640 1 -- 172.21.15.167:0/3972599109 shutdown_connections 2024-05-11T12:11:38.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.112+0000 7f226f200640 1 --2- 172.21.15.167:0/3972599109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22701f5390 0x7f22701f5840 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:38.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.112+0000 7f226f200640 1 --2- 172.21.15.167:0/3972599109 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f224c0762d0 0x7f224c078790 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:38.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.112+0000 7f226f200640 1 --2- 172.21.15.167:0/3972599109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2270162780 0x7f22700cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:38.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.112+0000 7f226f200640 1 --2- 172.21.15.167:0/3972599109 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f227006d310 0x7f22701e7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:38.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.112+0000 7f226f200640 1 -- 172.21.15.167:0/3972599109 >> 172.21.15.167:0/3972599109 conn(0x7f22700d1bc0 msgr2=0x7f22700ba440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:38.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.113+0000 7f226f200640 1 -- 172.21.15.167:0/3972599109 shutdown_connections 2024-05-11T12:11:38.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.113+0000 7f226f200640 1 -- 172.21.15.167:0/3972599109 wait complete. 2024-05-11T12:11:38.134 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-11T12:11:38.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.342+0000 7f993be00640 1 Processor -- start 2024-05-11T12:11:38.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.343+0000 7f993be00640 1 -- start start 2024-05-11T12:11:38.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.344+0000 7f993be00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f993415ac80 0x7f9934074260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:38.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.344+0000 7f993be00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9934161380 0x7f9934072950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:38.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.344+0000 7f993be00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99340b65e0 con 0x7f99340b3d50 2024-05-11T12:11:38.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.344+0000 7f993be00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f993406e3c0 con 0x7f993415ac80 2024-05-11T12:11:38.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.344+0000 7f993be00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f993415b060 con 0x7f9934161380 2024-05-11T12:11:38.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.345+0000 7f9939400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f993415ac80 0x7f9934074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:38.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.345+0000 7f9938a00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f99340b3d50 0x7f9934160c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:54914/0 (socket says 172.21.15.167:54914) 2024-05-11T12:11:38.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.345+0000 7f9938a00640 1 -- 172.21.15.167:0/2824134196 learned_addr learned my addr 172.21.15.167:0/2824134196 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:38.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.345+0000 7f9939e00640 1 --2- 172.21.15.167:0/2824134196 >> v2:172.21.15.204:3300/0 conn(0x7f9934161380 0x7f9934072950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:38.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.345+0000 7f9939400640 1 -- 172.21.15.167:0/2824134196 >> v2:172.21.15.204:3300/0 conn(0x7f9934161380 msgr2=0x7f9934072950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:38.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.345+0000 7f9939400640 1 --2- 172.21.15.167:0/2824134196 >> v2:172.21.15.204:3300/0 conn(0x7f9934161380 0x7f9934072950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:38.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.345+0000 7f9939400640 1 -- 172.21.15.167:0/2824134196 >> v1:172.21.15.167:6789/0 conn(0x7f99340b3d50 legacy=0x7f9934160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:38.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.345+0000 7f9939400640 1 -- 172.21.15.167:0/2824134196 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99340b68c0 con 0x7f993415ac80 2024-05-11T12:11:38.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.346+0000 7f9939400640 1 --2- 172.21.15.167:0/2824134196 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f993415ac80 0x7f9934074260 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f99240097a0 tx=0x7f992402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71832218ea5c62e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:38.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.347+0000 7f9933e00640 1 -- 172.21.15.167:0/2824134196 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9924037070 con 0x7f993415ac80 2024-05-11T12:11:38.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.347+0000 7f9933e00640 1 -- 172.21.15.167:0/2824134196 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9924032440 con 0x7f993415ac80 2024-05-11T12:11:38.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.347+0000 7f9933e00640 1 -- 172.21.15.167:0/2824134196 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9924032810 con 0x7f993415ac80 2024-05-11T12:11:38.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.347+0000 7f993be00640 1 -- 172.21.15.167:0/2824134196 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f993415ac80 msgr2=0x7f9934074260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:38.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.347+0000 7f993be00640 1 --2- 172.21.15.167:0/2824134196 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f993415ac80 0x7f9934074260 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f99240097a0 tx=0x7f992402a1e0 comp rx=0 tx=0).stop 2024-05-11T12:11:38.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.348+0000 7f993be00640 1 -- 172.21.15.167:0/2824134196 shutdown_connections 2024-05-11T12:11:38.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.348+0000 7f993be00640 1 --2- 172.21.15.167:0/2824134196 >> v2:172.21.15.204:3300/0 conn(0x7f9934161380 0x7f9934072950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:38.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.348+0000 7f993be00640 1 --2- 172.21.15.167:0/2824134196 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f993415ac80 0x7f9934074260 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:38.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.348+0000 7f993be00640 1 -- 172.21.15.167:0/2824134196 >> 172.21.15.167:0/2824134196 conn(0x7f993406c450 msgr2=0x7f993406c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:38.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.348+0000 7f993be00640 1 -- 172.21.15.167:0/2824134196 shutdown_connections 2024-05-11T12:11:38.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.348+0000 7f993be00640 1 -- 172.21.15.167:0/2824134196 wait complete. 2024-05-11T12:11:38.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.349+0000 7f993be00640 1 Processor -- start 2024-05-11T12:11:38.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.350+0000 7f993be00640 1 -- start start 2024-05-11T12:11:38.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.350+0000 7f993be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99340b3d50 0x7f99340d0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:38.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.350+0000 7f993be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f993415ac80 0x7f99341446f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:38.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.350+0000 7f993be00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9934161380 0x7f99340ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:38.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.351+0000 7f993be00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f99340b44f0 con 0x7f993415ac80 2024-05-11T12:11:38.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.351+0000 7f993be00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f99340b41e0 con 0x7f99340b3d50 2024-05-11T12:11:38.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.351+0000 7f993be00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f9934072e90 con 0x7f9934161380 2024-05-11T12:11:38.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.351+0000 7f9938a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f993415ac80 0x7f99341446f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:38.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.351+0000 7f9938a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f993415ac80 0x7f99341446f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:51384/0 (socket says 172.21.15.167:51384) 2024-05-11T12:11:38.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.351+0000 7f9938a00640 1 -- 172.21.15.167:0/1720894329 learned_addr learned my addr 172.21.15.167:0/1720894329 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:38.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.351+0000 7f9939e00640 1 --2- 172.21.15.167:0/1720894329 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9934161380 0x7f99340ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:38.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.351+0000 7f9939400640 1 --2- 172.21.15.167:0/1720894329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99340b3d50 0x7f99340d0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:38.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.351+0000 7f9938a00640 1 -- 172.21.15.167:0/1720894329 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9934161380 msgr2=0x7f99340ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:38.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.351+0000 7f9938a00640 1 --2- 172.21.15.167:0/1720894329 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9934161380 0x7f99340ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:38.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.351+0000 7f9938a00640 1 -- 172.21.15.167:0/1720894329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99340b3d50 msgr2=0x7f99340d0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:38.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.351+0000 7f9938a00640 1 --2- 172.21.15.167:0/1720894329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99340b3d50 0x7f99340d0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:38.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.351+0000 7f9938a00640 1 -- 172.21.15.167:0/1720894329 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99340cf460 con 0x7f993415ac80 2024-05-11T12:11:38.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.351+0000 7f9939400640 1 --2- 172.21.15.167:0/1720894329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99340b3d50 0x7f99340d0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:38.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.352+0000 7f9938a00640 1 --2- 172.21.15.167:0/1720894329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f993415ac80 0x7f99341446f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f992800c500 tx=0x7f992800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:38.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.352+0000 7f9931600640 1 -- 172.21.15.167:0/1720894329 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9928017070 con 0x7f993415ac80 2024-05-11T12:11:38.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.352+0000 7f9931600640 1 -- 172.21.15.167:0/1720894329 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9928005d00 con 0x7f993415ac80 2024-05-11T12:11:38.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.352+0000 7f9931600640 1 -- 172.21.15.167:0/1720894329 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9928007330 con 0x7f993415ac80 2024-05-11T12:11:38.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.352+0000 7f993be00640 1 -- 172.21.15.167:0/1720894329 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99340cf740 con 0x7f993415ac80 2024-05-11T12:11:38.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.352+0000 7f993be00640 1 -- 172.21.15.167:0/1720894329 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99340cffe0 con 0x7f993415ac80 2024-05-11T12:11:38.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.353+0000 7f993be00640 1 -- 172.21.15.167:0/1720894329 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9934073ff0 con 0x7f993415ac80 2024-05-11T12:11:38.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.354+0000 7f9931600640 1 -- 172.21.15.167:0/1720894329 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f992801a070 con 0x7f993415ac80 2024-05-11T12:11:38.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.356+0000 7f9931600640 1 --2- 172.21.15.167:0/1720894329 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f99180762d0 0x7f9918078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:38.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.356+0000 7f9931600640 1 -- 172.21.15.167:0/1720894329 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6735+0+0 (secure 0 0 0) 0x7f9928096db0 con 0x7f993415ac80 2024-05-11T12:11:38.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.356+0000 7f9939400640 1 --2- 172.21.15.167:0/1720894329 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f99180762d0 0x7f9918078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:38.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.357+0000 7f9939400640 1 --2- 172.21.15.167:0/1720894329 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f99180762d0 0x7f9918078790 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9924034000 tx=0x7f992402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:38.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.360+0000 7f9931600640 1 -- 172.21.15.167:0/1720894329 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9934073ff0 con 0x7f993415ac80 2024-05-11T12:11:38.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.567+0000 7f993be00640 1 -- 172.21.15.167:0/1720894329 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f99340d01e0 con 0x7f993415ac80 2024-05-11T12:11:38.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.574+0000 7f9931600640 1 -- 172.21.15.167:0/1720894329 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f99340d01e0 con 0x7f993415ac80 2024-05-11T12:11:38.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.577+0000 7f993be00640 1 -- 172.21.15.167:0/1720894329 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f99180762d0 msgr2=0x7f9918078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:38.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.577+0000 7f993be00640 1 --2- 172.21.15.167:0/1720894329 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f99180762d0 0x7f9918078790 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9924034000 tx=0x7f992402e000 comp rx=0 tx=0).stop 2024-05-11T12:11:38.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.577+0000 7f993be00640 1 -- 172.21.15.167:0/1720894329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f993415ac80 msgr2=0x7f99341446f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:38.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.577+0000 7f993be00640 1 --2- 172.21.15.167:0/1720894329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f993415ac80 0x7f99341446f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f992800c500 tx=0x7f992800c9c0 comp rx=0 tx=0).stop 2024-05-11T12:11:38.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.577+0000 7f993be00640 1 -- 172.21.15.167:0/1720894329 shutdown_connections 2024-05-11T12:11:38.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.577+0000 7f993be00640 1 --2- 172.21.15.167:0/1720894329 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9934161380 0x7f99340ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:38.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.577+0000 7f993be00640 1 --2- 172.21.15.167:0/1720894329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f993415ac80 0x7f99341446f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:38.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.577+0000 7f993be00640 1 --2- 172.21.15.167:0/1720894329 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f99180762d0 0x7f9918078790 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:38.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.577+0000 7f993be00640 1 --2- 172.21.15.167:0/1720894329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99340b3d50 0x7f99340d0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:38.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.577+0000 7f993be00640 1 -- 172.21.15.167:0/1720894329 >> 172.21.15.167:0/1720894329 conn(0x7f993406c450 msgr2=0x7f993406e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:38.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.577+0000 7f993be00640 1 -- 172.21.15.167:0/1720894329 shutdown_connections 2024-05-11T12:11:38.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:38.577+0000 7f993be00640 1 -- 172.21.15.167:0/1720894329 wait complete. 2024-05-11T12:11:38.595 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-11T12:11:38.595 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-11T12:11:38.708 DEBUG:teuthology.orchestra.run.smithi204:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-11T12:11:38.822 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-11T12:11:38.823 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:11:38.823 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:11:38.880 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:11:38.881 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:11:38.935 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:11:38.935 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:11:38.988 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-11T12:11:39.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.192+0000 7f1be6000640 1 Processor -- start 2024-05-11T12:11:39.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.192+0000 7f1be6000640 1 -- start start 2024-05-11T12:11:39.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.194+0000 7f1be6000640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1be00d0f60 0x7f1be00d3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:39.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.194+0000 7f1be6000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1be00d4140 0x7f1be006cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:39.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.194+0000 7f1be6000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1be00d13f0 con 0x7f1be01625d0 2024-05-11T12:11:39.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.194+0000 7f1be6000640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f1be00d45b0 con 0x7f1be00d0f60 2024-05-11T12:11:39.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.194+0000 7f1be6000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1be00d2e40 con 0x7f1be00d4140 2024-05-11T12:11:39.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.194+0000 7f1bdfe00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1be00d0f60 0x7f1be00d3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:39.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.194+0000 7f1bdfe00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1be00d0f60 0x7f1be00d3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:53606/0 (socket says 172.21.15.167:53606) 2024-05-11T12:11:39.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.194+0000 7f1bdfe00640 1 -- 172.21.15.167:0/734132085 learned_addr learned my addr 172.21.15.167:0/734132085 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:39.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.195+0000 7f1be5600640 1 --2- 172.21.15.167:0/734132085 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1be00d4140 0x7f1be006cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:39.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.195+0000 7f1bdfe00640 1 -- 172.21.15.167:0/734132085 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1be00d4140 msgr2=0x7f1be006cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:39.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.195+0000 7f1bdfe00640 1 --2- 172.21.15.167:0/734132085 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1be00d4140 0x7f1be006cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:39.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.195+0000 7f1bdfe00640 1 -- 172.21.15.167:0/734132085 >> v1:172.21.15.204:6789/0 conn(0x7f1be01625d0 legacy=0x7f1be00b8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:39.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.195+0000 7f1bdfe00640 1 -- 172.21.15.167:0/734132085 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1be00d3120 con 0x7f1be00d0f60 2024-05-11T12:11:39.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.196+0000 7f1bdfe00640 1 --2- 172.21.15.167:0/734132085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1be00d0f60 0x7f1be00d3c00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1be00d48e0 tx=0x7f1bcc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5247159828d5df6 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:39.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.196+0000 7f1bdf400640 1 -- 172.21.15.167:0/734132085 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc037070 con 0x7f1be00d0f60 2024-05-11T12:11:39.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.196+0000 7f1bdf400640 1 -- 172.21.15.167:0/734132085 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bcc032440 con 0x7f1be00d0f60 2024-05-11T12:11:39.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.196+0000 7f1bdf400640 1 -- 172.21.15.167:0/734132085 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc032810 con 0x7f1be00d0f60 2024-05-11T12:11:39.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.197+0000 7f1be6000640 1 -- 172.21.15.167:0/734132085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1be00d0f60 msgr2=0x7f1be00d3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:39.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.197+0000 7f1be6000640 1 --2- 172.21.15.167:0/734132085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1be00d0f60 0x7f1be00d3c00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1be00d48e0 tx=0x7f1bcc02a180 comp rx=0 tx=0).stop 2024-05-11T12:11:39.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.197+0000 7f1be6000640 1 -- 172.21.15.167:0/734132085 shutdown_connections 2024-05-11T12:11:39.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.197+0000 7f1be6000640 1 --2- 172.21.15.167:0/734132085 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1be00d4140 0x7f1be006cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:39.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.197+0000 7f1be6000640 1 --2- 172.21.15.167:0/734132085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1be00d0f60 0x7f1be00d3c00 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:39.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.197+0000 7f1be6000640 1 -- 172.21.15.167:0/734132085 >> 172.21.15.167:0/734132085 conn(0x7f1be00d50f0 msgr2=0x7f1be00d5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:39.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.197+0000 7f1be6000640 1 -- 172.21.15.167:0/734132085 shutdown_connections 2024-05-11T12:11:39.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.197+0000 7f1be6000640 1 -- 172.21.15.167:0/734132085 wait complete. 2024-05-11T12:11:39.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.199+0000 7f1be6000640 1 Processor -- start 2024-05-11T12:11:39.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.199+0000 7f1be6000640 1 -- start start 2024-05-11T12:11:39.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.200+0000 7f1be6000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1be00d0f60 0x7f1be00ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:39.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.200+0000 7f1be6000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1be00d4140 0x7f1be00ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:39.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.200+0000 7f1be6000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1be01625d0 0x7f1be0155420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:39.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.200+0000 7f1be6000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1be00b8aa0 con 0x7f1be01625d0 2024-05-11T12:11:39.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.200+0000 7f1be6000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1be00d36e0 con 0x7f1be00d0f60 2024-05-11T12:11:39.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.200+0000 7f1be6000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1be01629b0 con 0x7f1be00d4140 2024-05-11T12:11:39.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.200+0000 7f1be5600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1be01625d0 0x7f1be0155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:39.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.200+0000 7f1be5600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1be01625d0 0x7f1be0155420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:53616/0 (socket says 172.21.15.167:53616) 2024-05-11T12:11:39.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.200+0000 7f1be5600640 1 -- 172.21.15.167:0/185767187 learned_addr learned my addr 172.21.15.167:0/185767187 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:39.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.201+0000 7f1bdfe00640 1 --2- 172.21.15.167:0/185767187 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1be00d4140 0x7f1be00ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:39.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.201+0000 7f1be4c00640 1 --2- 172.21.15.167:0/185767187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1be00d0f60 0x7f1be00ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:39.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.201+0000 7f1be5600640 1 -- 172.21.15.167:0/185767187 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1be00d4140 msgr2=0x7f1be00ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:39.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.201+0000 7f1be5600640 1 --2- 172.21.15.167:0/185767187 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1be00d4140 0x7f1be00ce7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:39.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.201+0000 7f1be5600640 1 -- 172.21.15.167:0/185767187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1be00d0f60 msgr2=0x7f1be00ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:39.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.201+0000 7f1be5600640 1 --2- 172.21.15.167:0/185767187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1be00d0f60 0x7f1be00ccbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:39.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.201+0000 7f1be5600640 1 -- 172.21.15.167:0/185767187 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1be00cd2d0 con 0x7f1be01625d0 2024-05-11T12:11:39.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.201+0000 7f1bdfe00640 1 --2- 172.21.15.167:0/185767187 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1be00d4140 0x7f1be00ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:39.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.201+0000 7f1be5600640 1 --2- 172.21.15.167:0/185767187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1be01625d0 0x7f1be0155420 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1bd800cd70 tx=0x7f1bd800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:39.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.201+0000 7f1bdcc00640 1 -- 172.21.15.167:0/185767187 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd8003b90 con 0x7f1be01625d0 2024-05-11T12:11:39.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.201+0000 7f1bdcc00640 1 -- 172.21.15.167:0/185767187 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bd8003d30 con 0x7f1be01625d0 2024-05-11T12:11:39.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.201+0000 7f1be6000640 1 -- 172.21.15.167:0/185767187 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1be00cd5b0 con 0x7f1be01625d0 2024-05-11T12:11:39.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.201+0000 7f1bdcc00640 1 -- 172.21.15.167:0/185767187 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd8004240 con 0x7f1be01625d0 2024-05-11T12:11:39.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.201+0000 7f1be6000640 1 -- 172.21.15.167:0/185767187 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1be00c8c60 con 0x7f1be01625d0 2024-05-11T12:11:39.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.202+0000 7f1be6000640 1 -- 172.21.15.167:0/185767187 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1be00694b0 con 0x7f1be01625d0 2024-05-11T12:11:39.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.204+0000 7f1bdcc00640 1 -- 172.21.15.167:0/185767187 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1bd80043e0 con 0x7f1be01625d0 2024-05-11T12:11:39.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.205+0000 7f1bdcc00640 1 --2- 172.21.15.167:0/185767187 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f1bc0076300 0x7f1bc00787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:39.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.205+0000 7f1bdcc00640 1 -- 172.21.15.167:0/185767187 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6735+0+0 (secure 0 0 0) 0x7f1bd8098520 con 0x7f1be01625d0 2024-05-11T12:11:39.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.205+0000 7f1be4c00640 1 --2- 172.21.15.167:0/185767187 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f1bc0076300 0x7f1bc00787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:39.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.206+0000 7f1be4c00640 1 --2- 172.21.15.167:0/185767187 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f1bc0076300 0x7f1bc00787c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1be00cb9a0 tx=0x7f1bd0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:39.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.209+0000 7f1bdcc00640 1 -- 172.21.15.167:0/185767187 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1bd8061a70 con 0x7f1be01625d0 2024-05-11T12:11:39.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.431+0000 7f1be6000640 1 -- 172.21.15.167:0/185767187 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1be00b9880 con 0x7f1be01625d0 2024-05-11T12:11:39.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.432+0000 7f1bdcc00640 1 -- 172.21.15.167:0/185767187 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) ==== 80+0+0 (secure 0 0 0) 0x7f1bd8061370 con 0x7f1be01625d0 2024-05-11T12:11:39.431 INFO:teuthology.orchestra.run.smithi167.stderr:listed 0 entries 2024-05-11T12:11:39.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.435+0000 7f1be6000640 1 -- 172.21.15.167:0/185767187 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f1bc0076300 msgr2=0x7f1bc00787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:39.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.435+0000 7f1be6000640 1 --2- 172.21.15.167:0/185767187 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f1bc0076300 0x7f1bc00787c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1be00cb9a0 tx=0x7f1bd0009040 comp rx=0 tx=0).stop 2024-05-11T12:11:39.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.435+0000 7f1be6000640 1 -- 172.21.15.167:0/185767187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1be01625d0 msgr2=0x7f1be0155420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:39.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.435+0000 7f1be6000640 1 --2- 172.21.15.167:0/185767187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1be01625d0 0x7f1be0155420 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1bd800cd70 tx=0x7f1bd800a660 comp rx=0 tx=0).stop 2024-05-11T12:11:39.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.435+0000 7f1be6000640 1 -- 172.21.15.167:0/185767187 shutdown_connections 2024-05-11T12:11:39.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.435+0000 7f1be6000640 1 --2- 172.21.15.167:0/185767187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1be01625d0 0x7f1be0155420 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:39.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.435+0000 7f1be6000640 1 --2- 172.21.15.167:0/185767187 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1be00d4140 0x7f1be00ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:39.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.435+0000 7f1be6000640 1 --2- 172.21.15.167:0/185767187 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f1bc0076300 0x7f1bc00787c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:39.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.435+0000 7f1be6000640 1 --2- 172.21.15.167:0/185767187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1be00d0f60 0x7f1be00ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:39.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.435+0000 7f1be6000640 1 -- 172.21.15.167:0/185767187 >> 172.21.15.167:0/185767187 conn(0x7f1be00d50f0 msgr2=0x7f1be0161790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:39.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.435+0000 7f1be6000640 1 -- 172.21.15.167:0/185767187 shutdown_connections 2024-05-11T12:11:39.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:39.435+0000 7f1be6000640 1 -- 172.21.15.167:0/185767187 wait complete. 2024-05-11T12:11:39.454 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-11T12:11:39.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:39.454 DEBUG:teuthology.orchestra.run.smithi167:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-05-11T12:11:39.542 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2024-05-11T12:11:39.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:39.543 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-11T12:11:39.589 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:11:39.589 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:11:39.645 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:11:39.646 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:11:39.693 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:11:39.693 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:11:39.790 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-11T12:11:40.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.003+0000 7fd96ba00640 1 Processor -- start 2024-05-11T12:11:40.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.003+0000 7fd96ba00640 1 -- start start 2024-05-11T12:11:40.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.005+0000 7fd96ba00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd964072860 0x7fd964163c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:40.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.005+0000 7fd96ba00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd964164580 con 0x7fd964164170 2024-05-11T12:11:40.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.005+0000 7fd96ba00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd964167520 con 0x7fd9641468b0 2024-05-11T12:11:40.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.005+0000 7fd96ba00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fd96406ea10 con 0x7fd964072860 2024-05-11T12:11:40.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.005+0000 7fd963e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd964072860 0x7fd964163c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:40.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.005+0000 7fd963e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd964072860 0x7fd964163c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:53632/0 (socket says 172.21.15.167:53632) 2024-05-11T12:11:40.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.005+0000 7fd963e00640 1 -- 172.21.15.167:0/2476287581 learned_addr learned my addr 172.21.15.167:0/2476287581 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:40.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.005+0000 7fd969a00640 1 --1- 172.21.15.167:0/2476287581 >> v1:172.21.15.167:6789/0 conn(0x7fd964164170 0x7fd96406eea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:50896/0 (socket says 172.21.15.167:50896) 2024-05-11T12:11:40.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.005+0000 7fd963e00640 1 -- 172.21.15.167:0/2476287581 >> v1:172.21.15.204:6789/0 conn(0x7fd9641468b0 legacy=0x7fd964072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:40.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.006+0000 7fd963e00640 1 -- 172.21.15.167:0/2476287581 >> v1:172.21.15.167:6789/0 conn(0x7fd964164170 legacy=0x7fd96406eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:40.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.006+0000 7fd963e00640 1 -- 172.21.15.167:0/2476287581 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd964164810 con 0x7fd964072860 2024-05-11T12:11:40.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.007+0000 7fd963e00640 1 --2- 172.21.15.167:0/2476287581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd964072860 0x7fd964163c30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd950009870 tx=0x7fd95002a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d58eb52dc1fa54d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:40.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.007+0000 7fd963400640 1 -- 172.21.15.167:0/2476287581 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd950037070 con 0x7fd964072860 2024-05-11T12:11:40.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.007+0000 7fd963400640 1 -- 172.21.15.167:0/2476287581 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd950032440 con 0x7fd964072860 2024-05-11T12:11:40.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.007+0000 7fd963400640 1 -- 172.21.15.167:0/2476287581 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd950032810 con 0x7fd964072860 2024-05-11T12:11:40.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.008+0000 7fd96ba00640 1 -- 172.21.15.167:0/2476287581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd964072860 msgr2=0x7fd964163c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:40.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.008+0000 7fd96ba00640 1 --2- 172.21.15.167:0/2476287581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd964072860 0x7fd964163c30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd950009870 tx=0x7fd95002a130 comp rx=0 tx=0).stop 2024-05-11T12:11:40.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.008+0000 7fd96ba00640 1 -- 172.21.15.167:0/2476287581 shutdown_connections 2024-05-11T12:11:40.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.008+0000 7fd96ba00640 1 --2- 172.21.15.167:0/2476287581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd964072860 0x7fd964163c30 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:40.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.008+0000 7fd96ba00640 1 -- 172.21.15.167:0/2476287581 >> 172.21.15.167:0/2476287581 conn(0x7fd96406c450 msgr2=0x7fd96406c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:40.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.008+0000 7fd96ba00640 1 -- 172.21.15.167:0/2476287581 shutdown_connections 2024-05-11T12:11:40.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.008+0000 7fd96ba00640 1 -- 172.21.15.167:0/2476287581 wait complete. 2024-05-11T12:11:40.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.010+0000 7fd96ba00640 1 Processor -- start 2024-05-11T12:11:40.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.010+0000 7fd96ba00640 1 -- start start 2024-05-11T12:11:40.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.011+0000 7fd96ba00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd964072860 0x7fd9641495c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:40.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.011+0000 7fd96ba00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9641468b0 0x7fd9640b44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:40.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.011+0000 7fd96ba00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd964164170 0x7fd96414c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:40.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.011+0000 7fd96ba00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd96406e6a0 con 0x7fd964072860 2024-05-11T12:11:40.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.011+0000 7fd96ba00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd96406e3c0 con 0x7fd9641468b0 2024-05-11T12:11:40.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.011+0000 7fd96ba00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd964146e20 con 0x7fd964164170 2024-05-11T12:11:40.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.011+0000 7fd969000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd964072860 0x7fd9641495c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:40.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.011+0000 7fd969000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd964072860 0x7fd9641495c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:53646/0 (socket says 172.21.15.167:53646) 2024-05-11T12:11:40.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.011+0000 7fd969000640 1 -- 172.21.15.167:0/4199259348 learned_addr learned my addr 172.21.15.167:0/4199259348 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:40.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.011+0000 7fd969000640 1 -- 172.21.15.167:0/4199259348 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd964164170 msgr2=0x7fd96414c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:40.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.011+0000 7fd969000640 1 --2- 172.21.15.167:0/4199259348 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd964164170 0x7fd96414c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:40.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.011+0000 7fd969000640 1 -- 172.21.15.167:0/4199259348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9641468b0 msgr2=0x7fd9640b44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:40.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.011+0000 7fd969000640 1 --2- 172.21.15.167:0/4199259348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9641468b0 0x7fd9640b44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:40.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.011+0000 7fd969000640 1 -- 172.21.15.167:0/4199259348 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9640b49e0 con 0x7fd964072860 2024-05-11T12:11:40.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.012+0000 7fd969000640 1 --2- 172.21.15.167:0/4199259348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd964072860 0x7fd9641495c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd95800b420 tx=0x7fd95800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:40.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.012+0000 7fd960c00640 1 -- 172.21.15.167:0/4199259348 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd958011830 con 0x7fd964072860 2024-05-11T12:11:40.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.012+0000 7fd960c00640 1 -- 172.21.15.167:0/4199259348 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd958016070 con 0x7fd964072860 2024-05-11T12:11:40.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.012+0000 7fd960c00640 1 -- 172.21.15.167:0/4199259348 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd958011c10 con 0x7fd964072860 2024-05-11T12:11:40.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.012+0000 7fd96ba00640 1 -- 172.21.15.167:0/4199259348 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd964149cf0 con 0x7fd964072860 2024-05-11T12:11:40.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.012+0000 7fd96ba00640 1 -- 172.21.15.167:0/4199259348 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd96414ba00 con 0x7fd964072860 2024-05-11T12:11:40.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.013+0000 7fd96ba00640 1 -- 172.21.15.167:0/4199259348 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9640694b0 con 0x7fd964072860 2024-05-11T12:11:40.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.014+0000 7fd960c00640 1 -- 172.21.15.167:0/4199259348 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd95801b460 con 0x7fd964072860 2024-05-11T12:11:40.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.015+0000 7fd960c00640 1 --2- 172.21.15.167:0/4199259348 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd9440762d0 0x7fd944078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:40.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.016+0000 7fd960c00640 1 -- 172.21.15.167:0/4199259348 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6735+0+0 (secure 0 0 0) 0x7fd9580974f0 con 0x7fd964072860 2024-05-11T12:11:40.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.016+0000 7fd963e00640 1 --2- 172.21.15.167:0/4199259348 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd9440762d0 0x7fd944078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:40.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.017+0000 7fd963e00640 1 --2- 172.21.15.167:0/4199259348 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd9440762d0 0x7fd944078790 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd950034000 tx=0x7fd95002c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:40.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.021+0000 7fd960c00640 1 -- 172.21.15.167:0/4199259348 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd958020020 con 0x7fd964072860 2024-05-11T12:11:40.245 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.246+0000 7fd96ba00640 1 -- 172.21.15.167:0/4199259348 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd964165c90 con 0x7fd964072860 2024-05-11T12:11:40.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.247+0000 7fd960c00640 1 -- 172.21.15.167:0/4199259348 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) ==== 80+0+0 (secure 0 0 0) 0x7fd958060a40 con 0x7fd964072860 2024-05-11T12:11:40.246 INFO:teuthology.orchestra.run.smithi167.stderr:listed 0 entries 2024-05-11T12:11:40.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.249+0000 7fd96ba00640 1 -- 172.21.15.167:0/4199259348 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd9440762d0 msgr2=0x7fd944078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:40.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.250+0000 7fd96ba00640 1 --2- 172.21.15.167:0/4199259348 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd9440762d0 0x7fd944078790 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd950034000 tx=0x7fd95002c000 comp rx=0 tx=0).stop 2024-05-11T12:11:40.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.250+0000 7fd96ba00640 1 -- 172.21.15.167:0/4199259348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd964072860 msgr2=0x7fd9641495c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:40.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.250+0000 7fd96ba00640 1 --2- 172.21.15.167:0/4199259348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd964072860 0x7fd9641495c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd95800b420 tx=0x7fd95800b8e0 comp rx=0 tx=0).stop 2024-05-11T12:11:40.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.250+0000 7fd96ba00640 1 -- 172.21.15.167:0/4199259348 shutdown_connections 2024-05-11T12:11:40.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.250+0000 7fd96ba00640 1 --2- 172.21.15.167:0/4199259348 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd964164170 0x7fd96414c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:40.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.250+0000 7fd96ba00640 1 --2- 172.21.15.167:0/4199259348 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd9440762d0 0x7fd944078790 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:40.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.250+0000 7fd96ba00640 1 --2- 172.21.15.167:0/4199259348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9641468b0 0x7fd9640b44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:40.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.250+0000 7fd96ba00640 1 --2- 172.21.15.167:0/4199259348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd964072860 0x7fd9641495c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:40.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.250+0000 7fd96ba00640 1 -- 172.21.15.167:0/4199259348 >> 172.21.15.167:0/4199259348 conn(0x7fd96406c450 msgr2=0x7fd964161850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:40.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.250+0000 7fd96ba00640 1 -- 172.21.15.167:0/4199259348 shutdown_connections 2024-05-11T12:11:40.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.250+0000 7fd96ba00640 1 -- 172.21.15.167:0/4199259348 wait complete. 2024-05-11T12:11:40.267 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-11T12:11:40.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:40.267 DEBUG:teuthology.orchestra.run.smithi204:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-05-11T12:11:40.366 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi204.front.sepia.ceph.com 2024-05-11T12:11:40.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:40.367 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-11T12:11:40.431 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:11:40.431 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:11:40.488 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:11:40.488 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:11:40.543 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:11:40.543 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:11:40.594 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-11T12:11:40.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.799+0000 7fbe2e600640 1 Processor -- start 2024-05-11T12:11:40.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.799+0000 7fbe2e600640 1 -- start start 2024-05-11T12:11:40.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.801+0000 7fbe2e600640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbe2815ac80 0x7fbe28074260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:40.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.801+0000 7fbe2e600640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe280b65e0 con 0x7fbe280b3d50 2024-05-11T12:11:40.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.801+0000 7fbe2e600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe280b68c0 con 0x7fbe28161380 2024-05-11T12:11:40.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.801+0000 7fbe2e600640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fbe2806e3c0 con 0x7fbe2815ac80 2024-05-11T12:11:40.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.801+0000 7fbe27400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbe2815ac80 0x7fbe28074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:40.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.801+0000 7fbe26a00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fbe280b3d50 0x7fbe28160c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:50898/0 (socket says 172.21.15.167:50898) 2024-05-11T12:11:40.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.801+0000 7fbe26a00640 1 -- 172.21.15.167:0/1310074278 learned_addr learned my addr 172.21.15.167:0/1310074278 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:40.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.802+0000 7fbe27400640 1 -- 172.21.15.167:0/1310074278 >> v1:172.21.15.204:6789/0 conn(0x7fbe28161380 legacy=0x7fbe28072950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:40.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.802+0000 7fbe27400640 1 -- 172.21.15.167:0/1310074278 >> v1:172.21.15.167:6789/0 conn(0x7fbe280b3d50 legacy=0x7fbe28160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:40.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.802+0000 7fbe27400640 1 -- 172.21.15.167:0/1310074278 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe280b6ba0 con 0x7fbe2815ac80 2024-05-11T12:11:40.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.802+0000 7fbe26000640 1 -- 172.21.15.167:0/1310074278 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1764978971 0 0) 0x7fbe280b65e0 con 0x7fbe280b3d50 2024-05-11T12:11:40.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.803+0000 7fbe27400640 1 --2- 172.21.15.167:0/1310074278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe2815ac80 0x7fbe28074260 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbe180097d0 tx=0x7fbe1802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e587aed8c7bb394 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:40.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.804+0000 7fbe26000640 1 -- 172.21.15.167:0/1310074278 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe18037070 con 0x7fbe2815ac80 2024-05-11T12:11:40.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.804+0000 7fbe26000640 1 -- 172.21.15.167:0/1310074278 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe18032440 con 0x7fbe2815ac80 2024-05-11T12:11:40.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.804+0000 7fbe26000640 1 -- 172.21.15.167:0/1310074278 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe18032810 con 0x7fbe2815ac80 2024-05-11T12:11:40.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.804+0000 7fbe2e600640 1 -- 172.21.15.167:0/1310074278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe2815ac80 msgr2=0x7fbe28074260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:40.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.804+0000 7fbe2e600640 1 --2- 172.21.15.167:0/1310074278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe2815ac80 0x7fbe28074260 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbe180097d0 tx=0x7fbe1802a1c0 comp rx=0 tx=0).stop 2024-05-11T12:11:40.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.804+0000 7fbe2e600640 1 -- 172.21.15.167:0/1310074278 shutdown_connections 2024-05-11T12:11:40.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.804+0000 7fbe2e600640 1 --2- 172.21.15.167:0/1310074278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe2815ac80 0x7fbe28074260 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:40.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.804+0000 7fbe2e600640 1 -- 172.21.15.167:0/1310074278 >> 172.21.15.167:0/1310074278 conn(0x7fbe2806c450 msgr2=0x7fbe2806c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:40.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.804+0000 7fbe2e600640 1 -- 172.21.15.167:0/1310074278 shutdown_connections 2024-05-11T12:11:40.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.804+0000 7fbe2e600640 1 -- 172.21.15.167:0/1310074278 wait complete. 2024-05-11T12:11:40.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.806+0000 7fbe2e600640 1 Processor -- start 2024-05-11T12:11:40.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.806+0000 7fbe2e600640 1 -- start start 2024-05-11T12:11:40.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.807+0000 7fbe2e600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe280b3d50 0x7fbe2814a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:40.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.807+0000 7fbe2e600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe2815ac80 0x7fbe28157bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:40.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.807+0000 7fbe2e600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe28161380 0x7fbe281f4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:40.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.807+0000 7fbe2e600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbe280b44f0 con 0x7fbe28161380 2024-05-11T12:11:40.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.807+0000 7fbe2e600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2815b060 con 0x7fbe2815ac80 2024-05-11T12:11:40.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.807+0000 7fbe2e600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbe28073060 con 0x7fbe280b3d50 2024-05-11T12:11:40.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.808+0000 7fbe27e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe28161380 0x7fbe281f4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:40.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.808+0000 7fbe27e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe28161380 0x7fbe281f4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:53656/0 (socket says 172.21.15.167:53656) 2024-05-11T12:11:40.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.808+0000 7fbe27e00640 1 -- 172.21.15.167:0/2516872759 learned_addr learned my addr 172.21.15.167:0/2516872759 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:40.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.808+0000 7fbe26a00640 1 --2- 172.21.15.167:0/2516872759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe2815ac80 0x7fbe28157bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:40.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.808+0000 7fbe27400640 1 --2- 172.21.15.167:0/2516872759 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe280b3d50 0x7fbe2814a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:40.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.808+0000 7fbe27e00640 1 -- 172.21.15.167:0/2516872759 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe280b3d50 msgr2=0x7fbe2814a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:40.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.808+0000 7fbe27e00640 1 --2- 172.21.15.167:0/2516872759 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe280b3d50 0x7fbe2814a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:40.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.808+0000 7fbe27e00640 1 -- 172.21.15.167:0/2516872759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe2815ac80 msgr2=0x7fbe28157bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:40.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.808+0000 7fbe27e00640 1 --2- 172.21.15.167:0/2516872759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe2815ac80 0x7fbe28157bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:40.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.808+0000 7fbe27e00640 1 -- 172.21.15.167:0/2516872759 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe28156b30 con 0x7fbe28161380 2024-05-11T12:11:40.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.808+0000 7fbe26a00640 1 --2- 172.21.15.167:0/2516872759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe2815ac80 0x7fbe28157bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:40.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.808+0000 7fbe27e00640 1 --2- 172.21.15.167:0/2516872759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe28161380 0x7fbe281f4f30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbe2000ba10 tx=0x7fbe2000bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:40.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.809+0000 7fbe1f400640 1 -- 172.21.15.167:0/2516872759 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe20019070 con 0x7fbe28161380 2024-05-11T12:11:40.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.809+0000 7fbe1f400640 1 -- 172.21.15.167:0/2516872759 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe2000cb40 con 0x7fbe28161380 2024-05-11T12:11:40.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.809+0000 7fbe1f400640 1 -- 172.21.15.167:0/2516872759 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe20014440 con 0x7fbe28161380 2024-05-11T12:11:40.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.809+0000 7fbe2e600640 1 -- 172.21.15.167:0/2516872759 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe28156e10 con 0x7fbe28161380 2024-05-11T12:11:40.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.809+0000 7fbe2e600640 1 -- 172.21.15.167:0/2516872759 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe281573d0 con 0x7fbe28161380 2024-05-11T12:11:40.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.810+0000 7fbe2e600640 1 -- 172.21.15.167:0/2516872759 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe280694b0 con 0x7fbe28161380 2024-05-11T12:11:40.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.811+0000 7fbe1f400640 1 -- 172.21.15.167:0/2516872759 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbe2001c070 con 0x7fbe28161380 2024-05-11T12:11:40.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.813+0000 7fbe1f400640 1 --2- 172.21.15.167:0/2516872759 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbe08076300 0x7fbe080787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:40.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.813+0000 7fbe1f400640 1 -- 172.21.15.167:0/2516872759 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6735+0+0 (secure 0 0 0) 0x7fbe20097610 con 0x7fbe28161380 2024-05-11T12:11:40.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.813+0000 7fbe27400640 1 --2- 172.21.15.167:0/2516872759 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbe08076300 0x7fbe080787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:40.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.814+0000 7fbe27400640 1 --2- 172.21.15.167:0/2516872759 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbe08076300 0x7fbe080787c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbe18034000 tx=0x7fbe1802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:40.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:40.817+0000 7fbe1f400640 1 -- 172.21.15.167:0/2516872759 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fbe20060ad0 con 0x7fbe28161380 2024-05-11T12:11:41.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.037+0000 7fbe2e600640 1 -- 172.21.15.167:0/2516872759 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbe28069ca0 con 0x7fbe28161380 2024-05-11T12:11:41.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.038+0000 7fbe1f400640 1 -- 172.21.15.167:0/2516872759 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) ==== 80+0+0 (secure 0 0 0) 0x7fbe200603d0 con 0x7fbe28161380 2024-05-11T12:11:41.037 INFO:teuthology.orchestra.run.smithi167.stderr:listed 0 entries 2024-05-11T12:11:41.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.041+0000 7fbe2e600640 1 -- 172.21.15.167:0/2516872759 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbe08076300 msgr2=0x7fbe080787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:41.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.041+0000 7fbe2e600640 1 --2- 172.21.15.167:0/2516872759 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbe08076300 0x7fbe080787c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbe18034000 tx=0x7fbe1802e000 comp rx=0 tx=0).stop 2024-05-11T12:11:41.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.041+0000 7fbe2e600640 1 -- 172.21.15.167:0/2516872759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe28161380 msgr2=0x7fbe281f4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:41.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.041+0000 7fbe2e600640 1 --2- 172.21.15.167:0/2516872759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe28161380 0x7fbe281f4f30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbe2000ba10 tx=0x7fbe2000bed0 comp rx=0 tx=0).stop 2024-05-11T12:11:41.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.041+0000 7fbe2e600640 1 -- 172.21.15.167:0/2516872759 shutdown_connections 2024-05-11T12:11:41.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.041+0000 7fbe2e600640 1 --2- 172.21.15.167:0/2516872759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe28161380 0x7fbe281f4f30 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:41.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.041+0000 7fbe2e600640 1 --2- 172.21.15.167:0/2516872759 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbe08076300 0x7fbe080787c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:41.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.041+0000 7fbe2e600640 1 --2- 172.21.15.167:0/2516872759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe2815ac80 0x7fbe28157bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:41.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.041+0000 7fbe2e600640 1 --2- 172.21.15.167:0/2516872759 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe280b3d50 0x7fbe2814a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:41.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.041+0000 7fbe2e600640 1 -- 172.21.15.167:0/2516872759 >> 172.21.15.167:0/2516872759 conn(0x7fbe2806c450 msgr2=0x7fbe280b5a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:41.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.041+0000 7fbe2e600640 1 -- 172.21.15.167:0/2516872759 shutdown_connections 2024-05-11T12:11:41.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.041+0000 7fbe2e600640 1 -- 172.21.15.167:0/2516872759 wait complete. 2024-05-11T12:11:41.059 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-05-11T12:11:41.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:41.060 DEBUG:teuthology.orchestra.run.smithi167:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-05-11T12:11:41.143 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2024-05-11T12:11:41.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:41.143 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-11T12:11:41.194 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:11:41.194 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:11:41.251 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:11:41.251 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:11:41.304 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:11:41.305 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:11:41.356 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-11T12:11:41.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.571+0000 7fcf5f200640 1 Processor -- start 2024-05-11T12:11:41.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.572+0000 7fcf5f200640 1 -- start start 2024-05-11T12:11:41.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.573+0000 7fcf5f200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fcf600d0f60 0x7fcf600d3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:41.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.573+0000 7fcf5f200640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf600d13f0 con 0x7fcf600d4140 2024-05-11T12:11:41.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.573+0000 7fcf5f200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf600d45b0 con 0x7fcf601625d0 2024-05-11T12:11:41.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.573+0000 7fcf5f200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fcf600d2e40 con 0x7fcf600d0f60 2024-05-11T12:11:41.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.573+0000 7fcf5e800640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fcf600d4140 0x7fcf6006cef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:50902/0 (socket says 172.21.15.167:50902) 2024-05-11T12:11:41.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.573+0000 7fcf5e800640 1 -- 172.21.15.167:0/472194238 learned_addr learned my addr 172.21.15.167:0/472194238 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:41.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.574+0000 7fcf5d400640 1 --2- 172.21.15.167:0/472194238 >> v2:172.21.15.204:3300/0 conn(0x7fcf600d0f60 0x7fcf600d3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:41.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.574+0000 7fcf5ca00640 1 -- 172.21.15.167:0/472194238 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 196645403 0 0) 0x7fcf600d13f0 con 0x7fcf600d4140 2024-05-11T12:11:41.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.574+0000 7fcf5ca00640 1 -- 172.21.15.167:0/472194238 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf44003520 con 0x7fcf600d4140 2024-05-11T12:11:41.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.574+0000 7fcf5ca00640 1 -- 172.21.15.167:0/472194238 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4250778414 0 0) 0x7fcf44003520 con 0x7fcf600d4140 2024-05-11T12:11:41.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.574+0000 7fcf5ca00640 1 -- 172.21.15.167:0/472194238 >> v2:172.21.15.204:3300/0 conn(0x7fcf600d0f60 msgr2=0x7fcf600d3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:41.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.574+0000 7fcf5ca00640 1 --2- 172.21.15.167:0/472194238 >> v2:172.21.15.204:3300/0 conn(0x7fcf600d0f60 0x7fcf600d3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:41.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.574+0000 7fcf5ca00640 1 -- 172.21.15.167:0/472194238 >> v1:172.21.15.204:6789/0 conn(0x7fcf601625d0 legacy=0x7fcf600b8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:41.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.574+0000 7fcf5ca00640 1 -- 172.21.15.167:0/472194238 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf600d3120 con 0x7fcf600d4140 2024-05-11T12:11:41.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.574+0000 7fcf5ca00640 1 -- 172.21.15.167:0/472194238 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7fcf58001a90 con 0x7fcf600d4140 2024-05-11T12:11:41.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.574+0000 7fcf5ca00640 1 -- 172.21.15.167:0/472194238 >> v1:172.21.15.167:6789/0 conn(0x7fcf600d4140 legacy=0x7fcf6006cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:41.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.574+0000 7fcf5ca00640 1 --2- 172.21.15.167:0/472194238 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcf44003b10 0x7fcf44003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:41.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.574+0000 7fcf5ca00640 1 --2- 172.21.15.167:0/472194238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf44004800 0x7fcf44004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:41.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.574+0000 7fcf5ca00640 1 --2- 172.21.15.167:0/472194238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf440053c0 0x7fcf4400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:41.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.574+0000 7fcf5ca00640 1 -- 172.21.15.167:0/472194238 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcf600d2e40 con 0x7fcf44004800 2024-05-11T12:11:41.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.574+0000 7fcf5ca00640 1 -- 172.21.15.167:0/472194238 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4400c260 con 0x7fcf440053c0 2024-05-11T12:11:41.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.574+0000 7fcf5ca00640 1 -- 172.21.15.167:0/472194238 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fcf4400c460 con 0x7fcf44003b10 2024-05-11T12:11:41.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.575+0000 7fcf5d400640 1 --2- 172.21.15.167:0/472194238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf44004800 0x7fcf44004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:41.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.575+0000 7fcf5d400640 1 -- 172.21.15.167:0/472194238 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcf44003b10 msgr2=0x7fcf44003f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:41.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.575+0000 7fcf5de00640 1 --2- 172.21.15.167:0/472194238 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcf44003b10 0x7fcf44003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:41.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.575+0000 7fcf5d400640 1 --2- 172.21.15.167:0/472194238 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcf44003b10 0x7fcf44003f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:41.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.575+0000 7fcf5e800640 1 --2- 172.21.15.167:0/472194238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf440053c0 0x7fcf4400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:41.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.575+0000 7fcf5d400640 1 -- 172.21.15.167:0/472194238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf440053c0 msgr2=0x7fcf4400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:41.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.575+0000 7fcf5d400640 1 --2- 172.21.15.167:0/472194238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf440053c0 0x7fcf4400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:41.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.575+0000 7fcf5d400640 1 -- 172.21.15.167:0/472194238 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf44003520 con 0x7fcf44004800 2024-05-11T12:11:41.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.575+0000 7fcf5e800640 1 -- 172.21.15.167:0/472194238 reap_dead start 2024-05-11T12:11:41.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.576+0000 7fcf5d400640 1 --2- 172.21.15.167:0/472194238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf44004800 0x7fcf44004c80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fcf4800ba30 tx=0x7fcf44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540d91ee3d9a9742 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:41.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.576+0000 7fcf5ca00640 1 -- 172.21.15.167:0/472194238 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf48030070 con 0x7fcf44004800 2024-05-11T12:11:41.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.576+0000 7fcf5ca00640 1 -- 172.21.15.167:0/472194238 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf4802ade0 con 0x7fcf44004800 2024-05-11T12:11:41.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.576+0000 7fcf5ca00640 1 -- 172.21.15.167:0/472194238 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4800b600 con 0x7fcf44004800 2024-05-11T12:11:41.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.577+0000 7fcf5f200640 1 -- 172.21.15.167:0/472194238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf44004800 msgr2=0x7fcf44004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:41.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.577+0000 7fcf5f200640 1 --2- 172.21.15.167:0/472194238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf44004800 0x7fcf44004c80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fcf4800ba30 tx=0x7fcf44005830 comp rx=0 tx=0).stop 2024-05-11T12:11:41.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.577+0000 7fcf5f200640 1 -- 172.21.15.167:0/472194238 shutdown_connections 2024-05-11T12:11:41.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.577+0000 7fcf5f200640 1 --2- 172.21.15.167:0/472194238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf44004800 0x7fcf44004c80 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:41.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.577+0000 7fcf5f200640 1 -- 172.21.15.167:0/472194238 >> 172.21.15.167:0/472194238 conn(0x7fcf600d50f0 msgr2=0x7fcf600d5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:41.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.577+0000 7fcf5f200640 1 -- 172.21.15.167:0/472194238 shutdown_connections 2024-05-11T12:11:41.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.577+0000 7fcf5f200640 1 -- 172.21.15.167:0/472194238 wait complete. 2024-05-11T12:11:41.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.579+0000 7fcf5f200640 1 Processor -- start 2024-05-11T12:11:41.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.579+0000 7fcf5f200640 1 -- start start 2024-05-11T12:11:41.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.580+0000 7fcf5f200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf44004800 0x7fcf600cc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:41.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.580+0000 7fcf5f200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcf601600b0 0x7fcf601f8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:41.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.580+0000 7fcf5f200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf601f87c0 0x7fcf601f4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:41.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.580+0000 7fcf5f200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601629b0 con 0x7fcf44004800 2024-05-11T12:11:41.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.580+0000 7fcf5f200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcf600d36e0 con 0x7fcf601f87c0 2024-05-11T12:11:41.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.580+0000 7fcf5f200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fcf60162b30 con 0x7fcf601600b0 2024-05-11T12:11:41.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.580+0000 7fcf5de00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf44004800 0x7fcf600cc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:41.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.580+0000 7fcf5de00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf44004800 0x7fcf600cc1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:53672/0 (socket says 172.21.15.167:53672) 2024-05-11T12:11:41.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.580+0000 7fcf5de00640 1 -- 172.21.15.167:0/1785455615 learned_addr learned my addr 172.21.15.167:0/1785455615 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:41.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.581+0000 7fcf5e800640 1 --2- 172.21.15.167:0/1785455615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf601f87c0 0x7fcf601f4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:41.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.581+0000 7fcf5d400640 1 --2- 172.21.15.167:0/1785455615 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcf601600b0 0x7fcf601f8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:41.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.581+0000 7fcf5de00640 1 -- 172.21.15.167:0/1785455615 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcf601600b0 msgr2=0x7fcf601f8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:41.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.581+0000 7fcf5de00640 1 --2- 172.21.15.167:0/1785455615 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcf601600b0 0x7fcf601f8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:41.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.581+0000 7fcf5de00640 1 -- 172.21.15.167:0/1785455615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf601f87c0 msgr2=0x7fcf601f4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:41.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.581+0000 7fcf5de00640 1 --2- 172.21.15.167:0/1785455615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf601f87c0 0x7fcf601f4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:41.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.581+0000 7fcf5de00640 1 -- 172.21.15.167:0/1785455615 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf601592f0 con 0x7fcf44004800 2024-05-11T12:11:41.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.581+0000 7fcf5d400640 1 --2- 172.21.15.167:0/1785455615 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcf601600b0 0x7fcf601f8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:41.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.581+0000 7fcf5de00640 1 --2- 172.21.15.167:0/1785455615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf44004800 0x7fcf600cc1c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcf5000d7d0 tx=0x7fcf5000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:41.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.581+0000 7fcf56000640 1 -- 172.21.15.167:0/1785455615 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf50005270 con 0x7fcf44004800 2024-05-11T12:11:41.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.582+0000 7fcf56000640 1 -- 172.21.15.167:0/1785455615 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf50017070 con 0x7fcf44004800 2024-05-11T12:11:41.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.582+0000 7fcf5f200640 1 -- 172.21.15.167:0/1785455615 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf601595d0 con 0x7fcf44004800 2024-05-11T12:11:41.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.582+0000 7fcf56000640 1 -- 172.21.15.167:0/1785455615 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5000b6b0 con 0x7fcf44004800 2024-05-11T12:11:41.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.582+0000 7fcf5f200640 1 -- 172.21.15.167:0/1785455615 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf60159b90 con 0x7fcf44004800 2024-05-11T12:11:41.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.583+0000 7fcf5f200640 1 -- 172.21.15.167:0/1785455615 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf600694b0 con 0x7fcf44004800 2024-05-11T12:11:41.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.584+0000 7fcf56000640 1 -- 172.21.15.167:0/1785455615 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fcf5000b870 con 0x7fcf44004800 2024-05-11T12:11:41.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.585+0000 7fcf56000640 1 --2- 172.21.15.167:0/1785455615 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fcf3c0762d0 0x7fcf3c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:41.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.585+0000 7fcf56000640 1 -- 172.21.15.167:0/1785455615 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6735+0+0 (secure 0 0 0) 0x7fcf500978b0 con 0x7fcf44004800 2024-05-11T12:11:41.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.585+0000 7fcf5d400640 1 --2- 172.21.15.167:0/1785455615 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fcf3c0762d0 0x7fcf3c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:41.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.586+0000 7fcf5d400640 1 --2- 172.21.15.167:0/1785455615 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fcf3c0762d0 0x7fcf3c078790 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fcf4800ba00 tx=0x7fcf48030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:41.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.589+0000 7fcf56000640 1 -- 172.21.15.167:0/1785455615 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fcf50060e00 con 0x7fcf44004800 2024-05-11T12:11:41.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.814+0000 7fcf5f200640 1 -- 172.21.15.167:0/1785455615 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fcf601f8ba0 con 0x7fcf44004800 2024-05-11T12:11:41.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.815+0000 7fcf56000640 1 -- 172.21.15.167:0/1785455615 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) ==== 80+0+0 (secure 0 0 0) 0x7fcf50001230 con 0x7fcf44004800 2024-05-11T12:11:41.815 INFO:teuthology.orchestra.run.smithi167.stderr:listed 0 entries 2024-05-11T12:11:41.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.818+0000 7fcf5f200640 1 -- 172.21.15.167:0/1785455615 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fcf3c0762d0 msgr2=0x7fcf3c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:41.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.818+0000 7fcf5f200640 1 --2- 172.21.15.167:0/1785455615 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fcf3c0762d0 0x7fcf3c078790 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fcf4800ba00 tx=0x7fcf48030000 comp rx=0 tx=0).stop 2024-05-11T12:11:41.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.818+0000 7fcf5f200640 1 -- 172.21.15.167:0/1785455615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf44004800 msgr2=0x7fcf600cc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:41.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.818+0000 7fcf5f200640 1 --2- 172.21.15.167:0/1785455615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf44004800 0x7fcf600cc1c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcf5000d7d0 tx=0x7fcf5000dc90 comp rx=0 tx=0).stop 2024-05-11T12:11:41.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.818+0000 7fcf5f200640 1 -- 172.21.15.167:0/1785455615 shutdown_connections 2024-05-11T12:11:41.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.818+0000 7fcf5f200640 1 --2- 172.21.15.167:0/1785455615 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fcf3c0762d0 0x7fcf3c078790 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:41.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.818+0000 7fcf5f200640 1 --2- 172.21.15.167:0/1785455615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf601f87c0 0x7fcf601f4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:41.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.818+0000 7fcf5f200640 1 --2- 172.21.15.167:0/1785455615 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcf601600b0 0x7fcf601f8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:41.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.818+0000 7fcf5f200640 1 --2- 172.21.15.167:0/1785455615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf44004800 0x7fcf600cc1c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:41.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.818+0000 7fcf5f200640 1 -- 172.21.15.167:0/1785455615 >> 172.21.15.167:0/1785455615 conn(0x7fcf600d50f0 msgr2=0x7fcf600b4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:41.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.819+0000 7fcf5f200640 1 -- 172.21.15.167:0/1785455615 shutdown_connections 2024-05-11T12:11:41.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:41.819+0000 7fcf5f200640 1 -- 172.21.15.167:0/1785455615 wait complete. 2024-05-11T12:11:41.837 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-05-11T12:11:41.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:41.837 DEBUG:teuthology.orchestra.run.smithi204:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-05-11T12:11:41.926 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi204.front.sepia.ceph.com 2024-05-11T12:11:41.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:11:41.927 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-11T12:11:41.976 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-11T12:11:41.977 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:11:42.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.188+0000 7f0a3ee00640 1 Processor -- start 2024-05-11T12:11:42.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.188+0000 7f0a3ee00640 1 -- start start 2024-05-11T12:11:42.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.189+0000 7f0a3ee00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0a380af5d0 0x7f0a38073ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:42.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.190+0000 7f0a3ee00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a3806fe10 con 0x7f0a380af1f0 2024-05-11T12:11:42.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.190+0000 7f0a3ee00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a3815fcc0 con 0x7f0a3806d460 2024-05-11T12:11:42.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.190+0000 7f0a3ee00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f0a381608d0 con 0x7f0a380af5d0 2024-05-11T12:11:42.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.190+0000 7f0a37e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f0a380af1f0 0x7f0a380cf1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:50910/0 (socket says 172.21.15.167:50910) 2024-05-11T12:11:42.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.190+0000 7f0a37e00640 1 -- 172.21.15.167:0/2410423415 learned_addr learned my addr 172.21.15.167:0/2410423415 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:42.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.190+0000 7f0a2f400640 1 --2- 172.21.15.167:0/2410423415 >> v2:172.21.15.167:3300/0 conn(0x7f0a380af5d0 0x7f0a38073ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:42.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.190+0000 7f0a37400640 1 -- 172.21.15.167:0/2410423415 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3531366799 0 0) 0x7f0a3806fe10 con 0x7f0a380af1f0 2024-05-11T12:11:42.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.190+0000 7f0a37400640 1 -- 172.21.15.167:0/2410423415 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0a1c003520 con 0x7f0a380af1f0 2024-05-11T12:11:42.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.191+0000 7f0a2f400640 1 -- 172.21.15.167:0/2410423415 >> v1:172.21.15.204:6789/0 conn(0x7f0a3806d460 legacy=0x7f0a3806e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:42.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.191+0000 7f0a2f400640 1 -- 172.21.15.167:0/2410423415 >> v1:172.21.15.167:6789/0 conn(0x7f0a380af1f0 legacy=0x7f0a380cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:42.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.191+0000 7f0a2f400640 1 -- 172.21.15.167:0/2410423415 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a3815ffa0 con 0x7f0a380af5d0 2024-05-11T12:11:42.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.192+0000 7f0a2f400640 1 --2- 172.21.15.167:0/2410423415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a380af5d0 0x7f0a38073ce0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0a200097a0 tx=0x7f0a2002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e2b162d84cf7b50 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:42.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.192+0000 7f0a37400640 1 -- 172.21.15.167:0/2410423415 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a20037070 con 0x7f0a380af5d0 2024-05-11T12:11:42.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.192+0000 7f0a37400640 1 -- 172.21.15.167:0/2410423415 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a2002ae50 con 0x7f0a380af5d0 2024-05-11T12:11:42.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.192+0000 7f0a37400640 1 -- 172.21.15.167:0/2410423415 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a20032660 con 0x7f0a380af5d0 2024-05-11T12:11:42.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.192+0000 7f0a3ee00640 1 -- 172.21.15.167:0/2410423415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a380af5d0 msgr2=0x7f0a38073ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:42.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.192+0000 7f0a3ee00640 1 --2- 172.21.15.167:0/2410423415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a380af5d0 0x7f0a38073ce0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0a200097a0 tx=0x7f0a2002a100 comp rx=0 tx=0).stop 2024-05-11T12:11:42.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.193+0000 7f0a3ee00640 1 -- 172.21.15.167:0/2410423415 shutdown_connections 2024-05-11T12:11:42.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.193+0000 7f0a3ee00640 1 --2- 172.21.15.167:0/2410423415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a380af5d0 0x7f0a38073ce0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:42.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.193+0000 7f0a3ee00640 1 -- 172.21.15.167:0/2410423415 >> 172.21.15.167:0/2410423415 conn(0x7f0a380cd8f0 msgr2=0x7f0a380cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:42.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.193+0000 7f0a3ee00640 1 -- 172.21.15.167:0/2410423415 shutdown_connections 2024-05-11T12:11:42.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.193+0000 7f0a3ee00640 1 -- 172.21.15.167:0/2410423415 wait complete. 2024-05-11T12:11:42.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.194+0000 7f0a3ee00640 1 Processor -- start 2024-05-11T12:11:42.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.194+0000 7f0a3ee00640 1 -- start start 2024-05-11T12:11:42.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.195+0000 7f0a3ee00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a3806d460 0x7f0a380b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:42.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.195+0000 7f0a3ee00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a380af1f0 0x7f0a38072bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:42.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.195+0000 7f0a3ee00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a380af5d0 0x7f0a381553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:42.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.195+0000 7f0a3ee00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3815f920 con 0x7f0a3806d460 2024-05-11T12:11:42.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.195+0000 7f0a3ee00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0a38160560 con 0x7f0a380af5d0 2024-05-11T12:11:42.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.195+0000 7f0a3ee00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0a3815faa0 con 0x7f0a380af1f0 2024-05-11T12:11:42.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.196+0000 7f0a37e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a3806d460 0x7f0a380b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:42.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.196+0000 7f0a37e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a3806d460 0x7f0a380b54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:53688/0 (socket says 172.21.15.167:53688) 2024-05-11T12:11:42.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.196+0000 7f0a37e00640 1 -- 172.21.15.167:0/1951097622 learned_addr learned my addr 172.21.15.167:0/1951097622 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:42.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.196+0000 7f0a37e00640 1 -- 172.21.15.167:0/1951097622 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a380af1f0 msgr2=0x7f0a38072bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:42.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.196+0000 7f0a37e00640 1 --2- 172.21.15.167:0/1951097622 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a380af1f0 0x7f0a38072bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:42.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.196+0000 7f0a37e00640 1 -- 172.21.15.167:0/1951097622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a380af5d0 msgr2=0x7f0a381553d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:42.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.196+0000 7f0a37e00640 1 --2- 172.21.15.167:0/1951097622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a380af5d0 0x7f0a381553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:42.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.196+0000 7f0a37e00640 1 -- 172.21.15.167:0/1951097622 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a381f4fb0 con 0x7f0a3806d460 2024-05-11T12:11:42.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.196+0000 7f0a37e00640 1 --2- 172.21.15.167:0/1951097622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a3806d460 0x7f0a380b54c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0a28001eb0 tx=0x7f0a2800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:42.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.197+0000 7f0a34c00640 1 -- 172.21.15.167:0/1951097622 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a28017070 con 0x7f0a3806d460 2024-05-11T12:11:42.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.197+0000 7f0a34c00640 1 -- 172.21.15.167:0/1951097622 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a28005ce0 con 0x7f0a3806d460 2024-05-11T12:11:42.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.197+0000 7f0a3ee00640 1 -- 172.21.15.167:0/1951097622 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a381f5290 con 0x7f0a3806d460 2024-05-11T12:11:42.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.197+0000 7f0a34c00640 1 -- 172.21.15.167:0/1951097622 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a28007270 con 0x7f0a3806d460 2024-05-11T12:11:42.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.197+0000 7f0a3ee00640 1 -- 172.21.15.167:0/1951097622 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a381f5850 con 0x7f0a3806d460 2024-05-11T12:11:42.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.198+0000 7f0a3ee00640 1 -- 172.21.15.167:0/1951097622 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a38069590 con 0x7f0a3806d460 2024-05-11T12:11:42.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.199+0000 7f0a34c00640 1 -- 172.21.15.167:0/1951097622 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0a2801a070 con 0x7f0a3806d460 2024-05-11T12:11:42.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.200+0000 7f0a34c00640 1 --2- 172.21.15.167:0/1951097622 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0a140762d0 0x7f0a14078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:42.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.200+0000 7f0a34c00640 1 -- 172.21.15.167:0/1951097622 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6735+0+0 (secure 0 0 0) 0x7f0a28096be0 con 0x7f0a3806d460 2024-05-11T12:11:42.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.200+0000 7f0a2f400640 1 --2- 172.21.15.167:0/1951097622 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0a140762d0 0x7f0a14078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:42.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.201+0000 7f0a2f400640 1 --2- 172.21.15.167:0/1951097622 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0a140762d0 0x7f0a14078790 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0a20034000 tx=0x7f0a2002c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:42.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.204+0000 7f0a34c00640 1 -- 172.21.15.167:0/1951097622 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0a2805faf0 con 0x7f0a3806d460 2024-05-11T12:11:42.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.452+0000 7f0a3ee00640 1 -- 172.21.15.167:0/1951097622 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a3806dae0 con 0x7f0a3806d460 2024-05-11T12:11:42.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.454+0000 7f0a34c00640 1 -- 172.21.15.167:0/1951097622 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5189 (secure 0 0 0) 0x7f0a28063df0 con 0x7f0a3806d460 2024-05-11T12:11:42.453 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:11:42.453 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3127277486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3127277486},{"type":"v1","addr":"172.21.15.167:6835","nonce":3127277486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2547133600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2547133600},{"type":"v1","addr":"172.21.15.167:6837","nonce":2547133600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3200960100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3200960100},{"type":"v1","addr":"172.21.15.204:6834","nonce":3200960100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:10:45.225745+0000","modified":"2024-05-11T12:11:09.405446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/1889842375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":1889842375},{"type":"v1","addr":"172.21.15.204:6835","nonce":1889842375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-05-11T12:11:42.453 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 9 2024-05-11T12:11:42.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.456+0000 7f0a3ee00640 1 -- 172.21.15.167:0/1951097622 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0a140762d0 msgr2=0x7f0a14078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:42.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.457+0000 7f0a3ee00640 1 --2- 172.21.15.167:0/1951097622 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0a140762d0 0x7f0a14078790 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0a20034000 tx=0x7f0a2002c000 comp rx=0 tx=0).stop 2024-05-11T12:11:42.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.457+0000 7f0a3ee00640 1 -- 172.21.15.167:0/1951097622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a3806d460 msgr2=0x7f0a380b54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:42.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.457+0000 7f0a3ee00640 1 --2- 172.21.15.167:0/1951097622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a3806d460 0x7f0a380b54c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0a28001eb0 tx=0x7f0a2800c960 comp rx=0 tx=0).stop 2024-05-11T12:11:42.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.457+0000 7f0a3ee00640 1 -- 172.21.15.167:0/1951097622 shutdown_connections 2024-05-11T12:11:42.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.457+0000 7f0a3ee00640 1 --2- 172.21.15.167:0/1951097622 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0a140762d0 0x7f0a14078790 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:42.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.457+0000 7f0a3ee00640 1 --2- 172.21.15.167:0/1951097622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a380af5d0 0x7f0a381553d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:42.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.457+0000 7f0a3ee00640 1 --2- 172.21.15.167:0/1951097622 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a380af1f0 0x7f0a38072bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:42.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.457+0000 7f0a3ee00640 1 --2- 172.21.15.167:0/1951097622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a3806d460 0x7f0a380b54c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:42.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.457+0000 7f0a3ee00640 1 -- 172.21.15.167:0/1951097622 >> 172.21.15.167:0/1951097622 conn(0x7f0a380cd8f0 msgr2=0x7f0a3806e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:42.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.457+0000 7f0a3ee00640 1 -- 172.21.15.167:0/1951097622 shutdown_connections 2024-05-11T12:11:42.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.457+0000 7f0a3ee00640 1 -- 172.21.15.167:0/1951097622 wait complete. 2024-05-11T12:11:42.475 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:11:42.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.676+0000 7f799dc00640 1 Processor -- start 2024-05-11T12:11:42.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.677+0000 7f799dc00640 1 -- start start 2024-05-11T12:11:42.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.678+0000 7f799dc00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f7998146770 0x7f79980ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:42.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.678+0000 7f799dc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79980cef10 0x7f799806c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:42.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.678+0000 7f799dc00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f799806fbc0 con 0x7f7998145b00 2024-05-11T12:11:42.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.678+0000 7f799dc00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f79980b4520 con 0x7f7998146770 2024-05-11T12:11:42.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.678+0000 7f799dc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f79980b4800 con 0x7f79980cef10 2024-05-11T12:11:42.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.679+0000 7f799d200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79980cef10 0x7f799806c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:42.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.679+0000 7f799d200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79980cef10 0x7f799806c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:57770/0 (socket says 172.21.15.167:57770) 2024-05-11T12:11:42.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.679+0000 7f799d200640 1 -- 172.21.15.167:0/3640430850 learned_addr learned my addr 172.21.15.167:0/3640430850 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:42.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.679+0000 7f798f400640 1 --2- 172.21.15.167:0/3640430850 >> v2:172.21.15.204:3300/0 conn(0x7f7998146770 0x7f79980ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:42.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.680+0000 7f799d200640 1 -- 172.21.15.167:0/3640430850 >> v2:172.21.15.204:3300/0 conn(0x7f7998146770 msgr2=0x7f79980ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:42.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.680+0000 7f799d200640 1 --2- 172.21.15.167:0/3640430850 >> v2:172.21.15.204:3300/0 conn(0x7f7998146770 0x7f79980ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:42.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.680+0000 7f799d200640 1 -- 172.21.15.167:0/3640430850 >> v1:172.21.15.204:6789/0 conn(0x7f7998145b00 legacy=0x7f799806f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:42.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.680+0000 7f799d200640 1 -- 172.21.15.167:0/3640430850 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79980b4ae0 con 0x7f79980cef10 2024-05-11T12:11:42.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.680+0000 7f7997400640 1 -- 172.21.15.167:0/3640430850 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2790027051 0 0) 0x7f799806fbc0 con 0x7f7998145b00 2024-05-11T12:11:42.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.681+0000 7f799d200640 1 --2- 172.21.15.167:0/3640430850 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79980cef10 0x7f799806c660 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7990009e50 tx=0x7f799002a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d013a1a2316cafa server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:42.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.682+0000 7f7997400640 1 -- 172.21.15.167:0/3640430850 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7990036070 con 0x7f79980cef10 2024-05-11T12:11:42.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.682+0000 7f7997400640 1 -- 172.21.15.167:0/3640430850 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7990031600 con 0x7f79980cef10 2024-05-11T12:11:42.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.682+0000 7f7997400640 1 -- 172.21.15.167:0/3640430850 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79900319d0 con 0x7f79980cef10 2024-05-11T12:11:42.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.682+0000 7f799dc00640 1 -- 172.21.15.167:0/3640430850 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79980cef10 msgr2=0x7f799806c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:42.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.682+0000 7f799dc00640 1 --2- 172.21.15.167:0/3640430850 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79980cef10 0x7f799806c660 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7990009e50 tx=0x7f799002a680 comp rx=0 tx=0).stop 2024-05-11T12:11:42.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.683+0000 7f799dc00640 1 -- 172.21.15.167:0/3640430850 shutdown_connections 2024-05-11T12:11:42.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.683+0000 7f799dc00640 1 --2- 172.21.15.167:0/3640430850 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79980cef10 0x7f799806c660 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:42.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.683+0000 7f799dc00640 1 --2- 172.21.15.167:0/3640430850 >> v2:172.21.15.204:3300/0 conn(0x7f7998146770 0x7f79980ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:42.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.683+0000 7f799dc00640 1 -- 172.21.15.167:0/3640430850 >> 172.21.15.167:0/3640430850 conn(0x7f79980b3460 msgr2=0x7f79980d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:42.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.683+0000 7f799dc00640 1 -- 172.21.15.167:0/3640430850 shutdown_connections 2024-05-11T12:11:42.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.683+0000 7f799dc00640 1 -- 172.21.15.167:0/3640430850 wait complete. 2024-05-11T12:11:42.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.684+0000 7f799dc00640 1 Processor -- start 2024-05-11T12:11:42.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.685+0000 7f799dc00640 1 -- start start 2024-05-11T12:11:42.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.686+0000 7f799dc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79980cef10 0x7f79981f4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:42.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.686+0000 7f799dc00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7998145b00 0x7f79980ae820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:42.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.686+0000 7f799dc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7998146770 0x7f79980aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:42.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.686+0000 7f799dc00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f799806cd20 con 0x7f7998145b00 2024-05-11T12:11:42.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.686+0000 7f799dc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f799806cba0 con 0x7f7998146770 2024-05-11T12:11:42.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.686+0000 7f799dc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f799806cea0 con 0x7f79980cef10 2024-05-11T12:11:42.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.686+0000 7f798f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7998145b00 0x7f79980ae820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:42.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.686+0000 7f798f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7998145b00 0x7f79980ae820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:53690/0 (socket says 172.21.15.167:53690) 2024-05-11T12:11:42.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.686+0000 7f798f400640 1 -- 172.21.15.167:0/2835896887 learned_addr learned my addr 172.21.15.167:0/2835896887 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:42.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.686+0000 7f799d200640 1 --2- 172.21.15.167:0/2835896887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7998146770 0x7f79980aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:42.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.686+0000 7f7997e00640 1 --2- 172.21.15.167:0/2835896887 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79980cef10 0x7f79981f4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:42.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.686+0000 7f798f400640 1 -- 172.21.15.167:0/2835896887 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79980cef10 msgr2=0x7f79981f4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:42.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.686+0000 7f798f400640 1 --2- 172.21.15.167:0/2835896887 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79980cef10 0x7f79981f4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:42.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.686+0000 7f798f400640 1 -- 172.21.15.167:0/2835896887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7998146770 msgr2=0x7f79980aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:42.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.686+0000 7f798f400640 1 --2- 172.21.15.167:0/2835896887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7998146770 0x7f79980aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:42.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.687+0000 7f798f400640 1 -- 172.21.15.167:0/2835896887 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79980b0430 con 0x7f7998145b00 2024-05-11T12:11:42.687 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.687+0000 7f798f400640 1 --2- 172.21.15.167:0/2835896887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7998145b00 0x7f79980ae820 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f798000e9e0 tx=0x7f798000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:42.687 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.687+0000 7f7994c00640 1 -- 172.21.15.167:0/2835896887 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7980017070 con 0x7f7998145b00 2024-05-11T12:11:42.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.687+0000 7f7994c00640 1 -- 172.21.15.167:0/2835896887 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f798000c9d0 con 0x7f7998145b00 2024-05-11T12:11:42.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.687+0000 7f799dc00640 1 -- 172.21.15.167:0/2835896887 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79980af390 con 0x7f7998145b00 2024-05-11T12:11:42.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.687+0000 7f7994c00640 1 -- 172.21.15.167:0/2835896887 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798000cdc0 con 0x7f7998145b00 2024-05-11T12:11:42.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.687+0000 7f799dc00640 1 -- 172.21.15.167:0/2835896887 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79980af950 con 0x7f7998145b00 2024-05-11T12:11:42.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.689+0000 7f799dc00640 1 -- 172.21.15.167:0/2835896887 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79980694b0 con 0x7f7998145b00 2024-05-11T12:11:42.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.690+0000 7f7994c00640 1 -- 172.21.15.167:0/2835896887 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f798001a070 con 0x7f7998145b00 2024-05-11T12:11:42.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.691+0000 7f7994c00640 1 --2- 172.21.15.167:0/2835896887 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f79740762d0 0x7f7974078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:42.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.691+0000 7f7994c00640 1 -- 172.21.15.167:0/2835896887 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6735+0+0 (secure 0 0 0) 0x7f7980097e50 con 0x7f7998145b00 2024-05-11T12:11:42.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.691+0000 7f7997e00640 1 --2- 172.21.15.167:0/2835896887 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f79740762d0 0x7f7974078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:42.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.692+0000 7f7997e00640 1 --2- 172.21.15.167:0/2835896887 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f79740762d0 0x7f7974078790 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7988000f60 tx=0x7f7988009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:42.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.695+0000 7f7994c00640 1 -- 172.21.15.167:0/2835896887 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7980061310 con 0x7f7998145b00 2024-05-11T12:11:42.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.953+0000 7f799dc00640 1 -- 172.21.15.167:0/2835896887 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7998072240 con 0x7f7998145b00 2024-05-11T12:11:42.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.954+0000 7f7994c00640 1 -- 172.21.15.167:0/2835896887 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5189 (secure 0 0 0) 0x7f7980060c10 con 0x7f7998145b00 2024-05-11T12:11:42.953 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:11:42.953 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3127277486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3127277486},{"type":"v1","addr":"172.21.15.167:6835","nonce":3127277486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2547133600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2547133600},{"type":"v1","addr":"172.21.15.167:6837","nonce":2547133600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3200960100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3200960100},{"type":"v1","addr":"172.21.15.204:6834","nonce":3200960100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:10:45.225745+0000","modified":"2024-05-11T12:11:09.405446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/1889842375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":1889842375},{"type":"v1","addr":"172.21.15.204:6835","nonce":1889842375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-05-11T12:11:42.953 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 9 2024-05-11T12:11:42.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.956+0000 7f799dc00640 1 -- 172.21.15.167:0/2835896887 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f79740762d0 msgr2=0x7f7974078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:42.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.956+0000 7f799dc00640 1 --2- 172.21.15.167:0/2835896887 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f79740762d0 0x7f7974078790 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7988000f60 tx=0x7f7988009040 comp rx=0 tx=0).stop 2024-05-11T12:11:42.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.956+0000 7f799dc00640 1 -- 172.21.15.167:0/2835896887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7998145b00 msgr2=0x7f79980ae820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:42.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.956+0000 7f799dc00640 1 --2- 172.21.15.167:0/2835896887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7998145b00 0x7f79980ae820 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f798000e9e0 tx=0x7f798000eea0 comp rx=0 tx=0).stop 2024-05-11T12:11:42.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.956+0000 7f799dc00640 1 -- 172.21.15.167:0/2835896887 shutdown_connections 2024-05-11T12:11:42.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.956+0000 7f799dc00640 1 --2- 172.21.15.167:0/2835896887 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f79740762d0 0x7f7974078790 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:42.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.956+0000 7f799dc00640 1 --2- 172.21.15.167:0/2835896887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7998146770 0x7f79980aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:42.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.956+0000 7f799dc00640 1 --2- 172.21.15.167:0/2835896887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7998145b00 0x7f79980ae820 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:42.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.956+0000 7f799dc00640 1 --2- 172.21.15.167:0/2835896887 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79980cef10 0x7f79981f4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:42.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.956+0000 7f799dc00640 1 -- 172.21.15.167:0/2835896887 >> 172.21.15.167:0/2835896887 conn(0x7f79980b3460 msgr2=0x7f79980d01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:42.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.957+0000 7f799dc00640 1 -- 172.21.15.167:0/2835896887 shutdown_connections 2024-05-11T12:11:42.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:42.957+0000 7f799dc00640 1 -- 172.21.15.167:0/2835896887 wait complete. 2024-05-11T12:11:42.975 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:11:43.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.169+0000 7fcff7a00640 1 Processor -- start 2024-05-11T12:11:43.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.169+0000 7fcff7a00640 1 -- start start 2024-05-11T12:11:43.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.171+0000 7fcff7a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcff0157750 0x7fcff015d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:43.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.171+0000 7fcff7a00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fcff006d460 0x7fcff006d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:43.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.171+0000 7fcff7a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcff015dc80 con 0x7fcff00b3ea0 2024-05-11T12:11:43.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.171+0000 7fcff7a00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fcff0161000 con 0x7fcff006d460 2024-05-11T12:11:43.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.171+0000 7fcff7a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fcff015df30 con 0x7fcff0157750 2024-05-11T12:11:43.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.171+0000 7fcff5a00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fcff006d460 0x7fcff006d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:43.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.171+0000 7fcff5a00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fcff006d460 0x7fcff006d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:53704/0 (socket says 172.21.15.167:53704) 2024-05-11T12:11:43.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.171+0000 7fcff5a00640 1 -- 172.21.15.167:0/3988750594 learned_addr learned my addr 172.21.15.167:0/3988750594 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:43.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.171+0000 7fcff5000640 1 --2- 172.21.15.167:0/3988750594 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcff0157750 0x7fcff015d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:43.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.172+0000 7fcff5a00640 1 -- 172.21.15.167:0/3988750594 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcff0157750 msgr2=0x7fcff015d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:43.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.172+0000 7fcff5a00640 1 --2- 172.21.15.167:0/3988750594 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcff0157750 0x7fcff015d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:43.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.172+0000 7fcff5a00640 1 -- 172.21.15.167:0/3988750594 >> v1:172.21.15.204:6789/0 conn(0x7fcff00b3ea0 legacy=0x7fcff0161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:43.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.172+0000 7fcff5a00640 1 -- 172.21.15.167:0/3988750594 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcff015e210 con 0x7fcff006d460 2024-05-11T12:11:43.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.173+0000 7fcff5a00640 1 --2- 172.21.15.167:0/3988750594 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcff006d460 0x7fcff006d8f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8009e50 tx=0x7fcfe802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cb7414bb363337d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:43.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.173+0000 7fcfef400640 1 -- 172.21.15.167:0/3988750594 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe8036070 con 0x7fcff006d460 2024-05-11T12:11:43.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.173+0000 7fcfef400640 1 -- 172.21.15.167:0/3988750594 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfe8031600 con 0x7fcff006d460 2024-05-11T12:11:43.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.173+0000 7fcfef400640 1 -- 172.21.15.167:0/3988750594 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe80319d0 con 0x7fcff006d460 2024-05-11T12:11:43.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.174+0000 7fcff7a00640 1 -- 172.21.15.167:0/3988750594 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcff006d460 msgr2=0x7fcff006d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:43.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.174+0000 7fcff7a00640 1 --2- 172.21.15.167:0/3988750594 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcff006d460 0x7fcff006d8f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8009e50 tx=0x7fcfe802a680 comp rx=0 tx=0).stop 2024-05-11T12:11:43.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.174+0000 7fcff7a00640 1 -- 172.21.15.167:0/3988750594 shutdown_connections 2024-05-11T12:11:43.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.174+0000 7fcff7a00640 1 --2- 172.21.15.167:0/3988750594 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcff006d460 0x7fcff006d8f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:43.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.174+0000 7fcff7a00640 1 --2- 172.21.15.167:0/3988750594 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcff0157750 0x7fcff015d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:43.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.174+0000 7fcff7a00640 1 -- 172.21.15.167:0/3988750594 >> 172.21.15.167:0/3988750594 conn(0x7fcff00cd8f0 msgr2=0x7fcff00cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:43.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.174+0000 7fcff7a00640 1 -- 172.21.15.167:0/3988750594 shutdown_connections 2024-05-11T12:11:43.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.174+0000 7fcff7a00640 1 -- 172.21.15.167:0/3988750594 wait complete. 2024-05-11T12:11:43.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.176+0000 7fcff7a00640 1 Processor -- start 2024-05-11T12:11:43.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.176+0000 7fcff7a00640 1 -- start start 2024-05-11T12:11:43.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.177+0000 7fcff7a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcff006d460 0x7fcff0073240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:43.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.177+0000 7fcff7a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcff00b3ea0 0x7fcff013b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:43.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.177+0000 7fcff7a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcff0157750 0x7fcff01591f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:43.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.177+0000 7fcff7a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcff0161ba0 con 0x7fcff00b3ea0 2024-05-11T12:11:43.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.177+0000 7fcff7a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcff0157b30 con 0x7fcff006d460 2024-05-11T12:11:43.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.177+0000 7fcff7a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fcff00cf1c0 con 0x7fcff0157750 2024-05-11T12:11:43.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.178+0000 7fcfefe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcff00b3ea0 0x7fcff013b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:43.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.178+0000 7fcfefe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcff00b3ea0 0x7fcff013b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:53706/0 (socket says 172.21.15.167:53706) 2024-05-11T12:11:43.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.178+0000 7fcfefe00640 1 -- 172.21.15.167:0/3104752221 learned_addr learned my addr 172.21.15.167:0/3104752221 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:43.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.178+0000 7fcff5a00640 1 --2- 172.21.15.167:0/3104752221 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcff0157750 0x7fcff01591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:43.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.178+0000 7fcff5000640 1 --2- 172.21.15.167:0/3104752221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcff006d460 0x7fcff0073240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:43.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.178+0000 7fcfefe00640 1 -- 172.21.15.167:0/3104752221 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcff0157750 msgr2=0x7fcff01591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:43.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.178+0000 7fcfefe00640 1 --2- 172.21.15.167:0/3104752221 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcff0157750 0x7fcff01591f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:43.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.178+0000 7fcfefe00640 1 -- 172.21.15.167:0/3104752221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcff006d460 msgr2=0x7fcff0073240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:43.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.178+0000 7fcfefe00640 1 --2- 172.21.15.167:0/3104752221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcff006d460 0x7fcff0073240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:43.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.178+0000 7fcfefe00640 1 -- 172.21.15.167:0/3104752221 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcff013bd50 con 0x7fcff00b3ea0 2024-05-11T12:11:43.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.178+0000 7fcff5000640 1 --2- 172.21.15.167:0/3104752221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcff006d460 0x7fcff0073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:43.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.178+0000 7fcfefe00640 1 --2- 172.21.15.167:0/3104752221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcff00b3ea0 0x7fcff013b810 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc00c430 tx=0x7fcfdc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:43.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.179+0000 7fcfecc00640 1 -- 172.21.15.167:0/3104752221 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc011790 con 0x7fcff00b3ea0 2024-05-11T12:11:43.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.179+0000 7fcfecc00640 1 -- 172.21.15.167:0/3104752221 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfdc017070 con 0x7fcff00b3ea0 2024-05-11T12:11:43.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.179+0000 7fcfecc00640 1 -- 172.21.15.167:0/3104752221 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc011bb0 con 0x7fcff00b3ea0 2024-05-11T12:11:43.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.179+0000 7fcff7a00640 1 -- 172.21.15.167:0/3104752221 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcff0159ad0 con 0x7fcff00b3ea0 2024-05-11T12:11:43.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.179+0000 7fcff7a00640 1 -- 172.21.15.167:0/3104752221 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcff015a090 con 0x7fcff00b3ea0 2024-05-11T12:11:43.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.180+0000 7fcff7a00640 1 -- 172.21.15.167:0/3104752221 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcff00b29c0 con 0x7fcff00b3ea0 2024-05-11T12:11:43.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.181+0000 7fcfecc00640 1 -- 172.21.15.167:0/3104752221 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fcfdc01c460 con 0x7fcff00b3ea0 2024-05-11T12:11:43.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.182+0000 7fcfecc00640 1 --2- 172.21.15.167:0/3104752221 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fcfd00762d0 0x7fcfd0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:43.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.182+0000 7fcfecc00640 1 -- 172.21.15.167:0/3104752221 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6735+0+0 (secure 0 0 0) 0x7fcfdc097890 con 0x7fcff00b3ea0 2024-05-11T12:11:43.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.183+0000 7fcff5000640 1 --2- 172.21.15.167:0/3104752221 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fcfd00762d0 0x7fcfd0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:43.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.183+0000 7fcff5000640 1 --2- 172.21.15.167:0/3104752221 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fcfd00762d0 0x7fcfd0078790 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4006fd0 tx=0x7fcfe4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:43.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.187+0000 7fcfecc00640 1 -- 172.21.15.167:0/3104752221 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fcff00b29c0 con 0x7fcff00b3ea0 2024-05-11T12:11:43.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.394+0000 7fcff7a00640 1 -- 172.21.15.167:0/3104752221 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcff00b26e0 con 0x7fcff00b3ea0 2024-05-11T12:11:43.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.397+0000 7fcfecc00640 1 -- 172.21.15.167:0/3104752221 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18950 (secure 0 0 0) 0x7fcfdc060de0 con 0x7fcff00b3ea0 2024-05-11T12:11:43.397 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:11:43.397 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":29,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:11:27.038079+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:10:43.084511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T12:10:44.157782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-11T12:10:47.814674+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:11:43.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.400+0000 7fcff7a00640 1 -- 172.21.15.167:0/3104752221 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fcfd00762d0 msgr2=0x7fcfd0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:43.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.400+0000 7fcff7a00640 1 --2- 172.21.15.167:0/3104752221 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fcfd00762d0 0x7fcfd0078790 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4006fd0 tx=0x7fcfe4008040 comp rx=0 tx=0).stop 2024-05-11T12:11:43.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.400+0000 7fcff7a00640 1 -- 172.21.15.167:0/3104752221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcff00b3ea0 msgr2=0x7fcff013b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:43.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.400+0000 7fcff7a00640 1 --2- 172.21.15.167:0/3104752221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcff00b3ea0 0x7fcff013b810 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc00c430 tx=0x7fcfdc00c8f0 comp rx=0 tx=0).stop 2024-05-11T12:11:43.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.400+0000 7fcff7a00640 1 -- 172.21.15.167:0/3104752221 shutdown_connections 2024-05-11T12:11:43.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.401+0000 7fcff7a00640 1 --2- 172.21.15.167:0/3104752221 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcff0157750 0x7fcff01591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:43.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.401+0000 7fcff7a00640 1 --2- 172.21.15.167:0/3104752221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcff00b3ea0 0x7fcff013b810 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:43.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.401+0000 7fcff7a00640 1 --2- 172.21.15.167:0/3104752221 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fcfd00762d0 0x7fcfd0078790 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:43.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.401+0000 7fcff7a00640 1 --2- 172.21.15.167:0/3104752221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcff006d460 0x7fcff0073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:43.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.401+0000 7fcff7a00640 1 -- 172.21.15.167:0/3104752221 >> 172.21.15.167:0/3104752221 conn(0x7fcff00cd8f0 msgr2=0x7fcff0161490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:43.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.401+0000 7fcff7a00640 1 -- 172.21.15.167:0/3104752221 shutdown_connections 2024-05-11T12:11:43.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.401+0000 7fcff7a00640 1 -- 172.21.15.167:0/3104752221 wait complete. 2024-05-11T12:11:43.420 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-11T12:11:43.420 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:11:43.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.625+0000 7f66c7e00640 1 Processor -- start 2024-05-11T12:11:43.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.625+0000 7f66c7e00640 1 -- start start 2024-05-11T12:11:43.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.626+0000 7f66c7e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f66c81458f0 0x7f66c806d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:43.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.626+0000 7f66c7e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66c80b3a80 0x7f66c80b4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:43.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.627+0000 7f66c7e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f66c80b5220 0x7f66c80b56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:43.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.627+0000 7f66c7e00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f66c80b5bf0 con 0x7f66c80b5220 2024-05-11T12:11:43.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.627+0000 7f66c7e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f66c806db80 con 0x7f66c81458f0 2024-05-11T12:11:43.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.627+0000 7f66c7e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f66c8071f30 con 0x7f66c80b3a80 2024-05-11T12:11:43.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.627+0000 7f66c7400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f66c80b5220 0x7f66c80b56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:43.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.627+0000 7f66c7400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f66c80b5220 0x7f66c80b56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:53714/0 (socket says 172.21.15.167:53714) 2024-05-11T12:11:43.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.627+0000 7f66c7400640 1 -- 172.21.15.167:0/2407151729 learned_addr learned my addr 172.21.15.167:0/2407151729 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:43.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.627+0000 7f66bfe00640 1 --2- 172.21.15.167:0/2407151729 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66c80b3a80 0x7f66c80b4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:43.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.627+0000 7f66c6a00640 1 --2- 172.21.15.167:0/2407151729 >> v2:172.21.15.204:3300/0 conn(0x7f66c81458f0 0x7f66c806d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:43.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.627+0000 7f66c7400640 1 -- 172.21.15.167:0/2407151729 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66c80b3a80 msgr2=0x7f66c80b4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:43.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.627+0000 7f66c7400640 1 --2- 172.21.15.167:0/2407151729 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66c80b3a80 0x7f66c80b4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:43.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.627+0000 7f66c7400640 1 -- 172.21.15.167:0/2407151729 >> v2:172.21.15.204:3300/0 conn(0x7f66c81458f0 msgr2=0x7f66c806d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:43.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.627+0000 7f66c7400640 1 --2- 172.21.15.167:0/2407151729 >> v2:172.21.15.204:3300/0 conn(0x7f66c81458f0 0x7f66c806d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:43.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.627+0000 7f66c7400640 1 -- 172.21.15.167:0/2407151729 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66c8072210 con 0x7f66c80b5220 2024-05-11T12:11:43.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.628+0000 7f66c7400640 1 --2- 172.21.15.167:0/2407151729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66c80b5220 0x7f66c80b56b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f66c80d0980 tx=0x7f66c002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e4489afb280aba server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:43.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.629+0000 7f66c6000640 1 -- 172.21.15.167:0/2407151729 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c0036070 con 0x7f66c80b5220 2024-05-11T12:11:43.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.629+0000 7f66c6000640 1 -- 172.21.15.167:0/2407151729 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66c0031600 con 0x7f66c80b5220 2024-05-11T12:11:43.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.629+0000 7f66c6000640 1 -- 172.21.15.167:0/2407151729 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c00319d0 con 0x7f66c80b5220 2024-05-11T12:11:43.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.629+0000 7f66c7e00640 1 -- 172.21.15.167:0/2407151729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66c80b5220 msgr2=0x7f66c80b56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:43.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.629+0000 7f66c7e00640 1 --2- 172.21.15.167:0/2407151729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66c80b5220 0x7f66c80b56b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f66c80d0980 tx=0x7f66c002a6a0 comp rx=0 tx=0).stop 2024-05-11T12:11:43.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.629+0000 7f66c7e00640 1 -- 172.21.15.167:0/2407151729 shutdown_connections 2024-05-11T12:11:43.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.629+0000 7f66c7e00640 1 --2- 172.21.15.167:0/2407151729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66c80b5220 0x7f66c80b56b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:43.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.629+0000 7f66c7e00640 1 --2- 172.21.15.167:0/2407151729 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66c80b3a80 0x7f66c80b4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:43.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.629+0000 7f66c7e00640 1 --2- 172.21.15.167:0/2407151729 >> v2:172.21.15.204:3300/0 conn(0x7f66c81458f0 0x7f66c806d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:43.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.630+0000 7f66c7e00640 1 -- 172.21.15.167:0/2407151729 >> 172.21.15.167:0/2407151729 conn(0x7f66c80b3460 msgr2=0x7f66c80d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:43.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.630+0000 7f66c7e00640 1 -- 172.21.15.167:0/2407151729 shutdown_connections 2024-05-11T12:11:43.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.630+0000 7f66c7e00640 1 -- 172.21.15.167:0/2407151729 wait complete. 2024-05-11T12:11:43.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.631+0000 7f66c7e00640 1 Processor -- start 2024-05-11T12:11:43.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.631+0000 7f66c7e00640 1 -- start start 2024-05-11T12:11:43.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.632+0000 7f66c7e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66c80b3a80 0x7f66c815bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:43.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.632+0000 7f66c7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66c80b5220 0x7f66c813d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:43.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.632+0000 7f66c7e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66c81458f0 0x7f66c8143b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:43.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.632+0000 7f66c7e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f66c8072ab0 con 0x7f66c80b5220 2024-05-11T12:11:43.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.632+0000 7f66c7e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f66c80727d0 con 0x7f66c81458f0 2024-05-11T12:11:43.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.632+0000 7f66c7e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f66c80b4eb0 con 0x7f66c80b3a80 2024-05-11T12:11:43.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.633+0000 7f66bfe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66c80b5220 0x7f66c813d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:43.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.633+0000 7f66bfe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66c80b5220 0x7f66c813d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:53730/0 (socket says 172.21.15.167:53730) 2024-05-11T12:11:43.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.633+0000 7f66bfe00640 1 -- 172.21.15.167:0/2640855950 learned_addr learned my addr 172.21.15.167:0/2640855950 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:43.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.633+0000 7f66c6a00640 1 --2- 172.21.15.167:0/2640855950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66c80b3a80 0x7f66c815bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:43.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.633+0000 7f66bfe00640 1 -- 172.21.15.167:0/2640855950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66c80b3a80 msgr2=0x7f66c815bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:43.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.633+0000 7f66bfe00640 1 --2- 172.21.15.167:0/2640855950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66c80b3a80 0x7f66c815bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:43.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.633+0000 7f66bfe00640 1 -- 172.21.15.167:0/2640855950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66c81458f0 msgr2=0x7f66c8143b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:43.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.633+0000 7f66bfe00640 1 --2- 172.21.15.167:0/2640855950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66c81458f0 0x7f66c8143b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:43.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.633+0000 7f66bfe00640 1 -- 172.21.15.167:0/2640855950 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66c815c070 con 0x7f66c80b5220 2024-05-11T12:11:43.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.633+0000 7f66bfe00640 1 --2- 172.21.15.167:0/2640855950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66c80b5220 0x7f66c813d110 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f66b000d800 tx=0x7f66b000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:43.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.634+0000 7f66bea00640 1 -- 172.21.15.167:0/2640855950 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b0018020 con 0x7f66c80b5220 2024-05-11T12:11:43.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.634+0000 7f66bea00640 1 -- 172.21.15.167:0/2640855950 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66b0009640 con 0x7f66c80b5220 2024-05-11T12:11:43.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.634+0000 7f66bea00640 1 -- 172.21.15.167:0/2640855950 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b0009a10 con 0x7f66c80b5220 2024-05-11T12:11:43.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.634+0000 7f66c7e00640 1 -- 172.21.15.167:0/2640855950 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66c813a950 con 0x7f66c80b5220 2024-05-11T12:11:43.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.634+0000 7f66c7e00640 1 -- 172.21.15.167:0/2640855950 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66c815cb80 con 0x7f66c80b5220 2024-05-11T12:11:43.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.635+0000 7f66c7e00640 1 -- 172.21.15.167:0/2640855950 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66c80694b0 con 0x7f66c80b5220 2024-05-11T12:11:43.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.636+0000 7f66bea00640 1 -- 172.21.15.167:0/2640855950 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f66b001b020 con 0x7f66c80b5220 2024-05-11T12:11:43.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.637+0000 7f66bea00640 1 --2- 172.21.15.167:0/2640855950 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f66a40762d0 0x7f66a4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:43.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.637+0000 7f66bea00640 1 -- 172.21.15.167:0/2640855950 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6735+0+0 (secure 0 0 0) 0x7f66b0097c50 con 0x7f66c80b5220 2024-05-11T12:11:43.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.637+0000 7f66c6a00640 1 --2- 172.21.15.167:0/2640855950 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f66a40762d0 0x7f66a4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:43.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.638+0000 7f66c6a00640 1 --2- 172.21.15.167:0/2640855950 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f66a40762d0 0x7f66a4078790 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f66b8006fd0 tx=0x7f66b8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:43.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.642+0000 7f66bea00640 1 -- 172.21.15.167:0/2640855950 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f66b0022030 con 0x7f66c80b5220 2024-05-11T12:11:43.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.888+0000 7f66c7e00640 1 -- 172.21.15.167:0/2640855950 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66c80ceb80 con 0x7f66c80b5220 2024-05-11T12:11:43.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.889+0000 7f66bea00640 1 -- 172.21.15.167:0/2640855950 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5189 (secure 0 0 0) 0x7f66b0061110 con 0x7f66c80b5220 2024-05-11T12:11:43.889 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:11:43.889 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3127277486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3127277486},{"type":"v1","addr":"172.21.15.167:6835","nonce":3127277486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2547133600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2547133600},{"type":"v1","addr":"172.21.15.167:6837","nonce":2547133600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3200960100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3200960100},{"type":"v1","addr":"172.21.15.204:6834","nonce":3200960100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:10:45.225745+0000","modified":"2024-05-11T12:11:09.405446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/1889842375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":1889842375},{"type":"v1","addr":"172.21.15.204:6835","nonce":1889842375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-05-11T12:11:43.889 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 9 2024-05-11T12:11:43.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.892+0000 7f66c7e00640 1 -- 172.21.15.167:0/2640855950 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f66a40762d0 msgr2=0x7f66a4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:43.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.892+0000 7f66c7e00640 1 --2- 172.21.15.167:0/2640855950 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f66a40762d0 0x7f66a4078790 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f66b8006fd0 tx=0x7f66b8008040 comp rx=0 tx=0).stop 2024-05-11T12:11:43.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.892+0000 7f66c7e00640 1 -- 172.21.15.167:0/2640855950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66c80b5220 msgr2=0x7f66c813d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:43.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.892+0000 7f66c7e00640 1 --2- 172.21.15.167:0/2640855950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66c80b5220 0x7f66c813d110 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f66b000d800 tx=0x7f66b000dcc0 comp rx=0 tx=0).stop 2024-05-11T12:11:43.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.892+0000 7f66c7e00640 1 -- 172.21.15.167:0/2640855950 shutdown_connections 2024-05-11T12:11:43.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.892+0000 7f66c7e00640 1 --2- 172.21.15.167:0/2640855950 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f66a40762d0 0x7f66a4078790 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:43.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.892+0000 7f66c7e00640 1 --2- 172.21.15.167:0/2640855950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66c81458f0 0x7f66c8143b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:43.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.892+0000 7f66c7e00640 1 --2- 172.21.15.167:0/2640855950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66c80b5220 0x7f66c813d110 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:43.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.892+0000 7f66c7e00640 1 --2- 172.21.15.167:0/2640855950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66c80b3a80 0x7f66c815bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:43.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.892+0000 7f66c7e00640 1 -- 172.21.15.167:0/2640855950 >> 172.21.15.167:0/2640855950 conn(0x7f66c80b3460 msgr2=0x7f66c80cbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:43.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.892+0000 7f66c7e00640 1 -- 172.21.15.167:0/2640855950 shutdown_connections 2024-05-11T12:11:43.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:43.892+0000 7f66c7e00640 1 -- 172.21.15.167:0/2640855950 wait complete. 2024-05-11T12:11:43.912 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:11:44.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.113+0000 7f4176c00640 1 Processor -- start 2024-05-11T12:11:44.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.113+0000 7f4176c00640 1 -- start start 2024-05-11T12:11:44.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.115+0000 7f4176c00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f41700afcd0 0x7f41701601f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:44.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.115+0000 7f4176c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41700d04a0 0x7f41700b55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:44.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.115+0000 7f4176c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41700b4560 con 0x7f41700cf260 2024-05-11T12:11:44.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.115+0000 7f4176c00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f41700b5130 con 0x7f41700afcd0 2024-05-11T12:11:44.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.115+0000 7f4176c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f41700b4800 con 0x7f41700d04a0 2024-05-11T12:11:44.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.115+0000 7f4175800640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f41700afcd0 0x7f41701601f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:44.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.115+0000 7f4175800640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f41700afcd0 0x7f41701601f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:53740/0 (socket says 172.21.15.167:53740) 2024-05-11T12:11:44.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.115+0000 7f4175800640 1 -- 172.21.15.167:0/1068153378 learned_addr learned my addr 172.21.15.167:0/1068153378 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:44.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.116+0000 7f4175800640 1 -- 172.21.15.167:0/1068153378 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41700d04a0 msgr2=0x7f41700b55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:44.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.116+0000 7f4175800640 1 --2- 172.21.15.167:0/1068153378 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41700d04a0 0x7f41700b55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:44.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.116+0000 7f4175800640 1 -- 172.21.15.167:0/1068153378 >> v1:172.21.15.204:6789/0 conn(0x7f41700cf260 legacy=0x7f41700721d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:44.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.116+0000 7f4175800640 1 -- 172.21.15.167:0/1068153378 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41700b4ae0 con 0x7f41700afcd0 2024-05-11T12:11:44.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.117+0000 7f4175800640 1 --2- 172.21.15.167:0/1068153378 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41700afcd0 0x7f41701601f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4164009870 tx=0x7f416402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=628d98e5722d0e87 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:44.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.117+0000 7f416fe00640 1 -- 172.21.15.167:0/1068153378 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4164037070 con 0x7f41700afcd0 2024-05-11T12:11:44.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.117+0000 7f416fe00640 1 -- 172.21.15.167:0/1068153378 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4164032440 con 0x7f41700afcd0 2024-05-11T12:11:44.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.117+0000 7f416fe00640 1 -- 172.21.15.167:0/1068153378 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4164032810 con 0x7f41700afcd0 2024-05-11T12:11:44.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.118+0000 7f4176c00640 1 -- 172.21.15.167:0/1068153378 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41700afcd0 msgr2=0x7f41701601f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:44.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.118+0000 7f4176c00640 1 --2- 172.21.15.167:0/1068153378 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41700afcd0 0x7f41701601f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4164009870 tx=0x7f416402a1c0 comp rx=0 tx=0).stop 2024-05-11T12:11:44.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.118+0000 7f4176c00640 1 -- 172.21.15.167:0/1068153378 shutdown_connections 2024-05-11T12:11:44.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.118+0000 7f4176c00640 1 --2- 172.21.15.167:0/1068153378 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41700d04a0 0x7f41700b55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:44.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.118+0000 7f4176c00640 1 --2- 172.21.15.167:0/1068153378 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41700afcd0 0x7f41701601f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:44.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.118+0000 7f4176c00640 1 -- 172.21.15.167:0/1068153378 >> 172.21.15.167:0/1068153378 conn(0x7f41700b3460 msgr2=0x7f41700d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:44.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.118+0000 7f4176c00640 1 -- 172.21.15.167:0/1068153378 shutdown_connections 2024-05-11T12:11:44.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.118+0000 7f4176c00640 1 -- 172.21.15.167:0/1068153378 wait complete. 2024-05-11T12:11:44.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.120+0000 7f4176c00640 1 Processor -- start 2024-05-11T12:11:44.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.120+0000 7f4176c00640 1 -- start start 2024-05-11T12:11:44.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.121+0000 7f4176c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41700afcd0 0x7f4170152ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:44.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.121+0000 7f4176c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41700cf260 0x7f417014e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:44.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.121+0000 7f4176c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41700d04a0 0x7f41701ea600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:44.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.121+0000 7f4176c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f41700cff40 con 0x7f41700cf260 2024-05-11T12:11:44.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.121+0000 7f4176c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f41700b00b0 con 0x7f41700d04a0 2024-05-11T12:11:44.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.121+0000 7f4176c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f41700d00c0 con 0x7f41700afcd0 2024-05-11T12:11:44.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.121+0000 7f4174e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41700cf260 0x7f417014e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:44.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.121+0000 7f4174e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41700cf260 0x7f417014e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:53746/0 (socket says 172.21.15.167:53746) 2024-05-11T12:11:44.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.121+0000 7f4174e00640 1 -- 172.21.15.167:0/2154620346 learned_addr learned my addr 172.21.15.167:0/2154620346 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:44.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.121+0000 7f4175800640 1 --2- 172.21.15.167:0/2154620346 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41700afcd0 0x7f4170152ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:44.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.121+0000 7f4176200640 1 --2- 172.21.15.167:0/2154620346 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41700d04a0 0x7f41701ea600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:44.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.121+0000 7f4174e00640 1 -- 172.21.15.167:0/2154620346 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41700afcd0 msgr2=0x7f4170152ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:44.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.121+0000 7f4174e00640 1 --2- 172.21.15.167:0/2154620346 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41700afcd0 0x7f4170152ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:44.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.121+0000 7f4174e00640 1 -- 172.21.15.167:0/2154620346 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41700d04a0 msgr2=0x7f41701ea600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:44.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.122+0000 7f4174e00640 1 --2- 172.21.15.167:0/2154620346 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41700d04a0 0x7f41701ea600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:44.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.122+0000 7f4174e00640 1 -- 172.21.15.167:0/2154620346 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f417014ecb0 con 0x7f41700cf260 2024-05-11T12:11:44.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.122+0000 7f4176200640 1 --2- 172.21.15.167:0/2154620346 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41700d04a0 0x7f41701ea600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:44.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.122+0000 7f4174e00640 1 --2- 172.21.15.167:0/2154620346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41700cf260 0x7f417014e770 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f416000b410 tx=0x7f416000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:44.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.122+0000 7f416d600640 1 -- 172.21.15.167:0/2154620346 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4160015020 con 0x7f41700cf260 2024-05-11T12:11:44.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.122+0000 7f416d600640 1 -- 172.21.15.167:0/2154620346 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4160011850 con 0x7f41700cf260 2024-05-11T12:11:44.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.122+0000 7f416d600640 1 -- 172.21.15.167:0/2154620346 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4160011c40 con 0x7f41700cf260 2024-05-11T12:11:44.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.122+0000 7f4176c00640 1 -- 172.21.15.167:0/2154620346 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4170161440 con 0x7f41700cf260 2024-05-11T12:11:44.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.122+0000 7f4176c00640 1 -- 172.21.15.167:0/2154620346 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41701eb130 con 0x7f41700cf260 2024-05-11T12:11:44.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.123+0000 7f4176c00640 1 -- 172.21.15.167:0/2154620346 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41700694b0 con 0x7f41700cf260 2024-05-11T12:11:44.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.124+0000 7f416d600640 1 -- 172.21.15.167:0/2154620346 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4160018020 con 0x7f41700cf260 2024-05-11T12:11:44.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.126+0000 7f416d600640 1 --2- 172.21.15.167:0/2154620346 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4154076300 0x7f41540787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:44.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.126+0000 7f416d600640 1 -- 172.21.15.167:0/2154620346 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6735+0+0 (secure 0 0 0) 0x7f4160097980 con 0x7f41700cf260 2024-05-11T12:11:44.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.126+0000 7f4175800640 1 --2- 172.21.15.167:0/2154620346 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4154076300 0x7f41540787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:44.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.127+0000 7f4175800640 1 --2- 172.21.15.167:0/2154620346 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4154076300 0x7f41540787c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4164034000 tx=0x7f416402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:44.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.130+0000 7f416d600640 1 -- 172.21.15.167:0/2154620346 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f416001f020 con 0x7f41700cf260 2024-05-11T12:11:44.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.370+0000 7f4176c00640 1 -- 172.21.15.167:0/2154620346 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41701593d0 con 0x7f41700cf260 2024-05-11T12:11:44.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.370+0000 7f416d600640 1 -- 172.21.15.167:0/2154620346 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5189 (secure 0 0 0) 0x7f4160060e40 con 0x7f41700cf260 2024-05-11T12:11:44.370 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:11:44.370 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3127277486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3127277486},{"type":"v1","addr":"172.21.15.167:6835","nonce":3127277486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2547133600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2547133600},{"type":"v1","addr":"172.21.15.167:6837","nonce":2547133600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3200960100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3200960100},{"type":"v1","addr":"172.21.15.204:6834","nonce":3200960100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:10:45.225745+0000","modified":"2024-05-11T12:11:09.405446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/1889842375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":1889842375},{"type":"v1","addr":"172.21.15.204:6835","nonce":1889842375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-05-11T12:11:44.370 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 9 2024-05-11T12:11:44.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.373+0000 7f4176c00640 1 -- 172.21.15.167:0/2154620346 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4154076300 msgr2=0x7f41540787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:44.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.373+0000 7f4176c00640 1 --2- 172.21.15.167:0/2154620346 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4154076300 0x7f41540787c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4164034000 tx=0x7f416402e000 comp rx=0 tx=0).stop 2024-05-11T12:11:44.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.373+0000 7f4176c00640 1 -- 172.21.15.167:0/2154620346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41700cf260 msgr2=0x7f417014e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:44.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.373+0000 7f4176c00640 1 --2- 172.21.15.167:0/2154620346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41700cf260 0x7f417014e770 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f416000b410 tx=0x7f416000b8d0 comp rx=0 tx=0).stop 2024-05-11T12:11:44.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.373+0000 7f4176c00640 1 -- 172.21.15.167:0/2154620346 shutdown_connections 2024-05-11T12:11:44.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.373+0000 7f4176c00640 1 --2- 172.21.15.167:0/2154620346 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4154076300 0x7f41540787c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:44.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.373+0000 7f4176c00640 1 --2- 172.21.15.167:0/2154620346 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41700d04a0 0x7f41701ea600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:44.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.373+0000 7f4176c00640 1 --2- 172.21.15.167:0/2154620346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41700cf260 0x7f417014e770 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:44.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.373+0000 7f4176c00640 1 --2- 172.21.15.167:0/2154620346 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41700afcd0 0x7f4170152ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:44.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.373+0000 7f4176c00640 1 -- 172.21.15.167:0/2154620346 >> 172.21.15.167:0/2154620346 conn(0x7f41700b3460 msgr2=0x7f4170072f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:44.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.374+0000 7f4176c00640 1 -- 172.21.15.167:0/2154620346 shutdown_connections 2024-05-11T12:11:44.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.374+0000 7f4176c00640 1 -- 172.21.15.167:0/2154620346 wait complete. 2024-05-11T12:11:44.390 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:11:44.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.575+0000 7f8639600640 1 Processor -- start 2024-05-11T12:11:44.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.575+0000 7f8639600640 1 -- start start 2024-05-11T12:11:44.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.577+0000 7f8639600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8634157750 0x7f863415d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:44.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.577+0000 7f8639600640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f863406d460 0x7f863406d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:44.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.577+0000 7f8639600640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f863415dc80 con 0x7f86340b3ea0 2024-05-11T12:11:44.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.577+0000 7f8639600640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f8634161000 con 0x7f863406d460 2024-05-11T12:11:44.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.577+0000 7f8639600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f863406ddd0 con 0x7f8634157750 2024-05-11T12:11:44.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.577+0000 7f8633400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f863406d460 0x7f863406d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:44.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.577+0000 7f8633400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f863406d460 0x7f863406d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:53748/0 (socket says 172.21.15.167:53748) 2024-05-11T12:11:44.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.577+0000 7f8633400640 1 -- 172.21.15.167:0/2859873929 learned_addr learned my addr 172.21.15.167:0/2859873929 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:44.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.577+0000 7f862a000640 1 --1- 172.21.15.167:0/2859873929 >> v1:172.21.15.167:6789/0 conn(0x7f86340b3ea0 0x7f8634161490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:50922/0 (socket says 172.21.15.167:50922) 2024-05-11T12:11:44.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.577+0000 7f8632a00640 1 --2- 172.21.15.167:0/2859873929 >> v2:172.21.15.204:3300/0 conn(0x7f8634157750 0x7f863415d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:44.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.577+0000 7f8633400640 1 -- 172.21.15.167:0/2859873929 >> v2:172.21.15.204:3300/0 conn(0x7f8634157750 msgr2=0x7f863415d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:44.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.578+0000 7f8633400640 1 --2- 172.21.15.167:0/2859873929 >> v2:172.21.15.204:3300/0 conn(0x7f8634157750 0x7f863415d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:44.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.578+0000 7f8633400640 1 -- 172.21.15.167:0/2859873929 >> v1:172.21.15.167:6789/0 conn(0x7f86340b3ea0 legacy=0x7f8634161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:44.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.578+0000 7f8633400640 1 -- 172.21.15.167:0/2859873929 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f863415df30 con 0x7f863406d460 2024-05-11T12:11:44.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.578+0000 7f8632000640 1 -- 172.21.15.167:0/2859873929 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 995781548 0 0) 0x7f863415dc80 con 0x7f86340b3ea0 2024-05-11T12:11:44.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.578+0000 7f8632a00640 1 --2- 172.21.15.167:0/2859873929 >> v2:172.21.15.204:3300/0 conn(0x7f8634157750 0x7f863415d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:44.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.579+0000 7f8633400640 1 --2- 172.21.15.167:0/2859873929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f863406d460 0x7f863406d890 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f86340ce840 tx=0x7f862c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a0ee4cd0765636a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:44.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.579+0000 7f8632000640 1 -- 172.21.15.167:0/2859873929 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862c037070 con 0x7f863406d460 2024-05-11T12:11:44.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.579+0000 7f8632000640 1 -- 172.21.15.167:0/2859873929 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f862c032440 con 0x7f863406d460 2024-05-11T12:11:44.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.579+0000 7f8632000640 1 -- 172.21.15.167:0/2859873929 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862c032810 con 0x7f863406d460 2024-05-11T12:11:44.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.579+0000 7f8639600640 1 -- 172.21.15.167:0/2859873929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f863406d460 msgr2=0x7f863406d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:44.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.579+0000 7f8639600640 1 --2- 172.21.15.167:0/2859873929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f863406d460 0x7f863406d890 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f86340ce840 tx=0x7f862c02a270 comp rx=0 tx=0).stop 2024-05-11T12:11:44.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.579+0000 7f8639600640 1 -- 172.21.15.167:0/2859873929 shutdown_connections 2024-05-11T12:11:44.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.580+0000 7f8639600640 1 --2- 172.21.15.167:0/2859873929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f863406d460 0x7f863406d890 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:44.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.580+0000 7f8639600640 1 --2- 172.21.15.167:0/2859873929 >> v2:172.21.15.204:3300/0 conn(0x7f8634157750 0x7f863415d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:44.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.580+0000 7f8639600640 1 -- 172.21.15.167:0/2859873929 >> 172.21.15.167:0/2859873929 conn(0x7f86340cd8f0 msgr2=0x7f86340cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:44.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.580+0000 7f8639600640 1 -- 172.21.15.167:0/2859873929 shutdown_connections 2024-05-11T12:11:44.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.580+0000 7f8639600640 1 -- 172.21.15.167:0/2859873929 wait complete. 2024-05-11T12:11:44.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.581+0000 7f8639600640 1 Processor -- start 2024-05-11T12:11:44.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.581+0000 7f8639600640 1 -- start start 2024-05-11T12:11:44.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.582+0000 7f8639600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f863406d460 0x7f8634158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:44.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.582+0000 7f8639600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86340b3ea0 0x7f86340af880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:44.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.582+0000 7f8639600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8634157750 0x7f86340afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:44.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.582+0000 7f8639600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8634157b30 con 0x7f8634157750 2024-05-11T12:11:44.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.582+0000 7f8639600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f863415e4f0 con 0x7f863406d460 2024-05-11T12:11:44.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.582+0000 7f8639600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f8634161ba0 con 0x7f86340b3ea0 2024-05-11T12:11:44.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.583+0000 7f8633400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8634157750 0x7f86340afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:44.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.583+0000 7f8633400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8634157750 0x7f86340afdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:53756/0 (socket says 172.21.15.167:53756) 2024-05-11T12:11:44.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.583+0000 7f8633400640 1 -- 172.21.15.167:0/2775417783 learned_addr learned my addr 172.21.15.167:0/2775417783 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:44.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.583+0000 7f8632a00640 1 --2- 172.21.15.167:0/2775417783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f863406d460 0x7f8634158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:44.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.583+0000 7f8633400640 1 -- 172.21.15.167:0/2775417783 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86340b3ea0 msgr2=0x7f86340af880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:44.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.583+0000 7f8633400640 1 --2- 172.21.15.167:0/2775417783 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86340b3ea0 0x7f86340af880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:44.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.583+0000 7f8633400640 1 -- 172.21.15.167:0/2775417783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f863406d460 msgr2=0x7f8634158f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:44.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.583+0000 7f8633400640 1 --2- 172.21.15.167:0/2775417783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f863406d460 0x7f8634158f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:44.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.583+0000 7f8633400640 1 -- 172.21.15.167:0/2775417783 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86340ae820 con 0x7f8634157750 2024-05-11T12:11:44.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.583+0000 7f8632a00640 1 --2- 172.21.15.167:0/2775417783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f863406d460 0x7f8634158f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:44.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.583+0000 7f8633400640 1 --2- 172.21.15.167:0/2775417783 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8634157750 0x7f86340afdc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f862c035040 tx=0x7f862c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:44.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.584+0000 7f862b400640 1 -- 172.21.15.167:0/2775417783 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862c037040 con 0x7f8634157750 2024-05-11T12:11:44.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.584+0000 7f862b400640 1 -- 172.21.15.167:0/2775417783 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f862c004030 con 0x7f8634157750 2024-05-11T12:11:44.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.584+0000 7f862b400640 1 -- 172.21.15.167:0/2775417783 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862c004400 con 0x7f8634157750 2024-05-11T12:11:44.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.584+0000 7f8639600640 1 -- 172.21.15.167:0/2775417783 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86340aeac0 con 0x7f8634157750 2024-05-11T12:11:44.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.584+0000 7f8639600640 1 -- 172.21.15.167:0/2775417783 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86340af080 con 0x7f8634157750 2024-05-11T12:11:44.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.585+0000 7f8639600640 1 -- 172.21.15.167:0/2775417783 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8634069590 con 0x7f8634157750 2024-05-11T12:11:44.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.586+0000 7f862b400640 1 -- 172.21.15.167:0/2775417783 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f862c03a070 con 0x7f8634157750 2024-05-11T12:11:44.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.587+0000 7f862b400640 1 --2- 172.21.15.167:0/2775417783 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f86100762d0 0x7f8610078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:44.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.587+0000 7f862b400640 1 -- 172.21.15.167:0/2775417783 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6735+0+0 (secure 0 0 0) 0x7f862c0b82c0 con 0x7f8634157750 2024-05-11T12:11:44.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.587+0000 7f8632a00640 1 --2- 172.21.15.167:0/2775417783 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f86100762d0 0x7f8610078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:44.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.588+0000 7f8632a00640 1 --2- 172.21.15.167:0/2775417783 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f86100762d0 0x7f8610078790 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f862400d430 tx=0x7f8624007530 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:44.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.591+0000 7f862b400640 1 -- 172.21.15.167:0/2775417783 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f862c041020 con 0x7f8634157750 2024-05-11T12:11:44.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.793+0000 7f8639600640 1 -- 172.21.15.167:0/2775417783 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8634161530 con 0x7f8634157750 2024-05-11T12:11:44.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.795+0000 7f862b400640 1 -- 172.21.15.167:0/2775417783 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18950 (secure 0 0 0) 0x7f862c081840 con 0x7f8634157750 2024-05-11T12:11:44.794 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:11:44.795 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":29,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:11:27.038079+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:10:43.084511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T12:10:44.157782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-11T12:10:47.814674+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:11:44.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.797+0000 7f8639600640 1 -- 172.21.15.167:0/2775417783 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f86100762d0 msgr2=0x7f8610078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:44.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.797+0000 7f8639600640 1 --2- 172.21.15.167:0/2775417783 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f86100762d0 0x7f8610078790 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f862400d430 tx=0x7f8624007530 comp rx=0 tx=0).stop 2024-05-11T12:11:44.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.797+0000 7f8639600640 1 -- 172.21.15.167:0/2775417783 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8634157750 msgr2=0x7f86340afdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:44.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.797+0000 7f8639600640 1 --2- 172.21.15.167:0/2775417783 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8634157750 0x7f86340afdc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f862c035040 tx=0x7f862c0046c0 comp rx=0 tx=0).stop 2024-05-11T12:11:44.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.798+0000 7f8639600640 1 -- 172.21.15.167:0/2775417783 shutdown_connections 2024-05-11T12:11:44.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.798+0000 7f8639600640 1 --2- 172.21.15.167:0/2775417783 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8634157750 0x7f86340afdc0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:44.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.798+0000 7f8639600640 1 --2- 172.21.15.167:0/2775417783 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86340b3ea0 0x7f86340af880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:44.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.798+0000 7f8639600640 1 --2- 172.21.15.167:0/2775417783 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f86100762d0 0x7f8610078790 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:44.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.798+0000 7f8639600640 1 --2- 172.21.15.167:0/2775417783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f863406d460 0x7f8634158f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:44.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.798+0000 7f8639600640 1 -- 172.21.15.167:0/2775417783 >> 172.21.15.167:0/2775417783 conn(0x7f86340cd8f0 msgr2=0x7f8634145710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:44.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.798+0000 7f8639600640 1 -- 172.21.15.167:0/2775417783 shutdown_connections 2024-05-11T12:11:44.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:44.798+0000 7f8639600640 1 -- 172.21.15.167:0/2775417783 wait complete. 2024-05-11T12:11:44.814 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-11T12:11:45.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.007+0000 7fba73200640 1 Processor -- start 2024-05-11T12:11:45.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.007+0000 7fba73200640 1 -- start start 2024-05-11T12:11:45.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.009+0000 7fba73200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fba741611f0 0x7fba740cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:45.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.009+0000 7fba73200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fba74140010 0x7fba740cfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:45.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.009+0000 7fba73200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba740d0220 con 0x7fba7413fc30 2024-05-11T12:11:45.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.009+0000 7fba73200640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fba740d0000 con 0x7fba74140010 2024-05-11T12:11:45.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.009+0000 7fba73200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fba741619d0 con 0x7fba741611f0 2024-05-11T12:11:45.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.009+0000 7fba72800640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fba74140010 0x7fba740cfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:45.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.009+0000 7fba72800640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fba74140010 0x7fba740cfac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:53758/0 (socket says 172.21.15.167:53758) 2024-05-11T12:11:45.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.009+0000 7fba72800640 1 -- 172.21.15.167:0/3749269532 learned_addr learned my addr 172.21.15.167:0/3749269532 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:45.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.009+0000 7fba71400640 1 --2- 172.21.15.167:0/3749269532 >> v2:172.21.15.204:3300/0 conn(0x7fba741611f0 0x7fba740cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:45.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.009+0000 7fba72800640 1 -- 172.21.15.167:0/3749269532 >> v2:172.21.15.204:3300/0 conn(0x7fba741611f0 msgr2=0x7fba740cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:45.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.009+0000 7fba72800640 1 --2- 172.21.15.167:0/3749269532 >> v2:172.21.15.204:3300/0 conn(0x7fba741611f0 0x7fba740cf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:45.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.009+0000 7fba72800640 1 -- 172.21.15.167:0/3749269532 >> v1:172.21.15.204:6789/0 conn(0x7fba7413fc30 legacy=0x7fba74160ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:45.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.010+0000 7fba72800640 1 -- 172.21.15.167:0/3749269532 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba740d0500 con 0x7fba74140010 2024-05-11T12:11:45.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.010+0000 7fba71400640 1 --2- 172.21.15.167:0/3749269532 >> v2:172.21.15.204:3300/0 conn(0x7fba741611f0 0x7fba740cf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:45.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.011+0000 7fba72800640 1 --2- 172.21.15.167:0/3749269532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba74140010 0x7fba740cfac0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fba741545e0 tx=0x7fba6c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3498d15067496c05 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:45.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.011+0000 7fba70a00640 1 -- 172.21.15.167:0/3749269532 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c036070 con 0x7fba74140010 2024-05-11T12:11:45.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.011+0000 7fba70a00640 1 -- 172.21.15.167:0/3749269532 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba6c031600 con 0x7fba74140010 2024-05-11T12:11:45.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.011+0000 7fba70a00640 1 -- 172.21.15.167:0/3749269532 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c0319d0 con 0x7fba74140010 2024-05-11T12:11:45.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.012+0000 7fba73200640 1 -- 172.21.15.167:0/3749269532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba74140010 msgr2=0x7fba740cfac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:45.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.012+0000 7fba73200640 1 --2- 172.21.15.167:0/3749269532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba74140010 0x7fba740cfac0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fba741545e0 tx=0x7fba6c02a610 comp rx=0 tx=0).stop 2024-05-11T12:11:45.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.012+0000 7fba73200640 1 -- 172.21.15.167:0/3749269532 shutdown_connections 2024-05-11T12:11:45.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.012+0000 7fba73200640 1 --2- 172.21.15.167:0/3749269532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba74140010 0x7fba740cfac0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:45.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.012+0000 7fba73200640 1 --2- 172.21.15.167:0/3749269532 >> v2:172.21.15.204:3300/0 conn(0x7fba741611f0 0x7fba740cf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:45.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.012+0000 7fba73200640 1 -- 172.21.15.167:0/3749269532 >> 172.21.15.167:0/3749269532 conn(0x7fba7406c450 msgr2=0x7fba7406c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:45.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.012+0000 7fba73200640 1 -- 172.21.15.167:0/3749269532 shutdown_connections 2024-05-11T12:11:45.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.012+0000 7fba73200640 1 -- 172.21.15.167:0/3749269532 wait complete. 2024-05-11T12:11:45.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.014+0000 7fba73200640 1 Processor -- start 2024-05-11T12:11:45.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.014+0000 7fba73200640 1 -- start start 2024-05-11T12:11:45.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.015+0000 7fba73200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba7413fc30 0x7fba7415f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:45.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.015+0000 7fba73200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fba74140010 0x7fba7415fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:45.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.015+0000 7fba73200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba741611f0 0x7fba741308b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:45.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.015+0000 7fba73200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fba7406e590 con 0x7fba7413fc30 2024-05-11T12:11:45.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.015+0000 7fba73200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fba740d0ac0 con 0x7fba741611f0 2024-05-11T12:11:45.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.015+0000 7fba73200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fba7406eab0 con 0x7fba74140010 2024-05-11T12:11:45.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.015+0000 7fba71e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba7413fc30 0x7fba7415f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:45.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.015+0000 7fba71e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba7413fc30 0x7fba7415f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:53760/0 (socket says 172.21.15.167:53760) 2024-05-11T12:11:45.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.015+0000 7fba71e00640 1 -- 172.21.15.167:0/3468440980 learned_addr learned my addr 172.21.15.167:0/3468440980 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:45.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.016+0000 7fba71e00640 1 -- 172.21.15.167:0/3468440980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fba74140010 msgr2=0x7fba7415fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:45.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.016+0000 7fba71e00640 1 --2- 172.21.15.167:0/3468440980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fba74140010 0x7fba7415fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:45.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.016+0000 7fba71e00640 1 -- 172.21.15.167:0/3468440980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba741611f0 msgr2=0x7fba741308b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:45.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.016+0000 7fba72800640 1 --2- 172.21.15.167:0/3468440980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba741611f0 0x7fba741308b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:45.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.016+0000 7fba71e00640 1 --2- 172.21.15.167:0/3468440980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba741611f0 0x7fba741308b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:45.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.016+0000 7fba71e00640 1 -- 172.21.15.167:0/3468440980 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba741f9260 con 0x7fba7413fc30 2024-05-11T12:11:45.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.016+0000 7fba71e00640 1 --2- 172.21.15.167:0/3468440980 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba7413fc30 0x7fba7415f670 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fba6400c460 tx=0x7fba6400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:45.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.016+0000 7fba6a000640 1 -- 172.21.15.167:0/3468440980 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba64016020 con 0x7fba7413fc30 2024-05-11T12:11:45.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.016+0000 7fba6a000640 1 -- 172.21.15.167:0/3468440980 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba64012870 con 0x7fba7413fc30 2024-05-11T12:11:45.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.016+0000 7fba6a000640 1 -- 172.21.15.167:0/3468440980 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba64012c60 con 0x7fba7413fc30 2024-05-11T12:11:45.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.017+0000 7fba73200640 1 -- 172.21.15.167:0/3468440980 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba741f9540 con 0x7fba7413fc30 2024-05-11T12:11:45.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.017+0000 7fba73200640 1 -- 172.21.15.167:0/3468440980 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba741f9b00 con 0x7fba7413fc30 2024-05-11T12:11:45.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.018+0000 7fba73200640 1 -- 172.21.15.167:0/3468440980 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba740694b0 con 0x7fba7413fc30 2024-05-11T12:11:45.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.019+0000 7fba6a000640 1 -- 172.21.15.167:0/3468440980 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fba64019020 con 0x7fba7413fc30 2024-05-11T12:11:45.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.020+0000 7fba6a000640 1 --2- 172.21.15.167:0/3468440980 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fba540762d0 0x7fba54078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:45.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.020+0000 7fba6a000640 1 -- 172.21.15.167:0/3468440980 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6735+0+0 (secure 0 0 0) 0x7fba640977d0 con 0x7fba7413fc30 2024-05-11T12:11:45.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.020+0000 7fba71400640 1 --2- 172.21.15.167:0/3468440980 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fba540762d0 0x7fba54078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:45.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.021+0000 7fba71400640 1 --2- 172.21.15.167:0/3468440980 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fba540762d0 0x7fba54078790 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fba60006fd0 tx=0x7fba60008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:45.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.025+0000 7fba6a000640 1 -- 172.21.15.167:0/3468440980 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fba64020030 con 0x7fba7413fc30 2024-05-11T12:11:45.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.272+0000 7fba73200640 1 -- 172.21.15.167:0/3468440980 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fba74157610 con 0x7fba7413fc30 2024-05-11T12:11:45.272 INFO:tasks.ceph.mon.a.smithi167.stderr:2024-05-11T12:11:45.273+0000 7f3aac000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-11T12:11:45.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.277+0000 7fba6a000640 1 -- 172.21.15.167:0/3468440980 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) ==== 161+0+0 (secure 0 0 0) 0x7fba64060c90 con 0x7fba7413fc30 2024-05-11T12:11:45.276 INFO:teuthology.orchestra.run.smithi167.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-11T12:11:45.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.280+0000 7fba73200640 1 -- 172.21.15.167:0/3468440980 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fba540762d0 msgr2=0x7fba54078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:45.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.280+0000 7fba73200640 1 --2- 172.21.15.167:0/3468440980 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fba540762d0 0x7fba54078790 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fba60006fd0 tx=0x7fba60008040 comp rx=0 tx=0).stop 2024-05-11T12:11:45.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.280+0000 7fba73200640 1 -- 172.21.15.167:0/3468440980 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba7413fc30 msgr2=0x7fba7415f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:45.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.280+0000 7fba73200640 1 --2- 172.21.15.167:0/3468440980 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba7413fc30 0x7fba7415f670 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fba6400c460 tx=0x7fba6400c920 comp rx=0 tx=0).stop 2024-05-11T12:11:45.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.280+0000 7fba73200640 1 -- 172.21.15.167:0/3468440980 shutdown_connections 2024-05-11T12:11:45.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.280+0000 7fba73200640 1 --2- 172.21.15.167:0/3468440980 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fba540762d0 0x7fba54078790 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:45.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.280+0000 7fba73200640 1 --2- 172.21.15.167:0/3468440980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba741611f0 0x7fba741308b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:45.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.280+0000 7fba73200640 1 --2- 172.21.15.167:0/3468440980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fba74140010 0x7fba7415fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:45.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.280+0000 7fba73200640 1 --2- 172.21.15.167:0/3468440980 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba7413fc30 0x7fba7415f670 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:45.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.280+0000 7fba73200640 1 -- 172.21.15.167:0/3468440980 >> 172.21.15.167:0/3468440980 conn(0x7fba7406c450 msgr2=0x7fba7415c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:45.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.280+0000 7fba73200640 1 -- 172.21.15.167:0/3468440980 shutdown_connections 2024-05-11T12:11:45.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.281+0000 7fba73200640 1 -- 172.21.15.167:0/3468440980 wait complete. 2024-05-11T12:11:45.301 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-11T12:11:45.312 INFO:tasks.ceph.mds.b.smithi204.stderr: -4990> 2024-05-11T12:10:50.421+0000 7f8b5e200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-11T12:11:45.402 INFO:tasks.ceph.mds.b.smithi204.stdout:starting mds.b at 2024-05-11T12:11:45.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.498+0000 7f3f7ae00640 1 Processor -- start 2024-05-11T12:11:45.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.498+0000 7f3f7ae00640 1 -- start start 2024-05-11T12:11:45.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.500+0000 7f3f7ae00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3f7415f130 0x7f3f74071e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:45.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.500+0000 7f3f7ae00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f740b4b40 con 0x7f3f740afb60 2024-05-11T12:11:45.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.500+0000 7f3f7ae00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f740b4de0 con 0x7f3f740b62b0 2024-05-11T12:11:45.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.500+0000 7f3f7ae00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f3f740b50c0 con 0x7f3f7415f130 2024-05-11T12:11:45.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.500+0000 7f3f79000640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f3f740afb60 0x7f3f740b5ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:50928/0 (socket says 172.21.15.167:50928) 2024-05-11T12:11:45.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.500+0000 7f3f79000640 1 -- 172.21.15.167:0/1087076638 learned_addr learned my addr 172.21.15.167:0/1087076638 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:45.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.501+0000 7f3f73e00640 1 -- 172.21.15.167:0/1087076638 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1621832676 0 0) 0x7f3f740b4b40 con 0x7f3f740afb60 2024-05-11T12:11:45.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.501+0000 7f3f73e00640 1 -- 172.21.15.167:0/1087076638 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f5c003520 con 0x7f3f740afb60 2024-05-11T12:11:45.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.501+0000 7f3f79a00640 1 --2- 172.21.15.167:0/1087076638 >> v2:172.21.15.204:3300/0 conn(0x7f3f7415f130 0x7f3f74071e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:45.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.501+0000 7f3f73e00640 1 -- 172.21.15.167:0/1087076638 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3335248561 0 0) 0x7f3f5c003520 con 0x7f3f740afb60 2024-05-11T12:11:45.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.501+0000 7f3f73e00640 1 -- 172.21.15.167:0/1087076638 >> v2:172.21.15.204:3300/0 conn(0x7f3f7415f130 msgr2=0x7f3f74071e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:45.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.501+0000 7f3f73e00640 1 --2- 172.21.15.167:0/1087076638 >> v2:172.21.15.204:3300/0 conn(0x7f3f7415f130 0x7f3f74071e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:45.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.501+0000 7f3f73e00640 1 -- 172.21.15.167:0/1087076638 >> v1:172.21.15.204:6789/0 conn(0x7f3f740b62b0 legacy=0x7f3f7406d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:45.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.501+0000 7f3f73e00640 1 -- 172.21.15.167:0/1087076638 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f740b53a0 con 0x7f3f740afb60 2024-05-11T12:11:45.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.501+0000 7f3f73e00640 1 -- 172.21.15.167:0/1087076638 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f3f680016f0 con 0x7f3f740afb60 2024-05-11T12:11:45.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.501+0000 7f3f73e00640 1 -- 172.21.15.167:0/1087076638 >> v1:172.21.15.167:6789/0 conn(0x7f3f740afb60 legacy=0x7f3f740b5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:45.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.501+0000 7f3f73e00640 1 --2- 172.21.15.167:0/1087076638 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f5c003aa0 0x7f3f5c003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:45.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.501+0000 7f3f73e00640 1 --2- 172.21.15.167:0/1087076638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f5c004800 0x7f3f5c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:45.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.501+0000 7f3f73e00640 1 --2- 172.21.15.167:0/1087076638 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f5c0052e0 0x7f3f5c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:45.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.501+0000 7f3f73e00640 1 -- 172.21.15.167:0/1087076638 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3f740b50c0 con 0x7f3f5c003aa0 2024-05-11T12:11:45.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.501+0000 7f3f73e00640 1 -- 172.21.15.167:0/1087076638 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5c00c200 con 0x7f3f5c004800 2024-05-11T12:11:45.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.501+0000 7f3f73e00640 1 -- 172.21.15.167:0/1087076638 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3f5c00c400 con 0x7f3f5c0052e0 2024-05-11T12:11:45.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.502+0000 7f3f79a00640 1 --2- 172.21.15.167:0/1087076638 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f5c003aa0 0x7f3f5c003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:45.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.502+0000 7f3f79000640 1 --2- 172.21.15.167:0/1087076638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f5c004800 0x7f3f5c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:45.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.502+0000 7f3f79a00640 1 -- 172.21.15.167:0/1087076638 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f5c0052e0 msgr2=0x7f3f5c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:45.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.502+0000 7f3f79a00640 1 --2- 172.21.15.167:0/1087076638 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f5c0052e0 0x7f3f5c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:45.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.502+0000 7f3f79a00640 1 -- 172.21.15.167:0/1087076638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f5c004800 msgr2=0x7f3f5c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:45.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.502+0000 7f3f79a00640 1 --2- 172.21.15.167:0/1087076638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f5c004800 0x7f3f5c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:45.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.502+0000 7f3f79a00640 1 -- 172.21.15.167:0/1087076638 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f5c003520 con 0x7f3f5c003aa0 2024-05-11T12:11:45.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.502+0000 7f3f79000640 1 --2- 172.21.15.167:0/1087076638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f5c004800 0x7f3f5c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:45.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.502+0000 7f3f7a400640 1 -- 172.21.15.167:0/1087076638 reap_dead start 2024-05-11T12:11:45.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.503+0000 7f3f79a00640 1 --2- 172.21.15.167:0/1087076638 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f5c003aa0 0x7f3f5c003ec0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3f6000ba30 tx=0x7f3f5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4440ba3ada371fb server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:45.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.503+0000 7f3f73e00640 1 -- 172.21.15.167:0/1087076638 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f60030070 con 0x7f3f5c003aa0 2024-05-11T12:11:45.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.503+0000 7f3f73e00640 1 -- 172.21.15.167:0/1087076638 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f6002ae10 con 0x7f3f5c003aa0 2024-05-11T12:11:45.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.503+0000 7f3f73e00640 1 -- 172.21.15.167:0/1087076638 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6000b600 con 0x7f3f5c003aa0 2024-05-11T12:11:45.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.504+0000 7f3f7ae00640 1 -- 172.21.15.167:0/1087076638 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f5c003aa0 msgr2=0x7f3f5c003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:45.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.504+0000 7f3f7ae00640 1 --2- 172.21.15.167:0/1087076638 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f5c003aa0 0x7f3f5c003ec0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3f6000ba30 tx=0x7f3f5c005830 comp rx=0 tx=0).stop 2024-05-11T12:11:45.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.504+0000 7f3f7ae00640 1 -- 172.21.15.167:0/1087076638 shutdown_connections 2024-05-11T12:11:45.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.504+0000 7f3f7ae00640 1 --2- 172.21.15.167:0/1087076638 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f5c003aa0 0x7f3f5c003ec0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:45.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.504+0000 7f3f7ae00640 1 -- 172.21.15.167:0/1087076638 >> 172.21.15.167:0/1087076638 conn(0x7f3f740b2a20 msgr2=0x7f3f740b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:45.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.504+0000 7f3f7ae00640 1 -- 172.21.15.167:0/1087076638 shutdown_connections 2024-05-11T12:11:45.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.504+0000 7f3f7ae00640 1 -- 172.21.15.167:0/1087076638 wait complete. 2024-05-11T12:11:45.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.506+0000 7f3f7ae00640 1 Processor -- start 2024-05-11T12:11:45.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.506+0000 7f3f7ae00640 1 -- start start 2024-05-11T12:11:45.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.507+0000 7f3f7ae00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f5c003aa0 0x7f3f741495b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:45.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.507+0000 7f3f7ae00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f74160f70 0x7f3f741599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:45.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.507+0000 7f3f7ae00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f741613f0 0x7f3f74159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:45.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.507+0000 7f3f7ae00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3f740b0350 con 0x7f3f5c003aa0 2024-05-11T12:11:45.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.507+0000 7f3f7ae00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3f740b01d0 con 0x7f3f74160f70 2024-05-11T12:11:45.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.507+0000 7f3f7ae00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3f740b04d0 con 0x7f3f741613f0 2024-05-11T12:11:45.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.507+0000 7f3f79a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f5c003aa0 0x7f3f741495b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:45.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.507+0000 7f3f79a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f5c003aa0 0x7f3f741495b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:53772/0 (socket says 172.21.15.167:53772) 2024-05-11T12:11:45.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.507+0000 7f3f79a00640 1 -- 172.21.15.167:0/2332092763 learned_addr learned my addr 172.21.15.167:0/2332092763 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:45.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.508+0000 7f3f79000640 1 --2- 172.21.15.167:0/2332092763 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f74160f70 0x7f3f741599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:45.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.508+0000 7f3f7a400640 1 --2- 172.21.15.167:0/2332092763 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f741613f0 0x7f3f74159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:45.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.508+0000 7f3f79a00640 1 -- 172.21.15.167:0/2332092763 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f741613f0 msgr2=0x7f3f74159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:45.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.508+0000 7f3f79a00640 1 --2- 172.21.15.167:0/2332092763 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f741613f0 0x7f3f74159f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:45.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.508+0000 7f3f79a00640 1 -- 172.21.15.167:0/2332092763 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f74160f70 msgr2=0x7f3f741599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:45.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.508+0000 7f3f79a00640 1 --2- 172.21.15.167:0/2332092763 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f74160f70 0x7f3f741599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:45.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.508+0000 7f3f79a00640 1 -- 172.21.15.167:0/2332092763 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f74157230 con 0x7f3f5c003aa0 2024-05-11T12:11:45.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.508+0000 7f3f79a00640 1 --2- 172.21.15.167:0/2332092763 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f5c003aa0 0x7f3f741495b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3f6000bb20 tx=0x7f3f60004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:45.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.508+0000 7f3f71600640 1 -- 172.21.15.167:0/2332092763 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f60042030 con 0x7f3f5c003aa0 2024-05-11T12:11:45.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.508+0000 7f3f71600640 1 -- 172.21.15.167:0/2332092763 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f6002ac10 con 0x7f3f5c003aa0 2024-05-11T12:11:45.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.508+0000 7f3f71600640 1 -- 172.21.15.167:0/2332092763 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6000b600 con 0x7f3f5c003aa0 2024-05-11T12:11:45.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.508+0000 7f3f7ae00640 1 -- 172.21.15.167:0/2332092763 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f74157510 con 0x7f3f5c003aa0 2024-05-11T12:11:45.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.509+0000 7f3f7ae00640 1 -- 172.21.15.167:0/2332092763 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f74156b50 con 0x7f3f5c003aa0 2024-05-11T12:11:45.510 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.510+0000 7f3f7ae00640 1 -- 172.21.15.167:0/2332092763 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f740694b0 con 0x7f3f5c003aa0 2024-05-11T12:11:45.510 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.511+0000 7f3f71600640 1 -- 172.21.15.167:0/2332092763 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3f60045070 con 0x7f3f5c003aa0 2024-05-11T12:11:45.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.512+0000 7f3f71600640 1 --2- 172.21.15.167:0/2332092763 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3f54076300 0x7f3f540787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:45.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.512+0000 7f3f71600640 1 -- 172.21.15.167:0/2332092763 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6821+0+0 (secure 0 0 0) 0x7f3f600b8150 con 0x7f3f5c003aa0 2024-05-11T12:11:45.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.512+0000 7f3f79000640 1 --2- 172.21.15.167:0/2332092763 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3f54076300 0x7f3f540787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:45.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.513+0000 7f3f79000640 1 --2- 172.21.15.167:0/2332092763 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3f54076300 0x7f3f540787c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3f740b2360 tx=0x7f3f68009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:45.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.516+0000 7f3f71600640 1 -- 172.21.15.167:0/2332092763 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3f60081520 con 0x7f3f5c003aa0 2024-05-11T12:11:45.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:45.757+0000 7f3f7ae00640 1 -- 172.21.15.167:0/2332092763 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f3f740af050 con 0x7f3f5c003aa0 2024-05-11T12:11:46.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.280+0000 7f3f71600640 1 -- 172.21.15.167:0/2332092763 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f3f60080e20 con 0x7f3f5c003aa0 2024-05-11T12:11:46.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.284+0000 7f3f7ae00640 1 -- 172.21.15.167:0/2332092763 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3f54076300 msgr2=0x7f3f540787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:46.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.284+0000 7f3f7ae00640 1 --2- 172.21.15.167:0/2332092763 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3f54076300 0x7f3f540787c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3f740b2360 tx=0x7f3f68009040 comp rx=0 tx=0).stop 2024-05-11T12:11:46.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.284+0000 7f3f7ae00640 1 -- 172.21.15.167:0/2332092763 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f5c003aa0 msgr2=0x7f3f741495b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:46.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.284+0000 7f3f7ae00640 1 --2- 172.21.15.167:0/2332092763 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f5c003aa0 0x7f3f741495b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3f6000bb20 tx=0x7f3f60004730 comp rx=0 tx=0).stop 2024-05-11T12:11:46.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.284+0000 7f3f7ae00640 1 -- 172.21.15.167:0/2332092763 shutdown_connections 2024-05-11T12:11:46.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.284+0000 7f3f7ae00640 1 --2- 172.21.15.167:0/2332092763 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f741613f0 0x7f3f74159f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:46.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.284+0000 7f3f7ae00640 1 --2- 172.21.15.167:0/2332092763 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3f54076300 0x7f3f540787c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:46.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.284+0000 7f3f7ae00640 1 --2- 172.21.15.167:0/2332092763 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f74160f70 0x7f3f741599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:46.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.284+0000 7f3f7ae00640 1 --2- 172.21.15.167:0/2332092763 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f5c003aa0 0x7f3f741495b0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:46.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.284+0000 7f3f7ae00640 1 -- 172.21.15.167:0/2332092763 >> 172.21.15.167:0/2332092763 conn(0x7f3f740b2a20 msgr2=0x7f3f740b5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:46.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.285+0000 7f3f7ae00640 1 -- 172.21.15.167:0/2332092763 shutdown_connections 2024-05-11T12:11:46.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.285+0000 7f3f7ae00640 1 -- 172.21.15.167:0/2332092763 wait complete. 2024-05-11T12:11:46.305 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-11T12:11:46.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.509+0000 7f87df200640 1 Processor -- start 2024-05-11T12:11:46.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.510+0000 7f87df200640 1 -- start start 2024-05-11T12:11:46.510 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.511+0000 7f87df200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f87e01428d0 0x7f87e006d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:46.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.511+0000 7f87df200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87e006d9a0 0x7f87e00cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:46.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.511+0000 7f87df200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87e0160e90 con 0x7f87e013ffd0 2024-05-11T12:11:46.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.511+0000 7f87df200640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f87e0160c40 con 0x7f87e01428d0 2024-05-11T12:11:46.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.511+0000 7f87df200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f87e0161aa0 con 0x7f87e006d9a0 2024-05-11T12:11:46.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.511+0000 7f87d5400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f87e01428d0 0x7f87e006d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:46.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.512+0000 7f87d5400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f87e01428d0 0x7f87e006d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:53776/0 (socket says 172.21.15.167:53776) 2024-05-11T12:11:46.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.512+0000 7f87d5400640 1 -- 172.21.15.167:0/1825050405 learned_addr learned my addr 172.21.15.167:0/1825050405 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:46.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.512+0000 7f87d5400640 1 -- 172.21.15.167:0/1825050405 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87e006d9a0 msgr2=0x7f87e00cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:46.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.512+0000 7f87de800640 1 --2- 172.21.15.167:0/1825050405 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87e006d9a0 0x7f87e00cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:46.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.512+0000 7f87d5400640 1 --2- 172.21.15.167:0/1825050405 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87e006d9a0 0x7f87e00cf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:46.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.512+0000 7f87d5400640 1 -- 172.21.15.167:0/1825050405 >> v1:172.21.15.204:6789/0 conn(0x7f87e013ffd0 legacy=0x7f87e0143170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:46.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.512+0000 7f87d5400640 1 -- 172.21.15.167:0/1825050405 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87e0161170 con 0x7f87e01428d0 2024-05-11T12:11:46.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.512+0000 7f87de800640 1 --2- 172.21.15.167:0/1825050405 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87e006d9a0 0x7f87e00cf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:46.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.513+0000 7f87d5400640 1 --2- 172.21.15.167:0/1825050405 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87e01428d0 0x7f87e006d460 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f87c8009870 tx=0x7f87c802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6287b9196ef4f989 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:46.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.513+0000 7f87dd400640 1 -- 172.21.15.167:0/1825050405 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c8037070 con 0x7f87e01428d0 2024-05-11T12:11:46.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.513+0000 7f87dd400640 1 -- 172.21.15.167:0/1825050405 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87c8032440 con 0x7f87e01428d0 2024-05-11T12:11:46.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.514+0000 7f87dd400640 1 -- 172.21.15.167:0/1825050405 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c8032810 con 0x7f87e01428d0 2024-05-11T12:11:46.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.514+0000 7f87df200640 1 -- 172.21.15.167:0/1825050405 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87e01428d0 msgr2=0x7f87e006d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:46.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.514+0000 7f87df200640 1 --2- 172.21.15.167:0/1825050405 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87e01428d0 0x7f87e006d460 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f87c8009870 tx=0x7f87c802a1c0 comp rx=0 tx=0).stop 2024-05-11T12:11:46.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.514+0000 7f87df200640 1 -- 172.21.15.167:0/1825050405 shutdown_connections 2024-05-11T12:11:46.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.514+0000 7f87df200640 1 --2- 172.21.15.167:0/1825050405 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87e006d9a0 0x7f87e00cf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:46.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.514+0000 7f87df200640 1 --2- 172.21.15.167:0/1825050405 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87e01428d0 0x7f87e006d460 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:46.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.514+0000 7f87df200640 1 -- 172.21.15.167:0/1825050405 >> 172.21.15.167:0/1825050405 conn(0x7f87e00b32a0 msgr2=0x7f87e00d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:46.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.514+0000 7f87df200640 1 -- 172.21.15.167:0/1825050405 shutdown_connections 2024-05-11T12:11:46.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.514+0000 7f87df200640 1 -- 172.21.15.167:0/1825050405 wait complete. 2024-05-11T12:11:46.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.516+0000 7f87df200640 1 Processor -- start 2024-05-11T12:11:46.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.516+0000 7f87df200640 1 -- start start 2024-05-11T12:11:46.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.517+0000 7f87df200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87e006d9a0 0x7f87e01394d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:46.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.517+0000 7f87df200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87e013ffd0 0x7f87e013a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:46.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.517+0000 7f87df200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87e01428d0 0x7f87e015fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:46.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.517+0000 7f87df200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f87e0161730 con 0x7f87e01428d0 2024-05-11T12:11:46.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.517+0000 7f87df200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f87e006dde0 con 0x7f87e013ffd0 2024-05-11T12:11:46.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.517+0000 7f87df200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f87e01403b0 con 0x7f87e006d9a0 2024-05-11T12:11:46.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.517+0000 7f87de800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87e01428d0 0x7f87e015fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:46.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.517+0000 7f87de800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87e01428d0 0x7f87e015fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:53792/0 (socket says 172.21.15.167:53792) 2024-05-11T12:11:46.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.517+0000 7f87de800640 1 -- 172.21.15.167:0/3824647422 learned_addr learned my addr 172.21.15.167:0/3824647422 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:46.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.518+0000 7f87dde00640 1 --2- 172.21.15.167:0/3824647422 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87e006d9a0 0x7f87e01394d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:46.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.518+0000 7f87de800640 1 -- 172.21.15.167:0/3824647422 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87e006d9a0 msgr2=0x7f87e01394d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:46.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.518+0000 7f87d5400640 1 --2- 172.21.15.167:0/3824647422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87e013ffd0 0x7f87e013a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:46.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.518+0000 7f87de800640 1 --2- 172.21.15.167:0/3824647422 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87e006d9a0 0x7f87e01394d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:46.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.518+0000 7f87de800640 1 -- 172.21.15.167:0/3824647422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87e013ffd0 msgr2=0x7f87e013a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:46.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.518+0000 7f87de800640 1 --2- 172.21.15.167:0/3824647422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87e013ffd0 0x7f87e013a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:46.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.518+0000 7f87de800640 1 -- 172.21.15.167:0/3824647422 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87e00721d0 con 0x7f87e01428d0 2024-05-11T12:11:46.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.518+0000 7f87de800640 1 --2- 172.21.15.167:0/3824647422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87e01428d0 0x7f87e015fe90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f87d800cf20 tx=0x7f87d800a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:46.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.518+0000 7f87d6a00640 1 -- 172.21.15.167:0/3824647422 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d801a070 con 0x7f87e01428d0 2024-05-11T12:11:46.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.518+0000 7f87d6a00640 1 -- 172.21.15.167:0/3824647422 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87d800d840 con 0x7f87e01428d0 2024-05-11T12:11:46.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.518+0000 7f87d6a00640 1 -- 172.21.15.167:0/3824647422 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d800dc30 con 0x7f87e01428d0 2024-05-11T12:11:46.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.518+0000 7f87df200640 1 -- 172.21.15.167:0/3824647422 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87e00724b0 con 0x7f87e01428d0 2024-05-11T12:11:46.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.518+0000 7f87df200640 1 -- 172.21.15.167:0/3824647422 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87e0072a70 con 0x7f87e01428d0 2024-05-11T12:11:46.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.520+0000 7f87df200640 1 -- 172.21.15.167:0/3824647422 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87e00694b0 con 0x7f87e01428d0 2024-05-11T12:11:46.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.521+0000 7f87d6a00640 1 -- 172.21.15.167:0/3824647422 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f87d801d070 con 0x7f87e01428d0 2024-05-11T12:11:46.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.522+0000 7f87d6a00640 1 --2- 172.21.15.167:0/3824647422 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f87bc0762d0 0x7f87bc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:46.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.522+0000 7f87d6a00640 1 -- 172.21.15.167:0/3824647422 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6821+0+0 (secure 0 0 0) 0x7f87d80983b0 con 0x7f87e01428d0 2024-05-11T12:11:46.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.522+0000 7f87dde00640 1 --2- 172.21.15.167:0/3824647422 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f87bc0762d0 0x7f87bc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:46.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.522+0000 7f87dde00640 1 --2- 172.21.15.167:0/3824647422 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f87bc0762d0 0x7f87bc078790 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f87d0003d90 tx=0x7f87d0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:46.525 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.526+0000 7f87d6a00640 1 -- 172.21.15.167:0/3824647422 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f87d80618a0 con 0x7f87e01428d0 2024-05-11T12:11:46.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:46.733+0000 7f87df200640 1 -- 172.21.15.167:0/3824647422 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f87e006f900 con 0x7f87e01428d0 2024-05-11T12:11:47.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.284+0000 7f87d6a00640 1 -- 172.21.15.167:0/3824647422 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v31) ==== 181+0+0 (secure 0 0 0) 0x7f87d80611a0 con 0x7f87e01428d0 2024-05-11T12:11:47.284 INFO:teuthology.orchestra.run.smithi167.stderr:pool 'cephfs_metadata' removed 2024-05-11T12:11:47.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.289+0000 7f87df200640 1 -- 172.21.15.167:0/3824647422 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f87bc0762d0 msgr2=0x7f87bc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:47.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.289+0000 7f87df200640 1 --2- 172.21.15.167:0/3824647422 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f87bc0762d0 0x7f87bc078790 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f87d0003d90 tx=0x7f87d0009040 comp rx=0 tx=0).stop 2024-05-11T12:11:47.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.289+0000 7f87df200640 1 -- 172.21.15.167:0/3824647422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87e01428d0 msgr2=0x7f87e015fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:47.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.289+0000 7f87df200640 1 --2- 172.21.15.167:0/3824647422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87e01428d0 0x7f87e015fe90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f87d800cf20 tx=0x7f87d800a7d0 comp rx=0 tx=0).stop 2024-05-11T12:11:47.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.290+0000 7f87df200640 1 -- 172.21.15.167:0/3824647422 shutdown_connections 2024-05-11T12:11:47.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.290+0000 7f87df200640 1 --2- 172.21.15.167:0/3824647422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87e01428d0 0x7f87e015fe90 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:47.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.290+0000 7f87df200640 1 --2- 172.21.15.167:0/3824647422 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f87bc0762d0 0x7f87bc078790 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:47.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.290+0000 7f87df200640 1 --2- 172.21.15.167:0/3824647422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87e013ffd0 0x7f87e013a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:47.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.290+0000 7f87df200640 1 --2- 172.21.15.167:0/3824647422 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87e006d9a0 0x7f87e01394d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:47.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.290+0000 7f87df200640 1 -- 172.21.15.167:0/3824647422 >> 172.21.15.167:0/3824647422 conn(0x7f87e00b32a0 msgr2=0x7f87e00d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:47.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.295+0000 7f87df200640 1 -- 172.21.15.167:0/3824647422 shutdown_connections 2024-05-11T12:11:47.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.295+0000 7f87df200640 1 -- 172.21.15.167:0/3824647422 wait complete. 2024-05-11T12:11:47.319 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-11T12:11:47.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.517+0000 7f8897200640 1 Processor -- start 2024-05-11T12:11:47.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.517+0000 7f8897200640 1 -- start start 2024-05-11T12:11:47.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.519+0000 7f8897200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f889806d060 0x7f889806d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:47.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.519+0000 7f8897200640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f889815f640 con 0x7f889815f260 2024-05-11T12:11:47.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.519+0000 7f8897200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88980d2d30 con 0x7f88980cfd90 2024-05-11T12:11:47.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.519+0000 7f8897200640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f889806da10 con 0x7f889806d060 2024-05-11T12:11:47.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.519+0000 7f8896800640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f889806d060 0x7f889806d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:47.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.519+0000 7f8895e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f889815f260 0x7f88981637e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:50936/0 (socket says 172.21.15.167:50936) 2024-05-11T12:11:47.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.519+0000 7f8895e00640 1 -- 172.21.15.167:0/2446210684 learned_addr learned my addr 172.21.15.167:0/2446210684 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:47.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.520+0000 7f8896800640 1 -- 172.21.15.167:0/2446210684 >> v1:172.21.15.204:6789/0 conn(0x7f88980cfd90 legacy=0x7f88980d3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:47.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.520+0000 7f8896800640 1 -- 172.21.15.167:0/2446210684 >> v1:172.21.15.167:6789/0 conn(0x7f889815f260 legacy=0x7f88981637e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:47.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.520+0000 7f8896800640 1 -- 172.21.15.167:0/2446210684 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88980d3010 con 0x7f889806d060 2024-05-11T12:11:47.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.521+0000 7f8896800640 1 --2- 172.21.15.167:0/2446210684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f889806d060 0x7f889806d4d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f88900097a0 tx=0x7f889002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=972846b84dd7416d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:47.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.521+0000 7f8894a00640 1 -- 172.21.15.167:0/2446210684 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8890037070 con 0x7f889806d060 2024-05-11T12:11:47.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.521+0000 7f8894a00640 1 -- 172.21.15.167:0/2446210684 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8890032440 con 0x7f889806d060 2024-05-11T12:11:47.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.521+0000 7f8894a00640 1 -- 172.21.15.167:0/2446210684 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8890032810 con 0x7f889806d060 2024-05-11T12:11:47.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.522+0000 7f8897200640 1 -- 172.21.15.167:0/2446210684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f889806d060 msgr2=0x7f889806d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:47.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.522+0000 7f8897200640 1 --2- 172.21.15.167:0/2446210684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f889806d060 0x7f889806d4d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f88900097a0 tx=0x7f889002a180 comp rx=0 tx=0).stop 2024-05-11T12:11:47.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.522+0000 7f8897200640 1 -- 172.21.15.167:0/2446210684 shutdown_connections 2024-05-11T12:11:47.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.522+0000 7f8897200640 1 --2- 172.21.15.167:0/2446210684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f889806d060 0x7f889806d4d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:47.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.522+0000 7f8897200640 1 -- 172.21.15.167:0/2446210684 >> 172.21.15.167:0/2446210684 conn(0x7f88980b6cd0 msgr2=0x7f88980b70e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:47.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.522+0000 7f8897200640 1 -- 172.21.15.167:0/2446210684 shutdown_connections 2024-05-11T12:11:47.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.522+0000 7f8897200640 1 -- 172.21.15.167:0/2446210684 wait complete. 2024-05-11T12:11:47.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.523+0000 7f8897200640 1 Processor -- start 2024-05-11T12:11:47.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.524+0000 7f8897200640 1 -- start start 2024-05-11T12:11:47.525 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.525+0000 7f8897200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f889806d060 0x7f889815d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:47.525 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.525+0000 7f8897200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f88980cfd90 0x7f8898070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:47.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.525+0000 7f8897200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f889815f260 0x7f8898070850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:47.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.525+0000 7f8897200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f88980d35d0 con 0x7f889815f260 2024-05-11T12:11:47.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.525+0000 7f8897200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f88980d3940 con 0x7f889806d060 2024-05-11T12:11:47.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.525+0000 7f8897200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f88980d44e0 con 0x7f88980cfd90 2024-05-11T12:11:47.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.525+0000 7f8896800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f889815f260 0x7f8898070850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:47.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.525+0000 7f8896800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f889815f260 0x7f8898070850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:53802/0 (socket says 172.21.15.167:53802) 2024-05-11T12:11:47.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.525+0000 7f8896800640 1 -- 172.21.15.167:0/1662016300 learned_addr learned my addr 172.21.15.167:0/1662016300 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:47.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.525+0000 7f8896800640 1 -- 172.21.15.167:0/1662016300 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f88980cfd90 msgr2=0x7f8898070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:47.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.525+0000 7f8895400640 1 --2- 172.21.15.167:0/1662016300 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f88980cfd90 0x7f8898070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:47.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.525+0000 7f8895e00640 1 --2- 172.21.15.167:0/1662016300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f889806d060 0x7f889815d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:47.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.525+0000 7f8896800640 1 --2- 172.21.15.167:0/1662016300 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f88980cfd90 0x7f8898070310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:47.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.525+0000 7f8896800640 1 -- 172.21.15.167:0/1662016300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f889806d060 msgr2=0x7f889815d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:47.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.525+0000 7f8896800640 1 --2- 172.21.15.167:0/1662016300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f889806d060 0x7f889815d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:47.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.525+0000 7f8896800640 1 -- 172.21.15.167:0/1662016300 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88980b6800 con 0x7f889815f260 2024-05-11T12:11:47.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.526+0000 7f8895400640 1 --2- 172.21.15.167:0/1662016300 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f88980cfd90 0x7f8898070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:47.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.526+0000 7f8895e00640 1 --2- 172.21.15.167:0/1662016300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f889806d060 0x7f889815d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:47.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.526+0000 7f8896800640 1 --2- 172.21.15.167:0/1662016300 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f889815f260 0x7f8898070850 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f88900097a0 tx=0x7f88900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:47.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.526+0000 7f888e000640 1 -- 172.21.15.167:0/1662016300 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8890041070 con 0x7f889815f260 2024-05-11T12:11:47.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.526+0000 7f888e000640 1 -- 172.21.15.167:0/1662016300 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f889003c870 con 0x7f889815f260 2024-05-11T12:11:47.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.526+0000 7f888e000640 1 -- 172.21.15.167:0/1662016300 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8890037040 con 0x7f889815f260 2024-05-11T12:11:47.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.526+0000 7f8897200640 1 -- 172.21.15.167:0/1662016300 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f889815da90 con 0x7f889815f260 2024-05-11T12:11:47.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.526+0000 7f8897200640 1 -- 172.21.15.167:0/1662016300 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88980cc530 con 0x7f889815f260 2024-05-11T12:11:47.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.528+0000 7f8897200640 1 -- 172.21.15.167:0/1662016300 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88980694b0 con 0x7f889815f260 2024-05-11T12:11:47.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.529+0000 7f888e000640 1 -- 172.21.15.167:0/1662016300 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8890044070 con 0x7f889815f260 2024-05-11T12:11:47.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.530+0000 7f888e000640 1 --2- 172.21.15.167:0/1662016300 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f88740762d0 0x7f8874078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:47.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.530+0000 7f888e000640 1 -- 172.21.15.167:0/1662016300 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6383+0+0 (secure 0 0 0) 0x7f88900b84d0 con 0x7f889815f260 2024-05-11T12:11:47.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.530+0000 7f8895e00640 1 --2- 172.21.15.167:0/1662016300 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f88740762d0 0x7f8874078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:47.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.531+0000 7f8895e00640 1 --2- 172.21.15.167:0/1662016300 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f88740762d0 0x7f8874078790 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8888000f60 tx=0x7f8888009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:47.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.534+0000 7f888e000640 1 -- 172.21.15.167:0/1662016300 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8890081af0 con 0x7f889815f260 2024-05-11T12:11:47.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:47.742+0000 7f8897200640 1 -- 172.21.15.167:0/1662016300 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f88980759b0 con 0x7f889815f260 2024-05-11T12:11:48.295 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.295+0000 7f888e000640 1 -- 172.21.15.167:0/1662016300 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v32) ==== 169+0+0 (secure 0 0 0) 0x7f88900813f0 con 0x7f889815f260 2024-05-11T12:11:48.296 INFO:teuthology.orchestra.run.smithi167.stderr:pool 'cephfs_data' removed 2024-05-11T12:11:48.302 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.302+0000 7f8897200640 1 -- 172.21.15.167:0/1662016300 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f88740762d0 msgr2=0x7f8874078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:48.302 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.302+0000 7f8897200640 1 --2- 172.21.15.167:0/1662016300 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f88740762d0 0x7f8874078790 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8888000f60 tx=0x7f8888009040 comp rx=0 tx=0).stop 2024-05-11T12:11:48.302 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.302+0000 7f8897200640 1 -- 172.21.15.167:0/1662016300 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f889815f260 msgr2=0x7f8898070850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:48.302 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.302+0000 7f8897200640 1 --2- 172.21.15.167:0/1662016300 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f889815f260 0x7f8898070850 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f88900097a0 tx=0x7f88900046c0 comp rx=0 tx=0).stop 2024-05-11T12:11:48.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.302+0000 7f8897200640 1 -- 172.21.15.167:0/1662016300 shutdown_connections 2024-05-11T12:11:48.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.302+0000 7f8897200640 1 --2- 172.21.15.167:0/1662016300 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f889815f260 0x7f8898070850 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:48.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.302+0000 7f8897200640 1 --2- 172.21.15.167:0/1662016300 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f88980cfd90 0x7f8898070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:48.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.302+0000 7f8897200640 1 --2- 172.21.15.167:0/1662016300 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f88740762d0 0x7f8874078790 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:48.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.302+0000 7f8897200640 1 --2- 172.21.15.167:0/1662016300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f889806d060 0x7f889815d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:48.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.302+0000 7f8897200640 1 -- 172.21.15.167:0/1662016300 >> 172.21.15.167:0/1662016300 conn(0x7f88980b6cd0 msgr2=0x7f8898161ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:48.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.305+0000 7f8897200640 1 -- 172.21.15.167:0/1662016300 shutdown_connections 2024-05-11T12:11:48.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.307+0000 7f8897200640 1 -- 172.21.15.167:0/1662016300 wait complete. 2024-05-11T12:11:48.331 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-05-11T12:11:48.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.517+0000 7f422e800640 1 Processor -- start 2024-05-11T12:11:48.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.518+0000 7f422e800640 1 -- start start 2024-05-11T12:11:48.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.519+0000 7f422e800640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f422815cda0 0x7f42280b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:48.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.519+0000 7f422e800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f42280b5eb0 0x7f422806fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:48.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.519+0000 7f422e800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f422806ff70 0x7f4228162e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:48.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.519+0000 7f422e800640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f422815f970 con 0x7f422815cda0 2024-05-11T12:11:48.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.519+0000 7f422e800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f422815fc50 con 0x7f42280b5eb0 2024-05-11T12:11:48.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.519+0000 7f422e800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f422815ff30 con 0x7f422806ff70 2024-05-11T12:11:48.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.519+0000 7f4227400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f422815cda0 0x7f42280b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:48.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.519+0000 7f4227400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f422815cda0 0x7f42280b5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:53808/0 (socket says 172.21.15.167:53808) 2024-05-11T12:11:48.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.519+0000 7f4227400640 1 -- 172.21.15.167:0/1185749548 learned_addr learned my addr 172.21.15.167:0/1185749548 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:48.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.520+0000 7f4227400640 1 -- 172.21.15.167:0/1185749548 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f422806ff70 msgr2=0x7f4228162e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:48.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.520+0000 7f4227400640 1 --2- 172.21.15.167:0/1185749548 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f422806ff70 0x7f4228162e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:48.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.520+0000 7f4227400640 1 -- 172.21.15.167:0/1185749548 >> v2:172.21.15.204:3300/0 conn(0x7f42280b5eb0 msgr2=0x7f422806fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:48.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.520+0000 7f4227400640 1 --2- 172.21.15.167:0/1185749548 >> v2:172.21.15.204:3300/0 conn(0x7f42280b5eb0 0x7f422806fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:48.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.520+0000 7f4227400640 1 -- 172.21.15.167:0/1185749548 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4228160210 con 0x7f422815cda0 2024-05-11T12:11:48.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.521+0000 7f4227400640 1 --2- 172.21.15.167:0/1185749548 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f422815cda0 0x7f42280b5970 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f42280d1b70 tx=0x7f421802a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53b0f497be7be6c server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:48.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.521+0000 7f4226a00640 1 -- 172.21.15.167:0/1185749548 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4218036070 con 0x7f422815cda0 2024-05-11T12:11:48.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.521+0000 7f4226a00640 1 -- 172.21.15.167:0/1185749548 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4218031600 con 0x7f422815cda0 2024-05-11T12:11:48.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.521+0000 7f4226a00640 1 -- 172.21.15.167:0/1185749548 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42180319d0 con 0x7f422815cda0 2024-05-11T12:11:48.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.522+0000 7f422e800640 1 -- 172.21.15.167:0/1185749548 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f422815cda0 msgr2=0x7f42280b5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:48.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.522+0000 7f422e800640 1 --2- 172.21.15.167:0/1185749548 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f422815cda0 0x7f42280b5970 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f42280d1b70 tx=0x7f421802a690 comp rx=0 tx=0).stop 2024-05-11T12:11:48.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.522+0000 7f422e800640 1 -- 172.21.15.167:0/1185749548 shutdown_connections 2024-05-11T12:11:48.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.522+0000 7f422e800640 1 --2- 172.21.15.167:0/1185749548 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f422806ff70 0x7f4228162e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:48.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.522+0000 7f422e800640 1 --2- 172.21.15.167:0/1185749548 >> v2:172.21.15.204:3300/0 conn(0x7f42280b5eb0 0x7f422806fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:48.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.522+0000 7f422e800640 1 --2- 172.21.15.167:0/1185749548 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f422815cda0 0x7f42280b5970 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:48.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.522+0000 7f422e800640 1 -- 172.21.15.167:0/1185749548 >> 172.21.15.167:0/1185749548 conn(0x7f42280d0c70 msgr2=0x7f42280d1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:48.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.522+0000 7f422e800640 1 -- 172.21.15.167:0/1185749548 shutdown_connections 2024-05-11T12:11:48.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.522+0000 7f422e800640 1 -- 172.21.15.167:0/1185749548 wait complete. 2024-05-11T12:11:48.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.524+0000 7f422e800640 1 Processor -- start 2024-05-11T12:11:48.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.524+0000 7f422e800640 1 -- start start 2024-05-11T12:11:48.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.525+0000 7f422e800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f422806ff70 0x7f42280ceb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:48.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.525+0000 7f422e800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42280b5eb0 0x7f42281540c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:48.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.525+0000 7f422e800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f422815cda0 0x7f42280b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:48.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.525+0000 7f422e800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f42280703a0 con 0x7f422815cda0 2024-05-11T12:11:48.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.525+0000 7f422e800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f422815d180 con 0x7f422806ff70 2024-05-11T12:11:48.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.525+0000 7f422e800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f42280af6c0 con 0x7f42280b5eb0 2024-05-11T12:11:48.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.526+0000 7f4227e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f422815cda0 0x7f42280b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:48.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.526+0000 7f4227e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f422815cda0 0x7f42280b3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:53810/0 (socket says 172.21.15.167:53810) 2024-05-11T12:11:48.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.526+0000 7f4227e00640 1 -- 172.21.15.167:0/3485400260 learned_addr learned my addr 172.21.15.167:0/3485400260 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:48.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.526+0000 7f4227400640 1 --2- 172.21.15.167:0/3485400260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f422806ff70 0x7f42280ceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:48.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.526+0000 7f421ea00640 1 --2- 172.21.15.167:0/3485400260 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42280b5eb0 0x7f42281540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:48.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.526+0000 7f4227e00640 1 -- 172.21.15.167:0/3485400260 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42280b5eb0 msgr2=0x7f42281540c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:48.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.526+0000 7f4227e00640 1 --2- 172.21.15.167:0/3485400260 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42280b5eb0 0x7f42281540c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:48.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.526+0000 7f4227e00640 1 -- 172.21.15.167:0/3485400260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f422806ff70 msgr2=0x7f42280ceb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:48.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.526+0000 7f4227e00640 1 --2- 172.21.15.167:0/3485400260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f422806ff70 0x7f42280ceb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:48.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.526+0000 7f4227e00640 1 -- 172.21.15.167:0/3485400260 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4228161b50 con 0x7f422815cda0 2024-05-11T12:11:48.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.527+0000 7f4227e00640 1 --2- 172.21.15.167:0/3485400260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f422815cda0 0x7f42280b3de0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4220009fd0 tx=0x7f422000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:48.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.527+0000 7f421fe00640 1 -- 172.21.15.167:0/3485400260 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4220011830 con 0x7f422815cda0 2024-05-11T12:11:48.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.527+0000 7f421fe00640 1 -- 172.21.15.167:0/3485400260 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4220016070 con 0x7f422815cda0 2024-05-11T12:11:48.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.527+0000 7f421fe00640 1 -- 172.21.15.167:0/3485400260 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4220011bf0 con 0x7f422815cda0 2024-05-11T12:11:48.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.527+0000 7f422e800640 1 -- 172.21.15.167:0/3485400260 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4228161dd0 con 0x7f422815cda0 2024-05-11T12:11:48.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.527+0000 7f422e800640 1 -- 172.21.15.167:0/3485400260 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4228162390 con 0x7f422815cda0 2024-05-11T12:11:48.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.528+0000 7f422e800640 1 -- 172.21.15.167:0/3485400260 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4228069590 con 0x7f422815cda0 2024-05-11T12:11:48.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.530+0000 7f421fe00640 1 -- 172.21.15.167:0/3485400260 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f422001b460 con 0x7f422815cda0 2024-05-11T12:11:48.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.531+0000 7f421fe00640 1 --2- 172.21.15.167:0/3485400260 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f42040762d0 0x7f4204078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:48.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.531+0000 7f421fe00640 1 -- 172.21.15.167:0/3485400260 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5853+0+0 (secure 0 0 0) 0x7f4220097380 con 0x7f422815cda0 2024-05-11T12:11:48.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.532+0000 7f4227400640 1 --2- 172.21.15.167:0/3485400260 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f42040762d0 0x7f4204078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:48.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.532+0000 7f4227400640 1 --2- 172.21.15.167:0/3485400260 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f42040762d0 0x7f4204078790 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4218033000 tx=0x7f4218009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:48.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.536+0000 7f421fe00640 1 -- 172.21.15.167:0/3485400260 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4228161b50 con 0x7f422815cda0 2024-05-11T12:11:48.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:48.742+0000 7f422e800640 1 -- 172.21.15.167:0/3485400260 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4228158360 con 0x7f422815cda0 2024-05-11T12:11:49.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.298+0000 7f421fe00640 1 -- 172.21.15.167:0/3485400260 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v33) ==== 178+0+0 (secure 0 0 0) 0x7f4220060c40 con 0x7f422815cda0 2024-05-11T12:11:49.299 INFO:teuthology.orchestra.run.smithi167.stderr:pool 'cephfs_data_ec' removed 2024-05-11T12:11:49.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.303+0000 7f422e800640 1 -- 172.21.15.167:0/3485400260 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f42040762d0 msgr2=0x7f4204078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:49.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.303+0000 7f422e800640 1 --2- 172.21.15.167:0/3485400260 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f42040762d0 0x7f4204078790 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4218033000 tx=0x7f4218009000 comp rx=0 tx=0).stop 2024-05-11T12:11:49.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.303+0000 7f422e800640 1 -- 172.21.15.167:0/3485400260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f422815cda0 msgr2=0x7f42280b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:49.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.303+0000 7f422e800640 1 --2- 172.21.15.167:0/3485400260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f422815cda0 0x7f42280b3de0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4220009fd0 tx=0x7f422000b8e0 comp rx=0 tx=0).stop 2024-05-11T12:11:49.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.304+0000 7f422e800640 1 -- 172.21.15.167:0/3485400260 shutdown_connections 2024-05-11T12:11:49.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.305+0000 7f422e800640 1 --2- 172.21.15.167:0/3485400260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f422815cda0 0x7f42280b3de0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:49.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.305+0000 7f422e800640 1 --2- 172.21.15.167:0/3485400260 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42280b5eb0 0x7f42281540c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:49.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.305+0000 7f422e800640 1 --2- 172.21.15.167:0/3485400260 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f42040762d0 0x7f4204078790 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:49.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.305+0000 7f422e800640 1 --2- 172.21.15.167:0/3485400260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f422806ff70 0x7f42280ceb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:49.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.305+0000 7f422e800640 1 -- 172.21.15.167:0/3485400260 >> 172.21.15.167:0/3485400260 conn(0x7f42280d0c70 msgr2=0x7f422806d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:49.306 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.306+0000 7f422e800640 1 -- 172.21.15.167:0/3485400260 shutdown_connections 2024-05-11T12:11:49.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.307+0000 7f422e800640 1 -- 172.21.15.167:0/3485400260 wait complete. 2024-05-11T12:11:49.329 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-11T12:11:49.329 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-11T12:11:49.329 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-11T12:11:49.329 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T12:11:49.329 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-11T12:11:49.329 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-11T12:11:49.330 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-11T12:11:49.330 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T12:11:49.330 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-11T12:11:49.330 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-11T12:11:49.330 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-11T12:11:49.330 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T12:11:49.331 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-11T12:11:49.331 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-11T12:11:49.331 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-11T12:11:49.331 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T12:11:49.349 INFO:tasks.ceph.mds.a:Stopped 2024-05-11T12:11:49.350 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-11T12:11:49.353 INFO:tasks.ceph.mds.b:Stopped 2024-05-11T12:11:49.353 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-11T12:11:49.353 INFO:tasks.ceph.mds.d:Stopped 2024-05-11T12:11:49.353 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-11T12:11:49.355 INFO:tasks.ceph.mds.c:Stopped 2024-05-11T12:11:49.355 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-11T12:11:49.400 INFO:tasks.ceph.mds.b:Started 2024-05-11T12:11:49.401 DEBUG:teuthology.parallel:result is None 2024-05-11T12:11:49.401 INFO:tasks.ceph.mds.a:Started 2024-05-11T12:11:49.402 DEBUG:teuthology.parallel:result is None 2024-05-11T12:11:49.406 INFO:tasks.ceph.mds.d:Started 2024-05-11T12:11:49.406 DEBUG:teuthology.parallel:result is None 2024-05-11T12:11:49.408 INFO:tasks.ceph.mds.c:Started 2024-05-11T12:11:49.408 DEBUG:teuthology.parallel:result is None 2024-05-11T12:11:49.408 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-11T12:11:49.561 INFO:tasks.ceph.mds.b.smithi204.stdout:starting mds.b at 2024-05-11T12:11:49.561 INFO:tasks.ceph.mds.a.smithi167.stdout:starting mds.a at 2024-05-11T12:11:49.561 INFO:tasks.ceph.mds.d.smithi204.stdout:starting mds.d at 2024-05-11T12:11:49.562 INFO:tasks.ceph.mds.c.smithi167.stdout:starting mds.c at 2024-05-11T12:11:49.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.598+0000 7fb130800640 1 Processor -- start 2024-05-11T12:11:49.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.598+0000 7fb130800640 1 -- start start 2024-05-11T12:11:49.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.600+0000 7fb130800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb12815ac80 0x7fb128074260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:49.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.600+0000 7fb130800640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb128161380 0x7fb128072950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:49.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.600+0000 7fb130800640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1280b65e0 con 0x7fb1280b3d50 2024-05-11T12:11:49.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.600+0000 7fb130800640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fb12806e3c0 con 0x7fb128161380 2024-05-11T12:11:49.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.600+0000 7fb130800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fb12815b060 con 0x7fb12815ac80 2024-05-11T12:11:49.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.600+0000 7fb12e800640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb128161380 0x7fb128072950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:49.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.600+0000 7fb12d400640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fb1280b3d50 0x7fb128160c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:57670/0 (socket says 172.21.15.167:57670) 2024-05-11T12:11:49.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.600+0000 7fb12d400640 1 -- 172.21.15.167:0/1016977862 learned_addr learned my addr 172.21.15.167:0/1016977862 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:49.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.600+0000 7fb12de00640 1 --2- 172.21.15.167:0/1016977862 >> v2:172.21.15.204:3300/0 conn(0x7fb12815ac80 0x7fb128074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:49.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.601+0000 7fb12e800640 1 -- 172.21.15.167:0/1016977862 >> v2:172.21.15.204:3300/0 conn(0x7fb12815ac80 msgr2=0x7fb128074260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:49.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.601+0000 7fb12e800640 1 --2- 172.21.15.167:0/1016977862 >> v2:172.21.15.204:3300/0 conn(0x7fb12815ac80 0x7fb128074260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:49.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.601+0000 7fb12e800640 1 -- 172.21.15.167:0/1016977862 >> v1:172.21.15.167:6789/0 conn(0x7fb1280b3d50 legacy=0x7fb128160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:49.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.601+0000 7fb12e800640 1 -- 172.21.15.167:0/1016977862 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1280b68c0 con 0x7fb128161380 2024-05-11T12:11:49.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.601+0000 7fb12de00640 1 --2- 172.21.15.167:0/1016977862 >> v2:172.21.15.204:3300/0 conn(0x7fb12815ac80 0x7fb128074260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:11:49.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.602+0000 7fb12e800640 1 --2- 172.21.15.167:0/1016977862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb128161380 0x7fb128072950 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb124009d90 tx=0x7fb12402a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99e4f1d2a3ef4965 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:49.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.602+0000 7fb12ca00640 1 -- 172.21.15.167:0/1016977862 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb124007740 con 0x7fb128161380 2024-05-11T12:11:49.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.602+0000 7fb12ca00640 1 -- 172.21.15.167:0/1016977862 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1240078e0 con 0x7fb128161380 2024-05-11T12:11:49.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.602+0000 7fb12ca00640 1 -- 172.21.15.167:0/1016977862 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb124007cb0 con 0x7fb128161380 2024-05-11T12:11:49.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.602+0000 7fb130800640 1 -- 172.21.15.167:0/1016977862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb128161380 msgr2=0x7fb128072950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:49.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.603+0000 7fb130800640 1 --2- 172.21.15.167:0/1016977862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb128161380 0x7fb128072950 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb124009d90 tx=0x7fb12402a5b0 comp rx=0 tx=0).stop 2024-05-11T12:11:49.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.603+0000 7fb130800640 1 -- 172.21.15.167:0/1016977862 shutdown_connections 2024-05-11T12:11:49.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.603+0000 7fb130800640 1 --2- 172.21.15.167:0/1016977862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb128161380 0x7fb128072950 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:49.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.603+0000 7fb130800640 1 --2- 172.21.15.167:0/1016977862 >> v2:172.21.15.204:3300/0 conn(0x7fb12815ac80 0x7fb128074260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:49.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.603+0000 7fb130800640 1 -- 172.21.15.167:0/1016977862 >> 172.21.15.167:0/1016977862 conn(0x7fb12806c450 msgr2=0x7fb12806c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:49.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.603+0000 7fb130800640 1 -- 172.21.15.167:0/1016977862 shutdown_connections 2024-05-11T12:11:49.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.603+0000 7fb130800640 1 -- 172.21.15.167:0/1016977862 wait complete. 2024-05-11T12:11:49.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.604+0000 7fb130800640 1 Processor -- start 2024-05-11T12:11:49.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.604+0000 7fb130800640 1 -- start start 2024-05-11T12:11:49.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.605+0000 7fb130800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1280b3d50 0x7fb128156a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:49.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.605+0000 7fb130800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb12815ac80 0x7fb1280aa840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:49.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.605+0000 7fb130800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb128161380 0x7fb1280aad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:49.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.605+0000 7fb130800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb1280b44f0 con 0x7fb128161380 2024-05-11T12:11:49.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.605+0000 7fb130800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb1280b41e0 con 0x7fb12815ac80 2024-05-11T12:11:49.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.605+0000 7fb130800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb128072e90 con 0x7fb1280b3d50 2024-05-11T12:11:49.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.606+0000 7fb12e800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb128161380 0x7fb1280aad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:49.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.606+0000 7fb12e800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb128161380 0x7fb1280aad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:33114/0 (socket says 172.21.15.167:33114) 2024-05-11T12:11:49.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.606+0000 7fb12e800640 1 -- 172.21.15.167:0/1653707353 learned_addr learned my addr 172.21.15.167:0/1653707353 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:49.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.606+0000 7fb12d400640 1 --2- 172.21.15.167:0/1653707353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb12815ac80 0x7fb1280aa840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:49.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.606+0000 7fb12de00640 1 --2- 172.21.15.167:0/1653707353 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1280b3d50 0x7fb128156a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:49.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.606+0000 7fb12e800640 1 -- 172.21.15.167:0/1653707353 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1280b3d50 msgr2=0x7fb128156a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:49.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.606+0000 7fb12e800640 1 --2- 172.21.15.167:0/1653707353 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1280b3d50 0x7fb128156a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:49.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.606+0000 7fb12e800640 1 -- 172.21.15.167:0/1653707353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb12815ac80 msgr2=0x7fb1280aa840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:49.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.606+0000 7fb12e800640 1 --2- 172.21.15.167:0/1653707353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb12815ac80 0x7fb1280aa840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:49.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.606+0000 7fb12e800640 1 -- 172.21.15.167:0/1653707353 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb128157480 con 0x7fb128161380 2024-05-11T12:11:49.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.606+0000 7fb12de00640 1 --2- 172.21.15.167:0/1653707353 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1280b3d50 0x7fb128156a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:11:49.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.606+0000 7fb12e800640 1 --2- 172.21.15.167:0/1653707353 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb128161380 0x7fb1280aad80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb124009eb0 tx=0x7fb124004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:49.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.607+0000 7fb122000640 1 -- 172.21.15.167:0/1653707353 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12403f070 con 0x7fb128161380 2024-05-11T12:11:49.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.607+0000 7fb122000640 1 -- 172.21.15.167:0/1653707353 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb124031bc0 con 0x7fb128161380 2024-05-11T12:11:49.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.607+0000 7fb122000640 1 -- 172.21.15.167:0/1653707353 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb124003990 con 0x7fb128161380 2024-05-11T12:11:49.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.607+0000 7fb130800640 1 -- 172.21.15.167:0/1653707353 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb12815c080 con 0x7fb128161380 2024-05-11T12:11:49.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.607+0000 7fb130800640 1 -- 172.21.15.167:0/1653707353 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb12815c640 con 0x7fb128161380 2024-05-11T12:11:49.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.608+0000 7fb130800640 1 -- 172.21.15.167:0/1653707353 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1280b19d0 con 0x7fb128161380 2024-05-11T12:11:49.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.609+0000 7fb122000640 1 -- 172.21.15.167:0/1653707353 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb12402ad10 con 0x7fb128161380 2024-05-11T12:11:49.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.610+0000 7fb122000640 1 --2- 172.21.15.167:0/1653707353 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb10c076300 0x7fb10c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:49.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.610+0000 7fb122000640 1 -- 172.21.15.167:0/1653707353 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5335+0+0 (secure 0 0 0) 0x7fb1240b8380 con 0x7fb128161380 2024-05-11T12:11:49.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.610+0000 7fb12de00640 1 --2- 172.21.15.167:0/1653707353 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb10c076300 0x7fb10c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:49.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.611+0000 7fb12de00640 1 --2- 172.21.15.167:0/1653707353 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb10c076300 0x7fb10c0787c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb11c00dbe0 tx=0x7fb11c00f040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:49.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.614+0000 7fb122000640 1 -- 172.21.15.167:0/1653707353 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb1280b19d0 con 0x7fb128161380 2024-05-11T12:11:49.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:49.833+0000 7fb130800640 1 -- 172.21.15.167:0/1653707353 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fb12815e570 con 0x7fb128161380 2024-05-11T12:11:50.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.337+0000 7fb122000640 1 -- 172.21.15.167:0/1653707353 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v35) ==== 97+0+0 (secure 0 0 0) 0x7fb12815e570 con 0x7fb128161380 2024-05-11T12:11:50.338 INFO:teuthology.orchestra.run.smithi167.stderr: removed all blocklist entries 2024-05-11T12:11:50.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.341+0000 7fb130800640 1 -- 172.21.15.167:0/1653707353 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb10c076300 msgr2=0x7fb10c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:50.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.341+0000 7fb130800640 1 --2- 172.21.15.167:0/1653707353 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb10c076300 0x7fb10c0787c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb11c00dbe0 tx=0x7fb11c00f040 comp rx=0 tx=0).stop 2024-05-11T12:11:50.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.341+0000 7fb130800640 1 -- 172.21.15.167:0/1653707353 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb128161380 msgr2=0x7fb1280aad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:50.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.341+0000 7fb130800640 1 --2- 172.21.15.167:0/1653707353 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb128161380 0x7fb1280aad80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb124009eb0 tx=0x7fb124004a40 comp rx=0 tx=0).stop 2024-05-11T12:11:50.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.342+0000 7fb130800640 1 -- 172.21.15.167:0/1653707353 shutdown_connections 2024-05-11T12:11:50.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.342+0000 7fb130800640 1 --2- 172.21.15.167:0/1653707353 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb128161380 0x7fb1280aad80 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:50.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.342+0000 7fb130800640 1 --2- 172.21.15.167:0/1653707353 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb10c076300 0x7fb10c0787c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:50.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.342+0000 7fb130800640 1 --2- 172.21.15.167:0/1653707353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb12815ac80 0x7fb1280aa840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:50.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.342+0000 7fb130800640 1 --2- 172.21.15.167:0/1653707353 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1280b3d50 0x7fb128156a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:50.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.342+0000 7fb130800640 1 -- 172.21.15.167:0/1653707353 >> 172.21.15.167:0/1653707353 conn(0x7fb12806c450 msgr2=0x7fb1280b22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:50.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.342+0000 7fb130800640 1 -- 172.21.15.167:0/1653707353 shutdown_connections 2024-05-11T12:11:50.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.342+0000 7fb130800640 1 -- 172.21.15.167:0/1653707353 wait complete. 2024-05-11T12:11:50.365 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-11T12:11:50.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.559+0000 7f7325c00640 1 Processor -- start 2024-05-11T12:11:50.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.559+0000 7f7325c00640 1 -- start start 2024-05-11T12:11:50.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.561+0000 7f7325c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7320157750 0x7f732015d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:50.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.561+0000 7f7325c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f73200b3ea0 0x7f7320161490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:50.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.561+0000 7f7325c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f732015dc80 con 0x7f73201619d0 2024-05-11T12:11:50.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.561+0000 7f7325c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f7320161000 con 0x7f73200b3ea0 2024-05-11T12:11:50.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.561+0000 7f7325c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f732015df30 con 0x7f7320157750 2024-05-11T12:11:50.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.561+0000 7f7316000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f73200b3ea0 0x7f7320161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:50.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.561+0000 7f731ea00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7320157750 0x7f732015d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:50.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.562+0000 7f731f400640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f73201619d0 0x7f732006d460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.167:32962/0 (socket says 172.21.15.167:32962) 2024-05-11T12:11:50.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.562+0000 7f7316000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f73200b3ea0 0x7f7320161490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.167:49350/0 (socket says 172.21.15.167:49350) 2024-05-11T12:11:50.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.562+0000 7f731f400640 1 -- 172.21.15.167:0/3105445603 learned_addr learned my addr 172.21.15.167:0/3105445603 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:50.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.562+0000 7f7316000640 1 -- 172.21.15.167:0/3105445603 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7320157750 msgr2=0x7f732015d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:50.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.562+0000 7f7316000640 1 --2- 172.21.15.167:0/3105445603 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7320157750 0x7f732015d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:50.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.562+0000 7f7316000640 1 -- 172.21.15.167:0/3105445603 >> v1:172.21.15.204:6789/0 conn(0x7f73201619d0 legacy=0x7f732006d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:50.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.562+0000 7f7316000640 1 -- 172.21.15.167:0/3105445603 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f732015e210 con 0x7f73200b3ea0 2024-05-11T12:11:50.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.562+0000 7f731ea00640 1 --2- 172.21.15.167:0/3105445603 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7320157750 0x7f732015d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:11:50.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.564+0000 7f7316000640 1 --2- 172.21.15.167:0/3105445603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73200b3ea0 0x7f7320161490 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f73080097a0 tx=0x7f730802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb72d1ff10a991e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:50.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.564+0000 7f731e000640 1 -- 172.21.15.167:0/3105445603 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7308037070 con 0x7f73200b3ea0 2024-05-11T12:11:50.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.564+0000 7f731e000640 1 -- 172.21.15.167:0/3105445603 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7308032440 con 0x7f73200b3ea0 2024-05-11T12:11:50.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.564+0000 7f731e000640 1 -- 172.21.15.167:0/3105445603 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7308032810 con 0x7f73200b3ea0 2024-05-11T12:11:50.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.565+0000 7f7325c00640 1 -- 172.21.15.167:0/3105445603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73200b3ea0 msgr2=0x7f7320161490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:50.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.565+0000 7f7325c00640 1 --2- 172.21.15.167:0/3105445603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73200b3ea0 0x7f7320161490 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f73080097a0 tx=0x7f730802a180 comp rx=0 tx=0).stop 2024-05-11T12:11:50.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.565+0000 7f7325c00640 1 -- 172.21.15.167:0/3105445603 shutdown_connections 2024-05-11T12:11:50.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.565+0000 7f7325c00640 1 --2- 172.21.15.167:0/3105445603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73200b3ea0 0x7f7320161490 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:50.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.565+0000 7f7325c00640 1 --2- 172.21.15.167:0/3105445603 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7320157750 0x7f732015d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:50.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.565+0000 7f7325c00640 1 -- 172.21.15.167:0/3105445603 >> 172.21.15.167:0/3105445603 conn(0x7f73200cd8f0 msgr2=0x7f73200cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:50.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.565+0000 7f7325c00640 1 -- 172.21.15.167:0/3105445603 shutdown_connections 2024-05-11T12:11:50.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.565+0000 7f7325c00640 1 -- 172.21.15.167:0/3105445603 wait complete. 2024-05-11T12:11:50.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.567+0000 7f7325c00640 1 Processor -- start 2024-05-11T12:11:50.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.567+0000 7f7325c00640 1 -- start start 2024-05-11T12:11:50.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.568+0000 7f7325c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73200b3ea0 0x7f732013a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:50.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.568+0000 7f7325c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7320157750 0x7f73200730c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:50.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.568+0000 7f7325c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f73201619d0 0x7f7320073600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:50.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.568+0000 7f7325c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f732006db70 con 0x7f7320157750 2024-05-11T12:11:50.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.568+0000 7f7325c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7320157b30 con 0x7f73200b3ea0 2024-05-11T12:11:50.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.568+0000 7f7325c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f732006dcf0 con 0x7f73201619d0 2024-05-11T12:11:50.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.568+0000 7f7316000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7320157750 0x7f73200730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:50.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.568+0000 7f7316000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7320157750 0x7f73200730c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:33120/0 (socket says 172.21.15.167:33120) 2024-05-11T12:11:50.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.568+0000 7f7316000640 1 -- 172.21.15.167:0/2415332829 learned_addr learned my addr 172.21.15.167:0/2415332829 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:50.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.569+0000 7f731ea00640 1 --2- 172.21.15.167:0/2415332829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73200b3ea0 0x7f732013a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:50.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.569+0000 7f731f400640 1 --2- 172.21.15.167:0/2415332829 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f73201619d0 0x7f7320073600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:50.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.569+0000 7f7316000640 1 -- 172.21.15.167:0/2415332829 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f73201619d0 msgr2=0x7f7320073600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:50.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.569+0000 7f7316000640 1 --2- 172.21.15.167:0/2415332829 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f73201619d0 0x7f7320073600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:50.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.569+0000 7f7316000640 1 -- 172.21.15.167:0/2415332829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73200b3ea0 msgr2=0x7f732013a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:50.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.569+0000 7f7316000640 1 --2- 172.21.15.167:0/2415332829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73200b3ea0 0x7f732013a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:50.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.569+0000 7f7316000640 1 -- 172.21.15.167:0/2415332829 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7320157f50 con 0x7f7320157750 2024-05-11T12:11:50.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.569+0000 7f7316000640 1 --2- 172.21.15.167:0/2415332829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7320157750 0x7f73200730c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f730802a770 tx=0x7f73080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:50.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.570+0000 7f7317400640 1 -- 172.21.15.167:0/2415332829 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7308037050 con 0x7f7320157750 2024-05-11T12:11:50.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.570+0000 7f7317400640 1 -- 172.21.15.167:0/2415332829 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7308043070 con 0x7f7320157750 2024-05-11T12:11:50.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.570+0000 7f7317400640 1 -- 172.21.15.167:0/2415332829 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730803cb30 con 0x7f7320157750 2024-05-11T12:11:50.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.570+0000 7f7325c00640 1 -- 172.21.15.167:0/2415332829 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7320158230 con 0x7f7320157750 2024-05-11T12:11:50.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.570+0000 7f7325c00640 1 -- 172.21.15.167:0/2415332829 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73201f5ed0 con 0x7f7320157750 2024-05-11T12:11:50.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.571+0000 7f7325c00640 1 -- 172.21.15.167:0/2415332829 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7320069590 con 0x7f7320157750 2024-05-11T12:11:50.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.572+0000 7f7317400640 1 -- 172.21.15.167:0/2415332829 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f730803ccd0 con 0x7f7320157750 2024-05-11T12:11:50.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.573+0000 7f7317400640 1 --2- 172.21.15.167:0/2415332829 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f72fc0762d0 0x7f72fc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:50.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.573+0000 7f7317400640 1 -- 172.21.15.167:0/2415332829 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7f73080b8140 con 0x7f7320157750 2024-05-11T12:11:50.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.573+0000 7f731ea00640 1 --2- 172.21.15.167:0/2415332829 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f72fc0762d0 0x7f72fc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:50.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.574+0000 7f731ea00640 1 --2- 172.21.15.167:0/2415332829 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f72fc0762d0 0x7f72fc078790 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7320073f00 tx=0x7f731000b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:50.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.577+0000 7f7317400640 1 -- 172.21.15.167:0/2415332829 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7308081c50 con 0x7f7320157750 2024-05-11T12:11:50.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.865+0000 7f7325c00640 1 -- 172.21.15.167:0/2415332829 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f7320069d60 con 0x7f7320157750 2024-05-11T12:11:50.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.866+0000 7f7317400640 1 -- 172.21.15.167:0/2415332829 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v8) ==== 80+0+6645 (secure 0 0 0) 0x7f7308081550 con 0x7f7320157750 2024-05-11T12:11:50.867 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:11:50.867 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T12:11:50.867 INFO:teuthology.orchestra.run.smithi167.stdout: "auth_dump": [ 2024-05-11T12:11:50.867 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:11:50.867 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "mds.a", 2024-05-11T12:11:50.867 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAEYD9msMGnFBAAamqokXQ+znxPHtHhhs611g==", 2024-05-11T12:11:50.867 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:11:50.867 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T12:11:50.868 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow *", 2024-05-11T12:11:50.868 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow *", 2024-05-11T12:11:50.868 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:11:50.868 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:11:50.868 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:11:50.868 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:11:50.868 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "mds.b", 2024-05-11T12:11:50.868 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAEYD9mqbCgMhAA5CNUf1cz77nUqLli2bTFcQ==", 2024-05-11T12:11:50.868 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:11:50.868 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T12:11:50.868 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow *", 2024-05-11T12:11:50.868 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow *", 2024-05-11T12:11:50.868 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:11:50.868 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:11:50.868 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:11:50.869 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:11:50.869 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "mds.c", 2024-05-11T12:11:50.869 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAEYD9mhylUIxAAqrIRYuNIHKfoI00zx+Gh1Q==", 2024-05-11T12:11:50.869 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:11:50.869 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T12:11:50.869 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow *", 2024-05-11T12:11:50.869 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow *", 2024-05-11T12:11:50.869 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:11:50.869 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:11:50.869 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:11:50.869 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:11:50.869 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "mds.d", 2024-05-11T12:11:50.869 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAFYD9mcUjpBRAAbx54iDHcqexfNXGKk/avlw==", 2024-05-11T12:11:50.869 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:11:50.870 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T12:11:50.870 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow *", 2024-05-11T12:11:50.870 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow *", 2024-05-11T12:11:50.870 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:11:50.870 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:11:50.870 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:11:50.870 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:11:50.870 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.0", 2024-05-11T12:11:50.870 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAJYD9mb71mBxAAKMSzkfppZVbEevwELKrqvw==", 2024-05-11T12:11:50.870 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:11:50.870 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T12:11:50.870 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T12:11:50.870 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:11:50.870 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:11:50.871 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:11:50.871 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:11:50.871 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.1", 2024-05-11T12:11:50.871 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAMYD9m/qx7EhAAUcZntottFNBwqjdc/CqFbw==", 2024-05-11T12:11:50.871 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:11:50.871 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T12:11:50.871 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T12:11:50.871 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:11:50.871 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:11:50.871 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:11:50.871 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:11:50.871 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.2", 2024-05-11T12:11:50.871 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAPYD9m5PKRKxAAFQuzh7dAN8PVEHBq+UdQvg==", 2024-05-11T12:11:50.871 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:11:50.871 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T12:11:50.871 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T12:11:50.872 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:11:50.872 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:11:50.872 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:11:50.872 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:11:50.872 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.3", 2024-05-11T12:11:50.872 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQASYD9mZAy+NhAA49k6QbSvtSxA/5t8bWbPuQ==", 2024-05-11T12:11:50.872 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:11:50.872 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T12:11:50.872 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T12:11:50.872 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:11:50.872 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:11:50.872 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:11:50.872 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:11:50.872 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.4", 2024-05-11T12:11:50.872 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAZYD9mEbsvIBAA3nkQXQTbIf7ubq1SJtUeIA==", 2024-05-11T12:11:50.872 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:11:50.873 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T12:11:50.873 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T12:11:50.873 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:11:50.873 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:11:50.873 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:11:50.873 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:11:50.873 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.5", 2024-05-11T12:11:50.873 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAcYD9mtdsHOxAAvNUSkMgQlDc2n1ak/lfRgg==", 2024-05-11T12:11:50.873 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:11:50.873 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T12:11:50.873 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T12:11:50.873 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:11:50.873 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:11:50.873 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:11:50.873 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:11:50.873 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.6", 2024-05-11T12:11:50.874 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAgYD9mr3u8CRAAm2ShwV0lJpIVudodmviusg==", 2024-05-11T12:11:50.874 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:11:50.874 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T12:11:50.874 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T12:11:50.874 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:11:50.874 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:11:50.874 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:11:50.874 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:11:50.874 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.7", 2024-05-11T12:11:50.874 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAjYD9m+0oyFRAAUVQW0kQmvtU8IE599nrWJQ==", 2024-05-11T12:11:50.874 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:11:50.874 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T12:11:50.874 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T12:11:50.874 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:11:50.874 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:11:50.875 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:11:50.875 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:11:50.875 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "client.0", 2024-05-11T12:11:50.875 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAFYD9mLK/lEhAAyERTYMl250KkBvIrAXWYSw==", 2024-05-11T12:11:50.875 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:11:50.875 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T12:11:50.875 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow r", 2024-05-11T12:11:50.875 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow rw", 2024-05-11T12:11:50.875 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow rwx" 2024-05-11T12:11:50.875 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:11:50.875 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:11:50.875 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:11:50.875 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "client.1", 2024-05-11T12:11:50.875 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAFYD9m2oSjJhAAz53H1SvPStkxGq5V+2VFeA==", 2024-05-11T12:11:50.875 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:11:50.875 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T12:11:50.876 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow r", 2024-05-11T12:11:50.876 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow rw", 2024-05-11T12:11:50.876 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow rwx" 2024-05-11T12:11:50.876 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:11:50.876 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:11:50.876 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:11:50.876 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "client.2", 2024-05-11T12:11:50.876 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAFYD9mMzXnHBAAmJ//hwSofaZpk+6ykxVM/A==", 2024-05-11T12:11:50.876 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:11:50.876 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T12:11:50.876 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow r", 2024-05-11T12:11:50.876 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow rw", 2024-05-11T12:11:50.876 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow rwx" 2024-05-11T12:11:50.876 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:11:50.876 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:11:50.877 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:11:50.877 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "client.3", 2024-05-11T12:11:50.877 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAFYD9muOmsLxAA/tYFT4YNy3adENgVNLljdw==", 2024-05-11T12:11:50.877 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:11:50.877 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T12:11:50.877 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow r", 2024-05-11T12:11:50.877 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow rw", 2024-05-11T12:11:50.877 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow rwx" 2024-05-11T12:11:50.877 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:11:50.877 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:11:50.877 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:11:50.877 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "client.admin", 2024-05-11T12:11:50.877 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQADYD9mItegDBAALtuDP6qzAvbrwiROf94biw==", 2024-05-11T12:11:50.877 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:11:50.878 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow *", 2024-05-11T12:11:50.878 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow *", 2024-05-11T12:11:50.878 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow *", 2024-05-11T12:11:50.878 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:11:50.878 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:11:50.878 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:11:50.878 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:11:50.878 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "client.bootstrap-mds", 2024-05-11T12:11:50.878 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAsYD9mshszAhAA9QMWwPkCHQJeMCyaO/AftQ==", 2024-05-11T12:11:50.878 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:11:50.878 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile bootstrap-mds" 2024-05-11T12:11:50.878 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:11:50.878 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:11:50.878 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:11:50.879 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "client.bootstrap-mgr", 2024-05-11T12:11:50.879 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAsYD9mLzczAhAArx5Pxt+7axf+ooydcjxYdQ==", 2024-05-11T12:11:50.879 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:11:50.879 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile bootstrap-mgr" 2024-05-11T12:11:50.879 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:11:50.879 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:11:50.879 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:11:50.879 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "client.bootstrap-osd", 2024-05-11T12:11:50.879 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAsYD9mG1AzAhAAQzXJzgU6niaGwMndIBgbQg==", 2024-05-11T12:11:50.879 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:11:50.879 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile bootstrap-osd" 2024-05-11T12:11:50.879 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:11:50.879 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:11:50.879 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:11:50.879 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "client.bootstrap-rbd", 2024-05-11T12:11:50.880 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAsYD9m3mgzAhAAqetfYyIrBO2XMl3M/nhdMg==", 2024-05-11T12:11:50.880 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:11:50.880 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile bootstrap-rbd" 2024-05-11T12:11:50.880 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:11:50.880 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:11:50.880 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:11:50.880 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-05-11T12:11:50.880 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAsYD9mTIEzAhAABkSefs0GljEW022FCrOykQ==", 2024-05-11T12:11:50.880 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:11:50.880 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-05-11T12:11:50.880 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:11:50.880 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:11:50.880 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:11:50.880 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "client.bootstrap-rgw", 2024-05-11T12:11:50.880 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAsYD9mkZozAhAAHZ7W5DD/7hp+on9/ogXZtg==", 2024-05-11T12:11:50.881 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:11:50.881 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile bootstrap-rgw" 2024-05-11T12:11:50.881 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:11:50.881 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:11:50.881 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:11:50.881 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "mgr.x", 2024-05-11T12:11:50.881 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAEYD9mAZTKCRAAJbY821O+4pUMC0DgxRa3og==", 2024-05-11T12:11:50.881 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:11:50.881 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow *", 2024-05-11T12:11:50.881 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile mgr", 2024-05-11T12:11:50.881 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:11:50.881 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:11:50.881 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:11:50.881 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:11:50.881 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "mgr.y", 2024-05-11T12:11:50.881 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQADYD9m2cZRORAAX9kYcT5/t4f5jQRZwXHHzw==", 2024-05-11T12:11:50.882 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:11:50.882 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow *", 2024-05-11T12:11:50.882 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile mgr", 2024-05-11T12:11:50.882 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:11:50.882 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:11:50.882 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:11:50.882 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-05-11T12:11:50.882 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T12:11:50.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.869+0000 7f7325c00640 1 -- 172.21.15.167:0/2415332829 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f72fc0762d0 msgr2=0x7f72fc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:50.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.869+0000 7f7325c00640 1 --2- 172.21.15.167:0/2415332829 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f72fc0762d0 0x7f72fc078790 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7320073f00 tx=0x7f731000b040 comp rx=0 tx=0).stop 2024-05-11T12:11:50.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.869+0000 7f7325c00640 1 -- 172.21.15.167:0/2415332829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7320157750 msgr2=0x7f73200730c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:50.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.869+0000 7f7325c00640 1 --2- 172.21.15.167:0/2415332829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7320157750 0x7f73200730c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f730802a770 tx=0x7f73080046c0 comp rx=0 tx=0).stop 2024-05-11T12:11:50.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.869+0000 7f7325c00640 1 -- 172.21.15.167:0/2415332829 shutdown_connections 2024-05-11T12:11:50.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.869+0000 7f7325c00640 1 --2- 172.21.15.167:0/2415332829 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f73201619d0 0x7f7320073600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:50.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.869+0000 7f7325c00640 1 --2- 172.21.15.167:0/2415332829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7320157750 0x7f73200730c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:50.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.869+0000 7f7325c00640 1 --2- 172.21.15.167:0/2415332829 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f72fc0762d0 0x7f72fc078790 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:50.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.869+0000 7f7325c00640 1 --2- 172.21.15.167:0/2415332829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73200b3ea0 0x7f732013a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:50.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.869+0000 7f7325c00640 1 -- 172.21.15.167:0/2415332829 >> 172.21.15.167:0/2415332829 conn(0x7f73200cd8f0 msgr2=0x7f73200cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:50.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.870+0000 7f7325c00640 1 -- 172.21.15.167:0/2415332829 shutdown_connections 2024-05-11T12:11:50.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:50.870+0000 7f7325c00640 1 -- 172.21.15.167:0/2415332829 wait complete. 2024-05-11T12:11:50.889 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-05-11T12:11:51.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.082+0000 7f3665400640 1 Processor -- start 2024-05-11T12:11:51.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.082+0000 7f3665400640 1 -- start start 2024-05-11T12:11:51.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.083+0000 7f3665400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f366015d320 0x7f366006cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:51.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.083+0000 7f3665400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f36600d1710 0x7f366015d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:51.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.083+0000 7f3665400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f366006d760 con 0x7f366006d2c0 2024-05-11T12:11:51.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.083+0000 7f3665400640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f366006da40 con 0x7f36600d1710 2024-05-11T12:11:51.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.083+0000 7f3665400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f36600b1e10 con 0x7f366015d320 2024-05-11T12:11:51.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.084+0000 7f365f200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f36600d1710 0x7f366015d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:51.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.084+0000 7f365f200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f36600d1710 0x7f366015d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:33130/0 (socket says 172.21.15.167:33130) 2024-05-11T12:11:51.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.084+0000 7f365f200640 1 -- 172.21.15.167:0/476061941 learned_addr learned my addr 172.21.15.167:0/476061941 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:51.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.084+0000 7f365e800640 1 --2- 172.21.15.167:0/476061941 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f366015d320 0x7f366006cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:51.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.084+0000 7f365f200640 1 -- 172.21.15.167:0/476061941 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f366015d320 msgr2=0x7f366006cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:51.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.084+0000 7f365f200640 1 --2- 172.21.15.167:0/476061941 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f366015d320 0x7f366006cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:51.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.084+0000 7f365f200640 1 -- 172.21.15.167:0/476061941 >> v1:172.21.15.204:6789/0 conn(0x7f366006d2c0 legacy=0x7f36600d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:51.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.084+0000 7f365f200640 1 -- 172.21.15.167:0/476061941 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36600b20f0 con 0x7f36600d1710 2024-05-11T12:11:51.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.084+0000 7f365e800640 1 --2- 172.21.15.167:0/476061941 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f366015d320 0x7f366006cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:51.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.086+0000 7f365f200640 1 --2- 172.21.15.167:0/476061941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36600d1710 0x7f366015d700 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3658009e50 tx=0x7f365802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5483d225e2e2135 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:51.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.086+0000 7f365de00640 1 -- 172.21.15.167:0/476061941 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3658036070 con 0x7f36600d1710 2024-05-11T12:11:51.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.086+0000 7f365de00640 1 -- 172.21.15.167:0/476061941 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3658031600 con 0x7f36600d1710 2024-05-11T12:11:51.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.086+0000 7f365de00640 1 -- 172.21.15.167:0/476061941 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36580319d0 con 0x7f36600d1710 2024-05-11T12:11:51.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.086+0000 7f3665400640 1 -- 172.21.15.167:0/476061941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36600d1710 msgr2=0x7f366015d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:51.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.086+0000 7f3665400640 1 --2- 172.21.15.167:0/476061941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36600d1710 0x7f366015d700 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3658009e50 tx=0x7f365802a680 comp rx=0 tx=0).stop 2024-05-11T12:11:51.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.087+0000 7f3665400640 1 -- 172.21.15.167:0/476061941 shutdown_connections 2024-05-11T12:11:51.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.087+0000 7f3665400640 1 --2- 172.21.15.167:0/476061941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36600d1710 0x7f366015d700 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:51.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.087+0000 7f3665400640 1 --2- 172.21.15.167:0/476061941 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f366015d320 0x7f366006cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:51.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.087+0000 7f3665400640 1 -- 172.21.15.167:0/476061941 >> 172.21.15.167:0/476061941 conn(0x7f36600b5e20 msgr2=0x7f36600b64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:51.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.087+0000 7f3665400640 1 -- 172.21.15.167:0/476061941 shutdown_connections 2024-05-11T12:11:51.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.087+0000 7f3665400640 1 -- 172.21.15.167:0/476061941 wait complete. 2024-05-11T12:11:51.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.089+0000 7f3665400640 1 Processor -- start 2024-05-11T12:11:51.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.089+0000 7f3665400640 1 -- start start 2024-05-11T12:11:51.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.090+0000 7f3665400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f366006d2c0 0x7f366014c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:51.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.090+0000 7f3665400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36600d1710 0x7f366015fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:51.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.090+0000 7f3665400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f366015d320 0x7f3660156fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:51.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.090+0000 7f3665400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f36600b6ec0 con 0x7f366015d320 2024-05-11T12:11:51.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.090+0000 7f3665400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f36600b6d40 con 0x7f366006d2c0 2024-05-11T12:11:51.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.090+0000 7f3665400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f36600b28f0 con 0x7f36600d1710 2024-05-11T12:11:51.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.090+0000 7f365f200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f366015d320 0x7f3660156fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:51.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.090+0000 7f365f200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f366015d320 0x7f3660156fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:33134/0 (socket says 172.21.15.167:33134) 2024-05-11T12:11:51.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.090+0000 7f365f200640 1 -- 172.21.15.167:0/3823521552 learned_addr learned my addr 172.21.15.167:0/3823521552 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:51.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.090+0000 7f365e800640 1 --2- 172.21.15.167:0/3823521552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f366006d2c0 0x7f366014c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:51.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.090+0000 7f3655e00640 1 --2- 172.21.15.167:0/3823521552 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36600d1710 0x7f366015fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:51.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.091+0000 7f365f200640 1 -- 172.21.15.167:0/3823521552 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36600d1710 msgr2=0x7f366015fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:51.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.091+0000 7f365f200640 1 --2- 172.21.15.167:0/3823521552 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36600d1710 0x7f366015fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:51.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.091+0000 7f365f200640 1 -- 172.21.15.167:0/3823521552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f366006d2c0 msgr2=0x7f366014c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:51.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.091+0000 7f365f200640 1 --2- 172.21.15.167:0/3823521552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f366006d2c0 0x7f366014c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:51.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.091+0000 7f365f200640 1 -- 172.21.15.167:0/3823521552 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f366015e960 con 0x7f366015d320 2024-05-11T12:11:51.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.091+0000 7f365f200640 1 --2- 172.21.15.167:0/3823521552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f366015d320 0x7f3660156fb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3658034040 tx=0x7f3658004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:51.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.091+0000 7f3657400640 1 -- 172.21.15.167:0/3823521552 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3658036050 con 0x7f366015d320 2024-05-11T12:11:51.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.091+0000 7f3657400640 1 -- 172.21.15.167:0/3823521552 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3658042070 con 0x7f366015d320 2024-05-11T12:11:51.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.091+0000 7f3665400640 1 -- 172.21.15.167:0/3823521552 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f366015ec40 con 0x7f366015d320 2024-05-11T12:11:51.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.091+0000 7f3657400640 1 -- 172.21.15.167:0/3823521552 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3658009510 con 0x7f366015d320 2024-05-11T12:11:51.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.092+0000 7f3665400640 1 -- 172.21.15.167:0/3823521552 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f366015f200 con 0x7f366015d320 2024-05-11T12:11:51.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.093+0000 7f3665400640 1 -- 172.21.15.167:0/3823521552 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3660069bc0 con 0x7f366015d320 2024-05-11T12:11:51.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.094+0000 7f3657400640 1 -- 172.21.15.167:0/3823521552 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3658007bd0 con 0x7f366015d320 2024-05-11T12:11:51.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.095+0000 7f3657400640 1 --2- 172.21.15.167:0/3823521552 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f363c0762d0 0x7f363c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:51.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.095+0000 7f3657400640 1 -- 172.21.15.167:0/3823521552 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7f36580b8370 con 0x7f366015d320 2024-05-11T12:11:51.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.095+0000 7f365e800640 1 --2- 172.21.15.167:0/3823521552 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f363c0762d0 0x7f363c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:51.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.096+0000 7f365e800640 1 --2- 172.21.15.167:0/3823521552 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f363c0762d0 0x7f363c078790 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f36600d0730 tx=0x7f3650008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:51.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.099+0000 7f3657400640 1 -- 172.21.15.167:0/3823521552 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3658081e80 con 0x7f366015d320 2024-05-11T12:11:51.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.390+0000 7f3665400640 1 -- 172.21.15.167:0/3823521552 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f3660154b50 con 0x7f366015d320 2024-05-11T12:11:51.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.394+0000 7f3657400640 1 -- 172.21.15.167:0/3823521552 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v9) ==== 90+0+0 (secure 0 0 0) 0x7f3658088e20 con 0x7f366015d320 2024-05-11T12:11:51.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.397+0000 7f3665400640 1 -- 172.21.15.167:0/3823521552 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f363c0762d0 msgr2=0x7f363c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:51.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.397+0000 7f3665400640 1 --2- 172.21.15.167:0/3823521552 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f363c0762d0 0x7f363c078790 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f36600d0730 tx=0x7f3650008040 comp rx=0 tx=0).stop 2024-05-11T12:11:51.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.397+0000 7f3665400640 1 -- 172.21.15.167:0/3823521552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f366015d320 msgr2=0x7f3660156fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:51.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.397+0000 7f3665400640 1 --2- 172.21.15.167:0/3823521552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f366015d320 0x7f3660156fb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3658034040 tx=0x7f3658004a40 comp rx=0 tx=0).stop 2024-05-11T12:11:51.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.397+0000 7f3665400640 1 -- 172.21.15.167:0/3823521552 shutdown_connections 2024-05-11T12:11:51.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.397+0000 7f3665400640 1 --2- 172.21.15.167:0/3823521552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f366015d320 0x7f3660156fb0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:51.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.397+0000 7f3665400640 1 --2- 172.21.15.167:0/3823521552 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36600d1710 0x7f366015fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:51.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.397+0000 7f3665400640 1 --2- 172.21.15.167:0/3823521552 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f363c0762d0 0x7f363c078790 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:51.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.397+0000 7f3665400640 1 --2- 172.21.15.167:0/3823521552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f366006d2c0 0x7f366014c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:51.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.397+0000 7f3665400640 1 -- 172.21.15.167:0/3823521552 >> 172.21.15.167:0/3823521552 conn(0x7f36600b5e20 msgr2=0x7f366015c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:51.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.397+0000 7f3665400640 1 -- 172.21.15.167:0/3823521552 shutdown_connections 2024-05-11T12:11:51.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.397+0000 7f3665400640 1 -- 172.21.15.167:0/3823521552 wait complete. 2024-05-11T12:11:51.417 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-05-11T12:11:51.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.613+0000 7fb843200640 1 Processor -- start 2024-05-11T12:11:51.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.613+0000 7fb843200640 1 -- start start 2024-05-11T12:11:51.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.614+0000 7fb843200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb84415f2e0 0x7fb8440d1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:51.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.614+0000 7fb843200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb84406c980 0x7fb8440b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:51.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.614+0000 7fb843200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb84415f6c0 con 0x7fb8440b8840 2024-05-11T12:11:51.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.614+0000 7fb843200640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fb8440d2c30 con 0x7fb84406c980 2024-05-11T12:11:51.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.614+0000 7fb843200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb8440d2e80 con 0x7fb84415f2e0 2024-05-11T12:11:51.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.615+0000 7fb841400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb84406c980 0x7fb8440b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:51.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.615+0000 7fb841400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb84406c980 0x7fb8440b8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:33144/0 (socket says 172.21.15.167:33144) 2024-05-11T12:11:51.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.615+0000 7fb841400640 1 -- 172.21.15.167:0/779280950 learned_addr learned my addr 172.21.15.167:0/779280950 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:51.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.615+0000 7fb841400640 1 -- 172.21.15.167:0/779280950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb84415f2e0 msgr2=0x7fb8440d1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:51.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.615+0000 7fb841e00640 1 --2- 172.21.15.167:0/779280950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb84415f2e0 0x7fb8440d1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:51.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.615+0000 7fb841400640 1 --2- 172.21.15.167:0/779280950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb84415f2e0 0x7fb8440d1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:51.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.615+0000 7fb841400640 1 -- 172.21.15.167:0/779280950 >> v1:172.21.15.204:6789/0 conn(0x7fb8440b8840 legacy=0x7fb8440d3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:51.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.615+0000 7fb841400640 1 -- 172.21.15.167:0/779280950 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8440d3160 con 0x7fb84406c980 2024-05-11T12:11:51.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.615+0000 7fb841e00640 1 --2- 172.21.15.167:0/779280950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb84415f2e0 0x7fb8440d1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:51.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.616+0000 7fb841400640 1 --2- 172.21.15.167:0/779280950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb84406c980 0x7fb8440b8300 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb8440d2db0 tx=0x7fb83002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7676f21d1c803ae server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:51.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.617+0000 7fb840a00640 1 -- 172.21.15.167:0/779280950 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb830037070 con 0x7fb84406c980 2024-05-11T12:11:51.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.617+0000 7fb840a00640 1 -- 172.21.15.167:0/779280950 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb830032440 con 0x7fb84406c980 2024-05-11T12:11:51.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.617+0000 7fb840a00640 1 -- 172.21.15.167:0/779280950 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83002ae60 con 0x7fb84406c980 2024-05-11T12:11:51.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.617+0000 7fb843200640 1 -- 172.21.15.167:0/779280950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb84406c980 msgr2=0x7fb8440b8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:51.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.617+0000 7fb843200640 1 --2- 172.21.15.167:0/779280950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb84406c980 0x7fb8440b8300 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb8440d2db0 tx=0x7fb83002a110 comp rx=0 tx=0).stop 2024-05-11T12:11:51.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.617+0000 7fb843200640 1 -- 172.21.15.167:0/779280950 shutdown_connections 2024-05-11T12:11:51.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.617+0000 7fb843200640 1 --2- 172.21.15.167:0/779280950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb84406c980 0x7fb8440b8300 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:51.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.617+0000 7fb843200640 1 --2- 172.21.15.167:0/779280950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb84415f2e0 0x7fb8440d1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:51.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.617+0000 7fb843200640 1 -- 172.21.15.167:0/779280950 >> 172.21.15.167:0/779280950 conn(0x7fb8440d5060 msgr2=0x7fb8440d5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:51.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.618+0000 7fb843200640 1 -- 172.21.15.167:0/779280950 shutdown_connections 2024-05-11T12:11:51.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.618+0000 7fb843200640 1 -- 172.21.15.167:0/779280950 wait complete. 2024-05-11T12:11:51.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.619+0000 7fb843200640 1 Processor -- start 2024-05-11T12:11:51.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.619+0000 7fb843200640 1 -- start start 2024-05-11T12:11:51.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.620+0000 7fb843200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb84406c980 0x7fb8441607d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:51.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.620+0000 7fb843200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8440b8840 0x7fb8441e8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:51.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.620+0000 7fb843200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb84415f2e0 0x7fb8441e8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:51.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.620+0000 7fb843200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb84406ce10 con 0x7fb84415f2e0 2024-05-11T12:11:51.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.620+0000 7fb843200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb8440d3720 con 0x7fb8440b8840 2024-05-11T12:11:51.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.620+0000 7fb843200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb8440d4350 con 0x7fb84406c980 2024-05-11T12:11:51.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.621+0000 7fb842800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb84415f2e0 0x7fb8441e8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:51.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.621+0000 7fb842800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb84415f2e0 0x7fb8441e8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:33160/0 (socket says 172.21.15.167:33160) 2024-05-11T12:11:51.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.621+0000 7fb842800640 1 -- 172.21.15.167:0/1132099623 learned_addr learned my addr 172.21.15.167:0/1132099623 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:51.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.621+0000 7fb841e00640 1 --2- 172.21.15.167:0/1132099623 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb84406c980 0x7fb8441607d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:51.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.621+0000 7fb841400640 1 --2- 172.21.15.167:0/1132099623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8440b8840 0x7fb8441e8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:51.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.621+0000 7fb842800640 1 -- 172.21.15.167:0/1132099623 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb84406c980 msgr2=0x7fb8441607d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:51.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.621+0000 7fb842800640 1 --2- 172.21.15.167:0/1132099623 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb84406c980 0x7fb8441607d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:51.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.621+0000 7fb842800640 1 -- 172.21.15.167:0/1132099623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8440b8840 msgr2=0x7fb8441e8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:51.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.621+0000 7fb842800640 1 --2- 172.21.15.167:0/1132099623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8440b8840 0x7fb8441e8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:51.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.621+0000 7fb842800640 1 -- 172.21.15.167:0/1132099623 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8440ce3d0 con 0x7fb84415f2e0 2024-05-11T12:11:51.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.621+0000 7fb841400640 1 --2- 172.21.15.167:0/1132099623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8440b8840 0x7fb8441e8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:51.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.621+0000 7fb842800640 1 --2- 172.21.15.167:0/1132099623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb84415f2e0 0x7fb8441e8ea0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb83c00c780 tx=0x7fb83c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:51.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.622+0000 7fb83a000640 1 -- 172.21.15.167:0/1132099623 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c004b60 con 0x7fb84415f2e0 2024-05-11T12:11:51.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.622+0000 7fb83a000640 1 -- 172.21.15.167:0/1132099623 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb83c004d00 con 0x7fb84415f2e0 2024-05-11T12:11:51.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.622+0000 7fb843200640 1 -- 172.21.15.167:0/1132099623 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8440ce6b0 con 0x7fb84415f2e0 2024-05-11T12:11:51.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.622+0000 7fb83a000640 1 -- 172.21.15.167:0/1132099623 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c00aa90 con 0x7fb84415f2e0 2024-05-11T12:11:51.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.622+0000 7fb843200640 1 -- 172.21.15.167:0/1132099623 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8440cec70 con 0x7fb84415f2e0 2024-05-11T12:11:51.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.623+0000 7fb843200640 1 -- 172.21.15.167:0/1132099623 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8440694b0 con 0x7fb84415f2e0 2024-05-11T12:11:51.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.624+0000 7fb83a000640 1 -- 172.21.15.167:0/1132099623 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb83c00ac30 con 0x7fb84415f2e0 2024-05-11T12:11:51.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.625+0000 7fb83a000640 1 --2- 172.21.15.167:0/1132099623 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb8240762d0 0x7fb824078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:51.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.625+0000 7fb83a000640 1 -- 172.21.15.167:0/1132099623 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7fb83c0976d0 con 0x7fb84415f2e0 2024-05-11T12:11:51.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.625+0000 7fb841e00640 1 --2- 172.21.15.167:0/1132099623 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb8240762d0 0x7fb824078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:51.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.626+0000 7fb841e00640 1 --2- 172.21.15.167:0/1132099623 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb8240762d0 0x7fb824078790 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb83400b840 tx=0x7fb834008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:51.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.630+0000 7fb83a000640 1 -- 172.21.15.167:0/1132099623 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb83c020070 con 0x7fb84415f2e0 2024-05-11T12:11:51.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.909+0000 7fb843200640 1 -- 172.21.15.167:0/1132099623 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7fb84406ed60 con 0x7fb84415f2e0 2024-05-11T12:11:51.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.914+0000 7fb83a000640 1 -- 172.21.15.167:0/1132099623 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7fb83c0611e0 con 0x7fb84415f2e0 2024-05-11T12:11:51.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.917+0000 7fb843200640 1 -- 172.21.15.167:0/1132099623 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb8240762d0 msgr2=0x7fb824078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:51.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.917+0000 7fb843200640 1 --2- 172.21.15.167:0/1132099623 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb8240762d0 0x7fb824078790 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb83400b840 tx=0x7fb834008040 comp rx=0 tx=0).stop 2024-05-11T12:11:51.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.917+0000 7fb843200640 1 -- 172.21.15.167:0/1132099623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb84415f2e0 msgr2=0x7fb8441e8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:51.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.917+0000 7fb843200640 1 --2- 172.21.15.167:0/1132099623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb84415f2e0 0x7fb8441e8ea0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb83c00c780 tx=0x7fb83c00cc40 comp rx=0 tx=0).stop 2024-05-11T12:11:51.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.917+0000 7fb843200640 1 -- 172.21.15.167:0/1132099623 shutdown_connections 2024-05-11T12:11:51.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.917+0000 7fb843200640 1 --2- 172.21.15.167:0/1132099623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb84415f2e0 0x7fb8441e8ea0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:51.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.917+0000 7fb843200640 1 --2- 172.21.15.167:0/1132099623 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb8240762d0 0x7fb824078790 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:51.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.917+0000 7fb843200640 1 --2- 172.21.15.167:0/1132099623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8440b8840 0x7fb8441e8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:51.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.917+0000 7fb843200640 1 --2- 172.21.15.167:0/1132099623 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb84406c980 0x7fb8441607d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:51.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.917+0000 7fb843200640 1 -- 172.21.15.167:0/1132099623 >> 172.21.15.167:0/1132099623 conn(0x7fb8440d5060 msgr2=0x7fb8440b4ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:51.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.917+0000 7fb843200640 1 -- 172.21.15.167:0/1132099623 shutdown_connections 2024-05-11T12:11:51.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:51.917+0000 7fb843200640 1 -- 172.21.15.167:0/1132099623 wait complete. 2024-05-11T12:11:51.934 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-05-11T12:11:52.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.120+0000 7f3992c00640 1 Processor -- start 2024-05-11T12:11:52.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.121+0000 7f3992c00640 1 -- start start 2024-05-11T12:11:52.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.122+0000 7f3992c00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f398c15ac80 0x7f398c074260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:52.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.122+0000 7f3992c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f398c161380 0x7f398c072950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:52.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.122+0000 7f3992c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f398c0b65e0 con 0x7f398c0b3d50 2024-05-11T12:11:52.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.122+0000 7f3992c00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f398c06e3c0 con 0x7f398c15ac80 2024-05-11T12:11:52.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.122+0000 7f3992c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f398c15b060 con 0x7f398c161380 2024-05-11T12:11:52.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.122+0000 7f398be00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f398c15ac80 0x7f398c074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:52.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.123+0000 7f398be00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f398c15ac80 0x7f398c074260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:33172/0 (socket says 172.21.15.167:33172) 2024-05-11T12:11:52.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.123+0000 7f398be00640 1 -- 172.21.15.167:0/364418255 learned_addr learned my addr 172.21.15.167:0/364418255 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:52.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.123+0000 7f398be00640 1 -- 172.21.15.167:0/364418255 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f398c161380 msgr2=0x7f398c072950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:52.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.123+0000 7f398be00640 1 --2- 172.21.15.167:0/364418255 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f398c161380 0x7f398c072950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:52.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.123+0000 7f398be00640 1 -- 172.21.15.167:0/364418255 >> v1:172.21.15.204:6789/0 conn(0x7f398c0b3d50 legacy=0x7f398c160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:52.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.123+0000 7f398be00640 1 -- 172.21.15.167:0/364418255 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f398c0b68c0 con 0x7f398c15ac80 2024-05-11T12:11:52.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.124+0000 7f398be00640 1 --2- 172.21.15.167:0/364418255 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f398c15ac80 0x7f398c074260 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f397c0098a0 tx=0x7f397c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=102d0bb13a014668 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:52.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.124+0000 7f398aa00640 1 -- 172.21.15.167:0/364418255 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c037070 con 0x7f398c15ac80 2024-05-11T12:11:52.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.124+0000 7f398aa00640 1 -- 172.21.15.167:0/364418255 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f397c032440 con 0x7f398c15ac80 2024-05-11T12:11:52.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.124+0000 7f398aa00640 1 -- 172.21.15.167:0/364418255 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c032810 con 0x7f398c15ac80 2024-05-11T12:11:52.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.125+0000 7f3992c00640 1 -- 172.21.15.167:0/364418255 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f398c15ac80 msgr2=0x7f398c074260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:52.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.125+0000 7f3992c00640 1 --2- 172.21.15.167:0/364418255 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f398c15ac80 0x7f398c074260 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f397c0098a0 tx=0x7f397c02a210 comp rx=0 tx=0).stop 2024-05-11T12:11:52.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.125+0000 7f3992c00640 1 -- 172.21.15.167:0/364418255 shutdown_connections 2024-05-11T12:11:52.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.125+0000 7f3992c00640 1 --2- 172.21.15.167:0/364418255 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f398c161380 0x7f398c072950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:52.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.125+0000 7f3992c00640 1 --2- 172.21.15.167:0/364418255 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f398c15ac80 0x7f398c074260 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:52.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.125+0000 7f3992c00640 1 -- 172.21.15.167:0/364418255 >> 172.21.15.167:0/364418255 conn(0x7f398c06c450 msgr2=0x7f398c06c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:52.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.125+0000 7f3992c00640 1 -- 172.21.15.167:0/364418255 shutdown_connections 2024-05-11T12:11:52.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.125+0000 7f3992c00640 1 -- 172.21.15.167:0/364418255 wait complete. 2024-05-11T12:11:52.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.127+0000 7f3992c00640 1 Processor -- start 2024-05-11T12:11:52.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.127+0000 7f3992c00640 1 -- start start 2024-05-11T12:11:52.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.129+0000 7f3992c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f398c0b3d50 0x7f398c06caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:52.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.129+0000 7f3992c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f398c15ac80 0x7f398c1595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:52.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.129+0000 7f3992c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f398c161380 0x7f398c159af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:52.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.129+0000 7f3992c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f398c0b44f0 con 0x7f398c161380 2024-05-11T12:11:52.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.129+0000 7f3992c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f398c0b41e0 con 0x7f398c15ac80 2024-05-11T12:11:52.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.129+0000 7f3992c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f398c072e90 con 0x7f398c0b3d50 2024-05-11T12:11:52.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.129+0000 7f3990c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f398c161380 0x7f398c159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:52.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.129+0000 7f3990c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f398c161380 0x7f398c159af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:33188/0 (socket says 172.21.15.167:33188) 2024-05-11T12:11:52.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.129+0000 7f3990c00640 1 -- 172.21.15.167:0/488383869 learned_addr learned my addr 172.21.15.167:0/488383869 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:52.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.129+0000 7f398be00640 1 --2- 172.21.15.167:0/488383869 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f398c0b3d50 0x7f398c06caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:52.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.129+0000 7f398b400640 1 --2- 172.21.15.167:0/488383869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f398c15ac80 0x7f398c1595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:52.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.129+0000 7f3990c00640 1 -- 172.21.15.167:0/488383869 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f398c0b3d50 msgr2=0x7f398c06caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:52.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.129+0000 7f3990c00640 1 --2- 172.21.15.167:0/488383869 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f398c0b3d50 0x7f398c06caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:52.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.129+0000 7f3990c00640 1 -- 172.21.15.167:0/488383869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f398c15ac80 msgr2=0x7f398c1595b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:52.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.129+0000 7f3990c00640 1 --2- 172.21.15.167:0/488383869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f398c15ac80 0x7f398c1595b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:52.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.129+0000 7f3990c00640 1 -- 172.21.15.167:0/488383869 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f398c1432d0 con 0x7f398c161380 2024-05-11T12:11:52.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.130+0000 7f398b400640 1 --2- 172.21.15.167:0/488383869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f398c15ac80 0x7f398c1595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:52.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.130+0000 7f3990c00640 1 --2- 172.21.15.167:0/488383869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f398c161380 0x7f398c159af0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f398400b9e0 tx=0x7f398400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:52.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.130+0000 7f3983e00640 1 -- 172.21.15.167:0/488383869 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3984019070 con 0x7f398c161380 2024-05-11T12:11:52.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.130+0000 7f3983e00640 1 -- 172.21.15.167:0/488383869 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f398400cb40 con 0x7f398c161380 2024-05-11T12:11:52.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.130+0000 7f3983e00640 1 -- 172.21.15.167:0/488383869 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3984014440 con 0x7f398c161380 2024-05-11T12:11:52.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.130+0000 7f3992c00640 1 -- 172.21.15.167:0/488383869 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f398c1435b0 con 0x7f398c161380 2024-05-11T12:11:52.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.130+0000 7f3992c00640 1 -- 172.21.15.167:0/488383869 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f398c143b70 con 0x7f398c161380 2024-05-11T12:11:52.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.132+0000 7f3992c00640 1 -- 172.21.15.167:0/488383869 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f398c143d70 con 0x7f398c161380 2024-05-11T12:11:52.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.133+0000 7f3983e00640 1 -- 172.21.15.167:0/488383869 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f398401c070 con 0x7f398c161380 2024-05-11T12:11:52.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.134+0000 7f3983e00640 1 --2- 172.21.15.167:0/488383869 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f396c0762d0 0x7f396c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:52.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.135+0000 7f3983e00640 1 -- 172.21.15.167:0/488383869 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7f39840971b0 con 0x7f398c161380 2024-05-11T12:11:52.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.135+0000 7f398be00640 1 --2- 172.21.15.167:0/488383869 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f396c0762d0 0x7f396c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:52.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.135+0000 7f398be00640 1 --2- 172.21.15.167:0/488383869 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f396c0762d0 0x7f396c078790 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f397c034000 tx=0x7f397c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:52.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.140+0000 7f3983e00640 1 -- 172.21.15.167:0/488383869 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f398c143d70 con 0x7f398c161380 2024-05-11T12:11:52.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.418+0000 7f3992c00640 1 -- 172.21.15.167:0/488383869 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f398c0694b0 con 0x7f398c161380 2024-05-11T12:11:52.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.423+0000 7f3983e00640 1 -- 172.21.15.167:0/488383869 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v11) ==== 90+0+0 (secure 0 0 0) 0x7f3984060c30 con 0x7f398c161380 2024-05-11T12:11:52.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.425+0000 7f3992c00640 1 -- 172.21.15.167:0/488383869 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f396c0762d0 msgr2=0x7f396c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:52.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.425+0000 7f3992c00640 1 --2- 172.21.15.167:0/488383869 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f396c0762d0 0x7f396c078790 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f397c034000 tx=0x7f397c02e000 comp rx=0 tx=0).stop 2024-05-11T12:11:52.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.425+0000 7f3992c00640 1 -- 172.21.15.167:0/488383869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f398c161380 msgr2=0x7f398c159af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:52.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.425+0000 7f3992c00640 1 --2- 172.21.15.167:0/488383869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f398c161380 0x7f398c159af0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f398400b9e0 tx=0x7f398400bea0 comp rx=0 tx=0).stop 2024-05-11T12:11:52.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.425+0000 7f3992c00640 1 -- 172.21.15.167:0/488383869 shutdown_connections 2024-05-11T12:11:52.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.425+0000 7f3992c00640 1 --2- 172.21.15.167:0/488383869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f398c161380 0x7f398c159af0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:52.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.425+0000 7f3992c00640 1 --2- 172.21.15.167:0/488383869 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f396c0762d0 0x7f396c078790 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:52.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.425+0000 7f3992c00640 1 --2- 172.21.15.167:0/488383869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f398c15ac80 0x7f398c1595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:52.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.425+0000 7f3992c00640 1 --2- 172.21.15.167:0/488383869 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f398c0b3d50 0x7f398c06caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:52.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.425+0000 7f3992c00640 1 -- 172.21.15.167:0/488383869 >> 172.21.15.167:0/488383869 conn(0x7f398c06c450 msgr2=0x7f398c06d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:52.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.426+0000 7f3992c00640 1 -- 172.21.15.167:0/488383869 shutdown_connections 2024-05-11T12:11:52.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.426+0000 7f3992c00640 1 -- 172.21.15.167:0/488383869 wait complete. 2024-05-11T12:11:52.444 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-05-11T12:11:52.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.633+0000 7ffb5b000640 1 Processor -- start 2024-05-11T12:11:52.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.633+0000 7ffb5b000640 1 -- start start 2024-05-11T12:11:52.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.635+0000 7ffb5b000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb54161380 0x7ffb54074260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:52.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.635+0000 7ffb5b000640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ffb540b3d50 0x7ffb54072950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:52.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.635+0000 7ffb5b000640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb540b65e0 con 0x7ffb5415ac80 2024-05-11T12:11:52.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.635+0000 7ffb5b000640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7ffb5406e3c0 con 0x7ffb540b3d50 2024-05-11T12:11:52.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.635+0000 7ffb5b000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ffb5415b060 con 0x7ffb54161380 2024-05-11T12:11:52.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.635+0000 7ffb53e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7ffb5415ac80 0x7ffb54160c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:57686/0 (socket says 172.21.15.167:57686) 2024-05-11T12:11:52.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.635+0000 7ffb53e00640 1 -- 172.21.15.167:0/3062052629 learned_addr learned my addr 172.21.15.167:0/3062052629 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:52.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.636+0000 7ffb59000640 1 --2- 172.21.15.167:0/3062052629 >> v2:172.21.15.167:3300/0 conn(0x7ffb540b3d50 0x7ffb54072950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:52.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.636+0000 7ffb53400640 1 --2- 172.21.15.167:0/3062052629 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb54161380 0x7ffb54074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:52.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.636+0000 7ffb52a00640 1 -- 172.21.15.167:0/3062052629 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3415443494 0 0) 0x7ffb540b65e0 con 0x7ffb5415ac80 2024-05-11T12:11:52.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.636+0000 7ffb59000640 1 -- 172.21.15.167:0/3062052629 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb54161380 msgr2=0x7ffb54074260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:52.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.636+0000 7ffb59000640 1 --2- 172.21.15.167:0/3062052629 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb54161380 0x7ffb54074260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:52.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.636+0000 7ffb59000640 1 -- 172.21.15.167:0/3062052629 >> v1:172.21.15.167:6789/0 conn(0x7ffb5415ac80 legacy=0x7ffb54160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:52.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.636+0000 7ffb59000640 1 -- 172.21.15.167:0/3062052629 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb540b68c0 con 0x7ffb540b3d50 2024-05-11T12:11:52.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.637+0000 7ffb59000640 1 --2- 172.21.15.167:0/3062052629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb540b3d50 0x7ffb54072950 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c0097a0 tx=0x7ffb4c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd64599fd8e23bd5 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:52.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.637+0000 7ffb52a00640 1 -- 172.21.15.167:0/3062052629 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c037070 con 0x7ffb540b3d50 2024-05-11T12:11:52.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.637+0000 7ffb52a00640 1 -- 172.21.15.167:0/3062052629 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb4c032440 con 0x7ffb540b3d50 2024-05-11T12:11:52.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.637+0000 7ffb52a00640 1 -- 172.21.15.167:0/3062052629 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c032810 con 0x7ffb540b3d50 2024-05-11T12:11:52.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.638+0000 7ffb5b000640 1 -- 172.21.15.167:0/3062052629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb540b3d50 msgr2=0x7ffb54072950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:52.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.638+0000 7ffb5b000640 1 --2- 172.21.15.167:0/3062052629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb540b3d50 0x7ffb54072950 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c0097a0 tx=0x7ffb4c02a180 comp rx=0 tx=0).stop 2024-05-11T12:11:52.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.638+0000 7ffb5b000640 1 -- 172.21.15.167:0/3062052629 shutdown_connections 2024-05-11T12:11:52.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.638+0000 7ffb5b000640 1 --2- 172.21.15.167:0/3062052629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb540b3d50 0x7ffb54072950 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:52.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.638+0000 7ffb5b000640 1 --2- 172.21.15.167:0/3062052629 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb54161380 0x7ffb54074260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:52.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.638+0000 7ffb5b000640 1 -- 172.21.15.167:0/3062052629 >> 172.21.15.167:0/3062052629 conn(0x7ffb5406c450 msgr2=0x7ffb5406c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:52.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.638+0000 7ffb5b000640 1 -- 172.21.15.167:0/3062052629 shutdown_connections 2024-05-11T12:11:52.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.638+0000 7ffb5b000640 1 -- 172.21.15.167:0/3062052629 wait complete. 2024-05-11T12:11:52.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.640+0000 7ffb5b000640 1 Processor -- start 2024-05-11T12:11:52.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.640+0000 7ffb5b000640 1 -- start start 2024-05-11T12:11:52.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.641+0000 7ffb5b000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb540b3d50 0x7ffb541f8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:52.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.641+0000 7ffb5b000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb5415ac80 0x7ffb5415ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:52.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.641+0000 7ffb5b000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb54161380 0x7ffb5415f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:52.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.641+0000 7ffb5b000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ffb540b44f0 con 0x7ffb54161380 2024-05-11T12:11:52.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.641+0000 7ffb5b000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ffb540b41e0 con 0x7ffb540b3d50 2024-05-11T12:11:52.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.641+0000 7ffb5b000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ffb54072e90 con 0x7ffb5415ac80 2024-05-11T12:11:52.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.641+0000 7ffb59000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb54161380 0x7ffb5415f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:52.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.641+0000 7ffb59000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb54161380 0x7ffb5415f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:33214/0 (socket says 172.21.15.167:33214) 2024-05-11T12:11:52.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.641+0000 7ffb59000640 1 -- 172.21.15.167:0/2200003134 learned_addr learned my addr 172.21.15.167:0/2200003134 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:52.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.642+0000 7ffb59000640 1 -- 172.21.15.167:0/2200003134 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb5415ac80 msgr2=0x7ffb5415ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:52.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.642+0000 7ffb59000640 1 --2- 172.21.15.167:0/2200003134 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb5415ac80 0x7ffb5415ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:52.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.642+0000 7ffb59000640 1 -- 172.21.15.167:0/2200003134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb540b3d50 msgr2=0x7ffb541f8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:52.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.642+0000 7ffb59000640 1 --2- 172.21.15.167:0/2200003134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb540b3d50 0x7ffb541f8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:52.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.642+0000 7ffb59000640 1 -- 172.21.15.167:0/2200003134 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb541f8000 con 0x7ffb54161380 2024-05-11T12:11:52.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.642+0000 7ffb59000640 1 --2- 172.21.15.167:0/2200003134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb54161380 0x7ffb5415f810 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c0097a0 tx=0x7ffb4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:52.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.642+0000 7ffb4be00640 1 -- 172.21.15.167:0/2200003134 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c037050 con 0x7ffb54161380 2024-05-11T12:11:52.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.642+0000 7ffb4be00640 1 -- 172.21.15.167:0/2200003134 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb4c038040 con 0x7ffb54161380 2024-05-11T12:11:52.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.642+0000 7ffb5b000640 1 -- 172.21.15.167:0/2200003134 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb54160240 con 0x7ffb54161380 2024-05-11T12:11:52.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.642+0000 7ffb4be00640 1 -- 172.21.15.167:0/2200003134 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c006e10 con 0x7ffb54161380 2024-05-11T12:11:52.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.643+0000 7ffb5b000640 1 -- 172.21.15.167:0/2200003134 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb541f8550 con 0x7ffb54161380 2024-05-11T12:11:52.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.644+0000 7ffb5b000640 1 -- 172.21.15.167:0/2200003134 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb5415e3c0 con 0x7ffb54161380 2024-05-11T12:11:52.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.645+0000 7ffb4be00640 1 -- 172.21.15.167:0/2200003134 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ffb4c030980 con 0x7ffb54161380 2024-05-11T12:11:52.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.646+0000 7ffb4be00640 1 --2- 172.21.15.167:0/2200003134 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ffb300762d0 0x7ffb30078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:52.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.647+0000 7ffb4be00640 1 -- 172.21.15.167:0/2200003134 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7ffb4c0b7f60 con 0x7ffb54161380 2024-05-11T12:11:52.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.647+0000 7ffb53e00640 1 --2- 172.21.15.167:0/2200003134 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ffb300762d0 0x7ffb30078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:52.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.647+0000 7ffb53e00640 1 --2- 172.21.15.167:0/2200003134 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ffb300762d0 0x7ffb30078790 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ffb440015d0 tx=0x7ffb440078d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:52.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.651+0000 7ffb4be00640 1 -- 172.21.15.167:0/2200003134 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ffb4c032e20 con 0x7ffb54161380 2024-05-11T12:11:52.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.915+0000 7ffb5b000640 1 -- 172.21.15.167:0/2200003134 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7ffb540694b0 con 0x7ffb54161380 2024-05-11T12:11:52.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.919+0000 7ffb4be00640 1 -- 172.21.15.167:0/2200003134 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v12) ==== 90+0+0 (secure 0 0 0) 0x7ffb4c0819e0 con 0x7ffb54161380 2024-05-11T12:11:52.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.922+0000 7ffb5b000640 1 -- 172.21.15.167:0/2200003134 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ffb300762d0 msgr2=0x7ffb30078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:52.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.922+0000 7ffb5b000640 1 --2- 172.21.15.167:0/2200003134 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ffb300762d0 0x7ffb30078790 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ffb440015d0 tx=0x7ffb440078d0 comp rx=0 tx=0).stop 2024-05-11T12:11:52.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.922+0000 7ffb5b000640 1 -- 172.21.15.167:0/2200003134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb54161380 msgr2=0x7ffb5415f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:52.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.922+0000 7ffb5b000640 1 --2- 172.21.15.167:0/2200003134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb54161380 0x7ffb5415f810 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c0097a0 tx=0x7ffb4c0046c0 comp rx=0 tx=0).stop 2024-05-11T12:11:52.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.922+0000 7ffb5b000640 1 -- 172.21.15.167:0/2200003134 shutdown_connections 2024-05-11T12:11:52.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.922+0000 7ffb5b000640 1 --2- 172.21.15.167:0/2200003134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb54161380 0x7ffb5415f810 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:52.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.922+0000 7ffb5b000640 1 --2- 172.21.15.167:0/2200003134 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb5415ac80 0x7ffb5415ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:52.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.922+0000 7ffb5b000640 1 --2- 172.21.15.167:0/2200003134 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ffb300762d0 0x7ffb30078790 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:52.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.922+0000 7ffb5b000640 1 --2- 172.21.15.167:0/2200003134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb540b3d50 0x7ffb541f8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:52.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.922+0000 7ffb5b000640 1 -- 172.21.15.167:0/2200003134 >> 172.21.15.167:0/2200003134 conn(0x7ffb5406c450 msgr2=0x7ffb5406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:52.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.922+0000 7ffb5b000640 1 -- 172.21.15.167:0/2200003134 shutdown_connections 2024-05-11T12:11:52.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:52.923+0000 7ffb5b000640 1 -- 172.21.15.167:0/2200003134 wait complete. 2024-05-11T12:11:52.943 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-05-11T12:11:53.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.131+0000 7f2709600640 1 Processor -- start 2024-05-11T12:11:53.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.132+0000 7f2709600640 1 -- start start 2024-05-11T12:11:53.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.133+0000 7f2709600640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f27040b82c0 0x7f27040b8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:53.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.133+0000 7f2709600640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27040d1840 con 0x7f27040d3c70 2024-05-11T12:11:53.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.133+0000 7f2709600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f270406c6a0 con 0x7f270415f330 2024-05-11T12:11:53.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.133+0000 7f2709600640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f270406c450 con 0x7f27040b82c0 2024-05-11T12:11:53.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.134+0000 7f2702000640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f27040b82c0 0x7f27040b8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:53.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.134+0000 7f2703400640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f27040d3c70 0x7f270406d740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:57692/0 (socket says 172.21.15.167:57692) 2024-05-11T12:11:53.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.134+0000 7f2703400640 1 -- 172.21.15.167:0/1120735886 learned_addr learned my addr 172.21.15.167:0/1120735886 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:53.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.134+0000 7f2702000640 1 -- 172.21.15.167:0/1120735886 >> v1:172.21.15.204:6789/0 conn(0x7f270415f330 legacy=0x7f27040d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:53.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.134+0000 7f2702000640 1 -- 172.21.15.167:0/1120735886 >> v1:172.21.15.167:6789/0 conn(0x7f27040d3c70 legacy=0x7f270406d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:53.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.134+0000 7f2702000640 1 -- 172.21.15.167:0/1120735886 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f270406c980 con 0x7f27040b82c0 2024-05-11T12:11:53.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.134+0000 7f2701600640 1 -- 172.21.15.167:0/1120735886 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 307806129 0 0) 0x7f27040d1840 con 0x7f27040d3c70 2024-05-11T12:11:53.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.135+0000 7f2702000640 1 --2- 172.21.15.167:0/1120735886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27040b82c0 0x7f27040b8720 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f26ec009d90 tx=0x7f26ec02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=654d667be3d2825a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:53.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.135+0000 7f2701600640 1 -- 172.21.15.167:0/1120735886 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ec007740 con 0x7f27040b82c0 2024-05-11T12:11:53.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.135+0000 7f2701600640 1 -- 172.21.15.167:0/1120735886 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26ec0078e0 con 0x7f27040b82c0 2024-05-11T12:11:53.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.135+0000 7f2701600640 1 -- 172.21.15.167:0/1120735886 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ec007cb0 con 0x7f27040b82c0 2024-05-11T12:11:53.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.136+0000 7f2709600640 1 -- 172.21.15.167:0/1120735886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27040b82c0 msgr2=0x7f27040b8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:53.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.136+0000 7f2709600640 1 --2- 172.21.15.167:0/1120735886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27040b82c0 0x7f27040b8720 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f26ec009d90 tx=0x7f26ec02a660 comp rx=0 tx=0).stop 2024-05-11T12:11:53.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.136+0000 7f2709600640 1 -- 172.21.15.167:0/1120735886 shutdown_connections 2024-05-11T12:11:53.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.136+0000 7f2709600640 1 --2- 172.21.15.167:0/1120735886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27040b82c0 0x7f27040b8720 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:53.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.136+0000 7f2709600640 1 -- 172.21.15.167:0/1120735886 >> 172.21.15.167:0/1120735886 conn(0x7f27040b6b80 msgr2=0x7f27040b6f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:53.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.137+0000 7f2709600640 1 -- 172.21.15.167:0/1120735886 shutdown_connections 2024-05-11T12:11:53.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.137+0000 7f2709600640 1 -- 172.21.15.167:0/1120735886 wait complete. 2024-05-11T12:11:53.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.138+0000 7f2709600640 1 Processor -- start 2024-05-11T12:11:53.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.138+0000 7f2709600640 1 -- start start 2024-05-11T12:11:53.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.139+0000 7f2709600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27040b82c0 0x7f27040b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:53.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.139+0000 7f2709600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27040d3c70 0x7f27041f4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:53.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.139+0000 7f2709600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f270415f330 0x7f27041f6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:53.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.139+0000 7f2709600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f270406cf40 con 0x7f270415f330 2024-05-11T12:11:53.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.139+0000 7f2709600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f270406d2b0 con 0x7f27040b82c0 2024-05-11T12:11:53.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.139+0000 7f2709600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f270415f710 con 0x7f27040d3c70 2024-05-11T12:11:53.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.140+0000 7f2703400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f270415f330 0x7f27041f6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:53.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.140+0000 7f2703400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f270415f330 0x7f27041f6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:33234/0 (socket says 172.21.15.167:33234) 2024-05-11T12:11:53.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.140+0000 7f2703400640 1 -- 172.21.15.167:0/1461390139 learned_addr learned my addr 172.21.15.167:0/1461390139 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:53.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.140+0000 7f2702a00640 1 --2- 172.21.15.167:0/1461390139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27040b82c0 0x7f27040b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:53.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.140+0000 7f2702000640 1 --2- 172.21.15.167:0/1461390139 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27040d3c70 0x7f27041f4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:53.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.140+0000 7f2703400640 1 -- 172.21.15.167:0/1461390139 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27040d3c70 msgr2=0x7f27041f4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:53.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.140+0000 7f2703400640 1 --2- 172.21.15.167:0/1461390139 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27040d3c70 0x7f27041f4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:53.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.140+0000 7f2703400640 1 -- 172.21.15.167:0/1461390139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27040b82c0 msgr2=0x7f27040b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:53.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.140+0000 7f2703400640 1 --2- 172.21.15.167:0/1461390139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27040b82c0 0x7f27040b3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:53.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.140+0000 7f2703400640 1 -- 172.21.15.167:0/1461390139 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27041f57c0 con 0x7f270415f330 2024-05-11T12:11:53.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.140+0000 7f2702a00640 1 --2- 172.21.15.167:0/1461390139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27040b82c0 0x7f27040b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:53.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.140+0000 7f2703400640 1 --2- 172.21.15.167:0/1461390139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f270415f330 0x7f27041f6010 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f26fc00c410 tx=0x7f26fc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:53.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.141+0000 7f26faa00640 1 -- 172.21.15.167:0/1461390139 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc0074e0 con 0x7f270415f330 2024-05-11T12:11:53.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.141+0000 7f26faa00640 1 -- 172.21.15.167:0/1461390139 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26fc0047e0 con 0x7f270415f330 2024-05-11T12:11:53.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.141+0000 7f26faa00640 1 -- 172.21.15.167:0/1461390139 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc004bb0 con 0x7f270415f330 2024-05-11T12:11:53.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.141+0000 7f2709600640 1 -- 172.21.15.167:0/1461390139 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27041f5aa0 con 0x7f270415f330 2024-05-11T12:11:53.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.141+0000 7f2709600640 1 -- 172.21.15.167:0/1461390139 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2704159e30 con 0x7f270415f330 2024-05-11T12:11:53.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.142+0000 7f2709600640 1 -- 172.21.15.167:0/1461390139 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27040694b0 con 0x7f270415f330 2024-05-11T12:11:53.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.144+0000 7f26faa00640 1 -- 172.21.15.167:0/1461390139 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f26fc011070 con 0x7f270415f330 2024-05-11T12:11:53.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.144+0000 7f26faa00640 1 --2- 172.21.15.167:0/1461390139 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f26e00762d0 0x7f26e0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:53.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.145+0000 7f26faa00640 1 -- 172.21.15.167:0/1461390139 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7f26fc096ff0 con 0x7f270415f330 2024-05-11T12:11:53.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.145+0000 7f2702a00640 1 --2- 172.21.15.167:0/1461390139 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f26e00762d0 0x7f26e0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:53.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.145+0000 7f2702a00640 1 --2- 172.21.15.167:0/1461390139 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f26e00762d0 0x7f26e0078790 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f26f400a830 tx=0x7f26f4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:53.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.149+0000 7f26faa00640 1 -- 172.21.15.167:0/1461390139 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f26fc060b10 con 0x7f270415f330 2024-05-11T12:11:53.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.415+0000 7f2709600640 1 -- 172.21.15.167:0/1461390139 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f27040d4c10 con 0x7f270415f330 2024-05-11T12:11:53.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.420+0000 7f26faa00640 1 -- 172.21.15.167:0/1461390139 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v13) ==== 97+0+0 (secure 0 0 0) 0x7f26fc060410 con 0x7f270415f330 2024-05-11T12:11:53.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.422+0000 7f2709600640 1 -- 172.21.15.167:0/1461390139 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f26e00762d0 msgr2=0x7f26e0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:53.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.422+0000 7f2709600640 1 --2- 172.21.15.167:0/1461390139 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f26e00762d0 0x7f26e0078790 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f26f400a830 tx=0x7f26f4008040 comp rx=0 tx=0).stop 2024-05-11T12:11:53.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.422+0000 7f2709600640 1 -- 172.21.15.167:0/1461390139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f270415f330 msgr2=0x7f27041f6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:53.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.422+0000 7f2709600640 1 --2- 172.21.15.167:0/1461390139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f270415f330 0x7f27041f6010 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f26fc00c410 tx=0x7f26fc00c8d0 comp rx=0 tx=0).stop 2024-05-11T12:11:53.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.422+0000 7f2709600640 1 -- 172.21.15.167:0/1461390139 shutdown_connections 2024-05-11T12:11:53.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.422+0000 7f2709600640 1 --2- 172.21.15.167:0/1461390139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f270415f330 0x7f27041f6010 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:53.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.422+0000 7f2709600640 1 --2- 172.21.15.167:0/1461390139 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27040d3c70 0x7f27041f4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:53.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.422+0000 7f2709600640 1 --2- 172.21.15.167:0/1461390139 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f26e00762d0 0x7f26e0078790 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:53.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.422+0000 7f2709600640 1 --2- 172.21.15.167:0/1461390139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27040b82c0 0x7f27040b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:53.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.422+0000 7f2709600640 1 -- 172.21.15.167:0/1461390139 >> 172.21.15.167:0/1461390139 conn(0x7f27040b6b80 msgr2=0x7f2704160950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:53.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.423+0000 7f2709600640 1 -- 172.21.15.167:0/1461390139 shutdown_connections 2024-05-11T12:11:53.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.423+0000 7f2709600640 1 -- 172.21.15.167:0/1461390139 wait complete. 2024-05-11T12:11:53.440 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-05-11T12:11:53.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.635+0000 7fb384c00640 1 Processor -- start 2024-05-11T12:11:53.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.636+0000 7fb384c00640 1 -- start start 2024-05-11T12:11:53.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.637+0000 7fb384c00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb380145a00 0x7fb3800afe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:53.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.637+0000 7fb384c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb3800ceb00 0x7fb3800cef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:53.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.637+0000 7fb384c00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3800aed90 con 0x7fb3800ae820 2024-05-11T12:11:53.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.637+0000 7fb384c00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fb3800af9a0 con 0x7fb380145a00 2024-05-11T12:11:53.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.637+0000 7fb384c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fb3800af070 con 0x7fb3800ceb00 2024-05-11T12:11:53.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.638+0000 7fb377e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fb3800ae820 0x7fb38006f8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:57702/0 (socket says 172.21.15.167:57702) 2024-05-11T12:11:53.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.638+0000 7fb377e00640 1 -- 172.21.15.167:0/3273430953 learned_addr learned my addr 172.21.15.167:0/3273430953 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:53.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.638+0000 7fb37de00640 1 --2- 172.21.15.167:0/3273430953 >> v2:172.21.15.167:3300/0 conn(0x7fb380145a00 0x7fb3800afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:53.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.638+0000 7fb37e800640 1 --2- 172.21.15.167:0/3273430953 >> v2:172.21.15.204:3300/0 conn(0x7fb3800ceb00 0x7fb3800cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:53.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.638+0000 7fb37d400640 1 -- 172.21.15.167:0/3273430953 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3274494794 0 0) 0x7fb3800aed90 con 0x7fb3800ae820 2024-05-11T12:11:53.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.638+0000 7fb37d400640 1 -- 172.21.15.167:0/3273430953 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb364003520 con 0x7fb3800ae820 2024-05-11T12:11:53.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.638+0000 7fb37de00640 1 -- 172.21.15.167:0/3273430953 >> v2:172.21.15.204:3300/0 conn(0x7fb3800ceb00 msgr2=0x7fb3800cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:53.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.638+0000 7fb37de00640 1 --2- 172.21.15.167:0/3273430953 >> v2:172.21.15.204:3300/0 conn(0x7fb3800ceb00 0x7fb3800cef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:53.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.638+0000 7fb37de00640 1 -- 172.21.15.167:0/3273430953 >> v1:172.21.15.167:6789/0 conn(0x7fb3800ae820 legacy=0x7fb38006f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:53.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.638+0000 7fb37de00640 1 -- 172.21.15.167:0/3273430953 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3800af350 con 0x7fb380145a00 2024-05-11T12:11:53.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.639+0000 7fb37de00640 1 --2- 172.21.15.167:0/3273430953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb380145a00 0x7fb3800afe30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb3800b06d0 tx=0x7fb37002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4512586d5a8ea3f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:53.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.640+0000 7fb37d400640 1 -- 172.21.15.167:0/3273430953 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb370037070 con 0x7fb380145a00 2024-05-11T12:11:53.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.640+0000 7fb37d400640 1 -- 172.21.15.167:0/3273430953 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb370032440 con 0x7fb380145a00 2024-05-11T12:11:53.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.640+0000 7fb37d400640 1 -- 172.21.15.167:0/3273430953 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37002ae60 con 0x7fb380145a00 2024-05-11T12:11:53.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.640+0000 7fb384c00640 1 -- 172.21.15.167:0/3273430953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb380145a00 msgr2=0x7fb3800afe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:53.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.640+0000 7fb384c00640 1 --2- 172.21.15.167:0/3273430953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb380145a00 0x7fb3800afe30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb3800b06d0 tx=0x7fb37002a110 comp rx=0 tx=0).stop 2024-05-11T12:11:53.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.641+0000 7fb384c00640 1 -- 172.21.15.167:0/3273430953 shutdown_connections 2024-05-11T12:11:53.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.641+0000 7fb384c00640 1 --2- 172.21.15.167:0/3273430953 >> v2:172.21.15.204:3300/0 conn(0x7fb3800ceb00 0x7fb3800cef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:53.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.641+0000 7fb384c00640 1 --2- 172.21.15.167:0/3273430953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb380145a00 0x7fb3800afe30 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:53.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.641+0000 7fb384c00640 1 -- 172.21.15.167:0/3273430953 >> 172.21.15.167:0/3273430953 conn(0x7fb38006c450 msgr2=0x7fb38006c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:53.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.641+0000 7fb384c00640 1 -- 172.21.15.167:0/3273430953 shutdown_connections 2024-05-11T12:11:53.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.641+0000 7fb384c00640 1 -- 172.21.15.167:0/3273430953 wait complete. 2024-05-11T12:11:53.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.642+0000 7fb384c00640 1 Processor -- start 2024-05-11T12:11:53.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.643+0000 7fb384c00640 1 -- start start 2024-05-11T12:11:53.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.643+0000 7fb384c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3800ae820 0x7fb380142b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:53.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.643+0000 7fb384c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3800ceb00 0x7fb380143ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:53.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.644+0000 7fb384c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb380145a00 0x7fb380148060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:53.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.644+0000 7fb384c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb3800723b0 con 0x7fb3800ceb00 2024-05-11T12:11:53.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.644+0000 7fb384c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb380072230 con 0x7fb380145a00 2024-05-11T12:11:53.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.644+0000 7fb384c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb380072530 con 0x7fb3800ae820 2024-05-11T12:11:53.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.644+0000 7fb377e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3800ceb00 0x7fb380143ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:53.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.644+0000 7fb377e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3800ceb00 0x7fb380143ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:33248/0 (socket says 172.21.15.167:33248) 2024-05-11T12:11:53.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.644+0000 7fb377e00640 1 -- 172.21.15.167:0/1937756918 learned_addr learned my addr 172.21.15.167:0/1937756918 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:53.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.644+0000 7fb377e00640 1 -- 172.21.15.167:0/1937756918 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3800ae820 msgr2=0x7fb380142b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:53.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.644+0000 7fb377e00640 1 --2- 172.21.15.167:0/1937756918 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3800ae820 0x7fb380142b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:53.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.644+0000 7fb37e800640 1 --2- 172.21.15.167:0/1937756918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb380145a00 0x7fb380148060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:53.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.644+0000 7fb377e00640 1 -- 172.21.15.167:0/1937756918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb380145a00 msgr2=0x7fb380148060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:53.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.644+0000 7fb377e00640 1 --2- 172.21.15.167:0/1937756918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb380145a00 0x7fb380148060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:53.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.644+0000 7fb377e00640 1 -- 172.21.15.167:0/1937756918 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3801f4c00 con 0x7fb3800ceb00 2024-05-11T12:11:53.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.645+0000 7fb377e00640 1 --2- 172.21.15.167:0/1937756918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3800ceb00 0x7fb380143ce0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb36800c530 tx=0x7fb36800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:53.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.645+0000 7fb376000640 1 -- 172.21.15.167:0/1937756918 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb368017070 con 0x7fb3800ceb00 2024-05-11T12:11:53.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.645+0000 7fb376000640 1 -- 172.21.15.167:0/1937756918 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb368005ce0 con 0x7fb3800ceb00 2024-05-11T12:11:53.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.645+0000 7fb376000640 1 -- 172.21.15.167:0/1937756918 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb368007270 con 0x7fb3800ceb00 2024-05-11T12:11:53.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.645+0000 7fb384c00640 1 -- 172.21.15.167:0/1937756918 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3801f86d0 con 0x7fb3800ceb00 2024-05-11T12:11:53.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.645+0000 7fb384c00640 1 -- 172.21.15.167:0/1937756918 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3801f8c90 con 0x7fb3800ceb00 2024-05-11T12:11:53.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.646+0000 7fb384c00640 1 -- 172.21.15.167:0/1937756918 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3800694b0 con 0x7fb3800ceb00 2024-05-11T12:11:53.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.647+0000 7fb376000640 1 -- 172.21.15.167:0/1937756918 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb36801a070 con 0x7fb3800ceb00 2024-05-11T12:11:53.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.648+0000 7fb376000640 1 --2- 172.21.15.167:0/1937756918 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb35c0762d0 0x7fb35c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:53.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.648+0000 7fb376000640 1 -- 172.21.15.167:0/1937756918 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7fb368096690 con 0x7fb3800ceb00 2024-05-11T12:11:53.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.648+0000 7fb37de00640 1 --2- 172.21.15.167:0/1937756918 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb35c0762d0 0x7fb35c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:53.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.649+0000 7fb37de00640 1 --2- 172.21.15.167:0/1937756918 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb35c0762d0 0x7fb35c078790 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb370034000 tx=0x7fb37002c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:53.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.653+0000 7fb376000640 1 -- 172.21.15.167:0/1937756918 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb3801f4c00 con 0x7fb3800ceb00 2024-05-11T12:11:53.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.938+0000 7fb384c00640 1 -- 172.21.15.167:0/1937756918 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7fb380073a00 con 0x7fb3800ceb00 2024-05-11T12:11:53.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.942+0000 7fb376000640 1 -- 172.21.15.167:0/1937756918 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v14) ==== 90+0+0 (secure 0 0 0) 0x7fb36805fac0 con 0x7fb3800ceb00 2024-05-11T12:11:53.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.945+0000 7fb384c00640 1 -- 172.21.15.167:0/1937756918 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb35c0762d0 msgr2=0x7fb35c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:53.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.945+0000 7fb384c00640 1 --2- 172.21.15.167:0/1937756918 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb35c0762d0 0x7fb35c078790 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb370034000 tx=0x7fb37002c000 comp rx=0 tx=0).stop 2024-05-11T12:11:53.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.945+0000 7fb384c00640 1 -- 172.21.15.167:0/1937756918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3800ceb00 msgr2=0x7fb380143ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:53.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.945+0000 7fb384c00640 1 --2- 172.21.15.167:0/1937756918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3800ceb00 0x7fb380143ce0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb36800c530 tx=0x7fb36800c9f0 comp rx=0 tx=0).stop 2024-05-11T12:11:53.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.945+0000 7fb384c00640 1 -- 172.21.15.167:0/1937756918 shutdown_connections 2024-05-11T12:11:53.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.945+0000 7fb384c00640 1 --2- 172.21.15.167:0/1937756918 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb35c0762d0 0x7fb35c078790 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:53.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.945+0000 7fb384c00640 1 --2- 172.21.15.167:0/1937756918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb380145a00 0x7fb380148060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:53.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.945+0000 7fb384c00640 1 --2- 172.21.15.167:0/1937756918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3800ceb00 0x7fb380143ce0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:53.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.945+0000 7fb384c00640 1 --2- 172.21.15.167:0/1937756918 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3800ae820 0x7fb380142b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:53.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.945+0000 7fb384c00640 1 -- 172.21.15.167:0/1937756918 >> 172.21.15.167:0/1937756918 conn(0x7fb38006c450 msgr2=0x7fb3800b3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:53.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.946+0000 7fb384c00640 1 -- 172.21.15.167:0/1937756918 shutdown_connections 2024-05-11T12:11:53.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:53.946+0000 7fb384c00640 1 -- 172.21.15.167:0/1937756918 wait complete. 2024-05-11T12:11:53.964 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-11T12:11:54.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.157+0000 7f4918e00640 1 Processor -- start 2024-05-11T12:11:54.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.157+0000 7f4918e00640 1 -- start start 2024-05-11T12:11:54.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.159+0000 7f4918e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f491406c660 0x7f491406ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:54.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.159+0000 7f4918e00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f491406cfb0 con 0x7f4914145b00 2024-05-11T12:11:54.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.159+0000 7f4918e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f491406fbc0 con 0x7f4914146770 2024-05-11T12:11:54.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.159+0000 7f4918e00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f49140b4520 con 0x7f491406c660 2024-05-11T12:11:54.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.159+0000 7f4913400640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f4914145b00 0x7f491406f2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:57708/0 (socket says 172.21.15.167:57708) 2024-05-11T12:11:54.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.159+0000 7f4913400640 1 -- 172.21.15.167:0/1265031342 learned_addr learned my addr 172.21.15.167:0/1265031342 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:54.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.159+0000 7f4913e00640 1 --2- 172.21.15.167:0/1265031342 >> v2:172.21.15.167:3300/0 conn(0x7f491406c660 0x7f491406ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:54.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.159+0000 7f4912a00640 1 -- 172.21.15.167:0/1265031342 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3651369546 0 0) 0x7f4904001110 con 0x7f4914145b00 2024-05-11T12:11:54.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.159+0000 7f4912a00640 1 -- 172.21.15.167:0/1265031342 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f48f8003520 con 0x7f4914145b00 2024-05-11T12:11:54.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.160+0000 7f4913e00640 1 -- 172.21.15.167:0/1265031342 >> v1:172.21.15.204:6789/0 conn(0x7f4914146770 legacy=0x7f49140ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:54.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.160+0000 7f4913e00640 1 -- 172.21.15.167:0/1265031342 >> v1:172.21.15.167:6789/0 conn(0x7f4914145b00 legacy=0x7f491406f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:54.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.160+0000 7f4913e00640 1 -- 172.21.15.167:0/1265031342 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49140b4800 con 0x7f491406c660 2024-05-11T12:11:54.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.161+0000 7f4913e00640 1 --2- 172.21.15.167:0/1265031342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f491406c660 0x7f491406ca70 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f490c009d90 tx=0x7f490c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c8275d0c03c82e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:54.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.161+0000 7f4912a00640 1 -- 172.21.15.167:0/1265031342 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c031600 con 0x7f491406c660 2024-05-11T12:11:54.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.161+0000 7f4912a00640 1 -- 172.21.15.167:0/1265031342 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f490c0317a0 con 0x7f491406c660 2024-05-11T12:11:54.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.161+0000 7f4912a00640 1 -- 172.21.15.167:0/1265031342 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c031b70 con 0x7f491406c660 2024-05-11T12:11:54.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.161+0000 7f4918e00640 1 -- 172.21.15.167:0/1265031342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f491406c660 msgr2=0x7f491406ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:54.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.161+0000 7f4918e00640 1 --2- 172.21.15.167:0/1265031342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f491406c660 0x7f491406ca70 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f490c009d90 tx=0x7f490c02a5d0 comp rx=0 tx=0).stop 2024-05-11T12:11:54.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.161+0000 7f4918e00640 1 -- 172.21.15.167:0/1265031342 shutdown_connections 2024-05-11T12:11:54.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.161+0000 7f4918e00640 1 --2- 172.21.15.167:0/1265031342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f491406c660 0x7f491406ca70 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:54.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.161+0000 7f4918e00640 1 -- 172.21.15.167:0/1265031342 >> 172.21.15.167:0/1265031342 conn(0x7f49140b3460 msgr2=0x7f49140d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:54.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.162+0000 7f4918e00640 1 -- 172.21.15.167:0/1265031342 shutdown_connections 2024-05-11T12:11:54.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.162+0000 7f4918e00640 1 -- 172.21.15.167:0/1265031342 wait complete. 2024-05-11T12:11:54.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.163+0000 7f4918e00640 1 Processor -- start 2024-05-11T12:11:54.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.163+0000 7f4918e00640 1 -- start start 2024-05-11T12:11:54.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.164+0000 7f4918e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f491406c660 0x7f491415c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:54.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.164+0000 7f4918e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4914145b00 0x7f4914141f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:54.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.164+0000 7f4918e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4914146770 0x7f4914142460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:54.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.164+0000 7f4918e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4914145ee0 con 0x7f4914146770 2024-05-11T12:11:54.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.164+0000 7f4918e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f49140cf0e0 con 0x7f491406c660 2024-05-11T12:11:54.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.164+0000 7f4918e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f49140b5000 con 0x7f4914145b00 2024-05-11T12:11:54.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.165+0000 7f4913e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4914146770 0x7f4914142460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:54.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.165+0000 7f4913e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4914146770 0x7f4914142460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:33264/0 (socket says 172.21.15.167:33264) 2024-05-11T12:11:54.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.165+0000 7f4913e00640 1 -- 172.21.15.167:0/3059402780 learned_addr learned my addr 172.21.15.167:0/3059402780 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:54.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.165+0000 7f490aa00640 1 --2- 172.21.15.167:0/3059402780 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4914145b00 0x7f4914141f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:54.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.165+0000 7f4913400640 1 --2- 172.21.15.167:0/3059402780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f491406c660 0x7f491415c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:54.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.165+0000 7f4913e00640 1 -- 172.21.15.167:0/3059402780 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4914145b00 msgr2=0x7f4914141f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:54.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.165+0000 7f4913e00640 1 --2- 172.21.15.167:0/3059402780 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4914145b00 0x7f4914141f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:54.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.165+0000 7f4913e00640 1 -- 172.21.15.167:0/3059402780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f491406c660 msgr2=0x7f491415c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:54.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.165+0000 7f4913e00640 1 --2- 172.21.15.167:0/3059402780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f491406c660 0x7f491415c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:54.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.165+0000 7f4913e00640 1 -- 172.21.15.167:0/3059402780 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f491415cbb0 con 0x7f4914146770 2024-05-11T12:11:54.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.165+0000 7f4913e00640 1 --2- 172.21.15.167:0/3059402780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4914146770 0x7f4914142460 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f490c009d90 tx=0x7f490c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:54.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.166+0000 7f490be00640 1 -- 172.21.15.167:0/3059402780 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c040070 con 0x7f4914146770 2024-05-11T12:11:54.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.166+0000 7f490be00640 1 -- 172.21.15.167:0/3059402780 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f490c030b60 con 0x7f4914146770 2024-05-11T12:11:54.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.166+0000 7f4918e00640 1 -- 172.21.15.167:0/3059402780 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f491415fc50 con 0x7f4914146770 2024-05-11T12:11:54.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.166+0000 7f490be00640 1 -- 172.21.15.167:0/3059402780 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c009450 con 0x7f4914146770 2024-05-11T12:11:54.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.166+0000 7f4918e00640 1 -- 172.21.15.167:0/3059402780 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f491415f690 con 0x7f4914146770 2024-05-11T12:11:54.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.167+0000 7f4918e00640 1 -- 172.21.15.167:0/3059402780 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49140694b0 con 0x7f4914146770 2024-05-11T12:11:54.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.168+0000 7f490be00640 1 -- 172.21.15.167:0/3059402780 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f490c039040 con 0x7f4914146770 2024-05-11T12:11:54.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.170+0000 7f490be00640 1 --2- 172.21.15.167:0/3059402780 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f48f00762d0 0x7f48f0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:54.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.170+0000 7f490be00640 1 -- 172.21.15.167:0/3059402780 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7f490c0b8720 con 0x7f4914146770 2024-05-11T12:11:54.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.170+0000 7f4913400640 1 --2- 172.21.15.167:0/3059402780 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f48f00762d0 0x7f48f0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:54.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.171+0000 7f4913400640 1 --2- 172.21.15.167:0/3059402780 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f48f00762d0 0x7f48f0078790 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4904005880 tx=0x7f4904009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:54.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.174+0000 7f490be00640 1 -- 172.21.15.167:0/3059402780 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f491415cbb0 con 0x7f4914146770 2024-05-11T12:11:54.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.385+0000 7f4918e00640 1 -- 172.21.15.167:0/3059402780 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f491415fa30 con 0x7f4914146770 2024-05-11T12:11:54.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.386+0000 7f490be00640 1 -- 172.21.15.167:0/3059402780 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v37) ==== 84+0+65 (secure 0 0 0) 0x7f490c081be0 con 0x7f4914146770 2024-05-11T12:11:54.386 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:11:54.386 INFO:teuthology.orchestra.run.smithi167.stdout:[ 2024-05-11T12:11:54.386 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:11:54.386 INFO:teuthology.orchestra.run.smithi167.stdout: "poolnum": 1, 2024-05-11T12:11:54.387 INFO:teuthology.orchestra.run.smithi167.stdout: "poolname": ".mgr" 2024-05-11T12:11:54.387 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:11:54.387 INFO:teuthology.orchestra.run.smithi167.stdout:] 2024-05-11T12:11:54.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.388+0000 7f4918e00640 1 -- 172.21.15.167:0/3059402780 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f48f00762d0 msgr2=0x7f48f0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:54.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.388+0000 7f4918e00640 1 --2- 172.21.15.167:0/3059402780 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f48f00762d0 0x7f48f0078790 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4904005880 tx=0x7f4904009040 comp rx=0 tx=0).stop 2024-05-11T12:11:54.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.389+0000 7f4918e00640 1 -- 172.21.15.167:0/3059402780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4914146770 msgr2=0x7f4914142460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:54.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.389+0000 7f4918e00640 1 --2- 172.21.15.167:0/3059402780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4914146770 0x7f4914142460 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f490c009d90 tx=0x7f490c004a40 comp rx=0 tx=0).stop 2024-05-11T12:11:54.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.389+0000 7f4918e00640 1 -- 172.21.15.167:0/3059402780 shutdown_connections 2024-05-11T12:11:54.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.389+0000 7f4918e00640 1 --2- 172.21.15.167:0/3059402780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4914146770 0x7f4914142460 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:54.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.389+0000 7f4918e00640 1 --2- 172.21.15.167:0/3059402780 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4914145b00 0x7f4914141f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:54.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.389+0000 7f4918e00640 1 --2- 172.21.15.167:0/3059402780 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f48f00762d0 0x7f48f0078790 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:54.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.389+0000 7f4918e00640 1 --2- 172.21.15.167:0/3059402780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f491406c660 0x7f491415c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:54.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.389+0000 7f4918e00640 1 -- 172.21.15.167:0/3059402780 >> 172.21.15.167:0/3059402780 conn(0x7f49140b3460 msgr2=0x7f4914146060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:54.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.389+0000 7f4918e00640 1 -- 172.21.15.167:0/3059402780 shutdown_connections 2024-05-11T12:11:54.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.389+0000 7f4918e00640 1 -- 172.21.15.167:0/3059402780 wait complete. 2024-05-11T12:11:54.408 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-11T12:11:54.408 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-11T12:11:54.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.611+0000 7f5b75400640 1 Processor -- start 2024-05-11T12:11:54.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.612+0000 7f5b75400640 1 -- start start 2024-05-11T12:11:54.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.613+0000 7f5b75400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b7006cce0 0x7f5b7006d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:54.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.613+0000 7f5b75400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5b70072150 0x7f5b700725e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:54.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.613+0000 7f5b75400640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b700702d0 con 0x7f5b7015b8b0 2024-05-11T12:11:54.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.613+0000 7f5b75400640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f5b7006d680 con 0x7f5b70072150 2024-05-11T12:11:54.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.613+0000 7f5b75400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5b700ceb70 con 0x7f5b7006cce0 2024-05-11T12:11:54.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.614+0000 7f5b6e800640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f5b7015b8b0 0x7f5b7006fbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:57710/0 (socket says 172.21.15.167:57710) 2024-05-11T12:11:54.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.614+0000 7f5b6e800640 1 -- 172.21.15.167:0/4118194776 learned_addr learned my addr 172.21.15.167:0/4118194776 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:54.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.614+0000 7f5b6f200640 1 --2- 172.21.15.167:0/4118194776 >> v2:172.21.15.167:3300/0 conn(0x7f5b70072150 0x7f5b700725e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:54.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.614+0000 7f5b67e00640 1 --2- 172.21.15.167:0/4118194776 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b7006cce0 0x7f5b7006d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:54.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.614+0000 7f5b6de00640 1 -- 172.21.15.167:0/4118194776 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3143935226 0 0) 0x7f5b700702d0 con 0x7f5b7015b8b0 2024-05-11T12:11:54.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.614+0000 7f5b6f200640 1 -- 172.21.15.167:0/4118194776 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b7006cce0 msgr2=0x7f5b7006d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:54.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.614+0000 7f5b6f200640 1 --2- 172.21.15.167:0/4118194776 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b7006cce0 0x7f5b7006d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:54.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.614+0000 7f5b6f200640 1 -- 172.21.15.167:0/4118194776 >> v1:172.21.15.167:6789/0 conn(0x7f5b7015b8b0 legacy=0x7f5b7006fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:54.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.614+0000 7f5b6f200640 1 -- 172.21.15.167:0/4118194776 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b700cedc0 con 0x7f5b70072150 2024-05-11T12:11:54.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.615+0000 7f5b6f200640 1 --2- 172.21.15.167:0/4118194776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b70072150 0x7f5b700725e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5b68009d90 tx=0x7f5b6802c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16db15633746725c server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:54.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.615+0000 7f5b6de00640 1 -- 172.21.15.167:0/4118194776 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b68032750 con 0x7f5b70072150 2024-05-11T12:11:54.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.616+0000 7f5b6de00640 1 -- 172.21.15.167:0/4118194776 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b68030050 con 0x7f5b70072150 2024-05-11T12:11:54.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.616+0000 7f5b6de00640 1 -- 172.21.15.167:0/4118194776 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b68032b10 con 0x7f5b70072150 2024-05-11T12:11:54.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.616+0000 7f5b75400640 1 -- 172.21.15.167:0/4118194776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b70072150 msgr2=0x7f5b700725e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:54.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.616+0000 7f5b75400640 1 --2- 172.21.15.167:0/4118194776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b70072150 0x7f5b700725e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5b68009d90 tx=0x7f5b6802c850 comp rx=0 tx=0).stop 2024-05-11T12:11:54.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.616+0000 7f5b75400640 1 -- 172.21.15.167:0/4118194776 shutdown_connections 2024-05-11T12:11:54.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.616+0000 7f5b75400640 1 --2- 172.21.15.167:0/4118194776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b70072150 0x7f5b700725e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:54.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.616+0000 7f5b75400640 1 --2- 172.21.15.167:0/4118194776 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b7006cce0 0x7f5b7006d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:54.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.616+0000 7f5b75400640 1 -- 172.21.15.167:0/4118194776 >> 172.21.15.167:0/4118194776 conn(0x7f5b700d0c70 msgr2=0x7f5b700d1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:54.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.616+0000 7f5b75400640 1 -- 172.21.15.167:0/4118194776 shutdown_connections 2024-05-11T12:11:54.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.616+0000 7f5b75400640 1 -- 172.21.15.167:0/4118194776 wait complete. 2024-05-11T12:11:54.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.618+0000 7f5b75400640 1 Processor -- start 2024-05-11T12:11:54.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.618+0000 7f5b75400640 1 -- start start 2024-05-11T12:11:54.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.619+0000 7f5b75400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b7006cce0 0x7f5b7015bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:54.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.619+0000 7f5b75400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b70072150 0x7f5b7015e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:54.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.619+0000 7f5b75400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b7015b8b0 0x7f5b7014fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:54.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.619+0000 7f5b75400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5b700cf660 con 0x7f5b7015b8b0 2024-05-11T12:11:54.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.619+0000 7f5b75400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5b700cf380 con 0x7f5b7006cce0 2024-05-11T12:11:54.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.619+0000 7f5b75400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5b700d0240 con 0x7f5b70072150 2024-05-11T12:11:54.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.619+0000 7f5b6f200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b7015b8b0 0x7f5b7014fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:54.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.619+0000 7f5b6f200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b7015b8b0 0x7f5b7014fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:33282/0 (socket says 172.21.15.167:33282) 2024-05-11T12:11:54.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.619+0000 7f5b6f200640 1 -- 172.21.15.167:0/2726711844 learned_addr learned my addr 172.21.15.167:0/2726711844 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:54.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.620+0000 7f5b6f200640 1 -- 172.21.15.167:0/2726711844 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b70072150 msgr2=0x7f5b7015e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:54.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.620+0000 7f5b6e800640 1 --2- 172.21.15.167:0/2726711844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b7006cce0 0x7f5b7015bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:54.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.620+0000 7f5b67e00640 1 --2- 172.21.15.167:0/2726711844 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b70072150 0x7f5b7015e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:54.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.620+0000 7f5b6f200640 1 --2- 172.21.15.167:0/2726711844 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b70072150 0x7f5b7015e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:54.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.620+0000 7f5b6f200640 1 -- 172.21.15.167:0/2726711844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b7006cce0 msgr2=0x7f5b7015bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:54.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.620+0000 7f5b6f200640 1 --2- 172.21.15.167:0/2726711844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b7006cce0 0x7f5b7015bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:54.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.620+0000 7f5b6f200640 1 -- 172.21.15.167:0/2726711844 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b70150250 con 0x7f5b7015b8b0 2024-05-11T12:11:54.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.620+0000 7f5b67e00640 1 --2- 172.21.15.167:0/2726711844 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b70072150 0x7f5b7015e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:54.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.620+0000 7f5b6e800640 1 --2- 172.21.15.167:0/2726711844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b7006cce0 0x7f5b7015bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:54.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.620+0000 7f5b6f200640 1 --2- 172.21.15.167:0/2726711844 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b7015b8b0 0x7f5b7014fae0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5b68009d90 tx=0x7f5b68004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:54.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.621+0000 7f5b66a00640 1 -- 172.21.15.167:0/2726711844 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b68043070 con 0x7f5b7015b8b0 2024-05-11T12:11:54.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.621+0000 7f5b66a00640 1 -- 172.21.15.167:0/2726711844 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b68034c80 con 0x7f5b7015b8b0 2024-05-11T12:11:54.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.621+0000 7f5b75400640 1 -- 172.21.15.167:0/2726711844 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b70150530 con 0x7f5b7015b8b0 2024-05-11T12:11:54.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.621+0000 7f5b66a00640 1 -- 172.21.15.167:0/2726711844 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b68009450 con 0x7f5b7015b8b0 2024-05-11T12:11:54.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.621+0000 7f5b75400640 1 -- 172.21.15.167:0/2726711844 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b70150af0 con 0x7f5b7015b8b0 2024-05-11T12:11:54.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.622+0000 7f5b75400640 1 -- 172.21.15.167:0/2726711844 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b70069590 con 0x7f5b7015b8b0 2024-05-11T12:11:54.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.623+0000 7f5b66a00640 1 -- 172.21.15.167:0/2726711844 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5b6803c040 con 0x7f5b7015b8b0 2024-05-11T12:11:54.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.624+0000 7f5b66a00640 1 --2- 172.21.15.167:0/2726711844 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5b4c076320 0x7f5b4c0787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:54.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.624+0000 7f5b66a00640 1 -- 172.21.15.167:0/2726711844 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7f5b680ba1c0 con 0x7f5b7015b8b0 2024-05-11T12:11:54.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.624+0000 7f5b6e800640 1 --2- 172.21.15.167:0/2726711844 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5b4c076320 0x7f5b4c0787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:54.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.625+0000 7f5b6e800640 1 --2- 172.21.15.167:0/2726711844 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5b4c076320 0x7f5b4c0787e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5b60000f60 tx=0x7f5b60009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:54.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.628+0000 7f5b66a00640 1 -- 172.21.15.167:0/2726711844 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5b68083d50 con 0x7f5b7015b8b0 2024-05-11T12:11:54.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:54.835+0000 7f5b75400640 1 -- 172.21.15.167:0/2726711844 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f5b700744d0 con 0x7f5b7015b8b0 2024-05-11T12:11:55.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.359+0000 7f5b66a00640 1 -- 172.21.15.167:0/2726711844 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v38) ==== 138+0+0 (secure 0 0 0) 0x7f5b68083650 con 0x7f5b7015b8b0 2024-05-11T12:11:55.360 INFO:teuthology.orchestra.run.smithi167.stderr:pool 'cephfs_metadata' created 2024-05-11T12:11:55.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.363+0000 7f5b75400640 1 -- 172.21.15.167:0/2726711844 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5b4c076320 msgr2=0x7f5b4c0787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:55.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.363+0000 7f5b75400640 1 --2- 172.21.15.167:0/2726711844 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5b4c076320 0x7f5b4c0787e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5b60000f60 tx=0x7f5b60009040 comp rx=0 tx=0).stop 2024-05-11T12:11:55.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.363+0000 7f5b75400640 1 -- 172.21.15.167:0/2726711844 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b7015b8b0 msgr2=0x7f5b7014fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:55.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.363+0000 7f5b75400640 1 --2- 172.21.15.167:0/2726711844 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b7015b8b0 0x7f5b7014fae0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5b68009d90 tx=0x7f5b68004a40 comp rx=0 tx=0).stop 2024-05-11T12:11:55.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.364+0000 7f5b75400640 1 -- 172.21.15.167:0/2726711844 shutdown_connections 2024-05-11T12:11:55.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.364+0000 7f5b75400640 1 --2- 172.21.15.167:0/2726711844 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b7015b8b0 0x7f5b7014fae0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:55.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.364+0000 7f5b75400640 1 --2- 172.21.15.167:0/2726711844 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b70072150 0x7f5b7015e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:55.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.364+0000 7f5b75400640 1 --2- 172.21.15.167:0/2726711844 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5b4c076320 0x7f5b4c0787e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:55.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.364+0000 7f5b75400640 1 --2- 172.21.15.167:0/2726711844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b7006cce0 0x7f5b7015bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:55.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.364+0000 7f5b75400640 1 -- 172.21.15.167:0/2726711844 >> 172.21.15.167:0/2726711844 conn(0x7f5b700d0c70 msgr2=0x7f5b70148850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:55.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.364+0000 7f5b75400640 1 -- 172.21.15.167:0/2726711844 shutdown_connections 2024-05-11T12:11:55.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.364+0000 7f5b75400640 1 -- 172.21.15.167:0/2726711844 wait complete. 2024-05-11T12:11:55.394 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-11T12:11:55.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.589+0000 7f5017e00640 1 Processor -- start 2024-05-11T12:11:55.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.590+0000 7f5017e00640 1 -- start start 2024-05-11T12:11:55.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.591+0000 7f5017e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f501814c3f0 0x7f501806ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:55.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.591+0000 7f5017e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f50181672e0 0x7f501806d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:55.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.591+0000 7f5017e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f501806d9a0 0x7f50180b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:55.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.591+0000 7f5017e00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f501814c7d0 con 0x7f50181672e0 2024-05-11T12:11:55.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.591+0000 7f5017e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f5018167a50 con 0x7f501806d9a0 2024-05-11T12:11:55.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.591+0000 7f5017e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5018167d30 con 0x7f501814c3f0 2024-05-11T12:11:55.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.591+0000 7f500fe00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f50181672e0 0x7f501806d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:55.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.592+0000 7f500fe00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f50181672e0 0x7f501806d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:33288/0 (socket says 172.21.15.167:33288) 2024-05-11T12:11:55.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.592+0000 7f500fe00640 1 -- 172.21.15.167:0/3593094876 learned_addr learned my addr 172.21.15.167:0/3593094876 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:55.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.592+0000 7f5016a00640 1 --2- 172.21.15.167:0/3593094876 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f501814c3f0 0x7f501806ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:55.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.592+0000 7f5017400640 1 --2- 172.21.15.167:0/3593094876 >> v2:172.21.15.204:3300/0 conn(0x7f501806d9a0 0x7f50180b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:55.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.592+0000 7f500fe00640 1 -- 172.21.15.167:0/3593094876 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f501814c3f0 msgr2=0x7f501806ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:55.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.592+0000 7f500fe00640 1 --2- 172.21.15.167:0/3593094876 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f501814c3f0 0x7f501806ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:55.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.592+0000 7f500fe00640 1 -- 172.21.15.167:0/3593094876 >> v2:172.21.15.204:3300/0 conn(0x7f501806d9a0 msgr2=0x7f50180b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:55.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.592+0000 7f500fe00640 1 --2- 172.21.15.167:0/3593094876 >> v2:172.21.15.204:3300/0 conn(0x7f501806d9a0 0x7f50180b3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:55.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.592+0000 7f500fe00640 1 -- 172.21.15.167:0/3593094876 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5018168010 con 0x7f50181672e0 2024-05-11T12:11:55.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.593+0000 7f500fe00640 1 --2- 172.21.15.167:0/3593094876 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50181672e0 0x7f501806d460 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f50180b61f0 tx=0x7f500002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98b22163ca963b14 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:55.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.593+0000 7f5016000640 1 -- 172.21.15.167:0/3593094876 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5000037070 con 0x7f50181672e0 2024-05-11T12:11:55.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.593+0000 7f5016000640 1 -- 172.21.15.167:0/3593094876 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5000032440 con 0x7f50181672e0 2024-05-11T12:11:55.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.593+0000 7f5016000640 1 -- 172.21.15.167:0/3593094876 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5000032810 con 0x7f50181672e0 2024-05-11T12:11:55.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.594+0000 7f5017e00640 1 -- 172.21.15.167:0/3593094876 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50181672e0 msgr2=0x7f501806d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:55.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.594+0000 7f5017e00640 1 --2- 172.21.15.167:0/3593094876 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50181672e0 0x7f501806d460 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f50180b61f0 tx=0x7f500002a1e0 comp rx=0 tx=0).stop 2024-05-11T12:11:55.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.594+0000 7f5017e00640 1 -- 172.21.15.167:0/3593094876 shutdown_connections 2024-05-11T12:11:55.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.594+0000 7f5017e00640 1 --2- 172.21.15.167:0/3593094876 >> v2:172.21.15.204:3300/0 conn(0x7f501806d9a0 0x7f50180b3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:55.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.594+0000 7f5017e00640 1 --2- 172.21.15.167:0/3593094876 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50181672e0 0x7f501806d460 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:55.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.594+0000 7f5017e00640 1 --2- 172.21.15.167:0/3593094876 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f501814c3f0 0x7f501806ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:55.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.594+0000 7f5017e00640 1 -- 172.21.15.167:0/3593094876 >> 172.21.15.167:0/3593094876 conn(0x7f50180b32a0 msgr2=0x7f50180d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:55.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.594+0000 7f5017e00640 1 -- 172.21.15.167:0/3593094876 shutdown_connections 2024-05-11T12:11:55.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.594+0000 7f5017e00640 1 -- 172.21.15.167:0/3593094876 wait complete. 2024-05-11T12:11:55.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.596+0000 7f5017e00640 1 Processor -- start 2024-05-11T12:11:55.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.596+0000 7f5017e00640 1 -- start start 2024-05-11T12:11:55.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.597+0000 7f5017e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f501806d9a0 0x7f501813da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:55.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.597+0000 7f5017e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f501814c3f0 0x7f50180abaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:55.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.597+0000 7f5017e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50181672e0 0x7f50180ac030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:55.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.597+0000 7f5017e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f50180b4080 con 0x7f501806d9a0 2024-05-11T12:11:55.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.597+0000 7f5017e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5018167740 con 0x7f501814c3f0 2024-05-11T12:11:55.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.597+0000 7f5017e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f50180b4200 con 0x7f50181672e0 2024-05-11T12:11:55.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.597+0000 7f5016a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f501806d9a0 0x7f501813da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:55.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.597+0000 7f5016a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f501806d9a0 0x7f501813da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:33298/0 (socket says 172.21.15.167:33298) 2024-05-11T12:11:55.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.597+0000 7f5016a00640 1 -- 172.21.15.167:0/264619663 learned_addr learned my addr 172.21.15.167:0/264619663 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:55.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.598+0000 7f5016a00640 1 -- 172.21.15.167:0/264619663 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50181672e0 msgr2=0x7f50180ac030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:55.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.598+0000 7f5016a00640 1 --2- 172.21.15.167:0/264619663 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50181672e0 0x7f50180ac030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:55.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.598+0000 7f5016a00640 1 -- 172.21.15.167:0/264619663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f501814c3f0 msgr2=0x7f50180abaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:55.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.598+0000 7f5016a00640 1 --2- 172.21.15.167:0/264619663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f501814c3f0 0x7f50180abaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:55.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.598+0000 7f5016a00640 1 -- 172.21.15.167:0/264619663 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5018147de0 con 0x7f501806d9a0 2024-05-11T12:11:55.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.598+0000 7f5016a00640 1 --2- 172.21.15.167:0/264619663 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f501806d9a0 0x7f501813da00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f500800c820 tx=0x7f500800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:55.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.598+0000 7f500ea00640 1 -- 172.21.15.167:0/264619663 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5008015070 con 0x7f501806d9a0 2024-05-11T12:11:55.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.598+0000 7f500ea00640 1 -- 172.21.15.167:0/264619663 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5008004240 con 0x7f501806d9a0 2024-05-11T12:11:55.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.598+0000 7f500ea00640 1 -- 172.21.15.167:0/264619663 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500800ce10 con 0x7f501806d9a0 2024-05-11T12:11:55.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.599+0000 7f5017e00640 1 -- 172.21.15.167:0/264619663 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5018163350 con 0x7f501806d9a0 2024-05-11T12:11:55.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.599+0000 7f5017e00640 1 -- 172.21.15.167:0/264619663 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5018161ac0 con 0x7f501806d9a0 2024-05-11T12:11:55.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.600+0000 7f5017e00640 1 -- 172.21.15.167:0/264619663 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50180694b0 con 0x7f501806d9a0 2024-05-11T12:11:55.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.601+0000 7f500ea00640 1 -- 172.21.15.167:0/264619663 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5008007830 con 0x7f501806d9a0 2024-05-11T12:11:55.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.602+0000 7f500ea00640 1 --2- 172.21.15.167:0/264619663 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4ff40762d0 0x7f4ff4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:55.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.602+0000 7f500ea00640 1 -- 172.21.15.167:0/264619663 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5619+0+0 (secure 0 0 0) 0x7f50080973a0 con 0x7f501806d9a0 2024-05-11T12:11:55.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.602+0000 7f500fe00640 1 --2- 172.21.15.167:0/264619663 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4ff40762d0 0x7f4ff4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:55.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.603+0000 7f500fe00640 1 --2- 172.21.15.167:0/264619663 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4ff40762d0 0x7f4ff4078790 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5000034000 tx=0x7f500002c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:55.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.608+0000 7f500ea00640 1 -- 172.21.15.167:0/264619663 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5018147de0 con 0x7f501806d9a0 2024-05-11T12:11:55.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:55.818+0000 7f5017e00640 1 -- 172.21.15.167:0/264619663 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f50180d0e40 con 0x7f501806d9a0 2024-05-11T12:11:56.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.366+0000 7f500ea00640 1 -- 172.21.15.167:0/264619663 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v39) ==== 170+0+0 (secure 0 0 0) 0x7f5008060d40 con 0x7f501806d9a0 2024-05-11T12:11:56.367 INFO:teuthology.orchestra.run.smithi167.stderr:pool 'cephfs_data' created 2024-05-11T12:11:56.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.370+0000 7f5017e00640 1 -- 172.21.15.167:0/264619663 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4ff40762d0 msgr2=0x7f4ff4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:56.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.370+0000 7f5017e00640 1 --2- 172.21.15.167:0/264619663 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4ff40762d0 0x7f4ff4078790 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5000034000 tx=0x7f500002c000 comp rx=0 tx=0).stop 2024-05-11T12:11:56.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.370+0000 7f5017e00640 1 -- 172.21.15.167:0/264619663 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f501806d9a0 msgr2=0x7f501813da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:56.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.370+0000 7f5017e00640 1 --2- 172.21.15.167:0/264619663 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f501806d9a0 0x7f501813da00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f500800c820 tx=0x7f500800cce0 comp rx=0 tx=0).stop 2024-05-11T12:11:56.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.371+0000 7f5017e00640 1 -- 172.21.15.167:0/264619663 shutdown_connections 2024-05-11T12:11:56.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.371+0000 7f5017e00640 1 --2- 172.21.15.167:0/264619663 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50181672e0 0x7f50180ac030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:56.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.371+0000 7f5017e00640 1 --2- 172.21.15.167:0/264619663 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4ff40762d0 0x7f4ff4078790 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:56.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.371+0000 7f5017e00640 1 --2- 172.21.15.167:0/264619663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f501814c3f0 0x7f50180abaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:56.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.371+0000 7f5017e00640 1 --2- 172.21.15.167:0/264619663 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f501806d9a0 0x7f501813da00 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:56.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.371+0000 7f5017e00640 1 -- 172.21.15.167:0/264619663 >> 172.21.15.167:0/264619663 conn(0x7f50180b32a0 msgr2=0x7f50180b0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:56.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.372+0000 7f5017e00640 1 -- 172.21.15.167:0/264619663 shutdown_connections 2024-05-11T12:11:56.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.372+0000 7f5017e00640 1 -- 172.21.15.167:0/264619663 wait complete. 2024-05-11T12:11:56.406 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-11T12:11:56.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.582+0000 7fd086000640 1 Processor -- start 2024-05-11T12:11:56.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.583+0000 7fd086000640 1 -- start start 2024-05-11T12:11:56.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.584+0000 7fd086000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd08015ac80 0x7fd080074260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:56.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.584+0000 7fd086000640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd0800b3d50 0x7fd080160c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:56.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.584+0000 7fd086000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0800b65e0 con 0x7fd0801611b0 2024-05-11T12:11:56.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.584+0000 7fd086000640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fd08006e3c0 con 0x7fd0800b3d50 2024-05-11T12:11:56.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.584+0000 7fd086000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fd080161600 con 0x7fd08015ac80 2024-05-11T12:11:56.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.584+0000 7fd077e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd0800b3d50 0x7fd080160c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:56.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.585+0000 7fd077e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd0800b3d50 0x7fd080160c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:33304/0 (socket says 172.21.15.167:33304) 2024-05-11T12:11:56.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.585+0000 7fd077e00640 1 -- 172.21.15.167:0/924489025 learned_addr learned my addr 172.21.15.167:0/924489025 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:56.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.585+0000 7fd07f400640 1 --2- 172.21.15.167:0/924489025 >> v2:172.21.15.204:3300/0 conn(0x7fd08015ac80 0x7fd080074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:56.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.585+0000 7fd077e00640 1 -- 172.21.15.167:0/924489025 >> v2:172.21.15.204:3300/0 conn(0x7fd08015ac80 msgr2=0x7fd080074260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:56.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.585+0000 7fd077e00640 1 --2- 172.21.15.167:0/924489025 >> v2:172.21.15.204:3300/0 conn(0x7fd08015ac80 0x7fd080074260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:56.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.585+0000 7fd077e00640 1 -- 172.21.15.167:0/924489025 >> v1:172.21.15.204:6789/0 conn(0x7fd0801611b0 legacy=0x7fd08015d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:56.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.585+0000 7fd077e00640 1 -- 172.21.15.167:0/924489025 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0800b68c0 con 0x7fd0800b3d50 2024-05-11T12:11:56.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.586+0000 7fd077e00640 1 --2- 172.21.15.167:0/924489025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0800b3d50 0x7fd080160c70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd068009870 tx=0x7fd06802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b7ab9b61e8324a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:56.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.586+0000 7fd07ea00640 1 -- 172.21.15.167:0/924489025 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd068037070 con 0x7fd0800b3d50 2024-05-11T12:11:56.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.586+0000 7fd07ea00640 1 -- 172.21.15.167:0/924489025 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd068032440 con 0x7fd0800b3d50 2024-05-11T12:11:56.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.587+0000 7fd07ea00640 1 -- 172.21.15.167:0/924489025 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd068032810 con 0x7fd0800b3d50 2024-05-11T12:11:56.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.587+0000 7fd086000640 1 -- 172.21.15.167:0/924489025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0800b3d50 msgr2=0x7fd080160c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:56.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.587+0000 7fd086000640 1 --2- 172.21.15.167:0/924489025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0800b3d50 0x7fd080160c70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd068009870 tx=0x7fd06802a210 comp rx=0 tx=0).stop 2024-05-11T12:11:56.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.587+0000 7fd086000640 1 -- 172.21.15.167:0/924489025 shutdown_connections 2024-05-11T12:11:56.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.587+0000 7fd086000640 1 --2- 172.21.15.167:0/924489025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0800b3d50 0x7fd080160c70 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:56.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.587+0000 7fd086000640 1 --2- 172.21.15.167:0/924489025 >> v2:172.21.15.204:3300/0 conn(0x7fd08015ac80 0x7fd080074260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:56.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.587+0000 7fd086000640 1 -- 172.21.15.167:0/924489025 >> 172.21.15.167:0/924489025 conn(0x7fd08006c450 msgr2=0x7fd08006c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:56.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.587+0000 7fd086000640 1 -- 172.21.15.167:0/924489025 shutdown_connections 2024-05-11T12:11:56.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.587+0000 7fd086000640 1 -- 172.21.15.167:0/924489025 wait complete. 2024-05-11T12:11:56.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.589+0000 7fd086000640 1 Processor -- start 2024-05-11T12:11:56.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.589+0000 7fd086000640 1 -- start start 2024-05-11T12:11:56.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.590+0000 7fd086000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0800b3d50 0x7fd080138590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:56.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.590+0000 7fd086000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd08015ac80 0x7fd0800ac2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:56.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.590+0000 7fd086000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0801611b0 0x7fd080143f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:56.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.590+0000 7fd086000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd0800b41e0 con 0x7fd0801611b0 2024-05-11T12:11:56.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.590+0000 7fd086000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd08015b060 con 0x7fd08015ac80 2024-05-11T12:11:56.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.590+0000 7fd086000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd0800b44f0 con 0x7fd0800b3d50 2024-05-11T12:11:56.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.590+0000 7fd07fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0801611b0 0x7fd080143f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:56.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.590+0000 7fd07fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0801611b0 0x7fd080143f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:33310/0 (socket says 172.21.15.167:33310) 2024-05-11T12:11:56.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.590+0000 7fd07fe00640 1 -- 172.21.15.167:0/96301556 learned_addr learned my addr 172.21.15.167:0/96301556 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:56.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.591+0000 7fd077e00640 1 --2- 172.21.15.167:0/96301556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd08015ac80 0x7fd0800ac2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:56.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.591+0000 7fd07fe00640 1 -- 172.21.15.167:0/96301556 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0800b3d50 msgr2=0x7fd080138590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:56.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.591+0000 7fd07fe00640 1 --2- 172.21.15.167:0/96301556 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0800b3d50 0x7fd080138590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:56.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.591+0000 7fd07fe00640 1 -- 172.21.15.167:0/96301556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd08015ac80 msgr2=0x7fd0800ac2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:56.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.591+0000 7fd07fe00640 1 --2- 172.21.15.167:0/96301556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd08015ac80 0x7fd0800ac2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:56.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.591+0000 7fd07fe00640 1 -- 172.21.15.167:0/96301556 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd080138ad0 con 0x7fd0801611b0 2024-05-11T12:11:56.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.591+0000 7fd07fe00640 1 --2- 172.21.15.167:0/96301556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0801611b0 0x7fd080143f60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd07800b970 tx=0x7fd07800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:56.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.591+0000 7fd077400640 1 -- 172.21.15.167:0/96301556 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd078019070 con 0x7fd0801611b0 2024-05-11T12:11:56.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.591+0000 7fd077400640 1 -- 172.21.15.167:0/96301556 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd07800cb40 con 0x7fd0801611b0 2024-05-11T12:11:56.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.591+0000 7fd077400640 1 -- 172.21.15.167:0/96301556 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd078014440 con 0x7fd0801611b0 2024-05-11T12:11:56.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.591+0000 7fd086000640 1 -- 172.21.15.167:0/96301556 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0800aa7f0 con 0x7fd0801611b0 2024-05-11T12:11:56.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.592+0000 7fd086000640 1 -- 172.21.15.167:0/96301556 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0800aadb0 con 0x7fd0801611b0 2024-05-11T12:11:56.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.593+0000 7fd086000640 1 -- 172.21.15.167:0/96301556 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd080069520 con 0x7fd0801611b0 2024-05-11T12:11:56.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.595+0000 7fd077400640 1 -- 172.21.15.167:0/96301556 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd07801c070 con 0x7fd0801611b0 2024-05-11T12:11:56.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.595+0000 7fd077400640 1 --2- 172.21.15.167:0/96301556 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd05c0762d0 0x7fd05c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:56.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.596+0000 7fd077400640 1 -- 172.21.15.167:0/96301556 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6001+0+0 (secure 0 0 0) 0x7fd078097380 con 0x7fd0801611b0 2024-05-11T12:11:56.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.596+0000 7fd07f400640 1 --2- 172.21.15.167:0/96301556 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd05c0762d0 0x7fd05c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:56.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.596+0000 7fd07f400640 1 --2- 172.21.15.167:0/96301556 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd05c0762d0 0x7fd05c078790 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd0700098f0 tx=0x7fd070008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:56.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.601+0000 7fd077400640 1 -- 172.21.15.167:0/96301556 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd078060b10 con 0x7fd0801611b0 2024-05-11T12:11:56.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:56.850+0000 7fd086000640 1 -- 172.21.15.167:0/96301556 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd080070120 con 0x7fd0801611b0 2024-05-11T12:11:57.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.412+0000 7fd077400640 1 -- 172.21.15.167:0/96301556 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) ==== 172+0+0 (secure 0 0 0) 0x7fd078060410 con 0x7fd0801611b0 2024-05-11T12:11:57.413 INFO:teuthology.orchestra.run.smithi167.stderr:new fs with metadata pool 5 and data pool 6 2024-05-11T12:11:57.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.423+0000 7fd086000640 1 -- 172.21.15.167:0/96301556 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd05c0762d0 msgr2=0x7fd05c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:57.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.423+0000 7fd086000640 1 --2- 172.21.15.167:0/96301556 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd05c0762d0 0x7fd05c078790 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd0700098f0 tx=0x7fd070008040 comp rx=0 tx=0).stop 2024-05-11T12:11:57.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.423+0000 7fd086000640 1 -- 172.21.15.167:0/96301556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0801611b0 msgr2=0x7fd080143f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:57.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.423+0000 7fd086000640 1 --2- 172.21.15.167:0/96301556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0801611b0 0x7fd080143f60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd07800b970 tx=0x7fd07800be30 comp rx=0 tx=0).stop 2024-05-11T12:11:57.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.423+0000 7fd086000640 1 -- 172.21.15.167:0/96301556 shutdown_connections 2024-05-11T12:11:57.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.423+0000 7fd086000640 1 --2- 172.21.15.167:0/96301556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0801611b0 0x7fd080143f60 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:57.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.423+0000 7fd086000640 1 --2- 172.21.15.167:0/96301556 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd05c0762d0 0x7fd05c078790 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:57.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.423+0000 7fd086000640 1 --2- 172.21.15.167:0/96301556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd08015ac80 0x7fd0800ac2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:57.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.423+0000 7fd086000640 1 --2- 172.21.15.167:0/96301556 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0800b3d50 0x7fd080138590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:57.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.423+0000 7fd086000640 1 -- 172.21.15.167:0/96301556 >> 172.21.15.167:0/96301556 conn(0x7fd08006c450 msgr2=0x7fd08006f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:57.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.425+0000 7fd086000640 1 -- 172.21.15.167:0/96301556 shutdown_connections 2024-05-11T12:11:57.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.425+0000 7fd086000640 1 -- 172.21.15.167:0/96301556 wait complete. 2024-05-11T12:11:57.457 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:11:57.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.628+0000 7f1ec9600640 1 Processor -- start 2024-05-11T12:11:57.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.628+0000 7f1ec9600640 1 -- start start 2024-05-11T12:11:57.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.630+0000 7f1ec9600640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1ec4148a50 0x7f1ec40d4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:57.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.630+0000 7f1ec9600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1ec40b95a0 0x7f1ec415f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:57.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.630+0000 7f1ec9600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ec406d600 con 0x7f1ec406ce60 2024-05-11T12:11:57.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.630+0000 7f1ec9600640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f1ec406e210 con 0x7f1ec4148a50 2024-05-11T12:11:57.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.630+0000 7f1ec9600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1ec406d8e0 con 0x7f1ec40b95a0 2024-05-11T12:11:57.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.630+0000 7f1ec3e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1ec4148a50 0x7f1ec40d4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:57.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.630+0000 7f1ec3e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1ec4148a50 0x7f1ec40d4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:33312/0 (socket says 172.21.15.167:33312) 2024-05-11T12:11:57.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.630+0000 7f1ec3e00640 1 -- 172.21.15.167:0/243769259 learned_addr learned my addr 172.21.15.167:0/243769259 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:57.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.631+0000 7f1ec8c00640 1 --2- 172.21.15.167:0/243769259 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1ec40b95a0 0x7f1ec415f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:57.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.631+0000 7f1ec3e00640 1 -- 172.21.15.167:0/243769259 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1ec40b95a0 msgr2=0x7f1ec415f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:57.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.631+0000 7f1ec3e00640 1 --2- 172.21.15.167:0/243769259 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1ec40b95a0 0x7f1ec415f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:57.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.631+0000 7f1ec3e00640 1 -- 172.21.15.167:0/243769259 >> v1:172.21.15.204:6789/0 conn(0x7f1ec406ce60 legacy=0x7f1ec40b8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:57.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.631+0000 7f1ec3e00640 1 -- 172.21.15.167:0/243769259 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec406dbc0 con 0x7f1ec4148a50 2024-05-11T12:11:57.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.632+0000 7f1ec3e00640 1 --2- 172.21.15.167:0/243769259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ec4148a50 0x7f1ec40d4940 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4009870 tx=0x7f1eb402a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58e56bba1872b38c server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:57.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.632+0000 7f1ec3400640 1 -- 172.21.15.167:0/243769259 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4037070 con 0x7f1ec4148a50 2024-05-11T12:11:57.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.632+0000 7f1ec3400640 1 -- 172.21.15.167:0/243769259 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1eb4032440 con 0x7f1ec4148a50 2024-05-11T12:11:57.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.632+0000 7f1ec3400640 1 -- 172.21.15.167:0/243769259 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4032810 con 0x7f1ec4148a50 2024-05-11T12:11:57.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.633+0000 7f1ec9600640 1 -- 172.21.15.167:0/243769259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ec4148a50 msgr2=0x7f1ec40d4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:57.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.633+0000 7f1ec9600640 1 --2- 172.21.15.167:0/243769259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ec4148a50 0x7f1ec40d4940 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4009870 tx=0x7f1eb402a130 comp rx=0 tx=0).stop 2024-05-11T12:11:57.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.633+0000 7f1ec9600640 1 -- 172.21.15.167:0/243769259 shutdown_connections 2024-05-11T12:11:57.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.633+0000 7f1ec9600640 1 --2- 172.21.15.167:0/243769259 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1ec40b95a0 0x7f1ec415f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:57.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.633+0000 7f1ec9600640 1 --2- 172.21.15.167:0/243769259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ec4148a50 0x7f1ec40d4940 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:57.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.633+0000 7f1ec9600640 1 -- 172.21.15.167:0/243769259 >> 172.21.15.167:0/243769259 conn(0x7f1ec40d4f10 msgr2=0x7f1ec40d5320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:57.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.633+0000 7f1ec9600640 1 -- 172.21.15.167:0/243769259 shutdown_connections 2024-05-11T12:11:57.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.633+0000 7f1ec9600640 1 -- 172.21.15.167:0/243769259 wait complete. 2024-05-11T12:11:57.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.635+0000 7f1ec9600640 1 Processor -- start 2024-05-11T12:11:57.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.635+0000 7f1ec9600640 1 -- start start 2024-05-11T12:11:57.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.636+0000 7f1ec9600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ec406ce60 0x7f1ec415cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:57.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.636+0000 7f1ec9600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ec40b95a0 0x7f1ec415af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:57.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.636+0000 7f1ec9600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1ec4148a50 0x7f1ec415dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:57.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.636+0000 7f1ec9600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec4148e30 con 0x7f1ec40b95a0 2024-05-11T12:11:57.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.636+0000 7f1ec9600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec406d2c0 con 0x7f1ec406ce60 2024-05-11T12:11:57.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.636+0000 7f1ec9600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1ec406e6a0 con 0x7f1ec4148a50 2024-05-11T12:11:57.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.636+0000 7f1ebb400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ec40b95a0 0x7f1ec415af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:57.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.636+0000 7f1ebb400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ec40b95a0 0x7f1ec415af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:33318/0 (socket says 172.21.15.167:33318) 2024-05-11T12:11:57.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.636+0000 7f1ebb400640 1 -- 172.21.15.167:0/1476376978 learned_addr learned my addr 172.21.15.167:0/1476376978 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:57.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.636+0000 7f1ec3e00640 1 --2- 172.21.15.167:0/1476376978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ec406ce60 0x7f1ec415cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:57.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.636+0000 7f1ec8c00640 1 --2- 172.21.15.167:0/1476376978 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1ec4148a50 0x7f1ec415dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:57.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.636+0000 7f1ebb400640 1 -- 172.21.15.167:0/1476376978 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1ec4148a50 msgr2=0x7f1ec415dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:57.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.636+0000 7f1ebb400640 1 --2- 172.21.15.167:0/1476376978 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1ec4148a50 0x7f1ec415dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:57.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.636+0000 7f1ebb400640 1 -- 172.21.15.167:0/1476376978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ec406ce60 msgr2=0x7f1ec415cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:57.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.636+0000 7f1ebb400640 1 --2- 172.21.15.167:0/1476376978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ec406ce60 0x7f1ec415cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:57.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.636+0000 7f1ebb400640 1 -- 172.21.15.167:0/1476376978 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec415ba90 con 0x7f1ec40b95a0 2024-05-11T12:11:57.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.637+0000 7f1ebb400640 1 --2- 172.21.15.167:0/1476376978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ec40b95a0 0x7f1ec415af00 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1eac00c430 tx=0x7f1eac00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:57.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.637+0000 7f1ec0c00640 1 -- 172.21.15.167:0/1476376978 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eac011790 con 0x7f1ec40b95a0 2024-05-11T12:11:57.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.637+0000 7f1ec0c00640 1 -- 172.21.15.167:0/1476376978 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1eac017070 con 0x7f1ec40b95a0 2024-05-11T12:11:57.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.637+0000 7f1ec0c00640 1 -- 172.21.15.167:0/1476376978 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eac011b70 con 0x7f1ec40b95a0 2024-05-11T12:11:57.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.637+0000 7f1ec9600640 1 -- 172.21.15.167:0/1476376978 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ec415bd70 con 0x7f1ec40b95a0 2024-05-11T12:11:57.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.637+0000 7f1ec9600640 1 -- 172.21.15.167:0/1476376978 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ec415c330 con 0x7f1ec40b95a0 2024-05-11T12:11:57.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.638+0000 7f1ec9600640 1 -- 172.21.15.167:0/1476376978 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ec40694b0 con 0x7f1ec40b95a0 2024-05-11T12:11:57.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.639+0000 7f1ec0c00640 1 -- 172.21.15.167:0/1476376978 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1eac011d10 con 0x7f1ec40b95a0 2024-05-11T12:11:57.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.640+0000 7f1ec0c00640 1 --2- 172.21.15.167:0/1476376978 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f1ea00762d0 0x7f1ea0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:57.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.640+0000 7f1ec0c00640 1 -- 172.21.15.167:0/1476376978 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f1eac0975d0 con 0x7f1ec40b95a0 2024-05-11T12:11:57.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.641+0000 7f1ec3e00640 1 --2- 172.21.15.167:0/1476376978 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f1ea00762d0 0x7f1ea0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:57.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.641+0000 7f1ec3e00640 1 --2- 172.21.15.167:0/1476376978 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f1ea00762d0 0x7f1ea0078790 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4034000 tx=0x7f1eb402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:57.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.645+0000 7f1ec0c00640 1 -- 172.21.15.167:0/1476376978 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1eac060d90 con 0x7f1ec40b95a0 2024-05-11T12:11:57.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.851+0000 7f1ec9600640 1 -- 172.21.15.167:0/1476376978 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1ec40b9e20 con 0x7f1ec40b95a0 2024-05-11T12:11:57.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.854+0000 7f1ec0c00640 1 -- 172.21.15.167:0/1476376978 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+17003 (secure 0 0 0) 0x7f1eac060690 con 0x7f1ec40b95a0 2024-05-11T12:11:57.854 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:11:57.854 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":40,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:11:56.852551+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:11:54.849622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-11T12:11:55.831334+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:11:57.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.856+0000 7f1ec9600640 1 -- 172.21.15.167:0/1476376978 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f1ea00762d0 msgr2=0x7f1ea0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:57.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.856+0000 7f1ec9600640 1 --2- 172.21.15.167:0/1476376978 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f1ea00762d0 0x7f1ea0078790 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4034000 tx=0x7f1eb402e000 comp rx=0 tx=0).stop 2024-05-11T12:11:57.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.856+0000 7f1ec9600640 1 -- 172.21.15.167:0/1476376978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ec40b95a0 msgr2=0x7f1ec415af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:57.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.856+0000 7f1ec9600640 1 --2- 172.21.15.167:0/1476376978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ec40b95a0 0x7f1ec415af00 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1eac00c430 tx=0x7f1eac00c8f0 comp rx=0 tx=0).stop 2024-05-11T12:11:57.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.856+0000 7f1ec9600640 1 -- 172.21.15.167:0/1476376978 shutdown_connections 2024-05-11T12:11:57.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.856+0000 7f1ec9600640 1 --2- 172.21.15.167:0/1476376978 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1ec4148a50 0x7f1ec415dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:57.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.856+0000 7f1ec9600640 1 --2- 172.21.15.167:0/1476376978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ec40b95a0 0x7f1ec415af00 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:57.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.856+0000 7f1ec9600640 1 --2- 172.21.15.167:0/1476376978 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f1ea00762d0 0x7f1ea0078790 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:57.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.856+0000 7f1ec9600640 1 --2- 172.21.15.167:0/1476376978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ec406ce60 0x7f1ec415cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:57.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.856+0000 7f1ec9600640 1 -- 172.21.15.167:0/1476376978 >> 172.21.15.167:0/1476376978 conn(0x7f1ec40d4f10 msgr2=0x7f1ec4146c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:57.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.857+0000 7f1ec9600640 1 -- 172.21.15.167:0/1476376978 shutdown_connections 2024-05-11T12:11:57.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:57.857+0000 7f1ec9600640 1 -- 172.21.15.167:0/1476376978 wait complete. 2024-05-11T12:11:57.876 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:11:58.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.057+0000 7f4b96200640 1 Processor -- start 2024-05-11T12:11:58.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.057+0000 7f4b96200640 1 -- start start 2024-05-11T12:11:58.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.059+0000 7f4b96200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4b90157750 0x7f4b9015d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:58.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.059+0000 7f4b96200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b900b3ea0 0x7f4b90161490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:58.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.059+0000 7f4b96200640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b9015dc80 con 0x7f4b901619d0 2024-05-11T12:11:58.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.059+0000 7f4b96200640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f4b90161000 con 0x7f4b90157750 2024-05-11T12:11:58.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.059+0000 7f4b96200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4b9015df30 con 0x7f4b900b3ea0 2024-05-11T12:11:58.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.059+0000 7f4b8f400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4b90157750 0x7f4b9015d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:58.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.059+0000 7f4b8fe00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f4b901619d0 0x7f4b9006d460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:57724/0 (socket says 172.21.15.167:57724) 2024-05-11T12:11:58.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.059+0000 7f4b8fe00640 1 -- 172.21.15.167:0/303271219 learned_addr learned my addr 172.21.15.167:0/303271219 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:58.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.059+0000 7f4b8ea00640 1 --2- 172.21.15.167:0/303271219 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b900b3ea0 0x7f4b90161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:58.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.059+0000 7f4b8f400640 1 -- 172.21.15.167:0/303271219 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b900b3ea0 msgr2=0x7f4b90161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:58.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.059+0000 7f4b8f400640 1 --2- 172.21.15.167:0/303271219 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b900b3ea0 0x7f4b90161490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:58.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.059+0000 7f4b8f400640 1 -- 172.21.15.167:0/303271219 >> v1:172.21.15.167:6789/0 conn(0x7f4b901619d0 legacy=0x7f4b9006d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:58.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.059+0000 7f4b8f400640 1 -- 172.21.15.167:0/303271219 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b9015e210 con 0x7f4b90157750 2024-05-11T12:11:58.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.059+0000 7f4b8e000640 1 -- 172.21.15.167:0/303271219 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1039442725 0 0) 0x7f4b9015dc80 con 0x7f4b901619d0 2024-05-11T12:11:58.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.061+0000 7f4b8f400640 1 --2- 172.21.15.167:0/303271219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b90157750 0x7f4b9015d740 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4b800097a0 tx=0x7f4b8002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbd3653c17ec58e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:58.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.061+0000 7f4b8e000640 1 -- 172.21.15.167:0/303271219 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b80037070 con 0x7f4b90157750 2024-05-11T12:11:58.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.061+0000 7f4b8e000640 1 -- 172.21.15.167:0/303271219 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b80032440 con 0x7f4b90157750 2024-05-11T12:11:58.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.061+0000 7f4b8e000640 1 -- 172.21.15.167:0/303271219 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b80032810 con 0x7f4b90157750 2024-05-11T12:11:58.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.061+0000 7f4b96200640 1 -- 172.21.15.167:0/303271219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b90157750 msgr2=0x7f4b9015d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:58.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.061+0000 7f4b96200640 1 --2- 172.21.15.167:0/303271219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b90157750 0x7f4b9015d740 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4b800097a0 tx=0x7f4b8002a180 comp rx=0 tx=0).stop 2024-05-11T12:11:58.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.061+0000 7f4b96200640 1 -- 172.21.15.167:0/303271219 shutdown_connections 2024-05-11T12:11:58.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.061+0000 7f4b96200640 1 --2- 172.21.15.167:0/303271219 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b900b3ea0 0x7f4b90161490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:58.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.061+0000 7f4b96200640 1 --2- 172.21.15.167:0/303271219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b90157750 0x7f4b9015d740 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:58.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.061+0000 7f4b96200640 1 -- 172.21.15.167:0/303271219 >> 172.21.15.167:0/303271219 conn(0x7f4b900cd8f0 msgr2=0x7f4b900cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:58.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.062+0000 7f4b96200640 1 -- 172.21.15.167:0/303271219 shutdown_connections 2024-05-11T12:11:58.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.062+0000 7f4b96200640 1 -- 172.21.15.167:0/303271219 wait complete. 2024-05-11T12:11:58.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.063+0000 7f4b96200640 1 Processor -- start 2024-05-11T12:11:58.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.063+0000 7f4b96200640 1 -- start start 2024-05-11T12:11:58.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.064+0000 7f4b96200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b900b3ea0 0x7f4b901376d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:58.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.064+0000 7f4b96200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b90157750 0x7f4b9015b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:58.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.064+0000 7f4b96200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b901619d0 0x7f4b9015d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:58.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.064+0000 7f4b96200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9006db70 con 0x7f4b900b3ea0 2024-05-11T12:11:58.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.064+0000 7f4b96200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4b90157b30 con 0x7f4b90157750 2024-05-11T12:11:58.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.064+0000 7f4b96200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4b9006dcf0 con 0x7f4b901619d0 2024-05-11T12:11:58.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.064+0000 7f4b8f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b900b3ea0 0x7f4b901376d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:58.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.065+0000 7f4b8f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b900b3ea0 0x7f4b901376d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:33336/0 (socket says 172.21.15.167:33336) 2024-05-11T12:11:58.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.065+0000 7f4b8f400640 1 -- 172.21.15.167:0/2983649709 learned_addr learned my addr 172.21.15.167:0/2983649709 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:58.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.065+0000 7f4b8f400640 1 -- 172.21.15.167:0/2983649709 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b901619d0 msgr2=0x7f4b9015d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:58.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.065+0000 7f4b8f400640 1 --2- 172.21.15.167:0/2983649709 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b901619d0 0x7f4b9015d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:58.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.065+0000 7f4b8f400640 1 -- 172.21.15.167:0/2983649709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b90157750 msgr2=0x7f4b9015b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:58.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.065+0000 7f4b8f400640 1 --2- 172.21.15.167:0/2983649709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b90157750 0x7f4b9015b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:58.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.065+0000 7f4b8f400640 1 -- 172.21.15.167:0/2983649709 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b90159970 con 0x7f4b900b3ea0 2024-05-11T12:11:58.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.065+0000 7f4b8f400640 1 --2- 172.21.15.167:0/2983649709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b900b3ea0 0x7f4b901376d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4b80000c00 tx=0x7f4b800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:58.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.066+0000 7f4b87400640 1 -- 172.21.15.167:0/2983649709 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b80037040 con 0x7f4b900b3ea0 2024-05-11T12:11:58.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.066+0000 7f4b87400640 1 -- 172.21.15.167:0/2983649709 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b80042070 con 0x7f4b900b3ea0 2024-05-11T12:11:58.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.066+0000 7f4b96200640 1 -- 172.21.15.167:0/2983649709 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b90159c50 con 0x7f4b900b3ea0 2024-05-11T12:11:58.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.066+0000 7f4b87400640 1 -- 172.21.15.167:0/2983649709 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b80030a70 con 0x7f4b900b3ea0 2024-05-11T12:11:58.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.066+0000 7f4b96200640 1 -- 172.21.15.167:0/2983649709 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b9015a210 con 0x7f4b900b3ea0 2024-05-11T12:11:58.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.067+0000 7f4b96200640 1 -- 172.21.15.167:0/2983649709 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b90069590 con 0x7f4b900b3ea0 2024-05-11T12:11:58.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.068+0000 7f4b87400640 1 -- 172.21.15.167:0/2983649709 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4b80030c10 con 0x7f4b900b3ea0 2024-05-11T12:11:58.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.069+0000 7f4b87400640 1 --2- 172.21.15.167:0/2983649709 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4b700762d0 0x7f4b70078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:58.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.069+0000 7f4b87400640 1 -- 172.21.15.167:0/2983649709 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f4b800b81c0 con 0x7f4b900b3ea0 2024-05-11T12:11:58.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.069+0000 7f4b8ea00640 1 --2- 172.21.15.167:0/2983649709 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4b700762d0 0x7f4b70078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:58.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.070+0000 7f4b8ea00640 1 --2- 172.21.15.167:0/2983649709 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4b700762d0 0x7f4b70078790 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c006fd0 tx=0x7f4b7c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:58.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.073+0000 7f4b87400640 1 -- 172.21.15.167:0/2983649709 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4b80081a20 con 0x7f4b900b3ea0 2024-05-11T12:11:58.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.282+0000 7f4b96200640 1 -- 172.21.15.167:0/2983649709 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4b900b4570 con 0x7f4b900b3ea0 2024-05-11T12:11:58.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.284+0000 7f4b87400640 1 -- 172.21.15.167:0/2983649709 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+17003 (secure 0 0 0) 0x7f4b80081320 con 0x7f4b900b3ea0 2024-05-11T12:11:58.285 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:11:58.285 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":40,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:11:56.852551+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:11:54.849622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-11T12:11:55.831334+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:11:58.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.287+0000 7f4b96200640 1 -- 172.21.15.167:0/2983649709 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4b700762d0 msgr2=0x7f4b70078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:58.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.287+0000 7f4b96200640 1 --2- 172.21.15.167:0/2983649709 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4b700762d0 0x7f4b70078790 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c006fd0 tx=0x7f4b7c008040 comp rx=0 tx=0).stop 2024-05-11T12:11:58.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.287+0000 7f4b96200640 1 -- 172.21.15.167:0/2983649709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b900b3ea0 msgr2=0x7f4b901376d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:58.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.287+0000 7f4b96200640 1 --2- 172.21.15.167:0/2983649709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b900b3ea0 0x7f4b901376d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4b80000c00 tx=0x7f4b800046c0 comp rx=0 tx=0).stop 2024-05-11T12:11:58.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.287+0000 7f4b96200640 1 -- 172.21.15.167:0/2983649709 shutdown_connections 2024-05-11T12:11:58.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.287+0000 7f4b96200640 1 --2- 172.21.15.167:0/2983649709 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b901619d0 0x7f4b9015d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:58.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.287+0000 7f4b96200640 1 --2- 172.21.15.167:0/2983649709 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4b700762d0 0x7f4b70078790 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:58.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.287+0000 7f4b96200640 1 --2- 172.21.15.167:0/2983649709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b90157750 0x7f4b9015b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:58.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.287+0000 7f4b96200640 1 --2- 172.21.15.167:0/2983649709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b900b3ea0 0x7f4b901376d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:58.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.287+0000 7f4b96200640 1 -- 172.21.15.167:0/2983649709 >> 172.21.15.167:0/2983649709 conn(0x7f4b900cd8f0 msgr2=0x7f4b900cd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:58.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.287+0000 7f4b96200640 1 -- 172.21.15.167:0/2983649709 shutdown_connections 2024-05-11T12:11:58.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.287+0000 7f4b96200640 1 -- 172.21.15.167:0/2983649709 wait complete. 2024-05-11T12:11:58.305 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-11T12:11:58.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.515+0000 7fc4b4a00640 1 Processor -- start 2024-05-11T12:11:58.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.515+0000 7fc4b4a00640 1 -- start start 2024-05-11T12:11:58.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.517+0000 7fc4b4a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc4b006cef0 0x7fc4b006d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:58.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.517+0000 7fc4b4a00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4b00d2c50 con 0x7fc4b015f0d0 2024-05-11T12:11:58.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.517+0000 7fc4b4a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4b00d2f30 con 0x7fc4b00cfda0 2024-05-11T12:11:58.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.517+0000 7fc4b4a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc4b00d3210 con 0x7fc4b006cef0 2024-05-11T12:11:58.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.517+0000 7fc4af400640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fc4b015f0d0 0x7fc4b00d0ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:57734/0 (socket says 172.21.15.167:57734) 2024-05-11T12:11:58.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.517+0000 7fc4af400640 1 -- 172.21.15.167:0/422670772 learned_addr learned my addr 172.21.15.167:0/422670772 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:58.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.517+0000 7fc4afe00640 1 --2- 172.21.15.167:0/422670772 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc4b006cef0 0x7fc4b006d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:58.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.517+0000 7fc4aea00640 1 -- 172.21.15.167:0/422670772 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2270196990 0 0) 0x7fc4b00d2c50 con 0x7fc4b015f0d0 2024-05-11T12:11:58.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.517+0000 7fc4aea00640 1 -- 172.21.15.167:0/422670772 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc494003520 con 0x7fc4b015f0d0 2024-05-11T12:11:58.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.518+0000 7fc4aea00640 1 -- 172.21.15.167:0/422670772 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 163509002 0 0) 0x7fc494003520 con 0x7fc4b015f0d0 2024-05-11T12:11:58.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.518+0000 7fc4aea00640 1 -- 172.21.15.167:0/422670772 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc4b006cef0 msgr2=0x7fc4b006d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:58.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.518+0000 7fc4aea00640 1 --2- 172.21.15.167:0/422670772 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc4b006cef0 0x7fc4b006d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:58.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.518+0000 7fc4aea00640 1 -- 172.21.15.167:0/422670772 >> v1:172.21.15.204:6789/0 conn(0x7fc4b00cfda0 legacy=0x7fc4b00d3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:58.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.518+0000 7fc4aea00640 1 -- 172.21.15.167:0/422670772 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b00d34f0 con 0x7fc4b015f0d0 2024-05-11T12:11:58.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.518+0000 7fc4aea00640 1 -- 172.21.15.167:0/422670772 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7fc4a00016f0 con 0x7fc4b015f0d0 2024-05-11T12:11:58.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.518+0000 7fc4aea00640 1 -- 172.21.15.167:0/422670772 >> v1:172.21.15.167:6789/0 conn(0x7fc4b015f0d0 legacy=0x7fc4b00d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:58.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.518+0000 7fc4aea00640 1 --2- 172.21.15.167:0/422670772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc494003a80 0x7fc494003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:58.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.518+0000 7fc4aea00640 1 --2- 172.21.15.167:0/422670772 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc4b006cef0 0x7fc494004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:58.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.518+0000 7fc4aea00640 1 --2- 172.21.15.167:0/422670772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc494004e60 0x7fc49400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:58.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.518+0000 7fc4aea00640 1 -- 172.21.15.167:0/422670772 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b00d3210 con 0x7fc494003a80 2024-05-11T12:11:58.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.518+0000 7fc4aea00640 1 -- 172.21.15.167:0/422670772 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc4940054f0 con 0x7fc494004e60 2024-05-11T12:11:58.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.518+0000 7fc4aea00640 1 -- 172.21.15.167:0/422670772 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc49400bf00 con 0x7fc4b006cef0 2024-05-11T12:11:58.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.518+0000 7fc4af400640 1 --2- 172.21.15.167:0/422670772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc494003a80 0x7fc494003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:58.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.519+0000 7fc4afe00640 1 --2- 172.21.15.167:0/422670772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc494004e60 0x7fc49400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:58.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.519+0000 7fc4a7e00640 1 --2- 172.21.15.167:0/422670772 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc4b006cef0 0x7fc494004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:58.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.519+0000 7fc4af400640 1 -- 172.21.15.167:0/422670772 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc4b006cef0 msgr2=0x7fc494004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:58.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.519+0000 7fc4af400640 1 --2- 172.21.15.167:0/422670772 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc4b006cef0 0x7fc494004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:58.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.519+0000 7fc4af400640 1 -- 172.21.15.167:0/422670772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc494004e60 msgr2=0x7fc49400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:58.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.519+0000 7fc4af400640 1 --2- 172.21.15.167:0/422670772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc494004e60 0x7fc49400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:58.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.519+0000 7fc4af400640 1 -- 172.21.15.167:0/422670772 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc494003520 con 0x7fc494003a80 2024-05-11T12:11:58.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.519+0000 7fc4afe00640 1 --2- 172.21.15.167:0/422670772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc494004e60 0x7fc49400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:11:58.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.520+0000 7fc4af400640 1 --2- 172.21.15.167:0/422670772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc494003a80 0x7fc494003ea0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc4a000c210 tx=0x7fc494005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0dc45d9b76eb98 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:58.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.520+0000 7fc4aea00640 1 -- 172.21.15.167:0/422670772 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a0007ce0 con 0x7fc494003a80 2024-05-11T12:11:58.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.520+0000 7fc4aea00640 1 -- 172.21.15.167:0/422670772 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4a0005af0 con 0x7fc494003a80 2024-05-11T12:11:58.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.520+0000 7fc4aea00640 1 -- 172.21.15.167:0/422670772 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a0007070 con 0x7fc494003a80 2024-05-11T12:11:58.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.520+0000 7fc4b4a00640 1 -- 172.21.15.167:0/422670772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc494003a80 msgr2=0x7fc494003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:58.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.520+0000 7fc4b4a00640 1 --2- 172.21.15.167:0/422670772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc494003a80 0x7fc494003ea0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc4a000c210 tx=0x7fc494005830 comp rx=0 tx=0).stop 2024-05-11T12:11:58.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.520+0000 7fc4afe00640 1 -- 172.21.15.167:0/422670772 reap_dead start 2024-05-11T12:11:58.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.521+0000 7fc4b4a00640 1 -- 172.21.15.167:0/422670772 shutdown_connections 2024-05-11T12:11:58.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.521+0000 7fc4b4a00640 1 -- 172.21.15.167:0/422670772 >> 172.21.15.167:0/422670772 conn(0x7fc4b00b6af0 msgr2=0x7fc4b00b6f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:58.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.521+0000 7fc4b4a00640 1 -- 172.21.15.167:0/422670772 shutdown_connections 2024-05-11T12:11:58.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.521+0000 7fc4b4a00640 1 -- 172.21.15.167:0/422670772 wait complete. 2024-05-11T12:11:58.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.522+0000 7fc4b4a00640 1 Processor -- start 2024-05-11T12:11:58.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.523+0000 7fc4b4a00640 1 -- start start 2024-05-11T12:11:58.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.523+0000 7fc4b4a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc494003a80 0x7fc4b0162270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:58.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.524+0000 7fc4b4a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4b015ccd0 0x7fc4b0152040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:58.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.524+0000 7fc4b4a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4b015d150 0x7fc4b0152580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:58.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.524+0000 7fc4b4a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b00d4400 con 0x7fc4b015ccd0 2024-05-11T12:11:58.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.524+0000 7fc4b4a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b015f4b0 con 0x7fc4b015d150 2024-05-11T12:11:58.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.524+0000 7fc4b4a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc4b00d4580 con 0x7fc494003a80 2024-05-11T12:11:58.525 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.524+0000 7fc4a7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4b015ccd0 0x7fc4b0152040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:58.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.524+0000 7fc4a7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4b015ccd0 0x7fc4b0152040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:33354/0 (socket says 172.21.15.167:33354) 2024-05-11T12:11:58.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.524+0000 7fc4a7e00640 1 -- 172.21.15.167:0/3451725123 learned_addr learned my addr 172.21.15.167:0/3451725123 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:58.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.524+0000 7fc4af400640 1 --2- 172.21.15.167:0/3451725123 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc494003a80 0x7fc4b0162270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:58.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.524+0000 7fc4afe00640 1 --2- 172.21.15.167:0/3451725123 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4b015d150 0x7fc4b0152580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:58.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.525+0000 7fc4a7e00640 1 -- 172.21.15.167:0/3451725123 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc494003a80 msgr2=0x7fc4b0162270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:58.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.525+0000 7fc4a7e00640 1 --2- 172.21.15.167:0/3451725123 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc494003a80 0x7fc4b0162270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:58.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.525+0000 7fc4a7e00640 1 -- 172.21.15.167:0/3451725123 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4b015d150 msgr2=0x7fc4b0152580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:58.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.525+0000 7fc4a7e00640 1 --2- 172.21.15.167:0/3451725123 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4b015d150 0x7fc4b0152580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:58.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.525+0000 7fc4a7e00640 1 -- 172.21.15.167:0/3451725123 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b0160a40 con 0x7fc4b015ccd0 2024-05-11T12:11:58.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.525+0000 7fc4a7e00640 1 --2- 172.21.15.167:0/3451725123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4b015ccd0 0x7fc4b0152040 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc49800d5a0 tx=0x7fc49800da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:58.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.525+0000 7fc4a7400640 1 -- 172.21.15.167:0/3451725123 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc498005480 con 0x7fc4b015ccd0 2024-05-11T12:11:58.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.525+0000 7fc4a7400640 1 -- 172.21.15.167:0/3451725123 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc498017070 con 0x7fc4b015ccd0 2024-05-11T12:11:58.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.525+0000 7fc4a7400640 1 -- 172.21.15.167:0/3451725123 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc498003810 con 0x7fc4b015ccd0 2024-05-11T12:11:58.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.525+0000 7fc4b4a00640 1 -- 172.21.15.167:0/3451725123 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4b0160d20 con 0x7fc4b015ccd0 2024-05-11T12:11:58.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.525+0000 7fc4b4a00640 1 -- 172.21.15.167:0/3451725123 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4b01612e0 con 0x7fc4b015ccd0 2024-05-11T12:11:58.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.526+0000 7fc4b4a00640 1 -- 172.21.15.167:0/3451725123 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4b00694b0 con 0x7fc4b015ccd0 2024-05-11T12:11:58.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.527+0000 7fc4a7400640 1 -- 172.21.15.167:0/3451725123 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc4980039b0 con 0x7fc4b015ccd0 2024-05-11T12:11:58.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.528+0000 7fc4a7400640 1 --2- 172.21.15.167:0/3451725123 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc48c0762d0 0x7fc48c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:58.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.529+0000 7fc4a7400640 1 -- 172.21.15.167:0/3451725123 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7fc4980975e0 con 0x7fc4b015ccd0 2024-05-11T12:11:58.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.529+0000 7fc4af400640 1 --2- 172.21.15.167:0/3451725123 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc48c0762d0 0x7fc48c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:58.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.529+0000 7fc4af400640 1 --2- 172.21.15.167:0/3451725123 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc48c0762d0 0x7fc48c078790 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0032000 tx=0x7fc4a0009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:58.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.533+0000 7fc4a7400640 1 -- 172.21.15.167:0/3451725123 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc498060d10 con 0x7fc4b015ccd0 2024-05-11T12:11:58.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:58.782+0000 7fc4b4a00640 1 -- 172.21.15.167:0/3451725123 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fc4b006d500 con 0x7fc4b015ccd0 2024-05-11T12:11:59.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.427+0000 7fc4a7400640 1 -- 172.21.15.167:0/3451725123 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7fc498060610 con 0x7fc4b015ccd0 2024-05-11T12:11:59.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.431+0000 7fc4b4a00640 1 -- 172.21.15.167:0/3451725123 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc48c0762d0 msgr2=0x7fc48c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:59.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.431+0000 7fc4b4a00640 1 --2- 172.21.15.167:0/3451725123 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc48c0762d0 0x7fc48c078790 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0032000 tx=0x7fc4a0009030 comp rx=0 tx=0).stop 2024-05-11T12:11:59.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.431+0000 7fc4b4a00640 1 -- 172.21.15.167:0/3451725123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4b015ccd0 msgr2=0x7fc4b0152040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:59.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.431+0000 7fc4b4a00640 1 --2- 172.21.15.167:0/3451725123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4b015ccd0 0x7fc4b0152040 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc49800d5a0 tx=0x7fc49800da60 comp rx=0 tx=0).stop 2024-05-11T12:11:59.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.431+0000 7fc4b4a00640 1 -- 172.21.15.167:0/3451725123 shutdown_connections 2024-05-11T12:11:59.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.431+0000 7fc4b4a00640 1 --2- 172.21.15.167:0/3451725123 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc48c0762d0 0x7fc48c078790 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:59.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.431+0000 7fc4b4a00640 1 --2- 172.21.15.167:0/3451725123 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4b015d150 0x7fc4b0152580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:59.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.431+0000 7fc4b4a00640 1 --2- 172.21.15.167:0/3451725123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4b015ccd0 0x7fc4b0152040 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:59.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.431+0000 7fc4b4a00640 1 --2- 172.21.15.167:0/3451725123 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc494003a80 0x7fc4b0162270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:59.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.431+0000 7fc4b4a00640 1 -- 172.21.15.167:0/3451725123 >> 172.21.15.167:0/3451725123 conn(0x7fc4b00b6af0 msgr2=0x7fc4b00b9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:59.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.432+0000 7fc4b4a00640 1 -- 172.21.15.167:0/3451725123 shutdown_connections 2024-05-11T12:11:59.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.432+0000 7fc4b4a00640 1 -- 172.21.15.167:0/3451725123 wait complete. 2024-05-11T12:11:59.453 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-11T12:11:59.453 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-11T12:11:59.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.651+0000 7fd9bd600640 1 Processor -- start 2024-05-11T12:11:59.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.651+0000 7fd9bd600640 1 -- start start 2024-05-11T12:11:59.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.652+0000 7fd9bd600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd9b815f0d0 0x7fd9b80d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:59.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.653+0000 7fd9bd600640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9b80d2c50 con 0x7fd9b80cfda0 2024-05-11T12:11:59.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.653+0000 7fd9bd600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9b80d2f30 con 0x7fd9b806cef0 2024-05-11T12:11:59.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.653+0000 7fd9bd600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fd9b80d3210 con 0x7fd9b815f0d0 2024-05-11T12:11:59.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.653+0000 7fd9b7400640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fd9b80cfda0 0x7fd9b80d3cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:42146/0 (socket says 172.21.15.167:42146) 2024-05-11T12:11:59.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.653+0000 7fd9b7400640 1 -- 172.21.15.167:0/2324683004 learned_addr learned my addr 172.21.15.167:0/2324683004 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:11:59.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.653+0000 7fd9b7e00640 1 --2- 172.21.15.167:0/2324683004 >> v2:172.21.15.204:3300/0 conn(0x7fd9b815f0d0 0x7fd9b80d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:59.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.653+0000 7fd9b6a00640 1 -- 172.21.15.167:0/2324683004 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1257030544 0 0) 0x7fd9b80d2c50 con 0x7fd9b80cfda0 2024-05-11T12:11:59.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.653+0000 7fd9b6a00640 1 -- 172.21.15.167:0/2324683004 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd99c003520 con 0x7fd9b80cfda0 2024-05-11T12:11:59.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.654+0000 7fd9b6a00640 1 -- 172.21.15.167:0/2324683004 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4038957605 0 0) 0x7fd99c003520 con 0x7fd9b80cfda0 2024-05-11T12:11:59.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.654+0000 7fd9b6a00640 1 -- 172.21.15.167:0/2324683004 >> v2:172.21.15.204:3300/0 conn(0x7fd9b815f0d0 msgr2=0x7fd9b80d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:59.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.654+0000 7fd9b6a00640 1 --2- 172.21.15.167:0/2324683004 >> v2:172.21.15.204:3300/0 conn(0x7fd9b815f0d0 0x7fd9b80d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:59.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.654+0000 7fd9b6a00640 1 -- 172.21.15.167:0/2324683004 >> v1:172.21.15.204:6789/0 conn(0x7fd9b806cef0 legacy=0x7fd9b80b81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:59.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.654+0000 7fd9b6a00640 1 -- 172.21.15.167:0/2324683004 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9b80d34f0 con 0x7fd9b80cfda0 2024-05-11T12:11:59.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.654+0000 7fd9b6a00640 1 -- 172.21.15.167:0/2324683004 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7fd9a00016f0 con 0x7fd9b80cfda0 2024-05-11T12:11:59.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.654+0000 7fd9b6a00640 1 -- 172.21.15.167:0/2324683004 >> v1:172.21.15.167:6789/0 conn(0x7fd9b80cfda0 legacy=0x7fd9b80d3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:59.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.654+0000 7fd9b6a00640 1 --2- 172.21.15.167:0/2324683004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd99c003a00 0x7fd99c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:59.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.654+0000 7fd9b6a00640 1 --2- 172.21.15.167:0/2324683004 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd99c004800 0x7fd99c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:59.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.654+0000 7fd9b6a00640 1 --2- 172.21.15.167:0/2324683004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd99c005210 0x7fd99c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:59.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.654+0000 7fd9b6a00640 1 -- 172.21.15.167:0/2324683004 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b80d3210 con 0x7fd99c005210 2024-05-11T12:11:59.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.654+0000 7fd9b6a00640 1 -- 172.21.15.167:0/2324683004 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c00c140 con 0x7fd99c003a00 2024-05-11T12:11:59.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.654+0000 7fd9b6a00640 1 -- 172.21.15.167:0/2324683004 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd99c00c340 con 0x7fd99c004800 2024-05-11T12:11:59.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.654+0000 7fd9bcc00640 1 --2- 172.21.15.167:0/2324683004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd99c005210 0x7fd99c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:59.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.655+0000 7fd9b7e00640 1 --2- 172.21.15.167:0/2324683004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd99c003a00 0x7fd99c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:59.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.655+0000 7fd9bcc00640 1 -- 172.21.15.167:0/2324683004 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd99c004800 msgr2=0x7fd99c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:59.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.655+0000 7fd9bcc00640 1 --2- 172.21.15.167:0/2324683004 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd99c004800 0x7fd99c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:59.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.655+0000 7fd9bcc00640 1 -- 172.21.15.167:0/2324683004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd99c003a00 msgr2=0x7fd99c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:11:59.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.655+0000 7fd9bcc00640 1 --2- 172.21.15.167:0/2324683004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd99c003a00 0x7fd99c003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:59.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.655+0000 7fd9bcc00640 1 -- 172.21.15.167:0/2324683004 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd99c003520 con 0x7fd99c005210 2024-05-11T12:11:59.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.655+0000 7fd9bcc00640 1 -- 172.21.15.167:0/2324683004 reap_dead start 2024-05-11T12:11:59.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.656+0000 7fd9bcc00640 1 --2- 172.21.15.167:0/2324683004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd99c005210 0x7fd99c00b9c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0003d80 tx=0x7fd99c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d01b31e81282ac9 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:59.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.656+0000 7fd9b6a00640 1 -- 172.21.15.167:0/2324683004 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0003980 con 0x7fd99c005210 2024-05-11T12:11:59.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.656+0000 7fd9b6a00640 1 -- 172.21.15.167:0/2324683004 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9b0007240 con 0x7fd99c005210 2024-05-11T12:11:59.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.656+0000 7fd9b6a00640 1 -- 172.21.15.167:0/2324683004 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0007630 con 0x7fd99c005210 2024-05-11T12:11:59.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.656+0000 7fd9bd600640 1 -- 172.21.15.167:0/2324683004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd99c005210 msgr2=0x7fd99c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:59.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.656+0000 7fd9bd600640 1 --2- 172.21.15.167:0/2324683004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd99c005210 0x7fd99c00b9c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0003d80 tx=0x7fd99c005830 comp rx=0 tx=0).stop 2024-05-11T12:11:59.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.657+0000 7fd9bd600640 1 -- 172.21.15.167:0/2324683004 shutdown_connections 2024-05-11T12:11:59.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.657+0000 7fd9bd600640 1 --2- 172.21.15.167:0/2324683004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd99c005210 0x7fd99c00b9c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:59.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.657+0000 7fd9bd600640 1 -- 172.21.15.167:0/2324683004 >> 172.21.15.167:0/2324683004 conn(0x7fd9b80b6af0 msgr2=0x7fd9b80b6f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:11:59.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.657+0000 7fd9bd600640 1 -- 172.21.15.167:0/2324683004 shutdown_connections 2024-05-11T12:11:59.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.657+0000 7fd9bd600640 1 -- 172.21.15.167:0/2324683004 wait complete. 2024-05-11T12:11:59.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.659+0000 7fd9bd600640 1 Processor -- start 2024-05-11T12:11:59.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.659+0000 7fd9bd600640 1 -- start start 2024-05-11T12:11:59.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.660+0000 7fd9bd600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd99c005210 0x7fd9b8150ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:59.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.660+0000 7fd9bd600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9b80ccec0 0x7fd9b815fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:59.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.660+0000 7fd9bd600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9b8161a80 0x7fd9b8161f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:59.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.660+0000 7fd9bd600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b806d360 con 0x7fd9b80ccec0 2024-05-11T12:11:59.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.660+0000 7fd9bd600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b815f4b0 con 0x7fd9b8161a80 2024-05-11T12:11:59.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.660+0000 7fd9bd600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd9b806d4e0 con 0x7fd99c005210 2024-05-11T12:11:59.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.661+0000 7fd9b7400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9b80ccec0 0x7fd9b815fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:59.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.661+0000 7fd9b7400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9b80ccec0 0x7fd9b815fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41792/0 (socket says 172.21.15.167:41792) 2024-05-11T12:11:59.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.661+0000 7fd9b7400640 1 -- 172.21.15.167:0/3613537438 learned_addr learned my addr 172.21.15.167:0/3613537438 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:11:59.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.661+0000 7fd9b7e00640 1 --2- 172.21.15.167:0/3613537438 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd99c005210 0x7fd9b8150ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:59.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.661+0000 7fd9b7400640 1 -- 172.21.15.167:0/3613537438 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd99c005210 msgr2=0x7fd9b8150ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:59.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.661+0000 7fd9b7400640 1 --2- 172.21.15.167:0/3613537438 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd99c005210 0x7fd9b8150ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:59.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.661+0000 7fd9b7400640 1 -- 172.21.15.167:0/3613537438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9b8161a80 msgr2=0x7fd9b8161f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:11:59.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.661+0000 7fd9bcc00640 1 --2- 172.21.15.167:0/3613537438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9b8161a80 0x7fd9b8161f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:59.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.661+0000 7fd9b7400640 1 --2- 172.21.15.167:0/3613537438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9b8161a80 0x7fd9b8161f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:11:59.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.661+0000 7fd9b7400640 1 -- 172.21.15.167:0/3613537438 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9b80d0ea0 con 0x7fd9b80ccec0 2024-05-11T12:11:59.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.661+0000 7fd9b7400640 1 --2- 172.21.15.167:0/3613537438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9b80ccec0 0x7fd9b815fe40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd9a000c760 tx=0x7fd9a000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:59.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.662+0000 7fd9afe00640 1 -- 172.21.15.167:0/3613537438 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a00074e0 con 0x7fd9b80ccec0 2024-05-11T12:11:59.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.662+0000 7fd9afe00640 1 -- 172.21.15.167:0/3613537438 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9a000ce10 con 0x7fd9b80ccec0 2024-05-11T12:11:59.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.662+0000 7fd9afe00640 1 -- 172.21.15.167:0/3613537438 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a0004a20 con 0x7fd9b80ccec0 2024-05-11T12:11:59.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.662+0000 7fd9bd600640 1 -- 172.21.15.167:0/3613537438 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9b80d1140 con 0x7fd9b80ccec0 2024-05-11T12:11:59.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.662+0000 7fd9bd600640 1 -- 172.21.15.167:0/3613537438 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9b80d1700 con 0x7fd9b80ccec0 2024-05-11T12:11:59.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.663+0000 7fd9bd600640 1 -- 172.21.15.167:0/3613537438 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9b80694b0 con 0x7fd9b80ccec0 2024-05-11T12:11:59.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.664+0000 7fd9afe00640 1 -- 172.21.15.167:0/3613537438 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd9a0011070 con 0x7fd9b80ccec0 2024-05-11T12:11:59.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.665+0000 7fd9afe00640 1 --2- 172.21.15.167:0/3613537438 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd9940762d0 0x7fd994078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:11:59.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.666+0000 7fd9afe00640 1 -- 172.21.15.167:0/3613537438 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7fd9a00974e0 con 0x7fd9b80ccec0 2024-05-11T12:11:59.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.666+0000 7fd9b7e00640 1 --2- 172.21.15.167:0/3613537438 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd9940762d0 0x7fd994078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:11:59.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.666+0000 7fd9b7e00640 1 --2- 172.21.15.167:0/3613537438 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd9940762d0 0x7fd994078790 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8009df0 tx=0x7fd9a800b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:11:59.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.670+0000 7fd9afe00640 1 -- 172.21.15.167:0/3613537438 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd9a0060c10 con 0x7fd9b80ccec0 2024-05-11T12:11:59.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:11:59.923+0000 7fd9bd600640 1 -- 172.21.15.167:0/3613537438 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fd9b806c450 con 0x7fd9b80ccec0 2024-05-11T12:12:00.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.437+0000 7fd9afe00640 1 -- 172.21.15.167:0/3613537438 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7fd9a0060510 con 0x7fd9b80ccec0 2024-05-11T12:12:00.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.442+0000 7fd9bd600640 1 -- 172.21.15.167:0/3613537438 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd9940762d0 msgr2=0x7fd994078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:00.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.442+0000 7fd9bd600640 1 --2- 172.21.15.167:0/3613537438 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd9940762d0 0x7fd994078790 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8009df0 tx=0x7fd9a800b040 comp rx=0 tx=0).stop 2024-05-11T12:12:00.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.442+0000 7fd9bd600640 1 -- 172.21.15.167:0/3613537438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9b80ccec0 msgr2=0x7fd9b815fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:00.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.442+0000 7fd9bd600640 1 --2- 172.21.15.167:0/3613537438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9b80ccec0 0x7fd9b815fe40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd9a000c760 tx=0x7fd9a000cc20 comp rx=0 tx=0).stop 2024-05-11T12:12:00.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.442+0000 7fd9bd600640 1 -- 172.21.15.167:0/3613537438 shutdown_connections 2024-05-11T12:12:00.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.442+0000 7fd9bd600640 1 --2- 172.21.15.167:0/3613537438 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd9940762d0 0x7fd994078790 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:00.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.442+0000 7fd9bd600640 1 --2- 172.21.15.167:0/3613537438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9b8161a80 0x7fd9b8161f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:00.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.442+0000 7fd9bd600640 1 --2- 172.21.15.167:0/3613537438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9b80ccec0 0x7fd9b815fe40 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:00.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.442+0000 7fd9bd600640 1 --2- 172.21.15.167:0/3613537438 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd99c005210 0x7fd9b8150ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:00.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.442+0000 7fd9bd600640 1 -- 172.21.15.167:0/3613537438 >> 172.21.15.167:0/3613537438 conn(0x7fd9b80b6af0 msgr2=0x7fd9b80b6270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:00.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.442+0000 7fd9bd600640 1 -- 172.21.15.167:0/3613537438 shutdown_connections 2024-05-11T12:12:00.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.442+0000 7fd9bd600640 1 -- 172.21.15.167:0/3613537438 wait complete. 2024-05-11T12:12:00.465 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-11T12:12:00.465 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:12:00.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.663+0000 7f5884a00640 1 Processor -- start 2024-05-11T12:12:00.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.663+0000 7f5884a00640 1 -- start start 2024-05-11T12:12:00.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.664+0000 7f5884a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f58800b3ea0 0x7f5880161490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:00.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.665+0000 7f5884a00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f588015df30 con 0x7f5880157750 2024-05-11T12:12:00.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.665+0000 7f5884a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f588015e210 con 0x7f58801619d0 2024-05-11T12:12:00.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.665+0000 7f5884a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f5880161000 con 0x7f58800b3ea0 2024-05-11T12:12:00.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.665+0000 7f587de00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f5880157750 0x7f588015d740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:42160/0 (socket says 172.21.15.167:42160) 2024-05-11T12:12:00.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.665+0000 7f587de00640 1 -- 172.21.15.167:0/3184808867 learned_addr learned my addr 172.21.15.167:0/3184808867 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:00.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.665+0000 7f587d400640 1 -- 172.21.15.167:0/3184808867 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3428196346 0 0) 0x7f588015df30 con 0x7f5880157750 2024-05-11T12:12:00.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.665+0000 7f587d400640 1 -- 172.21.15.167:0/3184808867 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5864003520 con 0x7f5880157750 2024-05-11T12:12:00.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.665+0000 7f5877e00640 1 --2- 172.21.15.167:0/3184808867 >> v2:172.21.15.204:3300/0 conn(0x7f58800b3ea0 0x7f5880161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:00.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.666+0000 7f587d400640 1 -- 172.21.15.167:0/3184808867 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3403450001 0 0) 0x7f5864003520 con 0x7f5880157750 2024-05-11T12:12:00.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.666+0000 7f587d400640 1 -- 172.21.15.167:0/3184808867 >> v2:172.21.15.204:3300/0 conn(0x7f58800b3ea0 msgr2=0x7f5880161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:00.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.666+0000 7f587d400640 1 --2- 172.21.15.167:0/3184808867 >> v2:172.21.15.204:3300/0 conn(0x7f58800b3ea0 0x7f5880161490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:00.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.666+0000 7f587d400640 1 -- 172.21.15.167:0/3184808867 >> v1:172.21.15.204:6789/0 conn(0x7f58801619d0 legacy=0x7f588006d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:00.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.666+0000 7f587d400640 1 -- 172.21.15.167:0/3184808867 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f588015e4f0 con 0x7f5880157750 2024-05-11T12:12:00.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.666+0000 7f587d400640 1 -- 172.21.15.167:0/3184808867 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f58700016f0 con 0x7f5880157750 2024-05-11T12:12:00.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.666+0000 7f587d400640 1 -- 172.21.15.167:0/3184808867 >> v1:172.21.15.167:6789/0 conn(0x7f5880157750 legacy=0x7f588015d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:00.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.666+0000 7f587d400640 1 --2- 172.21.15.167:0/3184808867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5864003a50 0x7f5864003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:00.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.666+0000 7f587d400640 1 --2- 172.21.15.167:0/3184808867 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5864004800 0x7f5864004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:00.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.666+0000 7f587d400640 1 --2- 172.21.15.167:0/3184808867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58640052e0 0x7f586400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:00.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.666+0000 7f587d400640 1 -- 172.21.15.167:0/3184808867 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5880161000 con 0x7f5864003a50 2024-05-11T12:12:00.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.666+0000 7f587d400640 1 -- 172.21.15.167:0/3184808867 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f586400c190 con 0x7f58640052e0 2024-05-11T12:12:00.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.666+0000 7f587d400640 1 -- 172.21.15.167:0/3184808867 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f586400c390 con 0x7f5864004800 2024-05-11T12:12:00.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.666+0000 7f587de00640 1 --2- 172.21.15.167:0/3184808867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5864003a50 0x7f5864003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:00.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.666+0000 7f5877e00640 1 --2- 172.21.15.167:0/3184808867 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5864004800 0x7f5864004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:00.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.666+0000 7f587de00640 1 -- 172.21.15.167:0/3184808867 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5864004800 msgr2=0x7f5864004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:00.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.666+0000 7f587e800640 1 --2- 172.21.15.167:0/3184808867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58640052e0 0x7f586400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:00.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.667+0000 7f587de00640 1 --2- 172.21.15.167:0/3184808867 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5864004800 0x7f5864004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:00.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.667+0000 7f587de00640 1 -- 172.21.15.167:0/3184808867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58640052e0 msgr2=0x7f586400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:00.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.667+0000 7f587de00640 1 --2- 172.21.15.167:0/3184808867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58640052e0 0x7f586400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:00.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.667+0000 7f587de00640 1 -- 172.21.15.167:0/3184808867 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5864003520 con 0x7f5864003a50 2024-05-11T12:12:00.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.667+0000 7f587e800640 1 -- 172.21.15.167:0/3184808867 reap_dead start 2024-05-11T12:12:00.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.668+0000 7f587de00640 1 --2- 172.21.15.167:0/3184808867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5864003a50 0x7f5864003e50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5870007fd0 tx=0x7f5864005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ede30279edec4e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:00.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.668+0000 7f587d400640 1 -- 172.21.15.167:0/3184808867 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5870007ce0 con 0x7f5864003a50 2024-05-11T12:12:00.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.668+0000 7f587d400640 1 -- 172.21.15.167:0/3184808867 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5870005af0 con 0x7f5864003a50 2024-05-11T12:12:00.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.668+0000 7f587d400640 1 -- 172.21.15.167:0/3184808867 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5870007070 con 0x7f5864003a50 2024-05-11T12:12:00.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.668+0000 7f5884a00640 1 -- 172.21.15.167:0/3184808867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5864003a50 msgr2=0x7f5864003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:00.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.668+0000 7f5884a00640 1 --2- 172.21.15.167:0/3184808867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5864003a50 0x7f5864003e50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5870007fd0 tx=0x7f5864005830 comp rx=0 tx=0).stop 2024-05-11T12:12:00.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.669+0000 7f5884a00640 1 -- 172.21.15.167:0/3184808867 shutdown_connections 2024-05-11T12:12:00.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.669+0000 7f5884a00640 1 --2- 172.21.15.167:0/3184808867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5864003a50 0x7f5864003e50 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:00.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.669+0000 7f5884a00640 1 -- 172.21.15.167:0/3184808867 >> 172.21.15.167:0/3184808867 conn(0x7f58800cd8f0 msgr2=0x7f58800cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:00.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.669+0000 7f5884a00640 1 -- 172.21.15.167:0/3184808867 shutdown_connections 2024-05-11T12:12:00.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.669+0000 7f5884a00640 1 -- 172.21.15.167:0/3184808867 wait complete. 2024-05-11T12:12:00.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.670+0000 7f5884a00640 1 Processor -- start 2024-05-11T12:12:00.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.671+0000 7f5884a00640 1 -- start start 2024-05-11T12:12:00.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.672+0000 7f5884a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5864003a50 0x7f5880135df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:00.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.672+0000 7f5884a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58801f7f00 0x7f58800af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:00.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.672+0000 7f5884a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f58800afce0 0x7f58801f8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:00.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.672+0000 7f5884a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5880161610 con 0x7f5864003a50 2024-05-11T12:12:00.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.672+0000 7f5884a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5880161490 con 0x7f58801f7f00 2024-05-11T12:12:00.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.672+0000 7f5884a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5880161790 con 0x7f58800afce0 2024-05-11T12:12:00.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.672+0000 7f587de00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5864003a50 0x7f5880135df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:00.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.672+0000 7f587de00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5864003a50 0x7f5880135df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41816/0 (socket says 172.21.15.167:41816) 2024-05-11T12:12:00.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.672+0000 7f587de00640 1 -- 172.21.15.167:0/73639689 learned_addr learned my addr 172.21.15.167:0/73639689 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:00.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.673+0000 7f5877e00640 1 --2- 172.21.15.167:0/73639689 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58801f7f00 0x7f58800af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:00.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.673+0000 7f587e800640 1 --2- 172.21.15.167:0/73639689 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f58800afce0 0x7f58801f8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:00.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.673+0000 7f587de00640 1 -- 172.21.15.167:0/73639689 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f58800afce0 msgr2=0x7f58801f8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:00.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.673+0000 7f587de00640 1 --2- 172.21.15.167:0/73639689 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f58800afce0 0x7f58801f8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:00.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.673+0000 7f587de00640 1 -- 172.21.15.167:0/73639689 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58801f7f00 msgr2=0x7f58800af7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:00.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.673+0000 7f587de00640 1 --2- 172.21.15.167:0/73639689 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58801f7f00 0x7f58800af7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:00.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.673+0000 7f587de00640 1 -- 172.21.15.167:0/73639689 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58800721f0 con 0x7f5864003a50 2024-05-11T12:12:00.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.673+0000 7f587de00640 1 --2- 172.21.15.167:0/73639689 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5864003a50 0x7f5880135df0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f587000c250 tx=0x7f588015e100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:00.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.673+0000 7f5876000640 1 -- 172.21.15.167:0/73639689 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58700016f0 con 0x7f5864003a50 2024-05-11T12:12:00.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.673+0000 7f5876000640 1 -- 172.21.15.167:0/73639689 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5870042070 con 0x7f5864003a50 2024-05-11T12:12:00.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.673+0000 7f5876000640 1 -- 172.21.15.167:0/73639689 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5870007270 con 0x7f5864003a50 2024-05-11T12:12:00.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.673+0000 7f5884a00640 1 -- 172.21.15.167:0/73639689 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58801f9440 con 0x7f5864003a50 2024-05-11T12:12:00.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.674+0000 7f5884a00640 1 -- 172.21.15.167:0/73639689 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5880072790 con 0x7f5864003a50 2024-05-11T12:12:00.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.675+0000 7f5884a00640 1 -- 172.21.15.167:0/73639689 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5880069590 con 0x7f5864003a50 2024-05-11T12:12:00.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.676+0000 7f5876000640 1 -- 172.21.15.167:0/73639689 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5870007410 con 0x7f5864003a50 2024-05-11T12:12:00.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.676+0000 7f5876000640 1 --2- 172.21.15.167:0/73639689 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f585c076300 0x7f585c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:00.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.677+0000 7f5876000640 1 -- 172.21.15.167:0/73639689 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f58700b7800 con 0x7f5864003a50 2024-05-11T12:12:00.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.677+0000 7f5877e00640 1 --2- 172.21.15.167:0/73639689 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f585c076300 0x7f585c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:00.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.678+0000 7f5877e00640 1 --2- 172.21.15.167:0/73639689 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f585c076300 0x7f585c0787c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f58800ad750 tx=0x7f5868008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:00.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.682+0000 7f5876000640 1 -- 172.21.15.167:0/73639689 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5870080fc0 con 0x7f5864003a50 2024-05-11T12:12:00.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.927+0000 7f5884a00640 1 -- 172.21.15.167:0/73639689 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58800d0900 con 0x7f5864003a50 2024-05-11T12:12:00.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.928+0000 7f5876000640 1 -- 172.21.15.167:0/73639689 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5193 (secure 0 0 0) 0x7f58700808c0 con 0x7f5864003a50 2024-05-11T12:12:00.929 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:12:00.929 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4499,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3271527772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3271527772},{"type":"v1","addr":"172.21.15.167:6836","nonce":3271527772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4648,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2293636645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2293636645},{"type":"v1","addr":"172.21.15.167:6837","nonce":2293636645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4902,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3093163428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3093163428},{"type":"v1","addr":"172.21.15.204:6834","nonce":3093163428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:11:56.852478+0000","modified":"2024-05-11T12:11:59.926074+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4908},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4908":{"gid":4908,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/3248611739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":3248611739},{"type":"v1","addr":"172.21.15.204:6835","nonce":3248611739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4908,"qdb_cluster":[4908]},"id":2}]} 2024-05-11T12:12:00.929 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 18 2024-05-11T12:12:00.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.931+0000 7f5884a00640 1 -- 172.21.15.167:0/73639689 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f585c076300 msgr2=0x7f585c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:00.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.931+0000 7f5884a00640 1 --2- 172.21.15.167:0/73639689 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f585c076300 0x7f585c0787c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f58800ad750 tx=0x7f5868008040 comp rx=0 tx=0).stop 2024-05-11T12:12:00.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.931+0000 7f5884a00640 1 -- 172.21.15.167:0/73639689 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5864003a50 msgr2=0x7f5880135df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:00.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.931+0000 7f5884a00640 1 --2- 172.21.15.167:0/73639689 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5864003a50 0x7f5880135df0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f587000c250 tx=0x7f588015e100 comp rx=0 tx=0).stop 2024-05-11T12:12:00.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.931+0000 7f5884a00640 1 -- 172.21.15.167:0/73639689 shutdown_connections 2024-05-11T12:12:00.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.931+0000 7f5884a00640 1 --2- 172.21.15.167:0/73639689 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f58800afce0 0x7f58801f8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:00.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.931+0000 7f5884a00640 1 --2- 172.21.15.167:0/73639689 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f585c076300 0x7f585c0787c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:00.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.931+0000 7f5884a00640 1 --2- 172.21.15.167:0/73639689 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58801f7f00 0x7f58800af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:00.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.931+0000 7f5884a00640 1 --2- 172.21.15.167:0/73639689 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5864003a50 0x7f5880135df0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:00.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.931+0000 7f5884a00640 1 -- 172.21.15.167:0/73639689 >> 172.21.15.167:0/73639689 conn(0x7f58800cd8f0 msgr2=0x7f5880144ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:00.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.931+0000 7f5884a00640 1 -- 172.21.15.167:0/73639689 shutdown_connections 2024-05-11T12:12:00.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:00.931+0000 7f5884a00640 1 -- 172.21.15.167:0/73639689 wait complete. 2024-05-11T12:12:00.951 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:12:01.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.151+0000 7f6f7ea00640 1 Processor -- start 2024-05-11T12:12:01.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.151+0000 7f6f7ea00640 1 -- start start 2024-05-11T12:12:01.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.152+0000 7f6f7ea00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f780afcd0 0x7f6f781601f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:01.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.152+0000 7f6f7ea00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6f780cf970 0x7f6f780b55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:01.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.152+0000 7f6f7ea00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f780b4560 con 0x7f6f780d04a0 2024-05-11T12:12:01.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.152+0000 7f6f7ea00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f6f780b5130 con 0x7f6f780cf970 2024-05-11T12:12:01.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.152+0000 7f6f7ea00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6f780b4800 con 0x7f6f780afcd0 2024-05-11T12:12:01.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.153+0000 7f6f7e000640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6f780cf970 0x7f6f780b55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:01.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.153+0000 7f6f7cc00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f6f780d04a0 0x7f6f780cf260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:42172/0 (socket says 172.21.15.167:42172) 2024-05-11T12:12:01.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.153+0000 7f6f7cc00640 1 -- 172.21.15.167:0/1946794675 learned_addr learned my addr 172.21.15.167:0/1946794675 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:01.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.153+0000 7f6f7d600640 1 --2- 172.21.15.167:0/1946794675 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f780afcd0 0x7f6f781601f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:01.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.153+0000 7f6f77e00640 1 -- 172.21.15.167:0/1946794675 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3100643356 0 0) 0x7f6f780b4560 con 0x7f6f780d04a0 2024-05-11T12:12:01.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.153+0000 7f6f7e000640 1 -- 172.21.15.167:0/1946794675 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f780afcd0 msgr2=0x7f6f781601f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:01.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.153+0000 7f6f7e000640 1 --2- 172.21.15.167:0/1946794675 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f780afcd0 0x7f6f781601f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:01.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.153+0000 7f6f7e000640 1 -- 172.21.15.167:0/1946794675 >> v1:172.21.15.167:6789/0 conn(0x7f6f780d04a0 legacy=0x7f6f780cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:01.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.153+0000 7f6f7e000640 1 -- 172.21.15.167:0/1946794675 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f780b4ae0 con 0x7f6f780cf970 2024-05-11T12:12:01.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.154+0000 7f6f7e000640 1 --2- 172.21.15.167:0/1946794675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f780cf970 0x7f6f780b55c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6f780b4730 tx=0x7f6f7002a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=788886aeb4643f78 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:01.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.155+0000 7f6f77e00640 1 -- 172.21.15.167:0/1946794675 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7002ae60 con 0x7f6f780cf970 2024-05-11T12:12:01.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.155+0000 7f6f77e00640 1 -- 172.21.15.167:0/1946794675 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f70007740 con 0x7f6f780cf970 2024-05-11T12:12:01.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.155+0000 7f6f77e00640 1 -- 172.21.15.167:0/1946794675 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f70007b10 con 0x7f6f780cf970 2024-05-11T12:12:01.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.155+0000 7f6f7ea00640 1 -- 172.21.15.167:0/1946794675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f780cf970 msgr2=0x7f6f780b55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:01.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.155+0000 7f6f7ea00640 1 --2- 172.21.15.167:0/1946794675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f780cf970 0x7f6f780b55c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6f780b4730 tx=0x7f6f7002a640 comp rx=0 tx=0).stop 2024-05-11T12:12:01.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.155+0000 7f6f7ea00640 1 -- 172.21.15.167:0/1946794675 shutdown_connections 2024-05-11T12:12:01.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.155+0000 7f6f7ea00640 1 --2- 172.21.15.167:0/1946794675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f780cf970 0x7f6f780b55c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:01.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.155+0000 7f6f7ea00640 1 --2- 172.21.15.167:0/1946794675 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f780afcd0 0x7f6f781601f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:01.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.155+0000 7f6f7ea00640 1 -- 172.21.15.167:0/1946794675 >> 172.21.15.167:0/1946794675 conn(0x7f6f780b3460 msgr2=0x7f6f780d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:01.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.155+0000 7f6f7ea00640 1 -- 172.21.15.167:0/1946794675 shutdown_connections 2024-05-11T12:12:01.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.155+0000 7f6f7ea00640 1 -- 172.21.15.167:0/1946794675 wait complete. 2024-05-11T12:12:01.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.157+0000 7f6f7ea00640 1 Processor -- start 2024-05-11T12:12:01.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.157+0000 7f6f7ea00640 1 -- start start 2024-05-11T12:12:01.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.158+0000 7f6f7ea00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f780afcd0 0x7f6f78158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:01.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.158+0000 7f6f7ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f780cf970 0x7f6f7815cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:01.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.158+0000 7f6f7ea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f780d04a0 0x7f6f7815b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:01.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.158+0000 7f6f7ea00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6f780cff40 con 0x7f6f780d04a0 2024-05-11T12:12:01.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.158+0000 7f6f7ea00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6f780b00b0 con 0x7f6f780cf970 2024-05-11T12:12:01.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.158+0000 7f6f7ea00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6f780d00c0 con 0x7f6f780afcd0 2024-05-11T12:12:01.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.159+0000 7f6f7e000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f780d04a0 0x7f6f7815b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:01.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.159+0000 7f6f7e000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f780d04a0 0x7f6f7815b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41844/0 (socket says 172.21.15.167:41844) 2024-05-11T12:12:01.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.159+0000 7f6f7e000640 1 -- 172.21.15.167:0/203789289 learned_addr learned my addr 172.21.15.167:0/203789289 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:01.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.159+0000 7f6f7cc00640 1 --2- 172.21.15.167:0/203789289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f780cf970 0x7f6f7815cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:01.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.159+0000 7f6f7d600640 1 --2- 172.21.15.167:0/203789289 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f780afcd0 0x7f6f78158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:01.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.159+0000 7f6f7e000640 1 -- 172.21.15.167:0/203789289 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f780afcd0 msgr2=0x7f6f78158540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:01.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.159+0000 7f6f7e000640 1 --2- 172.21.15.167:0/203789289 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f780afcd0 0x7f6f78158540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:01.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.159+0000 7f6f7e000640 1 -- 172.21.15.167:0/203789289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f780cf970 msgr2=0x7f6f7815cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:01.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.159+0000 7f6f7e000640 1 --2- 172.21.15.167:0/203789289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f780cf970 0x7f6f7815cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:01.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.159+0000 7f6f7e000640 1 -- 172.21.15.167:0/203789289 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f7815b7d0 con 0x7f6f780d04a0 2024-05-11T12:12:01.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.159+0000 7f6f7e000640 1 --2- 172.21.15.167:0/203789289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f780d04a0 0x7f6f7815b290 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6f70009e50 tx=0x7f6f70004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:01.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.160+0000 7f6f75600640 1 -- 172.21.15.167:0/203789289 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f70040070 con 0x7f6f780d04a0 2024-05-11T12:12:01.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.160+0000 7f6f75600640 1 -- 172.21.15.167:0/203789289 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f70031d70 con 0x7f6f780d04a0 2024-05-11T12:12:01.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.160+0000 7f6f7ea00640 1 -- 172.21.15.167:0/203789289 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f7815a1f0 con 0x7f6f780d04a0 2024-05-11T12:12:01.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.160+0000 7f6f75600640 1 -- 172.21.15.167:0/203789289 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f70009450 con 0x7f6f780d04a0 2024-05-11T12:12:01.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.160+0000 7f6f7ea00640 1 -- 172.21.15.167:0/203789289 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f7815a7b0 con 0x7f6f780d04a0 2024-05-11T12:12:01.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.161+0000 7f6f7ea00640 1 -- 172.21.15.167:0/203789289 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f780694b0 con 0x7f6f780d04a0 2024-05-11T12:12:01.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.162+0000 7f6f75600640 1 -- 172.21.15.167:0/203789289 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6f70039040 con 0x7f6f780d04a0 2024-05-11T12:12:01.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.163+0000 7f6f75600640 1 --2- 172.21.15.167:0/203789289 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f6f580762d0 0x7f6f58078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:01.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.164+0000 7f6f75600640 1 -- 172.21.15.167:0/203789289 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f6f700b8b90 con 0x7f6f780d04a0 2024-05-11T12:12:01.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.164+0000 7f6f7d600640 1 --2- 172.21.15.167:0/203789289 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f6f580762d0 0x7f6f58078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:01.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.164+0000 7f6f7d600640 1 --2- 172.21.15.167:0/203789289 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f6f580762d0 0x7f6f58078790 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c006fd0 tx=0x7f6f6c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:01.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.168+0000 7f6f75600640 1 -- 172.21.15.167:0/203789289 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6f70082350 con 0x7f6f780d04a0 2024-05-11T12:12:01.350 INFO:tasks.ceph.mds.d.smithi204.stderr:2024-05-11T12:12:01.349+0000 7f8452800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-11T12:12:01.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.380+0000 7f6f7ea00640 1 -- 172.21.15.167:0/203789289 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6f780cf040 con 0x7f6f780d04a0 2024-05-11T12:12:01.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.383+0000 7f6f75600640 1 -- 172.21.15.167:0/203789289 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+16990 (secure 0 0 0) 0x7f6f70081c50 con 0x7f6f780d04a0 2024-05-11T12:12:01.383 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:12:01.383 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":41,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:11:58.413721+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:11:54.849622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-11T12:11:55.831334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:12:01.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.386+0000 7f6f7ea00640 1 -- 172.21.15.167:0/203789289 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f6f580762d0 msgr2=0x7f6f58078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:01.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.386+0000 7f6f7ea00640 1 --2- 172.21.15.167:0/203789289 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f6f580762d0 0x7f6f58078790 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c006fd0 tx=0x7f6f6c008040 comp rx=0 tx=0).stop 2024-05-11T12:12:01.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.386+0000 7f6f7ea00640 1 -- 172.21.15.167:0/203789289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f780d04a0 msgr2=0x7f6f7815b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:01.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.386+0000 7f6f7ea00640 1 --2- 172.21.15.167:0/203789289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f780d04a0 0x7f6f7815b290 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6f70009e50 tx=0x7f6f70004a40 comp rx=0 tx=0).stop 2024-05-11T12:12:01.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.386+0000 7f6f7ea00640 1 -- 172.21.15.167:0/203789289 shutdown_connections 2024-05-11T12:12:01.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.386+0000 7f6f7ea00640 1 --2- 172.21.15.167:0/203789289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f780d04a0 0x7f6f7815b290 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:01.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.386+0000 7f6f7ea00640 1 --2- 172.21.15.167:0/203789289 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f6f580762d0 0x7f6f58078790 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:01.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.386+0000 7f6f7ea00640 1 --2- 172.21.15.167:0/203789289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f780cf970 0x7f6f7815cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:01.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.386+0000 7f6f7ea00640 1 --2- 172.21.15.167:0/203789289 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f780afcd0 0x7f6f78158540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:01.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.386+0000 7f6f7ea00640 1 -- 172.21.15.167:0/203789289 >> 172.21.15.167:0/203789289 conn(0x7f6f780b3460 msgr2=0x7f6f781596a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:01.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.386+0000 7f6f7ea00640 1 -- 172.21.15.167:0/203789289 shutdown_connections 2024-05-11T12:12:01.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.386+0000 7f6f7ea00640 1 -- 172.21.15.167:0/203789289 wait complete. 2024-05-11T12:12:01.403 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-11T12:12:01.404 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T12:12:01.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.603+0000 7f802a800640 1 Processor -- start 2024-05-11T12:12:01.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.603+0000 7f802a800640 1 -- start start 2024-05-11T12:12:01.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.605+0000 7f802a800640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f8024157750 0x7f802415d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:01.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.605+0000 7f802a800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f80240b3ea0 0x7f8024161490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:01.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.605+0000 7f802a800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f802415dc80 con 0x7f80241619d0 2024-05-11T12:12:01.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.605+0000 7f802a800640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f8024161000 con 0x7f8024157750 2024-05-11T12:12:01.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.605+0000 7f802a800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f802415df30 con 0x7f80240b3ea0 2024-05-11T12:12:01.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.605+0000 7f8023400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f8024157750 0x7f802415d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:01.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.605+0000 7f8023400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f8024157750 0x7f802415d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41850/0 (socket says 172.21.15.167:41850) 2024-05-11T12:12:01.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.605+0000 7f8023400640 1 -- 172.21.15.167:0/1251816091 learned_addr learned my addr 172.21.15.167:0/1251816091 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:01.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.606+0000 7f801aa00640 1 --2- 172.21.15.167:0/1251816091 >> v2:172.21.15.204:3300/0 conn(0x7f80240b3ea0 0x7f8024161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:01.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.606+0000 7f8023400640 1 -- 172.21.15.167:0/1251816091 >> v2:172.21.15.204:3300/0 conn(0x7f80240b3ea0 msgr2=0x7f8024161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:01.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.606+0000 7f8023400640 1 --2- 172.21.15.167:0/1251816091 >> v2:172.21.15.204:3300/0 conn(0x7f80240b3ea0 0x7f8024161490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:01.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.606+0000 7f8023400640 1 -- 172.21.15.167:0/1251816091 >> v1:172.21.15.204:6789/0 conn(0x7f80241619d0 legacy=0x7f802406d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:01.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.606+0000 7f8023400640 1 -- 172.21.15.167:0/1251816091 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f802415e210 con 0x7f8024157750 2024-05-11T12:12:01.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.607+0000 7f8023400640 1 --2- 172.21.15.167:0/1251816091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8024157750 0x7f802415d740 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f80240702f0 tx=0x7f801402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c4ea346ec4af366 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:01.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.607+0000 7f8022a00640 1 -- 172.21.15.167:0/1251816091 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8014037070 con 0x7f8024157750 2024-05-11T12:12:01.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.607+0000 7f8022a00640 1 -- 172.21.15.167:0/1251816091 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8014032440 con 0x7f8024157750 2024-05-11T12:12:01.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.607+0000 7f8022a00640 1 -- 172.21.15.167:0/1251816091 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8014032810 con 0x7f8024157750 2024-05-11T12:12:01.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.608+0000 7f802a800640 1 -- 172.21.15.167:0/1251816091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8024157750 msgr2=0x7f802415d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:01.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.608+0000 7f802a800640 1 --2- 172.21.15.167:0/1251816091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8024157750 0x7f802415d740 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f80240702f0 tx=0x7f801402a1e0 comp rx=0 tx=0).stop 2024-05-11T12:12:01.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.608+0000 7f802a800640 1 -- 172.21.15.167:0/1251816091 shutdown_connections 2024-05-11T12:12:01.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.608+0000 7f802a800640 1 --2- 172.21.15.167:0/1251816091 >> v2:172.21.15.204:3300/0 conn(0x7f80240b3ea0 0x7f8024161490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:01.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.608+0000 7f802a800640 1 --2- 172.21.15.167:0/1251816091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8024157750 0x7f802415d740 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:01.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.608+0000 7f802a800640 1 -- 172.21.15.167:0/1251816091 >> 172.21.15.167:0/1251816091 conn(0x7f80240cd8f0 msgr2=0x7f80240cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:01.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.608+0000 7f802a800640 1 -- 172.21.15.167:0/1251816091 shutdown_connections 2024-05-11T12:12:01.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.608+0000 7f802a800640 1 -- 172.21.15.167:0/1251816091 wait complete. 2024-05-11T12:12:01.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.610+0000 7f802a800640 1 Processor -- start 2024-05-11T12:12:01.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.610+0000 7f802a800640 1 -- start start 2024-05-11T12:12:01.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.611+0000 7f802a800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f80240b3ea0 0x7f80241405a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:01.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.611+0000 7f802a800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8024157750 0x7f802415c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:01.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.611+0000 7f802a800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80241619d0 0x7f802415c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:01.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.611+0000 7f802a800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f802406db70 con 0x7f8024157750 2024-05-11T12:12:01.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.611+0000 7f802a800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8024157b30 con 0x7f80241619d0 2024-05-11T12:12:01.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.611+0000 7f802a800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f802406dcf0 con 0x7f80240b3ea0 2024-05-11T12:12:01.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.611+0000 7f801aa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8024157750 0x7f802415c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:01.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.611+0000 7f801aa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8024157750 0x7f802415c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41862/0 (socket says 172.21.15.167:41862) 2024-05-11T12:12:01.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.611+0000 7f801aa00640 1 -- 172.21.15.167:0/933945375 learned_addr learned my addr 172.21.15.167:0/933945375 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:01.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.612+0000 7f801aa00640 1 -- 172.21.15.167:0/933945375 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f80240b3ea0 msgr2=0x7f80241405a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:01.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.612+0000 7f801aa00640 1 --2- 172.21.15.167:0/933945375 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f80240b3ea0 0x7f80241405a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:01.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.612+0000 7f801aa00640 1 -- 172.21.15.167:0/933945375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80241619d0 msgr2=0x7f802415c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:01.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.612+0000 7f801aa00640 1 --2- 172.21.15.167:0/933945375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80241619d0 0x7f802415c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:01.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.612+0000 7f801aa00640 1 -- 172.21.15.167:0/933945375 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8024143790 con 0x7f8024157750 2024-05-11T12:12:01.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.612+0000 7f801aa00640 1 --2- 172.21.15.167:0/933945375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8024157750 0x7f802415c0c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f800c00c7f0 tx=0x7f800c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:01.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.612+0000 7f801be00640 1 -- 172.21.15.167:0/933945375 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800c015070 con 0x7f8024157750 2024-05-11T12:12:01.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.612+0000 7f801be00640 1 -- 172.21.15.167:0/933945375 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f800c004240 con 0x7f8024157750 2024-05-11T12:12:01.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.612+0000 7f801be00640 1 -- 172.21.15.167:0/933945375 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800c007620 con 0x7f8024157750 2024-05-11T12:12:01.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.612+0000 7f802a800640 1 -- 172.21.15.167:0/933945375 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80241426f0 con 0x7f8024157750 2024-05-11T12:12:01.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.612+0000 7f802a800640 1 -- 172.21.15.167:0/933945375 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8024142cb0 con 0x7f8024157750 2024-05-11T12:12:01.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.614+0000 7f802a800640 1 -- 172.21.15.167:0/933945375 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8024069590 con 0x7f8024157750 2024-05-11T12:12:01.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.615+0000 7f801be00640 1 -- 172.21.15.167:0/933945375 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f800c0079d0 con 0x7f8024157750 2024-05-11T12:12:01.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.616+0000 7f801be00640 1 --2- 172.21.15.167:0/933945375 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f80000762d0 0x7f8000078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:01.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.616+0000 7f801be00640 1 -- 172.21.15.167:0/933945375 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f800c097c20 con 0x7f8024157750 2024-05-11T12:12:01.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.616+0000 7f8023400640 1 --2- 172.21.15.167:0/933945375 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f80000762d0 0x7f8000078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:01.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.617+0000 7f8023400640 1 --2- 172.21.15.167:0/933945375 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f80000762d0 0x7f8000078790 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8014034000 tx=0x7f801402c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:01.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.621+0000 7f801be00640 1 -- 172.21.15.167:0/933945375 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f800c0613e0 con 0x7f8024157750 2024-05-11T12:12:01.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.820+0000 7f802a800640 1 -- 172.21.15.167:0/933945375 --> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f802415f290 con 0x7f80000762d0 2024-05-11T12:12:01.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.830+0000 7f801be00640 1 -- 172.21.15.167:0/933945375 <== mgr.4106 v2:172.21.15.167:6832/578352559 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248595 (secure 0 0 0) 0x7f802415f290 con 0x7f80000762d0 2024-05-11T12:12:01.831 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:12:01.836 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2024-05-11T12:12:01.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.836+0000 7f802a800640 1 -- 172.21.15.167:0/933945375 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f80000762d0 msgr2=0x7f8000078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:01.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.836+0000 7f802a800640 1 --2- 172.21.15.167:0/933945375 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f80000762d0 0x7f8000078790 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8014034000 tx=0x7f801402c000 comp rx=0 tx=0).stop 2024-05-11T12:12:01.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.836+0000 7f802a800640 1 -- 172.21.15.167:0/933945375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8024157750 msgr2=0x7f802415c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:01.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.836+0000 7f802a800640 1 --2- 172.21.15.167:0/933945375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8024157750 0x7f802415c0c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f800c00c7f0 tx=0x7f800c00ccb0 comp rx=0 tx=0).stop 2024-05-11T12:12:01.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.836+0000 7f802a800640 1 -- 172.21.15.167:0/933945375 shutdown_connections 2024-05-11T12:12:01.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.836+0000 7f802a800640 1 --2- 172.21.15.167:0/933945375 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f80000762d0 0x7f8000078790 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:01.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.836+0000 7f802a800640 1 --2- 172.21.15.167:0/933945375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80241619d0 0x7f802415c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:01.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.836+0000 7f802a800640 1 --2- 172.21.15.167:0/933945375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8024157750 0x7f802415c0c0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:01.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.836+0000 7f802a800640 1 --2- 172.21.15.167:0/933945375 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f80240b3ea0 0x7f80241405a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:01.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.836+0000 7f802a800640 1 -- 172.21.15.167:0/933945375 >> 172.21.15.167:0/933945375 conn(0x7f80240cd8f0 msgr2=0x7f802415f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:01.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.836+0000 7f802a800640 1 -- 172.21.15.167:0/933945375 shutdown_connections 2024-05-11T12:12:01.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:01.836+0000 7f802a800640 1 -- 172.21.15.167:0/933945375 wait complete. 2024-05-11T12:12:01.856 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":79,"stamp":"2024-05-11T12:12:01.258163+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591062,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":131,"num_write_kb":1631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":96,"ondisk_log_size":96,"up":182,"acting":182,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":100,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220844,"kb_used_data":4656,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753876,"statfs":{"total":773094113280,"available":772867969024,"internally_reserved":0,"allocated":4767744,"data_stored":2686341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":694,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.895997"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.383260+0000","last_change":"2024-05-11T12:11:56.383260+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.383260+0000","last_undegraded":"2024-05-11T12:11:56.383260+0000","last_fullsized":"2024-05-11T12:11:56.383260+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.381407+0000","last_change":"2024-05-11T12:11:56.381407+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.381407+0000","last_undegraded":"2024-05-11T12:11:56.381407+0000","last_fullsized":"2024-05-11T12:11:56.381407+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.388722+0000","last_change":"2024-05-11T12:11:56.388722+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.388722+0000","last_undegraded":"2024-05-11T12:11:56.388722+0000","last_fullsized":"2024-05-11T12:11:56.388722+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T12:11:56.360286+0000","last_change":"2024-05-11T12:11:56.360286+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.360286+0000","last_undegraded":"2024-05-11T12:11:56.360286+0000","last_fullsized":"2024-05-11T12:11:56.360286+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T12:11:56.360286+0000","last_change":"2024-05-11T12:11:56.360286+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.360286+0000","last_undegraded":"2024-05-11T12:11:56.360286+0000","last_fullsized":"2024-05-11T12:11:56.360286+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.386336+0000","last_change":"2024-05-11T12:11:56.386336+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.386336+0000","last_undegraded":"2024-05-11T12:11:56.386336+0000","last_fullsized":"2024-05-11T12:11:56.386336+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.437906+0000","last_change":"2024-05-11T12:11:57.432877+0000","last_active":"2024-05-11T12:11:57.437906+0000","last_peered":"2024-05-11T12:11:57.437906+0000","last_clean":"2024-05-11T12:11:57.437906+0000","last_became_active":"2024-05-11T12:11:57.432581+0000","last_became_peered":"2024-05-11T12:11:57.432581+0000","last_unstale":"2024-05-11T12:11:57.437906+0000","last_undegraded":"2024-05-11T12:11:57.437906+0000","last_fullsized":"2024-05-11T12:11:57.437906+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:47:49.623473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.381742+0000","last_change":"2024-05-11T12:11:56.381742+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.381742+0000","last_undegraded":"2024-05-11T12:11:56.381742+0000","last_fullsized":"2024-05-11T12:11:56.381742+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.384226+0000","last_change":"2024-05-11T12:11:56.384226+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.384226+0000","last_undegraded":"2024-05-11T12:11:56.384226+0000","last_fullsized":"2024-05-11T12:11:56.384226+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.439811+0000","last_change":"2024-05-11T12:11:57.434958+0000","last_active":"2024-05-11T12:11:57.439811+0000","last_peered":"2024-05-11T12:11:57.439811+0000","last_clean":"2024-05-11T12:11:57.439811+0000","last_became_active":"2024-05-11T12:11:57.429221+0000","last_became_peered":"2024-05-11T12:11:57.429221+0000","last_unstale":"2024-05-11T12:11:57.439811+0000","last_undegraded":"2024-05-11T12:11:57.439811+0000","last_fullsized":"2024-05-11T12:11:57.439811+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:32:26.804013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.455023+0000","last_change":"2024-05-11T12:11:57.449552+0000","last_active":"2024-05-11T12:11:57.455023+0000","last_peered":"2024-05-11T12:11:57.455023+0000","last_clean":"2024-05-11T12:11:57.455023+0000","last_became_active":"2024-05-11T12:11:57.441862+0000","last_became_peered":"2024-05-11T12:11:57.441862+0000","last_unstale":"2024-05-11T12:11:57.455023+0000","last_undegraded":"2024-05-11T12:11:57.455023+0000","last_fullsized":"2024-05-11T12:11:57.455023+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:04:26.168833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.386630+0000","last_change":"2024-05-11T12:11:56.386630+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.386630+0000","last_undegraded":"2024-05-11T12:11:56.386630+0000","last_fullsized":"2024-05-11T12:11:56.386630+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.432544+0000","last_change":"2024-05-11T12:11:57.429706+0000","last_active":"2024-05-11T12:11:57.432544+0000","last_peered":"2024-05-11T12:11:57.432544+0000","last_clean":"2024-05-11T12:11:57.432544+0000","last_became_active":"2024-05-11T12:11:57.429387+0000","last_became_peered":"2024-05-11T12:11:57.429387+0000","last_unstale":"2024-05-11T12:11:57.432544+0000","last_undegraded":"2024-05-11T12:11:57.432544+0000","last_fullsized":"2024-05-11T12:11:57.432544+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:55:35.468623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.445013+0000","last_change":"2024-05-11T12:11:57.430917+0000","last_active":"2024-05-11T12:11:57.445013+0000","last_peered":"2024-05-11T12:11:57.445013+0000","last_clean":"2024-05-11T12:11:57.445013+0000","last_became_active":"2024-05-11T12:11:57.430613+0000","last_became_peered":"2024-05-11T12:11:57.430613+0000","last_unstale":"2024-05-11T12:11:57.445013+0000","last_undegraded":"2024-05-11T12:11:57.445013+0000","last_fullsized":"2024-05-11T12:11:57.445013+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:25:16.350527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.454620+0000","last_change":"2024-05-11T12:11:57.438016+0000","last_active":"2024-05-11T12:11:57.454620+0000","last_peered":"2024-05-11T12:11:57.454620+0000","last_clean":"2024-05-11T12:11:57.454620+0000","last_became_active":"2024-05-11T12:11:57.437680+0000","last_became_peered":"2024-05-11T12:11:57.437680+0000","last_unstale":"2024-05-11T12:11:57.454620+0000","last_undegraded":"2024-05-11T12:11:57.454620+0000","last_fullsized":"2024-05-11T12:11:57.454620+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:28:33.430453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.453506+0000","last_change":"2024-05-11T12:11:57.448672+0000","last_active":"2024-05-11T12:11:57.453506+0000","last_peered":"2024-05-11T12:11:57.453506+0000","last_clean":"2024-05-11T12:11:57.453506+0000","last_became_active":"2024-05-11T12:11:57.447489+0000","last_became_peered":"2024-05-11T12:11:57.447489+0000","last_unstale":"2024-05-11T12:11:57.453506+0000","last_undegraded":"2024-05-11T12:11:57.453506+0000","last_fullsized":"2024-05-11T12:11:57.453506+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:53:03.480916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.451781+0000","last_change":"2024-05-11T12:11:57.447831+0000","last_active":"2024-05-11T12:11:57.451781+0000","last_peered":"2024-05-11T12:11:57.451781+0000","last_clean":"2024-05-11T12:11:57.451781+0000","last_became_active":"2024-05-11T12:11:57.443540+0000","last_became_peered":"2024-05-11T12:11:57.443540+0000","last_unstale":"2024-05-11T12:11:57.451781+0000","last_undegraded":"2024-05-11T12:11:57.451781+0000","last_fullsized":"2024-05-11T12:11:57.451781+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:26:34.393703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.435968+0000","last_change":"2024-05-11T12:11:57.431125+0000","last_active":"2024-05-11T12:11:57.435968+0000","last_peered":"2024-05-11T12:11:57.435968+0000","last_clean":"2024-05-11T12:11:57.435968+0000","last_became_active":"2024-05-11T12:11:57.430835+0000","last_became_peered":"2024-05-11T12:11:57.430835+0000","last_unstale":"2024-05-11T12:11:57.435968+0000","last_undegraded":"2024-05-11T12:11:57.435968+0000","last_fullsized":"2024-05-11T12:11:57.435968+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:23:28.626421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T12:11:56.360286+0000","last_change":"2024-05-11T12:11:56.360286+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.360286+0000","last_undegraded":"2024-05-11T12:11:56.360286+0000","last_fullsized":"2024-05-11T12:11:56.360286+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.436449+0000","last_change":"2024-05-11T12:11:57.433084+0000","last_active":"2024-05-11T12:11:57.436449+0000","last_peered":"2024-05-11T12:11:57.436449+0000","last_clean":"2024-05-11T12:11:57.436449+0000","last_became_active":"2024-05-11T12:11:57.432645+0000","last_became_peered":"2024-05-11T12:11:57.432645+0000","last_unstale":"2024-05-11T12:11:57.436449+0000","last_undegraded":"2024-05-11T12:11:57.436449+0000","last_fullsized":"2024-05-11T12:11:57.436449+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:32:45.644260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.439852+0000","last_change":"2024-05-11T12:11:57.434801+0000","last_active":"2024-05-11T12:11:57.439852+0000","last_peered":"2024-05-11T12:11:57.439852+0000","last_clean":"2024-05-11T12:11:57.439852+0000","last_became_active":"2024-05-11T12:11:57.433969+0000","last_became_peered":"2024-05-11T12:11:57.433969+0000","last_unstale":"2024-05-11T12:11:57.439852+0000","last_undegraded":"2024-05-11T12:11:57.439852+0000","last_fullsized":"2024-05-11T12:11:57.439852+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:22:33.211172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.449144+0000","last_change":"2024-05-11T12:11:57.445771+0000","last_active":"2024-05-11T12:11:57.449144+0000","last_peered":"2024-05-11T12:11:57.449144+0000","last_clean":"2024-05-11T12:11:57.449144+0000","last_became_active":"2024-05-11T12:11:57.445474+0000","last_became_peered":"2024-05-11T12:11:57.445474+0000","last_unstale":"2024-05-11T12:11:57.449144+0000","last_undegraded":"2024-05-11T12:11:57.449144+0000","last_fullsized":"2024-05-11T12:11:57.449144+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:23:28.626421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.451922+0000","last_change":"2024-05-11T12:11:57.447508+0000","last_active":"2024-05-11T12:11:57.451922+0000","last_peered":"2024-05-11T12:11:57.451922+0000","last_clean":"2024-05-11T12:11:57.451922+0000","last_became_active":"2024-05-11T12:11:57.444377+0000","last_became_peered":"2024-05-11T12:11:57.444377+0000","last_unstale":"2024-05-11T12:11:57.451922+0000","last_undegraded":"2024-05-11T12:11:57.451922+0000","last_fullsized":"2024-05-11T12:11:57.451922+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:46:06.174919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T12:11:56.360286+0000","last_change":"2024-05-11T12:11:56.360286+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.360286+0000","last_undegraded":"2024-05-11T12:11:56.360286+0000","last_fullsized":"2024-05-11T12:11:56.360286+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.430674+0000","last_change":"2024-05-11T12:11:57.426462+0000","last_active":"2024-05-11T12:11:57.430674+0000","last_peered":"2024-05-11T12:11:57.430674+0000","last_clean":"2024-05-11T12:11:57.430674+0000","last_became_active":"2024-05-11T12:11:57.426087+0000","last_became_peered":"2024-05-11T12:11:57.426087+0000","last_unstale":"2024-05-11T12:11:57.430674+0000","last_undegraded":"2024-05-11T12:11:57.430674+0000","last_fullsized":"2024-05-11T12:11:57.430674+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:30:57.830893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.390804+0000","last_change":"2024-05-11T12:11:56.390804+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.390804+0000","last_undegraded":"2024-05-11T12:11:56.390804+0000","last_fullsized":"2024-05-11T12:11:56.390804+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.432318+0000","last_change":"2024-05-11T12:11:57.428594+0000","last_active":"2024-05-11T12:11:57.432318+0000","last_peered":"2024-05-11T12:11:57.432318+0000","last_clean":"2024-05-11T12:11:57.432318+0000","last_became_active":"2024-05-11T12:11:57.428245+0000","last_became_peered":"2024-05-11T12:11:57.428245+0000","last_unstale":"2024-05-11T12:11:57.432318+0000","last_undegraded":"2024-05-11T12:11:57.432318+0000","last_fullsized":"2024-05-11T12:11:57.432318+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:28:44.586634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.430392+0000","last_change":"2024-05-11T12:11:57.417635+0000","last_active":"2024-05-11T12:11:57.430392+0000","last_peered":"2024-05-11T12:11:57.430392+0000","last_clean":"2024-05-11T12:11:57.430392+0000","last_became_active":"2024-05-11T12:11:57.415536+0000","last_became_peered":"2024-05-11T12:11:57.415536+0000","last_unstale":"2024-05-11T12:11:57.430392+0000","last_undegraded":"2024-05-11T12:11:57.430392+0000","last_fullsized":"2024-05-11T12:11:57.430392+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:51:43.649698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.432559+0000","last_change":"2024-05-11T12:11:57.427847+0000","last_active":"2024-05-11T12:11:57.432559+0000","last_peered":"2024-05-11T12:11:57.432559+0000","last_clean":"2024-05-11T12:11:57.432559+0000","last_became_active":"2024-05-11T12:11:57.427416+0000","last_became_peered":"2024-05-11T12:11:57.427416+0000","last_unstale":"2024-05-11T12:11:57.432559+0000","last_undegraded":"2024-05-11T12:11:57.432559+0000","last_fullsized":"2024-05-11T12:11:57.432559+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:07:22.774535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.423604+0000","last_change":"2024-05-11T12:11:57.420465+0000","last_active":"2024-05-11T12:11:57.423604+0000","last_peered":"2024-05-11T12:11:57.423604+0000","last_clean":"2024-05-11T12:11:57.423604+0000","last_became_active":"2024-05-11T12:11:57.419963+0000","last_became_peered":"2024-05-11T12:11:57.419963+0000","last_unstale":"2024-05-11T12:11:57.423604+0000","last_undegraded":"2024-05-11T12:11:57.423604+0000","last_fullsized":"2024-05-11T12:11:57.423604+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:36:41.799582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.451900+0000","last_change":"2024-05-11T12:11:57.447209+0000","last_active":"2024-05-11T12:11:57.451900+0000","last_peered":"2024-05-11T12:11:57.451900+0000","last_clean":"2024-05-11T12:11:57.451900+0000","last_became_active":"2024-05-11T12:11:57.443801+0000","last_became_peered":"2024-05-11T12:11:57.443801+0000","last_unstale":"2024-05-11T12:11:57.451900+0000","last_undegraded":"2024-05-11T12:11:57.451900+0000","last_fullsized":"2024-05-11T12:11:57.451900+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:10:05.034718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.389994+0000","last_change":"2024-05-11T12:11:56.389994+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.389994+0000","last_undegraded":"2024-05-11T12:11:56.389994+0000","last_fullsized":"2024-05-11T12:11:56.389994+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.451196+0000","last_change":"2024-05-11T12:11:57.444554+0000","last_active":"2024-05-11T12:11:57.451196+0000","last_peered":"2024-05-11T12:11:57.451196+0000","last_clean":"2024-05-11T12:11:57.451196+0000","last_became_active":"2024-05-11T12:11:57.443836+0000","last_became_peered":"2024-05-11T12:11:57.443836+0000","last_unstale":"2024-05-11T12:11:57.451196+0000","last_undegraded":"2024-05-11T12:11:57.451196+0000","last_fullsized":"2024-05-11T12:11:57.451196+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:20:14.138435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"40'4","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.451553+0000","last_change":"2024-05-11T12:11:56.409094+0000","last_active":"2024-05-11T12:11:57.451553+0000","last_peered":"2024-05-11T12:11:57.451553+0000","last_clean":"2024-05-11T12:11:57.451553+0000","last_became_active":"2024-05-11T12:11:56.408820+0000","last_became_peered":"2024-05-11T12:11:56.408820+0000","last_unstale":"2024-05-11T12:11:57.451553+0000","last_undegraded":"2024-05-11T12:11:57.451553+0000","last_fullsized":"2024-05-11T12:11:57.451553+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:11:42.632883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.389689+0000","last_change":"2024-05-11T12:11:56.389689+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.389689+0000","last_undegraded":"2024-05-11T12:11:56.389689+0000","last_fullsized":"2024-05-11T12:11:56.389689+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T12:11:56.407872+0000","last_change":"2024-05-11T12:11:56.403002+0000","last_active":"2024-05-11T12:11:56.407872+0000","last_peered":"2024-05-11T12:11:56.407872+0000","last_clean":"2024-05-11T12:11:56.407872+0000","last_became_active":"2024-05-11T12:11:56.402721+0000","last_became_peered":"2024-05-11T12:11:56.402721+0000","last_unstale":"2024-05-11T12:11:56.407872+0000","last_undegraded":"2024-05-11T12:11:56.407872+0000","last_fullsized":"2024-05-11T12:11:56.407872+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:36:40.785205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.450328+0000","last_change":"2024-05-11T12:11:57.435281+0000","last_active":"2024-05-11T12:11:57.450328+0000","last_peered":"2024-05-11T12:11:57.450328+0000","last_clean":"2024-05-11T12:11:57.450328+0000","last_became_active":"2024-05-11T12:11:57.434672+0000","last_became_peered":"2024-05-11T12:11:57.434672+0000","last_unstale":"2024-05-11T12:11:57.450328+0000","last_undegraded":"2024-05-11T12:11:57.450328+0000","last_fullsized":"2024-05-11T12:11:57.450328+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:30:57.830893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T12:11:56.417027+0000","last_change":"2024-05-11T12:11:56.410313+0000","last_active":"2024-05-11T12:11:56.417027+0000","last_peered":"2024-05-11T12:11:56.417027+0000","last_clean":"2024-05-11T12:11:56.417027+0000","last_became_active":"2024-05-11T12:11:56.409971+0000","last_became_peered":"2024-05-11T12:11:56.409971+0000","last_unstale":"2024-05-11T12:11:56.417027+0000","last_undegraded":"2024-05-11T12:11:56.417027+0000","last_fullsized":"2024-05-11T12:11:56.417027+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:11:42.632883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.382086+0000","last_change":"2024-05-11T12:11:56.382086+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.382086+0000","last_undegraded":"2024-05-11T12:11:56.382086+0000","last_fullsized":"2024-05-11T12:11:56.382086+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"40'2","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.454119+0000","last_change":"2024-05-11T12:11:56.408353+0000","last_active":"2024-05-11T12:11:57.454119+0000","last_peered":"2024-05-11T12:11:57.454119+0000","last_clean":"2024-05-11T12:11:57.454119+0000","last_became_active":"2024-05-11T12:11:56.408078+0000","last_became_peered":"2024-05-11T12:11:56.408078+0000","last_unstale":"2024-05-11T12:11:57.454119+0000","last_undegraded":"2024-05-11T12:11:57.454119+0000","last_fullsized":"2024-05-11T12:11:57.454119+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:16:00.825967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.384699+0000","last_change":"2024-05-11T12:11:56.384699+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.384699+0000","last_undegraded":"2024-05-11T12:11:56.384699+0000","last_fullsized":"2024-05-11T12:11:56.384699+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"40'1","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.455051+0000","last_change":"2024-05-11T12:11:56.404989+0000","last_active":"2024-05-11T12:11:57.455051+0000","last_peered":"2024-05-11T12:11:57.455051+0000","last_clean":"2024-05-11T12:11:57.455051+0000","last_became_active":"2024-05-11T12:11:56.404069+0000","last_became_peered":"2024-05-11T12:11:56.404069+0000","last_unstale":"2024-05-11T12:11:57.455051+0000","last_undegraded":"2024-05-11T12:11:57.455051+0000","last_fullsized":"2024-05-11T12:11:57.455051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:26:33.379326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.454982+0000","last_change":"2024-05-11T12:11:57.442633+0000","last_active":"2024-05-11T12:11:57.454982+0000","last_peered":"2024-05-11T12:11:57.454982+0000","last_clean":"2024-05-11T12:11:57.454982+0000","last_became_active":"2024-05-11T12:11:57.442361+0000","last_became_peered":"2024-05-11T12:11:57.442361+0000","last_unstale":"2024-05-11T12:11:57.454982+0000","last_undegraded":"2024-05-11T12:11:57.454982+0000","last_fullsized":"2024-05-11T12:11:57.454982+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:43:20.345625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"40'1","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.451754+0000","last_change":"2024-05-11T12:11:56.411689+0000","last_active":"2024-05-11T12:11:57.451754+0000","last_peered":"2024-05-11T12:11:57.451754+0000","last_clean":"2024-05-11T12:11:57.451754+0000","last_became_active":"2024-05-11T12:11:56.409937+0000","last_became_peered":"2024-05-11T12:11:56.409937+0000","last_unstale":"2024-05-11T12:11:57.451754+0000","last_undegraded":"2024-05-11T12:11:57.451754+0000","last_fullsized":"2024-05-11T12:11:57.451754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:36:40.785205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.448085+0000","last_change":"2024-05-11T12:11:57.435197+0000","last_active":"2024-05-11T12:11:57.448085+0000","last_peered":"2024-05-11T12:11:57.448085+0000","last_clean":"2024-05-11T12:11:57.448085+0000","last_became_active":"2024-05-11T12:11:57.434472+0000","last_became_peered":"2024-05-11T12:11:57.434472+0000","last_unstale":"2024-05-11T12:11:57.448085+0000","last_undegraded":"2024-05-11T12:11:57.448085+0000","last_fullsized":"2024-05-11T12:11:57.448085+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:36:41.799582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T12:11:56.416996+0000","last_change":"2024-05-11T12:11:56.409731+0000","last_active":"2024-05-11T12:11:56.416996+0000","last_peered":"2024-05-11T12:11:56.416996+0000","last_clean":"2024-05-11T12:11:56.416996+0000","last_became_active":"2024-05-11T12:11:56.409450+0000","last_became_peered":"2024-05-11T12:11:56.409450+0000","last_unstale":"2024-05-11T12:11:56.416996+0000","last_undegraded":"2024-05-11T12:11:56.416996+0000","last_fullsized":"2024-05-11T12:11:56.416996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:34:29.698881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.381254+0000","last_change":"2024-05-11T12:11:56.381254+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.381254+0000","last_undegraded":"2024-05-11T12:11:56.381254+0000","last_fullsized":"2024-05-11T12:11:56.381254+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.417407+0000","last_change":"2024-05-11T12:11:56.406587+0000","last_active":"2024-05-11T12:11:57.417407+0000","last_peered":"2024-05-11T12:11:57.417407+0000","last_clean":"2024-05-11T12:11:57.417407+0000","last_became_active":"2024-05-11T12:11:56.405746+0000","last_became_peered":"2024-05-11T12:11:56.405746+0000","last_unstale":"2024-05-11T12:11:57.417407+0000","last_undegraded":"2024-05-11T12:11:57.417407+0000","last_fullsized":"2024-05-11T12:11:57.417407+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:11:10.100041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T12:11:56.360286+0000","last_change":"2024-05-11T12:11:56.360286+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.360286+0000","last_undegraded":"2024-05-11T12:11:56.360286+0000","last_fullsized":"2024-05-11T12:11:56.360286+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T12:11:56.402312+0000","last_change":"2024-05-11T12:11:56.398172+0000","last_active":"2024-05-11T12:11:56.402312+0000","last_peered":"2024-05-11T12:11:56.402312+0000","last_clean":"2024-05-11T12:11:56.402312+0000","last_became_active":"2024-05-11T12:11:56.397860+0000","last_became_peered":"2024-05-11T12:11:56.397860+0000","last_unstale":"2024-05-11T12:11:56.402312+0000","last_undegraded":"2024-05-11T12:11:56.402312+0000","last_fullsized":"2024-05-11T12:11:56.402312+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:52:13.234125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.450935+0000","last_change":"2024-05-11T12:11:57.442952+0000","last_active":"2024-05-11T12:11:57.450935+0000","last_peered":"2024-05-11T12:11:57.450935+0000","last_clean":"2024-05-11T12:11:57.450935+0000","last_became_active":"2024-05-11T12:11:57.436880+0000","last_became_peered":"2024-05-11T12:11:57.436880+0000","last_unstale":"2024-05-11T12:11:57.450935+0000","last_undegraded":"2024-05-11T12:11:57.450935+0000","last_fullsized":"2024-05-11T12:11:57.450935+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:23:28.626421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.405805+0000","last_change":"2024-05-11T12:11:56.411981+0000","last_active":"2024-05-11T12:11:57.405805+0000","last_peered":"2024-05-11T12:11:57.405805+0000","last_clean":"2024-05-11T12:11:57.405805+0000","last_became_active":"2024-05-11T12:11:56.409613+0000","last_became_peered":"2024-05-11T12:11:56.409613+0000","last_unstale":"2024-05-11T12:11:57.405805+0000","last_undegraded":"2024-05-11T12:11:57.405805+0000","last_fullsized":"2024-05-11T12:11:57.405805+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:46:05.160542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.384922+0000","last_change":"2024-05-11T12:11:56.384922+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.384922+0000","last_undegraded":"2024-05-11T12:11:56.384922+0000","last_fullsized":"2024-05-11T12:11:56.384922+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"40'3","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.456654+0000","last_change":"2024-05-11T12:11:56.407591+0000","last_active":"2024-05-11T12:11:57.456654+0000","last_peered":"2024-05-11T12:11:57.456654+0000","last_clean":"2024-05-11T12:11:57.456654+0000","last_became_active":"2024-05-11T12:11:56.407173+0000","last_became_peered":"2024-05-11T12:11:56.407173+0000","last_unstale":"2024-05-11T12:11:57.456654+0000","last_undegraded":"2024-05-11T12:11:57.456654+0000","last_fullsized":"2024-05-11T12:11:57.456654+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:52:13.234125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.450953+0000","last_change":"2024-05-11T12:11:57.447257+0000","last_active":"2024-05-11T12:11:57.450953+0000","last_peered":"2024-05-11T12:11:57.450953+0000","last_clean":"2024-05-11T12:11:57.450953+0000","last_became_active":"2024-05-11T12:11:57.445351+0000","last_became_peered":"2024-05-11T12:11:57.445351+0000","last_unstale":"2024-05-11T12:11:57.450953+0000","last_undegraded":"2024-05-11T12:11:57.450953+0000","last_fullsized":"2024-05-11T12:11:57.450953+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:32:45.644260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T12:11:56.395987+0000","last_change":"2024-05-11T12:11:56.394318+0000","last_active":"2024-05-11T12:11:56.395987+0000","last_peered":"2024-05-11T12:11:56.395987+0000","last_clean":"2024-05-11T12:11:56.395987+0000","last_became_active":"2024-05-11T12:11:56.393560+0000","last_became_peered":"2024-05-11T12:11:56.393560+0000","last_unstale":"2024-05-11T12:11:56.395987+0000","last_undegraded":"2024-05-11T12:11:56.395987+0000","last_fullsized":"2024-05-11T12:11:56.395987+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:48:53.180988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.383778+0000","last_change":"2024-05-11T12:11:56.383778+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.383778+0000","last_undegraded":"2024-05-11T12:11:56.383778+0000","last_fullsized":"2024-05-11T12:11:56.383778+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T12:11:56.402064+0000","last_change":"2024-05-11T12:11:56.397825+0000","last_active":"2024-05-11T12:11:56.402064+0000","last_peered":"2024-05-11T12:11:56.402064+0000","last_clean":"2024-05-11T12:11:56.402064+0000","last_became_active":"2024-05-11T12:11:56.397506+0000","last_became_peered":"2024-05-11T12:11:56.397506+0000","last_unstale":"2024-05-11T12:11:56.402064+0000","last_undegraded":"2024-05-11T12:11:56.402064+0000","last_fullsized":"2024-05-11T12:11:56.402064+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:20:13.124057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.380190+0000","last_change":"2024-05-11T12:11:56.380190+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.380190+0000","last_undegraded":"2024-05-11T12:11:56.380190+0000","last_fullsized":"2024-05-11T12:11:56.380190+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T12:11:56.417002+0000","last_change":"2024-05-11T12:11:56.409815+0000","last_active":"2024-05-11T12:11:56.417002+0000","last_peered":"2024-05-11T12:11:56.417002+0000","last_clean":"2024-05-11T12:11:56.417002+0000","last_became_active":"2024-05-11T12:11:56.409411+0000","last_became_peered":"2024-05-11T12:11:56.409411+0000","last_unstale":"2024-05-11T12:11:56.417002+0000","last_undegraded":"2024-05-11T12:11:56.417002+0000","last_fullsized":"2024-05-11T12:11:56.417002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:16:00.825967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.450828+0000","last_change":"2024-05-11T12:11:57.442669+0000","last_active":"2024-05-11T12:11:57.450828+0000","last_peered":"2024-05-11T12:11:57.450828+0000","last_clean":"2024-05-11T12:11:57.450828+0000","last_became_active":"2024-05-11T12:11:57.437559+0000","last_became_peered":"2024-05-11T12:11:57.437559+0000","last_unstale":"2024-05-11T12:11:57.450828+0000","last_undegraded":"2024-05-11T12:11:57.450828+0000","last_fullsized":"2024-05-11T12:11:57.450828+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:28:44.586634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.405638+0000","last_change":"2024-05-11T12:11:56.408411+0000","last_active":"2024-05-11T12:11:57.405638+0000","last_peered":"2024-05-11T12:11:57.405638+0000","last_clean":"2024-05-11T12:11:57.405638+0000","last_became_active":"2024-05-11T12:11:56.408100+0000","last_became_peered":"2024-05-11T12:11:56.408100+0000","last_unstale":"2024-05-11T12:11:57.405638+0000","last_undegraded":"2024-05-11T12:11:57.405638+0000","last_fullsized":"2024-05-11T12:11:57.405638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:25:15.336149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.380730+0000","last_change":"2024-05-11T12:11:56.380730+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.380730+0000","last_undegraded":"2024-05-11T12:11:56.380730+0000","last_fullsized":"2024-05-11T12:11:56.380730+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"40'8","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.457343+0000","last_change":"2024-05-11T12:11:56.398517+0000","last_active":"2024-05-11T12:11:57.457343+0000","last_peered":"2024-05-11T12:11:57.457343+0000","last_clean":"2024-05-11T12:11:57.457343+0000","last_became_active":"2024-05-11T12:11:56.398204+0000","last_became_peered":"2024-05-11T12:11:56.398204+0000","last_unstale":"2024-05-11T12:11:57.457343+0000","last_undegraded":"2024-05-11T12:11:57.457343+0000","last_fullsized":"2024-05-11T12:11:57.457343+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:25:15.336149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.379813+0000","last_change":"2024-05-11T12:11:56.379813+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.379813+0000","last_undegraded":"2024-05-11T12:11:56.379813+0000","last_fullsized":"2024-05-11T12:11:56.379813+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"40'1","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.455210+0000","last_change":"2024-05-11T12:11:56.410975+0000","last_active":"2024-05-11T12:11:57.455210+0000","last_peered":"2024-05-11T12:11:57.455210+0000","last_clean":"2024-05-11T12:11:57.455210+0000","last_became_active":"2024-05-11T12:11:56.410634+0000","last_became_peered":"2024-05-11T12:11:56.410634+0000","last_unstale":"2024-05-11T12:11:57.455210+0000","last_undegraded":"2024-05-11T12:11:57.455210+0000","last_fullsized":"2024-05-11T12:11:57.455210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:10:04.020340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.433269+0000","last_change":"2024-05-11T12:11:57.428161+0000","last_active":"2024-05-11T12:11:57.433269+0000","last_peered":"2024-05-11T12:11:57.433269+0000","last_clean":"2024-05-11T12:11:57.433269+0000","last_became_active":"2024-05-11T12:11:57.427826+0000","last_became_peered":"2024-05-11T12:11:57.427826+0000","last_unstale":"2024-05-11T12:11:57.433269+0000","last_undegraded":"2024-05-11T12:11:57.433269+0000","last_fullsized":"2024-05-11T12:11:57.433269+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:11:37.571584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"40'2","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.459816+0000","last_change":"2024-05-11T12:11:56.406473+0000","last_active":"2024-05-11T12:11:57.459816+0000","last_peered":"2024-05-11T12:11:57.459816+0000","last_clean":"2024-05-11T12:11:57.459816+0000","last_became_active":"2024-05-11T12:11:56.405923+0000","last_became_peered":"2024-05-11T12:11:56.405923+0000","last_unstale":"2024-05-11T12:11:57.459816+0000","last_undegraded":"2024-05-11T12:11:57.459816+0000","last_fullsized":"2024-05-11T12:11:57.459816+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:11:42.632883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.450353+0000","last_change":"2024-05-11T12:11:57.436042+0000","last_active":"2024-05-11T12:11:57.450353+0000","last_peered":"2024-05-11T12:11:57.450353+0000","last_clean":"2024-05-11T12:11:57.450353+0000","last_became_active":"2024-05-11T12:11:57.435781+0000","last_became_peered":"2024-05-11T12:11:57.435781+0000","last_unstale":"2024-05-11T12:11:57.450353+0000","last_undegraded":"2024-05-11T12:11:57.450353+0000","last_fullsized":"2024-05-11T12:11:57.450353+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:07:22.774535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-11T12:11:55.388590+0000","last_change":"2024-05-11T12:11:55.388590+0000","last_active":"2024-05-11T12:11:55.345908+0000","last_peered":"2024-05-11T12:11:55.345908+0000","last_clean":"2024-05-11T12:11:55.345908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:55.388590+0000","last_undegraded":"2024-05-11T12:11:55.388590+0000","last_fullsized":"2024-05-11T12:11:55.388590+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T12:11:56.360286+0000","last_change":"2024-05-11T12:11:56.360286+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.360286+0000","last_undegraded":"2024-05-11T12:11:56.360286+0000","last_fullsized":"2024-05-11T12:11:56.360286+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T12:11:56.416911+0000","last_change":"2024-05-11T12:11:56.409279+0000","last_active":"2024-05-11T12:11:56.416911+0000","last_peered":"2024-05-11T12:11:56.416911+0000","last_clean":"2024-05-11T12:11:56.416911+0000","last_became_active":"2024-05-11T12:11:56.408985+0000","last_became_peered":"2024-05-11T12:11:56.408985+0000","last_unstale":"2024-05-11T12:11:56.416911+0000","last_undegraded":"2024-05-11T12:11:56.416911+0000","last_fullsized":"2024-05-11T12:11:56.416911+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:11:10.100041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.453397+0000","last_change":"2024-05-11T12:11:57.443914+0000","last_active":"2024-05-11T12:11:57.453397+0000","last_peered":"2024-05-11T12:11:57.453397+0000","last_clean":"2024-05-11T12:11:57.453397+0000","last_became_active":"2024-05-11T12:11:57.443680+0000","last_became_peered":"2024-05-11T12:11:57.443680+0000","last_unstale":"2024-05-11T12:11:57.453397+0000","last_undegraded":"2024-05-11T12:11:57.453397+0000","last_fullsized":"2024-05-11T12:11:57.453397+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:32:45.644260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-11T12:11:55.388296+0000","last_change":"2024-05-11T12:11:55.388296+0000","last_active":"2024-05-11T12:11:55.345908+0000","last_peered":"2024-05-11T12:11:55.345908+0000","last_clean":"2024-05-11T12:11:55.345908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:55.388296+0000","last_undegraded":"2024-05-11T12:11:55.388296+0000","last_fullsized":"2024-05-11T12:11:55.388296+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.383307+0000","last_change":"2024-05-11T12:11:56.383307+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.383307+0000","last_undegraded":"2024-05-11T12:11:56.383307+0000","last_fullsized":"2024-05-11T12:11:56.383307+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-11T12:11:55.388497+0000","last_change":"2024-05-11T12:11:55.388497+0000","last_active":"2024-05-11T12:11:55.345908+0000","last_peered":"2024-05-11T12:11:55.345908+0000","last_clean":"2024-05-11T12:11:55.345908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:55.388497+0000","last_undegraded":"2024-05-11T12:11:55.388497+0000","last_fullsized":"2024-05-11T12:11:55.388497+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.431245+0000","last_change":"2024-05-11T12:11:57.419098+0000","last_active":"2024-05-11T12:11:57.431245+0000","last_peered":"2024-05-11T12:11:57.431245+0000","last_clean":"2024-05-11T12:11:57.431245+0000","last_became_active":"2024-05-11T12:11:57.417628+0000","last_became_peered":"2024-05-11T12:11:57.417628+0000","last_unstale":"2024-05-11T12:11:57.431245+0000","last_undegraded":"2024-05-11T12:11:57.431245+0000","last_fullsized":"2024-05-11T12:11:57.431245+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:17:59.309336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.414852+0000","last_change":"2024-05-11T12:11:56.395623+0000","last_active":"2024-05-11T12:11:57.414852+0000","last_peered":"2024-05-11T12:11:57.414852+0000","last_clean":"2024-05-11T12:11:57.414852+0000","last_became_active":"2024-05-11T12:11:56.395250+0000","last_became_peered":"2024-05-11T12:11:56.395250+0000","last_unstale":"2024-05-11T12:11:57.414852+0000","last_undegraded":"2024-05-11T12:11:57.414852+0000","last_fullsized":"2024-05-11T12:11:57.414852+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:30:56.816516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.379844+0000","last_change":"2024-05-11T12:11:56.379844+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.379844+0000","last_undegraded":"2024-05-11T12:11:56.379844+0000","last_fullsized":"2024-05-11T12:11:56.379844+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-11T12:11:55.388481+0000","last_change":"2024-05-11T12:11:55.388481+0000","last_active":"2024-05-11T12:11:55.345908+0000","last_peered":"2024-05-11T12:11:55.345908+0000","last_clean":"2024-05-11T12:11:55.345908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:55.388481+0000","last_undegraded":"2024-05-11T12:11:55.388481+0000","last_fullsized":"2024-05-11T12:11:55.388481+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.386779+0000","last_change":"2024-05-11T12:11:56.386779+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.386779+0000","last_undegraded":"2024-05-11T12:11:56.386779+0000","last_fullsized":"2024-05-11T12:11:56.386779+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"40'2","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.455023+0000","last_change":"2024-05-11T12:11:56.409581+0000","last_active":"2024-05-11T12:11:57.455023+0000","last_peered":"2024-05-11T12:11:57.455023+0000","last_clean":"2024-05-11T12:11:57.455023+0000","last_became_active":"2024-05-11T12:11:56.408010+0000","last_became_peered":"2024-05-11T12:11:56.408010+0000","last_unstale":"2024-05-11T12:11:57.455023+0000","last_undegraded":"2024-05-11T12:11:57.455023+0000","last_fullsized":"2024-05-11T12:11:57.455023+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:20:13.124057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.451683+0000","last_change":"2024-05-11T12:11:57.447191+0000","last_active":"2024-05-11T12:11:57.451683+0000","last_peered":"2024-05-11T12:11:57.451683+0000","last_clean":"2024-05-11T12:11:57.451683+0000","last_became_active":"2024-05-11T12:11:57.442656+0000","last_became_peered":"2024-05-11T12:11:57.442656+0000","last_unstale":"2024-05-11T12:11:57.451683+0000","last_undegraded":"2024-05-11T12:11:57.451683+0000","last_fullsized":"2024-05-11T12:11:57.451683+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:52:14.248503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-11T12:11:55.388530+0000","last_change":"2024-05-11T12:11:55.388530+0000","last_active":"2024-05-11T12:11:55.345908+0000","last_peered":"2024-05-11T12:11:55.345908+0000","last_clean":"2024-05-11T12:11:55.345908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:55.388530+0000","last_undegraded":"2024-05-11T12:11:55.388530+0000","last_fullsized":"2024-05-11T12:11:55.388530+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.378928+0000","last_change":"2024-05-11T12:11:56.378928+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.378928+0000","last_undegraded":"2024-05-11T12:11:56.378928+0000","last_fullsized":"2024-05-11T12:11:56.378928+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T12:11:56.402024+0000","last_change":"2024-05-11T12:11:56.397553+0000","last_active":"2024-05-11T12:11:56.402024+0000","last_peered":"2024-05-11T12:11:56.402024+0000","last_clean":"2024-05-11T12:11:56.402024+0000","last_became_active":"2024-05-11T12:11:56.397219+0000","last_became_peered":"2024-05-11T12:11:56.397219+0000","last_unstale":"2024-05-11T12:11:56.402024+0000","last_undegraded":"2024-05-11T12:11:56.402024+0000","last_fullsized":"2024-05-11T12:11:56.402024+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:25:15.336149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.383035+0000","last_change":"2024-05-11T12:11:56.383035+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.383035+0000","last_undegraded":"2024-05-11T12:11:56.383035+0000","last_fullsized":"2024-05-11T12:11:56.383035+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T12:11:56.407895+0000","last_change":"2024-05-11T12:11:56.404006+0000","last_active":"2024-05-11T12:11:56.407895+0000","last_peered":"2024-05-11T12:11:56.407895+0000","last_clean":"2024-05-11T12:11:56.407895+0000","last_became_active":"2024-05-11T12:11:56.403672+0000","last_became_peered":"2024-05-11T12:11:56.403672+0000","last_unstale":"2024-05-11T12:11:56.407895+0000","last_undegraded":"2024-05-11T12:11:56.407895+0000","last_fullsized":"2024-05-11T12:11:56.407895+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:30:56.816516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.388205+0000","last_change":"2024-05-11T12:11:56.388205+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.388205+0000","last_undegraded":"2024-05-11T12:11:56.388205+0000","last_fullsized":"2024-05-11T12:11:56.388205+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.401153+0000","last_change":"2024-05-11T12:11:56.399122+0000","last_active":"2024-05-11T12:11:57.401153+0000","last_peered":"2024-05-11T12:11:57.401153+0000","last_clean":"2024-05-11T12:11:57.401153+0000","last_became_active":"2024-05-11T12:11:56.398845+0000","last_became_peered":"2024-05-11T12:11:56.398845+0000","last_unstale":"2024-05-11T12:11:57.401153+0000","last_undegraded":"2024-05-11T12:11:57.401153+0000","last_fullsized":"2024-05-11T12:11:57.401153+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:16:00.825967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.449098+0000","last_change":"2024-05-11T12:11:57.442240+0000","last_active":"2024-05-11T12:11:57.449098+0000","last_peered":"2024-05-11T12:11:57.449098+0000","last_clean":"2024-05-11T12:11:57.449098+0000","last_became_active":"2024-05-11T12:11:57.440831+0000","last_became_peered":"2024-05-11T12:11:57.440831+0000","last_unstale":"2024-05-11T12:11:57.449098+0000","last_undegraded":"2024-05-11T12:11:57.449098+0000","last_fullsized":"2024-05-11T12:11:57.449098+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:28:44.586634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.418097+0000","last_change":"2024-05-11T12:11:56.402880+0000","last_active":"2024-05-11T12:11:57.418097+0000","last_peered":"2024-05-11T12:11:57.418097+0000","last_clean":"2024-05-11T12:11:57.418097+0000","last_became_active":"2024-05-11T12:11:56.402608+0000","last_became_peered":"2024-05-11T12:11:56.402608+0000","last_unstale":"2024-05-11T12:11:57.418097+0000","last_undegraded":"2024-05-11T12:11:57.418097+0000","last_fullsized":"2024-05-11T12:11:57.418097+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:34:29.698881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.451131+0000","last_change":"2024-05-11T12:11:57.444173+0000","last_active":"2024-05-11T12:11:57.451131+0000","last_peered":"2024-05-11T12:11:57.451131+0000","last_clean":"2024-05-11T12:11:57.451131+0000","last_became_active":"2024-05-11T12:11:57.441809+0000","last_became_peered":"2024-05-11T12:11:57.441809+0000","last_unstale":"2024-05-11T12:11:57.451131+0000","last_undegraded":"2024-05-11T12:11:57.451131+0000","last_fullsized":"2024-05-11T12:11:57.451131+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:48:54.195365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T12:11:56.405633+0000","last_change":"2024-05-11T12:11:56.399468+0000","last_active":"2024-05-11T12:11:56.405633+0000","last_peered":"2024-05-11T12:11:56.405633+0000","last_clean":"2024-05-11T12:11:56.405633+0000","last_became_active":"2024-05-11T12:11:56.399205+0000","last_became_peered":"2024-05-11T12:11:56.399205+0000","last_unstale":"2024-05-11T12:11:56.405633+0000","last_undegraded":"2024-05-11T12:11:56.405633+0000","last_fullsized":"2024-05-11T12:11:56.405633+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:10:04.020340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.430065+0000","last_change":"2024-05-11T12:11:57.417010+0000","last_active":"2024-05-11T12:11:57.430065+0000","last_peered":"2024-05-11T12:11:57.430065+0000","last_clean":"2024-05-11T12:11:57.430065+0000","last_became_active":"2024-05-11T12:11:57.416499+0000","last_became_peered":"2024-05-11T12:11:57.416499+0000","last_unstale":"2024-05-11T12:11:57.430065+0000","last_undegraded":"2024-05-11T12:11:57.430065+0000","last_fullsized":"2024-05-11T12:11:57.430065+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:53:03.480916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.418829+0000","last_change":"2024-05-11T12:11:56.400810+0000","last_active":"2024-05-11T12:11:57.418829+0000","last_peered":"2024-05-11T12:11:57.418829+0000","last_clean":"2024-05-11T12:11:57.418829+0000","last_became_active":"2024-05-11T12:11:56.399824+0000","last_became_peered":"2024-05-11T12:11:56.399824+0000","last_unstale":"2024-05-11T12:11:57.418829+0000","last_undegraded":"2024-05-11T12:11:57.418829+0000","last_fullsized":"2024-05-11T12:11:57.418829+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:07:21.760158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":139,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.414520+0000","last_change":"2024-05-11T12:10:35.066673+0000","last_active":"2024-05-11T12:11:57.414520+0000","last_peered":"2024-05-11T12:11:57.414520+0000","last_clean":"2024-05-11T12:11:57.414520+0000","last_became_active":"2024-05-11T12:10:35.065942+0000","last_became_peered":"2024-05-11T12:10:35.065942+0000","last_unstale":"2024-05-11T12:11:57.414520+0000","last_undegraded":"2024-05-11T12:11:57.414520+0000","last_fullsized":"2024-05-11T12:11:57.414520+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_clean_scrub_stamp":"2024-05-11T12:10:33.601136+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:12:02.133182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":116,"acting":116,"num_store_stats":7},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":694,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24,"ondisk_log_size":24,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476756,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28064,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343776,"statfs":{"total":96636764160,"available":96608026624,"internally_reserved":0,"allocated":1060864,"data_stored":784295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476757,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":192827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476756,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":192357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476758,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":192849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509461,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":438272,"data_stored":185078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509462,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":438272,"data_stored":185078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509461,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343808,"statfs":{"total":96636764160,"available":96608059392,"internally_reserved":0,"allocated":1032192,"data_stored":775446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542165,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":178411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T12:12:01.860 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T12:12:02.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.098+0000 7f1994a00640 1 Processor -- start 2024-05-11T12:12:02.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.098+0000 7f1994a00640 1 -- start start 2024-05-11T12:12:02.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.100+0000 7f1994a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f199006d460 0x7f199006d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:02.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.100+0000 7f1994a00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f199006fe10 con 0x7f19900af5d0 2024-05-11T12:12:02.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.100+0000 7f1994a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f199015fcc0 con 0x7f19900af1f0 2024-05-11T12:12:02.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.100+0000 7f1994a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f19901608d0 con 0x7f199006d460 2024-05-11T12:12:02.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.100+0000 7f198d400640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f19900af5d0 0x7f1990073ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:42180/0 (socket says 172.21.15.167:42180) 2024-05-11T12:12:02.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.100+0000 7f198d400640 1 -- 172.21.15.167:0/1659033880 learned_addr learned my addr 172.21.15.167:0/1659033880 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:02.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.100+0000 7f198e800640 1 --2- 172.21.15.167:0/1659033880 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f199006d460 0x7f199006d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:02.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.101+0000 7f198ca00640 1 -- 172.21.15.167:0/1659033880 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1117002041 0 0) 0x7f199006fe10 con 0x7f19900af5d0 2024-05-11T12:12:02.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.101+0000 7f198ca00640 1 -- 172.21.15.167:0/1659033880 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1974003520 con 0x7f19900af5d0 2024-05-11T12:12:02.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.101+0000 7f198ca00640 1 -- 172.21.15.167:0/1659033880 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1113333045 0 0) 0x7f1974003520 con 0x7f19900af5d0 2024-05-11T12:12:02.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.101+0000 7f198ca00640 1 -- 172.21.15.167:0/1659033880 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f199006d460 msgr2=0x7f199006d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:02.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.101+0000 7f198ca00640 1 --2- 172.21.15.167:0/1659033880 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f199006d460 0x7f199006d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:02.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.101+0000 7f198ca00640 1 -- 172.21.15.167:0/1659033880 >> v1:172.21.15.204:6789/0 conn(0x7f19900af1f0 legacy=0x7f19900cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:02.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.101+0000 7f198ca00640 1 -- 172.21.15.167:0/1659033880 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f199015ffa0 con 0x7f19900af5d0 2024-05-11T12:12:02.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.101+0000 7f198ca00640 1 -- 172.21.15.167:0/1659033880 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f197c0016f0 con 0x7f19900af5d0 2024-05-11T12:12:02.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.101+0000 7f198ca00640 1 -- 172.21.15.167:0/1659033880 >> v1:172.21.15.167:6789/0 conn(0x7f19900af5d0 legacy=0x7f1990073ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:02.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.101+0000 7f198ca00640 1 --2- 172.21.15.167:0/1659033880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1974003a80 0x7f1974003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:02.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.101+0000 7f198ca00640 1 --2- 172.21.15.167:0/1659033880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1974004800 0x7f1974004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:02.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.101+0000 7f198ca00640 1 --2- 172.21.15.167:0/1659033880 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f199006d460 0x7f197400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:02.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.101+0000 7f198ca00640 1 -- 172.21.15.167:0/1659033880 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f19901608d0 con 0x7f1974003a80 2024-05-11T12:12:02.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.101+0000 7f198ca00640 1 -- 172.21.15.167:0/1659033880 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1974005550 con 0x7f1974004800 2024-05-11T12:12:02.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.101+0000 7f198ca00640 1 -- 172.21.15.167:0/1659033880 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f197400bf00 con 0x7f199006d460 2024-05-11T12:12:02.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.102+0000 7f198de00640 1 --2- 172.21.15.167:0/1659033880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1974003a80 0x7f1974003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:02.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.102+0000 7f198e800640 1 --2- 172.21.15.167:0/1659033880 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f199006d460 0x7f197400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:02.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.102+0000 7f198d400640 1 --2- 172.21.15.167:0/1659033880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1974004800 0x7f1974004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:02.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.102+0000 7f198de00640 1 -- 172.21.15.167:0/1659033880 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f199006d460 msgr2=0x7f197400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:02.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.102+0000 7f198de00640 1 --2- 172.21.15.167:0/1659033880 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f199006d460 0x7f197400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:02.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.102+0000 7f198de00640 1 -- 172.21.15.167:0/1659033880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1974004800 msgr2=0x7f1974004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:02.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.102+0000 7f198de00640 1 --2- 172.21.15.167:0/1659033880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1974004800 0x7f1974004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:02.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.102+0000 7f198de00640 1 -- 172.21.15.167:0/1659033880 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1974003520 con 0x7f1974003a80 2024-05-11T12:12:02.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.103+0000 7f198de00640 1 --2- 172.21.15.167:0/1659033880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1974003a80 0x7f1974003ea0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1980001e90 tx=0x7f1974005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a344581b2afb53d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:02.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.104+0000 7f198ca00640 1 -- 172.21.15.167:0/1659033880 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1980001ab0 con 0x7f1974003a80 2024-05-11T12:12:02.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.104+0000 7f198ca00640 1 -- 172.21.15.167:0/1659033880 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f198002ae50 con 0x7f1974003a80 2024-05-11T12:12:02.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.104+0000 7f198ca00640 1 -- 172.21.15.167:0/1659033880 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1980003b70 con 0x7f1974003a80 2024-05-11T12:12:02.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.104+0000 7f1994a00640 1 -- 172.21.15.167:0/1659033880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1974003a80 msgr2=0x7f1974003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:02.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.104+0000 7f1994a00640 1 --2- 172.21.15.167:0/1659033880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1974003a80 0x7f1974003ea0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1980001e90 tx=0x7f1974005830 comp rx=0 tx=0).stop 2024-05-11T12:12:02.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.104+0000 7f198e800640 1 -- 172.21.15.167:0/1659033880 reap_dead start 2024-05-11T12:12:02.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.104+0000 7f1994a00640 1 -- 172.21.15.167:0/1659033880 shutdown_connections 2024-05-11T12:12:02.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.104+0000 7f1994a00640 1 -- 172.21.15.167:0/1659033880 >> 172.21.15.167:0/1659033880 conn(0x7f19900cd8f0 msgr2=0x7f19900cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:02.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.104+0000 7f1994a00640 1 -- 172.21.15.167:0/1659033880 shutdown_connections 2024-05-11T12:12:02.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.104+0000 7f1994a00640 1 -- 172.21.15.167:0/1659033880 wait complete. 2024-05-11T12:12:02.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.106+0000 7f1994a00640 1 Processor -- start 2024-05-11T12:12:02.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.106+0000 7f1994a00640 1 -- start start 2024-05-11T12:12:02.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.107+0000 7f1994a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1974003a80 0x7f1990159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:02.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.107+0000 7f1994a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f199015c390 0x7f1990162ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:02.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.107+0000 7f1994a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1990154610 0x7f1990154a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:02.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.107+0000 7f1994a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1990160560 con 0x7f1990154610 2024-05-11T12:12:02.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.107+0000 7f1994a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f199006de10 con 0x7f1974003a80 2024-05-11T12:12:02.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.107+0000 7f1994a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f199015f920 con 0x7f199015c390 2024-05-11T12:12:02.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.108+0000 7f198e800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1990154610 0x7f1990154a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:02.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.108+0000 7f198e800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1990154610 0x7f1990154a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41872/0 (socket says 172.21.15.167:41872) 2024-05-11T12:12:02.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.108+0000 7f198e800640 1 -- 172.21.15.167:0/907291550 learned_addr learned my addr 172.21.15.167:0/907291550 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:02.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.108+0000 7f198de00640 1 --2- 172.21.15.167:0/907291550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1974003a80 0x7f1990159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:02.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.108+0000 7f198d400640 1 --2- 172.21.15.167:0/907291550 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f199015c390 0x7f1990162ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:02.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.108+0000 7f198e800640 1 -- 172.21.15.167:0/907291550 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f199015c390 msgr2=0x7f1990162ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:02.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.108+0000 7f198e800640 1 --2- 172.21.15.167:0/907291550 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f199015c390 0x7f1990162ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:02.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.108+0000 7f198e800640 1 -- 172.21.15.167:0/907291550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1974003a80 msgr2=0x7f1990159cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:02.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.108+0000 7f198e800640 1 --2- 172.21.15.167:0/907291550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1974003a80 0x7f1990159cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:02.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.108+0000 7f198e800640 1 -- 172.21.15.167:0/907291550 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1990154f90 con 0x7f1990154610 2024-05-11T12:12:02.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.109+0000 7f198e800640 1 --2- 172.21.15.167:0/907291550 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1990154610 0x7f1990154a50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1988011490 tx=0x7f1988011950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:02.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.109+0000 7f1986000640 1 -- 172.21.15.167:0/907291550 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1988003610 con 0x7f1990154610 2024-05-11T12:12:02.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.109+0000 7f1986000640 1 -- 172.21.15.167:0/907291550 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f198801b070 con 0x7f1990154610 2024-05-11T12:12:02.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.109+0000 7f1994a00640 1 -- 172.21.15.167:0/907291550 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f199015d3d0 con 0x7f1990154610 2024-05-11T12:12:02.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.109+0000 7f1986000640 1 -- 172.21.15.167:0/907291550 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19880039f0 con 0x7f1990154610 2024-05-11T12:12:02.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.109+0000 7f1994a00640 1 -- 172.21.15.167:0/907291550 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f199015ce10 con 0x7f1990154610 2024-05-11T12:12:02.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.110+0000 7f1994a00640 1 -- 172.21.15.167:0/907291550 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1990069590 con 0x7f1990154610 2024-05-11T12:12:02.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.111+0000 7f1986000640 1 -- 172.21.15.167:0/907291550 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1988003b90 con 0x7f1990154610 2024-05-11T12:12:02.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.112+0000 7f1986000640 1 --2- 172.21.15.167:0/907291550 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f19700762d0 0x7f1970078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:02.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.112+0000 7f1986000640 1 -- 172.21.15.167:0/907291550 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f1988097480 con 0x7f1990154610 2024-05-11T12:12:02.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.112+0000 7f198de00640 1 --2- 172.21.15.167:0/907291550 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f19700762d0 0x7f1970078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:02.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.113+0000 7f198de00640 1 --2- 172.21.15.167:0/907291550 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f19700762d0 0x7f1970078790 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1980032000 tx=0x7f198002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:02.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.117+0000 7f1986000640 1 -- 172.21.15.167:0/907291550 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1988024020 con 0x7f1990154610 2024-05-11T12:12:02.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.322+0000 7f1994a00640 1 -- 172.21.15.167:0/907291550 --> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f199006f0e0 con 0x7f19700762d0 2024-05-11T12:12:02.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.340+0000 7f1986000640 1 -- 172.21.15.167:0/907291550 <== mgr.4106 v2:172.21.15.167:6832/578352559 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248595 (secure 0 0 0) 0x7f198007eb30 con 0x7f19700762d0 2024-05-11T12:12:02.344 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:12:02.345 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2024-05-11T12:12:02.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.346+0000 7f1994a00640 1 -- 172.21.15.167:0/907291550 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f19700762d0 msgr2=0x7f1970078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:02.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.346+0000 7f1994a00640 1 --2- 172.21.15.167:0/907291550 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f19700762d0 0x7f1970078790 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1980032000 tx=0x7f198002e000 comp rx=0 tx=0).stop 2024-05-11T12:12:02.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.346+0000 7f1994a00640 1 -- 172.21.15.167:0/907291550 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1990154610 msgr2=0x7f1990154a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:02.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.346+0000 7f1994a00640 1 --2- 172.21.15.167:0/907291550 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1990154610 0x7f1990154a50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1988011490 tx=0x7f1988011950 comp rx=0 tx=0).stop 2024-05-11T12:12:02.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.346+0000 7f1994a00640 1 -- 172.21.15.167:0/907291550 shutdown_connections 2024-05-11T12:12:02.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.346+0000 7f1994a00640 1 --2- 172.21.15.167:0/907291550 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1990154610 0x7f1990154a50 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:02.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.346+0000 7f1994a00640 1 --2- 172.21.15.167:0/907291550 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f199015c390 0x7f1990162ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:02.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.346+0000 7f1994a00640 1 --2- 172.21.15.167:0/907291550 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f19700762d0 0x7f1970078790 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:02.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.346+0000 7f1994a00640 1 --2- 172.21.15.167:0/907291550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1974003a80 0x7f1990159cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:02.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.346+0000 7f1994a00640 1 -- 172.21.15.167:0/907291550 >> 172.21.15.167:0/907291550 conn(0x7f19900cd8f0 msgr2=0x7f19900cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:02.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.346+0000 7f1994a00640 1 -- 172.21.15.167:0/907291550 shutdown_connections 2024-05-11T12:12:02.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.346+0000 7f1994a00640 1 -- 172.21.15.167:0/907291550 wait complete. 2024-05-11T12:12:02.366 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":79,"stamp":"2024-05-11T12:12:01.258163+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591062,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":131,"num_write_kb":1631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":96,"ondisk_log_size":96,"up":182,"acting":182,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":100,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220844,"kb_used_data":4656,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753876,"statfs":{"total":773094113280,"available":772867969024,"internally_reserved":0,"allocated":4767744,"data_stored":2686341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":694,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.895997"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.383260+0000","last_change":"2024-05-11T12:11:56.383260+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.383260+0000","last_undegraded":"2024-05-11T12:11:56.383260+0000","last_fullsized":"2024-05-11T12:11:56.383260+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.381407+0000","last_change":"2024-05-11T12:11:56.381407+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.381407+0000","last_undegraded":"2024-05-11T12:11:56.381407+0000","last_fullsized":"2024-05-11T12:11:56.381407+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.388722+0000","last_change":"2024-05-11T12:11:56.388722+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.388722+0000","last_undegraded":"2024-05-11T12:11:56.388722+0000","last_fullsized":"2024-05-11T12:11:56.388722+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T12:11:56.360286+0000","last_change":"2024-05-11T12:11:56.360286+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.360286+0000","last_undegraded":"2024-05-11T12:11:56.360286+0000","last_fullsized":"2024-05-11T12:11:56.360286+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T12:11:56.360286+0000","last_change":"2024-05-11T12:11:56.360286+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.360286+0000","last_undegraded":"2024-05-11T12:11:56.360286+0000","last_fullsized":"2024-05-11T12:11:56.360286+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.386336+0000","last_change":"2024-05-11T12:11:56.386336+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.386336+0000","last_undegraded":"2024-05-11T12:11:56.386336+0000","last_fullsized":"2024-05-11T12:11:56.386336+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.437906+0000","last_change":"2024-05-11T12:11:57.432877+0000","last_active":"2024-05-11T12:11:57.437906+0000","last_peered":"2024-05-11T12:11:57.437906+0000","last_clean":"2024-05-11T12:11:57.437906+0000","last_became_active":"2024-05-11T12:11:57.432581+0000","last_became_peered":"2024-05-11T12:11:57.432581+0000","last_unstale":"2024-05-11T12:11:57.437906+0000","last_undegraded":"2024-05-11T12:11:57.437906+0000","last_fullsized":"2024-05-11T12:11:57.437906+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:47:49.623473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.381742+0000","last_change":"2024-05-11T12:11:56.381742+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.381742+0000","last_undegraded":"2024-05-11T12:11:56.381742+0000","last_fullsized":"2024-05-11T12:11:56.381742+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.384226+0000","last_change":"2024-05-11T12:11:56.384226+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.384226+0000","last_undegraded":"2024-05-11T12:11:56.384226+0000","last_fullsized":"2024-05-11T12:11:56.384226+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.439811+0000","last_change":"2024-05-11T12:11:57.434958+0000","last_active":"2024-05-11T12:11:57.439811+0000","last_peered":"2024-05-11T12:11:57.439811+0000","last_clean":"2024-05-11T12:11:57.439811+0000","last_became_active":"2024-05-11T12:11:57.429221+0000","last_became_peered":"2024-05-11T12:11:57.429221+0000","last_unstale":"2024-05-11T12:11:57.439811+0000","last_undegraded":"2024-05-11T12:11:57.439811+0000","last_fullsized":"2024-05-11T12:11:57.439811+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:32:26.804013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.455023+0000","last_change":"2024-05-11T12:11:57.449552+0000","last_active":"2024-05-11T12:11:57.455023+0000","last_peered":"2024-05-11T12:11:57.455023+0000","last_clean":"2024-05-11T12:11:57.455023+0000","last_became_active":"2024-05-11T12:11:57.441862+0000","last_became_peered":"2024-05-11T12:11:57.441862+0000","last_unstale":"2024-05-11T12:11:57.455023+0000","last_undegraded":"2024-05-11T12:11:57.455023+0000","last_fullsized":"2024-05-11T12:11:57.455023+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:04:26.168833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.386630+0000","last_change":"2024-05-11T12:11:56.386630+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.386630+0000","last_undegraded":"2024-05-11T12:11:56.386630+0000","last_fullsized":"2024-05-11T12:11:56.386630+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.432544+0000","last_change":"2024-05-11T12:11:57.429706+0000","last_active":"2024-05-11T12:11:57.432544+0000","last_peered":"2024-05-11T12:11:57.432544+0000","last_clean":"2024-05-11T12:11:57.432544+0000","last_became_active":"2024-05-11T12:11:57.429387+0000","last_became_peered":"2024-05-11T12:11:57.429387+0000","last_unstale":"2024-05-11T12:11:57.432544+0000","last_undegraded":"2024-05-11T12:11:57.432544+0000","last_fullsized":"2024-05-11T12:11:57.432544+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:55:35.468623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.445013+0000","last_change":"2024-05-11T12:11:57.430917+0000","last_active":"2024-05-11T12:11:57.445013+0000","last_peered":"2024-05-11T12:11:57.445013+0000","last_clean":"2024-05-11T12:11:57.445013+0000","last_became_active":"2024-05-11T12:11:57.430613+0000","last_became_peered":"2024-05-11T12:11:57.430613+0000","last_unstale":"2024-05-11T12:11:57.445013+0000","last_undegraded":"2024-05-11T12:11:57.445013+0000","last_fullsized":"2024-05-11T12:11:57.445013+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:25:16.350527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.454620+0000","last_change":"2024-05-11T12:11:57.438016+0000","last_active":"2024-05-11T12:11:57.454620+0000","last_peered":"2024-05-11T12:11:57.454620+0000","last_clean":"2024-05-11T12:11:57.454620+0000","last_became_active":"2024-05-11T12:11:57.437680+0000","last_became_peered":"2024-05-11T12:11:57.437680+0000","last_unstale":"2024-05-11T12:11:57.454620+0000","last_undegraded":"2024-05-11T12:11:57.454620+0000","last_fullsized":"2024-05-11T12:11:57.454620+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:28:33.430453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.453506+0000","last_change":"2024-05-11T12:11:57.448672+0000","last_active":"2024-05-11T12:11:57.453506+0000","last_peered":"2024-05-11T12:11:57.453506+0000","last_clean":"2024-05-11T12:11:57.453506+0000","last_became_active":"2024-05-11T12:11:57.447489+0000","last_became_peered":"2024-05-11T12:11:57.447489+0000","last_unstale":"2024-05-11T12:11:57.453506+0000","last_undegraded":"2024-05-11T12:11:57.453506+0000","last_fullsized":"2024-05-11T12:11:57.453506+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:53:03.480916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.451781+0000","last_change":"2024-05-11T12:11:57.447831+0000","last_active":"2024-05-11T12:11:57.451781+0000","last_peered":"2024-05-11T12:11:57.451781+0000","last_clean":"2024-05-11T12:11:57.451781+0000","last_became_active":"2024-05-11T12:11:57.443540+0000","last_became_peered":"2024-05-11T12:11:57.443540+0000","last_unstale":"2024-05-11T12:11:57.451781+0000","last_undegraded":"2024-05-11T12:11:57.451781+0000","last_fullsized":"2024-05-11T12:11:57.451781+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:26:34.393703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.435968+0000","last_change":"2024-05-11T12:11:57.431125+0000","last_active":"2024-05-11T12:11:57.435968+0000","last_peered":"2024-05-11T12:11:57.435968+0000","last_clean":"2024-05-11T12:11:57.435968+0000","last_became_active":"2024-05-11T12:11:57.430835+0000","last_became_peered":"2024-05-11T12:11:57.430835+0000","last_unstale":"2024-05-11T12:11:57.435968+0000","last_undegraded":"2024-05-11T12:11:57.435968+0000","last_fullsized":"2024-05-11T12:11:57.435968+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:23:28.626421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T12:11:56.360286+0000","last_change":"2024-05-11T12:11:56.360286+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.360286+0000","last_undegraded":"2024-05-11T12:11:56.360286+0000","last_fullsized":"2024-05-11T12:11:56.360286+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.436449+0000","last_change":"2024-05-11T12:11:57.433084+0000","last_active":"2024-05-11T12:11:57.436449+0000","last_peered":"2024-05-11T12:11:57.436449+0000","last_clean":"2024-05-11T12:11:57.436449+0000","last_became_active":"2024-05-11T12:11:57.432645+0000","last_became_peered":"2024-05-11T12:11:57.432645+0000","last_unstale":"2024-05-11T12:11:57.436449+0000","last_undegraded":"2024-05-11T12:11:57.436449+0000","last_fullsized":"2024-05-11T12:11:57.436449+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:32:45.644260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.439852+0000","last_change":"2024-05-11T12:11:57.434801+0000","last_active":"2024-05-11T12:11:57.439852+0000","last_peered":"2024-05-11T12:11:57.439852+0000","last_clean":"2024-05-11T12:11:57.439852+0000","last_became_active":"2024-05-11T12:11:57.433969+0000","last_became_peered":"2024-05-11T12:11:57.433969+0000","last_unstale":"2024-05-11T12:11:57.439852+0000","last_undegraded":"2024-05-11T12:11:57.439852+0000","last_fullsized":"2024-05-11T12:11:57.439852+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:22:33.211172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.449144+0000","last_change":"2024-05-11T12:11:57.445771+0000","last_active":"2024-05-11T12:11:57.449144+0000","last_peered":"2024-05-11T12:11:57.449144+0000","last_clean":"2024-05-11T12:11:57.449144+0000","last_became_active":"2024-05-11T12:11:57.445474+0000","last_became_peered":"2024-05-11T12:11:57.445474+0000","last_unstale":"2024-05-11T12:11:57.449144+0000","last_undegraded":"2024-05-11T12:11:57.449144+0000","last_fullsized":"2024-05-11T12:11:57.449144+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:23:28.626421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.451922+0000","last_change":"2024-05-11T12:11:57.447508+0000","last_active":"2024-05-11T12:11:57.451922+0000","last_peered":"2024-05-11T12:11:57.451922+0000","last_clean":"2024-05-11T12:11:57.451922+0000","last_became_active":"2024-05-11T12:11:57.444377+0000","last_became_peered":"2024-05-11T12:11:57.444377+0000","last_unstale":"2024-05-11T12:11:57.451922+0000","last_undegraded":"2024-05-11T12:11:57.451922+0000","last_fullsized":"2024-05-11T12:11:57.451922+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:46:06.174919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T12:11:56.360286+0000","last_change":"2024-05-11T12:11:56.360286+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.360286+0000","last_undegraded":"2024-05-11T12:11:56.360286+0000","last_fullsized":"2024-05-11T12:11:56.360286+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.430674+0000","last_change":"2024-05-11T12:11:57.426462+0000","last_active":"2024-05-11T12:11:57.430674+0000","last_peered":"2024-05-11T12:11:57.430674+0000","last_clean":"2024-05-11T12:11:57.430674+0000","last_became_active":"2024-05-11T12:11:57.426087+0000","last_became_peered":"2024-05-11T12:11:57.426087+0000","last_unstale":"2024-05-11T12:11:57.430674+0000","last_undegraded":"2024-05-11T12:11:57.430674+0000","last_fullsized":"2024-05-11T12:11:57.430674+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:30:57.830893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.390804+0000","last_change":"2024-05-11T12:11:56.390804+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.390804+0000","last_undegraded":"2024-05-11T12:11:56.390804+0000","last_fullsized":"2024-05-11T12:11:56.390804+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.432318+0000","last_change":"2024-05-11T12:11:57.428594+0000","last_active":"2024-05-11T12:11:57.432318+0000","last_peered":"2024-05-11T12:11:57.432318+0000","last_clean":"2024-05-11T12:11:57.432318+0000","last_became_active":"2024-05-11T12:11:57.428245+0000","last_became_peered":"2024-05-11T12:11:57.428245+0000","last_unstale":"2024-05-11T12:11:57.432318+0000","last_undegraded":"2024-05-11T12:11:57.432318+0000","last_fullsized":"2024-05-11T12:11:57.432318+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:28:44.586634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.430392+0000","last_change":"2024-05-11T12:11:57.417635+0000","last_active":"2024-05-11T12:11:57.430392+0000","last_peered":"2024-05-11T12:11:57.430392+0000","last_clean":"2024-05-11T12:11:57.430392+0000","last_became_active":"2024-05-11T12:11:57.415536+0000","last_became_peered":"2024-05-11T12:11:57.415536+0000","last_unstale":"2024-05-11T12:11:57.430392+0000","last_undegraded":"2024-05-11T12:11:57.430392+0000","last_fullsized":"2024-05-11T12:11:57.430392+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:51:43.649698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.432559+0000","last_change":"2024-05-11T12:11:57.427847+0000","last_active":"2024-05-11T12:11:57.432559+0000","last_peered":"2024-05-11T12:11:57.432559+0000","last_clean":"2024-05-11T12:11:57.432559+0000","last_became_active":"2024-05-11T12:11:57.427416+0000","last_became_peered":"2024-05-11T12:11:57.427416+0000","last_unstale":"2024-05-11T12:11:57.432559+0000","last_undegraded":"2024-05-11T12:11:57.432559+0000","last_fullsized":"2024-05-11T12:11:57.432559+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:07:22.774535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.423604+0000","last_change":"2024-05-11T12:11:57.420465+0000","last_active":"2024-05-11T12:11:57.423604+0000","last_peered":"2024-05-11T12:11:57.423604+0000","last_clean":"2024-05-11T12:11:57.423604+0000","last_became_active":"2024-05-11T12:11:57.419963+0000","last_became_peered":"2024-05-11T12:11:57.419963+0000","last_unstale":"2024-05-11T12:11:57.423604+0000","last_undegraded":"2024-05-11T12:11:57.423604+0000","last_fullsized":"2024-05-11T12:11:57.423604+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:36:41.799582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.451900+0000","last_change":"2024-05-11T12:11:57.447209+0000","last_active":"2024-05-11T12:11:57.451900+0000","last_peered":"2024-05-11T12:11:57.451900+0000","last_clean":"2024-05-11T12:11:57.451900+0000","last_became_active":"2024-05-11T12:11:57.443801+0000","last_became_peered":"2024-05-11T12:11:57.443801+0000","last_unstale":"2024-05-11T12:11:57.451900+0000","last_undegraded":"2024-05-11T12:11:57.451900+0000","last_fullsized":"2024-05-11T12:11:57.451900+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:10:05.034718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.389994+0000","last_change":"2024-05-11T12:11:56.389994+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.389994+0000","last_undegraded":"2024-05-11T12:11:56.389994+0000","last_fullsized":"2024-05-11T12:11:56.389994+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.451196+0000","last_change":"2024-05-11T12:11:57.444554+0000","last_active":"2024-05-11T12:11:57.451196+0000","last_peered":"2024-05-11T12:11:57.451196+0000","last_clean":"2024-05-11T12:11:57.451196+0000","last_became_active":"2024-05-11T12:11:57.443836+0000","last_became_peered":"2024-05-11T12:11:57.443836+0000","last_unstale":"2024-05-11T12:11:57.451196+0000","last_undegraded":"2024-05-11T12:11:57.451196+0000","last_fullsized":"2024-05-11T12:11:57.451196+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:20:14.138435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"40'4","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.451553+0000","last_change":"2024-05-11T12:11:56.409094+0000","last_active":"2024-05-11T12:11:57.451553+0000","last_peered":"2024-05-11T12:11:57.451553+0000","last_clean":"2024-05-11T12:11:57.451553+0000","last_became_active":"2024-05-11T12:11:56.408820+0000","last_became_peered":"2024-05-11T12:11:56.408820+0000","last_unstale":"2024-05-11T12:11:57.451553+0000","last_undegraded":"2024-05-11T12:11:57.451553+0000","last_fullsized":"2024-05-11T12:11:57.451553+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:11:42.632883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.389689+0000","last_change":"2024-05-11T12:11:56.389689+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.389689+0000","last_undegraded":"2024-05-11T12:11:56.389689+0000","last_fullsized":"2024-05-11T12:11:56.389689+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T12:11:56.407872+0000","last_change":"2024-05-11T12:11:56.403002+0000","last_active":"2024-05-11T12:11:56.407872+0000","last_peered":"2024-05-11T12:11:56.407872+0000","last_clean":"2024-05-11T12:11:56.407872+0000","last_became_active":"2024-05-11T12:11:56.402721+0000","last_became_peered":"2024-05-11T12:11:56.402721+0000","last_unstale":"2024-05-11T12:11:56.407872+0000","last_undegraded":"2024-05-11T12:11:56.407872+0000","last_fullsized":"2024-05-11T12:11:56.407872+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:36:40.785205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.450328+0000","last_change":"2024-05-11T12:11:57.435281+0000","last_active":"2024-05-11T12:11:57.450328+0000","last_peered":"2024-05-11T12:11:57.450328+0000","last_clean":"2024-05-11T12:11:57.450328+0000","last_became_active":"2024-05-11T12:11:57.434672+0000","last_became_peered":"2024-05-11T12:11:57.434672+0000","last_unstale":"2024-05-11T12:11:57.450328+0000","last_undegraded":"2024-05-11T12:11:57.450328+0000","last_fullsized":"2024-05-11T12:11:57.450328+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:30:57.830893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T12:11:56.417027+0000","last_change":"2024-05-11T12:11:56.410313+0000","last_active":"2024-05-11T12:11:56.417027+0000","last_peered":"2024-05-11T12:11:56.417027+0000","last_clean":"2024-05-11T12:11:56.417027+0000","last_became_active":"2024-05-11T12:11:56.409971+0000","last_became_peered":"2024-05-11T12:11:56.409971+0000","last_unstale":"2024-05-11T12:11:56.417027+0000","last_undegraded":"2024-05-11T12:11:56.417027+0000","last_fullsized":"2024-05-11T12:11:56.417027+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:11:42.632883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.382086+0000","last_change":"2024-05-11T12:11:56.382086+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.382086+0000","last_undegraded":"2024-05-11T12:11:56.382086+0000","last_fullsized":"2024-05-11T12:11:56.382086+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"40'2","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.454119+0000","last_change":"2024-05-11T12:11:56.408353+0000","last_active":"2024-05-11T12:11:57.454119+0000","last_peered":"2024-05-11T12:11:57.454119+0000","last_clean":"2024-05-11T12:11:57.454119+0000","last_became_active":"2024-05-11T12:11:56.408078+0000","last_became_peered":"2024-05-11T12:11:56.408078+0000","last_unstale":"2024-05-11T12:11:57.454119+0000","last_undegraded":"2024-05-11T12:11:57.454119+0000","last_fullsized":"2024-05-11T12:11:57.454119+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:16:00.825967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.384699+0000","last_change":"2024-05-11T12:11:56.384699+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.384699+0000","last_undegraded":"2024-05-11T12:11:56.384699+0000","last_fullsized":"2024-05-11T12:11:56.384699+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"40'1","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.455051+0000","last_change":"2024-05-11T12:11:56.404989+0000","last_active":"2024-05-11T12:11:57.455051+0000","last_peered":"2024-05-11T12:11:57.455051+0000","last_clean":"2024-05-11T12:11:57.455051+0000","last_became_active":"2024-05-11T12:11:56.404069+0000","last_became_peered":"2024-05-11T12:11:56.404069+0000","last_unstale":"2024-05-11T12:11:57.455051+0000","last_undegraded":"2024-05-11T12:11:57.455051+0000","last_fullsized":"2024-05-11T12:11:57.455051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:26:33.379326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.454982+0000","last_change":"2024-05-11T12:11:57.442633+0000","last_active":"2024-05-11T12:11:57.454982+0000","last_peered":"2024-05-11T12:11:57.454982+0000","last_clean":"2024-05-11T12:11:57.454982+0000","last_became_active":"2024-05-11T12:11:57.442361+0000","last_became_peered":"2024-05-11T12:11:57.442361+0000","last_unstale":"2024-05-11T12:11:57.454982+0000","last_undegraded":"2024-05-11T12:11:57.454982+0000","last_fullsized":"2024-05-11T12:11:57.454982+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:43:20.345625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"40'1","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.451754+0000","last_change":"2024-05-11T12:11:56.411689+0000","last_active":"2024-05-11T12:11:57.451754+0000","last_peered":"2024-05-11T12:11:57.451754+0000","last_clean":"2024-05-11T12:11:57.451754+0000","last_became_active":"2024-05-11T12:11:56.409937+0000","last_became_peered":"2024-05-11T12:11:56.409937+0000","last_unstale":"2024-05-11T12:11:57.451754+0000","last_undegraded":"2024-05-11T12:11:57.451754+0000","last_fullsized":"2024-05-11T12:11:57.451754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:36:40.785205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.448085+0000","last_change":"2024-05-11T12:11:57.435197+0000","last_active":"2024-05-11T12:11:57.448085+0000","last_peered":"2024-05-11T12:11:57.448085+0000","last_clean":"2024-05-11T12:11:57.448085+0000","last_became_active":"2024-05-11T12:11:57.434472+0000","last_became_peered":"2024-05-11T12:11:57.434472+0000","last_unstale":"2024-05-11T12:11:57.448085+0000","last_undegraded":"2024-05-11T12:11:57.448085+0000","last_fullsized":"2024-05-11T12:11:57.448085+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:36:41.799582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T12:11:56.416996+0000","last_change":"2024-05-11T12:11:56.409731+0000","last_active":"2024-05-11T12:11:56.416996+0000","last_peered":"2024-05-11T12:11:56.416996+0000","last_clean":"2024-05-11T12:11:56.416996+0000","last_became_active":"2024-05-11T12:11:56.409450+0000","last_became_peered":"2024-05-11T12:11:56.409450+0000","last_unstale":"2024-05-11T12:11:56.416996+0000","last_undegraded":"2024-05-11T12:11:56.416996+0000","last_fullsized":"2024-05-11T12:11:56.416996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:34:29.698881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.381254+0000","last_change":"2024-05-11T12:11:56.381254+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.381254+0000","last_undegraded":"2024-05-11T12:11:56.381254+0000","last_fullsized":"2024-05-11T12:11:56.381254+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.417407+0000","last_change":"2024-05-11T12:11:56.406587+0000","last_active":"2024-05-11T12:11:57.417407+0000","last_peered":"2024-05-11T12:11:57.417407+0000","last_clean":"2024-05-11T12:11:57.417407+0000","last_became_active":"2024-05-11T12:11:56.405746+0000","last_became_peered":"2024-05-11T12:11:56.405746+0000","last_unstale":"2024-05-11T12:11:57.417407+0000","last_undegraded":"2024-05-11T12:11:57.417407+0000","last_fullsized":"2024-05-11T12:11:57.417407+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:11:10.100041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T12:11:56.360286+0000","last_change":"2024-05-11T12:11:56.360286+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.360286+0000","last_undegraded":"2024-05-11T12:11:56.360286+0000","last_fullsized":"2024-05-11T12:11:56.360286+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T12:11:56.402312+0000","last_change":"2024-05-11T12:11:56.398172+0000","last_active":"2024-05-11T12:11:56.402312+0000","last_peered":"2024-05-11T12:11:56.402312+0000","last_clean":"2024-05-11T12:11:56.402312+0000","last_became_active":"2024-05-11T12:11:56.397860+0000","last_became_peered":"2024-05-11T12:11:56.397860+0000","last_unstale":"2024-05-11T12:11:56.402312+0000","last_undegraded":"2024-05-11T12:11:56.402312+0000","last_fullsized":"2024-05-11T12:11:56.402312+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:52:13.234125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.450935+0000","last_change":"2024-05-11T12:11:57.442952+0000","last_active":"2024-05-11T12:11:57.450935+0000","last_peered":"2024-05-11T12:11:57.450935+0000","last_clean":"2024-05-11T12:11:57.450935+0000","last_became_active":"2024-05-11T12:11:57.436880+0000","last_became_peered":"2024-05-11T12:11:57.436880+0000","last_unstale":"2024-05-11T12:11:57.450935+0000","last_undegraded":"2024-05-11T12:11:57.450935+0000","last_fullsized":"2024-05-11T12:11:57.450935+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:23:28.626421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.405805+0000","last_change":"2024-05-11T12:11:56.411981+0000","last_active":"2024-05-11T12:11:57.405805+0000","last_peered":"2024-05-11T12:11:57.405805+0000","last_clean":"2024-05-11T12:11:57.405805+0000","last_became_active":"2024-05-11T12:11:56.409613+0000","last_became_peered":"2024-05-11T12:11:56.409613+0000","last_unstale":"2024-05-11T12:11:57.405805+0000","last_undegraded":"2024-05-11T12:11:57.405805+0000","last_fullsized":"2024-05-11T12:11:57.405805+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:46:05.160542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.384922+0000","last_change":"2024-05-11T12:11:56.384922+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.384922+0000","last_undegraded":"2024-05-11T12:11:56.384922+0000","last_fullsized":"2024-05-11T12:11:56.384922+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"40'3","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.456654+0000","last_change":"2024-05-11T12:11:56.407591+0000","last_active":"2024-05-11T12:11:57.456654+0000","last_peered":"2024-05-11T12:11:57.456654+0000","last_clean":"2024-05-11T12:11:57.456654+0000","last_became_active":"2024-05-11T12:11:56.407173+0000","last_became_peered":"2024-05-11T12:11:56.407173+0000","last_unstale":"2024-05-11T12:11:57.456654+0000","last_undegraded":"2024-05-11T12:11:57.456654+0000","last_fullsized":"2024-05-11T12:11:57.456654+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:52:13.234125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.450953+0000","last_change":"2024-05-11T12:11:57.447257+0000","last_active":"2024-05-11T12:11:57.450953+0000","last_peered":"2024-05-11T12:11:57.450953+0000","last_clean":"2024-05-11T12:11:57.450953+0000","last_became_active":"2024-05-11T12:11:57.445351+0000","last_became_peered":"2024-05-11T12:11:57.445351+0000","last_unstale":"2024-05-11T12:11:57.450953+0000","last_undegraded":"2024-05-11T12:11:57.450953+0000","last_fullsized":"2024-05-11T12:11:57.450953+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:32:45.644260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T12:11:56.395987+0000","last_change":"2024-05-11T12:11:56.394318+0000","last_active":"2024-05-11T12:11:56.395987+0000","last_peered":"2024-05-11T12:11:56.395987+0000","last_clean":"2024-05-11T12:11:56.395987+0000","last_became_active":"2024-05-11T12:11:56.393560+0000","last_became_peered":"2024-05-11T12:11:56.393560+0000","last_unstale":"2024-05-11T12:11:56.395987+0000","last_undegraded":"2024-05-11T12:11:56.395987+0000","last_fullsized":"2024-05-11T12:11:56.395987+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:48:53.180988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.383778+0000","last_change":"2024-05-11T12:11:56.383778+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.383778+0000","last_undegraded":"2024-05-11T12:11:56.383778+0000","last_fullsized":"2024-05-11T12:11:56.383778+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T12:11:56.402064+0000","last_change":"2024-05-11T12:11:56.397825+0000","last_active":"2024-05-11T12:11:56.402064+0000","last_peered":"2024-05-11T12:11:56.402064+0000","last_clean":"2024-05-11T12:11:56.402064+0000","last_became_active":"2024-05-11T12:11:56.397506+0000","last_became_peered":"2024-05-11T12:11:56.397506+0000","last_unstale":"2024-05-11T12:11:56.402064+0000","last_undegraded":"2024-05-11T12:11:56.402064+0000","last_fullsized":"2024-05-11T12:11:56.402064+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:20:13.124057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.380190+0000","last_change":"2024-05-11T12:11:56.380190+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.380190+0000","last_undegraded":"2024-05-11T12:11:56.380190+0000","last_fullsized":"2024-05-11T12:11:56.380190+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T12:11:56.417002+0000","last_change":"2024-05-11T12:11:56.409815+0000","last_active":"2024-05-11T12:11:56.417002+0000","last_peered":"2024-05-11T12:11:56.417002+0000","last_clean":"2024-05-11T12:11:56.417002+0000","last_became_active":"2024-05-11T12:11:56.409411+0000","last_became_peered":"2024-05-11T12:11:56.409411+0000","last_unstale":"2024-05-11T12:11:56.417002+0000","last_undegraded":"2024-05-11T12:11:56.417002+0000","last_fullsized":"2024-05-11T12:11:56.417002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:16:00.825967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.450828+0000","last_change":"2024-05-11T12:11:57.442669+0000","last_active":"2024-05-11T12:11:57.450828+0000","last_peered":"2024-05-11T12:11:57.450828+0000","last_clean":"2024-05-11T12:11:57.450828+0000","last_became_active":"2024-05-11T12:11:57.437559+0000","last_became_peered":"2024-05-11T12:11:57.437559+0000","last_unstale":"2024-05-11T12:11:57.450828+0000","last_undegraded":"2024-05-11T12:11:57.450828+0000","last_fullsized":"2024-05-11T12:11:57.450828+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:28:44.586634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.405638+0000","last_change":"2024-05-11T12:11:56.408411+0000","last_active":"2024-05-11T12:11:57.405638+0000","last_peered":"2024-05-11T12:11:57.405638+0000","last_clean":"2024-05-11T12:11:57.405638+0000","last_became_active":"2024-05-11T12:11:56.408100+0000","last_became_peered":"2024-05-11T12:11:56.408100+0000","last_unstale":"2024-05-11T12:11:57.405638+0000","last_undegraded":"2024-05-11T12:11:57.405638+0000","last_fullsized":"2024-05-11T12:11:57.405638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:25:15.336149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.380730+0000","last_change":"2024-05-11T12:11:56.380730+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.380730+0000","last_undegraded":"2024-05-11T12:11:56.380730+0000","last_fullsized":"2024-05-11T12:11:56.380730+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"40'8","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.457343+0000","last_change":"2024-05-11T12:11:56.398517+0000","last_active":"2024-05-11T12:11:57.457343+0000","last_peered":"2024-05-11T12:11:57.457343+0000","last_clean":"2024-05-11T12:11:57.457343+0000","last_became_active":"2024-05-11T12:11:56.398204+0000","last_became_peered":"2024-05-11T12:11:56.398204+0000","last_unstale":"2024-05-11T12:11:57.457343+0000","last_undegraded":"2024-05-11T12:11:57.457343+0000","last_fullsized":"2024-05-11T12:11:57.457343+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:25:15.336149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.379813+0000","last_change":"2024-05-11T12:11:56.379813+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.379813+0000","last_undegraded":"2024-05-11T12:11:56.379813+0000","last_fullsized":"2024-05-11T12:11:56.379813+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"40'1","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.455210+0000","last_change":"2024-05-11T12:11:56.410975+0000","last_active":"2024-05-11T12:11:57.455210+0000","last_peered":"2024-05-11T12:11:57.455210+0000","last_clean":"2024-05-11T12:11:57.455210+0000","last_became_active":"2024-05-11T12:11:56.410634+0000","last_became_peered":"2024-05-11T12:11:56.410634+0000","last_unstale":"2024-05-11T12:11:57.455210+0000","last_undegraded":"2024-05-11T12:11:57.455210+0000","last_fullsized":"2024-05-11T12:11:57.455210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:10:04.020340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.433269+0000","last_change":"2024-05-11T12:11:57.428161+0000","last_active":"2024-05-11T12:11:57.433269+0000","last_peered":"2024-05-11T12:11:57.433269+0000","last_clean":"2024-05-11T12:11:57.433269+0000","last_became_active":"2024-05-11T12:11:57.427826+0000","last_became_peered":"2024-05-11T12:11:57.427826+0000","last_unstale":"2024-05-11T12:11:57.433269+0000","last_undegraded":"2024-05-11T12:11:57.433269+0000","last_fullsized":"2024-05-11T12:11:57.433269+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:11:37.571584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"40'2","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.459816+0000","last_change":"2024-05-11T12:11:56.406473+0000","last_active":"2024-05-11T12:11:57.459816+0000","last_peered":"2024-05-11T12:11:57.459816+0000","last_clean":"2024-05-11T12:11:57.459816+0000","last_became_active":"2024-05-11T12:11:56.405923+0000","last_became_peered":"2024-05-11T12:11:56.405923+0000","last_unstale":"2024-05-11T12:11:57.459816+0000","last_undegraded":"2024-05-11T12:11:57.459816+0000","last_fullsized":"2024-05-11T12:11:57.459816+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:11:42.632883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.450353+0000","last_change":"2024-05-11T12:11:57.436042+0000","last_active":"2024-05-11T12:11:57.450353+0000","last_peered":"2024-05-11T12:11:57.450353+0000","last_clean":"2024-05-11T12:11:57.450353+0000","last_became_active":"2024-05-11T12:11:57.435781+0000","last_became_peered":"2024-05-11T12:11:57.435781+0000","last_unstale":"2024-05-11T12:11:57.450353+0000","last_undegraded":"2024-05-11T12:11:57.450353+0000","last_fullsized":"2024-05-11T12:11:57.450353+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:07:22.774535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-11T12:11:55.388590+0000","last_change":"2024-05-11T12:11:55.388590+0000","last_active":"2024-05-11T12:11:55.345908+0000","last_peered":"2024-05-11T12:11:55.345908+0000","last_clean":"2024-05-11T12:11:55.345908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:55.388590+0000","last_undegraded":"2024-05-11T12:11:55.388590+0000","last_fullsized":"2024-05-11T12:11:55.388590+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T12:11:56.360286+0000","last_change":"2024-05-11T12:11:56.360286+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.360286+0000","last_undegraded":"2024-05-11T12:11:56.360286+0000","last_fullsized":"2024-05-11T12:11:56.360286+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T12:11:56.416911+0000","last_change":"2024-05-11T12:11:56.409279+0000","last_active":"2024-05-11T12:11:56.416911+0000","last_peered":"2024-05-11T12:11:56.416911+0000","last_clean":"2024-05-11T12:11:56.416911+0000","last_became_active":"2024-05-11T12:11:56.408985+0000","last_became_peered":"2024-05-11T12:11:56.408985+0000","last_unstale":"2024-05-11T12:11:56.416911+0000","last_undegraded":"2024-05-11T12:11:56.416911+0000","last_fullsized":"2024-05-11T12:11:56.416911+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:11:10.100041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.453397+0000","last_change":"2024-05-11T12:11:57.443914+0000","last_active":"2024-05-11T12:11:57.453397+0000","last_peered":"2024-05-11T12:11:57.453397+0000","last_clean":"2024-05-11T12:11:57.453397+0000","last_became_active":"2024-05-11T12:11:57.443680+0000","last_became_peered":"2024-05-11T12:11:57.443680+0000","last_unstale":"2024-05-11T12:11:57.453397+0000","last_undegraded":"2024-05-11T12:11:57.453397+0000","last_fullsized":"2024-05-11T12:11:57.453397+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:32:45.644260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-11T12:11:55.388296+0000","last_change":"2024-05-11T12:11:55.388296+0000","last_active":"2024-05-11T12:11:55.345908+0000","last_peered":"2024-05-11T12:11:55.345908+0000","last_clean":"2024-05-11T12:11:55.345908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:55.388296+0000","last_undegraded":"2024-05-11T12:11:55.388296+0000","last_fullsized":"2024-05-11T12:11:55.388296+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.383307+0000","last_change":"2024-05-11T12:11:56.383307+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.383307+0000","last_undegraded":"2024-05-11T12:11:56.383307+0000","last_fullsized":"2024-05-11T12:11:56.383307+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-11T12:11:55.388497+0000","last_change":"2024-05-11T12:11:55.388497+0000","last_active":"2024-05-11T12:11:55.345908+0000","last_peered":"2024-05-11T12:11:55.345908+0000","last_clean":"2024-05-11T12:11:55.345908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:55.388497+0000","last_undegraded":"2024-05-11T12:11:55.388497+0000","last_fullsized":"2024-05-11T12:11:55.388497+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.431245+0000","last_change":"2024-05-11T12:11:57.419098+0000","last_active":"2024-05-11T12:11:57.431245+0000","last_peered":"2024-05-11T12:11:57.431245+0000","last_clean":"2024-05-11T12:11:57.431245+0000","last_became_active":"2024-05-11T12:11:57.417628+0000","last_became_peered":"2024-05-11T12:11:57.417628+0000","last_unstale":"2024-05-11T12:11:57.431245+0000","last_undegraded":"2024-05-11T12:11:57.431245+0000","last_fullsized":"2024-05-11T12:11:57.431245+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:17:59.309336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.414852+0000","last_change":"2024-05-11T12:11:56.395623+0000","last_active":"2024-05-11T12:11:57.414852+0000","last_peered":"2024-05-11T12:11:57.414852+0000","last_clean":"2024-05-11T12:11:57.414852+0000","last_became_active":"2024-05-11T12:11:56.395250+0000","last_became_peered":"2024-05-11T12:11:56.395250+0000","last_unstale":"2024-05-11T12:11:57.414852+0000","last_undegraded":"2024-05-11T12:11:57.414852+0000","last_fullsized":"2024-05-11T12:11:57.414852+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:30:56.816516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.379844+0000","last_change":"2024-05-11T12:11:56.379844+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.379844+0000","last_undegraded":"2024-05-11T12:11:56.379844+0000","last_fullsized":"2024-05-11T12:11:56.379844+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-11T12:11:55.388481+0000","last_change":"2024-05-11T12:11:55.388481+0000","last_active":"2024-05-11T12:11:55.345908+0000","last_peered":"2024-05-11T12:11:55.345908+0000","last_clean":"2024-05-11T12:11:55.345908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:55.388481+0000","last_undegraded":"2024-05-11T12:11:55.388481+0000","last_fullsized":"2024-05-11T12:11:55.388481+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.386779+0000","last_change":"2024-05-11T12:11:56.386779+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.386779+0000","last_undegraded":"2024-05-11T12:11:56.386779+0000","last_fullsized":"2024-05-11T12:11:56.386779+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"40'2","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.455023+0000","last_change":"2024-05-11T12:11:56.409581+0000","last_active":"2024-05-11T12:11:57.455023+0000","last_peered":"2024-05-11T12:11:57.455023+0000","last_clean":"2024-05-11T12:11:57.455023+0000","last_became_active":"2024-05-11T12:11:56.408010+0000","last_became_peered":"2024-05-11T12:11:56.408010+0000","last_unstale":"2024-05-11T12:11:57.455023+0000","last_undegraded":"2024-05-11T12:11:57.455023+0000","last_fullsized":"2024-05-11T12:11:57.455023+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:20:13.124057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.451683+0000","last_change":"2024-05-11T12:11:57.447191+0000","last_active":"2024-05-11T12:11:57.451683+0000","last_peered":"2024-05-11T12:11:57.451683+0000","last_clean":"2024-05-11T12:11:57.451683+0000","last_became_active":"2024-05-11T12:11:57.442656+0000","last_became_peered":"2024-05-11T12:11:57.442656+0000","last_unstale":"2024-05-11T12:11:57.451683+0000","last_undegraded":"2024-05-11T12:11:57.451683+0000","last_fullsized":"2024-05-11T12:11:57.451683+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:52:14.248503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-11T12:11:55.388530+0000","last_change":"2024-05-11T12:11:55.388530+0000","last_active":"2024-05-11T12:11:55.345908+0000","last_peered":"2024-05-11T12:11:55.345908+0000","last_clean":"2024-05-11T12:11:55.345908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:55.388530+0000","last_undegraded":"2024-05-11T12:11:55.388530+0000","last_fullsized":"2024-05-11T12:11:55.388530+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.378928+0000","last_change":"2024-05-11T12:11:56.378928+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.378928+0000","last_undegraded":"2024-05-11T12:11:56.378928+0000","last_fullsized":"2024-05-11T12:11:56.378928+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T12:11:56.402024+0000","last_change":"2024-05-11T12:11:56.397553+0000","last_active":"2024-05-11T12:11:56.402024+0000","last_peered":"2024-05-11T12:11:56.402024+0000","last_clean":"2024-05-11T12:11:56.402024+0000","last_became_active":"2024-05-11T12:11:56.397219+0000","last_became_peered":"2024-05-11T12:11:56.397219+0000","last_unstale":"2024-05-11T12:11:56.402024+0000","last_undegraded":"2024-05-11T12:11:56.402024+0000","last_fullsized":"2024-05-11T12:11:56.402024+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:25:15.336149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.383035+0000","last_change":"2024-05-11T12:11:56.383035+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.383035+0000","last_undegraded":"2024-05-11T12:11:56.383035+0000","last_fullsized":"2024-05-11T12:11:56.383035+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T12:11:56.407895+0000","last_change":"2024-05-11T12:11:56.404006+0000","last_active":"2024-05-11T12:11:56.407895+0000","last_peered":"2024-05-11T12:11:56.407895+0000","last_clean":"2024-05-11T12:11:56.407895+0000","last_became_active":"2024-05-11T12:11:56.403672+0000","last_became_peered":"2024-05-11T12:11:56.403672+0000","last_unstale":"2024-05-11T12:11:56.407895+0000","last_undegraded":"2024-05-11T12:11:56.407895+0000","last_fullsized":"2024-05-11T12:11:56.407895+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:30:56.816516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.388205+0000","last_change":"2024-05-11T12:11:56.388205+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.388205+0000","last_undegraded":"2024-05-11T12:11:56.388205+0000","last_fullsized":"2024-05-11T12:11:56.388205+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.401153+0000","last_change":"2024-05-11T12:11:56.399122+0000","last_active":"2024-05-11T12:11:57.401153+0000","last_peered":"2024-05-11T12:11:57.401153+0000","last_clean":"2024-05-11T12:11:57.401153+0000","last_became_active":"2024-05-11T12:11:56.398845+0000","last_became_peered":"2024-05-11T12:11:56.398845+0000","last_unstale":"2024-05-11T12:11:57.401153+0000","last_undegraded":"2024-05-11T12:11:57.401153+0000","last_fullsized":"2024-05-11T12:11:57.401153+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:16:00.825967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.449098+0000","last_change":"2024-05-11T12:11:57.442240+0000","last_active":"2024-05-11T12:11:57.449098+0000","last_peered":"2024-05-11T12:11:57.449098+0000","last_clean":"2024-05-11T12:11:57.449098+0000","last_became_active":"2024-05-11T12:11:57.440831+0000","last_became_peered":"2024-05-11T12:11:57.440831+0000","last_unstale":"2024-05-11T12:11:57.449098+0000","last_undegraded":"2024-05-11T12:11:57.449098+0000","last_fullsized":"2024-05-11T12:11:57.449098+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:28:44.586634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.418097+0000","last_change":"2024-05-11T12:11:56.402880+0000","last_active":"2024-05-11T12:11:57.418097+0000","last_peered":"2024-05-11T12:11:57.418097+0000","last_clean":"2024-05-11T12:11:57.418097+0000","last_became_active":"2024-05-11T12:11:56.402608+0000","last_became_peered":"2024-05-11T12:11:56.402608+0000","last_unstale":"2024-05-11T12:11:57.418097+0000","last_undegraded":"2024-05-11T12:11:57.418097+0000","last_fullsized":"2024-05-11T12:11:57.418097+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:34:29.698881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.451131+0000","last_change":"2024-05-11T12:11:57.444173+0000","last_active":"2024-05-11T12:11:57.451131+0000","last_peered":"2024-05-11T12:11:57.451131+0000","last_clean":"2024-05-11T12:11:57.451131+0000","last_became_active":"2024-05-11T12:11:57.441809+0000","last_became_peered":"2024-05-11T12:11:57.441809+0000","last_unstale":"2024-05-11T12:11:57.451131+0000","last_undegraded":"2024-05-11T12:11:57.451131+0000","last_fullsized":"2024-05-11T12:11:57.451131+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:48:54.195365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T12:11:56.405633+0000","last_change":"2024-05-11T12:11:56.399468+0000","last_active":"2024-05-11T12:11:56.405633+0000","last_peered":"2024-05-11T12:11:56.405633+0000","last_clean":"2024-05-11T12:11:56.405633+0000","last_became_active":"2024-05-11T12:11:56.399205+0000","last_became_peered":"2024-05-11T12:11:56.399205+0000","last_unstale":"2024-05-11T12:11:56.405633+0000","last_undegraded":"2024-05-11T12:11:56.405633+0000","last_fullsized":"2024-05-11T12:11:56.405633+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:10:04.020340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.430065+0000","last_change":"2024-05-11T12:11:57.417010+0000","last_active":"2024-05-11T12:11:57.430065+0000","last_peered":"2024-05-11T12:11:57.430065+0000","last_clean":"2024-05-11T12:11:57.430065+0000","last_became_active":"2024-05-11T12:11:57.416499+0000","last_became_peered":"2024-05-11T12:11:57.416499+0000","last_unstale":"2024-05-11T12:11:57.430065+0000","last_undegraded":"2024-05-11T12:11:57.430065+0000","last_fullsized":"2024-05-11T12:11:57.430065+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:53:03.480916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.418829+0000","last_change":"2024-05-11T12:11:56.400810+0000","last_active":"2024-05-11T12:11:57.418829+0000","last_peered":"2024-05-11T12:11:57.418829+0000","last_clean":"2024-05-11T12:11:57.418829+0000","last_became_active":"2024-05-11T12:11:56.399824+0000","last_became_peered":"2024-05-11T12:11:56.399824+0000","last_unstale":"2024-05-11T12:11:57.418829+0000","last_undegraded":"2024-05-11T12:11:57.418829+0000","last_fullsized":"2024-05-11T12:11:57.418829+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:07:21.760158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":139,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.414520+0000","last_change":"2024-05-11T12:10:35.066673+0000","last_active":"2024-05-11T12:11:57.414520+0000","last_peered":"2024-05-11T12:11:57.414520+0000","last_clean":"2024-05-11T12:11:57.414520+0000","last_became_active":"2024-05-11T12:10:35.065942+0000","last_became_peered":"2024-05-11T12:10:35.065942+0000","last_unstale":"2024-05-11T12:11:57.414520+0000","last_undegraded":"2024-05-11T12:11:57.414520+0000","last_fullsized":"2024-05-11T12:11:57.414520+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_clean_scrub_stamp":"2024-05-11T12:10:33.601136+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:12:02.133182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":116,"acting":116,"num_store_stats":7},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":694,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24,"ondisk_log_size":24,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476756,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28064,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343776,"statfs":{"total":96636764160,"available":96608026624,"internally_reserved":0,"allocated":1060864,"data_stored":784295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476757,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":192827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476756,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":192357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476758,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":192849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509461,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":438272,"data_stored":185078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509462,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":438272,"data_stored":185078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509461,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343808,"statfs":{"total":96636764160,"available":96608059392,"internally_reserved":0,"allocated":1032192,"data_stored":775446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542165,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":178411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T12:12:02.370 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T12:12:02.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.612+0000 7f480f200640 1 Processor -- start 2024-05-11T12:12:02.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.612+0000 7f480f200640 1 -- start start 2024-05-11T12:12:02.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.613+0000 7f480f200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4810145b00 0x7f481006f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:02.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.614+0000 7f480f200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f481006c660 0x7f481006ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:02.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.614+0000 7f480f200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f481006fbc0 con 0x7f4810146770 2024-05-11T12:12:02.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.614+0000 7f480f200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f481006cfd0 con 0x7f481006c660 2024-05-11T12:12:02.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.614+0000 7f480f200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f48100b4520 con 0x7f4810145b00 2024-05-11T12:12:02.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.614+0000 7f480de00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4810145b00 0x7f481006f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:02.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.614+0000 7f480e800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f481006c660 0x7f481006ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:02.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.614+0000 7f480de00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4810145b00 0x7f481006f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:59722/0 (socket says 172.21.15.167:59722) 2024-05-11T12:12:02.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.614+0000 7f480de00640 1 -- 172.21.15.167:0/1164047499 learned_addr learned my addr 172.21.15.167:0/1164047499 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:02.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.615+0000 7f480de00640 1 -- 172.21.15.167:0/1164047499 >> v2:172.21.15.204:3300/0 conn(0x7f481006c660 msgr2=0x7f481006ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:02.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.615+0000 7f480de00640 1 --2- 172.21.15.167:0/1164047499 >> v2:172.21.15.204:3300/0 conn(0x7f481006c660 0x7f481006ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:02.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.615+0000 7f480de00640 1 -- 172.21.15.167:0/1164047499 >> v1:172.21.15.204:6789/0 conn(0x7f4810146770 legacy=0x7f48100ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:02.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.615+0000 7f480de00640 1 -- 172.21.15.167:0/1164047499 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48100b4800 con 0x7f4810145b00 2024-05-11T12:12:02.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.615+0000 7f480ca00640 1 -- 172.21.15.167:0/1164047499 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2478358264 0 0) 0x7f481006fbc0 con 0x7f4810146770 2024-05-11T12:12:02.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.615+0000 7f480e800640 1 --2- 172.21.15.167:0/1164047499 >> v2:172.21.15.204:3300/0 conn(0x7f481006c660 0x7f481006ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:12:02.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.617+0000 7f480de00640 1 --2- 172.21.15.167:0/1164047499 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4810145b00 0x7f481006f2c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4800009870 tx=0x7f480002a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=675ea516cfd7ee47 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:02.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.618+0000 7f480ca00640 1 -- 172.21.15.167:0/1164047499 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4800037070 con 0x7f4810145b00 2024-05-11T12:12:02.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.618+0000 7f480ca00640 1 -- 172.21.15.167:0/1164047499 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4800032440 con 0x7f4810145b00 2024-05-11T12:12:02.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.618+0000 7f480ca00640 1 -- 172.21.15.167:0/1164047499 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4800032810 con 0x7f4810145b00 2024-05-11T12:12:02.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.618+0000 7f480f200640 1 -- 172.21.15.167:0/1164047499 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4810145b00 msgr2=0x7f481006f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:02.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.618+0000 7f480f200640 1 --2- 172.21.15.167:0/1164047499 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4810145b00 0x7f481006f2c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4800009870 tx=0x7f480002a1c0 comp rx=0 tx=0).stop 2024-05-11T12:12:02.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.618+0000 7f480f200640 1 -- 172.21.15.167:0/1164047499 shutdown_connections 2024-05-11T12:12:02.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.618+0000 7f480f200640 1 --2- 172.21.15.167:0/1164047499 >> v2:172.21.15.204:3300/0 conn(0x7f481006c660 0x7f481006ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:02.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.618+0000 7f480f200640 1 --2- 172.21.15.167:0/1164047499 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4810145b00 0x7f481006f2c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:02.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.618+0000 7f480f200640 1 -- 172.21.15.167:0/1164047499 >> 172.21.15.167:0/1164047499 conn(0x7f48100b3460 msgr2=0x7f48100d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:02.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.619+0000 7f480f200640 1 -- 172.21.15.167:0/1164047499 shutdown_connections 2024-05-11T12:12:02.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.619+0000 7f480f200640 1 -- 172.21.15.167:0/1164047499 wait complete. 2024-05-11T12:12:02.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.620+0000 7f480f200640 1 Processor -- start 2024-05-11T12:12:02.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.620+0000 7f480f200640 1 -- start start 2024-05-11T12:12:02.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.621+0000 7f480f200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f481006c660 0x7f481013f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:02.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.621+0000 7f480f200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4810145b00 0x7f48101f4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:02.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.621+0000 7f480f200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4810146770 0x7f4810160900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:02.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.621+0000 7f480f200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4810145ee0 con 0x7f4810145b00 2024-05-11T12:12:02.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.621+0000 7f480f200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f48100cf0e0 con 0x7f481006c660 2024-05-11T12:12:02.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.621+0000 7f480f200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f481006f800 con 0x7f4810146770 2024-05-11T12:12:02.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.622+0000 7f480d400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4810145b00 0x7f48101f4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:02.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.622+0000 7f480d400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4810145b00 0x7f48101f4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41880/0 (socket says 172.21.15.167:41880) 2024-05-11T12:12:02.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.622+0000 7f480d400640 1 -- 172.21.15.167:0/521796358 learned_addr learned my addr 172.21.15.167:0/521796358 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:02.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.622+0000 7f480e800640 1 --2- 172.21.15.167:0/521796358 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4810146770 0x7f4810160900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:02.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.622+0000 7f480de00640 1 --2- 172.21.15.167:0/521796358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f481006c660 0x7f481013f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:02.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.622+0000 7f480d400640 1 -- 172.21.15.167:0/521796358 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4810146770 msgr2=0x7f4810160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:02.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.622+0000 7f480d400640 1 --2- 172.21.15.167:0/521796358 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4810146770 0x7f4810160900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:02.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.622+0000 7f480d400640 1 -- 172.21.15.167:0/521796358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f481006c660 msgr2=0x7f481013f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:02.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.622+0000 7f480d400640 1 --2- 172.21.15.167:0/521796358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f481006c660 0x7f481013f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:02.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.622+0000 7f480d400640 1 -- 172.21.15.167:0/521796358 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f481013d000 con 0x7f4810145b00 2024-05-11T12:12:02.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.622+0000 7f480d400640 1 --2- 172.21.15.167:0/521796358 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4810145b00 0x7f48101f4200 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f47f800c500 tx=0x7f47f800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:02.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.623+0000 7f4806000640 1 -- 172.21.15.167:0/521796358 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f8018070 con 0x7f4810145b00 2024-05-11T12:12:02.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.623+0000 7f4806000640 1 -- 172.21.15.167:0/521796358 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47f80035f0 con 0x7f4810145b00 2024-05-11T12:12:02.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.623+0000 7f4806000640 1 -- 172.21.15.167:0/521796358 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f8003a60 con 0x7f4810145b00 2024-05-11T12:12:02.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.623+0000 7f480f200640 1 -- 172.21.15.167:0/521796358 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f481013fbb0 con 0x7f4810145b00 2024-05-11T12:12:02.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.623+0000 7f480f200640 1 -- 172.21.15.167:0/521796358 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48101f8140 con 0x7f4810145b00 2024-05-11T12:12:02.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.624+0000 7f480f200640 1 -- 172.21.15.167:0/521796358 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48100694b0 con 0x7f4810145b00 2024-05-11T12:12:02.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.625+0000 7f4806000640 1 -- 172.21.15.167:0/521796358 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f47f8003c00 con 0x7f4810145b00 2024-05-11T12:12:02.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.627+0000 7f4806000640 1 --2- 172.21.15.167:0/521796358 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f47ec0762d0 0x7f47ec078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:02.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.627+0000 7f4806000640 1 -- 172.21.15.167:0/521796358 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f47f8096b40 con 0x7f4810145b00 2024-05-11T12:12:02.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.627+0000 7f480de00640 1 --2- 172.21.15.167:0/521796358 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f47ec0762d0 0x7f47ec078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:02.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.628+0000 7f480de00640 1 --2- 172.21.15.167:0/521796358 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f47ec0762d0 0x7f47ec078790 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4800034000 tx=0x7f480002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:02.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.631+0000 7f4806000640 1 -- 172.21.15.167:0/521796358 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f47f8060280 con 0x7f4810145b00 2024-05-11T12:12:02.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.838+0000 7f480f200640 1 -- 172.21.15.167:0/521796358 --> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f48100b3bd0 con 0x7f47ec0762d0 2024-05-11T12:12:02.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.857+0000 7f4806000640 1 -- 172.21.15.167:0/521796358 <== mgr.4106 v2:172.21.15.167:6832/578352559 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248595 (secure 0 0 0) 0x7f48100b3bd0 con 0x7f47ec0762d0 2024-05-11T12:12:02.858 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:12:02.862 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2024-05-11T12:12:02.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.864+0000 7f480f200640 1 -- 172.21.15.167:0/521796358 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f47ec0762d0 msgr2=0x7f47ec078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:02.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.864+0000 7f480f200640 1 --2- 172.21.15.167:0/521796358 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f47ec0762d0 0x7f47ec078790 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4800034000 tx=0x7f480002e000 comp rx=0 tx=0).stop 2024-05-11T12:12:02.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.864+0000 7f480f200640 1 -- 172.21.15.167:0/521796358 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4810145b00 msgr2=0x7f48101f4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:02.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.864+0000 7f480f200640 1 --2- 172.21.15.167:0/521796358 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4810145b00 0x7f48101f4200 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f47f800c500 tx=0x7f47f800c9c0 comp rx=0 tx=0).stop 2024-05-11T12:12:02.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.864+0000 7f480f200640 1 -- 172.21.15.167:0/521796358 shutdown_connections 2024-05-11T12:12:02.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.864+0000 7f480f200640 1 --2- 172.21.15.167:0/521796358 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4810146770 0x7f4810160900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:02.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.864+0000 7f480f200640 1 --2- 172.21.15.167:0/521796358 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4810145b00 0x7f48101f4200 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:02.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.864+0000 7f480f200640 1 --2- 172.21.15.167:0/521796358 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f47ec0762d0 0x7f47ec078790 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:02.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.864+0000 7f480f200640 1 --2- 172.21.15.167:0/521796358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f481006c660 0x7f481013f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:02.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.864+0000 7f480f200640 1 -- 172.21.15.167:0/521796358 >> 172.21.15.167:0/521796358 conn(0x7f48100b3460 msgr2=0x7f48100ce9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:02.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.864+0000 7f480f200640 1 -- 172.21.15.167:0/521796358 shutdown_connections 2024-05-11T12:12:02.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:02.864+0000 7f480f200640 1 -- 172.21.15.167:0/521796358 wait complete. 2024-05-11T12:12:02.884 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":79,"stamp":"2024-05-11T12:12:01.258163+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591062,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":131,"num_write_kb":1631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":96,"ondisk_log_size":96,"up":182,"acting":182,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":100,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220844,"kb_used_data":4656,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753876,"statfs":{"total":773094113280,"available":772867969024,"internally_reserved":0,"allocated":4767744,"data_stored":2686341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":694,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.895997"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.383260+0000","last_change":"2024-05-11T12:11:56.383260+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.383260+0000","last_undegraded":"2024-05-11T12:11:56.383260+0000","last_fullsized":"2024-05-11T12:11:56.383260+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.381407+0000","last_change":"2024-05-11T12:11:56.381407+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.381407+0000","last_undegraded":"2024-05-11T12:11:56.381407+0000","last_fullsized":"2024-05-11T12:11:56.381407+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.388722+0000","last_change":"2024-05-11T12:11:56.388722+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.388722+0000","last_undegraded":"2024-05-11T12:11:56.388722+0000","last_fullsized":"2024-05-11T12:11:56.388722+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T12:11:56.360286+0000","last_change":"2024-05-11T12:11:56.360286+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.360286+0000","last_undegraded":"2024-05-11T12:11:56.360286+0000","last_fullsized":"2024-05-11T12:11:56.360286+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T12:11:56.360286+0000","last_change":"2024-05-11T12:11:56.360286+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.360286+0000","last_undegraded":"2024-05-11T12:11:56.360286+0000","last_fullsized":"2024-05-11T12:11:56.360286+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.386336+0000","last_change":"2024-05-11T12:11:56.386336+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.386336+0000","last_undegraded":"2024-05-11T12:11:56.386336+0000","last_fullsized":"2024-05-11T12:11:56.386336+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.437906+0000","last_change":"2024-05-11T12:11:57.432877+0000","last_active":"2024-05-11T12:11:57.437906+0000","last_peered":"2024-05-11T12:11:57.437906+0000","last_clean":"2024-05-11T12:11:57.437906+0000","last_became_active":"2024-05-11T12:11:57.432581+0000","last_became_peered":"2024-05-11T12:11:57.432581+0000","last_unstale":"2024-05-11T12:11:57.437906+0000","last_undegraded":"2024-05-11T12:11:57.437906+0000","last_fullsized":"2024-05-11T12:11:57.437906+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:47:49.623473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.381742+0000","last_change":"2024-05-11T12:11:56.381742+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.381742+0000","last_undegraded":"2024-05-11T12:11:56.381742+0000","last_fullsized":"2024-05-11T12:11:56.381742+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.384226+0000","last_change":"2024-05-11T12:11:56.384226+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.384226+0000","last_undegraded":"2024-05-11T12:11:56.384226+0000","last_fullsized":"2024-05-11T12:11:56.384226+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.439811+0000","last_change":"2024-05-11T12:11:57.434958+0000","last_active":"2024-05-11T12:11:57.439811+0000","last_peered":"2024-05-11T12:11:57.439811+0000","last_clean":"2024-05-11T12:11:57.439811+0000","last_became_active":"2024-05-11T12:11:57.429221+0000","last_became_peered":"2024-05-11T12:11:57.429221+0000","last_unstale":"2024-05-11T12:11:57.439811+0000","last_undegraded":"2024-05-11T12:11:57.439811+0000","last_fullsized":"2024-05-11T12:11:57.439811+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:32:26.804013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.455023+0000","last_change":"2024-05-11T12:11:57.449552+0000","last_active":"2024-05-11T12:11:57.455023+0000","last_peered":"2024-05-11T12:11:57.455023+0000","last_clean":"2024-05-11T12:11:57.455023+0000","last_became_active":"2024-05-11T12:11:57.441862+0000","last_became_peered":"2024-05-11T12:11:57.441862+0000","last_unstale":"2024-05-11T12:11:57.455023+0000","last_undegraded":"2024-05-11T12:11:57.455023+0000","last_fullsized":"2024-05-11T12:11:57.455023+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:04:26.168833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.386630+0000","last_change":"2024-05-11T12:11:56.386630+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.386630+0000","last_undegraded":"2024-05-11T12:11:56.386630+0000","last_fullsized":"2024-05-11T12:11:56.386630+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.432544+0000","last_change":"2024-05-11T12:11:57.429706+0000","last_active":"2024-05-11T12:11:57.432544+0000","last_peered":"2024-05-11T12:11:57.432544+0000","last_clean":"2024-05-11T12:11:57.432544+0000","last_became_active":"2024-05-11T12:11:57.429387+0000","last_became_peered":"2024-05-11T12:11:57.429387+0000","last_unstale":"2024-05-11T12:11:57.432544+0000","last_undegraded":"2024-05-11T12:11:57.432544+0000","last_fullsized":"2024-05-11T12:11:57.432544+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:55:35.468623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.445013+0000","last_change":"2024-05-11T12:11:57.430917+0000","last_active":"2024-05-11T12:11:57.445013+0000","last_peered":"2024-05-11T12:11:57.445013+0000","last_clean":"2024-05-11T12:11:57.445013+0000","last_became_active":"2024-05-11T12:11:57.430613+0000","last_became_peered":"2024-05-11T12:11:57.430613+0000","last_unstale":"2024-05-11T12:11:57.445013+0000","last_undegraded":"2024-05-11T12:11:57.445013+0000","last_fullsized":"2024-05-11T12:11:57.445013+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:25:16.350527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.454620+0000","last_change":"2024-05-11T12:11:57.438016+0000","last_active":"2024-05-11T12:11:57.454620+0000","last_peered":"2024-05-11T12:11:57.454620+0000","last_clean":"2024-05-11T12:11:57.454620+0000","last_became_active":"2024-05-11T12:11:57.437680+0000","last_became_peered":"2024-05-11T12:11:57.437680+0000","last_unstale":"2024-05-11T12:11:57.454620+0000","last_undegraded":"2024-05-11T12:11:57.454620+0000","last_fullsized":"2024-05-11T12:11:57.454620+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:28:33.430453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.453506+0000","last_change":"2024-05-11T12:11:57.448672+0000","last_active":"2024-05-11T12:11:57.453506+0000","last_peered":"2024-05-11T12:11:57.453506+0000","last_clean":"2024-05-11T12:11:57.453506+0000","last_became_active":"2024-05-11T12:11:57.447489+0000","last_became_peered":"2024-05-11T12:11:57.447489+0000","last_unstale":"2024-05-11T12:11:57.453506+0000","last_undegraded":"2024-05-11T12:11:57.453506+0000","last_fullsized":"2024-05-11T12:11:57.453506+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:53:03.480916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.451781+0000","last_change":"2024-05-11T12:11:57.447831+0000","last_active":"2024-05-11T12:11:57.451781+0000","last_peered":"2024-05-11T12:11:57.451781+0000","last_clean":"2024-05-11T12:11:57.451781+0000","last_became_active":"2024-05-11T12:11:57.443540+0000","last_became_peered":"2024-05-11T12:11:57.443540+0000","last_unstale":"2024-05-11T12:11:57.451781+0000","last_undegraded":"2024-05-11T12:11:57.451781+0000","last_fullsized":"2024-05-11T12:11:57.451781+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:26:34.393703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.435968+0000","last_change":"2024-05-11T12:11:57.431125+0000","last_active":"2024-05-11T12:11:57.435968+0000","last_peered":"2024-05-11T12:11:57.435968+0000","last_clean":"2024-05-11T12:11:57.435968+0000","last_became_active":"2024-05-11T12:11:57.430835+0000","last_became_peered":"2024-05-11T12:11:57.430835+0000","last_unstale":"2024-05-11T12:11:57.435968+0000","last_undegraded":"2024-05-11T12:11:57.435968+0000","last_fullsized":"2024-05-11T12:11:57.435968+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:23:28.626421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T12:11:56.360286+0000","last_change":"2024-05-11T12:11:56.360286+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.360286+0000","last_undegraded":"2024-05-11T12:11:56.360286+0000","last_fullsized":"2024-05-11T12:11:56.360286+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.436449+0000","last_change":"2024-05-11T12:11:57.433084+0000","last_active":"2024-05-11T12:11:57.436449+0000","last_peered":"2024-05-11T12:11:57.436449+0000","last_clean":"2024-05-11T12:11:57.436449+0000","last_became_active":"2024-05-11T12:11:57.432645+0000","last_became_peered":"2024-05-11T12:11:57.432645+0000","last_unstale":"2024-05-11T12:11:57.436449+0000","last_undegraded":"2024-05-11T12:11:57.436449+0000","last_fullsized":"2024-05-11T12:11:57.436449+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:32:45.644260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.439852+0000","last_change":"2024-05-11T12:11:57.434801+0000","last_active":"2024-05-11T12:11:57.439852+0000","last_peered":"2024-05-11T12:11:57.439852+0000","last_clean":"2024-05-11T12:11:57.439852+0000","last_became_active":"2024-05-11T12:11:57.433969+0000","last_became_peered":"2024-05-11T12:11:57.433969+0000","last_unstale":"2024-05-11T12:11:57.439852+0000","last_undegraded":"2024-05-11T12:11:57.439852+0000","last_fullsized":"2024-05-11T12:11:57.439852+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:22:33.211172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.449144+0000","last_change":"2024-05-11T12:11:57.445771+0000","last_active":"2024-05-11T12:11:57.449144+0000","last_peered":"2024-05-11T12:11:57.449144+0000","last_clean":"2024-05-11T12:11:57.449144+0000","last_became_active":"2024-05-11T12:11:57.445474+0000","last_became_peered":"2024-05-11T12:11:57.445474+0000","last_unstale":"2024-05-11T12:11:57.449144+0000","last_undegraded":"2024-05-11T12:11:57.449144+0000","last_fullsized":"2024-05-11T12:11:57.449144+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:23:28.626421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.451922+0000","last_change":"2024-05-11T12:11:57.447508+0000","last_active":"2024-05-11T12:11:57.451922+0000","last_peered":"2024-05-11T12:11:57.451922+0000","last_clean":"2024-05-11T12:11:57.451922+0000","last_became_active":"2024-05-11T12:11:57.444377+0000","last_became_peered":"2024-05-11T12:11:57.444377+0000","last_unstale":"2024-05-11T12:11:57.451922+0000","last_undegraded":"2024-05-11T12:11:57.451922+0000","last_fullsized":"2024-05-11T12:11:57.451922+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:46:06.174919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T12:11:56.360286+0000","last_change":"2024-05-11T12:11:56.360286+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.360286+0000","last_undegraded":"2024-05-11T12:11:56.360286+0000","last_fullsized":"2024-05-11T12:11:56.360286+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.430674+0000","last_change":"2024-05-11T12:11:57.426462+0000","last_active":"2024-05-11T12:11:57.430674+0000","last_peered":"2024-05-11T12:11:57.430674+0000","last_clean":"2024-05-11T12:11:57.430674+0000","last_became_active":"2024-05-11T12:11:57.426087+0000","last_became_peered":"2024-05-11T12:11:57.426087+0000","last_unstale":"2024-05-11T12:11:57.430674+0000","last_undegraded":"2024-05-11T12:11:57.430674+0000","last_fullsized":"2024-05-11T12:11:57.430674+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:30:57.830893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.390804+0000","last_change":"2024-05-11T12:11:56.390804+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.390804+0000","last_undegraded":"2024-05-11T12:11:56.390804+0000","last_fullsized":"2024-05-11T12:11:56.390804+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.432318+0000","last_change":"2024-05-11T12:11:57.428594+0000","last_active":"2024-05-11T12:11:57.432318+0000","last_peered":"2024-05-11T12:11:57.432318+0000","last_clean":"2024-05-11T12:11:57.432318+0000","last_became_active":"2024-05-11T12:11:57.428245+0000","last_became_peered":"2024-05-11T12:11:57.428245+0000","last_unstale":"2024-05-11T12:11:57.432318+0000","last_undegraded":"2024-05-11T12:11:57.432318+0000","last_fullsized":"2024-05-11T12:11:57.432318+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:28:44.586634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.430392+0000","last_change":"2024-05-11T12:11:57.417635+0000","last_active":"2024-05-11T12:11:57.430392+0000","last_peered":"2024-05-11T12:11:57.430392+0000","last_clean":"2024-05-11T12:11:57.430392+0000","last_became_active":"2024-05-11T12:11:57.415536+0000","last_became_peered":"2024-05-11T12:11:57.415536+0000","last_unstale":"2024-05-11T12:11:57.430392+0000","last_undegraded":"2024-05-11T12:11:57.430392+0000","last_fullsized":"2024-05-11T12:11:57.430392+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:51:43.649698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.432559+0000","last_change":"2024-05-11T12:11:57.427847+0000","last_active":"2024-05-11T12:11:57.432559+0000","last_peered":"2024-05-11T12:11:57.432559+0000","last_clean":"2024-05-11T12:11:57.432559+0000","last_became_active":"2024-05-11T12:11:57.427416+0000","last_became_peered":"2024-05-11T12:11:57.427416+0000","last_unstale":"2024-05-11T12:11:57.432559+0000","last_undegraded":"2024-05-11T12:11:57.432559+0000","last_fullsized":"2024-05-11T12:11:57.432559+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:07:22.774535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.423604+0000","last_change":"2024-05-11T12:11:57.420465+0000","last_active":"2024-05-11T12:11:57.423604+0000","last_peered":"2024-05-11T12:11:57.423604+0000","last_clean":"2024-05-11T12:11:57.423604+0000","last_became_active":"2024-05-11T12:11:57.419963+0000","last_became_peered":"2024-05-11T12:11:57.419963+0000","last_unstale":"2024-05-11T12:11:57.423604+0000","last_undegraded":"2024-05-11T12:11:57.423604+0000","last_fullsized":"2024-05-11T12:11:57.423604+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:36:41.799582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.451900+0000","last_change":"2024-05-11T12:11:57.447209+0000","last_active":"2024-05-11T12:11:57.451900+0000","last_peered":"2024-05-11T12:11:57.451900+0000","last_clean":"2024-05-11T12:11:57.451900+0000","last_became_active":"2024-05-11T12:11:57.443801+0000","last_became_peered":"2024-05-11T12:11:57.443801+0000","last_unstale":"2024-05-11T12:11:57.451900+0000","last_undegraded":"2024-05-11T12:11:57.451900+0000","last_fullsized":"2024-05-11T12:11:57.451900+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:10:05.034718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.389994+0000","last_change":"2024-05-11T12:11:56.389994+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.389994+0000","last_undegraded":"2024-05-11T12:11:56.389994+0000","last_fullsized":"2024-05-11T12:11:56.389994+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.451196+0000","last_change":"2024-05-11T12:11:57.444554+0000","last_active":"2024-05-11T12:11:57.451196+0000","last_peered":"2024-05-11T12:11:57.451196+0000","last_clean":"2024-05-11T12:11:57.451196+0000","last_became_active":"2024-05-11T12:11:57.443836+0000","last_became_peered":"2024-05-11T12:11:57.443836+0000","last_unstale":"2024-05-11T12:11:57.451196+0000","last_undegraded":"2024-05-11T12:11:57.451196+0000","last_fullsized":"2024-05-11T12:11:57.451196+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:20:14.138435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"40'4","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.451553+0000","last_change":"2024-05-11T12:11:56.409094+0000","last_active":"2024-05-11T12:11:57.451553+0000","last_peered":"2024-05-11T12:11:57.451553+0000","last_clean":"2024-05-11T12:11:57.451553+0000","last_became_active":"2024-05-11T12:11:56.408820+0000","last_became_peered":"2024-05-11T12:11:56.408820+0000","last_unstale":"2024-05-11T12:11:57.451553+0000","last_undegraded":"2024-05-11T12:11:57.451553+0000","last_fullsized":"2024-05-11T12:11:57.451553+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:11:42.632883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.389689+0000","last_change":"2024-05-11T12:11:56.389689+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.389689+0000","last_undegraded":"2024-05-11T12:11:56.389689+0000","last_fullsized":"2024-05-11T12:11:56.389689+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T12:11:56.407872+0000","last_change":"2024-05-11T12:11:56.403002+0000","last_active":"2024-05-11T12:11:56.407872+0000","last_peered":"2024-05-11T12:11:56.407872+0000","last_clean":"2024-05-11T12:11:56.407872+0000","last_became_active":"2024-05-11T12:11:56.402721+0000","last_became_peered":"2024-05-11T12:11:56.402721+0000","last_unstale":"2024-05-11T12:11:56.407872+0000","last_undegraded":"2024-05-11T12:11:56.407872+0000","last_fullsized":"2024-05-11T12:11:56.407872+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:36:40.785205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.450328+0000","last_change":"2024-05-11T12:11:57.435281+0000","last_active":"2024-05-11T12:11:57.450328+0000","last_peered":"2024-05-11T12:11:57.450328+0000","last_clean":"2024-05-11T12:11:57.450328+0000","last_became_active":"2024-05-11T12:11:57.434672+0000","last_became_peered":"2024-05-11T12:11:57.434672+0000","last_unstale":"2024-05-11T12:11:57.450328+0000","last_undegraded":"2024-05-11T12:11:57.450328+0000","last_fullsized":"2024-05-11T12:11:57.450328+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:30:57.830893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T12:11:56.417027+0000","last_change":"2024-05-11T12:11:56.410313+0000","last_active":"2024-05-11T12:11:56.417027+0000","last_peered":"2024-05-11T12:11:56.417027+0000","last_clean":"2024-05-11T12:11:56.417027+0000","last_became_active":"2024-05-11T12:11:56.409971+0000","last_became_peered":"2024-05-11T12:11:56.409971+0000","last_unstale":"2024-05-11T12:11:56.417027+0000","last_undegraded":"2024-05-11T12:11:56.417027+0000","last_fullsized":"2024-05-11T12:11:56.417027+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:11:42.632883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.382086+0000","last_change":"2024-05-11T12:11:56.382086+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.382086+0000","last_undegraded":"2024-05-11T12:11:56.382086+0000","last_fullsized":"2024-05-11T12:11:56.382086+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"40'2","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.454119+0000","last_change":"2024-05-11T12:11:56.408353+0000","last_active":"2024-05-11T12:11:57.454119+0000","last_peered":"2024-05-11T12:11:57.454119+0000","last_clean":"2024-05-11T12:11:57.454119+0000","last_became_active":"2024-05-11T12:11:56.408078+0000","last_became_peered":"2024-05-11T12:11:56.408078+0000","last_unstale":"2024-05-11T12:11:57.454119+0000","last_undegraded":"2024-05-11T12:11:57.454119+0000","last_fullsized":"2024-05-11T12:11:57.454119+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:16:00.825967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.384699+0000","last_change":"2024-05-11T12:11:56.384699+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.384699+0000","last_undegraded":"2024-05-11T12:11:56.384699+0000","last_fullsized":"2024-05-11T12:11:56.384699+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"40'1","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.455051+0000","last_change":"2024-05-11T12:11:56.404989+0000","last_active":"2024-05-11T12:11:57.455051+0000","last_peered":"2024-05-11T12:11:57.455051+0000","last_clean":"2024-05-11T12:11:57.455051+0000","last_became_active":"2024-05-11T12:11:56.404069+0000","last_became_peered":"2024-05-11T12:11:56.404069+0000","last_unstale":"2024-05-11T12:11:57.455051+0000","last_undegraded":"2024-05-11T12:11:57.455051+0000","last_fullsized":"2024-05-11T12:11:57.455051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:26:33.379326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.454982+0000","last_change":"2024-05-11T12:11:57.442633+0000","last_active":"2024-05-11T12:11:57.454982+0000","last_peered":"2024-05-11T12:11:57.454982+0000","last_clean":"2024-05-11T12:11:57.454982+0000","last_became_active":"2024-05-11T12:11:57.442361+0000","last_became_peered":"2024-05-11T12:11:57.442361+0000","last_unstale":"2024-05-11T12:11:57.454982+0000","last_undegraded":"2024-05-11T12:11:57.454982+0000","last_fullsized":"2024-05-11T12:11:57.454982+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:43:20.345625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"40'1","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.451754+0000","last_change":"2024-05-11T12:11:56.411689+0000","last_active":"2024-05-11T12:11:57.451754+0000","last_peered":"2024-05-11T12:11:57.451754+0000","last_clean":"2024-05-11T12:11:57.451754+0000","last_became_active":"2024-05-11T12:11:56.409937+0000","last_became_peered":"2024-05-11T12:11:56.409937+0000","last_unstale":"2024-05-11T12:11:57.451754+0000","last_undegraded":"2024-05-11T12:11:57.451754+0000","last_fullsized":"2024-05-11T12:11:57.451754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:36:40.785205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.448085+0000","last_change":"2024-05-11T12:11:57.435197+0000","last_active":"2024-05-11T12:11:57.448085+0000","last_peered":"2024-05-11T12:11:57.448085+0000","last_clean":"2024-05-11T12:11:57.448085+0000","last_became_active":"2024-05-11T12:11:57.434472+0000","last_became_peered":"2024-05-11T12:11:57.434472+0000","last_unstale":"2024-05-11T12:11:57.448085+0000","last_undegraded":"2024-05-11T12:11:57.448085+0000","last_fullsized":"2024-05-11T12:11:57.448085+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:36:41.799582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T12:11:56.416996+0000","last_change":"2024-05-11T12:11:56.409731+0000","last_active":"2024-05-11T12:11:56.416996+0000","last_peered":"2024-05-11T12:11:56.416996+0000","last_clean":"2024-05-11T12:11:56.416996+0000","last_became_active":"2024-05-11T12:11:56.409450+0000","last_became_peered":"2024-05-11T12:11:56.409450+0000","last_unstale":"2024-05-11T12:11:56.416996+0000","last_undegraded":"2024-05-11T12:11:56.416996+0000","last_fullsized":"2024-05-11T12:11:56.416996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:34:29.698881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.381254+0000","last_change":"2024-05-11T12:11:56.381254+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.381254+0000","last_undegraded":"2024-05-11T12:11:56.381254+0000","last_fullsized":"2024-05-11T12:11:56.381254+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.417407+0000","last_change":"2024-05-11T12:11:56.406587+0000","last_active":"2024-05-11T12:11:57.417407+0000","last_peered":"2024-05-11T12:11:57.417407+0000","last_clean":"2024-05-11T12:11:57.417407+0000","last_became_active":"2024-05-11T12:11:56.405746+0000","last_became_peered":"2024-05-11T12:11:56.405746+0000","last_unstale":"2024-05-11T12:11:57.417407+0000","last_undegraded":"2024-05-11T12:11:57.417407+0000","last_fullsized":"2024-05-11T12:11:57.417407+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:11:10.100041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T12:11:56.360286+0000","last_change":"2024-05-11T12:11:56.360286+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.360286+0000","last_undegraded":"2024-05-11T12:11:56.360286+0000","last_fullsized":"2024-05-11T12:11:56.360286+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T12:11:56.402312+0000","last_change":"2024-05-11T12:11:56.398172+0000","last_active":"2024-05-11T12:11:56.402312+0000","last_peered":"2024-05-11T12:11:56.402312+0000","last_clean":"2024-05-11T12:11:56.402312+0000","last_became_active":"2024-05-11T12:11:56.397860+0000","last_became_peered":"2024-05-11T12:11:56.397860+0000","last_unstale":"2024-05-11T12:11:56.402312+0000","last_undegraded":"2024-05-11T12:11:56.402312+0000","last_fullsized":"2024-05-11T12:11:56.402312+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:52:13.234125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.450935+0000","last_change":"2024-05-11T12:11:57.442952+0000","last_active":"2024-05-11T12:11:57.450935+0000","last_peered":"2024-05-11T12:11:57.450935+0000","last_clean":"2024-05-11T12:11:57.450935+0000","last_became_active":"2024-05-11T12:11:57.436880+0000","last_became_peered":"2024-05-11T12:11:57.436880+0000","last_unstale":"2024-05-11T12:11:57.450935+0000","last_undegraded":"2024-05-11T12:11:57.450935+0000","last_fullsized":"2024-05-11T12:11:57.450935+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:23:28.626421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.405805+0000","last_change":"2024-05-11T12:11:56.411981+0000","last_active":"2024-05-11T12:11:57.405805+0000","last_peered":"2024-05-11T12:11:57.405805+0000","last_clean":"2024-05-11T12:11:57.405805+0000","last_became_active":"2024-05-11T12:11:56.409613+0000","last_became_peered":"2024-05-11T12:11:56.409613+0000","last_unstale":"2024-05-11T12:11:57.405805+0000","last_undegraded":"2024-05-11T12:11:57.405805+0000","last_fullsized":"2024-05-11T12:11:57.405805+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:46:05.160542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.384922+0000","last_change":"2024-05-11T12:11:56.384922+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.384922+0000","last_undegraded":"2024-05-11T12:11:56.384922+0000","last_fullsized":"2024-05-11T12:11:56.384922+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"40'3","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.456654+0000","last_change":"2024-05-11T12:11:56.407591+0000","last_active":"2024-05-11T12:11:57.456654+0000","last_peered":"2024-05-11T12:11:57.456654+0000","last_clean":"2024-05-11T12:11:57.456654+0000","last_became_active":"2024-05-11T12:11:56.407173+0000","last_became_peered":"2024-05-11T12:11:56.407173+0000","last_unstale":"2024-05-11T12:11:57.456654+0000","last_undegraded":"2024-05-11T12:11:57.456654+0000","last_fullsized":"2024-05-11T12:11:57.456654+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:52:13.234125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.450953+0000","last_change":"2024-05-11T12:11:57.447257+0000","last_active":"2024-05-11T12:11:57.450953+0000","last_peered":"2024-05-11T12:11:57.450953+0000","last_clean":"2024-05-11T12:11:57.450953+0000","last_became_active":"2024-05-11T12:11:57.445351+0000","last_became_peered":"2024-05-11T12:11:57.445351+0000","last_unstale":"2024-05-11T12:11:57.450953+0000","last_undegraded":"2024-05-11T12:11:57.450953+0000","last_fullsized":"2024-05-11T12:11:57.450953+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:32:45.644260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T12:11:56.395987+0000","last_change":"2024-05-11T12:11:56.394318+0000","last_active":"2024-05-11T12:11:56.395987+0000","last_peered":"2024-05-11T12:11:56.395987+0000","last_clean":"2024-05-11T12:11:56.395987+0000","last_became_active":"2024-05-11T12:11:56.393560+0000","last_became_peered":"2024-05-11T12:11:56.393560+0000","last_unstale":"2024-05-11T12:11:56.395987+0000","last_undegraded":"2024-05-11T12:11:56.395987+0000","last_fullsized":"2024-05-11T12:11:56.395987+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:48:53.180988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.383778+0000","last_change":"2024-05-11T12:11:56.383778+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.383778+0000","last_undegraded":"2024-05-11T12:11:56.383778+0000","last_fullsized":"2024-05-11T12:11:56.383778+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T12:11:56.402064+0000","last_change":"2024-05-11T12:11:56.397825+0000","last_active":"2024-05-11T12:11:56.402064+0000","last_peered":"2024-05-11T12:11:56.402064+0000","last_clean":"2024-05-11T12:11:56.402064+0000","last_became_active":"2024-05-11T12:11:56.397506+0000","last_became_peered":"2024-05-11T12:11:56.397506+0000","last_unstale":"2024-05-11T12:11:56.402064+0000","last_undegraded":"2024-05-11T12:11:56.402064+0000","last_fullsized":"2024-05-11T12:11:56.402064+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:20:13.124057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.380190+0000","last_change":"2024-05-11T12:11:56.380190+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.380190+0000","last_undegraded":"2024-05-11T12:11:56.380190+0000","last_fullsized":"2024-05-11T12:11:56.380190+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T12:11:56.417002+0000","last_change":"2024-05-11T12:11:56.409815+0000","last_active":"2024-05-11T12:11:56.417002+0000","last_peered":"2024-05-11T12:11:56.417002+0000","last_clean":"2024-05-11T12:11:56.417002+0000","last_became_active":"2024-05-11T12:11:56.409411+0000","last_became_peered":"2024-05-11T12:11:56.409411+0000","last_unstale":"2024-05-11T12:11:56.417002+0000","last_undegraded":"2024-05-11T12:11:56.417002+0000","last_fullsized":"2024-05-11T12:11:56.417002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:16:00.825967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.450828+0000","last_change":"2024-05-11T12:11:57.442669+0000","last_active":"2024-05-11T12:11:57.450828+0000","last_peered":"2024-05-11T12:11:57.450828+0000","last_clean":"2024-05-11T12:11:57.450828+0000","last_became_active":"2024-05-11T12:11:57.437559+0000","last_became_peered":"2024-05-11T12:11:57.437559+0000","last_unstale":"2024-05-11T12:11:57.450828+0000","last_undegraded":"2024-05-11T12:11:57.450828+0000","last_fullsized":"2024-05-11T12:11:57.450828+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:28:44.586634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.405638+0000","last_change":"2024-05-11T12:11:56.408411+0000","last_active":"2024-05-11T12:11:57.405638+0000","last_peered":"2024-05-11T12:11:57.405638+0000","last_clean":"2024-05-11T12:11:57.405638+0000","last_became_active":"2024-05-11T12:11:56.408100+0000","last_became_peered":"2024-05-11T12:11:56.408100+0000","last_unstale":"2024-05-11T12:11:57.405638+0000","last_undegraded":"2024-05-11T12:11:57.405638+0000","last_fullsized":"2024-05-11T12:11:57.405638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:25:15.336149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.380730+0000","last_change":"2024-05-11T12:11:56.380730+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.380730+0000","last_undegraded":"2024-05-11T12:11:56.380730+0000","last_fullsized":"2024-05-11T12:11:56.380730+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"40'8","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.457343+0000","last_change":"2024-05-11T12:11:56.398517+0000","last_active":"2024-05-11T12:11:57.457343+0000","last_peered":"2024-05-11T12:11:57.457343+0000","last_clean":"2024-05-11T12:11:57.457343+0000","last_became_active":"2024-05-11T12:11:56.398204+0000","last_became_peered":"2024-05-11T12:11:56.398204+0000","last_unstale":"2024-05-11T12:11:57.457343+0000","last_undegraded":"2024-05-11T12:11:57.457343+0000","last_fullsized":"2024-05-11T12:11:57.457343+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:25:15.336149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.379813+0000","last_change":"2024-05-11T12:11:56.379813+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.379813+0000","last_undegraded":"2024-05-11T12:11:56.379813+0000","last_fullsized":"2024-05-11T12:11:56.379813+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"40'1","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.455210+0000","last_change":"2024-05-11T12:11:56.410975+0000","last_active":"2024-05-11T12:11:57.455210+0000","last_peered":"2024-05-11T12:11:57.455210+0000","last_clean":"2024-05-11T12:11:57.455210+0000","last_became_active":"2024-05-11T12:11:56.410634+0000","last_became_peered":"2024-05-11T12:11:56.410634+0000","last_unstale":"2024-05-11T12:11:57.455210+0000","last_undegraded":"2024-05-11T12:11:57.455210+0000","last_fullsized":"2024-05-11T12:11:57.455210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:10:04.020340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.433269+0000","last_change":"2024-05-11T12:11:57.428161+0000","last_active":"2024-05-11T12:11:57.433269+0000","last_peered":"2024-05-11T12:11:57.433269+0000","last_clean":"2024-05-11T12:11:57.433269+0000","last_became_active":"2024-05-11T12:11:57.427826+0000","last_became_peered":"2024-05-11T12:11:57.427826+0000","last_unstale":"2024-05-11T12:11:57.433269+0000","last_undegraded":"2024-05-11T12:11:57.433269+0000","last_fullsized":"2024-05-11T12:11:57.433269+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:11:37.571584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"40'2","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.459816+0000","last_change":"2024-05-11T12:11:56.406473+0000","last_active":"2024-05-11T12:11:57.459816+0000","last_peered":"2024-05-11T12:11:57.459816+0000","last_clean":"2024-05-11T12:11:57.459816+0000","last_became_active":"2024-05-11T12:11:56.405923+0000","last_became_peered":"2024-05-11T12:11:56.405923+0000","last_unstale":"2024-05-11T12:11:57.459816+0000","last_undegraded":"2024-05-11T12:11:57.459816+0000","last_fullsized":"2024-05-11T12:11:57.459816+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:11:42.632883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.450353+0000","last_change":"2024-05-11T12:11:57.436042+0000","last_active":"2024-05-11T12:11:57.450353+0000","last_peered":"2024-05-11T12:11:57.450353+0000","last_clean":"2024-05-11T12:11:57.450353+0000","last_became_active":"2024-05-11T12:11:57.435781+0000","last_became_peered":"2024-05-11T12:11:57.435781+0000","last_unstale":"2024-05-11T12:11:57.450353+0000","last_undegraded":"2024-05-11T12:11:57.450353+0000","last_fullsized":"2024-05-11T12:11:57.450353+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:07:22.774535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-11T12:11:55.388590+0000","last_change":"2024-05-11T12:11:55.388590+0000","last_active":"2024-05-11T12:11:55.345908+0000","last_peered":"2024-05-11T12:11:55.345908+0000","last_clean":"2024-05-11T12:11:55.345908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:55.388590+0000","last_undegraded":"2024-05-11T12:11:55.388590+0000","last_fullsized":"2024-05-11T12:11:55.388590+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T12:11:56.360286+0000","last_change":"2024-05-11T12:11:56.360286+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.360286+0000","last_undegraded":"2024-05-11T12:11:56.360286+0000","last_fullsized":"2024-05-11T12:11:56.360286+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T12:11:56.416911+0000","last_change":"2024-05-11T12:11:56.409279+0000","last_active":"2024-05-11T12:11:56.416911+0000","last_peered":"2024-05-11T12:11:56.416911+0000","last_clean":"2024-05-11T12:11:56.416911+0000","last_became_active":"2024-05-11T12:11:56.408985+0000","last_became_peered":"2024-05-11T12:11:56.408985+0000","last_unstale":"2024-05-11T12:11:56.416911+0000","last_undegraded":"2024-05-11T12:11:56.416911+0000","last_fullsized":"2024-05-11T12:11:56.416911+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:11:10.100041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.453397+0000","last_change":"2024-05-11T12:11:57.443914+0000","last_active":"2024-05-11T12:11:57.453397+0000","last_peered":"2024-05-11T12:11:57.453397+0000","last_clean":"2024-05-11T12:11:57.453397+0000","last_became_active":"2024-05-11T12:11:57.443680+0000","last_became_peered":"2024-05-11T12:11:57.443680+0000","last_unstale":"2024-05-11T12:11:57.453397+0000","last_undegraded":"2024-05-11T12:11:57.453397+0000","last_fullsized":"2024-05-11T12:11:57.453397+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:32:45.644260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-11T12:11:55.388296+0000","last_change":"2024-05-11T12:11:55.388296+0000","last_active":"2024-05-11T12:11:55.345908+0000","last_peered":"2024-05-11T12:11:55.345908+0000","last_clean":"2024-05-11T12:11:55.345908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:55.388296+0000","last_undegraded":"2024-05-11T12:11:55.388296+0000","last_fullsized":"2024-05-11T12:11:55.388296+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.383307+0000","last_change":"2024-05-11T12:11:56.383307+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.383307+0000","last_undegraded":"2024-05-11T12:11:56.383307+0000","last_fullsized":"2024-05-11T12:11:56.383307+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-11T12:11:55.388497+0000","last_change":"2024-05-11T12:11:55.388497+0000","last_active":"2024-05-11T12:11:55.345908+0000","last_peered":"2024-05-11T12:11:55.345908+0000","last_clean":"2024-05-11T12:11:55.345908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:55.388497+0000","last_undegraded":"2024-05-11T12:11:55.388497+0000","last_fullsized":"2024-05-11T12:11:55.388497+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.431245+0000","last_change":"2024-05-11T12:11:57.419098+0000","last_active":"2024-05-11T12:11:57.431245+0000","last_peered":"2024-05-11T12:11:57.431245+0000","last_clean":"2024-05-11T12:11:57.431245+0000","last_became_active":"2024-05-11T12:11:57.417628+0000","last_became_peered":"2024-05-11T12:11:57.417628+0000","last_unstale":"2024-05-11T12:11:57.431245+0000","last_undegraded":"2024-05-11T12:11:57.431245+0000","last_fullsized":"2024-05-11T12:11:57.431245+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:17:59.309336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.414852+0000","last_change":"2024-05-11T12:11:56.395623+0000","last_active":"2024-05-11T12:11:57.414852+0000","last_peered":"2024-05-11T12:11:57.414852+0000","last_clean":"2024-05-11T12:11:57.414852+0000","last_became_active":"2024-05-11T12:11:56.395250+0000","last_became_peered":"2024-05-11T12:11:56.395250+0000","last_unstale":"2024-05-11T12:11:57.414852+0000","last_undegraded":"2024-05-11T12:11:57.414852+0000","last_fullsized":"2024-05-11T12:11:57.414852+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:30:56.816516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.379844+0000","last_change":"2024-05-11T12:11:56.379844+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.379844+0000","last_undegraded":"2024-05-11T12:11:56.379844+0000","last_fullsized":"2024-05-11T12:11:56.379844+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-11T12:11:55.388481+0000","last_change":"2024-05-11T12:11:55.388481+0000","last_active":"2024-05-11T12:11:55.345908+0000","last_peered":"2024-05-11T12:11:55.345908+0000","last_clean":"2024-05-11T12:11:55.345908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:55.388481+0000","last_undegraded":"2024-05-11T12:11:55.388481+0000","last_fullsized":"2024-05-11T12:11:55.388481+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.386779+0000","last_change":"2024-05-11T12:11:56.386779+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.386779+0000","last_undegraded":"2024-05-11T12:11:56.386779+0000","last_fullsized":"2024-05-11T12:11:56.386779+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"40'2","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.455023+0000","last_change":"2024-05-11T12:11:56.409581+0000","last_active":"2024-05-11T12:11:57.455023+0000","last_peered":"2024-05-11T12:11:57.455023+0000","last_clean":"2024-05-11T12:11:57.455023+0000","last_became_active":"2024-05-11T12:11:56.408010+0000","last_became_peered":"2024-05-11T12:11:56.408010+0000","last_unstale":"2024-05-11T12:11:57.455023+0000","last_undegraded":"2024-05-11T12:11:57.455023+0000","last_fullsized":"2024-05-11T12:11:57.455023+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:20:13.124057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.451683+0000","last_change":"2024-05-11T12:11:57.447191+0000","last_active":"2024-05-11T12:11:57.451683+0000","last_peered":"2024-05-11T12:11:57.451683+0000","last_clean":"2024-05-11T12:11:57.451683+0000","last_became_active":"2024-05-11T12:11:57.442656+0000","last_became_peered":"2024-05-11T12:11:57.442656+0000","last_unstale":"2024-05-11T12:11:57.451683+0000","last_undegraded":"2024-05-11T12:11:57.451683+0000","last_fullsized":"2024-05-11T12:11:57.451683+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:52:14.248503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-11T12:11:55.388530+0000","last_change":"2024-05-11T12:11:55.388530+0000","last_active":"2024-05-11T12:11:55.345908+0000","last_peered":"2024-05-11T12:11:55.345908+0000","last_clean":"2024-05-11T12:11:55.345908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:55.388530+0000","last_undegraded":"2024-05-11T12:11:55.388530+0000","last_fullsized":"2024-05-11T12:11:55.388530+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.378928+0000","last_change":"2024-05-11T12:11:56.378928+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.378928+0000","last_undegraded":"2024-05-11T12:11:56.378928+0000","last_fullsized":"2024-05-11T12:11:56.378928+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T12:11:56.402024+0000","last_change":"2024-05-11T12:11:56.397553+0000","last_active":"2024-05-11T12:11:56.402024+0000","last_peered":"2024-05-11T12:11:56.402024+0000","last_clean":"2024-05-11T12:11:56.402024+0000","last_became_active":"2024-05-11T12:11:56.397219+0000","last_became_peered":"2024-05-11T12:11:56.397219+0000","last_unstale":"2024-05-11T12:11:56.402024+0000","last_undegraded":"2024-05-11T12:11:56.402024+0000","last_fullsized":"2024-05-11T12:11:56.402024+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:25:15.336149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.383035+0000","last_change":"2024-05-11T12:11:56.383035+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.383035+0000","last_undegraded":"2024-05-11T12:11:56.383035+0000","last_fullsized":"2024-05-11T12:11:56.383035+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T12:11:56.407895+0000","last_change":"2024-05-11T12:11:56.404006+0000","last_active":"2024-05-11T12:11:56.407895+0000","last_peered":"2024-05-11T12:11:56.407895+0000","last_clean":"2024-05-11T12:11:56.407895+0000","last_became_active":"2024-05-11T12:11:56.403672+0000","last_became_peered":"2024-05-11T12:11:56.403672+0000","last_unstale":"2024-05-11T12:11:56.407895+0000","last_undegraded":"2024-05-11T12:11:56.407895+0000","last_fullsized":"2024-05-11T12:11:56.407895+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:30:56.816516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T12:11:56.388205+0000","last_change":"2024-05-11T12:11:56.388205+0000","last_active":"2024-05-11T12:11:56.360286+0000","last_peered":"2024-05-11T12:11:56.360286+0000","last_clean":"2024-05-11T12:11:56.360286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:11:56.388205+0000","last_undegraded":"2024-05-11T12:11:56.388205+0000","last_fullsized":"2024-05-11T12:11:56.388205+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.401153+0000","last_change":"2024-05-11T12:11:56.399122+0000","last_active":"2024-05-11T12:11:57.401153+0000","last_peered":"2024-05-11T12:11:57.401153+0000","last_clean":"2024-05-11T12:11:57.401153+0000","last_became_active":"2024-05-11T12:11:56.398845+0000","last_became_peered":"2024-05-11T12:11:56.398845+0000","last_unstale":"2024-05-11T12:11:57.401153+0000","last_undegraded":"2024-05-11T12:11:57.401153+0000","last_fullsized":"2024-05-11T12:11:57.401153+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:16:00.825967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.449098+0000","last_change":"2024-05-11T12:11:57.442240+0000","last_active":"2024-05-11T12:11:57.449098+0000","last_peered":"2024-05-11T12:11:57.449098+0000","last_clean":"2024-05-11T12:11:57.449098+0000","last_became_active":"2024-05-11T12:11:57.440831+0000","last_became_peered":"2024-05-11T12:11:57.440831+0000","last_unstale":"2024-05-11T12:11:57.449098+0000","last_undegraded":"2024-05-11T12:11:57.449098+0000","last_fullsized":"2024-05-11T12:11:57.449098+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:28:44.586634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.418097+0000","last_change":"2024-05-11T12:11:56.402880+0000","last_active":"2024-05-11T12:11:57.418097+0000","last_peered":"2024-05-11T12:11:57.418097+0000","last_clean":"2024-05-11T12:11:57.418097+0000","last_became_active":"2024-05-11T12:11:56.402608+0000","last_became_peered":"2024-05-11T12:11:56.402608+0000","last_unstale":"2024-05-11T12:11:57.418097+0000","last_undegraded":"2024-05-11T12:11:57.418097+0000","last_fullsized":"2024-05-11T12:11:57.418097+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:34:29.698881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.451131+0000","last_change":"2024-05-11T12:11:57.444173+0000","last_active":"2024-05-11T12:11:57.451131+0000","last_peered":"2024-05-11T12:11:57.451131+0000","last_clean":"2024-05-11T12:11:57.451131+0000","last_became_active":"2024-05-11T12:11:57.441809+0000","last_became_peered":"2024-05-11T12:11:57.441809+0000","last_unstale":"2024-05-11T12:11:57.451131+0000","last_undegraded":"2024-05-11T12:11:57.451131+0000","last_fullsized":"2024-05-11T12:11:57.451131+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:48:54.195365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T12:11:56.405633+0000","last_change":"2024-05-11T12:11:56.399468+0000","last_active":"2024-05-11T12:11:56.405633+0000","last_peered":"2024-05-11T12:11:56.405633+0000","last_clean":"2024-05-11T12:11:56.405633+0000","last_became_active":"2024-05-11T12:11:56.399205+0000","last_became_peered":"2024-05-11T12:11:56.399205+0000","last_unstale":"2024-05-11T12:11:56.405633+0000","last_undegraded":"2024-05-11T12:11:56.405633+0000","last_fullsized":"2024-05-11T12:11:56.405633+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:10:04.020340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.430065+0000","last_change":"2024-05-11T12:11:57.417010+0000","last_active":"2024-05-11T12:11:57.430065+0000","last_peered":"2024-05-11T12:11:57.430065+0000","last_clean":"2024-05-11T12:11:57.430065+0000","last_became_active":"2024-05-11T12:11:57.416499+0000","last_became_peered":"2024-05-11T12:11:57.416499+0000","last_unstale":"2024-05-11T12:11:57.430065+0000","last_undegraded":"2024-05-11T12:11:57.430065+0000","last_fullsized":"2024-05-11T12:11:57.430065+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:53:03.480916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.418829+0000","last_change":"2024-05-11T12:11:56.400810+0000","last_active":"2024-05-11T12:11:57.418829+0000","last_peered":"2024-05-11T12:11:57.418829+0000","last_clean":"2024-05-11T12:11:57.418829+0000","last_became_active":"2024-05-11T12:11:56.399824+0000","last_became_peered":"2024-05-11T12:11:56.399824+0000","last_unstale":"2024-05-11T12:11:57.418829+0000","last_undegraded":"2024-05-11T12:11:57.418829+0000","last_fullsized":"2024-05-11T12:11:57.418829+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:07:21.760158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":139,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.414520+0000","last_change":"2024-05-11T12:10:35.066673+0000","last_active":"2024-05-11T12:11:57.414520+0000","last_peered":"2024-05-11T12:11:57.414520+0000","last_clean":"2024-05-11T12:11:57.414520+0000","last_became_active":"2024-05-11T12:10:35.065942+0000","last_became_peered":"2024-05-11T12:10:35.065942+0000","last_unstale":"2024-05-11T12:11:57.414520+0000","last_undegraded":"2024-05-11T12:11:57.414520+0000","last_fullsized":"2024-05-11T12:11:57.414520+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_clean_scrub_stamp":"2024-05-11T12:10:33.601136+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:12:02.133182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":116,"acting":116,"num_store_stats":7},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":694,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24,"ondisk_log_size":24,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476756,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28064,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343776,"statfs":{"total":96636764160,"available":96608026624,"internally_reserved":0,"allocated":1060864,"data_stored":784295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476757,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":192827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476756,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":192357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476758,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":192849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509461,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":438272,"data_stored":185078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509462,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":438272,"data_stored":185078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509461,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343808,"statfs":{"total":96636764160,"available":96608059392,"internally_reserved":0,"allocated":1032192,"data_stored":775446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542165,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":178411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T12:12:02.888 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3f is not active+clean 2024-05-11T12:12:02.888 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:11:56.383260+0000', 'last_change': '2024-05-11T12:11:56.383260+0000', 'last_active': '2024-05-11T12:11:56.360286+0000', 'last_peered': '2024-05-11T12:11:56.360286+0000', 'last_clean': '2024-05-11T12:11:56.360286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:11:56.383260+0000', 'last_undegraded': '2024-05-11T12:11:56.383260+0000', 'last_fullsized': '2024-05-11T12:11:56.383260+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_clean_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-11T12:12:02.888 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3e is not active+clean 2024-05-11T12:12:02.888 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:11:56.381407+0000', 'last_change': '2024-05-11T12:11:56.381407+0000', 'last_active': '2024-05-11T12:11:56.360286+0000', 'last_peered': '2024-05-11T12:11:56.360286+0000', 'last_clean': '2024-05-11T12:11:56.360286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:11:56.381407+0000', 'last_undegraded': '2024-05-11T12:11:56.381407+0000', 'last_fullsized': '2024-05-11T12:11:56.381407+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_clean_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-11T12:12:02.888 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3d is not active+clean 2024-05-11T12:12:02.889 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:11:56.388722+0000', 'last_change': '2024-05-11T12:11:56.388722+0000', 'last_active': '2024-05-11T12:11:56.360286+0000', 'last_peered': '2024-05-11T12:11:56.360286+0000', 'last_clean': '2024-05-11T12:11:56.360286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:11:56.388722+0000', 'last_undegraded': '2024-05-11T12:11:56.388722+0000', 'last_fullsized': '2024-05-11T12:11:56.388722+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_clean_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-11T12:12:02.889 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3c is not active+clean 2024-05-11T12:12:02.889 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T12:11:56.360286+0000', 'last_change': '2024-05-11T12:11:56.360286+0000', 'last_active': '2024-05-11T12:11:56.360286+0000', 'last_peered': '2024-05-11T12:11:56.360286+0000', 'last_clean': '2024-05-11T12:11:56.360286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:11:56.360286+0000', 'last_undegraded': '2024-05-11T12:11:56.360286+0000', 'last_fullsized': '2024-05-11T12:11:56.360286+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_clean_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T12:12:02.889 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3b is not active+clean 2024-05-11T12:12:02.889 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T12:11:56.360286+0000', 'last_change': '2024-05-11T12:11:56.360286+0000', 'last_active': '2024-05-11T12:11:56.360286+0000', 'last_peered': '2024-05-11T12:11:56.360286+0000', 'last_clean': '2024-05-11T12:11:56.360286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:11:56.360286+0000', 'last_undegraded': '2024-05-11T12:11:56.360286+0000', 'last_fullsized': '2024-05-11T12:11:56.360286+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_clean_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T12:12:02.889 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3a is not active+clean 2024-05-11T12:12:02.889 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:11:56.386336+0000', 'last_change': '2024-05-11T12:11:56.386336+0000', 'last_active': '2024-05-11T12:11:56.360286+0000', 'last_peered': '2024-05-11T12:11:56.360286+0000', 'last_clean': '2024-05-11T12:11:56.360286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:11:56.386336+0000', 'last_undegraded': '2024-05-11T12:11:56.386336+0000', 'last_fullsized': '2024-05-11T12:11:56.386336+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_clean_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-11T12:12:02.889 INFO:tasks.ceph.ceph_manager.ceph:PG 6.38 is not active+clean 2024-05-11T12:12:02.889 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:11:56.381742+0000', 'last_change': '2024-05-11T12:11:56.381742+0000', 'last_active': '2024-05-11T12:11:56.360286+0000', 'last_peered': '2024-05-11T12:11:56.360286+0000', 'last_clean': '2024-05-11T12:11:56.360286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:11:56.381742+0000', 'last_undegraded': '2024-05-11T12:11:56.381742+0000', 'last_fullsized': '2024-05-11T12:11:56.381742+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_clean_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-11T12:12:02.890 INFO:tasks.ceph.ceph_manager.ceph:PG 6.37 is not active+clean 2024-05-11T12:12:02.890 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:11:56.384226+0000', 'last_change': '2024-05-11T12:11:56.384226+0000', 'last_active': '2024-05-11T12:11:56.360286+0000', 'last_peered': '2024-05-11T12:11:56.360286+0000', 'last_clean': '2024-05-11T12:11:56.360286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:11:56.384226+0000', 'last_undegraded': '2024-05-11T12:11:56.384226+0000', 'last_fullsized': '2024-05-11T12:11:56.384226+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_clean_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-11T12:12:02.890 INFO:tasks.ceph.ceph_manager.ceph:PG 6.34 is not active+clean 2024-05-11T12:12:02.890 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:11:56.386630+0000', 'last_change': '2024-05-11T12:11:56.386630+0000', 'last_active': '2024-05-11T12:11:56.360286+0000', 'last_peered': '2024-05-11T12:11:56.360286+0000', 'last_clean': '2024-05-11T12:11:56.360286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:11:56.386630+0000', 'last_undegraded': '2024-05-11T12:11:56.386630+0000', 'last_fullsized': '2024-05-11T12:11:56.386630+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_clean_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-11T12:12:02.890 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2d is not active+clean 2024-05-11T12:12:02.890 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T12:11:56.360286+0000', 'last_change': '2024-05-11T12:11:56.360286+0000', 'last_active': '2024-05-11T12:11:56.360286+0000', 'last_peered': '2024-05-11T12:11:56.360286+0000', 'last_clean': '2024-05-11T12:11:56.360286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:11:56.360286+0000', 'last_undegraded': '2024-05-11T12:11:56.360286+0000', 'last_fullsized': '2024-05-11T12:11:56.360286+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_clean_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T12:12:02.890 INFO:tasks.ceph.ceph_manager.ceph:PG 6.28 is not active+clean 2024-05-11T12:12:02.890 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T12:11:56.360286+0000', 'last_change': '2024-05-11T12:11:56.360286+0000', 'last_active': '2024-05-11T12:11:56.360286+0000', 'last_peered': '2024-05-11T12:11:56.360286+0000', 'last_clean': '2024-05-11T12:11:56.360286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:11:56.360286+0000', 'last_undegraded': '2024-05-11T12:11:56.360286+0000', 'last_fullsized': '2024-05-11T12:11:56.360286+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_clean_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T12:12:02.890 INFO:tasks.ceph.ceph_manager.ceph:PG 6.26 is not active+clean 2024-05-11T12:12:02.891 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:11:56.390804+0000', 'last_change': '2024-05-11T12:11:56.390804+0000', 'last_active': '2024-05-11T12:11:56.360286+0000', 'last_peered': '2024-05-11T12:11:56.360286+0000', 'last_clean': '2024-05-11T12:11:56.360286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:11:56.390804+0000', 'last_undegraded': '2024-05-11T12:11:56.390804+0000', 'last_fullsized': '2024-05-11T12:11:56.390804+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_clean_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-11T12:12:02.891 INFO:tasks.ceph.ceph_manager.ceph:PG 6.20 is not active+clean 2024-05-11T12:12:02.891 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:11:56.389994+0000', 'last_change': '2024-05-11T12:11:56.389994+0000', 'last_active': '2024-05-11T12:11:56.360286+0000', 'last_peered': '2024-05-11T12:11:56.360286+0000', 'last_clean': '2024-05-11T12:11:56.360286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:11:56.389994+0000', 'last_undegraded': '2024-05-11T12:11:56.389994+0000', 'last_fullsized': '2024-05-11T12:11:56.389994+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_clean_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-11T12:12:02.891 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1d is not active+clean 2024-05-11T12:12:02.891 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:11:56.389689+0000', 'last_change': '2024-05-11T12:11:56.389689+0000', 'last_active': '2024-05-11T12:11:56.360286+0000', 'last_peered': '2024-05-11T12:11:56.360286+0000', 'last_clean': '2024-05-11T12:11:56.360286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:11:56.389689+0000', 'last_undegraded': '2024-05-11T12:11:56.389689+0000', 'last_fullsized': '2024-05-11T12:11:56.389689+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_clean_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-11T12:12:02.891 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1f is not active+clean 2024-05-11T12:12:02.891 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:11:56.382086+0000', 'last_change': '2024-05-11T12:11:56.382086+0000', 'last_active': '2024-05-11T12:11:56.360286+0000', 'last_peered': '2024-05-11T12:11:56.360286+0000', 'last_clean': '2024-05-11T12:11:56.360286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:11:56.382086+0000', 'last_undegraded': '2024-05-11T12:11:56.382086+0000', 'last_fullsized': '2024-05-11T12:11:56.382086+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_clean_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-11T12:12:02.891 INFO:tasks.ceph.ceph_manager.ceph:PG 6.18 is not active+clean 2024-05-11T12:12:02.891 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:11:56.384699+0000', 'last_change': '2024-05-11T12:11:56.384699+0000', 'last_active': '2024-05-11T12:11:56.360286+0000', 'last_peered': '2024-05-11T12:11:56.360286+0000', 'last_clean': '2024-05-11T12:11:56.360286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:11:56.384699+0000', 'last_undegraded': '2024-05-11T12:11:56.384699+0000', 'last_fullsized': '2024-05-11T12:11:56.384699+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_clean_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-11T12:12:02.891 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1b is not active+clean 2024-05-11T12:12:02.891 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:11:56.381254+0000', 'last_change': '2024-05-11T12:11:56.381254+0000', 'last_active': '2024-05-11T12:11:56.360286+0000', 'last_peered': '2024-05-11T12:11:56.360286+0000', 'last_clean': '2024-05-11T12:11:56.360286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:11:56.381254+0000', 'last_undegraded': '2024-05-11T12:11:56.381254+0000', 'last_fullsized': '2024-05-11T12:11:56.381254+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_clean_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-11T12:12:02.891 INFO:tasks.ceph.ceph_manager.ceph:PG 6.14 is not active+clean 2024-05-11T12:12:02.892 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T12:11:56.360286+0000', 'last_change': '2024-05-11T12:11:56.360286+0000', 'last_active': '2024-05-11T12:11:56.360286+0000', 'last_peered': '2024-05-11T12:11:56.360286+0000', 'last_clean': '2024-05-11T12:11:56.360286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:11:56.360286+0000', 'last_undegraded': '2024-05-11T12:11:56.360286+0000', 'last_fullsized': '2024-05-11T12:11:56.360286+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_clean_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T12:12:02.892 INFO:tasks.ceph.ceph_manager.ceph:PG 6.16 is not active+clean 2024-05-11T12:12:02.892 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:11:56.384922+0000', 'last_change': '2024-05-11T12:11:56.384922+0000', 'last_active': '2024-05-11T12:11:56.360286+0000', 'last_peered': '2024-05-11T12:11:56.360286+0000', 'last_clean': '2024-05-11T12:11:56.360286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:11:56.384922+0000', 'last_undegraded': '2024-05-11T12:11:56.384922+0000', 'last_fullsized': '2024-05-11T12:11:56.384922+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_clean_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-11T12:12:02.892 INFO:tasks.ceph.ceph_manager.ceph:PG 6.10 is not active+clean 2024-05-11T12:12:02.892 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:11:56.383778+0000', 'last_change': '2024-05-11T12:11:56.383778+0000', 'last_active': '2024-05-11T12:11:56.360286+0000', 'last_peered': '2024-05-11T12:11:56.360286+0000', 'last_clean': '2024-05-11T12:11:56.360286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:11:56.383778+0000', 'last_undegraded': '2024-05-11T12:11:56.383778+0000', 'last_fullsized': '2024-05-11T12:11:56.383778+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_clean_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-11T12:12:02.892 INFO:tasks.ceph.ceph_manager.ceph:PG 6.11 is not active+clean 2024-05-11T12:12:02.892 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:11:56.380190+0000', 'last_change': '2024-05-11T12:11:56.380190+0000', 'last_active': '2024-05-11T12:11:56.360286+0000', 'last_peered': '2024-05-11T12:11:56.360286+0000', 'last_clean': '2024-05-11T12:11:56.360286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:11:56.380190+0000', 'last_undegraded': '2024-05-11T12:11:56.380190+0000', 'last_fullsized': '2024-05-11T12:11:56.380190+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_clean_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-11T12:12:02.892 INFO:tasks.ceph.ceph_manager.ceph:PG 6.13 is not active+clean 2024-05-11T12:12:02.892 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:11:56.380730+0000', 'last_change': '2024-05-11T12:11:56.380730+0000', 'last_active': '2024-05-11T12:11:56.360286+0000', 'last_peered': '2024-05-11T12:11:56.360286+0000', 'last_clean': '2024-05-11T12:11:56.360286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:11:56.380730+0000', 'last_undegraded': '2024-05-11T12:11:56.380730+0000', 'last_fullsized': '2024-05-11T12:11:56.380730+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_clean_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-11T12:12:02.892 INFO:tasks.ceph.ceph_manager.ceph:PG 6.c is not active+clean 2024-05-11T12:12:02.893 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:11:56.379813+0000', 'last_change': '2024-05-11T12:11:56.379813+0000', 'last_active': '2024-05-11T12:11:56.360286+0000', 'last_peered': '2024-05-11T12:11:56.360286+0000', 'last_clean': '2024-05-11T12:11:56.360286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:11:56.379813+0000', 'last_undegraded': '2024-05-11T12:11:56.379813+0000', 'last_fullsized': '2024-05-11T12:11:56.379813+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_clean_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-11T12:12:02.893 INFO:tasks.ceph.ceph_manager.ceph:PG 5.d is not active+clean 2024-05-11T12:12:02.893 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:11:55.388590+0000', 'last_change': '2024-05-11T12:11:55.388590+0000', 'last_active': '2024-05-11T12:11:55.345908+0000', 'last_peered': '2024-05-11T12:11:55.345908+0000', 'last_clean': '2024-05-11T12:11:55.345908+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:11:55.388590+0000', 'last_undegraded': '2024-05-11T12:11:55.388590+0000', 'last_fullsized': '2024-05-11T12:11:55.388590+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:11:55.345908+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:11:55.345908+0000', 'last_clean_scrub_stamp': '2024-05-11T12:11:55.345908+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-11T12:12:02.893 INFO:tasks.ceph.ceph_manager.ceph:PG 6.f is not active+clean 2024-05-11T12:12:02.893 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T12:11:56.360286+0000', 'last_change': '2024-05-11T12:11:56.360286+0000', 'last_active': '2024-05-11T12:11:56.360286+0000', 'last_peered': '2024-05-11T12:11:56.360286+0000', 'last_clean': '2024-05-11T12:11:56.360286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:11:56.360286+0000', 'last_undegraded': '2024-05-11T12:11:56.360286+0000', 'last_fullsized': '2024-05-11T12:11:56.360286+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_clean_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T12:12:02.893 INFO:tasks.ceph.ceph_manager.ceph:PG 5.b is not active+clean 2024-05-11T12:12:02.893 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:11:55.388296+0000', 'last_change': '2024-05-11T12:11:55.388296+0000', 'last_active': '2024-05-11T12:11:55.345908+0000', 'last_peered': '2024-05-11T12:11:55.345908+0000', 'last_clean': '2024-05-11T12:11:55.345908+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:11:55.388296+0000', 'last_undegraded': '2024-05-11T12:11:55.388296+0000', 'last_fullsized': '2024-05-11T12:11:55.388296+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:11:55.345908+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:11:55.345908+0000', 'last_clean_scrub_stamp': '2024-05-11T12:11:55.345908+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-11T12:12:02.893 INFO:tasks.ceph.ceph_manager.ceph:PG 6.9 is not active+clean 2024-05-11T12:12:02.893 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:11:56.383307+0000', 'last_change': '2024-05-11T12:11:56.383307+0000', 'last_active': '2024-05-11T12:11:56.360286+0000', 'last_peered': '2024-05-11T12:11:56.360286+0000', 'last_clean': '2024-05-11T12:11:56.360286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:11:56.383307+0000', 'last_undegraded': '2024-05-11T12:11:56.383307+0000', 'last_fullsized': '2024-05-11T12:11:56.383307+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_clean_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-11T12:12:02.893 INFO:tasks.ceph.ceph_manager.ceph:PG 5.a is not active+clean 2024-05-11T12:12:02.893 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:11:55.388497+0000', 'last_change': '2024-05-11T12:11:55.388497+0000', 'last_active': '2024-05-11T12:11:55.345908+0000', 'last_peered': '2024-05-11T12:11:55.345908+0000', 'last_clean': '2024-05-11T12:11:55.345908+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:11:55.388497+0000', 'last_undegraded': '2024-05-11T12:11:55.388497+0000', 'last_fullsized': '2024-05-11T12:11:55.388497+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:11:55.345908+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:11:55.345908+0000', 'last_clean_scrub_stamp': '2024-05-11T12:11:55.345908+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-11T12:12:02.893 INFO:tasks.ceph.ceph_manager.ceph:PG 6.b is not active+clean 2024-05-11T12:12:02.894 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:11:56.379844+0000', 'last_change': '2024-05-11T12:11:56.379844+0000', 'last_active': '2024-05-11T12:11:56.360286+0000', 'last_peered': '2024-05-11T12:11:56.360286+0000', 'last_clean': '2024-05-11T12:11:56.360286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:11:56.379844+0000', 'last_undegraded': '2024-05-11T12:11:56.379844+0000', 'last_fullsized': '2024-05-11T12:11:56.379844+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_clean_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-11T12:12:02.894 INFO:tasks.ceph.ceph_manager.ceph:PG 5.8 is not active+clean 2024-05-11T12:12:02.894 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:11:55.388481+0000', 'last_change': '2024-05-11T12:11:55.388481+0000', 'last_active': '2024-05-11T12:11:55.345908+0000', 'last_peered': '2024-05-11T12:11:55.345908+0000', 'last_clean': '2024-05-11T12:11:55.345908+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:11:55.388481+0000', 'last_undegraded': '2024-05-11T12:11:55.388481+0000', 'last_fullsized': '2024-05-11T12:11:55.388481+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:11:55.345908+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:11:55.345908+0000', 'last_clean_scrub_stamp': '2024-05-11T12:11:55.345908+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-11T12:12:02.894 INFO:tasks.ceph.ceph_manager.ceph:PG 6.4 is not active+clean 2024-05-11T12:12:02.894 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:11:56.386779+0000', 'last_change': '2024-05-11T12:11:56.386779+0000', 'last_active': '2024-05-11T12:11:56.360286+0000', 'last_peered': '2024-05-11T12:11:56.360286+0000', 'last_clean': '2024-05-11T12:11:56.360286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:11:56.386779+0000', 'last_undegraded': '2024-05-11T12:11:56.386779+0000', 'last_fullsized': '2024-05-11T12:11:56.386779+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_clean_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-11T12:12:02.894 INFO:tasks.ceph.ceph_manager.ceph:PG 5.6 is not active+clean 2024-05-11T12:12:02.894 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:11:55.388530+0000', 'last_change': '2024-05-11T12:11:55.388530+0000', 'last_active': '2024-05-11T12:11:55.345908+0000', 'last_peered': '2024-05-11T12:11:55.345908+0000', 'last_clean': '2024-05-11T12:11:55.345908+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:11:55.388530+0000', 'last_undegraded': '2024-05-11T12:11:55.388530+0000', 'last_fullsized': '2024-05-11T12:11:55.388530+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:11:55.345908+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:11:55.345908+0000', 'last_clean_scrub_stamp': '2024-05-11T12:11:55.345908+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-11T12:12:02.894 INFO:tasks.ceph.ceph_manager.ceph:PG 6.6 is not active+clean 2024-05-11T12:12:02.894 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:11:56.378928+0000', 'last_change': '2024-05-11T12:11:56.378928+0000', 'last_active': '2024-05-11T12:11:56.360286+0000', 'last_peered': '2024-05-11T12:11:56.360286+0000', 'last_clean': '2024-05-11T12:11:56.360286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:11:56.378928+0000', 'last_undegraded': '2024-05-11T12:11:56.378928+0000', 'last_fullsized': '2024-05-11T12:11:56.378928+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_clean_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-11T12:12:02.894 INFO:tasks.ceph.ceph_manager.ceph:PG 6.0 is not active+clean 2024-05-11T12:12:02.894 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:11:56.383035+0000', 'last_change': '2024-05-11T12:11:56.383035+0000', 'last_active': '2024-05-11T12:11:56.360286+0000', 'last_peered': '2024-05-11T12:11:56.360286+0000', 'last_clean': '2024-05-11T12:11:56.360286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:11:56.383035+0000', 'last_undegraded': '2024-05-11T12:11:56.383035+0000', 'last_fullsized': '2024-05-11T12:11:56.383035+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_clean_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-11T12:12:02.894 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1 is not active+clean 2024-05-11T12:12:02.895 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:11:56.388205+0000', 'last_change': '2024-05-11T12:11:56.388205+0000', 'last_active': '2024-05-11T12:11:56.360286+0000', 'last_peered': '2024-05-11T12:11:56.360286+0000', 'last_clean': '2024-05-11T12:11:56.360286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:11:56.388205+0000', 'last_undegraded': '2024-05-11T12:11:56.388205+0000', 'last_fullsized': '2024-05-11T12:11:56.388205+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'last_clean_scrub_stamp': '2024-05-11T12:11:56.360286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-11T12:12:02.895 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-11T12:12:03.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.125+0000 7fc087e00640 1 Processor -- start 2024-05-11T12:12:03.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.125+0000 7fc087e00640 1 -- start start 2024-05-11T12:12:03.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.126+0000 7fc087e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0881458f0 0x7fc08806d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:03.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.126+0000 7fc087e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc0880b5220 0x7fc0880b56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:03.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.127+0000 7fc087e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc088071f30 con 0x7fc0880b3a80 2024-05-11T12:12:03.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.127+0000 7fc087e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fc0880b5bf0 con 0x7fc0880b5220 2024-05-11T12:12:03.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.127+0000 7fc087e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc08806db80 con 0x7fc0881458f0 2024-05-11T12:12:03.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.127+0000 7fc086a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0881458f0 0x7fc08806d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:03.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.127+0000 7fc087400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc0880b5220 0x7fc0880b56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:03.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.127+0000 7fc087400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc0880b5220 0x7fc0880b56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.167:33954/0 (socket says 172.21.15.167:33954) 2024-05-11T12:12:03.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.127+0000 7fc087400640 1 -- 172.21.15.167:0/2462348252 learned_addr learned my addr 172.21.15.167:0/2462348252 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:03.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.128+0000 7fc087400640 1 -- 172.21.15.167:0/2462348252 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0881458f0 msgr2=0x7fc08806d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:03.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.128+0000 7fc087400640 1 --2- 172.21.15.167:0/2462348252 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0881458f0 0x7fc08806d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:03.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.128+0000 7fc087400640 1 -- 172.21.15.167:0/2462348252 >> v1:172.21.15.204:6789/0 conn(0x7fc0880b3a80 legacy=0x7fc0880b4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:03.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.128+0000 7fc087400640 1 -- 172.21.15.167:0/2462348252 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc088072210 con 0x7fc0880b5220 2024-05-11T12:12:03.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.130+0000 7fc087400640 1 --2- 172.21.15.167:0/2462348252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0880b5220 0x7fc0880b56b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc080009e50 tx=0x7fc08002a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2e782f9aef553ea server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:03.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.130+0000 7fc085600640 1 -- 172.21.15.167:0/2462348252 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc080036070 con 0x7fc0880b5220 2024-05-11T12:12:03.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.130+0000 7fc085600640 1 -- 172.21.15.167:0/2462348252 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc080031600 con 0x7fc0880b5220 2024-05-11T12:12:03.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.130+0000 7fc085600640 1 -- 172.21.15.167:0/2462348252 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0800319d0 con 0x7fc0880b5220 2024-05-11T12:12:03.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.131+0000 7fc087e00640 1 -- 172.21.15.167:0/2462348252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0880b5220 msgr2=0x7fc0880b56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:03.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.131+0000 7fc087e00640 1 --2- 172.21.15.167:0/2462348252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0880b5220 0x7fc0880b56b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc080009e50 tx=0x7fc08002a710 comp rx=0 tx=0).stop 2024-05-11T12:12:03.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.131+0000 7fc087e00640 1 -- 172.21.15.167:0/2462348252 shutdown_connections 2024-05-11T12:12:03.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.131+0000 7fc087e00640 1 --2- 172.21.15.167:0/2462348252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0880b5220 0x7fc0880b56b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:03.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.131+0000 7fc087e00640 1 --2- 172.21.15.167:0/2462348252 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0881458f0 0x7fc08806d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:03.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.131+0000 7fc087e00640 1 -- 172.21.15.167:0/2462348252 >> 172.21.15.167:0/2462348252 conn(0x7fc0880b3460 msgr2=0x7fc0880d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:03.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.131+0000 7fc087e00640 1 -- 172.21.15.167:0/2462348252 shutdown_connections 2024-05-11T12:12:03.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.131+0000 7fc087e00640 1 -- 172.21.15.167:0/2462348252 wait complete. 2024-05-11T12:12:03.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.133+0000 7fc087e00640 1 Processor -- start 2024-05-11T12:12:03.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.133+0000 7fc087e00640 1 -- start start 2024-05-11T12:12:03.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.134+0000 7fc087e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0880b3a80 0x7fc08815d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:03.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.134+0000 7fc087e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0880b5220 0x7fc08813c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:03.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.134+0000 7fc087e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0881458f0 0x7fc08815e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:03.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.134+0000 7fc087e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc088072ab0 con 0x7fc0880b5220 2024-05-11T12:12:03.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.134+0000 7fc087e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc0880727d0 con 0x7fc0881458f0 2024-05-11T12:12:03.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.134+0000 7fc087e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc088072fd0 con 0x7fc0880b3a80 2024-05-11T12:12:03.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.134+0000 7fc086000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0880b5220 0x7fc08813c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:03.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.135+0000 7fc086000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0880b5220 0x7fc08813c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41886/0 (socket says 172.21.15.167:41886) 2024-05-11T12:12:03.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.135+0000 7fc086000640 1 -- 172.21.15.167:0/4240758456 learned_addr learned my addr 172.21.15.167:0/4240758456 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:03.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.135+0000 7fc086a00640 1 --2- 172.21.15.167:0/4240758456 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0880b3a80 0x7fc08815d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:03.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.135+0000 7fc087400640 1 --2- 172.21.15.167:0/4240758456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0881458f0 0x7fc08815e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:03.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.135+0000 7fc086000640 1 -- 172.21.15.167:0/4240758456 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0880b3a80 msgr2=0x7fc08815d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:03.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.135+0000 7fc086000640 1 --2- 172.21.15.167:0/4240758456 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0880b3a80 0x7fc08815d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:03.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.135+0000 7fc086000640 1 -- 172.21.15.167:0/4240758456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0881458f0 msgr2=0x7fc08815e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:03.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.135+0000 7fc086000640 1 --2- 172.21.15.167:0/4240758456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0881458f0 0x7fc08815e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:03.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.135+0000 7fc086000640 1 -- 172.21.15.167:0/4240758456 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0880b05b0 con 0x7fc0880b5220 2024-05-11T12:12:03.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.135+0000 7fc086000640 1 --2- 172.21.15.167:0/4240758456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0880b5220 0x7fc08813c2d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc07400c450 tx=0x7fc07400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:03.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.136+0000 7fc07f400640 1 -- 172.21.15.167:0/4240758456 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc074017070 con 0x7fc0880b5220 2024-05-11T12:12:03.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.136+0000 7fc07f400640 1 -- 172.21.15.167:0/4240758456 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0740052d0 con 0x7fc0880b5220 2024-05-11T12:12:03.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.136+0000 7fc07f400640 1 -- 172.21.15.167:0/4240758456 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc074003830 con 0x7fc0880b5220 2024-05-11T12:12:03.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.136+0000 7fc087e00640 1 -- 172.21.15.167:0/4240758456 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc08815e9a0 con 0x7fc0880b5220 2024-05-11T12:12:03.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.136+0000 7fc087e00640 1 -- 172.21.15.167:0/4240758456 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0880b0b50 con 0x7fc0880b5220 2024-05-11T12:12:03.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.137+0000 7fc087e00640 1 -- 172.21.15.167:0/4240758456 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0880694b0 con 0x7fc0880b5220 2024-05-11T12:12:03.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.138+0000 7fc07f400640 1 -- 172.21.15.167:0/4240758456 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc07401a070 con 0x7fc0880b5220 2024-05-11T12:12:03.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.139+0000 7fc07f400640 1 --2- 172.21.15.167:0/4240758456 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc0680762d0 0x7fc068078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:03.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.139+0000 7fc07f400640 1 -- 172.21.15.167:0/4240758456 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7fc074096bc0 con 0x7fc0880b5220 2024-05-11T12:12:03.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.139+0000 7fc086a00640 1 --2- 172.21.15.167:0/4240758456 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc0680762d0 0x7fc068078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:03.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.140+0000 7fc086a00640 1 --2- 172.21.15.167:0/4240758456 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc0680762d0 0x7fc068078790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc078009a30 tx=0x7fc078008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:03.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.144+0000 7fc07f400640 1 -- 172.21.15.167:0/4240758456 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc0740602f0 con 0x7fc0880b5220 2024-05-11T12:12:03.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.426+0000 7fc087e00640 1 -- 172.21.15.167:0/4240758456 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fc08813c810 con 0x7fc0880b5220 2024-05-11T12:12:03.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.427+0000 7fc07f400640 1 -- 172.21.15.167:0/4240758456 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1335 (secure 0 0 0) 0x7fc07405fbf0 con 0x7fc0880b5220 2024-05-11T12:12:03.428 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:12:03.428 INFO:teuthology.orchestra.run.smithi167.stdout:{"fsid":"5f32d344-7289-4c27-881b-1533419ab678","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":103,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":8,"num_up_osds":8,"osd_up_since":1715429432,"num_in_osds":8,"osd_in_since":1715429423,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":62},{"state_name":"creating+peering","count":29},{"state_name":"unknown","count":6}],"num_pgs":97,"num_pools":3,"num_objects":14,"data_bytes":591062,"bytes_used":226144256,"bytes_avail":772867969024,"bytes_total":773094113280,"unknown_pgs_ratio":0.061855670064687729,"inactive_pgs_ratio":0.29896906018257141,"write_bytes_sec":521,"read_op_per_sec":0,"write_op_per_sec":5},"fsmap":{"epoch":18,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"d","status":"up:active","gid":4908}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-05-11T12:11:41.251279+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-11T12:12:03.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.430+0000 7fc087e00640 1 -- 172.21.15.167:0/4240758456 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc0680762d0 msgr2=0x7fc068078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:03.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.430+0000 7fc087e00640 1 --2- 172.21.15.167:0/4240758456 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc0680762d0 0x7fc068078790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc078009a30 tx=0x7fc078008040 comp rx=0 tx=0).stop 2024-05-11T12:12:03.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.430+0000 7fc087e00640 1 -- 172.21.15.167:0/4240758456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0880b5220 msgr2=0x7fc08813c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:03.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.430+0000 7fc087e00640 1 --2- 172.21.15.167:0/4240758456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0880b5220 0x7fc08813c2d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc07400c450 tx=0x7fc07400c910 comp rx=0 tx=0).stop 2024-05-11T12:12:03.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.430+0000 7fc087e00640 1 -- 172.21.15.167:0/4240758456 shutdown_connections 2024-05-11T12:12:03.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.430+0000 7fc087e00640 1 --2- 172.21.15.167:0/4240758456 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc0680762d0 0x7fc068078790 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:03.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.430+0000 7fc087e00640 1 --2- 172.21.15.167:0/4240758456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0881458f0 0x7fc08815e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:03.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.430+0000 7fc087e00640 1 --2- 172.21.15.167:0/4240758456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0880b5220 0x7fc08813c2d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:03.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.430+0000 7fc087e00640 1 --2- 172.21.15.167:0/4240758456 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0880b3a80 0x7fc08815d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:03.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.430+0000 7fc087e00640 1 -- 172.21.15.167:0/4240758456 >> 172.21.15.167:0/4240758456 conn(0x7fc0880b3460 msgr2=0x7fc0880ae820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:03.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.430+0000 7fc087e00640 1 -- 172.21.15.167:0/4240758456 shutdown_connections 2024-05-11T12:12:03.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.430+0000 7fc087e00640 1 -- 172.21.15.167:0/4240758456 wait complete. 2024-05-11T12:12:03.449 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-05-11T12:12:03.449 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T12:12:03.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.646+0000 7fd6c7e00640 1 Processor -- start 2024-05-11T12:12:03.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.646+0000 7fd6c7e00640 1 -- start start 2024-05-11T12:12:03.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.647+0000 7fd6c7e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6c806c660 0x7fd6c806ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:03.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.647+0000 7fd6c7e00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6c806cfb0 con 0x7fd6c8146770 2024-05-11T12:12:03.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.647+0000 7fd6c7e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6c806fbc0 con 0x7fd6c8145b00 2024-05-11T12:12:03.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.647+0000 7fd6c7e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd6c80b4520 con 0x7fd6c806c660 2024-05-11T12:12:03.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.648+0000 7fd6be000640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fd6c8146770 0x7fd6c80ce9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:42182/0 (socket says 172.21.15.167:42182) 2024-05-11T12:12:03.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.648+0000 7fd6be000640 1 -- 172.21.15.167:0/1273655909 learned_addr learned my addr 172.21.15.167:0/1273655909 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:03.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.648+0000 7fd6c7400640 1 --2- 172.21.15.167:0/1273655909 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6c806c660 0x7fd6c806ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:03.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.648+0000 7fd6c6000640 1 -- 172.21.15.167:0/1273655909 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1885027646 0 0) 0x7fd6b0001110 con 0x7fd6c8146770 2024-05-11T12:12:03.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.648+0000 7fd6c6000640 1 -- 172.21.15.167:0/1273655909 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6ac003520 con 0x7fd6c8146770 2024-05-11T12:12:03.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.649+0000 7fd6c6000640 1 -- 172.21.15.167:0/1273655909 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3674161244 0 0) 0x7fd6ac003520 con 0x7fd6c8146770 2024-05-11T12:12:03.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.649+0000 7fd6c6000640 1 -- 172.21.15.167:0/1273655909 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6c806c660 msgr2=0x7fd6c806ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:03.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.649+0000 7fd6c6000640 1 --2- 172.21.15.167:0/1273655909 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6c806c660 0x7fd6c806ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:03.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.649+0000 7fd6c6000640 1 -- 172.21.15.167:0/1273655909 >> v1:172.21.15.204:6789/0 conn(0x7fd6c8145b00 legacy=0x7fd6c806f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:03.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.649+0000 7fd6c6000640 1 -- 172.21.15.167:0/1273655909 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6c80b4800 con 0x7fd6c8146770 2024-05-11T12:12:03.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.649+0000 7fd6c6000640 1 -- 172.21.15.167:0/1273655909 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7fd6b00018c0 con 0x7fd6c8146770 2024-05-11T12:12:03.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.649+0000 7fd6c6000640 1 -- 172.21.15.167:0/1273655909 >> v1:172.21.15.167:6789/0 conn(0x7fd6c8146770 legacy=0x7fd6c80ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:03.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.649+0000 7fd6c6000640 1 --2- 172.21.15.167:0/1273655909 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6c806c660 0x7fd6ac003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:03.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.649+0000 7fd6c6000640 1 --2- 172.21.15.167:0/1273655909 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6ac0040b0 0x7fd6ac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:03.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.649+0000 7fd6c6000640 1 --2- 172.21.15.167:0/1273655909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6ac004d90 0x7fd6ac005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:03.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.649+0000 7fd6c6000640 1 -- 172.21.15.167:0/1273655909 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c80b4520 con 0x7fd6ac0040b0 2024-05-11T12:12:03.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.649+0000 7fd6c6000640 1 -- 172.21.15.167:0/1273655909 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ac00bca0 con 0x7fd6ac004d90 2024-05-11T12:12:03.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.649+0000 7fd6c6000640 1 -- 172.21.15.167:0/1273655909 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd6ac00bea0 con 0x7fd6c806c660 2024-05-11T12:12:03.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.649+0000 7fd6be000640 1 --2- 172.21.15.167:0/1273655909 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6ac0040b0 0x7fd6ac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:03.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.649+0000 7fd6c6a00640 1 --2- 172.21.15.167:0/1273655909 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6c806c660 0x7fd6ac003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:03.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.649+0000 7fd6c7400640 1 --2- 172.21.15.167:0/1273655909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6ac004d90 0x7fd6ac005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:03.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.650+0000 7fd6be000640 1 -- 172.21.15.167:0/1273655909 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6c806c660 msgr2=0x7fd6ac003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:03.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.650+0000 7fd6be000640 1 --2- 172.21.15.167:0/1273655909 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6c806c660 0x7fd6ac003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:03.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.650+0000 7fd6be000640 1 -- 172.21.15.167:0/1273655909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6ac004d90 msgr2=0x7fd6ac005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:03.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.650+0000 7fd6be000640 1 --2- 172.21.15.167:0/1273655909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6ac004d90 0x7fd6ac005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:03.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.650+0000 7fd6be000640 1 -- 172.21.15.167:0/1273655909 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6ac003520 con 0x7fd6ac0040b0 2024-05-11T12:12:03.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.650+0000 7fd6c7400640 1 --2- 172.21.15.167:0/1273655909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6ac004d90 0x7fd6ac005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:03.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.651+0000 7fd6be000640 1 --2- 172.21.15.167:0/1273655909 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6ac0040b0 0x7fd6ac004800 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd6b000c490 tx=0x7fd6ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9459f998ecbdd4f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:03.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.651+0000 7fd6c6000640 1 -- 172.21.15.167:0/1273655909 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0035070 con 0x7fd6ac0040b0 2024-05-11T12:12:03.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.651+0000 7fd6c6000640 1 -- 172.21.15.167:0/1273655909 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6b0005ad0 con 0x7fd6ac0040b0 2024-05-11T12:12:03.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.651+0000 7fd6c6000640 1 -- 172.21.15.167:0/1273655909 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b000c0e0 con 0x7fd6ac0040b0 2024-05-11T12:12:03.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.651+0000 7fd6c7e00640 1 -- 172.21.15.167:0/1273655909 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6ac0040b0 msgr2=0x7fd6ac004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:03.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.651+0000 7fd6c7e00640 1 --2- 172.21.15.167:0/1273655909 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6ac0040b0 0x7fd6ac004800 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd6b000c490 tx=0x7fd6ac005830 comp rx=0 tx=0).stop 2024-05-11T12:12:03.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.651+0000 7fd6c7400640 1 -- 172.21.15.167:0/1273655909 reap_dead start 2024-05-11T12:12:03.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.652+0000 7fd6c7e00640 1 -- 172.21.15.167:0/1273655909 shutdown_connections 2024-05-11T12:12:03.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.652+0000 7fd6c7e00640 1 -- 172.21.15.167:0/1273655909 >> 172.21.15.167:0/1273655909 conn(0x7fd6c80b3460 msgr2=0x7fd6c80d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:03.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.652+0000 7fd6c7e00640 1 -- 172.21.15.167:0/1273655909 shutdown_connections 2024-05-11T12:12:03.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.652+0000 7fd6c7e00640 1 -- 172.21.15.167:0/1273655909 wait complete. 2024-05-11T12:12:03.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.653+0000 7fd6c7e00640 1 Processor -- start 2024-05-11T12:12:03.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.654+0000 7fd6c7e00640 1 -- start start 2024-05-11T12:12:03.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.654+0000 7fd6c7e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6ac0040b0 0x7fd6c8141f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:03.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.654+0000 7fd6c7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6c815d970 0x7fd6c815c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:03.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.655+0000 7fd6c7e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6c8072f80 0x7fd6c8073430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:03.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.655+0000 7fd6c7e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c8145ee0 con 0x7fd6c815d970 2024-05-11T12:12:03.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.655+0000 7fd6c7e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c80cf0e0 con 0x7fd6ac0040b0 2024-05-11T12:12:03.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.655+0000 7fd6c7e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd6c806ca70 con 0x7fd6c8072f80 2024-05-11T12:12:03.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.655+0000 7fd6be000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6c815d970 0x7fd6c815c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:03.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.655+0000 7fd6be000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6c815d970 0x7fd6c815c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41916/0 (socket says 172.21.15.167:41916) 2024-05-11T12:12:03.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.655+0000 7fd6be000640 1 -- 172.21.15.167:0/2839498518 learned_addr learned my addr 172.21.15.167:0/2839498518 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:03.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.655+0000 7fd6c7400640 1 --2- 172.21.15.167:0/2839498518 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6c8072f80 0x7fd6c8073430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:03.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.655+0000 7fd6c6a00640 1 --2- 172.21.15.167:0/2839498518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6ac0040b0 0x7fd6c8141f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:03.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.655+0000 7fd6be000640 1 -- 172.21.15.167:0/2839498518 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6c8072f80 msgr2=0x7fd6c8073430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:03.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.655+0000 7fd6be000640 1 --2- 172.21.15.167:0/2839498518 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6c8072f80 0x7fd6c8073430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:03.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.655+0000 7fd6be000640 1 -- 172.21.15.167:0/2839498518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6ac0040b0 msgr2=0x7fd6c8141f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:03.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.655+0000 7fd6be000640 1 --2- 172.21.15.167:0/2839498518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6ac0040b0 0x7fd6c8141f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:03.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.655+0000 7fd6be000640 1 -- 172.21.15.167:0/2839498518 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6c8143650 con 0x7fd6c815d970 2024-05-11T12:12:03.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.655+0000 7fd6c6a00640 1 --2- 172.21.15.167:0/2839498518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6ac0040b0 0x7fd6c8141f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:03.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.655+0000 7fd6c7400640 1 --2- 172.21.15.167:0/2839498518 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6c8072f80 0x7fd6c8073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:03.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.656+0000 7fd6be000640 1 --2- 172.21.15.167:0/2839498518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6c815d970 0x7fd6c815c580 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0033040 tx=0x7fd6c80b49d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:03.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.656+0000 7fd6bf400640 1 -- 172.21.15.167:0/2839498518 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0035040 con 0x7fd6c815d970 2024-05-11T12:12:03.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.656+0000 7fd6bf400640 1 -- 172.21.15.167:0/2839498518 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6b0040020 con 0x7fd6c815d970 2024-05-11T12:12:03.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.656+0000 7fd6bf400640 1 -- 172.21.15.167:0/2839498518 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b002c040 con 0x7fd6c815d970 2024-05-11T12:12:03.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.656+0000 7fd6c7e00640 1 -- 172.21.15.167:0/2839498518 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6c81430b0 con 0x7fd6c815d970 2024-05-11T12:12:03.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.656+0000 7fd6c7e00640 1 -- 172.21.15.167:0/2839498518 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6c8142af0 con 0x7fd6c815d970 2024-05-11T12:12:03.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.658+0000 7fd6c7e00640 1 -- 172.21.15.167:0/2839498518 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6c80694b0 con 0x7fd6c815d970 2024-05-11T12:12:03.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.659+0000 7fd6bf400640 1 -- 172.21.15.167:0/2839498518 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd6b00016c0 con 0x7fd6c815d970 2024-05-11T12:12:03.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.659+0000 7fd6bf400640 1 --2- 172.21.15.167:0/2839498518 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd6a40762d0 0x7fd6a4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:03.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.660+0000 7fd6bf400640 1 -- 172.21.15.167:0/2839498518 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7fd6b00b72c0 con 0x7fd6c815d970 2024-05-11T12:12:03.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.660+0000 7fd6c6a00640 1 --2- 172.21.15.167:0/2839498518 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd6a40762d0 0x7fd6a4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:03.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.660+0000 7fd6c6a00640 1 --2- 172.21.15.167:0/2839498518 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd6a40762d0 0x7fd6a4078790 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8001750 tx=0x7fd6b8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:03.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.664+0000 7fd6bf400640 1 -- 172.21.15.167:0/2839498518 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd6c806cfb0 con 0x7fd6c815d970 2024-05-11T12:12:03.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.868+0000 7fd6c7e00640 1 -- 172.21.15.167:0/2839498518 --> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd6c80b3960 con 0x7fd6a40762d0 2024-05-11T12:12:03.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.878+0000 7fd6bf400640 1 -- 172.21.15.167:0/2839498518 <== mgr.4106 v2:172.21.15.167:6832/578352559 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251914 (secure 0 0 0) 0x7fd6c80b3960 con 0x7fd6a40762d0 2024-05-11T12:12:03.881 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:12:03.884 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2024-05-11T12:12:03.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.885+0000 7fd6c7e00640 1 -- 172.21.15.167:0/2839498518 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd6a40762d0 msgr2=0x7fd6a4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:03.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.885+0000 7fd6c7e00640 1 --2- 172.21.15.167:0/2839498518 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd6a40762d0 0x7fd6a4078790 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8001750 tx=0x7fd6b8009040 comp rx=0 tx=0).stop 2024-05-11T12:12:03.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.885+0000 7fd6c7e00640 1 -- 172.21.15.167:0/2839498518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6c815d970 msgr2=0x7fd6c815c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:03.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.885+0000 7fd6c7e00640 1 --2- 172.21.15.167:0/2839498518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6c815d970 0x7fd6c815c580 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0033040 tx=0x7fd6c80b49d0 comp rx=0 tx=0).stop 2024-05-11T12:12:03.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.886+0000 7fd6c7e00640 1 -- 172.21.15.167:0/2839498518 shutdown_connections 2024-05-11T12:12:03.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.886+0000 7fd6c7e00640 1 --2- 172.21.15.167:0/2839498518 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6c8072f80 0x7fd6c8073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:03.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.886+0000 7fd6c7e00640 1 --2- 172.21.15.167:0/2839498518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6c815d970 0x7fd6c815c580 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:03.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.886+0000 7fd6c7e00640 1 --2- 172.21.15.167:0/2839498518 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd6a40762d0 0x7fd6a4078790 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:03.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.886+0000 7fd6c7e00640 1 --2- 172.21.15.167:0/2839498518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6ac0040b0 0x7fd6c8141f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:03.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.886+0000 7fd6c7e00640 1 -- 172.21.15.167:0/2839498518 >> 172.21.15.167:0/2839498518 conn(0x7fd6c80b3460 msgr2=0x7fd6c8146060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:03.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.886+0000 7fd6c7e00640 1 -- 172.21.15.167:0/2839498518 shutdown_connections 2024-05-11T12:12:03.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:03.886+0000 7fd6c7e00640 1 -- 172.21.15.167:0/2839498518 wait complete. 2024-05-11T12:12:03.904 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":80,"stamp":"2024-05-11T12:12:03.258878+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":183,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220984,"kb_used_data":4792,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753736,"statfs":{"total":773094113280,"available":772867825664,"internally_reserved":0,"allocated":4907008,"data_stored":2750284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":49,"apply_latency_ms":49,"commit_latency_ns":49000000,"apply_latency_ns":49000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.889132"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.430959+0000","last_change":"2024-05-11T12:11:57.428217+0000","last_active":"2024-05-11T12:11:58.430959+0000","last_peered":"2024-05-11T12:11:58.430959+0000","last_clean":"2024-05-11T12:11:58.430959+0000","last_became_active":"2024-05-11T12:11:57.425926+0000","last_became_peered":"2024-05-11T12:11:57.425926+0000","last_unstale":"2024-05-11T12:11:58.430959+0000","last_undegraded":"2024-05-11T12:11:58.430959+0000","last_fullsized":"2024-05-11T12:11:58.430959+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:46:47.695505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.430603+0000","last_change":"2024-05-11T12:11:57.430689+0000","last_active":"2024-05-11T12:11:58.430603+0000","last_peered":"2024-05-11T12:11:58.430603+0000","last_clean":"2024-05-11T12:11:58.430603+0000","last_became_active":"2024-05-11T12:11:57.430424+0000","last_became_peered":"2024-05-11T12:11:57.430424+0000","last_unstale":"2024-05-11T12:11:58.430603+0000","last_undegraded":"2024-05-11T12:11:58.430603+0000","last_fullsized":"2024-05-11T12:11:58.430603+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:32:11.650224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:59.749549+0000","last_change":"2024-05-11T12:11:57.432468+0000","last_active":"2024-05-11T12:11:59.749549+0000","last_peered":"2024-05-11T12:11:59.749549+0000","last_clean":"2024-05-11T12:11:59.749549+0000","last_became_active":"2024-05-11T12:11:57.432189+0000","last_became_peered":"2024-05-11T12:11:57.432189+0000","last_unstale":"2024-05-11T12:11:59.749549+0000","last_undegraded":"2024-05-11T12:11:59.749549+0000","last_fullsized":"2024-05-11T12:11:59.749549+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:55:35.468623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.436535+0000","last_change":"2024-05-11T12:11:57.433107+0000","last_active":"2024-05-11T12:11:58.436535+0000","last_peered":"2024-05-11T12:11:58.436535+0000","last_clean":"2024-05-11T12:11:58.436535+0000","last_became_active":"2024-05-11T12:11:57.432829+0000","last_became_peered":"2024-05-11T12:11:57.432829+0000","last_unstale":"2024-05-11T12:11:58.436535+0000","last_undegraded":"2024-05-11T12:11:58.436535+0000","last_fullsized":"2024-05-11T12:11:58.436535+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:31:31.558337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.435925+0000","last_change":"2024-05-11T12:11:57.436577+0000","last_active":"2024-05-11T12:11:58.435925+0000","last_peered":"2024-05-11T12:11:58.435925+0000","last_clean":"2024-05-11T12:11:58.435925+0000","last_became_active":"2024-05-11T12:11:57.436245+0000","last_became_peered":"2024-05-11T12:11:57.436245+0000","last_unstale":"2024-05-11T12:11:58.435925+0000","last_undegraded":"2024-05-11T12:11:58.435925+0000","last_fullsized":"2024-05-11T12:11:58.435925+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:20:16.274615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.432396+0000","last_change":"2024-05-11T12:11:57.429489+0000","last_active":"2024-05-11T12:11:58.432396+0000","last_peered":"2024-05-11T12:11:58.432396+0000","last_clean":"2024-05-11T12:11:58.432396+0000","last_became_active":"2024-05-11T12:11:57.429216+0000","last_became_peered":"2024-05-11T12:11:57.429216+0000","last_unstale":"2024-05-11T12:11:58.432396+0000","last_undegraded":"2024-05-11T12:11:58.432396+0000","last_fullsized":"2024-05-11T12:11:58.432396+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:26:34.393703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.437906+0000","last_change":"2024-05-11T12:11:57.432877+0000","last_active":"2024-05-11T12:11:57.437906+0000","last_peered":"2024-05-11T12:11:57.437906+0000","last_clean":"2024-05-11T12:11:57.437906+0000","last_became_active":"2024-05-11T12:11:57.432581+0000","last_became_peered":"2024-05-11T12:11:57.432581+0000","last_unstale":"2024-05-11T12:11:57.437906+0000","last_undegraded":"2024-05-11T12:11:57.437906+0000","last_fullsized":"2024-05-11T12:11:57.437906+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:47:49.623473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.428593+0000","last_change":"2024-05-11T12:11:57.417730+0000","last_active":"2024-05-11T12:11:58.428593+0000","last_peered":"2024-05-11T12:11:58.428593+0000","last_clean":"2024-05-11T12:11:58.428593+0000","last_became_active":"2024-05-11T12:11:57.413336+0000","last_became_peered":"2024-05-11T12:11:57.413336+0000","last_unstale":"2024-05-11T12:11:58.428593+0000","last_undegraded":"2024-05-11T12:11:58.428593+0000","last_fullsized":"2024-05-11T12:11:58.428593+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:36:05.818955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:59.750283+0000","last_change":"2024-05-11T12:11:57.433333+0000","last_active":"2024-05-11T12:11:59.750283+0000","last_peered":"2024-05-11T12:11:59.750283+0000","last_clean":"2024-05-11T12:11:59.750283+0000","last_became_active":"2024-05-11T12:11:57.433046+0000","last_became_peered":"2024-05-11T12:11:57.433046+0000","last_unstale":"2024-05-11T12:11:59.750283+0000","last_undegraded":"2024-05-11T12:11:59.750283+0000","last_fullsized":"2024-05-11T12:11:59.750283+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:43:20.345625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.439811+0000","last_change":"2024-05-11T12:11:57.434958+0000","last_active":"2024-05-11T12:11:57.439811+0000","last_peered":"2024-05-11T12:11:57.439811+0000","last_clean":"2024-05-11T12:11:57.439811+0000","last_became_active":"2024-05-11T12:11:57.429221+0000","last_became_peered":"2024-05-11T12:11:57.429221+0000","last_unstale":"2024-05-11T12:11:57.439811+0000","last_undegraded":"2024-05-11T12:11:57.439811+0000","last_fullsized":"2024-05-11T12:11:57.439811+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:32:26.804013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.455023+0000","last_change":"2024-05-11T12:11:57.449552+0000","last_active":"2024-05-11T12:11:57.455023+0000","last_peered":"2024-05-11T12:11:57.455023+0000","last_clean":"2024-05-11T12:11:57.455023+0000","last_became_active":"2024-05-11T12:11:57.441862+0000","last_became_peered":"2024-05-11T12:11:57.441862+0000","last_unstale":"2024-05-11T12:11:57.455023+0000","last_undegraded":"2024-05-11T12:11:57.455023+0000","last_fullsized":"2024-05-11T12:11:57.455023+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:04:26.168833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:59.749971+0000","last_change":"2024-05-11T12:11:57.431974+0000","last_active":"2024-05-11T12:11:59.749971+0000","last_peered":"2024-05-11T12:11:59.749971+0000","last_clean":"2024-05-11T12:11:59.749971+0000","last_became_active":"2024-05-11T12:11:57.431619+0000","last_became_peered":"2024-05-11T12:11:57.431619+0000","last_unstale":"2024-05-11T12:11:59.749971+0000","last_undegraded":"2024-05-11T12:11:59.749971+0000","last_fullsized":"2024-05-11T12:11:59.749971+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:28:33.430453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.432544+0000","last_change":"2024-05-11T12:11:57.429706+0000","last_active":"2024-05-11T12:11:57.432544+0000","last_peered":"2024-05-11T12:11:57.432544+0000","last_clean":"2024-05-11T12:11:57.432544+0000","last_became_active":"2024-05-11T12:11:57.429387+0000","last_became_peered":"2024-05-11T12:11:57.429387+0000","last_unstale":"2024-05-11T12:11:57.432544+0000","last_undegraded":"2024-05-11T12:11:57.432544+0000","last_fullsized":"2024-05-11T12:11:57.432544+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:55:35.468623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.445013+0000","last_change":"2024-05-11T12:11:57.430917+0000","last_active":"2024-05-11T12:11:57.445013+0000","last_peered":"2024-05-11T12:11:57.445013+0000","last_clean":"2024-05-11T12:11:57.445013+0000","last_became_active":"2024-05-11T12:11:57.430613+0000","last_became_peered":"2024-05-11T12:11:57.430613+0000","last_unstale":"2024-05-11T12:11:57.445013+0000","last_undegraded":"2024-05-11T12:11:57.445013+0000","last_fullsized":"2024-05-11T12:11:57.445013+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:25:16.350527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.454620+0000","last_change":"2024-05-11T12:11:57.438016+0000","last_active":"2024-05-11T12:11:57.454620+0000","last_peered":"2024-05-11T12:11:57.454620+0000","last_clean":"2024-05-11T12:11:57.454620+0000","last_became_active":"2024-05-11T12:11:57.437680+0000","last_became_peered":"2024-05-11T12:11:57.437680+0000","last_unstale":"2024-05-11T12:11:57.454620+0000","last_undegraded":"2024-05-11T12:11:57.454620+0000","last_fullsized":"2024-05-11T12:11:57.454620+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:28:33.430453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.453506+0000","last_change":"2024-05-11T12:11:57.448672+0000","last_active":"2024-05-11T12:11:57.453506+0000","last_peered":"2024-05-11T12:11:57.453506+0000","last_clean":"2024-05-11T12:11:57.453506+0000","last_became_active":"2024-05-11T12:11:57.447489+0000","last_became_peered":"2024-05-11T12:11:57.447489+0000","last_unstale":"2024-05-11T12:11:57.453506+0000","last_undegraded":"2024-05-11T12:11:57.453506+0000","last_fullsized":"2024-05-11T12:11:57.453506+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:53:03.480916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.451781+0000","last_change":"2024-05-11T12:11:57.447831+0000","last_active":"2024-05-11T12:11:57.451781+0000","last_peered":"2024-05-11T12:11:57.451781+0000","last_clean":"2024-05-11T12:11:57.451781+0000","last_became_active":"2024-05-11T12:11:57.443540+0000","last_became_peered":"2024-05-11T12:11:57.443540+0000","last_unstale":"2024-05-11T12:11:57.451781+0000","last_undegraded":"2024-05-11T12:11:57.451781+0000","last_fullsized":"2024-05-11T12:11:57.451781+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:26:34.393703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.435968+0000","last_change":"2024-05-11T12:11:57.431125+0000","last_active":"2024-05-11T12:11:57.435968+0000","last_peered":"2024-05-11T12:11:57.435968+0000","last_clean":"2024-05-11T12:11:57.435968+0000","last_became_active":"2024-05-11T12:11:57.430835+0000","last_became_peered":"2024-05-11T12:11:57.430835+0000","last_unstale":"2024-05-11T12:11:57.435968+0000","last_undegraded":"2024-05-11T12:11:57.435968+0000","last_fullsized":"2024-05-11T12:11:57.435968+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:23:28.626421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.436454+0000","last_change":"2024-05-11T12:11:57.438506+0000","last_active":"2024-05-11T12:11:58.436454+0000","last_peered":"2024-05-11T12:11:58.436454+0000","last_clean":"2024-05-11T12:11:58.436454+0000","last_became_active":"2024-05-11T12:11:57.437706+0000","last_became_peered":"2024-05-11T12:11:57.437706+0000","last_unstale":"2024-05-11T12:11:58.436454+0000","last_undegraded":"2024-05-11T12:11:58.436454+0000","last_fullsized":"2024-05-11T12:11:58.436454+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:05:23.821514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.436449+0000","last_change":"2024-05-11T12:11:57.433084+0000","last_active":"2024-05-11T12:11:57.436449+0000","last_peered":"2024-05-11T12:11:57.436449+0000","last_clean":"2024-05-11T12:11:57.436449+0000","last_became_active":"2024-05-11T12:11:57.432645+0000","last_became_peered":"2024-05-11T12:11:57.432645+0000","last_unstale":"2024-05-11T12:11:57.436449+0000","last_undegraded":"2024-05-11T12:11:57.436449+0000","last_fullsized":"2024-05-11T12:11:57.436449+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:32:45.644260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.439852+0000","last_change":"2024-05-11T12:11:57.434801+0000","last_active":"2024-05-11T12:11:57.439852+0000","last_peered":"2024-05-11T12:11:57.439852+0000","last_clean":"2024-05-11T12:11:57.439852+0000","last_became_active":"2024-05-11T12:11:57.433969+0000","last_became_peered":"2024-05-11T12:11:57.433969+0000","last_unstale":"2024-05-11T12:11:57.439852+0000","last_undegraded":"2024-05-11T12:11:57.439852+0000","last_fullsized":"2024-05-11T12:11:57.439852+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:22:33.211172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.449144+0000","last_change":"2024-05-11T12:11:57.445771+0000","last_active":"2024-05-11T12:11:57.449144+0000","last_peered":"2024-05-11T12:11:57.449144+0000","last_clean":"2024-05-11T12:11:57.449144+0000","last_became_active":"2024-05-11T12:11:57.445474+0000","last_became_peered":"2024-05-11T12:11:57.445474+0000","last_unstale":"2024-05-11T12:11:57.449144+0000","last_undegraded":"2024-05-11T12:11:57.449144+0000","last_fullsized":"2024-05-11T12:11:57.449144+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:23:28.626421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.451922+0000","last_change":"2024-05-11T12:11:57.447508+0000","last_active":"2024-05-11T12:11:57.451922+0000","last_peered":"2024-05-11T12:11:57.451922+0000","last_clean":"2024-05-11T12:11:57.451922+0000","last_became_active":"2024-05-11T12:11:57.444377+0000","last_became_peered":"2024-05-11T12:11:57.444377+0000","last_unstale":"2024-05-11T12:11:57.451922+0000","last_undegraded":"2024-05-11T12:11:57.451922+0000","last_fullsized":"2024-05-11T12:11:57.451922+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:46:06.174919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.435737+0000","last_change":"2024-05-11T12:11:57.437303+0000","last_active":"2024-05-11T12:11:58.435737+0000","last_peered":"2024-05-11T12:11:58.435737+0000","last_clean":"2024-05-11T12:11:58.435737+0000","last_became_active":"2024-05-11T12:11:57.436970+0000","last_became_peered":"2024-05-11T12:11:57.436970+0000","last_unstale":"2024-05-11T12:11:58.435737+0000","last_undegraded":"2024-05-11T12:11:58.435737+0000","last_fullsized":"2024-05-11T12:11:58.435737+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:58:38.525115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.430674+0000","last_change":"2024-05-11T12:11:57.426462+0000","last_active":"2024-05-11T12:11:57.430674+0000","last_peered":"2024-05-11T12:11:57.430674+0000","last_clean":"2024-05-11T12:11:57.430674+0000","last_became_active":"2024-05-11T12:11:57.426087+0000","last_became_peered":"2024-05-11T12:11:57.426087+0000","last_unstale":"2024-05-11T12:11:57.430674+0000","last_undegraded":"2024-05-11T12:11:57.430674+0000","last_fullsized":"2024-05-11T12:11:57.430674+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:30:57.830893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.433780+0000","last_change":"2024-05-11T12:11:57.434107+0000","last_active":"2024-05-11T12:11:58.433780+0000","last_peered":"2024-05-11T12:11:58.433780+0000","last_clean":"2024-05-11T12:11:58.433780+0000","last_became_active":"2024-05-11T12:11:57.433812+0000","last_became_peered":"2024-05-11T12:11:57.433812+0000","last_unstale":"2024-05-11T12:11:58.433780+0000","last_undegraded":"2024-05-11T12:11:58.433780+0000","last_fullsized":"2024-05-11T12:11:58.433780+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:17:59.309336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.432318+0000","last_change":"2024-05-11T12:11:57.428594+0000","last_active":"2024-05-11T12:11:57.432318+0000","last_peered":"2024-05-11T12:11:57.432318+0000","last_clean":"2024-05-11T12:11:57.432318+0000","last_became_active":"2024-05-11T12:11:57.428245+0000","last_became_peered":"2024-05-11T12:11:57.428245+0000","last_unstale":"2024-05-11T12:11:57.432318+0000","last_undegraded":"2024-05-11T12:11:57.432318+0000","last_fullsized":"2024-05-11T12:11:57.432318+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:28:44.586634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.430392+0000","last_change":"2024-05-11T12:11:57.417635+0000","last_active":"2024-05-11T12:11:57.430392+0000","last_peered":"2024-05-11T12:11:57.430392+0000","last_clean":"2024-05-11T12:11:57.430392+0000","last_became_active":"2024-05-11T12:11:57.415536+0000","last_became_peered":"2024-05-11T12:11:57.415536+0000","last_unstale":"2024-05-11T12:11:57.430392+0000","last_undegraded":"2024-05-11T12:11:57.430392+0000","last_fullsized":"2024-05-11T12:11:57.430392+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:51:43.649698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.432559+0000","last_change":"2024-05-11T12:11:57.427847+0000","last_active":"2024-05-11T12:11:57.432559+0000","last_peered":"2024-05-11T12:11:57.432559+0000","last_clean":"2024-05-11T12:11:57.432559+0000","last_became_active":"2024-05-11T12:11:57.427416+0000","last_became_peered":"2024-05-11T12:11:57.427416+0000","last_unstale":"2024-05-11T12:11:57.432559+0000","last_undegraded":"2024-05-11T12:11:57.432559+0000","last_fullsized":"2024-05-11T12:11:57.432559+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:07:22.774535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.423604+0000","last_change":"2024-05-11T12:11:57.420465+0000","last_active":"2024-05-11T12:11:57.423604+0000","last_peered":"2024-05-11T12:11:57.423604+0000","last_clean":"2024-05-11T12:11:57.423604+0000","last_became_active":"2024-05-11T12:11:57.419963+0000","last_became_peered":"2024-05-11T12:11:57.419963+0000","last_unstale":"2024-05-11T12:11:57.423604+0000","last_undegraded":"2024-05-11T12:11:57.423604+0000","last_fullsized":"2024-05-11T12:11:57.423604+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:36:41.799582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.451900+0000","last_change":"2024-05-11T12:11:57.447209+0000","last_active":"2024-05-11T12:11:57.451900+0000","last_peered":"2024-05-11T12:11:57.451900+0000","last_clean":"2024-05-11T12:11:57.451900+0000","last_became_active":"2024-05-11T12:11:57.443801+0000","last_became_peered":"2024-05-11T12:11:57.443801+0000","last_unstale":"2024-05-11T12:11:57.451900+0000","last_undegraded":"2024-05-11T12:11:57.451900+0000","last_fullsized":"2024-05-11T12:11:57.451900+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:10:05.034718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.432358+0000","last_change":"2024-05-11T12:11:57.436143+0000","last_active":"2024-05-11T12:11:58.432358+0000","last_peered":"2024-05-11T12:11:58.432358+0000","last_clean":"2024-05-11T12:11:58.432358+0000","last_became_active":"2024-05-11T12:11:57.435874+0000","last_became_peered":"2024-05-11T12:11:57.435874+0000","last_unstale":"2024-05-11T12:11:58.432358+0000","last_undegraded":"2024-05-11T12:11:58.432358+0000","last_fullsized":"2024-05-11T12:11:58.432358+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:10:05.034718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.451196+0000","last_change":"2024-05-11T12:11:57.444554+0000","last_active":"2024-05-11T12:11:57.451196+0000","last_peered":"2024-05-11T12:11:57.451196+0000","last_clean":"2024-05-11T12:11:57.451196+0000","last_became_active":"2024-05-11T12:11:57.443836+0000","last_became_peered":"2024-05-11T12:11:57.443836+0000","last_unstale":"2024-05-11T12:11:57.451196+0000","last_undegraded":"2024-05-11T12:11:57.451196+0000","last_fullsized":"2024-05-11T12:11:57.451196+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:20:14.138435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"40'4","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.451553+0000","last_change":"2024-05-11T12:11:56.409094+0000","last_active":"2024-05-11T12:11:57.451553+0000","last_peered":"2024-05-11T12:11:57.451553+0000","last_clean":"2024-05-11T12:11:57.451553+0000","last_became_active":"2024-05-11T12:11:56.408820+0000","last_became_peered":"2024-05-11T12:11:56.408820+0000","last_unstale":"2024-05-11T12:11:57.451553+0000","last_undegraded":"2024-05-11T12:11:57.451553+0000","last_fullsized":"2024-05-11T12:11:57.451553+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:11:42.632883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.433251+0000","last_change":"2024-05-11T12:11:57.436017+0000","last_active":"2024-05-11T12:11:58.433251+0000","last_peered":"2024-05-11T12:11:58.433251+0000","last_clean":"2024-05-11T12:11:58.433251+0000","last_became_active":"2024-05-11T12:11:57.435070+0000","last_became_peered":"2024-05-11T12:11:57.435070+0000","last_unstale":"2024-05-11T12:11:58.433251+0000","last_undegraded":"2024-05-11T12:11:58.433251+0000","last_fullsized":"2024-05-11T12:11:58.433251+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:51:43.649698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"40'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:59.749979+0000","last_change":"2024-05-11T12:11:56.403002+0000","last_active":"2024-05-11T12:11:59.749979+0000","last_peered":"2024-05-11T12:11:59.749979+0000","last_clean":"2024-05-11T12:11:59.749979+0000","last_became_active":"2024-05-11T12:11:56.402721+0000","last_became_peered":"2024-05-11T12:11:56.402721+0000","last_unstale":"2024-05-11T12:11:59.749979+0000","last_undegraded":"2024-05-11T12:11:59.749979+0000","last_fullsized":"2024-05-11T12:11:59.749979+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:23:27.612044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.450328+0000","last_change":"2024-05-11T12:11:57.435281+0000","last_active":"2024-05-11T12:11:57.450328+0000","last_peered":"2024-05-11T12:11:57.450328+0000","last_clean":"2024-05-11T12:11:57.450328+0000","last_became_active":"2024-05-11T12:11:57.434672+0000","last_became_peered":"2024-05-11T12:11:57.434672+0000","last_unstale":"2024-05-11T12:11:57.450328+0000","last_undegraded":"2024-05-11T12:11:57.450328+0000","last_fullsized":"2024-05-11T12:11:57.450328+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:30:57.830893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.433425+0000","last_change":"2024-05-11T12:11:56.410313+0000","last_active":"2024-05-11T12:11:58.433425+0000","last_peered":"2024-05-11T12:11:58.433425+0000","last_clean":"2024-05-11T12:11:58.433425+0000","last_became_active":"2024-05-11T12:11:56.409971+0000","last_became_peered":"2024-05-11T12:11:56.409971+0000","last_unstale":"2024-05-11T12:11:58.433425+0000","last_undegraded":"2024-05-11T12:11:58.433425+0000","last_fullsized":"2024-05-11T12:11:58.433425+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:07:21.760158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.430958+0000","last_change":"2024-05-11T12:11:57.419505+0000","last_active":"2024-05-11T12:11:58.430958+0000","last_peered":"2024-05-11T12:11:58.430958+0000","last_clean":"2024-05-11T12:11:58.430958+0000","last_became_active":"2024-05-11T12:11:57.418984+0000","last_became_peered":"2024-05-11T12:11:57.418984+0000","last_unstale":"2024-05-11T12:11:58.430958+0000","last_undegraded":"2024-05-11T12:11:58.430958+0000","last_fullsized":"2024-05-11T12:11:58.430958+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:07:26.576379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"40'2","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.454119+0000","last_change":"2024-05-11T12:11:56.408353+0000","last_active":"2024-05-11T12:11:57.454119+0000","last_peered":"2024-05-11T12:11:57.454119+0000","last_clean":"2024-05-11T12:11:57.454119+0000","last_became_active":"2024-05-11T12:11:56.408078+0000","last_became_peered":"2024-05-11T12:11:56.408078+0000","last_unstale":"2024-05-11T12:11:57.454119+0000","last_undegraded":"2024-05-11T12:11:57.454119+0000","last_fullsized":"2024-05-11T12:11:57.454119+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:16:00.825967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:59.749207+0000","last_change":"2024-05-11T12:11:57.438954+0000","last_active":"2024-05-11T12:11:59.749207+0000","last_peered":"2024-05-11T12:11:59.749207+0000","last_clean":"2024-05-11T12:11:59.749207+0000","last_became_active":"2024-05-11T12:11:57.437487+0000","last_became_peered":"2024-05-11T12:11:57.437487+0000","last_unstale":"2024-05-11T12:11:59.749207+0000","last_undegraded":"2024-05-11T12:11:59.749207+0000","last_fullsized":"2024-05-11T12:11:59.749207+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:17:59.309336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"40'1","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.455051+0000","last_change":"2024-05-11T12:11:56.404989+0000","last_active":"2024-05-11T12:11:57.455051+0000","last_peered":"2024-05-11T12:11:57.455051+0000","last_clean":"2024-05-11T12:11:57.455051+0000","last_became_active":"2024-05-11T12:11:56.404069+0000","last_became_peered":"2024-05-11T12:11:56.404069+0000","last_unstale":"2024-05-11T12:11:57.455051+0000","last_undegraded":"2024-05-11T12:11:57.455051+0000","last_fullsized":"2024-05-11T12:11:57.455051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:26:33.379326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.454982+0000","last_change":"2024-05-11T12:11:57.442633+0000","last_active":"2024-05-11T12:11:57.454982+0000","last_peered":"2024-05-11T12:11:57.454982+0000","last_clean":"2024-05-11T12:11:57.454982+0000","last_became_active":"2024-05-11T12:11:57.442361+0000","last_became_peered":"2024-05-11T12:11:57.442361+0000","last_unstale":"2024-05-11T12:11:57.454982+0000","last_undegraded":"2024-05-11T12:11:57.454982+0000","last_fullsized":"2024-05-11T12:11:57.454982+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:43:20.345625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"40'1","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.451754+0000","last_change":"2024-05-11T12:11:56.411689+0000","last_active":"2024-05-11T12:11:57.451754+0000","last_peered":"2024-05-11T12:11:57.451754+0000","last_clean":"2024-05-11T12:11:57.451754+0000","last_became_active":"2024-05-11T12:11:56.409937+0000","last_became_peered":"2024-05-11T12:11:56.409937+0000","last_unstale":"2024-05-11T12:11:57.451754+0000","last_undegraded":"2024-05-11T12:11:57.451754+0000","last_fullsized":"2024-05-11T12:11:57.451754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:36:40.785205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.448085+0000","last_change":"2024-05-11T12:11:57.435197+0000","last_active":"2024-05-11T12:11:57.448085+0000","last_peered":"2024-05-11T12:11:57.448085+0000","last_clean":"2024-05-11T12:11:57.448085+0000","last_became_active":"2024-05-11T12:11:57.434472+0000","last_became_peered":"2024-05-11T12:11:57.434472+0000","last_unstale":"2024-05-11T12:11:57.448085+0000","last_undegraded":"2024-05-11T12:11:57.448085+0000","last_fullsized":"2024-05-11T12:11:57.448085+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:36:41.799582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.432452+0000","last_change":"2024-05-11T12:11:56.409731+0000","last_active":"2024-05-11T12:11:58.432452+0000","last_peered":"2024-05-11T12:11:58.432452+0000","last_clean":"2024-05-11T12:11:58.432452+0000","last_became_active":"2024-05-11T12:11:56.409450+0000","last_became_peered":"2024-05-11T12:11:56.409450+0000","last_unstale":"2024-05-11T12:11:58.432452+0000","last_undegraded":"2024-05-11T12:11:58.432452+0000","last_fullsized":"2024-05-11T12:11:58.432452+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:25:15.336149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.429278+0000","last_change":"2024-05-11T12:11:57.433051+0000","last_active":"2024-05-11T12:11:58.429278+0000","last_peered":"2024-05-11T12:11:58.429278+0000","last_clean":"2024-05-11T12:11:58.429278+0000","last_became_active":"2024-05-11T12:11:57.432381+0000","last_became_peered":"2024-05-11T12:11:57.432381+0000","last_unstale":"2024-05-11T12:11:58.429278+0000","last_undegraded":"2024-05-11T12:11:58.429278+0000","last_fullsized":"2024-05-11T12:11:58.429278+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:03:20.730869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.417407+0000","last_change":"2024-05-11T12:11:56.406587+0000","last_active":"2024-05-11T12:11:57.417407+0000","last_peered":"2024-05-11T12:11:57.417407+0000","last_clean":"2024-05-11T12:11:57.417407+0000","last_became_active":"2024-05-11T12:11:56.405746+0000","last_became_peered":"2024-05-11T12:11:56.405746+0000","last_unstale":"2024-05-11T12:11:57.417407+0000","last_undegraded":"2024-05-11T12:11:57.417407+0000","last_fullsized":"2024-05-11T12:11:57.417407+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:11:10.100041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.436332+0000","last_change":"2024-05-11T12:11:57.438714+0000","last_active":"2024-05-11T12:11:58.436332+0000","last_peered":"2024-05-11T12:11:58.436332+0000","last_clean":"2024-05-11T12:11:58.436332+0000","last_became_active":"2024-05-11T12:11:57.437691+0000","last_became_peered":"2024-05-11T12:11:57.437691+0000","last_unstale":"2024-05-11T12:11:58.436332+0000","last_undegraded":"2024-05-11T12:11:58.436332+0000","last_fullsized":"2024-05-11T12:11:58.436332+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:42:57.665986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.431241+0000","last_change":"2024-05-11T12:11:56.398172+0000","last_active":"2024-05-11T12:11:58.431241+0000","last_peered":"2024-05-11T12:11:58.431241+0000","last_clean":"2024-05-11T12:11:58.431241+0000","last_became_active":"2024-05-11T12:11:56.397860+0000","last_became_peered":"2024-05-11T12:11:56.397860+0000","last_unstale":"2024-05-11T12:11:58.431241+0000","last_undegraded":"2024-05-11T12:11:58.431241+0000","last_fullsized":"2024-05-11T12:11:58.431241+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:51:42.635321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.450935+0000","last_change":"2024-05-11T12:11:57.442952+0000","last_active":"2024-05-11T12:11:57.450935+0000","last_peered":"2024-05-11T12:11:57.450935+0000","last_clean":"2024-05-11T12:11:57.450935+0000","last_became_active":"2024-05-11T12:11:57.436880+0000","last_became_peered":"2024-05-11T12:11:57.436880+0000","last_unstale":"2024-05-11T12:11:57.450935+0000","last_undegraded":"2024-05-11T12:11:57.450935+0000","last_fullsized":"2024-05-11T12:11:57.450935+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:23:28.626421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.405805+0000","last_change":"2024-05-11T12:11:56.411981+0000","last_active":"2024-05-11T12:11:57.405805+0000","last_peered":"2024-05-11T12:11:57.405805+0000","last_clean":"2024-05-11T12:11:57.405805+0000","last_became_active":"2024-05-11T12:11:56.409613+0000","last_became_peered":"2024-05-11T12:11:56.409613+0000","last_unstale":"2024-05-11T12:11:57.405805+0000","last_undegraded":"2024-05-11T12:11:57.405805+0000","last_fullsized":"2024-05-11T12:11:57.405805+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:46:05.160542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:59.749748+0000","last_change":"2024-05-11T12:11:57.429655+0000","last_active":"2024-05-11T12:11:59.749748+0000","last_peered":"2024-05-11T12:11:59.749748+0000","last_clean":"2024-05-11T12:11:59.749748+0000","last_became_active":"2024-05-11T12:11:57.429394+0000","last_became_peered":"2024-05-11T12:11:57.429394+0000","last_unstale":"2024-05-11T12:11:59.749748+0000","last_undegraded":"2024-05-11T12:11:59.749748+0000","last_fullsized":"2024-05-11T12:11:59.749748+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:32:26.804013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"40'3","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.456654+0000","last_change":"2024-05-11T12:11:56.407591+0000","last_active":"2024-05-11T12:11:57.456654+0000","last_peered":"2024-05-11T12:11:57.456654+0000","last_clean":"2024-05-11T12:11:57.456654+0000","last_became_active":"2024-05-11T12:11:56.407173+0000","last_became_peered":"2024-05-11T12:11:56.407173+0000","last_unstale":"2024-05-11T12:11:57.456654+0000","last_undegraded":"2024-05-11T12:11:57.456654+0000","last_fullsized":"2024-05-11T12:11:57.456654+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:52:13.234125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.450953+0000","last_change":"2024-05-11T12:11:57.447257+0000","last_active":"2024-05-11T12:11:57.450953+0000","last_peered":"2024-05-11T12:11:57.450953+0000","last_clean":"2024-05-11T12:11:57.450953+0000","last_became_active":"2024-05-11T12:11:57.445351+0000","last_became_peered":"2024-05-11T12:11:57.445351+0000","last_unstale":"2024-05-11T12:11:57.450953+0000","last_undegraded":"2024-05-11T12:11:57.450953+0000","last_fullsized":"2024-05-11T12:11:57.450953+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:32:45.644260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"40'6","reported_seq":20,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.430111+0000","last_change":"2024-05-11T12:11:56.394318+0000","last_active":"2024-05-11T12:11:58.430111+0000","last_peered":"2024-05-11T12:11:58.430111+0000","last_clean":"2024-05-11T12:11:58.430111+0000","last_became_active":"2024-05-11T12:11:56.393560+0000","last_became_peered":"2024-05-11T12:11:56.393560+0000","last_unstale":"2024-05-11T12:11:58.430111+0000","last_undegraded":"2024-05-11T12:11:58.430111+0000","last_fullsized":"2024-05-11T12:11:58.430111+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:26:33.379326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:59.749673+0000","last_change":"2024-05-11T12:11:57.440633+0000","last_active":"2024-05-11T12:11:59.749673+0000","last_peered":"2024-05-11T12:11:59.749673+0000","last_clean":"2024-05-11T12:11:59.749673+0000","last_became_active":"2024-05-11T12:11:57.435381+0000","last_became_peered":"2024-05-11T12:11:57.435381+0000","last_unstale":"2024-05-11T12:11:59.749673+0000","last_undegraded":"2024-05-11T12:11:59.749673+0000","last_fullsized":"2024-05-11T12:11:59.749673+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:47:49.623473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.429956+0000","last_change":"2024-05-11T12:11:56.397825+0000","last_active":"2024-05-11T12:11:58.429956+0000","last_peered":"2024-05-11T12:11:58.429956+0000","last_clean":"2024-05-11T12:11:58.429956+0000","last_became_active":"2024-05-11T12:11:56.397506+0000","last_became_peered":"2024-05-11T12:11:56.397506+0000","last_unstale":"2024-05-11T12:11:58.429956+0000","last_undegraded":"2024-05-11T12:11:58.429956+0000","last_fullsized":"2024-05-11T12:11:58.429956+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:53:02.466538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.429706+0000","last_change":"2024-05-11T12:11:57.426061+0000","last_active":"2024-05-11T12:11:58.429706+0000","last_peered":"2024-05-11T12:11:58.429706+0000","last_clean":"2024-05-11T12:11:58.429706+0000","last_became_active":"2024-05-11T12:11:57.425702+0000","last_became_peered":"2024-05-11T12:11:57.425702+0000","last_unstale":"2024-05-11T12:11:58.429706+0000","last_undegraded":"2024-05-11T12:11:58.429706+0000","last_fullsized":"2024-05-11T12:11:58.429706+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:27:58.937922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.432528+0000","last_change":"2024-05-11T12:11:56.409815+0000","last_active":"2024-05-11T12:11:58.432528+0000","last_peered":"2024-05-11T12:11:58.432528+0000","last_clean":"2024-05-11T12:11:58.432528+0000","last_became_active":"2024-05-11T12:11:56.409411+0000","last_became_peered":"2024-05-11T12:11:56.409411+0000","last_unstale":"2024-05-11T12:11:58.432528+0000","last_undegraded":"2024-05-11T12:11:58.432528+0000","last_fullsized":"2024-05-11T12:11:58.432528+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:36:40.785205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.450828+0000","last_change":"2024-05-11T12:11:57.442669+0000","last_active":"2024-05-11T12:11:57.450828+0000","last_peered":"2024-05-11T12:11:57.450828+0000","last_clean":"2024-05-11T12:11:57.450828+0000","last_became_active":"2024-05-11T12:11:57.437559+0000","last_became_peered":"2024-05-11T12:11:57.437559+0000","last_unstale":"2024-05-11T12:11:57.450828+0000","last_undegraded":"2024-05-11T12:11:57.450828+0000","last_fullsized":"2024-05-11T12:11:57.450828+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:28:44.586634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.405638+0000","last_change":"2024-05-11T12:11:56.408411+0000","last_active":"2024-05-11T12:11:57.405638+0000","last_peered":"2024-05-11T12:11:57.405638+0000","last_clean":"2024-05-11T12:11:57.405638+0000","last_became_active":"2024-05-11T12:11:56.408100+0000","last_became_peered":"2024-05-11T12:11:56.408100+0000","last_unstale":"2024-05-11T12:11:57.405638+0000","last_undegraded":"2024-05-11T12:11:57.405638+0000","last_fullsized":"2024-05-11T12:11:57.405638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:25:15.336149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.429718+0000","last_change":"2024-05-11T12:11:57.430055+0000","last_active":"2024-05-11T12:11:58.429718+0000","last_peered":"2024-05-11T12:11:58.429718+0000","last_clean":"2024-05-11T12:11:58.429718+0000","last_became_active":"2024-05-11T12:11:57.426614+0000","last_became_peered":"2024-05-11T12:11:57.426614+0000","last_unstale":"2024-05-11T12:11:58.429718+0000","last_undegraded":"2024-05-11T12:11:58.429718+0000","last_fullsized":"2024-05-11T12:11:58.429718+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:58:40.171928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"40'8","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.457343+0000","last_change":"2024-05-11T12:11:56.398517+0000","last_active":"2024-05-11T12:11:57.457343+0000","last_peered":"2024-05-11T12:11:57.457343+0000","last_clean":"2024-05-11T12:11:57.457343+0000","last_became_active":"2024-05-11T12:11:56.398204+0000","last_became_peered":"2024-05-11T12:11:56.398204+0000","last_unstale":"2024-05-11T12:11:57.457343+0000","last_undegraded":"2024-05-11T12:11:57.457343+0000","last_fullsized":"2024-05-11T12:11:57.457343+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:25:15.336149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.429946+0000","last_change":"2024-05-11T12:11:57.422076+0000","last_active":"2024-05-11T12:11:58.429946+0000","last_peered":"2024-05-11T12:11:58.429946+0000","last_clean":"2024-05-11T12:11:58.429946+0000","last_became_active":"2024-05-11T12:11:57.421535+0000","last_became_peered":"2024-05-11T12:11:57.421535+0000","last_unstale":"2024-05-11T12:11:58.429946+0000","last_undegraded":"2024-05-11T12:11:58.429946+0000","last_fullsized":"2024-05-11T12:11:58.429946+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:07:26.210928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"40'1","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.455210+0000","last_change":"2024-05-11T12:11:56.410975+0000","last_active":"2024-05-11T12:11:57.455210+0000","last_peered":"2024-05-11T12:11:57.455210+0000","last_clean":"2024-05-11T12:11:57.455210+0000","last_became_active":"2024-05-11T12:11:56.410634+0000","last_became_peered":"2024-05-11T12:11:56.410634+0000","last_unstale":"2024-05-11T12:11:57.455210+0000","last_undegraded":"2024-05-11T12:11:57.455210+0000","last_fullsized":"2024-05-11T12:11:57.455210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:10:04.020340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.433269+0000","last_change":"2024-05-11T12:11:57.428161+0000","last_active":"2024-05-11T12:11:57.433269+0000","last_peered":"2024-05-11T12:11:57.433269+0000","last_clean":"2024-05-11T12:11:57.433269+0000","last_became_active":"2024-05-11T12:11:57.427826+0000","last_became_peered":"2024-05-11T12:11:57.427826+0000","last_unstale":"2024-05-11T12:11:57.433269+0000","last_undegraded":"2024-05-11T12:11:57.433269+0000","last_fullsized":"2024-05-11T12:11:57.433269+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:11:37.571584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"40'2","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.459816+0000","last_change":"2024-05-11T12:11:56.406473+0000","last_active":"2024-05-11T12:11:57.459816+0000","last_peered":"2024-05-11T12:11:57.459816+0000","last_clean":"2024-05-11T12:11:57.459816+0000","last_became_active":"2024-05-11T12:11:56.405923+0000","last_became_peered":"2024-05-11T12:11:56.405923+0000","last_unstale":"2024-05-11T12:11:57.459816+0000","last_undegraded":"2024-05-11T12:11:57.459816+0000","last_fullsized":"2024-05-11T12:11:57.459816+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:11:42.632883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.450353+0000","last_change":"2024-05-11T12:11:57.436042+0000","last_active":"2024-05-11T12:11:57.450353+0000","last_peered":"2024-05-11T12:11:57.450353+0000","last_clean":"2024-05-11T12:11:57.450353+0000","last_became_active":"2024-05-11T12:11:57.435781+0000","last_became_peered":"2024-05-11T12:11:57.435781+0000","last_unstale":"2024-05-11T12:11:57.450353+0000","last_undegraded":"2024-05-11T12:11:57.450353+0000","last_fullsized":"2024-05-11T12:11:57.450353+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:07:22.774535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"40'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.436439+0000","last_change":"2024-05-11T12:11:56.406200+0000","last_active":"2024-05-11T12:11:58.436439+0000","last_peered":"2024-05-11T12:11:58.436439+0000","last_clean":"2024-05-11T12:11:58.436439+0000","last_became_active":"2024-05-11T12:11:56.405912+0000","last_became_peered":"2024-05-11T12:11:56.405912+0000","last_unstale":"2024-05-11T12:11:58.436439+0000","last_undegraded":"2024-05-11T12:11:58.436439+0000","last_fullsized":"2024-05-11T12:11:58.436439+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:31:16.917994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.436708+0000","last_change":"2024-05-11T12:11:57.440226+0000","last_active":"2024-05-11T12:11:58.436708+0000","last_peered":"2024-05-11T12:11:58.436708+0000","last_clean":"2024-05-11T12:11:58.436708+0000","last_became_active":"2024-05-11T12:11:57.438861+0000","last_became_peered":"2024-05-11T12:11:57.438861+0000","last_unstale":"2024-05-11T12:11:58.436708+0000","last_undegraded":"2024-05-11T12:11:58.436708+0000","last_fullsized":"2024-05-11T12:11:58.436708+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:23:47.093149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.432963+0000","last_change":"2024-05-11T12:11:56.409279+0000","last_active":"2024-05-11T12:11:58.432963+0000","last_peered":"2024-05-11T12:11:58.432963+0000","last_clean":"2024-05-11T12:11:58.432963+0000","last_became_active":"2024-05-11T12:11:56.408985+0000","last_became_peered":"2024-05-11T12:11:56.408985+0000","last_unstale":"2024-05-11T12:11:58.432963+0000","last_undegraded":"2024-05-11T12:11:58.432963+0000","last_fullsized":"2024-05-11T12:11:58.432963+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:30:56.816516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.453397+0000","last_change":"2024-05-11T12:11:57.443914+0000","last_active":"2024-05-11T12:11:57.453397+0000","last_peered":"2024-05-11T12:11:57.453397+0000","last_clean":"2024-05-11T12:11:57.453397+0000","last_became_active":"2024-05-11T12:11:57.443680+0000","last_became_peered":"2024-05-11T12:11:57.443680+0000","last_unstale":"2024-05-11T12:11:57.453397+0000","last_undegraded":"2024-05-11T12:11:57.453397+0000","last_fullsized":"2024-05-11T12:11:57.453397+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:32:45.644260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.435987+0000","last_change":"2024-05-11T12:11:56.398721+0000","last_active":"2024-05-11T12:11:58.435987+0000","last_peered":"2024-05-11T12:11:58.435987+0000","last_clean":"2024-05-11T12:11:58.435987+0000","last_became_active":"2024-05-11T12:11:56.398181+0000","last_became_peered":"2024-05-11T12:11:56.398181+0000","last_unstale":"2024-05-11T12:11:58.435987+0000","last_undegraded":"2024-05-11T12:11:58.435987+0000","last_fullsized":"2024-05-11T12:11:58.435987+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:24:06.036447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:59.749297+0000","last_change":"2024-05-11T12:11:57.436889+0000","last_active":"2024-05-11T12:11:59.749297+0000","last_peered":"2024-05-11T12:11:59.749297+0000","last_clean":"2024-05-11T12:11:59.749297+0000","last_became_active":"2024-05-11T12:11:57.436592+0000","last_became_peered":"2024-05-11T12:11:57.436592+0000","last_unstale":"2024-05-11T12:11:59.749297+0000","last_undegraded":"2024-05-11T12:11:59.749297+0000","last_fullsized":"2024-05-11T12:11:59.749297+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:11:37.571584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.436163+0000","last_change":"2024-05-11T12:11:56.409035+0000","last_active":"2024-05-11T12:11:58.436163+0000","last_peered":"2024-05-11T12:11:58.436163+0000","last_clean":"2024-05-11T12:11:58.436163+0000","last_became_active":"2024-05-11T12:11:56.408751+0000","last_became_peered":"2024-05-11T12:11:56.408751+0000","last_unstale":"2024-05-11T12:11:58.436163+0000","last_undegraded":"2024-05-11T12:11:58.436163+0000","last_fullsized":"2024-05-11T12:11:58.436163+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:11:47.612399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.431245+0000","last_change":"2024-05-11T12:11:57.419098+0000","last_active":"2024-05-11T12:11:57.431245+0000","last_peered":"2024-05-11T12:11:57.431245+0000","last_clean":"2024-05-11T12:11:57.431245+0000","last_became_active":"2024-05-11T12:11:57.417628+0000","last_became_peered":"2024-05-11T12:11:57.417628+0000","last_unstale":"2024-05-11T12:11:57.431245+0000","last_undegraded":"2024-05-11T12:11:57.431245+0000","last_fullsized":"2024-05-11T12:11:57.431245+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:17:59.309336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.414852+0000","last_change":"2024-05-11T12:11:56.395623+0000","last_active":"2024-05-11T12:11:57.414852+0000","last_peered":"2024-05-11T12:11:57.414852+0000","last_clean":"2024-05-11T12:11:57.414852+0000","last_became_active":"2024-05-11T12:11:56.395250+0000","last_became_peered":"2024-05-11T12:11:56.395250+0000","last_unstale":"2024-05-11T12:11:57.414852+0000","last_undegraded":"2024-05-11T12:11:57.414852+0000","last_fullsized":"2024-05-11T12:11:57.414852+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:30:56.816516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.430168+0000","last_change":"2024-05-11T12:11:57.433272+0000","last_active":"2024-05-11T12:11:58.430168+0000","last_peered":"2024-05-11T12:11:58.430168+0000","last_clean":"2024-05-11T12:11:58.430168+0000","last_became_active":"2024-05-11T12:11:57.432767+0000","last_became_peered":"2024-05-11T12:11:57.432767+0000","last_unstale":"2024-05-11T12:11:58.430168+0000","last_undegraded":"2024-05-11T12:11:58.430168+0000","last_fullsized":"2024-05-11T12:11:58.430168+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:27:46.224897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.435741+0000","last_change":"2024-05-11T12:11:56.400668+0000","last_active":"2024-05-11T12:11:58.435741+0000","last_peered":"2024-05-11T12:11:58.435741+0000","last_clean":"2024-05-11T12:11:58.435741+0000","last_became_active":"2024-05-11T12:11:56.397950+0000","last_became_peered":"2024-05-11T12:11:56.397950+0000","last_unstale":"2024-05-11T12:11:58.435741+0000","last_undegraded":"2024-05-11T12:11:58.435741+0000","last_fullsized":"2024-05-11T12:11:58.435741+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:13:01.143192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.433130+0000","last_change":"2024-05-11T12:11:57.434141+0000","last_active":"2024-05-11T12:11:58.433130+0000","last_peered":"2024-05-11T12:11:58.433130+0000","last_clean":"2024-05-11T12:11:58.433130+0000","last_became_active":"2024-05-11T12:11:57.433852+0000","last_became_peered":"2024-05-11T12:11:57.433852+0000","last_unstale":"2024-05-11T12:11:58.433130+0000","last_undegraded":"2024-05-11T12:11:58.433130+0000","last_fullsized":"2024-05-11T12:11:58.433130+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:53:03.480916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"40'2","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.455023+0000","last_change":"2024-05-11T12:11:56.409581+0000","last_active":"2024-05-11T12:11:57.455023+0000","last_peered":"2024-05-11T12:11:57.455023+0000","last_clean":"2024-05-11T12:11:57.455023+0000","last_became_active":"2024-05-11T12:11:56.408010+0000","last_became_peered":"2024-05-11T12:11:56.408010+0000","last_unstale":"2024-05-11T12:11:57.455023+0000","last_undegraded":"2024-05-11T12:11:57.455023+0000","last_fullsized":"2024-05-11T12:11:57.455023+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:20:13.124057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.451683+0000","last_change":"2024-05-11T12:11:57.447191+0000","last_active":"2024-05-11T12:11:57.451683+0000","last_peered":"2024-05-11T12:11:57.451683+0000","last_clean":"2024-05-11T12:11:57.451683+0000","last_became_active":"2024-05-11T12:11:57.442656+0000","last_became_peered":"2024-05-11T12:11:57.442656+0000","last_unstale":"2024-05-11T12:11:57.451683+0000","last_undegraded":"2024-05-11T12:11:57.451683+0000","last_fullsized":"2024-05-11T12:11:57.451683+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:52:14.248503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"41'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.436582+0000","last_change":"2024-05-11T12:11:56.404335+0000","last_active":"2024-05-11T12:11:58.436582+0000","last_peered":"2024-05-11T12:11:58.436582+0000","last_clean":"2024-05-11T12:11:58.436582+0000","last_became_active":"2024-05-11T12:11:56.404069+0000","last_became_peered":"2024-05-11T12:11:56.404069+0000","last_unstale":"2024-05-11T12:11:58.436582+0000","last_undegraded":"2024-05-11T12:11:58.436582+0000","last_fullsized":"2024-05-11T12:11:58.436582+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:50:43.515250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.430571+0000","last_change":"2024-05-11T12:11:57.432694+0000","last_active":"2024-05-11T12:11:58.430571+0000","last_peered":"2024-05-11T12:11:58.430571+0000","last_clean":"2024-05-11T12:11:58.430571+0000","last_became_active":"2024-05-11T12:11:57.431359+0000","last_became_peered":"2024-05-11T12:11:57.431359+0000","last_unstale":"2024-05-11T12:11:58.430571+0000","last_undegraded":"2024-05-11T12:11:58.430571+0000","last_fullsized":"2024-05-11T12:11:58.430571+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:12:00.162149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:59.749622+0000","last_change":"2024-05-11T12:11:56.397553+0000","last_active":"2024-05-11T12:11:59.749622+0000","last_peered":"2024-05-11T12:11:59.749622+0000","last_clean":"2024-05-11T12:11:59.749622+0000","last_became_active":"2024-05-11T12:11:56.397219+0000","last_became_peered":"2024-05-11T12:11:56.397219+0000","last_unstale":"2024-05-11T12:11:59.749622+0000","last_undegraded":"2024-05-11T12:11:59.749622+0000","last_fullsized":"2024-05-11T12:11:59.749622+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:07:21.760158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:59.749916+0000","last_change":"2024-05-11T12:11:57.433279+0000","last_active":"2024-05-11T12:11:59.749916+0000","last_peered":"2024-05-11T12:11:59.749916+0000","last_clean":"2024-05-11T12:11:59.749916+0000","last_became_active":"2024-05-11T12:11:57.432993+0000","last_became_peered":"2024-05-11T12:11:57.432993+0000","last_unstale":"2024-05-11T12:11:59.749916+0000","last_undegraded":"2024-05-11T12:11:59.749916+0000","last_fullsized":"2024-05-11T12:11:59.749916+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:22:33.211172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:59.749802+0000","last_change":"2024-05-11T12:11:56.404006+0000","last_active":"2024-05-11T12:11:59.749802+0000","last_peered":"2024-05-11T12:11:59.749802+0000","last_clean":"2024-05-11T12:11:59.749802+0000","last_became_active":"2024-05-11T12:11:56.403672+0000","last_became_peered":"2024-05-11T12:11:56.403672+0000","last_unstale":"2024-05-11T12:11:59.749802+0000","last_undegraded":"2024-05-11T12:11:59.749802+0000","last_fullsized":"2024-05-11T12:11:59.749802+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:28:43.572256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.432618+0000","last_change":"2024-05-11T12:11:57.436933+0000","last_active":"2024-05-11T12:11:58.432618+0000","last_peered":"2024-05-11T12:11:58.432618+0000","last_clean":"2024-05-11T12:11:58.432618+0000","last_became_active":"2024-05-11T12:11:57.435829+0000","last_became_peered":"2024-05-11T12:11:57.435829+0000","last_unstale":"2024-05-11T12:11:58.432618+0000","last_undegraded":"2024-05-11T12:11:58.432618+0000","last_fullsized":"2024-05-11T12:11:58.432618+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:46:06.174919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.401153+0000","last_change":"2024-05-11T12:11:56.399122+0000","last_active":"2024-05-11T12:11:57.401153+0000","last_peered":"2024-05-11T12:11:57.401153+0000","last_clean":"2024-05-11T12:11:57.401153+0000","last_became_active":"2024-05-11T12:11:56.398845+0000","last_became_peered":"2024-05-11T12:11:56.398845+0000","last_unstale":"2024-05-11T12:11:57.401153+0000","last_undegraded":"2024-05-11T12:11:57.401153+0000","last_fullsized":"2024-05-11T12:11:57.401153+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:16:00.825967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.449098+0000","last_change":"2024-05-11T12:11:57.442240+0000","last_active":"2024-05-11T12:11:57.449098+0000","last_peered":"2024-05-11T12:11:57.449098+0000","last_clean":"2024-05-11T12:11:57.449098+0000","last_became_active":"2024-05-11T12:11:57.440831+0000","last_became_peered":"2024-05-11T12:11:57.440831+0000","last_unstale":"2024-05-11T12:11:57.449098+0000","last_undegraded":"2024-05-11T12:11:57.449098+0000","last_fullsized":"2024-05-11T12:11:57.449098+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:28:44.586634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.418097+0000","last_change":"2024-05-11T12:11:56.402880+0000","last_active":"2024-05-11T12:11:57.418097+0000","last_peered":"2024-05-11T12:11:57.418097+0000","last_clean":"2024-05-11T12:11:57.418097+0000","last_became_active":"2024-05-11T12:11:56.402608+0000","last_became_peered":"2024-05-11T12:11:56.402608+0000","last_unstale":"2024-05-11T12:11:57.418097+0000","last_undegraded":"2024-05-11T12:11:57.418097+0000","last_fullsized":"2024-05-11T12:11:57.418097+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:34:29.698881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.451131+0000","last_change":"2024-05-11T12:11:57.444173+0000","last_active":"2024-05-11T12:11:57.451131+0000","last_peered":"2024-05-11T12:11:57.451131+0000","last_clean":"2024-05-11T12:11:57.451131+0000","last_became_active":"2024-05-11T12:11:57.441809+0000","last_became_peered":"2024-05-11T12:11:57.441809+0000","last_unstale":"2024-05-11T12:11:57.451131+0000","last_undegraded":"2024-05-11T12:11:57.451131+0000","last_fullsized":"2024-05-11T12:11:57.451131+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:48:54.195365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.428722+0000","last_change":"2024-05-11T12:11:56.399468+0000","last_active":"2024-05-11T12:11:58.428722+0000","last_peered":"2024-05-11T12:11:58.428722+0000","last_clean":"2024-05-11T12:11:58.428722+0000","last_became_active":"2024-05-11T12:11:56.399205+0000","last_became_peered":"2024-05-11T12:11:56.399205+0000","last_unstale":"2024-05-11T12:11:58.428722+0000","last_undegraded":"2024-05-11T12:11:58.428722+0000","last_fullsized":"2024-05-11T12:11:58.428722+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:46:05.160542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.430065+0000","last_change":"2024-05-11T12:11:57.417010+0000","last_active":"2024-05-11T12:11:57.430065+0000","last_peered":"2024-05-11T12:11:57.430065+0000","last_clean":"2024-05-11T12:11:57.430065+0000","last_became_active":"2024-05-11T12:11:57.416499+0000","last_became_peered":"2024-05-11T12:11:57.416499+0000","last_unstale":"2024-05-11T12:11:57.430065+0000","last_undegraded":"2024-05-11T12:11:57.430065+0000","last_fullsized":"2024-05-11T12:11:57.430065+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:53:03.480916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.418829+0000","last_change":"2024-05-11T12:11:56.400810+0000","last_active":"2024-05-11T12:11:57.418829+0000","last_peered":"2024-05-11T12:11:57.418829+0000","last_clean":"2024-05-11T12:11:57.418829+0000","last_became_active":"2024-05-11T12:11:56.399824+0000","last_became_peered":"2024-05-11T12:11:56.399824+0000","last_unstale":"2024-05-11T12:11:57.418829+0000","last_undegraded":"2024-05-11T12:11:57.418829+0000","last_fullsized":"2024-05-11T12:11:57.418829+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:07:21.760158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":139,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-11T12:11:57.414520+0000","last_change":"2024-05-11T12:10:35.066673+0000","last_active":"2024-05-11T12:11:57.414520+0000","last_peered":"2024-05-11T12:11:57.414520+0000","last_clean":"2024-05-11T12:11:57.414520+0000","last_became_active":"2024-05-11T12:10:35.065942+0000","last_became_peered":"2024-05-11T12:10:35.065942+0000","last_unstale":"2024-05-11T12:11:57.414520+0000","last_undegraded":"2024-05-11T12:11:57.414520+0000","last_fullsized":"2024-05-11T12:11:57.414520+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_clean_scrub_stamp":"2024-05-11T12:10:33.601136+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:12:02.133182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476756,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28064,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343776,"statfs":{"total":96636764160,"available":96608026624,"internally_reserved":0,"allocated":1060864,"data_stored":784295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476757,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":192827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476756,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":192357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476758,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":192849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":479232,"data_stored":200583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509463,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":466944,"data_stored":199027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509462,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1060864,"data_stored":789339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542166,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":466944,"data_stored":199007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T12:12:06.909 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T12:12:06.932 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T12:12:06.969 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T12:12:07.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.114+0000 7fdfbd600640 1 Processor -- start 2024-05-11T12:12:07.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.114+0000 7fdfbd600640 1 -- start start 2024-05-11T12:12:07.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.115+0000 7fdfbd600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fdfb80b3a80 0x7fdfb806d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:07.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.115+0000 7fdfbd600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdfb80b5220 0x7fdfb80b56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:07.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.115+0000 7fdfbd600640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfb8071f30 con 0x7fdfb81458f0 2024-05-11T12:12:07.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.115+0000 7fdfbd600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fdfb80b5bf0 con 0x7fdfb80b3a80 2024-05-11T12:12:07.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.115+0000 7fdfbd600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fdfb806db80 con 0x7fdfb80b5220 2024-05-11T12:12:07.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.116+0000 7fdfb7e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fdfb81458f0 0x7fdfb80b4970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:42196/0 (socket says 172.21.15.167:42196) 2024-05-11T12:12:07.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.116+0000 7fdfb7e00640 1 -- 172.21.15.167:0/1676264715 learned_addr learned my addr 172.21.15.167:0/1676264715 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:07.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.116+0000 7fdfaf400640 1 --2- 172.21.15.167:0/1676264715 >> v2:172.21.15.204:3300/0 conn(0x7fdfb80b3a80 0x7fdfb806d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:07.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.116+0000 7fdfbcc00640 1 --2- 172.21.15.167:0/1676264715 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdfb80b5220 0x7fdfb80b56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:07.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.116+0000 7fdfb7400640 1 -- 172.21.15.167:0/1676264715 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3496693280 0 0) 0x7fdfb8071f30 con 0x7fdfb81458f0 2024-05-11T12:12:07.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.116+0000 7fdfb7400640 1 -- 172.21.15.167:0/1676264715 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf9c003520 con 0x7fdfb81458f0 2024-05-11T12:12:07.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.117+0000 7fdfb7400640 1 -- 172.21.15.167:0/1676264715 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2267641365 0 0) 0x7fdf9c003520 con 0x7fdfb81458f0 2024-05-11T12:12:07.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.117+0000 7fdfb7400640 1 -- 172.21.15.167:0/1676264715 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdfb80b5220 msgr2=0x7fdfb80b56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:07.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.117+0000 7fdfb7400640 1 --2- 172.21.15.167:0/1676264715 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdfb80b5220 0x7fdfb80b56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:07.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.117+0000 7fdfb7400640 1 -- 172.21.15.167:0/1676264715 >> v2:172.21.15.204:3300/0 conn(0x7fdfb80b3a80 msgr2=0x7fdfb806d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:07.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.117+0000 7fdfb7400640 1 --2- 172.21.15.167:0/1676264715 >> v2:172.21.15.204:3300/0 conn(0x7fdfb80b3a80 0x7fdfb806d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:07.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.117+0000 7fdfb7400640 1 -- 172.21.15.167:0/1676264715 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfb8072210 con 0x7fdfb81458f0 2024-05-11T12:12:07.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.117+0000 7fdfb7400640 1 -- 172.21.15.167:0/1676264715 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7fdfa80016f0 con 0x7fdfb81458f0 2024-05-11T12:12:07.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.117+0000 7fdfaf400640 1 --2- 172.21.15.167:0/1676264715 >> v2:172.21.15.204:3300/0 conn(0x7fdfb80b3a80 0x7fdfb806d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:12:07.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.117+0000 7fdfb7400640 1 -- 172.21.15.167:0/1676264715 >> v1:172.21.15.167:6789/0 conn(0x7fdfb81458f0 legacy=0x7fdfb80b4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:07.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.117+0000 7fdfb7400640 1 --2- 172.21.15.167:0/1676264715 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdf9c003a00 0x7fdf9c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:07.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.117+0000 7fdfb7400640 1 --2- 172.21.15.167:0/1676264715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf9c004800 0x7fdf9c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:07.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.117+0000 7fdfb7400640 1 --2- 172.21.15.167:0/1676264715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf9c0051c0 0x7fdf9c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:07.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.117+0000 7fdfb7400640 1 -- 172.21.15.167:0/1676264715 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb80b5bf0 con 0x7fdf9c0051c0 2024-05-11T12:12:07.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.117+0000 7fdfb7400640 1 -- 172.21.15.167:0/1676264715 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c00c010 con 0x7fdf9c004800 2024-05-11T12:12:07.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.117+0000 7fdfb7400640 1 -- 172.21.15.167:0/1676264715 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fdf9c00c210 con 0x7fdf9c003a00 2024-05-11T12:12:07.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.117+0000 7fdfbcc00640 1 --2- 172.21.15.167:0/1676264715 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdfb80b5220 0x7fdfb80b56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:12:07.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.117+0000 7fdfbcc00640 1 --2- 172.21.15.167:0/1676264715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf9c0051c0 0x7fdf9c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:07.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.117+0000 7fdfb7e00640 1 --2- 172.21.15.167:0/1676264715 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdf9c003a00 0x7fdf9c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:07.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.117+0000 7fdfaf400640 1 --2- 172.21.15.167:0/1676264715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf9c004800 0x7fdf9c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:07.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.118+0000 7fdfbcc00640 1 -- 172.21.15.167:0/1676264715 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdf9c003a00 msgr2=0x7fdf9c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:07.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.118+0000 7fdfbcc00640 1 --2- 172.21.15.167:0/1676264715 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdf9c003a00 0x7fdf9c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:07.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.118+0000 7fdfbcc00640 1 -- 172.21.15.167:0/1676264715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf9c004800 msgr2=0x7fdf9c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:07.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.118+0000 7fdfbcc00640 1 --2- 172.21.15.167:0/1676264715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf9c004800 0x7fdf9c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:07.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.118+0000 7fdfbcc00640 1 -- 172.21.15.167:0/1676264715 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf9c003520 con 0x7fdf9c0051c0 2024-05-11T12:12:07.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.119+0000 7fdfbcc00640 1 --2- 172.21.15.167:0/1676264715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf9c0051c0 0x7fdf9c00b9c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0006fd0 tx=0x7fdf9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=841a0f798575f8ab server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:07.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.119+0000 7fdfb7400640 1 -- 172.21.15.167:0/1676264715 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0037070 con 0x7fdf9c0051c0 2024-05-11T12:12:07.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.119+0000 7fdfb7400640 1 -- 172.21.15.167:0/1676264715 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfb002aae0 con 0x7fdf9c0051c0 2024-05-11T12:12:07.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.119+0000 7fdfb7400640 1 -- 172.21.15.167:0/1676264715 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb000f040 con 0x7fdf9c0051c0 2024-05-11T12:12:07.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.119+0000 7fdfbd600640 1 -- 172.21.15.167:0/1676264715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf9c0051c0 msgr2=0x7fdf9c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:07.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.119+0000 7fdfbd600640 1 --2- 172.21.15.167:0/1676264715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf9c0051c0 0x7fdf9c00b9c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0006fd0 tx=0x7fdf9c005830 comp rx=0 tx=0).stop 2024-05-11T12:12:07.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.119+0000 7fdfbcc00640 1 -- 172.21.15.167:0/1676264715 reap_dead start 2024-05-11T12:12:07.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.119+0000 7fdfbd600640 1 -- 172.21.15.167:0/1676264715 shutdown_connections 2024-05-11T12:12:07.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.119+0000 7fdfbd600640 1 -- 172.21.15.167:0/1676264715 >> 172.21.15.167:0/1676264715 conn(0x7fdfb80b3460 msgr2=0x7fdfb80d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:07.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.120+0000 7fdfbd600640 1 -- 172.21.15.167:0/1676264715 shutdown_connections 2024-05-11T12:12:07.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.120+0000 7fdfbd600640 1 -- 172.21.15.167:0/1676264715 wait complete. 2024-05-11T12:12:07.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.121+0000 7fdfbd600640 1 Processor -- start 2024-05-11T12:12:07.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.121+0000 7fdfbd600640 1 -- start start 2024-05-11T12:12:07.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.122+0000 7fdfbd600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdf9c0051c0 0x7fdfb81407c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:07.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.122+0000 7fdfbd600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdfb813fc90 0x7fdfb815da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:07.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.122+0000 7fdfbd600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdfb813da50 0x7fdfb813ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:07.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.122+0000 7fdfbd600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb8072ab0 con 0x7fdfb813da50 2024-05-11T12:12:07.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.122+0000 7fdfbd600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb80727d0 con 0x7fdfb813fc90 2024-05-11T12:12:07.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.122+0000 7fdfbd600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fdfb80b56b0 con 0x7fdf9c0051c0 2024-05-11T12:12:07.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.123+0000 7fdfb7e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdf9c0051c0 0x7fdfb81407c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:07.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.123+0000 7fdfbcc00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdfb813da50 0x7fdfb813ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:07.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.123+0000 7fdfb7e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdf9c0051c0 0x7fdfb81407c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:59792/0 (socket says 172.21.15.167:59792) 2024-05-11T12:12:07.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.123+0000 7fdfb7e00640 1 -- 172.21.15.167:0/1658938192 learned_addr learned my addr 172.21.15.167:0/1658938192 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:07.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.123+0000 7fdfaf400640 1 --2- 172.21.15.167:0/1658938192 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdfb813fc90 0x7fdfb815da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:07.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.123+0000 7fdfbcc00640 1 -- 172.21.15.167:0/1658938192 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdf9c0051c0 msgr2=0x7fdfb81407c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:07.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.123+0000 7fdfbcc00640 1 --2- 172.21.15.167:0/1658938192 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdf9c0051c0 0x7fdfb81407c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:07.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.123+0000 7fdfbcc00640 1 -- 172.21.15.167:0/1658938192 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdfb813fc90 msgr2=0x7fdfb815da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:07.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.123+0000 7fdfbcc00640 1 --2- 172.21.15.167:0/1658938192 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdfb813fc90 0x7fdfb815da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:07.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.123+0000 7fdfbcc00640 1 -- 172.21.15.167:0/1658938192 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfb815e2e0 con 0x7fdfb813da50 2024-05-11T12:12:07.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.124+0000 7fdfbcc00640 1 --2- 172.21.15.167:0/1658938192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdfb813da50 0x7fdfb813ded0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0031000 tx=0x7fdfb0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:07.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.124+0000 7fdfb4c00640 1 -- 172.21.15.167:0/1658938192 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0037070 con 0x7fdfb813da50 2024-05-11T12:12:07.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.124+0000 7fdfb4c00640 1 -- 172.21.15.167:0/1658938192 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfb0042070 con 0x7fdfb813da50 2024-05-11T12:12:07.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.124+0000 7fdfb4c00640 1 -- 172.21.15.167:0/1658938192 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb003c850 con 0x7fdfb813da50 2024-05-11T12:12:07.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.124+0000 7fdfbd600640 1 -- 172.21.15.167:0/1658938192 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfb815e5c0 con 0x7fdfb813da50 2024-05-11T12:12:07.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.124+0000 7fdfbd600640 1 -- 172.21.15.167:0/1658938192 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfb815eb80 con 0x7fdfb813da50 2024-05-11T12:12:07.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.126+0000 7fdfbd600640 1 -- 172.21.15.167:0/1658938192 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfb80694b0 con 0x7fdfb813da50 2024-05-11T12:12:07.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.127+0000 7fdfb4c00640 1 -- 172.21.15.167:0/1658938192 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fdfb003c9f0 con 0x7fdfb813da50 2024-05-11T12:12:07.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.128+0000 7fdfb4c00640 1 --2- 172.21.15.167:0/1658938192 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fdf940762d0 0x7fdf94078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:07.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.128+0000 7fdfb4c00640 1 -- 172.21.15.167:0/1658938192 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7fdfb00b7710 con 0x7fdfb813da50 2024-05-11T12:12:07.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.128+0000 7fdfb7e00640 1 --2- 172.21.15.167:0/1658938192 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fdf940762d0 0x7fdf94078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:07.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.129+0000 7fdfb7e00640 1 --2- 172.21.15.167:0/1658938192 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fdf940762d0 0x7fdf94078790 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8005c10 tx=0x7fdfa8007040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:07.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.132+0000 7fdfb4c00640 1 -- 172.21.15.167:0/1658938192 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fdfb0080890 con 0x7fdfb813da50 2024-05-11T12:12:07.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.333+0000 7fdfbd600640 1 -- 172.21.15.167:0/1658938192 --> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdfb815f380 con 0x7fdf940762d0 2024-05-11T12:12:07.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.351+0000 7fdfb4c00640 1 -- 172.21.15.167:0/1658938192 <== mgr.4106 v2:172.21.15.167:6832/578352559 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251936 (secure 0 0 0) 0x7fdfa8005d60 con 0x7fdf940762d0 2024-05-11T12:12:07.355 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:12:07.356 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2024-05-11T12:12:07.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.357+0000 7fdfbd600640 1 -- 172.21.15.167:0/1658938192 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fdf940762d0 msgr2=0x7fdf94078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:07.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.357+0000 7fdfbd600640 1 --2- 172.21.15.167:0/1658938192 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fdf940762d0 0x7fdf94078790 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8005c10 tx=0x7fdfa8007040 comp rx=0 tx=0).stop 2024-05-11T12:12:07.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.357+0000 7fdfbd600640 1 -- 172.21.15.167:0/1658938192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdfb813da50 msgr2=0x7fdfb813ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:07.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.357+0000 7fdfbd600640 1 --2- 172.21.15.167:0/1658938192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdfb813da50 0x7fdfb813ded0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0031000 tx=0x7fdfb0004730 comp rx=0 tx=0).stop 2024-05-11T12:12:07.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.358+0000 7fdfbd600640 1 -- 172.21.15.167:0/1658938192 shutdown_connections 2024-05-11T12:12:07.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.358+0000 7fdfbd600640 1 --2- 172.21.15.167:0/1658938192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdfb813da50 0x7fdfb813ded0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:07.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.358+0000 7fdfbd600640 1 --2- 172.21.15.167:0/1658938192 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fdf940762d0 0x7fdf94078790 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:07.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.358+0000 7fdfbd600640 1 --2- 172.21.15.167:0/1658938192 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdfb813fc90 0x7fdfb815da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:07.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.358+0000 7fdfbd600640 1 --2- 172.21.15.167:0/1658938192 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdf9c0051c0 0x7fdfb81407c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:07.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.358+0000 7fdfbd600640 1 -- 172.21.15.167:0/1658938192 >> 172.21.15.167:0/1658938192 conn(0x7fdfb80b3460 msgr2=0x7fdfb806da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:07.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.358+0000 7fdfbd600640 1 -- 172.21.15.167:0/1658938192 shutdown_connections 2024-05-11T12:12:07.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.358+0000 7fdfbd600640 1 -- 172.21.15.167:0/1658938192 wait complete. 2024-05-11T12:12:07.375 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":82,"stamp":"2024-05-11T12:12:07.260503+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221024,"kb_used_data":4840,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753696,"statfs":{"total":773094113280,"available":772867784704,"internally_reserved":0,"allocated":4956160,"data_stored":2777052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.867515"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.430959+0000","last_change":"2024-05-11T12:11:57.428217+0000","last_active":"2024-05-11T12:11:58.430959+0000","last_peered":"2024-05-11T12:11:58.430959+0000","last_clean":"2024-05-11T12:11:58.430959+0000","last_became_active":"2024-05-11T12:11:57.425926+0000","last_became_peered":"2024-05-11T12:11:57.425926+0000","last_unstale":"2024-05-11T12:11:58.430959+0000","last_undegraded":"2024-05-11T12:11:58.430959+0000","last_fullsized":"2024-05-11T12:11:58.430959+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:46:47.695505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.430603+0000","last_change":"2024-05-11T12:11:57.430689+0000","last_active":"2024-05-11T12:11:58.430603+0000","last_peered":"2024-05-11T12:11:58.430603+0000","last_clean":"2024-05-11T12:11:58.430603+0000","last_became_active":"2024-05-11T12:11:57.430424+0000","last_became_peered":"2024-05-11T12:11:57.430424+0000","last_unstale":"2024-05-11T12:11:58.430603+0000","last_undegraded":"2024-05-11T12:11:58.430603+0000","last_fullsized":"2024-05-11T12:11:58.430603+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:32:11.650224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:59.749549+0000","last_change":"2024-05-11T12:11:57.432468+0000","last_active":"2024-05-11T12:11:59.749549+0000","last_peered":"2024-05-11T12:11:59.749549+0000","last_clean":"2024-05-11T12:11:59.749549+0000","last_became_active":"2024-05-11T12:11:57.432189+0000","last_became_peered":"2024-05-11T12:11:57.432189+0000","last_unstale":"2024-05-11T12:11:59.749549+0000","last_undegraded":"2024-05-11T12:11:59.749549+0000","last_fullsized":"2024-05-11T12:11:59.749549+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:55:35.468623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.436535+0000","last_change":"2024-05-11T12:11:57.433107+0000","last_active":"2024-05-11T12:11:58.436535+0000","last_peered":"2024-05-11T12:11:58.436535+0000","last_clean":"2024-05-11T12:11:58.436535+0000","last_became_active":"2024-05-11T12:11:57.432829+0000","last_became_peered":"2024-05-11T12:11:57.432829+0000","last_unstale":"2024-05-11T12:11:58.436535+0000","last_undegraded":"2024-05-11T12:11:58.436535+0000","last_fullsized":"2024-05-11T12:11:58.436535+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:31:31.558337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.435925+0000","last_change":"2024-05-11T12:11:57.436577+0000","last_active":"2024-05-11T12:11:58.435925+0000","last_peered":"2024-05-11T12:11:58.435925+0000","last_clean":"2024-05-11T12:11:58.435925+0000","last_became_active":"2024-05-11T12:11:57.436245+0000","last_became_peered":"2024-05-11T12:11:57.436245+0000","last_unstale":"2024-05-11T12:11:58.435925+0000","last_undegraded":"2024-05-11T12:11:58.435925+0000","last_fullsized":"2024-05-11T12:11:58.435925+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:20:16.274615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.432396+0000","last_change":"2024-05-11T12:11:57.429489+0000","last_active":"2024-05-11T12:11:58.432396+0000","last_peered":"2024-05-11T12:11:58.432396+0000","last_clean":"2024-05-11T12:11:58.432396+0000","last_became_active":"2024-05-11T12:11:57.429216+0000","last_became_peered":"2024-05-11T12:11:57.429216+0000","last_unstale":"2024-05-11T12:11:58.432396+0000","last_undegraded":"2024-05-11T12:11:58.432396+0000","last_fullsized":"2024-05-11T12:11:58.432396+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:26:34.393703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.430852+0000","last_change":"2024-05-11T12:11:57.432877+0000","last_active":"2024-05-11T12:11:58.430852+0000","last_peered":"2024-05-11T12:11:58.430852+0000","last_clean":"2024-05-11T12:11:58.430852+0000","last_became_active":"2024-05-11T12:11:57.432581+0000","last_became_peered":"2024-05-11T12:11:57.432581+0000","last_unstale":"2024-05-11T12:11:58.430852+0000","last_undegraded":"2024-05-11T12:11:58.430852+0000","last_fullsized":"2024-05-11T12:11:58.430852+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:36:05.818955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.428593+0000","last_change":"2024-05-11T12:11:57.417730+0000","last_active":"2024-05-11T12:11:58.428593+0000","last_peered":"2024-05-11T12:11:58.428593+0000","last_clean":"2024-05-11T12:11:58.428593+0000","last_became_active":"2024-05-11T12:11:57.413336+0000","last_became_peered":"2024-05-11T12:11:57.413336+0000","last_unstale":"2024-05-11T12:11:58.428593+0000","last_undegraded":"2024-05-11T12:11:58.428593+0000","last_fullsized":"2024-05-11T12:11:58.428593+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:36:05.818955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:59.750283+0000","last_change":"2024-05-11T12:11:57.433333+0000","last_active":"2024-05-11T12:11:59.750283+0000","last_peered":"2024-05-11T12:11:59.750283+0000","last_clean":"2024-05-11T12:11:59.750283+0000","last_became_active":"2024-05-11T12:11:57.433046+0000","last_became_peered":"2024-05-11T12:11:57.433046+0000","last_unstale":"2024-05-11T12:11:59.750283+0000","last_undegraded":"2024-05-11T12:11:59.750283+0000","last_fullsized":"2024-05-11T12:11:59.750283+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:43:20.345625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.430797+0000","last_change":"2024-05-11T12:11:57.434958+0000","last_active":"2024-05-11T12:11:58.430797+0000","last_peered":"2024-05-11T12:11:58.430797+0000","last_clean":"2024-05-11T12:11:58.430797+0000","last_became_active":"2024-05-11T12:11:57.429221+0000","last_became_peered":"2024-05-11T12:11:57.429221+0000","last_unstale":"2024-05-11T12:11:58.430797+0000","last_undegraded":"2024-05-11T12:11:58.430797+0000","last_fullsized":"2024-05-11T12:11:58.430797+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:28:44.183789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.431380+0000","last_change":"2024-05-11T12:11:57.449552+0000","last_active":"2024-05-11T12:11:58.431380+0000","last_peered":"2024-05-11T12:11:58.431380+0000","last_clean":"2024-05-11T12:11:58.431380+0000","last_became_active":"2024-05-11T12:11:57.441862+0000","last_became_peered":"2024-05-11T12:11:57.441862+0000","last_unstale":"2024-05-11T12:11:58.431380+0000","last_undegraded":"2024-05-11T12:11:58.431380+0000","last_fullsized":"2024-05-11T12:11:58.431380+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:07:26.210928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:59.749971+0000","last_change":"2024-05-11T12:11:57.431974+0000","last_active":"2024-05-11T12:11:59.749971+0000","last_peered":"2024-05-11T12:11:59.749971+0000","last_clean":"2024-05-11T12:11:59.749971+0000","last_became_active":"2024-05-11T12:11:57.431619+0000","last_became_peered":"2024-05-11T12:11:57.431619+0000","last_unstale":"2024-05-11T12:11:59.749971+0000","last_undegraded":"2024-05-11T12:11:59.749971+0000","last_fullsized":"2024-05-11T12:11:59.749971+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:28:33.430453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.430639+0000","last_change":"2024-05-11T12:11:57.429706+0000","last_active":"2024-05-11T12:11:58.430639+0000","last_peered":"2024-05-11T12:11:58.430639+0000","last_clean":"2024-05-11T12:11:58.430639+0000","last_became_active":"2024-05-11T12:11:57.429387+0000","last_became_peered":"2024-05-11T12:11:57.429387+0000","last_unstale":"2024-05-11T12:11:58.430639+0000","last_undegraded":"2024-05-11T12:11:58.430639+0000","last_fullsized":"2024-05-11T12:11:58.430639+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:32:11.650224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:12:00.076303+0000","last_change":"2024-05-11T12:11:57.430917+0000","last_active":"2024-05-11T12:12:00.076303+0000","last_peered":"2024-05-11T12:12:00.076303+0000","last_clean":"2024-05-11T12:12:00.076303+0000","last_became_active":"2024-05-11T12:11:57.430613+0000","last_became_peered":"2024-05-11T12:11:57.430613+0000","last_unstale":"2024-05-11T12:12:00.076303+0000","last_undegraded":"2024-05-11T12:12:00.076303+0000","last_fullsized":"2024-05-11T12:12:00.076303+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:20:14.138435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.432076+0000","last_change":"2024-05-11T12:11:57.438016+0000","last_active":"2024-05-11T12:11:58.432076+0000","last_peered":"2024-05-11T12:11:58.432076+0000","last_clean":"2024-05-11T12:11:58.432076+0000","last_became_active":"2024-05-11T12:11:57.437680+0000","last_became_peered":"2024-05-11T12:11:57.437680+0000","last_unstale":"2024-05-11T12:11:58.432076+0000","last_undegraded":"2024-05-11T12:11:58.432076+0000","last_fullsized":"2024-05-11T12:11:58.432076+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:12:00.162149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:12:00.076944+0000","last_change":"2024-05-11T12:11:57.448672+0000","last_active":"2024-05-11T12:12:00.076944+0000","last_peered":"2024-05-11T12:12:00.076944+0000","last_clean":"2024-05-11T12:12:00.076944+0000","last_became_active":"2024-05-11T12:11:57.447489+0000","last_became_peered":"2024-05-11T12:11:57.447489+0000","last_unstale":"2024-05-11T12:12:00.076944+0000","last_undegraded":"2024-05-11T12:12:00.076944+0000","last_fullsized":"2024-05-11T12:12:00.076944+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:55:35.468623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:12:00.078413+0000","last_change":"2024-05-11T12:11:57.447831+0000","last_active":"2024-05-11T12:12:00.078413+0000","last_peered":"2024-05-11T12:12:00.078413+0000","last_clean":"2024-05-11T12:12:00.078413+0000","last_became_active":"2024-05-11T12:11:57.443540+0000","last_became_peered":"2024-05-11T12:11:57.443540+0000","last_unstale":"2024-05-11T12:12:00.078413+0000","last_undegraded":"2024-05-11T12:12:00.078413+0000","last_fullsized":"2024-05-11T12:12:00.078413+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:28:33.430453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:12:00.079017+0000","last_change":"2024-05-11T12:11:57.431125+0000","last_active":"2024-05-11T12:12:00.079017+0000","last_peered":"2024-05-11T12:12:00.079017+0000","last_clean":"2024-05-11T12:12:00.079017+0000","last_became_active":"2024-05-11T12:11:57.430835+0000","last_became_peered":"2024-05-11T12:11:57.430835+0000","last_unstale":"2024-05-11T12:12:00.079017+0000","last_undegraded":"2024-05-11T12:12:00.079017+0000","last_fullsized":"2024-05-11T12:12:00.079017+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:10:05.034718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.436454+0000","last_change":"2024-05-11T12:11:57.438506+0000","last_active":"2024-05-11T12:11:58.436454+0000","last_peered":"2024-05-11T12:11:58.436454+0000","last_clean":"2024-05-11T12:11:58.436454+0000","last_became_active":"2024-05-11T12:11:57.437706+0000","last_became_peered":"2024-05-11T12:11:57.437706+0000","last_unstale":"2024-05-11T12:11:58.436454+0000","last_undegraded":"2024-05-11T12:11:58.436454+0000","last_fullsized":"2024-05-11T12:11:58.436454+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:05:23.821514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:12:00.079212+0000","last_change":"2024-05-11T12:11:57.433084+0000","last_active":"2024-05-11T12:12:00.079212+0000","last_peered":"2024-05-11T12:12:00.079212+0000","last_clean":"2024-05-11T12:12:00.079212+0000","last_became_active":"2024-05-11T12:11:57.432645+0000","last_became_peered":"2024-05-11T12:11:57.432645+0000","last_unstale":"2024-05-11T12:12:00.079212+0000","last_undegraded":"2024-05-11T12:12:00.079212+0000","last_fullsized":"2024-05-11T12:12:00.079212+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:46:06.174919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.431366+0000","last_change":"2024-05-11T12:11:57.434801+0000","last_active":"2024-05-11T12:11:58.431366+0000","last_peered":"2024-05-11T12:11:58.431366+0000","last_clean":"2024-05-11T12:11:58.431366+0000","last_became_active":"2024-05-11T12:11:57.433969+0000","last_became_peered":"2024-05-11T12:11:57.433969+0000","last_unstale":"2024-05-11T12:11:58.431366+0000","last_undegraded":"2024-05-11T12:11:58.431366+0000","last_fullsized":"2024-05-11T12:11:58.431366+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:58:40.171928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:12:00.080858+0000","last_change":"2024-05-11T12:11:57.445771+0000","last_active":"2024-05-11T12:12:00.080858+0000","last_peered":"2024-05-11T12:12:00.080858+0000","last_clean":"2024-05-11T12:12:00.080858+0000","last_became_active":"2024-05-11T12:11:57.445474+0000","last_became_peered":"2024-05-11T12:11:57.445474+0000","last_unstale":"2024-05-11T12:12:00.080858+0000","last_undegraded":"2024-05-11T12:12:00.080858+0000","last_fullsized":"2024-05-11T12:12:00.080858+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:51:43.649698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:12:00.077021+0000","last_change":"2024-05-11T12:11:57.447508+0000","last_active":"2024-05-11T12:12:00.077021+0000","last_peered":"2024-05-11T12:12:00.077021+0000","last_clean":"2024-05-11T12:12:00.077021+0000","last_became_active":"2024-05-11T12:11:57.444377+0000","last_became_peered":"2024-05-11T12:11:57.444377+0000","last_unstale":"2024-05-11T12:12:00.077021+0000","last_undegraded":"2024-05-11T12:12:00.077021+0000","last_fullsized":"2024-05-11T12:12:00.077021+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:47:49.623473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.435737+0000","last_change":"2024-05-11T12:11:57.437303+0000","last_active":"2024-05-11T12:11:58.435737+0000","last_peered":"2024-05-11T12:11:58.435737+0000","last_clean":"2024-05-11T12:11:58.435737+0000","last_became_active":"2024-05-11T12:11:57.436970+0000","last_became_peered":"2024-05-11T12:11:57.436970+0000","last_unstale":"2024-05-11T12:11:58.435737+0000","last_undegraded":"2024-05-11T12:11:58.435737+0000","last_fullsized":"2024-05-11T12:11:58.435737+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:58:38.525115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:12:00.079530+0000","last_change":"2024-05-11T12:11:57.426462+0000","last_active":"2024-05-11T12:12:00.079530+0000","last_peered":"2024-05-11T12:12:00.079530+0000","last_clean":"2024-05-11T12:12:00.079530+0000","last_became_active":"2024-05-11T12:11:57.426087+0000","last_became_peered":"2024-05-11T12:11:57.426087+0000","last_unstale":"2024-05-11T12:12:00.079530+0000","last_undegraded":"2024-05-11T12:12:00.079530+0000","last_fullsized":"2024-05-11T12:12:00.079530+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:53:03.480916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.433780+0000","last_change":"2024-05-11T12:11:57.434107+0000","last_active":"2024-05-11T12:11:58.433780+0000","last_peered":"2024-05-11T12:11:58.433780+0000","last_clean":"2024-05-11T12:11:58.433780+0000","last_became_active":"2024-05-11T12:11:57.433812+0000","last_became_peered":"2024-05-11T12:11:57.433812+0000","last_unstale":"2024-05-11T12:11:58.433780+0000","last_undegraded":"2024-05-11T12:11:58.433780+0000","last_fullsized":"2024-05-11T12:11:58.433780+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:17:59.309336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:12:00.079355+0000","last_change":"2024-05-11T12:11:57.428594+0000","last_active":"2024-05-11T12:12:00.079355+0000","last_peered":"2024-05-11T12:12:00.079355+0000","last_clean":"2024-05-11T12:12:00.079355+0000","last_became_active":"2024-05-11T12:11:57.428245+0000","last_became_peered":"2024-05-11T12:11:57.428245+0000","last_unstale":"2024-05-11T12:12:00.079355+0000","last_undegraded":"2024-05-11T12:12:00.079355+0000","last_fullsized":"2024-05-11T12:12:00.079355+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:26:34.393703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.431314+0000","last_change":"2024-05-11T12:11:57.417635+0000","last_active":"2024-05-11T12:11:58.431314+0000","last_peered":"2024-05-11T12:11:58.431314+0000","last_clean":"2024-05-11T12:11:58.431314+0000","last_became_active":"2024-05-11T12:11:57.415536+0000","last_became_peered":"2024-05-11T12:11:57.415536+0000","last_unstale":"2024-05-11T12:11:58.431314+0000","last_undegraded":"2024-05-11T12:11:58.431314+0000","last_fullsized":"2024-05-11T12:11:58.431314+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:27:58.937922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:12:00.078590+0000","last_change":"2024-05-11T12:11:57.427847+0000","last_active":"2024-05-11T12:12:00.078590+0000","last_peered":"2024-05-11T12:12:00.078590+0000","last_clean":"2024-05-11T12:12:00.078590+0000","last_became_active":"2024-05-11T12:11:57.427416+0000","last_became_peered":"2024-05-11T12:11:57.427416+0000","last_unstale":"2024-05-11T12:12:00.078590+0000","last_undegraded":"2024-05-11T12:12:00.078590+0000","last_fullsized":"2024-05-11T12:12:00.078590+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:52:14.248503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:12:00.077534+0000","last_change":"2024-05-11T12:11:57.420465+0000","last_active":"2024-05-11T12:12:00.077534+0000","last_peered":"2024-05-11T12:12:00.077534+0000","last_clean":"2024-05-11T12:12:00.077534+0000","last_became_active":"2024-05-11T12:11:57.419963+0000","last_became_peered":"2024-05-11T12:11:57.419963+0000","last_unstale":"2024-05-11T12:12:00.077534+0000","last_undegraded":"2024-05-11T12:12:00.077534+0000","last_fullsized":"2024-05-11T12:12:00.077534+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:20:14.138435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:12:00.078648+0000","last_change":"2024-05-11T12:11:57.447209+0000","last_active":"2024-05-11T12:12:00.078648+0000","last_peered":"2024-05-11T12:12:00.078648+0000","last_clean":"2024-05-11T12:12:00.078648+0000","last_became_active":"2024-05-11T12:11:57.443801+0000","last_became_peered":"2024-05-11T12:11:57.443801+0000","last_unstale":"2024-05-11T12:12:00.078648+0000","last_undegraded":"2024-05-11T12:12:00.078648+0000","last_fullsized":"2024-05-11T12:12:00.078648+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:28:44.183789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.432358+0000","last_change":"2024-05-11T12:11:57.436143+0000","last_active":"2024-05-11T12:11:58.432358+0000","last_peered":"2024-05-11T12:11:58.432358+0000","last_clean":"2024-05-11T12:11:58.432358+0000","last_became_active":"2024-05-11T12:11:57.435874+0000","last_became_peered":"2024-05-11T12:11:57.435874+0000","last_unstale":"2024-05-11T12:11:58.432358+0000","last_undegraded":"2024-05-11T12:11:58.432358+0000","last_fullsized":"2024-05-11T12:11:58.432358+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:10:05.034718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:12:00.078393+0000","last_change":"2024-05-11T12:11:57.444554+0000","last_active":"2024-05-11T12:12:00.078393+0000","last_peered":"2024-05-11T12:12:00.078393+0000","last_clean":"2024-05-11T12:12:00.078393+0000","last_became_active":"2024-05-11T12:11:57.443836+0000","last_became_peered":"2024-05-11T12:11:57.443836+0000","last_unstale":"2024-05-11T12:12:00.078393+0000","last_undegraded":"2024-05-11T12:12:00.078393+0000","last_fullsized":"2024-05-11T12:12:00.078393+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:43:20.345625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"40'4","reported_seq":18,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:12:00.079778+0000","last_change":"2024-05-11T12:11:56.409094+0000","last_active":"2024-05-11T12:12:00.079778+0000","last_peered":"2024-05-11T12:12:00.079778+0000","last_clean":"2024-05-11T12:12:00.079778+0000","last_became_active":"2024-05-11T12:11:56.408820+0000","last_became_peered":"2024-05-11T12:11:56.408820+0000","last_unstale":"2024-05-11T12:12:00.079778+0000","last_undegraded":"2024-05-11T12:12:00.079778+0000","last_fullsized":"2024-05-11T12:12:00.079778+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:11:42.632883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.433251+0000","last_change":"2024-05-11T12:11:57.436017+0000","last_active":"2024-05-11T12:11:58.433251+0000","last_peered":"2024-05-11T12:11:58.433251+0000","last_clean":"2024-05-11T12:11:58.433251+0000","last_became_active":"2024-05-11T12:11:57.435070+0000","last_became_peered":"2024-05-11T12:11:57.435070+0000","last_unstale":"2024-05-11T12:11:58.433251+0000","last_undegraded":"2024-05-11T12:11:58.433251+0000","last_fullsized":"2024-05-11T12:11:58.433251+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:51:43.649698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"40'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:59.749979+0000","last_change":"2024-05-11T12:11:56.403002+0000","last_active":"2024-05-11T12:11:59.749979+0000","last_peered":"2024-05-11T12:11:59.749979+0000","last_clean":"2024-05-11T12:11:59.749979+0000","last_became_active":"2024-05-11T12:11:56.402721+0000","last_became_peered":"2024-05-11T12:11:56.402721+0000","last_unstale":"2024-05-11T12:11:59.749979+0000","last_undegraded":"2024-05-11T12:11:59.749979+0000","last_fullsized":"2024-05-11T12:11:59.749979+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:23:27.612044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:12:00.078430+0000","last_change":"2024-05-11T12:11:57.435281+0000","last_active":"2024-05-11T12:12:00.078430+0000","last_peered":"2024-05-11T12:12:00.078430+0000","last_clean":"2024-05-11T12:12:00.078430+0000","last_became_active":"2024-05-11T12:11:57.434672+0000","last_became_peered":"2024-05-11T12:11:57.434672+0000","last_unstale":"2024-05-11T12:12:00.078430+0000","last_undegraded":"2024-05-11T12:12:00.078430+0000","last_fullsized":"2024-05-11T12:12:00.078430+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:46:06.174919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.433425+0000","last_change":"2024-05-11T12:11:56.410313+0000","last_active":"2024-05-11T12:11:58.433425+0000","last_peered":"2024-05-11T12:11:58.433425+0000","last_clean":"2024-05-11T12:11:58.433425+0000","last_became_active":"2024-05-11T12:11:56.409971+0000","last_became_peered":"2024-05-11T12:11:56.409971+0000","last_unstale":"2024-05-11T12:11:58.433425+0000","last_undegraded":"2024-05-11T12:11:58.433425+0000","last_fullsized":"2024-05-11T12:11:58.433425+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:07:21.760158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.430958+0000","last_change":"2024-05-11T12:11:57.419505+0000","last_active":"2024-05-11T12:11:58.430958+0000","last_peered":"2024-05-11T12:11:58.430958+0000","last_clean":"2024-05-11T12:11:58.430958+0000","last_became_active":"2024-05-11T12:11:57.418984+0000","last_became_peered":"2024-05-11T12:11:57.418984+0000","last_unstale":"2024-05-11T12:11:58.430958+0000","last_undegraded":"2024-05-11T12:11:58.430958+0000","last_fullsized":"2024-05-11T12:11:58.430958+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:07:26.576379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:12:00.078210+0000","last_change":"2024-05-11T12:11:56.408353+0000","last_active":"2024-05-11T12:12:00.078210+0000","last_peered":"2024-05-11T12:12:00.078210+0000","last_clean":"2024-05-11T12:12:00.078210+0000","last_became_active":"2024-05-11T12:11:56.408078+0000","last_became_peered":"2024-05-11T12:11:56.408078+0000","last_unstale":"2024-05-11T12:12:00.078210+0000","last_undegraded":"2024-05-11T12:12:00.078210+0000","last_fullsized":"2024-05-11T12:12:00.078210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:16:00.825967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:59.749207+0000","last_change":"2024-05-11T12:11:57.438954+0000","last_active":"2024-05-11T12:11:59.749207+0000","last_peered":"2024-05-11T12:11:59.749207+0000","last_clean":"2024-05-11T12:11:59.749207+0000","last_became_active":"2024-05-11T12:11:57.437487+0000","last_became_peered":"2024-05-11T12:11:57.437487+0000","last_unstale":"2024-05-11T12:11:59.749207+0000","last_undegraded":"2024-05-11T12:11:59.749207+0000","last_fullsized":"2024-05-11T12:11:59.749207+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:17:59.309336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"40'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.431672+0000","last_change":"2024-05-11T12:11:56.404989+0000","last_active":"2024-05-11T12:11:58.431672+0000","last_peered":"2024-05-11T12:11:58.431672+0000","last_clean":"2024-05-11T12:11:58.431672+0000","last_became_active":"2024-05-11T12:11:56.404069+0000","last_became_peered":"2024-05-11T12:11:56.404069+0000","last_unstale":"2024-05-11T12:11:58.431672+0000","last_undegraded":"2024-05-11T12:11:58.431672+0000","last_fullsized":"2024-05-11T12:11:58.431672+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:26:33.379326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.432507+0000","last_change":"2024-05-11T12:11:57.442633+0000","last_active":"2024-05-11T12:11:58.432507+0000","last_peered":"2024-05-11T12:11:58.432507+0000","last_clean":"2024-05-11T12:11:58.432507+0000","last_became_active":"2024-05-11T12:11:57.442361+0000","last_became_peered":"2024-05-11T12:11:57.442361+0000","last_unstale":"2024-05-11T12:11:58.432507+0000","last_undegraded":"2024-05-11T12:11:58.432507+0000","last_fullsized":"2024-05-11T12:11:58.432507+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:07:26.576379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"40'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:12:00.076820+0000","last_change":"2024-05-11T12:11:56.411689+0000","last_active":"2024-05-11T12:12:00.076820+0000","last_peered":"2024-05-11T12:12:00.076820+0000","last_clean":"2024-05-11T12:12:00.076820+0000","last_became_active":"2024-05-11T12:11:56.409937+0000","last_became_peered":"2024-05-11T12:11:56.409937+0000","last_unstale":"2024-05-11T12:12:00.076820+0000","last_undegraded":"2024-05-11T12:12:00.076820+0000","last_fullsized":"2024-05-11T12:12:00.076820+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:36:40.785205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:12:00.080041+0000","last_change":"2024-05-11T12:11:57.435197+0000","last_active":"2024-05-11T12:12:00.080041+0000","last_peered":"2024-05-11T12:12:00.080041+0000","last_clean":"2024-05-11T12:12:00.080041+0000","last_became_active":"2024-05-11T12:11:57.434472+0000","last_became_peered":"2024-05-11T12:11:57.434472+0000","last_unstale":"2024-05-11T12:12:00.080041+0000","last_undegraded":"2024-05-11T12:12:00.080041+0000","last_fullsized":"2024-05-11T12:12:00.080041+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:53:03.480916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.432452+0000","last_change":"2024-05-11T12:11:56.409731+0000","last_active":"2024-05-11T12:11:58.432452+0000","last_peered":"2024-05-11T12:11:58.432452+0000","last_clean":"2024-05-11T12:11:58.432452+0000","last_became_active":"2024-05-11T12:11:56.409450+0000","last_became_peered":"2024-05-11T12:11:56.409450+0000","last_unstale":"2024-05-11T12:11:58.432452+0000","last_undegraded":"2024-05-11T12:11:58.432452+0000","last_fullsized":"2024-05-11T12:11:58.432452+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:25:15.336149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.429278+0000","last_change":"2024-05-11T12:11:57.433051+0000","last_active":"2024-05-11T12:11:58.429278+0000","last_peered":"2024-05-11T12:11:58.429278+0000","last_clean":"2024-05-11T12:11:58.429278+0000","last_became_active":"2024-05-11T12:11:57.432381+0000","last_became_peered":"2024-05-11T12:11:57.432381+0000","last_unstale":"2024-05-11T12:11:58.429278+0000","last_undegraded":"2024-05-11T12:11:58.429278+0000","last_fullsized":"2024-05-11T12:11:58.429278+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:03:20.730869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:12:00.075950+0000","last_change":"2024-05-11T12:11:56.406587+0000","last_active":"2024-05-11T12:12:00.075950+0000","last_peered":"2024-05-11T12:12:00.075950+0000","last_clean":"2024-05-11T12:12:00.075950+0000","last_became_active":"2024-05-11T12:11:56.405746+0000","last_became_peered":"2024-05-11T12:11:56.405746+0000","last_unstale":"2024-05-11T12:12:00.075950+0000","last_undegraded":"2024-05-11T12:12:00.075950+0000","last_fullsized":"2024-05-11T12:12:00.075950+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:11:10.100041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.436332+0000","last_change":"2024-05-11T12:11:57.438714+0000","last_active":"2024-05-11T12:11:58.436332+0000","last_peered":"2024-05-11T12:11:58.436332+0000","last_clean":"2024-05-11T12:11:58.436332+0000","last_became_active":"2024-05-11T12:11:57.437691+0000","last_became_peered":"2024-05-11T12:11:57.437691+0000","last_unstale":"2024-05-11T12:11:58.436332+0000","last_undegraded":"2024-05-11T12:11:58.436332+0000","last_fullsized":"2024-05-11T12:11:58.436332+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:42:57.665986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.431241+0000","last_change":"2024-05-11T12:11:56.398172+0000","last_active":"2024-05-11T12:11:58.431241+0000","last_peered":"2024-05-11T12:11:58.431241+0000","last_clean":"2024-05-11T12:11:58.431241+0000","last_became_active":"2024-05-11T12:11:56.397860+0000","last_became_peered":"2024-05-11T12:11:56.397860+0000","last_unstale":"2024-05-11T12:11:58.431241+0000","last_undegraded":"2024-05-11T12:11:58.431241+0000","last_fullsized":"2024-05-11T12:11:58.431241+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:51:42.635321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:12:00.076942+0000","last_change":"2024-05-11T12:11:57.442952+0000","last_active":"2024-05-11T12:12:00.076942+0000","last_peered":"2024-05-11T12:12:00.076942+0000","last_clean":"2024-05-11T12:12:00.076942+0000","last_became_active":"2024-05-11T12:11:57.436880+0000","last_became_peered":"2024-05-11T12:11:57.436880+0000","last_unstale":"2024-05-11T12:12:00.076942+0000","last_undegraded":"2024-05-11T12:12:00.076942+0000","last_fullsized":"2024-05-11T12:12:00.076942+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:11:37.571584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.432578+0000","last_change":"2024-05-11T12:11:56.411981+0000","last_active":"2024-05-11T12:11:58.432578+0000","last_peered":"2024-05-11T12:11:58.432578+0000","last_clean":"2024-05-11T12:11:58.432578+0000","last_became_active":"2024-05-11T12:11:56.409613+0000","last_became_peered":"2024-05-11T12:11:56.409613+0000","last_unstale":"2024-05-11T12:11:58.432578+0000","last_undegraded":"2024-05-11T12:11:58.432578+0000","last_fullsized":"2024-05-11T12:11:58.432578+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:46:05.160542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:59.749748+0000","last_change":"2024-05-11T12:11:57.429655+0000","last_active":"2024-05-11T12:11:59.749748+0000","last_peered":"2024-05-11T12:11:59.749748+0000","last_clean":"2024-05-11T12:11:59.749748+0000","last_became_active":"2024-05-11T12:11:57.429394+0000","last_became_peered":"2024-05-11T12:11:57.429394+0000","last_unstale":"2024-05-11T12:11:59.749748+0000","last_undegraded":"2024-05-11T12:11:59.749748+0000","last_fullsized":"2024-05-11T12:11:59.749748+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:32:26.804013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"40'3","reported_seq":17,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.432042+0000","last_change":"2024-05-11T12:11:56.407591+0000","last_active":"2024-05-11T12:11:58.432042+0000","last_peered":"2024-05-11T12:11:58.432042+0000","last_clean":"2024-05-11T12:11:58.432042+0000","last_became_active":"2024-05-11T12:11:56.407173+0000","last_became_peered":"2024-05-11T12:11:56.407173+0000","last_unstale":"2024-05-11T12:11:58.432042+0000","last_undegraded":"2024-05-11T12:11:58.432042+0000","last_fullsized":"2024-05-11T12:11:58.432042+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:52:13.234125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:12:00.078092+0000","last_change":"2024-05-11T12:11:57.447257+0000","last_active":"2024-05-11T12:12:00.078092+0000","last_peered":"2024-05-11T12:12:00.078092+0000","last_clean":"2024-05-11T12:12:00.078092+0000","last_became_active":"2024-05-11T12:11:57.445351+0000","last_became_peered":"2024-05-11T12:11:57.445351+0000","last_unstale":"2024-05-11T12:12:00.078092+0000","last_undegraded":"2024-05-11T12:12:00.078092+0000","last_fullsized":"2024-05-11T12:12:00.078092+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:52:14.248503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"40'6","reported_seq":20,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.430111+0000","last_change":"2024-05-11T12:11:56.394318+0000","last_active":"2024-05-11T12:11:58.430111+0000","last_peered":"2024-05-11T12:11:58.430111+0000","last_clean":"2024-05-11T12:11:58.430111+0000","last_became_active":"2024-05-11T12:11:56.393560+0000","last_became_peered":"2024-05-11T12:11:56.393560+0000","last_unstale":"2024-05-11T12:11:58.430111+0000","last_undegraded":"2024-05-11T12:11:58.430111+0000","last_fullsized":"2024-05-11T12:11:58.430111+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:26:33.379326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:59.749673+0000","last_change":"2024-05-11T12:11:57.440633+0000","last_active":"2024-05-11T12:11:59.749673+0000","last_peered":"2024-05-11T12:11:59.749673+0000","last_clean":"2024-05-11T12:11:59.749673+0000","last_became_active":"2024-05-11T12:11:57.435381+0000","last_became_peered":"2024-05-11T12:11:57.435381+0000","last_unstale":"2024-05-11T12:11:59.749673+0000","last_undegraded":"2024-05-11T12:11:59.749673+0000","last_fullsized":"2024-05-11T12:11:59.749673+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:47:49.623473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.429956+0000","last_change":"2024-05-11T12:11:56.397825+0000","last_active":"2024-05-11T12:11:58.429956+0000","last_peered":"2024-05-11T12:11:58.429956+0000","last_clean":"2024-05-11T12:11:58.429956+0000","last_became_active":"2024-05-11T12:11:56.397506+0000","last_became_peered":"2024-05-11T12:11:56.397506+0000","last_unstale":"2024-05-11T12:11:58.429956+0000","last_undegraded":"2024-05-11T12:11:58.429956+0000","last_fullsized":"2024-05-11T12:11:58.429956+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:53:02.466538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.429706+0000","last_change":"2024-05-11T12:11:57.426061+0000","last_active":"2024-05-11T12:11:58.429706+0000","last_peered":"2024-05-11T12:11:58.429706+0000","last_clean":"2024-05-11T12:11:58.429706+0000","last_became_active":"2024-05-11T12:11:57.425702+0000","last_became_peered":"2024-05-11T12:11:57.425702+0000","last_unstale":"2024-05-11T12:11:58.429706+0000","last_undegraded":"2024-05-11T12:11:58.429706+0000","last_fullsized":"2024-05-11T12:11:58.429706+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:27:58.937922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.432528+0000","last_change":"2024-05-11T12:11:56.409815+0000","last_active":"2024-05-11T12:11:58.432528+0000","last_peered":"2024-05-11T12:11:58.432528+0000","last_clean":"2024-05-11T12:11:58.432528+0000","last_became_active":"2024-05-11T12:11:56.409411+0000","last_became_peered":"2024-05-11T12:11:56.409411+0000","last_unstale":"2024-05-11T12:11:58.432528+0000","last_undegraded":"2024-05-11T12:11:58.432528+0000","last_fullsized":"2024-05-11T12:11:58.432528+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:36:40.785205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:12:00.076527+0000","last_change":"2024-05-11T12:11:57.442669+0000","last_active":"2024-05-11T12:12:00.076527+0000","last_peered":"2024-05-11T12:12:00.076527+0000","last_clean":"2024-05-11T12:12:00.076527+0000","last_became_active":"2024-05-11T12:11:57.437559+0000","last_became_peered":"2024-05-11T12:11:57.437559+0000","last_unstale":"2024-05-11T12:12:00.076527+0000","last_undegraded":"2024-05-11T12:12:00.076527+0000","last_fullsized":"2024-05-11T12:12:00.076527+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:17:59.309336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:12:00.077291+0000","last_change":"2024-05-11T12:11:56.408411+0000","last_active":"2024-05-11T12:12:00.077291+0000","last_peered":"2024-05-11T12:12:00.077291+0000","last_clean":"2024-05-11T12:12:00.077291+0000","last_became_active":"2024-05-11T12:11:56.408100+0000","last_became_peered":"2024-05-11T12:11:56.408100+0000","last_unstale":"2024-05-11T12:12:00.077291+0000","last_undegraded":"2024-05-11T12:12:00.077291+0000","last_fullsized":"2024-05-11T12:12:00.077291+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:25:15.336149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.429718+0000","last_change":"2024-05-11T12:11:57.430055+0000","last_active":"2024-05-11T12:11:58.429718+0000","last_peered":"2024-05-11T12:11:58.429718+0000","last_clean":"2024-05-11T12:11:58.429718+0000","last_became_active":"2024-05-11T12:11:57.426614+0000","last_became_peered":"2024-05-11T12:11:57.426614+0000","last_unstale":"2024-05-11T12:11:58.429718+0000","last_undegraded":"2024-05-11T12:11:58.429718+0000","last_fullsized":"2024-05-11T12:11:58.429718+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:58:40.171928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"40'8","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:12:00.077240+0000","last_change":"2024-05-11T12:11:56.398517+0000","last_active":"2024-05-11T12:12:00.077240+0000","last_peered":"2024-05-11T12:12:00.077240+0000","last_clean":"2024-05-11T12:12:00.077240+0000","last_became_active":"2024-05-11T12:11:56.398204+0000","last_became_peered":"2024-05-11T12:11:56.398204+0000","last_unstale":"2024-05-11T12:12:00.077240+0000","last_undegraded":"2024-05-11T12:12:00.077240+0000","last_fullsized":"2024-05-11T12:12:00.077240+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:25:15.336149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.429946+0000","last_change":"2024-05-11T12:11:57.422076+0000","last_active":"2024-05-11T12:11:58.429946+0000","last_peered":"2024-05-11T12:11:58.429946+0000","last_clean":"2024-05-11T12:11:58.429946+0000","last_became_active":"2024-05-11T12:11:57.421535+0000","last_became_peered":"2024-05-11T12:11:57.421535+0000","last_unstale":"2024-05-11T12:11:58.429946+0000","last_undegraded":"2024-05-11T12:11:58.429946+0000","last_fullsized":"2024-05-11T12:11:58.429946+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:07:26.210928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"40'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.431179+0000","last_change":"2024-05-11T12:11:56.410975+0000","last_active":"2024-05-11T12:11:58.431179+0000","last_peered":"2024-05-11T12:11:58.431179+0000","last_clean":"2024-05-11T12:11:58.431179+0000","last_became_active":"2024-05-11T12:11:56.410634+0000","last_became_peered":"2024-05-11T12:11:56.410634+0000","last_unstale":"2024-05-11T12:11:58.431179+0000","last_undegraded":"2024-05-11T12:11:58.431179+0000","last_fullsized":"2024-05-11T12:11:58.431179+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:10:04.020340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.432003+0000","last_change":"2024-05-11T12:11:57.428161+0000","last_active":"2024-05-11T12:11:58.432003+0000","last_peered":"2024-05-11T12:11:58.432003+0000","last_clean":"2024-05-11T12:11:58.432003+0000","last_became_active":"2024-05-11T12:11:57.427826+0000","last_became_peered":"2024-05-11T12:11:57.427826+0000","last_unstale":"2024-05-11T12:11:58.432003+0000","last_undegraded":"2024-05-11T12:11:58.432003+0000","last_fullsized":"2024-05-11T12:11:58.432003+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:27:46.224897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:12:00.078711+0000","last_change":"2024-05-11T12:11:56.406473+0000","last_active":"2024-05-11T12:12:00.078711+0000","last_peered":"2024-05-11T12:12:00.078711+0000","last_clean":"2024-05-11T12:12:00.078711+0000","last_became_active":"2024-05-11T12:11:56.405923+0000","last_became_peered":"2024-05-11T12:11:56.405923+0000","last_unstale":"2024-05-11T12:12:00.078711+0000","last_undegraded":"2024-05-11T12:12:00.078711+0000","last_fullsized":"2024-05-11T12:12:00.078711+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:11:42.632883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:12:00.078134+0000","last_change":"2024-05-11T12:11:57.436042+0000","last_active":"2024-05-11T12:12:00.078134+0000","last_peered":"2024-05-11T12:12:00.078134+0000","last_clean":"2024-05-11T12:12:00.078134+0000","last_became_active":"2024-05-11T12:11:57.435781+0000","last_became_peered":"2024-05-11T12:11:57.435781+0000","last_unstale":"2024-05-11T12:12:00.078134+0000","last_undegraded":"2024-05-11T12:12:00.078134+0000","last_fullsized":"2024-05-11T12:12:00.078134+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:10:05.034718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"40'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.436439+0000","last_change":"2024-05-11T12:11:56.406200+0000","last_active":"2024-05-11T12:11:58.436439+0000","last_peered":"2024-05-11T12:11:58.436439+0000","last_clean":"2024-05-11T12:11:58.436439+0000","last_became_active":"2024-05-11T12:11:56.405912+0000","last_became_peered":"2024-05-11T12:11:56.405912+0000","last_unstale":"2024-05-11T12:11:58.436439+0000","last_undegraded":"2024-05-11T12:11:58.436439+0000","last_fullsized":"2024-05-11T12:11:58.436439+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:31:16.917994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.436708+0000","last_change":"2024-05-11T12:11:57.440226+0000","last_active":"2024-05-11T12:11:58.436708+0000","last_peered":"2024-05-11T12:11:58.436708+0000","last_clean":"2024-05-11T12:11:58.436708+0000","last_became_active":"2024-05-11T12:11:57.438861+0000","last_became_peered":"2024-05-11T12:11:57.438861+0000","last_unstale":"2024-05-11T12:11:58.436708+0000","last_undegraded":"2024-05-11T12:11:58.436708+0000","last_fullsized":"2024-05-11T12:11:58.436708+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:23:47.093149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.432963+0000","last_change":"2024-05-11T12:11:56.409279+0000","last_active":"2024-05-11T12:11:58.432963+0000","last_peered":"2024-05-11T12:11:58.432963+0000","last_clean":"2024-05-11T12:11:58.432963+0000","last_became_active":"2024-05-11T12:11:56.408985+0000","last_became_peered":"2024-05-11T12:11:56.408985+0000","last_unstale":"2024-05-11T12:11:58.432963+0000","last_undegraded":"2024-05-11T12:11:58.432963+0000","last_fullsized":"2024-05-11T12:11:58.432963+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:30:56.816516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:12:00.077734+0000","last_change":"2024-05-11T12:11:57.443914+0000","last_active":"2024-05-11T12:12:00.077734+0000","last_peered":"2024-05-11T12:12:00.077734+0000","last_clean":"2024-05-11T12:12:00.077734+0000","last_became_active":"2024-05-11T12:11:57.443680+0000","last_became_peered":"2024-05-11T12:11:57.443680+0000","last_unstale":"2024-05-11T12:12:00.077734+0000","last_undegraded":"2024-05-11T12:12:00.077734+0000","last_fullsized":"2024-05-11T12:12:00.077734+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:22:33.211172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.435987+0000","last_change":"2024-05-11T12:11:56.398721+0000","last_active":"2024-05-11T12:11:58.435987+0000","last_peered":"2024-05-11T12:11:58.435987+0000","last_clean":"2024-05-11T12:11:58.435987+0000","last_became_active":"2024-05-11T12:11:56.398181+0000","last_became_peered":"2024-05-11T12:11:56.398181+0000","last_unstale":"2024-05-11T12:11:58.435987+0000","last_undegraded":"2024-05-11T12:11:58.435987+0000","last_fullsized":"2024-05-11T12:11:58.435987+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:24:06.036447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:59.749297+0000","last_change":"2024-05-11T12:11:57.436889+0000","last_active":"2024-05-11T12:11:59.749297+0000","last_peered":"2024-05-11T12:11:59.749297+0000","last_clean":"2024-05-11T12:11:59.749297+0000","last_became_active":"2024-05-11T12:11:57.436592+0000","last_became_peered":"2024-05-11T12:11:57.436592+0000","last_unstale":"2024-05-11T12:11:59.749297+0000","last_undegraded":"2024-05-11T12:11:59.749297+0000","last_fullsized":"2024-05-11T12:11:59.749297+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:11:37.571584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.436163+0000","last_change":"2024-05-11T12:11:56.409035+0000","last_active":"2024-05-11T12:11:58.436163+0000","last_peered":"2024-05-11T12:11:58.436163+0000","last_clean":"2024-05-11T12:11:58.436163+0000","last_became_active":"2024-05-11T12:11:56.408751+0000","last_became_peered":"2024-05-11T12:11:56.408751+0000","last_unstale":"2024-05-11T12:11:58.436163+0000","last_undegraded":"2024-05-11T12:11:58.436163+0000","last_fullsized":"2024-05-11T12:11:58.436163+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:11:47.612399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.432400+0000","last_change":"2024-05-11T12:11:57.419098+0000","last_active":"2024-05-11T12:11:58.432400+0000","last_peered":"2024-05-11T12:11:58.432400+0000","last_clean":"2024-05-11T12:11:58.432400+0000","last_became_active":"2024-05-11T12:11:57.417628+0000","last_became_peered":"2024-05-11T12:11:57.417628+0000","last_unstale":"2024-05-11T12:11:58.432400+0000","last_undegraded":"2024-05-11T12:11:58.432400+0000","last_fullsized":"2024-05-11T12:11:58.432400+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:46:47.695505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:12:00.077629+0000","last_change":"2024-05-11T12:11:56.395623+0000","last_active":"2024-05-11T12:12:00.077629+0000","last_peered":"2024-05-11T12:12:00.077629+0000","last_clean":"2024-05-11T12:12:00.077629+0000","last_became_active":"2024-05-11T12:11:56.395250+0000","last_became_peered":"2024-05-11T12:11:56.395250+0000","last_unstale":"2024-05-11T12:12:00.077629+0000","last_undegraded":"2024-05-11T12:12:00.077629+0000","last_fullsized":"2024-05-11T12:12:00.077629+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:30:56.816516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.430168+0000","last_change":"2024-05-11T12:11:57.433272+0000","last_active":"2024-05-11T12:11:58.430168+0000","last_peered":"2024-05-11T12:11:58.430168+0000","last_clean":"2024-05-11T12:11:58.430168+0000","last_became_active":"2024-05-11T12:11:57.432767+0000","last_became_peered":"2024-05-11T12:11:57.432767+0000","last_unstale":"2024-05-11T12:11:58.430168+0000","last_undegraded":"2024-05-11T12:11:58.430168+0000","last_fullsized":"2024-05-11T12:11:58.430168+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:27:46.224897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.435741+0000","last_change":"2024-05-11T12:11:56.400668+0000","last_active":"2024-05-11T12:11:58.435741+0000","last_peered":"2024-05-11T12:11:58.435741+0000","last_clean":"2024-05-11T12:11:58.435741+0000","last_became_active":"2024-05-11T12:11:56.397950+0000","last_became_peered":"2024-05-11T12:11:56.397950+0000","last_unstale":"2024-05-11T12:11:58.435741+0000","last_undegraded":"2024-05-11T12:11:58.435741+0000","last_fullsized":"2024-05-11T12:11:58.435741+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:13:01.143192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.433130+0000","last_change":"2024-05-11T12:11:57.434141+0000","last_active":"2024-05-11T12:11:58.433130+0000","last_peered":"2024-05-11T12:11:58.433130+0000","last_clean":"2024-05-11T12:11:58.433130+0000","last_became_active":"2024-05-11T12:11:57.433852+0000","last_became_peered":"2024-05-11T12:11:57.433852+0000","last_unstale":"2024-05-11T12:11:58.433130+0000","last_undegraded":"2024-05-11T12:11:58.433130+0000","last_fullsized":"2024-05-11T12:11:58.433130+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:53:03.480916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.431383+0000","last_change":"2024-05-11T12:11:56.409581+0000","last_active":"2024-05-11T12:11:58.431383+0000","last_peered":"2024-05-11T12:11:58.431383+0000","last_clean":"2024-05-11T12:11:58.431383+0000","last_became_active":"2024-05-11T12:11:56.408010+0000","last_became_peered":"2024-05-11T12:11:56.408010+0000","last_unstale":"2024-05-11T12:11:58.431383+0000","last_undegraded":"2024-05-11T12:11:58.431383+0000","last_fullsized":"2024-05-11T12:11:58.431383+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:20:13.124057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:12:00.077372+0000","last_change":"2024-05-11T12:11:57.447191+0000","last_active":"2024-05-11T12:12:00.077372+0000","last_peered":"2024-05-11T12:12:00.077372+0000","last_clean":"2024-05-11T12:12:00.077372+0000","last_became_active":"2024-05-11T12:11:57.442656+0000","last_became_peered":"2024-05-11T12:11:57.442656+0000","last_unstale":"2024-05-11T12:12:00.077372+0000","last_undegraded":"2024-05-11T12:12:00.077372+0000","last_fullsized":"2024-05-11T12:12:00.077372+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:32:26.804013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"41'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.436582+0000","last_change":"2024-05-11T12:11:56.404335+0000","last_active":"2024-05-11T12:11:58.436582+0000","last_peered":"2024-05-11T12:11:58.436582+0000","last_clean":"2024-05-11T12:11:58.436582+0000","last_became_active":"2024-05-11T12:11:56.404069+0000","last_became_peered":"2024-05-11T12:11:56.404069+0000","last_unstale":"2024-05-11T12:11:58.436582+0000","last_undegraded":"2024-05-11T12:11:58.436582+0000","last_fullsized":"2024-05-11T12:11:58.436582+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:50:43.515250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.430571+0000","last_change":"2024-05-11T12:11:57.432694+0000","last_active":"2024-05-11T12:11:58.430571+0000","last_peered":"2024-05-11T12:11:58.430571+0000","last_clean":"2024-05-11T12:11:58.430571+0000","last_became_active":"2024-05-11T12:11:57.431359+0000","last_became_peered":"2024-05-11T12:11:57.431359+0000","last_unstale":"2024-05-11T12:11:58.430571+0000","last_undegraded":"2024-05-11T12:11:58.430571+0000","last_fullsized":"2024-05-11T12:11:58.430571+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:12:00.162149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:59.749622+0000","last_change":"2024-05-11T12:11:56.397553+0000","last_active":"2024-05-11T12:11:59.749622+0000","last_peered":"2024-05-11T12:11:59.749622+0000","last_clean":"2024-05-11T12:11:59.749622+0000","last_became_active":"2024-05-11T12:11:56.397219+0000","last_became_peered":"2024-05-11T12:11:56.397219+0000","last_unstale":"2024-05-11T12:11:59.749622+0000","last_undegraded":"2024-05-11T12:11:59.749622+0000","last_fullsized":"2024-05-11T12:11:59.749622+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:07:21.760158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:59.749916+0000","last_change":"2024-05-11T12:11:57.433279+0000","last_active":"2024-05-11T12:11:59.749916+0000","last_peered":"2024-05-11T12:11:59.749916+0000","last_clean":"2024-05-11T12:11:59.749916+0000","last_became_active":"2024-05-11T12:11:57.432993+0000","last_became_peered":"2024-05-11T12:11:57.432993+0000","last_unstale":"2024-05-11T12:11:59.749916+0000","last_undegraded":"2024-05-11T12:11:59.749916+0000","last_fullsized":"2024-05-11T12:11:59.749916+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:22:33.211172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:59.749802+0000","last_change":"2024-05-11T12:11:56.404006+0000","last_active":"2024-05-11T12:11:59.749802+0000","last_peered":"2024-05-11T12:11:59.749802+0000","last_clean":"2024-05-11T12:11:59.749802+0000","last_became_active":"2024-05-11T12:11:56.403672+0000","last_became_peered":"2024-05-11T12:11:56.403672+0000","last_unstale":"2024-05-11T12:11:59.749802+0000","last_undegraded":"2024-05-11T12:11:59.749802+0000","last_fullsized":"2024-05-11T12:11:59.749802+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:28:43.572256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.432618+0000","last_change":"2024-05-11T12:11:57.436933+0000","last_active":"2024-05-11T12:11:58.432618+0000","last_peered":"2024-05-11T12:11:58.432618+0000","last_clean":"2024-05-11T12:11:58.432618+0000","last_became_active":"2024-05-11T12:11:57.435829+0000","last_became_peered":"2024-05-11T12:11:57.435829+0000","last_unstale":"2024-05-11T12:11:58.432618+0000","last_undegraded":"2024-05-11T12:11:58.432618+0000","last_fullsized":"2024-05-11T12:11:58.432618+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:46:06.174919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:12:00.077781+0000","last_change":"2024-05-11T12:11:56.399122+0000","last_active":"2024-05-11T12:12:00.077781+0000","last_peered":"2024-05-11T12:12:00.077781+0000","last_clean":"2024-05-11T12:12:00.077781+0000","last_became_active":"2024-05-11T12:11:56.398845+0000","last_became_peered":"2024-05-11T12:11:56.398845+0000","last_unstale":"2024-05-11T12:12:00.077781+0000","last_undegraded":"2024-05-11T12:12:00.077781+0000","last_fullsized":"2024-05-11T12:12:00.077781+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:16:00.825967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:12:00.079641+0000","last_change":"2024-05-11T12:11:57.442240+0000","last_active":"2024-05-11T12:12:00.079641+0000","last_peered":"2024-05-11T12:12:00.079641+0000","last_clean":"2024-05-11T12:12:00.079641+0000","last_became_active":"2024-05-11T12:11:57.440831+0000","last_became_peered":"2024-05-11T12:11:57.440831+0000","last_unstale":"2024-05-11T12:12:00.079641+0000","last_undegraded":"2024-05-11T12:12:00.079641+0000","last_fullsized":"2024-05-11T12:12:00.079641+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:26:34.393703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:12:00.076175+0000","last_change":"2024-05-11T12:11:56.402880+0000","last_active":"2024-05-11T12:12:00.076175+0000","last_peered":"2024-05-11T12:12:00.076175+0000","last_clean":"2024-05-11T12:12:00.076175+0000","last_became_active":"2024-05-11T12:11:56.402608+0000","last_became_peered":"2024-05-11T12:11:56.402608+0000","last_unstale":"2024-05-11T12:12:00.076175+0000","last_undegraded":"2024-05-11T12:12:00.076175+0000","last_fullsized":"2024-05-11T12:12:00.076175+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:34:29.698881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:12:00.078634+0000","last_change":"2024-05-11T12:11:57.444173+0000","last_active":"2024-05-11T12:12:00.078634+0000","last_peered":"2024-05-11T12:12:00.078634+0000","last_clean":"2024-05-11T12:12:00.078634+0000","last_became_active":"2024-05-11T12:11:57.441809+0000","last_became_peered":"2024-05-11T12:11:57.441809+0000","last_unstale":"2024-05-11T12:12:00.078634+0000","last_undegraded":"2024-05-11T12:12:00.078634+0000","last_fullsized":"2024-05-11T12:12:00.078634+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:04:26.168833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.428722+0000","last_change":"2024-05-11T12:11:56.399468+0000","last_active":"2024-05-11T12:11:58.428722+0000","last_peered":"2024-05-11T12:11:58.428722+0000","last_clean":"2024-05-11T12:11:58.428722+0000","last_became_active":"2024-05-11T12:11:56.399205+0000","last_became_peered":"2024-05-11T12:11:56.399205+0000","last_unstale":"2024-05-11T12:11:58.428722+0000","last_undegraded":"2024-05-11T12:11:58.428722+0000","last_fullsized":"2024-05-11T12:11:58.428722+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:46:05.160542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:11:58.430983+0000","last_change":"2024-05-11T12:11:57.417010+0000","last_active":"2024-05-11T12:11:58.430983+0000","last_peered":"2024-05-11T12:11:58.430983+0000","last_clean":"2024-05-11T12:11:58.430983+0000","last_became_active":"2024-05-11T12:11:57.416499+0000","last_became_peered":"2024-05-11T12:11:57.416499+0000","last_unstale":"2024-05-11T12:11:58.430983+0000","last_undegraded":"2024-05-11T12:11:58.430983+0000","last_fullsized":"2024-05-11T12:11:58.430983+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:56.360286+0000","last_clean_scrub_stamp":"2024-05-11T12:11:56.360286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:03:20.730869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:12:00.077851+0000","last_change":"2024-05-11T12:11:56.400810+0000","last_active":"2024-05-11T12:12:00.077851+0000","last_peered":"2024-05-11T12:12:00.077851+0000","last_clean":"2024-05-11T12:12:00.077851+0000","last_became_active":"2024-05-11T12:11:56.399824+0000","last_became_peered":"2024-05-11T12:11:56.399824+0000","last_unstale":"2024-05-11T12:12:00.077851+0000","last_undegraded":"2024-05-11T12:12:00.077851+0000","last_fullsized":"2024-05-11T12:12:00.077851+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:11:55.345908+0000","last_clean_scrub_stamp":"2024-05-11T12:11:55.345908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:07:21.760158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":141,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T12:12:00.077973+0000","last_change":"2024-05-11T12:10:35.066673+0000","last_active":"2024-05-11T12:12:00.077973+0000","last_peered":"2024-05-11T12:12:00.077973+0000","last_clean":"2024-05-11T12:12:00.077973+0000","last_became_active":"2024-05-11T12:10:35.065942+0000","last_became_peered":"2024-05-11T12:10:35.065942+0000","last_unstale":"2024-05-11T12:12:00.077973+0000","last_undegraded":"2024-05-11T12:12:00.077973+0000","last_fullsized":"2024-05-11T12:12:00.077973+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_clean_scrub_stamp":"2024-05-11T12:10:33.601136+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:12:02.133182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476757,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28068,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343772,"statfs":{"total":96636764160,"available":96608022528,"internally_reserved":0,"allocated":1073152,"data_stored":790987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476758,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":466944,"data_stored":199519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476757,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":471040,"data_stored":199049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476759,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":471040,"data_stored":199541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":479232,"data_stored":200583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509463,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":466944,"data_stored":199027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509462,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1060864,"data_stored":789339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542167,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":466944,"data_stored":199007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T12:12:07.379 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-11T12:12:07.379 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-11T12:12:07.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.626+0000 7fafc3400640 1 Processor -- start 2024-05-11T12:12:07.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.626+0000 7fafc3400640 1 -- start start 2024-05-11T12:12:07.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.628+0000 7fafc3400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fafbc15f430 0x7fafbc06e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:07.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.628+0000 7fafc3400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fafbc0d03b0 0x7fafbc06e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:07.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.628+0000 7fafc3400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafbc0cf0f0 con 0x7fafbc0b59f0 2024-05-11T12:12:07.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.628+0000 7fafc3400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fafbc06c450 con 0x7fafbc0d03b0 2024-05-11T12:12:07.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.628+0000 7fafc3400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fafbc06d2b0 con 0x7fafbc15f430 2024-05-11T12:12:07.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.629+0000 7fafc1400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fafbc0d03b0 0x7fafbc06e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:07.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.629+0000 7fafc0a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fafbc15f430 0x7fafbc06e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:07.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.629+0000 7fafbbe00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fafbc0b59f0 0x7fafbc162f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.167:41776/0 (socket says 172.21.15.167:41776) 2024-05-11T12:12:07.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.629+0000 7fafbbe00640 1 -- 172.21.15.167:0/3176241063 learned_addr learned my addr 172.21.15.167:0/3176241063 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:07.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.630+0000 7fafc1400640 1 -- 172.21.15.167:0/3176241063 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fafbc15f430 msgr2=0x7fafbc06e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:07.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.630+0000 7fafc1400640 1 --2- 172.21.15.167:0/3176241063 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fafbc15f430 0x7fafbc06e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:07.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.630+0000 7fafc1400640 1 -- 172.21.15.167:0/3176241063 >> v1:172.21.15.204:6789/0 conn(0x7fafbc0b59f0 legacy=0x7fafbc162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:07.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.630+0000 7fafc1400640 1 -- 172.21.15.167:0/3176241063 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafbc06c6a0 con 0x7fafbc0d03b0 2024-05-11T12:12:07.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.630+0000 7fafc0a00640 1 --2- 172.21.15.167:0/3176241063 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fafbc15f430 0x7fafbc06e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:12:07.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.631+0000 7fafc1400640 1 --2- 172.21.15.167:0/3176241063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafbc0d03b0 0x7fafbc06e890 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fafb40097a0 tx=0x7fafb402a0e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d797458cba6e544 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:07.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.632+0000 7fafbb400640 1 -- 172.21.15.167:0/3176241063 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb4037070 con 0x7fafbc0d03b0 2024-05-11T12:12:07.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.632+0000 7fafbb400640 1 -- 172.21.15.167:0/3176241063 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafb402ae30 con 0x7fafbc0d03b0 2024-05-11T12:12:07.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.632+0000 7fafbb400640 1 -- 172.21.15.167:0/3176241063 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb4032640 con 0x7fafbc0d03b0 2024-05-11T12:12:07.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.633+0000 7fafc3400640 1 -- 172.21.15.167:0/3176241063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafbc0d03b0 msgr2=0x7fafbc06e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:07.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.633+0000 7fafc3400640 1 --2- 172.21.15.167:0/3176241063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafbc0d03b0 0x7fafbc06e890 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fafb40097a0 tx=0x7fafb402a0e0 comp rx=0 tx=0).stop 2024-05-11T12:12:07.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.633+0000 7fafc3400640 1 -- 172.21.15.167:0/3176241063 shutdown_connections 2024-05-11T12:12:07.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.633+0000 7fafc3400640 1 --2- 172.21.15.167:0/3176241063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafbc0d03b0 0x7fafbc06e890 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:07.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.633+0000 7fafc3400640 1 --2- 172.21.15.167:0/3176241063 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fafbc15f430 0x7fafbc06e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:07.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.633+0000 7fafc3400640 1 -- 172.21.15.167:0/3176241063 >> 172.21.15.167:0/3176241063 conn(0x7fafbc0b2f60 msgr2=0x7fafbc0b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:07.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.633+0000 7fafc3400640 1 -- 172.21.15.167:0/3176241063 shutdown_connections 2024-05-11T12:12:07.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.633+0000 7fafc3400640 1 -- 172.21.15.167:0/3176241063 wait complete. 2024-05-11T12:12:07.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.634+0000 7fafc3400640 1 Processor -- start 2024-05-11T12:12:07.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.635+0000 7fafc3400640 1 -- start start 2024-05-11T12:12:07.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.635+0000 7fafc3400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fafbc0b59f0 0x7fafbc148f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:07.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.635+0000 7fafc3400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafbc0d03b0 0x7fafbc15a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:07.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.636+0000 7fafc3400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafbc15f430 0x7fafbc15bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:07.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.636+0000 7fafc3400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc06cf40 con 0x7fafbc0d03b0 2024-05-11T12:12:07.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.636+0000 7fafc3400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc06cc60 con 0x7fafbc15f430 2024-05-11T12:12:07.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.636+0000 7fafc3400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fafbc0ae820 con 0x7fafbc0b59f0 2024-05-11T12:12:07.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.636+0000 7fafbbe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafbc0d03b0 0x7fafbc15a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:07.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.636+0000 7fafbbe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafbc0d03b0 0x7fafbc15a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41948/0 (socket says 172.21.15.167:41948) 2024-05-11T12:12:07.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.636+0000 7fafbbe00640 1 -- 172.21.15.167:0/999763780 learned_addr learned my addr 172.21.15.167:0/999763780 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:07.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.636+0000 7fafc0a00640 1 --2- 172.21.15.167:0/999763780 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fafbc0b59f0 0x7fafbc148f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:07.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.636+0000 7fafc1400640 1 --2- 172.21.15.167:0/999763780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafbc15f430 0x7fafbc15bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:07.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.636+0000 7fafbbe00640 1 -- 172.21.15.167:0/999763780 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fafbc0b59f0 msgr2=0x7fafbc148f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:07.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.636+0000 7fafbbe00640 1 --2- 172.21.15.167:0/999763780 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fafbc0b59f0 0x7fafbc148f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:07.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.636+0000 7fafbbe00640 1 -- 172.21.15.167:0/999763780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafbc15f430 msgr2=0x7fafbc15bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:07.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.636+0000 7fafbbe00640 1 --2- 172.21.15.167:0/999763780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafbc15f430 0x7fafbc15bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:07.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.636+0000 7fafbbe00640 1 -- 172.21.15.167:0/999763780 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafbc15aa60 con 0x7fafbc0d03b0 2024-05-11T12:12:07.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.637+0000 7fafbbe00640 1 --2- 172.21.15.167:0/999763780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafbc0d03b0 0x7fafbc15a520 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fafa800c500 tx=0x7fafa800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:07.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.637+0000 7fafb8c00640 1 -- 172.21.15.167:0/999763780 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa8017070 con 0x7fafbc0d03b0 2024-05-11T12:12:07.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.637+0000 7fafb8c00640 1 -- 172.21.15.167:0/999763780 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafa8005d00 con 0x7fafbc0d03b0 2024-05-11T12:12:07.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.637+0000 7fafc3400640 1 -- 172.21.15.167:0/999763780 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafbc159760 con 0x7fafbc0d03b0 2024-05-11T12:12:07.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.637+0000 7fafb8c00640 1 -- 172.21.15.167:0/999763780 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa8007330 con 0x7fafbc0d03b0 2024-05-11T12:12:07.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.637+0000 7fafc3400640 1 -- 172.21.15.167:0/999763780 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafbc159d20 con 0x7fafbc0d03b0 2024-05-11T12:12:07.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.638+0000 7fafc3400640 1 -- 172.21.15.167:0/999763780 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafbc069630 con 0x7fafbc0d03b0 2024-05-11T12:12:07.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.639+0000 7fafb8c00640 1 -- 172.21.15.167:0/999763780 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fafa801a070 con 0x7fafbc0d03b0 2024-05-11T12:12:07.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.640+0000 7fafb8c00640 1 --2- 172.21.15.167:0/999763780 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7faf9c0762d0 0x7faf9c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:07.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.640+0000 7fafb8c00640 1 -- 172.21.15.167:0/999763780 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7fafa8096c00 con 0x7fafbc0d03b0 2024-05-11T12:12:07.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.640+0000 7fafc0a00640 1 --2- 172.21.15.167:0/999763780 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7faf9c0762d0 0x7faf9c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:07.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.641+0000 7fafc0a00640 1 --2- 172.21.15.167:0/999763780 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7faf9c0762d0 0x7faf9c078790 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fafac009fd0 tx=0x7fafac00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:07.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.645+0000 7fafb8c00640 1 -- 172.21.15.167:0/999763780 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fafbc15aa60 con 0x7fafbc0d03b0 2024-05-11T12:12:07.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.927+0000 7fafc3400640 1 -- 172.21.15.167:0/999763780 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fafbc06da00 con 0x7fafbc0d03b0 2024-05-11T12:12:07.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.931+0000 7fafb8c00640 1 -- 172.21.15.167:0/999763780 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7fafa8060330 con 0x7fafbc0d03b0 2024-05-11T12:12:07.931 INFO:teuthology.orchestra.run.smithi167.stdout:[client.0] 2024-05-11T12:12:07.931 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQAFYD9mLK/lEhAAyERTYMl250KkBvIrAXWYSw== 2024-05-11T12:12:07.931 INFO:teuthology.orchestra.run.smithi167.stdout: caps mds = "allow" 2024-05-11T12:12:07.931 INFO:teuthology.orchestra.run.smithi167.stdout: caps mon = "allow r" 2024-05-11T12:12:07.932 INFO:teuthology.orchestra.run.smithi167.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-11T12:12:07.932 INFO:teuthology.orchestra.run.smithi167.stderr:updated caps for client.0 2024-05-11T12:12:07.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.934+0000 7fafc3400640 1 -- 172.21.15.167:0/999763780 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7faf9c0762d0 msgr2=0x7faf9c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:07.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.934+0000 7fafc3400640 1 --2- 172.21.15.167:0/999763780 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7faf9c0762d0 0x7faf9c078790 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fafac009fd0 tx=0x7fafac00b040 comp rx=0 tx=0).stop 2024-05-11T12:12:07.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.934+0000 7fafc3400640 1 -- 172.21.15.167:0/999763780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafbc0d03b0 msgr2=0x7fafbc15a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:07.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.934+0000 7fafc3400640 1 --2- 172.21.15.167:0/999763780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafbc0d03b0 0x7fafbc15a520 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fafa800c500 tx=0x7fafa800c9c0 comp rx=0 tx=0).stop 2024-05-11T12:12:07.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.934+0000 7fafc3400640 1 -- 172.21.15.167:0/999763780 shutdown_connections 2024-05-11T12:12:07.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.934+0000 7fafc3400640 1 --2- 172.21.15.167:0/999763780 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7faf9c0762d0 0x7faf9c078790 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:07.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.934+0000 7fafc3400640 1 --2- 172.21.15.167:0/999763780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafbc15f430 0x7fafbc15bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:07.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.934+0000 7fafc3400640 1 --2- 172.21.15.167:0/999763780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafbc0d03b0 0x7fafbc15a520 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:07.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.934+0000 7fafc3400640 1 --2- 172.21.15.167:0/999763780 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fafbc0b59f0 0x7fafbc148f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:07.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.934+0000 7fafc3400640 1 -- 172.21.15.167:0/999763780 >> 172.21.15.167:0/999763780 conn(0x7fafbc0b2f60 msgr2=0x7fafbc0aec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:07.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.934+0000 7fafc3400640 1 -- 172.21.15.167:0/999763780 shutdown_connections 2024-05-11T12:12:07.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:07.934+0000 7fafc3400640 1 -- 172.21.15.167:0/999763780 wait complete. 2024-05-11T12:12:07.954 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:12:08.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.150+0000 7f0aada00640 1 Processor -- start 2024-05-11T12:12:08.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.150+0000 7f0aada00640 1 -- start start 2024-05-11T12:12:08.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.151+0000 7f0aada00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0aa80b3ea0 0x7f0aa8161490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:08.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.151+0000 7f0aada00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0aa81619d0 0x7f0aa806d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:08.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.151+0000 7f0aada00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0aa815df30 con 0x7f0aa8157750 2024-05-11T12:12:08.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.151+0000 7f0aada00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f0aa8161000 con 0x7f0aa81619d0 2024-05-11T12:12:08.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.151+0000 7f0aada00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0aa815e210 con 0x7f0aa80b3ea0 2024-05-11T12:12:08.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.152+0000 7f0aa6a00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f0aa8157750 0x7f0aa815d740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:42208/0 (socket says 172.21.15.167:42208) 2024-05-11T12:12:08.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.152+0000 7f0aa6a00640 1 -- 172.21.15.167:0/2473079853 learned_addr learned my addr 172.21.15.167:0/2473079853 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:08.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.152+0000 7f0aa7400640 1 --2- 172.21.15.167:0/2473079853 >> v2:172.21.15.204:3300/0 conn(0x7f0aa81619d0 0x7f0aa806d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:08.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.152+0000 7f0aa6000640 1 --2- 172.21.15.167:0/2473079853 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0aa80b3ea0 0x7f0aa8161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:08.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.152+0000 7f0aa5600640 1 -- 172.21.15.167:0/2473079853 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3404060480 0 0) 0x7f0aa815df30 con 0x7f0aa8157750 2024-05-11T12:12:08.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.152+0000 7f0aa5600640 1 -- 172.21.15.167:0/2473079853 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0a8c003520 con 0x7f0aa8157750 2024-05-11T12:12:08.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.153+0000 7f0aa5600640 1 -- 172.21.15.167:0/2473079853 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3036607665 0 0) 0x7f0a8c003520 con 0x7f0aa8157750 2024-05-11T12:12:08.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.153+0000 7f0aa5600640 1 -- 172.21.15.167:0/2473079853 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0aa80b3ea0 msgr2=0x7f0aa8161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:08.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.153+0000 7f0aa5600640 1 --2- 172.21.15.167:0/2473079853 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0aa80b3ea0 0x7f0aa8161490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:08.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.153+0000 7f0aa5600640 1 -- 172.21.15.167:0/2473079853 >> v2:172.21.15.204:3300/0 conn(0x7f0aa81619d0 msgr2=0x7f0aa806d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:08.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.153+0000 7f0aa5600640 1 --2- 172.21.15.167:0/2473079853 >> v2:172.21.15.204:3300/0 conn(0x7f0aa81619d0 0x7f0aa806d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:08.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.153+0000 7f0aa5600640 1 -- 172.21.15.167:0/2473079853 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aa815e4f0 con 0x7f0aa8157750 2024-05-11T12:12:08.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.153+0000 7f0aa5600640 1 -- 172.21.15.167:0/2473079853 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f0a980016f0 con 0x7f0aa8157750 2024-05-11T12:12:08.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.153+0000 7f0aa6000640 1 --2- 172.21.15.167:0/2473079853 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0aa80b3ea0 0x7f0aa8161490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:12:08.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.153+0000 7f0aa5600640 1 -- 172.21.15.167:0/2473079853 >> v1:172.21.15.167:6789/0 conn(0x7f0aa8157750 legacy=0x7f0aa815d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:08.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.153+0000 7f0aa5600640 1 --2- 172.21.15.167:0/2473079853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a8c003a00 0x7f0a8c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:08.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.153+0000 7f0aa5600640 1 --2- 172.21.15.167:0/2473079853 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0aa80b3ea0 0x7f0a8c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:08.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.153+0000 7f0aa5600640 1 --2- 172.21.15.167:0/2473079853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a8c004d40 0x7f0a8c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:08.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.153+0000 7f0aa5600640 1 -- 172.21.15.167:0/2473079853 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa8161000 con 0x7f0a8c004d40 2024-05-11T12:12:08.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.153+0000 7f0aa5600640 1 -- 172.21.15.167:0/2473079853 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa815e210 con 0x7f0a8c003a00 2024-05-11T12:12:08.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.153+0000 7f0aa5600640 1 -- 172.21.15.167:0/2473079853 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0a8c00ba40 con 0x7f0aa80b3ea0 2024-05-11T12:12:08.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.153+0000 7f0aa7400640 1 --2- 172.21.15.167:0/2473079853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a8c004d40 0x7f0a8c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:08.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.153+0000 7f0aa6a00640 1 --2- 172.21.15.167:0/2473079853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a8c003a00 0x7f0a8c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:08.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.153+0000 7f0aa6000640 1 --2- 172.21.15.167:0/2473079853 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0aa80b3ea0 0x7f0a8c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:08.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.154+0000 7f0aa7400640 1 -- 172.21.15.167:0/2473079853 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0aa80b3ea0 msgr2=0x7f0a8c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:08.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.154+0000 7f0aa7400640 1 --2- 172.21.15.167:0/2473079853 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0aa80b3ea0 0x7f0a8c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:08.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.154+0000 7f0aa7400640 1 -- 172.21.15.167:0/2473079853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a8c003a00 msgr2=0x7f0a8c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:08.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.154+0000 7f0aa7400640 1 --2- 172.21.15.167:0/2473079853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a8c003a00 0x7f0a8c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:08.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.154+0000 7f0aa7400640 1 -- 172.21.15.167:0/2473079853 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a8c003520 con 0x7f0a8c004d40 2024-05-11T12:12:08.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.154+0000 7f0aa6000640 1 --2- 172.21.15.167:0/2473079853 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0aa80b3ea0 0x7f0a8c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:08.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.155+0000 7f0aa7400640 1 --2- 172.21.15.167:0/2473079853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a8c004d40 0x7f0a8c0051c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0009f90 tx=0x7f0a8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b816b0b5d2ff00a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:08.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.155+0000 7f0aa5600640 1 -- 172.21.15.167:0/2473079853 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa0009c10 con 0x7f0a8c004d40 2024-05-11T12:12:08.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.155+0000 7f0aa5600640 1 -- 172.21.15.167:0/2473079853 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0aa0036070 con 0x7f0a8c004d40 2024-05-11T12:12:08.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.155+0000 7f0aa5600640 1 -- 172.21.15.167:0/2473079853 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa0031880 con 0x7f0a8c004d40 2024-05-11T12:12:08.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.155+0000 7f0aada00640 1 -- 172.21.15.167:0/2473079853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a8c004d40 msgr2=0x7f0a8c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:08.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.155+0000 7f0aada00640 1 --2- 172.21.15.167:0/2473079853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a8c004d40 0x7f0a8c0051c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0009f90 tx=0x7f0a8c005830 comp rx=0 tx=0).stop 2024-05-11T12:12:08.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.155+0000 7f0aa7400640 1 -- 172.21.15.167:0/2473079853 reap_dead start 2024-05-11T12:12:08.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.156+0000 7f0aada00640 1 -- 172.21.15.167:0/2473079853 shutdown_connections 2024-05-11T12:12:08.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.156+0000 7f0aada00640 1 -- 172.21.15.167:0/2473079853 >> 172.21.15.167:0/2473079853 conn(0x7f0aa80cd8f0 msgr2=0x7f0aa80cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:08.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.156+0000 7f0aada00640 1 -- 172.21.15.167:0/2473079853 shutdown_connections 2024-05-11T12:12:08.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.156+0000 7f0aada00640 1 -- 172.21.15.167:0/2473079853 wait complete. 2024-05-11T12:12:08.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.157+0000 7f0aada00640 1 Processor -- start 2024-05-11T12:12:08.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.158+0000 7f0aada00640 1 -- start start 2024-05-11T12:12:08.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.158+0000 7f0aada00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0aa8145f20 0x7f0aa8157da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:08.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.158+0000 7f0aada00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0aa8146300 0x7f0aa81f59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:08.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.158+0000 7f0aada00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0aa815a260 0x7f0aa815a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:08.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.159+0000 7f0aada00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa8161610 con 0x7f0aa8146300 2024-05-11T12:12:08.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.159+0000 7f0aada00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa8161490 con 0x7f0aa815a260 2024-05-11T12:12:08.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.159+0000 7f0aada00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0aa8161790 con 0x7f0aa8145f20 2024-05-11T12:12:08.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.159+0000 7f0aa6000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0aa8146300 0x7f0aa81f59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:08.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.159+0000 7f0aa6000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0aa8146300 0x7f0aa81f59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41962/0 (socket says 172.21.15.167:41962) 2024-05-11T12:12:08.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.159+0000 7f0aa6000640 1 -- 172.21.15.167:0/3983219640 learned_addr learned my addr 172.21.15.167:0/3983219640 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:08.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.159+0000 7f0aa6000640 1 -- 172.21.15.167:0/3983219640 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0aa8145f20 msgr2=0x7f0aa8157da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:08.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.159+0000 7f0aa6000640 1 --2- 172.21.15.167:0/3983219640 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0aa8145f20 0x7f0aa8157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:08.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.159+0000 7f0aa6000640 1 -- 172.21.15.167:0/3983219640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0aa815a260 msgr2=0x7f0aa815a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:08.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.159+0000 7f0aa6000640 1 --2- 172.21.15.167:0/3983219640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0aa815a260 0x7f0aa815a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:08.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.159+0000 7f0aa6000640 1 -- 172.21.15.167:0/3983219640 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aa815ac30 con 0x7f0aa8146300 2024-05-11T12:12:08.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.159+0000 7f0aa6000640 1 --2- 172.21.15.167:0/3983219640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0aa8146300 0x7f0aa81f59d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0a9400f550 tx=0x7f0a9400fa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:08.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.160+0000 7f0a9ea00640 1 -- 172.21.15.167:0/3983219640 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a94018070 con 0x7f0aa8146300 2024-05-11T12:12:08.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.160+0000 7f0a9ea00640 1 -- 172.21.15.167:0/3983219640 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a94014890 con 0x7f0aa8146300 2024-05-11T12:12:08.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.160+0000 7f0a9ea00640 1 -- 172.21.15.167:0/3983219640 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a94014c40 con 0x7f0aa8146300 2024-05-11T12:12:08.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.160+0000 7f0aada00640 1 -- 172.21.15.167:0/3983219640 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0aa8158460 con 0x7f0aa8146300 2024-05-11T12:12:08.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.160+0000 7f0aada00640 1 -- 172.21.15.167:0/3983219640 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0aa8158a20 con 0x7f0aa8146300 2024-05-11T12:12:08.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.161+0000 7f0aada00640 1 -- 172.21.15.167:0/3983219640 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0aa8069590 con 0x7f0aa8146300 2024-05-11T12:12:08.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.162+0000 7f0a9ea00640 1 -- 172.21.15.167:0/3983219640 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0a9401b070 con 0x7f0aa8146300 2024-05-11T12:12:08.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.163+0000 7f0a9ea00640 1 --2- 172.21.15.167:0/3983219640 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0a88076320 0x7f0a880787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:08.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.163+0000 7f0a9ea00640 1 -- 172.21.15.167:0/3983219640 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f0a940978d0 con 0x7f0aa8146300 2024-05-11T12:12:08.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.163+0000 7f0aa6a00640 1 --2- 172.21.15.167:0/3983219640 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0a88076320 0x7f0a880787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:08.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.164+0000 7f0aa6a00640 1 --2- 172.21.15.167:0/3983219640 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0a88076320 0x7f0a880787e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0aa815d230 tx=0x7f0a98000ed0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:08.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.167+0000 7f0a9ea00640 1 -- 172.21.15.167:0/3983219640 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0a94061180 con 0x7f0aa8146300 2024-05-11T12:12:08.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.413+0000 7f0aada00640 1 -- 172.21.15.167:0/3983219640 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0aa80b4640 con 0x7f0aa8146300 2024-05-11T12:12:08.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.414+0000 7f0a9ea00640 1 -- 172.21.15.167:0/3983219640 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5193 (secure 0 0 0) 0x7f0a94060a80 con 0x7f0aa8146300 2024-05-11T12:12:08.415 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:12:08.415 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4499,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3271527772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3271527772},{"type":"v1","addr":"172.21.15.167:6836","nonce":3271527772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4648,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2293636645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2293636645},{"type":"v1","addr":"172.21.15.167:6837","nonce":2293636645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4902,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3093163428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3093163428},{"type":"v1","addr":"172.21.15.204:6834","nonce":3093163428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:11:56.852478+0000","modified":"2024-05-11T12:11:59.926074+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4908},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4908":{"gid":4908,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/3248611739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":3248611739},{"type":"v1","addr":"172.21.15.204:6835","nonce":3248611739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4908,"qdb_cluster":[4908]},"id":2}]} 2024-05-11T12:12:08.415 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 18 2024-05-11T12:12:08.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.417+0000 7f0aada00640 1 -- 172.21.15.167:0/3983219640 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0a88076320 msgr2=0x7f0a880787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:08.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.417+0000 7f0aada00640 1 --2- 172.21.15.167:0/3983219640 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0a88076320 0x7f0a880787e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0aa815d230 tx=0x7f0a98000ed0 comp rx=0 tx=0).stop 2024-05-11T12:12:08.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.417+0000 7f0aada00640 1 -- 172.21.15.167:0/3983219640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0aa8146300 msgr2=0x7f0aa81f59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:08.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.417+0000 7f0aada00640 1 --2- 172.21.15.167:0/3983219640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0aa8146300 0x7f0aa81f59d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0a9400f550 tx=0x7f0a9400fa10 comp rx=0 tx=0).stop 2024-05-11T12:12:08.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.417+0000 7f0aada00640 1 -- 172.21.15.167:0/3983219640 shutdown_connections 2024-05-11T12:12:08.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.417+0000 7f0aada00640 1 --2- 172.21.15.167:0/3983219640 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0a88076320 0x7f0a880787e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:08.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.417+0000 7f0aada00640 1 --2- 172.21.15.167:0/3983219640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0aa815a260 0x7f0aa815a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:08.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.417+0000 7f0aada00640 1 --2- 172.21.15.167:0/3983219640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0aa8146300 0x7f0aa81f59d0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:08.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.417+0000 7f0aada00640 1 --2- 172.21.15.167:0/3983219640 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0aa8145f20 0x7f0aa8157da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:08.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.417+0000 7f0aada00640 1 -- 172.21.15.167:0/3983219640 >> 172.21.15.167:0/3983219640 conn(0x7f0aa80cd8f0 msgr2=0x7f0aa815d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:08.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.417+0000 7f0aada00640 1 -- 172.21.15.167:0/3983219640 shutdown_connections 2024-05-11T12:12:08.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:08.417+0000 7f0aada00640 1 -- 172.21.15.167:0/3983219640 wait complete. 2024-05-11T12:12:08.839 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-11T12:11:56.852478+0000', 'modified': '2024-05-11T12:11:59.926074+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4908}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4908': {'gid': 4908, 'name': 'd', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.204:6835/3248611739', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6833', 'nonce': 3248611739}, {'type': 'v1', 'addr': '172.21.15.204:6835', 'nonce': 3248611739}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4908, 'qdb_cluster': [4908]} 2024-05-11T12:12:08.839 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-11T12:12:08.839 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-11T12:12:09.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.049+0000 7f532ea00640 1 Processor -- start 2024-05-11T12:12:09.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.049+0000 7f532ea00640 1 -- start start 2024-05-11T12:12:09.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.051+0000 7f532ea00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f532815ac80 0x7f5328074260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:09.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.051+0000 7f532ea00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53280b65e0 con 0x7f5328161380 2024-05-11T12:12:09.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.051+0000 7f532ea00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53280b68c0 con 0x7f53280b3d50 2024-05-11T12:12:09.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.051+0000 7f532ea00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f532806e3c0 con 0x7f532815ac80 2024-05-11T12:12:09.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.051+0000 7f532ca00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f5328161380 0x7f5328072950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:50098/0 (socket says 172.21.15.167:50098) 2024-05-11T12:12:09.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.052+0000 7f532ca00640 1 -- 172.21.15.167:0/926672533 learned_addr learned my addr 172.21.15.167:0/926672533 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:09.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.052+0000 7f5327e00640 1 --2- 172.21.15.167:0/926672533 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f532815ac80 0x7f5328074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:09.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.052+0000 7f5326a00640 1 -- 172.21.15.167:0/926672533 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 282353882 0 0) 0x7f53280b65e0 con 0x7f5328161380 2024-05-11T12:12:09.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.052+0000 7f5326a00640 1 -- 172.21.15.167:0/926672533 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f530c003520 con 0x7f5328161380 2024-05-11T12:12:09.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.052+0000 7f5326a00640 1 -- 172.21.15.167:0/926672533 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4161024621 0 0) 0x7f530c003520 con 0x7f5328161380 2024-05-11T12:12:09.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.052+0000 7f5326a00640 1 -- 172.21.15.167:0/926672533 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f532815ac80 msgr2=0x7f5328074260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:09.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.052+0000 7f5326a00640 1 --2- 172.21.15.167:0/926672533 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f532815ac80 0x7f5328074260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:09.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.052+0000 7f5326a00640 1 -- 172.21.15.167:0/926672533 >> v1:172.21.15.204:6789/0 conn(0x7f53280b3d50 legacy=0x7f5328160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:09.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.052+0000 7f5326a00640 1 -- 172.21.15.167:0/926672533 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53280b6ba0 con 0x7f5328161380 2024-05-11T12:12:09.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.052+0000 7f5326a00640 1 -- 172.21.15.167:0/926672533 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f5320001a70 con 0x7f5328161380 2024-05-11T12:12:09.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.052+0000 7f5326a00640 1 -- 172.21.15.167:0/926672533 >> v1:172.21.15.167:6789/0 conn(0x7f5328161380 legacy=0x7f5328072950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:09.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.053+0000 7f5326a00640 1 --2- 172.21.15.167:0/926672533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f530c003a30 0x7f530c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:09.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.053+0000 7f5326a00640 1 --2- 172.21.15.167:0/926672533 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f532815ac80 0x7f530c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:09.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.053+0000 7f5326a00640 1 --2- 172.21.15.167:0/926672533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f530c004d40 0x7f530c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:09.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.053+0000 7f5326a00640 1 -- 172.21.15.167:0/926672533 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f530c00bac0 con 0x7f530c003a30 2024-05-11T12:12:09.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.053+0000 7f5326a00640 1 -- 172.21.15.167:0/926672533 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f530c00bc70 con 0x7f530c004d40 2024-05-11T12:12:09.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.053+0000 7f5326a00640 1 -- 172.21.15.167:0/926672533 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f530c00be70 con 0x7f532815ac80 2024-05-11T12:12:09.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.053+0000 7f5327e00640 1 --2- 172.21.15.167:0/926672533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f530c003a30 0x7f530c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:09.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.053+0000 7f5327400640 1 --2- 172.21.15.167:0/926672533 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f532815ac80 0x7f530c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:09.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.053+0000 7f5327e00640 1 -- 172.21.15.167:0/926672533 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f532815ac80 msgr2=0x7f530c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:09.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.053+0000 7f5327e00640 1 --2- 172.21.15.167:0/926672533 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f532815ac80 0x7f530c004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:09.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.053+0000 7f5327e00640 1 -- 172.21.15.167:0/926672533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f530c004d40 msgr2=0x7f530c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:09.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.053+0000 7f532ca00640 1 --2- 172.21.15.167:0/926672533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f530c004d40 0x7f530c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:09.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.053+0000 7f5327e00640 1 --2- 172.21.15.167:0/926672533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f530c004d40 0x7f530c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:09.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.053+0000 7f5327e00640 1 -- 172.21.15.167:0/926672533 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f530c003520 con 0x7f530c003a30 2024-05-11T12:12:09.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.053+0000 7f532ca00640 1 --2- 172.21.15.167:0/926672533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f530c004d40 0x7f530c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:09.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.054+0000 7f5327e00640 1 --2- 172.21.15.167:0/926672533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f530c003a30 0x7f530c003e50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5318009ac0 tx=0x7f530c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f8839b4b2ea6a72 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:09.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.055+0000 7f5326a00640 1 -- 172.21.15.167:0/926672533 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5318036070 con 0x7f530c003a30 2024-05-11T12:12:09.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.055+0000 7f5326a00640 1 -- 172.21.15.167:0/926672533 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5318031400 con 0x7f530c003a30 2024-05-11T12:12:09.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.055+0000 7f5326a00640 1 -- 172.21.15.167:0/926672533 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5318009690 con 0x7f530c003a30 2024-05-11T12:12:09.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.055+0000 7f532ea00640 1 -- 172.21.15.167:0/926672533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f530c003a30 msgr2=0x7f530c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:09.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.055+0000 7f532ea00640 1 --2- 172.21.15.167:0/926672533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f530c003a30 0x7f530c003e50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5318009ac0 tx=0x7f530c005830 comp rx=0 tx=0).stop 2024-05-11T12:12:09.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.055+0000 7f532ca00640 1 -- 172.21.15.167:0/926672533 reap_dead start 2024-05-11T12:12:09.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.055+0000 7f532ea00640 1 -- 172.21.15.167:0/926672533 shutdown_connections 2024-05-11T12:12:09.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.055+0000 7f532ea00640 1 -- 172.21.15.167:0/926672533 >> 172.21.15.167:0/926672533 conn(0x7f532806c450 msgr2=0x7f532806c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:09.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.055+0000 7f532ea00640 1 -- 172.21.15.167:0/926672533 shutdown_connections 2024-05-11T12:12:09.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.055+0000 7f532ea00640 1 -- 172.21.15.167:0/926672533 wait complete. 2024-05-11T12:12:09.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.057+0000 7f532ea00640 1 Processor -- start 2024-05-11T12:12:09.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.057+0000 7f532ea00640 1 -- start start 2024-05-11T12:12:09.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.058+0000 7f532ea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f530c003a30 0x7f53281f4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:09.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.058+0000 7f532ea00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f53281f8460 0x7f5328144330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:09.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.058+0000 7f532ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f532815f810 0x7f532815fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:09.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.058+0000 7f532ea00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f53280b44f0 con 0x7f530c003a30 2024-05-11T12:12:09.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.058+0000 7f532ea00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f532815b060 con 0x7f532815f810 2024-05-11T12:12:09.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.058+0000 7f532ea00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5328074260 con 0x7f53281f8460 2024-05-11T12:12:09.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.059+0000 7f5327e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f530c003a30 0x7f53281f4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:09.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.059+0000 7f5327e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f530c003a30 0x7f53281f4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59758/0 (socket says 172.21.15.167:59758) 2024-05-11T12:12:09.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.059+0000 7f5327e00640 1 -- 172.21.15.167:0/3992988102 learned_addr learned my addr 172.21.15.167:0/3992988102 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:09.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.059+0000 7f5327400640 1 --2- 172.21.15.167:0/3992988102 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f53281f8460 0x7f5328144330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:09.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.059+0000 7f532ca00640 1 --2- 172.21.15.167:0/3992988102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f532815f810 0x7f532815fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:09.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.059+0000 7f5327e00640 1 -- 172.21.15.167:0/3992988102 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f53281f8460 msgr2=0x7f5328144330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:09.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.059+0000 7f5327e00640 1 --2- 172.21.15.167:0/3992988102 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f53281f8460 0x7f5328144330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:09.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.059+0000 7f5327e00640 1 -- 172.21.15.167:0/3992988102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f532815f810 msgr2=0x7f532815fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:09.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.059+0000 7f5327e00640 1 --2- 172.21.15.167:0/3992988102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f532815f810 0x7f532815fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:09.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.059+0000 7f5327e00640 1 -- 172.21.15.167:0/3992988102 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53280ce990 con 0x7f530c003a30 2024-05-11T12:12:09.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.059+0000 7f5327400640 1 --2- 172.21.15.167:0/3992988102 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f53281f8460 0x7f5328144330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:09.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.060+0000 7f5327e00640 1 --2- 172.21.15.167:0/3992988102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f530c003a30 0x7f53281f4e70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5318034040 tx=0x7f5318004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:09.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.060+0000 7f531fe00640 1 -- 172.21.15.167:0/3992988102 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5318040020 con 0x7f530c003a30 2024-05-11T12:12:09.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.060+0000 7f531fe00640 1 -- 172.21.15.167:0/3992988102 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5318003bc0 con 0x7f530c003a30 2024-05-11T12:12:09.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.060+0000 7f531fe00640 1 -- 172.21.15.167:0/3992988102 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5318036040 con 0x7f530c003a30 2024-05-11T12:12:09.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.060+0000 7f532ea00640 1 -- 172.21.15.167:0/3992988102 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53281f54c0 con 0x7f530c003a30 2024-05-11T12:12:09.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.060+0000 7f532ea00640 1 -- 172.21.15.167:0/3992988102 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53281f5a80 con 0x7f530c003a30 2024-05-11T12:12:09.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.062+0000 7f531fe00640 1 -- 172.21.15.167:0/3992988102 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5318043020 con 0x7f530c003a30 2024-05-11T12:12:09.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.063+0000 7f531fe00640 1 --2- 172.21.15.167:0/3992988102 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5308076300 0x7f53080787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:09.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.064+0000 7f531fe00640 1 -- 172.21.15.167:0/3992988102 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f53180b78b0 con 0x7f530c003a30 2024-05-11T12:12:09.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.064+0000 7f5327400640 1 --2- 172.21.15.167:0/3992988102 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5308076300 0x7f53080787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:09.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.064+0000 7f5331804740 1 Processor -- start 2024-05-11T12:12:09.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.064+0000 7f5331804740 1 -- start start 2024-05-11T12:12:09.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.064+0000 7f5327400640 1 --2- 172.21.15.167:0/3992988102 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5308076300 0x7f53080787c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5314000c00 tx=0x7f5314009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:09.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.065+0000 7f5331804740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560e28fb840 0x5560e28fbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:09.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.065+0000 7f5331804740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5560e28f66d0 0x5560e28fa440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:09.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.065+0000 7f5331804740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5560e28fa980 0x5560e28fadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:09.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.065+0000 7f5331804740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5560e285f790 con 0x5560e28fb840 2024-05-11T12:12:09.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.065+0000 7f5331804740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5560e2804f20 con 0x5560e28f66d0 2024-05-11T12:12:09.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.065+0000 7f5331804740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x5560e285b750 con 0x5560e28fa980 2024-05-11T12:12:09.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.065+0000 7f5327e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560e28fb840 0x5560e28fbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:09.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.065+0000 7f5327e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560e28fb840 0x5560e28fbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59764/0 (socket says 172.21.15.167:59764) 2024-05-11T12:12:09.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.065+0000 7f5327e00640 1 -- 172.21.15.167:0/1558120780 learned_addr learned my addr 172.21.15.167:0/1558120780 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:09.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.066+0000 7f5327e00640 1 -- 172.21.15.167:0/1558120780 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5560e28fa980 msgr2=0x5560e28fadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:09.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.066+0000 7f5327e00640 1 --2- 172.21.15.167:0/1558120780 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5560e28fa980 0x5560e28fadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:09.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.066+0000 7f5327e00640 1 -- 172.21.15.167:0/1558120780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5560e28f66d0 msgr2=0x5560e28fa440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:09.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.066+0000 7f5327e00640 1 --2- 172.21.15.167:0/1558120780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5560e28f66d0 0x5560e28fa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:09.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.066+0000 7f5327e00640 1 -- 172.21.15.167:0/1558120780 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560e25712d0 con 0x5560e28fb840 2024-05-11T12:12:09.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.066+0000 7f5327e00640 1 --2- 172.21.15.167:0/1558120780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560e28fb840 0x5560e28fbc20 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5318080900 tx=0x7f5318009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=400301dc8ad3a77f server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:09.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.066+0000 7f5313e00640 1 -- 172.21.15.167:0/1558120780 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53180044e0 con 0x5560e28fb840 2024-05-11T12:12:09.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.066+0000 7f5313e00640 1 -- 172.21.15.167:0/1558120780 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5318081070 con 0x5560e28fb840 2024-05-11T12:12:09.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.066+0000 7f5313e00640 1 -- 172.21.15.167:0/1558120780 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53180394d0 con 0x5560e28fb840 2024-05-11T12:12:09.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.067+0000 7f5331804740 1 -- 172.21.15.167:0/1558120780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560e28fb840 msgr2=0x5560e28fbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:09.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.067+0000 7f5331804740 1 --2- 172.21.15.167:0/1558120780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560e28fb840 0x5560e28fbc20 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5318080900 tx=0x7f5318009750 comp rx=0 tx=0).stop 2024-05-11T12:12:09.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.067+0000 7f5331804740 1 -- 172.21.15.167:0/1558120780 shutdown_connections 2024-05-11T12:12:09.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.067+0000 7f5331804740 1 --2- 172.21.15.167:0/1558120780 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5560e28fa980 0x5560e28fadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:09.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.067+0000 7f5331804740 1 --2- 172.21.15.167:0/1558120780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5560e28f66d0 0x5560e28fa440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:09.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.067+0000 7f5331804740 1 --2- 172.21.15.167:0/1558120780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560e28fb840 0x5560e28fbc20 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:09.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.067+0000 7f5331804740 1 -- 172.21.15.167:0/1558120780 >> 172.21.15.167:0/1558120780 conn(0x5560e28f8940 msgr2=0x5560e2941400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:09.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.067+0000 7f5331804740 1 -- 172.21.15.167:0/1558120780 shutdown_connections 2024-05-11T12:12:09.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.067+0000 7f5331804740 1 -- 172.21.15.167:0/1558120780 wait complete. 2024-05-11T12:12:09.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.068+0000 7f5331804740 1 -- start start 2024-05-11T12:12:09.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.068+0000 7f5331804740 1 Processor -- start 2024-05-11T12:12:09.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.069+0000 7f5331804740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5560e28fa980 0x5560e2942d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:09.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.069+0000 7f5331804740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560e28fb840 0x5560e2928800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:09.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.069+0000 7f5331804740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5560e29545b0 0x5560e2928d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:09.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.069+0000 7f5331804740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5560e285b750 con 0x5560e28fb840 2024-05-11T12:12:09.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.069+0000 7f5331804740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5560e28189c0 con 0x5560e29545b0 2024-05-11T12:12:09.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.069+0000 7f5331804740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x5560e285f790 con 0x5560e28fa980 2024-05-11T12:12:09.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.069+0000 7f5331804740 10 client.0 _ll_register_callbacks cb 0x5560e27e5120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:12:09.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.069+0000 7f5327400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560e28fb840 0x5560e2928800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:09.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.069+0000 7f5327400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560e28fb840 0x5560e2928800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59766/0 (socket says 172.21.15.167:59766) 2024-05-11T12:12:09.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.069+0000 7f5327400640 1 -- 172.21.15.167:0/3701962287 learned_addr learned my addr 172.21.15.167:0/3701962287 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:09.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.070+0000 7f5327e00640 1 --2- 172.21.15.167:0/3701962287 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5560e28fa980 0x5560e2942d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:09.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.070+0000 7f532ca00640 1 --2- 172.21.15.167:0/3701962287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5560e29545b0 0x5560e2928d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:09.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.070+0000 7f5327400640 1 -- 172.21.15.167:0/3701962287 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5560e28fa980 msgr2=0x5560e2942d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:09.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.070+0000 7f5327400640 1 --2- 172.21.15.167:0/3701962287 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5560e28fa980 0x5560e2942d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:09.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.070+0000 7f5327400640 1 -- 172.21.15.167:0/3701962287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5560e29545b0 msgr2=0x5560e2928d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:09.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.070+0000 7f5327400640 1 --2- 172.21.15.167:0/3701962287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5560e29545b0 0x5560e2928d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:09.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.070+0000 7f5327400640 1 -- 172.21.15.167:0/3701962287 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560e263b6b0 con 0x5560e28fb840 2024-05-11T12:12:09.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.070+0000 7f5327400640 1 --2- 172.21.15.167:0/3701962287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560e28fb840 0x5560e2928800 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f531400ef50 tx=0x7f531400ef80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58956a9090ba20bb server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:09.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.070+0000 7f5310c00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-11T12:12:09.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.070+0000 7f5310c00640 1 -- 172.21.15.167:0/3701962287 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5314024060 con 0x5560e28fb840 2024-05-11T12:12:09.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.071+0000 7f5310c00640 1 -- 172.21.15.167:0/3701962287 <== mon.0 v2:172.21.15.167:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f531401f400 con 0x5560e28fb840 2024-05-11T12:12:09.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.071+0000 7f5331804740 1 -- 172.21.15.167:0/3701962287 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5318081070 con 0x5560e28fb840 2024-05-11T12:12:09.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.071+0000 7f5310c00640 1 -- 172.21.15.167:0/3701962287 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f531402a050 con 0x5560e28fb840 2024-05-11T12:12:09.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.071+0000 7f5331804740 10 client.5091 fetch_fsmap learned FSMap version 18 2024-05-11T12:12:09.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.071+0000 7f5331804740 1 -- 172.21.15.167:0/3701962287 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x5560e2944c00 con 0x5560e28fb840 2024-05-11T12:12:09.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.071+0000 7f5310c00640 1 -- 172.21.15.167:0/3701962287 <== mon.0 v2:172.21.15.167:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5318081070 con 0x5560e28fb840 2024-05-11T12:12:09.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.071+0000 7f5310c00640 1 -- 172.21.15.167:0/3701962287 <== mon.0 v2:172.21.15.167:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f531402d020 con 0x5560e28fb840 2024-05-11T12:12:09.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.071+0000 7f5331804740 10 client.5091 fetch_fsmap finished waiting for FSMap version 18 2024-05-11T12:12:09.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.071+0000 7f5331804740 10 client.5091 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-11T12:12:09.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.071+0000 7f5331804740 20 client.5091 populate_metadata read hostname 'smithi167' 2024-05-11T12:12:09.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.071+0000 7f5331804740 1 --2- 172.21.15.167:0/3701962287 >> [v2:172.21.15.204:6833/3248611739,v1:172.21.15.204:6835/3248611739] conn(0x5560e2951aa0 0x5560e2951e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:09.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.071+0000 7f5331804740 4 client.5091 mds_command: new command op to 4908 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:12:09.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.071+0000 7f5331804740 1 -- 172.21.15.167:0/3701962287 --> [v2:172.21.15.204:6833/3248611739,v1:172.21.15.204:6835/3248611739] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560e286d3a0 con 0x5560e2951aa0 2024-05-11T12:12:09.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.072+0000 7f5327e00640 1 --2- 172.21.15.167:0/3701962287 >> [v2:172.21.15.204:6833/3248611739,v1:172.21.15.204:6835/3248611739] conn(0x5560e2951aa0 0x5560e2951e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:09.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.073+0000 7f5327e00640 1 --2- 172.21.15.167:0/3701962287 >> [v2:172.21.15.204:6833/3248611739,v1:172.21.15.204:6835/3248611739] conn(0x5560e2951aa0 0x5560e2951e80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe39afb7819b8873 server_cookie=3062909d6c7f1af5 in_seq=0 out_seq=0 2024-05-11T12:12:09.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.073+0000 7f5310c00640 10 client.5091 ms_handle_connect on v2:172.21.15.204:6833/3248611739 2024-05-11T12:12:09.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.075+0000 7f5310c00640 1 -- 172.21.15.167:0/3701962287 <== mds.0 v2:172.21.15.204:6833/3248611739 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5560e286d3a0 con 0x5560e2951aa0 2024-05-11T12:12:09.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.075+0000 7f5310c00640 10 client.5091 handle_command_reply: tid=0 2024-05-11T12:12:09.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.075+0000 7f5331804740 1 client.5091 shutdown 2024-05-11T12:12:09.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.075+0000 7f5331804740 1 -- 172.21.15.167:0/3701962287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560e28fb840 msgr2=0x5560e2928800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:09.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.075+0000 7f5331804740 1 --2- 172.21.15.167:0/3701962287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560e28fb840 0x5560e2928800 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f531400ef50 tx=0x7f531400ef80 comp rx=0 tx=0).stop 2024-05-11T12:12:09.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.075+0000 7f5331804740 1 -- 172.21.15.167:0/3701962287 shutdown_connections 2024-05-11T12:12:09.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.075+0000 7f5331804740 1 --2- 172.21.15.167:0/3701962287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5560e29545b0 0x5560e2928d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:09.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.075+0000 7f5331804740 1 --2- 172.21.15.167:0/3701962287 >> [v2:172.21.15.204:6833/3248611739,v1:172.21.15.204:6835/3248611739] conn(0x5560e2951aa0 0x5560e2951e80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:09.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.075+0000 7f5331804740 1 --2- 172.21.15.167:0/3701962287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560e28fb840 0x5560e2928800 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:09.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.075+0000 7f5331804740 1 --2- 172.21.15.167:0/3701962287 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5560e28fa980 0x5560e2942d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:09.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.075+0000 7f5310c00640 0 client.5091 ms_handle_reset on v2:172.21.15.204:6833/3248611739 2024-05-11T12:12:09.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.075+0000 7f5331804740 1 -- 172.21.15.167:0/3701962287 >> 172.21.15.167:0/3701962287 conn(0x5560e28f66d0 msgr2=0x5560e28f5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:09.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.075+0000 7f5331804740 1 -- 172.21.15.167:0/3701962287 shutdown_connections 2024-05-11T12:12:09.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.075+0000 7f5331804740 1 -- 172.21.15.167:0/3701962287 wait complete. 2024-05-11T12:12:09.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.075+0000 7f5331804740 20 client.5091 trim_cache size 0 max 16384 2024-05-11T12:12:09.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.095+0000 7f5331804740 1 Processor -- start 2024-05-11T12:12:09.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.096+0000 7f5331804740 1 -- start start 2024-05-11T12:12:09.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.096+0000 7f5331804740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5560e28fa980 0x5560e2943460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:09.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.096+0000 7f5331804740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5560e28fb840 0x5560e2951e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:09.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.096+0000 7f5331804740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560e29545b0 0x5560e29523c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:09.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.096+0000 7f5331804740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f53282026f0 con 0x5560e29545b0 2024-05-11T12:12:09.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.096+0000 7f5331804740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5328201800 con 0x5560e28fb840 2024-05-11T12:12:09.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.096+0000 7f5331804740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f53280cfc20 con 0x5560e28fa980 2024-05-11T12:12:09.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.097+0000 7f532ca00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560e29545b0 0x5560e29523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:09.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.097+0000 7f532ca00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560e29545b0 0x5560e29523c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59782/0 (socket says 172.21.15.167:59782) 2024-05-11T12:12:09.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.097+0000 7f532ca00640 1 -- 172.21.15.167:0/1144204871 learned_addr learned my addr 172.21.15.167:0/1144204871 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:09.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.097+0000 7f532ca00640 1 -- 172.21.15.167:0/1144204871 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5560e28fa980 msgr2=0x5560e2943460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:09.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.097+0000 7f532ca00640 1 --2- 172.21.15.167:0/1144204871 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5560e28fa980 0x5560e2943460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:09.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.097+0000 7f532ca00640 1 -- 172.21.15.167:0/1144204871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5560e28fb840 msgr2=0x5560e2951e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:09.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.097+0000 7f5327400640 1 --2- 172.21.15.167:0/1144204871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5560e28fb840 0x5560e2951e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:09.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.097+0000 7f532ca00640 1 --2- 172.21.15.167:0/1144204871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5560e28fb840 0x5560e2951e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:09.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.097+0000 7f532ca00640 1 -- 172.21.15.167:0/1144204871 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560e28f7640 con 0x5560e29545b0 2024-05-11T12:12:09.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.098+0000 7f532ca00640 1 --2- 172.21.15.167:0/1144204871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560e29545b0 0x5560e29523c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5328161d50 tx=0x7f5320005f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48fc9d3f5d7e4452 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:09.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.098+0000 7f5313400640 1 -- 172.21.15.167:0/1144204871 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5320007730 con 0x5560e29545b0 2024-05-11T12:12:09.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.098+0000 7f5313400640 1 -- 172.21.15.167:0/1144204871 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53200078d0 con 0x5560e29545b0 2024-05-11T12:12:09.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.098+0000 7f5313400640 1 -- 172.21.15.167:0/1144204871 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5320014620 con 0x5560e29545b0 2024-05-11T12:12:09.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.098+0000 7f5331804740 1 -- 172.21.15.167:0/1144204871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560e29545b0 msgr2=0x5560e29523c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:09.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.098+0000 7f5331804740 1 --2- 172.21.15.167:0/1144204871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560e29545b0 0x5560e29523c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5328161d50 tx=0x7f5320005f40 comp rx=0 tx=0).stop 2024-05-11T12:12:09.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.099+0000 7f5331804740 1 -- 172.21.15.167:0/1144204871 shutdown_connections 2024-05-11T12:12:09.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.099+0000 7f5331804740 1 --2- 172.21.15.167:0/1144204871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560e29545b0 0x5560e29523c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:09.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.099+0000 7f5331804740 1 --2- 172.21.15.167:0/1144204871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5560e28fb840 0x5560e2951e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:09.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.099+0000 7f5331804740 1 --2- 172.21.15.167:0/1144204871 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5560e28fa980 0x5560e2943460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:09.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.099+0000 7f5331804740 1 -- 172.21.15.167:0/1144204871 >> 172.21.15.167:0/1144204871 conn(0x5560e28f66d0 msgr2=0x5560e28f1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:09.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.099+0000 7f5331804740 1 -- 172.21.15.167:0/1144204871 shutdown_connections 2024-05-11T12:12:09.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.099+0000 7f5331804740 1 -- 172.21.15.167:0/1144204871 wait complete. 2024-05-11T12:12:09.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.099+0000 7f5331804740 1 -- start start 2024-05-11T12:12:09.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.100+0000 7f5331804740 1 Processor -- start 2024-05-11T12:12:09.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.100+0000 7f5331804740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5560e28fa980 0x5560e29aa870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:09.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.101+0000 7f5331804740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560e28fb840 0x5560e29ad060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:09.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.101+0000 7f5331804740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5560e29545b0 0x5560e2950510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:09.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.101+0000 7f5331804740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5560e29439c0 con 0x5560e28fb840 2024-05-11T12:12:09.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.101+0000 7f5331804740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5560e2804f20 con 0x5560e29545b0 2024-05-11T12:12:09.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.101+0000 7f5331804740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x5560e2917ce0 con 0x5560e28fa980 2024-05-11T12:12:09.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.101+0000 7f5331804740 10 client.0 _ll_register_callbacks cb 0x7f5314001430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:12:09.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.101+0000 7f5327400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560e28fb840 0x5560e29ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:09.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.101+0000 7f5327400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560e28fb840 0x5560e29ad060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59784/0 (socket says 172.21.15.167:59784) 2024-05-11T12:12:09.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.101+0000 7f5327400640 1 -- 172.21.15.167:0/1450516752 learned_addr learned my addr 172.21.15.167:0/1450516752 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:09.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.101+0000 7f5327e00640 1 --2- 172.21.15.167:0/1450516752 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5560e28fa980 0x5560e29aa870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:09.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.101+0000 7f532ca00640 1 --2- 172.21.15.167:0/1450516752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5560e29545b0 0x5560e2950510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:09.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.101+0000 7f5327400640 1 -- 172.21.15.167:0/1450516752 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5560e28fa980 msgr2=0x5560e29aa870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:09.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.101+0000 7f5327400640 1 --2- 172.21.15.167:0/1450516752 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5560e28fa980 0x5560e29aa870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:09.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.101+0000 7f5327400640 1 -- 172.21.15.167:0/1450516752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5560e29545b0 msgr2=0x5560e2950510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:09.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.101+0000 7f5327400640 1 --2- 172.21.15.167:0/1450516752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5560e29545b0 0x5560e2950510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:09.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.101+0000 7f5327400640 1 -- 172.21.15.167:0/1450516752 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560e29add20 con 0x5560e28fb840 2024-05-11T12:12:09.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.102+0000 7f5327400640 1 --2- 172.21.15.167:0/1450516752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560e28fb840 0x5560e29ad060 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f531400ee10 tx=0x7f5314014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a50712da52b4be2 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:09.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.102+0000 7f5310c00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-11T12:12:09.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.102+0000 7f5310c00640 1 -- 172.21.15.167:0/1450516752 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53140143f0 con 0x5560e28fb840 2024-05-11T12:12:09.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.102+0000 7f5310c00640 1 -- 172.21.15.167:0/1450516752 <== mon.0 v2:172.21.15.167:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f5314013cd0 con 0x5560e28fb840 2024-05-11T12:12:09.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.102+0000 7f5331804740 1 -- 172.21.15.167:0/1450516752 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53200078d0 con 0x5560e28fb840 2024-05-11T12:12:09.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.102+0000 7f5310c00640 1 -- 172.21.15.167:0/1450516752 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f531403b050 con 0x5560e28fb840 2024-05-11T12:12:09.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.102+0000 7f5331804740 10 client.5097 fetch_fsmap learned FSMap version 18 2024-05-11T12:12:09.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.102+0000 7f5331804740 1 -- 172.21.15.167:0/1450516752 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x5560e28f5cc0 con 0x5560e28fb840 2024-05-11T12:12:09.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.102+0000 7f5310c00640 1 -- 172.21.15.167:0/1450516752 <== mon.0 v2:172.21.15.167:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53200078d0 con 0x5560e28fb840 2024-05-11T12:12:09.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.103+0000 7f5310c00640 1 -- 172.21.15.167:0/1450516752 <== mon.0 v2:172.21.15.167:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f531403e020 con 0x5560e28fb840 2024-05-11T12:12:09.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.103+0000 7f5331804740 10 client.5097 fetch_fsmap finished waiting for FSMap version 18 2024-05-11T12:12:09.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.103+0000 7f5331804740 10 client.5097 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-11T12:12:09.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.103+0000 7f5331804740 20 client.5097 populate_metadata read hostname 'smithi167' 2024-05-11T12:12:09.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.103+0000 7f5331804740 1 --2- 172.21.15.167:0/1450516752 >> [v2:172.21.15.204:6833/3248611739,v1:172.21.15.204:6835/3248611739] conn(0x5560e29cbb20 0x5560e2916070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:09.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.103+0000 7f5331804740 4 client.5097 mds_command: new command op to 4908 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-11T12:12:09.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.103+0000 7f5331804740 1 -- 172.21.15.167:0/1450516752 --> [v2:172.21.15.204:6833/3248611739,v1:172.21.15.204:6835/3248611739] -- command(tid 0: {"prefix": "status"}) -- 0x5560e2945250 con 0x5560e29cbb20 2024-05-11T12:12:09.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.103+0000 7f5327e00640 1 --2- 172.21.15.167:0/1450516752 >> [v2:172.21.15.204:6833/3248611739,v1:172.21.15.204:6835/3248611739] conn(0x5560e29cbb20 0x5560e2916070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:09.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.104+0000 7f5327e00640 1 --2- 172.21.15.167:0/1450516752 >> [v2:172.21.15.204:6833/3248611739,v1:172.21.15.204:6835/3248611739] conn(0x5560e29cbb20 0x5560e2916070 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f489bcde2b79ff24 server_cookie=698c65e75abc9fee in_seq=0 out_seq=0 2024-05-11T12:12:09.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.104+0000 7f5310c00640 10 client.5097 ms_handle_connect on v2:172.21.15.204:6833/3248611739 2024-05-11T12:12:09.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.105+0000 7f5310c00640 1 -- 172.21.15.167:0/1450516752 <== mds.0 v2:172.21.15.204:6833/3248611739 1 ==== command_reply(tid 0: 0 ) ==== 8+0+318 (crc 0 0 0) 0x5560e2945250 con 0x5560e29cbb20 2024-05-11T12:12:09.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.105+0000 7f5310c00640 10 client.5097 handle_command_reply: tid=0 2024-05-11T12:12:09.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.105+0000 7f5331804740 1 client.5097 shutdown 2024-05-11T12:12:09.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.105+0000 7f5331804740 1 -- 172.21.15.167:0/1450516752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560e28fb840 msgr2=0x5560e29ad060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:09.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.105+0000 7f5331804740 1 --2- 172.21.15.167:0/1450516752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560e28fb840 0x5560e29ad060 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f531400ee10 tx=0x7f5314014000 comp rx=0 tx=0).stop 2024-05-11T12:12:09.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.105+0000 7f5331804740 1 -- 172.21.15.167:0/1450516752 shutdown_connections 2024-05-11T12:12:09.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.105+0000 7f5331804740 1 --2- 172.21.15.167:0/1450516752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5560e29545b0 0x5560e2950510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:09.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.105+0000 7f5331804740 1 --2- 172.21.15.167:0/1450516752 >> [v2:172.21.15.204:6833/3248611739,v1:172.21.15.204:6835/3248611739] conn(0x5560e29cbb20 0x5560e2916070 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:09.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.105+0000 7f5331804740 1 --2- 172.21.15.167:0/1450516752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560e28fb840 0x5560e29ad060 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:09.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.105+0000 7f5331804740 1 --2- 172.21.15.167:0/1450516752 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5560e28fa980 0x5560e29aa870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:09.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.105+0000 7f5331804740 1 -- 172.21.15.167:0/1450516752 >> 172.21.15.167:0/1450516752 conn(0x5560e28f66d0 msgr2=0x5560e28f1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:09.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.105+0000 7f5310c00640 0 client.5097 ms_handle_reset on v2:172.21.15.204:6833/3248611739 2024-05-11T12:12:09.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.105+0000 7f5331804740 1 -- 172.21.15.167:0/1450516752 shutdown_connections 2024-05-11T12:12:09.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.105+0000 7f5331804740 1 -- 172.21.15.167:0/1450516752 wait complete. 2024-05-11T12:12:09.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.105+0000 7f5331804740 20 client.5097 trim_cache size 0 max 16384 2024-05-11T12:12:09.108 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T12:12:09.108 INFO:teuthology.orchestra.run.smithi167.stdout: "cluster_fsid": "5f32d344-7289-4c27-881b-1533419ab678", 2024-05-11T12:12:09.108 INFO:teuthology.orchestra.run.smithi167.stdout: "whoami": 0, 2024-05-11T12:12:09.108 INFO:teuthology.orchestra.run.smithi167.stdout: "id": 4908, 2024-05-11T12:12:09.108 INFO:teuthology.orchestra.run.smithi167.stdout: "want_state": "up:active", 2024-05-11T12:12:09.108 INFO:teuthology.orchestra.run.smithi167.stdout: "state": "up:active", 2024-05-11T12:12:09.108 INFO:teuthology.orchestra.run.smithi167.stdout: "fs_name": "cephfs", 2024-05-11T12:12:09.108 INFO:teuthology.orchestra.run.smithi167.stdout: "rank_uptime": 11.689942301, 2024-05-11T12:12:09.108 INFO:teuthology.orchestra.run.smithi167.stdout: "mdsmap_epoch": 18, 2024-05-11T12:12:09.108 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_epoch": 41, 2024-05-11T12:12:09.108 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_epoch_barrier": 41, 2024-05-11T12:12:09.108 INFO:teuthology.orchestra.run.smithi167.stdout: "uptime": 19.53968485 2024-05-11T12:12:09.109 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T12:12:09.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.106+0000 7f532ea00640 1 -- 172.21.15.167:0/3992988102 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5308076300 msgr2=0x7f53080787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:09.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.106+0000 7f532ea00640 1 --2- 172.21.15.167:0/3992988102 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5308076300 0x7f53080787c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5314000c00 tx=0x7f5314009040 comp rx=0 tx=0).stop 2024-05-11T12:12:09.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.106+0000 7f532ea00640 1 -- 172.21.15.167:0/3992988102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f530c003a30 msgr2=0x7f53281f4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:09.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.106+0000 7f532ea00640 1 --2- 172.21.15.167:0/3992988102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f530c003a30 0x7f53281f4e70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5318034040 tx=0x7f5318004730 comp rx=0 tx=0).stop 2024-05-11T12:12:09.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.106+0000 7f532ea00640 1 -- 172.21.15.167:0/3992988102 shutdown_connections 2024-05-11T12:12:09.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.106+0000 7f532ea00640 1 --2- 172.21.15.167:0/3992988102 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5308076300 0x7f53080787c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:09.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.106+0000 7f532ea00640 1 --2- 172.21.15.167:0/3992988102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f532815f810 0x7f532815fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:09.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.106+0000 7f532ea00640 1 --2- 172.21.15.167:0/3992988102 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f53281f8460 0x7f5328144330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:09.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.106+0000 7f532ea00640 1 --2- 172.21.15.167:0/3992988102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f530c003a30 0x7f53281f4e70 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:09.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.106+0000 7f532ea00640 1 -- 172.21.15.167:0/3992988102 >> 172.21.15.167:0/3992988102 conn(0x7f532806c450 msgr2=0x7f5328072850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:09.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.106+0000 7f532ea00640 1 -- 172.21.15.167:0/3992988102 shutdown_connections 2024-05-11T12:12:09.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.106+0000 7f532ea00640 1 -- 172.21.15.167:0/3992988102 wait complete. 2024-05-11T12:12:09.124 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-11T12:12:09.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.327+0000 7fb73ac00640 1 Processor -- start 2024-05-11T12:12:09.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.327+0000 7fb73ac00640 1 -- start start 2024-05-11T12:12:09.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.329+0000 7fb73ac00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb734157750 0x7fb73415d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:09.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.329+0000 7fb73ac00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb7340b3ea0 0x7fb734161490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:09.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.329+0000 7fb73ac00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb73415dc80 con 0x7fb7341619d0 2024-05-11T12:12:09.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.329+0000 7fb73ac00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fb734161000 con 0x7fb734157750 2024-05-11T12:12:09.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.329+0000 7fb73ac00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb73415df30 con 0x7fb7340b3ea0 2024-05-11T12:12:09.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.330+0000 7fb733e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb734157750 0x7fb73415d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:09.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.330+0000 7fb738c00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fb7341619d0 0x7fb73406d460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:50104/0 (socket says 172.21.15.167:50104) 2024-05-11T12:12:09.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.330+0000 7fb738c00640 1 -- 172.21.15.167:0/1068507907 learned_addr learned my addr 172.21.15.167:0/1068507907 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:09.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.330+0000 7fb733400640 1 --2- 172.21.15.167:0/1068507907 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb7340b3ea0 0x7fb734161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:09.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.330+0000 7fb733e00640 1 -- 172.21.15.167:0/1068507907 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb7340b3ea0 msgr2=0x7fb734161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:09.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.330+0000 7fb733e00640 1 --2- 172.21.15.167:0/1068507907 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb7340b3ea0 0x7fb734161490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:09.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.330+0000 7fb733e00640 1 -- 172.21.15.167:0/1068507907 >> v1:172.21.15.167:6789/0 conn(0x7fb7341619d0 legacy=0x7fb73406d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:09.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.330+0000 7fb732a00640 1 -- 172.21.15.167:0/1068507907 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2284407889 0 0) 0x7fb73415dc80 con 0x7fb7341619d0 2024-05-11T12:12:09.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.330+0000 7fb733e00640 1 -- 172.21.15.167:0/1068507907 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb73415e210 con 0x7fb734157750 2024-05-11T12:12:09.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.330+0000 7fb733400640 1 --2- 172.21.15.167:0/1068507907 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb7340b3ea0 0x7fb734161490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:09.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.331+0000 7fb733e00640 1 --2- 172.21.15.167:0/1068507907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb734157750 0x7fb73415d740 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb7240097a0 tx=0x7fb72402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fbdd1342eee7404 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:09.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.332+0000 7fb732a00640 1 -- 172.21.15.167:0/1068507907 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb724037070 con 0x7fb734157750 2024-05-11T12:12:09.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.332+0000 7fb732a00640 1 -- 172.21.15.167:0/1068507907 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb724032440 con 0x7fb734157750 2024-05-11T12:12:09.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.332+0000 7fb732a00640 1 -- 172.21.15.167:0/1068507907 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb724032810 con 0x7fb734157750 2024-05-11T12:12:09.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.332+0000 7fb73ac00640 1 -- 172.21.15.167:0/1068507907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb734157750 msgr2=0x7fb73415d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:09.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.332+0000 7fb73ac00640 1 --2- 172.21.15.167:0/1068507907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb734157750 0x7fb73415d740 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb7240097a0 tx=0x7fb72402a180 comp rx=0 tx=0).stop 2024-05-11T12:12:09.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.332+0000 7fb73ac00640 1 -- 172.21.15.167:0/1068507907 shutdown_connections 2024-05-11T12:12:09.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.332+0000 7fb73ac00640 1 --2- 172.21.15.167:0/1068507907 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb7340b3ea0 0x7fb734161490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:09.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.332+0000 7fb73ac00640 1 --2- 172.21.15.167:0/1068507907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb734157750 0x7fb73415d740 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:09.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.332+0000 7fb73ac00640 1 -- 172.21.15.167:0/1068507907 >> 172.21.15.167:0/1068507907 conn(0x7fb7340cd8f0 msgr2=0x7fb7340cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:09.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.332+0000 7fb73ac00640 1 -- 172.21.15.167:0/1068507907 shutdown_connections 2024-05-11T12:12:09.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.332+0000 7fb73ac00640 1 -- 172.21.15.167:0/1068507907 wait complete. 2024-05-11T12:12:09.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.334+0000 7fb73ac00640 1 Processor -- start 2024-05-11T12:12:09.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.334+0000 7fb73ac00640 1 -- start start 2024-05-11T12:12:09.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.335+0000 7fb73ac00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb7340b3ea0 0x7fb7340af0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:09.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.335+0000 7fb73ac00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb734157750 0x7fb7340adda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:09.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.335+0000 7fb73ac00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7341619d0 0x7fb7340ae2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:09.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.335+0000 7fb73ac00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb73406db70 con 0x7fb7341619d0 2024-05-11T12:12:09.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.335+0000 7fb73ac00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb734157b30 con 0x7fb734157750 2024-05-11T12:12:09.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.335+0000 7fb73ac00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb73406dcf0 con 0x7fb7340b3ea0 2024-05-11T12:12:09.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.335+0000 7fb738c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7341619d0 0x7fb7340ae2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:09.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.336+0000 7fb738c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7341619d0 0x7fb7340ae2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59790/0 (socket says 172.21.15.167:59790) 2024-05-11T12:12:09.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.336+0000 7fb738c00640 1 -- 172.21.15.167:0/1662024507 learned_addr learned my addr 172.21.15.167:0/1662024507 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:09.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.336+0000 7fb738c00640 1 -- 172.21.15.167:0/1662024507 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb7340b3ea0 msgr2=0x7fb7340af0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:09.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.336+0000 7fb738c00640 1 --2- 172.21.15.167:0/1662024507 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb7340b3ea0 0x7fb7340af0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:09.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.336+0000 7fb738c00640 1 -- 172.21.15.167:0/1662024507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb734157750 msgr2=0x7fb7340adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:09.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.336+0000 7fb738c00640 1 --2- 172.21.15.167:0/1662024507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb734157750 0x7fb7340adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:09.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.336+0000 7fb738c00640 1 -- 172.21.15.167:0/1662024507 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7340aae20 con 0x7fb7341619d0 2024-05-11T12:12:09.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.336+0000 7fb738c00640 1 --2- 172.21.15.167:0/1662024507 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7341619d0 0x7fb7340ae2e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb72c00c680 tx=0x7fb72c00cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:09.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.336+0000 7fb72be00640 1 -- 172.21.15.167:0/1662024507 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb72c018070 con 0x7fb7341619d0 2024-05-11T12:12:09.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.337+0000 7fb72be00640 1 -- 172.21.15.167:0/1662024507 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb72c004d60 con 0x7fb7341619d0 2024-05-11T12:12:09.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.337+0000 7fb73ac00640 1 -- 172.21.15.167:0/1662024507 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7340ab0c0 con 0x7fb7341619d0 2024-05-11T12:12:09.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.337+0000 7fb72be00640 1 -- 172.21.15.167:0/1662024507 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb72c00a880 con 0x7fb7341619d0 2024-05-11T12:12:09.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.337+0000 7fb73ac00640 1 -- 172.21.15.167:0/1662024507 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7340ab680 con 0x7fb7341619d0 2024-05-11T12:12:09.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.338+0000 7fb73ac00640 1 -- 172.21.15.167:0/1662024507 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb734069590 con 0x7fb7341619d0 2024-05-11T12:12:09.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.339+0000 7fb72be00640 1 -- 172.21.15.167:0/1662024507 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb72c01b070 con 0x7fb7341619d0 2024-05-11T12:12:09.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.340+0000 7fb72be00640 1 --2- 172.21.15.167:0/1662024507 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb7140762d0 0x7fb714078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:09.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.340+0000 7fb72be00640 1 -- 172.21.15.167:0/1662024507 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7fb72c096cc0 con 0x7fb7341619d0 2024-05-11T12:12:09.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.340+0000 7fb733e00640 1 --2- 172.21.15.167:0/1662024507 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb7140762d0 0x7fb714078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:09.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.341+0000 7fb733e00640 1 --2- 172.21.15.167:0/1662024507 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb7140762d0 0x7fb714078790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb724034000 tx=0x7fb72402c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:09.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.344+0000 7fb72be00640 1 -- 172.21.15.167:0/1662024507 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb72c09d080 con 0x7fb7341619d0 2024-05-11T12:12:09.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.600+0000 7fb73ac00640 1 -- 172.21.15.167:0/1662024507 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb734146c30 con 0x7fb7341619d0 2024-05-11T12:12:09.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.600+0000 7fb72be00640 1 -- 172.21.15.167:0/1662024507 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) ==== 53+0+73 (secure 0 0 0) 0x7fb72c0603f0 con 0x7fb7341619d0 2024-05-11T12:12:09.601 INFO:teuthology.orchestra.run.smithi167.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-11T12:12:09.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.603+0000 7fb73ac00640 1 -- 172.21.15.167:0/1662024507 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb7140762d0 msgr2=0x7fb714078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:09.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.603+0000 7fb73ac00640 1 --2- 172.21.15.167:0/1662024507 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb7140762d0 0x7fb714078790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb724034000 tx=0x7fb72402c000 comp rx=0 tx=0).stop 2024-05-11T12:12:09.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.603+0000 7fb73ac00640 1 -- 172.21.15.167:0/1662024507 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7341619d0 msgr2=0x7fb7340ae2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:09.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.603+0000 7fb73ac00640 1 --2- 172.21.15.167:0/1662024507 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7341619d0 0x7fb7340ae2e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb72c00c680 tx=0x7fb72c00cb40 comp rx=0 tx=0).stop 2024-05-11T12:12:09.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.603+0000 7fb73ac00640 1 -- 172.21.15.167:0/1662024507 shutdown_connections 2024-05-11T12:12:09.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.603+0000 7fb73ac00640 1 --2- 172.21.15.167:0/1662024507 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7341619d0 0x7fb7340ae2e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:09.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.603+0000 7fb73ac00640 1 --2- 172.21.15.167:0/1662024507 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb7140762d0 0x7fb714078790 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:09.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.603+0000 7fb73ac00640 1 --2- 172.21.15.167:0/1662024507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb734157750 0x7fb7340adda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:09.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.603+0000 7fb73ac00640 1 --2- 172.21.15.167:0/1662024507 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb7340b3ea0 0x7fb7340af0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:09.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.603+0000 7fb73ac00640 1 -- 172.21.15.167:0/1662024507 >> 172.21.15.167:0/1662024507 conn(0x7fb7340cd8f0 msgr2=0x7fb73406c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:09.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.604+0000 7fb73ac00640 1 -- 172.21.15.167:0/1662024507 shutdown_connections 2024-05-11T12:12:09.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09.604+0000 7fb73ac00640 1 -- 172.21.15.167:0/1662024507 wait complete. 2024-05-11T12:12:09.623 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-11T12:12:09.623 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-11T12:12:09.623 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi167.front.sepia.ceph.com 2024-05-11T12:12:09.624 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-11T12:12:09.624 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-11T12:12:09.624 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-11T12:12:09.624 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-11T12:12:09.624 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-11T12:12:09.624 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-11T12:12:09.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:12:09.624 DEBUG:teuthology.orchestra.run.smithi167:> ip addr 2024-05-11T12:12:09.675 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-11T12:12:09.675 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-11T12:12:09.675 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2024-05-11T12:12:09.675 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:12:09.675 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2024-05-11T12:12:09.675 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:12:09.675 INFO:teuthology.orchestra.run.smithi167.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:12:09.675 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e6 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:12:09.676 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f0 2024-05-11T12:12:09.676 INFO:teuthology.orchestra.run.smithi167.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:12:09.676 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e7 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:12:09.676 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f1 2024-05-11T12:12:09.676 INFO:teuthology.orchestra.run.smithi167.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:12:09.676 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:08 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:12:09.676 INFO:teuthology.orchestra.run.smithi167.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-11T12:12:09.676 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:09 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:12:09.676 INFO:teuthology.orchestra.run.smithi167.stdout: inet 172.21.15.167/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-11T12:12:09.676 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft 42539sec preferred_lft 42539sec 2024-05-11T12:12:09.676 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::ec4:7aff:fe88:7309/64 scope link 2024-05-11T12:12:09.676 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:12:09.676 INFO:teuthology.orchestra.run.smithi167.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-11T12:12:09.676 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 1a:a8:8d:b3:f4:32 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:12:09.677 INFO:teuthology.orchestra.run.smithi167.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-11T12:12:09.677 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:12:09.677 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::b017:48ff:fea0:7183/64 scope link 2024-05-11T12:12:09.677 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:12:09.677 INFO:teuthology.orchestra.run.smithi167.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-11T12:12:09.677 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 1a:a8:8d:b3:f4:32 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-11T12:12:09.677 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::18a8:8dff:feb3:f432/64 scope link 2024-05-11T12:12:09.677 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:12:09.677 INFO:teuthology.orchestra.run.smithi167.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-11T12:12:09.677 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 7e:79:1f:cb:d2:83 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-11T12:12:09.677 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::7c79:1fff:fecb:d283/64 scope link 2024-05-11T12:12:09.677 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:12:09.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:12:09.678 DEBUG:teuthology.orchestra.run.smithi167:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-11T12:12:09.757 INFO:teuthology.orchestra.run.smithi167.stdout:1 2024-05-11T12:12:09.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:12:09.761 DEBUG:teuthology.orchestra.run.smithi167:> route 2024-05-11T12:12:09.815 INFO:teuthology.orchestra.run.smithi167.stdout:Kernel IP routing table 2024-05-11T12:12:09.816 INFO:teuthology.orchestra.run.smithi167.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-11T12:12:09.816 INFO:teuthology.orchestra.run.smithi167.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-11T12:12:09.816 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-11T12:12:09.816 INFO:teuthology.orchestra.run.smithi167.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-11T12:12:09.817 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T12:12:09.817 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-05-11T12:12:09.817 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-11T12:12:09.817 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-11T12:12:09.817 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-11T12:12:09.817 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-05-11T12:12:09.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T12:12:10.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:12:10.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:12:10.030 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2024-05-11T12:12:10.077 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-11T12:12:10.077 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-11T12:12:10.077 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-11T12:12:10.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:12:10.078 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set brx.0 up 2024-05-11T12:12:10.145 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T12:12:10.145 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-11T12:12:10.192 INFO:teuthology.orchestra.run.smithi167.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-11T12:12:10.192 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T12:12:10.192 DEBUG:teuthology.orchestra.run.smithi167:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-11T12:12:10.240 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-05-11T12:12:10.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:12:10.240 DEBUG:teuthology.orchestra.run.smithi167:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:12:10.365 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:12:10.365 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-11T12:12:10.402 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:12:10.402 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-11T12:12:10.402 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.167:3300,172.21.15.204:3300,172.21.15.204:3301" 2024-05-11T12:12:10.403 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: trying mount with new device syntax: 0@5f32d344-7289-4c27-881b-1533419ab678.cephfs=/ 2024-05-11T12:12:10.403 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.167:3300/172.21.15.204:3300/172.21.15.204:3301" will pass to kernel 2024-05-11T12:12:10.404 INFO:teuthology.orchestra.run.smithi167.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-11T12:12:10.404 INFO:teuthology.orchestra.run.smithi167.stdout: You just mounted a file system that supports labels which does not 2024-05-11T12:12:10.404 INFO:teuthology.orchestra.run.smithi167.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-11T12:12:10.404 INFO:teuthology.orchestra.run.smithi167.stdout: applications will generate AVC messages and not be allowed access to 2024-05-11T12:12:10.404 INFO:teuthology.orchestra.run.smithi167.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-11T12:12:10.408 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-11T12:12:10.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:12:10.408 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-11T12:12:10.477 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-11T12:12:10.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-11T12:12:10.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:12:10.612 INFO:teuthology.orchestra.run.smithi167.stdout:client5109 2024-05-11T12:12:10.612 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:12:10.612 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:12:10.665 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-11T12:12:10.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.861+0000 7f1944c00640 1 Processor -- start 2024-05-11T12:12:10.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.861+0000 7f1944c00640 1 -- start start 2024-05-11T12:12:10.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.862+0000 7f1944c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f194015b860 0x7f194006f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:10.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.862+0000 7f1944c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19400736a0 0x7f1940072b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:10.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.862+0000 7f1944c00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1940072640 0x7f1940073040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:10.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.862+0000 7f1944c00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f194015bc40 con 0x7f1940072640 2024-05-11T12:12:10.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.862+0000 7f1944c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f194006e410 con 0x7f194015b860 2024-05-11T12:12:10.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.862+0000 7f1944c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f194006e6f0 con 0x7f19400736a0 2024-05-11T12:12:10.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.863+0000 7f193e800640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1940072640 0x7f1940073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:10.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.863+0000 7f193e800640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1940072640 0x7f1940073040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59800/0 (socket says 172.21.15.167:59800) 2024-05-11T12:12:10.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.863+0000 7f193e800640 1 -- 172.21.15.167:0/278316042 learned_addr learned my addr 172.21.15.167:0/278316042 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:10.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.863+0000 7f193de00640 1 --2- 172.21.15.167:0/278316042 >> v2:172.21.15.204:3300/0 conn(0x7f194015b860 0x7f194006f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:10.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.863+0000 7f1937e00640 1 --2- 172.21.15.167:0/278316042 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19400736a0 0x7f1940072b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:10.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.863+0000 7f193e800640 1 -- 172.21.15.167:0/278316042 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19400736a0 msgr2=0x7f1940072b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:10.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.863+0000 7f193e800640 1 --2- 172.21.15.167:0/278316042 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19400736a0 0x7f1940072b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:10.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.863+0000 7f193e800640 1 -- 172.21.15.167:0/278316042 >> v2:172.21.15.204:3300/0 conn(0x7f194015b860 msgr2=0x7f194006f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:10.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.863+0000 7f193e800640 1 --2- 172.21.15.167:0/278316042 >> v2:172.21.15.204:3300/0 conn(0x7f194015b860 0x7f194006f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:10.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.863+0000 7f193e800640 1 -- 172.21.15.167:0/278316042 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f194006e9d0 con 0x7f1940072640 2024-05-11T12:12:10.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.863+0000 7f193de00640 1 --2- 172.21.15.167:0/278316042 >> v2:172.21.15.204:3300/0 conn(0x7f194015b860 0x7f194006f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:10.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.863+0000 7f1937e00640 1 --2- 172.21.15.167:0/278316042 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19400736a0 0x7f1940072b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:10.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.865+0000 7f193e800640 1 --2- 172.21.15.167:0/278316042 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1940072640 0x7f1940073040 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f194006e130 tx=0x7f193802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c5ab9f67f654044 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:10.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.865+0000 7f193d400640 1 -- 172.21.15.167:0/278316042 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1938036070 con 0x7f1940072640 2024-05-11T12:12:10.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.865+0000 7f193d400640 1 -- 172.21.15.167:0/278316042 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1938031600 con 0x7f1940072640 2024-05-11T12:12:10.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.865+0000 7f193d400640 1 -- 172.21.15.167:0/278316042 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19380319d0 con 0x7f1940072640 2024-05-11T12:12:10.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.866+0000 7f1944c00640 1 -- 172.21.15.167:0/278316042 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1940072640 msgr2=0x7f1940073040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:10.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.866+0000 7f1944c00640 1 --2- 172.21.15.167:0/278316042 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1940072640 0x7f1940073040 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f194006e130 tx=0x7f193802a6a0 comp rx=0 tx=0).stop 2024-05-11T12:12:10.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.866+0000 7f1944c00640 1 -- 172.21.15.167:0/278316042 shutdown_connections 2024-05-11T12:12:10.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.866+0000 7f1944c00640 1 --2- 172.21.15.167:0/278316042 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1940072640 0x7f1940073040 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:10.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.866+0000 7f1944c00640 1 --2- 172.21.15.167:0/278316042 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19400736a0 0x7f1940072b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:10.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.866+0000 7f1944c00640 1 --2- 172.21.15.167:0/278316042 >> v2:172.21.15.204:3300/0 conn(0x7f194015b860 0x7f194006f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:10.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.866+0000 7f1944c00640 1 -- 172.21.15.167:0/278316042 >> 172.21.15.167:0/278316042 conn(0x7f19400cd8f0 msgr2=0x7f19400cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:10.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.866+0000 7f1944c00640 1 -- 172.21.15.167:0/278316042 shutdown_connections 2024-05-11T12:12:10.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.866+0000 7f1944c00640 1 -- 172.21.15.167:0/278316042 wait complete. 2024-05-11T12:12:10.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.868+0000 7f1944c00640 1 Processor -- start 2024-05-11T12:12:10.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.868+0000 7f1944c00640 1 -- start start 2024-05-11T12:12:10.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.869+0000 7f1944c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1940072640 0x7f1940150840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:10.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.869+0000 7f1944c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19400736a0 0x7f19400aae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:10.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.869+0000 7f1944c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f194015b860 0x7f19401f4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:10.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.869+0000 7f1944c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f194006f9f0 con 0x7f194015b860 2024-05-11T12:12:10.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.869+0000 7f1944c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f194006ef90 con 0x7f1940072640 2024-05-11T12:12:10.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.869+0000 7f1944c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f194006fb70 con 0x7f19400736a0 2024-05-11T12:12:10.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.869+0000 7f193e800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f194015b860 0x7f19401f4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:10.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.869+0000 7f193e800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f194015b860 0x7f19401f4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59812/0 (socket says 172.21.15.167:59812) 2024-05-11T12:12:10.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.869+0000 7f193e800640 1 -- 172.21.15.167:0/115587172 learned_addr learned my addr 172.21.15.167:0/115587172 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:10.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.869+0000 7f193de00640 1 --2- 172.21.15.167:0/115587172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1940072640 0x7f1940150840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:10.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.869+0000 7f1937e00640 1 --2- 172.21.15.167:0/115587172 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19400736a0 0x7f19400aae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:10.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.869+0000 7f193e800640 1 -- 172.21.15.167:0/115587172 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19400736a0 msgr2=0x7f19400aae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:10.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.869+0000 7f193e800640 1 --2- 172.21.15.167:0/115587172 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19400736a0 0x7f19400aae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:10.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.869+0000 7f193e800640 1 -- 172.21.15.167:0/115587172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1940072640 msgr2=0x7f1940150840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:10.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.870+0000 7f193e800640 1 --2- 172.21.15.167:0/115587172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1940072640 0x7f1940150840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:10.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.870+0000 7f193e800640 1 -- 172.21.15.167:0/115587172 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19400ab360 con 0x7f194015b860 2024-05-11T12:12:10.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.870+0000 7f193de00640 1 --2- 172.21.15.167:0/115587172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1940072640 0x7f1940150840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:10.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.870+0000 7f193e800640 1 --2- 172.21.15.167:0/115587172 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f194015b860 0x7f19401f4d70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1938034040 tx=0x7f1938004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:10.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.870+0000 7f1936000640 1 -- 172.21.15.167:0/115587172 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1938036050 con 0x7f194015b860 2024-05-11T12:12:10.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.870+0000 7f1936000640 1 -- 172.21.15.167:0/115587172 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1938042070 con 0x7f194015b860 2024-05-11T12:12:10.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.870+0000 7f1936000640 1 -- 172.21.15.167:0/115587172 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1938009510 con 0x7f194015b860 2024-05-11T12:12:10.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.870+0000 7f1944c00640 1 -- 172.21.15.167:0/115587172 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19400ab630 con 0x7f194015b860 2024-05-11T12:12:10.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.870+0000 7f1944c00640 1 -- 172.21.15.167:0/115587172 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1940153650 con 0x7f194015b860 2024-05-11T12:12:10.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.871+0000 7f1944c00640 1 -- 172.21.15.167:0/115587172 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1940069590 con 0x7f194015b860 2024-05-11T12:12:10.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.873+0000 7f1936000640 1 -- 172.21.15.167:0/115587172 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1938007bd0 con 0x7f194015b860 2024-05-11T12:12:10.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.874+0000 7f1936000640 1 --2- 172.21.15.167:0/115587172 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f191c0762d0 0x7f191c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:10.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.874+0000 7f1936000640 1 -- 172.21.15.167:0/115587172 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f19380b8790 con 0x7f194015b860 2024-05-11T12:12:10.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.874+0000 7f193de00640 1 --2- 172.21.15.167:0/115587172 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f191c0762d0 0x7f191c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:10.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.875+0000 7f193de00640 1 --2- 172.21.15.167:0/115587172 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f191c0762d0 0x7f191c078790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1930006fd0 tx=0x7f1930008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:10.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:10.879+0000 7f1936000640 1 -- 172.21.15.167:0/115587172 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1938081ec0 con 0x7f194015b860 2024-05-11T12:12:11.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:11.128+0000 7f1944c00640 1 -- 172.21.15.167:0/115587172 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f19400cf1d0 con 0x7f194015b860 2024-05-11T12:12:11.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:11.129+0000 7f1936000640 1 -- 172.21.15.167:0/115587172 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) ==== 53+0+73 (secure 0 0 0) 0x7f19380817c0 con 0x7f194015b860 2024-05-11T12:12:11.129 INFO:teuthology.orchestra.run.smithi167.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-11T12:12:11.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:11.131+0000 7f1944c00640 1 -- 172.21.15.167:0/115587172 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f191c0762d0 msgr2=0x7f191c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:11.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:11.131+0000 7f1944c00640 1 --2- 172.21.15.167:0/115587172 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f191c0762d0 0x7f191c078790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1930006fd0 tx=0x7f1930008040 comp rx=0 tx=0).stop 2024-05-11T12:12:11.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:11.131+0000 7f1944c00640 1 -- 172.21.15.167:0/115587172 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f194015b860 msgr2=0x7f19401f4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:11.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:11.131+0000 7f1944c00640 1 --2- 172.21.15.167:0/115587172 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f194015b860 0x7f19401f4d70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1938034040 tx=0x7f1938004a40 comp rx=0 tx=0).stop 2024-05-11T12:12:11.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:11.131+0000 7f1944c00640 1 -- 172.21.15.167:0/115587172 shutdown_connections 2024-05-11T12:12:11.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:11.131+0000 7f1944c00640 1 --2- 172.21.15.167:0/115587172 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f194015b860 0x7f19401f4d70 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:11.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:11.131+0000 7f1944c00640 1 --2- 172.21.15.167:0/115587172 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19400736a0 0x7f19400aae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:11.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:11.131+0000 7f1944c00640 1 --2- 172.21.15.167:0/115587172 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f191c0762d0 0x7f191c078790 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:11.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:11.131+0000 7f1944c00640 1 --2- 172.21.15.167:0/115587172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1940072640 0x7f1940150840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:11.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:11.131+0000 7f1944c00640 1 -- 172.21.15.167:0/115587172 >> 172.21.15.167:0/115587172 conn(0x7f19400cd8f0 msgr2=0x7f1940152930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:11.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:11.132+0000 7f1944c00640 1 -- 172.21.15.167:0/115587172 shutdown_connections 2024-05-11T12:12:11.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:11.132+0000 7f1944c00640 1 -- 172.21.15.167:0/115587172 wait complete. 2024-05-11T12:12:11.151 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-11T12:12:11.151 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-11T12:12:11.151 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi204.front.sepia.ceph.com 2024-05-11T12:12:11.151 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-11T12:12:11.151 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-11T12:12:11.151 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-11T12:12:11.151 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-11T12:12:11.151 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-11T12:12:11.151 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-11T12:12:11.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:12:11.152 DEBUG:teuthology.orchestra.run.smithi204:> ip addr 2024-05-11T12:12:11.210 INFO:teuthology.orchestra.run.smithi204.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-11T12:12:11.210 INFO:teuthology.orchestra.run.smithi204.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-11T12:12:11.210 INFO:teuthology.orchestra.run.smithi204.stdout: inet 127.0.0.1/8 scope host lo 2024-05-11T12:12:11.210 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:12:11.210 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 ::1/128 scope host 2024-05-11T12:12:11.210 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:12:11.210 INFO:teuthology.orchestra.run.smithi204.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:12:11.210 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2a brd ff:ff:ff:ff:ff:ff 2024-05-11T12:12:11.210 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f0 2024-05-11T12:12:11.210 INFO:teuthology.orchestra.run.smithi204.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:12:11.210 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2b brd ff:ff:ff:ff:ff:ff 2024-05-11T12:12:11.210 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f1 2024-05-11T12:12:11.211 INFO:teuthology.orchestra.run.smithi204.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:12:11.211 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:aa brd ff:ff:ff:ff:ff:ff 2024-05-11T12:12:11.211 INFO:teuthology.orchestra.run.smithi204.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-11T12:12:11.211 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:ab brd ff:ff:ff:ff:ff:ff 2024-05-11T12:12:11.211 INFO:teuthology.orchestra.run.smithi204.stdout: inet 172.21.15.204/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-11T12:12:11.211 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft 42533sec preferred_lft 42533sec 2024-05-11T12:12:11.211 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::ec4:7aff:fe88:77ab/64 scope link 2024-05-11T12:12:11.211 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:12:11.211 INFO:teuthology.orchestra.run.smithi204.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-11T12:12:11.211 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether e2:9e:5b:47:45:5d brd ff:ff:ff:ff:ff:ff 2024-05-11T12:12:11.211 INFO:teuthology.orchestra.run.smithi204.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-11T12:12:11.211 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:12:11.211 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::a498:7aff:fe50:48c3/64 scope link 2024-05-11T12:12:11.211 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:12:11.212 INFO:teuthology.orchestra.run.smithi204.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-11T12:12:11.212 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether fa:0a:7f:e4:30:35 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-11T12:12:11.212 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::f80a:7fff:fee4:3035/64 scope link 2024-05-11T12:12:11.212 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:12:11.212 INFO:teuthology.orchestra.run.smithi204.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-11T12:12:11.212 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether e2:9e:5b:47:45:5d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-11T12:12:11.212 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::e09e:5bff:fe47:455d/64 scope link 2024-05-11T12:12:11.212 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:12:11.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:12:11.213 DEBUG:teuthology.orchestra.run.smithi204:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-11T12:12:11.295 INFO:teuthology.orchestra.run.smithi204.stdout:1 2024-05-11T12:12:11.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:12:11.300 DEBUG:teuthology.orchestra.run.smithi204:> route 2024-05-11T12:12:11.355 INFO:teuthology.orchestra.run.smithi204.stdout:Kernel IP routing table 2024-05-11T12:12:11.355 INFO:teuthology.orchestra.run.smithi204.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-11T12:12:11.355 INFO:teuthology.orchestra.run.smithi204.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-11T12:12:11.355 INFO:teuthology.orchestra.run.smithi204.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-11T12:12:11.355 INFO:teuthology.orchestra.run.smithi204.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-11T12:12:11.356 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T12:12:11.356 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-05-11T12:12:11.356 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-11T12:12:11.356 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-11T12:12:11.356 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-11T12:12:11.356 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-05-11T12:12:11.453 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:12:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T12:12:11.553 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:12:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:12:11.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:12:11.563 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list 2024-05-11T12:12:11.614 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-11T12:12:11.615 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-11T12:12:11.615 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-11T12:12:11.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:12:11.616 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link set brx.0 up 2024-05-11T12:12:11.698 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T12:12:11.698 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-11T12:12:11.753 INFO:teuthology.orchestra.run.smithi204.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-11T12:12:11.753 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T12:12:11.754 DEBUG:teuthology.orchestra.run.smithi204:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-11T12:12:11.807 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-05-11T12:12:11.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:12:11.807 DEBUG:teuthology.orchestra.run.smithi204:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:12:11.935 INFO:teuthology.orchestra.run.smithi204.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:12:11.935 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-11T12:12:11.969 INFO:teuthology.orchestra.run.smithi204.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:12:11.969 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-11T12:12:11.969 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: resolved to: "172.21.15.167:3300,172.21.15.204:3300,172.21.15.204:3301" 2024-05-11T12:12:11.969 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: trying mount with new device syntax: 1@5f32d344-7289-4c27-881b-1533419ab678.cephfs=/ 2024-05-11T12:12:11.969 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.167:3300/172.21.15.204:3300/172.21.15.204:3301" will pass to kernel 2024-05-11T12:12:11.973 INFO:teuthology.orchestra.run.smithi204.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-05-11T12:12:11.973 INFO:teuthology.orchestra.run.smithi204.stdout: You just mounted a file system that supports labels which does not 2024-05-11T12:12:11.973 INFO:teuthology.orchestra.run.smithi204.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-11T12:12:11.973 INFO:teuthology.orchestra.run.smithi204.stdout: applications will generate AVC messages and not be allowed access to 2024-05-11T12:12:11.973 INFO:teuthology.orchestra.run.smithi204.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-11T12:12:11.976 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-11T12:12:11.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:12:11.976 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-11T12:12:12.047 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-11T12:12:12.149 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:12:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-11T12:12:12.154 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:12:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:12:12.164 INFO:teuthology.orchestra.run.smithi204.stdout:client4574 2024-05-11T12:12:12.164 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:12:12.164 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:12:12.211 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds ms_async_reap_threshold 1 2024-05-11T12:12:12.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.430+0000 7f5995800640 1 Processor -- start 2024-05-11T12:12:12.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.430+0000 7f5995800640 1 -- start start 2024-05-11T12:12:12.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.431+0000 7f5995800640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f599013fe30 0x7f599015de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:12.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.431+0000 7f5995800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f59901610c0 0x7f599006c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:12.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.431+0000 7f5995800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f599015e380 con 0x7f59900d0660 2024-05-11T12:12:12.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.431+0000 7f5995800640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f59900cf5c0 con 0x7f599013fe30 2024-05-11T12:12:12.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.431+0000 7f5995800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f59900cf8a0 con 0x7f59901610c0 2024-05-11T12:12:12.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.432+0000 7f598fe00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f599013fe30 0x7f599015de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:12.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.432+0000 7f598fe00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f599013fe30 0x7f599015de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59822/0 (socket says 172.21.15.167:59822) 2024-05-11T12:12:12.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.432+0000 7f598fe00640 1 -- 172.21.15.167:0/2395255949 learned_addr learned my addr 172.21.15.167:0/2395255949 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:12.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.432+0000 7f5994e00640 1 --2- 172.21.15.167:0/2395255949 >> v2:172.21.15.204:3300/0 conn(0x7f59901610c0 0x7f599006c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:12.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.432+0000 7f598fe00640 1 -- 172.21.15.167:0/2395255949 >> v2:172.21.15.204:3300/0 conn(0x7f59901610c0 msgr2=0x7f599006c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:12.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.432+0000 7f598fe00640 1 --2- 172.21.15.167:0/2395255949 >> v2:172.21.15.204:3300/0 conn(0x7f59901610c0 0x7f599006c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:12.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.432+0000 7f598fe00640 1 -- 172.21.15.167:0/2395255949 >> v1:172.21.15.204:6789/0 conn(0x7f59900d0660 legacy=0x7f59901609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:12.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.432+0000 7f598fe00640 1 -- 172.21.15.167:0/2395255949 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59900cfb80 con 0x7f599013fe30 2024-05-11T12:12:12.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.434+0000 7f598fe00640 1 --2- 172.21.15.167:0/2395255949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f599013fe30 0x7f599015de40 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f59900b3df0 tx=0x7f598002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9883f2e39b71083f server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:12.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.434+0000 7f598ea00640 1 -- 172.21.15.167:0/2395255949 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5980037070 con 0x7f599013fe30 2024-05-11T12:12:12.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.434+0000 7f598ea00640 1 -- 172.21.15.167:0/2395255949 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5980032440 con 0x7f599013fe30 2024-05-11T12:12:12.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.434+0000 7f598ea00640 1 -- 172.21.15.167:0/2395255949 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5980032810 con 0x7f599013fe30 2024-05-11T12:12:12.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.435+0000 7f5995800640 1 -- 172.21.15.167:0/2395255949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f599013fe30 msgr2=0x7f599015de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:12.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.435+0000 7f5995800640 1 --2- 172.21.15.167:0/2395255949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f599013fe30 0x7f599015de40 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f59900b3df0 tx=0x7f598002a1e0 comp rx=0 tx=0).stop 2024-05-11T12:12:12.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.435+0000 7f5995800640 1 -- 172.21.15.167:0/2395255949 shutdown_connections 2024-05-11T12:12:12.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.435+0000 7f5995800640 1 --2- 172.21.15.167:0/2395255949 >> v2:172.21.15.204:3300/0 conn(0x7f59901610c0 0x7f599006c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:12.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.435+0000 7f5995800640 1 --2- 172.21.15.167:0/2395255949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f599013fe30 0x7f599015de40 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:12.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.435+0000 7f5995800640 1 -- 172.21.15.167:0/2395255949 >> 172.21.15.167:0/2395255949 conn(0x7f59900b2a20 msgr2=0x7f59900b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:12.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.435+0000 7f5995800640 1 -- 172.21.15.167:0/2395255949 shutdown_connections 2024-05-11T12:12:12.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.435+0000 7f5995800640 1 -- 172.21.15.167:0/2395255949 wait complete. 2024-05-11T12:12:12.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.437+0000 7f5995800640 1 Processor -- start 2024-05-11T12:12:12.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.437+0000 7f5995800640 1 -- start start 2024-05-11T12:12:12.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.438+0000 7f5995800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59900d0660 0x7f5990150bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:12.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.438+0000 7f5995800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f599013fe30 0x7f5990073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:12.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.438+0000 7f5995800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59901610c0 0x7f59901f8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:12.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.438+0000 7f5995800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f59901616a0 con 0x7f59900d0660 2024-05-11T12:12:12.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.438+0000 7f5995800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f59900d0140 con 0x7f59901610c0 2024-05-11T12:12:12.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.438+0000 7f5995800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5990161820 con 0x7f599013fe30 2024-05-11T12:12:12.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.438+0000 7f598fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59900d0660 0x7f5990150bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:12.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.438+0000 7f598fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59900d0660 0x7f5990150bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59824/0 (socket says 172.21.15.167:59824) 2024-05-11T12:12:12.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.438+0000 7f598fe00640 1 -- 172.21.15.167:0/1535527923 learned_addr learned my addr 172.21.15.167:0/1535527923 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:12.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.439+0000 7f598fe00640 1 -- 172.21.15.167:0/1535527923 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f599013fe30 msgr2=0x7f5990073140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:12.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.439+0000 7f598fe00640 1 --2- 172.21.15.167:0/1535527923 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f599013fe30 0x7f5990073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:12.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.439+0000 7f598fe00640 1 -- 172.21.15.167:0/1535527923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59901610c0 msgr2=0x7f59901f8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:12.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.439+0000 7f5994e00640 1 --2- 172.21.15.167:0/1535527923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59901610c0 0x7f59901f8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:12.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.439+0000 7f598fe00640 1 --2- 172.21.15.167:0/1535527923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59901610c0 0x7f59901f8ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:12.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.439+0000 7f598fe00640 1 -- 172.21.15.167:0/1535527923 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5990134fb0 con 0x7f59900d0660 2024-05-11T12:12:12.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.439+0000 7f598fe00640 1 --2- 172.21.15.167:0/1535527923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59900d0660 0x7f5990150bd0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5980000c00 tx=0x7f59800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:12.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.439+0000 7f5987e00640 1 -- 172.21.15.167:0/1535527923 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5980041070 con 0x7f59900d0660 2024-05-11T12:12:12.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.439+0000 7f5987e00640 1 -- 172.21.15.167:0/1535527923 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f598003c870 con 0x7f59900d0660 2024-05-11T12:12:12.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.439+0000 7f5987e00640 1 -- 172.21.15.167:0/1535527923 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5980037040 con 0x7f59900d0660 2024-05-11T12:12:12.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.439+0000 7f5995800640 1 -- 172.21.15.167:0/1535527923 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5990135290 con 0x7f59900d0660 2024-05-11T12:12:12.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.440+0000 7f5995800640 1 -- 172.21.15.167:0/1535527923 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5990137680 con 0x7f59900d0660 2024-05-11T12:12:12.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.441+0000 7f5995800640 1 -- 172.21.15.167:0/1535527923 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59900694b0 con 0x7f59900d0660 2024-05-11T12:12:12.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.442+0000 7f5987e00640 1 -- 172.21.15.167:0/1535527923 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5980044070 con 0x7f59900d0660 2024-05-11T12:12:12.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.443+0000 7f5987e00640 1 --2- 172.21.15.167:0/1535527923 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f59700762d0 0x7f5970078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:12.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.443+0000 7f5987e00640 1 -- 172.21.15.167:0/1535527923 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f59800b8170 con 0x7f59900d0660 2024-05-11T12:12:12.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.443+0000 7f598f400640 1 --2- 172.21.15.167:0/1535527923 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f59700762d0 0x7f5970078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:12.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.444+0000 7f598f400640 1 --2- 172.21.15.167:0/1535527923 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f59700762d0 0x7f5970078790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5990135590 tx=0x7f597c0079b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:12.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.447+0000 7f5987e00640 1 -- 172.21.15.167:0/1535527923 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f599015e380 con 0x7f59900d0660 2024-05-11T12:12:12.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.659+0000 7f5995800640 1 -- 172.21.15.167:0/1535527923 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command([{prefix=config set, name=ms_async_reap_threshold}] v 0) -- 0x7f59900b41b0 con 0x7f59900d0660 2024-05-11T12:12:12.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.666+0000 7f5987e00640 1 -- 172.21.15.167:0/1535527923 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{prefix=config set, name=ms_async_reap_threshold}]=0 v11)=0 v11) ==== 121+0+0 (secure 0 0 0) 0x7f5980081940 con 0x7f59900d0660 2024-05-11T12:12:12.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.669+0000 7f5995800640 1 -- 172.21.15.167:0/1535527923 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f59700762d0 msgr2=0x7f5970078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:12.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.669+0000 7f5995800640 1 --2- 172.21.15.167:0/1535527923 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f59700762d0 0x7f5970078790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5990135590 tx=0x7f597c0079b0 comp rx=0 tx=0).stop 2024-05-11T12:12:12.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.669+0000 7f5995800640 1 -- 172.21.15.167:0/1535527923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59900d0660 msgr2=0x7f5990150bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:12.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.669+0000 7f5995800640 1 --2- 172.21.15.167:0/1535527923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59900d0660 0x7f5990150bd0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5980000c00 tx=0x7f59800046c0 comp rx=0 tx=0).stop 2024-05-11T12:12:12.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.669+0000 7f5995800640 1 -- 172.21.15.167:0/1535527923 shutdown_connections 2024-05-11T12:12:12.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.669+0000 7f5995800640 1 --2- 172.21.15.167:0/1535527923 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f59700762d0 0x7f5970078790 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:12.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.670+0000 7f5995800640 1 --2- 172.21.15.167:0/1535527923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59901610c0 0x7f59901f8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:12.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.670+0000 7f5995800640 1 --2- 172.21.15.167:0/1535527923 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f599013fe30 0x7f5990073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:12.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.670+0000 7f5995800640 1 --2- 172.21.15.167:0/1535527923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59900d0660 0x7f5990150bd0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:12.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.670+0000 7f5995800640 1 -- 172.21.15.167:0/1535527923 >> 172.21.15.167:0/1535527923 conn(0x7f59900b2a20 msgr2=0x7f5990136270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:12.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.670+0000 7f5995800640 1 -- 172.21.15.167:0/1535527923 shutdown_connections 2024-05-11T12:12:12.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:12.670+0000 7f5995800640 1 -- 172.21.15.167:0/1535527923 wait complete. 2024-05-11T12:12:12.690 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:12:12.690 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:12:12.738 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:12:12.738 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:12:12.833 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-11T12:12:13.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.040+0000 7f10aee00640 1 Processor -- start 2024-05-11T12:12:13.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.041+0000 7f10aee00640 1 -- start start 2024-05-11T12:12:13.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.042+0000 7f10aee00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f10a8161380 0x7f10a8074260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:13.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.042+0000 7f10aee00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10a80b65e0 con 0x7f10a815ac80 2024-05-11T12:12:13.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.042+0000 7f10aee00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10a80b68c0 con 0x7f10a80b3d50 2024-05-11T12:12:13.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.042+0000 7f10aee00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f10a806e3c0 con 0x7f10a8161380 2024-05-11T12:12:13.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.043+0000 7f109f400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f10a8161380 0x7f10a8074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:13.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.043+0000 7f10a7e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f10a815ac80 0x7f10a8160c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:50110/0 (socket says 172.21.15.167:50110) 2024-05-11T12:12:13.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.043+0000 7f10a7e00640 1 -- 172.21.15.167:0/2582957093 learned_addr learned my addr 172.21.15.167:0/2582957093 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:13.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.043+0000 7f109f400640 1 -- 172.21.15.167:0/2582957093 >> v1:172.21.15.204:6789/0 conn(0x7f10a80b3d50 legacy=0x7f10a8072950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:13.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.043+0000 7f109f400640 1 -- 172.21.15.167:0/2582957093 >> v1:172.21.15.167:6789/0 conn(0x7f10a815ac80 legacy=0x7f10a8160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:13.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.043+0000 7f109f400640 1 -- 172.21.15.167:0/2582957093 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10a80b6ba0 con 0x7f10a8161380 2024-05-11T12:12:13.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.043+0000 7f10a7400640 1 -- 172.21.15.167:0/2582957093 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 612377248 0 0) 0x7f10a80b65e0 con 0x7f10a815ac80 2024-05-11T12:12:13.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.044+0000 7f109f400640 1 --2- 172.21.15.167:0/2582957093 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10a8161380 0x7f10a8074260 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f10900097a0 tx=0x7f109002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bccf48bb2625c790 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:13.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.045+0000 7f10a7400640 1 -- 172.21.15.167:0/2582957093 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1090037070 con 0x7f10a8161380 2024-05-11T12:12:13.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.045+0000 7f10a7400640 1 -- 172.21.15.167:0/2582957093 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1090032440 con 0x7f10a8161380 2024-05-11T12:12:13.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.045+0000 7f10a7400640 1 -- 172.21.15.167:0/2582957093 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1090032810 con 0x7f10a8161380 2024-05-11T12:12:13.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.045+0000 7f10aee00640 1 -- 172.21.15.167:0/2582957093 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10a8161380 msgr2=0x7f10a8074260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:13.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.045+0000 7f10aee00640 1 --2- 172.21.15.167:0/2582957093 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10a8161380 0x7f10a8074260 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f10900097a0 tx=0x7f109002a180 comp rx=0 tx=0).stop 2024-05-11T12:12:13.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.045+0000 7f10aee00640 1 -- 172.21.15.167:0/2582957093 shutdown_connections 2024-05-11T12:12:13.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.045+0000 7f10aee00640 1 --2- 172.21.15.167:0/2582957093 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10a8161380 0x7f10a8074260 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:13.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.045+0000 7f10aee00640 1 -- 172.21.15.167:0/2582957093 >> 172.21.15.167:0/2582957093 conn(0x7f10a806c450 msgr2=0x7f10a806c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:13.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.045+0000 7f10aee00640 1 -- 172.21.15.167:0/2582957093 shutdown_connections 2024-05-11T12:12:13.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.045+0000 7f10aee00640 1 -- 172.21.15.167:0/2582957093 wait complete. 2024-05-11T12:12:13.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.047+0000 7f10aee00640 1 Processor -- start 2024-05-11T12:12:13.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.047+0000 7f10aee00640 1 -- start start 2024-05-11T12:12:13.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.048+0000 7f10aee00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10a80b3d50 0x7f10a8143cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:13.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.048+0000 7f10aee00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10a815ac80 0x7f10a815c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:13.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.048+0000 7f10aee00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f10a8161380 0x7f10a815c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:13.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.048+0000 7f10aee00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f10a80b4160 con 0x7f10a80b3d50 2024-05-11T12:12:13.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.048+0000 7f10aee00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f10a815b060 con 0x7f10a815ac80 2024-05-11T12:12:13.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.048+0000 7f10aee00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f10a8073060 con 0x7f10a8161380 2024-05-11T12:12:13.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.048+0000 7f10a7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10a80b3d50 0x7f10a8143cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:13.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.048+0000 7f10a7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10a80b3d50 0x7f10a8143cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59840/0 (socket says 172.21.15.167:59840) 2024-05-11T12:12:13.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.048+0000 7f10a7e00640 1 -- 172.21.15.167:0/2492268297 learned_addr learned my addr 172.21.15.167:0/2492268297 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:13.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.049+0000 7f10a7e00640 1 -- 172.21.15.167:0/2492268297 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f10a8161380 msgr2=0x7f10a815c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:13.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.049+0000 7f10a7e00640 1 --2- 172.21.15.167:0/2492268297 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f10a8161380 0x7f10a815c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:13.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.049+0000 7f10a7e00640 1 -- 172.21.15.167:0/2492268297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10a815ac80 msgr2=0x7f10a815c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:13.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.049+0000 7f10a7e00640 1 --2- 172.21.15.167:0/2492268297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10a815ac80 0x7f10a815c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:13.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.049+0000 7f10a7e00640 1 -- 172.21.15.167:0/2492268297 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10a80abb20 con 0x7f10a80b3d50 2024-05-11T12:12:13.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.049+0000 7f10a7e00640 1 --2- 172.21.15.167:0/2492268297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10a80b3d50 0x7f10a8143cc0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f109800c490 tx=0x7f109800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:13.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.049+0000 7f10a4c00640 1 -- 172.21.15.167:0/2492268297 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10980075a0 con 0x7f10a80b3d50 2024-05-11T12:12:13.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.049+0000 7f10a4c00640 1 -- 172.21.15.167:0/2492268297 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10980047e0 con 0x7f10a80b3d50 2024-05-11T12:12:13.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.049+0000 7f10a4c00640 1 -- 172.21.15.167:0/2492268297 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1098004bd0 con 0x7f10a80b3d50 2024-05-11T12:12:13.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.049+0000 7f10aee00640 1 -- 172.21.15.167:0/2492268297 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10a80abe00 con 0x7f10a80b3d50 2024-05-11T12:12:13.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.050+0000 7f10aee00640 1 -- 172.21.15.167:0/2492268297 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10a80ac3c0 con 0x7f10a80b3d50 2024-05-11T12:12:13.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.051+0000 7f10aee00640 1 -- 172.21.15.167:0/2492268297 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10a80694b0 con 0x7f10a80b3d50 2024-05-11T12:12:13.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.052+0000 7f10a4c00640 1 -- 172.21.15.167:0/2492268297 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1098011070 con 0x7f10a80b3d50 2024-05-11T12:12:13.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.053+0000 7f10a4c00640 1 --2- 172.21.15.167:0/2492268297 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f10840762d0 0x7f1084078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:13.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.053+0000 7f10a4c00640 1 -- 172.21.15.167:0/2492268297 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f10980973b0 con 0x7f10a80b3d50 2024-05-11T12:12:13.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.053+0000 7f109f400640 1 --2- 172.21.15.167:0/2492268297 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f10840762d0 0x7f1084078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:13.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.054+0000 7f109f400640 1 --2- 172.21.15.167:0/2492268297 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f10840762d0 0x7f1084078790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1090034000 tx=0x7f109002c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:13.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.058+0000 7f10a4c00640 1 -- 172.21.15.167:0/2492268297 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1098060c10 con 0x7f10a80b3d50 2024-05-11T12:12:13.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.280+0000 7f10aee00640 1 -- 172.21.15.167:0/2492268297 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f10a8070120 con 0x7f10a80b3d50 2024-05-11T12:12:13.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.280+0000 7f10a4c00640 1 -- 172.21.15.167:0/2492268297 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v41) ==== 80+0+0 (secure 0 0 0) 0x7f1098060510 con 0x7f10a80b3d50 2024-05-11T12:12:13.281 INFO:teuthology.orchestra.run.smithi167.stderr:listed 0 entries 2024-05-11T12:12:13.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.283+0000 7f10aee00640 1 -- 172.21.15.167:0/2492268297 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f10840762d0 msgr2=0x7f1084078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:13.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.283+0000 7f10aee00640 1 --2- 172.21.15.167:0/2492268297 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f10840762d0 0x7f1084078790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1090034000 tx=0x7f109002c000 comp rx=0 tx=0).stop 2024-05-11T12:12:13.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.283+0000 7f10aee00640 1 -- 172.21.15.167:0/2492268297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10a80b3d50 msgr2=0x7f10a8143cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:13.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.283+0000 7f10aee00640 1 --2- 172.21.15.167:0/2492268297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10a80b3d50 0x7f10a8143cc0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f109800c490 tx=0x7f109800c950 comp rx=0 tx=0).stop 2024-05-11T12:12:13.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.284+0000 7f10aee00640 1 -- 172.21.15.167:0/2492268297 shutdown_connections 2024-05-11T12:12:13.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.284+0000 7f10aee00640 1 --2- 172.21.15.167:0/2492268297 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f10a8161380 0x7f10a815c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:13.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.284+0000 7f10aee00640 1 --2- 172.21.15.167:0/2492268297 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f10840762d0 0x7f1084078790 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:13.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.284+0000 7f10aee00640 1 --2- 172.21.15.167:0/2492268297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10a815ac80 0x7f10a815c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:13.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.284+0000 7f10aee00640 1 --2- 172.21.15.167:0/2492268297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10a80b3d50 0x7f10a8143cc0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:13.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.284+0000 7f10aee00640 1 -- 172.21.15.167:0/2492268297 >> 172.21.15.167:0/2492268297 conn(0x7f10a806c450 msgr2=0x7f10a80b5a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:13.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.284+0000 7f10aee00640 1 -- 172.21.15.167:0/2492268297 shutdown_connections 2024-05-11T12:12:13.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.284+0000 7f10aee00640 1 -- 172.21.15.167:0/2492268297 wait complete. 2024-05-11T12:12:13.303 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-11T12:12:13.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:12:13.303 DEBUG:teuthology.orchestra.run.smithi167:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-11T12:12:13.399 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2024-05-11T12:12:13.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:12:13.399 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-11T12:12:13.448 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:12:13.448 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:12:13.505 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:12:13.505 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:12:13.556 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-11T12:12:13.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.767+0000 7f6917e00640 1 Processor -- start 2024-05-11T12:12:13.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.768+0000 7f6917e00640 1 -- start start 2024-05-11T12:12:13.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.769+0000 7f6917e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f69181625d0 0x7f69180d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:13.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.769+0000 7f6917e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69180b8390 0x7f69180b8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:13.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.769+0000 7f6917e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f69180d3c00 0x7f69180d4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:13.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.769+0000 7f6917e00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f69180d45b0 con 0x7f69180d3c00 2024-05-11T12:12:13.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.769+0000 7f6917e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f69180d2e40 con 0x7f69181625d0 2024-05-11T12:12:13.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.769+0000 7f6917e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f69180d3120 con 0x7f69180b8390 2024-05-11T12:12:13.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.769+0000 7f6917400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f69180d3c00 0x7f69180d4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:13.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.770+0000 7f6917400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f69180d3c00 0x7f69180d4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59852/0 (socket says 172.21.15.167:59852) 2024-05-11T12:12:13.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.770+0000 7f6917400640 1 -- 172.21.15.167:0/4099331347 learned_addr learned my addr 172.21.15.167:0/4099331347 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:13.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.770+0000 7f690fe00640 1 --2- 172.21.15.167:0/4099331347 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69180b8390 0x7f69180b8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:13.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.770+0000 7f6916a00640 1 --2- 172.21.15.167:0/4099331347 >> v2:172.21.15.204:3300/0 conn(0x7f69181625d0 0x7f69180d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:13.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.770+0000 7f6917400640 1 -- 172.21.15.167:0/4099331347 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69180b8390 msgr2=0x7f69180b8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:13.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.770+0000 7f6917400640 1 --2- 172.21.15.167:0/4099331347 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69180b8390 0x7f69180b8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:13.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.770+0000 7f6917400640 1 -- 172.21.15.167:0/4099331347 >> v2:172.21.15.204:3300/0 conn(0x7f69181625d0 msgr2=0x7f69180d0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:13.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.770+0000 7f6917400640 1 --2- 172.21.15.167:0/4099331347 >> v2:172.21.15.204:3300/0 conn(0x7f69181625d0 0x7f69180d0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:13.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.770+0000 7f6917400640 1 -- 172.21.15.167:0/4099331347 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69180d3400 con 0x7f69180d3c00 2024-05-11T12:12:13.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.771+0000 7f6917400640 1 --2- 172.21.15.167:0/4099331347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69180d3c00 0x7f69180d4070 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f69180b7c10 tx=0x7f691002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=115d54dd476833fe server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:13.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.771+0000 7f6916000640 1 -- 172.21.15.167:0/4099331347 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6910036070 con 0x7f69180d3c00 2024-05-11T12:12:13.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.771+0000 7f6916000640 1 -- 172.21.15.167:0/4099331347 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6910031600 con 0x7f69180d3c00 2024-05-11T12:12:13.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.771+0000 7f6916000640 1 -- 172.21.15.167:0/4099331347 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69100319d0 con 0x7f69180d3c00 2024-05-11T12:12:13.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.772+0000 7f6917e00640 1 -- 172.21.15.167:0/4099331347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69180d3c00 msgr2=0x7f69180d4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:13.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.772+0000 7f6917e00640 1 --2- 172.21.15.167:0/4099331347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69180d3c00 0x7f69180d4070 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f69180b7c10 tx=0x7f691002a6a0 comp rx=0 tx=0).stop 2024-05-11T12:12:13.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.772+0000 7f6917e00640 1 -- 172.21.15.167:0/4099331347 shutdown_connections 2024-05-11T12:12:13.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.772+0000 7f6917e00640 1 --2- 172.21.15.167:0/4099331347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69180d3c00 0x7f69180d4070 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:13.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.772+0000 7f6917e00640 1 --2- 172.21.15.167:0/4099331347 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69180b8390 0x7f69180b8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:13.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.772+0000 7f6917e00640 1 --2- 172.21.15.167:0/4099331347 >> v2:172.21.15.204:3300/0 conn(0x7f69181625d0 0x7f69180d0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:13.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.772+0000 7f6917e00640 1 -- 172.21.15.167:0/4099331347 >> 172.21.15.167:0/4099331347 conn(0x7f69180d50f0 msgr2=0x7f69180d5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:13.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.772+0000 7f6917e00640 1 -- 172.21.15.167:0/4099331347 shutdown_connections 2024-05-11T12:12:13.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.772+0000 7f6917e00640 1 -- 172.21.15.167:0/4099331347 wait complete. 2024-05-11T12:12:13.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.774+0000 7f6917e00640 1 Processor -- start 2024-05-11T12:12:13.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.774+0000 7f6917e00640 1 -- start start 2024-05-11T12:12:13.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.775+0000 7f6917e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69180b8390 0x7f6918149380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:13.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.775+0000 7f6917e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69180d3c00 0x7f69180cc250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:13.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.775+0000 7f6917e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69181625d0 0x7f691814be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:13.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.775+0000 7f6917e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6918162b30 con 0x7f69180d3c00 2024-05-11T12:12:13.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.775+0000 7f6917e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f69181629b0 con 0x7f69181625d0 2024-05-11T12:12:13.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.775+0000 7f6917e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6918162d50 con 0x7f69180b8390 2024-05-11T12:12:13.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.775+0000 7f690fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69180d3c00 0x7f69180cc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:13.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.775+0000 7f690fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69180d3c00 0x7f69180cc250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59862/0 (socket says 172.21.15.167:59862) 2024-05-11T12:12:13.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.775+0000 7f690fe00640 1 -- 172.21.15.167:0/454650780 learned_addr learned my addr 172.21.15.167:0/454650780 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:13.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.776+0000 7f690fe00640 1 -- 172.21.15.167:0/454650780 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69180b8390 msgr2=0x7f6918149380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:13.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.776+0000 7f690fe00640 1 --2- 172.21.15.167:0/454650780 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69180b8390 0x7f6918149380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:13.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.776+0000 7f6917400640 1 --2- 172.21.15.167:0/454650780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69181625d0 0x7f691814be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:13.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.776+0000 7f690fe00640 1 -- 172.21.15.167:0/454650780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69181625d0 msgr2=0x7f691814be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:13.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.776+0000 7f690fe00640 1 --2- 172.21.15.167:0/454650780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69181625d0 0x7f691814be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:13.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.776+0000 7f690fe00640 1 -- 172.21.15.167:0/454650780 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f691814c360 con 0x7f69180d3c00 2024-05-11T12:12:13.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.776+0000 7f690fe00640 1 --2- 172.21.15.167:0/454650780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69180d3c00 0x7f69180cc250 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f690000d7d0 tx=0x7f690000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:13.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.776+0000 7f690ea00640 1 -- 172.21.15.167:0/454650780 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6900018030 con 0x7f69180d3c00 2024-05-11T12:12:13.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.776+0000 7f690ea00640 1 -- 172.21.15.167:0/454650780 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6900004020 con 0x7f69180d3c00 2024-05-11T12:12:13.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.776+0000 7f690ea00640 1 -- 172.21.15.167:0/454650780 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6900004230 con 0x7f69180d3c00 2024-05-11T12:12:13.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.776+0000 7f6917e00640 1 -- 172.21.15.167:0/454650780 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69180cdd30 con 0x7f69180d3c00 2024-05-11T12:12:13.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.776+0000 7f6917e00640 1 -- 172.21.15.167:0/454650780 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69180ce2b0 con 0x7f69180d3c00 2024-05-11T12:12:13.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.777+0000 7f6917e00640 1 -- 172.21.15.167:0/454650780 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69180694b0 con 0x7f69180d3c00 2024-05-11T12:12:13.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.779+0000 7f690ea00640 1 -- 172.21.15.167:0/454650780 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f690001b070 con 0x7f69180d3c00 2024-05-11T12:12:13.780 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.780+0000 7f690ea00640 1 --2- 172.21.15.167:0/454650780 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f68f4076300 0x7f68f40787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:13.780 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.780+0000 7f690ea00640 1 -- 172.21.15.167:0/454650780 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f69000978c0 con 0x7f69180d3c00 2024-05-11T12:12:13.780 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.780+0000 7f6916a00640 1 --2- 172.21.15.167:0/454650780 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f68f4076300 0x7f68f40787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:13.780 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.780+0000 7f6916a00640 1 --2- 172.21.15.167:0/454650780 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f68f4076300 0x7f68f40787c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6908006fd0 tx=0x7f6908008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:13.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:13.784+0000 7f690ea00640 1 -- 172.21.15.167:0/454650780 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f691814c360 con 0x7f69180d3c00 2024-05-11T12:12:14.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.004+0000 7f6917e00640 1 -- 172.21.15.167:0/454650780 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f69180d4080 con 0x7f69180d3c00 2024-05-11T12:12:14.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.005+0000 7f690ea00640 1 -- 172.21.15.167:0/454650780 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v41) ==== 80+0+0 (secure 0 0 0) 0x7f6900061080 con 0x7f69180d3c00 2024-05-11T12:12:14.005 INFO:teuthology.orchestra.run.smithi167.stderr:listed 0 entries 2024-05-11T12:12:14.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.008+0000 7f6917e00640 1 -- 172.21.15.167:0/454650780 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f68f4076300 msgr2=0x7f68f40787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:14.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.008+0000 7f6917e00640 1 --2- 172.21.15.167:0/454650780 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f68f4076300 0x7f68f40787c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6908006fd0 tx=0x7f6908008040 comp rx=0 tx=0).stop 2024-05-11T12:12:14.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.008+0000 7f6917e00640 1 -- 172.21.15.167:0/454650780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69180d3c00 msgr2=0x7f69180cc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:14.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.008+0000 7f6917e00640 1 --2- 172.21.15.167:0/454650780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69180d3c00 0x7f69180cc250 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f690000d7d0 tx=0x7f690000dc90 comp rx=0 tx=0).stop 2024-05-11T12:12:14.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.008+0000 7f6917e00640 1 -- 172.21.15.167:0/454650780 shutdown_connections 2024-05-11T12:12:14.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.008+0000 7f6917e00640 1 --2- 172.21.15.167:0/454650780 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f68f4076300 0x7f68f40787c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:14.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.008+0000 7f6917e00640 1 --2- 172.21.15.167:0/454650780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69181625d0 0x7f691814be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:14.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.008+0000 7f6917e00640 1 --2- 172.21.15.167:0/454650780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69180d3c00 0x7f69180cc250 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:14.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.008+0000 7f6917e00640 1 --2- 172.21.15.167:0/454650780 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69180b8390 0x7f6918149380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:14.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.008+0000 7f6917e00640 1 -- 172.21.15.167:0/454650780 >> 172.21.15.167:0/454650780 conn(0x7f69180d50f0 msgr2=0x7f691806ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:14.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.008+0000 7f6917e00640 1 -- 172.21.15.167:0/454650780 shutdown_connections 2024-05-11T12:12:14.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.008+0000 7f6917e00640 1 -- 172.21.15.167:0/454650780 wait complete. 2024-05-11T12:12:14.026 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-11T12:12:14.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:12:14.027 DEBUG:teuthology.orchestra.run.smithi204:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-05-11T12:12:14.128 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi204.front.sepia.ceph.com 2024-05-11T12:12:14.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:12:14.129 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-11T12:12:14.183 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:12:14.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.394+0000 7f9f30000640 1 Processor -- start 2024-05-11T12:12:14.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.394+0000 7f9f30000640 1 -- start start 2024-05-11T12:12:14.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.396+0000 7f9f30000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9f280af1f0 0x7f9f280cf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:14.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.396+0000 7f9f30000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f280af5d0 0x7f9f28073ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:14.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.396+0000 7f9f30000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f280cf700 con 0x7f9f2806d460 2024-05-11T12:12:14.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.396+0000 7f9f30000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f9f281608d0 con 0x7f9f280af1f0 2024-05-11T12:12:14.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.396+0000 7f9f30000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f9f2806fe10 con 0x7f9f280af5d0 2024-05-11T12:12:14.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.397+0000 7f9f2cc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f280af5d0 0x7f9f28073ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:14.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.397+0000 7f9f2cc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f280af5d0 0x7f9f28073ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:57042/0 (socket says 172.21.15.167:57042) 2024-05-11T12:12:14.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.397+0000 7f9f2cc00640 1 -- 172.21.15.167:0/1667816901 learned_addr learned my addr 172.21.15.167:0/1667816901 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:14.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.397+0000 7f9f2d600640 1 --2- 172.21.15.167:0/1667816901 >> v2:172.21.15.204:3300/0 conn(0x7f9f280af1f0 0x7f9f280cf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:14.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.398+0000 7f9f2cc00640 1 -- 172.21.15.167:0/1667816901 >> v2:172.21.15.204:3300/0 conn(0x7f9f280af1f0 msgr2=0x7f9f280cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:14.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.398+0000 7f9f2cc00640 1 --2- 172.21.15.167:0/1667816901 >> v2:172.21.15.204:3300/0 conn(0x7f9f280af1f0 0x7f9f280cf1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:14.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.398+0000 7f9f2cc00640 1 -- 172.21.15.167:0/1667816901 >> v1:172.21.15.204:6789/0 conn(0x7f9f2806d460 legacy=0x7f9f2806e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:14.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.398+0000 7f9f2cc00640 1 -- 172.21.15.167:0/1667816901 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f2815fcc0 con 0x7f9f280af5d0 2024-05-11T12:12:14.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.399+0000 7f9f2cc00640 1 --2- 172.21.15.167:0/1667816901 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f280af5d0 0x7f9f28073ce0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9f14009870 tx=0x7f9f1402a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d143b648a3dd3199 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:14.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.400+0000 7f9f27e00640 1 -- 172.21.15.167:0/1667816901 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f14037070 con 0x7f9f280af5d0 2024-05-11T12:12:14.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.400+0000 7f9f27e00640 1 -- 172.21.15.167:0/1667816901 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f14032440 con 0x7f9f280af5d0 2024-05-11T12:12:14.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.400+0000 7f9f27e00640 1 -- 172.21.15.167:0/1667816901 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f14032810 con 0x7f9f280af5d0 2024-05-11T12:12:14.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.401+0000 7f9f30000640 1 -- 172.21.15.167:0/1667816901 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f280af5d0 msgr2=0x7f9f28073ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:14.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.401+0000 7f9f30000640 1 --2- 172.21.15.167:0/1667816901 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f280af5d0 0x7f9f28073ce0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9f14009870 tx=0x7f9f1402a1c0 comp rx=0 tx=0).stop 2024-05-11T12:12:14.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.401+0000 7f9f30000640 1 -- 172.21.15.167:0/1667816901 shutdown_connections 2024-05-11T12:12:14.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.401+0000 7f9f30000640 1 --2- 172.21.15.167:0/1667816901 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f280af5d0 0x7f9f28073ce0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:14.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.401+0000 7f9f30000640 1 --2- 172.21.15.167:0/1667816901 >> v2:172.21.15.204:3300/0 conn(0x7f9f280af1f0 0x7f9f280cf1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:14.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.401+0000 7f9f30000640 1 -- 172.21.15.167:0/1667816901 >> 172.21.15.167:0/1667816901 conn(0x7f9f280cd8f0 msgr2=0x7f9f280cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:14.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.401+0000 7f9f30000640 1 -- 172.21.15.167:0/1667816901 shutdown_connections 2024-05-11T12:12:14.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.401+0000 7f9f30000640 1 -- 172.21.15.167:0/1667816901 wait complete. 2024-05-11T12:12:14.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.403+0000 7f9f30000640 1 Processor -- start 2024-05-11T12:12:14.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.403+0000 7f9f30000640 1 -- start start 2024-05-11T12:12:14.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.404+0000 7f9f30000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f2806d460 0x7f9f28156560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:14.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.404+0000 7f9f30000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f280af1f0 0x7f9f2815a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:14.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.404+0000 7f9f30000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f280af5d0 0x7f9f2815aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:14.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.404+0000 7f9f30000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9f28160560 con 0x7f9f280af1f0 2024-05-11T12:12:14.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.404+0000 7f9f30000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9f28160280 con 0x7f9f2806d460 2024-05-11T12:12:14.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.404+0000 7f9f30000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f9f2815f920 con 0x7f9f280af5d0 2024-05-11T12:12:14.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.404+0000 7f9f2cc00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f280af1f0 0x7f9f2815a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:14.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.404+0000 7f9f2cc00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f280af1f0 0x7f9f2815a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59872/0 (socket says 172.21.15.167:59872) 2024-05-11T12:12:14.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.404+0000 7f9f2cc00640 1 -- 172.21.15.167:0/2237632733 learned_addr learned my addr 172.21.15.167:0/2237632733 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:14.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.404+0000 7f9f2e000640 1 --2- 172.21.15.167:0/2237632733 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f280af5d0 0x7f9f2815aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:14.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.404+0000 7f9f2d600640 1 --2- 172.21.15.167:0/2237632733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f2806d460 0x7f9f28156560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:14.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.404+0000 7f9f2cc00640 1 -- 172.21.15.167:0/2237632733 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f280af5d0 msgr2=0x7f9f2815aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:14.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.404+0000 7f9f2cc00640 1 --2- 172.21.15.167:0/2237632733 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f280af5d0 0x7f9f2815aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:14.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.404+0000 7f9f2cc00640 1 -- 172.21.15.167:0/2237632733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f2806d460 msgr2=0x7f9f28156560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:14.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.404+0000 7f9f2cc00640 1 --2- 172.21.15.167:0/2237632733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f2806d460 0x7f9f28156560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:14.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.404+0000 7f9f2cc00640 1 -- 172.21.15.167:0/2237632733 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f280b23d0 con 0x7f9f280af1f0 2024-05-11T12:12:14.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.404+0000 7f9f2d600640 1 --2- 172.21.15.167:0/2237632733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f2806d460 0x7f9f28156560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:14.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.405+0000 7f9f2cc00640 1 --2- 172.21.15.167:0/2237632733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f280af1f0 0x7f9f2815a550 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9f14000c00 tx=0x7f9f140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:14.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.405+0000 7f9f25600640 1 -- 172.21.15.167:0/2237632733 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f14037040 con 0x7f9f280af1f0 2024-05-11T12:12:14.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.405+0000 7f9f25600640 1 -- 172.21.15.167:0/2237632733 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f14042070 con 0x7f9f280af1f0 2024-05-11T12:12:14.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.405+0000 7f9f30000640 1 -- 172.21.15.167:0/2237632733 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f280b26b0 con 0x7f9f280af1f0 2024-05-11T12:12:14.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.405+0000 7f9f25600640 1 -- 172.21.15.167:0/2237632733 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f14030940 con 0x7f9f280af1f0 2024-05-11T12:12:14.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.405+0000 7f9f30000640 1 -- 172.21.15.167:0/2237632733 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f2815b7f0 con 0x7f9f280af1f0 2024-05-11T12:12:14.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.406+0000 7f9f30000640 1 -- 172.21.15.167:0/2237632733 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f28069590 con 0x7f9f280af1f0 2024-05-11T12:12:14.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.408+0000 7f9f25600640 1 -- 172.21.15.167:0/2237632733 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9f1402ad10 con 0x7f9f280af1f0 2024-05-11T12:12:14.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.409+0000 7f9f25600640 1 --2- 172.21.15.167:0/2237632733 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f9f080762d0 0x7f9f08078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:14.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.409+0000 7f9f25600640 1 -- 172.21.15.167:0/2237632733 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f9f140b7d70 con 0x7f9f280af1f0 2024-05-11T12:12:14.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.409+0000 7f9f2d600640 1 --2- 172.21.15.167:0/2237632733 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f9f080762d0 0x7f9f08078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:14.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.410+0000 7f9f2d600640 1 --2- 172.21.15.167:0/2237632733 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f9f080762d0 0x7f9f08078790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c009ac0 tx=0x7f9f1c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:14.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.413+0000 7f9f25600640 1 -- 172.21.15.167:0/2237632733 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9f140814a0 con 0x7f9f280af1f0 2024-05-11T12:12:14.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.661+0000 7f9f30000640 1 -- 172.21.15.167:0/2237632733 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f2815b080 con 0x7f9f280af1f0 2024-05-11T12:12:14.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.662+0000 7f9f25600640 1 -- 172.21.15.167:0/2237632733 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5193 (secure 0 0 0) 0x7f9f14080da0 con 0x7f9f280af1f0 2024-05-11T12:12:14.662 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:12:14.663 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4499,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3271527772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3271527772},{"type":"v1","addr":"172.21.15.167:6836","nonce":3271527772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4648,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2293636645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2293636645},{"type":"v1","addr":"172.21.15.167:6837","nonce":2293636645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4902,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3093163428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3093163428},{"type":"v1","addr":"172.21.15.204:6834","nonce":3093163428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:11:56.852478+0000","modified":"2024-05-11T12:11:59.926074+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4908},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4908":{"gid":4908,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/3248611739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":3248611739},{"type":"v1","addr":"172.21.15.204:6835","nonce":3248611739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4908,"qdb_cluster":[4908]},"id":2}]} 2024-05-11T12:12:14.663 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 18 2024-05-11T12:12:14.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.665+0000 7f9f30000640 1 -- 172.21.15.167:0/2237632733 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f9f080762d0 msgr2=0x7f9f08078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:14.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.665+0000 7f9f30000640 1 --2- 172.21.15.167:0/2237632733 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f9f080762d0 0x7f9f08078790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c009ac0 tx=0x7f9f1c008040 comp rx=0 tx=0).stop 2024-05-11T12:12:14.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.665+0000 7f9f30000640 1 -- 172.21.15.167:0/2237632733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f280af1f0 msgr2=0x7f9f2815a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:14.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.665+0000 7f9f30000640 1 --2- 172.21.15.167:0/2237632733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f280af1f0 0x7f9f2815a550 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9f14000c00 tx=0x7f9f140046c0 comp rx=0 tx=0).stop 2024-05-11T12:12:14.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.665+0000 7f9f30000640 1 -- 172.21.15.167:0/2237632733 shutdown_connections 2024-05-11T12:12:14.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.665+0000 7f9f30000640 1 --2- 172.21.15.167:0/2237632733 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f280af5d0 0x7f9f2815aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:14.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.665+0000 7f9f30000640 1 --2- 172.21.15.167:0/2237632733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f280af1f0 0x7f9f2815a550 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:14.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.665+0000 7f9f30000640 1 --2- 172.21.15.167:0/2237632733 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f9f080762d0 0x7f9f08078790 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:14.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.665+0000 7f9f30000640 1 --2- 172.21.15.167:0/2237632733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f2806d460 0x7f9f28156560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:14.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.665+0000 7f9f30000640 1 -- 172.21.15.167:0/2237632733 >> 172.21.15.167:0/2237632733 conn(0x7f9f280cd8f0 msgr2=0x7f9f28163690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:14.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.665+0000 7f9f30000640 1 -- 172.21.15.167:0/2237632733 shutdown_connections 2024-05-11T12:12:14.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:14.665+0000 7f9f30000640 1 -- 172.21.15.167:0/2237632733 wait complete. 2024-05-11T12:12:14.684 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json perf dump 2024-05-11T12:12:14.911 INFO:teuthology.orchestra.run.smithi204.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":35,"msgr_send_messages":34,"msgr_recv_bytes":9400,"msgr_send_bytes":22099,"msgr_created_connections":8,"msgr_active_connections":2,"msgr_running_total_time":0.016301901,"msgr_running_send_time":0.002174909,"msgr_running_recv_time":0.007363609,"msgr_running_fast_dispatch_time":0.002892308,"msgr_send_messages_queue_lat":{"avgcount":32,"sum":0.097234330,"avgtime":0.003038572},"msgr_handle_ack_lat":{"avgcount":13,"sum":0.000004249,"avgtime":0.000000326},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"AsyncMessenger::Worker-1":{"msgr_recv_messages":70,"msgr_send_messages":63,"msgr_recv_bytes":124963,"msgr_send_bytes":17153,"msgr_created_connections":5,"msgr_active_connections":4,"msgr_running_total_time":0.020942360,"msgr_running_send_time":0.002807877,"msgr_running_recv_time":0.024174955,"msgr_running_fast_dispatch_time":0.000683061,"msgr_send_messages_queue_lat":{"avgcount":63,"sum":0.066166317,"avgtime":0.001050259},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":122144,"msgr_send_encrypted_bytes":12368},"AsyncMessenger::Worker-2":{"msgr_recv_messages":45,"msgr_send_messages":48,"msgr_recv_bytes":12809,"msgr_send_bytes":135877,"msgr_created_connections":8,"msgr_active_connections":3,"msgr_running_total_time":0.027775560,"msgr_running_send_time":0.004946011,"msgr_running_recv_time":0.022007756,"msgr_running_fast_dispatch_time":0.001499946,"msgr_send_messages_queue_lat":{"avgcount":47,"sum":0.079103978,"avgtime":0.001683063},"msgr_handle_ack_lat":{"avgcount":58,"sum":0.000021757,"avgtime":0.000000375},"msgr_recv_encrypted_bytes":1728,"msgr_send_encrypted_bytes":94352},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":54,"sum":0.003453088,"avgtime":0.000063946}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":2,"sum":0.000024224,"avgtime":0.000012112}},"mds":{"request":29,"reply":29,"reply_latency":{"avgcount":29,"sum":0.012356649,"avgtime":0.000426091},"slow_reply":0,"forward":0,"dir_fetch_complete":0,"dir_fetch_keys":0,"dir_commit":12,"dir_split":0,"dir_merge":0,"inodes":11,"inodes_top":0,"inodes_bottom":11,"inodes_pin_tail":0,"inodes_pinned":10,"inodes_expired":0,"inodes_with_caps":0,"caps":0,"subtrees":2,"traverse":55,"traverse_hit":53,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":0,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":0,"handle_client_caps_dirty":0,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_quiesce_path":0,"ireq_quiesce_inode":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evlrg":0,"evadd":8,"evex":0,"evtrm":0,"ev":8,"evexg":0,"evexd":0,"segadd":2,"segex":0,"segtrm":0,"seg":2,"segmjr":2,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4197029,"rdpos":0,"jlat":{"avgcount":5,"sum":0.020889364,"avgtime":0.004177872},"replayed":0},"mds_mem":{"ino":13,"ino+":13,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":11,"dn+":11,"dn-":0,"cap":0,"cap+":2,"cap-":2,"rss":41216,"heap":231788},"mds_server":{"dispatch_client_request":29,"dispatch_server_request":0,"handle_client_request":29,"handle_client_session":6,"handle_peer_request":0,"req_create_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_getattr_latency":{"avgcount":26,"sum":0.011224873,"avgtime":0.000431725},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":2,"sum":0.000478114,"avgtime":0.000239057},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":1,"sum":0.000653662,"avgtime":0.000653662},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":0,"session_add":2,"session_remove":2,"sessions_open":0,"sessions_stale":0,"total_load":28,"average_load":14,"avg_session_uptime":0,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":224915,"buffer_anon_items":81,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5424,"osdmap_items":76,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":53064,"mds_co_items":216,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":44,"op_send_bytes":13935,"op_resend":37,"op_reply":44,"op_latency":{"avgcount":44,"sum":0.827989677,"avgtime":0.018817947},"op_inflight":0,"oplen_avg":{"avgcount":44,"sum":68},"op":44,"op_r":0,"op_w":44,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":11,"osdop_read":0,"osdop_write":5,"osdop_writefull":8,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":22,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":41,"map_full":0,"map_inc":1,"osd_sessions":8,"osd_session_open":8,"osd_session_close":0,"osd_laggy":0,"omap_wr":15,"omap_rd":0,"omap_del":1},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":0,"omap_total_updates":0,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":145,"get_sum":129657,"get_or_fail_fail":0,"get_or_fail_success":145,"take":0,"take_sum":0,"put":145,"put_sum":129657,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":44,"take_sum":13935,"put":44,"put_sum":13935,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":44,"take_sum":44,"put":44,"put_sum":44,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x55747bba4500":{"val":0,"max":3758096384,"get_started":0,"get":8,"get_sum":2725,"get_or_fail_fail":0,"get_or_fail_success":8,"take":0,"take_sum":0,"put":5,"put_sum":2725,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-05-11T12:12:14.912 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 2, "msgr_created_connections": 8, "msgr_handle_ack_lat": { "avgcount": 13, "avgtime": 3.26e-07, "sum": 4.249e-06 }, "msgr_recv_bytes": 9400, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 35, "msgr_running_fast_dispatch_time": 0.002892308, "msgr_running_recv_time": 0.007363609, "msgr_running_send_time": 0.002174909, "msgr_running_total_time": 0.016301901, "msgr_send_bytes": 22099, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 34, "msgr_send_messages_queue_lat": { "avgcount": 32, "avgtime": 0.003038572, "sum": 0.09723433 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 4, "msgr_created_connections": 5, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 124963, "msgr_recv_encrypted_bytes": 122144, "msgr_recv_messages": 70, "msgr_running_fast_dispatch_time": 0.000683061, "msgr_running_recv_time": 0.024174955, "msgr_running_send_time": 0.002807877, "msgr_running_total_time": 0.02094236, "msgr_send_bytes": 17153, "msgr_send_encrypted_bytes": 12368, "msgr_send_messages": 63, "msgr_send_messages_queue_lat": { "avgcount": 63, "avgtime": 0.001050259, "sum": 0.066166317 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 3, "msgr_created_connections": 8, "msgr_handle_ack_lat": { "avgcount": 58, "avgtime": 3.75e-07, "sum": 2.1757e-05 }, "msgr_recv_bytes": 12809, "msgr_recv_encrypted_bytes": 1728, "msgr_recv_messages": 45, "msgr_running_fast_dispatch_time": 0.001499946, "msgr_running_recv_time": 0.022007756, "msgr_running_send_time": 0.004946011, "msgr_running_total_time": 0.02777556, "msgr_send_bytes": 135877, "msgr_send_encrypted_bytes": 94352, "msgr_send_messages": 48, "msgr_send_messages_queue_lat": { "avgcount": 47, "avgtime": 0.001683063, "sum": 0.079103978 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 54, "avgtime": 6.3946e-05, "sum": 0.003453088 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 2, "avgtime": 1.2112e-05, "sum": 2.4224e-05 }, "queue_len": 0 }, "mds": { "caps": 0, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 12, "dir_fetch_complete": 0, "dir_fetch_keys": 0, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 0, "handle_client_caps_dirty": 0, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 11, "inodes_bottom": 11, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 10, "inodes_top": 0, "inodes_with_caps": 0, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 29, "reply_latency": { "avgcount": 29, "avgtime": 0.000426091, "sum": 0.012356649 }, "request": 29, "root_rbytes": 0, "root_rfiles": 0, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 55, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 53, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "ireq_quiesce_inode": 0, "ireq_quiesce_path": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 8, "evadd": 8, "evex": 0, "evexd": 0, "evexg": 0, "evlrg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 5, "avgtime": 0.004177872, "sum": 0.020889364 }, "rdpos": 0, "replayed": 0, "seg": 2, "segadd": 2, "segex": 0, "segexd": 0, "segexg": 0, "segmjr": 2, "segtrm": 0, "wrpos": 4197029 }, "mds_mem": { "cap": 0, "cap+": 2, "cap-": 2, "dir": 12, "dir+": 12, "dir-": 0, "dn": 11, "dn+": 11, "dn-": 0, "heap": 231788, "ino": 13, "ino+": 13, "ino-": 0, "rss": 41216 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 29, "dispatch_server_request": 0, "handle_client_request": 29, "handle_client_session": 6, "handle_peer_request": 0, "req_create_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getattr_latency": { "avgcount": 26, "avgtime": 0.000431725, "sum": 0.011224873 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 2, "avgtime": 0.000239057, "sum": 0.000478114 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 1, "avgtime": 0.000653662, "sum": 0.000653662 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 14, "avg_session_uptime": 0, "mdthresh_evicted": 0, "session_add": 2, "session_count": 0, "session_remove": 2, "sessions_open": 0, "sessions_stale": 0, "total_load": 28 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 224915, "buffer_anon_items": 81, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 53064, "mds_co_items": 216, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5424, "osdmap_items": 76, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 41, "map_full": 0, "map_inc": 1, "omap_del": 1, "omap_rd": 0, "omap_wr": 15, "op": 44, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 44, "avgtime": 0.018817947, "sum": 0.827989677 }, "op_pg": 0, "op_r": 0, "op_reply": 44, "op_resend": 37, "op_rmw": 0, "op_send": 44, "op_send_bytes": 13935, "op_w": 44, "oplen_avg": { "avgcount": 44, "sum": 68 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 8, "osd_sessions": 8, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 11, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 22, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 5, "osdop_writefull": 8, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 0, "omap_total_objs": 1, "omap_total_removes": 0, "omap_total_updates": 0 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 145, "get_or_fail_fail": 0, "get_or_fail_success": 145, "get_started": 0, "get_sum": 129657, "max": 104857600, "put": 145, "put_sum": 129657, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 44, "put_sum": 13935, "take": 44, "take_sum": 13935, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 44, "put_sum": 44, "take": 44, "take_sum": 44, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x55747bba4500": { "get": 8, "get_or_fail_fail": 0, "get_or_fail_success": 8, "get_started": 0, "get_sum": 2725, "max": 3758096384, "put": 5, "put_sum": 2725, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-05-11T12:12:14.913 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-11T12:12:15.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.124+0000 7f5cef600640 1 Processor -- start 2024-05-11T12:12:15.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.124+0000 7f5cef600640 1 -- start start 2024-05-11T12:12:15.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.126+0000 7f5cef600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ce80af1f0 0x7f5ce80cf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:15.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.126+0000 7f5cef600640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5ce806d460 0x7f5ce806d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:15.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.126+0000 7f5cef600640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ce80cf700 con 0x7f5ce80af5d0 2024-05-11T12:12:15.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.126+0000 7f5cef600640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f5ce81608d0 con 0x7f5ce806d460 2024-05-11T12:12:15.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.126+0000 7f5cef600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5ce806fe10 con 0x7f5ce80af1f0 2024-05-11T12:12:15.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.127+0000 7f5ce7e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f5ce80af5d0 0x7f5ce8073ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:50112/0 (socket says 172.21.15.167:50112) 2024-05-11T12:12:15.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.127+0000 7f5ce7e00640 1 -- 172.21.15.167:0/4243679987 learned_addr learned my addr 172.21.15.167:0/4243679987 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:15.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.127+0000 7f5ced600640 1 --2- 172.21.15.167:0/4243679987 >> v2:172.21.15.167:3300/0 conn(0x7f5ce806d460 0x7f5ce806d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:15.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.127+0000 7f5cecc00640 1 --2- 172.21.15.167:0/4243679987 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ce80af1f0 0x7f5ce80cf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:15.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.127+0000 7f5ced600640 1 -- 172.21.15.167:0/4243679987 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ce80af1f0 msgr2=0x7f5ce80cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:15.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.127+0000 7f5ced600640 1 --2- 172.21.15.167:0/4243679987 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ce80af1f0 0x7f5ce80cf1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:15.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.127+0000 7f5ced600640 1 -- 172.21.15.167:0/4243679987 >> v1:172.21.15.167:6789/0 conn(0x7f5ce80af5d0 legacy=0x7f5ce8073ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:15.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.127+0000 7f5ced600640 1 -- 172.21.15.167:0/4243679987 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ce815fcc0 con 0x7f5ce806d460 2024-05-11T12:12:15.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.127+0000 7f5cecc00640 1 --2- 172.21.15.167:0/4243679987 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ce80af1f0 0x7f5ce80cf1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:15.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.128+0000 7f5ced600640 1 --2- 172.21.15.167:0/4243679987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ce806d460 0x7f5ce806d8f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0009d90 tx=0x7f5ce002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b19a36df40aafe79 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:15.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.128+0000 7f5ce7400640 1 -- 172.21.15.167:0/4243679987 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0031600 con 0x7f5ce806d460 2024-05-11T12:12:15.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.128+0000 7f5ce7400640 1 -- 172.21.15.167:0/4243679987 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ce00317a0 con 0x7f5ce806d460 2024-05-11T12:12:15.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.128+0000 7f5ce7400640 1 -- 172.21.15.167:0/4243679987 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0031b70 con 0x7f5ce806d460 2024-05-11T12:12:15.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.129+0000 7f5cef600640 1 -- 172.21.15.167:0/4243679987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ce806d460 msgr2=0x7f5ce806d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:15.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.129+0000 7f5cef600640 1 --2- 172.21.15.167:0/4243679987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ce806d460 0x7f5ce806d8f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0009d90 tx=0x7f5ce002a5d0 comp rx=0 tx=0).stop 2024-05-11T12:12:15.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.129+0000 7f5cef600640 1 -- 172.21.15.167:0/4243679987 shutdown_connections 2024-05-11T12:12:15.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.129+0000 7f5cef600640 1 --2- 172.21.15.167:0/4243679987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ce806d460 0x7f5ce806d8f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:15.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.129+0000 7f5cef600640 1 --2- 172.21.15.167:0/4243679987 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ce80af1f0 0x7f5ce80cf1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:15.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.129+0000 7f5cef600640 1 -- 172.21.15.167:0/4243679987 >> 172.21.15.167:0/4243679987 conn(0x7f5ce80cd8f0 msgr2=0x7f5ce80cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:15.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.129+0000 7f5cef600640 1 -- 172.21.15.167:0/4243679987 shutdown_connections 2024-05-11T12:12:15.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.129+0000 7f5cef600640 1 -- 172.21.15.167:0/4243679987 wait complete. 2024-05-11T12:12:15.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.131+0000 7f5cef600640 1 Processor -- start 2024-05-11T12:12:15.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.131+0000 7f5cef600640 1 -- start start 2024-05-11T12:12:15.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.132+0000 7f5cef600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ce806d460 0x7f5ce8150a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:15.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.132+0000 7f5cef600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ce80af1f0 0x7f5ce81f8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:15.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.132+0000 7f5cef600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ce80af5d0 0x7f5ce8156130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:15.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.132+0000 7f5cef600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce8160560 con 0x7f5ce80af5d0 2024-05-11T12:12:15.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.132+0000 7f5cef600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce8160280 con 0x7f5ce806d460 2024-05-11T12:12:15.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.132+0000 7f5cef600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5ce815f920 con 0x7f5ce80af1f0 2024-05-11T12:12:15.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.132+0000 7f5ced600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ce80af5d0 0x7f5ce8156130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:15.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.132+0000 7f5ced600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ce80af5d0 0x7f5ce8156130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59886/0 (socket says 172.21.15.167:59886) 2024-05-11T12:12:15.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.132+0000 7f5ced600640 1 -- 172.21.15.167:0/1819623305 learned_addr learned my addr 172.21.15.167:0/1819623305 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:15.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.133+0000 7f5cecc00640 1 --2- 172.21.15.167:0/1819623305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ce806d460 0x7f5ce8150a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:15.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.133+0000 7f5ce7e00640 1 --2- 172.21.15.167:0/1819623305 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ce80af1f0 0x7f5ce81f8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:15.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.133+0000 7f5ced600640 1 -- 172.21.15.167:0/1819623305 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ce80af1f0 msgr2=0x7f5ce81f8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:15.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.133+0000 7f5ced600640 1 --2- 172.21.15.167:0/1819623305 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ce80af1f0 0x7f5ce81f8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:15.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.133+0000 7f5ced600640 1 -- 172.21.15.167:0/1819623305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ce806d460 msgr2=0x7f5ce8150a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:15.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.133+0000 7f5ced600640 1 --2- 172.21.15.167:0/1819623305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ce806d460 0x7f5ce8150a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:15.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.133+0000 7f5ced600640 1 -- 172.21.15.167:0/1819623305 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ce815ac60 con 0x7f5ce80af5d0 2024-05-11T12:12:15.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.133+0000 7f5cecc00640 1 --2- 172.21.15.167:0/1819623305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ce806d460 0x7f5ce8150a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:15.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.133+0000 7f5ce7e00640 1 --2- 172.21.15.167:0/1819623305 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ce80af1f0 0x7f5ce81f8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:15.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.133+0000 7f5ced600640 1 --2- 172.21.15.167:0/1819623305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ce80af5d0 0x7f5ce8156130 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0009eb0 tx=0x7f5ce0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:15.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.134+0000 7f5ce4c00640 1 -- 172.21.15.167:0/1819623305 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0040070 con 0x7f5ce80af5d0 2024-05-11T12:12:15.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.134+0000 7f5ce4c00640 1 -- 172.21.15.167:0/1819623305 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ce0030b60 con 0x7f5ce80af5d0 2024-05-11T12:12:15.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.134+0000 7f5ce4c00640 1 -- 172.21.15.167:0/1819623305 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0045460 con 0x7f5ce80af5d0 2024-05-11T12:12:15.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.134+0000 7f5cef600640 1 -- 172.21.15.167:0/1819623305 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ce81f9520 con 0x7f5ce80af5d0 2024-05-11T12:12:15.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.134+0000 7f5cef600640 1 -- 172.21.15.167:0/1819623305 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ce8154f60 con 0x7f5ce80af5d0 2024-05-11T12:12:15.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.135+0000 7f5cef600640 1 -- 172.21.15.167:0/1819623305 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ce8069590 con 0x7f5ce80af5d0 2024-05-11T12:12:15.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.136+0000 7f5ce4c00640 1 -- 172.21.15.167:0/1819623305 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5ce0039040 con 0x7f5ce80af5d0 2024-05-11T12:12:15.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.137+0000 7f5ce4c00640 1 --2- 172.21.15.167:0/1819623305 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5cc80762d0 0x7f5cc8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:15.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.137+0000 7f5ce4c00640 1 -- 172.21.15.167:0/1819623305 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f5ce00b9000 con 0x7f5ce80af5d0 2024-05-11T12:12:15.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.137+0000 7f5cecc00640 1 --2- 172.21.15.167:0/1819623305 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5cc80762d0 0x7f5cc8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:15.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.138+0000 7f5cecc00640 1 --2- 172.21.15.167:0/1819623305 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5cc80762d0 0x7f5cc8078790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5ce80b2b70 tx=0x7f5cdc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:15.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.142+0000 7f5ce4c00640 1 -- 172.21.15.167:0/1819623305 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5ce81f9520 con 0x7f5ce80af5d0 2024-05-11T12:12:15.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.391+0000 7f5cef600640 1 -- 172.21.15.167:0/1819623305 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5ce815f110 con 0x7f5ce80af5d0 2024-05-11T12:12:15.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.391+0000 7f5ce4c00640 1 -- 172.21.15.167:0/1819623305 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) ==== 53+0+73 (secure 0 0 0) 0x7f5ce815ac60 con 0x7f5ce80af5d0 2024-05-11T12:12:15.392 INFO:teuthology.orchestra.run.smithi167.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-11T12:12:15.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.394+0000 7f5cef600640 1 -- 172.21.15.167:0/1819623305 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5cc80762d0 msgr2=0x7f5cc8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:15.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.394+0000 7f5cef600640 1 --2- 172.21.15.167:0/1819623305 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5cc80762d0 0x7f5cc8078790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5ce80b2b70 tx=0x7f5cdc008040 comp rx=0 tx=0).stop 2024-05-11T12:12:15.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.394+0000 7f5cef600640 1 -- 172.21.15.167:0/1819623305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ce80af5d0 msgr2=0x7f5ce8156130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:15.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.394+0000 7f5cef600640 1 --2- 172.21.15.167:0/1819623305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ce80af5d0 0x7f5ce8156130 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0009eb0 tx=0x7f5ce0004a40 comp rx=0 tx=0).stop 2024-05-11T12:12:15.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.394+0000 7f5cef600640 1 -- 172.21.15.167:0/1819623305 shutdown_connections 2024-05-11T12:12:15.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.394+0000 7f5cef600640 1 --2- 172.21.15.167:0/1819623305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ce80af5d0 0x7f5ce8156130 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:15.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.394+0000 7f5cef600640 1 --2- 172.21.15.167:0/1819623305 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ce80af1f0 0x7f5ce81f8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:15.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.394+0000 7f5cef600640 1 --2- 172.21.15.167:0/1819623305 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5cc80762d0 0x7f5cc8078790 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:15.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.394+0000 7f5cef600640 1 --2- 172.21.15.167:0/1819623305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ce806d460 0x7f5ce8150a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:15.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.394+0000 7f5cef600640 1 -- 172.21.15.167:0/1819623305 >> 172.21.15.167:0/1819623305 conn(0x7f5ce80cd8f0 msgr2=0x7f5ce8160a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:15.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.394+0000 7f5cef600640 1 -- 172.21.15.167:0/1819623305 shutdown_connections 2024-05-11T12:12:15.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15.395+0000 7f5cef600640 1 -- 172.21.15.167:0/1819623305 wait complete. 2024-05-11T12:12:15.415 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-11T12:12:15.415 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-11T12:12:15.415 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi167.front.sepia.ceph.com 2024-05-11T12:12:15.415 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-11T12:12:15.415 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-11T12:12:15.415 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-11T12:12:15.416 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-11T12:12:15.416 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-11T12:12:15.416 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-11T12:12:15.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:12:15.416 DEBUG:teuthology.orchestra.run.smithi167:> ip addr 2024-05-11T12:12:15.470 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-11T12:12:15.470 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-11T12:12:15.470 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2024-05-11T12:12:15.471 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:12:15.471 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2024-05-11T12:12:15.471 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:12:15.471 INFO:teuthology.orchestra.run.smithi167.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:12:15.471 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e6 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:12:15.471 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f0 2024-05-11T12:12:15.471 INFO:teuthology.orchestra.run.smithi167.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:12:15.471 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e7 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:12:15.471 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f1 2024-05-11T12:12:15.471 INFO:teuthology.orchestra.run.smithi167.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:12:15.471 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:08 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:12:15.471 INFO:teuthology.orchestra.run.smithi167.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-11T12:12:15.471 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:09 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:12:15.471 INFO:teuthology.orchestra.run.smithi167.stdout: inet 172.21.15.167/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-11T12:12:15.472 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft 42533sec preferred_lft 42533sec 2024-05-11T12:12:15.472 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::ec4:7aff:fe88:7309/64 scope link 2024-05-11T12:12:15.472 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:12:15.472 INFO:teuthology.orchestra.run.smithi167.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-11T12:12:15.472 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 1a:a8:8d:b3:f4:32 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:12:15.472 INFO:teuthology.orchestra.run.smithi167.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-11T12:12:15.472 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:12:15.472 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::b017:48ff:fea0:7183/64 scope link 2024-05-11T12:12:15.472 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:12:15.472 INFO:teuthology.orchestra.run.smithi167.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-11T12:12:15.472 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 1a:a8:8d:b3:f4:32 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-11T12:12:15.472 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::18a8:8dff:feb3:f432/64 scope link 2024-05-11T12:12:15.472 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:12:15.472 INFO:teuthology.orchestra.run.smithi167.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-11T12:12:15.473 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 7e:79:1f:cb:d2:83 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-11T12:12:15.473 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::7c79:1fff:fecb:d283/64 scope link 2024-05-11T12:12:15.473 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:12:15.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:12:15.474 DEBUG:teuthology.orchestra.run.smithi167:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-11T12:12:15.551 INFO:teuthology.orchestra.run.smithi167.stdout:1 2024-05-11T12:12:15.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:12:15.556 DEBUG:teuthology.orchestra.run.smithi167:> route 2024-05-11T12:12:15.607 INFO:teuthology.orchestra.run.smithi167.stdout:Kernel IP routing table 2024-05-11T12:12:15.608 INFO:teuthology.orchestra.run.smithi167.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-11T12:12:15.608 INFO:teuthology.orchestra.run.smithi167.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-11T12:12:15.608 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-11T12:12:15.608 INFO:teuthology.orchestra.run.smithi167.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-11T12:12:15.609 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T12:12:15.609 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-05-11T12:12:15.609 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-11T12:12:15.609 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-11T12:12:15.609 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-11T12:12:15.609 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-05-11T12:12:15.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T12:12:15.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:12:15.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:12:15.821 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2024-05-11T12:12:15.872 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-11T12:12:15.872 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-11T12:12:15.873 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-11T12:12:15.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:12:15.873 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set brx.0 up 2024-05-11T12:12:15.950 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T12:12:15.950 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-11T12:12:16.003 INFO:teuthology.orchestra.run.smithi167.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-11T12:12:16.003 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T12:12:16.003 DEBUG:teuthology.orchestra.run.smithi167:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-11T12:12:16.054 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-05-11T12:12:16.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:12:16.055 DEBUG:teuthology.orchestra.run.smithi167:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:12:16.180 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:12:16.180 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-11T12:12:16.214 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:12:16.215 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-11T12:12:16.215 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.167:3300,172.21.15.204:3300,172.21.15.204:3301" 2024-05-11T12:12:16.215 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: trying mount with new device syntax: 0@5f32d344-7289-4c27-881b-1533419ab678.cephfs=/ 2024-05-11T12:12:16.215 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.167:3300/172.21.15.204:3300/172.21.15.204:3301" will pass to kernel 2024-05-11T12:12:16.217 INFO:teuthology.orchestra.run.smithi167.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-11T12:12:16.217 INFO:teuthology.orchestra.run.smithi167.stdout: You just mounted a file system that supports labels which does not 2024-05-11T12:12:16.217 INFO:teuthology.orchestra.run.smithi167.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-11T12:12:16.217 INFO:teuthology.orchestra.run.smithi167.stdout: applications will generate AVC messages and not be allowed access to 2024-05-11T12:12:16.217 INFO:teuthology.orchestra.run.smithi167.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-11T12:12:16.223 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-11T12:12:16.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:12:16.223 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-11T12:12:16.295 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-11T12:12:16.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-11T12:12:16.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:12:16.420 INFO:teuthology.orchestra.run.smithi167.stdout:client4586 2024-05-11T12:12:16.421 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:12:16.421 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:12:16.475 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json perf dump 2024-05-11T12:12:16.705 INFO:teuthology.orchestra.run.smithi204.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":36,"msgr_send_messages":35,"msgr_recv_bytes":9646,"msgr_send_bytes":22675,"msgr_created_connections":8,"msgr_active_connections":2,"msgr_running_total_time":0.016746639,"msgr_running_send_time":0.002245025,"msgr_running_recv_time":0.007435735,"msgr_running_fast_dispatch_time":0.002966984,"msgr_send_messages_queue_lat":{"avgcount":33,"sum":0.097296872,"avgtime":0.002948390},"msgr_handle_ack_lat":{"avgcount":13,"sum":0.000004249,"avgtime":0.000000326},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"AsyncMessenger::Worker-1":{"msgr_recv_messages":70,"msgr_send_messages":63,"msgr_recv_bytes":124963,"msgr_send_bytes":17153,"msgr_created_connections":5,"msgr_active_connections":4,"msgr_running_total_time":0.020942360,"msgr_running_send_time":0.002807877,"msgr_running_recv_time":0.024174955,"msgr_running_fast_dispatch_time":0.000683061,"msgr_send_messages_queue_lat":{"avgcount":63,"sum":0.066166317,"avgtime":0.001050259},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":122144,"msgr_send_encrypted_bytes":12368},"AsyncMessenger::Worker-2":{"msgr_recv_messages":50,"msgr_send_messages":54,"msgr_recv_bytes":14224,"msgr_send_bytes":141844,"msgr_created_connections":9,"msgr_active_connections":4,"msgr_running_total_time":0.029864859,"msgr_running_send_time":0.005680385,"msgr_running_recv_time":0.023408808,"msgr_running_fast_dispatch_time":0.001499946,"msgr_send_messages_queue_lat":{"avgcount":53,"sum":0.079682606,"avgtime":0.001503445},"msgr_handle_ack_lat":{"avgcount":67,"sum":0.000025752,"avgtime":0.000000384},"msgr_recv_encrypted_bytes":1728,"msgr_send_encrypted_bytes":98080},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":56,"sum":0.003594572,"avgtime":0.000064188}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":2,"sum":0.000024224,"avgtime":0.000012112}},"mds":{"request":33,"reply":33,"reply_latency":{"avgcount":33,"sum":0.013944159,"avgtime":0.000422550},"slow_reply":0,"forward":0,"dir_fetch_complete":0,"dir_fetch_keys":0,"dir_commit":12,"dir_split":0,"dir_merge":0,"inodes":11,"inodes_top":0,"inodes_bottom":11,"inodes_pin_tail":0,"inodes_pinned":10,"inodes_expired":0,"inodes_with_caps":1,"caps":1,"subtrees":2,"traverse":62,"traverse_hit":59,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":0,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":0,"handle_client_caps_dirty":0,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_quiesce_path":0,"ireq_quiesce_inode":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evlrg":0,"evadd":9,"evex":0,"evtrm":0,"ev":9,"evexg":0,"evexd":0,"segadd":2,"segex":0,"segtrm":0,"seg":2,"segmjr":2,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4197287,"rdpos":0,"jlat":{"avgcount":6,"sum":0.024966791,"avgtime":0.004161131},"replayed":0},"mds_mem":{"ino":13,"ino+":13,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":11,"dn+":11,"dn-":0,"cap":1,"cap+":3,"cap-":2,"rss":41472,"heap":231788},"mds_server":{"dispatch_client_request":33,"dispatch_server_request":0,"handle_client_request":33,"handle_client_session":7,"handle_peer_request":0,"req_create_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_getattr_latency":{"avgcount":29,"sum":0.012473740,"avgtime":0.000430128},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":3,"sum":0.000816757,"avgtime":0.000272252},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":1,"sum":0.000653662,"avgtime":0.000653662},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":1,"session_add":3,"session_remove":2,"sessions_open":1,"sessions_stale":0,"total_load":31,"average_load":31,"avg_session_uptime":0,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":224915,"buffer_anon_items":81,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5424,"osdmap_items":76,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":53064,"mds_co_items":216,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":45,"op_send_bytes":14193,"op_resend":37,"op_reply":45,"op_latency":{"avgcount":45,"sum":0.831989627,"avgtime":0.018488658},"op_inflight":0,"oplen_avg":{"avgcount":45,"sum":69},"op":45,"op_r":0,"op_w":45,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":11,"osdop_read":0,"osdop_write":6,"osdop_writefull":8,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":22,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":41,"map_full":0,"map_inc":1,"osd_sessions":8,"osd_session_open":8,"osd_session_close":0,"osd_laggy":0,"omap_wr":15,"omap_rd":0,"omap_del":1},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":0,"omap_total_updates":0,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":151,"get_sum":130778,"get_or_fail_fail":0,"get_or_fail_success":151,"take":0,"take_sum":0,"put":151,"put_sum":130778,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":45,"take_sum":14193,"put":45,"put_sum":14193,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":45,"take_sum":45,"put":45,"put_sum":45,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x55747bba4500":{"val":0,"max":3758096384,"get_started":0,"get":9,"get_sum":2983,"get_or_fail_fail":0,"get_or_fail_success":9,"take":0,"take_sum":0,"put":6,"put_sum":2983,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-05-11T12:12:16.706 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 2, "msgr_created_connections": 8, "msgr_handle_ack_lat": { "avgcount": 13, "avgtime": 3.26e-07, "sum": 4.249e-06 }, "msgr_recv_bytes": 9646, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 36, "msgr_running_fast_dispatch_time": 0.002966984, "msgr_running_recv_time": 0.007435735, "msgr_running_send_time": 0.002245025, "msgr_running_total_time": 0.016746639, "msgr_send_bytes": 22675, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 35, "msgr_send_messages_queue_lat": { "avgcount": 33, "avgtime": 0.00294839, "sum": 0.097296872 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 4, "msgr_created_connections": 5, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 124963, "msgr_recv_encrypted_bytes": 122144, "msgr_recv_messages": 70, "msgr_running_fast_dispatch_time": 0.000683061, "msgr_running_recv_time": 0.024174955, "msgr_running_send_time": 0.002807877, "msgr_running_total_time": 0.02094236, "msgr_send_bytes": 17153, "msgr_send_encrypted_bytes": 12368, "msgr_send_messages": 63, "msgr_send_messages_queue_lat": { "avgcount": 63, "avgtime": 0.001050259, "sum": 0.066166317 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 4, "msgr_created_connections": 9, "msgr_handle_ack_lat": { "avgcount": 67, "avgtime": 3.84e-07, "sum": 2.5752e-05 }, "msgr_recv_bytes": 14224, "msgr_recv_encrypted_bytes": 1728, "msgr_recv_messages": 50, "msgr_running_fast_dispatch_time": 0.001499946, "msgr_running_recv_time": 0.023408808, "msgr_running_send_time": 0.005680385, "msgr_running_total_time": 0.029864859, "msgr_send_bytes": 141844, "msgr_send_encrypted_bytes": 98080, "msgr_send_messages": 54, "msgr_send_messages_queue_lat": { "avgcount": 53, "avgtime": 0.001503445, "sum": 0.079682606 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 56, "avgtime": 6.4188e-05, "sum": 0.003594572 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 2, "avgtime": 1.2112e-05, "sum": 2.4224e-05 }, "queue_len": 0 }, "mds": { "caps": 1, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 12, "dir_fetch_complete": 0, "dir_fetch_keys": 0, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 0, "handle_client_caps_dirty": 0, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 11, "inodes_bottom": 11, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 10, "inodes_top": 0, "inodes_with_caps": 1, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 33, "reply_latency": { "avgcount": 33, "avgtime": 0.00042255, "sum": 0.013944159 }, "request": 33, "root_rbytes": 0, "root_rfiles": 0, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 62, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 59, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "ireq_quiesce_inode": 0, "ireq_quiesce_path": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 9, "evadd": 9, "evex": 0, "evexd": 0, "evexg": 0, "evlrg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 6, "avgtime": 0.004161131, "sum": 0.024966791 }, "rdpos": 0, "replayed": 0, "seg": 2, "segadd": 2, "segex": 0, "segexd": 0, "segexg": 0, "segmjr": 2, "segtrm": 0, "wrpos": 4197287 }, "mds_mem": { "cap": 1, "cap+": 3, "cap-": 2, "dir": 12, "dir+": 12, "dir-": 0, "dn": 11, "dn+": 11, "dn-": 0, "heap": 231788, "ino": 13, "ino+": 13, "ino-": 0, "rss": 41472 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 33, "dispatch_server_request": 0, "handle_client_request": 33, "handle_client_session": 7, "handle_peer_request": 0, "req_create_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getattr_latency": { "avgcount": 29, "avgtime": 0.000430128, "sum": 0.01247374 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 3, "avgtime": 0.000272252, "sum": 0.000816757 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 1, "avgtime": 0.000653662, "sum": 0.000653662 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 31, "avg_session_uptime": 0, "mdthresh_evicted": 0, "session_add": 3, "session_count": 1, "session_remove": 2, "sessions_open": 1, "sessions_stale": 0, "total_load": 31 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 224915, "buffer_anon_items": 81, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 53064, "mds_co_items": 216, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5424, "osdmap_items": 76, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 41, "map_full": 0, "map_inc": 1, "omap_del": 1, "omap_rd": 0, "omap_wr": 15, "op": 45, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 45, "avgtime": 0.018488658, "sum": 0.831989627 }, "op_pg": 0, "op_r": 0, "op_reply": 45, "op_resend": 37, "op_rmw": 0, "op_send": 45, "op_send_bytes": 14193, "op_w": 45, "oplen_avg": { "avgcount": 45, "sum": 69 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 8, "osd_sessions": 8, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 11, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 22, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 6, "osdop_writefull": 8, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 0, "omap_total_objs": 1, "omap_total_removes": 0, "omap_total_updates": 0 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 151, "get_or_fail_fail": 0, "get_or_fail_success": 151, "get_started": 0, "get_sum": 130778, "max": 104857600, "put": 151, "put_sum": 130778, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 45, "put_sum": 14193, "take": 45, "take_sum": 14193, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 45, "put_sum": 45, "take": 45, "take_sum": 45, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x55747bba4500": { "get": 9, "get_or_fail_fail": 0, "get_or_fail_success": 9, "get_started": 0, "get_sum": 2983, "max": 3758096384, "put": 6, "put_sum": 2983, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-05-11T12:12:16.706 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:12:16.706 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:12:16.764 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:12:16.765 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:12:16.816 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-11T12:12:17.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.014+0000 7f534d800640 1 Processor -- start 2024-05-11T12:12:17.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.015+0000 7f534d800640 1 -- start start 2024-05-11T12:12:17.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.016+0000 7f534d800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f53481611f0 0x7f53480cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:17.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.016+0000 7f534d800640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53480d0220 con 0x7f5348140010 2024-05-11T12:12:17.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.016+0000 7f534d800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53480d0500 con 0x7f534813fc30 2024-05-11T12:12:17.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.016+0000 7f534d800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f53481619d0 con 0x7f53481611f0 2024-05-11T12:12:17.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.017+0000 7f534ce00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f5348140010 0x7f534806eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:50116/0 (socket says 172.21.15.167:50116) 2024-05-11T12:12:17.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.017+0000 7f534ce00640 1 -- 172.21.15.167:0/589351339 learned_addr learned my addr 172.21.15.167:0/589351339 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:17.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.017+0000 7f5346a00640 1 -- 172.21.15.167:0/589351339 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4002333574 0 0) 0x7f53480d0220 con 0x7f5348140010 2024-05-11T12:12:17.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.017+0000 7f5346a00640 1 -- 172.21.15.167:0/589351339 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f532c003520 con 0x7f5348140010 2024-05-11T12:12:17.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.017+0000 7f5347400640 1 --2- 172.21.15.167:0/589351339 >> v2:172.21.15.204:3300/0 conn(0x7f53481611f0 0x7f53480cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:17.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.017+0000 7f5346a00640 1 -- 172.21.15.167:0/589351339 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 285221389 0 0) 0x7f532c003520 con 0x7f5348140010 2024-05-11T12:12:17.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.017+0000 7f5346a00640 1 -- 172.21.15.167:0/589351339 >> v2:172.21.15.204:3300/0 conn(0x7f53481611f0 msgr2=0x7f53480cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:17.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.017+0000 7f5346a00640 1 --2- 172.21.15.167:0/589351339 >> v2:172.21.15.204:3300/0 conn(0x7f53481611f0 0x7f53480cf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:17.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.017+0000 7f5346a00640 1 -- 172.21.15.167:0/589351339 >> v1:172.21.15.204:6789/0 conn(0x7f534813fc30 legacy=0x7f5348160ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:17.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.017+0000 7f5346a00640 1 -- 172.21.15.167:0/589351339 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53480d07e0 con 0x7f5348140010 2024-05-11T12:12:17.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.017+0000 7f5346a00640 1 -- 172.21.15.167:0/589351339 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f5340001a70 con 0x7f5348140010 2024-05-11T12:12:17.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.017+0000 7f5346a00640 1 -- 172.21.15.167:0/589351339 >> v1:172.21.15.167:6789/0 conn(0x7f5348140010 legacy=0x7f534806eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:17.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.018+0000 7f5346a00640 1 --2- 172.21.15.167:0/589351339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f532c003ac0 0x7f532c003ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:17.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.018+0000 7f5346a00640 1 --2- 172.21.15.167:0/589351339 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f532c004800 0x7f532c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:17.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.018+0000 7f5346a00640 1 --2- 172.21.15.167:0/589351339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f532c005370 0x7f532c00fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:17.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.018+0000 7f5346a00640 1 -- 172.21.15.167:0/589351339 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f53481619d0 con 0x7f532c003ac0 2024-05-11T12:12:17.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.018+0000 7f5346a00640 1 -- 172.21.15.167:0/589351339 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f532c0106d0 con 0x7f532c005370 2024-05-11T12:12:17.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.018+0000 7f5346a00640 1 -- 172.21.15.167:0/589351339 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f532c0108d0 con 0x7f532c004800 2024-05-11T12:12:17.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.018+0000 7f5347e00640 1 --2- 172.21.15.167:0/589351339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f532c003ac0 0x7f532c003ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:17.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.018+0000 7f534ce00640 1 --2- 172.21.15.167:0/589351339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f532c005370 0x7f532c00fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:17.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.018+0000 7f5347400640 1 --2- 172.21.15.167:0/589351339 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f532c004800 0x7f532c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:17.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.018+0000 7f5347e00640 1 -- 172.21.15.167:0/589351339 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f532c004800 msgr2=0x7f532c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:17.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.018+0000 7f5347e00640 1 --2- 172.21.15.167:0/589351339 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f532c004800 0x7f532c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:17.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.018+0000 7f5347e00640 1 -- 172.21.15.167:0/589351339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f532c005370 msgr2=0x7f532c00fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:17.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.018+0000 7f5347e00640 1 --2- 172.21.15.167:0/589351339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f532c005370 0x7f532c00fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:17.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.018+0000 7f534ce00640 1 -- 172.21.15.167:0/589351339 reap_dead start 2024-05-11T12:12:17.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.018+0000 7f5347e00640 1 -- 172.21.15.167:0/589351339 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f532c003520 con 0x7f532c003ac0 2024-05-11T12:12:17.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.018+0000 7f5347400640 1 --2- 172.21.15.167:0/589351339 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f532c004800 0x7f532c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:17.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.019+0000 7f5347e00640 1 --2- 172.21.15.167:0/589351339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f532c003ac0 0x7f532c003ee0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5330003db0 tx=0x7f532c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de1106686736c7fb server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:17.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.020+0000 7f5346a00640 1 -- 172.21.15.167:0/589351339 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5330037070 con 0x7f532c003ac0 2024-05-11T12:12:17.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.020+0000 7f5346a00640 1 -- 172.21.15.167:0/589351339 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f533002a9f0 con 0x7f532c003ac0 2024-05-11T12:12:17.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.020+0000 7f5346a00640 1 -- 172.21.15.167:0/589351339 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5330003980 con 0x7f532c003ac0 2024-05-11T12:12:17.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.020+0000 7f534d800640 1 -- 172.21.15.167:0/589351339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f532c003ac0 msgr2=0x7f532c003ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:17.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.020+0000 7f534d800640 1 --2- 172.21.15.167:0/589351339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f532c003ac0 0x7f532c003ee0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5330003db0 tx=0x7f532c005830 comp rx=0 tx=0).stop 2024-05-11T12:12:17.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.020+0000 7f534d800640 1 -- 172.21.15.167:0/589351339 shutdown_connections 2024-05-11T12:12:17.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.020+0000 7f534d800640 1 --2- 172.21.15.167:0/589351339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f532c003ac0 0x7f532c003ee0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:17.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.020+0000 7f534d800640 1 -- 172.21.15.167:0/589351339 >> 172.21.15.167:0/589351339 conn(0x7f534806c450 msgr2=0x7f534806c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:17.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.021+0000 7f534d800640 1 -- 172.21.15.167:0/589351339 shutdown_connections 2024-05-11T12:12:17.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.021+0000 7f534d800640 1 -- 172.21.15.167:0/589351339 wait complete. 2024-05-11T12:12:17.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.023+0000 7f534d800640 1 Processor -- start 2024-05-11T12:12:17.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.023+0000 7f534d800640 1 -- start start 2024-05-11T12:12:17.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.024+0000 7f534d800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f532c003ac0 0x7f53480b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:17.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.024+0000 7f534d800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f53481f1ac0 0x7f534813afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:17.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.024+0000 7f534d800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f534815f580 0x7f534815fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:17.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.024+0000 7f534d800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5348160ae0 con 0x7f532c003ac0 2024-05-11T12:12:17.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.024+0000 7f534d800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f53480d0ac0 con 0x7f534815f580 2024-05-11T12:12:17.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.024+0000 7f534d800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5348160c60 con 0x7f53481f1ac0 2024-05-11T12:12:17.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.024+0000 7f5347e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f532c003ac0 0x7f53480b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:17.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.024+0000 7f5347e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f532c003ac0 0x7f53480b47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59902/0 (socket says 172.21.15.167:59902) 2024-05-11T12:12:17.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.024+0000 7f5347e00640 1 -- 172.21.15.167:0/2741221685 learned_addr learned my addr 172.21.15.167:0/2741221685 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:17.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.025+0000 7f5347e00640 1 -- 172.21.15.167:0/2741221685 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f53481f1ac0 msgr2=0x7f534813afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:17.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.025+0000 7f5347400640 1 --2- 172.21.15.167:0/2741221685 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f53481f1ac0 0x7f534813afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:17.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.025+0000 7f5347e00640 1 --2- 172.21.15.167:0/2741221685 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f53481f1ac0 0x7f534813afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:17.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.025+0000 7f5347e00640 1 -- 172.21.15.167:0/2741221685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f534815f580 msgr2=0x7f534815fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:17.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.025+0000 7f534ce00640 1 --2- 172.21.15.167:0/2741221685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f534815f580 0x7f534815fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:17.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.025+0000 7f5347e00640 1 --2- 172.21.15.167:0/2741221685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f534815f580 0x7f534815fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:17.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.025+0000 7f5347e00640 1 -- 172.21.15.167:0/2741221685 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53480b4e60 con 0x7f532c003ac0 2024-05-11T12:12:17.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.025+0000 7f5347e00640 1 --2- 172.21.15.167:0/2741221685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f532c003ac0 0x7f53480b47c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5330003db0 tx=0x7f53300036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:17.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.025+0000 7f533fe00640 1 -- 172.21.15.167:0/2741221685 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5330037050 con 0x7f532c003ac0 2024-05-11T12:12:17.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.025+0000 7f533fe00640 1 -- 172.21.15.167:0/2741221685 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5330038040 con 0x7f532c003ac0 2024-05-11T12:12:17.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.025+0000 7f533fe00640 1 -- 172.21.15.167:0/2741221685 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5330006e50 con 0x7f532c003ac0 2024-05-11T12:12:17.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.025+0000 7f534d800640 1 -- 172.21.15.167:0/2741221685 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53480b5140 con 0x7f532c003ac0 2024-05-11T12:12:17.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.025+0000 7f534d800640 1 -- 172.21.15.167:0/2741221685 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53480b5700 con 0x7f532c003ac0 2024-05-11T12:12:17.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.027+0000 7f534d800640 1 -- 172.21.15.167:0/2741221685 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5348076390 con 0x7f532c003ac0 2024-05-11T12:12:17.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.028+0000 7f533fe00640 1 -- 172.21.15.167:0/2741221685 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5330045020 con 0x7f532c003ac0 2024-05-11T12:12:17.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.029+0000 7f533fe00640 1 --2- 172.21.15.167:0/2741221685 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f53240764a0 0x7f5324078960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:17.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.029+0000 7f533fe00640 1 -- 172.21.15.167:0/2741221685 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f53300b7520 con 0x7f532c003ac0 2024-05-11T12:12:17.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.029+0000 7f5347400640 1 --2- 172.21.15.167:0/2741221685 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f53240764a0 0x7f5324078960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:17.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.030+0000 7f5347400640 1 --2- 172.21.15.167:0/2741221685 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f53240764a0 0x7f5324078960 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f534815cd90 tx=0x7f5338008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:17.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.033+0000 7f533fe00640 1 -- 172.21.15.167:0/2741221685 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5348076390 con 0x7f532c003ac0 2024-05-11T12:12:17.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.261+0000 7f534d800640 1 -- 172.21.15.167:0/2741221685 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f53480694b0 con 0x7f532c003ac0 2024-05-11T12:12:17.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.261+0000 7f533fe00640 1 -- 172.21.15.167:0/2741221685 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v41) ==== 80+0+0 (secure 0 0 0) 0x7f5330080c50 con 0x7f532c003ac0 2024-05-11T12:12:17.262 INFO:teuthology.orchestra.run.smithi167.stderr:listed 0 entries 2024-05-11T12:12:17.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.264+0000 7f534d800640 1 -- 172.21.15.167:0/2741221685 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f53240764a0 msgr2=0x7f5324078960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:17.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.264+0000 7f534d800640 1 --2- 172.21.15.167:0/2741221685 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f53240764a0 0x7f5324078960 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f534815cd90 tx=0x7f5338008040 comp rx=0 tx=0).stop 2024-05-11T12:12:17.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.264+0000 7f534d800640 1 -- 172.21.15.167:0/2741221685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f532c003ac0 msgr2=0x7f53480b47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:17.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.264+0000 7f534d800640 1 --2- 172.21.15.167:0/2741221685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f532c003ac0 0x7f53480b47c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5330003db0 tx=0x7f53300036f0 comp rx=0 tx=0).stop 2024-05-11T12:12:17.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.264+0000 7f534d800640 1 -- 172.21.15.167:0/2741221685 shutdown_connections 2024-05-11T12:12:17.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.264+0000 7f534d800640 1 --2- 172.21.15.167:0/2741221685 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f53240764a0 0x7f5324078960 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:17.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.264+0000 7f534d800640 1 --2- 172.21.15.167:0/2741221685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f534815f580 0x7f534815fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:17.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.264+0000 7f534d800640 1 --2- 172.21.15.167:0/2741221685 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f53481f1ac0 0x7f534813afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:17.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.264+0000 7f534d800640 1 --2- 172.21.15.167:0/2741221685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f532c003ac0 0x7f53480b47c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:17.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.264+0000 7f534d800640 1 -- 172.21.15.167:0/2741221685 >> 172.21.15.167:0/2741221685 conn(0x7f534806c450 msgr2=0x7f534813c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:17.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.264+0000 7f534d800640 1 -- 172.21.15.167:0/2741221685 shutdown_connections 2024-05-11T12:12:17.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:17.264+0000 7f534d800640 1 -- 172.21.15.167:0/2741221685 wait complete. 2024-05-11T12:12:17.283 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-11T12:12:17.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:12:17.283 DEBUG:teuthology.orchestra.run.smithi167:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-11T12:12:17.368 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2024-05-11T12:12:17.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:12:17.368 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-11T12:12:17.418 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json perf dump 2024-05-11T12:12:17.650 INFO:teuthology.orchestra.run.smithi204.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":37,"msgr_send_messages":36,"msgr_recv_bytes":9892,"msgr_send_bytes":23134,"msgr_created_connections":8,"msgr_active_connections":2,"msgr_running_total_time":0.016987029,"msgr_running_send_time":0.002318418,"msgr_running_recv_time":0.007484619,"msgr_running_fast_dispatch_time":0.003025856,"msgr_send_messages_queue_lat":{"avgcount":34,"sum":0.097414961,"avgtime":0.002865145},"msgr_handle_ack_lat":{"avgcount":13,"sum":0.000004249,"avgtime":0.000000326},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"AsyncMessenger::Worker-1":{"msgr_recv_messages":71,"msgr_send_messages":64,"msgr_recv_bytes":125235,"msgr_send_bytes":17793,"msgr_created_connections":5,"msgr_active_connections":4,"msgr_running_total_time":0.021083912,"msgr_running_send_time":0.002862535,"msgr_running_recv_time":0.024215365,"msgr_running_fast_dispatch_time":0.000717340,"msgr_send_messages_queue_lat":{"avgcount":64,"sum":0.066226392,"avgtime":0.001034787},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":122416,"msgr_send_encrypted_bytes":13008},"AsyncMessenger::Worker-2":{"msgr_recv_messages":52,"msgr_send_messages":55,"msgr_recv_bytes":14460,"msgr_send_bytes":141962,"msgr_created_connections":9,"msgr_active_connections":3,"msgr_running_total_time":0.030333470,"msgr_running_send_time":0.005850827,"msgr_running_recv_time":0.023656037,"msgr_running_fast_dispatch_time":0.001499946,"msgr_send_messages_queue_lat":{"avgcount":54,"sum":0.079750772,"avgtime":0.001476866},"msgr_handle_ack_lat":{"avgcount":69,"sum":0.000026125,"avgtime":0.000000378},"msgr_recv_encrypted_bytes":1728,"msgr_send_encrypted_bytes":98080},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":58,"sum":0.003762757,"avgtime":0.000064875}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":2,"sum":0.000024224,"avgtime":0.000012112}},"mds":{"request":33,"reply":33,"reply_latency":{"avgcount":33,"sum":0.013944159,"avgtime":0.000422550},"slow_reply":0,"forward":0,"dir_fetch_complete":0,"dir_fetch_keys":0,"dir_commit":12,"dir_split":0,"dir_merge":0,"inodes":11,"inodes_top":0,"inodes_bottom":11,"inodes_pin_tail":0,"inodes_pinned":10,"inodes_expired":0,"inodes_with_caps":1,"caps":1,"subtrees":2,"traverse":62,"traverse_hit":59,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":0,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":0,"handle_client_caps_dirty":0,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_quiesce_path":0,"ireq_quiesce_inode":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evlrg":0,"evadd":10,"evex":0,"evtrm":0,"ev":10,"evexg":0,"evexd":0,"segadd":2,"segex":0,"segtrm":0,"seg":2,"segmjr":2,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4197428,"rdpos":0,"jlat":{"avgcount":7,"sum":0.028963076,"avgtime":0.004137582},"replayed":0},"mds_mem":{"ino":13,"ino+":13,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":11,"dn+":11,"dn-":0,"cap":0,"cap+":3,"cap-":3,"rss":41600,"heap":231788},"mds_server":{"dispatch_client_request":33,"dispatch_server_request":0,"handle_client_request":33,"handle_client_session":9,"handle_peer_request":0,"req_create_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_getattr_latency":{"avgcount":29,"sum":0.012473740,"avgtime":0.000430128},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":3,"sum":0.000816757,"avgtime":0.000272252},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":1,"sum":0.000653662,"avgtime":0.000653662},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":0,"session_add":3,"session_remove":3,"sessions_open":0,"sessions_stale":0,"total_load":31,"average_load":31,"avg_session_uptime":0,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":224915,"buffer_anon_items":81,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5424,"osdmap_items":76,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":53064,"mds_co_items":216,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":46,"op_send_bytes":14334,"op_resend":37,"op_reply":46,"op_latency":{"avgcount":46,"sum":0.835989577,"avgtime":0.018173686},"op_inflight":0,"oplen_avg":{"avgcount":46,"sum":70},"op":46,"op_r":0,"op_w":46,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":11,"osdop_read":0,"osdop_write":7,"osdop_writefull":8,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":22,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":41,"map_full":0,"map_inc":1,"osd_sessions":8,"osd_session_open":8,"osd_session_close":0,"osd_laggy":0,"omap_wr":15,"omap_rd":0,"omap_del":1},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":0,"omap_total_updates":0,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":155,"get_sum":131120,"get_or_fail_fail":0,"get_or_fail_success":155,"take":0,"take_sum":0,"put":155,"put_sum":131120,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":46,"take_sum":14334,"put":46,"put_sum":14334,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":46,"take_sum":46,"put":46,"put_sum":46,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x55747bba4500":{"val":0,"max":3758096384,"get_started":0,"get":10,"get_sum":3124,"get_or_fail_fail":0,"get_or_fail_success":10,"take":0,"take_sum":0,"put":7,"put_sum":3124,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-05-11T12:12:17.651 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 2, "msgr_created_connections": 8, "msgr_handle_ack_lat": { "avgcount": 13, "avgtime": 3.26e-07, "sum": 4.249e-06 }, "msgr_recv_bytes": 9892, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 37, "msgr_running_fast_dispatch_time": 0.003025856, "msgr_running_recv_time": 0.007484619, "msgr_running_send_time": 0.002318418, "msgr_running_total_time": 0.016987029, "msgr_send_bytes": 23134, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 36, "msgr_send_messages_queue_lat": { "avgcount": 34, "avgtime": 0.002865145, "sum": 0.097414961 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 4, "msgr_created_connections": 5, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 125235, "msgr_recv_encrypted_bytes": 122416, "msgr_recv_messages": 71, "msgr_running_fast_dispatch_time": 0.00071734, "msgr_running_recv_time": 0.024215365, "msgr_running_send_time": 0.002862535, "msgr_running_total_time": 0.021083912, "msgr_send_bytes": 17793, "msgr_send_encrypted_bytes": 13008, "msgr_send_messages": 64, "msgr_send_messages_queue_lat": { "avgcount": 64, "avgtime": 0.001034787, "sum": 0.066226392 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 3, "msgr_created_connections": 9, "msgr_handle_ack_lat": { "avgcount": 69, "avgtime": 3.78e-07, "sum": 2.6125e-05 }, "msgr_recv_bytes": 14460, "msgr_recv_encrypted_bytes": 1728, "msgr_recv_messages": 52, "msgr_running_fast_dispatch_time": 0.001499946, "msgr_running_recv_time": 0.023656037, "msgr_running_send_time": 0.005850827, "msgr_running_total_time": 0.03033347, "msgr_send_bytes": 141962, "msgr_send_encrypted_bytes": 98080, "msgr_send_messages": 55, "msgr_send_messages_queue_lat": { "avgcount": 54, "avgtime": 0.001476866, "sum": 0.079750772 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 58, "avgtime": 6.4875e-05, "sum": 0.003762757 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 2, "avgtime": 1.2112e-05, "sum": 2.4224e-05 }, "queue_len": 0 }, "mds": { "caps": 1, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 12, "dir_fetch_complete": 0, "dir_fetch_keys": 0, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 0, "handle_client_caps_dirty": 0, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 11, "inodes_bottom": 11, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 10, "inodes_top": 0, "inodes_with_caps": 1, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 33, "reply_latency": { "avgcount": 33, "avgtime": 0.00042255, "sum": 0.013944159 }, "request": 33, "root_rbytes": 0, "root_rfiles": 0, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 62, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 59, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "ireq_quiesce_inode": 0, "ireq_quiesce_path": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 10, "evadd": 10, "evex": 0, "evexd": 0, "evexg": 0, "evlrg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 7, "avgtime": 0.004137582, "sum": 0.028963076 }, "rdpos": 0, "replayed": 0, "seg": 2, "segadd": 2, "segex": 0, "segexd": 0, "segexg": 0, "segmjr": 2, "segtrm": 0, "wrpos": 4197428 }, "mds_mem": { "cap": 0, "cap+": 3, "cap-": 3, "dir": 12, "dir+": 12, "dir-": 0, "dn": 11, "dn+": 11, "dn-": 0, "heap": 231788, "ino": 13, "ino+": 13, "ino-": 0, "rss": 41600 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 33, "dispatch_server_request": 0, "handle_client_request": 33, "handle_client_session": 9, "handle_peer_request": 0, "req_create_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getattr_latency": { "avgcount": 29, "avgtime": 0.000430128, "sum": 0.01247374 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 3, "avgtime": 0.000272252, "sum": 0.000816757 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 1, "avgtime": 0.000653662, "sum": 0.000653662 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 31, "avg_session_uptime": 0, "mdthresh_evicted": 0, "session_add": 3, "session_count": 0, "session_remove": 3, "sessions_open": 0, "sessions_stale": 0, "total_load": 31 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 224915, "buffer_anon_items": 81, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 53064, "mds_co_items": 216, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5424, "osdmap_items": 76, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 41, "map_full": 0, "map_inc": 1, "omap_del": 1, "omap_rd": 0, "omap_wr": 15, "op": 46, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 46, "avgtime": 0.018173686, "sum": 0.835989577 }, "op_pg": 0, "op_r": 0, "op_reply": 46, "op_resend": 37, "op_rmw": 0, "op_send": 46, "op_send_bytes": 14334, "op_w": 46, "oplen_avg": { "avgcount": 46, "sum": 70 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 8, "osd_sessions": 8, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 11, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 22, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 7, "osdop_writefull": 8, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 0, "omap_total_objs": 1, "omap_total_removes": 0, "omap_total_updates": 0 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 155, "get_or_fail_fail": 0, "get_or_fail_success": 155, "get_started": 0, "get_sum": 131120, "max": 104857600, "put": 155, "put_sum": 131120, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 46, "put_sum": 14334, "take": 46, "take_sum": 14334, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 46, "put_sum": 46, "take": 46, "take_sum": 46, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x55747bba4500": { "get": 10, "get_or_fail_fail": 0, "get_or_fail_success": 10, "get_started": 0, "get_sum": 3124, "max": 3758096384, "put": 7, "put_sum": 3124, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-05-11T12:12:17.651 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-11T12:12:17.652 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-11T12:12:17.652 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-11T12:12:17.753 DEBUG:teuthology.orchestra.run.smithi204:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-11T12:12:17.854 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-11T12:12:17.854 INFO:tasks.cephfs.mount:Terminating background process 2024-05-11T12:12:17.855 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:12:17.855 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:12:17.910 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-11T12:12:17.910 INFO:tasks.cephfs.mount:Terminating background process 2024-05-11T12:12:17.910 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:12:17.910 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:12:17.967 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052713,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-11T12:12:17.967 INFO:tasks.cephfs.mount:Terminating background process 2024-05-11T12:12:17.967 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:12:17.967 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:12:18.025 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-11T12:12:18.025 INFO:tasks.cephfs.mount:Terminating background process 2024-05-11T12:12:18.025 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:12:18.025 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:12:18.082 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052713,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-11T12:12:18.082 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:12:18.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.290+0000 7f7720c00640 1 Processor -- start 2024-05-11T12:12:18.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.290+0000 7f7720c00640 1 -- start start 2024-05-11T12:12:18.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.292+0000 7f7720c00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f771c06d460 0x7f771c06d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:18.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.292+0000 7f7720c00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f771c15df30 con 0x7f771c157750 2024-05-11T12:12:18.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.292+0000 7f7720c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f771c15e210 con 0x7f771c0b3ea0 2024-05-11T12:12:18.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.292+0000 7f7720c00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f771c06ddf0 con 0x7f771c06d460 2024-05-11T12:12:18.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.292+0000 7f771a800640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f771c06d460 0x7f771c06d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:18.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.293+0000 7f7719e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f771c157750 0x7f771c15d740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:50126/0 (socket says 172.21.15.167:50126) 2024-05-11T12:12:18.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.293+0000 7f7719e00640 1 -- 172.21.15.167:0/52004404 learned_addr learned my addr 172.21.15.167:0/52004404 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:18.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.293+0000 7f771a800640 1 -- 172.21.15.167:0/52004404 >> v1:172.21.15.204:6789/0 conn(0x7f771c0b3ea0 legacy=0x7f771c161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:18.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.293+0000 7f771a800640 1 -- 172.21.15.167:0/52004404 >> v1:172.21.15.167:6789/0 conn(0x7f771c157750 legacy=0x7f771c15d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:18.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.293+0000 7f771a800640 1 -- 172.21.15.167:0/52004404 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f771c15e4f0 con 0x7f771c06d460 2024-05-11T12:12:18.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.294+0000 7f771a800640 1 --2- 172.21.15.167:0/52004404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f771c06d460 0x7f771c06d8b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7714009d90 tx=0x7f771402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aff26c52e9848aa9 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:18.295 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.294+0000 7f7719400640 1 -- 172.21.15.167:0/52004404 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7714007740 con 0x7f771c06d460 2024-05-11T12:12:18.295 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.294+0000 7f7719400640 1 -- 172.21.15.167:0/52004404 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77140078e0 con 0x7f771c06d460 2024-05-11T12:12:18.295 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.294+0000 7f7719400640 1 -- 172.21.15.167:0/52004404 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7714007cb0 con 0x7f771c06d460 2024-05-11T12:12:18.295 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.295+0000 7f7720c00640 1 -- 172.21.15.167:0/52004404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f771c06d460 msgr2=0x7f771c06d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:18.295 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.295+0000 7f7720c00640 1 --2- 172.21.15.167:0/52004404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f771c06d460 0x7f771c06d8b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7714009d90 tx=0x7f771402a660 comp rx=0 tx=0).stop 2024-05-11T12:12:18.296 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.295+0000 7f7720c00640 1 -- 172.21.15.167:0/52004404 shutdown_connections 2024-05-11T12:12:18.296 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.295+0000 7f7720c00640 1 --2- 172.21.15.167:0/52004404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f771c06d460 0x7f771c06d8b0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:18.296 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.295+0000 7f7720c00640 1 -- 172.21.15.167:0/52004404 >> 172.21.15.167:0/52004404 conn(0x7f771c0cd8f0 msgr2=0x7f771c0cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:18.296 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.295+0000 7f7720c00640 1 -- 172.21.15.167:0/52004404 shutdown_connections 2024-05-11T12:12:18.296 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.295+0000 7f7720c00640 1 -- 172.21.15.167:0/52004404 wait complete. 2024-05-11T12:12:18.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.297+0000 7f7720c00640 1 Processor -- start 2024-05-11T12:12:18.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.297+0000 7f7720c00640 1 -- start start 2024-05-11T12:12:18.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.298+0000 7f7720c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f771c06d460 0x7f771c1428e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:18.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.298+0000 7f7720c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f771c0b3ea0 0x7f771c1f5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:18.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.298+0000 7f7720c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f771c157750 0x7f771c1f7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:18.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.298+0000 7f7720c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f771c161ba0 con 0x7f771c0b3ea0 2024-05-11T12:12:18.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.298+0000 7f7720c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f771c161000 con 0x7f771c06d460 2024-05-11T12:12:18.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.298+0000 7f7720c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f771c0cf1c0 con 0x7f771c157750 2024-05-11T12:12:18.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.298+0000 7f7711400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f771c0b3ea0 0x7f771c1f5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:18.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.298+0000 7f7711400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f771c0b3ea0 0x7f771c1f5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59920/0 (socket says 172.21.15.167:59920) 2024-05-11T12:12:18.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.298+0000 7f7711400640 1 -- 172.21.15.167:0/3520869834 learned_addr learned my addr 172.21.15.167:0/3520869834 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:18.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.299+0000 7f7719e00640 1 --2- 172.21.15.167:0/3520869834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f771c06d460 0x7f771c1428e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:18.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.299+0000 7f7711400640 1 -- 172.21.15.167:0/3520869834 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f771c157750 msgr2=0x7f771c1f7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:18.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.299+0000 7f7711400640 1 --2- 172.21.15.167:0/3520869834 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f771c157750 0x7f771c1f7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:18.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.299+0000 7f7711400640 1 -- 172.21.15.167:0/3520869834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f771c06d460 msgr2=0x7f771c1428e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:18.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.299+0000 7f7711400640 1 --2- 172.21.15.167:0/3520869834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f771c06d460 0x7f771c1428e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:18.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.299+0000 7f7711400640 1 -- 172.21.15.167:0/3520869834 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f771c1f5850 con 0x7f771c0b3ea0 2024-05-11T12:12:18.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.299+0000 7f7711400640 1 --2- 172.21.15.167:0/3520869834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f771c0b3ea0 0x7f771c1f5000 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f770400c430 tx=0x7f770400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:18.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.299+0000 7f7712a00640 1 -- 172.21.15.167:0/3520869834 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7704011790 con 0x7f771c0b3ea0 2024-05-11T12:12:18.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.299+0000 7f7712a00640 1 -- 172.21.15.167:0/3520869834 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7704017070 con 0x7f771c0b3ea0 2024-05-11T12:12:18.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.299+0000 7f7720c00640 1 -- 172.21.15.167:0/3520869834 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f771c1f5b30 con 0x7f771c0b3ea0 2024-05-11T12:12:18.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.299+0000 7f7712a00640 1 -- 172.21.15.167:0/3520869834 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7704011bb0 con 0x7f771c0b3ea0 2024-05-11T12:12:18.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.299+0000 7f7720c00640 1 -- 172.21.15.167:0/3520869834 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f771c140390 con 0x7f771c0b3ea0 2024-05-11T12:12:18.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.301+0000 7f7720c00640 1 -- 172.21.15.167:0/3520869834 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f771c140180 con 0x7f771c0b3ea0 2024-05-11T12:12:18.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.302+0000 7f7712a00640 1 -- 172.21.15.167:0/3520869834 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f770401c460 con 0x7f771c0b3ea0 2024-05-11T12:12:18.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.303+0000 7f7712a00640 1 --2- 172.21.15.167:0/3520869834 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f76f80762d0 0x7f76f8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:18.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.303+0000 7f7712a00640 1 -- 172.21.15.167:0/3520869834 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f77040975c0 con 0x7f771c0b3ea0 2024-05-11T12:12:18.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.303+0000 7f7719e00640 1 --2- 172.21.15.167:0/3520869834 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f76f80762d0 0x7f76f8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:18.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.304+0000 7f7719e00640 1 --2- 172.21.15.167:0/3520869834 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f76f80762d0 0x7f76f8078790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f770c000f60 tx=0x7f770c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:18.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.307+0000 7f7712a00640 1 -- 172.21.15.167:0/3520869834 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7704060d80 con 0x7f771c0b3ea0 2024-05-11T12:12:18.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.551+0000 7f7720c00640 1 -- 172.21.15.167:0/3520869834 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f771c069590 con 0x7f771c0b3ea0 2024-05-11T12:12:18.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.553+0000 7f7712a00640 1 -- 172.21.15.167:0/3520869834 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5193 (secure 0 0 0) 0x7f7704060680 con 0x7f771c0b3ea0 2024-05-11T12:12:18.553 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:12:18.553 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4499,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3271527772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3271527772},{"type":"v1","addr":"172.21.15.167:6836","nonce":3271527772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4648,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2293636645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2293636645},{"type":"v1","addr":"172.21.15.167:6837","nonce":2293636645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4902,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3093163428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3093163428},{"type":"v1","addr":"172.21.15.204:6834","nonce":3093163428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:11:56.852478+0000","modified":"2024-05-11T12:11:59.926074+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4908},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4908":{"gid":4908,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/3248611739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":3248611739},{"type":"v1","addr":"172.21.15.204:6835","nonce":3248611739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4908,"qdb_cluster":[4908]},"id":2}]} 2024-05-11T12:12:18.554 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 18 2024-05-11T12:12:18.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.555+0000 7f7720c00640 1 -- 172.21.15.167:0/3520869834 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f76f80762d0 msgr2=0x7f76f8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:18.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.555+0000 7f7720c00640 1 --2- 172.21.15.167:0/3520869834 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f76f80762d0 0x7f76f8078790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f770c000f60 tx=0x7f770c009040 comp rx=0 tx=0).stop 2024-05-11T12:12:18.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.555+0000 7f7720c00640 1 -- 172.21.15.167:0/3520869834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f771c0b3ea0 msgr2=0x7f771c1f5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:18.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.555+0000 7f7720c00640 1 --2- 172.21.15.167:0/3520869834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f771c0b3ea0 0x7f771c1f5000 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f770400c430 tx=0x7f770400c8f0 comp rx=0 tx=0).stop 2024-05-11T12:12:18.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.556+0000 7f7720c00640 1 -- 172.21.15.167:0/3520869834 shutdown_connections 2024-05-11T12:12:18.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.556+0000 7f7720c00640 1 --2- 172.21.15.167:0/3520869834 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f771c157750 0x7f771c1f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:18.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.556+0000 7f7720c00640 1 --2- 172.21.15.167:0/3520869834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f771c0b3ea0 0x7f771c1f5000 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:18.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.556+0000 7f7720c00640 1 --2- 172.21.15.167:0/3520869834 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f76f80762d0 0x7f76f8078790 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:18.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.556+0000 7f7720c00640 1 --2- 172.21.15.167:0/3520869834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f771c06d460 0x7f771c1428e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:18.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.556+0000 7f7720c00640 1 -- 172.21.15.167:0/3520869834 >> 172.21.15.167:0/3520869834 conn(0x7f771c0cd8f0 msgr2=0x7f771c146750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:18.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.556+0000 7f7720c00640 1 -- 172.21.15.167:0/3520869834 shutdown_connections 2024-05-11T12:12:18.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.556+0000 7f7720c00640 1 -- 172.21.15.167:0/3520869834 wait complete. 2024-05-11T12:12:18.576 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:12:18.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.779+0000 7fa652000640 1 Processor -- start 2024-05-11T12:12:18.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.779+0000 7fa652000640 1 -- start start 2024-05-11T12:12:18.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.780+0000 7fa652000640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa64c0b3ea0 0x7fa64c161490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:18.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.781+0000 7fa652000640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa64c15df30 con 0x7fa64c157750 2024-05-11T12:12:18.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.781+0000 7fa652000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa64c15e210 con 0x7fa64c1619d0 2024-05-11T12:12:18.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.781+0000 7fa652000640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fa64c161000 con 0x7fa64c0b3ea0 2024-05-11T12:12:18.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.781+0000 7fa64b400640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fa64c157750 0x7fa64c15d740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:50140/0 (socket says 172.21.15.167:50140) 2024-05-11T12:12:18.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.781+0000 7fa64b400640 1 -- 172.21.15.167:0/986758371 learned_addr learned my addr 172.21.15.167:0/986758371 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:18.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.781+0000 7fa643e00640 1 --2- 172.21.15.167:0/986758371 >> v2:172.21.15.167:3300/0 conn(0x7fa64c0b3ea0 0x7fa64c161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:18.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.781+0000 7fa64aa00640 1 -- 172.21.15.167:0/986758371 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3905861836 0 0) 0x7fa64c15df30 con 0x7fa64c157750 2024-05-11T12:12:18.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.781+0000 7fa64aa00640 1 -- 172.21.15.167:0/986758371 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa630003520 con 0x7fa64c157750 2024-05-11T12:12:18.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.781+0000 7fa643e00640 1 -- 172.21.15.167:0/986758371 >> v1:172.21.15.204:6789/0 conn(0x7fa64c1619d0 legacy=0x7fa64c06d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:18.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.781+0000 7fa643e00640 1 -- 172.21.15.167:0/986758371 >> v1:172.21.15.167:6789/0 conn(0x7fa64c157750 legacy=0x7fa64c15d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:18.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.782+0000 7fa643e00640 1 -- 172.21.15.167:0/986758371 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa64c15e4f0 con 0x7fa64c0b3ea0 2024-05-11T12:12:18.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.783+0000 7fa643e00640 1 --2- 172.21.15.167:0/986758371 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa64c0b3ea0 0x7fa64c161490 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa6340097a0 tx=0x7fa63402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d12e45f2252e7502 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:18.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.783+0000 7fa64aa00640 1 -- 172.21.15.167:0/986758371 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa634037070 con 0x7fa64c0b3ea0 2024-05-11T12:12:18.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.783+0000 7fa64aa00640 1 -- 172.21.15.167:0/986758371 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa634032440 con 0x7fa64c0b3ea0 2024-05-11T12:12:18.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.783+0000 7fa64aa00640 1 -- 172.21.15.167:0/986758371 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa634032810 con 0x7fa64c0b3ea0 2024-05-11T12:12:18.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.783+0000 7fa652000640 1 -- 172.21.15.167:0/986758371 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa64c0b3ea0 msgr2=0x7fa64c161490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:18.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.783+0000 7fa652000640 1 --2- 172.21.15.167:0/986758371 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa64c0b3ea0 0x7fa64c161490 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa6340097a0 tx=0x7fa63402a180 comp rx=0 tx=0).stop 2024-05-11T12:12:18.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.784+0000 7fa652000640 1 -- 172.21.15.167:0/986758371 shutdown_connections 2024-05-11T12:12:18.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.784+0000 7fa652000640 1 --2- 172.21.15.167:0/986758371 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa64c0b3ea0 0x7fa64c161490 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:18.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.784+0000 7fa652000640 1 -- 172.21.15.167:0/986758371 >> 172.21.15.167:0/986758371 conn(0x7fa64c0cd8f0 msgr2=0x7fa64c0cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:18.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.784+0000 7fa652000640 1 -- 172.21.15.167:0/986758371 shutdown_connections 2024-05-11T12:12:18.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.784+0000 7fa652000640 1 -- 172.21.15.167:0/986758371 wait complete. 2024-05-11T12:12:18.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.785+0000 7fa652000640 1 Processor -- start 2024-05-11T12:12:18.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.785+0000 7fa652000640 1 -- start start 2024-05-11T12:12:18.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.786+0000 7fa652000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa64c0b3ea0 0x7fa64c156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:18.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.786+0000 7fa652000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa64c157750 0x7fa64c141c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:18.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.786+0000 7fa652000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa64c1619d0 0x7fa64c12f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:18.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.786+0000 7fa652000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa64c06dcf0 con 0x7fa64c157750 2024-05-11T12:12:18.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.786+0000 7fa652000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa64c06db70 con 0x7fa64c1619d0 2024-05-11T12:12:18.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.786+0000 7fa652000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa64c0cf1c0 con 0x7fa64c0b3ea0 2024-05-11T12:12:18.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.787+0000 7fa643e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa64c157750 0x7fa64c141c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:18.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.787+0000 7fa643e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa64c157750 0x7fa64c141c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59944/0 (socket says 172.21.15.167:59944) 2024-05-11T12:12:18.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.787+0000 7fa643e00640 1 -- 172.21.15.167:0/2346116888 learned_addr learned my addr 172.21.15.167:0/2346116888 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:18.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.787+0000 7fa64b400640 1 --2- 172.21.15.167:0/2346116888 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa64c0b3ea0 0x7fa64c156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:18.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.787+0000 7fa643e00640 1 -- 172.21.15.167:0/2346116888 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa64c0b3ea0 msgr2=0x7fa64c156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:18.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.787+0000 7fa643e00640 1 --2- 172.21.15.167:0/2346116888 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa64c0b3ea0 0x7fa64c156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:18.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.787+0000 7fa64be00640 1 --2- 172.21.15.167:0/2346116888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa64c1619d0 0x7fa64c12f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:18.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.787+0000 7fa643e00640 1 -- 172.21.15.167:0/2346116888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa64c1619d0 msgr2=0x7fa64c12f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:18.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.787+0000 7fa643e00640 1 --2- 172.21.15.167:0/2346116888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa64c1619d0 0x7fa64c12f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:18.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.787+0000 7fa643e00640 1 -- 172.21.15.167:0/2346116888 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa64c142460 con 0x7fa64c157750 2024-05-11T12:12:18.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.788+0000 7fa643e00640 1 --2- 172.21.15.167:0/2346116888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa64c157750 0x7fa64c141c10 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa634000c00 tx=0x7fa6340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:18.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.788+0000 7fa643400640 1 -- 172.21.15.167:0/2346116888 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa634037040 con 0x7fa64c157750 2024-05-11T12:12:18.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.788+0000 7fa643400640 1 -- 172.21.15.167:0/2346116888 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa634042070 con 0x7fa64c157750 2024-05-11T12:12:18.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.788+0000 7fa643400640 1 -- 172.21.15.167:0/2346116888 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa634030940 con 0x7fa64c157750 2024-05-11T12:12:18.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.788+0000 7fa652000640 1 -- 172.21.15.167:0/2346116888 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa64c13fc30 con 0x7fa64c157750 2024-05-11T12:12:18.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.788+0000 7fa652000640 1 -- 172.21.15.167:0/2346116888 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa64c1401b0 con 0x7fa64c157750 2024-05-11T12:12:18.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.789+0000 7fa652000640 1 -- 172.21.15.167:0/2346116888 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa64c069590 con 0x7fa64c157750 2024-05-11T12:12:18.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.790+0000 7fa643400640 1 -- 172.21.15.167:0/2346116888 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa634030ae0 con 0x7fa64c157750 2024-05-11T12:12:18.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.791+0000 7fa643400640 1 --2- 172.21.15.167:0/2346116888 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa628076300 0x7fa6280787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:18.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.791+0000 7fa643400640 1 -- 172.21.15.167:0/2346116888 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7fa6340b8130 con 0x7fa64c157750 2024-05-11T12:12:18.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.792+0000 7fa64b400640 1 --2- 172.21.15.167:0/2346116888 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa628076300 0x7fa6280787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:18.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.792+0000 7fa64b400640 1 --2- 172.21.15.167:0/2346116888 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa628076300 0x7fa6280787c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa63c001220 tx=0x7fa63c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:18.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:18.796+0000 7fa643400640 1 -- 172.21.15.167:0/2346116888 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa634081860 con 0x7fa64c157750 2024-05-11T12:12:19.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.048+0000 7fa652000640 1 -- 172.21.15.167:0/2346116888 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa64c0b4c00 con 0x7fa64c157750 2024-05-11T12:12:19.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.049+0000 7fa643400640 1 -- 172.21.15.167:0/2346116888 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5193 (secure 0 0 0) 0x7fa634081160 con 0x7fa64c157750 2024-05-11T12:12:19.049 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:12:19.050 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4499,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3271527772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3271527772},{"type":"v1","addr":"172.21.15.167:6836","nonce":3271527772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4648,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2293636645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2293636645},{"type":"v1","addr":"172.21.15.167:6837","nonce":2293636645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4902,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3093163428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3093163428},{"type":"v1","addr":"172.21.15.204:6834","nonce":3093163428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:11:56.852478+0000","modified":"2024-05-11T12:11:59.926074+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4908},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4908":{"gid":4908,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/3248611739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":3248611739},{"type":"v1","addr":"172.21.15.204:6835","nonce":3248611739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4908,"qdb_cluster":[4908]},"id":2}]} 2024-05-11T12:12:19.050 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 18 2024-05-11T12:12:19.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.052+0000 7fa652000640 1 -- 172.21.15.167:0/2346116888 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa628076300 msgr2=0x7fa6280787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:19.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.052+0000 7fa652000640 1 --2- 172.21.15.167:0/2346116888 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa628076300 0x7fa6280787c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa63c001220 tx=0x7fa63c009040 comp rx=0 tx=0).stop 2024-05-11T12:12:19.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.052+0000 7fa652000640 1 -- 172.21.15.167:0/2346116888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa64c157750 msgr2=0x7fa64c141c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:19.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.052+0000 7fa652000640 1 --2- 172.21.15.167:0/2346116888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa64c157750 0x7fa64c141c10 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa634000c00 tx=0x7fa6340046c0 comp rx=0 tx=0).stop 2024-05-11T12:12:19.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.052+0000 7fa652000640 1 -- 172.21.15.167:0/2346116888 shutdown_connections 2024-05-11T12:12:19.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.052+0000 7fa652000640 1 --2- 172.21.15.167:0/2346116888 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa628076300 0x7fa6280787c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:19.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.052+0000 7fa652000640 1 --2- 172.21.15.167:0/2346116888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa64c1619d0 0x7fa64c12f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:19.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.052+0000 7fa652000640 1 --2- 172.21.15.167:0/2346116888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa64c157750 0x7fa64c141c10 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:19.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.052+0000 7fa652000640 1 --2- 172.21.15.167:0/2346116888 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa64c0b3ea0 0x7fa64c156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:19.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.052+0000 7fa652000640 1 -- 172.21.15.167:0/2346116888 >> 172.21.15.167:0/2346116888 conn(0x7fa64c0cd8f0 msgr2=0x7fa64c0b4360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:19.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.052+0000 7fa652000640 1 -- 172.21.15.167:0/2346116888 shutdown_connections 2024-05-11T12:12:19.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.052+0000 7fa652000640 1 -- 172.21.15.167:0/2346116888 wait complete. 2024-05-11T12:12:19.071 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:12:19.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.266+0000 7fb3dd800640 1 Processor -- start 2024-05-11T12:12:19.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.266+0000 7fb3dd800640 1 -- start start 2024-05-11T12:12:19.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.268+0000 7fb3dd800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3d80cd4e0 0x7fb3d80b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:19.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.268+0000 7fb3dd800640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3d8160b50 con 0x7fb3d80b4fa0 2024-05-11T12:12:19.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.268+0000 7fb3dd800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3d8160df0 con 0x7fb3d815ab50 2024-05-11T12:12:19.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.268+0000 7fb3dd800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb3d815af30 con 0x7fb3d80cd4e0 2024-05-11T12:12:19.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.268+0000 7fb3dce00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fb3d80b4fa0 0x7fb3d815d4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:54522/0 (socket says 172.21.15.167:54522) 2024-05-11T12:12:19.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.268+0000 7fb3dce00640 1 -- 172.21.15.167:0/398709470 learned_addr learned my addr 172.21.15.167:0/398709470 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:19.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.269+0000 7fb3d7400640 1 --2- 172.21.15.167:0/398709470 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3d80cd4e0 0x7fb3d80b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:19.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.269+0000 7fb3d6a00640 1 -- 172.21.15.167:0/398709470 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3137964678 0 0) 0x7fb3d8160b50 con 0x7fb3d80b4fa0 2024-05-11T12:12:19.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.269+0000 7fb3d6a00640 1 -- 172.21.15.167:0/398709470 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb3b8003520 con 0x7fb3d80b4fa0 2024-05-11T12:12:19.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.269+0000 7fb3d6a00640 1 -- 172.21.15.167:0/398709470 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1942270752 0 0) 0x7fb3b8003520 con 0x7fb3d80b4fa0 2024-05-11T12:12:19.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.269+0000 7fb3d6a00640 1 -- 172.21.15.167:0/398709470 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3d80cd4e0 msgr2=0x7fb3d80b4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:19.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.269+0000 7fb3d6a00640 1 --2- 172.21.15.167:0/398709470 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3d80cd4e0 0x7fb3d80b4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:19.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.269+0000 7fb3d6a00640 1 -- 172.21.15.167:0/398709470 >> v1:172.21.15.204:6789/0 conn(0x7fb3d815ab50 legacy=0x7fb3d80d2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:19.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.269+0000 7fb3d6a00640 1 -- 172.21.15.167:0/398709470 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3d81610d0 con 0x7fb3d80b4fa0 2024-05-11T12:12:19.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.269+0000 7fb3d6a00640 1 -- 172.21.15.167:0/398709470 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7fb3d00016f0 con 0x7fb3d80b4fa0 2024-05-11T12:12:19.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.269+0000 7fb3d6a00640 1 -- 172.21.15.167:0/398709470 >> v1:172.21.15.167:6789/0 conn(0x7fb3d80b4fa0 legacy=0x7fb3d815d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:19.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.269+0000 7fb3d6a00640 1 --2- 172.21.15.167:0/398709470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3b8003a80 0x7fb3b8003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:19.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.270+0000 7fb3d6a00640 1 --2- 172.21.15.167:0/398709470 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3d80cd4e0 0x7fb3b8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:19.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.270+0000 7fb3d6a00640 1 --2- 172.21.15.167:0/398709470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3b8004d40 0x7fb3b80051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:19.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.270+0000 7fb3d6a00640 1 -- 172.21.15.167:0/398709470 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d815af30 con 0x7fb3b8004d40 2024-05-11T12:12:19.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.270+0000 7fb3d6a00640 1 -- 172.21.15.167:0/398709470 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b800bb50 con 0x7fb3b8003a80 2024-05-11T12:12:19.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.270+0000 7fb3d6a00640 1 -- 172.21.15.167:0/398709470 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb3b800bd50 con 0x7fb3d80cd4e0 2024-05-11T12:12:19.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.270+0000 7fb3dce00640 1 --2- 172.21.15.167:0/398709470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3b8004d40 0x7fb3b80051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:19.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.270+0000 7fb3d7400640 1 --2- 172.21.15.167:0/398709470 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3d80cd4e0 0x7fb3b8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:19.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.270+0000 7fb3dce00640 1 -- 172.21.15.167:0/398709470 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3d80cd4e0 msgr2=0x7fb3b8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:19.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.270+0000 7fb3dce00640 1 --2- 172.21.15.167:0/398709470 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3d80cd4e0 0x7fb3b8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:19.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.270+0000 7fb3dce00640 1 -- 172.21.15.167:0/398709470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3b8003a80 msgr2=0x7fb3b8003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:19.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.270+0000 7fb3d7e00640 1 --2- 172.21.15.167:0/398709470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3b8003a80 0x7fb3b8003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:19.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.270+0000 7fb3dce00640 1 --2- 172.21.15.167:0/398709470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3b8003a80 0x7fb3b8003ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:19.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.270+0000 7fb3dce00640 1 -- 172.21.15.167:0/398709470 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b8003520 con 0x7fb3b8004d40 2024-05-11T12:12:19.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.271+0000 7fb3dce00640 1 --2- 172.21.15.167:0/398709470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3b8004d40 0x7fb3b80051a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb3d000c190 tx=0x7fb3b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7dd99782b9a2230 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:19.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.272+0000 7fb3d6a00640 1 -- 172.21.15.167:0/398709470 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d0007c60 con 0x7fb3b8004d40 2024-05-11T12:12:19.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.272+0000 7fb3d6a00640 1 -- 172.21.15.167:0/398709470 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3d0005af0 con 0x7fb3b8004d40 2024-05-11T12:12:19.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.272+0000 7fb3d6a00640 1 -- 172.21.15.167:0/398709470 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d0007070 con 0x7fb3b8004d40 2024-05-11T12:12:19.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.272+0000 7fb3dd800640 1 -- 172.21.15.167:0/398709470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3b8004d40 msgr2=0x7fb3b80051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:19.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.272+0000 7fb3dd800640 1 --2- 172.21.15.167:0/398709470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3b8004d40 0x7fb3b80051a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb3d000c190 tx=0x7fb3b8005830 comp rx=0 tx=0).stop 2024-05-11T12:12:19.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.272+0000 7fb3dce00640 1 -- 172.21.15.167:0/398709470 reap_dead start 2024-05-11T12:12:19.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.272+0000 7fb3dd800640 1 -- 172.21.15.167:0/398709470 shutdown_connections 2024-05-11T12:12:19.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.272+0000 7fb3dd800640 1 -- 172.21.15.167:0/398709470 >> 172.21.15.167:0/398709470 conn(0x7fb3d80d2e20 msgr2=0x7fb3d80d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:19.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.272+0000 7fb3dd800640 1 -- 172.21.15.167:0/398709470 shutdown_connections 2024-05-11T12:12:19.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.272+0000 7fb3dd800640 1 -- 172.21.15.167:0/398709470 wait complete. 2024-05-11T12:12:19.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.274+0000 7fb3dd800640 1 Processor -- start 2024-05-11T12:12:19.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.274+0000 7fb3dd800640 1 -- start start 2024-05-11T12:12:19.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.275+0000 7fb3dd800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3b8004d40 0x7fb3d8150ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:19.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.275+0000 7fb3dd800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3d81f57d0 0x7fb3d8153ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:19.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.275+0000 7fb3dd800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3d8156530 0x7fb3d81569c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:19.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.275+0000 7fb3dd800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d80d0960 con 0x7fb3b8004d40 2024-05-11T12:12:19.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.275+0000 7fb3dd800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d8161690 con 0x7fb3d8156530 2024-05-11T12:12:19.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.275+0000 7fb3dd800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb3d80d0ae0 con 0x7fb3d81f57d0 2024-05-11T12:12:19.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.275+0000 7fb3d7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3b8004d40 0x7fb3d8150ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:19.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.276+0000 7fb3d7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3b8004d40 0x7fb3d8150ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39870/0 (socket says 172.21.15.167:39870) 2024-05-11T12:12:19.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.276+0000 7fb3d7e00640 1 -- 172.21.15.167:0/3690862426 learned_addr learned my addr 172.21.15.167:0/3690862426 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:19.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.276+0000 7fb3d7e00640 1 -- 172.21.15.167:0/3690862426 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3d81f57d0 msgr2=0x7fb3d8153ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:19.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.276+0000 7fb3d7400640 1 --2- 172.21.15.167:0/3690862426 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3d81f57d0 0x7fb3d8153ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:19.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.276+0000 7fb3d7e00640 1 --2- 172.21.15.167:0/3690862426 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3d81f57d0 0x7fb3d8153ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:19.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.276+0000 7fb3dce00640 1 --2- 172.21.15.167:0/3690862426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3d8156530 0x7fb3d81569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:19.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.276+0000 7fb3d7e00640 1 -- 172.21.15.167:0/3690862426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3d8156530 msgr2=0x7fb3d81569c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:19.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.276+0000 7fb3d7e00640 1 --2- 172.21.15.167:0/3690862426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3d8156530 0x7fb3d81569c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:19.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.276+0000 7fb3d7e00640 1 -- 172.21.15.167:0/3690862426 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3d81547d0 con 0x7fb3b8004d40 2024-05-11T12:12:19.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.276+0000 7fb3d7e00640 1 --2- 172.21.15.167:0/3690862426 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3b8004d40 0x7fb3d8150ab0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb3c800d7a0 tx=0x7fb3c800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:19.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.276+0000 7fb3cfe00640 1 -- 172.21.15.167:0/3690862426 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c8005270 con 0x7fb3b8004d40 2024-05-11T12:12:19.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.277+0000 7fb3cfe00640 1 -- 172.21.15.167:0/3690862426 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3c8017070 con 0x7fb3b8004d40 2024-05-11T12:12:19.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.277+0000 7fb3cfe00640 1 -- 172.21.15.167:0/3690862426 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c800b6b0 con 0x7fb3b8004d40 2024-05-11T12:12:19.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.277+0000 7fb3dd800640 1 -- 172.21.15.167:0/3690862426 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3d8151b70 con 0x7fb3b8004d40 2024-05-11T12:12:19.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.277+0000 7fb3dd800640 1 -- 172.21.15.167:0/3690862426 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3d81523b0 con 0x7fb3b8004d40 2024-05-11T12:12:19.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.278+0000 7fb3dd800640 1 -- 172.21.15.167:0/3690862426 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3d8069bc0 con 0x7fb3b8004d40 2024-05-11T12:12:19.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.279+0000 7fb3cfe00640 1 -- 172.21.15.167:0/3690862426 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb3c800b850 con 0x7fb3b8004d40 2024-05-11T12:12:19.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.280+0000 7fb3cfe00640 1 --2- 172.21.15.167:0/3690862426 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb3b4076300 0x7fb3b40787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:19.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.280+0000 7fb3cfe00640 1 -- 172.21.15.167:0/3690862426 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7fb3c8097660 con 0x7fb3b8004d40 2024-05-11T12:12:19.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.280+0000 7fb3d7400640 1 --2- 172.21.15.167:0/3690862426 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb3b4076300 0x7fb3b40787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:19.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.281+0000 7fb3d7400640 1 --2- 172.21.15.167:0/3690862426 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb3b4076300 0x7fb3b40787c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb3d81f5c10 tx=0x7fb3c000b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:19.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.284+0000 7fb3cfe00640 1 -- 172.21.15.167:0/3690862426 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb3d8151b70 con 0x7fb3b8004d40 2024-05-11T12:12:19.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.496+0000 7fb3dd800640 1 -- 172.21.15.167:0/3690862426 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb3d8071560 con 0x7fb3b8004d40 2024-05-11T12:12:19.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.498+0000 7fb3cfe00640 1 -- 172.21.15.167:0/3690862426 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+16990 (secure 0 0 0) 0x7fb3c8060d90 con 0x7fb3b8004d40 2024-05-11T12:12:19.498 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:12:19.499 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":41,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:11:58.413721+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:11:54.849622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-11T12:11:55.831334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:12:19.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.501+0000 7fb3dd800640 1 -- 172.21.15.167:0/3690862426 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb3b4076300 msgr2=0x7fb3b40787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:19.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.501+0000 7fb3dd800640 1 --2- 172.21.15.167:0/3690862426 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb3b4076300 0x7fb3b40787c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb3d81f5c10 tx=0x7fb3c000b040 comp rx=0 tx=0).stop 2024-05-11T12:12:19.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.501+0000 7fb3dd800640 1 -- 172.21.15.167:0/3690862426 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3b8004d40 msgr2=0x7fb3d8150ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:19.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.501+0000 7fb3dd800640 1 --2- 172.21.15.167:0/3690862426 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3b8004d40 0x7fb3d8150ab0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb3c800d7a0 tx=0x7fb3c800dc60 comp rx=0 tx=0).stop 2024-05-11T12:12:19.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.501+0000 7fb3dd800640 1 -- 172.21.15.167:0/3690862426 shutdown_connections 2024-05-11T12:12:19.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.501+0000 7fb3dd800640 1 --2- 172.21.15.167:0/3690862426 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb3b4076300 0x7fb3b40787c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:19.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.501+0000 7fb3dd800640 1 --2- 172.21.15.167:0/3690862426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3d8156530 0x7fb3d81569c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:19.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.501+0000 7fb3dd800640 1 --2- 172.21.15.167:0/3690862426 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3d81f57d0 0x7fb3d8153ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:19.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.501+0000 7fb3dd800640 1 --2- 172.21.15.167:0/3690862426 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3b8004d40 0x7fb3d8150ab0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:19.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.501+0000 7fb3dd800640 1 -- 172.21.15.167:0/3690862426 >> 172.21.15.167:0/3690862426 conn(0x7fb3d80d2e20 msgr2=0x7fb3d8159180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:19.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.501+0000 7fb3dd800640 1 -- 172.21.15.167:0/3690862426 shutdown_connections 2024-05-11T12:12:19.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.501+0000 7fb3dd800640 1 -- 172.21.15.167:0/3690862426 wait complete. 2024-05-11T12:12:19.520 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-11T12:12:19.520 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:12:19.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.718+0000 7f2809400640 1 Processor -- start 2024-05-11T12:12:19.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.719+0000 7f2809400640 1 -- start start 2024-05-11T12:12:19.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.720+0000 7f2809400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f280415b860 0x7f280406f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:19.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.720+0000 7f2809400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f28040736a0 0x7f2804072b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:19.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.720+0000 7f2809400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f280415bc40 con 0x7f2804072640 2024-05-11T12:12:19.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.720+0000 7f2809400640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f280406e410 con 0x7f28040736a0 2024-05-11T12:12:19.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.720+0000 7f2809400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f280406e6f0 con 0x7f280415b860 2024-05-11T12:12:19.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.720+0000 7f2801e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f28040736a0 0x7f2804072b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:19.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.720+0000 7f2801e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f28040736a0 0x7f2804072b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39882/0 (socket says 172.21.15.167:39882) 2024-05-11T12:12:19.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.720+0000 7f2801e00640 1 -- 172.21.15.167:0/801597430 learned_addr learned my addr 172.21.15.167:0/801597430 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:19.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.721+0000 7f2801e00640 1 -- 172.21.15.167:0/801597430 >> v2:172.21.15.204:3300/0 conn(0x7f280415b860 msgr2=0x7f280406f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:19.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.721+0000 7f2801e00640 1 --2- 172.21.15.167:0/801597430 >> v2:172.21.15.204:3300/0 conn(0x7f280415b860 0x7f280406f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:19.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.721+0000 7f2801e00640 1 -- 172.21.15.167:0/801597430 >> v1:172.21.15.204:6789/0 conn(0x7f2804072640 legacy=0x7f28040b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:19.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.721+0000 7f2801e00640 1 -- 172.21.15.167:0/801597430 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f280406e9d0 con 0x7f28040736a0 2024-05-11T12:12:19.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.722+0000 7f2801e00640 1 --2- 172.21.15.167:0/801597430 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28040736a0 0x7f2804072b00 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f28040700b0 tx=0x7f27ec02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1b6a6beb623683a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:19.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.722+0000 7f2801400640 1 -- 172.21.15.167:0/801597430 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ec037070 con 0x7f28040736a0 2024-05-11T12:12:19.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.722+0000 7f2801400640 1 -- 172.21.15.167:0/801597430 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27ec032440 con 0x7f28040736a0 2024-05-11T12:12:19.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.722+0000 7f2801400640 1 -- 172.21.15.167:0/801597430 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ec032810 con 0x7f28040736a0 2024-05-11T12:12:19.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.723+0000 7f2809400640 1 -- 172.21.15.167:0/801597430 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28040736a0 msgr2=0x7f2804072b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:19.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.723+0000 7f2809400640 1 --2- 172.21.15.167:0/801597430 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28040736a0 0x7f2804072b00 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f28040700b0 tx=0x7f27ec02a1c0 comp rx=0 tx=0).stop 2024-05-11T12:12:19.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.723+0000 7f2809400640 1 -- 172.21.15.167:0/801597430 shutdown_connections 2024-05-11T12:12:19.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.723+0000 7f2809400640 1 --2- 172.21.15.167:0/801597430 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28040736a0 0x7f2804072b00 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:19.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.723+0000 7f2809400640 1 --2- 172.21.15.167:0/801597430 >> v2:172.21.15.204:3300/0 conn(0x7f280415b860 0x7f280406f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:19.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.723+0000 7f2809400640 1 -- 172.21.15.167:0/801597430 >> 172.21.15.167:0/801597430 conn(0x7f28040cd8f0 msgr2=0x7f28040cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:19.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.723+0000 7f2809400640 1 -- 172.21.15.167:0/801597430 shutdown_connections 2024-05-11T12:12:19.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.723+0000 7f2809400640 1 -- 172.21.15.167:0/801597430 wait complete. 2024-05-11T12:12:19.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.725+0000 7f2809400640 1 Processor -- start 2024-05-11T12:12:19.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.725+0000 7f2809400640 1 -- start start 2024-05-11T12:12:19.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.726+0000 7f2809400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2804072640 0x7f28041544c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:19.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.726+0000 7f2809400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28040736a0 0x7f280414dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:19.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.726+0000 7f2809400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f280415b860 0x7f2804140980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:19.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.726+0000 7f2809400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f28040b4ee0 con 0x7f280415b860 2024-05-11T12:12:19.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.726+0000 7f2809400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f280406ef90 con 0x7f2804072640 2024-05-11T12:12:19.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.726+0000 7f2809400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f28040b5060 con 0x7f28040736a0 2024-05-11T12:12:19.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.726+0000 7f2803200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f280415b860 0x7f2804140980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:19.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.726+0000 7f2803200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f280415b860 0x7f2804140980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39896/0 (socket says 172.21.15.167:39896) 2024-05-11T12:12:19.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.726+0000 7f2803200640 1 -- 172.21.15.167:0/2312031027 learned_addr learned my addr 172.21.15.167:0/2312031027 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:19.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.726+0000 7f2802800640 1 --2- 172.21.15.167:0/2312031027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2804072640 0x7f28041544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:19.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.726+0000 7f2801e00640 1 --2- 172.21.15.167:0/2312031027 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28040736a0 0x7f280414dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:19.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.726+0000 7f2803200640 1 -- 172.21.15.167:0/2312031027 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28040736a0 msgr2=0x7f280414dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:19.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.726+0000 7f2803200640 1 --2- 172.21.15.167:0/2312031027 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28040736a0 0x7f280414dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:19.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.726+0000 7f2803200640 1 -- 172.21.15.167:0/2312031027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2804072640 msgr2=0x7f28041544c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:19.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.726+0000 7f2803200640 1 --2- 172.21.15.167:0/2312031027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2804072640 0x7f28041544c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:19.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.726+0000 7f2803200640 1 -- 172.21.15.167:0/2312031027 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f280414e320 con 0x7f280415b860 2024-05-11T12:12:19.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.726+0000 7f2802800640 1 --2- 172.21.15.167:0/2312031027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2804072640 0x7f28041544c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:19.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.727+0000 7f2803200640 1 --2- 172.21.15.167:0/2312031027 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f280415b860 0x7f2804140980 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f27fc00c3c0 tx=0x7f27fc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:19.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.727+0000 7f27faa00640 1 -- 172.21.15.167:0/2312031027 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27fc017070 con 0x7f280415b860 2024-05-11T12:12:19.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.727+0000 7f27faa00640 1 -- 172.21.15.167:0/2312031027 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27fc0035f0 con 0x7f280415b860 2024-05-11T12:12:19.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.727+0000 7f2809400640 1 -- 172.21.15.167:0/2312031027 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28041539e0 con 0x7f280415b860 2024-05-11T12:12:19.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.727+0000 7f27faa00640 1 -- 172.21.15.167:0/2312031027 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27fc0039e0 con 0x7f280415b860 2024-05-11T12:12:19.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.727+0000 7f2809400640 1 -- 172.21.15.167:0/2312031027 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2804153fa0 con 0x7f280415b860 2024-05-11T12:12:19.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.728+0000 7f2809400640 1 -- 172.21.15.167:0/2312031027 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2804069590 con 0x7f280415b860 2024-05-11T12:12:19.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.729+0000 7f27faa00640 1 -- 172.21.15.167:0/2312031027 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f27fc01a070 con 0x7f280415b860 2024-05-11T12:12:19.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.730+0000 7f27faa00640 1 --2- 172.21.15.167:0/2312031027 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f27e0076300 0x7f27e00787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:19.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.730+0000 7f27faa00640 1 -- 172.21.15.167:0/2312031027 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f27fc096ae0 con 0x7f280415b860 2024-05-11T12:12:19.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.730+0000 7f2802800640 1 --2- 172.21.15.167:0/2312031027 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f27e0076300 0x7f27e00787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:19.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.731+0000 7f2802800640 1 --2- 172.21.15.167:0/2312031027 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f27e0076300 0x7f27e00787c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2804152690 tx=0x7f27f40073e0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:19.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.735+0000 7f27faa00640 1 -- 172.21.15.167:0/2312031027 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f27fc0602a0 con 0x7f280415b860 2024-05-11T12:12:19.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.982+0000 7f2809400640 1 -- 172.21.15.167:0/2312031027 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28040cf1d0 con 0x7f280415b860 2024-05-11T12:12:19.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.983+0000 7f27faa00640 1 -- 172.21.15.167:0/2312031027 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5193 (secure 0 0 0) 0x7f27fc068290 con 0x7f280415b860 2024-05-11T12:12:19.984 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:12:19.984 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4499,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3271527772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3271527772},{"type":"v1","addr":"172.21.15.167:6836","nonce":3271527772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4648,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2293636645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2293636645},{"type":"v1","addr":"172.21.15.167:6837","nonce":2293636645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4902,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3093163428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3093163428},{"type":"v1","addr":"172.21.15.204:6834","nonce":3093163428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:11:56.852478+0000","modified":"2024-05-11T12:11:59.926074+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4908},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4908":{"gid":4908,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/3248611739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":3248611739},{"type":"v1","addr":"172.21.15.204:6835","nonce":3248611739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4908,"qdb_cluster":[4908]},"id":2}]} 2024-05-11T12:12:19.984 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 18 2024-05-11T12:12:19.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.986+0000 7f2809400640 1 -- 172.21.15.167:0/2312031027 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f27e0076300 msgr2=0x7f27e00787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:19.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.986+0000 7f2809400640 1 --2- 172.21.15.167:0/2312031027 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f27e0076300 0x7f27e00787c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2804152690 tx=0x7f27f40073e0 comp rx=0 tx=0).stop 2024-05-11T12:12:19.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.986+0000 7f2809400640 1 -- 172.21.15.167:0/2312031027 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f280415b860 msgr2=0x7f2804140980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:19.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.986+0000 7f2809400640 1 --2- 172.21.15.167:0/2312031027 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f280415b860 0x7f2804140980 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f27fc00c3c0 tx=0x7f27fc00c880 comp rx=0 tx=0).stop 2024-05-11T12:12:19.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.986+0000 7f2809400640 1 -- 172.21.15.167:0/2312031027 shutdown_connections 2024-05-11T12:12:19.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.986+0000 7f2809400640 1 --2- 172.21.15.167:0/2312031027 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f280415b860 0x7f2804140980 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:19.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.986+0000 7f2809400640 1 --2- 172.21.15.167:0/2312031027 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28040736a0 0x7f280414dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:19.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.986+0000 7f2809400640 1 --2- 172.21.15.167:0/2312031027 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f27e0076300 0x7f27e00787c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:19.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.986+0000 7f2809400640 1 --2- 172.21.15.167:0/2312031027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2804072640 0x7f28041544c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:19.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.986+0000 7f2809400640 1 -- 172.21.15.167:0/2312031027 >> 172.21.15.167:0/2312031027 conn(0x7f28040cd8f0 msgr2=0x7f2804072ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:19.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.986+0000 7f2809400640 1 -- 172.21.15.167:0/2312031027 shutdown_connections 2024-05-11T12:12:19.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:19.986+0000 7f2809400640 1 -- 172.21.15.167:0/2312031027 wait complete. 2024-05-11T12:12:20.005 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:12:20.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.208+0000 7f5326200640 1 Processor -- start 2024-05-11T12:12:20.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.208+0000 7f5326200640 1 -- start start 2024-05-11T12:12:20.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.210+0000 7f5326200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f532013f290 0x7f53200735c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:20.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.210+0000 7f5326200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f53200b50a0 0x7f5320073b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:20.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.210+0000 7f5326200640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f532015fb90 con 0x7f532013eeb0 2024-05-11T12:12:20.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.210+0000 7f5326200640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f532015d5b0 con 0x7f532013f290 2024-05-11T12:12:20.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.210+0000 7f5326200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f532015fe70 con 0x7f53200b50a0 2024-05-11T12:12:20.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.210+0000 7f531f400640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f532013eeb0 0x7f532006f480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:54534/0 (socket says 172.21.15.167:54534) 2024-05-11T12:12:20.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.210+0000 7f531f400640 1 -- 172.21.15.167:0/1117466700 learned_addr learned my addr 172.21.15.167:0/1117466700 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:20.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.210+0000 7f5316a00640 1 --2- 172.21.15.167:0/1117466700 >> v2:172.21.15.167:3300/0 conn(0x7f532013f290 0x7f53200735c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:20.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.211+0000 7f531fe00640 1 --2- 172.21.15.167:0/1117466700 >> v2:172.21.15.204:3300/0 conn(0x7f53200b50a0 0x7f5320073b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:20.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.211+0000 7f531ea00640 1 -- 172.21.15.167:0/1117466700 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3887255477 0 0) 0x7f532015fb90 con 0x7f532013eeb0 2024-05-11T12:12:20.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.211+0000 7f531ea00640 1 -- 172.21.15.167:0/1117466700 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5304003520 con 0x7f532013eeb0 2024-05-11T12:12:20.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.211+0000 7f5316a00640 1 -- 172.21.15.167:0/1117466700 >> v2:172.21.15.204:3300/0 conn(0x7f53200b50a0 msgr2=0x7f5320073b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:20.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.211+0000 7f5316a00640 1 --2- 172.21.15.167:0/1117466700 >> v2:172.21.15.204:3300/0 conn(0x7f53200b50a0 0x7f5320073b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:20.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.211+0000 7f5316a00640 1 -- 172.21.15.167:0/1117466700 >> v1:172.21.15.167:6789/0 conn(0x7f532013eeb0 legacy=0x7f532006f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:20.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.211+0000 7f5316a00640 1 -- 172.21.15.167:0/1117466700 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5320160150 con 0x7f532013f290 2024-05-11T12:12:20.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.212+0000 7f5316a00640 1 --2- 172.21.15.167:0/1117466700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f532013f290 0x7f53200735c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f532015faf0 tx=0x7f530802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=516188efef45bf73 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:20.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.212+0000 7f531ea00640 1 -- 172.21.15.167:0/1117466700 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5308037070 con 0x7f532013f290 2024-05-11T12:12:20.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.212+0000 7f531ea00640 1 -- 172.21.15.167:0/1117466700 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5308032440 con 0x7f532013f290 2024-05-11T12:12:20.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.212+0000 7f531ea00640 1 -- 172.21.15.167:0/1117466700 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530802ae60 con 0x7f532013f290 2024-05-11T12:12:20.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.213+0000 7f5326200640 1 -- 172.21.15.167:0/1117466700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f532013f290 msgr2=0x7f53200735c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:20.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.213+0000 7f5326200640 1 --2- 172.21.15.167:0/1117466700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f532013f290 0x7f53200735c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f532015faf0 tx=0x7f530802a110 comp rx=0 tx=0).stop 2024-05-11T12:12:20.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.213+0000 7f5326200640 1 -- 172.21.15.167:0/1117466700 shutdown_connections 2024-05-11T12:12:20.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.213+0000 7f5326200640 1 --2- 172.21.15.167:0/1117466700 >> v2:172.21.15.204:3300/0 conn(0x7f53200b50a0 0x7f5320073b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:20.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.213+0000 7f5326200640 1 --2- 172.21.15.167:0/1117466700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f532013f290 0x7f53200735c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:20.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.213+0000 7f5326200640 1 -- 172.21.15.167:0/1117466700 >> 172.21.15.167:0/1117466700 conn(0x7f53200cd8f0 msgr2=0x7f53200cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:20.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.213+0000 7f5326200640 1 -- 172.21.15.167:0/1117466700 shutdown_connections 2024-05-11T12:12:20.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.213+0000 7f5326200640 1 -- 172.21.15.167:0/1117466700 wait complete. 2024-05-11T12:12:20.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.215+0000 7f5326200640 1 Processor -- start 2024-05-11T12:12:20.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.215+0000 7f5326200640 1 -- start start 2024-05-11T12:12:20.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.216+0000 7f5326200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53200b50a0 0x7f532015b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:20.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.216+0000 7f5326200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f532013eeb0 0x7f532015b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:20.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.216+0000 7f5326200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f532013f290 0x7f53201e8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:20.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.216+0000 7f5326200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5320074040 con 0x7f532013f290 2024-05-11T12:12:20.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.216+0000 7f5326200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5320160710 con 0x7f53200b50a0 2024-05-11T12:12:20.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.216+0000 7f5326200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f532006fb90 con 0x7f532013eeb0 2024-05-11T12:12:20.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.216+0000 7f531fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f532013f290 0x7f53201e8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:20.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.216+0000 7f531fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f532013f290 0x7f53201e8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39918/0 (socket says 172.21.15.167:39918) 2024-05-11T12:12:20.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.216+0000 7f531fe00640 1 -- 172.21.15.167:0/2828764315 learned_addr learned my addr 172.21.15.167:0/2828764315 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:20.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.216+0000 7f531f400640 1 --2- 172.21.15.167:0/2828764315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53200b50a0 0x7f532015b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:20.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.216+0000 7f5316a00640 1 --2- 172.21.15.167:0/2828764315 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f532013eeb0 0x7f532015b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:20.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.216+0000 7f531fe00640 1 -- 172.21.15.167:0/2828764315 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f532013eeb0 msgr2=0x7f532015b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:20.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.216+0000 7f531fe00640 1 --2- 172.21.15.167:0/2828764315 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f532013eeb0 0x7f532015b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:20.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.216+0000 7f531fe00640 1 -- 172.21.15.167:0/2828764315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53200b50a0 msgr2=0x7f532015b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:20.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.216+0000 7f531fe00640 1 --2- 172.21.15.167:0/2828764315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53200b50a0 0x7f532015b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:20.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.216+0000 7f531fe00640 1 -- 172.21.15.167:0/2828764315 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53201e8f50 con 0x7f532013f290 2024-05-11T12:12:20.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.217+0000 7f531fe00640 1 --2- 172.21.15.167:0/2828764315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f532013f290 0x7f53201e8370 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5318007e40 tx=0x7f531800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:20.219 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.217+0000 7f5317e00640 1 -- 172.21.15.167:0/2828764315 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5318014620 con 0x7f532013f290 2024-05-11T12:12:20.219 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.217+0000 7f5317e00640 1 -- 172.21.15.167:0/2828764315 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5318019070 con 0x7f532013f290 2024-05-11T12:12:20.219 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.217+0000 7f5317e00640 1 -- 172.21.15.167:0/2828764315 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5318014a20 con 0x7f532013f290 2024-05-11T12:12:20.219 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.217+0000 7f5326200640 1 -- 172.21.15.167:0/2828764315 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53200cd2b0 con 0x7f532013f290 2024-05-11T12:12:20.219 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.217+0000 7f5326200640 1 -- 172.21.15.167:0/2828764315 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53201e7950 con 0x7f532013f290 2024-05-11T12:12:20.219 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.218+0000 7f5326200640 1 -- 172.21.15.167:0/2828764315 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5320069590 con 0x7f532013f290 2024-05-11T12:12:20.219 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.219+0000 7f5317e00640 1 -- 172.21.15.167:0/2828764315 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5318014bc0 con 0x7f532013f290 2024-05-11T12:12:20.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.220+0000 7f5317e00640 1 --2- 172.21.15.167:0/2828764315 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f52fc0762d0 0x7f52fc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:20.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.220+0000 7f5317e00640 1 -- 172.21.15.167:0/2828764315 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f5318098120 con 0x7f532013f290 2024-05-11T12:12:20.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.221+0000 7f531f400640 1 --2- 172.21.15.167:0/2828764315 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f52fc0762d0 0x7f52fc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:20.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.221+0000 7f531f400640 1 --2- 172.21.15.167:0/2828764315 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f52fc0762d0 0x7f52fc078790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5310001220 tx=0x7f5310009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:20.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.225+0000 7f5317e00640 1 -- 172.21.15.167:0/2828764315 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5318061850 con 0x7f532013f290 2024-05-11T12:12:20.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.475+0000 7f5326200640 1 -- 172.21.15.167:0/2828764315 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53201ea840 con 0x7f532013f290 2024-05-11T12:12:20.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.476+0000 7f5317e00640 1 -- 172.21.15.167:0/2828764315 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5193 (secure 0 0 0) 0x7f5318061150 con 0x7f532013f290 2024-05-11T12:12:20.476 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:12:20.476 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4499,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3271527772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3271527772},{"type":"v1","addr":"172.21.15.167:6836","nonce":3271527772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4648,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2293636645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2293636645},{"type":"v1","addr":"172.21.15.167:6837","nonce":2293636645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4902,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3093163428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3093163428},{"type":"v1","addr":"172.21.15.204:6834","nonce":3093163428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:11:56.852478+0000","modified":"2024-05-11T12:11:59.926074+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4908},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4908":{"gid":4908,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/3248611739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":3248611739},{"type":"v1","addr":"172.21.15.204:6835","nonce":3248611739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4908,"qdb_cluster":[4908]},"id":2}]} 2024-05-11T12:12:20.476 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 18 2024-05-11T12:12:20.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.479+0000 7f5326200640 1 -- 172.21.15.167:0/2828764315 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f52fc0762d0 msgr2=0x7f52fc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:20.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.479+0000 7f5326200640 1 --2- 172.21.15.167:0/2828764315 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f52fc0762d0 0x7f52fc078790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5310001220 tx=0x7f5310009040 comp rx=0 tx=0).stop 2024-05-11T12:12:20.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.479+0000 7f5326200640 1 -- 172.21.15.167:0/2828764315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f532013f290 msgr2=0x7f53201e8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:20.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.479+0000 7f5326200640 1 --2- 172.21.15.167:0/2828764315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f532013f290 0x7f53201e8370 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5318007e40 tx=0x7f531800e4b0 comp rx=0 tx=0).stop 2024-05-11T12:12:20.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.479+0000 7f5326200640 1 -- 172.21.15.167:0/2828764315 shutdown_connections 2024-05-11T12:12:20.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.479+0000 7f5326200640 1 --2- 172.21.15.167:0/2828764315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f532013f290 0x7f53201e8370 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:20.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.479+0000 7f5326200640 1 --2- 172.21.15.167:0/2828764315 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f532013eeb0 0x7f532015b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:20.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.479+0000 7f5326200640 1 --2- 172.21.15.167:0/2828764315 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f52fc0762d0 0x7f52fc078790 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:20.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.479+0000 7f5326200640 1 --2- 172.21.15.167:0/2828764315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53200b50a0 0x7f532015b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:20.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.479+0000 7f5326200640 1 -- 172.21.15.167:0/2828764315 >> 172.21.15.167:0/2828764315 conn(0x7f53200cd8f0 msgr2=0x7f532006d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:20.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.479+0000 7f5326200640 1 -- 172.21.15.167:0/2828764315 shutdown_connections 2024-05-11T12:12:20.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.479+0000 7f5326200640 1 -- 172.21.15.167:0/2828764315 wait complete. 2024-05-11T12:12:20.498 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:12:20.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.694+0000 7f75bee00640 1 Processor -- start 2024-05-11T12:12:20.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.694+0000 7f75bee00640 1 -- start start 2024-05-11T12:12:20.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.695+0000 7f75bee00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75b8157750 0x7f75b815d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:20.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.695+0000 7f75bee00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f75b80b3ea0 0x7f75b8161490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:20.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.695+0000 7f75bee00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f75b81619d0 0x7f75b806d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:20.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.695+0000 7f75bee00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f75b8161000 con 0x7f75b81619d0 2024-05-11T12:12:20.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.695+0000 7f75bee00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f75b815dc80 con 0x7f75b80b3ea0 2024-05-11T12:12:20.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.695+0000 7f75bee00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f75b815df30 con 0x7f75b8157750 2024-05-11T12:12:20.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.696+0000 7f75bce00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f75b81619d0 0x7f75b806d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:20.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.696+0000 7f75bce00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f75b81619d0 0x7f75b806d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39924/0 (socket says 172.21.15.167:39924) 2024-05-11T12:12:20.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.696+0000 7f75bce00640 1 -- 172.21.15.167:0/313427343 learned_addr learned my addr 172.21.15.167:0/313427343 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:20.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.696+0000 7f75b7e00640 1 --2- 172.21.15.167:0/313427343 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75b8157750 0x7f75b815d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:20.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.696+0000 7f75b7400640 1 --2- 172.21.15.167:0/313427343 >> v2:172.21.15.204:3300/0 conn(0x7f75b80b3ea0 0x7f75b8161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:20.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.696+0000 7f75bce00640 1 -- 172.21.15.167:0/313427343 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75b8157750 msgr2=0x7f75b815d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:20.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.696+0000 7f75bce00640 1 --2- 172.21.15.167:0/313427343 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75b8157750 0x7f75b815d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:20.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.696+0000 7f75bce00640 1 -- 172.21.15.167:0/313427343 >> v2:172.21.15.204:3300/0 conn(0x7f75b80b3ea0 msgr2=0x7f75b8161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:20.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.696+0000 7f75bce00640 1 --2- 172.21.15.167:0/313427343 >> v2:172.21.15.204:3300/0 conn(0x7f75b80b3ea0 0x7f75b8161490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:20.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.696+0000 7f75bce00640 1 -- 172.21.15.167:0/313427343 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b815e210 con 0x7f75b81619d0 2024-05-11T12:12:20.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.696+0000 7f75b7400640 1 --2- 172.21.15.167:0/313427343 >> v2:172.21.15.204:3300/0 conn(0x7f75b80b3ea0 0x7f75b8161490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:20.698 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.696+0000 7f75b7e00640 1 --2- 172.21.15.167:0/313427343 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75b8157750 0x7f75b815d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:20.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.698+0000 7f75bce00640 1 --2- 172.21.15.167:0/313427343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75b81619d0 0x7f75b806d460 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f75b806e0a0 tx=0x7f75b002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3c1389135cf4213 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:20.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.698+0000 7f75b6a00640 1 -- 172.21.15.167:0/313427343 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b0036070 con 0x7f75b81619d0 2024-05-11T12:12:20.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.698+0000 7f75b6a00640 1 -- 172.21.15.167:0/313427343 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75b0031600 con 0x7f75b81619d0 2024-05-11T12:12:20.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.698+0000 7f75b6a00640 1 -- 172.21.15.167:0/313427343 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b00319d0 con 0x7f75b81619d0 2024-05-11T12:12:20.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.698+0000 7f75bee00640 1 -- 172.21.15.167:0/313427343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75b81619d0 msgr2=0x7f75b806d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:20.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.698+0000 7f75bee00640 1 --2- 172.21.15.167:0/313427343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75b81619d0 0x7f75b806d460 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f75b806e0a0 tx=0x7f75b002a6a0 comp rx=0 tx=0).stop 2024-05-11T12:12:20.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.699+0000 7f75bee00640 1 -- 172.21.15.167:0/313427343 shutdown_connections 2024-05-11T12:12:20.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.699+0000 7f75bee00640 1 --2- 172.21.15.167:0/313427343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75b81619d0 0x7f75b806d460 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:20.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.699+0000 7f75bee00640 1 --2- 172.21.15.167:0/313427343 >> v2:172.21.15.204:3300/0 conn(0x7f75b80b3ea0 0x7f75b8161490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:20.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.699+0000 7f75bee00640 1 --2- 172.21.15.167:0/313427343 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75b8157750 0x7f75b815d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:20.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.699+0000 7f75bee00640 1 -- 172.21.15.167:0/313427343 >> 172.21.15.167:0/313427343 conn(0x7f75b80cd8f0 msgr2=0x7f75b80cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:20.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.699+0000 7f75bee00640 1 -- 172.21.15.167:0/313427343 shutdown_connections 2024-05-11T12:12:20.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.699+0000 7f75bee00640 1 -- 172.21.15.167:0/313427343 wait complete. 2024-05-11T12:12:20.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.700+0000 7f75bee00640 1 Processor -- start 2024-05-11T12:12:20.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.701+0000 7f75bee00640 1 -- start start 2024-05-11T12:12:20.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.702+0000 7f75bee00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75b80b3ea0 0x7f75b81f2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:20.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.702+0000 7f75bee00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75b8157750 0x7f75b815c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:20.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.702+0000 7f75bee00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75b81619d0 0x7f75b80af9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:20.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.702+0000 7f75bee00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f75b806d9a0 con 0x7f75b8157750 2024-05-11T12:12:20.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.702+0000 7f75bee00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f75b8157b30 con 0x7f75b80b3ea0 2024-05-11T12:12:20.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.702+0000 7f75bee00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f75b806db20 con 0x7f75b81619d0 2024-05-11T12:12:20.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.702+0000 7f75b7400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75b8157750 0x7f75b815c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:20.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.702+0000 7f75b7400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75b8157750 0x7f75b815c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39936/0 (socket says 172.21.15.167:39936) 2024-05-11T12:12:20.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.702+0000 7f75b7400640 1 -- 172.21.15.167:0/2631595672 learned_addr learned my addr 172.21.15.167:0/2631595672 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:20.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.702+0000 7f75b7e00640 1 --2- 172.21.15.167:0/2631595672 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75b80b3ea0 0x7f75b81f2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:20.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.702+0000 7f75bce00640 1 --2- 172.21.15.167:0/2631595672 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75b81619d0 0x7f75b80af9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:20.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.702+0000 7f75b7400640 1 -- 172.21.15.167:0/2631595672 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75b81619d0 msgr2=0x7f75b80af9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:20.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.702+0000 7f75b7400640 1 --2- 172.21.15.167:0/2631595672 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75b81619d0 0x7f75b80af9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:20.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.702+0000 7f75b7400640 1 -- 172.21.15.167:0/2631595672 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75b80b3ea0 msgr2=0x7f75b81f2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:20.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.702+0000 7f75b7400640 1 --2- 172.21.15.167:0/2631595672 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75b80b3ea0 0x7f75b81f2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:20.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.702+0000 7f75b7400640 1 -- 172.21.15.167:0/2631595672 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b815ad10 con 0x7f75b8157750 2024-05-11T12:12:20.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.703+0000 7f75b7400640 1 --2- 172.21.15.167:0/2631595672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75b8157750 0x7f75b815c090 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f75a400c820 tx=0x7f75a400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:20.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.703+0000 7f75afe00640 1 -- 172.21.15.167:0/2631595672 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a4015070 con 0x7f75b8157750 2024-05-11T12:12:20.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.703+0000 7f75afe00640 1 -- 172.21.15.167:0/2631595672 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75a4007a30 con 0x7f75b8157750 2024-05-11T12:12:20.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.703+0000 7f75afe00640 1 -- 172.21.15.167:0/2631595672 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a4007e20 con 0x7f75b8157750 2024-05-11T12:12:20.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.703+0000 7f75bee00640 1 -- 172.21.15.167:0/2631595672 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75b815aff0 con 0x7f75b8157750 2024-05-11T12:12:20.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.703+0000 7f75bee00640 1 -- 172.21.15.167:0/2631595672 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75b815b5b0 con 0x7f75b8157750 2024-05-11T12:12:20.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.704+0000 7f75bee00640 1 -- 172.21.15.167:0/2631595672 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75b8069590 con 0x7f75b8157750 2024-05-11T12:12:20.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.706+0000 7f75afe00640 1 -- 172.21.15.167:0/2631595672 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f75a4018070 con 0x7f75b8157750 2024-05-11T12:12:20.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.707+0000 7f75afe00640 1 --2- 172.21.15.167:0/2631595672 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f75980762d0 0x7f7598078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:20.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.707+0000 7f75afe00640 1 -- 172.21.15.167:0/2631595672 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f75a4097780 con 0x7f75b8157750 2024-05-11T12:12:20.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.707+0000 7f75b7e00640 1 --2- 172.21.15.167:0/2631595672 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f75980762d0 0x7f7598078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:20.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.708+0000 7f75b7e00640 1 --2- 172.21.15.167:0/2631595672 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f75980762d0 0x7f7598078790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f75a800a950 tx=0x7f75a8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:20.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.711+0000 7f75afe00640 1 -- 172.21.15.167:0/2631595672 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f75a4060e30 con 0x7f75b8157750 2024-05-11T12:12:20.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.921+0000 7f75bee00640 1 -- 172.21.15.167:0/2631595672 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f75b80cdfe0 con 0x7f75b8157750 2024-05-11T12:12:20.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.923+0000 7f75afe00640 1 -- 172.21.15.167:0/2631595672 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+16990 (secure 0 0 0) 0x7f75a4060730 con 0x7f75b8157750 2024-05-11T12:12:20.924 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:12:20.924 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":41,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:11:58.413721+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:11:54.849622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-11T12:11:55.831334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:12:20.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.926+0000 7f75bee00640 1 -- 172.21.15.167:0/2631595672 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f75980762d0 msgr2=0x7f7598078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:20.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.926+0000 7f75bee00640 1 --2- 172.21.15.167:0/2631595672 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f75980762d0 0x7f7598078790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f75a800a950 tx=0x7f75a8008040 comp rx=0 tx=0).stop 2024-05-11T12:12:20.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.926+0000 7f75bee00640 1 -- 172.21.15.167:0/2631595672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75b8157750 msgr2=0x7f75b815c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:20.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.926+0000 7f75bee00640 1 --2- 172.21.15.167:0/2631595672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75b8157750 0x7f75b815c090 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f75a400c820 tx=0x7f75a400cce0 comp rx=0 tx=0).stop 2024-05-11T12:12:20.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.926+0000 7f75bee00640 1 -- 172.21.15.167:0/2631595672 shutdown_connections 2024-05-11T12:12:20.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.926+0000 7f75bee00640 1 --2- 172.21.15.167:0/2631595672 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75b81619d0 0x7f75b80af9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:20.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.926+0000 7f75bee00640 1 --2- 172.21.15.167:0/2631595672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75b8157750 0x7f75b815c090 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:20.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.926+0000 7f75bee00640 1 --2- 172.21.15.167:0/2631595672 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f75980762d0 0x7f7598078790 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:20.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.926+0000 7f75bee00640 1 --2- 172.21.15.167:0/2631595672 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75b80b3ea0 0x7f75b81f2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:20.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.926+0000 7f75bee00640 1 -- 172.21.15.167:0/2631595672 >> 172.21.15.167:0/2631595672 conn(0x7f75b80cd8f0 msgr2=0x7f75b8160100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:20.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.926+0000 7f75bee00640 1 -- 172.21.15.167:0/2631595672 shutdown_connections 2024-05-11T12:12:20.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:20.926+0000 7f75bee00640 1 -- 172.21.15.167:0/2631595672 wait complete. 2024-05-11T12:12:20.945 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-11T12:12:21.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.145+0000 7fe5e3a00640 1 Processor -- start 2024-05-11T12:12:21.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.146+0000 7fe5e3a00640 1 -- start start 2024-05-11T12:12:21.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.147+0000 7fe5e3a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe5dc0d18e0 0x7fe5dc06e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:21.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.147+0000 7fe5e3a00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe5dc06ec80 0x7fe5dc166c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:21.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.147+0000 7fe5e3a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5dc070aa0 con 0x7fe5dc1469a0 2024-05-11T12:12:21.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.147+0000 7fe5e3a00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fe5dc146d80 con 0x7fe5dc06ec80 2024-05-11T12:12:21.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.147+0000 7fe5e3a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fe5dc070870 con 0x7fe5dc0d18e0 2024-05-11T12:12:21.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.148+0000 7fe5e1a00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe5dc06ec80 0x7fe5dc166c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:21.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.148+0000 7fe5e1a00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe5dc06ec80 0x7fe5dc166c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39940/0 (socket says 172.21.15.167:39940) 2024-05-11T12:12:21.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.148+0000 7fe5e1a00640 1 -- 172.21.15.167:0/265454875 learned_addr learned my addr 172.21.15.167:0/265454875 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:21.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.148+0000 7fe5dbe00640 1 --2- 172.21.15.167:0/265454875 >> v2:172.21.15.204:3300/0 conn(0x7fe5dc0d18e0 0x7fe5dc06e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:21.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.148+0000 7fe5e1a00640 1 -- 172.21.15.167:0/265454875 >> v2:172.21.15.204:3300/0 conn(0x7fe5dc0d18e0 msgr2=0x7fe5dc06e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:21.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.148+0000 7fe5e1a00640 1 --2- 172.21.15.167:0/265454875 >> v2:172.21.15.204:3300/0 conn(0x7fe5dc0d18e0 0x7fe5dc06e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:21.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.148+0000 7fe5e1a00640 1 -- 172.21.15.167:0/265454875 >> v1:172.21.15.204:6789/0 conn(0x7fe5dc1469a0 legacy=0x7fe5dc06d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:21.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.148+0000 7fe5e1a00640 1 -- 172.21.15.167:0/265454875 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5dc070d80 con 0x7fe5dc06ec80 2024-05-11T12:12:21.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.150+0000 7fe5e1a00640 1 --2- 172.21.15.167:0/265454875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5dc06ec80 0x7fe5dc166c90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc0b7370 tx=0x7fe5d402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae9d76b18d43e41c server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:21.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.150+0000 7fe5db400640 1 -- 172.21.15.167:0/265454875 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4037070 con 0x7fe5dc06ec80 2024-05-11T12:12:21.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.150+0000 7fe5db400640 1 -- 172.21.15.167:0/265454875 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5d4032440 con 0x7fe5dc06ec80 2024-05-11T12:12:21.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.150+0000 7fe5db400640 1 -- 172.21.15.167:0/265454875 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4032810 con 0x7fe5dc06ec80 2024-05-11T12:12:21.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.151+0000 7fe5e3a00640 1 -- 172.21.15.167:0/265454875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5dc06ec80 msgr2=0x7fe5dc166c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:21.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.151+0000 7fe5e3a00640 1 --2- 172.21.15.167:0/265454875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5dc06ec80 0x7fe5dc166c90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc0b7370 tx=0x7fe5d402a270 comp rx=0 tx=0).stop 2024-05-11T12:12:21.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.151+0000 7fe5e3a00640 1 -- 172.21.15.167:0/265454875 shutdown_connections 2024-05-11T12:12:21.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.151+0000 7fe5e3a00640 1 --2- 172.21.15.167:0/265454875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5dc06ec80 0x7fe5dc166c90 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:21.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.151+0000 7fe5e3a00640 1 --2- 172.21.15.167:0/265454875 >> v2:172.21.15.204:3300/0 conn(0x7fe5dc0d18e0 0x7fe5dc06e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:21.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.151+0000 7fe5e3a00640 1 -- 172.21.15.167:0/265454875 >> 172.21.15.167:0/265454875 conn(0x7fe5dc0d2010 msgr2=0x7fe5dc0d2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:21.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.151+0000 7fe5e3a00640 1 -- 172.21.15.167:0/265454875 shutdown_connections 2024-05-11T12:12:21.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.151+0000 7fe5e3a00640 1 -- 172.21.15.167:0/265454875 wait complete. 2024-05-11T12:12:21.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.153+0000 7fe5e3a00640 1 Processor -- start 2024-05-11T12:12:21.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.153+0000 7fe5e3a00640 1 -- start start 2024-05-11T12:12:21.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.154+0000 7fe5e3a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe5dc06ec80 0x7fe5dc0afea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:21.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.154+0000 7fe5e3a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5dc0d18e0 0x7fe5dc0b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:21.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.154+0000 7fe5e3a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5dc1469a0 0x7fe5dc0ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:21.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.154+0000 7fe5e3a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc071340 con 0x7fe5dc0d18e0 2024-05-11T12:12:21.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.154+0000 7fe5e3a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc0716b0 con 0x7fe5dc1469a0 2024-05-11T12:12:21.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.154+0000 7fe5e3a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe5dc071b40 con 0x7fe5dc06ec80 2024-05-11T12:12:21.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.154+0000 7fe5dbe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5dc0d18e0 0x7fe5dc0b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:21.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.154+0000 7fe5dbe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5dc0d18e0 0x7fe5dc0b03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39948/0 (socket says 172.21.15.167:39948) 2024-05-11T12:12:21.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.154+0000 7fe5dbe00640 1 -- 172.21.15.167:0/1233924051 learned_addr learned my addr 172.21.15.167:0/1233924051 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:21.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.155+0000 7fe5e1a00640 1 --2- 172.21.15.167:0/1233924051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5dc1469a0 0x7fe5dc0ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:21.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.155+0000 7fe5e1000640 1 --2- 172.21.15.167:0/1233924051 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe5dc06ec80 0x7fe5dc0afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:21.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.155+0000 7fe5dbe00640 1 -- 172.21.15.167:0/1233924051 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe5dc06ec80 msgr2=0x7fe5dc0afea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:21.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.155+0000 7fe5dbe00640 1 --2- 172.21.15.167:0/1233924051 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe5dc06ec80 0x7fe5dc0afea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:21.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.155+0000 7fe5dbe00640 1 -- 172.21.15.167:0/1233924051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5dc1469a0 msgr2=0x7fe5dc0ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:21.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.155+0000 7fe5dbe00640 1 --2- 172.21.15.167:0/1233924051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5dc1469a0 0x7fe5dc0ccc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:21.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.155+0000 7fe5dbe00640 1 -- 172.21.15.167:0/1233924051 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5dc0cd9c0 con 0x7fe5dc0d18e0 2024-05-11T12:12:21.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.155+0000 7fe5e1a00640 1 --2- 172.21.15.167:0/1233924051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5dc1469a0 0x7fe5dc0ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:21.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.155+0000 7fe5dbe00640 1 --2- 172.21.15.167:0/1233924051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5dc0d18e0 0x7fe5dc0b03e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4007ec0 tx=0x7fe5c400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:21.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.155+0000 7fe5d8c00640 1 -- 172.21.15.167:0/1233924051 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c4014620 con 0x7fe5dc0d18e0 2024-05-11T12:12:21.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.155+0000 7fe5d8c00640 1 -- 172.21.15.167:0/1233924051 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5c4019070 con 0x7fe5dc0d18e0 2024-05-11T12:12:21.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.155+0000 7fe5d8c00640 1 -- 172.21.15.167:0/1233924051 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c4014a20 con 0x7fe5dc0d18e0 2024-05-11T12:12:21.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.155+0000 7fe5e3a00640 1 -- 172.21.15.167:0/1233924051 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5dc0cdca0 con 0x7fe5dc0d18e0 2024-05-11T12:12:21.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.155+0000 7fe5e3a00640 1 -- 172.21.15.167:0/1233924051 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5dc138df0 con 0x7fe5dc0d18e0 2024-05-11T12:12:21.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.157+0000 7fe5e3a00640 1 -- 172.21.15.167:0/1233924051 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5dc0ce1c0 con 0x7fe5dc0d18e0 2024-05-11T12:12:21.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.158+0000 7fe5d8c00640 1 -- 172.21.15.167:0/1233924051 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe5c4014bc0 con 0x7fe5dc0d18e0 2024-05-11T12:12:21.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.159+0000 7fe5d8c00640 1 --2- 172.21.15.167:0/1233924051 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe5b80762d0 0x7fe5b8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:21.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.159+0000 7fe5d8c00640 1 -- 172.21.15.167:0/1233924051 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7fe5c4098120 con 0x7fe5dc0d18e0 2024-05-11T12:12:21.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.159+0000 7fe5e1000640 1 --2- 172.21.15.167:0/1233924051 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe5b80762d0 0x7fe5b8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:21.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.160+0000 7fe5e1000640 1 --2- 172.21.15.167:0/1233924051 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe5b80762d0 0x7fe5b8078790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc0015a0 tx=0x7fe5cc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:21.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.163+0000 7fe5d8c00640 1 -- 172.21.15.167:0/1233924051 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe5dc0ce1c0 con 0x7fe5dc0d18e0 2024-05-11T12:12:21.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.404+0000 7fe5e3a00640 1 -- 172.21.15.167:0/1233924051 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe5dc069bc0 con 0x7fe5dc0d18e0 2024-05-11T12:12:21.405 INFO:tasks.ceph.mon.a.smithi167.stderr:2024-05-11T12:12:21.405+0000 7f3aac000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-11T12:12:21.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.408+0000 7fe5d8c00640 1 -- 172.21.15.167:0/1233924051 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) ==== 161+0+0 (secure 0 0 0) 0x7fe5c40618e0 con 0x7fe5dc0d18e0 2024-05-11T12:12:21.408 INFO:teuthology.orchestra.run.smithi167.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-11T12:12:21.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.411+0000 7fe5e3a00640 1 -- 172.21.15.167:0/1233924051 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe5b80762d0 msgr2=0x7fe5b8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:21.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.411+0000 7fe5e3a00640 1 --2- 172.21.15.167:0/1233924051 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe5b80762d0 0x7fe5b8078790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc0015a0 tx=0x7fe5cc009040 comp rx=0 tx=0).stop 2024-05-11T12:12:21.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.411+0000 7fe5e3a00640 1 -- 172.21.15.167:0/1233924051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5dc0d18e0 msgr2=0x7fe5dc0b03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:21.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.411+0000 7fe5e3a00640 1 --2- 172.21.15.167:0/1233924051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5dc0d18e0 0x7fe5dc0b03e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4007ec0 tx=0x7fe5c400e4b0 comp rx=0 tx=0).stop 2024-05-11T12:12:21.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.411+0000 7fe5e3a00640 1 -- 172.21.15.167:0/1233924051 shutdown_connections 2024-05-11T12:12:21.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.411+0000 7fe5e3a00640 1 --2- 172.21.15.167:0/1233924051 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe5b80762d0 0x7fe5b8078790 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:21.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.411+0000 7fe5e3a00640 1 --2- 172.21.15.167:0/1233924051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5dc1469a0 0x7fe5dc0ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:21.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.411+0000 7fe5e3a00640 1 --2- 172.21.15.167:0/1233924051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5dc0d18e0 0x7fe5dc0b03e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:21.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.411+0000 7fe5e3a00640 1 --2- 172.21.15.167:0/1233924051 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe5dc06ec80 0x7fe5dc0afea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:21.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.411+0000 7fe5e3a00640 1 -- 172.21.15.167:0/1233924051 >> 172.21.15.167:0/1233924051 conn(0x7fe5dc0d2010 msgr2=0x7fe5dc073330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:21.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.411+0000 7fe5e3a00640 1 -- 172.21.15.167:0/1233924051 shutdown_connections 2024-05-11T12:12:21.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.411+0000 7fe5e3a00640 1 -- 172.21.15.167:0/1233924051 wait complete. 2024-05-11T12:12:21.431 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-11T12:12:21.439 INFO:tasks.ceph.mds.d.smithi204.stderr: -2616> 2024-05-11T12:12:01.349+0000 7f8452800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-11T12:12:21.513 INFO:tasks.ceph.mds.d.smithi204.stdout:starting mds.d at 2024-05-11T12:12:21.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.613+0000 7efcaf600640 1 Processor -- start 2024-05-11T12:12:21.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.613+0000 7efcaf600640 1 -- start start 2024-05-11T12:12:21.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.615+0000 7efcaf600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efca815ac80 0x7efca8074260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:21.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.615+0000 7efcaf600640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7efca8161380 0x7efca8072950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:21.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.615+0000 7efcaf600640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efca80b65e0 con 0x7efca80b3d50 2024-05-11T12:12:21.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.615+0000 7efcaf600640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7efca806e3c0 con 0x7efca8161380 2024-05-11T12:12:21.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.615+0000 7efcaf600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7efca815b060 con 0x7efca815ac80 2024-05-11T12:12:21.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.615+0000 7efcad600640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7efca8161380 0x7efca8072950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:21.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.615+0000 7efc9fe00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7efca80b3d50 0x7efca8160c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:54536/0 (socket says 172.21.15.167:54536) 2024-05-11T12:12:21.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.615+0000 7efc9fe00640 1 -- 172.21.15.167:0/317871839 learned_addr learned my addr 172.21.15.167:0/317871839 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:21.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.616+0000 7efcacc00640 1 --2- 172.21.15.167:0/317871839 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efca815ac80 0x7efca8074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:21.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.616+0000 7efcad600640 1 -- 172.21.15.167:0/317871839 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efca815ac80 msgr2=0x7efca8074260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:21.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.616+0000 7efcad600640 1 --2- 172.21.15.167:0/317871839 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efca815ac80 0x7efca8074260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:21.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.616+0000 7efcad600640 1 -- 172.21.15.167:0/317871839 >> v1:172.21.15.167:6789/0 conn(0x7efca80b3d50 legacy=0x7efca8160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:21.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.616+0000 7efcad600640 1 -- 172.21.15.167:0/317871839 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efca80b68c0 con 0x7efca8161380 2024-05-11T12:12:21.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.616+0000 7efca7e00640 1 -- 172.21.15.167:0/317871839 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4049256338 0 0) 0x7efca80b65e0 con 0x7efca80b3d50 2024-05-11T12:12:21.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.617+0000 7efcad600640 1 --2- 172.21.15.167:0/317871839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efca8161380 0x7efca8072950 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7efca0009d90 tx=0x7efca002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8896e8aeb4b2e597 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:21.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.617+0000 7efca7e00640 1 -- 172.21.15.167:0/317871839 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca0007740 con 0x7efca8161380 2024-05-11T12:12:21.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.618+0000 7efca7e00640 1 -- 172.21.15.167:0/317871839 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efca00078e0 con 0x7efca8161380 2024-05-11T12:12:21.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.618+0000 7efca7e00640 1 -- 172.21.15.167:0/317871839 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca0007cb0 con 0x7efca8161380 2024-05-11T12:12:21.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.618+0000 7efcaf600640 1 -- 172.21.15.167:0/317871839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efca8161380 msgr2=0x7efca8072950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:21.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.618+0000 7efcaf600640 1 --2- 172.21.15.167:0/317871839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efca8161380 0x7efca8072950 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7efca0009d90 tx=0x7efca002a660 comp rx=0 tx=0).stop 2024-05-11T12:12:21.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.618+0000 7efcaf600640 1 -- 172.21.15.167:0/317871839 shutdown_connections 2024-05-11T12:12:21.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.618+0000 7efcaf600640 1 --2- 172.21.15.167:0/317871839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efca8161380 0x7efca8072950 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:21.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.618+0000 7efcaf600640 1 --2- 172.21.15.167:0/317871839 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efca815ac80 0x7efca8074260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:21.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.618+0000 7efcaf600640 1 -- 172.21.15.167:0/317871839 >> 172.21.15.167:0/317871839 conn(0x7efca806c450 msgr2=0x7efca806c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:21.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.618+0000 7efcaf600640 1 -- 172.21.15.167:0/317871839 shutdown_connections 2024-05-11T12:12:21.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.618+0000 7efcaf600640 1 -- 172.21.15.167:0/317871839 wait complete. 2024-05-11T12:12:21.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.620+0000 7efcaf600640 1 Processor -- start 2024-05-11T12:12:21.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.620+0000 7efcaf600640 1 -- start start 2024-05-11T12:12:21.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.621+0000 7efcaf600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efca80b3d50 0x7efca80aa410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:21.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.621+0000 7efcaf600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efca815ac80 0x7efca81f2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:21.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.621+0000 7efcaf600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efca8161380 0x7efca81f5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:21.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.621+0000 7efcaf600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efca80b44f0 con 0x7efca815ac80 2024-05-11T12:12:21.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.621+0000 7efcaf600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7efca80b41b0 con 0x7efca8161380 2024-05-11T12:12:21.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.621+0000 7efcaf600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7efca8072e90 con 0x7efca80b3d50 2024-05-11T12:12:21.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.621+0000 7efc9fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efca815ac80 0x7efca81f2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:21.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.621+0000 7efc9fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efca815ac80 0x7efca81f2fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39958/0 (socket says 172.21.15.167:39958) 2024-05-11T12:12:21.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.621+0000 7efc9fe00640 1 -- 172.21.15.167:0/1746144911 learned_addr learned my addr 172.21.15.167:0/1746144911 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:21.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.622+0000 7efc9fe00640 1 -- 172.21.15.167:0/1746144911 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efca80b3d50 msgr2=0x7efca80aa410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:21.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.622+0000 7efc9fe00640 1 --2- 172.21.15.167:0/1746144911 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efca80b3d50 0x7efca80aa410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:21.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.622+0000 7efc9fe00640 1 -- 172.21.15.167:0/1746144911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efca8161380 msgr2=0x7efca81f5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:21.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.622+0000 7efc9fe00640 1 --2- 172.21.15.167:0/1746144911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efca8161380 0x7efca81f5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:21.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.622+0000 7efc9fe00640 1 -- 172.21.15.167:0/1746144911 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efca80ab070 con 0x7efca815ac80 2024-05-11T12:12:21.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.622+0000 7efc9fe00640 1 --2- 172.21.15.167:0/1746144911 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efca815ac80 0x7efca81f2fd0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7efc9000c490 tx=0x7efc9000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:21.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.622+0000 7efca5600640 1 -- 172.21.15.167:0/1746144911 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc90018070 con 0x7efca815ac80 2024-05-11T12:12:21.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.622+0000 7efca5600640 1 -- 172.21.15.167:0/1746144911 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc90007500 con 0x7efca815ac80 2024-05-11T12:12:21.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.622+0000 7efcaf600640 1 -- 172.21.15.167:0/1746144911 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efca81e9010 con 0x7efca815ac80 2024-05-11T12:12:21.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.622+0000 7efca5600640 1 -- 172.21.15.167:0/1746144911 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc90004a20 con 0x7efca815ac80 2024-05-11T12:12:21.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.623+0000 7efcaf600640 1 -- 172.21.15.167:0/1746144911 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efca81e95d0 con 0x7efca815ac80 2024-05-11T12:12:21.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.624+0000 7efcaf600640 1 -- 172.21.15.167:0/1746144911 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efca8069520 con 0x7efca815ac80 2024-05-11T12:12:21.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.625+0000 7efca5600640 1 -- 172.21.15.167:0/1746144911 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7efc9001b070 con 0x7efca815ac80 2024-05-11T12:12:21.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.626+0000 7efca5600640 1 --2- 172.21.15.167:0/1746144911 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7efc840762d0 0x7efc84078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:21.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.626+0000 7efca5600640 1 -- 172.21.15.167:0/1746144911 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6187+0+0 (secure 0 0 0) 0x7efc90096ad0 con 0x7efca815ac80 2024-05-11T12:12:21.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.626+0000 7efcacc00640 1 --2- 172.21.15.167:0/1746144911 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7efc840762d0 0x7efc84078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:21.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.626+0000 7efcacc00640 1 --2- 172.21.15.167:0/1746144911 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7efc840762d0 0x7efc84078790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7efc980098f0 tx=0x7efc98008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:21.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.630+0000 7efca5600640 1 -- 172.21.15.167:0/1746144911 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7efc90060240 con 0x7efca815ac80 2024-05-11T12:12:21.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:21.857+0000 7efcaf600640 1 -- 172.21.15.167:0/1746144911 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7efca80b7000 con 0x7efca815ac80 2024-05-11T12:12:22.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.412+0000 7efca5600640 1 -- 172.21.15.167:0/1746144911 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) ==== 104+0+0 (secure 0 0 0) 0x7efc9005fb40 con 0x7efca815ac80 2024-05-11T12:12:22.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.416+0000 7efcaf600640 1 -- 172.21.15.167:0/1746144911 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7efc840762d0 msgr2=0x7efc84078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:22.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.416+0000 7efcaf600640 1 --2- 172.21.15.167:0/1746144911 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7efc840762d0 0x7efc84078790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7efc980098f0 tx=0x7efc98008040 comp rx=0 tx=0).stop 2024-05-11T12:12:22.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.416+0000 7efcaf600640 1 -- 172.21.15.167:0/1746144911 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efca815ac80 msgr2=0x7efca81f2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:22.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.416+0000 7efcaf600640 1 --2- 172.21.15.167:0/1746144911 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efca815ac80 0x7efca81f2fd0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7efc9000c490 tx=0x7efc9000c950 comp rx=0 tx=0).stop 2024-05-11T12:12:22.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.416+0000 7efcaf600640 1 -- 172.21.15.167:0/1746144911 shutdown_connections 2024-05-11T12:12:22.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.416+0000 7efcaf600640 1 --2- 172.21.15.167:0/1746144911 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7efc840762d0 0x7efc84078790 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:22.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.416+0000 7efcaf600640 1 --2- 172.21.15.167:0/1746144911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efca8161380 0x7efca81f5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:22.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.416+0000 7efcaf600640 1 --2- 172.21.15.167:0/1746144911 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efca815ac80 0x7efca81f2fd0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:22.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.416+0000 7efcaf600640 1 --2- 172.21.15.167:0/1746144911 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efca80b3d50 0x7efca80aa410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:22.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.416+0000 7efcaf600640 1 -- 172.21.15.167:0/1746144911 >> 172.21.15.167:0/1746144911 conn(0x7efca806c450 msgr2=0x7efca8145140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:22.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.416+0000 7efcaf600640 1 -- 172.21.15.167:0/1746144911 shutdown_connections 2024-05-11T12:12:22.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.416+0000 7efcaf600640 1 -- 172.21.15.167:0/1746144911 wait complete. 2024-05-11T12:12:22.439 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-11T12:12:22.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.645+0000 7feefbc00640 1 Processor -- start 2024-05-11T12:12:22.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.645+0000 7feefbc00640 1 -- start start 2024-05-11T12:12:22.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.647+0000 7feefbc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feef40b5290 0x7feef406f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:22.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.647+0000 7feefbc00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feef406fb40 con 0x7feef406da20 2024-05-11T12:12:22.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.647+0000 7feefbc00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feef406fe20 con 0x7feef406ed60 2024-05-11T12:12:22.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.647+0000 7feefbc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7feef40703e0 con 0x7feef40b5290 2024-05-11T12:12:22.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.647+0000 7feef9c00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7feef406da20 0x7feef416d280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:54544/0 (socket says 172.21.15.167:54544) 2024-05-11T12:12:22.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.647+0000 7feef9c00640 1 -- 172.21.15.167:0/3794869521 learned_addr learned my addr 172.21.15.167:0/3794869521 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:22.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.647+0000 7feef9200640 1 --2- 172.21.15.167:0/3794869521 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feef40b5290 0x7feef406f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:22.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.648+0000 7feef3400640 1 -- 172.21.15.167:0/3794869521 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 751656749 0 0) 0x7feef406fb40 con 0x7feef406da20 2024-05-11T12:12:22.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.648+0000 7feef3400640 1 -- 172.21.15.167:0/3794869521 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feed8003520 con 0x7feef406da20 2024-05-11T12:12:22.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.648+0000 7feef3400640 1 -- 172.21.15.167:0/3794869521 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2300301999 0 0) 0x7feed8003520 con 0x7feef406da20 2024-05-11T12:12:22.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.648+0000 7feef3400640 1 -- 172.21.15.167:0/3794869521 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feef40b5290 msgr2=0x7feef406f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:22.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.648+0000 7feef3400640 1 --2- 172.21.15.167:0/3794869521 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feef40b5290 0x7feef406f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:22.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.648+0000 7feef3400640 1 -- 172.21.15.167:0/3794869521 >> v1:172.21.15.204:6789/0 conn(0x7feef406ed60 legacy=0x7feef40b5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:22.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.648+0000 7feef3400640 1 -- 172.21.15.167:0/3794869521 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feef4070100 con 0x7feef406da20 2024-05-11T12:12:22.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.648+0000 7feef3400640 1 -- 172.21.15.167:0/3794869521 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7feeec001710 con 0x7feef406da20 2024-05-11T12:12:22.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.648+0000 7feef3400640 1 -- 172.21.15.167:0/3794869521 >> v1:172.21.15.167:6789/0 conn(0x7feef406da20 legacy=0x7feef416d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:22.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.648+0000 7feef3400640 1 --2- 172.21.15.167:0/3794869521 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feef40b5290 0x7feed80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:22.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.648+0000 7feef3400640 1 --2- 172.21.15.167:0/3794869521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feed8003f70 0x7feed8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:22.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.648+0000 7feef3400640 1 --2- 172.21.15.167:0/3794869521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feed8004d40 0x7feed80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:22.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.648+0000 7feef3400640 1 -- 172.21.15.167:0/3794869521 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7feef40703e0 con 0x7feed8004d40 2024-05-11T12:12:22.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.648+0000 7feef3400640 1 -- 172.21.15.167:0/3794869521 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7feed800bad0 con 0x7feed8003f70 2024-05-11T12:12:22.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.648+0000 7feef3400640 1 -- 172.21.15.167:0/3794869521 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7feed800bcd0 con 0x7feef40b5290 2024-05-11T12:12:22.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.649+0000 7feef9c00640 1 --2- 172.21.15.167:0/3794869521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feed8004d40 0x7feed80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:22.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.649+0000 7feef9200640 1 --2- 172.21.15.167:0/3794869521 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feef40b5290 0x7feed80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:22.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.649+0000 7feef3e00640 1 --2- 172.21.15.167:0/3794869521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feed8003f70 0x7feed8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:22.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.649+0000 7feef9c00640 1 -- 172.21.15.167:0/3794869521 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feef40b5290 msgr2=0x7feed80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:22.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.649+0000 7feef9c00640 1 --2- 172.21.15.167:0/3794869521 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feef40b5290 0x7feed80039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:22.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.649+0000 7feef9c00640 1 -- 172.21.15.167:0/3794869521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feed8003f70 msgr2=0x7feed8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:22.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.649+0000 7feef9c00640 1 --2- 172.21.15.167:0/3794869521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feed8003f70 0x7feed8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:22.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.649+0000 7feef9c00640 1 -- 172.21.15.167:0/3794869521 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed8003520 con 0x7feed8004d40 2024-05-11T12:12:22.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.650+0000 7feef9c00640 1 --2- 172.21.15.167:0/3794869521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feed8004d40 0x7feed80051c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7feef40cd700 tx=0x7feed8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=681b38040589f030 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:22.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.650+0000 7feef3400640 1 -- 172.21.15.167:0/3794869521 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec007d70 con 0x7feed8004d40 2024-05-11T12:12:22.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.650+0000 7feef3400640 1 -- 172.21.15.167:0/3794869521 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeec005af0 con 0x7feed8004d40 2024-05-11T12:12:22.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.650+0000 7feef3400640 1 -- 172.21.15.167:0/3794869521 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec007070 con 0x7feed8004d40 2024-05-11T12:12:22.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.651+0000 7feefbc00640 1 -- 172.21.15.167:0/3794869521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feed8004d40 msgr2=0x7feed80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:22.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.651+0000 7feefbc00640 1 --2- 172.21.15.167:0/3794869521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feed8004d40 0x7feed80051c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7feef40cd700 tx=0x7feed8005830 comp rx=0 tx=0).stop 2024-05-11T12:12:22.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.651+0000 7feef9c00640 1 -- 172.21.15.167:0/3794869521 reap_dead start 2024-05-11T12:12:22.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.651+0000 7feefbc00640 1 -- 172.21.15.167:0/3794869521 shutdown_connections 2024-05-11T12:12:22.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.651+0000 7feefbc00640 1 -- 172.21.15.167:0/3794869521 >> 172.21.15.167:0/3794869521 conn(0x7feef40d0c70 msgr2=0x7feef40d1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:22.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.651+0000 7feefbc00640 1 -- 172.21.15.167:0/3794869521 shutdown_connections 2024-05-11T12:12:22.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.651+0000 7feefbc00640 1 -- 172.21.15.167:0/3794869521 wait complete. 2024-05-11T12:12:22.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.653+0000 7feefbc00640 1 Processor -- start 2024-05-11T12:12:22.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.653+0000 7feefbc00640 1 -- start start 2024-05-11T12:12:22.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.654+0000 7feefbc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feed8004d40 0x7feef4163100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:22.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.654+0000 7feefbc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feef40aaf00 0x7feef40ab380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:22.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.654+0000 7feefbc00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feef4073ac0 0x7feef4073f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:22.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.654+0000 7feefbc00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7feef40b5d80 con 0x7feef4073ac0 2024-05-11T12:12:22.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.654+0000 7feefbc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7feef406c580 con 0x7feef40aaf00 2024-05-11T12:12:22.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.654+0000 7feefbc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7feef40b5f00 con 0x7feed8004d40 2024-05-11T12:12:22.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.654+0000 7feef9c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feef4073ac0 0x7feef4073f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:22.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.654+0000 7feef9c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feef4073ac0 0x7feef4073f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39984/0 (socket says 172.21.15.167:39984) 2024-05-11T12:12:22.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.654+0000 7feef9c00640 1 -- 172.21.15.167:0/2224905909 learned_addr learned my addr 172.21.15.167:0/2224905909 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:22.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.655+0000 7feef3e00640 1 --2- 172.21.15.167:0/2224905909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feef40aaf00 0x7feef40ab380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:22.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.655+0000 7feef9c00640 1 -- 172.21.15.167:0/2224905909 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feed8004d40 msgr2=0x7feef4163100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:22.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.655+0000 7feef9c00640 1 --2- 172.21.15.167:0/2224905909 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feed8004d40 0x7feef4163100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:22.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.655+0000 7feef9c00640 1 -- 172.21.15.167:0/2224905909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feef40aaf00 msgr2=0x7feef40ab380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:22.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.655+0000 7feef9c00640 1 --2- 172.21.15.167:0/2224905909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feef40aaf00 0x7feef40ab380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:22.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.655+0000 7feef9c00640 1 -- 172.21.15.167:0/2224905909 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feef4163640 con 0x7feef4073ac0 2024-05-11T12:12:22.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.655+0000 7feef9c00640 1 --2- 172.21.15.167:0/2224905909 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feef4073ac0 0x7feef4073f70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7feeec02ad70 tx=0x7feef40702d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:22.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.655+0000 7feef0c00640 1 -- 172.21.15.167:0/2224905909 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec043070 con 0x7feef4073ac0 2024-05-11T12:12:22.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.655+0000 7feef0c00640 1 -- 172.21.15.167:0/2224905909 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeec038040 con 0x7feef4073ac0 2024-05-11T12:12:22.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.655+0000 7feefbc00640 1 -- 172.21.15.167:0/2224905909 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feef41f93c0 con 0x7feef4073ac0 2024-05-11T12:12:22.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.655+0000 7feef0c00640 1 -- 172.21.15.167:0/2224905909 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec001cb0 con 0x7feef4073ac0 2024-05-11T12:12:22.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.656+0000 7feefbc00640 1 -- 172.21.15.167:0/2224905909 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feef41f9980 con 0x7feef4073ac0 2024-05-11T12:12:22.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.657+0000 7feefbc00640 1 -- 172.21.15.167:0/2224905909 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feef4069590 con 0x7feef4073ac0 2024-05-11T12:12:22.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.658+0000 7feef0c00640 1 -- 172.21.15.167:0/2224905909 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7feeec007270 con 0x7feef4073ac0 2024-05-11T12:12:22.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.659+0000 7feef0c00640 1 --2- 172.21.15.167:0/2224905909 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7feed0076300 0x7feed00787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:22.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.659+0000 7feef0c00640 1 -- 172.21.15.167:0/2224905909 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6187+0+0 (secure 0 0 0) 0x7feeec0b7080 con 0x7feef4073ac0 2024-05-11T12:12:22.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.659+0000 7feef9200640 1 --2- 172.21.15.167:0/2224905909 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7feed0076300 0x7feed00787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:22.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.660+0000 7feef9200640 1 --2- 172.21.15.167:0/2224905909 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7feed0076300 0x7feed00787c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7feef40cd3a0 tx=0x7feee4009750 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:22.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.663+0000 7feef0c00640 1 -- 172.21.15.167:0/2224905909 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7feef4163640 con 0x7feef4073ac0 2024-05-11T12:12:22.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:22.873+0000 7feefbc00640 1 -- 172.21.15.167:0/2224905909 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7feef40b0a80 con 0x7feef4073ac0 2024-05-11T12:12:23.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.739+0000 7feef0c00640 1 -- 172.21.15.167:0/2224905909 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v43) ==== 181+0+0 (secure 0 0 0) 0x7feeec080080 con 0x7feef4073ac0 2024-05-11T12:12:23.739 INFO:teuthology.orchestra.run.smithi167.stderr:pool 'cephfs_metadata' removed 2024-05-11T12:12:23.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.743+0000 7feefbc00640 1 -- 172.21.15.167:0/2224905909 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7feed0076300 msgr2=0x7feed00787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:23.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.743+0000 7feefbc00640 1 --2- 172.21.15.167:0/2224905909 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7feed0076300 0x7feed00787c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7feef40cd3a0 tx=0x7feee4009750 comp rx=0 tx=0).stop 2024-05-11T12:12:23.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.743+0000 7feefbc00640 1 -- 172.21.15.167:0/2224905909 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feef4073ac0 msgr2=0x7feef4073f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:23.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.743+0000 7feefbc00640 1 --2- 172.21.15.167:0/2224905909 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feef4073ac0 0x7feef4073f70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7feeec02ad70 tx=0x7feef40702d0 comp rx=0 tx=0).stop 2024-05-11T12:12:23.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.744+0000 7feefbc00640 1 -- 172.21.15.167:0/2224905909 shutdown_connections 2024-05-11T12:12:23.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.744+0000 7feefbc00640 1 --2- 172.21.15.167:0/2224905909 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feef4073ac0 0x7feef4073f70 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:23.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.744+0000 7feefbc00640 1 --2- 172.21.15.167:0/2224905909 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7feed0076300 0x7feed00787c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:23.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.744+0000 7feefbc00640 1 --2- 172.21.15.167:0/2224905909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feef40aaf00 0x7feef40ab380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:23.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.744+0000 7feefbc00640 1 --2- 172.21.15.167:0/2224905909 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feed8004d40 0x7feef4163100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:23.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.744+0000 7feefbc00640 1 -- 172.21.15.167:0/2224905909 >> 172.21.15.167:0/2224905909 conn(0x7feef40d0c70 msgr2=0x7feef40724c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:23.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.744+0000 7feefbc00640 1 -- 172.21.15.167:0/2224905909 shutdown_connections 2024-05-11T12:12:23.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.744+0000 7feefbc00640 1 -- 172.21.15.167:0/2224905909 wait complete. 2024-05-11T12:12:23.769 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-11T12:12:23.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.972+0000 7efc68a00640 1 Processor -- start 2024-05-11T12:12:23.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.973+0000 7efc68a00640 1 -- start start 2024-05-11T12:12:23.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.974+0000 7efc68a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc6415cb90 0x7efc6415fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:23.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.974+0000 7efc68a00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7efc6406db70 0x7efc64072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:23.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.974+0000 7efc68a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc64160340 con 0x7efc6415cf70 2024-05-11T12:12:23.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.974+0000 7efc68a00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7efc641600e0 con 0x7efc6406db70 2024-05-11T12:12:23.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.974+0000 7efc68a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7efc64160620 con 0x7efc6415cb90 2024-05-11T12:12:23.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.974+0000 7efc63200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7efc6406db70 0x7efc64072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:23.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.974+0000 7efc63200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7efc6406db70 0x7efc64072680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:40000/0 (socket says 172.21.15.167:40000) 2024-05-11T12:12:23.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.974+0000 7efc63200640 1 -- 172.21.15.167:0/3442392894 learned_addr learned my addr 172.21.15.167:0/3442392894 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:23.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.975+0000 7efc63200640 1 -- 172.21.15.167:0/3442392894 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc6415cb90 msgr2=0x7efc6415fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:23.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.975+0000 7efc63200640 1 --2- 172.21.15.167:0/3442392894 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc6415cb90 0x7efc6415fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:23.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.975+0000 7efc63200640 1 -- 172.21.15.167:0/3442392894 >> v1:172.21.15.204:6789/0 conn(0x7efc6415cf70 legacy=0x7efc6406d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:23.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.975+0000 7efc63200640 1 -- 172.21.15.167:0/3442392894 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc64160900 con 0x7efc6406db70 2024-05-11T12:12:23.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.976+0000 7efc63200640 1 --2- 172.21.15.167:0/3442392894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc6406db70 0x7efc64072680 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7efc5c009e50 tx=0x7efc5c02c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7daef7125d03ac03 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:23.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.976+0000 7efc61e00640 1 -- 172.21.15.167:0/3442392894 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5c039070 con 0x7efc6406db70 2024-05-11T12:12:23.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.976+0000 7efc61e00640 1 -- 172.21.15.167:0/3442392894 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc5c034680 con 0x7efc6406db70 2024-05-11T12:12:23.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.976+0000 7efc61e00640 1 -- 172.21.15.167:0/3442392894 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5c034a50 con 0x7efc6406db70 2024-05-11T12:12:23.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.977+0000 7efc68a00640 1 -- 172.21.15.167:0/3442392894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc6406db70 msgr2=0x7efc64072680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:23.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.977+0000 7efc68a00640 1 --2- 172.21.15.167:0/3442392894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc6406db70 0x7efc64072680 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7efc5c009e50 tx=0x7efc5c02c890 comp rx=0 tx=0).stop 2024-05-11T12:12:23.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.977+0000 7efc68a00640 1 -- 172.21.15.167:0/3442392894 shutdown_connections 2024-05-11T12:12:23.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.977+0000 7efc68a00640 1 --2- 172.21.15.167:0/3442392894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc6406db70 0x7efc64072680 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:23.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.977+0000 7efc68a00640 1 --2- 172.21.15.167:0/3442392894 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc6415cb90 0x7efc6415fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:23.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.977+0000 7efc68a00640 1 -- 172.21.15.167:0/3442392894 >> 172.21.15.167:0/3442392894 conn(0x7efc640b32a0 msgr2=0x7efc640d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:23.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.977+0000 7efc68a00640 1 -- 172.21.15.167:0/3442392894 shutdown_connections 2024-05-11T12:12:23.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.977+0000 7efc68a00640 1 -- 172.21.15.167:0/3442392894 wait complete. 2024-05-11T12:12:23.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.979+0000 7efc68a00640 1 Processor -- start 2024-05-11T12:12:23.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.979+0000 7efc68a00640 1 -- start start 2024-05-11T12:12:23.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.980+0000 7efc68a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc6406db70 0x7efc641578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:23.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.980+0000 7efc68a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc6415cb90 0x7efc641556a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:23.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.980+0000 7efc68a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc6415cf70 0x7efc6415b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:23.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.980+0000 7efc68a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efc6415d570 con 0x7efc6415cf70 2024-05-11T12:12:23.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.980+0000 7efc68a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7efc6415d3f0 con 0x7efc6406db70 2024-05-11T12:12:23.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.980+0000 7efc68a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7efc64072bc0 con 0x7efc6415cb90 2024-05-11T12:12:23.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.980+0000 7efc63200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc6415cf70 0x7efc6415b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:23.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.981+0000 7efc63200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc6415cf70 0x7efc6415b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:40002/0 (socket says 172.21.15.167:40002) 2024-05-11T12:12:23.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.981+0000 7efc63200640 1 -- 172.21.15.167:0/1129462363 learned_addr learned my addr 172.21.15.167:0/1129462363 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:23.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.981+0000 7efc63200640 1 -- 172.21.15.167:0/1129462363 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc6415cb90 msgr2=0x7efc641556a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:23.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.981+0000 7efc62800640 1 --2- 172.21.15.167:0/1129462363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc6406db70 0x7efc641578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:23.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.981+0000 7efc59e00640 1 --2- 172.21.15.167:0/1129462363 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc6415cb90 0x7efc641556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:23.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.981+0000 7efc63200640 1 --2- 172.21.15.167:0/1129462363 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc6415cb90 0x7efc641556a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:23.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.981+0000 7efc63200640 1 -- 172.21.15.167:0/1129462363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc6406db70 msgr2=0x7efc641578b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:23.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.981+0000 7efc63200640 1 --2- 172.21.15.167:0/1129462363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc6406db70 0x7efc641578b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:23.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.981+0000 7efc63200640 1 -- 172.21.15.167:0/1129462363 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc64156df0 con 0x7efc6415cf70 2024-05-11T12:12:23.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.981+0000 7efc62800640 1 --2- 172.21.15.167:0/1129462363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc6406db70 0x7efc641578b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:23.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.981+0000 7efc59e00640 1 --2- 172.21.15.167:0/1129462363 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc6415cb90 0x7efc641556a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:23.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.981+0000 7efc63200640 1 --2- 172.21.15.167:0/1129462363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc6415cf70 0x7efc6415b430 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7efc5c009e50 tx=0x7efc5c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:23.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.982+0000 7efc5b400640 1 -- 172.21.15.167:0/1129462363 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5c044070 con 0x7efc6415cf70 2024-05-11T12:12:23.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.982+0000 7efc5b400640 1 -- 172.21.15.167:0/1129462363 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc5c034b60 con 0x7efc6415cf70 2024-05-11T12:12:23.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.982+0000 7efc5b400640 1 -- 172.21.15.167:0/1129462363 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5c039040 con 0x7efc6415cf70 2024-05-11T12:12:23.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.982+0000 7efc68a00640 1 -- 172.21.15.167:0/1129462363 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc641fa7a0 con 0x7efc6415cf70 2024-05-11T12:12:23.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.982+0000 7efc68a00640 1 -- 172.21.15.167:0/1129462363 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc64157390 con 0x7efc6415cf70 2024-05-11T12:12:23.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.983+0000 7efc68a00640 1 -- 172.21.15.167:0/1129462363 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc6415fe80 con 0x7efc6415cf70 2024-05-11T12:12:23.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.984+0000 7efc5b400640 1 -- 172.21.15.167:0/1129462363 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7efc5c03e3d0 con 0x7efc6415cf70 2024-05-11T12:12:23.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.985+0000 7efc5b400640 1 --2- 172.21.15.167:0/1129462363 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7efc400762d0 0x7efc40078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:23.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.985+0000 7efc5b400640 1 -- 172.21.15.167:0/1129462363 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5749+0+0 (secure 0 0 0) 0x7efc5c0b7510 con 0x7efc6415cf70 2024-05-11T12:12:23.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.985+0000 7efc62800640 1 --2- 172.21.15.167:0/1129462363 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7efc400762d0 0x7efc40078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:23.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.986+0000 7efc62800640 1 --2- 172.21.15.167:0/1129462363 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7efc400762d0 0x7efc40078790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7efc54006fd0 tx=0x7efc54008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:23.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:23.989+0000 7efc5b400640 1 -- 172.21.15.167:0/1129462363 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7efc6415fe80 con 0x7efc6415cf70 2024-05-11T12:12:24.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.200+0000 7efc68a00640 1 -- 172.21.15.167:0/1129462363 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7efc640694b0 con 0x7efc6415cf70 2024-05-11T12:12:24.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.752+0000 7efc5b400640 1 -- 172.21.15.167:0/1129462363 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v44) ==== 169+0+0 (secure 0 0 0) 0x7efc5c080e30 con 0x7efc6415cf70 2024-05-11T12:12:24.752 INFO:teuthology.orchestra.run.smithi167.stderr:pool 'cephfs_data' removed 2024-05-11T12:12:24.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.756+0000 7efc68a00640 1 -- 172.21.15.167:0/1129462363 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7efc400762d0 msgr2=0x7efc40078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:24.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.756+0000 7efc68a00640 1 --2- 172.21.15.167:0/1129462363 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7efc400762d0 0x7efc40078790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7efc54006fd0 tx=0x7efc54008040 comp rx=0 tx=0).stop 2024-05-11T12:12:24.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.756+0000 7efc68a00640 1 -- 172.21.15.167:0/1129462363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc6415cf70 msgr2=0x7efc6415b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:24.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.756+0000 7efc68a00640 1 --2- 172.21.15.167:0/1129462363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc6415cf70 0x7efc6415b430 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7efc5c009e50 tx=0x7efc5c004a40 comp rx=0 tx=0).stop 2024-05-11T12:12:24.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.757+0000 7efc68a00640 1 -- 172.21.15.167:0/1129462363 shutdown_connections 2024-05-11T12:12:24.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.758+0000 7efc68a00640 1 --2- 172.21.15.167:0/1129462363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc6415cf70 0x7efc6415b430 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:24.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.758+0000 7efc68a00640 1 --2- 172.21.15.167:0/1129462363 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc6415cb90 0x7efc641556a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:24.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.758+0000 7efc68a00640 1 --2- 172.21.15.167:0/1129462363 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7efc400762d0 0x7efc40078790 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:24.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.758+0000 7efc68a00640 1 --2- 172.21.15.167:0/1129462363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc6406db70 0x7efc641578b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:24.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.758+0000 7efc68a00640 1 -- 172.21.15.167:0/1129462363 >> 172.21.15.167:0/1129462363 conn(0x7efc640b32a0 msgr2=0x7efc640b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:24.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.758+0000 7efc68a00640 1 -- 172.21.15.167:0/1129462363 shutdown_connections 2024-05-11T12:12:24.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.758+0000 7efc68a00640 1 -- 172.21.15.167:0/1129462363 wait complete. 2024-05-11T12:12:24.786 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds ms_async_reap_threshold 2024-05-11T12:12:24.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.975+0000 7f19d2800640 1 Processor -- start 2024-05-11T12:12:24.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.975+0000 7f19d2800640 1 -- start start 2024-05-11T12:12:24.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.976+0000 7f19d2800640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f19cc15ac80 0x7f19cc074260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:24.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.977+0000 7f19d2800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f19cc161380 0x7f19cc072950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:24.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.977+0000 7f19d2800640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19cc0b65e0 con 0x7f19cc0b3d50 2024-05-11T12:12:24.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.977+0000 7f19d2800640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f19cc06e3c0 con 0x7f19cc15ac80 2024-05-11T12:12:24.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.977+0000 7f19d2800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f19cc15b060 con 0x7f19cc161380 2024-05-11T12:12:24.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.977+0000 7f19cb400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f19cc15ac80 0x7f19cc074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:24.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.977+0000 7f19caa00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f19cc0b3d50 0x7f19cc160c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:54546/0 (socket says 172.21.15.167:54546) 2024-05-11T12:12:24.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.977+0000 7f19caa00640 1 -- 172.21.15.167:0/2299740305 learned_addr learned my addr 172.21.15.167:0/2299740305 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:24.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.977+0000 7f19cb400640 1 -- 172.21.15.167:0/2299740305 >> v2:172.21.15.204:3300/0 conn(0x7f19cc161380 msgr2=0x7f19cc072950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:24.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.977+0000 7f19cbe00640 1 --2- 172.21.15.167:0/2299740305 >> v2:172.21.15.204:3300/0 conn(0x7f19cc161380 0x7f19cc072950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:24.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.978+0000 7f19ca000640 1 -- 172.21.15.167:0/2299740305 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2049878173 0 0) 0x7f19cc0b65e0 con 0x7f19cc0b3d50 2024-05-11T12:12:24.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.978+0000 7f19cb400640 1 --2- 172.21.15.167:0/2299740305 >> v2:172.21.15.204:3300/0 conn(0x7f19cc161380 0x7f19cc072950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:24.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.978+0000 7f19cb400640 1 -- 172.21.15.167:0/2299740305 >> v1:172.21.15.167:6789/0 conn(0x7f19cc0b3d50 legacy=0x7f19cc160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:24.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.978+0000 7f19cb400640 1 -- 172.21.15.167:0/2299740305 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19cc0b68c0 con 0x7f19cc15ac80 2024-05-11T12:12:24.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.978+0000 7f19cbe00640 1 --2- 172.21.15.167:0/2299740305 >> v2:172.21.15.204:3300/0 conn(0x7f19cc161380 0x7f19cc072950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:24.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.979+0000 7f19cb400640 1 --2- 172.21.15.167:0/2299740305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19cc15ac80 0x7f19cc074260 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f19bc0097a0 tx=0x7f19bc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad8174fae64b00b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:24.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.979+0000 7f19ca000640 1 -- 172.21.15.167:0/2299740305 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19bc037070 con 0x7f19cc15ac80 2024-05-11T12:12:24.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.979+0000 7f19ca000640 1 -- 172.21.15.167:0/2299740305 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19bc032440 con 0x7f19cc15ac80 2024-05-11T12:12:24.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.979+0000 7f19ca000640 1 -- 172.21.15.167:0/2299740305 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19bc032810 con 0x7f19cc15ac80 2024-05-11T12:12:24.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.980+0000 7f19d2800640 1 -- 172.21.15.167:0/2299740305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19cc15ac80 msgr2=0x7f19cc074260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:24.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.980+0000 7f19d2800640 1 --2- 172.21.15.167:0/2299740305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19cc15ac80 0x7f19cc074260 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f19bc0097a0 tx=0x7f19bc02a1e0 comp rx=0 tx=0).stop 2024-05-11T12:12:24.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.980+0000 7f19d2800640 1 -- 172.21.15.167:0/2299740305 shutdown_connections 2024-05-11T12:12:24.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.980+0000 7f19d2800640 1 --2- 172.21.15.167:0/2299740305 >> v2:172.21.15.204:3300/0 conn(0x7f19cc161380 0x7f19cc072950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:24.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.980+0000 7f19d2800640 1 --2- 172.21.15.167:0/2299740305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19cc15ac80 0x7f19cc074260 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:24.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.980+0000 7f19d2800640 1 -- 172.21.15.167:0/2299740305 >> 172.21.15.167:0/2299740305 conn(0x7f19cc06c450 msgr2=0x7f19cc06c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:24.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.980+0000 7f19d2800640 1 -- 172.21.15.167:0/2299740305 shutdown_connections 2024-05-11T12:12:24.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.980+0000 7f19d2800640 1 -- 172.21.15.167:0/2299740305 wait complete. 2024-05-11T12:12:24.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.982+0000 7f19d2800640 1 Processor -- start 2024-05-11T12:12:24.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.982+0000 7f19d2800640 1 -- start start 2024-05-11T12:12:24.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.983+0000 7f19d2800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19cc0b3d50 0x7f19cc159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:24.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.983+0000 7f19d2800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19cc15ac80 0x7f19cc06e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:24.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.983+0000 7f19d2800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19cc161380 0x7f19cc06ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:24.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.983+0000 7f19d2800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f19cc0b44f0 con 0x7f19cc161380 2024-05-11T12:12:24.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.983+0000 7f19d2800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f19cc0b41e0 con 0x7f19cc15ac80 2024-05-11T12:12:24.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.983+0000 7f19d2800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f19cc072e90 con 0x7f19cc0b3d50 2024-05-11T12:12:24.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.983+0000 7f19cbe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19cc161380 0x7f19cc06ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:24.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.983+0000 7f19cbe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19cc161380 0x7f19cc06ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:40010/0 (socket says 172.21.15.167:40010) 2024-05-11T12:12:24.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.983+0000 7f19cbe00640 1 -- 172.21.15.167:0/1220577654 learned_addr learned my addr 172.21.15.167:0/1220577654 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:24.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.983+0000 7f19caa00640 1 --2- 172.21.15.167:0/1220577654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19cc15ac80 0x7f19cc06e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:24.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.983+0000 7f19cb400640 1 --2- 172.21.15.167:0/1220577654 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19cc0b3d50 0x7f19cc159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:24.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.983+0000 7f19cbe00640 1 -- 172.21.15.167:0/1220577654 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19cc0b3d50 msgr2=0x7f19cc159890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:24.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.983+0000 7f19cbe00640 1 --2- 172.21.15.167:0/1220577654 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19cc0b3d50 0x7f19cc159890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:24.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.984+0000 7f19cbe00640 1 -- 172.21.15.167:0/1220577654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19cc15ac80 msgr2=0x7f19cc06e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:24.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.984+0000 7f19cbe00640 1 --2- 172.21.15.167:0/1220577654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19cc15ac80 0x7f19cc06e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:24.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.984+0000 7f19cbe00640 1 -- 172.21.15.167:0/1220577654 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19cc0d0420 con 0x7f19cc161380 2024-05-11T12:12:24.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.984+0000 7f19caa00640 1 --2- 172.21.15.167:0/1220577654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19cc15ac80 0x7f19cc06e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:24.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.984+0000 7f19cb400640 1 --2- 172.21.15.167:0/1220577654 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19cc0b3d50 0x7f19cc159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:24.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.984+0000 7f19cbe00640 1 --2- 172.21.15.167:0/1220577654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19cc161380 0x7f19cc06ea90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f19c400c5f0 tx=0x7f19c400cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:24.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.984+0000 7f19c3400640 1 -- 172.21.15.167:0/1220577654 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c4015070 con 0x7f19cc161380 2024-05-11T12:12:24.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.984+0000 7f19c3400640 1 -- 172.21.15.167:0/1220577654 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19c4011820 con 0x7f19cc161380 2024-05-11T12:12:24.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.984+0000 7f19d2800640 1 -- 172.21.15.167:0/1220577654 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19cc0d0700 con 0x7f19cc161380 2024-05-11T12:12:24.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.984+0000 7f19c3400640 1 -- 172.21.15.167:0/1220577654 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c4011c10 con 0x7f19cc161380 2024-05-11T12:12:24.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.984+0000 7f19d2800640 1 -- 172.21.15.167:0/1220577654 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19cc159dd0 con 0x7f19cc161380 2024-05-11T12:12:24.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.986+0000 7f19d2800640 1 -- 172.21.15.167:0/1220577654 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19cc1e6a90 con 0x7f19cc161380 2024-05-11T12:12:24.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.987+0000 7f19c3400640 1 -- 172.21.15.167:0/1220577654 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f19c4018070 con 0x7f19cc161380 2024-05-11T12:12:24.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.988+0000 7f19c3400640 1 --2- 172.21.15.167:0/1220577654 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f19a8076300 0x7f19a80787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:24.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.988+0000 7f19c3400640 1 -- 172.21.15.167:0/1220577654 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f19c40972d0 con 0x7f19cc161380 2024-05-11T12:12:24.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.988+0000 7f19cb400640 1 --2- 172.21.15.167:0/1220577654 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f19a8076300 0x7f19a80787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:24.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.989+0000 7f19cb400640 1 --2- 172.21.15.167:0/1220577654 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f19a8076300 0x7f19a80787c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f19bc034000 tx=0x7f19bc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:24.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:24.993+0000 7f19c3400640 1 -- 172.21.15.167:0/1220577654 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f19cc1e6a90 con 0x7f19cc161380 2024-05-11T12:12:25.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.205+0000 7f19d2800640 1 -- 172.21.15.167:0/1220577654 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "ms_async_reap_threshold"} v 0) -- 0x7f19cc069520 con 0x7f19cc161380 2024-05-11T12:12:25.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.209+0000 7f19c3400640 1 -- 172.21.15.167:0/1220577654 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "ms_async_reap_threshold"}]=0 v12) ==== 106+0+0 (secure 0 0 0) 0x7f19c4060db0 con 0x7f19cc161380 2024-05-11T12:12:25.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.213+0000 7f19d2800640 1 -- 172.21.15.167:0/1220577654 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f19a8076300 msgr2=0x7f19a80787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:25.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.213+0000 7f19d2800640 1 --2- 172.21.15.167:0/1220577654 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f19a8076300 0x7f19a80787c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f19bc034000 tx=0x7f19bc02e000 comp rx=0 tx=0).stop 2024-05-11T12:12:25.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.213+0000 7f19d2800640 1 -- 172.21.15.167:0/1220577654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19cc161380 msgr2=0x7f19cc06ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:25.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.213+0000 7f19d2800640 1 --2- 172.21.15.167:0/1220577654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19cc161380 0x7f19cc06ea90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f19c400c5f0 tx=0x7f19c400cab0 comp rx=0 tx=0).stop 2024-05-11T12:12:25.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.213+0000 7f19d2800640 1 -- 172.21.15.167:0/1220577654 shutdown_connections 2024-05-11T12:12:25.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.213+0000 7f19d2800640 1 --2- 172.21.15.167:0/1220577654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19cc161380 0x7f19cc06ea90 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:25.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.213+0000 7f19d2800640 1 --2- 172.21.15.167:0/1220577654 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f19a8076300 0x7f19a80787c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:25.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.213+0000 7f19d2800640 1 --2- 172.21.15.167:0/1220577654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19cc15ac80 0x7f19cc06e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:25.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.213+0000 7f19d2800640 1 --2- 172.21.15.167:0/1220577654 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19cc0b3d50 0x7f19cc159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:25.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.213+0000 7f19d2800640 1 -- 172.21.15.167:0/1220577654 >> 172.21.15.167:0/1220577654 conn(0x7f19cc06c450 msgr2=0x7f19cc0af020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:25.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.213+0000 7f19d2800640 1 -- 172.21.15.167:0/1220577654 shutdown_connections 2024-05-11T12:12:25.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.214+0000 7f19d2800640 1 -- 172.21.15.167:0/1220577654 wait complete. 2024-05-11T12:12:25.231 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-11T12:12:25.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.443+0000 7fc76ec00640 1 Processor -- start 2024-05-11T12:12:25.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.443+0000 7fc76ec00640 1 -- start start 2024-05-11T12:12:25.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.445+0000 7fc76ec00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc768163a20 0x7fc7680710f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:25.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.445+0000 7fc76ec00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc768071630 0x7fc768160930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:25.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.445+0000 7fc76ec00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7680b4fe0 con 0x7fc7681627f0 2024-05-11T12:12:25.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.445+0000 7fc76ec00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fc768160730 con 0x7fc768163a20 2024-05-11T12:12:25.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.445+0000 7fc76ec00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc7680b52c0 con 0x7fc768071630 2024-05-11T12:12:25.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.446+0000 7fc767e00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fc7681627f0 0x7fc768160020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.167:50990/0 (socket says 172.21.15.167:50990) 2024-05-11T12:12:25.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.446+0000 7fc767e00640 1 -- 172.21.15.167:0/95527233 learned_addr learned my addr 172.21.15.167:0/95527233 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:25.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.446+0000 7fc75fe00640 1 --2- 172.21.15.167:0/95527233 >> v2:172.21.15.204:3300/0 conn(0x7fc768163a20 0x7fc7680710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:25.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.446+0000 7fc76cc00640 1 --2- 172.21.15.167:0/95527233 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc768071630 0x7fc768160930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:25.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.447+0000 7fc75fe00640 1 -- 172.21.15.167:0/95527233 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc768071630 msgr2=0x7fc768160930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:25.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.447+0000 7fc75fe00640 1 --2- 172.21.15.167:0/95527233 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc768071630 0x7fc768160930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:25.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.447+0000 7fc75fe00640 1 -- 172.21.15.167:0/95527233 >> v1:172.21.15.204:6789/0 conn(0x7fc7681627f0 legacy=0x7fc768160020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:25.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.447+0000 7fc767400640 1 -- 172.21.15.167:0/95527233 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1199799807 0 0) 0x7fc7680b4fe0 con 0x7fc7681627f0 2024-05-11T12:12:25.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.447+0000 7fc75fe00640 1 -- 172.21.15.167:0/95527233 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7680b55a0 con 0x7fc768163a20 2024-05-11T12:12:25.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.449+0000 7fc75fe00640 1 --2- 172.21.15.167:0/95527233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc768163a20 0x7fc7680710f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc7500097a0 tx=0x7fc75002a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42ccfacc33982e16 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:25.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.450+0000 7fc767400640 1 -- 172.21.15.167:0/95527233 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc750037070 con 0x7fc768163a20 2024-05-11T12:12:25.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.450+0000 7fc767400640 1 -- 172.21.15.167:0/95527233 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc75002ae50 con 0x7fc768163a20 2024-05-11T12:12:25.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.450+0000 7fc767400640 1 -- 172.21.15.167:0/95527233 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc750032660 con 0x7fc768163a20 2024-05-11T12:12:25.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.451+0000 7fc76ec00640 1 -- 172.21.15.167:0/95527233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc768163a20 msgr2=0x7fc7680710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:25.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.451+0000 7fc76ec00640 1 --2- 172.21.15.167:0/95527233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc768163a20 0x7fc7680710f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc7500097a0 tx=0x7fc75002a100 comp rx=0 tx=0).stop 2024-05-11T12:12:25.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.451+0000 7fc76ec00640 1 -- 172.21.15.167:0/95527233 shutdown_connections 2024-05-11T12:12:25.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.451+0000 7fc76ec00640 1 --2- 172.21.15.167:0/95527233 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc768071630 0x7fc768160930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:25.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.451+0000 7fc76ec00640 1 --2- 172.21.15.167:0/95527233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc768163a20 0x7fc7680710f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:25.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.451+0000 7fc76ec00640 1 -- 172.21.15.167:0/95527233 >> 172.21.15.167:0/95527233 conn(0x7fc7680d2000 msgr2=0x7fc7680d2410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:25.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.451+0000 7fc76ec00640 1 -- 172.21.15.167:0/95527233 shutdown_connections 2024-05-11T12:12:25.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.451+0000 7fc76ec00640 1 -- 172.21.15.167:0/95527233 wait complete. 2024-05-11T12:12:25.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.453+0000 7fc76ec00640 1 Processor -- start 2024-05-11T12:12:25.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.453+0000 7fc76ec00640 1 -- start start 2024-05-11T12:12:25.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.454+0000 7fc76ec00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc768071630 0x7fc768154150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:25.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.454+0000 7fc76ec00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7681627f0 0x7fc7680b2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:25.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.454+0000 7fc76ec00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc768163a20 0x7fc7680ad7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:25.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.454+0000 7fc76ec00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc768160e70 con 0x7fc7681627f0 2024-05-11T12:12:25.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.454+0000 7fc76ec00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc7680b5b60 con 0x7fc768071630 2024-05-11T12:12:25.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.454+0000 7fc76ec00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc768160ff0 con 0x7fc768163a20 2024-05-11T12:12:25.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.454+0000 7fc75fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7681627f0 0x7fc7680b2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:25.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.454+0000 7fc75fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7681627f0 0x7fc7680b2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:40014/0 (socket says 172.21.15.167:40014) 2024-05-11T12:12:25.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.455+0000 7fc75fe00640 1 -- 172.21.15.167:0/3965319585 learned_addr learned my addr 172.21.15.167:0/3965319585 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:25.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.455+0000 7fc767e00640 1 --2- 172.21.15.167:0/3965319585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc768071630 0x7fc768154150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:25.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.455+0000 7fc76cc00640 1 --2- 172.21.15.167:0/3965319585 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc768163a20 0x7fc7680ad7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:25.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.455+0000 7fc75fe00640 1 -- 172.21.15.167:0/3965319585 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc768163a20 msgr2=0x7fc7680ad7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:25.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.455+0000 7fc75fe00640 1 --2- 172.21.15.167:0/3965319585 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc768163a20 0x7fc7680ad7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:25.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.455+0000 7fc75fe00640 1 -- 172.21.15.167:0/3965319585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc768071630 msgr2=0x7fc768154150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:25.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.455+0000 7fc75fe00640 1 --2- 172.21.15.167:0/3965319585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc768071630 0x7fc768154150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:25.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.455+0000 7fc75fe00640 1 -- 172.21.15.167:0/3965319585 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc768154980 con 0x7fc7681627f0 2024-05-11T12:12:25.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.455+0000 7fc75fe00640 1 --2- 172.21.15.167:0/3965319585 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7681627f0 0x7fc7680b2ea0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc75002a6f0 tx=0x7fc7500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:25.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.456+0000 7fc764c00640 1 -- 172.21.15.167:0/3965319585 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc750041020 con 0x7fc7681627f0 2024-05-11T12:12:25.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.456+0000 7fc764c00640 1 -- 172.21.15.167:0/3965319585 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc750038040 con 0x7fc7681627f0 2024-05-11T12:12:25.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.456+0000 7fc76ec00640 1 -- 172.21.15.167:0/3965319585 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc768154c60 con 0x7fc7681627f0 2024-05-11T12:12:25.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.456+0000 7fc764c00640 1 -- 172.21.15.167:0/3965319585 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc750037030 con 0x7fc7681627f0 2024-05-11T12:12:25.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.456+0000 7fc76ec00640 1 -- 172.21.15.167:0/3965319585 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc768155220 con 0x7fc7681627f0 2024-05-11T12:12:25.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.457+0000 7fc76ec00640 1 -- 172.21.15.167:0/3965319585 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc768069bc0 con 0x7fc7681627f0 2024-05-11T12:12:25.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.458+0000 7fc764c00640 1 -- 172.21.15.167:0/3965319585 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc750044020 con 0x7fc7681627f0 2024-05-11T12:12:25.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.459+0000 7fc764c00640 1 --2- 172.21.15.167:0/3965319585 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc7440762d0 0x7fc744078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:25.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.459+0000 7fc764c00640 1 -- 172.21.15.167:0/3965319585 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7fc7500b7740 con 0x7fc7681627f0 2024-05-11T12:12:25.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.460+0000 7fc767e00640 1 --2- 172.21.15.167:0/3965319585 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc7440762d0 0x7fc744078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:25.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.460+0000 7fc767e00640 1 --2- 172.21.15.167:0/3965319585 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc7440762d0 0x7fc744078790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc758000f60 tx=0x7fc758009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:25.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.463+0000 7fc764c00640 1 -- 172.21.15.167:0/3965319585 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc75004b030 con 0x7fc7681627f0 2024-05-11T12:12:25.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.684+0000 7fc76ec00640 1 -- 172.21.15.167:0/3965319585 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fc7681599e0 con 0x7fc7681627f0 2024-05-11T12:12:25.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.691+0000 7fc764c00640 1 -- 172.21.15.167:0/3965319585 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) ==== 104+0+0 (secure 0 0 0) 0x7fc750081200 con 0x7fc7681627f0 2024-05-11T12:12:25.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.694+0000 7fc76ec00640 1 -- 172.21.15.167:0/3965319585 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc7440762d0 msgr2=0x7fc744078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:25.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.694+0000 7fc76ec00640 1 --2- 172.21.15.167:0/3965319585 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc7440762d0 0x7fc744078790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc758000f60 tx=0x7fc758009040 comp rx=0 tx=0).stop 2024-05-11T12:12:25.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.694+0000 7fc76ec00640 1 -- 172.21.15.167:0/3965319585 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7681627f0 msgr2=0x7fc7680b2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:25.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.694+0000 7fc76ec00640 1 --2- 172.21.15.167:0/3965319585 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7681627f0 0x7fc7680b2ea0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc75002a6f0 tx=0x7fc7500046c0 comp rx=0 tx=0).stop 2024-05-11T12:12:25.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.695+0000 7fc76ec00640 1 -- 172.21.15.167:0/3965319585 shutdown_connections 2024-05-11T12:12:25.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.695+0000 7fc76ec00640 1 --2- 172.21.15.167:0/3965319585 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc768163a20 0x7fc7680ad7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:25.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.695+0000 7fc76ec00640 1 --2- 172.21.15.167:0/3965319585 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7681627f0 0x7fc7680b2ea0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:25.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.695+0000 7fc76ec00640 1 --2- 172.21.15.167:0/3965319585 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc7440762d0 0x7fc744078790 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:25.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.695+0000 7fc76ec00640 1 --2- 172.21.15.167:0/3965319585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc768071630 0x7fc768154150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:25.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.695+0000 7fc76ec00640 1 -- 172.21.15.167:0/3965319585 >> 172.21.15.167:0/3965319585 conn(0x7fc7680d2000 msgr2=0x7fc76815ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:25.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.695+0000 7fc76ec00640 1 -- 172.21.15.167:0/3965319585 shutdown_connections 2024-05-11T12:12:25.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.695+0000 7fc76ec00640 1 -- 172.21.15.167:0/3965319585 wait complete. 2024-05-11T12:12:25.712 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_mount_conn_close' 2024-05-11T12:12:25.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.906+0000 7f9b50000640 1 Processor -- start 2024-05-11T12:12:25.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.906+0000 7f9b50000640 1 -- start start 2024-05-11T12:12:25.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.908+0000 7f9b50000640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9b48071800 0x7f9b480b5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:25.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.908+0000 7f9b50000640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b48163e60 con 0x7f9b480b5a10 2024-05-11T12:12:25.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.908+0000 7f9b50000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b480b4f70 con 0x7f9b481488a0 2024-05-11T12:12:25.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.908+0000 7f9b50000640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f9b480b63b0 con 0x7f9b48071800 2024-05-11T12:12:25.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.908+0000 7f9b4cc00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f9b480b5a10 0x7f9b480710f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:54552/0 (socket says 172.21.15.167:54552) 2024-05-11T12:12:25.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.908+0000 7f9b4cc00640 1 -- 172.21.15.167:0/1599178321 learned_addr learned my addr 172.21.15.167:0/1599178321 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:25.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.908+0000 7f9b4e000640 1 --2- 172.21.15.167:0/1599178321 >> v2:172.21.15.167:3300/0 conn(0x7f9b48071800 0x7f9b480b5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:25.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.909+0000 7f9b47e00640 1 -- 172.21.15.167:0/1599178321 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3002617144 0 0) 0x7f9b48163e60 con 0x7f9b480b5a10 2024-05-11T12:12:25.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.909+0000 7f9b47e00640 1 -- 172.21.15.167:0/1599178321 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b2c003520 con 0x7f9b480b5a10 2024-05-11T12:12:25.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.909+0000 7f9b4e000640 1 -- 172.21.15.167:0/1599178321 >> v1:172.21.15.204:6789/0 conn(0x7f9b481488a0 legacy=0x7f9b4815f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:25.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.909+0000 7f9b4e000640 1 -- 172.21.15.167:0/1599178321 >> v1:172.21.15.167:6789/0 conn(0x7f9b480b5a10 legacy=0x7f9b480710f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:25.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.909+0000 7f9b4e000640 1 -- 172.21.15.167:0/1599178321 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b48163b80 con 0x7f9b48071800 2024-05-11T12:12:25.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.910+0000 7f9b4e000640 1 --2- 172.21.15.167:0/1599178321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b48071800 0x7f9b480b5e70 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9b400097d0 tx=0x7f9b4002a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf1a970bf8fe3265 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:25.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.910+0000 7f9b47e00640 1 -- 172.21.15.167:0/1599178321 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b40037070 con 0x7f9b48071800 2024-05-11T12:12:25.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.910+0000 7f9b47e00640 1 -- 172.21.15.167:0/1599178321 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b4002ae40 con 0x7f9b48071800 2024-05-11T12:12:25.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.910+0000 7f9b47e00640 1 -- 172.21.15.167:0/1599178321 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b40032660 con 0x7f9b48071800 2024-05-11T12:12:25.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.911+0000 7f9b50000640 1 -- 172.21.15.167:0/1599178321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b48071800 msgr2=0x7f9b480b5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:25.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.911+0000 7f9b50000640 1 --2- 172.21.15.167:0/1599178321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b48071800 0x7f9b480b5e70 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9b400097d0 tx=0x7f9b4002a130 comp rx=0 tx=0).stop 2024-05-11T12:12:25.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.911+0000 7f9b50000640 1 -- 172.21.15.167:0/1599178321 shutdown_connections 2024-05-11T12:12:25.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.911+0000 7f9b50000640 1 --2- 172.21.15.167:0/1599178321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b48071800 0x7f9b480b5e70 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:25.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.911+0000 7f9b50000640 1 -- 172.21.15.167:0/1599178321 >> 172.21.15.167:0/1599178321 conn(0x7f9b480d2550 msgr2=0x7f9b480ce890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:25.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.911+0000 7f9b50000640 1 -- 172.21.15.167:0/1599178321 shutdown_connections 2024-05-11T12:12:25.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.911+0000 7f9b50000640 1 -- 172.21.15.167:0/1599178321 wait complete. 2024-05-11T12:12:25.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.913+0000 7f9b50000640 1 Processor -- start 2024-05-11T12:12:25.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.913+0000 7f9b50000640 1 -- start start 2024-05-11T12:12:25.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.914+0000 7f9b50000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b48071800 0x7f9b481f8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:25.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.914+0000 7f9b50000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b480b5a10 0x7f9b48143a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:25.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.914+0000 7f9b50000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9b481488a0 0x7f9b481f4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:25.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.914+0000 7f9b50000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9b48160020 con 0x7f9b48071800 2024-05-11T12:12:25.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.914+0000 7f9b50000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9b480b5580 con 0x7f9b480b5a10 2024-05-11T12:12:25.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.914+0000 7f9b50000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f9b48148c80 con 0x7f9b481488a0 2024-05-11T12:12:25.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.914+0000 7f9b4d600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b48071800 0x7f9b481f8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:25.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.914+0000 7f9b4d600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b48071800 0x7f9b481f8f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:40030/0 (socket says 172.21.15.167:40030) 2024-05-11T12:12:25.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.914+0000 7f9b4d600640 1 -- 172.21.15.167:0/2571139265 learned_addr learned my addr 172.21.15.167:0/2571139265 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:25.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.915+0000 7f9b4d600640 1 -- 172.21.15.167:0/2571139265 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9b481488a0 msgr2=0x7f9b481f4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:25.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.915+0000 7f9b4d600640 1 --2- 172.21.15.167:0/2571139265 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9b481488a0 0x7f9b481f4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:25.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.915+0000 7f9b4d600640 1 -- 172.21.15.167:0/2571139265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b480b5a10 msgr2=0x7f9b48143a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:25.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.915+0000 7f9b4d600640 1 --2- 172.21.15.167:0/2571139265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b480b5a10 0x7f9b48143a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:25.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.915+0000 7f9b4d600640 1 -- 172.21.15.167:0/2571139265 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b48143fa0 con 0x7f9b48071800 2024-05-11T12:12:25.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.915+0000 7f9b4d600640 1 --2- 172.21.15.167:0/2571139265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b48071800 0x7f9b481f8f10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9b3800c7c0 tx=0x7f9b3800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:25.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.915+0000 7f9b45600640 1 -- 172.21.15.167:0/2571139265 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b38016020 con 0x7f9b48071800 2024-05-11T12:12:25.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.915+0000 7f9b45600640 1 -- 172.21.15.167:0/2571139265 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b3800aa50 con 0x7f9b48071800 2024-05-11T12:12:25.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.915+0000 7f9b45600640 1 -- 172.21.15.167:0/2571139265 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b38012420 con 0x7f9b48071800 2024-05-11T12:12:25.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.915+0000 7f9b50000640 1 -- 172.21.15.167:0/2571139265 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b481f59b0 con 0x7f9b48071800 2024-05-11T12:12:25.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.915+0000 7f9b50000640 1 -- 172.21.15.167:0/2571139265 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b481f8730 con 0x7f9b48071800 2024-05-11T12:12:25.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.917+0000 7f9b50000640 1 -- 172.21.15.167:0/2571139265 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b48069590 con 0x7f9b48071800 2024-05-11T12:12:25.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.918+0000 7f9b45600640 1 -- 172.21.15.167:0/2571139265 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9b38019020 con 0x7f9b48071800 2024-05-11T12:12:25.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.919+0000 7f9b45600640 1 --2- 172.21.15.167:0/2571139265 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f9b280762d0 0x7f9b28078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:25.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.919+0000 7f9b45600640 1 -- 172.21.15.167:0/2571139265 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f9b380971b0 con 0x7f9b48071800 2024-05-11T12:12:25.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.919+0000 7f9b4cc00640 1 --2- 172.21.15.167:0/2571139265 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f9b280762d0 0x7f9b28078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:25.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.920+0000 7f9b4cc00640 1 --2- 172.21.15.167:0/2571139265 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f9b280762d0 0x7f9b28078790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9b34001220 tx=0x7f9b340078f0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:25.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:25.924+0000 7f9b45600640 1 -- 172.21.15.167:0/2571139265 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9b38020070 con 0x7f9b48071800 2024-05-11T12:12:26.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.208+0000 7f9b50000640 1 -- 172.21.15.167:0/2571139265 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_mount_conn_close"]} v 0) -- 0x7f9b480b2500 con 0x7f9b48071800 2024-05-11T12:12:26.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.758+0000 7f9b45600640 1 -- 172.21.15.167:0/2571139265 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_mount_conn_close"]}]=0 v112) ==== 144+0+0 (secure 0 0 0) 0x7f9b38060c70 con 0x7f9b48071800 2024-05-11T12:12:26.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.762+0000 7f9b50000640 1 -- 172.21.15.167:0/2571139265 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f9b280762d0 msgr2=0x7f9b28078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:26.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.762+0000 7f9b50000640 1 --2- 172.21.15.167:0/2571139265 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f9b280762d0 0x7f9b28078790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9b34001220 tx=0x7f9b340078f0 comp rx=0 tx=0).stop 2024-05-11T12:12:26.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.762+0000 7f9b50000640 1 -- 172.21.15.167:0/2571139265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b48071800 msgr2=0x7f9b481f8f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:26.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.762+0000 7f9b50000640 1 --2- 172.21.15.167:0/2571139265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b48071800 0x7f9b481f8f10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9b3800c7c0 tx=0x7f9b3800cc80 comp rx=0 tx=0).stop 2024-05-11T12:12:26.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.762+0000 7f9b50000640 1 -- 172.21.15.167:0/2571139265 shutdown_connections 2024-05-11T12:12:26.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.762+0000 7f9b50000640 1 --2- 172.21.15.167:0/2571139265 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9b481488a0 0x7f9b481f4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:26.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.762+0000 7f9b50000640 1 --2- 172.21.15.167:0/2571139265 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f9b280762d0 0x7f9b28078790 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:26.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.762+0000 7f9b50000640 1 --2- 172.21.15.167:0/2571139265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b480b5a10 0x7f9b48143a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:26.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.763+0000 7f9b50000640 1 --2- 172.21.15.167:0/2571139265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b48071800 0x7f9b481f8f10 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:26.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.763+0000 7f9b50000640 1 -- 172.21.15.167:0/2571139265 >> 172.21.15.167:0/2571139265 conn(0x7f9b480d2550 msgr2=0x7f9b48160680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:26.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.763+0000 7f9b50000640 1 -- 172.21.15.167:0/2571139265 shutdown_connections 2024-05-11T12:12:26.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.763+0000 7f9b50000640 1 -- 172.21.15.167:0/2571139265 wait complete. 2024-05-11T12:12:26.784 INFO:tasks.cephfs_test_runner:That when a client unmounts, the thread count on the MDS goes back ... ok 2024-05-11T12:12:26.784 INFO:tasks.cephfs_test_runner:Starting test: test_session_evict_blocklisted (tasks.cephfs.test_sessionmap.TestSessionMap) Check that mds evicts blocklisted client 2024-05-11T12:12:26.784 INFO:tasks.cephfs_test_runner:test_session_evict_blocklisted (tasks.cephfs.test_sessionmap.TestSessionMap) 2024-05-11T12:12:26.785 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:12:26.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.989+0000 7fe252a00640 1 Processor -- start 2024-05-11T12:12:26.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.989+0000 7fe252a00640 1 -- start start 2024-05-11T12:12:26.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.991+0000 7fe252a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe24c0af1f0 0x7fe24c0cf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:26.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.991+0000 7fe252a00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe24c06d460 0x7fe24c06d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:26.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.991+0000 7fe252a00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe24c0cf700 con 0x7fe24c0af5d0 2024-05-11T12:12:26.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.991+0000 7fe252a00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fe24c1608d0 con 0x7fe24c06d460 2024-05-11T12:12:26.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.991+0000 7fe252a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe24c06fe10 con 0x7fe24c0af1f0 2024-05-11T12:12:26.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.991+0000 7fe250a00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe24c06d460 0x7fe24c06d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:26.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.991+0000 7fe250a00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe24c06d460 0x7fe24c06d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:40046/0 (socket says 172.21.15.167:40046) 2024-05-11T12:12:26.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.991+0000 7fe250a00640 1 -- 172.21.15.167:0/403772777 learned_addr learned my addr 172.21.15.167:0/403772777 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:26.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.991+0000 7fe24b400640 1 --1- 172.21.15.167:0/403772777 >> v1:172.21.15.167:6789/0 conn(0x7fe24c0af5d0 0x7fe24c073ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:54568/0 (socket says 172.21.15.167:54568) 2024-05-11T12:12:26.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.992+0000 7fe24be00640 1 --2- 172.21.15.167:0/403772777 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe24c0af1f0 0x7fe24c0cf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:26.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.992+0000 7fe250a00640 1 -- 172.21.15.167:0/403772777 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe24c0af1f0 msgr2=0x7fe24c0cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:26.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.992+0000 7fe250a00640 1 --2- 172.21.15.167:0/403772777 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe24c0af1f0 0x7fe24c0cf1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:26.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.992+0000 7fe250a00640 1 -- 172.21.15.167:0/403772777 >> v1:172.21.15.167:6789/0 conn(0x7fe24c0af5d0 legacy=0x7fe24c073ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:26.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.992+0000 7fe250a00640 1 -- 172.21.15.167:0/403772777 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe24c15fcc0 con 0x7fe24c06d460 2024-05-11T12:12:26.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.993+0000 7fe250a00640 1 --2- 172.21.15.167:0/403772777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe24c06d460 0x7fe24c06d8f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe244009e50 tx=0x7fe24402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f57a30da7c61052 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:26.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.993+0000 7fe24aa00640 1 -- 172.21.15.167:0/403772777 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe244036070 con 0x7fe24c06d460 2024-05-11T12:12:26.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.993+0000 7fe24aa00640 1 -- 172.21.15.167:0/403772777 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe244031600 con 0x7fe24c06d460 2024-05-11T12:12:26.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.993+0000 7fe24aa00640 1 -- 172.21.15.167:0/403772777 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2440319d0 con 0x7fe24c06d460 2024-05-11T12:12:26.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.994+0000 7fe252a00640 1 -- 172.21.15.167:0/403772777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe24c06d460 msgr2=0x7fe24c06d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:26.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.994+0000 7fe252a00640 1 --2- 172.21.15.167:0/403772777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe24c06d460 0x7fe24c06d8f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe244009e50 tx=0x7fe24402a6a0 comp rx=0 tx=0).stop 2024-05-11T12:12:26.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.994+0000 7fe252a00640 1 -- 172.21.15.167:0/403772777 shutdown_connections 2024-05-11T12:12:26.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.994+0000 7fe252a00640 1 --2- 172.21.15.167:0/403772777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe24c06d460 0x7fe24c06d8f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:26.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.994+0000 7fe252a00640 1 --2- 172.21.15.167:0/403772777 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe24c0af1f0 0x7fe24c0cf1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:26.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.994+0000 7fe252a00640 1 -- 172.21.15.167:0/403772777 >> 172.21.15.167:0/403772777 conn(0x7fe24c0cd8f0 msgr2=0x7fe24c0cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:26.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.994+0000 7fe252a00640 1 -- 172.21.15.167:0/403772777 shutdown_connections 2024-05-11T12:12:26.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.994+0000 7fe252a00640 1 -- 172.21.15.167:0/403772777 wait complete. 2024-05-11T12:12:26.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.996+0000 7fe252a00640 1 Processor -- start 2024-05-11T12:12:26.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.996+0000 7fe252a00640 1 -- start start 2024-05-11T12:12:26.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.997+0000 7fe252a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe24c06d460 0x7fe24c14d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:26.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.997+0000 7fe252a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe24c0af1f0 0x7fe24c0b0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:26.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.997+0000 7fe252a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe24c0af5d0 0x7fe24c0b1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:26.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.997+0000 7fe252a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe24c160560 con 0x7fe24c0af1f0 2024-05-11T12:12:26.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.997+0000 7fe252a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe24c160280 con 0x7fe24c06d460 2024-05-11T12:12:26.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.997+0000 7fe252a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe24c15f920 con 0x7fe24c0af5d0 2024-05-11T12:12:26.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.997+0000 7fe24b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe24c0af1f0 0x7fe24c0b0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:26.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.997+0000 7fe24b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe24c0af1f0 0x7fe24c0b0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:40060/0 (socket says 172.21.15.167:40060) 2024-05-11T12:12:27.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.997+0000 7fe24b400640 1 -- 172.21.15.167:0/1080319531 learned_addr learned my addr 172.21.15.167:0/1080319531 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:27.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.998+0000 7fe24be00640 1 --2- 172.21.15.167:0/1080319531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe24c06d460 0x7fe24c14d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:27.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.998+0000 7fe250a00640 1 --2- 172.21.15.167:0/1080319531 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe24c0af5d0 0x7fe24c0b1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:27.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.998+0000 7fe24b400640 1 -- 172.21.15.167:0/1080319531 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe24c0af5d0 msgr2=0x7fe24c0b1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:27.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.998+0000 7fe24b400640 1 --2- 172.21.15.167:0/1080319531 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe24c0af5d0 0x7fe24c0b1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:27.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.998+0000 7fe24b400640 1 -- 172.21.15.167:0/1080319531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe24c06d460 msgr2=0x7fe24c14d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:27.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.998+0000 7fe24b400640 1 --2- 172.21.15.167:0/1080319531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe24c06d460 0x7fe24c14d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:27.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.998+0000 7fe24b400640 1 -- 172.21.15.167:0/1080319531 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe24c15bf30 con 0x7fe24c0af1f0 2024-05-11T12:12:27.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.998+0000 7fe24be00640 1 --2- 172.21.15.167:0/1080319531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe24c06d460 0x7fe24c14d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:27.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.998+0000 7fe24b400640 1 --2- 172.21.15.167:0/1080319531 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe24c0af1f0 0x7fe24c0b0af0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe23800c400 tx=0x7fe23800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:27.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.998+0000 7fe243e00640 1 -- 172.21.15.167:0/1080319531 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2380074e0 con 0x7fe24c0af1f0 2024-05-11T12:12:27.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.998+0000 7fe243e00640 1 -- 172.21.15.167:0/1080319531 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe238007680 con 0x7fe24c0af1f0 2024-05-11T12:12:27.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.998+0000 7fe243e00640 1 -- 172.21.15.167:0/1080319531 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2380049e0 con 0x7fe24c0af1f0 2024-05-11T12:12:27.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.998+0000 7fe252a00640 1 -- 172.21.15.167:0/1080319531 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe24c15c210 con 0x7fe24c0af1f0 2024-05-11T12:12:27.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:26.999+0000 7fe252a00640 1 -- 172.21.15.167:0/1080319531 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe24c15c7d0 con 0x7fe24c0af1f0 2024-05-11T12:12:27.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.000+0000 7fe252a00640 1 -- 172.21.15.167:0/1080319531 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe24c0b05d0 con 0x7fe24c0af1f0 2024-05-11T12:12:27.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.001+0000 7fe243e00640 1 -- 172.21.15.167:0/1080319531 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe23801a020 con 0x7fe24c0af1f0 2024-05-11T12:12:27.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.002+0000 7fe243e00640 1 --2- 172.21.15.167:0/1080319531 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe22c0762d0 0x7fe22c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:27.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.002+0000 7fe243e00640 1 -- 172.21.15.167:0/1080319531 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7fe238096c60 con 0x7fe24c0af1f0 2024-05-11T12:12:27.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.002+0000 7fe24be00640 1 --2- 172.21.15.167:0/1080319531 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe22c0762d0 0x7fe22c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:27.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.003+0000 7fe24be00640 1 --2- 172.21.15.167:0/1080319531 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe22c0762d0 0x7fe22c078790 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe23c006fd0 tx=0x7fe23c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:27.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.006+0000 7fe243e00640 1 -- 172.21.15.167:0/1080319531 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe24c0b05d0 con 0x7fe24c0af1f0 2024-05-11T12:12:27.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.216+0000 7fe252a00640 1 -- 172.21.15.167:0/1080319531 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe24c069590 con 0x7fe24c0af1f0 2024-05-11T12:12:27.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.218+0000 7fe243e00640 1 -- 172.21.15.167:0/1080319531 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+13343 (secure 0 0 0) 0x7fe238060720 con 0x7fe24c0af1f0 2024-05-11T12:12:27.218 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:12:27.218 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":44,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:12:24.740830+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.204:6835/3248611739":"2024-05-12T12:12:21.406164+0000","172.21.15.204:6833/3248611739":"2024-05-12T12:12:21.406164+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:12:27.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.221+0000 7fe252a00640 1 -- 172.21.15.167:0/1080319531 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe22c0762d0 msgr2=0x7fe22c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:27.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.221+0000 7fe252a00640 1 --2- 172.21.15.167:0/1080319531 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe22c0762d0 0x7fe22c078790 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe23c006fd0 tx=0x7fe23c008040 comp rx=0 tx=0).stop 2024-05-11T12:12:27.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.221+0000 7fe252a00640 1 -- 172.21.15.167:0/1080319531 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe24c0af1f0 msgr2=0x7fe24c0b0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:27.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.221+0000 7fe252a00640 1 --2- 172.21.15.167:0/1080319531 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe24c0af1f0 0x7fe24c0b0af0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe23800c400 tx=0x7fe23800c8c0 comp rx=0 tx=0).stop 2024-05-11T12:12:27.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.221+0000 7fe252a00640 1 -- 172.21.15.167:0/1080319531 shutdown_connections 2024-05-11T12:12:27.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.221+0000 7fe252a00640 1 --2- 172.21.15.167:0/1080319531 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe24c0af5d0 0x7fe24c0b1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:27.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.221+0000 7fe252a00640 1 --2- 172.21.15.167:0/1080319531 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe24c0af1f0 0x7fe24c0b0af0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:27.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.221+0000 7fe252a00640 1 --2- 172.21.15.167:0/1080319531 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe22c0762d0 0x7fe22c078790 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:27.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.221+0000 7fe252a00640 1 --2- 172.21.15.167:0/1080319531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe24c06d460 0x7fe24c14d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:27.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.222+0000 7fe252a00640 1 -- 172.21.15.167:0/1080319531 >> 172.21.15.167:0/1080319531 conn(0x7fe24c0cd8f0 msgr2=0x7fe24c163510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:27.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.222+0000 7fe252a00640 1 -- 172.21.15.167:0/1080319531 shutdown_connections 2024-05-11T12:12:27.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.222+0000 7fe252a00640 1 -- 172.21.15.167:0/1080319531 wait complete. 2024-05-11T12:12:27.240 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-11T12:10:33.260270+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-11T12:12:27.240 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-11T12:12:27.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.445+0000 7f0276e00640 1 Processor -- start 2024-05-11T12:12:27.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.445+0000 7f0276e00640 1 -- start start 2024-05-11T12:12:27.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.447+0000 7f0276e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f027015d170 0x7f027006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:27.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.447+0000 7f0276e00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f027015c6d0 con 0x7f0270073270 2024-05-11T12:12:27.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.447+0000 7f0276e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02700700b0 con 0x7f02700ce8f0 2024-05-11T12:12:27.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.447+0000 7f0276e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0270160b00 con 0x7f027015d170 2024-05-11T12:12:27.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.447+0000 7f026fe00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f0270073270 0x7f0270073650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:54582/0 (socket says 172.21.15.167:54582) 2024-05-11T12:12:27.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.447+0000 7f026fe00640 1 -- 172.21.15.167:0/901014092 learned_addr learned my addr 172.21.15.167:0/901014092 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:27.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.447+0000 7f026f400640 1 --2- 172.21.15.167:0/901014092 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f027015d170 0x7f027006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:27.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.447+0000 7f026ea00640 1 -- 172.21.15.167:0/901014092 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3580809716 0 0) 0x7f027015c6d0 con 0x7f0270073270 2024-05-11T12:12:27.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.448+0000 7f026ea00640 1 -- 172.21.15.167:0/901014092 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0254003520 con 0x7f0270073270 2024-05-11T12:12:27.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.448+0000 7f026ea00640 1 -- 172.21.15.167:0/901014092 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3620782765 0 0) 0x7f0254003520 con 0x7f0270073270 2024-05-11T12:12:27.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.448+0000 7f026ea00640 1 -- 172.21.15.167:0/901014092 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f027015d170 msgr2=0x7f027006e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:27.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.448+0000 7f026ea00640 1 --2- 172.21.15.167:0/901014092 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f027015d170 0x7f027006e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:27.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.448+0000 7f026ea00640 1 -- 172.21.15.167:0/901014092 >> v1:172.21.15.204:6789/0 conn(0x7f02700ce8f0 legacy=0x7f02700ced80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:27.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.448+0000 7f026ea00640 1 -- 172.21.15.167:0/901014092 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f027015c970 con 0x7f0270073270 2024-05-11T12:12:27.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.448+0000 7f026ea00640 1 -- 172.21.15.167:0/901014092 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f02600016f0 con 0x7f0270073270 2024-05-11T12:12:27.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.448+0000 7f026ea00640 1 -- 172.21.15.167:0/901014092 >> v1:172.21.15.167:6789/0 conn(0x7f0270073270 legacy=0x7f0270073650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:27.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.448+0000 7f026ea00640 1 --2- 172.21.15.167:0/901014092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0254003a30 0x7f0254003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:27.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.448+0000 7f026ea00640 1 --2- 172.21.15.167:0/901014092 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f027015d170 0x7f0254004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:27.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.448+0000 7f026ea00640 1 --2- 172.21.15.167:0/901014092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0254004d40 0x7f02540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:27.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.448+0000 7f026ea00640 1 -- 172.21.15.167:0/901014092 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f025400baf0 con 0x7f0254003a30 2024-05-11T12:12:27.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.448+0000 7f026ea00640 1 -- 172.21.15.167:0/901014092 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f025400bca0 con 0x7f0254004d40 2024-05-11T12:12:27.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.448+0000 7f026ea00640 1 -- 172.21.15.167:0/901014092 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f025400e0b0 con 0x7f027015d170 2024-05-11T12:12:27.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.448+0000 7f026fe00640 1 --2- 172.21.15.167:0/901014092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0254003a30 0x7f0254003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:27.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.449+0000 7f0274e00640 1 --2- 172.21.15.167:0/901014092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0254004d40 0x7f02540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:27.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.449+0000 7f026f400640 1 --2- 172.21.15.167:0/901014092 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f027015d170 0x7f0254004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:27.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.449+0000 7f026fe00640 1 -- 172.21.15.167:0/901014092 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f027015d170 msgr2=0x7f0254004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:27.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.449+0000 7f026fe00640 1 --2- 172.21.15.167:0/901014092 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f027015d170 0x7f0254004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:27.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.449+0000 7f026fe00640 1 -- 172.21.15.167:0/901014092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0254004d40 msgr2=0x7f02540051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:27.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.449+0000 7f026fe00640 1 --2- 172.21.15.167:0/901014092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0254004d40 0x7f02540051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:27.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.449+0000 7f026fe00640 1 -- 172.21.15.167:0/901014092 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0254003520 con 0x7f0254003a30 2024-05-11T12:12:27.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.449+0000 7f0274e00640 1 --2- 172.21.15.167:0/901014092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0254004d40 0x7f02540051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:27.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.450+0000 7f026fe00640 1 --2- 172.21.15.167:0/901014092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0254003a30 0x7f0254003e50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f026000c240 tx=0x7f0254005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b77166920e63de3 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:27.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.450+0000 7f026ea00640 1 -- 172.21.15.167:0/901014092 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0260007d70 con 0x7f0254003a30 2024-05-11T12:12:27.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.450+0000 7f026ea00640 1 -- 172.21.15.167:0/901014092 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0260005ad0 con 0x7f0254003a30 2024-05-11T12:12:27.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.450+0000 7f026ea00640 1 -- 172.21.15.167:0/901014092 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0260007070 con 0x7f0254003a30 2024-05-11T12:12:27.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.451+0000 7f0276e00640 1 -- 172.21.15.167:0/901014092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0254003a30 msgr2=0x7f0254003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:27.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.451+0000 7f0276e00640 1 --2- 172.21.15.167:0/901014092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0254003a30 0x7f0254003e50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f026000c240 tx=0x7f0254005830 comp rx=0 tx=0).stop 2024-05-11T12:12:27.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.451+0000 7f0274e00640 1 -- 172.21.15.167:0/901014092 reap_dead start 2024-05-11T12:12:27.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.451+0000 7f0276e00640 1 -- 172.21.15.167:0/901014092 shutdown_connections 2024-05-11T12:12:27.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.451+0000 7f0276e00640 1 -- 172.21.15.167:0/901014092 >> 172.21.15.167:0/901014092 conn(0x7f027006c450 msgr2=0x7f027006c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:27.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.451+0000 7f0276e00640 1 -- 172.21.15.167:0/901014092 shutdown_connections 2024-05-11T12:12:27.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.451+0000 7f0276e00640 1 -- 172.21.15.167:0/901014092 wait complete. 2024-05-11T12:12:27.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.453+0000 7f0276e00640 1 Processor -- start 2024-05-11T12:12:27.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.453+0000 7f0276e00640 1 -- start start 2024-05-11T12:12:27.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.454+0000 7f0276e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0254003a30 0x7f027015e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:27.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.454+0000 7f0276e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f027015ac00 0x7f027015f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:27.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.454+0000 7f0276e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f02700af920 0x7f02700afda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:27.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.454+0000 7f0276e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0270145bb0 con 0x7f027015ac00 2024-05-11T12:12:27.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.454+0000 7f0276e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f027015cce0 con 0x7f0254003a30 2024-05-11T12:12:27.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.454+0000 7f0276e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0270145d30 con 0x7f02700af920 2024-05-11T12:12:27.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.454+0000 7f026f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f027015ac00 0x7f027015f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:27.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.454+0000 7f026f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f027015ac00 0x7f027015f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:40084/0 (socket says 172.21.15.167:40084) 2024-05-11T12:12:27.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.454+0000 7f026f400640 1 -- 172.21.15.167:0/1619832694 learned_addr learned my addr 172.21.15.167:0/1619832694 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:27.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.454+0000 7f026fe00640 1 --2- 172.21.15.167:0/1619832694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0254003a30 0x7f027015e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:27.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.454+0000 7f0274e00640 1 --2- 172.21.15.167:0/1619832694 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f02700af920 0x7f02700afda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:27.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.454+0000 7f026f400640 1 -- 172.21.15.167:0/1619832694 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f02700af920 msgr2=0x7f02700afda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:27.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.454+0000 7f026f400640 1 --2- 172.21.15.167:0/1619832694 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f02700af920 0x7f02700afda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:27.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.454+0000 7f026f400640 1 -- 172.21.15.167:0/1619832694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0254003a30 msgr2=0x7f027015e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:27.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.454+0000 7f026f400640 1 --2- 172.21.15.167:0/1619832694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0254003a30 0x7f027015e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:27.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.454+0000 7f026f400640 1 -- 172.21.15.167:0/1619832694 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02700ae8c0 con 0x7f027015ac00 2024-05-11T12:12:27.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.455+0000 7f026f400640 1 --2- 172.21.15.167:0/1619832694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f027015ac00 0x7f027015f450 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f025c011600 tx=0x7f025c011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:27.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.455+0000 7f0267e00640 1 -- 172.21.15.167:0/1619832694 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f025c00f920 con 0x7f027015ac00 2024-05-11T12:12:27.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.455+0000 7f0267e00640 1 -- 172.21.15.167:0/1619832694 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f025c00fac0 con 0x7f027015ac00 2024-05-11T12:12:27.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.455+0000 7f0276e00640 1 -- 172.21.15.167:0/1619832694 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02700aeb60 con 0x7f027015ac00 2024-05-11T12:12:27.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.455+0000 7f0267e00640 1 -- 172.21.15.167:0/1619832694 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f025c00fe70 con 0x7f027015ac00 2024-05-11T12:12:27.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.455+0000 7f0276e00640 1 -- 172.21.15.167:0/1619832694 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02700af120 con 0x7f027015ac00 2024-05-11T12:12:27.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.456+0000 7f0276e00640 1 -- 172.21.15.167:0/1619832694 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0270069ca0 con 0x7f027015ac00 2024-05-11T12:12:27.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.458+0000 7f0267e00640 1 -- 172.21.15.167:0/1619832694 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f025c015460 con 0x7f027015ac00 2024-05-11T12:12:27.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.459+0000 7f0267e00640 1 --2- 172.21.15.167:0/1619832694 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0250076300 0x7f02500787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:27.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.459+0000 7f0267e00640 1 -- 172.21.15.167:0/1619832694 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f025c097ce0 con 0x7f027015ac00 2024-05-11T12:12:27.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.459+0000 7f026fe00640 1 --2- 172.21.15.167:0/1619832694 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0250076300 0x7f02500787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:27.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.460+0000 7f026fe00640 1 --2- 172.21.15.167:0/1619832694 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0250076300 0x7f02500787c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0260032000 tx=0x7f0260034040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:27.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.464+0000 7f0267e00640 1 -- 172.21.15.167:0/1619832694 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0270069ca0 con 0x7f027015ac00 2024-05-11T12:12:27.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.666+0000 7f0276e00640 1 -- 172.21.15.167:0/1619832694 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0270069520 con 0x7f027015ac00 2024-05-11T12:12:27.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.667+0000 7f0267e00640 1 -- 172.21.15.167:0/1619832694 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v44) ==== 93+0+10 (secure 0 0 0) 0x7f025c019030 con 0x7f027015ac00 2024-05-11T12:12:27.666 INFO:teuthology.orchestra.run.smithi167.stdout:pg_num: 1 2024-05-11T12:12:27.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.669+0000 7f0276e00640 1 -- 172.21.15.167:0/1619832694 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0250076300 msgr2=0x7f02500787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:27.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.669+0000 7f0276e00640 1 --2- 172.21.15.167:0/1619832694 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0250076300 0x7f02500787c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0260032000 tx=0x7f0260034040 comp rx=0 tx=0).stop 2024-05-11T12:12:27.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.669+0000 7f0276e00640 1 -- 172.21.15.167:0/1619832694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f027015ac00 msgr2=0x7f027015f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:27.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.669+0000 7f0276e00640 1 --2- 172.21.15.167:0/1619832694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f027015ac00 0x7f027015f450 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f025c011600 tx=0x7f025c011ac0 comp rx=0 tx=0).stop 2024-05-11T12:12:27.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.669+0000 7f0276e00640 1 -- 172.21.15.167:0/1619832694 shutdown_connections 2024-05-11T12:12:27.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.669+0000 7f0276e00640 1 --2- 172.21.15.167:0/1619832694 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f02700af920 0x7f02700afda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:27.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.669+0000 7f0276e00640 1 --2- 172.21.15.167:0/1619832694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f027015ac00 0x7f027015f450 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:27.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.669+0000 7f0276e00640 1 --2- 172.21.15.167:0/1619832694 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0250076300 0x7f02500787c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:27.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.669+0000 7f0276e00640 1 --2- 172.21.15.167:0/1619832694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0254003a30 0x7f027015e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:27.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.669+0000 7f0276e00640 1 -- 172.21.15.167:0/1619832694 >> 172.21.15.167:0/1619832694 conn(0x7f027006c450 msgr2=0x7f027006f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:27.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.669+0000 7f0276e00640 1 -- 172.21.15.167:0/1619832694 shutdown_connections 2024-05-11T12:12:27.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.669+0000 7f0276e00640 1 -- 172.21.15.167:0/1619832694 wait complete. 2024-05-11T12:12:27.687 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_evict_blocklisted' 2024-05-11T12:12:27.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.882+0000 7feb5d400640 1 Processor -- start 2024-05-11T12:12:27.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.883+0000 7feb5d400640 1 -- start start 2024-05-11T12:12:27.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.884+0000 7feb5d400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7feb580b3ea0 0x7feb58161490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:27.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.884+0000 7feb5d400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb581619d0 0x7feb5806d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:27.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.884+0000 7feb5d400640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb5815df30 con 0x7feb58157750 2024-05-11T12:12:27.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.884+0000 7feb5d400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7feb58161000 con 0x7feb580b3ea0 2024-05-11T12:12:27.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.884+0000 7feb5d400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7feb5815e210 con 0x7feb581619d0 2024-05-11T12:12:27.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.884+0000 7feb56800640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7feb58157750 0x7feb5815d740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:54590/0 (socket says 172.21.15.167:54590) 2024-05-11T12:12:27.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.884+0000 7feb56800640 1 -- 172.21.15.167:0/975749251 learned_addr learned my addr 172.21.15.167:0/975749251 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:27.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.884+0000 7feb55e00640 1 -- 172.21.15.167:0/975749251 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2471820590 0 0) 0x7feb5815df30 con 0x7feb58157750 2024-05-11T12:12:27.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.885+0000 7feb4de00640 1 --2- 172.21.15.167:0/975749251 >> v2:172.21.15.204:3300/0 conn(0x7feb580b3ea0 0x7feb58161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:27.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.885+0000 7feb57200640 1 --2- 172.21.15.167:0/975749251 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb581619d0 0x7feb5806d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:27.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.885+0000 7feb55e00640 1 -- 172.21.15.167:0/975749251 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb3c003520 con 0x7feb58157750 2024-05-11T12:12:27.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.885+0000 7feb55e00640 1 -- 172.21.15.167:0/975749251 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 206170804 0 0) 0x7feb3c003520 con 0x7feb58157750 2024-05-11T12:12:27.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.885+0000 7feb55e00640 1 -- 172.21.15.167:0/975749251 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb581619d0 msgr2=0x7feb5806d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:27.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.885+0000 7feb55e00640 1 --2- 172.21.15.167:0/975749251 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb581619d0 0x7feb5806d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:27.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.885+0000 7feb55e00640 1 -- 172.21.15.167:0/975749251 >> v2:172.21.15.204:3300/0 conn(0x7feb580b3ea0 msgr2=0x7feb58161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:27.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.885+0000 7feb55e00640 1 --2- 172.21.15.167:0/975749251 >> v2:172.21.15.204:3300/0 conn(0x7feb580b3ea0 0x7feb58161490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:27.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.885+0000 7feb55e00640 1 -- 172.21.15.167:0/975749251 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb5815e4f0 con 0x7feb58157750 2024-05-11T12:12:27.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.885+0000 7feb55e00640 1 -- 172.21.15.167:0/975749251 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7feb480016f0 con 0x7feb58157750 2024-05-11T12:12:27.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.885+0000 7feb55e00640 1 -- 172.21.15.167:0/975749251 >> v1:172.21.15.167:6789/0 conn(0x7feb58157750 legacy=0x7feb5815d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:27.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.885+0000 7feb55e00640 1 --2- 172.21.15.167:0/975749251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb3c003a00 0x7feb3c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:27.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.885+0000 7feb55e00640 1 --2- 172.21.15.167:0/975749251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb3c004800 0x7feb3c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:27.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.885+0000 7feb55e00640 1 --2- 172.21.15.167:0/975749251 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb581619d0 0x7feb3c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:27.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.885+0000 7feb55e00640 1 -- 172.21.15.167:0/975749251 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7feb58161000 con 0x7feb3c004800 2024-05-11T12:12:27.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.885+0000 7feb55e00640 1 -- 172.21.15.167:0/975749251 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7feb5815e210 con 0x7feb3c003a00 2024-05-11T12:12:27.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.885+0000 7feb55e00640 1 -- 172.21.15.167:0/975749251 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7feb3c00bb20 con 0x7feb581619d0 2024-05-11T12:12:27.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.885+0000 7feb4de00640 1 --2- 172.21.15.167:0/975749251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb3c004800 0x7feb3c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:27.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.886+0000 7feb56800640 1 --2- 172.21.15.167:0/975749251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb3c003a00 0x7feb3c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:27.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.886+0000 7feb57200640 1 --2- 172.21.15.167:0/975749251 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb581619d0 0x7feb3c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:27.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.886+0000 7feb4de00640 1 -- 172.21.15.167:0/975749251 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb581619d0 msgr2=0x7feb3c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:27.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.886+0000 7feb4de00640 1 --2- 172.21.15.167:0/975749251 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb581619d0 0x7feb3c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:27.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.886+0000 7feb4de00640 1 -- 172.21.15.167:0/975749251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb3c003a00 msgr2=0x7feb3c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:27.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.886+0000 7feb4de00640 1 --2- 172.21.15.167:0/975749251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb3c003a00 0x7feb3c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:27.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.886+0000 7feb4de00640 1 -- 172.21.15.167:0/975749251 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb3c003520 con 0x7feb3c004800 2024-05-11T12:12:27.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.886+0000 7feb57200640 1 --2- 172.21.15.167:0/975749251 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb581619d0 0x7feb3c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:27.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.887+0000 7feb4de00640 1 --2- 172.21.15.167:0/975749251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb3c004800 0x7feb3c004c80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7feb40007330 tx=0x7feb3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b09651fbf780fa4 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:27.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.887+0000 7feb55e00640 1 -- 172.21.15.167:0/975749251 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb40033070 con 0x7feb3c004800 2024-05-11T12:12:27.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.887+0000 7feb55e00640 1 -- 172.21.15.167:0/975749251 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb4002f660 con 0x7feb3c004800 2024-05-11T12:12:27.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.887+0000 7feb55e00640 1 -- 172.21.15.167:0/975749251 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb40009be0 con 0x7feb3c004800 2024-05-11T12:12:27.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.888+0000 7feb5d400640 1 -- 172.21.15.167:0/975749251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb3c004800 msgr2=0x7feb3c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:27.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.888+0000 7feb5d400640 1 --2- 172.21.15.167:0/975749251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb3c004800 0x7feb3c004c80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7feb40007330 tx=0x7feb3c005830 comp rx=0 tx=0).stop 2024-05-11T12:12:27.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.888+0000 7feb57200640 1 -- 172.21.15.167:0/975749251 reap_dead start 2024-05-11T12:12:27.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.888+0000 7feb5d400640 1 -- 172.21.15.167:0/975749251 shutdown_connections 2024-05-11T12:12:27.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.888+0000 7feb5d400640 1 -- 172.21.15.167:0/975749251 >> 172.21.15.167:0/975749251 conn(0x7feb580cd8f0 msgr2=0x7feb580cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:27.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.888+0000 7feb5d400640 1 -- 172.21.15.167:0/975749251 shutdown_connections 2024-05-11T12:12:27.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.888+0000 7feb5d400640 1 -- 172.21.15.167:0/975749251 wait complete. 2024-05-11T12:12:27.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.890+0000 7feb5d400640 1 Processor -- start 2024-05-11T12:12:27.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.890+0000 7feb5d400640 1 -- start start 2024-05-11T12:12:27.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.891+0000 7feb5d400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb3c004800 0x7feb5813e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:27.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.891+0000 7feb5d400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb58157d40 0x7feb581581c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:27.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.891+0000 7feb5d400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb580cf980 0x7feb580cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:27.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.891+0000 7feb5d400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7feb5806d5e0 con 0x7feb58157d40 2024-05-11T12:12:27.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.891+0000 7feb5d400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7feb5806d460 con 0x7feb580cf980 2024-05-11T12:12:27.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.891+0000 7feb5d400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7feb5806d760 con 0x7feb3c004800 2024-05-11T12:12:27.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.891+0000 7feb4de00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb58157d40 0x7feb581581c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:27.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.891+0000 7feb4de00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb58157d40 0x7feb581581c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:40102/0 (socket says 172.21.15.167:40102) 2024-05-11T12:12:27.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.891+0000 7feb4de00640 1 -- 172.21.15.167:0/1443552916 learned_addr learned my addr 172.21.15.167:0/1443552916 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:27.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.891+0000 7feb4de00640 1 -- 172.21.15.167:0/1443552916 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb3c004800 msgr2=0x7feb5813e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:27.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.891+0000 7feb4de00640 1 --2- 172.21.15.167:0/1443552916 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb3c004800 0x7feb5813e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:27.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.892+0000 7feb4de00640 1 -- 172.21.15.167:0/1443552916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb580cf980 msgr2=0x7feb580cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:27.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.892+0000 7feb4de00640 1 --2- 172.21.15.167:0/1443552916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb580cf980 0x7feb580cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:27.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.892+0000 7feb4de00640 1 -- 172.21.15.167:0/1443552916 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb5813fcc0 con 0x7feb58157d40 2024-05-11T12:12:27.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.892+0000 7feb4de00640 1 --2- 172.21.15.167:0/1443552916 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb58157d40 0x7feb581581c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7feb4002b040 tx=0x7feb40004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:27.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.892+0000 7feb4f400640 1 -- 172.21.15.167:0/1443552916 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb40033040 con 0x7feb58157d40 2024-05-11T12:12:27.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.892+0000 7feb4f400640 1 -- 172.21.15.167:0/1443552916 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb4002f460 con 0x7feb58157d40 2024-05-11T12:12:27.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.892+0000 7feb4f400640 1 -- 172.21.15.167:0/1443552916 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb40009be0 con 0x7feb58157d40 2024-05-11T12:12:27.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.892+0000 7feb5d400640 1 -- 172.21.15.167:0/1443552916 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb5813eac0 con 0x7feb58157d40 2024-05-11T12:12:27.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.892+0000 7feb5d400640 1 -- 172.21.15.167:0/1443552916 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb58158aa0 con 0x7feb58157d40 2024-05-11T12:12:27.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.893+0000 7feb5d400640 1 -- 172.21.15.167:0/1443552916 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb580d0340 con 0x7feb58157d40 2024-05-11T12:12:27.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.895+0000 7feb4f400640 1 -- 172.21.15.167:0/1443552916 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7feb40005d10 con 0x7feb58157d40 2024-05-11T12:12:27.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.896+0000 7feb4f400640 1 --2- 172.21.15.167:0/1443552916 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7feb34076300 0x7feb340787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:27.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.896+0000 7feb4f400640 1 -- 172.21.15.167:0/1443552916 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7feb400b82d0 con 0x7feb58157d40 2024-05-11T12:12:27.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.896+0000 7feb56800640 1 --2- 172.21.15.167:0/1443552916 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7feb34076300 0x7feb340787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:27.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.896+0000 7feb56800640 1 --2- 172.21.15.167:0/1443552916 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7feb34076300 0x7feb340787c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7feb48005990 tx=0x7feb48000ed0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:27.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:27.900+0000 7feb4f400640 1 -- 172.21.15.167:0/1443552916 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7feb4002f630 con 0x7feb58157d40 2024-05-11T12:12:28.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:28.179+0000 7feb5d400640 1 -- 172.21.15.167:0/1443552916 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_evict_blocklisted"]} v 0) -- 0x7feb580b3c30 con 0x7feb58157d40 2024-05-11T12:12:28.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:28.773+0000 7feb4f400640 1 -- 172.21.15.167:0/1443552916 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_evict_blocklisted"]}]=0 v114) ==== 156+0+0 (secure 0 0 0) 0x7feb40081c70 con 0x7feb58157d40 2024-05-11T12:12:28.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:28.777+0000 7feb5d400640 1 -- 172.21.15.167:0/1443552916 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7feb34076300 msgr2=0x7feb340787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:28.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:28.777+0000 7feb5d400640 1 --2- 172.21.15.167:0/1443552916 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7feb34076300 0x7feb340787c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7feb48005990 tx=0x7feb48000ed0 comp rx=0 tx=0).stop 2024-05-11T12:12:28.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:28.777+0000 7feb5d400640 1 -- 172.21.15.167:0/1443552916 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb58157d40 msgr2=0x7feb581581c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:28.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:28.777+0000 7feb5d400640 1 --2- 172.21.15.167:0/1443552916 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb58157d40 0x7feb581581c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7feb4002b040 tx=0x7feb40004ab0 comp rx=0 tx=0).stop 2024-05-11T12:12:28.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:28.777+0000 7feb5d400640 1 -- 172.21.15.167:0/1443552916 shutdown_connections 2024-05-11T12:12:28.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:28.777+0000 7feb5d400640 1 --2- 172.21.15.167:0/1443552916 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7feb34076300 0x7feb340787c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:28.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:28.777+0000 7feb5d400640 1 --2- 172.21.15.167:0/1443552916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb580cf980 0x7feb580cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:28.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:28.777+0000 7feb5d400640 1 --2- 172.21.15.167:0/1443552916 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb58157d40 0x7feb581581c0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:28.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:28.777+0000 7feb5d400640 1 --2- 172.21.15.167:0/1443552916 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb3c004800 0x7feb5813e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:28.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:28.777+0000 7feb5d400640 1 -- 172.21.15.167:0/1443552916 >> 172.21.15.167:0/1443552916 conn(0x7feb580cd8f0 msgr2=0x7feb581467c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:28.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:28.777+0000 7feb5d400640 1 -- 172.21.15.167:0/1443552916 shutdown_connections 2024-05-11T12:12:28.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:28.777+0000 7feb5d400640 1 -- 172.21.15.167:0/1443552916 wait complete. 2024-05-11T12:12:28.800 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-11T12:12:29.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.006+0000 7f2422400640 1 Processor -- start 2024-05-11T12:12:29.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.006+0000 7f2422400640 1 -- start start 2024-05-11T12:12:29.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.008+0000 7f2422400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f241c147ff0 0x7f241c0d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:29.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.008+0000 7f2422400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f241c0d2e80 0x7f241c0d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:29.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.008+0000 7f2422400640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f241c0d37c0 con 0x7f241c148cf0 2024-05-11T12:12:29.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.008+0000 7f2422400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f241c0b64f0 con 0x7f241c147ff0 2024-05-11T12:12:29.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.008+0000 7f2422400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f241c0d2040 con 0x7f241c0d2e80 2024-05-11T12:12:29.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.008+0000 7f241aa00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f241c148cf0 0x7f241c0b5de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49278/0 (socket says 172.21.15.167:49278) 2024-05-11T12:12:29.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.008+0000 7f241aa00640 1 -- 172.21.15.167:0/1661039874 learned_addr learned my addr 172.21.15.167:0/1661039874 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:29.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.009+0000 7f241b400640 1 --2- 172.21.15.167:0/1661039874 >> v2:172.21.15.204:3300/0 conn(0x7f241c147ff0 0x7f241c0d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:29.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.009+0000 7f241be00640 1 --2- 172.21.15.167:0/1661039874 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f241c0d2e80 0x7f241c0d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:29.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.009+0000 7f241a000640 1 -- 172.21.15.167:0/1661039874 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 800122409 0 0) 0x7f241c0d37c0 con 0x7f241c148cf0 2024-05-11T12:12:29.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.009+0000 7f241a000640 1 -- 172.21.15.167:0/1661039874 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2400003520 con 0x7f241c148cf0 2024-05-11T12:12:29.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.009+0000 7f241a000640 1 -- 172.21.15.167:0/1661039874 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1191822507 0 0) 0x7f2400003520 con 0x7f241c148cf0 2024-05-11T12:12:29.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.009+0000 7f241a000640 1 -- 172.21.15.167:0/1661039874 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f241c0d2e80 msgr2=0x7f241c0d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:29.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.009+0000 7f241a000640 1 --2- 172.21.15.167:0/1661039874 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f241c0d2e80 0x7f241c0d3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:29.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.009+0000 7f241a000640 1 -- 172.21.15.167:0/1661039874 >> v2:172.21.15.204:3300/0 conn(0x7f241c147ff0 msgr2=0x7f241c0d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:29.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.009+0000 7f241a000640 1 --2- 172.21.15.167:0/1661039874 >> v2:172.21.15.204:3300/0 conn(0x7f241c147ff0 0x7f241c0d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:29.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.009+0000 7f241a000640 1 -- 172.21.15.167:0/1661039874 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f241c0d1430 con 0x7f241c148cf0 2024-05-11T12:12:29.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.009+0000 7f241a000640 1 -- 172.21.15.167:0/1661039874 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f24040016f0 con 0x7f241c148cf0 2024-05-11T12:12:29.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.010+0000 7f241a000640 1 -- 172.21.15.167:0/1661039874 >> v1:172.21.15.167:6789/0 conn(0x7f241c148cf0 legacy=0x7f241c0b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:29.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.010+0000 7f241a000640 1 --2- 172.21.15.167:0/1661039874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2400003a50 0x7f2400003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:29.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.010+0000 7f241a000640 1 --2- 172.21.15.167:0/1661039874 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2400004800 0x7f2400004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:29.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.010+0000 7f241a000640 1 --2- 172.21.15.167:0/1661039874 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f241c0d2e80 0x7f24000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:29.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.010+0000 7f241a000640 1 -- 172.21.15.167:0/1661039874 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f241c0d2040 con 0x7f2400003a50 2024-05-11T12:12:29.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.010+0000 7f241a000640 1 -- 172.21.15.167:0/1661039874 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f240000b9c0 con 0x7f2400004800 2024-05-11T12:12:29.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.010+0000 7f241a000640 1 -- 172.21.15.167:0/1661039874 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f240000bb90 con 0x7f241c0d2e80 2024-05-11T12:12:29.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.010+0000 7f241b400640 1 --2- 172.21.15.167:0/1661039874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2400003a50 0x7f2400003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:29.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.010+0000 7f241aa00640 1 --2- 172.21.15.167:0/1661039874 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2400004800 0x7f2400004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:29.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.010+0000 7f241b400640 1 -- 172.21.15.167:0/1661039874 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f241c0d2e80 msgr2=0x7f24000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:29.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.010+0000 7f241be00640 1 --2- 172.21.15.167:0/1661039874 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f241c0d2e80 0x7f24000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:29.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.010+0000 7f241b400640 1 --2- 172.21.15.167:0/1661039874 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f241c0d2e80 0x7f24000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:29.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.010+0000 7f241b400640 1 -- 172.21.15.167:0/1661039874 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2400004800 msgr2=0x7f2400004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:29.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.010+0000 7f241b400640 1 --2- 172.21.15.167:0/1661039874 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2400004800 0x7f2400004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:29.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.010+0000 7f241b400640 1 -- 172.21.15.167:0/1661039874 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2400003520 con 0x7f2400003a50 2024-05-11T12:12:29.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.010+0000 7f241aa00640 1 --2- 172.21.15.167:0/1661039874 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2400004800 0x7f2400004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:29.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.010+0000 7f241be00640 1 --2- 172.21.15.167:0/1661039874 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f241c0d2e80 0x7f24000051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:29.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.011+0000 7f241b400640 1 --2- 172.21.15.167:0/1661039874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2400003a50 0x7f2400003e70 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f240c009af0 tx=0x7f2400005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d783d2fec139c88 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:29.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.012+0000 7f241a000640 1 -- 172.21.15.167:0/1661039874 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240c036070 con 0x7f2400003a50 2024-05-11T12:12:29.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.012+0000 7f241a000640 1 -- 172.21.15.167:0/1661039874 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f240c031400 con 0x7f2400003a50 2024-05-11T12:12:29.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.012+0000 7f241a000640 1 -- 172.21.15.167:0/1661039874 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240c009690 con 0x7f2400003a50 2024-05-11T12:12:29.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.012+0000 7f2422400640 1 -- 172.21.15.167:0/1661039874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2400003a50 msgr2=0x7f2400003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:29.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.012+0000 7f2422400640 1 --2- 172.21.15.167:0/1661039874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2400003a50 0x7f2400003e70 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f240c009af0 tx=0x7f2400005830 comp rx=0 tx=0).stop 2024-05-11T12:12:29.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.012+0000 7f241be00640 1 -- 172.21.15.167:0/1661039874 reap_dead start 2024-05-11T12:12:29.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.013+0000 7f2422400640 1 -- 172.21.15.167:0/1661039874 shutdown_connections 2024-05-11T12:12:29.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.013+0000 7f2422400640 1 -- 172.21.15.167:0/1661039874 >> 172.21.15.167:0/1661039874 conn(0x7f241c0b8300 msgr2=0x7f241c0b86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:29.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.013+0000 7f2422400640 1 -- 172.21.15.167:0/1661039874 shutdown_connections 2024-05-11T12:12:29.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.013+0000 7f2422400640 1 -- 172.21.15.167:0/1661039874 wait complete. 2024-05-11T12:12:29.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.014+0000 7f2422400640 1 Processor -- start 2024-05-11T12:12:29.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.015+0000 7f2422400640 1 -- start start 2024-05-11T12:12:29.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.015+0000 7f2422400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2400003a50 0x7f241c15ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:29.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.015+0000 7f2422400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f241c13dde0 0x7f241c13f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:29.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.015+0000 7f2422400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f241c13e260 0x7f241c13f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:29.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.015+0000 7f2422400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f241c0d1cd0 con 0x7f241c13dde0 2024-05-11T12:12:29.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.015+0000 7f2422400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f241c0d19f0 con 0x7f2400003a50 2024-05-11T12:12:29.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.015+0000 7f2422400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f241c1483d0 con 0x7f241c13e260 2024-05-11T12:12:29.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.016+0000 7f241aa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f241c13dde0 0x7f241c13f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:29.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.016+0000 7f241aa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f241c13dde0 0x7f241c13f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:33148/0 (socket says 172.21.15.167:33148) 2024-05-11T12:12:29.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.016+0000 7f241aa00640 1 -- 172.21.15.167:0/1896937037 learned_addr learned my addr 172.21.15.167:0/1896937037 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:29.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.016+0000 7f241aa00640 1 -- 172.21.15.167:0/1896937037 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f241c13e260 msgr2=0x7f241c13f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:29.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.016+0000 7f241aa00640 1 --2- 172.21.15.167:0/1896937037 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f241c13e260 0x7f241c13f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:29.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.016+0000 7f241aa00640 1 -- 172.21.15.167:0/1896937037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2400003a50 msgr2=0x7f241c15ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:29.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.016+0000 7f241aa00640 1 --2- 172.21.15.167:0/1896937037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2400003a50 0x7f241c15ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:29.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.016+0000 7f241aa00640 1 -- 172.21.15.167:0/1896937037 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f241c13cd40 con 0x7f241c13dde0 2024-05-11T12:12:29.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.016+0000 7f241aa00640 1 --2- 172.21.15.167:0/1896937037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f241c13dde0 0x7f241c13f060 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f240400d770 tx=0x7f240400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:29.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.017+0000 7f2413400640 1 -- 172.21.15.167:0/1896937037 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24040074e0 con 0x7f241c13dde0 2024-05-11T12:12:29.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.017+0000 7f2413400640 1 -- 172.21.15.167:0/1896937037 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f240400de20 con 0x7f241c13dde0 2024-05-11T12:12:29.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.017+0000 7f2413400640 1 -- 172.21.15.167:0/1896937037 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24040049e0 con 0x7f241c13dde0 2024-05-11T12:12:29.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.017+0000 7f2422400640 1 -- 172.21.15.167:0/1896937037 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f241c13d020 con 0x7f241c13dde0 2024-05-11T12:12:29.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.017+0000 7f2422400640 1 -- 172.21.15.167:0/1896937037 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f241c13d5e0 con 0x7f241c13dde0 2024-05-11T12:12:29.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.018+0000 7f2422400640 1 -- 172.21.15.167:0/1896937037 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f241c069520 con 0x7f241c13dde0 2024-05-11T12:12:29.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.019+0000 7f2413400640 1 -- 172.21.15.167:0/1896937037 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f240401b020 con 0x7f241c13dde0 2024-05-11T12:12:29.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.020+0000 7f2413400640 1 --2- 172.21.15.167:0/1896937037 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f23f8076300 0x7f23f80787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:29.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.020+0000 7f2413400640 1 -- 172.21.15.167:0/1896937037 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f24040971b0 con 0x7f241c13dde0 2024-05-11T12:12:29.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.020+0000 7f241b400640 1 --2- 172.21.15.167:0/1896937037 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f23f8076300 0x7f23f80787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:29.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.021+0000 7f241b400640 1 --2- 172.21.15.167:0/1896937037 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f23f8076300 0x7f23f80787c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f240c033000 tx=0x7f240c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:29.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.024+0000 7f2413400640 1 -- 172.21.15.167:0/1896937037 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2404060c70 con 0x7f241c13dde0 2024-05-11T12:12:29.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.232+0000 7f2422400640 1 -- 172.21.15.167:0/1896937037 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f241c0b3b80 con 0x7f241c13dde0 2024-05-11T12:12:29.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.237+0000 7f2413400640 1 -- 172.21.15.167:0/1896937037 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7f2404060570 con 0x7f241c13dde0 2024-05-11T12:12:29.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.240+0000 7f2422400640 1 -- 172.21.15.167:0/1896937037 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f23f8076300 msgr2=0x7f23f80787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:29.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.240+0000 7f2422400640 1 --2- 172.21.15.167:0/1896937037 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f23f8076300 0x7f23f80787c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f240c033000 tx=0x7f240c02e000 comp rx=0 tx=0).stop 2024-05-11T12:12:29.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.240+0000 7f2422400640 1 -- 172.21.15.167:0/1896937037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f241c13dde0 msgr2=0x7f241c13f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:29.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.240+0000 7f2422400640 1 --2- 172.21.15.167:0/1896937037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f241c13dde0 0x7f241c13f060 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f240400d770 tx=0x7f240400dc30 comp rx=0 tx=0).stop 2024-05-11T12:12:29.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.240+0000 7f2422400640 1 -- 172.21.15.167:0/1896937037 shutdown_connections 2024-05-11T12:12:29.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.240+0000 7f2422400640 1 --2- 172.21.15.167:0/1896937037 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f241c13e260 0x7f241c13f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:29.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.240+0000 7f2422400640 1 --2- 172.21.15.167:0/1896937037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f241c13dde0 0x7f241c13f060 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:29.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.240+0000 7f2422400640 1 --2- 172.21.15.167:0/1896937037 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f23f8076300 0x7f23f80787c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:29.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.240+0000 7f2422400640 1 --2- 172.21.15.167:0/1896937037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2400003a50 0x7f241c15ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:29.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.240+0000 7f2422400640 1 -- 172.21.15.167:0/1896937037 >> 172.21.15.167:0/1896937037 conn(0x7f241c0b8300 msgr2=0x7f241c0d3260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:29.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.240+0000 7f2422400640 1 -- 172.21.15.167:0/1896937037 shutdown_connections 2024-05-11T12:12:29.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.240+0000 7f2422400640 1 -- 172.21.15.167:0/1896937037 wait complete. 2024-05-11T12:12:29.257 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-11T12:12:29.258 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-11T12:12:29.356 DEBUG:teuthology.orchestra.run.smithi204:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-11T12:12:29.469 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-11T12:12:29.470 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:12:29.470 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:12:29.528 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-11T12:12:29.528 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:12:29.529 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:12:29.586 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052713,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-11T12:12:29.586 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:12:29.586 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:12:29.645 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-11T12:12:29.645 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:12:29.645 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:12:29.704 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052713,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-11T12:12:29.704 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-11T12:12:29.704 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:12:29.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.916+0000 7f0698800640 1 Processor -- start 2024-05-11T12:12:29.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.916+0000 7f0698800640 1 -- start start 2024-05-11T12:12:29.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.918+0000 7f0698800640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f06900af5d0 0x7f0690073ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:29.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.918+0000 7f0698800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f069006d460 0x7f069006d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:29.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.918+0000 7f0698800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f069006fe10 con 0x7f06900af1f0 2024-05-11T12:12:29.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.918+0000 7f0698800640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f06901608d0 con 0x7f06900af5d0 2024-05-11T12:12:29.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.918+0000 7f0698800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f069006de10 con 0x7f069006d460 2024-05-11T12:12:29.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.918+0000 7f0695400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f06900af5d0 0x7f0690073ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:29.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.919+0000 7f0695400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f06900af5d0 0x7f0690073ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:33154/0 (socket says 172.21.15.167:33154) 2024-05-11T12:12:29.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.919+0000 7f0695400640 1 -- 172.21.15.167:0/2514476502 learned_addr learned my addr 172.21.15.167:0/2514476502 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:29.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.919+0000 7f0695400640 1 -- 172.21.15.167:0/2514476502 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f069006d460 msgr2=0x7f069006d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:29.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.919+0000 7f0695400640 1 --2- 172.21.15.167:0/2514476502 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f069006d460 0x7f069006d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:29.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.919+0000 7f0695400640 1 -- 172.21.15.167:0/2514476502 >> v1:172.21.15.204:6789/0 conn(0x7f06900af1f0 legacy=0x7f06900cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:29.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.919+0000 7f0695400640 1 -- 172.21.15.167:0/2514476502 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f069015fcc0 con 0x7f06900af5d0 2024-05-11T12:12:29.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.920+0000 7f0695400640 1 --2- 172.21.15.167:0/2514476502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06900af5d0 0x7f0690073ce0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0680009870 tx=0x7f068002a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db3042070dc01c58 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:29.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.920+0000 7f0694a00640 1 -- 172.21.15.167:0/2514476502 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0680037070 con 0x7f06900af5d0 2024-05-11T12:12:29.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.920+0000 7f0694a00640 1 -- 172.21.15.167:0/2514476502 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0680032440 con 0x7f06900af5d0 2024-05-11T12:12:29.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.920+0000 7f0694a00640 1 -- 172.21.15.167:0/2514476502 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0680032810 con 0x7f06900af5d0 2024-05-11T12:12:29.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.921+0000 7f0698800640 1 -- 172.21.15.167:0/2514476502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06900af5d0 msgr2=0x7f0690073ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:29.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.921+0000 7f0698800640 1 --2- 172.21.15.167:0/2514476502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06900af5d0 0x7f0690073ce0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0680009870 tx=0x7f068002a130 comp rx=0 tx=0).stop 2024-05-11T12:12:29.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.921+0000 7f0698800640 1 -- 172.21.15.167:0/2514476502 shutdown_connections 2024-05-11T12:12:29.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.921+0000 7f0698800640 1 --2- 172.21.15.167:0/2514476502 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f069006d460 0x7f069006d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:29.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.921+0000 7f0698800640 1 --2- 172.21.15.167:0/2514476502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06900af5d0 0x7f0690073ce0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:29.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.921+0000 7f0698800640 1 -- 172.21.15.167:0/2514476502 >> 172.21.15.167:0/2514476502 conn(0x7f06900cd8f0 msgr2=0x7f06900cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:29.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.921+0000 7f0698800640 1 -- 172.21.15.167:0/2514476502 shutdown_connections 2024-05-11T12:12:29.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.921+0000 7f0698800640 1 -- 172.21.15.167:0/2514476502 wait complete. 2024-05-11T12:12:29.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.923+0000 7f0698800640 1 Processor -- start 2024-05-11T12:12:29.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.923+0000 7f0698800640 1 -- start start 2024-05-11T12:12:29.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.924+0000 7f0698800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f069006d460 0x7f06900c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:29.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.924+0000 7f0698800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06900af1f0 0x7f06900b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:29.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.924+0000 7f0698800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f06900af5d0 0x7f06900723c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:29.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.924+0000 7f0698800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0690160560 con 0x7f06900af1f0 2024-05-11T12:12:29.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.924+0000 7f0698800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0690160280 con 0x7f069006d460 2024-05-11T12:12:29.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.924+0000 7f0698800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f069015f920 con 0x7f06900af5d0 2024-05-11T12:12:29.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.924+0000 7f0695400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06900af1f0 0x7f06900b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:29.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.924+0000 7f0695400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06900af1f0 0x7f06900b2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:33156/0 (socket says 172.21.15.167:33156) 2024-05-11T12:12:29.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.924+0000 7f0695400640 1 -- 172.21.15.167:0/950949396 learned_addr learned my addr 172.21.15.167:0/950949396 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:29.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.925+0000 7f0695400640 1 -- 172.21.15.167:0/950949396 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f06900af5d0 msgr2=0x7f06900723c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:29.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.925+0000 7f0695400640 1 --2- 172.21.15.167:0/950949396 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f06900af5d0 0x7f06900723c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:29.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.925+0000 7f0695400640 1 -- 172.21.15.167:0/950949396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f069006d460 msgr2=0x7f06900c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:29.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.925+0000 7f0695400640 1 --2- 172.21.15.167:0/950949396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f069006d460 0x7f06900c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:29.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.925+0000 7f0695400640 1 -- 172.21.15.167:0/950949396 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06901f88c0 con 0x7f06900af1f0 2024-05-11T12:12:29.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.925+0000 7f0695400640 1 --2- 172.21.15.167:0/950949396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06900af1f0 0x7f06900b2170 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f06800090b0 tx=0x7f06800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:29.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.925+0000 7f068a000640 1 -- 172.21.15.167:0/950949396 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0680041070 con 0x7f06900af1f0 2024-05-11T12:12:29.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.925+0000 7f068a000640 1 -- 172.21.15.167:0/950949396 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f068003c890 con 0x7f06900af1f0 2024-05-11T12:12:29.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.925+0000 7f068a000640 1 -- 172.21.15.167:0/950949396 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0680037040 con 0x7f06900af1f0 2024-05-11T12:12:29.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.925+0000 7f0698800640 1 -- 172.21.15.167:0/950949396 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06901f8ba0 con 0x7f06900af1f0 2024-05-11T12:12:29.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.925+0000 7f0698800640 1 -- 172.21.15.167:0/950949396 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06901f9160 con 0x7f06900af1f0 2024-05-11T12:12:29.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.926+0000 7f0698800640 1 -- 172.21.15.167:0/950949396 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06901f9540 con 0x7f06900af1f0 2024-05-11T12:12:29.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.928+0000 7f068a000640 1 -- 172.21.15.167:0/950949396 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0680044070 con 0x7f06900af1f0 2024-05-11T12:12:29.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.928+0000 7f068a000640 1 --2- 172.21.15.167:0/950949396 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f06740762d0 0x7f0674078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:29.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.929+0000 7f068a000640 1 -- 172.21.15.167:0/950949396 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f06800b80a0 con 0x7f06900af1f0 2024-05-11T12:12:29.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.929+0000 7f0695e00640 1 --2- 172.21.15.167:0/950949396 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f06740762d0 0x7f0674078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:29.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.930+0000 7f0695e00640 1 --2- 172.21.15.167:0/950949396 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f06740762d0 0x7f0674078790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f06900b1730 tx=0x7f06840079b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:29.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:29.933+0000 7f068a000640 1 -- 172.21.15.167:0/950949396 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f06901f9540 con 0x7f06900af1f0 2024-05-11T12:12:30.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.182+0000 7f0698800640 1 -- 172.21.15.167:0/950949396 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0690069590 con 0x7f06900af1f0 2024-05-11T12:12:30.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.183+0000 7f068a000640 1 -- 172.21.15.167:0/950949396 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+3881 (secure 0 0 0) 0x7f0680081b60 con 0x7f06900af1f0 2024-05-11T12:12:30.183 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:12:30.183 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4499,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3271527772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3271527772},{"type":"v1","addr":"172.21.15.167:6836","nonce":3271527772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/3566394913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":3566394913},{"type":"v1","addr":"172.21.15.204:6835","nonce":3566394913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":4648,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2293636645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2293636645},{"type":"v1","addr":"172.21.15.167:6837","nonce":2293636645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4902,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3093163428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3093163428},{"type":"v1","addr":"172.21.15.204:6834","nonce":3093163428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[]} 2024-05-11T12:12:30.183 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 20 2024-05-11T12:12:30.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.186+0000 7f0698800640 1 -- 172.21.15.167:0/950949396 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f06740762d0 msgr2=0x7f0674078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:30.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.186+0000 7f0698800640 1 --2- 172.21.15.167:0/950949396 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f06740762d0 0x7f0674078790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f06900b1730 tx=0x7f06840079b0 comp rx=0 tx=0).stop 2024-05-11T12:12:30.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.186+0000 7f0698800640 1 -- 172.21.15.167:0/950949396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06900af1f0 msgr2=0x7f06900b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:30.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.186+0000 7f0698800640 1 --2- 172.21.15.167:0/950949396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06900af1f0 0x7f06900b2170 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f06800090b0 tx=0x7f06800046c0 comp rx=0 tx=0).stop 2024-05-11T12:12:30.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.186+0000 7f0698800640 1 -- 172.21.15.167:0/950949396 shutdown_connections 2024-05-11T12:12:30.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.186+0000 7f0698800640 1 --2- 172.21.15.167:0/950949396 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f06900af5d0 0x7f06900723c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:30.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.186+0000 7f0698800640 1 --2- 172.21.15.167:0/950949396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06900af1f0 0x7f06900b2170 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:30.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.186+0000 7f0698800640 1 --2- 172.21.15.167:0/950949396 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f06740762d0 0x7f0674078790 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:30.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.186+0000 7f0698800640 1 --2- 172.21.15.167:0/950949396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f069006d460 0x7f06900c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:30.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.186+0000 7f0698800640 1 -- 172.21.15.167:0/950949396 >> 172.21.15.167:0/950949396 conn(0x7f06900cd8f0 msgr2=0x7f0690160a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:30.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.187+0000 7f0698800640 1 -- 172.21.15.167:0/950949396 shutdown_connections 2024-05-11T12:12:30.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.187+0000 7f0698800640 1 -- 172.21.15.167:0/950949396 wait complete. 2024-05-11T12:12:30.205 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-11T12:12:30.205 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-11T12:12:30.206 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-11T12:12:30.206 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T12:12:30.206 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-11T12:12:30.206 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-11T12:12:30.206 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-11T12:12:30.206 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T12:12:30.207 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-11T12:12:30.207 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-11T12:12:30.207 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-11T12:12:30.207 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T12:12:30.207 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-11T12:12:30.207 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-11T12:12:30.207 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-11T12:12:30.207 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T12:12:30.222 INFO:tasks.ceph.mds.a:Stopped 2024-05-11T12:12:30.222 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-11T12:12:30.228 INFO:tasks.ceph.mds.b:Stopped 2024-05-11T12:12:30.228 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-11T12:12:30.229 INFO:tasks.ceph.mds.d:Stopped 2024-05-11T12:12:30.229 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-11T12:12:30.232 INFO:tasks.ceph.mds.c:Stopped 2024-05-11T12:12:30.232 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-11T12:12:30.273 INFO:tasks.ceph.mds.b:Started 2024-05-11T12:12:30.273 DEBUG:teuthology.parallel:result is None 2024-05-11T12:12:30.276 INFO:tasks.ceph.mds.d:Started 2024-05-11T12:12:30.276 DEBUG:teuthology.parallel:result is None 2024-05-11T12:12:30.279 INFO:tasks.ceph.mds.a:Started 2024-05-11T12:12:30.279 DEBUG:teuthology.parallel:result is None 2024-05-11T12:12:30.285 INFO:tasks.ceph.mds.c:Started 2024-05-11T12:12:30.285 DEBUG:teuthology.parallel:result is None 2024-05-11T12:12:30.286 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-11T12:12:30.424 INFO:tasks.ceph.mds.b.smithi204.stdout:starting mds.b at 2024-05-11T12:12:30.425 INFO:tasks.ceph.mds.d.smithi204.stdout:starting mds.d at 2024-05-11T12:12:30.428 INFO:tasks.ceph.mds.a.smithi167.stdout:starting mds.a at 2024-05-11T12:12:30.431 INFO:tasks.ceph.mds.c.smithi167.stdout:starting mds.c at 2024-05-11T12:12:30.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.467+0000 7f5bca800640 1 Processor -- start 2024-05-11T12:12:30.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.467+0000 7f5bca800640 1 -- start start 2024-05-11T12:12:30.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.468+0000 7f5bca800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5bc4159970 0x7f5bc40d1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:30.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.468+0000 7f5bca800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5bc40d2db0 0x7f5bc40d3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:30.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.468+0000 7f5bca800640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bc4159d50 con 0x7f5bc40d17b0 2024-05-11T12:12:30.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.468+0000 7f5bca800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f5bc40d04b0 con 0x7f5bc4159970 2024-05-11T12:12:30.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.468+0000 7f5bca800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5bc40d0790 con 0x7f5bc40d2db0 2024-05-11T12:12:30.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.468+0000 7f5bc2a00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f5bc40d17b0 0x7f5bc406d200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49290/0 (socket says 172.21.15.167:49290) 2024-05-11T12:12:30.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.468+0000 7f5bc2a00640 1 -- 172.21.15.167:0/2748016530 learned_addr learned my addr 172.21.15.167:0/2748016530 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:30.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.468+0000 7f5bc3400640 1 --2- 172.21.15.167:0/2748016530 >> v2:172.21.15.204:3300/0 conn(0x7f5bc4159970 0x7f5bc40d1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:30.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.468+0000 7f5bc3e00640 1 --2- 172.21.15.167:0/2748016530 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5bc40d2db0 0x7f5bc40d3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:30.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.469+0000 7f5bc3e00640 1 -- 172.21.15.167:0/2748016530 >> v2:172.21.15.204:3300/0 conn(0x7f5bc4159970 msgr2=0x7f5bc40d1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:30.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.469+0000 7f5bc2000640 1 -- 172.21.15.167:0/2748016530 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1681180721 0 0) 0x7f5bc4159d50 con 0x7f5bc40d17b0 2024-05-11T12:12:30.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.469+0000 7f5bc3e00640 1 --2- 172.21.15.167:0/2748016530 >> v2:172.21.15.204:3300/0 conn(0x7f5bc4159970 0x7f5bc40d1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:30.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.469+0000 7f5bc3e00640 1 -- 172.21.15.167:0/2748016530 >> v1:172.21.15.167:6789/0 conn(0x7f5bc40d17b0 legacy=0x7f5bc406d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:30.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.469+0000 7f5bc3e00640 1 -- 172.21.15.167:0/2748016530 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bc40d0a70 con 0x7f5bc40d2db0 2024-05-11T12:12:30.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.470+0000 7f5bc3e00640 1 --2- 172.21.15.167:0/2748016530 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5bc40d2db0 0x7f5bc40d3240 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc009d90 tx=0x7f5bbc02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=141952bbc4a8749d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:30.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.470+0000 7f5bc2000640 1 -- 172.21.15.167:0/2748016530 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc007740 con 0x7f5bc40d2db0 2024-05-11T12:12:30.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.470+0000 7f5bc2000640 1 -- 172.21.15.167:0/2748016530 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bbc0078e0 con 0x7f5bc40d2db0 2024-05-11T12:12:30.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.470+0000 7f5bc2000640 1 -- 172.21.15.167:0/2748016530 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc007cb0 con 0x7f5bc40d2db0 2024-05-11T12:12:30.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.471+0000 7f5bca800640 1 -- 172.21.15.167:0/2748016530 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5bc40d2db0 msgr2=0x7f5bc40d3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:30.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.471+0000 7f5bca800640 1 --2- 172.21.15.167:0/2748016530 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5bc40d2db0 0x7f5bc40d3240 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc009d90 tx=0x7f5bbc02a660 comp rx=0 tx=0).stop 2024-05-11T12:12:30.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.471+0000 7f5bca800640 1 -- 172.21.15.167:0/2748016530 shutdown_connections 2024-05-11T12:12:30.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.471+0000 7f5bca800640 1 --2- 172.21.15.167:0/2748016530 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5bc40d2db0 0x7f5bc40d3240 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:30.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.471+0000 7f5bca800640 1 --2- 172.21.15.167:0/2748016530 >> v2:172.21.15.204:3300/0 conn(0x7f5bc4159970 0x7f5bc40d1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:30.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.471+0000 7f5bca800640 1 -- 172.21.15.167:0/2748016530 >> 172.21.15.167:0/2748016530 conn(0x7f5bc40d51e0 msgr2=0x7f5bc40d55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:30.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.471+0000 7f5bca800640 1 -- 172.21.15.167:0/2748016530 shutdown_connections 2024-05-11T12:12:30.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.471+0000 7f5bca800640 1 -- 172.21.15.167:0/2748016530 wait complete. 2024-05-11T12:12:30.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.473+0000 7f5bca800640 1 Processor -- start 2024-05-11T12:12:30.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.473+0000 7f5bca800640 1 -- start start 2024-05-11T12:12:30.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.474+0000 7f5bca800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bc40d17b0 0x7f5bc4158220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:30.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.474+0000 7f5bca800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bc40d2db0 0x7f5bc40c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:30.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.474+0000 7f5bca800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5bc4159970 0x7f5bc40c9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:30.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.474+0000 7f5bca800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc406d910 con 0x7f5bc40d2db0 2024-05-11T12:12:30.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.474+0000 7f5bca800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc4163d00 con 0x7f5bc40d17b0 2024-05-11T12:12:30.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.474+0000 7f5bca800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5bc406da90 con 0x7f5bc4159970 2024-05-11T12:12:30.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.474+0000 7f5bc2a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bc40d2db0 0x7f5bc40c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:30.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.474+0000 7f5bc3400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bc40d17b0 0x7f5bc4158220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:30.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.474+0000 7f5bc2a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bc40d2db0 0x7f5bc40c8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:33200/0 (socket says 172.21.15.167:33200) 2024-05-11T12:12:30.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.474+0000 7f5bc2a00640 1 -- 172.21.15.167:0/3227849640 learned_addr learned my addr 172.21.15.167:0/3227849640 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:30.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.474+0000 7f5bc3e00640 1 --2- 172.21.15.167:0/3227849640 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5bc4159970 0x7f5bc40c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:30.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.475+0000 7f5bc2a00640 1 -- 172.21.15.167:0/3227849640 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5bc4159970 msgr2=0x7f5bc40c9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:30.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.475+0000 7f5bc2a00640 1 --2- 172.21.15.167:0/3227849640 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5bc4159970 0x7f5bc40c9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:30.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.475+0000 7f5bc2a00640 1 -- 172.21.15.167:0/3227849640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bc40d17b0 msgr2=0x7f5bc4158220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:30.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.475+0000 7f5bc2a00640 1 --2- 172.21.15.167:0/3227849640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bc40d17b0 0x7f5bc4158220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:30.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.475+0000 7f5bc2a00640 1 -- 172.21.15.167:0/3227849640 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bc41f5c10 con 0x7f5bc40d2db0 2024-05-11T12:12:30.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.475+0000 7f5bc3400640 1 --2- 172.21.15.167:0/3227849640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bc40d17b0 0x7f5bc4158220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:12:30.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.475+0000 7f5bc2a00640 1 --2- 172.21.15.167:0/3227849640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bc40d2db0 0x7f5bc40c8f40 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5bb000c3d0 tx=0x7f5bb000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:30.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.475+0000 7f5bbb400640 1 -- 172.21.15.167:0/3227849640 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb00074e0 con 0x7f5bc40d2db0 2024-05-11T12:12:30.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.475+0000 7f5bbb400640 1 -- 172.21.15.167:0/3227849640 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bb00047e0 con 0x7f5bc40d2db0 2024-05-11T12:12:30.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.475+0000 7f5bca800640 1 -- 172.21.15.167:0/3227849640 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bc41f4e70 con 0x7f5bc40d2db0 2024-05-11T12:12:30.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.475+0000 7f5bbb400640 1 -- 172.21.15.167:0/3227849640 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb0004c10 con 0x7f5bc40d2db0 2024-05-11T12:12:30.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.475+0000 7f5bca800640 1 -- 172.21.15.167:0/3227849640 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bc41f5410 con 0x7f5bc40d2db0 2024-05-11T12:12:30.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.476+0000 7f5bca800640 1 -- 172.21.15.167:0/3227849640 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bc40694b0 con 0x7f5bc40d2db0 2024-05-11T12:12:30.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.477+0000 7f5bbb400640 1 -- 172.21.15.167:0/3227849640 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5bb0011070 con 0x7f5bc40d2db0 2024-05-11T12:12:30.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.479+0000 7f5bbb400640 1 --2- 172.21.15.167:0/3227849640 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5ba40762d0 0x7f5ba4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:30.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.479+0000 7f5bbb400640 1 -- 172.21.15.167:0/3227849640 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f5bb0096f40 con 0x7f5bc40d2db0 2024-05-11T12:12:30.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.479+0000 7f5bc3400640 1 --2- 172.21.15.167:0/3227849640 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5ba40762d0 0x7f5ba4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:30.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.480+0000 7f5bc3400640 1 --2- 172.21.15.167:0/3227849640 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5ba40762d0 0x7f5ba4078790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4156940 tx=0x7f5bb4009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:30.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.484+0000 7f5bbb400640 1 -- 172.21.15.167:0/3227849640 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5bb0060320 con 0x7f5bc40d2db0 2024-05-11T12:12:30.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:30.696+0000 7f5bca800640 1 -- 172.21.15.167:0/3227849640 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f5bc406e7e0 con 0x7f5bc40d2db0 2024-05-11T12:12:31.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:31.814+0000 7f5bbb400640 1 -- 172.21.15.167:0/3227849640 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v49) ==== 97+0+0 (secure 0 0 0) 0x7f5bb0064620 con 0x7f5bc40d2db0 2024-05-11T12:12:31.813 INFO:teuthology.orchestra.run.smithi167.stderr: removed all blocklist entries 2024-05-11T12:12:31.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:31.818+0000 7f5bca800640 1 -- 172.21.15.167:0/3227849640 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5ba40762d0 msgr2=0x7f5ba4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:31.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:31.818+0000 7f5bca800640 1 --2- 172.21.15.167:0/3227849640 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5ba40762d0 0x7f5ba4078790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4156940 tx=0x7f5bb4009190 comp rx=0 tx=0).stop 2024-05-11T12:12:31.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:31.818+0000 7f5bca800640 1 -- 172.21.15.167:0/3227849640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bc40d2db0 msgr2=0x7f5bc40c8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:31.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:31.818+0000 7f5bca800640 1 --2- 172.21.15.167:0/3227849640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bc40d2db0 0x7f5bc40c8f40 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5bb000c3d0 tx=0x7f5bb000c890 comp rx=0 tx=0).stop 2024-05-11T12:12:31.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:31.818+0000 7f5bca800640 1 -- 172.21.15.167:0/3227849640 shutdown_connections 2024-05-11T12:12:31.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:31.818+0000 7f5bca800640 1 --2- 172.21.15.167:0/3227849640 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5bc4159970 0x7f5bc40c9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:31.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:31.818+0000 7f5bca800640 1 --2- 172.21.15.167:0/3227849640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bc40d2db0 0x7f5bc40c8f40 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:31.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:31.818+0000 7f5bca800640 1 --2- 172.21.15.167:0/3227849640 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5ba40762d0 0x7f5ba4078790 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:31.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:31.818+0000 7f5bca800640 1 --2- 172.21.15.167:0/3227849640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bc40d17b0 0x7f5bc4158220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:31.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:31.818+0000 7f5bca800640 1 -- 172.21.15.167:0/3227849640 >> 172.21.15.167:0/3227849640 conn(0x7f5bc40d51e0 msgr2=0x7f5bc4162c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:31.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:31.819+0000 7f5bca800640 1 -- 172.21.15.167:0/3227849640 shutdown_connections 2024-05-11T12:12:31.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:31.819+0000 7f5bca800640 1 -- 172.21.15.167:0/3227849640 wait complete. 2024-05-11T12:12:31.839 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-11T12:12:32.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.040+0000 7fd5b1a00640 1 Processor -- start 2024-05-11T12:12:32.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.040+0000 7fd5b1a00640 1 -- start start 2024-05-11T12:12:32.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.042+0000 7fd5b1a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd5ac06cc40 0x7fd5ac0d3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:32.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.042+0000 7fd5b1a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5ac06db20 0x7fd5ac0d40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:32.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.042+0000 7fd5b1a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5ac0cf990 con 0x7fd5ac15f460 2024-05-11T12:12:32.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.042+0000 7fd5b1a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fd5ac0cfc70 con 0x7fd5ac06cc40 2024-05-11T12:12:32.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.042+0000 7fd5b1a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd5ac0cff50 con 0x7fd5ac06db20 2024-05-11T12:12:32.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.043+0000 7fd5ab400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd5ac06cc40 0x7fd5ac0d3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:32.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.043+0000 7fd5ab400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd5ac06cc40 0x7fd5ac0d3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.167:54142/0 (socket says 172.21.15.167:54142) 2024-05-11T12:12:32.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.043+0000 7fd5ab400640 1 -- 172.21.15.167:0/1166671218 learned_addr learned my addr 172.21.15.167:0/1166671218 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:32.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.043+0000 7fd5b1000640 1 --2- 172.21.15.167:0/1166671218 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5ac06db20 0x7fd5ac0d40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:32.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.043+0000 7fd5ab400640 1 -- 172.21.15.167:0/1166671218 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5ac06db20 msgr2=0x7fd5ac0d40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:32.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.043+0000 7fd5ab400640 1 --2- 172.21.15.167:0/1166671218 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5ac06db20 0x7fd5ac0d40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:32.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.043+0000 7fd5ab400640 1 -- 172.21.15.167:0/1166671218 >> v1:172.21.15.204:6789/0 conn(0x7fd5ac15f460 legacy=0x7fd5ac0b83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:32.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.044+0000 7fd5ab400640 1 -- 172.21.15.167:0/1166671218 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5ac0d0230 con 0x7fd5ac06cc40 2024-05-11T12:12:32.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.045+0000 7fd5ab400640 1 --2- 172.21.15.167:0/1166671218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5ac06cc40 0x7fd5ac0d3bb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd598009870 tx=0x7fd59802a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d82585982e3915c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:32.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.045+0000 7fd5aaa00640 1 -- 172.21.15.167:0/1166671218 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd598037070 con 0x7fd5ac06cc40 2024-05-11T12:12:32.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.045+0000 7fd5aaa00640 1 -- 172.21.15.167:0/1166671218 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd598032440 con 0x7fd5ac06cc40 2024-05-11T12:12:32.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.045+0000 7fd5aaa00640 1 -- 172.21.15.167:0/1166671218 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd598032810 con 0x7fd5ac06cc40 2024-05-11T12:12:32.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.046+0000 7fd5b1a00640 1 -- 172.21.15.167:0/1166671218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5ac06cc40 msgr2=0x7fd5ac0d3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:32.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.046+0000 7fd5b1a00640 1 --2- 172.21.15.167:0/1166671218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5ac06cc40 0x7fd5ac0d3bb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd598009870 tx=0x7fd59802a250 comp rx=0 tx=0).stop 2024-05-11T12:12:32.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.046+0000 7fd5b1a00640 1 -- 172.21.15.167:0/1166671218 shutdown_connections 2024-05-11T12:12:32.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.046+0000 7fd5b1a00640 1 --2- 172.21.15.167:0/1166671218 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5ac06db20 0x7fd5ac0d40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:32.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.046+0000 7fd5b1a00640 1 --2- 172.21.15.167:0/1166671218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5ac06cc40 0x7fd5ac0d3bb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:32.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.046+0000 7fd5b1a00640 1 -- 172.21.15.167:0/1166671218 >> 172.21.15.167:0/1166671218 conn(0x7fd5ac0d1a50 msgr2=0x7fd5ac0d1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:32.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.046+0000 7fd5b1a00640 1 -- 172.21.15.167:0/1166671218 shutdown_connections 2024-05-11T12:12:32.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.046+0000 7fd5b1a00640 1 -- 172.21.15.167:0/1166671218 wait complete. 2024-05-11T12:12:32.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.048+0000 7fd5b1a00640 1 Processor -- start 2024-05-11T12:12:32.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.048+0000 7fd5b1a00640 1 -- start start 2024-05-11T12:12:32.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.049+0000 7fd5b1a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5ac06cc40 0x7fd5ac0cd040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:32.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.050+0000 7fd5b1a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5ac06db20 0x7fd5ac0cb270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:32.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.050+0000 7fd5b1a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5ac15f460 0x7fd5ac0b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:32.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.050+0000 7fd5b1a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ac0cf620 con 0x7fd5ac06cc40 2024-05-11T12:12:32.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.050+0000 7fd5b1a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ac0b8ab0 con 0x7fd5ac15f460 2024-05-11T12:12:32.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.050+0000 7fd5b1a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd5ac0d0a30 con 0x7fd5ac06db20 2024-05-11T12:12:32.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.050+0000 7fd5abe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5ac06cc40 0x7fd5ac0cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:32.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.050+0000 7fd5abe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5ac06cc40 0x7fd5ac0cd040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:33212/0 (socket says 172.21.15.167:33212) 2024-05-11T12:12:32.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.050+0000 7fd5abe00640 1 -- 172.21.15.167:0/608434934 learned_addr learned my addr 172.21.15.167:0/608434934 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:32.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.050+0000 7fd5abe00640 1 -- 172.21.15.167:0/608434934 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5ac06db20 msgr2=0x7fd5ac0cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:32.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.050+0000 7fd5abe00640 1 --2- 172.21.15.167:0/608434934 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5ac06db20 0x7fd5ac0cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:32.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.050+0000 7fd5abe00640 1 -- 172.21.15.167:0/608434934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5ac15f460 msgr2=0x7fd5ac0b5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:32.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.050+0000 7fd5abe00640 1 --2- 172.21.15.167:0/608434934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5ac15f460 0x7fd5ac0b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:32.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.050+0000 7fd5abe00640 1 -- 172.21.15.167:0/608434934 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5ac157bd0 con 0x7fd5ac06cc40 2024-05-11T12:12:32.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.051+0000 7fd5abe00640 1 --2- 172.21.15.167:0/608434934 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5ac06cc40 0x7fd5ac0cd040 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd59c00c3a0 tx=0x7fd59c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:32.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.051+0000 7fd5a3e00640 1 -- 172.21.15.167:0/608434934 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59c017070 con 0x7fd5ac06cc40 2024-05-11T12:12:32.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.051+0000 7fd5a3e00640 1 -- 172.21.15.167:0/608434934 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd59c0052d0 con 0x7fd5ac06cc40 2024-05-11T12:12:32.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.051+0000 7fd5a3e00640 1 -- 172.21.15.167:0/608434934 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59c003830 con 0x7fd5ac06cc40 2024-05-11T12:12:32.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.051+0000 7fd5b1a00640 1 -- 172.21.15.167:0/608434934 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5ac15f8e0 con 0x7fd5ac06cc40 2024-05-11T12:12:32.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.051+0000 7fd5b1a00640 1 -- 172.21.15.167:0/608434934 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5ac157eb0 con 0x7fd5ac06cc40 2024-05-11T12:12:32.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.052+0000 7fd5b1a00640 1 -- 172.21.15.167:0/608434934 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5ac0694b0 con 0x7fd5ac06cc40 2024-05-11T12:12:32.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.053+0000 7fd5a3e00640 1 -- 172.21.15.167:0/608434934 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd59c01a070 con 0x7fd5ac06cc40 2024-05-11T12:12:32.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.054+0000 7fd5a3e00640 1 --2- 172.21.15.167:0/608434934 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd58c0762d0 0x7fd58c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:32.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.055+0000 7fd5a3e00640 1 -- 172.21.15.167:0/608434934 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5249+0+0 (secure 0 0 0) 0x7fd59c096750 con 0x7fd5ac06cc40 2024-05-11T12:12:32.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.055+0000 7fd5ab400640 1 --2- 172.21.15.167:0/608434934 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd58c0762d0 0x7fd58c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:32.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.056+0000 7fd5ab400640 1 --2- 172.21.15.167:0/608434934 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd58c0762d0 0x7fd58c078790 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd598034000 tx=0x7fd59802c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:32.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.059+0000 7fd5a3e00640 1 -- 172.21.15.167:0/608434934 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd59c060260 con 0x7fd5ac06cc40 2024-05-11T12:12:32.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.339+0000 7fd5b1a00640 1 -- 172.21.15.167:0/608434934 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fd5ac0d4900 con 0x7fd5ac06cc40 2024-05-11T12:12:32.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.340+0000 7fd5a3e00640 1 -- 172.21.15.167:0/608434934 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) ==== 80+0+5304 (secure 0 0 0) 0x7fd59c068250 con 0x7fd5ac06cc40 2024-05-11T12:12:32.340 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:12:32.340 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T12:12:32.340 INFO:teuthology.orchestra.run.smithi167.stdout: "auth_dump": [ 2024-05-11T12:12:32.340 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:12:32.340 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "mds.a", 2024-05-11T12:12:32.340 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAEYD9msMGnFBAAamqokXQ+znxPHtHhhs611g==", 2024-05-11T12:12:32.340 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:12:32.340 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T12:12:32.340 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow *", 2024-05-11T12:12:32.340 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow *", 2024-05-11T12:12:32.340 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:12:32.340 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:12:32.341 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:12:32.341 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:12:32.341 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "mds.b", 2024-05-11T12:12:32.341 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAEYD9mqbCgMhAA5CNUf1cz77nUqLli2bTFcQ==", 2024-05-11T12:12:32.341 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:12:32.341 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T12:12:32.341 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow *", 2024-05-11T12:12:32.341 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow *", 2024-05-11T12:12:32.341 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:12:32.341 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:12:32.341 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:12:32.341 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:12:32.341 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "mds.c", 2024-05-11T12:12:32.341 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAEYD9mhylUIxAAqrIRYuNIHKfoI00zx+Gh1Q==", 2024-05-11T12:12:32.342 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:12:32.342 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T12:12:32.342 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow *", 2024-05-11T12:12:32.342 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow *", 2024-05-11T12:12:32.342 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:12:32.342 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:12:32.342 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:12:32.342 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:12:32.342 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "mds.d", 2024-05-11T12:12:32.342 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAFYD9mcUjpBRAAbx54iDHcqexfNXGKk/avlw==", 2024-05-11T12:12:32.342 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:12:32.342 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T12:12:32.342 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow *", 2024-05-11T12:12:32.343 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow *", 2024-05-11T12:12:32.343 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:12:32.343 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:12:32.343 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:12:32.343 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:12:32.343 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.0", 2024-05-11T12:12:32.343 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAJYD9mb71mBxAAKMSzkfppZVbEevwELKrqvw==", 2024-05-11T12:12:32.343 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:12:32.343 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T12:12:32.343 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T12:12:32.343 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:12:32.343 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:12:32.343 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:12:32.343 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:12:32.343 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.1", 2024-05-11T12:12:32.344 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAMYD9m/qx7EhAAUcZntottFNBwqjdc/CqFbw==", 2024-05-11T12:12:32.344 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:12:32.344 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T12:12:32.344 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T12:12:32.344 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:12:32.344 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:12:32.344 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:12:32.344 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:12:32.344 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.2", 2024-05-11T12:12:32.344 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAPYD9m5PKRKxAAFQuzh7dAN8PVEHBq+UdQvg==", 2024-05-11T12:12:32.344 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:12:32.344 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T12:12:32.344 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T12:12:32.344 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:12:32.344 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:12:32.345 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:12:32.345 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:12:32.345 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.3", 2024-05-11T12:12:32.345 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQASYD9mZAy+NhAA49k6QbSvtSxA/5t8bWbPuQ==", 2024-05-11T12:12:32.345 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:12:32.345 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T12:12:32.345 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T12:12:32.345 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:12:32.345 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:12:32.345 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:12:32.345 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:12:32.345 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.4", 2024-05-11T12:12:32.345 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAZYD9mEbsvIBAA3nkQXQTbIf7ubq1SJtUeIA==", 2024-05-11T12:12:32.345 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:12:32.345 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T12:12:32.346 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T12:12:32.346 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:12:32.346 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:12:32.346 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:12:32.346 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:12:32.346 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.5", 2024-05-11T12:12:32.346 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAcYD9mtdsHOxAAvNUSkMgQlDc2n1ak/lfRgg==", 2024-05-11T12:12:32.346 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:12:32.346 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T12:12:32.346 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T12:12:32.346 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:12:32.346 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:12:32.346 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:12:32.347 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:12:32.347 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.6", 2024-05-11T12:12:32.347 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAgYD9mr3u8CRAAm2ShwV0lJpIVudodmviusg==", 2024-05-11T12:12:32.347 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:12:32.347 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T12:12:32.347 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T12:12:32.347 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:12:32.347 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:12:32.347 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:12:32.347 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:12:32.347 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.7", 2024-05-11T12:12:32.347 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAjYD9m+0oyFRAAUVQW0kQmvtU8IE599nrWJQ==", 2024-05-11T12:12:32.347 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:12:32.347 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T12:12:32.347 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T12:12:32.348 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:12:32.348 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:12:32.348 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:12:32.348 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:12:32.348 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "client.0", 2024-05-11T12:12:32.348 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAFYD9mLK/lEhAAyERTYMl250KkBvIrAXWYSw==", 2024-05-11T12:12:32.348 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:12:32.348 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T12:12:32.348 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow r", 2024-05-11T12:12:32.348 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-11T12:12:32.348 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:12:32.348 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:12:32.348 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:12:32.348 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "client.1", 2024-05-11T12:12:32.348 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAFYD9m2oSjJhAAz53H1SvPStkxGq5V+2VFeA==", 2024-05-11T12:12:32.349 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:12:32.349 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T12:12:32.349 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow r", 2024-05-11T12:12:32.349 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow rw", 2024-05-11T12:12:32.349 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow rwx" 2024-05-11T12:12:32.349 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:12:32.349 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:12:32.349 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:12:32.349 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "client.2", 2024-05-11T12:12:32.349 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAFYD9mMzXnHBAAmJ//hwSofaZpk+6ykxVM/A==", 2024-05-11T12:12:32.349 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:12:32.349 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T12:12:32.349 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow r", 2024-05-11T12:12:32.349 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow rw", 2024-05-11T12:12:32.349 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow rwx" 2024-05-11T12:12:32.350 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:12:32.350 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:12:32.350 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:12:32.350 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "client.3", 2024-05-11T12:12:32.350 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAFYD9muOmsLxAA/tYFT4YNy3adENgVNLljdw==", 2024-05-11T12:12:32.350 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:12:32.350 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T12:12:32.350 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow r", 2024-05-11T12:12:32.350 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow rw", 2024-05-11T12:12:32.350 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow rwx" 2024-05-11T12:12:32.350 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:12:32.350 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:12:32.350 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:12:32.350 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "client.admin", 2024-05-11T12:12:32.351 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQADYD9mItegDBAALtuDP6qzAvbrwiROf94biw==", 2024-05-11T12:12:32.351 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:12:32.351 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow *", 2024-05-11T12:12:32.351 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow *", 2024-05-11T12:12:32.351 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow *", 2024-05-11T12:12:32.351 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:12:32.351 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:12:32.351 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:12:32.351 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:12:32.351 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "mgr.x", 2024-05-11T12:12:32.351 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAEYD9mAZTKCRAAJbY821O+4pUMC0DgxRa3og==", 2024-05-11T12:12:32.351 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:12:32.351 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow *", 2024-05-11T12:12:32.351 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile mgr", 2024-05-11T12:12:32.351 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:12:32.352 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:12:32.352 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:12:32.352 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:12:32.352 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "mgr.y", 2024-05-11T12:12:32.352 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQADYD9m2cZRORAAX9kYcT5/t4f5jQRZwXHHzw==", 2024-05-11T12:12:32.352 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:12:32.352 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow *", 2024-05-11T12:12:32.352 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile mgr", 2024-05-11T12:12:32.352 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:12:32.352 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:12:32.352 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:12:32.352 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-05-11T12:12:32.352 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T12:12:32.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.343+0000 7fd5b1a00640 1 -- 172.21.15.167:0/608434934 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd58c0762d0 msgr2=0x7fd58c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:32.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.343+0000 7fd5b1a00640 1 --2- 172.21.15.167:0/608434934 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd58c0762d0 0x7fd58c078790 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd598034000 tx=0x7fd59802c000 comp rx=0 tx=0).stop 2024-05-11T12:12:32.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.343+0000 7fd5b1a00640 1 -- 172.21.15.167:0/608434934 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5ac06cc40 msgr2=0x7fd5ac0cd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:32.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.343+0000 7fd5b1a00640 1 --2- 172.21.15.167:0/608434934 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5ac06cc40 0x7fd5ac0cd040 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd59c00c3a0 tx=0x7fd59c00c860 comp rx=0 tx=0).stop 2024-05-11T12:12:32.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.343+0000 7fd5b1a00640 1 -- 172.21.15.167:0/608434934 shutdown_connections 2024-05-11T12:12:32.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.343+0000 7fd5b1a00640 1 --2- 172.21.15.167:0/608434934 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd58c0762d0 0x7fd58c078790 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:32.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.343+0000 7fd5b1a00640 1 --2- 172.21.15.167:0/608434934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5ac15f460 0x7fd5ac0b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:32.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.343+0000 7fd5b1a00640 1 --2- 172.21.15.167:0/608434934 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5ac06db20 0x7fd5ac0cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:32.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.343+0000 7fd5b1a00640 1 --2- 172.21.15.167:0/608434934 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5ac06cc40 0x7fd5ac0cd040 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:32.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.343+0000 7fd5b1a00640 1 -- 172.21.15.167:0/608434934 >> 172.21.15.167:0/608434934 conn(0x7fd5ac0d1a50 msgr2=0x7fd5ac0ba280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:32.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.343+0000 7fd5b1a00640 1 -- 172.21.15.167:0/608434934 shutdown_connections 2024-05-11T12:12:32.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.343+0000 7fd5b1a00640 1 -- 172.21.15.167:0/608434934 wait complete. 2024-05-11T12:12:32.359 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-11T12:12:32.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.562+0000 7fb189400640 1 Processor -- start 2024-05-11T12:12:32.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.562+0000 7fb189400640 1 -- start start 2024-05-11T12:12:32.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.563+0000 7fb189400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb18413eeb0 0x7fb18406f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:32.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.563+0000 7fb189400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb18413f290 0x7fb18406f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:32.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.563+0000 7fb189400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1840b50a0 0x7fb1840735c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:32.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.563+0000 7fb189400640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fb18415fb90 con 0x7fb18413f290 2024-05-11T12:12:32.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.563+0000 7fb189400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fb18415fe70 con 0x7fb18413eeb0 2024-05-11T12:12:32.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.563+0000 7fb189400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb184160150 con 0x7fb1840b50a0 2024-05-11T12:12:32.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.564+0000 7fb179e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb18413f290 0x7fb18406f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:32.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.564+0000 7fb179e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb18413f290 0x7fb18406f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:33222/0 (socket says 172.21.15.167:33222) 2024-05-11T12:12:32.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.564+0000 7fb179e00640 1 -- 172.21.15.167:0/1333567572 learned_addr learned my addr 172.21.15.167:0/1333567572 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:32.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.564+0000 7fb183200640 1 --2- 172.21.15.167:0/1333567572 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1840b50a0 0x7fb1840735c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:32.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.564+0000 7fb182800640 1 --2- 172.21.15.167:0/1333567572 >> v2:172.21.15.204:3300/0 conn(0x7fb18413eeb0 0x7fb18406f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:32.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.564+0000 7fb179e00640 1 -- 172.21.15.167:0/1333567572 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1840b50a0 msgr2=0x7fb1840735c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:32.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.564+0000 7fb179e00640 1 --2- 172.21.15.167:0/1333567572 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1840b50a0 0x7fb1840735c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:32.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.564+0000 7fb179e00640 1 -- 172.21.15.167:0/1333567572 >> v2:172.21.15.204:3300/0 conn(0x7fb18413eeb0 msgr2=0x7fb18406f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:32.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.564+0000 7fb179e00640 1 --2- 172.21.15.167:0/1333567572 >> v2:172.21.15.204:3300/0 conn(0x7fb18413eeb0 0x7fb18406f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:32.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.564+0000 7fb179e00640 1 -- 172.21.15.167:0/1333567572 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb184160430 con 0x7fb18413f290 2024-05-11T12:12:32.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.565+0000 7fb179e00640 1 --2- 172.21.15.167:0/1333567572 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb18413f290 0x7fb18406f9c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb1840b6590 tx=0x7fb16c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20d7d956729c79d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:32.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.566+0000 7fb181e00640 1 -- 172.21.15.167:0/1333567572 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16c037070 con 0x7fb18413f290 2024-05-11T12:12:32.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.566+0000 7fb181e00640 1 -- 172.21.15.167:0/1333567572 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb16c032440 con 0x7fb18413f290 2024-05-11T12:12:32.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.566+0000 7fb181e00640 1 -- 172.21.15.167:0/1333567572 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16c032810 con 0x7fb18413f290 2024-05-11T12:12:32.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.566+0000 7fb189400640 1 -- 172.21.15.167:0/1333567572 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb18413f290 msgr2=0x7fb18406f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:32.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.566+0000 7fb189400640 1 --2- 172.21.15.167:0/1333567572 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb18413f290 0x7fb18406f9c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb1840b6590 tx=0x7fb16c02a140 comp rx=0 tx=0).stop 2024-05-11T12:12:32.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.567+0000 7fb189400640 1 -- 172.21.15.167:0/1333567572 shutdown_connections 2024-05-11T12:12:32.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.567+0000 7fb189400640 1 --2- 172.21.15.167:0/1333567572 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1840b50a0 0x7fb1840735c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:32.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.567+0000 7fb189400640 1 --2- 172.21.15.167:0/1333567572 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb18413f290 0x7fb18406f9c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:32.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.567+0000 7fb189400640 1 --2- 172.21.15.167:0/1333567572 >> v2:172.21.15.204:3300/0 conn(0x7fb18413eeb0 0x7fb18406f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:32.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.567+0000 7fb189400640 1 -- 172.21.15.167:0/1333567572 >> 172.21.15.167:0/1333567572 conn(0x7fb1840cd8f0 msgr2=0x7fb1840cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:32.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.567+0000 7fb189400640 1 -- 172.21.15.167:0/1333567572 shutdown_connections 2024-05-11T12:12:32.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.567+0000 7fb189400640 1 -- 172.21.15.167:0/1333567572 wait complete. 2024-05-11T12:12:32.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.568+0000 7fb189400640 1 Processor -- start 2024-05-11T12:12:32.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.568+0000 7fb189400640 1 -- start start 2024-05-11T12:12:32.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.569+0000 7fb189400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1840b50a0 0x7fb1841f8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:32.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.569+0000 7fb189400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb18413eeb0 0x7fb1841fcd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:32.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.569+0000 7fb189400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb18413f290 0x7fb1841fd260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:32.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.569+0000 7fb189400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb184073c80 con 0x7fb18413f290 2024-05-11T12:12:32.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.570+0000 7fb189400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb184073b00 con 0x7fb1840b50a0 2024-05-11T12:12:32.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.570+0000 7fb189400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb184073e00 con 0x7fb18413eeb0 2024-05-11T12:12:32.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.570+0000 7fb183200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb18413f290 0x7fb1841fd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:32.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.570+0000 7fb183200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb18413f290 0x7fb1841fd260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:33232/0 (socket says 172.21.15.167:33232) 2024-05-11T12:12:32.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.570+0000 7fb183200640 1 -- 172.21.15.167:0/2907381370 learned_addr learned my addr 172.21.15.167:0/2907381370 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:32.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.570+0000 7fb182800640 1 --2- 172.21.15.167:0/2907381370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1840b50a0 0x7fb1841f8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:32.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.570+0000 7fb183200640 1 -- 172.21.15.167:0/2907381370 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb18413eeb0 msgr2=0x7fb1841fcd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:32.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.570+0000 7fb183200640 1 --2- 172.21.15.167:0/2907381370 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb18413eeb0 0x7fb1841fcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:32.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.570+0000 7fb183200640 1 -- 172.21.15.167:0/2907381370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1840b50a0 msgr2=0x7fb1841f8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:32.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.570+0000 7fb183200640 1 --2- 172.21.15.167:0/2907381370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1840b50a0 0x7fb1841f8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:32.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.570+0000 7fb183200640 1 -- 172.21.15.167:0/2907381370 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1840b1ff0 con 0x7fb18413f290 2024-05-11T12:12:32.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.571+0000 7fb183200640 1 --2- 172.21.15.167:0/2907381370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb18413f290 0x7fb1841fd260 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb17c007e40 tx=0x7fb17c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:32.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.571+0000 7fb17b400640 1 -- 172.21.15.167:0/2907381370 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17c014620 con 0x7fb18413f290 2024-05-11T12:12:32.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.571+0000 7fb17b400640 1 -- 172.21.15.167:0/2907381370 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb17c019070 con 0x7fb18413f290 2024-05-11T12:12:32.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.571+0000 7fb17b400640 1 -- 172.21.15.167:0/2907381370 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17c014a20 con 0x7fb18413f290 2024-05-11T12:12:32.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.571+0000 7fb189400640 1 -- 172.21.15.167:0/2907381370 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1840b22d0 con 0x7fb18413f290 2024-05-11T12:12:32.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.571+0000 7fb189400640 1 -- 172.21.15.167:0/2907381370 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb18415c7b0 con 0x7fb18413f290 2024-05-11T12:12:32.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.572+0000 7fb189400640 1 -- 172.21.15.167:0/2907381370 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb184069590 con 0x7fb18413f290 2024-05-11T12:12:32.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.574+0000 7fb17b400640 1 -- 172.21.15.167:0/2907381370 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb17c014bc0 con 0x7fb18413f290 2024-05-11T12:12:32.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.574+0000 7fb17b400640 1 --2- 172.21.15.167:0/2907381370 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb1600762d0 0x7fb160078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:32.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.575+0000 7fb17b400640 1 -- 172.21.15.167:0/2907381370 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5249+0+0 (secure 0 0 0) 0x7fb17c097dd0 con 0x7fb18413f290 2024-05-11T12:12:32.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.575+0000 7fb182800640 1 --2- 172.21.15.167:0/2907381370 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb1600762d0 0x7fb160078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:32.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.575+0000 7fb182800640 1 --2- 172.21.15.167:0/2907381370 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb1600762d0 0x7fb160078790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb174006fd0 tx=0x7fb174008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:32.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.579+0000 7fb17b400640 1 -- 172.21.15.167:0/2907381370 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb17c061850 con 0x7fb18413f290 2024-05-11T12:12:32.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.790+0000 7fb189400640 1 -- 172.21.15.167:0/2907381370 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb1841fdb00 con 0x7fb18413f290 2024-05-11T12:12:32.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.791+0000 7fb17b400640 1 -- 172.21.15.167:0/2907381370 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v49) ==== 84+0+65 (secure 0 0 0) 0x7fb17c061150 con 0x7fb18413f290 2024-05-11T12:12:32.790 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:12:32.790 INFO:teuthology.orchestra.run.smithi167.stdout:[ 2024-05-11T12:12:32.790 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:12:32.790 INFO:teuthology.orchestra.run.smithi167.stdout: "poolnum": 1, 2024-05-11T12:12:32.791 INFO:teuthology.orchestra.run.smithi167.stdout: "poolname": ".mgr" 2024-05-11T12:12:32.791 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:12:32.791 INFO:teuthology.orchestra.run.smithi167.stdout:] 2024-05-11T12:12:32.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.793+0000 7fb189400640 1 -- 172.21.15.167:0/2907381370 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb1600762d0 msgr2=0x7fb160078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:32.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.793+0000 7fb189400640 1 --2- 172.21.15.167:0/2907381370 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb1600762d0 0x7fb160078790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb174006fd0 tx=0x7fb174008040 comp rx=0 tx=0).stop 2024-05-11T12:12:32.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.793+0000 7fb189400640 1 -- 172.21.15.167:0/2907381370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb18413f290 msgr2=0x7fb1841fd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:32.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.793+0000 7fb189400640 1 --2- 172.21.15.167:0/2907381370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb18413f290 0x7fb1841fd260 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb17c007e40 tx=0x7fb17c00e4b0 comp rx=0 tx=0).stop 2024-05-11T12:12:32.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.793+0000 7fb189400640 1 -- 172.21.15.167:0/2907381370 shutdown_connections 2024-05-11T12:12:32.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.793+0000 7fb189400640 1 --2- 172.21.15.167:0/2907381370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb18413f290 0x7fb1841fd260 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:32.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.794+0000 7fb189400640 1 --2- 172.21.15.167:0/2907381370 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb18413eeb0 0x7fb1841fcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:32.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.794+0000 7fb189400640 1 --2- 172.21.15.167:0/2907381370 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb1600762d0 0x7fb160078790 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:32.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.794+0000 7fb189400640 1 --2- 172.21.15.167:0/2907381370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1840b50a0 0x7fb1841f8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:32.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.794+0000 7fb189400640 1 -- 172.21.15.167:0/2907381370 >> 172.21.15.167:0/2907381370 conn(0x7fb1840cd8f0 msgr2=0x7fb18406f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:32.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.794+0000 7fb189400640 1 -- 172.21.15.167:0/2907381370 shutdown_connections 2024-05-11T12:12:32.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:32.794+0000 7fb189400640 1 -- 172.21.15.167:0/2907381370 wait complete. 2024-05-11T12:12:32.812 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-11T12:12:32.812 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-11T12:12:33.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.019+0000 7f1746c00640 1 Processor -- start 2024-05-11T12:12:33.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.019+0000 7f1746c00640 1 -- start start 2024-05-11T12:12:33.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.021+0000 7f1746c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f174015f260 0x7f17401637e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:33.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.021+0000 7f1746c00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f17400cfd90 0x7f17400d3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:33.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.021+0000 7f1746c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f17400d4310 0x7f174006d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:33.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.021+0000 7f1746c00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f17400d3940 con 0x7f17400cfd90 2024-05-11T12:12:33.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.021+0000 7f1746c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f1740163d20 con 0x7f174015f260 2024-05-11T12:12:33.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.021+0000 7f1746c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f174015f640 con 0x7f17400d4310 2024-05-11T12:12:33.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.021+0000 7f1744e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f17400cfd90 0x7f17400d3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:33.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.022+0000 7f1744e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f17400cfd90 0x7f17400d3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:33236/0 (socket says 172.21.15.167:33236) 2024-05-11T12:12:33.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.022+0000 7f1744e00640 1 -- 172.21.15.167:0/1854592187 learned_addr learned my addr 172.21.15.167:0/1854592187 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:33.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.022+0000 7f1746200640 1 --2- 172.21.15.167:0/1854592187 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f17400d4310 0x7f174006d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:33.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.022+0000 7f1745800640 1 --2- 172.21.15.167:0/1854592187 >> v2:172.21.15.204:3300/0 conn(0x7f174015f260 0x7f17401637e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:33.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.022+0000 7f1744e00640 1 -- 172.21.15.167:0/1854592187 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f17400d4310 msgr2=0x7f174006d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:33.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.022+0000 7f1744e00640 1 --2- 172.21.15.167:0/1854592187 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f17400d4310 0x7f174006d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:33.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.022+0000 7f1744e00640 1 -- 172.21.15.167:0/1854592187 >> v2:172.21.15.204:3300/0 conn(0x7f174015f260 msgr2=0x7f17401637e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:33.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.022+0000 7f1744e00640 1 --2- 172.21.15.167:0/1854592187 >> v2:172.21.15.204:3300/0 conn(0x7f174015f260 0x7f17401637e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:33.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.022+0000 7f1744e00640 1 -- 172.21.15.167:0/1854592187 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17400d2d30 con 0x7f17400cfd90 2024-05-11T12:12:33.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.022+0000 7f1745800640 1 --2- 172.21.15.167:0/1854592187 >> v2:172.21.15.204:3300/0 conn(0x7f174015f260 0x7f17401637e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:33.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.023+0000 7f1744e00640 1 --2- 172.21.15.167:0/1854592187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17400cfd90 0x7f17400d3dd0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f174008d4e0 tx=0x7f172c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18046bbdb469b936 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:33.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.023+0000 7f173fe00640 1 -- 172.21.15.167:0/1854592187 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c037070 con 0x7f17400cfd90 2024-05-11T12:12:33.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.023+0000 7f173fe00640 1 -- 172.21.15.167:0/1854592187 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f172c032440 con 0x7f17400cfd90 2024-05-11T12:12:33.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.023+0000 7f173fe00640 1 -- 172.21.15.167:0/1854592187 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c032810 con 0x7f17400cfd90 2024-05-11T12:12:33.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.024+0000 7f1746c00640 1 -- 172.21.15.167:0/1854592187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17400cfd90 msgr2=0x7f17400d3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:33.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.024+0000 7f1746c00640 1 --2- 172.21.15.167:0/1854592187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17400cfd90 0x7f17400d3dd0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f174008d4e0 tx=0x7f172c02a1e0 comp rx=0 tx=0).stop 2024-05-11T12:12:33.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.024+0000 7f1746c00640 1 -- 172.21.15.167:0/1854592187 shutdown_connections 2024-05-11T12:12:33.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.024+0000 7f1746c00640 1 --2- 172.21.15.167:0/1854592187 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f17400d4310 0x7f174006d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:33.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.024+0000 7f1746c00640 1 --2- 172.21.15.167:0/1854592187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17400cfd90 0x7f17400d3dd0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:33.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.024+0000 7f1746c00640 1 --2- 172.21.15.167:0/1854592187 >> v2:172.21.15.204:3300/0 conn(0x7f174015f260 0x7f17401637e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:33.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.024+0000 7f1746c00640 1 -- 172.21.15.167:0/1854592187 >> 172.21.15.167:0/1854592187 conn(0x7f17400b6cd0 msgr2=0x7f17400b70e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:33.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.024+0000 7f1746c00640 1 -- 172.21.15.167:0/1854592187 shutdown_connections 2024-05-11T12:12:33.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.024+0000 7f1746c00640 1 -- 172.21.15.167:0/1854592187 wait complete. 2024-05-11T12:12:33.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.026+0000 7f1746c00640 1 Processor -- start 2024-05-11T12:12:33.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.026+0000 7f1746c00640 1 -- start start 2024-05-11T12:12:33.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.027+0000 7f1746c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17400cfd90 0x7f17400b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:33.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.027+0000 7f1746c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f17400d4310 0x7f17400aea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:33.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.027+0000 7f1746c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f174015f260 0x7f17400aefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:33.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.027+0000 7f1746c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f17400d35d0 con 0x7f174015f260 2024-05-11T12:12:33.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.027+0000 7f1746c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f17400d32f0 con 0x7f17400cfd90 2024-05-11T12:12:33.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.027+0000 7f1746c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f174006d5a0 con 0x7f17400d4310 2024-05-11T12:12:33.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.027+0000 7f1746200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f174015f260 0x7f17400aefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:33.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.027+0000 7f1746200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f174015f260 0x7f17400aefa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:33238/0 (socket says 172.21.15.167:33238) 2024-05-11T12:12:33.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.027+0000 7f1746200640 1 -- 172.21.15.167:0/1046502441 learned_addr learned my addr 172.21.15.167:0/1046502441 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:33.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.028+0000 7f1744e00640 1 --2- 172.21.15.167:0/1046502441 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f17400d4310 0x7f17400aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:33.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.028+0000 7f1746200640 1 -- 172.21.15.167:0/1046502441 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f17400d4310 msgr2=0x7f17400aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:33.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.028+0000 7f1746200640 1 --2- 172.21.15.167:0/1046502441 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f17400d4310 0x7f17400aea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:33.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.028+0000 7f1746200640 1 -- 172.21.15.167:0/1046502441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17400cfd90 msgr2=0x7f17400b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:33.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.028+0000 7f1746200640 1 --2- 172.21.15.167:0/1046502441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17400cfd90 0x7f17400b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:33.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.028+0000 7f1746200640 1 -- 172.21.15.167:0/1046502441 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17401f5010 con 0x7f174015f260 2024-05-11T12:12:33.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.028+0000 7f1744e00640 1 --2- 172.21.15.167:0/1046502441 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f17400d4310 0x7f17400aea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:33.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.028+0000 7f1746200640 1 --2- 172.21.15.167:0/1046502441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f174015f260 0x7f17400aefa0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1738007e30 tx=0x7f173800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:33.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.028+0000 7f173d600640 1 -- 172.21.15.167:0/1046502441 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1738014420 con 0x7f174015f260 2024-05-11T12:12:33.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.028+0000 7f173d600640 1 -- 172.21.15.167:0/1046502441 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1738019070 con 0x7f174015f260 2024-05-11T12:12:33.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.028+0000 7f173d600640 1 -- 172.21.15.167:0/1046502441 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1738014800 con 0x7f174015f260 2024-05-11T12:12:33.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.028+0000 7f1746c00640 1 -- 172.21.15.167:0/1046502441 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17400af5d0 con 0x7f174015f260 2024-05-11T12:12:33.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.029+0000 7f1746c00640 1 -- 172.21.15.167:0/1046502441 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17401f52b0 con 0x7f174015f260 2024-05-11T12:12:33.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.030+0000 7f1746c00640 1 -- 172.21.15.167:0/1046502441 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17400694b0 con 0x7f174015f260 2024-05-11T12:12:33.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.031+0000 7f173d600640 1 -- 172.21.15.167:0/1046502441 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f17380149a0 con 0x7f174015f260 2024-05-11T12:12:33.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.032+0000 7f173d600640 1 --2- 172.21.15.167:0/1046502441 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f17200762d0 0x7f1720078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:33.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.032+0000 7f173d600640 1 -- 172.21.15.167:0/1046502441 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5249+0+0 (secure 0 0 0) 0x7f1738097d20 con 0x7f174015f260 2024-05-11T12:12:33.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.032+0000 7f1745800640 1 --2- 172.21.15.167:0/1046502441 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f17200762d0 0x7f1720078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:33.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.033+0000 7f1745800640 1 --2- 172.21.15.167:0/1046502441 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f17200762d0 0x7f1720078790 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1730006fd0 tx=0x7f1730008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:33.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.037+0000 7f173d600640 1 -- 172.21.15.167:0/1046502441 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f17380617a0 con 0x7f174015f260 2024-05-11T12:12:33.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.245+0000 7f1746c00640 1 -- 172.21.15.167:0/1046502441 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f17400b8be0 con 0x7f174015f260 2024-05-11T12:12:33.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.831+0000 7f173d600640 1 -- 172.21.15.167:0/1046502441 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v50) ==== 138+0+0 (secure 0 0 0) 0x7f17380610a0 con 0x7f174015f260 2024-05-11T12:12:33.831 INFO:teuthology.orchestra.run.smithi167.stderr:pool 'cephfs_metadata' created 2024-05-11T12:12:33.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.835+0000 7f1746c00640 1 -- 172.21.15.167:0/1046502441 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f17200762d0 msgr2=0x7f1720078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:33.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.835+0000 7f1746c00640 1 --2- 172.21.15.167:0/1046502441 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f17200762d0 0x7f1720078790 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1730006fd0 tx=0x7f1730008040 comp rx=0 tx=0).stop 2024-05-11T12:12:33.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.835+0000 7f1746c00640 1 -- 172.21.15.167:0/1046502441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f174015f260 msgr2=0x7f17400aefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:33.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.835+0000 7f1746c00640 1 --2- 172.21.15.167:0/1046502441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f174015f260 0x7f17400aefa0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1738007e30 tx=0x7f173800e4b0 comp rx=0 tx=0).stop 2024-05-11T12:12:33.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.836+0000 7f1746c00640 1 -- 172.21.15.167:0/1046502441 shutdown_connections 2024-05-11T12:12:33.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.836+0000 7f1746c00640 1 --2- 172.21.15.167:0/1046502441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f174015f260 0x7f17400aefa0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:33.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.836+0000 7f1746c00640 1 --2- 172.21.15.167:0/1046502441 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f17400d4310 0x7f17400aea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:33.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.836+0000 7f1746c00640 1 --2- 172.21.15.167:0/1046502441 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f17200762d0 0x7f1720078790 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:33.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.836+0000 7f1746c00640 1 --2- 172.21.15.167:0/1046502441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17400cfd90 0x7f17400b62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:33.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.836+0000 7f1746c00640 1 -- 172.21.15.167:0/1046502441 >> 172.21.15.167:0/1046502441 conn(0x7f17400b6cd0 msgr2=0x7f17400b9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:33.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.836+0000 7f1746c00640 1 -- 172.21.15.167:0/1046502441 shutdown_connections 2024-05-11T12:12:33.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:33.837+0000 7f1746c00640 1 -- 172.21.15.167:0/1046502441 wait complete. 2024-05-11T12:12:33.863 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-11T12:12:34.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.054+0000 7fc222200640 1 Processor -- start 2024-05-11T12:12:34.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.054+0000 7fc222200640 1 -- start start 2024-05-11T12:12:34.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.056+0000 7fc222200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc21c162a40 0x7fc21c0b84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:34.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.056+0000 7fc222200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc21c0b3fb0 0x7fc21c0d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:34.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.056+0000 7fc222200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc21c06dec0 con 0x7fc21c0d3de0 2024-05-11T12:12:34.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.056+0000 7fc222200640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fc21c0b89f0 con 0x7fc21c162a40 2024-05-11T12:12:34.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.056+0000 7fc222200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fc21c0d2b70 con 0x7fc21c0b3fb0 2024-05-11T12:12:34.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.056+0000 7fc220e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc21c162a40 0x7fc21c0b84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:34.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.057+0000 7fc220e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc21c162a40 0x7fc21c0b84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:33242/0 (socket says 172.21.15.167:33242) 2024-05-11T12:12:34.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.057+0000 7fc220e00640 1 -- 172.21.15.167:0/2808213224 learned_addr learned my addr 172.21.15.167:0/2808213224 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:34.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.057+0000 7fc21be00640 1 --2- 172.21.15.167:0/2808213224 >> v2:172.21.15.204:3300/0 conn(0x7fc21c0b3fb0 0x7fc21c0d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:34.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.057+0000 7fc220e00640 1 -- 172.21.15.167:0/2808213224 >> v2:172.21.15.204:3300/0 conn(0x7fc21c0b3fb0 msgr2=0x7fc21c0d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:34.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.057+0000 7fc220e00640 1 --2- 172.21.15.167:0/2808213224 >> v2:172.21.15.204:3300/0 conn(0x7fc21c0b3fb0 0x7fc21c0d38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:34.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.057+0000 7fc220e00640 1 -- 172.21.15.167:0/2808213224 >> v1:172.21.15.204:6789/0 conn(0x7fc21c0d3de0 legacy=0x7fc21c0cff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:34.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.057+0000 7fc220e00640 1 -- 172.21.15.167:0/2808213224 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc21c0d2dc0 con 0x7fc21c162a40 2024-05-11T12:12:34.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.058+0000 7fc220e00640 1 --2- 172.21.15.167:0/2808213224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc21c162a40 0x7fc21c0b84b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc21c06cb90 tx=0x7fc21002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdba4a88cc010a1e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:34.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.059+0000 7fc21b400640 1 -- 172.21.15.167:0/2808213224 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc210037070 con 0x7fc21c162a40 2024-05-11T12:12:34.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.059+0000 7fc21b400640 1 -- 172.21.15.167:0/2808213224 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc210032440 con 0x7fc21c162a40 2024-05-11T12:12:34.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.059+0000 7fc21b400640 1 -- 172.21.15.167:0/2808213224 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc210032810 con 0x7fc21c162a40 2024-05-11T12:12:34.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.059+0000 7fc222200640 1 -- 172.21.15.167:0/2808213224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc21c162a40 msgr2=0x7fc21c0b84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:34.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.059+0000 7fc222200640 1 --2- 172.21.15.167:0/2808213224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc21c162a40 0x7fc21c0b84b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc21c06cb90 tx=0x7fc21002a270 comp rx=0 tx=0).stop 2024-05-11T12:12:34.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.059+0000 7fc222200640 1 -- 172.21.15.167:0/2808213224 shutdown_connections 2024-05-11T12:12:34.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.059+0000 7fc222200640 1 --2- 172.21.15.167:0/2808213224 >> v2:172.21.15.204:3300/0 conn(0x7fc21c0b3fb0 0x7fc21c0d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:34.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.059+0000 7fc222200640 1 --2- 172.21.15.167:0/2808213224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc21c162a40 0x7fc21c0b84b0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:34.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.059+0000 7fc222200640 1 -- 172.21.15.167:0/2808213224 >> 172.21.15.167:0/2808213224 conn(0x7fc21c0d1c00 msgr2=0x7fc21c0d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:34.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.060+0000 7fc222200640 1 -- 172.21.15.167:0/2808213224 shutdown_connections 2024-05-11T12:12:34.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.060+0000 7fc222200640 1 -- 172.21.15.167:0/2808213224 wait complete. 2024-05-11T12:12:34.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.061+0000 7fc222200640 1 Processor -- start 2024-05-11T12:12:34.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.061+0000 7fc222200640 1 -- start start 2024-05-11T12:12:34.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.062+0000 7fc222200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc21c0b3fb0 0x7fc21c1f4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:34.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.062+0000 7fc222200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc21c0d3de0 0x7fc21c160690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:34.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.062+0000 7fc222200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc21c162a40 0x7fc21c160bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:34.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.062+0000 7fc222200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc21c0d4270 con 0x7fc21c0b3fb0 2024-05-11T12:12:34.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.062+0000 7fc222200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc21c0b3c60 con 0x7fc21c162a40 2024-05-11T12:12:34.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.062+0000 7fc222200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc21c0d43f0 con 0x7fc21c0d3de0 2024-05-11T12:12:34.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.063+0000 7fc220e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc21c0b3fb0 0x7fc21c1f4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:34.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.063+0000 7fc220e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc21c0b3fb0 0x7fc21c1f4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:33244/0 (socket says 172.21.15.167:33244) 2024-05-11T12:12:34.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.063+0000 7fc220e00640 1 -- 172.21.15.167:0/3963415474 learned_addr learned my addr 172.21.15.167:0/3963415474 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:34.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.063+0000 7fc220e00640 1 -- 172.21.15.167:0/3963415474 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc21c0d3de0 msgr2=0x7fc21c160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:34.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.063+0000 7fc220e00640 1 --2- 172.21.15.167:0/3963415474 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc21c0d3de0 0x7fc21c160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:34.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.063+0000 7fc220e00640 1 -- 172.21.15.167:0/3963415474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc21c162a40 msgr2=0x7fc21c160bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:34.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.063+0000 7fc220e00640 1 --2- 172.21.15.167:0/3963415474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc21c162a40 0x7fc21c160bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:34.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.063+0000 7fc220e00640 1 -- 172.21.15.167:0/3963415474 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc21c0ca280 con 0x7fc21c0b3fb0 2024-05-11T12:12:34.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.063+0000 7fc220e00640 1 --2- 172.21.15.167:0/3963415474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc21c0b3fb0 0x7fc21c1f4dc0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc21002a2b0 tx=0x7fc2100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:34.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.064+0000 7fc218c00640 1 -- 172.21.15.167:0/3963415474 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc210037040 con 0x7fc21c0b3fb0 2024-05-11T12:12:34.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.064+0000 7fc218c00640 1 -- 172.21.15.167:0/3963415474 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc210042070 con 0x7fc21c0b3fb0 2024-05-11T12:12:34.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.064+0000 7fc218c00640 1 -- 172.21.15.167:0/3963415474 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2100308a0 con 0x7fc21c0b3fb0 2024-05-11T12:12:34.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.064+0000 7fc222200640 1 -- 172.21.15.167:0/3963415474 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc21c0b4a00 con 0x7fc21c0b3fb0 2024-05-11T12:12:34.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.064+0000 7fc222200640 1 -- 172.21.15.167:0/3963415474 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc21c0ca4f0 con 0x7fc21c0b3fb0 2024-05-11T12:12:34.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.065+0000 7fc222200640 1 -- 172.21.15.167:0/3963415474 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc21c0cfb30 con 0x7fc21c0b3fb0 2024-05-11T12:12:34.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.066+0000 7fc218c00640 1 -- 172.21.15.167:0/3963415474 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc210030a40 con 0x7fc21c0b3fb0 2024-05-11T12:12:34.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.067+0000 7fc218c00640 1 --2- 172.21.15.167:0/3963415474 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc2000762d0 0x7fc200078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:34.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.067+0000 7fc218c00640 1 -- 172.21.15.167:0/3963415474 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5619+0+0 (secure 0 0 0) 0x7fc2100b7f20 con 0x7fc21c0b3fb0 2024-05-11T12:12:34.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.067+0000 7fc21be00640 1 --2- 172.21.15.167:0/3963415474 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc2000762d0 0x7fc200078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:34.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.068+0000 7fc21be00640 1 --2- 172.21.15.167:0/3963415474 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc2000762d0 0x7fc200078790 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc20c006a30 tx=0x7fc20c0069c0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:34.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.072+0000 7fc218c00640 1 -- 172.21.15.167:0/3963415474 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc21c0cfb30 con 0x7fc21c0b3fb0 2024-05-11T12:12:34.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.276+0000 7fc222200640 1 -- 172.21.15.167:0/3963415474 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fc21c0694b0 con 0x7fc21c0b3fb0 2024-05-11T12:12:34.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.837+0000 7fc218c00640 1 -- 172.21.15.167:0/3963415474 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v51) ==== 170+0+0 (secure 0 0 0) 0x7fc210081830 con 0x7fc21c0b3fb0 2024-05-11T12:12:34.837 INFO:teuthology.orchestra.run.smithi167.stderr:pool 'cephfs_data' created 2024-05-11T12:12:34.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.842+0000 7fc222200640 1 -- 172.21.15.167:0/3963415474 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc2000762d0 msgr2=0x7fc200078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:34.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.842+0000 7fc222200640 1 --2- 172.21.15.167:0/3963415474 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc2000762d0 0x7fc200078790 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc20c006a30 tx=0x7fc20c0069c0 comp rx=0 tx=0).stop 2024-05-11T12:12:34.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.842+0000 7fc222200640 1 -- 172.21.15.167:0/3963415474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc21c0b3fb0 msgr2=0x7fc21c1f4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:34.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.842+0000 7fc222200640 1 --2- 172.21.15.167:0/3963415474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc21c0b3fb0 0x7fc21c1f4dc0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc21002a2b0 tx=0x7fc2100046c0 comp rx=0 tx=0).stop 2024-05-11T12:12:34.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.842+0000 7fc222200640 1 -- 172.21.15.167:0/3963415474 shutdown_connections 2024-05-11T12:12:34.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.842+0000 7fc222200640 1 --2- 172.21.15.167:0/3963415474 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc2000762d0 0x7fc200078790 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:34.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.842+0000 7fc222200640 1 --2- 172.21.15.167:0/3963415474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc21c162a40 0x7fc21c160bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:34.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.842+0000 7fc222200640 1 --2- 172.21.15.167:0/3963415474 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc21c0d3de0 0x7fc21c160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:34.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.842+0000 7fc222200640 1 --2- 172.21.15.167:0/3963415474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc21c0b3fb0 0x7fc21c1f4dc0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:34.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.842+0000 7fc222200640 1 -- 172.21.15.167:0/3963415474 >> 172.21.15.167:0/3963415474 conn(0x7fc21c0d1c00 msgr2=0x7fc21c0cd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:34.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.843+0000 7fc222200640 1 -- 172.21.15.167:0/3963415474 shutdown_connections 2024-05-11T12:12:34.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:34.843+0000 7fc222200640 1 -- 172.21.15.167:0/3963415474 wait complete. 2024-05-11T12:12:34.880 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-11T12:12:35.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.048+0000 7fbaf1c00640 1 Processor -- start 2024-05-11T12:12:35.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.048+0000 7fbaf1c00640 1 -- start start 2024-05-11T12:12:35.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.049+0000 7fbaf1c00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbaec15ac80 0x7fbaec074260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:35.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.049+0000 7fbaf1c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbaec161380 0x7fbaec072950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:35.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.049+0000 7fbaf1c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbaec0b65e0 con 0x7fbaec0b3d50 2024-05-11T12:12:35.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.049+0000 7fbaf1c00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fbaec06e3c0 con 0x7fbaec15ac80 2024-05-11T12:12:35.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.049+0000 7fbaf1c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbaec15b060 con 0x7fbaec161380 2024-05-11T12:12:35.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.050+0000 7fbaeaa00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbaec15ac80 0x7fbaec074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:35.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.050+0000 7fbaeaa00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbaec15ac80 0x7fbaec074260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:33250/0 (socket says 172.21.15.167:33250) 2024-05-11T12:12:35.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.050+0000 7fbaeaa00640 1 -- 172.21.15.167:0/772600835 learned_addr learned my addr 172.21.15.167:0/772600835 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:35.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.050+0000 7fbaeaa00640 1 -- 172.21.15.167:0/772600835 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbaec161380 msgr2=0x7fbaec072950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:35.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.050+0000 7fbaeaa00640 1 --2- 172.21.15.167:0/772600835 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbaec161380 0x7fbaec072950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:35.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.050+0000 7fbaeaa00640 1 -- 172.21.15.167:0/772600835 >> v1:172.21.15.204:6789/0 conn(0x7fbaec0b3d50 legacy=0x7fbaec160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:35.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.050+0000 7fbaeaa00640 1 -- 172.21.15.167:0/772600835 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaec0b68c0 con 0x7fbaec15ac80 2024-05-11T12:12:35.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.051+0000 7fbaeaa00640 1 --2- 172.21.15.167:0/772600835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbaec15ac80 0x7fbaec074260 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fbae4009e80 tx=0x7fbae402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23cdc37d24842128 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:35.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.051+0000 7fbae9600640 1 -- 172.21.15.167:0/772600835 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae4036070 con 0x7fbaec15ac80 2024-05-11T12:12:35.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.051+0000 7fbae9600640 1 -- 172.21.15.167:0/772600835 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbae4031600 con 0x7fbaec15ac80 2024-05-11T12:12:35.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.051+0000 7fbae9600640 1 -- 172.21.15.167:0/772600835 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae40319d0 con 0x7fbaec15ac80 2024-05-11T12:12:35.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.052+0000 7fbaf1c00640 1 -- 172.21.15.167:0/772600835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbaec15ac80 msgr2=0x7fbaec074260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:35.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.052+0000 7fbaf1c00640 1 --2- 172.21.15.167:0/772600835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbaec15ac80 0x7fbaec074260 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fbae4009e80 tx=0x7fbae402a6c0 comp rx=0 tx=0).stop 2024-05-11T12:12:35.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.052+0000 7fbaf1c00640 1 -- 172.21.15.167:0/772600835 shutdown_connections 2024-05-11T12:12:35.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.052+0000 7fbaf1c00640 1 --2- 172.21.15.167:0/772600835 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbaec161380 0x7fbaec072950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:35.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.052+0000 7fbaf1c00640 1 --2- 172.21.15.167:0/772600835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbaec15ac80 0x7fbaec074260 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:35.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.052+0000 7fbaf1c00640 1 -- 172.21.15.167:0/772600835 >> 172.21.15.167:0/772600835 conn(0x7fbaec06c450 msgr2=0x7fbaec06c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:35.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.052+0000 7fbaf1c00640 1 -- 172.21.15.167:0/772600835 shutdown_connections 2024-05-11T12:12:35.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.052+0000 7fbaf1c00640 1 -- 172.21.15.167:0/772600835 wait complete. 2024-05-11T12:12:35.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.054+0000 7fbaf1c00640 1 Processor -- start 2024-05-11T12:12:35.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.054+0000 7fbaf1c00640 1 -- start start 2024-05-11T12:12:35.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.055+0000 7fbaf1c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbaec0b3d50 0x7fbaec1577a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:35.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.055+0000 7fbaf1c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbaec15ac80 0x7fbaec141e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:35.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.055+0000 7fbaf1c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbaec161380 0x7fbaec144ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:35.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.055+0000 7fbaf1c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec0b44f0 con 0x7fbaec15ac80 2024-05-11T12:12:35.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.055+0000 7fbaf1c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec0b41e0 con 0x7fbaec161380 2024-05-11T12:12:35.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.055+0000 7fbaf1c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbaec072e90 con 0x7fbaec0b3d50 2024-05-11T12:12:35.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.055+0000 7fbaea000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbaec15ac80 0x7fbaec141e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:35.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.055+0000 7fbaea000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbaec15ac80 0x7fbaec141e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:33252/0 (socket says 172.21.15.167:33252) 2024-05-11T12:12:35.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.055+0000 7fbaea000640 1 -- 172.21.15.167:0/745426072 learned_addr learned my addr 172.21.15.167:0/745426072 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:35.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.055+0000 7fbaeaa00640 1 --2- 172.21.15.167:0/745426072 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbaec0b3d50 0x7fbaec1577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:35.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.055+0000 7fbaea000640 1 -- 172.21.15.167:0/745426072 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbaec0b3d50 msgr2=0x7fbaec1577a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:35.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.056+0000 7fbaea000640 1 --2- 172.21.15.167:0/745426072 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbaec0b3d50 0x7fbaec1577a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:35.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.056+0000 7fbaea000640 1 -- 172.21.15.167:0/745426072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbaec161380 msgr2=0x7fbaec144ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:35.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.056+0000 7fbaeb400640 1 --2- 172.21.15.167:0/745426072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbaec161380 0x7fbaec144ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:35.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.056+0000 7fbaea000640 1 --2- 172.21.15.167:0/745426072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbaec161380 0x7fbaec144ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:35.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.056+0000 7fbaea000640 1 -- 172.21.15.167:0/745426072 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaec158180 con 0x7fbaec15ac80 2024-05-11T12:12:35.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.056+0000 7fbaeb400640 1 --2- 172.21.15.167:0/745426072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbaec161380 0x7fbaec144ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:35.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.056+0000 7fbaea000640 1 --2- 172.21.15.167:0/745426072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbaec15ac80 0x7fbaec141e70 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fbad800c440 tx=0x7fbad800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:35.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.056+0000 7fbae2a00640 1 -- 172.21.15.167:0/745426072 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad8017070 con 0x7fbaec15ac80 2024-05-11T12:12:35.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.056+0000 7fbae2a00640 1 -- 172.21.15.167:0/745426072 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbad80035f0 con 0x7fbaec15ac80 2024-05-11T12:12:35.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.056+0000 7fbae2a00640 1 -- 172.21.15.167:0/745426072 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad80039e0 con 0x7fbaec15ac80 2024-05-11T12:12:35.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.056+0000 7fbaf1c00640 1 -- 172.21.15.167:0/745426072 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaec158460 con 0x7fbaec15ac80 2024-05-11T12:12:35.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.056+0000 7fbaf1c00640 1 -- 172.21.15.167:0/745426072 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaec158a20 con 0x7fbaec15ac80 2024-05-11T12:12:35.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.058+0000 7fbaf1c00640 1 -- 172.21.15.167:0/745426072 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaec069520 con 0x7fbaec15ac80 2024-05-11T12:12:35.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.059+0000 7fbae2a00640 1 -- 172.21.15.167:0/745426072 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbad801a070 con 0x7fbaec15ac80 2024-05-11T12:12:35.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.061+0000 7fbae2a00640 1 --2- 172.21.15.167:0/745426072 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbacc076300 0x7fbacc0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:35.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.061+0000 7fbae2a00640 1 -- 172.21.15.167:0/745426072 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6001+0+0 (secure 0 0 0) 0x7fbad8096db0 con 0x7fbaec15ac80 2024-05-11T12:12:35.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.061+0000 7fbaeaa00640 1 --2- 172.21.15.167:0/745426072 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbacc076300 0x7fbacc0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:35.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.062+0000 7fbaeaa00640 1 --2- 172.21.15.167:0/745426072 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbacc076300 0x7fbacc0787c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbae4033000 tx=0x7fbae4009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:35.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.065+0000 7fbae2a00640 1 -- 172.21.15.167:0/745426072 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fbad8021080 con 0x7fbaec15ac80 2024-05-11T12:12:35.313 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.313+0000 7fbaf1c00640 1 -- 172.21.15.167:0/745426072 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fbaec069d60 con 0x7fbaec15ac80 2024-05-11T12:12:35.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.864+0000 7fbae2a00640 1 -- 172.21.15.167:0/745426072 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v22) ==== 172+0+0 (secure 0 0 0) 0x7fbad80605c0 con 0x7fbaec15ac80 2024-05-11T12:12:35.864 INFO:teuthology.orchestra.run.smithi167.stderr:new fs with metadata pool 7 and data pool 8 2024-05-11T12:12:35.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.869+0000 7fbaf1c00640 1 -- 172.21.15.167:0/745426072 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbacc076300 msgr2=0x7fbacc0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:35.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.869+0000 7fbaf1c00640 1 --2- 172.21.15.167:0/745426072 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbacc076300 0x7fbacc0787c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbae4033000 tx=0x7fbae4009000 comp rx=0 tx=0).stop 2024-05-11T12:12:35.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.869+0000 7fbaf1c00640 1 -- 172.21.15.167:0/745426072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbaec15ac80 msgr2=0x7fbaec141e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:35.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.869+0000 7fbaf1c00640 1 --2- 172.21.15.167:0/745426072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbaec15ac80 0x7fbaec141e70 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fbad800c440 tx=0x7fbad800c900 comp rx=0 tx=0).stop 2024-05-11T12:12:35.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.869+0000 7fbaf1c00640 1 -- 172.21.15.167:0/745426072 shutdown_connections 2024-05-11T12:12:35.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.869+0000 7fbaf1c00640 1 --2- 172.21.15.167:0/745426072 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbacc076300 0x7fbacc0787c0 secure :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbae4033000 tx=0x7fbae4009000 comp rx=0 tx=0).stop 2024-05-11T12:12:35.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.869+0000 7fbaf1c00640 1 --2- 172.21.15.167:0/745426072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbaec161380 0x7fbaec144ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:35.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.869+0000 7fbaf1c00640 1 --2- 172.21.15.167:0/745426072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbaec15ac80 0x7fbaec141e70 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:35.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.869+0000 7fbaf1c00640 1 --2- 172.21.15.167:0/745426072 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbaec0b3d50 0x7fbaec1577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:35.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.869+0000 7fbaf1c00640 1 -- 172.21.15.167:0/745426072 >> 172.21.15.167:0/745426072 conn(0x7fbaec06c450 msgr2=0x7fbaec15e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:35.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.871+0000 7fbaf1c00640 1 -- 172.21.15.167:0/745426072 shutdown_connections 2024-05-11T12:12:35.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:35.872+0000 7fbaf1c00640 1 -- 172.21.15.167:0/745426072 wait complete. 2024-05-11T12:12:35.910 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:12:36.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.075+0000 7f2a46000640 1 Processor -- start 2024-05-11T12:12:36.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.075+0000 7f2a46000640 1 -- start start 2024-05-11T12:12:36.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.076+0000 7f2a46000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2a4013eeb0 0x7f2a4006f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:36.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.076+0000 7f2a46000640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2a4013f290 0x7f2a4006f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:36.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.077+0000 7f2a46000640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a4015fb90 con 0x7f2a400b50a0 2024-05-11T12:12:36.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.077+0000 7f2a46000640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f2a4015d5b0 con 0x7f2a4013f290 2024-05-11T12:12:36.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.077+0000 7f2a46000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2a4015fe70 con 0x7f2a4013eeb0 2024-05-11T12:12:36.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.077+0000 7f2a3fe00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f2a400b50a0 0x7f2a400735c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49306/0 (socket says 172.21.15.167:49306) 2024-05-11T12:12:36.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.077+0000 7f2a3fe00640 1 -- 172.21.15.167:0/520749608 learned_addr learned my addr 172.21.15.167:0/520749608 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:36.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.077+0000 7f2a3ea00640 1 --2- 172.21.15.167:0/520749608 >> v2:172.21.15.167:3300/0 conn(0x7f2a4013f290 0x7f2a4006f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:36.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.077+0000 7f2a3f400640 1 --2- 172.21.15.167:0/520749608 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2a4013eeb0 0x7f2a4006f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:36.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.078+0000 7f2a3ea00640 1 -- 172.21.15.167:0/520749608 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2a4013eeb0 msgr2=0x7f2a4006f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:36.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.078+0000 7f2a3ea00640 1 --2- 172.21.15.167:0/520749608 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2a4013eeb0 0x7f2a4006f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:36.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.078+0000 7f2a3ea00640 1 -- 172.21.15.167:0/520749608 >> v1:172.21.15.167:6789/0 conn(0x7f2a400b50a0 legacy=0x7f2a400735c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:36.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.078+0000 7f2a3ea00640 1 -- 172.21.15.167:0/520749608 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a40160150 con 0x7f2a4013f290 2024-05-11T12:12:36.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.078+0000 7f2a3e000640 1 -- 172.21.15.167:0/520749608 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3819576610 0 0) 0x7f2a4015fb90 con 0x7f2a400b50a0 2024-05-11T12:12:36.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.079+0000 7f2a3ea00640 1 --2- 172.21.15.167:0/520749608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a4013f290 0x7f2a4006f9c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2a28009d90 tx=0x7f2a2802c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef9ca2ea6ab948c server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:36.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.079+0000 7f2a3e000640 1 -- 172.21.15.167:0/520749608 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a28007740 con 0x7f2a4013f290 2024-05-11T12:12:36.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.079+0000 7f2a3e000640 1 -- 172.21.15.167:0/520749608 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a280078e0 con 0x7f2a4013f290 2024-05-11T12:12:36.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.079+0000 7f2a3e000640 1 -- 172.21.15.167:0/520749608 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a28007cb0 con 0x7f2a4013f290 2024-05-11T12:12:36.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.079+0000 7f2a46000640 1 -- 172.21.15.167:0/520749608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a4013f290 msgr2=0x7f2a4006f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:36.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.079+0000 7f2a46000640 1 --2- 172.21.15.167:0/520749608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a4013f290 0x7f2a4006f9c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2a28009d90 tx=0x7f2a2802c7e0 comp rx=0 tx=0).stop 2024-05-11T12:12:36.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.080+0000 7f2a46000640 1 -- 172.21.15.167:0/520749608 shutdown_connections 2024-05-11T12:12:36.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.080+0000 7f2a46000640 1 --2- 172.21.15.167:0/520749608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a4013f290 0x7f2a4006f9c0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:36.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.080+0000 7f2a46000640 1 --2- 172.21.15.167:0/520749608 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2a4013eeb0 0x7f2a4006f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:36.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.080+0000 7f2a46000640 1 -- 172.21.15.167:0/520749608 >> 172.21.15.167:0/520749608 conn(0x7f2a400cd8f0 msgr2=0x7f2a400cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:36.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.080+0000 7f2a46000640 1 -- 172.21.15.167:0/520749608 shutdown_connections 2024-05-11T12:12:36.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.080+0000 7f2a46000640 1 -- 172.21.15.167:0/520749608 wait complete. 2024-05-11T12:12:36.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.081+0000 7f2a46000640 1 Processor -- start 2024-05-11T12:12:36.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.081+0000 7f2a46000640 1 -- start start 2024-05-11T12:12:36.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.082+0000 7f2a46000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a400b50a0 0x7f2a401553e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:36.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.082+0000 7f2a46000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2a4013eeb0 0x7f2a40141950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:36.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.082+0000 7f2a46000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a40139e60 0x7f2a40071f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:36.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.082+0000 7f2a46000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2a40073cd0 con 0x7f2a400b50a0 2024-05-11T12:12:36.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.082+0000 7f2a46000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2a40160710 con 0x7f2a40139e60 2024-05-11T12:12:36.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.082+0000 7f2a46000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2a40073e50 con 0x7f2a4013eeb0 2024-05-11T12:12:36.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.082+0000 7f2a3f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a400b50a0 0x7f2a401553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:36.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.083+0000 7f2a3f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a400b50a0 0x7f2a401553e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:33262/0 (socket says 172.21.15.167:33262) 2024-05-11T12:12:36.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.083+0000 7f2a3f400640 1 -- 172.21.15.167:0/3821928263 learned_addr learned my addr 172.21.15.167:0/3821928263 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:36.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.083+0000 7f2a3fe00640 1 --2- 172.21.15.167:0/3821928263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a40139e60 0x7f2a40071f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:36.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.083+0000 7f2a3ea00640 1 --2- 172.21.15.167:0/3821928263 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2a4013eeb0 0x7f2a40141950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:36.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.083+0000 7f2a3f400640 1 -- 172.21.15.167:0/3821928263 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2a4013eeb0 msgr2=0x7f2a40141950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:36.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.083+0000 7f2a3f400640 1 --2- 172.21.15.167:0/3821928263 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2a4013eeb0 0x7f2a40141950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:36.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.083+0000 7f2a3f400640 1 -- 172.21.15.167:0/3821928263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a40139e60 msgr2=0x7f2a40071f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:36.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.083+0000 7f2a3f400640 1 --2- 172.21.15.167:0/3821928263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a40139e60 0x7f2a40071f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:36.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.083+0000 7f2a3f400640 1 -- 172.21.15.167:0/3821928263 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a40140b90 con 0x7f2a400b50a0 2024-05-11T12:12:36.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.083+0000 7f2a3f400640 1 --2- 172.21.15.167:0/3821928263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a400b50a0 0x7f2a401553e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2a3000d5f0 tx=0x7f2a3000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:36.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.084+0000 7f2a37400640 1 -- 172.21.15.167:0/3821928263 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a300125f0 con 0x7f2a400b50a0 2024-05-11T12:12:36.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.084+0000 7f2a37400640 1 -- 172.21.15.167:0/3821928263 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a30019070 con 0x7f2a400b50a0 2024-05-11T12:12:36.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.084+0000 7f2a37400640 1 -- 172.21.15.167:0/3821928263 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a300129d0 con 0x7f2a400b50a0 2024-05-11T12:12:36.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.084+0000 7f2a46000640 1 -- 172.21.15.167:0/3821928263 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a40140e70 con 0x7f2a400b50a0 2024-05-11T12:12:36.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.084+0000 7f2a46000640 1 -- 172.21.15.167:0/3821928263 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a40155920 con 0x7f2a400b50a0 2024-05-11T12:12:36.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.085+0000 7f2a46000640 1 -- 172.21.15.167:0/3821928263 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a40069590 con 0x7f2a400b50a0 2024-05-11T12:12:36.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.086+0000 7f2a37400640 1 -- 172.21.15.167:0/3821928263 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2a30012b70 con 0x7f2a400b50a0 2024-05-11T12:12:36.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.087+0000 7f2a37400640 1 --2- 172.21.15.167:0/3821928263 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2a1c0762d0 0x7f2a1c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:36.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.087+0000 7f2a37400640 1 -- 172.21.15.167:0/3821928263 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f2a30097c60 con 0x7f2a400b50a0 2024-05-11T12:12:36.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.087+0000 7f2a3ea00640 1 --2- 172.21.15.167:0/3821928263 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2a1c0762d0 0x7f2a1c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:36.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.088+0000 7f2a3ea00640 1 --2- 172.21.15.167:0/3821928263 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2a1c0762d0 0x7f2a1c078790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2a28036000 tx=0x7f2a28031000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:36.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.091+0000 7f2a37400640 1 -- 172.21.15.167:0/3821928263 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2a30061390 con 0x7f2a400b50a0 2024-05-11T12:12:36.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.287+0000 7f2a46000640 1 -- 172.21.15.167:0/3821928263 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a40141200 con 0x7f2a400b50a0 2024-05-11T12:12:36.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.290+0000 7f2a37400640 1 -- 172.21.15.167:0/3821928263 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) ==== 74+0+17003 (secure 0 0 0) 0x7f2a30060c90 con 0x7f2a400b50a0 2024-05-11T12:12:36.289 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:12:36.289 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":52,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:12:35.314743+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:12:33.261425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-11T12:12:34.290603+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:12:36.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.292+0000 7f2a46000640 1 -- 172.21.15.167:0/3821928263 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2a1c0762d0 msgr2=0x7f2a1c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:36.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.292+0000 7f2a46000640 1 --2- 172.21.15.167:0/3821928263 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2a1c0762d0 0x7f2a1c078790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2a28036000 tx=0x7f2a28031000 comp rx=0 tx=0).stop 2024-05-11T12:12:36.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.292+0000 7f2a46000640 1 -- 172.21.15.167:0/3821928263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a400b50a0 msgr2=0x7f2a401553e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:36.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.292+0000 7f2a46000640 1 --2- 172.21.15.167:0/3821928263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a400b50a0 0x7f2a401553e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2a3000d5f0 tx=0x7f2a3000dab0 comp rx=0 tx=0).stop 2024-05-11T12:12:36.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.292+0000 7f2a46000640 1 -- 172.21.15.167:0/3821928263 shutdown_connections 2024-05-11T12:12:36.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.292+0000 7f2a46000640 1 --2- 172.21.15.167:0/3821928263 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2a1c0762d0 0x7f2a1c078790 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:36.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.292+0000 7f2a46000640 1 --2- 172.21.15.167:0/3821928263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a40139e60 0x7f2a40071f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:36.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.292+0000 7f2a46000640 1 --2- 172.21.15.167:0/3821928263 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2a4013eeb0 0x7f2a40141950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:36.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.292+0000 7f2a46000640 1 --2- 172.21.15.167:0/3821928263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a400b50a0 0x7f2a401553e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:36.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.292+0000 7f2a46000640 1 -- 172.21.15.167:0/3821928263 >> 172.21.15.167:0/3821928263 conn(0x7f2a400cd8f0 msgr2=0x7f2a4015b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:36.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.293+0000 7f2a46000640 1 -- 172.21.15.167:0/3821928263 shutdown_connections 2024-05-11T12:12:36.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.293+0000 7f2a46000640 1 -- 172.21.15.167:0/3821928263 wait complete. 2024-05-11T12:12:36.309 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:12:36.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.503+0000 7f00d8200640 1 Processor -- start 2024-05-11T12:12:36.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.503+0000 7f00d8200640 1 -- start start 2024-05-11T12:12:36.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.505+0000 7f00d8200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f00d0157750 0x7f00d015d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:36.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.505+0000 7f00d8200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f00d006d460 0x7f00d006d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:36.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.505+0000 7f00d8200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00d015dc80 con 0x7f00d00b3ea0 2024-05-11T12:12:36.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.505+0000 7f00d8200640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f00d0161000 con 0x7f00d0157750 2024-05-11T12:12:36.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.505+0000 7f00d8200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f00d006ddd0 con 0x7f00d006d460 2024-05-11T12:12:36.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.505+0000 7f00d5800640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f00d0157750 0x7f00d015d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:36.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.505+0000 7f00d5800640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f00d0157750 0x7f00d015d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:33272/0 (socket says 172.21.15.167:33272) 2024-05-11T12:12:36.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.505+0000 7f00d5800640 1 -- 172.21.15.167:0/657377096 learned_addr learned my addr 172.21.15.167:0/657377096 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:36.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.505+0000 7f00d6200640 1 --2- 172.21.15.167:0/657377096 >> v2:172.21.15.204:3300/0 conn(0x7f00d006d460 0x7f00d006d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:36.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.506+0000 7f00d5800640 1 -- 172.21.15.167:0/657377096 >> v2:172.21.15.204:3300/0 conn(0x7f00d006d460 msgr2=0x7f00d006d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:36.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.506+0000 7f00d5800640 1 --2- 172.21.15.167:0/657377096 >> v2:172.21.15.204:3300/0 conn(0x7f00d006d460 0x7f00d006d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:36.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.506+0000 7f00d5800640 1 -- 172.21.15.167:0/657377096 >> v1:172.21.15.204:6789/0 conn(0x7f00d00b3ea0 legacy=0x7f00d0161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:36.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.506+0000 7f00d5800640 1 -- 172.21.15.167:0/657377096 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00d015df30 con 0x7f00d0157750 2024-05-11T12:12:36.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.507+0000 7f00d5800640 1 --2- 172.21.15.167:0/657377096 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00d0157750 0x7f00d015d740 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f00d00b0aa0 tx=0x7f00c402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4ea1c5b7831bb4d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:36.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.507+0000 7f00cfe00640 1 -- 172.21.15.167:0/657377096 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c4037070 con 0x7f00d0157750 2024-05-11T12:12:36.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.507+0000 7f00cfe00640 1 -- 172.21.15.167:0/657377096 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00c4032440 con 0x7f00d0157750 2024-05-11T12:12:36.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.507+0000 7f00cfe00640 1 -- 172.21.15.167:0/657377096 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c4032810 con 0x7f00d0157750 2024-05-11T12:12:36.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.508+0000 7f00d8200640 1 -- 172.21.15.167:0/657377096 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00d0157750 msgr2=0x7f00d015d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:36.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.508+0000 7f00d8200640 1 --2- 172.21.15.167:0/657377096 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00d0157750 0x7f00d015d740 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f00d00b0aa0 tx=0x7f00c402a1e0 comp rx=0 tx=0).stop 2024-05-11T12:12:36.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.508+0000 7f00d8200640 1 -- 172.21.15.167:0/657377096 shutdown_connections 2024-05-11T12:12:36.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.508+0000 7f00d8200640 1 --2- 172.21.15.167:0/657377096 >> v2:172.21.15.204:3300/0 conn(0x7f00d006d460 0x7f00d006d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:36.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.508+0000 7f00d8200640 1 --2- 172.21.15.167:0/657377096 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00d0157750 0x7f00d015d740 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:36.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.508+0000 7f00d8200640 1 -- 172.21.15.167:0/657377096 >> 172.21.15.167:0/657377096 conn(0x7f00d00cd8f0 msgr2=0x7f00d00cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:36.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.508+0000 7f00d8200640 1 -- 172.21.15.167:0/657377096 shutdown_connections 2024-05-11T12:12:36.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.508+0000 7f00d8200640 1 -- 172.21.15.167:0/657377096 wait complete. 2024-05-11T12:12:36.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.509+0000 7f00d8200640 1 Processor -- start 2024-05-11T12:12:36.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.510+0000 7f00d8200640 1 -- start start 2024-05-11T12:12:36.510 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.510+0000 7f00d8200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00d006d460 0x7f00d0156da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:36.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.510+0000 7f00d8200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00d00b3ea0 0x7f00d01f30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:36.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.510+0000 7f00d8200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00d0157750 0x7f00d015c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:36.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.511+0000 7f00d8200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f00d0157b30 con 0x7f00d00b3ea0 2024-05-11T12:12:36.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.511+0000 7f00d8200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f00d015e4f0 con 0x7f00d0157750 2024-05-11T12:12:36.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.511+0000 7f00d8200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f00d0161ba0 con 0x7f00d006d460 2024-05-11T12:12:36.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.511+0000 7f00d4e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00d00b3ea0 0x7f00d01f30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:36.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.511+0000 7f00d4e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00d00b3ea0 0x7f00d01f30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:33280/0 (socket says 172.21.15.167:33280) 2024-05-11T12:12:36.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.511+0000 7f00d4e00640 1 -- 172.21.15.167:0/1296749630 learned_addr learned my addr 172.21.15.167:0/1296749630 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:36.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.511+0000 7f00d5800640 1 --2- 172.21.15.167:0/1296749630 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00d006d460 0x7f00d0156da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:36.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.511+0000 7f00d6200640 1 --2- 172.21.15.167:0/1296749630 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00d0157750 0x7f00d015c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:36.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.511+0000 7f00d4e00640 1 -- 172.21.15.167:0/1296749630 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00d006d460 msgr2=0x7f00d0156da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:36.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.511+0000 7f00d4e00640 1 --2- 172.21.15.167:0/1296749630 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00d006d460 0x7f00d0156da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:36.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.511+0000 7f00d4e00640 1 -- 172.21.15.167:0/1296749630 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00d0157750 msgr2=0x7f00d015c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:36.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.511+0000 7f00d4e00640 1 --2- 172.21.15.167:0/1296749630 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00d0157750 0x7f00d015c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:36.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.511+0000 7f00d4e00640 1 -- 172.21.15.167:0/1296749630 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00d0158370 con 0x7f00d00b3ea0 2024-05-11T12:12:36.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.512+0000 7f00d4e00640 1 --2- 172.21.15.167:0/1296749630 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00d00b3ea0 0x7f00d01f30b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f00bc00c430 tx=0x7f00bc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:36.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.512+0000 7f00cd600640 1 -- 172.21.15.167:0/1296749630 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00bc011790 con 0x7f00d00b3ea0 2024-05-11T12:12:36.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.512+0000 7f00cd600640 1 -- 172.21.15.167:0/1296749630 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00bc017070 con 0x7f00d00b3ea0 2024-05-11T12:12:36.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.512+0000 7f00d8200640 1 -- 172.21.15.167:0/1296749630 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00d0158650 con 0x7f00d00b3ea0 2024-05-11T12:12:36.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.512+0000 7f00cd600640 1 -- 172.21.15.167:0/1296749630 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00bc011bb0 con 0x7f00d00b3ea0 2024-05-11T12:12:36.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.512+0000 7f00d8200640 1 -- 172.21.15.167:0/1296749630 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00d0158c10 con 0x7f00d00b3ea0 2024-05-11T12:12:36.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.513+0000 7f00d8200640 1 -- 172.21.15.167:0/1296749630 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00d0069590 con 0x7f00d00b3ea0 2024-05-11T12:12:36.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.514+0000 7f00cd600640 1 -- 172.21.15.167:0/1296749630 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f00bc01c460 con 0x7f00d00b3ea0 2024-05-11T12:12:36.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.515+0000 7f00cd600640 1 --2- 172.21.15.167:0/1296749630 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f00b00762d0 0x7f00b0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:36.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.516+0000 7f00cd600640 1 -- 172.21.15.167:0/1296749630 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f00bc0976e0 con 0x7f00d00b3ea0 2024-05-11T12:12:36.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.516+0000 7f00d5800640 1 --2- 172.21.15.167:0/1296749630 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f00b00762d0 0x7f00b0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:36.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.516+0000 7f00d5800640 1 --2- 172.21.15.167:0/1296749630 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f00b00762d0 0x7f00b0078790 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f00c4034000 tx=0x7f00c402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:36.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.520+0000 7f00cd600640 1 -- 172.21.15.167:0/1296749630 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f00bc060e10 con 0x7f00d00b3ea0 2024-05-11T12:12:36.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.718+0000 7f00d8200640 1 -- 172.21.15.167:0/1296749630 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f00d00b1ba0 con 0x7f00d00b3ea0 2024-05-11T12:12:36.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.720+0000 7f00cd600640 1 -- 172.21.15.167:0/1296749630 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) ==== 74+0+17003 (secure 0 0 0) 0x7f00bc060710 con 0x7f00d00b3ea0 2024-05-11T12:12:36.720 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:12:36.721 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":52,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:12:35.314743+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:12:33.261425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-11T12:12:34.290603+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:12:36.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.723+0000 7f00d8200640 1 -- 172.21.15.167:0/1296749630 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f00b00762d0 msgr2=0x7f00b0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:36.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.723+0000 7f00d8200640 1 --2- 172.21.15.167:0/1296749630 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f00b00762d0 0x7f00b0078790 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f00c4034000 tx=0x7f00c402e000 comp rx=0 tx=0).stop 2024-05-11T12:12:36.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.723+0000 7f00d8200640 1 -- 172.21.15.167:0/1296749630 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00d00b3ea0 msgr2=0x7f00d01f30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:36.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.723+0000 7f00d8200640 1 --2- 172.21.15.167:0/1296749630 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00d00b3ea0 0x7f00d01f30b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f00bc00c430 tx=0x7f00bc00c8f0 comp rx=0 tx=0).stop 2024-05-11T12:12:36.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.723+0000 7f00d8200640 1 -- 172.21.15.167:0/1296749630 shutdown_connections 2024-05-11T12:12:36.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.723+0000 7f00d8200640 1 --2- 172.21.15.167:0/1296749630 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f00b00762d0 0x7f00b0078790 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:36.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.723+0000 7f00d8200640 1 --2- 172.21.15.167:0/1296749630 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00d0157750 0x7f00d015c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:36.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.723+0000 7f00d8200640 1 --2- 172.21.15.167:0/1296749630 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00d00b3ea0 0x7f00d01f30b0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:36.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.723+0000 7f00d8200640 1 --2- 172.21.15.167:0/1296749630 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00d006d460 0x7f00d0156da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:36.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.723+0000 7f00d8200640 1 -- 172.21.15.167:0/1296749630 >> 172.21.15.167:0/1296749630 conn(0x7f00d00cd8f0 msgr2=0x7f00d00681c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:36.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.724+0000 7f00d8200640 1 -- 172.21.15.167:0/1296749630 shutdown_connections 2024-05-11T12:12:36.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.724+0000 7f00d8200640 1 -- 172.21.15.167:0/1296749630 wait complete. 2024-05-11T12:12:36.741 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-11T12:12:36.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.933+0000 7f336c200640 1 Processor -- start 2024-05-11T12:12:36.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.933+0000 7f336c200640 1 -- start start 2024-05-11T12:12:36.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.934+0000 7f336c200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f336415f910 0x7f336406c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:36.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.934+0000 7f336c200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f336406cc30 0x7f336415fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:36.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.935+0000 7f336c200640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f336406dcc0 con 0x7f3364160230 2024-05-11T12:12:36.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.935+0000 7f336c200640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f3364156820 con 0x7f336406cc30 2024-05-11T12:12:36.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.935+0000 7f336c200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f336406d140 con 0x7f336415f910 2024-05-11T12:12:36.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.935+0000 7f3368e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f336406cc30 0x7f336415fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:36.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.935+0000 7f336a200640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f3364160230 0x7f33640d3090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49312/0 (socket says 172.21.15.167:49312) 2024-05-11T12:12:36.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.935+0000 7f336a200640 1 -- 172.21.15.167:0/1504552198 learned_addr learned my addr 172.21.15.167:0/1504552198 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:36.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.935+0000 7f3369800640 1 --2- 172.21.15.167:0/1504552198 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f336415f910 0x7f336406c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:36.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.935+0000 7f3368e00640 1 -- 172.21.15.167:0/1504552198 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f336415f910 msgr2=0x7f336406c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:36.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.935+0000 7f3368e00640 1 --2- 172.21.15.167:0/1504552198 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f336415f910 0x7f336406c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:36.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.935+0000 7f3368e00640 1 -- 172.21.15.167:0/1504552198 >> v1:172.21.15.167:6789/0 conn(0x7f3364160230 legacy=0x7f33640d3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:36.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.935+0000 7f3368e00640 1 -- 172.21.15.167:0/1504552198 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f336406d420 con 0x7f336406cc30 2024-05-11T12:12:36.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.936+0000 7f3368e00640 1 --2- 172.21.15.167:0/1504552198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f336406cc30 0x7f336415fcf0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f33500097a0 tx=0x7f335002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d15d3c0f3b64ec server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:36.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.937+0000 7f3363e00640 1 -- 172.21.15.167:0/1504552198 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3350037070 con 0x7f336406cc30 2024-05-11T12:12:36.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.937+0000 7f3363e00640 1 -- 172.21.15.167:0/1504552198 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3350032440 con 0x7f336406cc30 2024-05-11T12:12:36.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.937+0000 7f3363e00640 1 -- 172.21.15.167:0/1504552198 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3350032810 con 0x7f336406cc30 2024-05-11T12:12:36.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.937+0000 7f336c200640 1 -- 172.21.15.167:0/1504552198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f336406cc30 msgr2=0x7f336415fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:36.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.937+0000 7f336c200640 1 --2- 172.21.15.167:0/1504552198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f336406cc30 0x7f336415fcf0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f33500097a0 tx=0x7f335002a180 comp rx=0 tx=0).stop 2024-05-11T12:12:36.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.937+0000 7f336c200640 1 -- 172.21.15.167:0/1504552198 shutdown_connections 2024-05-11T12:12:36.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.937+0000 7f336c200640 1 --2- 172.21.15.167:0/1504552198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f336406cc30 0x7f336415fcf0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:36.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.937+0000 7f336c200640 1 --2- 172.21.15.167:0/1504552198 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f336415f910 0x7f336406c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:36.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.937+0000 7f336c200640 1 -- 172.21.15.167:0/1504552198 >> 172.21.15.167:0/1504552198 conn(0x7f33640d51e0 msgr2=0x7f33640d55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:36.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.937+0000 7f336c200640 1 -- 172.21.15.167:0/1504552198 shutdown_connections 2024-05-11T12:12:36.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.937+0000 7f336c200640 1 -- 172.21.15.167:0/1504552198 wait complete. 2024-05-11T12:12:36.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.939+0000 7f336c200640 1 Processor -- start 2024-05-11T12:12:36.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.939+0000 7f336c200640 1 -- start start 2024-05-11T12:12:36.940 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.940+0000 7f336c200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f336406cc30 0x7f33641569a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:36.940 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.940+0000 7f336c200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f336415f910 0x7f33641f78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:36.940 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.940+0000 7f336c200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3364160230 0x7f3364154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:36.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.940+0000 7f336c200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f33641606b0 con 0x7f336406cc30 2024-05-11T12:12:36.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.940+0000 7f336c200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f336406d9e0 con 0x7f3364160230 2024-05-11T12:12:36.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.940+0000 7f336c200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f33640d0060 con 0x7f336415f910 2024-05-11T12:12:36.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.940+0000 7f3369800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f336406cc30 0x7f33641569a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:36.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.940+0000 7f3369800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f336406cc30 0x7f33641569a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:33302/0 (socket says 172.21.15.167:33302) 2024-05-11T12:12:36.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.940+0000 7f3369800640 1 -- 172.21.15.167:0/3903749008 learned_addr learned my addr 172.21.15.167:0/3903749008 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:36.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.941+0000 7f3368e00640 1 --2- 172.21.15.167:0/3903749008 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f336415f910 0x7f33641f78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:36.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.941+0000 7f3369800640 1 -- 172.21.15.167:0/3903749008 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f336415f910 msgr2=0x7f33641f78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:36.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.941+0000 7f3369800640 1 --2- 172.21.15.167:0/3903749008 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f336415f910 0x7f33641f78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:36.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.941+0000 7f3369800640 1 -- 172.21.15.167:0/3903749008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3364160230 msgr2=0x7f3364154180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:36.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.941+0000 7f3369800640 1 --2- 172.21.15.167:0/3903749008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3364160230 0x7f3364154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:36.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.941+0000 7f3369800640 1 -- 172.21.15.167:0/3903749008 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3364156ee0 con 0x7f336406cc30 2024-05-11T12:12:36.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.941+0000 7f3369800640 1 --2- 172.21.15.167:0/3903749008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f336406cc30 0x7f33641569a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f335400d830 tx=0x7f335400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:36.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.941+0000 7f3361600640 1 -- 172.21.15.167:0/3903749008 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335400b6c0 con 0x7f336406cc30 2024-05-11T12:12:36.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.941+0000 7f3361600640 1 -- 172.21.15.167:0/3903749008 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3354019070 con 0x7f336406cc30 2024-05-11T12:12:36.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.941+0000 7f3361600640 1 -- 172.21.15.167:0/3903749008 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335400ba80 con 0x7f336406cc30 2024-05-11T12:12:36.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.941+0000 7f336c200640 1 -- 172.21.15.167:0/3903749008 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33641f7de0 con 0x7f336406cc30 2024-05-11T12:12:36.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.941+0000 7f336c200640 1 -- 172.21.15.167:0/3903749008 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3364157460 con 0x7f336406cc30 2024-05-11T12:12:36.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.942+0000 7f336c200640 1 -- 172.21.15.167:0/3903749008 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3364069520 con 0x7f336406cc30 2024-05-11T12:12:36.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.944+0000 7f3361600640 1 -- 172.21.15.167:0/3903749008 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f335401e460 con 0x7f336406cc30 2024-05-11T12:12:36.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.944+0000 7f3361600640 1 --2- 172.21.15.167:0/3903749008 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3344076300 0x7f33440787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:36.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.945+0000 7f3361600640 1 -- 172.21.15.167:0/3903749008 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f3354097e10 con 0x7f336406cc30 2024-05-11T12:12:36.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.945+0000 7f3368e00640 1 --2- 172.21.15.167:0/3903749008 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3344076300 0x7f33440787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:36.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.945+0000 7f3368e00640 1 --2- 172.21.15.167:0/3903749008 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3344076300 0x7f33440787c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3350034000 tx=0x7f335002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:36.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:36.949+0000 7f3361600640 1 -- 172.21.15.167:0/3903749008 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f335409b030 con 0x7f336406cc30 2024-05-11T12:12:37.124 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T12:12:37.131 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T12:12:37.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:37.181+0000 7f336c200640 1 -- 172.21.15.167:0/3903749008 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f33640cee70 con 0x7f336406cc30 2024-05-11T12:12:37.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:37.904+0000 7f3361600640 1 -- 172.21.15.167:0/3903749008 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v25) ==== 118+0+0 (secure 0 0 0) 0x7f33540615d0 con 0x7f336406cc30 2024-05-11T12:12:37.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:37.908+0000 7f336c200640 1 -- 172.21.15.167:0/3903749008 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3344076300 msgr2=0x7f33440787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:37.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:37.908+0000 7f336c200640 1 --2- 172.21.15.167:0/3903749008 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3344076300 0x7f33440787c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3350034000 tx=0x7f335002e000 comp rx=0 tx=0).stop 2024-05-11T12:12:37.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:37.908+0000 7f336c200640 1 -- 172.21.15.167:0/3903749008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f336406cc30 msgr2=0x7f33641569a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:37.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:37.908+0000 7f336c200640 1 --2- 172.21.15.167:0/3903749008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f336406cc30 0x7f33641569a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f335400d830 tx=0x7f335400dcf0 comp rx=0 tx=0).stop 2024-05-11T12:12:37.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:37.908+0000 7f336c200640 1 -- 172.21.15.167:0/3903749008 shutdown_connections 2024-05-11T12:12:37.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:37.908+0000 7f336c200640 1 --2- 172.21.15.167:0/3903749008 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3344076300 0x7f33440787c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:37.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:37.908+0000 7f336c200640 1 --2- 172.21.15.167:0/3903749008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3364160230 0x7f3364154180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:37.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:37.908+0000 7f336c200640 1 --2- 172.21.15.167:0/3903749008 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f336415f910 0x7f33641f78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:37.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:37.908+0000 7f336c200640 1 --2- 172.21.15.167:0/3903749008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f336406cc30 0x7f33641569a0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:37.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:37.909+0000 7f336c200640 1 -- 172.21.15.167:0/3903749008 >> 172.21.15.167:0/3903749008 conn(0x7f33640d51e0 msgr2=0x7f33640b9330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:37.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:37.909+0000 7f336c200640 1 -- 172.21.15.167:0/3903749008 shutdown_connections 2024-05-11T12:12:37.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:37.909+0000 7f336c200640 1 -- 172.21.15.167:0/3903749008 wait complete. 2024-05-11T12:12:37.931 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-11T12:12:37.931 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-11T12:12:38.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.135+0000 7fed64a00640 1 Processor -- start 2024-05-11T12:12:38.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.136+0000 7fed64a00640 1 -- start start 2024-05-11T12:12:38.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.137+0000 7fed64a00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fed6015ac80 0x7fed60074260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:38.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.137+0000 7fed64a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fed600b3d50 0x7fed60160c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:38.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.137+0000 7fed64a00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed600b65e0 con 0x7fed601611b0 2024-05-11T12:12:38.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.137+0000 7fed64a00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fed6006e3c0 con 0x7fed6015ac80 2024-05-11T12:12:38.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.137+0000 7fed64a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fed60161600 con 0x7fed600b3d50 2024-05-11T12:12:38.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.137+0000 7fed5e800640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fed601611b0 0x7fed6015d680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49322/0 (socket says 172.21.15.167:49322) 2024-05-11T12:12:38.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.137+0000 7fed5de00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fed6015ac80 0x7fed60074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:38.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.137+0000 7fed5e800640 1 -- 172.21.15.167:0/3814924887 learned_addr learned my addr 172.21.15.167:0/3814924887 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:38.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.138+0000 7fed5ca00640 1 -- 172.21.15.167:0/3814924887 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 870094750 0 0) 0x7fed600b65e0 con 0x7fed601611b0 2024-05-11T12:12:38.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.138+0000 7fed5de00640 1 -- 172.21.15.167:0/3814924887 >> v2:172.21.15.204:3300/0 conn(0x7fed600b3d50 msgr2=0x7fed60160c70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T12:12:38.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.138+0000 7fed5de00640 1 --2- 172.21.15.167:0/3814924887 >> v2:172.21.15.204:3300/0 conn(0x7fed600b3d50 0x7fed60160c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:38.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.138+0000 7fed5de00640 1 -- 172.21.15.167:0/3814924887 >> v1:172.21.15.167:6789/0 conn(0x7fed601611b0 legacy=0x7fed6015d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:38.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.138+0000 7fed5de00640 1 -- 172.21.15.167:0/3814924887 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed600b68c0 con 0x7fed6015ac80 2024-05-11T12:12:38.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.139+0000 7fed5de00640 1 --2- 172.21.15.167:0/3814924887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed6015ac80 0x7fed60074260 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fed500097a0 tx=0x7fed5002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=655679810b2dd27b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:38.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.139+0000 7fed5ca00640 1 -- 172.21.15.167:0/3814924887 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed50037070 con 0x7fed6015ac80 2024-05-11T12:12:38.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.139+0000 7fed5ca00640 1 -- 172.21.15.167:0/3814924887 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed50032440 con 0x7fed6015ac80 2024-05-11T12:12:38.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.139+0000 7fed5ca00640 1 -- 172.21.15.167:0/3814924887 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed50032810 con 0x7fed6015ac80 2024-05-11T12:12:38.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.140+0000 7fed64a00640 1 -- 172.21.15.167:0/3814924887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed6015ac80 msgr2=0x7fed60074260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:38.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.140+0000 7fed64a00640 1 --2- 172.21.15.167:0/3814924887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed6015ac80 0x7fed60074260 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fed500097a0 tx=0x7fed5002a1e0 comp rx=0 tx=0).stop 2024-05-11T12:12:38.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.140+0000 7fed64a00640 1 -- 172.21.15.167:0/3814924887 shutdown_connections 2024-05-11T12:12:38.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.140+0000 7fed64a00640 1 --2- 172.21.15.167:0/3814924887 >> v2:172.21.15.204:3300/0 conn(0x7fed600b3d50 0x7fed60160c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:38.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.140+0000 7fed64a00640 1 --2- 172.21.15.167:0/3814924887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed6015ac80 0x7fed60074260 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:38.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.140+0000 7fed64a00640 1 -- 172.21.15.167:0/3814924887 >> 172.21.15.167:0/3814924887 conn(0x7fed6006c450 msgr2=0x7fed6006c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:38.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.140+0000 7fed64a00640 1 -- 172.21.15.167:0/3814924887 shutdown_connections 2024-05-11T12:12:38.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.140+0000 7fed64a00640 1 -- 172.21.15.167:0/3814924887 wait complete. 2024-05-11T12:12:38.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.142+0000 7fed64a00640 1 Processor -- start 2024-05-11T12:12:38.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.142+0000 7fed64a00640 1 -- start start 2024-05-11T12:12:38.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.143+0000 7fed64a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed600b3d50 0x7fed6015cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:38.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.143+0000 7fed64a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed6015ac80 0x7fed6013c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:38.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.143+0000 7fed64a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed601611b0 0x7fed6013fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:38.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.143+0000 7fed64a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fed600b41e0 con 0x7fed600b3d50 2024-05-11T12:12:38.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.143+0000 7fed64a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fed6015b060 con 0x7fed6015ac80 2024-05-11T12:12:38.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.143+0000 7fed64a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fed600b44f0 con 0x7fed601611b0 2024-05-11T12:12:38.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.143+0000 7fed5de00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed600b3d50 0x7fed6015cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:38.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.143+0000 7fed5de00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed600b3d50 0x7fed6015cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:33308/0 (socket says 172.21.15.167:33308) 2024-05-11T12:12:38.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.143+0000 7fed5de00640 1 -- 172.21.15.167:0/648111806 learned_addr learned my addr 172.21.15.167:0/648111806 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:38.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.143+0000 7fed5d400640 1 --2- 172.21.15.167:0/648111806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed6015ac80 0x7fed6013c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:38.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.143+0000 7fed5de00640 1 -- 172.21.15.167:0/648111806 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed601611b0 msgr2=0x7fed6013fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:38.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.143+0000 7fed5de00640 1 --2- 172.21.15.167:0/648111806 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed601611b0 0x7fed6013fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:38.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.143+0000 7fed5de00640 1 -- 172.21.15.167:0/648111806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed6015ac80 msgr2=0x7fed6013c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:38.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.143+0000 7fed5de00640 1 --2- 172.21.15.167:0/648111806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed6015ac80 0x7fed6013c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:38.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.143+0000 7fed5de00640 1 -- 172.21.15.167:0/648111806 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed6013c9d0 con 0x7fed600b3d50 2024-05-11T12:12:38.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.143+0000 7fed5d400640 1 --2- 172.21.15.167:0/648111806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed6015ac80 0x7fed6013c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:38.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.144+0000 7fed5de00640 1 --2- 172.21.15.167:0/648111806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed600b3d50 0x7fed6015cf90 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fed500097a0 tx=0x7fed500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:38.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.144+0000 7fed56000640 1 -- 172.21.15.167:0/648111806 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed50041070 con 0x7fed600b3d50 2024-05-11T12:12:38.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.144+0000 7fed56000640 1 -- 172.21.15.167:0/648111806 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed5003c870 con 0x7fed600b3d50 2024-05-11T12:12:38.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.144+0000 7fed56000640 1 -- 172.21.15.167:0/648111806 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed50037040 con 0x7fed600b3d50 2024-05-11T12:12:38.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.144+0000 7fed64a00640 1 -- 172.21.15.167:0/648111806 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed6013ccb0 con 0x7fed600b3d50 2024-05-11T12:12:38.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.144+0000 7fed64a00640 1 -- 172.21.15.167:0/648111806 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed6013d270 con 0x7fed600b3d50 2024-05-11T12:12:38.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.145+0000 7fed64a00640 1 -- 172.21.15.167:0/648111806 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed60069520 con 0x7fed600b3d50 2024-05-11T12:12:38.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.146+0000 7fed56000640 1 -- 172.21.15.167:0/648111806 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fed50044070 con 0x7fed600b3d50 2024-05-11T12:12:38.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.147+0000 7fed56000640 1 --2- 172.21.15.167:0/648111806 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fed3c0762d0 0x7fed3c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:38.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.147+0000 7fed56000640 1 -- 172.21.15.167:0/648111806 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fed500b8390 con 0x7fed600b3d50 2024-05-11T12:12:38.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.148+0000 7fed5d400640 1 --2- 172.21.15.167:0/648111806 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fed3c0762d0 0x7fed3c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:38.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.148+0000 7fed5d400640 1 --2- 172.21.15.167:0/648111806 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fed3c0762d0 0x7fed3c078790 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fed601f6250 tx=0x7fed480073c0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:38.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.152+0000 7fed56000640 1 -- 172.21.15.167:0/648111806 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fed50081ac0 con 0x7fed600b3d50 2024-05-11T12:12:38.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.381+0000 7fed64a00640 1 -- 172.21.15.167:0/648111806 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fed60070120 con 0x7fed600b3d50 2024-05-11T12:12:38.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.907+0000 7fed56000640 1 -- 172.21.15.167:0/648111806 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v26) ==== 122+0+0 (secure 0 0 0) 0x7fed500813c0 con 0x7fed600b3d50 2024-05-11T12:12:38.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.910+0000 7fed64a00640 1 -- 172.21.15.167:0/648111806 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fed3c0762d0 msgr2=0x7fed3c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:38.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.910+0000 7fed64a00640 1 --2- 172.21.15.167:0/648111806 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fed3c0762d0 0x7fed3c078790 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fed601f6250 tx=0x7fed480073c0 comp rx=0 tx=0).stop 2024-05-11T12:12:38.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.910+0000 7fed64a00640 1 -- 172.21.15.167:0/648111806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed600b3d50 msgr2=0x7fed6015cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:38.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.910+0000 7fed64a00640 1 --2- 172.21.15.167:0/648111806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed600b3d50 0x7fed6015cf90 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fed500097a0 tx=0x7fed500046c0 comp rx=0 tx=0).stop 2024-05-11T12:12:38.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.911+0000 7fed64a00640 1 -- 172.21.15.167:0/648111806 shutdown_connections 2024-05-11T12:12:38.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.911+0000 7fed64a00640 1 --2- 172.21.15.167:0/648111806 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed601611b0 0x7fed6013fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:38.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.911+0000 7fed64a00640 1 --2- 172.21.15.167:0/648111806 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fed3c0762d0 0x7fed3c078790 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:38.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.911+0000 7fed64a00640 1 --2- 172.21.15.167:0/648111806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed6015ac80 0x7fed6013c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:38.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.911+0000 7fed64a00640 1 --2- 172.21.15.167:0/648111806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed600b3d50 0x7fed6015cf90 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:38.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.911+0000 7fed64a00640 1 -- 172.21.15.167:0/648111806 >> 172.21.15.167:0/648111806 conn(0x7fed6006c450 msgr2=0x7fed6015e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:38.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.911+0000 7fed64a00640 1 -- 172.21.15.167:0/648111806 shutdown_connections 2024-05-11T12:12:38.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:38.911+0000 7fed64a00640 1 -- 172.21.15.167:0/648111806 wait complete. 2024-05-11T12:12:38.932 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-11T12:12:38.932 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:12:39.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.130+0000 7fdf62a00640 1 Processor -- start 2024-05-11T12:12:39.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.130+0000 7fdf62a00640 1 -- start start 2024-05-11T12:12:39.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.132+0000 7fdf62a00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fdf5c13f290 0x7fdf5c0735c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:39.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.132+0000 7fdf62a00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf5c15fb90 con 0x7fdf5c13eeb0 2024-05-11T12:12:39.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.132+0000 7fdf62a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf5c15fe70 con 0x7fdf5c0b50a0 2024-05-11T12:12:39.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.132+0000 7fdf62a00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fdf5c15d5b0 con 0x7fdf5c13f290 2024-05-11T12:12:39.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.132+0000 7fdf5be00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fdf5c13eeb0 0x7fdf5c06f480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:58082/0 (socket says 172.21.15.167:58082) 2024-05-11T12:12:39.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.132+0000 7fdf5be00640 1 -- 172.21.15.167:0/2860219320 learned_addr learned my addr 172.21.15.167:0/2860219320 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:39.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.132+0000 7fdf53400640 1 --2- 172.21.15.167:0/2860219320 >> v2:172.21.15.167:3300/0 conn(0x7fdf5c13f290 0x7fdf5c0735c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:39.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.133+0000 7fdf5b400640 1 -- 172.21.15.167:0/2860219320 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 119521644 0 0) 0x7fdf5c15fb90 con 0x7fdf5c13eeb0 2024-05-11T12:12:39.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.133+0000 7fdf5b400640 1 -- 172.21.15.167:0/2860219320 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf40003520 con 0x7fdf5c13eeb0 2024-05-11T12:12:39.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.133+0000 7fdf53400640 1 -- 172.21.15.167:0/2860219320 >> v1:172.21.15.204:6789/0 conn(0x7fdf5c0b50a0 legacy=0x7fdf5c073b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:39.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.133+0000 7fdf53400640 1 -- 172.21.15.167:0/2860219320 >> v1:172.21.15.167:6789/0 conn(0x7fdf5c13eeb0 legacy=0x7fdf5c06f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:39.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.133+0000 7fdf53400640 1 -- 172.21.15.167:0/2860219320 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf5c160150 con 0x7fdf5c13f290 2024-05-11T12:12:39.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.134+0000 7fdf53400640 1 --2- 172.21.15.167:0/2860219320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf5c13f290 0x7fdf5c0735c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fdf440097a0 tx=0x7fdf4402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7de92034b116d6e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:39.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.134+0000 7fdf5b400640 1 -- 172.21.15.167:0/2860219320 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf44037070 con 0x7fdf5c13f290 2024-05-11T12:12:39.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.134+0000 7fdf5b400640 1 -- 172.21.15.167:0/2860219320 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf44032440 con 0x7fdf5c13f290 2024-05-11T12:12:39.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.134+0000 7fdf5b400640 1 -- 172.21.15.167:0/2860219320 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf44032810 con 0x7fdf5c13f290 2024-05-11T12:12:39.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.135+0000 7fdf62a00640 1 -- 172.21.15.167:0/2860219320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf5c13f290 msgr2=0x7fdf5c0735c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:39.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.135+0000 7fdf62a00640 1 --2- 172.21.15.167:0/2860219320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf5c13f290 0x7fdf5c0735c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fdf440097a0 tx=0x7fdf4402a1a0 comp rx=0 tx=0).stop 2024-05-11T12:12:39.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.135+0000 7fdf62a00640 1 -- 172.21.15.167:0/2860219320 shutdown_connections 2024-05-11T12:12:39.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.135+0000 7fdf62a00640 1 --2- 172.21.15.167:0/2860219320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf5c13f290 0x7fdf5c0735c0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:39.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.135+0000 7fdf62a00640 1 -- 172.21.15.167:0/2860219320 >> 172.21.15.167:0/2860219320 conn(0x7fdf5c0cd8f0 msgr2=0x7fdf5c0cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:39.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.135+0000 7fdf62a00640 1 -- 172.21.15.167:0/2860219320 shutdown_connections 2024-05-11T12:12:39.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.135+0000 7fdf62a00640 1 -- 172.21.15.167:0/2860219320 wait complete. 2024-05-11T12:12:39.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.137+0000 7fdf62a00640 1 Processor -- start 2024-05-11T12:12:39.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.137+0000 7fdf62a00640 1 -- start start 2024-05-11T12:12:39.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.138+0000 7fdf62a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdf5c0b50a0 0x7fdf5c149080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:39.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.138+0000 7fdf62a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf5c13eeb0 0x7fdf5c13c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:39.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.138+0000 7fdf62a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf5c13f290 0x7fdf5c13b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:39.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.138+0000 7fdf62a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5c06fb90 con 0x7fdf5c13f290 2024-05-11T12:12:39.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.138+0000 7fdf62a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5c160710 con 0x7fdf5c13eeb0 2024-05-11T12:12:39.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.138+0000 7fdf62a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fdf5c06fd10 con 0x7fdf5c0b50a0 2024-05-11T12:12:39.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.138+0000 7fdf60a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf5c13f290 0x7fdf5c13b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:39.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.138+0000 7fdf60a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf5c13f290 0x7fdf5c13b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45534/0 (socket says 172.21.15.167:45534) 2024-05-11T12:12:39.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.138+0000 7fdf60a00640 1 -- 172.21.15.167:0/4103865946 learned_addr learned my addr 172.21.15.167:0/4103865946 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:39.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.139+0000 7fdf53400640 1 --2- 172.21.15.167:0/4103865946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf5c13eeb0 0x7fdf5c13c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:39.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.139+0000 7fdf5be00640 1 --2- 172.21.15.167:0/4103865946 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdf5c0b50a0 0x7fdf5c149080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:39.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.139+0000 7fdf60a00640 1 -- 172.21.15.167:0/4103865946 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdf5c0b50a0 msgr2=0x7fdf5c149080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:39.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.139+0000 7fdf60a00640 1 --2- 172.21.15.167:0/4103865946 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdf5c0b50a0 0x7fdf5c149080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:39.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.139+0000 7fdf60a00640 1 -- 172.21.15.167:0/4103865946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf5c13eeb0 msgr2=0x7fdf5c13c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:39.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.139+0000 7fdf60a00640 1 --2- 172.21.15.167:0/4103865946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf5c13eeb0 0x7fdf5c13c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:39.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.139+0000 7fdf60a00640 1 -- 172.21.15.167:0/4103865946 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf5c0b0bb0 con 0x7fdf5c13f290 2024-05-11T12:12:39.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.139+0000 7fdf53400640 1 --2- 172.21.15.167:0/4103865946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf5c13eeb0 0x7fdf5c13c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:12:39.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.139+0000 7fdf60a00640 1 --2- 172.21.15.167:0/4103865946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf5c13f290 0x7fdf5c13b8a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fdf5400c7a0 tx=0x7fdf5400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:39.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.140+0000 7fdf58c00640 1 -- 172.21.15.167:0/4103865946 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf54003b70 con 0x7fdf5c13f290 2024-05-11T12:12:39.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.140+0000 7fdf58c00640 1 -- 172.21.15.167:0/4103865946 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf54003d10 con 0x7fdf5c13f290 2024-05-11T12:12:39.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.140+0000 7fdf58c00640 1 -- 172.21.15.167:0/4103865946 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5400aa10 con 0x7fdf5c13f290 2024-05-11T12:12:39.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.140+0000 7fdf62a00640 1 -- 172.21.15.167:0/4103865946 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf5c0b0e90 con 0x7fdf5c13f290 2024-05-11T12:12:39.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.140+0000 7fdf62a00640 1 -- 172.21.15.167:0/4103865946 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf5c0b1450 con 0x7fdf5c13f290 2024-05-11T12:12:39.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.141+0000 7fdf62a00640 1 -- 172.21.15.167:0/4103865946 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf5c069590 con 0x7fdf5c13f290 2024-05-11T12:12:39.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.142+0000 7fdf58c00640 1 -- 172.21.15.167:0/4103865946 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fdf5400abb0 con 0x7fdf5c13f290 2024-05-11T12:12:39.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.143+0000 7fdf58c00640 1 --2- 172.21.15.167:0/4103865946 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fdf380762d0 0x7fdf38078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:39.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.143+0000 7fdf58c00640 1 -- 172.21.15.167:0/4103865946 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fdf540977b0 con 0x7fdf5c13f290 2024-05-11T12:12:39.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.143+0000 7fdf5be00640 1 --2- 172.21.15.167:0/4103865946 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fdf380762d0 0x7fdf38078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:39.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.144+0000 7fdf5be00640 1 --2- 172.21.15.167:0/4103865946 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fdf380762d0 0x7fdf38078790 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c007a60 tx=0x7fdf4c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:39.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.147+0000 7fdf58c00640 1 -- 172.21.15.167:0/4103865946 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fdf54060e50 con 0x7fdf5c13f290 2024-05-11T12:12:39.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.387+0000 7fdf62a00640 1 -- 172.21.15.167:0/4103865946 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf5c073880 con 0x7fdf5c13f290 2024-05-11T12:12:39.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.388+0000 7fdf58c00640 1 -- 172.21.15.167:0/4103865946 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5193 (secure 0 0 0) 0x7fdf54060750 con 0x7fdf5c13f290 2024-05-11T12:12:39.387 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:12:39.388 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4634,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2640837270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":2640837270},{"type":"v1","addr":"172.21.15.204:6835","nonce":2640837270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4807,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3516271493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3516271493},{"type":"v1","addr":"172.21.15.204:6834","nonce":3516271493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4816,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2070289869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2070289869},{"type":"v1","addr":"172.21.15.167:6837","nonce":2070289869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:12:35.314711+0000","modified":"2024-05-11T12:12:38.382446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6835/4016299199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4016299199},{"type":"v1","addr":"172.21.15.167:6835","nonce":4016299199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5310,"qdb_cluster":[5310]},"id":3}]} 2024-05-11T12:12:39.388 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 26 2024-05-11T12:12:39.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.390+0000 7fdf62a00640 1 -- 172.21.15.167:0/4103865946 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fdf380762d0 msgr2=0x7fdf38078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:39.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.390+0000 7fdf62a00640 1 --2- 172.21.15.167:0/4103865946 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fdf380762d0 0x7fdf38078790 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c007a60 tx=0x7fdf4c009040 comp rx=0 tx=0).stop 2024-05-11T12:12:39.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.390+0000 7fdf62a00640 1 -- 172.21.15.167:0/4103865946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf5c13f290 msgr2=0x7fdf5c13b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:39.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.390+0000 7fdf62a00640 1 --2- 172.21.15.167:0/4103865946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf5c13f290 0x7fdf5c13b8a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fdf5400c7a0 tx=0x7fdf5400cc60 comp rx=0 tx=0).stop 2024-05-11T12:12:39.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.391+0000 7fdf62a00640 1 -- 172.21.15.167:0/4103865946 shutdown_connections 2024-05-11T12:12:39.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.391+0000 7fdf62a00640 1 --2- 172.21.15.167:0/4103865946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf5c13f290 0x7fdf5c13b8a0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:39.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.391+0000 7fdf62a00640 1 --2- 172.21.15.167:0/4103865946 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fdf380762d0 0x7fdf38078790 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:39.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.391+0000 7fdf62a00640 1 --2- 172.21.15.167:0/4103865946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf5c13eeb0 0x7fdf5c13c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:39.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.391+0000 7fdf62a00640 1 --2- 172.21.15.167:0/4103865946 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdf5c0b50a0 0x7fdf5c149080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:39.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.391+0000 7fdf62a00640 1 -- 172.21.15.167:0/4103865946 >> 172.21.15.167:0/4103865946 conn(0x7fdf5c0cd8f0 msgr2=0x7fdf5c072800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:39.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.391+0000 7fdf62a00640 1 -- 172.21.15.167:0/4103865946 shutdown_connections 2024-05-11T12:12:39.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.391+0000 7fdf62a00640 1 -- 172.21.15.167:0/4103865946 wait complete. 2024-05-11T12:12:39.408 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:12:39.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.607+0000 7fb9a9600640 1 Processor -- start 2024-05-11T12:12:39.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.608+0000 7fb9a9600640 1 -- start start 2024-05-11T12:12:39.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.609+0000 7fb9a9600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb9a40af5d0 0x7fb9a4073ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:39.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.609+0000 7fb9a9600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb9a406d460 0x7fb9a406d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:39.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.609+0000 7fb9a9600640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9a406fe10 con 0x7fb9a40af1f0 2024-05-11T12:12:39.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.609+0000 7fb9a9600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fb9a41608d0 con 0x7fb9a40af5d0 2024-05-11T12:12:39.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.609+0000 7fb9a9600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb9a406de10 con 0x7fb9a406d460 2024-05-11T12:12:39.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.610+0000 7fb9a2a00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fb9a40af1f0 0x7fb9a40cf1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:58088/0 (socket says 172.21.15.167:58088) 2024-05-11T12:12:39.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.610+0000 7fb9a2a00640 1 -- 172.21.15.167:0/3226628189 learned_addr learned my addr 172.21.15.167:0/3226628189 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:39.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.610+0000 7fb9a3400640 1 --2- 172.21.15.167:0/3226628189 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb9a406d460 0x7fb9a406d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:39.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.610+0000 7fb9a2000640 1 --2- 172.21.15.167:0/3226628189 >> v2:172.21.15.204:3300/0 conn(0x7fb9a40af5d0 0x7fb9a4073ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:39.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.610+0000 7fb9a1600640 1 -- 172.21.15.167:0/3226628189 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2748467741 0 0) 0x7fb9a406fe10 con 0x7fb9a40af1f0 2024-05-11T12:12:39.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.610+0000 7fb9a1600640 1 -- 172.21.15.167:0/3226628189 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb988003520 con 0x7fb9a40af1f0 2024-05-11T12:12:39.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.611+0000 7fb9a1600640 1 -- 172.21.15.167:0/3226628189 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4184525498 0 0) 0x7fb988003520 con 0x7fb9a40af1f0 2024-05-11T12:12:39.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.611+0000 7fb9a1600640 1 -- 172.21.15.167:0/3226628189 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb9a406d460 msgr2=0x7fb9a406d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:39.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.611+0000 7fb9a1600640 1 --2- 172.21.15.167:0/3226628189 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb9a406d460 0x7fb9a406d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:39.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.611+0000 7fb9a1600640 1 -- 172.21.15.167:0/3226628189 >> v2:172.21.15.204:3300/0 conn(0x7fb9a40af5d0 msgr2=0x7fb9a4073ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:39.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.611+0000 7fb9a1600640 1 --2- 172.21.15.167:0/3226628189 >> v2:172.21.15.204:3300/0 conn(0x7fb9a40af5d0 0x7fb9a4073ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:39.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.611+0000 7fb9a1600640 1 -- 172.21.15.167:0/3226628189 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a415fcc0 con 0x7fb9a40af1f0 2024-05-11T12:12:39.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.611+0000 7fb9a1600640 1 -- 172.21.15.167:0/3226628189 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7fb9940016f0 con 0x7fb9a40af1f0 2024-05-11T12:12:39.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.611+0000 7fb9a1600640 1 -- 172.21.15.167:0/3226628189 >> v1:172.21.15.167:6789/0 conn(0x7fb9a40af1f0 legacy=0x7fb9a40cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:39.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.611+0000 7fb9a1600640 1 --2- 172.21.15.167:0/3226628189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb988003a00 0x7fb988003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:39.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.611+0000 7fb9a1600640 1 --2- 172.21.15.167:0/3226628189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb988004800 0x7fb988004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:39.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.611+0000 7fb9a1600640 1 --2- 172.21.15.167:0/3226628189 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb9a406d460 0x7fb9880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:39.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.611+0000 7fb9a1600640 1 -- 172.21.15.167:0/3226628189 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a41608d0 con 0x7fb988004800 2024-05-11T12:12:39.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.611+0000 7fb9a1600640 1 -- 172.21.15.167:0/3226628189 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a406de10 con 0x7fb988003a00 2024-05-11T12:12:39.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.611+0000 7fb9a1600640 1 -- 172.21.15.167:0/3226628189 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb98800ba70 con 0x7fb9a406d460 2024-05-11T12:12:39.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.611+0000 7fb9a2000640 1 --2- 172.21.15.167:0/3226628189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb988004800 0x7fb988004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:39.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.611+0000 7fb9a2a00640 1 --2- 172.21.15.167:0/3226628189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb988003a00 0x7fb988003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:39.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.612+0000 7fb9a2000640 1 -- 172.21.15.167:0/3226628189 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb9a406d460 msgr2=0x7fb9880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:39.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.612+0000 7fb9a3400640 1 --2- 172.21.15.167:0/3226628189 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb9a406d460 0x7fb9880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:39.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.612+0000 7fb9a2000640 1 --2- 172.21.15.167:0/3226628189 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb9a406d460 0x7fb9880051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:39.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.612+0000 7fb9a2000640 1 -- 172.21.15.167:0/3226628189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb988003a00 msgr2=0x7fb988003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:39.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.612+0000 7fb9a2000640 1 --2- 172.21.15.167:0/3226628189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb988003a00 0x7fb988003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:39.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.612+0000 7fb9a2000640 1 -- 172.21.15.167:0/3226628189 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb988003520 con 0x7fb988004800 2024-05-11T12:12:39.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.612+0000 7fb9a2a00640 1 --2- 172.21.15.167:0/3226628189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb988003a00 0x7fb988003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:39.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.612+0000 7fb9a3400640 1 --2- 172.21.15.167:0/3226628189 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb9a406d460 0x7fb9880051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:39.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.613+0000 7fb9a2000640 1 --2- 172.21.15.167:0/3226628189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb988004800 0x7fb988004c80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb98c009a90 tx=0x7fb988005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48ba8615788913ea server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:39.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.613+0000 7fb9a1600640 1 -- 172.21.15.167:0/3226628189 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98c036070 con 0x7fb988004800 2024-05-11T12:12:39.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.613+0000 7fb9a1600640 1 -- 172.21.15.167:0/3226628189 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb98c031400 con 0x7fb988004800 2024-05-11T12:12:39.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.613+0000 7fb9a1600640 1 -- 172.21.15.167:0/3226628189 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98c009690 con 0x7fb988004800 2024-05-11T12:12:39.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.613+0000 7fb9a9600640 1 -- 172.21.15.167:0/3226628189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb988004800 msgr2=0x7fb988004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:39.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.613+0000 7fb9a9600640 1 --2- 172.21.15.167:0/3226628189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb988004800 0x7fb988004c80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb98c009a90 tx=0x7fb988005830 comp rx=0 tx=0).stop 2024-05-11T12:12:39.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.613+0000 7fb9a3400640 1 -- 172.21.15.167:0/3226628189 reap_dead start 2024-05-11T12:12:39.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.614+0000 7fb9a9600640 1 -- 172.21.15.167:0/3226628189 shutdown_connections 2024-05-11T12:12:39.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.614+0000 7fb9a9600640 1 -- 172.21.15.167:0/3226628189 >> 172.21.15.167:0/3226628189 conn(0x7fb9a40cd8f0 msgr2=0x7fb9a40cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:39.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.614+0000 7fb9a9600640 1 -- 172.21.15.167:0/3226628189 shutdown_connections 2024-05-11T12:12:39.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.614+0000 7fb9a9600640 1 -- 172.21.15.167:0/3226628189 wait complete. 2024-05-11T12:12:39.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.615+0000 7fb9a9600640 1 Processor -- start 2024-05-11T12:12:39.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.616+0000 7fb9a9600640 1 -- start start 2024-05-11T12:12:39.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.616+0000 7fb9a9600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb988004800 0x7fb9a415bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:39.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.616+0000 7fb9a9600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9a40b4d90 0x7fb9a40b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:39.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.617+0000 7fb9a9600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb9a40b3c30 0x7fb9a40b40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:39.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.617+0000 7fb9a9600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a4160560 con 0x7fb9a40b4d90 2024-05-11T12:12:39.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.617+0000 7fb9a9600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a4160280 con 0x7fb988004800 2024-05-11T12:12:39.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.617+0000 7fb9a9600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb9a415f920 con 0x7fb9a40b3c30 2024-05-11T12:12:39.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.617+0000 7fb9a2000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9a40b4d90 0x7fb9a40b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:39.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.617+0000 7fb9a2000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9a40b4d90 0x7fb9a40b51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45558/0 (socket says 172.21.15.167:45558) 2024-05-11T12:12:39.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.617+0000 7fb9a2000640 1 -- 172.21.15.167:0/2284244672 learned_addr learned my addr 172.21.15.167:0/2284244672 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:39.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.617+0000 7fb9a2000640 1 -- 172.21.15.167:0/2284244672 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb9a40b3c30 msgr2=0x7fb9a40b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:39.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.617+0000 7fb9a2000640 1 --2- 172.21.15.167:0/2284244672 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb9a40b3c30 0x7fb9a40b40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:39.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.617+0000 7fb9a2a00640 1 --2- 172.21.15.167:0/2284244672 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb988004800 0x7fb9a415bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:39.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.617+0000 7fb9a2000640 1 -- 172.21.15.167:0/2284244672 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb988004800 msgr2=0x7fb9a415bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:39.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.617+0000 7fb9a2000640 1 --2- 172.21.15.167:0/2284244672 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb988004800 0x7fb9a415bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:39.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.617+0000 7fb9a2000640 1 -- 172.21.15.167:0/2284244672 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a41564a0 con 0x7fb9a40b4d90 2024-05-11T12:12:39.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.618+0000 7fb9a2000640 1 --2- 172.21.15.167:0/2284244672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9a40b4d90 0x7fb9a40b51b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb98c009a90 tx=0x7fb98c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:39.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.618+0000 7fb99aa00640 1 -- 172.21.15.167:0/2284244672 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98c041020 con 0x7fb9a40b4d90 2024-05-11T12:12:39.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.618+0000 7fb99aa00640 1 -- 172.21.15.167:0/2284244672 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb98c03c470 con 0x7fb9a40b4d90 2024-05-11T12:12:39.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.618+0000 7fb99aa00640 1 -- 172.21.15.167:0/2284244672 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98c036040 con 0x7fb9a40b4d90 2024-05-11T12:12:39.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.618+0000 7fb9a9600640 1 -- 172.21.15.167:0/2284244672 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9a4156780 con 0x7fb9a40b4d90 2024-05-11T12:12:39.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.618+0000 7fb9a9600640 1 -- 172.21.15.167:0/2284244672 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9a4156d40 con 0x7fb9a40b4d90 2024-05-11T12:12:39.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.619+0000 7fb9a9600640 1 -- 172.21.15.167:0/2284244672 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9a4157120 con 0x7fb9a40b4d90 2024-05-11T12:12:39.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.620+0000 7fb99aa00640 1 -- 172.21.15.167:0/2284244672 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb98c044020 con 0x7fb9a40b4d90 2024-05-11T12:12:39.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.621+0000 7fb99aa00640 1 --2- 172.21.15.167:0/2284244672 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb9800762d0 0x7fb980078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:39.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.621+0000 7fb99aa00640 1 -- 172.21.15.167:0/2284244672 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fb98c0b7fe0 con 0x7fb9a40b4d90 2024-05-11T12:12:39.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.622+0000 7fb9a2a00640 1 --2- 172.21.15.167:0/2284244672 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb9800762d0 0x7fb980078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:39.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.622+0000 7fb9a2a00640 1 --2- 172.21.15.167:0/2284244672 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb9800762d0 0x7fb980078790 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb9940078d0 tx=0x7fb994001b20 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:39.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.626+0000 7fb99aa00640 1 -- 172.21.15.167:0/2284244672 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb9a4157120 con 0x7fb9a40b4d90 2024-05-11T12:12:39.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.835+0000 7fb9a9600640 1 -- 172.21.15.167:0/2284244672 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb9a406f220 con 0x7fb9a40b4d90 2024-05-11T12:12:39.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.837+0000 7fb99aa00640 1 -- 172.21.15.167:0/2284244672 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) ==== 74+0+16990 (secure 0 0 0) 0x7fb98c04b030 con 0x7fb9a40b4d90 2024-05-11T12:12:39.837 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:12:39.837 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":53,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:12:36.879705+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:12:33.261425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-11T12:12:34.290603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:12:39.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.840+0000 7fb9a9600640 1 -- 172.21.15.167:0/2284244672 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb9800762d0 msgr2=0x7fb980078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:39.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.840+0000 7fb9a9600640 1 --2- 172.21.15.167:0/2284244672 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb9800762d0 0x7fb980078790 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb9940078d0 tx=0x7fb994001b20 comp rx=0 tx=0).stop 2024-05-11T12:12:39.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.840+0000 7fb9a9600640 1 -- 172.21.15.167:0/2284244672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9a40b4d90 msgr2=0x7fb9a40b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:39.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.840+0000 7fb9a9600640 1 --2- 172.21.15.167:0/2284244672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9a40b4d90 0x7fb9a40b51b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb98c009a90 tx=0x7fb98c004730 comp rx=0 tx=0).stop 2024-05-11T12:12:39.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.840+0000 7fb9a9600640 1 -- 172.21.15.167:0/2284244672 shutdown_connections 2024-05-11T12:12:39.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.840+0000 7fb9a9600640 1 --2- 172.21.15.167:0/2284244672 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb9a40b3c30 0x7fb9a40b40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:39.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.840+0000 7fb9a9600640 1 --2- 172.21.15.167:0/2284244672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9a40b4d90 0x7fb9a40b51b0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:39.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.840+0000 7fb9a9600640 1 --2- 172.21.15.167:0/2284244672 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb9800762d0 0x7fb980078790 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:39.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.840+0000 7fb9a9600640 1 --2- 172.21.15.167:0/2284244672 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb988004800 0x7fb9a415bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:39.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.840+0000 7fb9a9600640 1 -- 172.21.15.167:0/2284244672 >> 172.21.15.167:0/2284244672 conn(0x7fb9a40cd8f0 msgr2=0x7fb9a406e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:39.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.840+0000 7fb9a9600640 1 -- 172.21.15.167:0/2284244672 shutdown_connections 2024-05-11T12:12:39.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:39.840+0000 7fb9a9600640 1 -- 172.21.15.167:0/2284244672 wait complete. 2024-05-11T12:12:39.857 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-11T12:12:39.858 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T12:12:40.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.059+0000 7f7432e00640 1 Processor -- start 2024-05-11T12:12:40.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.059+0000 7f7432e00640 1 -- start start 2024-05-11T12:12:40.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.061+0000 7f7432e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f742c145b00 0x7f742c06f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:40.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.061+0000 7f7432e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f742c146770 0x7f742c0ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:40.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.061+0000 7f7432e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f742c0cef10 0x7f742c06c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:40.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.061+0000 7f7432e00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f742c06fbc0 con 0x7f742c145b00 2024-05-11T12:12:40.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.061+0000 7f7432e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f742c0b4520 con 0x7f742c0cef10 2024-05-11T12:12:40.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.061+0000 7f7432e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f742c0b4800 con 0x7f742c146770 2024-05-11T12:12:40.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.061+0000 7f7431a00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f742c145b00 0x7f742c06f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:40.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.061+0000 7f7431a00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f742c145b00 0x7f742c06f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45566/0 (socket says 172.21.15.167:45566) 2024-05-11T12:12:40.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.061+0000 7f7431a00640 1 -- 172.21.15.167:0/489156014 learned_addr learned my addr 172.21.15.167:0/489156014 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:40.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.062+0000 7f7431a00640 1 -- 172.21.15.167:0/489156014 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f742c146770 msgr2=0x7f742c0ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:40.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.062+0000 7f7431a00640 1 --2- 172.21.15.167:0/489156014 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f742c146770 0x7f742c0ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:40.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.062+0000 7f7431a00640 1 -- 172.21.15.167:0/489156014 >> v2:172.21.15.204:3300/0 conn(0x7f742c0cef10 msgr2=0x7f742c06c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:40.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.062+0000 7f7431a00640 1 --2- 172.21.15.167:0/489156014 >> v2:172.21.15.204:3300/0 conn(0x7f742c0cef10 0x7f742c06c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:40.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.062+0000 7f7431a00640 1 -- 172.21.15.167:0/489156014 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f742c0b4ae0 con 0x7f742c145b00 2024-05-11T12:12:40.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.063+0000 7f7431a00640 1 --2- 172.21.15.167:0/489156014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f742c145b00 0x7f742c06f2c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f742c0b4730 tx=0x7f742002a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af6ab57e4fbdc56 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:40.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.063+0000 7f742be00640 1 -- 172.21.15.167:0/489156014 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7420037070 con 0x7f742c145b00 2024-05-11T12:12:40.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.063+0000 7f742be00640 1 -- 172.21.15.167:0/489156014 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7420032440 con 0x7f742c145b00 2024-05-11T12:12:40.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.063+0000 7f742be00640 1 -- 172.21.15.167:0/489156014 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7420032810 con 0x7f742c145b00 2024-05-11T12:12:40.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.064+0000 7f7432e00640 1 -- 172.21.15.167:0/489156014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f742c145b00 msgr2=0x7f742c06f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:40.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.064+0000 7f7432e00640 1 --2- 172.21.15.167:0/489156014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f742c145b00 0x7f742c06f2c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f742c0b4730 tx=0x7f742002a130 comp rx=0 tx=0).stop 2024-05-11T12:12:40.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.064+0000 7f7432e00640 1 -- 172.21.15.167:0/489156014 shutdown_connections 2024-05-11T12:12:40.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.064+0000 7f7432e00640 1 --2- 172.21.15.167:0/489156014 >> v2:172.21.15.204:3300/0 conn(0x7f742c0cef10 0x7f742c06c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:40.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.064+0000 7f7432e00640 1 --2- 172.21.15.167:0/489156014 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f742c146770 0x7f742c0ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:40.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.064+0000 7f7432e00640 1 --2- 172.21.15.167:0/489156014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f742c145b00 0x7f742c06f2c0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:40.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.064+0000 7f7432e00640 1 -- 172.21.15.167:0/489156014 >> 172.21.15.167:0/489156014 conn(0x7f742c0b3460 msgr2=0x7f742c0d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:40.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.064+0000 7f7432e00640 1 -- 172.21.15.167:0/489156014 shutdown_connections 2024-05-11T12:12:40.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.064+0000 7f7432e00640 1 -- 172.21.15.167:0/489156014 wait complete. 2024-05-11T12:12:40.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.066+0000 7f7432e00640 1 Processor -- start 2024-05-11T12:12:40.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.066+0000 7f7432e00640 1 -- start start 2024-05-11T12:12:40.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.067+0000 7f7432e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f742c0cef10 0x7f742c1f8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:40.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.067+0000 7f7432e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f742c145b00 0x7f742c1f7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:40.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.067+0000 7f7432e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f742c146770 0x7f742c0b1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:40.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.067+0000 7f7432e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f742c06f980 con 0x7f742c146770 2024-05-11T12:12:40.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.067+0000 7f7432e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f742c06f800 con 0x7f742c0cef10 2024-05-11T12:12:40.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.067+0000 7f7432e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f742c06cba0 con 0x7f742c145b00 2024-05-11T12:12:40.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.067+0000 7f7432400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f742c146770 0x7f742c0b1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:40.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.067+0000 7f7432400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f742c146770 0x7f742c0b1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45580/0 (socket says 172.21.15.167:45580) 2024-05-11T12:12:40.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.067+0000 7f7432400640 1 -- 172.21.15.167:0/2781618869 learned_addr learned my addr 172.21.15.167:0/2781618869 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:40.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.068+0000 7f7432400640 1 -- 172.21.15.167:0/2781618869 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f742c145b00 msgr2=0x7f742c1f7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:40.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.068+0000 7f7432400640 1 --2- 172.21.15.167:0/2781618869 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f742c145b00 0x7f742c1f7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:40.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.068+0000 7f7432400640 1 -- 172.21.15.167:0/2781618869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f742c0cef10 msgr2=0x7f742c1f8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:40.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.068+0000 7f7432400640 1 --2- 172.21.15.167:0/2781618869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f742c0cef10 0x7f742c1f8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:40.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.068+0000 7f7432400640 1 -- 172.21.15.167:0/2781618869 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f742c1f9320 con 0x7f742c146770 2024-05-11T12:12:40.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.068+0000 7f7432400640 1 --2- 172.21.15.167:0/2781618869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f742c146770 0x7f742c0b1240 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f742400b9a0 tx=0x7f742400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:40.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.068+0000 7f7429600640 1 -- 172.21.15.167:0/2781618869 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7424019070 con 0x7f742c146770 2024-05-11T12:12:40.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.068+0000 7f7429600640 1 -- 172.21.15.167:0/2781618869 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f742400cb60 con 0x7f742c146770 2024-05-11T12:12:40.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.068+0000 7f7429600640 1 -- 172.21.15.167:0/2781618869 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7424014440 con 0x7f742c146770 2024-05-11T12:12:40.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.068+0000 7f7432e00640 1 -- 172.21.15.167:0/2781618869 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f742c1f5d90 con 0x7f742c146770 2024-05-11T12:12:40.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.068+0000 7f7432e00640 1 -- 172.21.15.167:0/2781618869 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f742c1f5460 con 0x7f742c146770 2024-05-11T12:12:40.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.070+0000 7f7432e00640 1 -- 172.21.15.167:0/2781618869 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f742c0694b0 con 0x7f742c146770 2024-05-11T12:12:40.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.071+0000 7f7429600640 1 -- 172.21.15.167:0/2781618869 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f742401c070 con 0x7f742c146770 2024-05-11T12:12:40.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.072+0000 7f7429600640 1 --2- 172.21.15.167:0/2781618869 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f740c0762d0 0x7f740c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:40.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.072+0000 7f7429600640 1 -- 172.21.15.167:0/2781618869 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f7424097550 con 0x7f742c146770 2024-05-11T12:12:40.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.072+0000 7f7431a00640 1 --2- 172.21.15.167:0/2781618869 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f740c0762d0 0x7f740c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:40.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.073+0000 7f7431a00640 1 --2- 172.21.15.167:0/2781618869 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f740c0762d0 0x7f740c078790 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7420034000 tx=0x7f742002c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:40.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.077+0000 7f7429600640 1 -- 172.21.15.167:0/2781618869 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f742c1f9320 con 0x7f742c146770 2024-05-11T12:12:40.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.285+0000 7f7432e00640 1 -- 172.21.15.167:0/2781618869 --> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f742c1f5fa0 con 0x7f740c0762d0 2024-05-11T12:12:40.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.294+0000 7f7429600640 1 -- 172.21.15.167:0/2781618869 <== mgr.4106 v2:172.21.15.167:6832/578352559 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251942 (secure 0 0 0) 0x7f742c1f5fa0 con 0x7f740c0762d0 2024-05-11T12:12:40.297 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:12:40.298 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2024-05-11T12:12:40.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.298+0000 7f7432e00640 1 -- 172.21.15.167:0/2781618869 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f740c0762d0 msgr2=0x7f740c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:40.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.298+0000 7f7432e00640 1 --2- 172.21.15.167:0/2781618869 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f740c0762d0 0x7f740c078790 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7420034000 tx=0x7f742002c000 comp rx=0 tx=0).stop 2024-05-11T12:12:40.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.299+0000 7f7432e00640 1 -- 172.21.15.167:0/2781618869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f742c146770 msgr2=0x7f742c0b1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:40.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.299+0000 7f7432e00640 1 --2- 172.21.15.167:0/2781618869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f742c146770 0x7f742c0b1240 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f742400b9a0 tx=0x7f742400be60 comp rx=0 tx=0).stop 2024-05-11T12:12:40.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.299+0000 7f7432e00640 1 -- 172.21.15.167:0/2781618869 shutdown_connections 2024-05-11T12:12:40.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.299+0000 7f7432e00640 1 --2- 172.21.15.167:0/2781618869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f742c146770 0x7f742c0b1240 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:40.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.299+0000 7f7432e00640 1 --2- 172.21.15.167:0/2781618869 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f742c145b00 0x7f742c1f7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:40.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.299+0000 7f7432e00640 1 --2- 172.21.15.167:0/2781618869 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f740c0762d0 0x7f740c078790 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:40.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.299+0000 7f7432e00640 1 --2- 172.21.15.167:0/2781618869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f742c0cef10 0x7f742c1f8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:40.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.299+0000 7f7432e00640 1 -- 172.21.15.167:0/2781618869 >> 172.21.15.167:0/2781618869 conn(0x7f742c0b3460 msgr2=0x7f742c0afbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:40.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.299+0000 7f7432e00640 1 -- 172.21.15.167:0/2781618869 shutdown_connections 2024-05-11T12:12:40.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.299+0000 7f7432e00640 1 -- 172.21.15.167:0/2781618869 wait complete. 2024-05-11T12:12:40.315 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2024-05-11T12:12:39.270766+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":161,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222228,"kb_used_data":6044,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752492,"statfs":{"total":773094113280,"available":772866551808,"internally_reserved":0,"allocated":6189056,"data_stored":3356112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.437495"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.897064+0000","last_change":"2024-05-11T12:12:35.902949+0000","last_active":"2024-05-11T12:12:36.897064+0000","last_peered":"2024-05-11T12:12:36.897064+0000","last_clean":"2024-05-11T12:12:36.897064+0000","last_became_active":"2024-05-11T12:12:35.902645+0000","last_became_peered":"2024-05-11T12:12:35.902645+0000","last_unstale":"2024-05-11T12:12:36.897064+0000","last_undegraded":"2024-05-11T12:12:36.897064+0000","last_fullsized":"2024-05-11T12:12:36.897064+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:07:59.083017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:37.001789+0000","last_change":"2024-05-11T12:12:35.911490+0000","last_active":"2024-05-11T12:12:37.001789+0000","last_peered":"2024-05-11T12:12:37.001789+0000","last_clean":"2024-05-11T12:12:37.001789+0000","last_became_active":"2024-05-11T12:12:35.909694+0000","last_became_peered":"2024-05-11T12:12:35.909694+0000","last_unstale":"2024-05-11T12:12:37.001789+0000","last_undegraded":"2024-05-11T12:12:37.001789+0000","last_fullsized":"2024-05-11T12:12:37.001789+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:54:53.304242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.996452+0000","last_change":"2024-05-11T12:12:35.911674+0000","last_active":"2024-05-11T12:12:36.996452+0000","last_peered":"2024-05-11T12:12:36.996452+0000","last_clean":"2024-05-11T12:12:36.996452+0000","last_became_active":"2024-05-11T12:12:35.909170+0000","last_became_peered":"2024-05-11T12:12:35.909170+0000","last_unstale":"2024-05-11T12:12:36.996452+0000","last_undegraded":"2024-05-11T12:12:36.996452+0000","last_fullsized":"2024-05-11T12:12:36.996452+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:27:00.117553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:37.001764+0000","last_change":"2024-05-11T12:12:35.907647+0000","last_active":"2024-05-11T12:12:37.001764+0000","last_peered":"2024-05-11T12:12:37.001764+0000","last_clean":"2024-05-11T12:12:37.001764+0000","last_became_active":"2024-05-11T12:12:35.907364+0000","last_became_peered":"2024-05-11T12:12:35.907364+0000","last_unstale":"2024-05-11T12:12:37.001764+0000","last_undegraded":"2024-05-11T12:12:37.001764+0000","last_fullsized":"2024-05-11T12:12:37.001764+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:08:20.888721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:37.000307+0000","last_change":"2024-05-11T12:12:35.909970+0000","last_active":"2024-05-11T12:12:37.000307+0000","last_peered":"2024-05-11T12:12:37.000307+0000","last_clean":"2024-05-11T12:12:37.000307+0000","last_became_active":"2024-05-11T12:12:35.907496+0000","last_became_peered":"2024-05-11T12:12:35.907496+0000","last_unstale":"2024-05-11T12:12:37.000307+0000","last_undegraded":"2024-05-11T12:12:37.000307+0000","last_fullsized":"2024-05-11T12:12:37.000307+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:41:26.025854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.895850+0000","last_change":"2024-05-11T12:12:35.894953+0000","last_active":"2024-05-11T12:12:36.895850+0000","last_peered":"2024-05-11T12:12:36.895850+0000","last_clean":"2024-05-11T12:12:36.895850+0000","last_became_active":"2024-05-11T12:12:35.894705+0000","last_became_peered":"2024-05-11T12:12:35.894705+0000","last_unstale":"2024-05-11T12:12:36.895850+0000","last_undegraded":"2024-05-11T12:12:36.895850+0000","last_fullsized":"2024-05-11T12:12:36.895850+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:05:33.189382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.995688+0000","last_change":"2024-05-11T12:12:35.910126+0000","last_active":"2024-05-11T12:12:36.995688+0000","last_peered":"2024-05-11T12:12:36.995688+0000","last_clean":"2024-05-11T12:12:36.995688+0000","last_became_active":"2024-05-11T12:12:35.904981+0000","last_became_peered":"2024-05-11T12:12:35.904981+0000","last_unstale":"2024-05-11T12:12:36.995688+0000","last_undegraded":"2024-05-11T12:12:36.995688+0000","last_fullsized":"2024-05-11T12:12:36.995688+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:45:55.136734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.894137+0000","last_change":"2024-05-11T12:12:35.892683+0000","last_active":"2024-05-11T12:12:36.894137+0000","last_peered":"2024-05-11T12:12:36.894137+0000","last_clean":"2024-05-11T12:12:36.894137+0000","last_became_active":"2024-05-11T12:12:35.892259+0000","last_became_peered":"2024-05-11T12:12:35.892259+0000","last_unstale":"2024-05-11T12:12:36.894137+0000","last_undegraded":"2024-05-11T12:12:36.894137+0000","last_fullsized":"2024-05-11T12:12:36.894137+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:16:51.794989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.895297+0000","last_change":"2024-05-11T12:12:35.901567+0000","last_active":"2024-05-11T12:12:36.895297+0000","last_peered":"2024-05-11T12:12:36.895297+0000","last_clean":"2024-05-11T12:12:36.895297+0000","last_became_active":"2024-05-11T12:12:35.901300+0000","last_became_peered":"2024-05-11T12:12:35.901300+0000","last_unstale":"2024-05-11T12:12:36.895297+0000","last_undegraded":"2024-05-11T12:12:36.895297+0000","last_fullsized":"2024-05-11T12:12:36.895297+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:04:29.795201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.896387+0000","last_change":"2024-05-11T12:12:35.905638+0000","last_active":"2024-05-11T12:12:36.896387+0000","last_peered":"2024-05-11T12:12:36.896387+0000","last_clean":"2024-05-11T12:12:36.896387+0000","last_became_active":"2024-05-11T12:12:35.903743+0000","last_became_peered":"2024-05-11T12:12:35.903743+0000","last_unstale":"2024-05-11T12:12:36.896387+0000","last_undegraded":"2024-05-11T12:12:36.896387+0000","last_fullsized":"2024-05-11T12:12:36.896387+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:05:46.465596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-11T12:12:35.906971+0000","last_change":"2024-05-11T12:12:35.900554+0000","last_active":"2024-05-11T12:12:35.906971+0000","last_peered":"2024-05-11T12:12:35.906971+0000","last_clean":"2024-05-11T12:12:35.906971+0000","last_became_active":"2024-05-11T12:12:35.899891+0000","last_became_peered":"2024-05-11T12:12:35.899891+0000","last_unstale":"2024-05-11T12:12:35.906971+0000","last_undegraded":"2024-05-11T12:12:35.906971+0000","last_fullsized":"2024-05-11T12:12:35.906971+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:13:52.913419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.894489+0000","last_change":"2024-05-11T12:12:35.906623+0000","last_active":"2024-05-11T12:12:36.894489+0000","last_peered":"2024-05-11T12:12:36.894489+0000","last_clean":"2024-05-11T12:12:36.894489+0000","last_became_active":"2024-05-11T12:12:35.906345+0000","last_became_peered":"2024-05-11T12:12:35.906345+0000","last_unstale":"2024-05-11T12:12:36.894489+0000","last_undegraded":"2024-05-11T12:12:36.894489+0000","last_fullsized":"2024-05-11T12:12:36.894489+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:55:51.311691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.896768+0000","last_change":"2024-05-11T12:12:35.909420+0000","last_active":"2024-05-11T12:12:36.896768+0000","last_peered":"2024-05-11T12:12:36.896768+0000","last_clean":"2024-05-11T12:12:36.896768+0000","last_became_active":"2024-05-11T12:12:35.909143+0000","last_became_peered":"2024-05-11T12:12:35.909143+0000","last_unstale":"2024-05-11T12:12:36.896768+0000","last_undegraded":"2024-05-11T12:12:36.896768+0000","last_fullsized":"2024-05-11T12:12:36.896768+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:22:23.535763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.996413+0000","last_change":"2024-05-11T12:12:35.905111+0000","last_active":"2024-05-11T12:12:36.996413+0000","last_peered":"2024-05-11T12:12:36.996413+0000","last_clean":"2024-05-11T12:12:36.996413+0000","last_became_active":"2024-05-11T12:12:35.904343+0000","last_became_peered":"2024-05-11T12:12:35.904343+0000","last_unstale":"2024-05-11T12:12:36.996413+0000","last_undegraded":"2024-05-11T12:12:36.996413+0000","last_fullsized":"2024-05-11T12:12:36.996413+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:06:13.461753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.999561+0000","last_change":"2024-05-11T12:12:35.912265+0000","last_active":"2024-05-11T12:12:36.999561+0000","last_peered":"2024-05-11T12:12:36.999561+0000","last_clean":"2024-05-11T12:12:36.999561+0000","last_became_active":"2024-05-11T12:12:35.911845+0000","last_became_peered":"2024-05-11T12:12:35.911845+0000","last_unstale":"2024-05-11T12:12:36.999561+0000","last_undegraded":"2024-05-11T12:12:36.999561+0000","last_fullsized":"2024-05-11T12:12:36.999561+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:07:59.083017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.895199+0000","last_change":"2024-05-11T12:12:35.903128+0000","last_active":"2024-05-11T12:12:36.895199+0000","last_peered":"2024-05-11T12:12:36.895199+0000","last_clean":"2024-05-11T12:12:36.895199+0000","last_became_active":"2024-05-11T12:12:35.902846+0000","last_became_peered":"2024-05-11T12:12:35.902846+0000","last_unstale":"2024-05-11T12:12:36.895199+0000","last_undegraded":"2024-05-11T12:12:36.895199+0000","last_fullsized":"2024-05-11T12:12:36.895199+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:21:13.315246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.997426+0000","last_change":"2024-05-11T12:12:35.904085+0000","last_active":"2024-05-11T12:12:36.997426+0000","last_peered":"2024-05-11T12:12:36.997426+0000","last_clean":"2024-05-11T12:12:36.997426+0000","last_became_active":"2024-05-11T12:12:35.901684+0000","last_became_peered":"2024-05-11T12:12:35.901684+0000","last_unstale":"2024-05-11T12:12:36.997426+0000","last_undegraded":"2024-05-11T12:12:36.997426+0000","last_fullsized":"2024-05-11T12:12:36.997426+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:11:40.633859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:37.001749+0000","last_change":"2024-05-11T12:12:35.913214+0000","last_active":"2024-05-11T12:12:37.001749+0000","last_peered":"2024-05-11T12:12:37.001749+0000","last_clean":"2024-05-11T12:12:37.001749+0000","last_became_active":"2024-05-11T12:12:35.912936+0000","last_became_peered":"2024-05-11T12:12:35.912936+0000","last_unstale":"2024-05-11T12:12:37.001749+0000","last_undegraded":"2024-05-11T12:12:37.001749+0000","last_fullsized":"2024-05-11T12:12:37.001749+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:55:13.936904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.996716+0000","last_change":"2024-05-11T12:12:35.909798+0000","last_active":"2024-05-11T12:12:36.996716+0000","last_peered":"2024-05-11T12:12:36.996716+0000","last_clean":"2024-05-11T12:12:36.996716+0000","last_became_active":"2024-05-11T12:12:35.909547+0000","last_became_peered":"2024-05-11T12:12:35.909547+0000","last_unstale":"2024-05-11T12:12:36.996716+0000","last_undegraded":"2024-05-11T12:12:36.996716+0000","last_fullsized":"2024-05-11T12:12:36.996716+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:55:51.311691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.998003+0000","last_change":"2024-05-11T12:12:35.908575+0000","last_active":"2024-05-11T12:12:36.998003+0000","last_peered":"2024-05-11T12:12:36.998003+0000","last_clean":"2024-05-11T12:12:36.998003+0000","last_became_active":"2024-05-11T12:12:35.908099+0000","last_became_peered":"2024-05-11T12:12:35.908099+0000","last_unstale":"2024-05-11T12:12:36.998003+0000","last_undegraded":"2024-05-11T12:12:36.998003+0000","last_fullsized":"2024-05-11T12:12:36.998003+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:26:03.633110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.997840+0000","last_change":"2024-05-11T12:12:35.911283+0000","last_active":"2024-05-11T12:12:36.997840+0000","last_peered":"2024-05-11T12:12:36.997840+0000","last_clean":"2024-05-11T12:12:36.997840+0000","last_became_active":"2024-05-11T12:12:35.910516+0000","last_became_peered":"2024-05-11T12:12:35.910516+0000","last_unstale":"2024-05-11T12:12:36.997840+0000","last_undegraded":"2024-05-11T12:12:36.997840+0000","last_fullsized":"2024-05-11T12:12:36.997840+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:05:46.465596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.894636+0000","last_change":"2024-05-11T12:12:35.901779+0000","last_active":"2024-05-11T12:12:36.894636+0000","last_peered":"2024-05-11T12:12:36.894636+0000","last_clean":"2024-05-11T12:12:36.894636+0000","last_became_active":"2024-05-11T12:12:35.901510+0000","last_became_peered":"2024-05-11T12:12:35.901510+0000","last_unstale":"2024-05-11T12:12:36.894636+0000","last_undegraded":"2024-05-11T12:12:36.894636+0000","last_fullsized":"2024-05-11T12:12:36.894636+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:28:07.216631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-11T12:12:35.889795+0000","last_change":"2024-05-11T12:12:35.887226+0000","last_active":"2024-05-11T12:12:35.889795+0000","last_peered":"2024-05-11T12:12:35.889795+0000","last_clean":"2024-05-11T12:12:35.889795+0000","last_became_active":"2024-05-11T12:12:35.886971+0000","last_became_peered":"2024-05-11T12:12:35.886971+0000","last_unstale":"2024-05-11T12:12:35.889795+0000","last_undegraded":"2024-05-11T12:12:35.889795+0000","last_fullsized":"2024-05-11T12:12:35.889795+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:49:59.788770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.995750+0000","last_change":"2024-05-11T12:12:35.908880+0000","last_active":"2024-05-11T12:12:36.995750+0000","last_peered":"2024-05-11T12:12:36.995750+0000","last_clean":"2024-05-11T12:12:36.995750+0000","last_became_active":"2024-05-11T12:12:35.908577+0000","last_became_peered":"2024-05-11T12:12:35.908577+0000","last_unstale":"2024-05-11T12:12:36.995750+0000","last_undegraded":"2024-05-11T12:12:36.995750+0000","last_fullsized":"2024-05-11T12:12:36.995750+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:03:54.906678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.997667+0000","last_change":"2024-05-11T12:12:35.899053+0000","last_active":"2024-05-11T12:12:36.997667+0000","last_peered":"2024-05-11T12:12:36.997667+0000","last_clean":"2024-05-11T12:12:36.997667+0000","last_became_active":"2024-05-11T12:12:35.898034+0000","last_became_peered":"2024-05-11T12:12:35.898034+0000","last_unstale":"2024-05-11T12:12:36.997667+0000","last_undegraded":"2024-05-11T12:12:36.997667+0000","last_fullsized":"2024-05-11T12:12:36.997667+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:45:25.587344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.896273+0000","last_change":"2024-05-11T12:12:35.892534+0000","last_active":"2024-05-11T12:12:36.896273+0000","last_peered":"2024-05-11T12:12:36.896273+0000","last_clean":"2024-05-11T12:12:36.896273+0000","last_became_active":"2024-05-11T12:12:35.892281+0000","last_became_peered":"2024-05-11T12:12:35.892281+0000","last_unstale":"2024-05-11T12:12:36.896273+0000","last_undegraded":"2024-05-11T12:12:36.896273+0000","last_fullsized":"2024-05-11T12:12:36.896273+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:38:37.221204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.896494+0000","last_change":"2024-05-11T12:12:35.907611+0000","last_active":"2024-05-11T12:12:36.896494+0000","last_peered":"2024-05-11T12:12:36.896494+0000","last_clean":"2024-05-11T12:12:36.896494+0000","last_became_active":"2024-05-11T12:12:35.905641+0000","last_became_peered":"2024-05-11T12:12:35.905641+0000","last_unstale":"2024-05-11T12:12:36.896494+0000","last_undegraded":"2024-05-11T12:12:36.896494+0000","last_fullsized":"2024-05-11T12:12:36.896494+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:26:03.633110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.998964+0000","last_change":"2024-05-11T12:12:35.911153+0000","last_active":"2024-05-11T12:12:36.998964+0000","last_peered":"2024-05-11T12:12:36.998964+0000","last_clean":"2024-05-11T12:12:36.998964+0000","last_became_active":"2024-05-11T12:12:35.910827+0000","last_became_peered":"2024-05-11T12:12:35.910827+0000","last_unstale":"2024-05-11T12:12:36.998964+0000","last_undegraded":"2024-05-11T12:12:36.998964+0000","last_fullsized":"2024-05-11T12:12:36.998964+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:15:29.274470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.995827+0000","last_change":"2024-05-11T12:12:35.912776+0000","last_active":"2024-05-11T12:12:36.995827+0000","last_peered":"2024-05-11T12:12:36.995827+0000","last_clean":"2024-05-11T12:12:36.995827+0000","last_became_active":"2024-05-11T12:12:35.909675+0000","last_became_peered":"2024-05-11T12:12:35.909675+0000","last_unstale":"2024-05-11T12:12:36.995827+0000","last_undegraded":"2024-05-11T12:12:36.995827+0000","last_fullsized":"2024-05-11T12:12:36.995827+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:52:12.110427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:37.000647+0000","last_change":"2024-05-11T12:12:35.902945+0000","last_active":"2024-05-11T12:12:37.000647+0000","last_peered":"2024-05-11T12:12:37.000647+0000","last_clean":"2024-05-11T12:12:37.000647+0000","last_became_active":"2024-05-11T12:12:35.901592+0000","last_became_peered":"2024-05-11T12:12:35.901592+0000","last_unstale":"2024-05-11T12:12:37.000647+0000","last_undegraded":"2024-05-11T12:12:37.000647+0000","last_fullsized":"2024-05-11T12:12:37.000647+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:39:11.359267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.996575+0000","last_change":"2024-05-11T12:12:35.887770+0000","last_active":"2024-05-11T12:12:36.996575+0000","last_peered":"2024-05-11T12:12:36.996575+0000","last_clean":"2024-05-11T12:12:36.996575+0000","last_became_active":"2024-05-11T12:12:35.887461+0000","last_became_peered":"2024-05-11T12:12:35.887461+0000","last_unstale":"2024-05-11T12:12:36.996575+0000","last_undegraded":"2024-05-11T12:12:36.996575+0000","last_fullsized":"2024-05-11T12:12:36.996575+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:18:23.487491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.896686+0000","last_change":"2024-05-11T12:12:35.891569+0000","last_active":"2024-05-11T12:12:36.896686+0000","last_peered":"2024-05-11T12:12:36.896686+0000","last_clean":"2024-05-11T12:12:36.896686+0000","last_became_active":"2024-05-11T12:12:35.890122+0000","last_became_peered":"2024-05-11T12:12:35.890122+0000","last_unstale":"2024-05-11T12:12:36.896686+0000","last_undegraded":"2024-05-11T12:12:36.896686+0000","last_fullsized":"2024-05-11T12:12:36.896686+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:08:24.690584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.998683+0000","last_change":"2024-05-11T12:12:35.897521+0000","last_active":"2024-05-11T12:12:36.998683+0000","last_peered":"2024-05-11T12:12:36.998683+0000","last_clean":"2024-05-11T12:12:36.998683+0000","last_became_active":"2024-05-11T12:12:35.897134+0000","last_became_peered":"2024-05-11T12:12:35.897134+0000","last_unstale":"2024-05-11T12:12:36.998683+0000","last_undegraded":"2024-05-11T12:12:36.998683+0000","last_fullsized":"2024-05-11T12:12:36.998683+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:57:27.618429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"52'4","reported_seq":18,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.998558+0000","last_change":"2024-05-11T12:12:34.875959+0000","last_active":"2024-05-11T12:12:36.998558+0000","last_peered":"2024-05-11T12:12:36.998558+0000","last_clean":"2024-05-11T12:12:36.998558+0000","last_became_active":"2024-05-11T12:12:34.875681+0000","last_became_peered":"2024-05-11T12:12:34.875681+0000","last_unstale":"2024-05-11T12:12:36.998558+0000","last_undegraded":"2024-05-11T12:12:36.998558+0000","last_fullsized":"2024-05-11T12:12:36.998558+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:58:57.419368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.896805+0000","last_change":"2024-05-11T12:12:35.899506+0000","last_active":"2024-05-11T12:12:36.896805+0000","last_peered":"2024-05-11T12:12:36.896805+0000","last_clean":"2024-05-11T12:12:36.896805+0000","last_became_active":"2024-05-11T12:12:35.899203+0000","last_became_peered":"2024-05-11T12:12:35.899203+0000","last_unstale":"2024-05-11T12:12:36.896805+0000","last_undegraded":"2024-05-11T12:12:36.896805+0000","last_fullsized":"2024-05-11T12:12:36.896805+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:45:55.136734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"52'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.896616+0000","last_change":"2024-05-11T12:12:34.880856+0000","last_active":"2024-05-11T12:12:36.896616+0000","last_peered":"2024-05-11T12:12:36.896616+0000","last_clean":"2024-05-11T12:12:36.896616+0000","last_became_active":"2024-05-11T12:12:34.880593+0000","last_became_peered":"2024-05-11T12:12:34.880593+0000","last_unstale":"2024-05-11T12:12:36.896616+0000","last_undegraded":"2024-05-11T12:12:36.896616+0000","last_fullsized":"2024-05-11T12:12:36.896616+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:47:25.153252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.997695+0000","last_change":"2024-05-11T12:12:35.907083+0000","last_active":"2024-05-11T12:12:36.997695+0000","last_peered":"2024-05-11T12:12:36.997695+0000","last_clean":"2024-05-11T12:12:36.997695+0000","last_became_active":"2024-05-11T12:12:35.906781+0000","last_became_peered":"2024-05-11T12:12:35.906781+0000","last_unstale":"2024-05-11T12:12:36.997695+0000","last_undegraded":"2024-05-11T12:12:36.997695+0000","last_fullsized":"2024-05-11T12:12:36.997695+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:04:52.423584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.896600+0000","last_change":"2024-05-11T12:12:34.871981+0000","last_active":"2024-05-11T12:12:36.896600+0000","last_peered":"2024-05-11T12:12:36.896600+0000","last_clean":"2024-05-11T12:12:36.896600+0000","last_became_active":"2024-05-11T12:12:34.871714+0000","last_became_peered":"2024-05-11T12:12:34.871714+0000","last_unstale":"2024-05-11T12:12:36.896600+0000","last_undegraded":"2024-05-11T12:12:36.896600+0000","last_fullsized":"2024-05-11T12:12:36.896600+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:08:04.034126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.894999+0000","last_change":"2024-05-11T12:12:35.891527+0000","last_active":"2024-05-11T12:12:36.894999+0000","last_peered":"2024-05-11T12:12:36.894999+0000","last_clean":"2024-05-11T12:12:36.894999+0000","last_became_active":"2024-05-11T12:12:35.891154+0000","last_became_peered":"2024-05-11T12:12:35.891154+0000","last_unstale":"2024-05-11T12:12:36.894999+0000","last_undegraded":"2024-05-11T12:12:36.894999+0000","last_fullsized":"2024-05-11T12:12:36.894999+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:11:40.633859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.897528+0000","last_change":"2024-05-11T12:12:34.870334+0000","last_active":"2024-05-11T12:12:36.897528+0000","last_peered":"2024-05-11T12:12:36.897528+0000","last_clean":"2024-05-11T12:12:36.897528+0000","last_became_active":"2024-05-11T12:12:34.870086+0000","last_became_peered":"2024-05-11T12:12:34.870086+0000","last_unstale":"2024-05-11T12:12:36.897528+0000","last_undegraded":"2024-05-11T12:12:36.897528+0000","last_fullsized":"2024-05-11T12:12:36.897528+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:57:26.604746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:37.001631+0000","last_change":"2024-05-11T12:12:35.918504+0000","last_active":"2024-05-11T12:12:37.001631+0000","last_peered":"2024-05-11T12:12:37.001631+0000","last_clean":"2024-05-11T12:12:37.001631+0000","last_became_active":"2024-05-11T12:12:35.918183+0000","last_became_peered":"2024-05-11T12:12:35.918183+0000","last_unstale":"2024-05-11T12:12:37.001631+0000","last_undegraded":"2024-05-11T12:12:37.001631+0000","last_fullsized":"2024-05-11T12:12:37.001631+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:05:51.978084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"52'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.996060+0000","last_change":"2024-05-11T12:12:34.879102+0000","last_active":"2024-05-11T12:12:36.996060+0000","last_peered":"2024-05-11T12:12:36.996060+0000","last_clean":"2024-05-11T12:12:36.996060+0000","last_became_active":"2024-05-11T12:12:34.878845+0000","last_became_peered":"2024-05-11T12:12:34.878845+0000","last_unstale":"2024-05-11T12:12:36.996060+0000","last_undegraded":"2024-05-11T12:12:36.996060+0000","last_fullsized":"2024-05-11T12:12:36.996060+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:23:49.239695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:37.001597+0000","last_change":"2024-05-11T12:12:35.902433+0000","last_active":"2024-05-11T12:12:37.001597+0000","last_peered":"2024-05-11T12:12:37.001597+0000","last_clean":"2024-05-11T12:12:37.001597+0000","last_became_active":"2024-05-11T12:12:35.902157+0000","last_became_peered":"2024-05-11T12:12:35.902157+0000","last_unstale":"2024-05-11T12:12:37.001597+0000","last_undegraded":"2024-05-11T12:12:37.001597+0000","last_fullsized":"2024-05-11T12:12:37.001597+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:21:37.077099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"52'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:37.000087+0000","last_change":"2024-05-11T12:12:34.875598+0000","last_active":"2024-05-11T12:12:37.000087+0000","last_peered":"2024-05-11T12:12:37.000087+0000","last_clean":"2024-05-11T12:12:37.000087+0000","last_became_active":"2024-05-11T12:12:34.875296+0000","last_became_peered":"2024-05-11T12:12:34.875296+0000","last_unstale":"2024-05-11T12:12:37.000087+0000","last_undegraded":"2024-05-11T12:12:37.000087+0000","last_fullsized":"2024-05-11T12:12:37.000087+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:12:37.619896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.896724+0000","last_change":"2024-05-11T12:12:35.893781+0000","last_active":"2024-05-11T12:12:36.896724+0000","last_peered":"2024-05-11T12:12:36.896724+0000","last_clean":"2024-05-11T12:12:36.896724+0000","last_became_active":"2024-05-11T12:12:35.893187+0000","last_became_peered":"2024-05-11T12:12:35.893187+0000","last_unstale":"2024-05-11T12:12:36.896724+0000","last_undegraded":"2024-05-11T12:12:36.896724+0000","last_fullsized":"2024-05-11T12:12:36.896724+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:18:07.792899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.997289+0000","last_change":"2024-05-11T12:12:34.880309+0000","last_active":"2024-05-11T12:12:36.997289+0000","last_peered":"2024-05-11T12:12:36.997289+0000","last_clean":"2024-05-11T12:12:36.997289+0000","last_became_active":"2024-05-11T12:12:34.879613+0000","last_became_peered":"2024-05-11T12:12:34.879613+0000","last_unstale":"2024-05-11T12:12:36.997289+0000","last_undegraded":"2024-05-11T12:12:36.997289+0000","last_fullsized":"2024-05-11T12:12:36.997289+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:08:04.034126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:37.001733+0000","last_change":"2024-05-11T12:12:35.919915+0000","last_active":"2024-05-11T12:12:37.001733+0000","last_peered":"2024-05-11T12:12:37.001733+0000","last_clean":"2024-05-11T12:12:37.001733+0000","last_became_active":"2024-05-11T12:12:35.919610+0000","last_became_peered":"2024-05-11T12:12:35.919610+0000","last_unstale":"2024-05-11T12:12:37.001733+0000","last_undegraded":"2024-05-11T12:12:37.001733+0000","last_fullsized":"2024-05-11T12:12:37.001733+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:59:23.453600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.995642+0000","last_change":"2024-05-11T12:12:34.877742+0000","last_active":"2024-05-11T12:12:36.995642+0000","last_peered":"2024-05-11T12:12:36.995642+0000","last_clean":"2024-05-11T12:12:36.995642+0000","last_became_active":"2024-05-11T12:12:34.877195+0000","last_became_peered":"2024-05-11T12:12:34.877195+0000","last_unstale":"2024-05-11T12:12:36.995642+0000","last_undegraded":"2024-05-11T12:12:36.995642+0000","last_fullsized":"2024-05-11T12:12:36.995642+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:04:51.409901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.999741+0000","last_change":"2024-05-11T12:12:35.899909+0000","last_active":"2024-05-11T12:12:36.999741+0000","last_peered":"2024-05-11T12:12:36.999741+0000","last_clean":"2024-05-11T12:12:36.999741+0000","last_became_active":"2024-05-11T12:12:35.899632+0000","last_became_peered":"2024-05-11T12:12:35.899632+0000","last_unstale":"2024-05-11T12:12:36.999741+0000","last_undegraded":"2024-05-11T12:12:36.999741+0000","last_fullsized":"2024-05-11T12:12:36.999741+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:09:12.219980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:37.001616+0000","last_change":"2024-05-11T12:12:34.877012+0000","last_active":"2024-05-11T12:12:37.001616+0000","last_peered":"2024-05-11T12:12:37.001616+0000","last_clean":"2024-05-11T12:12:37.001616+0000","last_became_active":"2024-05-11T12:12:34.876745+0000","last_became_peered":"2024-05-11T12:12:34.876745+0000","last_unstale":"2024-05-11T12:12:37.001616+0000","last_undegraded":"2024-05-11T12:12:37.001616+0000","last_fullsized":"2024-05-11T12:12:37.001616+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:05:50.964401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.896781+0000","last_change":"2024-05-11T12:12:35.908562+0000","last_active":"2024-05-11T12:12:36.896781+0000","last_peered":"2024-05-11T12:12:36.896781+0000","last_clean":"2024-05-11T12:12:36.896781+0000","last_became_active":"2024-05-11T12:12:35.908251+0000","last_became_peered":"2024-05-11T12:12:35.908251+0000","last_unstale":"2024-05-11T12:12:36.896781+0000","last_undegraded":"2024-05-11T12:12:36.896781+0000","last_fullsized":"2024-05-11T12:12:36.896781+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:15:08.594181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:37.000520+0000","last_change":"2024-05-11T12:12:34.879016+0000","last_active":"2024-05-11T12:12:37.000520+0000","last_peered":"2024-05-11T12:12:37.000520+0000","last_clean":"2024-05-11T12:12:37.000520+0000","last_became_active":"2024-05-11T12:12:34.878710+0000","last_became_peered":"2024-05-11T12:12:34.878710+0000","last_unstale":"2024-05-11T12:12:37.000520+0000","last_undegraded":"2024-05-11T12:12:37.000520+0000","last_fullsized":"2024-05-11T12:12:37.000520+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:47:25.153252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.996529+0000","last_change":"2024-05-11T12:12:35.907779+0000","last_active":"2024-05-11T12:12:36.996529+0000","last_peered":"2024-05-11T12:12:36.996529+0000","last_clean":"2024-05-11T12:12:36.996529+0000","last_became_active":"2024-05-11T12:12:35.904819+0000","last_became_peered":"2024-05-11T12:12:35.904819+0000","last_unstale":"2024-05-11T12:12:36.996529+0000","last_undegraded":"2024-05-11T12:12:36.996529+0000","last_fullsized":"2024-05-11T12:12:36.996529+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:16:51.794989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"52'3","reported_seq":17,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:37.001611+0000","last_change":"2024-05-11T12:12:34.876889+0000","last_active":"2024-05-11T12:12:37.001611+0000","last_peered":"2024-05-11T12:12:37.001611+0000","last_clean":"2024-05-11T12:12:37.001611+0000","last_became_active":"2024-05-11T12:12:34.876591+0000","last_became_peered":"2024-05-11T12:12:34.876591+0000","last_unstale":"2024-05-11T12:12:37.001611+0000","last_undegraded":"2024-05-11T12:12:37.001611+0000","last_fullsized":"2024-05-11T12:12:37.001611+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:41:25.012171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:37.000651+0000","last_change":"2024-05-11T12:12:35.911564+0000","last_active":"2024-05-11T12:12:37.000651+0000","last_peered":"2024-05-11T12:12:37.000651+0000","last_clean":"2024-05-11T12:12:37.000651+0000","last_became_active":"2024-05-11T12:12:35.911265+0000","last_became_peered":"2024-05-11T12:12:35.911265+0000","last_unstale":"2024-05-11T12:12:37.000651+0000","last_undegraded":"2024-05-11T12:12:37.000651+0000","last_fullsized":"2024-05-11T12:12:37.000651+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:23:50.253378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"52'6","reported_seq":20,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.894731+0000","last_change":"2024-05-11T12:12:34.852652+0000","last_active":"2024-05-11T12:12:36.894731+0000","last_peered":"2024-05-11T12:12:36.894731+0000","last_clean":"2024-05-11T12:12:36.894731+0000","last_became_active":"2024-05-11T12:12:34.852252+0000","last_became_peered":"2024-05-11T12:12:34.852252+0000","last_unstale":"2024-05-11T12:12:36.894731+0000","last_undegraded":"2024-05-11T12:12:36.894731+0000","last_fullsized":"2024-05-11T12:12:36.894731+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:21:36.063415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.897226+0000","last_change":"2024-05-11T12:12:35.899743+0000","last_active":"2024-05-11T12:12:36.897226+0000","last_peered":"2024-05-11T12:12:36.897226+0000","last_clean":"2024-05-11T12:12:36.897226+0000","last_became_active":"2024-05-11T12:12:35.898666+0000","last_became_peered":"2024-05-11T12:12:35.898666+0000","last_unstale":"2024-05-11T12:12:36.897226+0000","last_undegraded":"2024-05-11T12:12:36.897226+0000","last_fullsized":"2024-05-11T12:12:36.897226+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:52:12.110427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.996125+0000","last_change":"2024-05-11T12:12:34.876500+0000","last_active":"2024-05-11T12:12:36.996125+0000","last_peered":"2024-05-11T12:12:36.996125+0000","last_clean":"2024-05-11T12:12:36.996125+0000","last_became_active":"2024-05-11T12:12:34.875232+0000","last_became_peered":"2024-05-11T12:12:34.875232+0000","last_unstale":"2024-05-11T12:12:36.996125+0000","last_undegraded":"2024-05-11T12:12:36.996125+0000","last_fullsized":"2024-05-11T12:12:36.996125+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:05:50.964401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.997985+0000","last_change":"2024-05-11T12:12:35.909184+0000","last_active":"2024-05-11T12:12:36.997985+0000","last_peered":"2024-05-11T12:12:36.997985+0000","last_clean":"2024-05-11T12:12:36.997985+0000","last_became_active":"2024-05-11T12:12:35.903977+0000","last_became_peered":"2024-05-11T12:12:35.903977+0000","last_unstale":"2024-05-11T12:12:36.997985+0000","last_undegraded":"2024-05-11T12:12:36.997985+0000","last_fullsized":"2024-05-11T12:12:36.997985+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:04:38.991505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.997971+0000","last_change":"2024-05-11T12:12:34.884279+0000","last_active":"2024-05-11T12:12:36.997971+0000","last_peered":"2024-05-11T12:12:36.997971+0000","last_clean":"2024-05-11T12:12:36.997971+0000","last_became_active":"2024-05-11T12:12:34.883968+0000","last_became_peered":"2024-05-11T12:12:34.883968+0000","last_unstale":"2024-05-11T12:12:36.997971+0000","last_undegraded":"2024-05-11T12:12:36.997971+0000","last_fullsized":"2024-05-11T12:12:36.997971+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:49:37.334299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.896275+0000","last_change":"2024-05-11T12:12:35.897340+0000","last_active":"2024-05-11T12:12:36.896275+0000","last_peered":"2024-05-11T12:12:36.896275+0000","last_clean":"2024-05-11T12:12:36.896275+0000","last_became_active":"2024-05-11T12:12:35.897016+0000","last_became_peered":"2024-05-11T12:12:35.897016+0000","last_unstale":"2024-05-11T12:12:36.896275+0000","last_undegraded":"2024-05-11T12:12:36.896275+0000","last_fullsized":"2024-05-11T12:12:36.896275+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:56:47.387069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.999819+0000","last_change":"2024-05-11T12:12:34.878196+0000","last_active":"2024-05-11T12:12:36.999819+0000","last_peered":"2024-05-11T12:12:36.999819+0000","last_clean":"2024-05-11T12:12:36.999819+0000","last_became_active":"2024-05-11T12:12:34.876434+0000","last_became_peered":"2024-05-11T12:12:34.876434+0000","last_unstale":"2024-05-11T12:12:36.999819+0000","last_undegraded":"2024-05-11T12:12:36.999819+0000","last_fullsized":"2024-05-11T12:12:36.999819+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:24:43.843771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.997602+0000","last_change":"2024-05-11T12:12:35.899264+0000","last_active":"2024-05-11T12:12:36.997602+0000","last_peered":"2024-05-11T12:12:36.997602+0000","last_clean":"2024-05-11T12:12:36.997602+0000","last_became_active":"2024-05-11T12:12:35.898738+0000","last_became_peered":"2024-05-11T12:12:35.898738+0000","last_unstale":"2024-05-11T12:12:36.997602+0000","last_undegraded":"2024-05-11T12:12:36.997602+0000","last_fullsized":"2024-05-11T12:12:36.997602+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:04:29.795201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"52'8","reported_seq":22,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.894305+0000","last_change":"2024-05-11T12:12:34.855649+0000","last_active":"2024-05-11T12:12:36.894305+0000","last_peered":"2024-05-11T12:12:36.894305+0000","last_clean":"2024-05-11T12:12:36.894305+0000","last_became_active":"2024-05-11T12:12:34.854413+0000","last_became_peered":"2024-05-11T12:12:34.854413+0000","last_unstale":"2024-05-11T12:12:36.894305+0000","last_undegraded":"2024-05-11T12:12:36.894305+0000","last_fullsized":"2024-05-11T12:12:36.894305+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:39:10.345584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.894290+0000","last_change":"2024-05-11T12:12:35.890771+0000","last_active":"2024-05-11T12:12:36.894290+0000","last_peered":"2024-05-11T12:12:36.894290+0000","last_clean":"2024-05-11T12:12:36.894290+0000","last_became_active":"2024-05-11T12:12:35.890339+0000","last_became_peered":"2024-05-11T12:12:35.890339+0000","last_unstale":"2024-05-11T12:12:36.894290+0000","last_undegraded":"2024-05-11T12:12:36.894290+0000","last_fullsized":"2024-05-11T12:12:36.894290+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:18:23.487491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"52'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.997779+0000","last_change":"2024-05-11T12:12:34.874758+0000","last_active":"2024-05-11T12:12:36.997779+0000","last_peered":"2024-05-11T12:12:36.997779+0000","last_clean":"2024-05-11T12:12:36.997779+0000","last_became_active":"2024-05-11T12:12:34.872416+0000","last_became_peered":"2024-05-11T12:12:34.872416+0000","last_unstale":"2024-05-11T12:12:36.997779+0000","last_undegraded":"2024-05-11T12:12:36.997779+0000","last_fullsized":"2024-05-11T12:12:36.997779+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:29:52.188725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.895528+0000","last_change":"2024-05-11T12:12:35.915603+0000","last_active":"2024-05-11T12:12:36.895528+0000","last_peered":"2024-05-11T12:12:36.895528+0000","last_clean":"2024-05-11T12:12:36.895528+0000","last_became_active":"2024-05-11T12:12:35.915314+0000","last_became_peered":"2024-05-11T12:12:35.915314+0000","last_unstale":"2024-05-11T12:12:36.895528+0000","last_undegraded":"2024-05-11T12:12:36.895528+0000","last_fullsized":"2024-05-11T12:12:36.895528+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:29:53.202409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"52'2","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-11T12:12:35.896842+0000","last_change":"2024-05-11T12:12:34.868902+0000","last_active":"2024-05-11T12:12:35.896842+0000","last_peered":"2024-05-11T12:12:35.896842+0000","last_clean":"2024-05-11T12:12:35.896842+0000","last_became_active":"2024-05-11T12:12:34.868578+0000","last_became_peered":"2024-05-11T12:12:34.868578+0000","last_unstale":"2024-05-11T12:12:35.896842+0000","last_undegraded":"2024-05-11T12:12:35.896842+0000","last_fullsized":"2024-05-11T12:12:35.896842+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:41:00.444365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.999777+0000","last_change":"2024-05-11T12:12:35.918586+0000","last_active":"2024-05-11T12:12:36.999777+0000","last_peered":"2024-05-11T12:12:36.999777+0000","last_clean":"2024-05-11T12:12:36.999777+0000","last_became_active":"2024-05-11T12:12:35.918157+0000","last_became_peered":"2024-05-11T12:12:35.918157+0000","last_unstale":"2024-05-11T12:12:36.999777+0000","last_undegraded":"2024-05-11T12:12:36.999777+0000","last_fullsized":"2024-05-11T12:12:36.999777+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:24:44.857454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"52'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.998305+0000","last_change":"2024-05-11T12:12:34.877389+0000","last_active":"2024-05-11T12:12:36.998305+0000","last_peered":"2024-05-11T12:12:36.998305+0000","last_clean":"2024-05-11T12:12:36.998305+0000","last_became_active":"2024-05-11T12:12:34.876688+0000","last_became_peered":"2024-05-11T12:12:34.876688+0000","last_unstale":"2024-05-11T12:12:36.998305+0000","last_undegraded":"2024-05-11T12:12:36.998305+0000","last_fullsized":"2024-05-11T12:12:36.998305+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:28:53.318080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.896516+0000","last_change":"2024-05-11T12:12:35.895971+0000","last_active":"2024-05-11T12:12:36.896516+0000","last_peered":"2024-05-11T12:12:36.896516+0000","last_clean":"2024-05-11T12:12:36.896516+0000","last_became_active":"2024-05-11T12:12:35.895640+0000","last_became_peered":"2024-05-11T12:12:35.895640+0000","last_unstale":"2024-05-11T12:12:36.896516+0000","last_undegraded":"2024-05-11T12:12:36.896516+0000","last_fullsized":"2024-05-11T12:12:36.896516+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:15:29.274470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.998304+0000","last_change":"2024-05-11T12:12:34.881073+0000","last_active":"2024-05-11T12:12:36.998304+0000","last_peered":"2024-05-11T12:12:36.998304+0000","last_clean":"2024-05-11T12:12:36.998304+0000","last_became_active":"2024-05-11T12:12:34.880510+0000","last_became_peered":"2024-05-11T12:12:34.880510+0000","last_unstale":"2024-05-11T12:12:36.998304+0000","last_undegraded":"2024-05-11T12:12:36.998304+0000","last_fullsized":"2024-05-11T12:12:36.998304+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:07:15.197517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.897235+0000","last_change":"2024-05-11T12:12:35.904628+0000","last_active":"2024-05-11T12:12:36.897235+0000","last_peered":"2024-05-11T12:12:36.897235+0000","last_clean":"2024-05-11T12:12:36.897235+0000","last_became_active":"2024-05-11T12:12:35.904307+0000","last_became_peered":"2024-05-11T12:12:35.904307+0000","last_unstale":"2024-05-11T12:12:36.897235+0000","last_undegraded":"2024-05-11T12:12:36.897235+0000","last_fullsized":"2024-05-11T12:12:36.897235+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:32:18.738049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.896676+0000","last_change":"2024-05-11T12:12:34.876572+0000","last_active":"2024-05-11T12:12:36.896676+0000","last_peered":"2024-05-11T12:12:36.896676+0000","last_clean":"2024-05-11T12:12:36.896676+0000","last_became_active":"2024-05-11T12:12:34.875346+0000","last_became_peered":"2024-05-11T12:12:34.875346+0000","last_unstale":"2024-05-11T12:12:36.896676+0000","last_undegraded":"2024-05-11T12:12:36.896676+0000","last_fullsized":"2024-05-11T12:12:36.896676+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:32:49.107971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.997680+0000","last_change":"2024-05-11T12:12:35.906735+0000","last_active":"2024-05-11T12:12:36.997680+0000","last_peered":"2024-05-11T12:12:36.997680+0000","last_clean":"2024-05-11T12:12:36.997680+0000","last_became_active":"2024-05-11T12:12:35.906394+0000","last_became_peered":"2024-05-11T12:12:35.906394+0000","last_unstale":"2024-05-11T12:12:36.997680+0000","last_undegraded":"2024-05-11T12:12:36.997680+0000","last_fullsized":"2024-05-11T12:12:36.997680+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:05:33.189382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.894620+0000","last_change":"2024-05-11T12:12:34.849998+0000","last_active":"2024-05-11T12:12:36.894620+0000","last_peered":"2024-05-11T12:12:36.894620+0000","last_clean":"2024-05-11T12:12:36.894620+0000","last_became_active":"2024-05-11T12:12:34.849640+0000","last_became_peered":"2024-05-11T12:12:34.849640+0000","last_unstale":"2024-05-11T12:12:36.894620+0000","last_undegraded":"2024-05-11T12:12:36.894620+0000","last_fullsized":"2024-05-11T12:12:36.894620+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:59:22.439917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.997021+0000","last_change":"2024-05-11T12:12:35.911148+0000","last_active":"2024-05-11T12:12:36.997021+0000","last_peered":"2024-05-11T12:12:36.997021+0000","last_clean":"2024-05-11T12:12:36.997021+0000","last_became_active":"2024-05-11T12:12:35.909909+0000","last_became_peered":"2024-05-11T12:12:35.909909+0000","last_unstale":"2024-05-11T12:12:36.997021+0000","last_undegraded":"2024-05-11T12:12:36.997021+0000","last_fullsized":"2024-05-11T12:12:36.997021+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:28:07.216631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.995848+0000","last_change":"2024-05-11T12:12:34.873037+0000","last_active":"2024-05-11T12:12:36.995848+0000","last_peered":"2024-05-11T12:12:36.995848+0000","last_clean":"2024-05-11T12:12:36.995848+0000","last_became_active":"2024-05-11T12:12:34.870011+0000","last_became_peered":"2024-05-11T12:12:34.870011+0000","last_unstale":"2024-05-11T12:12:36.995848+0000","last_undegraded":"2024-05-11T12:12:36.995848+0000","last_fullsized":"2024-05-11T12:12:36.995848+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:41:25.012171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.896948+0000","last_change":"2024-05-11T12:12:35.893448+0000","last_active":"2024-05-11T12:12:36.896948+0000","last_peered":"2024-05-11T12:12:36.896948+0000","last_clean":"2024-05-11T12:12:36.896948+0000","last_became_active":"2024-05-11T12:12:35.893164+0000","last_became_peered":"2024-05-11T12:12:35.893164+0000","last_unstale":"2024-05-11T12:12:36.896948+0000","last_undegraded":"2024-05-11T12:12:36.896948+0000","last_fullsized":"2024-05-11T12:12:36.896948+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:57:27.618429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-11T12:12:35.872209+0000","last_change":"2024-05-11T12:12:34.860395+0000","last_active":"2024-05-11T12:12:35.872209+0000","last_peered":"2024-05-11T12:12:35.872209+0000","last_clean":"2024-05-11T12:12:35.872209+0000","last_became_active":"2024-05-11T12:12:34.860099+0000","last_became_peered":"2024-05-11T12:12:34.860099+0000","last_unstale":"2024-05-11T12:12:35.872209+0000","last_undegraded":"2024-05-11T12:12:35.872209+0000","last_fullsized":"2024-05-11T12:12:35.872209+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:13:59.633245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.896682+0000","last_change":"2024-05-11T12:12:35.893266+0000","last_active":"2024-05-11T12:12:36.896682+0000","last_peered":"2024-05-11T12:12:36.896682+0000","last_clean":"2024-05-11T12:12:36.896682+0000","last_became_active":"2024-05-11T12:12:35.891896+0000","last_became_peered":"2024-05-11T12:12:35.891896+0000","last_unstale":"2024-05-11T12:12:36.896682+0000","last_undegraded":"2024-05-11T12:12:36.896682+0000","last_fullsized":"2024-05-11T12:12:36.896682+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:11:03.801515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.998781+0000","last_change":"2024-05-11T12:12:34.882239+0000","last_active":"2024-05-11T12:12:36.998781+0000","last_peered":"2024-05-11T12:12:36.998781+0000","last_clean":"2024-05-11T12:12:36.998781+0000","last_became_active":"2024-05-11T12:12:34.881983+0000","last_became_peered":"2024-05-11T12:12:34.881983+0000","last_unstale":"2024-05-11T12:12:36.998781+0000","last_undegraded":"2024-05-11T12:12:36.998781+0000","last_fullsized":"2024-05-11T12:12:36.998781+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:26:35.169359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.896226+0000","last_change":"2024-05-11T12:12:35.894137+0000","last_active":"2024-05-11T12:12:36.896226+0000","last_peered":"2024-05-11T12:12:36.896226+0000","last_clean":"2024-05-11T12:12:36.896226+0000","last_became_active":"2024-05-11T12:12:35.893898+0000","last_became_peered":"2024-05-11T12:12:35.893898+0000","last_unstale":"2024-05-11T12:12:36.896226+0000","last_undegraded":"2024-05-11T12:12:36.896226+0000","last_fullsized":"2024-05-11T12:12:36.896226+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:04:52.423584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:37.001896+0000","last_change":"2024-05-11T12:12:34.877279+0000","last_active":"2024-05-11T12:12:37.001896+0000","last_peered":"2024-05-11T12:12:37.001896+0000","last_clean":"2024-05-11T12:12:37.001896+0000","last_became_active":"2024-05-11T12:12:34.876997+0000","last_became_peered":"2024-05-11T12:12:34.876997+0000","last_unstale":"2024-05-11T12:12:37.001896+0000","last_undegraded":"2024-05-11T12:12:37.001896+0000","last_fullsized":"2024-05-11T12:12:37.001896+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:23:49.239695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.895392+0000","last_change":"2024-05-11T12:12:35.901286+0000","last_active":"2024-05-11T12:12:36.895392+0000","last_peered":"2024-05-11T12:12:36.895392+0000","last_clean":"2024-05-11T12:12:36.895392+0000","last_became_active":"2024-05-11T12:12:35.901003+0000","last_became_peered":"2024-05-11T12:12:35.901003+0000","last_unstale":"2024-05-11T12:12:36.895392+0000","last_undegraded":"2024-05-11T12:12:36.895392+0000","last_fullsized":"2024-05-11T12:12:36.895392+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:45:25.587344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.996211+0000","last_change":"2024-05-11T12:12:34.881759+0000","last_active":"2024-05-11T12:12:36.996211+0000","last_peered":"2024-05-11T12:12:36.996211+0000","last_clean":"2024-05-11T12:12:36.996211+0000","last_became_active":"2024-05-11T12:12:34.878053+0000","last_became_peered":"2024-05-11T12:12:34.878053+0000","last_unstale":"2024-05-11T12:12:36.996211+0000","last_undegraded":"2024-05-11T12:12:36.996211+0000","last_fullsized":"2024-05-11T12:12:36.996211+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:21:12.301563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.996048+0000","last_change":"2024-05-11T12:12:35.897607+0000","last_active":"2024-05-11T12:12:36.996048+0000","last_peered":"2024-05-11T12:12:36.996048+0000","last_clean":"2024-05-11T12:12:36.996048+0000","last_became_active":"2024-05-11T12:12:35.897219+0000","last_became_peered":"2024-05-11T12:12:35.897219+0000","last_unstale":"2024-05-11T12:12:36.996048+0000","last_undegraded":"2024-05-11T12:12:36.996048+0000","last_fullsized":"2024-05-11T12:12:36.996048+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:32:18.738049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.895638+0000","last_change":"2024-05-11T12:12:34.860288+0000","last_active":"2024-05-11T12:12:36.895638+0000","last_peered":"2024-05-11T12:12:36.895638+0000","last_clean":"2024-05-11T12:12:36.895638+0000","last_became_active":"2024-05-11T12:12:34.858534+0000","last_became_peered":"2024-05-11T12:12:34.858534+0000","last_unstale":"2024-05-11T12:12:36.895638+0000","last_undegraded":"2024-05-11T12:12:36.895638+0000","last_fullsized":"2024-05-11T12:12:36.895638+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:55:12.923220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.897191+0000","last_change":"2024-05-11T12:12:35.900991+0000","last_active":"2024-05-11T12:12:36.897191+0000","last_peered":"2024-05-11T12:12:36.897191+0000","last_clean":"2024-05-11T12:12:36.897191+0000","last_became_active":"2024-05-11T12:12:35.899178+0000","last_became_peered":"2024-05-11T12:12:35.899178+0000","last_unstale":"2024-05-11T12:12:36.897191+0000","last_undegraded":"2024-05-11T12:12:36.897191+0000","last_fullsized":"2024-05-11T12:12:36.897191+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:03:54.906678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.894866+0000","last_change":"2024-05-11T12:12:34.878691+0000","last_active":"2024-05-11T12:12:36.894866+0000","last_peered":"2024-05-11T12:12:36.894866+0000","last_clean":"2024-05-11T12:12:36.894866+0000","last_became_active":"2024-05-11T12:12:34.878392+0000","last_became_peered":"2024-05-11T12:12:34.878392+0000","last_unstale":"2024-05-11T12:12:36.894866+0000","last_undegraded":"2024-05-11T12:12:36.894866+0000","last_fullsized":"2024-05-11T12:12:36.894866+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:08:19.875038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.996126+0000","last_change":"2024-05-11T12:12:35.904540+0000","last_active":"2024-05-11T12:12:36.996126+0000","last_peered":"2024-05-11T12:12:36.996126+0000","last_clean":"2024-05-11T12:12:36.996126+0000","last_became_active":"2024-05-11T12:12:35.904178+0000","last_became_peered":"2024-05-11T12:12:35.904178+0000","last_unstale":"2024-05-11T12:12:36.996126+0000","last_undegraded":"2024-05-11T12:12:36.996126+0000","last_fullsized":"2024-05-11T12:12:36.996126+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:20:14.406725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:37.000180+0000","last_change":"2024-05-11T12:12:34.877303+0000","last_active":"2024-05-11T12:12:37.000180+0000","last_peered":"2024-05-11T12:12:37.000180+0000","last_clean":"2024-05-11T12:12:37.000180+0000","last_became_active":"2024-05-11T12:12:34.876968+0000","last_became_peered":"2024-05-11T12:12:34.876968+0000","last_unstale":"2024-05-11T12:12:37.000180+0000","last_undegraded":"2024-05-11T12:12:37.000180+0000","last_fullsized":"2024-05-11T12:12:37.000180+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:32:49.107971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.895543+0000","last_change":"2024-05-11T12:12:35.892629+0000","last_active":"2024-05-11T12:12:36.895543+0000","last_peered":"2024-05-11T12:12:36.895543+0000","last_clean":"2024-05-11T12:12:36.895543+0000","last_became_active":"2024-05-11T12:12:35.892244+0000","last_became_peered":"2024-05-11T12:12:35.892244+0000","last_unstale":"2024-05-11T12:12:36.895543+0000","last_undegraded":"2024-05-11T12:12:36.895543+0000","last_fullsized":"2024-05-11T12:12:36.895543+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:04:38.991505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.897066+0000","last_change":"2024-05-11T12:12:34.873773+0000","last_active":"2024-05-11T12:12:36.897066+0000","last_peered":"2024-05-11T12:12:36.897066+0000","last_clean":"2024-05-11T12:12:36.897066+0000","last_became_active":"2024-05-11T12:12:34.872888+0000","last_became_peered":"2024-05-11T12:12:34.872888+0000","last_unstale":"2024-05-11T12:12:36.897066+0000","last_undegraded":"2024-05-11T12:12:36.897066+0000","last_fullsized":"2024-05-11T12:12:36.897066+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:22:22.522080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.997388+0000","last_change":"2024-05-11T12:12:35.903755+0000","last_active":"2024-05-11T12:12:36.997388+0000","last_peered":"2024-05-11T12:12:36.997388+0000","last_clean":"2024-05-11T12:12:36.997388+0000","last_became_active":"2024-05-11T12:12:35.902255+0000","last_became_peered":"2024-05-11T12:12:35.902255+0000","last_unstale":"2024-05-11T12:12:36.997388+0000","last_undegraded":"2024-05-11T12:12:36.997388+0000","last_fullsized":"2024-05-11T12:12:36.997388+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:29:53.202409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.997113+0000","last_change":"2024-05-11T12:12:34.881543+0000","last_active":"2024-05-11T12:12:36.997113+0000","last_peered":"2024-05-11T12:12:36.997113+0000","last_clean":"2024-05-11T12:12:36.997113+0000","last_became_active":"2024-05-11T12:12:34.881274+0000","last_became_peered":"2024-05-11T12:12:34.881274+0000","last_unstale":"2024-05-11T12:12:36.997113+0000","last_undegraded":"2024-05-11T12:12:36.997113+0000","last_fullsized":"2024-05-11T12:12:36.997113+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:05:32.175699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":165,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.999033+0000","last_change":"2024-05-11T12:10:35.066673+0000","last_active":"2024-05-11T12:12:36.999033+0000","last_peered":"2024-05-11T12:12:36.999033+0000","last_clean":"2024-05-11T12:12:36.999033+0000","last_became_active":"2024-05-11T12:10:35.065942+0000","last_became_peered":"2024-05-11T12:10:35.065942+0000","last_unstale":"2024-05-11T12:12:36.999033+0000","last_undegraded":"2024-05-11T12:12:36.999033+0000","last_fullsized":"2024-05-11T12:12:36.999033+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_clean_scrub_stamp":"2024-05-11T12:10:33.601136+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:12:02.133182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476764,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28220,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343620,"statfs":{"total":96636764160,"available":96607866880,"internally_reserved":0,"allocated":1220608,"data_stored":862648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476765,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":626688,"data_stored":272808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476764,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27624,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344216,"statfs":{"total":96636764160,"available":96608477184,"internally_reserved":0,"allocated":618496,"data_stored":272156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476766,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27628,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344212,"statfs":{"total":96636764160,"available":96608473088,"internally_reserved":0,"allocated":622592,"data_stored":272738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509469,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":634880,"data_stored":273790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509470,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":626688,"data_stored":272316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509469,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28220,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343620,"statfs":{"total":96636764160,"available":96607866880,"internally_reserved":0,"allocated":1220608,"data_stored":862580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542173,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27624,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344216,"statfs":{"total":96636764160,"available":96608477184,"internally_reserved":0,"allocated":618496,"data_stored":267076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T12:12:40.319 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T12:12:40.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.558+0000 7f4d0a800640 1 Processor -- start 2024-05-11T12:12:40.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.558+0000 7f4d0a800640 1 -- start start 2024-05-11T12:12:40.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.560+0000 7f4d0a800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4d040afcd0 0x7f4d041601f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:40.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.560+0000 7f4d0a800640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d040b4560 con 0x7f4d040cf970 2024-05-11T12:12:40.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.560+0000 7f4d0a800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d040b4800 con 0x7f4d040d04a0 2024-05-11T12:12:40.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.560+0000 7f4d0a800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4d040b5130 con 0x7f4d040afcd0 2024-05-11T12:12:40.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.560+0000 7f4d09e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f4d040cf970 0x7f4d040721d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:58104/0 (socket says 172.21.15.167:58104) 2024-05-11T12:12:40.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.560+0000 7f4d09e00640 1 -- 172.21.15.167:0/1088752557 learned_addr learned my addr 172.21.15.167:0/1088752557 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:40.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.561+0000 7f4d09400640 1 --2- 172.21.15.167:0/1088752557 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4d040afcd0 0x7f4d041601f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:40.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.561+0000 7f4d03e00640 1 -- 172.21.15.167:0/1088752557 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3637139035 0 0) 0x7f4d040b4560 con 0x7f4d040cf970 2024-05-11T12:12:40.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.561+0000 7f4d03e00640 1 -- 172.21.15.167:0/1088752557 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4cec003520 con 0x7f4d040cf970 2024-05-11T12:12:40.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.561+0000 7f4d03e00640 1 -- 172.21.15.167:0/1088752557 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4117062854 0 0) 0x7f4cec003520 con 0x7f4d040cf970 2024-05-11T12:12:40.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.561+0000 7f4d03e00640 1 -- 172.21.15.167:0/1088752557 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4d040afcd0 msgr2=0x7f4d041601f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:40.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.561+0000 7f4d03e00640 1 --2- 172.21.15.167:0/1088752557 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4d040afcd0 0x7f4d041601f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:40.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.561+0000 7f4d03e00640 1 -- 172.21.15.167:0/1088752557 >> v1:172.21.15.204:6789/0 conn(0x7f4d040d04a0 legacy=0x7f4d040cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:40.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.561+0000 7f4d03e00640 1 -- 172.21.15.167:0/1088752557 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d040b4ae0 con 0x7f4d040cf970 2024-05-11T12:12:40.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.561+0000 7f4d03e00640 1 -- 172.21.15.167:0/1088752557 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f4cfc001a70 con 0x7f4d040cf970 2024-05-11T12:12:40.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.561+0000 7f4d03e00640 1 -- 172.21.15.167:0/1088752557 >> v1:172.21.15.167:6789/0 conn(0x7f4d040cf970 legacy=0x7f4d040721d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:40.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.561+0000 7f4d03e00640 1 --2- 172.21.15.167:0/1088752557 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4d040afcd0 0x7f4cec003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:40.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.561+0000 7f4d03e00640 1 --2- 172.21.15.167:0/1088752557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4cec0040b0 0x7f4cec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:40.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.561+0000 7f4d03e00640 1 --2- 172.21.15.167:0/1088752557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cec004d90 0x7f4cec005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:40.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.561+0000 7f4d03e00640 1 -- 172.21.15.167:0/1088752557 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4d040b5130 con 0x7f4cec004d90 2024-05-11T12:12:40.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.561+0000 7f4d03e00640 1 -- 172.21.15.167:0/1088752557 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4cec00bc00 con 0x7f4cec0040b0 2024-05-11T12:12:40.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.561+0000 7f4d03e00640 1 -- 172.21.15.167:0/1088752557 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4cec00be00 con 0x7f4d040afcd0 2024-05-11T12:12:40.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.562+0000 7f4d09e00640 1 --2- 172.21.15.167:0/1088752557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cec004d90 0x7f4cec005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:40.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.562+0000 7f4d08a00640 1 --2- 172.21.15.167:0/1088752557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4cec0040b0 0x7f4cec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:40.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.562+0000 7f4d09400640 1 --2- 172.21.15.167:0/1088752557 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4d040afcd0 0x7f4cec003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:40.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.562+0000 7f4d09e00640 1 -- 172.21.15.167:0/1088752557 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4d040afcd0 msgr2=0x7f4cec003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:40.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.562+0000 7f4d09e00640 1 --2- 172.21.15.167:0/1088752557 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4d040afcd0 0x7f4cec003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:40.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.562+0000 7f4d09e00640 1 -- 172.21.15.167:0/1088752557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4cec0040b0 msgr2=0x7f4cec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:40.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.562+0000 7f4d09e00640 1 --2- 172.21.15.167:0/1088752557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4cec0040b0 0x7f4cec004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:40.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.562+0000 7f4d09e00640 1 -- 172.21.15.167:0/1088752557 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cec003520 con 0x7f4cec004d90 2024-05-11T12:12:40.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.563+0000 7f4d09e00640 1 --2- 172.21.15.167:0/1088752557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cec004d90 0x7f4cec005210 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4d04071d10 tx=0x7f4cec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6a456c4c3eb6104 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:40.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.564+0000 7f4d03e00640 1 -- 172.21.15.167:0/1088752557 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc00a4e0 con 0x7f4cec004d90 2024-05-11T12:12:40.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.564+0000 7f4d03e00640 1 -- 172.21.15.167:0/1088752557 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cfc02f070 con 0x7f4cec004d90 2024-05-11T12:12:40.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.564+0000 7f4d03e00640 1 -- 172.21.15.167:0/1088752557 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc02bbd0 con 0x7f4cec004d90 2024-05-11T12:12:40.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.564+0000 7f4d0a800640 1 -- 172.21.15.167:0/1088752557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cec004d90 msgr2=0x7f4cec005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:40.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.564+0000 7f4d0a800640 1 --2- 172.21.15.167:0/1088752557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cec004d90 0x7f4cec005210 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4d04071d10 tx=0x7f4cec005830 comp rx=0 tx=0).stop 2024-05-11T12:12:40.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.564+0000 7f4d09e00640 1 -- 172.21.15.167:0/1088752557 reap_dead start 2024-05-11T12:12:40.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.565+0000 7f4d0a800640 1 -- 172.21.15.167:0/1088752557 shutdown_connections 2024-05-11T12:12:40.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.565+0000 7f4d0a800640 1 -- 172.21.15.167:0/1088752557 >> 172.21.15.167:0/1088752557 conn(0x7f4d040b3460 msgr2=0x7f4d040d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:40.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.565+0000 7f4d0a800640 1 -- 172.21.15.167:0/1088752557 shutdown_connections 2024-05-11T12:12:40.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.565+0000 7f4d0a800640 1 -- 172.21.15.167:0/1088752557 wait complete. 2024-05-11T12:12:40.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.566+0000 7f4d0a800640 1 Processor -- start 2024-05-11T12:12:40.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.567+0000 7f4d0a800640 1 -- start start 2024-05-11T12:12:40.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.567+0000 7f4d0a800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4cec004d90 0x7f4d04151670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:40.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.567+0000 7f4d0a800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d0406c450 0x7f4d04071f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:40.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.567+0000 7f4d0a800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d0406c8d0 0x7f4d04072470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:40.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.568+0000 7f4d0a800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4d040cff40 con 0x7f4d0406c8d0 2024-05-11T12:12:40.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.568+0000 7f4d0a800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4d040b00b0 con 0x7f4d0406c450 2024-05-11T12:12:40.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.568+0000 7f4d0a800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4d040d00c0 con 0x7f4cec004d90 2024-05-11T12:12:40.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.568+0000 7f4d09e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d0406c8d0 0x7f4d04072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:40.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.568+0000 7f4d09e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d0406c8d0 0x7f4d04072470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45604/0 (socket says 172.21.15.167:45604) 2024-05-11T12:12:40.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.568+0000 7f4d09e00640 1 -- 172.21.15.167:0/1483165878 learned_addr learned my addr 172.21.15.167:0/1483165878 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:40.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.568+0000 7f4d09400640 1 --2- 172.21.15.167:0/1483165878 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4cec004d90 0x7f4d04151670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:40.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.568+0000 7f4d08a00640 1 --2- 172.21.15.167:0/1483165878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d0406c450 0x7f4d04071f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:40.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.568+0000 7f4d09e00640 1 -- 172.21.15.167:0/1483165878 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4cec004d90 msgr2=0x7f4d04151670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:40.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.568+0000 7f4d09e00640 1 --2- 172.21.15.167:0/1483165878 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4cec004d90 0x7f4d04151670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:40.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.568+0000 7f4d09e00640 1 -- 172.21.15.167:0/1483165878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d0406c450 msgr2=0x7f4d04071f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:40.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.568+0000 7f4d09e00640 1 --2- 172.21.15.167:0/1483165878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d0406c450 0x7f4d04071f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:40.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.568+0000 7f4d09e00640 1 -- 172.21.15.167:0/1483165878 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d04162640 con 0x7f4d0406c8d0 2024-05-11T12:12:40.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.568+0000 7f4d08a00640 1 --2- 172.21.15.167:0/1483165878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d0406c450 0x7f4d04071f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:40.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.569+0000 7f4d09e00640 1 --2- 172.21.15.167:0/1483165878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d0406c8d0 0x7f4d04072470 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc005c20 tx=0x7f4cfc001d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:40.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.569+0000 7f4d01600640 1 -- 172.21.15.167:0/1483165878 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc00a4e0 con 0x7f4d0406c8d0 2024-05-11T12:12:40.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.569+0000 7f4d01600640 1 -- 172.21.15.167:0/1483165878 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cfc034470 con 0x7f4d0406c8d0 2024-05-11T12:12:40.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.569+0000 7f4d01600640 1 -- 172.21.15.167:0/1483165878 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc002030 con 0x7f4d0406c8d0 2024-05-11T12:12:40.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.569+0000 7f4d0a800640 1 -- 172.21.15.167:0/1483165878 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d04162920 con 0x7f4d0406c8d0 2024-05-11T12:12:40.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.569+0000 7f4d0a800640 1 -- 172.21.15.167:0/1483165878 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d04153370 con 0x7f4d0406c8d0 2024-05-11T12:12:40.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.570+0000 7f4d0a800640 1 -- 172.21.15.167:0/1483165878 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d040694b0 con 0x7f4d0406c8d0 2024-05-11T12:12:40.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.571+0000 7f4d01600640 1 -- 172.21.15.167:0/1483165878 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4cfc034820 con 0x7f4d0406c8d0 2024-05-11T12:12:40.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.572+0000 7f4d01600640 1 --2- 172.21.15.167:0/1483165878 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4ce40762d0 0x7f4ce4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:40.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.572+0000 7f4d01600640 1 -- 172.21.15.167:0/1483165878 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f4cfc0b7440 con 0x7f4d0406c8d0 2024-05-11T12:12:40.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.573+0000 7f4d09400640 1 --2- 172.21.15.167:0/1483165878 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4ce40762d0 0x7f4ce4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:40.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.573+0000 7f4d09400640 1 --2- 172.21.15.167:0/1483165878 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4ce40762d0 0x7f4ce4078790 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8006fd0 tx=0x7f4cf8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:40.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.577+0000 7f4d01600640 1 -- 172.21.15.167:0/1483165878 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4cfc080c00 con 0x7f4d0406c8d0 2024-05-11T12:12:40.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.784+0000 7f4d0a800640 1 -- 172.21.15.167:0/1483165878 --> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4d040cf040 con 0x7f4ce40762d0 2024-05-11T12:12:40.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.802+0000 7f4d01600640 1 -- 172.21.15.167:0/1483165878 <== mgr.4106 v2:172.21.15.167:6832/578352559 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251942 (secure 0 0 0) 0x7f4d040cf040 con 0x7f4ce40762d0 2024-05-11T12:12:40.805 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:12:40.806 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2024-05-11T12:12:40.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.807+0000 7f4d0a800640 1 -- 172.21.15.167:0/1483165878 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4ce40762d0 msgr2=0x7f4ce4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:40.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.807+0000 7f4d0a800640 1 --2- 172.21.15.167:0/1483165878 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4ce40762d0 0x7f4ce4078790 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8006fd0 tx=0x7f4cf8008040 comp rx=0 tx=0).stop 2024-05-11T12:12:40.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.807+0000 7f4d0a800640 1 -- 172.21.15.167:0/1483165878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d0406c8d0 msgr2=0x7f4d04072470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:40.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.807+0000 7f4d0a800640 1 --2- 172.21.15.167:0/1483165878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d0406c8d0 0x7f4d04072470 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc005c20 tx=0x7f4cfc001d70 comp rx=0 tx=0).stop 2024-05-11T12:12:40.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.807+0000 7f4d0a800640 1 -- 172.21.15.167:0/1483165878 shutdown_connections 2024-05-11T12:12:40.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.807+0000 7f4d0a800640 1 --2- 172.21.15.167:0/1483165878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d0406c8d0 0x7f4d04072470 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:40.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.807+0000 7f4d0a800640 1 --2- 172.21.15.167:0/1483165878 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4ce40762d0 0x7f4ce4078790 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:40.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.808+0000 7f4d0a800640 1 --2- 172.21.15.167:0/1483165878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d0406c450 0x7f4d04071f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:40.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.808+0000 7f4d0a800640 1 --2- 172.21.15.167:0/1483165878 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4cec004d90 0x7f4d04151670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:40.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.808+0000 7f4d0a800640 1 -- 172.21.15.167:0/1483165878 >> 172.21.15.167:0/1483165878 conn(0x7f4d040b3460 msgr2=0x7f4d040cf260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:40.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.808+0000 7f4d0a800640 1 -- 172.21.15.167:0/1483165878 shutdown_connections 2024-05-11T12:12:40.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:40.808+0000 7f4d0a800640 1 -- 172.21.15.167:0/1483165878 wait complete. 2024-05-11T12:12:40.827 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2024-05-11T12:12:39.270766+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":161,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222228,"kb_used_data":6044,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752492,"statfs":{"total":773094113280,"available":772866551808,"internally_reserved":0,"allocated":6189056,"data_stored":3356112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.437495"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.897064+0000","last_change":"2024-05-11T12:12:35.902949+0000","last_active":"2024-05-11T12:12:36.897064+0000","last_peered":"2024-05-11T12:12:36.897064+0000","last_clean":"2024-05-11T12:12:36.897064+0000","last_became_active":"2024-05-11T12:12:35.902645+0000","last_became_peered":"2024-05-11T12:12:35.902645+0000","last_unstale":"2024-05-11T12:12:36.897064+0000","last_undegraded":"2024-05-11T12:12:36.897064+0000","last_fullsized":"2024-05-11T12:12:36.897064+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:07:59.083017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:37.001789+0000","last_change":"2024-05-11T12:12:35.911490+0000","last_active":"2024-05-11T12:12:37.001789+0000","last_peered":"2024-05-11T12:12:37.001789+0000","last_clean":"2024-05-11T12:12:37.001789+0000","last_became_active":"2024-05-11T12:12:35.909694+0000","last_became_peered":"2024-05-11T12:12:35.909694+0000","last_unstale":"2024-05-11T12:12:37.001789+0000","last_undegraded":"2024-05-11T12:12:37.001789+0000","last_fullsized":"2024-05-11T12:12:37.001789+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:54:53.304242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.996452+0000","last_change":"2024-05-11T12:12:35.911674+0000","last_active":"2024-05-11T12:12:36.996452+0000","last_peered":"2024-05-11T12:12:36.996452+0000","last_clean":"2024-05-11T12:12:36.996452+0000","last_became_active":"2024-05-11T12:12:35.909170+0000","last_became_peered":"2024-05-11T12:12:35.909170+0000","last_unstale":"2024-05-11T12:12:36.996452+0000","last_undegraded":"2024-05-11T12:12:36.996452+0000","last_fullsized":"2024-05-11T12:12:36.996452+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:27:00.117553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:37.001764+0000","last_change":"2024-05-11T12:12:35.907647+0000","last_active":"2024-05-11T12:12:37.001764+0000","last_peered":"2024-05-11T12:12:37.001764+0000","last_clean":"2024-05-11T12:12:37.001764+0000","last_became_active":"2024-05-11T12:12:35.907364+0000","last_became_peered":"2024-05-11T12:12:35.907364+0000","last_unstale":"2024-05-11T12:12:37.001764+0000","last_undegraded":"2024-05-11T12:12:37.001764+0000","last_fullsized":"2024-05-11T12:12:37.001764+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:08:20.888721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:37.000307+0000","last_change":"2024-05-11T12:12:35.909970+0000","last_active":"2024-05-11T12:12:37.000307+0000","last_peered":"2024-05-11T12:12:37.000307+0000","last_clean":"2024-05-11T12:12:37.000307+0000","last_became_active":"2024-05-11T12:12:35.907496+0000","last_became_peered":"2024-05-11T12:12:35.907496+0000","last_unstale":"2024-05-11T12:12:37.000307+0000","last_undegraded":"2024-05-11T12:12:37.000307+0000","last_fullsized":"2024-05-11T12:12:37.000307+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:41:26.025854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.895850+0000","last_change":"2024-05-11T12:12:35.894953+0000","last_active":"2024-05-11T12:12:36.895850+0000","last_peered":"2024-05-11T12:12:36.895850+0000","last_clean":"2024-05-11T12:12:36.895850+0000","last_became_active":"2024-05-11T12:12:35.894705+0000","last_became_peered":"2024-05-11T12:12:35.894705+0000","last_unstale":"2024-05-11T12:12:36.895850+0000","last_undegraded":"2024-05-11T12:12:36.895850+0000","last_fullsized":"2024-05-11T12:12:36.895850+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:05:33.189382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.995688+0000","last_change":"2024-05-11T12:12:35.910126+0000","last_active":"2024-05-11T12:12:36.995688+0000","last_peered":"2024-05-11T12:12:36.995688+0000","last_clean":"2024-05-11T12:12:36.995688+0000","last_became_active":"2024-05-11T12:12:35.904981+0000","last_became_peered":"2024-05-11T12:12:35.904981+0000","last_unstale":"2024-05-11T12:12:36.995688+0000","last_undegraded":"2024-05-11T12:12:36.995688+0000","last_fullsized":"2024-05-11T12:12:36.995688+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:45:55.136734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.894137+0000","last_change":"2024-05-11T12:12:35.892683+0000","last_active":"2024-05-11T12:12:36.894137+0000","last_peered":"2024-05-11T12:12:36.894137+0000","last_clean":"2024-05-11T12:12:36.894137+0000","last_became_active":"2024-05-11T12:12:35.892259+0000","last_became_peered":"2024-05-11T12:12:35.892259+0000","last_unstale":"2024-05-11T12:12:36.894137+0000","last_undegraded":"2024-05-11T12:12:36.894137+0000","last_fullsized":"2024-05-11T12:12:36.894137+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:16:51.794989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.895297+0000","last_change":"2024-05-11T12:12:35.901567+0000","last_active":"2024-05-11T12:12:36.895297+0000","last_peered":"2024-05-11T12:12:36.895297+0000","last_clean":"2024-05-11T12:12:36.895297+0000","last_became_active":"2024-05-11T12:12:35.901300+0000","last_became_peered":"2024-05-11T12:12:35.901300+0000","last_unstale":"2024-05-11T12:12:36.895297+0000","last_undegraded":"2024-05-11T12:12:36.895297+0000","last_fullsized":"2024-05-11T12:12:36.895297+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:04:29.795201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.896387+0000","last_change":"2024-05-11T12:12:35.905638+0000","last_active":"2024-05-11T12:12:36.896387+0000","last_peered":"2024-05-11T12:12:36.896387+0000","last_clean":"2024-05-11T12:12:36.896387+0000","last_became_active":"2024-05-11T12:12:35.903743+0000","last_became_peered":"2024-05-11T12:12:35.903743+0000","last_unstale":"2024-05-11T12:12:36.896387+0000","last_undegraded":"2024-05-11T12:12:36.896387+0000","last_fullsized":"2024-05-11T12:12:36.896387+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:05:46.465596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-11T12:12:35.906971+0000","last_change":"2024-05-11T12:12:35.900554+0000","last_active":"2024-05-11T12:12:35.906971+0000","last_peered":"2024-05-11T12:12:35.906971+0000","last_clean":"2024-05-11T12:12:35.906971+0000","last_became_active":"2024-05-11T12:12:35.899891+0000","last_became_peered":"2024-05-11T12:12:35.899891+0000","last_unstale":"2024-05-11T12:12:35.906971+0000","last_undegraded":"2024-05-11T12:12:35.906971+0000","last_fullsized":"2024-05-11T12:12:35.906971+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:13:52.913419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.894489+0000","last_change":"2024-05-11T12:12:35.906623+0000","last_active":"2024-05-11T12:12:36.894489+0000","last_peered":"2024-05-11T12:12:36.894489+0000","last_clean":"2024-05-11T12:12:36.894489+0000","last_became_active":"2024-05-11T12:12:35.906345+0000","last_became_peered":"2024-05-11T12:12:35.906345+0000","last_unstale":"2024-05-11T12:12:36.894489+0000","last_undegraded":"2024-05-11T12:12:36.894489+0000","last_fullsized":"2024-05-11T12:12:36.894489+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:55:51.311691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.896768+0000","last_change":"2024-05-11T12:12:35.909420+0000","last_active":"2024-05-11T12:12:36.896768+0000","last_peered":"2024-05-11T12:12:36.896768+0000","last_clean":"2024-05-11T12:12:36.896768+0000","last_became_active":"2024-05-11T12:12:35.909143+0000","last_became_peered":"2024-05-11T12:12:35.909143+0000","last_unstale":"2024-05-11T12:12:36.896768+0000","last_undegraded":"2024-05-11T12:12:36.896768+0000","last_fullsized":"2024-05-11T12:12:36.896768+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:22:23.535763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.996413+0000","last_change":"2024-05-11T12:12:35.905111+0000","last_active":"2024-05-11T12:12:36.996413+0000","last_peered":"2024-05-11T12:12:36.996413+0000","last_clean":"2024-05-11T12:12:36.996413+0000","last_became_active":"2024-05-11T12:12:35.904343+0000","last_became_peered":"2024-05-11T12:12:35.904343+0000","last_unstale":"2024-05-11T12:12:36.996413+0000","last_undegraded":"2024-05-11T12:12:36.996413+0000","last_fullsized":"2024-05-11T12:12:36.996413+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:06:13.461753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.999561+0000","last_change":"2024-05-11T12:12:35.912265+0000","last_active":"2024-05-11T12:12:36.999561+0000","last_peered":"2024-05-11T12:12:36.999561+0000","last_clean":"2024-05-11T12:12:36.999561+0000","last_became_active":"2024-05-11T12:12:35.911845+0000","last_became_peered":"2024-05-11T12:12:35.911845+0000","last_unstale":"2024-05-11T12:12:36.999561+0000","last_undegraded":"2024-05-11T12:12:36.999561+0000","last_fullsized":"2024-05-11T12:12:36.999561+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:07:59.083017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.895199+0000","last_change":"2024-05-11T12:12:35.903128+0000","last_active":"2024-05-11T12:12:36.895199+0000","last_peered":"2024-05-11T12:12:36.895199+0000","last_clean":"2024-05-11T12:12:36.895199+0000","last_became_active":"2024-05-11T12:12:35.902846+0000","last_became_peered":"2024-05-11T12:12:35.902846+0000","last_unstale":"2024-05-11T12:12:36.895199+0000","last_undegraded":"2024-05-11T12:12:36.895199+0000","last_fullsized":"2024-05-11T12:12:36.895199+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:21:13.315246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.997426+0000","last_change":"2024-05-11T12:12:35.904085+0000","last_active":"2024-05-11T12:12:36.997426+0000","last_peered":"2024-05-11T12:12:36.997426+0000","last_clean":"2024-05-11T12:12:36.997426+0000","last_became_active":"2024-05-11T12:12:35.901684+0000","last_became_peered":"2024-05-11T12:12:35.901684+0000","last_unstale":"2024-05-11T12:12:36.997426+0000","last_undegraded":"2024-05-11T12:12:36.997426+0000","last_fullsized":"2024-05-11T12:12:36.997426+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:11:40.633859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:37.001749+0000","last_change":"2024-05-11T12:12:35.913214+0000","last_active":"2024-05-11T12:12:37.001749+0000","last_peered":"2024-05-11T12:12:37.001749+0000","last_clean":"2024-05-11T12:12:37.001749+0000","last_became_active":"2024-05-11T12:12:35.912936+0000","last_became_peered":"2024-05-11T12:12:35.912936+0000","last_unstale":"2024-05-11T12:12:37.001749+0000","last_undegraded":"2024-05-11T12:12:37.001749+0000","last_fullsized":"2024-05-11T12:12:37.001749+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:55:13.936904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.996716+0000","last_change":"2024-05-11T12:12:35.909798+0000","last_active":"2024-05-11T12:12:36.996716+0000","last_peered":"2024-05-11T12:12:36.996716+0000","last_clean":"2024-05-11T12:12:36.996716+0000","last_became_active":"2024-05-11T12:12:35.909547+0000","last_became_peered":"2024-05-11T12:12:35.909547+0000","last_unstale":"2024-05-11T12:12:36.996716+0000","last_undegraded":"2024-05-11T12:12:36.996716+0000","last_fullsized":"2024-05-11T12:12:36.996716+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:55:51.311691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.998003+0000","last_change":"2024-05-11T12:12:35.908575+0000","last_active":"2024-05-11T12:12:36.998003+0000","last_peered":"2024-05-11T12:12:36.998003+0000","last_clean":"2024-05-11T12:12:36.998003+0000","last_became_active":"2024-05-11T12:12:35.908099+0000","last_became_peered":"2024-05-11T12:12:35.908099+0000","last_unstale":"2024-05-11T12:12:36.998003+0000","last_undegraded":"2024-05-11T12:12:36.998003+0000","last_fullsized":"2024-05-11T12:12:36.998003+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:26:03.633110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.997840+0000","last_change":"2024-05-11T12:12:35.911283+0000","last_active":"2024-05-11T12:12:36.997840+0000","last_peered":"2024-05-11T12:12:36.997840+0000","last_clean":"2024-05-11T12:12:36.997840+0000","last_became_active":"2024-05-11T12:12:35.910516+0000","last_became_peered":"2024-05-11T12:12:35.910516+0000","last_unstale":"2024-05-11T12:12:36.997840+0000","last_undegraded":"2024-05-11T12:12:36.997840+0000","last_fullsized":"2024-05-11T12:12:36.997840+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:05:46.465596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.894636+0000","last_change":"2024-05-11T12:12:35.901779+0000","last_active":"2024-05-11T12:12:36.894636+0000","last_peered":"2024-05-11T12:12:36.894636+0000","last_clean":"2024-05-11T12:12:36.894636+0000","last_became_active":"2024-05-11T12:12:35.901510+0000","last_became_peered":"2024-05-11T12:12:35.901510+0000","last_unstale":"2024-05-11T12:12:36.894636+0000","last_undegraded":"2024-05-11T12:12:36.894636+0000","last_fullsized":"2024-05-11T12:12:36.894636+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:28:07.216631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-11T12:12:35.889795+0000","last_change":"2024-05-11T12:12:35.887226+0000","last_active":"2024-05-11T12:12:35.889795+0000","last_peered":"2024-05-11T12:12:35.889795+0000","last_clean":"2024-05-11T12:12:35.889795+0000","last_became_active":"2024-05-11T12:12:35.886971+0000","last_became_peered":"2024-05-11T12:12:35.886971+0000","last_unstale":"2024-05-11T12:12:35.889795+0000","last_undegraded":"2024-05-11T12:12:35.889795+0000","last_fullsized":"2024-05-11T12:12:35.889795+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:49:59.788770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.995750+0000","last_change":"2024-05-11T12:12:35.908880+0000","last_active":"2024-05-11T12:12:36.995750+0000","last_peered":"2024-05-11T12:12:36.995750+0000","last_clean":"2024-05-11T12:12:36.995750+0000","last_became_active":"2024-05-11T12:12:35.908577+0000","last_became_peered":"2024-05-11T12:12:35.908577+0000","last_unstale":"2024-05-11T12:12:36.995750+0000","last_undegraded":"2024-05-11T12:12:36.995750+0000","last_fullsized":"2024-05-11T12:12:36.995750+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:03:54.906678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.997667+0000","last_change":"2024-05-11T12:12:35.899053+0000","last_active":"2024-05-11T12:12:36.997667+0000","last_peered":"2024-05-11T12:12:36.997667+0000","last_clean":"2024-05-11T12:12:36.997667+0000","last_became_active":"2024-05-11T12:12:35.898034+0000","last_became_peered":"2024-05-11T12:12:35.898034+0000","last_unstale":"2024-05-11T12:12:36.997667+0000","last_undegraded":"2024-05-11T12:12:36.997667+0000","last_fullsized":"2024-05-11T12:12:36.997667+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:45:25.587344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.896273+0000","last_change":"2024-05-11T12:12:35.892534+0000","last_active":"2024-05-11T12:12:36.896273+0000","last_peered":"2024-05-11T12:12:36.896273+0000","last_clean":"2024-05-11T12:12:36.896273+0000","last_became_active":"2024-05-11T12:12:35.892281+0000","last_became_peered":"2024-05-11T12:12:35.892281+0000","last_unstale":"2024-05-11T12:12:36.896273+0000","last_undegraded":"2024-05-11T12:12:36.896273+0000","last_fullsized":"2024-05-11T12:12:36.896273+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:38:37.221204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.896494+0000","last_change":"2024-05-11T12:12:35.907611+0000","last_active":"2024-05-11T12:12:36.896494+0000","last_peered":"2024-05-11T12:12:36.896494+0000","last_clean":"2024-05-11T12:12:36.896494+0000","last_became_active":"2024-05-11T12:12:35.905641+0000","last_became_peered":"2024-05-11T12:12:35.905641+0000","last_unstale":"2024-05-11T12:12:36.896494+0000","last_undegraded":"2024-05-11T12:12:36.896494+0000","last_fullsized":"2024-05-11T12:12:36.896494+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:26:03.633110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.998964+0000","last_change":"2024-05-11T12:12:35.911153+0000","last_active":"2024-05-11T12:12:36.998964+0000","last_peered":"2024-05-11T12:12:36.998964+0000","last_clean":"2024-05-11T12:12:36.998964+0000","last_became_active":"2024-05-11T12:12:35.910827+0000","last_became_peered":"2024-05-11T12:12:35.910827+0000","last_unstale":"2024-05-11T12:12:36.998964+0000","last_undegraded":"2024-05-11T12:12:36.998964+0000","last_fullsized":"2024-05-11T12:12:36.998964+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:15:29.274470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.995827+0000","last_change":"2024-05-11T12:12:35.912776+0000","last_active":"2024-05-11T12:12:36.995827+0000","last_peered":"2024-05-11T12:12:36.995827+0000","last_clean":"2024-05-11T12:12:36.995827+0000","last_became_active":"2024-05-11T12:12:35.909675+0000","last_became_peered":"2024-05-11T12:12:35.909675+0000","last_unstale":"2024-05-11T12:12:36.995827+0000","last_undegraded":"2024-05-11T12:12:36.995827+0000","last_fullsized":"2024-05-11T12:12:36.995827+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:52:12.110427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:37.000647+0000","last_change":"2024-05-11T12:12:35.902945+0000","last_active":"2024-05-11T12:12:37.000647+0000","last_peered":"2024-05-11T12:12:37.000647+0000","last_clean":"2024-05-11T12:12:37.000647+0000","last_became_active":"2024-05-11T12:12:35.901592+0000","last_became_peered":"2024-05-11T12:12:35.901592+0000","last_unstale":"2024-05-11T12:12:37.000647+0000","last_undegraded":"2024-05-11T12:12:37.000647+0000","last_fullsized":"2024-05-11T12:12:37.000647+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:39:11.359267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.996575+0000","last_change":"2024-05-11T12:12:35.887770+0000","last_active":"2024-05-11T12:12:36.996575+0000","last_peered":"2024-05-11T12:12:36.996575+0000","last_clean":"2024-05-11T12:12:36.996575+0000","last_became_active":"2024-05-11T12:12:35.887461+0000","last_became_peered":"2024-05-11T12:12:35.887461+0000","last_unstale":"2024-05-11T12:12:36.996575+0000","last_undegraded":"2024-05-11T12:12:36.996575+0000","last_fullsized":"2024-05-11T12:12:36.996575+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:18:23.487491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.896686+0000","last_change":"2024-05-11T12:12:35.891569+0000","last_active":"2024-05-11T12:12:36.896686+0000","last_peered":"2024-05-11T12:12:36.896686+0000","last_clean":"2024-05-11T12:12:36.896686+0000","last_became_active":"2024-05-11T12:12:35.890122+0000","last_became_peered":"2024-05-11T12:12:35.890122+0000","last_unstale":"2024-05-11T12:12:36.896686+0000","last_undegraded":"2024-05-11T12:12:36.896686+0000","last_fullsized":"2024-05-11T12:12:36.896686+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:08:24.690584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.998683+0000","last_change":"2024-05-11T12:12:35.897521+0000","last_active":"2024-05-11T12:12:36.998683+0000","last_peered":"2024-05-11T12:12:36.998683+0000","last_clean":"2024-05-11T12:12:36.998683+0000","last_became_active":"2024-05-11T12:12:35.897134+0000","last_became_peered":"2024-05-11T12:12:35.897134+0000","last_unstale":"2024-05-11T12:12:36.998683+0000","last_undegraded":"2024-05-11T12:12:36.998683+0000","last_fullsized":"2024-05-11T12:12:36.998683+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:57:27.618429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"52'4","reported_seq":18,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.998558+0000","last_change":"2024-05-11T12:12:34.875959+0000","last_active":"2024-05-11T12:12:36.998558+0000","last_peered":"2024-05-11T12:12:36.998558+0000","last_clean":"2024-05-11T12:12:36.998558+0000","last_became_active":"2024-05-11T12:12:34.875681+0000","last_became_peered":"2024-05-11T12:12:34.875681+0000","last_unstale":"2024-05-11T12:12:36.998558+0000","last_undegraded":"2024-05-11T12:12:36.998558+0000","last_fullsized":"2024-05-11T12:12:36.998558+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:58:57.419368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.896805+0000","last_change":"2024-05-11T12:12:35.899506+0000","last_active":"2024-05-11T12:12:36.896805+0000","last_peered":"2024-05-11T12:12:36.896805+0000","last_clean":"2024-05-11T12:12:36.896805+0000","last_became_active":"2024-05-11T12:12:35.899203+0000","last_became_peered":"2024-05-11T12:12:35.899203+0000","last_unstale":"2024-05-11T12:12:36.896805+0000","last_undegraded":"2024-05-11T12:12:36.896805+0000","last_fullsized":"2024-05-11T12:12:36.896805+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:45:55.136734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"52'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.896616+0000","last_change":"2024-05-11T12:12:34.880856+0000","last_active":"2024-05-11T12:12:36.896616+0000","last_peered":"2024-05-11T12:12:36.896616+0000","last_clean":"2024-05-11T12:12:36.896616+0000","last_became_active":"2024-05-11T12:12:34.880593+0000","last_became_peered":"2024-05-11T12:12:34.880593+0000","last_unstale":"2024-05-11T12:12:36.896616+0000","last_undegraded":"2024-05-11T12:12:36.896616+0000","last_fullsized":"2024-05-11T12:12:36.896616+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:47:25.153252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.997695+0000","last_change":"2024-05-11T12:12:35.907083+0000","last_active":"2024-05-11T12:12:36.997695+0000","last_peered":"2024-05-11T12:12:36.997695+0000","last_clean":"2024-05-11T12:12:36.997695+0000","last_became_active":"2024-05-11T12:12:35.906781+0000","last_became_peered":"2024-05-11T12:12:35.906781+0000","last_unstale":"2024-05-11T12:12:36.997695+0000","last_undegraded":"2024-05-11T12:12:36.997695+0000","last_fullsized":"2024-05-11T12:12:36.997695+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:04:52.423584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.896600+0000","last_change":"2024-05-11T12:12:34.871981+0000","last_active":"2024-05-11T12:12:36.896600+0000","last_peered":"2024-05-11T12:12:36.896600+0000","last_clean":"2024-05-11T12:12:36.896600+0000","last_became_active":"2024-05-11T12:12:34.871714+0000","last_became_peered":"2024-05-11T12:12:34.871714+0000","last_unstale":"2024-05-11T12:12:36.896600+0000","last_undegraded":"2024-05-11T12:12:36.896600+0000","last_fullsized":"2024-05-11T12:12:36.896600+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:08:04.034126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.894999+0000","last_change":"2024-05-11T12:12:35.891527+0000","last_active":"2024-05-11T12:12:36.894999+0000","last_peered":"2024-05-11T12:12:36.894999+0000","last_clean":"2024-05-11T12:12:36.894999+0000","last_became_active":"2024-05-11T12:12:35.891154+0000","last_became_peered":"2024-05-11T12:12:35.891154+0000","last_unstale":"2024-05-11T12:12:36.894999+0000","last_undegraded":"2024-05-11T12:12:36.894999+0000","last_fullsized":"2024-05-11T12:12:36.894999+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:11:40.633859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.897528+0000","last_change":"2024-05-11T12:12:34.870334+0000","last_active":"2024-05-11T12:12:36.897528+0000","last_peered":"2024-05-11T12:12:36.897528+0000","last_clean":"2024-05-11T12:12:36.897528+0000","last_became_active":"2024-05-11T12:12:34.870086+0000","last_became_peered":"2024-05-11T12:12:34.870086+0000","last_unstale":"2024-05-11T12:12:36.897528+0000","last_undegraded":"2024-05-11T12:12:36.897528+0000","last_fullsized":"2024-05-11T12:12:36.897528+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:57:26.604746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:37.001631+0000","last_change":"2024-05-11T12:12:35.918504+0000","last_active":"2024-05-11T12:12:37.001631+0000","last_peered":"2024-05-11T12:12:37.001631+0000","last_clean":"2024-05-11T12:12:37.001631+0000","last_became_active":"2024-05-11T12:12:35.918183+0000","last_became_peered":"2024-05-11T12:12:35.918183+0000","last_unstale":"2024-05-11T12:12:37.001631+0000","last_undegraded":"2024-05-11T12:12:37.001631+0000","last_fullsized":"2024-05-11T12:12:37.001631+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:05:51.978084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"52'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.996060+0000","last_change":"2024-05-11T12:12:34.879102+0000","last_active":"2024-05-11T12:12:36.996060+0000","last_peered":"2024-05-11T12:12:36.996060+0000","last_clean":"2024-05-11T12:12:36.996060+0000","last_became_active":"2024-05-11T12:12:34.878845+0000","last_became_peered":"2024-05-11T12:12:34.878845+0000","last_unstale":"2024-05-11T12:12:36.996060+0000","last_undegraded":"2024-05-11T12:12:36.996060+0000","last_fullsized":"2024-05-11T12:12:36.996060+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:23:49.239695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:37.001597+0000","last_change":"2024-05-11T12:12:35.902433+0000","last_active":"2024-05-11T12:12:37.001597+0000","last_peered":"2024-05-11T12:12:37.001597+0000","last_clean":"2024-05-11T12:12:37.001597+0000","last_became_active":"2024-05-11T12:12:35.902157+0000","last_became_peered":"2024-05-11T12:12:35.902157+0000","last_unstale":"2024-05-11T12:12:37.001597+0000","last_undegraded":"2024-05-11T12:12:37.001597+0000","last_fullsized":"2024-05-11T12:12:37.001597+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:21:37.077099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"52'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:37.000087+0000","last_change":"2024-05-11T12:12:34.875598+0000","last_active":"2024-05-11T12:12:37.000087+0000","last_peered":"2024-05-11T12:12:37.000087+0000","last_clean":"2024-05-11T12:12:37.000087+0000","last_became_active":"2024-05-11T12:12:34.875296+0000","last_became_peered":"2024-05-11T12:12:34.875296+0000","last_unstale":"2024-05-11T12:12:37.000087+0000","last_undegraded":"2024-05-11T12:12:37.000087+0000","last_fullsized":"2024-05-11T12:12:37.000087+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:12:37.619896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.896724+0000","last_change":"2024-05-11T12:12:35.893781+0000","last_active":"2024-05-11T12:12:36.896724+0000","last_peered":"2024-05-11T12:12:36.896724+0000","last_clean":"2024-05-11T12:12:36.896724+0000","last_became_active":"2024-05-11T12:12:35.893187+0000","last_became_peered":"2024-05-11T12:12:35.893187+0000","last_unstale":"2024-05-11T12:12:36.896724+0000","last_undegraded":"2024-05-11T12:12:36.896724+0000","last_fullsized":"2024-05-11T12:12:36.896724+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:18:07.792899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.997289+0000","last_change":"2024-05-11T12:12:34.880309+0000","last_active":"2024-05-11T12:12:36.997289+0000","last_peered":"2024-05-11T12:12:36.997289+0000","last_clean":"2024-05-11T12:12:36.997289+0000","last_became_active":"2024-05-11T12:12:34.879613+0000","last_became_peered":"2024-05-11T12:12:34.879613+0000","last_unstale":"2024-05-11T12:12:36.997289+0000","last_undegraded":"2024-05-11T12:12:36.997289+0000","last_fullsized":"2024-05-11T12:12:36.997289+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:08:04.034126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:37.001733+0000","last_change":"2024-05-11T12:12:35.919915+0000","last_active":"2024-05-11T12:12:37.001733+0000","last_peered":"2024-05-11T12:12:37.001733+0000","last_clean":"2024-05-11T12:12:37.001733+0000","last_became_active":"2024-05-11T12:12:35.919610+0000","last_became_peered":"2024-05-11T12:12:35.919610+0000","last_unstale":"2024-05-11T12:12:37.001733+0000","last_undegraded":"2024-05-11T12:12:37.001733+0000","last_fullsized":"2024-05-11T12:12:37.001733+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:59:23.453600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.995642+0000","last_change":"2024-05-11T12:12:34.877742+0000","last_active":"2024-05-11T12:12:36.995642+0000","last_peered":"2024-05-11T12:12:36.995642+0000","last_clean":"2024-05-11T12:12:36.995642+0000","last_became_active":"2024-05-11T12:12:34.877195+0000","last_became_peered":"2024-05-11T12:12:34.877195+0000","last_unstale":"2024-05-11T12:12:36.995642+0000","last_undegraded":"2024-05-11T12:12:36.995642+0000","last_fullsized":"2024-05-11T12:12:36.995642+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:04:51.409901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.999741+0000","last_change":"2024-05-11T12:12:35.899909+0000","last_active":"2024-05-11T12:12:36.999741+0000","last_peered":"2024-05-11T12:12:36.999741+0000","last_clean":"2024-05-11T12:12:36.999741+0000","last_became_active":"2024-05-11T12:12:35.899632+0000","last_became_peered":"2024-05-11T12:12:35.899632+0000","last_unstale":"2024-05-11T12:12:36.999741+0000","last_undegraded":"2024-05-11T12:12:36.999741+0000","last_fullsized":"2024-05-11T12:12:36.999741+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:09:12.219980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:37.001616+0000","last_change":"2024-05-11T12:12:34.877012+0000","last_active":"2024-05-11T12:12:37.001616+0000","last_peered":"2024-05-11T12:12:37.001616+0000","last_clean":"2024-05-11T12:12:37.001616+0000","last_became_active":"2024-05-11T12:12:34.876745+0000","last_became_peered":"2024-05-11T12:12:34.876745+0000","last_unstale":"2024-05-11T12:12:37.001616+0000","last_undegraded":"2024-05-11T12:12:37.001616+0000","last_fullsized":"2024-05-11T12:12:37.001616+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:05:50.964401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.896781+0000","last_change":"2024-05-11T12:12:35.908562+0000","last_active":"2024-05-11T12:12:36.896781+0000","last_peered":"2024-05-11T12:12:36.896781+0000","last_clean":"2024-05-11T12:12:36.896781+0000","last_became_active":"2024-05-11T12:12:35.908251+0000","last_became_peered":"2024-05-11T12:12:35.908251+0000","last_unstale":"2024-05-11T12:12:36.896781+0000","last_undegraded":"2024-05-11T12:12:36.896781+0000","last_fullsized":"2024-05-11T12:12:36.896781+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:15:08.594181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:37.000520+0000","last_change":"2024-05-11T12:12:34.879016+0000","last_active":"2024-05-11T12:12:37.000520+0000","last_peered":"2024-05-11T12:12:37.000520+0000","last_clean":"2024-05-11T12:12:37.000520+0000","last_became_active":"2024-05-11T12:12:34.878710+0000","last_became_peered":"2024-05-11T12:12:34.878710+0000","last_unstale":"2024-05-11T12:12:37.000520+0000","last_undegraded":"2024-05-11T12:12:37.000520+0000","last_fullsized":"2024-05-11T12:12:37.000520+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:47:25.153252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.996529+0000","last_change":"2024-05-11T12:12:35.907779+0000","last_active":"2024-05-11T12:12:36.996529+0000","last_peered":"2024-05-11T12:12:36.996529+0000","last_clean":"2024-05-11T12:12:36.996529+0000","last_became_active":"2024-05-11T12:12:35.904819+0000","last_became_peered":"2024-05-11T12:12:35.904819+0000","last_unstale":"2024-05-11T12:12:36.996529+0000","last_undegraded":"2024-05-11T12:12:36.996529+0000","last_fullsized":"2024-05-11T12:12:36.996529+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:16:51.794989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"52'3","reported_seq":17,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:37.001611+0000","last_change":"2024-05-11T12:12:34.876889+0000","last_active":"2024-05-11T12:12:37.001611+0000","last_peered":"2024-05-11T12:12:37.001611+0000","last_clean":"2024-05-11T12:12:37.001611+0000","last_became_active":"2024-05-11T12:12:34.876591+0000","last_became_peered":"2024-05-11T12:12:34.876591+0000","last_unstale":"2024-05-11T12:12:37.001611+0000","last_undegraded":"2024-05-11T12:12:37.001611+0000","last_fullsized":"2024-05-11T12:12:37.001611+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:41:25.012171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:37.000651+0000","last_change":"2024-05-11T12:12:35.911564+0000","last_active":"2024-05-11T12:12:37.000651+0000","last_peered":"2024-05-11T12:12:37.000651+0000","last_clean":"2024-05-11T12:12:37.000651+0000","last_became_active":"2024-05-11T12:12:35.911265+0000","last_became_peered":"2024-05-11T12:12:35.911265+0000","last_unstale":"2024-05-11T12:12:37.000651+0000","last_undegraded":"2024-05-11T12:12:37.000651+0000","last_fullsized":"2024-05-11T12:12:37.000651+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:23:50.253378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"52'6","reported_seq":20,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.894731+0000","last_change":"2024-05-11T12:12:34.852652+0000","last_active":"2024-05-11T12:12:36.894731+0000","last_peered":"2024-05-11T12:12:36.894731+0000","last_clean":"2024-05-11T12:12:36.894731+0000","last_became_active":"2024-05-11T12:12:34.852252+0000","last_became_peered":"2024-05-11T12:12:34.852252+0000","last_unstale":"2024-05-11T12:12:36.894731+0000","last_undegraded":"2024-05-11T12:12:36.894731+0000","last_fullsized":"2024-05-11T12:12:36.894731+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:21:36.063415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.897226+0000","last_change":"2024-05-11T12:12:35.899743+0000","last_active":"2024-05-11T12:12:36.897226+0000","last_peered":"2024-05-11T12:12:36.897226+0000","last_clean":"2024-05-11T12:12:36.897226+0000","last_became_active":"2024-05-11T12:12:35.898666+0000","last_became_peered":"2024-05-11T12:12:35.898666+0000","last_unstale":"2024-05-11T12:12:36.897226+0000","last_undegraded":"2024-05-11T12:12:36.897226+0000","last_fullsized":"2024-05-11T12:12:36.897226+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:52:12.110427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.996125+0000","last_change":"2024-05-11T12:12:34.876500+0000","last_active":"2024-05-11T12:12:36.996125+0000","last_peered":"2024-05-11T12:12:36.996125+0000","last_clean":"2024-05-11T12:12:36.996125+0000","last_became_active":"2024-05-11T12:12:34.875232+0000","last_became_peered":"2024-05-11T12:12:34.875232+0000","last_unstale":"2024-05-11T12:12:36.996125+0000","last_undegraded":"2024-05-11T12:12:36.996125+0000","last_fullsized":"2024-05-11T12:12:36.996125+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:05:50.964401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.997985+0000","last_change":"2024-05-11T12:12:35.909184+0000","last_active":"2024-05-11T12:12:36.997985+0000","last_peered":"2024-05-11T12:12:36.997985+0000","last_clean":"2024-05-11T12:12:36.997985+0000","last_became_active":"2024-05-11T12:12:35.903977+0000","last_became_peered":"2024-05-11T12:12:35.903977+0000","last_unstale":"2024-05-11T12:12:36.997985+0000","last_undegraded":"2024-05-11T12:12:36.997985+0000","last_fullsized":"2024-05-11T12:12:36.997985+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:04:38.991505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.997971+0000","last_change":"2024-05-11T12:12:34.884279+0000","last_active":"2024-05-11T12:12:36.997971+0000","last_peered":"2024-05-11T12:12:36.997971+0000","last_clean":"2024-05-11T12:12:36.997971+0000","last_became_active":"2024-05-11T12:12:34.883968+0000","last_became_peered":"2024-05-11T12:12:34.883968+0000","last_unstale":"2024-05-11T12:12:36.997971+0000","last_undegraded":"2024-05-11T12:12:36.997971+0000","last_fullsized":"2024-05-11T12:12:36.997971+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:49:37.334299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.896275+0000","last_change":"2024-05-11T12:12:35.897340+0000","last_active":"2024-05-11T12:12:36.896275+0000","last_peered":"2024-05-11T12:12:36.896275+0000","last_clean":"2024-05-11T12:12:36.896275+0000","last_became_active":"2024-05-11T12:12:35.897016+0000","last_became_peered":"2024-05-11T12:12:35.897016+0000","last_unstale":"2024-05-11T12:12:36.896275+0000","last_undegraded":"2024-05-11T12:12:36.896275+0000","last_fullsized":"2024-05-11T12:12:36.896275+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:56:47.387069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.999819+0000","last_change":"2024-05-11T12:12:34.878196+0000","last_active":"2024-05-11T12:12:36.999819+0000","last_peered":"2024-05-11T12:12:36.999819+0000","last_clean":"2024-05-11T12:12:36.999819+0000","last_became_active":"2024-05-11T12:12:34.876434+0000","last_became_peered":"2024-05-11T12:12:34.876434+0000","last_unstale":"2024-05-11T12:12:36.999819+0000","last_undegraded":"2024-05-11T12:12:36.999819+0000","last_fullsized":"2024-05-11T12:12:36.999819+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:24:43.843771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.997602+0000","last_change":"2024-05-11T12:12:35.899264+0000","last_active":"2024-05-11T12:12:36.997602+0000","last_peered":"2024-05-11T12:12:36.997602+0000","last_clean":"2024-05-11T12:12:36.997602+0000","last_became_active":"2024-05-11T12:12:35.898738+0000","last_became_peered":"2024-05-11T12:12:35.898738+0000","last_unstale":"2024-05-11T12:12:36.997602+0000","last_undegraded":"2024-05-11T12:12:36.997602+0000","last_fullsized":"2024-05-11T12:12:36.997602+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:04:29.795201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"52'8","reported_seq":22,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.894305+0000","last_change":"2024-05-11T12:12:34.855649+0000","last_active":"2024-05-11T12:12:36.894305+0000","last_peered":"2024-05-11T12:12:36.894305+0000","last_clean":"2024-05-11T12:12:36.894305+0000","last_became_active":"2024-05-11T12:12:34.854413+0000","last_became_peered":"2024-05-11T12:12:34.854413+0000","last_unstale":"2024-05-11T12:12:36.894305+0000","last_undegraded":"2024-05-11T12:12:36.894305+0000","last_fullsized":"2024-05-11T12:12:36.894305+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:39:10.345584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.894290+0000","last_change":"2024-05-11T12:12:35.890771+0000","last_active":"2024-05-11T12:12:36.894290+0000","last_peered":"2024-05-11T12:12:36.894290+0000","last_clean":"2024-05-11T12:12:36.894290+0000","last_became_active":"2024-05-11T12:12:35.890339+0000","last_became_peered":"2024-05-11T12:12:35.890339+0000","last_unstale":"2024-05-11T12:12:36.894290+0000","last_undegraded":"2024-05-11T12:12:36.894290+0000","last_fullsized":"2024-05-11T12:12:36.894290+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:18:23.487491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"52'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.997779+0000","last_change":"2024-05-11T12:12:34.874758+0000","last_active":"2024-05-11T12:12:36.997779+0000","last_peered":"2024-05-11T12:12:36.997779+0000","last_clean":"2024-05-11T12:12:36.997779+0000","last_became_active":"2024-05-11T12:12:34.872416+0000","last_became_peered":"2024-05-11T12:12:34.872416+0000","last_unstale":"2024-05-11T12:12:36.997779+0000","last_undegraded":"2024-05-11T12:12:36.997779+0000","last_fullsized":"2024-05-11T12:12:36.997779+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:29:52.188725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.895528+0000","last_change":"2024-05-11T12:12:35.915603+0000","last_active":"2024-05-11T12:12:36.895528+0000","last_peered":"2024-05-11T12:12:36.895528+0000","last_clean":"2024-05-11T12:12:36.895528+0000","last_became_active":"2024-05-11T12:12:35.915314+0000","last_became_peered":"2024-05-11T12:12:35.915314+0000","last_unstale":"2024-05-11T12:12:36.895528+0000","last_undegraded":"2024-05-11T12:12:36.895528+0000","last_fullsized":"2024-05-11T12:12:36.895528+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:29:53.202409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"52'2","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-11T12:12:35.896842+0000","last_change":"2024-05-11T12:12:34.868902+0000","last_active":"2024-05-11T12:12:35.896842+0000","last_peered":"2024-05-11T12:12:35.896842+0000","last_clean":"2024-05-11T12:12:35.896842+0000","last_became_active":"2024-05-11T12:12:34.868578+0000","last_became_peered":"2024-05-11T12:12:34.868578+0000","last_unstale":"2024-05-11T12:12:35.896842+0000","last_undegraded":"2024-05-11T12:12:35.896842+0000","last_fullsized":"2024-05-11T12:12:35.896842+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:41:00.444365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.999777+0000","last_change":"2024-05-11T12:12:35.918586+0000","last_active":"2024-05-11T12:12:36.999777+0000","last_peered":"2024-05-11T12:12:36.999777+0000","last_clean":"2024-05-11T12:12:36.999777+0000","last_became_active":"2024-05-11T12:12:35.918157+0000","last_became_peered":"2024-05-11T12:12:35.918157+0000","last_unstale":"2024-05-11T12:12:36.999777+0000","last_undegraded":"2024-05-11T12:12:36.999777+0000","last_fullsized":"2024-05-11T12:12:36.999777+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:24:44.857454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"52'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.998305+0000","last_change":"2024-05-11T12:12:34.877389+0000","last_active":"2024-05-11T12:12:36.998305+0000","last_peered":"2024-05-11T12:12:36.998305+0000","last_clean":"2024-05-11T12:12:36.998305+0000","last_became_active":"2024-05-11T12:12:34.876688+0000","last_became_peered":"2024-05-11T12:12:34.876688+0000","last_unstale":"2024-05-11T12:12:36.998305+0000","last_undegraded":"2024-05-11T12:12:36.998305+0000","last_fullsized":"2024-05-11T12:12:36.998305+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:28:53.318080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.896516+0000","last_change":"2024-05-11T12:12:35.895971+0000","last_active":"2024-05-11T12:12:36.896516+0000","last_peered":"2024-05-11T12:12:36.896516+0000","last_clean":"2024-05-11T12:12:36.896516+0000","last_became_active":"2024-05-11T12:12:35.895640+0000","last_became_peered":"2024-05-11T12:12:35.895640+0000","last_unstale":"2024-05-11T12:12:36.896516+0000","last_undegraded":"2024-05-11T12:12:36.896516+0000","last_fullsized":"2024-05-11T12:12:36.896516+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:15:29.274470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.998304+0000","last_change":"2024-05-11T12:12:34.881073+0000","last_active":"2024-05-11T12:12:36.998304+0000","last_peered":"2024-05-11T12:12:36.998304+0000","last_clean":"2024-05-11T12:12:36.998304+0000","last_became_active":"2024-05-11T12:12:34.880510+0000","last_became_peered":"2024-05-11T12:12:34.880510+0000","last_unstale":"2024-05-11T12:12:36.998304+0000","last_undegraded":"2024-05-11T12:12:36.998304+0000","last_fullsized":"2024-05-11T12:12:36.998304+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:07:15.197517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.897235+0000","last_change":"2024-05-11T12:12:35.904628+0000","last_active":"2024-05-11T12:12:36.897235+0000","last_peered":"2024-05-11T12:12:36.897235+0000","last_clean":"2024-05-11T12:12:36.897235+0000","last_became_active":"2024-05-11T12:12:35.904307+0000","last_became_peered":"2024-05-11T12:12:35.904307+0000","last_unstale":"2024-05-11T12:12:36.897235+0000","last_undegraded":"2024-05-11T12:12:36.897235+0000","last_fullsized":"2024-05-11T12:12:36.897235+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:32:18.738049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.896676+0000","last_change":"2024-05-11T12:12:34.876572+0000","last_active":"2024-05-11T12:12:36.896676+0000","last_peered":"2024-05-11T12:12:36.896676+0000","last_clean":"2024-05-11T12:12:36.896676+0000","last_became_active":"2024-05-11T12:12:34.875346+0000","last_became_peered":"2024-05-11T12:12:34.875346+0000","last_unstale":"2024-05-11T12:12:36.896676+0000","last_undegraded":"2024-05-11T12:12:36.896676+0000","last_fullsized":"2024-05-11T12:12:36.896676+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:32:49.107971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.997680+0000","last_change":"2024-05-11T12:12:35.906735+0000","last_active":"2024-05-11T12:12:36.997680+0000","last_peered":"2024-05-11T12:12:36.997680+0000","last_clean":"2024-05-11T12:12:36.997680+0000","last_became_active":"2024-05-11T12:12:35.906394+0000","last_became_peered":"2024-05-11T12:12:35.906394+0000","last_unstale":"2024-05-11T12:12:36.997680+0000","last_undegraded":"2024-05-11T12:12:36.997680+0000","last_fullsized":"2024-05-11T12:12:36.997680+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:05:33.189382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.894620+0000","last_change":"2024-05-11T12:12:34.849998+0000","last_active":"2024-05-11T12:12:36.894620+0000","last_peered":"2024-05-11T12:12:36.894620+0000","last_clean":"2024-05-11T12:12:36.894620+0000","last_became_active":"2024-05-11T12:12:34.849640+0000","last_became_peered":"2024-05-11T12:12:34.849640+0000","last_unstale":"2024-05-11T12:12:36.894620+0000","last_undegraded":"2024-05-11T12:12:36.894620+0000","last_fullsized":"2024-05-11T12:12:36.894620+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:59:22.439917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.997021+0000","last_change":"2024-05-11T12:12:35.911148+0000","last_active":"2024-05-11T12:12:36.997021+0000","last_peered":"2024-05-11T12:12:36.997021+0000","last_clean":"2024-05-11T12:12:36.997021+0000","last_became_active":"2024-05-11T12:12:35.909909+0000","last_became_peered":"2024-05-11T12:12:35.909909+0000","last_unstale":"2024-05-11T12:12:36.997021+0000","last_undegraded":"2024-05-11T12:12:36.997021+0000","last_fullsized":"2024-05-11T12:12:36.997021+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:28:07.216631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.995848+0000","last_change":"2024-05-11T12:12:34.873037+0000","last_active":"2024-05-11T12:12:36.995848+0000","last_peered":"2024-05-11T12:12:36.995848+0000","last_clean":"2024-05-11T12:12:36.995848+0000","last_became_active":"2024-05-11T12:12:34.870011+0000","last_became_peered":"2024-05-11T12:12:34.870011+0000","last_unstale":"2024-05-11T12:12:36.995848+0000","last_undegraded":"2024-05-11T12:12:36.995848+0000","last_fullsized":"2024-05-11T12:12:36.995848+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:41:25.012171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.896948+0000","last_change":"2024-05-11T12:12:35.893448+0000","last_active":"2024-05-11T12:12:36.896948+0000","last_peered":"2024-05-11T12:12:36.896948+0000","last_clean":"2024-05-11T12:12:36.896948+0000","last_became_active":"2024-05-11T12:12:35.893164+0000","last_became_peered":"2024-05-11T12:12:35.893164+0000","last_unstale":"2024-05-11T12:12:36.896948+0000","last_undegraded":"2024-05-11T12:12:36.896948+0000","last_fullsized":"2024-05-11T12:12:36.896948+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:57:27.618429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-11T12:12:35.872209+0000","last_change":"2024-05-11T12:12:34.860395+0000","last_active":"2024-05-11T12:12:35.872209+0000","last_peered":"2024-05-11T12:12:35.872209+0000","last_clean":"2024-05-11T12:12:35.872209+0000","last_became_active":"2024-05-11T12:12:34.860099+0000","last_became_peered":"2024-05-11T12:12:34.860099+0000","last_unstale":"2024-05-11T12:12:35.872209+0000","last_undegraded":"2024-05-11T12:12:35.872209+0000","last_fullsized":"2024-05-11T12:12:35.872209+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:13:59.633245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.896682+0000","last_change":"2024-05-11T12:12:35.893266+0000","last_active":"2024-05-11T12:12:36.896682+0000","last_peered":"2024-05-11T12:12:36.896682+0000","last_clean":"2024-05-11T12:12:36.896682+0000","last_became_active":"2024-05-11T12:12:35.891896+0000","last_became_peered":"2024-05-11T12:12:35.891896+0000","last_unstale":"2024-05-11T12:12:36.896682+0000","last_undegraded":"2024-05-11T12:12:36.896682+0000","last_fullsized":"2024-05-11T12:12:36.896682+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:11:03.801515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.998781+0000","last_change":"2024-05-11T12:12:34.882239+0000","last_active":"2024-05-11T12:12:36.998781+0000","last_peered":"2024-05-11T12:12:36.998781+0000","last_clean":"2024-05-11T12:12:36.998781+0000","last_became_active":"2024-05-11T12:12:34.881983+0000","last_became_peered":"2024-05-11T12:12:34.881983+0000","last_unstale":"2024-05-11T12:12:36.998781+0000","last_undegraded":"2024-05-11T12:12:36.998781+0000","last_fullsized":"2024-05-11T12:12:36.998781+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:26:35.169359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.896226+0000","last_change":"2024-05-11T12:12:35.894137+0000","last_active":"2024-05-11T12:12:36.896226+0000","last_peered":"2024-05-11T12:12:36.896226+0000","last_clean":"2024-05-11T12:12:36.896226+0000","last_became_active":"2024-05-11T12:12:35.893898+0000","last_became_peered":"2024-05-11T12:12:35.893898+0000","last_unstale":"2024-05-11T12:12:36.896226+0000","last_undegraded":"2024-05-11T12:12:36.896226+0000","last_fullsized":"2024-05-11T12:12:36.896226+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:04:52.423584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:37.001896+0000","last_change":"2024-05-11T12:12:34.877279+0000","last_active":"2024-05-11T12:12:37.001896+0000","last_peered":"2024-05-11T12:12:37.001896+0000","last_clean":"2024-05-11T12:12:37.001896+0000","last_became_active":"2024-05-11T12:12:34.876997+0000","last_became_peered":"2024-05-11T12:12:34.876997+0000","last_unstale":"2024-05-11T12:12:37.001896+0000","last_undegraded":"2024-05-11T12:12:37.001896+0000","last_fullsized":"2024-05-11T12:12:37.001896+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:23:49.239695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.895392+0000","last_change":"2024-05-11T12:12:35.901286+0000","last_active":"2024-05-11T12:12:36.895392+0000","last_peered":"2024-05-11T12:12:36.895392+0000","last_clean":"2024-05-11T12:12:36.895392+0000","last_became_active":"2024-05-11T12:12:35.901003+0000","last_became_peered":"2024-05-11T12:12:35.901003+0000","last_unstale":"2024-05-11T12:12:36.895392+0000","last_undegraded":"2024-05-11T12:12:36.895392+0000","last_fullsized":"2024-05-11T12:12:36.895392+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:45:25.587344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.996211+0000","last_change":"2024-05-11T12:12:34.881759+0000","last_active":"2024-05-11T12:12:36.996211+0000","last_peered":"2024-05-11T12:12:36.996211+0000","last_clean":"2024-05-11T12:12:36.996211+0000","last_became_active":"2024-05-11T12:12:34.878053+0000","last_became_peered":"2024-05-11T12:12:34.878053+0000","last_unstale":"2024-05-11T12:12:36.996211+0000","last_undegraded":"2024-05-11T12:12:36.996211+0000","last_fullsized":"2024-05-11T12:12:36.996211+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:21:12.301563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.996048+0000","last_change":"2024-05-11T12:12:35.897607+0000","last_active":"2024-05-11T12:12:36.996048+0000","last_peered":"2024-05-11T12:12:36.996048+0000","last_clean":"2024-05-11T12:12:36.996048+0000","last_became_active":"2024-05-11T12:12:35.897219+0000","last_became_peered":"2024-05-11T12:12:35.897219+0000","last_unstale":"2024-05-11T12:12:36.996048+0000","last_undegraded":"2024-05-11T12:12:36.996048+0000","last_fullsized":"2024-05-11T12:12:36.996048+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:32:18.738049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.895638+0000","last_change":"2024-05-11T12:12:34.860288+0000","last_active":"2024-05-11T12:12:36.895638+0000","last_peered":"2024-05-11T12:12:36.895638+0000","last_clean":"2024-05-11T12:12:36.895638+0000","last_became_active":"2024-05-11T12:12:34.858534+0000","last_became_peered":"2024-05-11T12:12:34.858534+0000","last_unstale":"2024-05-11T12:12:36.895638+0000","last_undegraded":"2024-05-11T12:12:36.895638+0000","last_fullsized":"2024-05-11T12:12:36.895638+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:55:12.923220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.897191+0000","last_change":"2024-05-11T12:12:35.900991+0000","last_active":"2024-05-11T12:12:36.897191+0000","last_peered":"2024-05-11T12:12:36.897191+0000","last_clean":"2024-05-11T12:12:36.897191+0000","last_became_active":"2024-05-11T12:12:35.899178+0000","last_became_peered":"2024-05-11T12:12:35.899178+0000","last_unstale":"2024-05-11T12:12:36.897191+0000","last_undegraded":"2024-05-11T12:12:36.897191+0000","last_fullsized":"2024-05-11T12:12:36.897191+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:03:54.906678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.894866+0000","last_change":"2024-05-11T12:12:34.878691+0000","last_active":"2024-05-11T12:12:36.894866+0000","last_peered":"2024-05-11T12:12:36.894866+0000","last_clean":"2024-05-11T12:12:36.894866+0000","last_became_active":"2024-05-11T12:12:34.878392+0000","last_became_peered":"2024-05-11T12:12:34.878392+0000","last_unstale":"2024-05-11T12:12:36.894866+0000","last_undegraded":"2024-05-11T12:12:36.894866+0000","last_fullsized":"2024-05-11T12:12:36.894866+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:08:19.875038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.996126+0000","last_change":"2024-05-11T12:12:35.904540+0000","last_active":"2024-05-11T12:12:36.996126+0000","last_peered":"2024-05-11T12:12:36.996126+0000","last_clean":"2024-05-11T12:12:36.996126+0000","last_became_active":"2024-05-11T12:12:35.904178+0000","last_became_peered":"2024-05-11T12:12:35.904178+0000","last_unstale":"2024-05-11T12:12:36.996126+0000","last_undegraded":"2024-05-11T12:12:36.996126+0000","last_fullsized":"2024-05-11T12:12:36.996126+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:20:14.406725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:37.000180+0000","last_change":"2024-05-11T12:12:34.877303+0000","last_active":"2024-05-11T12:12:37.000180+0000","last_peered":"2024-05-11T12:12:37.000180+0000","last_clean":"2024-05-11T12:12:37.000180+0000","last_became_active":"2024-05-11T12:12:34.876968+0000","last_became_peered":"2024-05-11T12:12:34.876968+0000","last_unstale":"2024-05-11T12:12:37.000180+0000","last_undegraded":"2024-05-11T12:12:37.000180+0000","last_fullsized":"2024-05-11T12:12:37.000180+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:32:49.107971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.895543+0000","last_change":"2024-05-11T12:12:35.892629+0000","last_active":"2024-05-11T12:12:36.895543+0000","last_peered":"2024-05-11T12:12:36.895543+0000","last_clean":"2024-05-11T12:12:36.895543+0000","last_became_active":"2024-05-11T12:12:35.892244+0000","last_became_peered":"2024-05-11T12:12:35.892244+0000","last_unstale":"2024-05-11T12:12:36.895543+0000","last_undegraded":"2024-05-11T12:12:36.895543+0000","last_fullsized":"2024-05-11T12:12:36.895543+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:04:38.991505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.897066+0000","last_change":"2024-05-11T12:12:34.873773+0000","last_active":"2024-05-11T12:12:36.897066+0000","last_peered":"2024-05-11T12:12:36.897066+0000","last_clean":"2024-05-11T12:12:36.897066+0000","last_became_active":"2024-05-11T12:12:34.872888+0000","last_became_peered":"2024-05-11T12:12:34.872888+0000","last_unstale":"2024-05-11T12:12:36.897066+0000","last_undegraded":"2024-05-11T12:12:36.897066+0000","last_fullsized":"2024-05-11T12:12:36.897066+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:22:22.522080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.997388+0000","last_change":"2024-05-11T12:12:35.903755+0000","last_active":"2024-05-11T12:12:36.997388+0000","last_peered":"2024-05-11T12:12:36.997388+0000","last_clean":"2024-05-11T12:12:36.997388+0000","last_became_active":"2024-05-11T12:12:35.902255+0000","last_became_peered":"2024-05-11T12:12:35.902255+0000","last_unstale":"2024-05-11T12:12:36.997388+0000","last_undegraded":"2024-05-11T12:12:36.997388+0000","last_fullsized":"2024-05-11T12:12:36.997388+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:34.831716+0000","last_clean_scrub_stamp":"2024-05-11T12:12:34.831716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:29:53.202409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.997113+0000","last_change":"2024-05-11T12:12:34.881543+0000","last_active":"2024-05-11T12:12:36.997113+0000","last_peered":"2024-05-11T12:12:36.997113+0000","last_clean":"2024-05-11T12:12:36.997113+0000","last_became_active":"2024-05-11T12:12:34.881274+0000","last_became_peered":"2024-05-11T12:12:34.881274+0000","last_unstale":"2024-05-11T12:12:36.997113+0000","last_undegraded":"2024-05-11T12:12:36.997113+0000","last_fullsized":"2024-05-11T12:12:36.997113+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:12:33.818033+0000","last_clean_scrub_stamp":"2024-05-11T12:12:33.818033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:05:32.175699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":165,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-05-11T12:12:36.999033+0000","last_change":"2024-05-11T12:10:35.066673+0000","last_active":"2024-05-11T12:12:36.999033+0000","last_peered":"2024-05-11T12:12:36.999033+0000","last_clean":"2024-05-11T12:12:36.999033+0000","last_became_active":"2024-05-11T12:10:35.065942+0000","last_became_peered":"2024-05-11T12:10:35.065942+0000","last_unstale":"2024-05-11T12:12:36.999033+0000","last_undegraded":"2024-05-11T12:12:36.999033+0000","last_fullsized":"2024-05-11T12:12:36.999033+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_clean_scrub_stamp":"2024-05-11T12:10:33.601136+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:12:02.133182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476764,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28220,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343620,"statfs":{"total":96636764160,"available":96607866880,"internally_reserved":0,"allocated":1220608,"data_stored":862648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476765,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":626688,"data_stored":272808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476764,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27624,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344216,"statfs":{"total":96636764160,"available":96608477184,"internally_reserved":0,"allocated":618496,"data_stored":272156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476766,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27628,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344212,"statfs":{"total":96636764160,"available":96608473088,"internally_reserved":0,"allocated":622592,"data_stored":272738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509469,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":634880,"data_stored":273790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509470,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":626688,"data_stored":272316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509469,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28220,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343620,"statfs":{"total":96636764160,"available":96607866880,"internally_reserved":0,"allocated":1220608,"data_stored":862580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542173,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27624,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344216,"statfs":{"total":96636764160,"available":96608477184,"internally_reserved":0,"allocated":618496,"data_stored":267076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T12:12:40.831 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-11T12:12:40.831 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-11T12:12:41.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.078+0000 7f664d000640 1 Processor -- start 2024-05-11T12:12:41.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.079+0000 7f664d000640 1 -- start start 2024-05-11T12:12:41.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.080+0000 7f664d000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f66480b84b0 0x7f664816a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:41.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.080+0000 7f664d000640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f664806dec0 con 0x7f6648162a40 2024-05-11T12:12:41.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.080+0000 7f664d000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66480d2dc0 con 0x7f66480b3fb0 2024-05-11T12:12:41.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.080+0000 7f664d000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f66480d2b70 con 0x7f66480b84b0 2024-05-11T12:12:41.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.080+0000 7f6647400640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f6648162a40 0x7f66480d38a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:58116/0 (socket says 172.21.15.167:58116) 2024-05-11T12:12:41.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.080+0000 7f6647400640 1 -- 172.21.15.167:0/2345939580 learned_addr learned my addr 172.21.15.167:0/2345939580 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:41.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.081+0000 7f6646000640 1 -- 172.21.15.167:0/2345939580 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3902395526 0 0) 0x7f664806dec0 con 0x7f6648162a40 2024-05-11T12:12:41.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.081+0000 7f6646000640 1 -- 172.21.15.167:0/2345939580 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f662c003520 con 0x7f6648162a40 2024-05-11T12:12:41.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.081+0000 7f6647e00640 1 --2- 172.21.15.167:0/2345939580 >> v2:172.21.15.204:3300/0 conn(0x7f66480b84b0 0x7f664816a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:41.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.081+0000 7f6646000640 1 -- 172.21.15.167:0/2345939580 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2592820836 0 0) 0x7f662c003520 con 0x7f6648162a40 2024-05-11T12:12:41.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.081+0000 7f6646000640 1 -- 172.21.15.167:0/2345939580 >> v2:172.21.15.204:3300/0 conn(0x7f66480b84b0 msgr2=0x7f664816a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:41.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.081+0000 7f6646000640 1 --2- 172.21.15.167:0/2345939580 >> v2:172.21.15.204:3300/0 conn(0x7f66480b84b0 0x7f664816a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:41.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.081+0000 7f6646000640 1 -- 172.21.15.167:0/2345939580 >> v1:172.21.15.204:6789/0 conn(0x7f66480b3fb0 legacy=0x7f66480d3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:41.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.081+0000 7f6646000640 1 -- 172.21.15.167:0/2345939580 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66480d30a0 con 0x7f6648162a40 2024-05-11T12:12:41.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.081+0000 7f6646000640 1 -- 172.21.15.167:0/2345939580 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f6638001a70 con 0x7f6648162a40 2024-05-11T12:12:41.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.081+0000 7f6646000640 1 -- 172.21.15.167:0/2345939580 >> v1:172.21.15.167:6789/0 conn(0x7f6648162a40 legacy=0x7f66480d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:41.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.081+0000 7f6646000640 1 --2- 172.21.15.167:0/2345939580 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f662c003ad0 0x7f662c003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:41.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.081+0000 7f6646000640 1 --2- 172.21.15.167:0/2345939580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f662c004800 0x7f662c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:41.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.082+0000 7f6646000640 1 --2- 172.21.15.167:0/2345939580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f662c0052e0 0x7f662c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:41.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.082+0000 7f6646000640 1 -- 172.21.15.167:0/2345939580 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f66480d2b70 con 0x7f662c0052e0 2024-05-11T12:12:41.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.082+0000 7f6646000640 1 -- 172.21.15.167:0/2345939580 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f662c00c230 con 0x7f662c004800 2024-05-11T12:12:41.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.082+0000 7f6646000640 1 -- 172.21.15.167:0/2345939580 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f662c00c430 con 0x7f662c003ad0 2024-05-11T12:12:41.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.082+0000 7f6647e00640 1 --2- 172.21.15.167:0/2345939580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f662c0052e0 0x7f662c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:41.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.082+0000 7f6647400640 1 --2- 172.21.15.167:0/2345939580 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f662c003ad0 0x7f662c003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:41.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.082+0000 7f6646a00640 1 --2- 172.21.15.167:0/2345939580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f662c004800 0x7f662c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:41.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.082+0000 7f6647e00640 1 -- 172.21.15.167:0/2345939580 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f662c003ad0 msgr2=0x7f662c003ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:41.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.082+0000 7f6647e00640 1 --2- 172.21.15.167:0/2345939580 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f662c003ad0 0x7f662c003ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:41.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.082+0000 7f6647e00640 1 -- 172.21.15.167:0/2345939580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f662c004800 msgr2=0x7f662c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:41.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.082+0000 7f6647e00640 1 --2- 172.21.15.167:0/2345939580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f662c004800 0x7f662c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:41.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.082+0000 7f6647e00640 1 -- 172.21.15.167:0/2345939580 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f662c003520 con 0x7f662c0052e0 2024-05-11T12:12:41.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.082+0000 7f6646a00640 1 --2- 172.21.15.167:0/2345939580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f662c004800 0x7f662c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:41.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.083+0000 7f6647e00640 1 -- 172.21.15.167:0/2345939580 reap_dead start 2024-05-11T12:12:41.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.083+0000 7f6647e00640 1 --2- 172.21.15.167:0/2345939580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f662c0052e0 0x7f662c00b9c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f664000ba00 tx=0x7f662c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=267414f204468eb2 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:41.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.084+0000 7f6646000640 1 -- 172.21.15.167:0/2345939580 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6640030070 con 0x7f662c0052e0 2024-05-11T12:12:41.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.084+0000 7f6646000640 1 -- 172.21.15.167:0/2345939580 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f664002ade0 con 0x7f662c0052e0 2024-05-11T12:12:41.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.084+0000 7f6646000640 1 -- 172.21.15.167:0/2345939580 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664000b600 con 0x7f662c0052e0 2024-05-11T12:12:41.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.084+0000 7f664d000640 1 -- 172.21.15.167:0/2345939580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f662c0052e0 msgr2=0x7f662c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:41.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.084+0000 7f664d000640 1 --2- 172.21.15.167:0/2345939580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f662c0052e0 0x7f662c00b9c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f664000ba00 tx=0x7f662c005830 comp rx=0 tx=0).stop 2024-05-11T12:12:41.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.084+0000 7f664d000640 1 -- 172.21.15.167:0/2345939580 shutdown_connections 2024-05-11T12:12:41.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.084+0000 7f664d000640 1 --2- 172.21.15.167:0/2345939580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f662c0052e0 0x7f662c00b9c0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:41.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.084+0000 7f664d000640 1 -- 172.21.15.167:0/2345939580 >> 172.21.15.167:0/2345939580 conn(0x7f66480d1c00 msgr2=0x7f66480d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:41.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.085+0000 7f664d000640 1 -- 172.21.15.167:0/2345939580 shutdown_connections 2024-05-11T12:12:41.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.085+0000 7f664d000640 1 -- 172.21.15.167:0/2345939580 wait complete. 2024-05-11T12:12:41.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.086+0000 7f664d000640 1 Processor -- start 2024-05-11T12:12:41.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.086+0000 7f664d000640 1 -- start start 2024-05-11T12:12:41.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.087+0000 7f664d000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f662c0052e0 0x7f664814a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:41.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.087+0000 7f664d000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66480cb2f0 0x7f664815d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:41.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.087+0000 7f664d000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66480cb750 0x7f664815dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:41.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.087+0000 7f664d000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f66480d38a0 con 0x7f66480cb750 2024-05-11T12:12:41.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.087+0000 7f664d000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f66480b3c60 con 0x7f662c0052e0 2024-05-11T12:12:41.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.087+0000 7f664d000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f66480d3a20 con 0x7f66480cb2f0 2024-05-11T12:12:41.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.088+0000 7f6647e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66480cb750 0x7f664815dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:41.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.088+0000 7f6647e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66480cb750 0x7f664815dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45614/0 (socket says 172.21.15.167:45614) 2024-05-11T12:12:41.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.088+0000 7f6647e00640 1 -- 172.21.15.167:0/1899067870 learned_addr learned my addr 172.21.15.167:0/1899067870 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:41.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.088+0000 7f6647400640 1 --2- 172.21.15.167:0/1899067870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f662c0052e0 0x7f664814a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:41.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.088+0000 7f6646a00640 1 --2- 172.21.15.167:0/1899067870 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66480cb2f0 0x7f664815d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:41.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.088+0000 7f6647e00640 1 -- 172.21.15.167:0/1899067870 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66480cb2f0 msgr2=0x7f664815d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:41.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.088+0000 7f6647e00640 1 --2- 172.21.15.167:0/1899067870 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66480cb2f0 0x7f664815d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:41.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.088+0000 7f6647e00640 1 -- 172.21.15.167:0/1899067870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f662c0052e0 msgr2=0x7f664814a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:41.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.088+0000 7f6647e00640 1 --2- 172.21.15.167:0/1899067870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f662c0052e0 0x7f664814a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:41.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.088+0000 7f6647e00640 1 -- 172.21.15.167:0/1899067870 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f664815e350 con 0x7f66480cb750 2024-05-11T12:12:41.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.089+0000 7f6647e00640 1 --2- 172.21.15.167:0/1899067870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66480cb750 0x7f664815dac0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f664002a8d0 tx=0x7f66400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:41.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.089+0000 7f663f400640 1 -- 172.21.15.167:0/1899067870 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664000b600 con 0x7f66480cb750 2024-05-11T12:12:41.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.089+0000 7f663f400640 1 -- 172.21.15.167:0/1899067870 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6640042070 con 0x7f66480cb750 2024-05-11T12:12:41.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.089+0000 7f663f400640 1 -- 172.21.15.167:0/1899067870 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66400044a0 con 0x7f66480cb750 2024-05-11T12:12:41.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.089+0000 7f664d000640 1 -- 172.21.15.167:0/1899067870 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f664815e630 con 0x7f66480cb750 2024-05-11T12:12:41.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.089+0000 7f664d000640 1 -- 172.21.15.167:0/1899067870 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f664815ebf0 con 0x7f66480cb750 2024-05-11T12:12:41.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.090+0000 7f664d000640 1 -- 172.21.15.167:0/1899067870 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66480694b0 con 0x7f66480cb750 2024-05-11T12:12:41.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.091+0000 7f663f400640 1 -- 172.21.15.167:0/1899067870 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6640047460 con 0x7f66480cb750 2024-05-11T12:12:41.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.092+0000 7f663f400640 1 --2- 172.21.15.167:0/1899067870 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f66240762d0 0x7f6624078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:41.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.092+0000 7f663f400640 1 -- 172.21.15.167:0/1899067870 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f66400b7570 con 0x7f66480cb750 2024-05-11T12:12:41.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.093+0000 7f6647400640 1 --2- 172.21.15.167:0/1899067870 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f66240762d0 0x7f6624078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:41.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.093+0000 7f6647400640 1 --2- 172.21.15.167:0/1899067870 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f66240762d0 0x7f6624078790 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6638001af0 tx=0x7f6638001290 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:41.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.097+0000 7f663f400640 1 -- 172.21.15.167:0/1899067870 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6640003e20 con 0x7f66480cb750 2024-05-11T12:12:41.258 INFO:tasks.ceph.mds.a.smithi167.stderr:2024-05-11T12:12:41.259+0000 7fcc07600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-11T12:12:41.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.381+0000 7f664d000640 1 -- 172.21.15.167:0/1899067870 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f664806d2c0 con 0x7f66480cb750 2024-05-11T12:12:41.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.385+0000 7f663f400640 1 -- 172.21.15.167:0/1899067870 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) ==== 190+0+147 (secure 0 0 0) 0x7f6640080c10 con 0x7f66480cb750 2024-05-11T12:12:41.385 INFO:teuthology.orchestra.run.smithi167.stdout:[client.0] 2024-05-11T12:12:41.385 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQAFYD9mLK/lEhAAyERTYMl250KkBvIrAXWYSw== 2024-05-11T12:12:41.385 INFO:teuthology.orchestra.run.smithi167.stdout: caps mds = "allow" 2024-05-11T12:12:41.385 INFO:teuthology.orchestra.run.smithi167.stdout: caps mon = "allow r" 2024-05-11T12:12:41.385 INFO:teuthology.orchestra.run.smithi167.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-11T12:12:41.386 INFO:teuthology.orchestra.run.smithi167.stderr:updated caps for client.0 2024-05-11T12:12:41.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.388+0000 7f664d000640 1 -- 172.21.15.167:0/1899067870 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f66240762d0 msgr2=0x7f6624078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:41.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.388+0000 7f664d000640 1 --2- 172.21.15.167:0/1899067870 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f66240762d0 0x7f6624078790 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6638001af0 tx=0x7f6638001290 comp rx=0 tx=0).stop 2024-05-11T12:12:41.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.388+0000 7f664d000640 1 -- 172.21.15.167:0/1899067870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66480cb750 msgr2=0x7f664815dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:41.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.388+0000 7f664d000640 1 --2- 172.21.15.167:0/1899067870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66480cb750 0x7f664815dac0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f664002a8d0 tx=0x7f66400046c0 comp rx=0 tx=0).stop 2024-05-11T12:12:41.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.388+0000 7f664d000640 1 -- 172.21.15.167:0/1899067870 shutdown_connections 2024-05-11T12:12:41.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.388+0000 7f664d000640 1 --2- 172.21.15.167:0/1899067870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66480cb750 0x7f664815dac0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:41.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.389+0000 7f664d000640 1 --2- 172.21.15.167:0/1899067870 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66480cb2f0 0x7f664815d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:41.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.389+0000 7f664d000640 1 --2- 172.21.15.167:0/1899067870 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f66240762d0 0x7f6624078790 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:41.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.389+0000 7f664d000640 1 --2- 172.21.15.167:0/1899067870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f662c0052e0 0x7f664814a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:41.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.389+0000 7f664d000640 1 -- 172.21.15.167:0/1899067870 >> 172.21.15.167:0/1899067870 conn(0x7f66480d1c00 msgr2=0x7f66480b8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:41.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.389+0000 7f664d000640 1 -- 172.21.15.167:0/1899067870 shutdown_connections 2024-05-11T12:12:41.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.389+0000 7f664d000640 1 -- 172.21.15.167:0/1899067870 wait complete. 2024-05-11T12:12:41.406 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:12:41.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.601+0000 7f1829800640 1 Processor -- start 2024-05-11T12:12:41.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.601+0000 7f1829800640 1 -- start start 2024-05-11T12:12:41.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.603+0000 7f1829800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f182415cf70 0x7f18240722f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:41.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.603+0000 7f1829800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f18240cf100 0x7f1824070000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:41.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.603+0000 7f1829800640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18240d23e0 con 0x7f1824070540 2024-05-11T12:12:41.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.603+0000 7f1829800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f1824072af0 con 0x7f18240cf100 2024-05-11T12:12:41.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.603+0000 7f1829800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f18240d1f50 con 0x7f182415cf70 2024-05-11T12:12:41.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.603+0000 7f1828e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f1824070540 0x7f18240cda50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:58126/0 (socket says 172.21.15.167:58126) 2024-05-11T12:12:41.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.604+0000 7f1828e00640 1 -- 172.21.15.167:0/1587363035 learned_addr learned my addr 172.21.15.167:0/1587363035 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:41.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.604+0000 7f1823e00640 1 --2- 172.21.15.167:0/1587363035 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f182415cf70 0x7f18240722f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:41.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.604+0000 7f181be00640 1 --2- 172.21.15.167:0/1587363035 >> v2:172.21.15.204:3300/0 conn(0x7f18240cf100 0x7f1824070000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:41.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.604+0000 7f1823400640 1 -- 172.21.15.167:0/1587363035 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1032724469 0 0) 0x7f18240d23e0 con 0x7f1824070540 2024-05-11T12:12:41.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.604+0000 7f1823400640 1 -- 172.21.15.167:0/1587363035 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1808003520 con 0x7f1824070540 2024-05-11T12:12:41.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.604+0000 7f1823400640 1 -- 172.21.15.167:0/1587363035 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1173109085 0 0) 0x7f1808003520 con 0x7f1824070540 2024-05-11T12:12:41.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.604+0000 7f1823400640 1 -- 172.21.15.167:0/1587363035 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f182415cf70 msgr2=0x7f18240722f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:41.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.604+0000 7f1823400640 1 --2- 172.21.15.167:0/1587363035 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f182415cf70 0x7f18240722f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:41.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.604+0000 7f1823400640 1 -- 172.21.15.167:0/1587363035 >> v2:172.21.15.204:3300/0 conn(0x7f18240cf100 msgr2=0x7f1824070000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:41.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.604+0000 7f1823400640 1 --2- 172.21.15.167:0/1587363035 >> v2:172.21.15.204:3300/0 conn(0x7f18240cf100 0x7f1824070000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:41.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.604+0000 7f1823400640 1 -- 172.21.15.167:0/1587363035 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1824072830 con 0x7f1824070540 2024-05-11T12:12:41.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.605+0000 7f1823400640 1 -- 172.21.15.167:0/1587363035 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f181c001a70 con 0x7f1824070540 2024-05-11T12:12:41.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.605+0000 7f1823400640 1 -- 172.21.15.167:0/1587363035 >> v1:172.21.15.167:6789/0 conn(0x7f1824070540 legacy=0x7f18240cda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:41.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.605+0000 7f1823400640 1 --2- 172.21.15.167:0/1587363035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1808003a50 0x7f1808003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:41.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.605+0000 7f1823400640 1 --2- 172.21.15.167:0/1587363035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1808004800 0x7f1808004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:41.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.605+0000 7f1823400640 1 --2- 172.21.15.167:0/1587363035 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f182415cf70 0x7f18080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:41.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.605+0000 7f1823400640 1 -- 172.21.15.167:0/1587363035 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1824072af0 con 0x7f1808003a50 2024-05-11T12:12:41.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.605+0000 7f1823400640 1 -- 172.21.15.167:0/1587363035 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f18240d1f50 con 0x7f1808004800 2024-05-11T12:12:41.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.605+0000 7f1823400640 1 -- 172.21.15.167:0/1587363035 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f180800bb40 con 0x7f182415cf70 2024-05-11T12:12:41.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.605+0000 7f1823e00640 1 --2- 172.21.15.167:0/1587363035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1808003a50 0x7f1808003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:41.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.605+0000 7f1823e00640 1 -- 172.21.15.167:0/1587363035 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f182415cf70 msgr2=0x7f18080051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:41.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.605+0000 7f1823e00640 1 --2- 172.21.15.167:0/1587363035 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f182415cf70 0x7f18080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:41.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.605+0000 7f1823e00640 1 -- 172.21.15.167:0/1587363035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1808004800 msgr2=0x7f1808004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:41.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.605+0000 7f181be00640 1 --2- 172.21.15.167:0/1587363035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1808004800 0x7f1808004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:41.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.605+0000 7f1823e00640 1 --2- 172.21.15.167:0/1587363035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1808004800 0x7f1808004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:41.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.605+0000 7f1823e00640 1 -- 172.21.15.167:0/1587363035 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1808003520 con 0x7f1808003a50 2024-05-11T12:12:41.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.606+0000 7f1823e00640 1 --2- 172.21.15.167:0/1587363035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1808003a50 0x7f1808003e70 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1814009ac0 tx=0x7f1808005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b5b335813a58bfc server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:41.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.607+0000 7f1823400640 1 -- 172.21.15.167:0/1587363035 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1814036070 con 0x7f1808003a50 2024-05-11T12:12:41.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.607+0000 7f1823400640 1 -- 172.21.15.167:0/1587363035 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1814031400 con 0x7f1808003a50 2024-05-11T12:12:41.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.607+0000 7f1823400640 1 -- 172.21.15.167:0/1587363035 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1814009690 con 0x7f1808003a50 2024-05-11T12:12:41.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.607+0000 7f1829800640 1 -- 172.21.15.167:0/1587363035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1808003a50 msgr2=0x7f1808003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:41.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.607+0000 7f1829800640 1 --2- 172.21.15.167:0/1587363035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1808003a50 0x7f1808003e70 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1814009ac0 tx=0x7f1808005830 comp rx=0 tx=0).stop 2024-05-11T12:12:41.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.607+0000 7f1828e00640 1 -- 172.21.15.167:0/1587363035 reap_dead start 2024-05-11T12:12:41.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.607+0000 7f1829800640 1 -- 172.21.15.167:0/1587363035 shutdown_connections 2024-05-11T12:12:41.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.607+0000 7f1829800640 1 -- 172.21.15.167:0/1587363035 >> 172.21.15.167:0/1587363035 conn(0x7f18240cffc0 msgr2=0x7f18240d2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:41.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.607+0000 7f1829800640 1 -- 172.21.15.167:0/1587363035 shutdown_connections 2024-05-11T12:12:41.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.607+0000 7f1829800640 1 -- 172.21.15.167:0/1587363035 wait complete. 2024-05-11T12:12:41.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.609+0000 7f1829800640 1 Processor -- start 2024-05-11T12:12:41.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.609+0000 7f1829800640 1 -- start start 2024-05-11T12:12:41.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.610+0000 7f1829800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1808003a50 0x7f18241527c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:41.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.610+0000 7f1829800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18241f8740 0x7f18240af590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:41.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.610+0000 7f1829800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18240afad0 0x7f18240b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:41.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.610+0000 7f1829800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f18240d1900 con 0x7f18241f8740 2024-05-11T12:12:41.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.610+0000 7f1829800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f18240d1620 con 0x7f18240afad0 2024-05-11T12:12:41.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.610+0000 7f1829800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f18240d1be0 con 0x7f1808003a50 2024-05-11T12:12:41.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.610+0000 7f181be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18241f8740 0x7f18240af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:41.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.610+0000 7f181be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18241f8740 0x7f18240af590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45628/0 (socket says 172.21.15.167:45628) 2024-05-11T12:12:41.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.610+0000 7f181be00640 1 -- 172.21.15.167:0/2651809113 learned_addr learned my addr 172.21.15.167:0/2651809113 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:41.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.611+0000 7f1823e00640 1 --2- 172.21.15.167:0/2651809113 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1808003a50 0x7f18241527c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:41.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.611+0000 7f181be00640 1 -- 172.21.15.167:0/2651809113 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1808003a50 msgr2=0x7f18241527c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:41.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.611+0000 7f181be00640 1 --2- 172.21.15.167:0/2651809113 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1808003a50 0x7f18241527c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:41.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.611+0000 7f1828e00640 1 --2- 172.21.15.167:0/2651809113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18240afad0 0x7f18240b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:41.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.611+0000 7f181be00640 1 -- 172.21.15.167:0/2651809113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18240afad0 msgr2=0x7f18240b49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:41.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.611+0000 7f181be00640 1 --2- 172.21.15.167:0/2651809113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18240afad0 0x7f18240b49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:41.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.611+0000 7f181be00640 1 -- 172.21.15.167:0/2651809113 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1824152d00 con 0x7f18241f8740 2024-05-11T12:12:41.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.611+0000 7f1828e00640 1 --2- 172.21.15.167:0/2651809113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18240afad0 0x7f18240b49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:41.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.611+0000 7f181be00640 1 --2- 172.21.15.167:0/2651809113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18241f8740 0x7f18240af590 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f180c010890 tx=0x7f180c010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:41.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.611+0000 7f1820c00640 1 -- 172.21.15.167:0/2651809113 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180c00f040 con 0x7f18241f8740 2024-05-11T12:12:41.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.611+0000 7f1820c00640 1 -- 172.21.15.167:0/2651809113 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f180c00eb00 con 0x7f18241f8740 2024-05-11T12:12:41.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.611+0000 7f1820c00640 1 -- 172.21.15.167:0/2651809113 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180c01c400 con 0x7f18241f8740 2024-05-11T12:12:41.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.612+0000 7f1829800640 1 -- 172.21.15.167:0/2651809113 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18241f9170 con 0x7f18241f8740 2024-05-11T12:12:41.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.612+0000 7f1829800640 1 -- 172.21.15.167:0/2651809113 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18240ca180 con 0x7f18241f8740 2024-05-11T12:12:41.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.613+0000 7f1829800640 1 -- 172.21.15.167:0/2651809113 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18241f9690 con 0x7f18241f8740 2024-05-11T12:12:41.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.614+0000 7f1820c00640 1 -- 172.21.15.167:0/2651809113 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f180c01c5a0 con 0x7f18241f8740 2024-05-11T12:12:41.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.615+0000 7f1820c00640 1 --2- 172.21.15.167:0/2651809113 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f18000762d0 0x7f1800078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:41.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.615+0000 7f1820c00640 1 -- 172.21.15.167:0/2651809113 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f180c097cb0 con 0x7f18241f8740 2024-05-11T12:12:41.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.615+0000 7f1823e00640 1 --2- 172.21.15.167:0/2651809113 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f18000762d0 0x7f1800078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:41.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.616+0000 7f1823e00640 1 --2- 172.21.15.167:0/2651809113 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f18000762d0 0x7f1800078790 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1814033000 tx=0x7f1814008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:41.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.619+0000 7f1820c00640 1 -- 172.21.15.167:0/2651809113 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f18241f9690 con 0x7f18241f8740 2024-05-11T12:12:41.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.868+0000 7f1829800640 1 -- 172.21.15.167:0/2651809113 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1824069bc0 con 0x7f18241f8740 2024-05-11T12:12:41.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.869+0000 7f1820c00640 1 -- 172.21.15.167:0/2651809113 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5193 (secure 0 0 0) 0x7f180c018030 con 0x7f18241f8740 2024-05-11T12:12:41.869 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:12:41.869 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4634,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2640837270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":2640837270},{"type":"v1","addr":"172.21.15.204:6835","nonce":2640837270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4807,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3516271493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3516271493},{"type":"v1","addr":"172.21.15.204:6834","nonce":3516271493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4816,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2070289869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2070289869},{"type":"v1","addr":"172.21.15.167:6837","nonce":2070289869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:12:35.314711+0000","modified":"2024-05-11T12:12:38.382446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6835/4016299199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4016299199},{"type":"v1","addr":"172.21.15.167:6835","nonce":4016299199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5310,"qdb_cluster":[5310]},"id":3}]} 2024-05-11T12:12:41.869 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 26 2024-05-11T12:12:41.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.872+0000 7f1829800640 1 -- 172.21.15.167:0/2651809113 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f18000762d0 msgr2=0x7f1800078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:41.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.872+0000 7f1829800640 1 --2- 172.21.15.167:0/2651809113 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f18000762d0 0x7f1800078790 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1814033000 tx=0x7f1814008030 comp rx=0 tx=0).stop 2024-05-11T12:12:41.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.872+0000 7f1829800640 1 -- 172.21.15.167:0/2651809113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18241f8740 msgr2=0x7f18240af590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:41.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.872+0000 7f1829800640 1 --2- 172.21.15.167:0/2651809113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18241f8740 0x7f18240af590 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f180c010890 tx=0x7f180c010d50 comp rx=0 tx=0).stop 2024-05-11T12:12:41.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.872+0000 7f1829800640 1 -- 172.21.15.167:0/2651809113 shutdown_connections 2024-05-11T12:12:41.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.872+0000 7f1829800640 1 --2- 172.21.15.167:0/2651809113 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f18000762d0 0x7f1800078790 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:41.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.872+0000 7f1829800640 1 --2- 172.21.15.167:0/2651809113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18240afad0 0x7f18240b49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:41.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.872+0000 7f1829800640 1 --2- 172.21.15.167:0/2651809113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18241f8740 0x7f18240af590 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:41.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.872+0000 7f1829800640 1 --2- 172.21.15.167:0/2651809113 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1808003a50 0x7f18241527c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:41.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.873+0000 7f1829800640 1 -- 172.21.15.167:0/2651809113 >> 172.21.15.167:0/2651809113 conn(0x7f18240cffc0 msgr2=0x7f182415acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:41.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.873+0000 7f1829800640 1 -- 172.21.15.167:0/2651809113 shutdown_connections 2024-05-11T12:12:41.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:41.873+0000 7f1829800640 1 -- 172.21.15.167:0/2651809113 wait complete. 2024-05-11T12:12:42.294 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-11T12:12:35.314711+0000', 'modified': '2024-05-11T12:12:38.382446+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5310}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5310': {'gid': 5310, 'name': 'a', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.167:6835/4016299199', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6834', 'nonce': 4016299199}, {'type': 'v1', 'addr': '172.21.15.167:6835', 'nonce': 4016299199}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5310, 'qdb_cluster': [5310]} 2024-05-11T12:12:42.295 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-11T12:12:42.295 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-11T12:12:42.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.507+0000 7f2963200640 1 Processor -- start 2024-05-11T12:12:42.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.507+0000 7f2963200640 1 -- start start 2024-05-11T12:12:42.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.508+0000 7f2963200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f295c1628c0 0x7f295c162e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:42.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.508+0000 7f2963200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f295c0d1860 0x7f295c1633a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:42.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.508+0000 7f2963200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f295c0d2d90 0x7f295c0d3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:42.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.508+0000 7f2963200640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f295c163ca0 con 0x7f295c0d1860 2024-05-11T12:12:42.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.508+0000 7f2963200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f295c0d0da0 con 0x7f295c1628c0 2024-05-11T12:12:42.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.508+0000 7f2963200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f295c0b4760 con 0x7f295c0d2d90 2024-05-11T12:12:42.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.509+0000 7f295b400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f295c0d1860 0x7f295c1633a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:42.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.509+0000 7f295b400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f295c0d1860 0x7f295c1633a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45640/0 (socket says 172.21.15.167:45640) 2024-05-11T12:12:42.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.509+0000 7f295b400640 1 -- 172.21.15.167:0/610773877 learned_addr learned my addr 172.21.15.167:0/610773877 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:42.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.509+0000 7f2961200640 1 --2- 172.21.15.167:0/610773877 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f295c0d2d90 0x7f295c0d3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:42.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.509+0000 7f295be00640 1 --2- 172.21.15.167:0/610773877 >> v2:172.21.15.204:3300/0 conn(0x7f295c1628c0 0x7f295c162e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:42.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.509+0000 7f295b400640 1 -- 172.21.15.167:0/610773877 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f295c0d2d90 msgr2=0x7f295c0d3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:42.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.509+0000 7f295b400640 1 --2- 172.21.15.167:0/610773877 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f295c0d2d90 0x7f295c0d3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:42.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.509+0000 7f295b400640 1 -- 172.21.15.167:0/610773877 >> v2:172.21.15.204:3300/0 conn(0x7f295c1628c0 msgr2=0x7f295c162e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:42.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.509+0000 7f295b400640 1 --2- 172.21.15.167:0/610773877 >> v2:172.21.15.204:3300/0 conn(0x7f295c1628c0 0x7f295c162e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:42.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.509+0000 7f295b400640 1 -- 172.21.15.167:0/610773877 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f295c0d1060 con 0x7f295c0d1860 2024-05-11T12:12:42.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.510+0000 7f295b400640 1 --2- 172.21.15.167:0/610773877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f295c0d1860 0x7f295c1633a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f295c0c8ca0 tx=0x7f294c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6ea6bc134e9a131 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:42.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.511+0000 7f295aa00640 1 -- 172.21.15.167:0/610773877 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c036070 con 0x7f295c0d1860 2024-05-11T12:12:42.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.511+0000 7f295aa00640 1 -- 172.21.15.167:0/610773877 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f294c031600 con 0x7f295c0d1860 2024-05-11T12:12:42.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.511+0000 7f295aa00640 1 -- 172.21.15.167:0/610773877 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c0319d0 con 0x7f295c0d1860 2024-05-11T12:12:42.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.511+0000 7f2963200640 1 -- 172.21.15.167:0/610773877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f295c0d1860 msgr2=0x7f295c1633a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:42.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.511+0000 7f2963200640 1 --2- 172.21.15.167:0/610773877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f295c0d1860 0x7f295c1633a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f295c0c8ca0 tx=0x7f294c02a6a0 comp rx=0 tx=0).stop 2024-05-11T12:12:42.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.511+0000 7f2963200640 1 -- 172.21.15.167:0/610773877 shutdown_connections 2024-05-11T12:12:42.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.511+0000 7f2963200640 1 --2- 172.21.15.167:0/610773877 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f295c0d2d90 0x7f295c0d3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:42.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.511+0000 7f2963200640 1 --2- 172.21.15.167:0/610773877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f295c0d1860 0x7f295c1633a0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:42.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.511+0000 7f2963200640 1 --2- 172.21.15.167:0/610773877 >> v2:172.21.15.204:3300/0 conn(0x7f295c1628c0 0x7f295c162e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:42.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.511+0000 7f2963200640 1 -- 172.21.15.167:0/610773877 >> 172.21.15.167:0/610773877 conn(0x7f295c0d51c0 msgr2=0x7f295c0d55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:42.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.511+0000 7f2963200640 1 -- 172.21.15.167:0/610773877 shutdown_connections 2024-05-11T12:12:42.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.512+0000 7f2963200640 1 -- 172.21.15.167:0/610773877 wait complete. 2024-05-11T12:12:42.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.513+0000 7f2963200640 1 Processor -- start 2024-05-11T12:12:42.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.513+0000 7f2963200640 1 -- start start 2024-05-11T12:12:42.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.514+0000 7f2963200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f295c0d1860 0x7f295c0c9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:42.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.514+0000 7f2963200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f295c0d2d90 0x7f295c1e8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:42.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.514+0000 7f2963200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f295c1628c0 0x7f295c1f8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:42.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.514+0000 7f2963200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f295c0b4380 con 0x7f295c0d1860 2024-05-11T12:12:42.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.514+0000 7f2963200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f295c0b4a40 con 0x7f295c1628c0 2024-05-11T12:12:42.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.514+0000 7f2963200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f295c0b4500 con 0x7f295c0d2d90 2024-05-11T12:12:42.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.515+0000 7f295be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f295c0d1860 0x7f295c0c9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:42.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.515+0000 7f295be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f295c0d1860 0x7f295c0c9430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45644/0 (socket says 172.21.15.167:45644) 2024-05-11T12:12:42.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.515+0000 7f295be00640 1 -- 172.21.15.167:0/3876503629 learned_addr learned my addr 172.21.15.167:0/3876503629 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:42.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.515+0000 7f295b400640 1 --2- 172.21.15.167:0/3876503629 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f295c0d2d90 0x7f295c1e8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:42.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.515+0000 7f295be00640 1 -- 172.21.15.167:0/3876503629 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f295c0d2d90 msgr2=0x7f295c1e8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:42.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.515+0000 7f295be00640 1 --2- 172.21.15.167:0/3876503629 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f295c0d2d90 0x7f295c1e8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:42.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.515+0000 7f295be00640 1 -- 172.21.15.167:0/3876503629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f295c1628c0 msgr2=0x7f295c1f8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:42.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.515+0000 7f2961200640 1 --2- 172.21.15.167:0/3876503629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f295c1628c0 0x7f295c1f8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:42.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.515+0000 7f295be00640 1 --2- 172.21.15.167:0/3876503629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f295c1628c0 0x7f295c1f8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:42.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.515+0000 7f295be00640 1 -- 172.21.15.167:0/3876503629 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f295c1f8550 con 0x7f295c0d1860 2024-05-11T12:12:42.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.515+0000 7f295b400640 1 --2- 172.21.15.167:0/3876503629 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f295c0d2d90 0x7f295c1e8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:42.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.516+0000 7f295be00640 1 --2- 172.21.15.167:0/3876503629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f295c0d1860 0x7f295c0c9430 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f294400d5f0 tx=0x7f294400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:42.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.516+0000 7f2953e00640 1 -- 172.21.15.167:0/3876503629 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29440125f0 con 0x7f295c0d1860 2024-05-11T12:12:42.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.516+0000 7f2953e00640 1 -- 172.21.15.167:0/3876503629 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2944019070 con 0x7f295c0d1860 2024-05-11T12:12:42.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.516+0000 7f2963200640 1 -- 172.21.15.167:0/3876503629 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f295c0c9ba0 con 0x7f295c0d1860 2024-05-11T12:12:42.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.516+0000 7f2953e00640 1 -- 172.21.15.167:0/3876503629 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29440129d0 con 0x7f295c0d1860 2024-05-11T12:12:42.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.516+0000 7f2963200640 1 -- 172.21.15.167:0/3876503629 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f295c0ca160 con 0x7f295c0d1860 2024-05-11T12:12:42.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.518+0000 7f2953e00640 1 -- 172.21.15.167:0/3876503629 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2944012b70 con 0x7f295c0d1860 2024-05-11T12:12:42.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.520+0000 7f2953e00640 1 --2- 172.21.15.167:0/3876503629 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f29380762d0 0x7f2938078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:42.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.520+0000 7f2953e00640 1 -- 172.21.15.167:0/3876503629 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f2944097be0 con 0x7f295c0d1860 2024-05-11T12:12:42.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.520+0000 7f295b400640 1 --2- 172.21.15.167:0/3876503629 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f29380762d0 0x7f2938078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:42.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.521+0000 7f2965f95740 1 Processor -- start 2024-05-11T12:12:42.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.521+0000 7f2965f95740 1 -- start start 2024-05-11T12:12:42.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.521+0000 7f295b400640 1 --2- 172.21.15.167:0/3876503629 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f29380762d0 0x7f2938078790 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f294c033000 tx=0x7f294c009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:42.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.522+0000 7f2965f95740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560568d49840 0x560568d49c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:42.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.522+0000 7f2965f95740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x560568d446d0 0x560568d48440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:42.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.522+0000 7f2965f95740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560568d48980 0x560568d48dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:42.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.522+0000 7f2965f95740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560568cad790 con 0x560568d49840 2024-05-11T12:12:42.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.522+0000 7f2965f95740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560568c52f20 con 0x560568d48980 2024-05-11T12:12:42.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.522+0000 7f2965f95740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x560568ca9750 con 0x560568d446d0 2024-05-11T12:12:42.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.522+0000 7f295be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560568d49840 0x560568d49c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:42.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.522+0000 7f295be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560568d49840 0x560568d49c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45652/0 (socket says 172.21.15.167:45652) 2024-05-11T12:12:42.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.522+0000 7f295be00640 1 -- 172.21.15.167:0/2672106355 learned_addr learned my addr 172.21.15.167:0/2672106355 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:42.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.522+0000 7f295b400640 1 --2- 172.21.15.167:0/2672106355 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x560568d446d0 0x560568d48440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:42.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.522+0000 7f295be00640 1 -- 172.21.15.167:0/2672106355 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x560568d446d0 msgr2=0x560568d48440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:42.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.522+0000 7f295be00640 1 --2- 172.21.15.167:0/2672106355 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x560568d446d0 0x560568d48440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:42.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.522+0000 7f295be00640 1 -- 172.21.15.167:0/2672106355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560568d48980 msgr2=0x560568d48dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:42.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.522+0000 7f295be00640 1 --2- 172.21.15.167:0/2672106355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560568d48980 0x560568d48dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:42.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.522+0000 7f295be00640 1 -- 172.21.15.167:0/2672106355 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605689bf2d0 con 0x560568d49840 2024-05-11T12:12:42.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.523+0000 7f295b400640 1 --2- 172.21.15.167:0/2672106355 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x560568d446d0 0x560568d48440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:42.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.523+0000 7f295be00640 1 --2- 172.21.15.167:0/2672106355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560568d49840 0x560568d49c20 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f294405d200 tx=0x7f2944060720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d3745145a54271b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:42.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.523+0000 7f2950c00640 1 -- 172.21.15.167:0/2672106355 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2944069330 con 0x560568d49840 2024-05-11T12:12:42.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.523+0000 7f2950c00640 1 -- 172.21.15.167:0/2672106355 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f294407a070 con 0x560568d49840 2024-05-11T12:12:42.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.523+0000 7f2950c00640 1 -- 172.21.15.167:0/2672106355 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2944011e10 con 0x560568d49840 2024-05-11T12:12:42.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.524+0000 7f2965f95740 1 -- 172.21.15.167:0/2672106355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560568d49840 msgr2=0x560568d49c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:42.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.524+0000 7f2965f95740 1 --2- 172.21.15.167:0/2672106355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560568d49840 0x560568d49c20 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f294405d200 tx=0x7f2944060720 comp rx=0 tx=0).stop 2024-05-11T12:12:42.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.524+0000 7f2965f95740 1 -- 172.21.15.167:0/2672106355 shutdown_connections 2024-05-11T12:12:42.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.524+0000 7f2965f95740 1 --2- 172.21.15.167:0/2672106355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560568d48980 0x560568d48dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:42.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.524+0000 7f2965f95740 1 --2- 172.21.15.167:0/2672106355 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x560568d446d0 0x560568d48440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:42.525 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.524+0000 7f2965f95740 1 --2- 172.21.15.167:0/2672106355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560568d49840 0x560568d49c20 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:42.525 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.524+0000 7f2965f95740 1 -- 172.21.15.167:0/2672106355 >> 172.21.15.167:0/2672106355 conn(0x560568d46940 msgr2=0x560568d8f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:42.525 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.524+0000 7f2965f95740 1 -- 172.21.15.167:0/2672106355 shutdown_connections 2024-05-11T12:12:42.525 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.524+0000 7f2965f95740 1 -- 172.21.15.167:0/2672106355 wait complete. 2024-05-11T12:12:42.525 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.524+0000 7f2965f95740 1 -- start start 2024-05-11T12:12:42.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.525+0000 7f2965f95740 1 Processor -- start 2024-05-11T12:12:42.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.526+0000 7f2965f95740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560568d48980 0x560568d60560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:42.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.526+0000 7f2965f95740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560568d49840 0x560568d67aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:42.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.526+0000 7f2965f95740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x560568d659b0 0x560568d65e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:42.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.526+0000 7f2965f95740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560568d753d0 con 0x560568d48980 2024-05-11T12:12:42.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.526+0000 7f2965f95740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560568c669c0 con 0x560568d49840 2024-05-11T12:12:42.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.526+0000 7f2965f95740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x560568d74850 con 0x560568d659b0 2024-05-11T12:12:42.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.526+0000 7f2965f95740 10 client.0 _ll_register_callbacks cb 0x560568c33120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:12:42.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.526+0000 7f295be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560568d48980 0x560568d60560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:42.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.526+0000 7f295be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560568d48980 0x560568d60560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45660/0 (socket says 172.21.15.167:45660) 2024-05-11T12:12:42.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.526+0000 7f295be00640 1 -- 172.21.15.167:0/2093991989 learned_addr learned my addr 172.21.15.167:0/2093991989 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:42.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.526+0000 7f295b400640 1 --2- 172.21.15.167:0/2093991989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560568d49840 0x560568d67aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:42.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.526+0000 7f2961200640 1 --2- 172.21.15.167:0/2093991989 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x560568d659b0 0x560568d65e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:42.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.526+0000 7f295be00640 1 -- 172.21.15.167:0/2093991989 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x560568d659b0 msgr2=0x560568d65e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:42.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.526+0000 7f295be00640 1 --2- 172.21.15.167:0/2093991989 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x560568d659b0 0x560568d65e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:42.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.526+0000 7f295be00640 1 -- 172.21.15.167:0/2093991989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560568d49840 msgr2=0x560568d67aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:42.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.526+0000 7f295be00640 1 --2- 172.21.15.167:0/2093991989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560568d49840 0x560568d67aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:42.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.526+0000 7f295be00640 1 -- 172.21.15.167:0/2093991989 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560568a896b0 con 0x560568d48980 2024-05-11T12:12:42.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.527+0000 7f295be00640 1 --2- 172.21.15.167:0/2093991989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560568d48980 0x560568d60560 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2944060910 tx=0x7f2944020070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f231d7a183c3c3c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:42.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.527+0000 7f2949600640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-11T12:12:42.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.527+0000 7f2949600640 1 -- 172.21.15.167:0/2093991989 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2944080060 con 0x560568d48980 2024-05-11T12:12:42.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.527+0000 7f2949600640 1 -- 172.21.15.167:0/2093991989 <== mon.0 v2:172.21.15.167:3300/0 2 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f2944011df0 con 0x560568d48980 2024-05-11T12:12:42.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.527+0000 7f2965f95740 1 -- 172.21.15.167:0/2093991989 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f294407a070 con 0x560568d48980 2024-05-11T12:12:42.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.527+0000 7f2949600640 1 -- 172.21.15.167:0/2093991989 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f294406e050 con 0x560568d48980 2024-05-11T12:12:42.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.527+0000 7f2965f95740 10 client.5430 fetch_fsmap learned FSMap version 26 2024-05-11T12:12:42.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.527+0000 7f2965f95740 1 -- 172.21.15.167:0/2093991989 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x560568d626e0 con 0x560568d48980 2024-05-11T12:12:42.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.527+0000 7f2949600640 1 -- 172.21.15.167:0/2093991989 <== mon.0 v2:172.21.15.167:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f294407a070 con 0x560568d48980 2024-05-11T12:12:42.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.527+0000 7f2949600640 1 -- 172.21.15.167:0/2093991989 <== mon.0 v2:172.21.15.167:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7f2944013070 con 0x560568d48980 2024-05-11T12:12:42.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.527+0000 7f2965f95740 10 client.5430 fetch_fsmap finished waiting for FSMap version 26 2024-05-11T12:12:42.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.527+0000 7f2965f95740 10 client.5430 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-11T12:12:42.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.527+0000 7f2965f95740 20 client.5430 populate_metadata read hostname 'smithi167' 2024-05-11T12:12:42.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.528+0000 7f2965f95740 1 --2- 172.21.15.167:0/2093991989 >> [v2:172.21.15.167:6834/4016299199,v1:172.21.15.167:6835/4016299199] conn(0x560568daaa90 0x560568dace80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:42.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.528+0000 7f2965f95740 4 client.5430 mds_command: new command op to 5310 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:12:42.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.528+0000 7f2965f95740 1 -- 172.21.15.167:0/2093991989 --> [v2:172.21.15.167:6834/4016299199,v1:172.21.15.167:6835/4016299199] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560568cbb3a0 con 0x560568daaa90 2024-05-11T12:12:42.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.528+0000 7f295b400640 1 --2- 172.21.15.167:0/2093991989 >> [v2:172.21.15.167:6834/4016299199,v1:172.21.15.167:6835/4016299199] conn(0x560568daaa90 0x560568dace80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:42.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.528+0000 7f295b400640 1 --2- 172.21.15.167:0/2093991989 >> [v2:172.21.15.167:6834/4016299199,v1:172.21.15.167:6835/4016299199] conn(0x560568daaa90 0x560568dace80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23b6eb546d4caa36 server_cookie=9afb1090c60f676e in_seq=0 out_seq=0 2024-05-11T12:12:42.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.528+0000 7f2949600640 10 client.5430 ms_handle_connect on v2:172.21.15.167:6834/4016299199 2024-05-11T12:12:42.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.529+0000 7f2949600640 1 -- 172.21.15.167:0/2093991989 <== mds.0 v2:172.21.15.167:6834/4016299199 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560568cbb3a0 con 0x560568daaa90 2024-05-11T12:12:42.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.529+0000 7f2949600640 10 client.5430 handle_command_reply: tid=0 2024-05-11T12:12:42.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.529+0000 7f2965f95740 1 client.5430 shutdown 2024-05-11T12:12:42.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.530+0000 7f2965f95740 1 -- 172.21.15.167:0/2093991989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560568d48980 msgr2=0x560568d60560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:42.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.530+0000 7f2965f95740 1 --2- 172.21.15.167:0/2093991989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560568d48980 0x560568d60560 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2944060910 tx=0x7f2944020070 comp rx=0 tx=0).stop 2024-05-11T12:12:42.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.530+0000 7f2965f95740 1 -- 172.21.15.167:0/2093991989 shutdown_connections 2024-05-11T12:12:42.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.530+0000 7f2965f95740 1 --2- 172.21.15.167:0/2093991989 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x560568d659b0 0x560568d65e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:42.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.530+0000 7f2965f95740 1 --2- 172.21.15.167:0/2093991989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560568d49840 0x560568d67aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:42.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.530+0000 7f2965f95740 1 --2- 172.21.15.167:0/2093991989 >> [v2:172.21.15.167:6834/4016299199,v1:172.21.15.167:6835/4016299199] conn(0x560568daaa90 0x560568dace80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:42.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.530+0000 7f2965f95740 1 --2- 172.21.15.167:0/2093991989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560568d48980 0x560568d60560 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:42.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.530+0000 7f2949600640 0 client.5430 ms_handle_reset on v2:172.21.15.167:6834/4016299199 2024-05-11T12:12:42.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.530+0000 7f2965f95740 1 -- 172.21.15.167:0/2093991989 >> 172.21.15.167:0/2093991989 conn(0x560568d446d0 msgr2=0x560568d43df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:42.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.530+0000 7f2965f95740 1 -- 172.21.15.167:0/2093991989 shutdown_connections 2024-05-11T12:12:42.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.530+0000 7f2965f95740 1 -- 172.21.15.167:0/2093991989 wait complete. 2024-05-11T12:12:42.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.530+0000 7f2965f95740 20 client.5430 trim_cache size 0 max 16384 2024-05-11T12:12:42.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.550+0000 7f2965f95740 1 Processor -- start 2024-05-11T12:12:42.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.551+0000 7f2965f95740 1 -- start start 2024-05-11T12:12:42.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.551+0000 7f2965f95740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x560568d446d0 0x560568d6b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:42.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.551+0000 7f2965f95740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560568d48980 0x560568d6ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:42.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.551+0000 7f2965f95740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560568d49840 0x560568d6c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:42.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.551+0000 7f2965f95740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f295c201060 con 0x560568d49840 2024-05-11T12:12:42.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.551+0000 7f2965f95740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f295c1f6680 con 0x560568d48980 2024-05-11T12:12:42.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.551+0000 7f2965f95740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f295c200a40 con 0x560568d446d0 2024-05-11T12:12:42.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.552+0000 7f2961200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560568d49840 0x560568d6c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:42.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.552+0000 7f2961200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560568d49840 0x560568d6c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45670/0 (socket says 172.21.15.167:45670) 2024-05-11T12:12:42.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.552+0000 7f2961200640 1 -- 172.21.15.167:0/2122855324 learned_addr learned my addr 172.21.15.167:0/2122855324 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:42.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.552+0000 7f295be00640 1 --2- 172.21.15.167:0/2122855324 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x560568d446d0 0x560568d6b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:42.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.552+0000 7f2961200640 1 -- 172.21.15.167:0/2122855324 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x560568d446d0 msgr2=0x560568d6b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:42.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.552+0000 7f295b400640 1 --2- 172.21.15.167:0/2122855324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560568d48980 0x560568d6ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:42.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.552+0000 7f2961200640 1 --2- 172.21.15.167:0/2122855324 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x560568d446d0 0x560568d6b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:42.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.552+0000 7f2961200640 1 -- 172.21.15.167:0/2122855324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560568d48980 msgr2=0x560568d6ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:42.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.552+0000 7f2961200640 1 --2- 172.21.15.167:0/2122855324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560568d48980 0x560568d6ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:42.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.552+0000 7f2961200640 1 -- 172.21.15.167:0/2122855324 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560568d62960 con 0x560568d49840 2024-05-11T12:12:42.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.552+0000 7f295be00640 1 --2- 172.21.15.167:0/2122855324 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x560568d446d0 0x560568d6b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:42.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.553+0000 7f2961200640 1 --2- 172.21.15.167:0/2122855324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560568d49840 0x560568d6c4b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f295400d500 tx=0x7f295400d9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8182c1d15936800 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:42.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.553+0000 7f294be00640 1 -- 172.21.15.167:0/2122855324 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2954018070 con 0x560568d49840 2024-05-11T12:12:42.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.553+0000 7f294be00640 1 -- 172.21.15.167:0/2122855324 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29540146a0 con 0x560568d49840 2024-05-11T12:12:42.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.553+0000 7f294be00640 1 -- 172.21.15.167:0/2122855324 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2954014a90 con 0x560568d49840 2024-05-11T12:12:42.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.553+0000 7f2965f95740 1 -- 172.21.15.167:0/2122855324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560568d49840 msgr2=0x560568d6c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:42.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.553+0000 7f2965f95740 1 --2- 172.21.15.167:0/2122855324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560568d49840 0x560568d6c4b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f295400d500 tx=0x7f295400d9c0 comp rx=0 tx=0).stop 2024-05-11T12:12:42.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.553+0000 7f2965f95740 1 -- 172.21.15.167:0/2122855324 shutdown_connections 2024-05-11T12:12:42.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.553+0000 7f2965f95740 1 --2- 172.21.15.167:0/2122855324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560568d49840 0x560568d6c4b0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:42.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.553+0000 7f2965f95740 1 --2- 172.21.15.167:0/2122855324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560568d48980 0x560568d6ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:42.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.553+0000 7f2965f95740 1 --2- 172.21.15.167:0/2122855324 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x560568d446d0 0x560568d6b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:42.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.553+0000 7f2965f95740 1 -- 172.21.15.167:0/2122855324 >> 172.21.15.167:0/2122855324 conn(0x560568d47850 msgr2=0x560568d3fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:42.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.554+0000 7f2965f95740 1 -- 172.21.15.167:0/2122855324 shutdown_connections 2024-05-11T12:12:42.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.554+0000 7f2965f95740 1 -- 172.21.15.167:0/2122855324 wait complete. 2024-05-11T12:12:42.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.554+0000 7f2965f95740 1 -- start start 2024-05-11T12:12:42.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.554+0000 7f2965f95740 1 Processor -- start 2024-05-11T12:12:42.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.555+0000 7f2965f95740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x560568d446d0 0x560568d9e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:42.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.555+0000 7f2965f95740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560568d48980 0x560568d6c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:42.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.555+0000 7f2965f95740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560568d49840 0x560568d6c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:42.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.555+0000 7f2965f95740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560568d753d0 con 0x560568d49840 2024-05-11T12:12:42.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.555+0000 7f2965f95740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560568c669c0 con 0x560568d48980 2024-05-11T12:12:42.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.555+0000 7f2965f95740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x560568d61d10 con 0x560568d446d0 2024-05-11T12:12:42.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.556+0000 7f2965f95740 10 client.0 _ll_register_callbacks cb 0x7f2944090020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:12:42.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.556+0000 7f2961200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560568d49840 0x560568d6c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:42.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.556+0000 7f2961200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560568d49840 0x560568d6c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45680/0 (socket says 172.21.15.167:45680) 2024-05-11T12:12:42.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.556+0000 7f2961200640 1 -- 172.21.15.167:0/1664781722 learned_addr learned my addr 172.21.15.167:0/1664781722 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:42.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.556+0000 7f295b400640 1 --2- 172.21.15.167:0/1664781722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560568d48980 0x560568d6c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:42.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.556+0000 7f2961200640 1 -- 172.21.15.167:0/1664781722 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x560568d446d0 msgr2=0x560568d9e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:42.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.556+0000 7f2961200640 1 --2- 172.21.15.167:0/1664781722 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x560568d446d0 0x560568d9e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:42.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.556+0000 7f2961200640 1 -- 172.21.15.167:0/1664781722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560568d48980 msgr2=0x560568d6c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:42.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.556+0000 7f2961200640 1 --2- 172.21.15.167:0/1664781722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560568d48980 0x560568d6c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:42.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.556+0000 7f2961200640 1 -- 172.21.15.167:0/1664781722 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560568d9eb20 con 0x560568d49840 2024-05-11T12:12:42.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.556+0000 7f2961200640 1 --2- 172.21.15.167:0/1664781722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560568d49840 0x560568d6c990 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f295400dec0 tx=0x7f2954004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=377430b77c84b7b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:42.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.556+0000 7f2949600640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-11T12:12:42.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.557+0000 7f2949600640 1 -- 172.21.15.167:0/1664781722 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2954018070 con 0x560568d49840 2024-05-11T12:12:42.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.557+0000 7f2965f95740 1 -- 172.21.15.167:0/1664781722 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29540146a0 con 0x560568d49840 2024-05-11T12:12:42.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.557+0000 7f2949600640 1 -- 172.21.15.167:0/1664781722 <== mon.0 v2:172.21.15.167:3300/0 2 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f2954003810 con 0x560568d49840 2024-05-11T12:12:42.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.557+0000 7f2949600640 1 -- 172.21.15.167:0/1664781722 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f295402e050 con 0x560568d49840 2024-05-11T12:12:42.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.557+0000 7f2965f95740 10 client.5436 fetch_fsmap learned FSMap version 26 2024-05-11T12:12:42.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.557+0000 7f2965f95740 1 -- 172.21.15.167:0/1664781722 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x560568d9e6e0 con 0x560568d49840 2024-05-11T12:12:42.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.557+0000 7f2949600640 1 -- 172.21.15.167:0/1664781722 <== mon.0 v2:172.21.15.167:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29540146a0 con 0x560568d49840 2024-05-11T12:12:42.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.557+0000 7f2949600640 1 -- 172.21.15.167:0/1664781722 <== mon.0 v2:172.21.15.167:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7f2954031020 con 0x560568d49840 2024-05-11T12:12:42.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.557+0000 7f2965f95740 10 client.5436 fetch_fsmap finished waiting for FSMap version 26 2024-05-11T12:12:42.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.557+0000 7f2965f95740 10 client.5436 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-11T12:12:42.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.557+0000 7f2965f95740 20 client.5436 populate_metadata read hostname 'smithi167' 2024-05-11T12:12:42.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.557+0000 7f2965f95740 1 --2- 172.21.15.167:0/1664781722 >> [v2:172.21.15.167:6834/4016299199,v1:172.21.15.167:6835/4016299199] conn(0x560568d659b0 0x560568e19580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:42.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.558+0000 7f2965f95740 4 client.5436 mds_command: new command op to 5310 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-11T12:12:42.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.558+0000 7f2965f95740 1 -- 172.21.15.167:0/1664781722 --> [v2:172.21.15.167:6834/4016299199,v1:172.21.15.167:6835/4016299199] -- command(tid 0: {"prefix": "status"}) -- 0x560568d45be0 con 0x560568d659b0 2024-05-11T12:12:42.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.558+0000 7f295be00640 1 --2- 172.21.15.167:0/1664781722 >> [v2:172.21.15.167:6834/4016299199,v1:172.21.15.167:6835/4016299199] conn(0x560568d659b0 0x560568e19580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:42.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.558+0000 7f295be00640 1 --2- 172.21.15.167:0/1664781722 >> [v2:172.21.15.167:6834/4016299199,v1:172.21.15.167:6835/4016299199] conn(0x560568d659b0 0x560568e19580 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8885ec98fecb06cf server_cookie=d370489f93a3d19 in_seq=0 out_seq=0 2024-05-11T12:12:42.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.558+0000 7f2949600640 10 client.5436 ms_handle_connect on v2:172.21.15.167:6834/4016299199 2024-05-11T12:12:42.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.558+0000 7f2949600640 1 -- 172.21.15.167:0/1664781722 <== mds.0 v2:172.21.15.167:6834/4016299199 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x560568d45be0 con 0x560568d659b0 2024-05-11T12:12:42.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.558+0000 7f2949600640 10 client.5436 handle_command_reply: tid=0 2024-05-11T12:12:42.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.558+0000 7f2965f95740 1 client.5436 shutdown 2024-05-11T12:12:42.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.559+0000 7f2965f95740 1 -- 172.21.15.167:0/1664781722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560568d49840 msgr2=0x560568d6c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:42.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.559+0000 7f2965f95740 1 --2- 172.21.15.167:0/1664781722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560568d49840 0x560568d6c990 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f295400dec0 tx=0x7f2954004730 comp rx=0 tx=0).stop 2024-05-11T12:12:42.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.559+0000 7f2965f95740 1 -- 172.21.15.167:0/1664781722 shutdown_connections 2024-05-11T12:12:42.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.559+0000 7f2965f95740 1 --2- 172.21.15.167:0/1664781722 >> [v2:172.21.15.167:6834/4016299199,v1:172.21.15.167:6835/4016299199] conn(0x560568d659b0 0x560568e19580 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:42.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.559+0000 7f2965f95740 1 --2- 172.21.15.167:0/1664781722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560568d49840 0x560568d6c990 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:42.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.559+0000 7f2965f95740 1 --2- 172.21.15.167:0/1664781722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560568d48980 0x560568d6c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:42.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.559+0000 7f2965f95740 1 --2- 172.21.15.167:0/1664781722 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x560568d446d0 0x560568d9e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:42.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.559+0000 7f2949600640 0 client.5436 ms_handle_reset on v2:172.21.15.167:6834/4016299199 2024-05-11T12:12:42.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.559+0000 7f2965f95740 1 -- 172.21.15.167:0/1664781722 >> 172.21.15.167:0/1664781722 conn(0x560568d47850 msgr2=0x560568d6b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:42.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.559+0000 7f2965f95740 1 -- 172.21.15.167:0/1664781722 shutdown_connections 2024-05-11T12:12:42.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.559+0000 7f2965f95740 1 -- 172.21.15.167:0/1664781722 wait complete. 2024-05-11T12:12:42.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.559+0000 7f2965f95740 20 client.5436 trim_cache size 0 max 16384 2024-05-11T12:12:42.562 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T12:12:42.562 INFO:teuthology.orchestra.run.smithi167.stdout: "cluster_fsid": "5f32d344-7289-4c27-881b-1533419ab678", 2024-05-11T12:12:42.562 INFO:teuthology.orchestra.run.smithi167.stdout: "whoami": 0, 2024-05-11T12:12:42.563 INFO:teuthology.orchestra.run.smithi167.stdout: "id": 5310, 2024-05-11T12:12:42.563 INFO:teuthology.orchestra.run.smithi167.stdout: "want_state": "up:active", 2024-05-11T12:12:42.563 INFO:teuthology.orchestra.run.smithi167.stdout: "state": "up:active", 2024-05-11T12:12:42.563 INFO:teuthology.orchestra.run.smithi167.stdout: "fs_name": "cephfs", 2024-05-11T12:12:42.563 INFO:teuthology.orchestra.run.smithi167.stdout: "rank_uptime": 6.6931353250000001, 2024-05-11T12:12:42.563 INFO:teuthology.orchestra.run.smithi167.stdout: "mdsmap_epoch": 26, 2024-05-11T12:12:42.563 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_epoch": 53, 2024-05-11T12:12:42.563 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_epoch_barrier": 53, 2024-05-11T12:12:42.563 INFO:teuthology.orchestra.run.smithi167.stdout: "uptime": 12.125149507 2024-05-11T12:12:42.563 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T12:12:42.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.560+0000 7f2963200640 1 -- 172.21.15.167:0/3876503629 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f29380762d0 msgr2=0x7f2938078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:42.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.560+0000 7f2963200640 1 --2- 172.21.15.167:0/3876503629 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f29380762d0 0x7f2938078790 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f294c033000 tx=0x7f294c009000 comp rx=0 tx=0).stop 2024-05-11T12:12:42.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.560+0000 7f2963200640 1 -- 172.21.15.167:0/3876503629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f295c0d1860 msgr2=0x7f295c0c9430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:42.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.560+0000 7f2963200640 1 --2- 172.21.15.167:0/3876503629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f295c0d1860 0x7f295c0c9430 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f294400d5f0 tx=0x7f294400dab0 comp rx=0 tx=0).stop 2024-05-11T12:12:42.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.560+0000 7f2963200640 1 -- 172.21.15.167:0/3876503629 shutdown_connections 2024-05-11T12:12:42.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.560+0000 7f2963200640 1 --2- 172.21.15.167:0/3876503629 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f29380762d0 0x7f2938078790 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:42.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.560+0000 7f2963200640 1 --2- 172.21.15.167:0/3876503629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f295c1628c0 0x7f295c1f8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:42.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.560+0000 7f2963200640 1 --2- 172.21.15.167:0/3876503629 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f295c0d2d90 0x7f295c1e8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:42.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.560+0000 7f2963200640 1 --2- 172.21.15.167:0/3876503629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f295c0d1860 0x7f295c0c9430 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:42.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.560+0000 7f2963200640 1 -- 172.21.15.167:0/3876503629 >> 172.21.15.167:0/3876503629 conn(0x7f295c0d51c0 msgr2=0x7f295c0b5cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:42.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.560+0000 7f2963200640 1 -- 172.21.15.167:0/3876503629 shutdown_connections 2024-05-11T12:12:42.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.560+0000 7f2963200640 1 -- 172.21.15.167:0/3876503629 wait complete. 2024-05-11T12:12:42.577 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-11T12:12:42.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.770+0000 7f7810800640 1 Processor -- start 2024-05-11T12:12:42.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.770+0000 7f7810800640 1 -- start start 2024-05-11T12:12:42.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.772+0000 7f7810800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7808157750 0x7f780815d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:42.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.772+0000 7f7810800640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f78080b3ea0 0x7f7808161490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:42.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.772+0000 7f7810800640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f780815dc80 con 0x7f78081619d0 2024-05-11T12:12:42.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.772+0000 7f7810800640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f7808161000 con 0x7f78080b3ea0 2024-05-11T12:12:42.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.772+0000 7f7810800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f780815df30 con 0x7f7808157750 2024-05-11T12:12:42.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.772+0000 7f780d400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f78080b3ea0 0x7f7808161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:42.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.772+0000 7f780e800640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f78081619d0 0x7f780806d460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:58136/0 (socket says 172.21.15.167:58136) 2024-05-11T12:12:42.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.772+0000 7f780e800640 1 -- 172.21.15.167:0/165311614 learned_addr learned my addr 172.21.15.167:0/165311614 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:42.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.773+0000 7f780de00640 1 --2- 172.21.15.167:0/165311614 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7808157750 0x7f780815d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:42.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.773+0000 7f780d400640 1 -- 172.21.15.167:0/165311614 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7808157750 msgr2=0x7f780815d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:42.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.773+0000 7f780d400640 1 --2- 172.21.15.167:0/165311614 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7808157750 0x7f780815d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:42.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.773+0000 7f780d400640 1 -- 172.21.15.167:0/165311614 >> v1:172.21.15.167:6789/0 conn(0x7f78081619d0 legacy=0x7f780806d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:42.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.773+0000 7f780d400640 1 -- 172.21.15.167:0/165311614 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f780815e210 con 0x7f78080b3ea0 2024-05-11T12:12:42.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.773+0000 7f780ca00640 1 -- 172.21.15.167:0/165311614 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1662425592 0 0) 0x7f780815dc80 con 0x7f78081619d0 2024-05-11T12:12:42.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.774+0000 7f780d400640 1 --2- 172.21.15.167:0/165311614 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78080b3ea0 0x7f7808161490 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f77f40097a0 tx=0x7f77f402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ab2a538c2316185 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:42.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.774+0000 7f780ca00640 1 -- 172.21.15.167:0/165311614 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f4037070 con 0x7f78080b3ea0 2024-05-11T12:12:42.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.774+0000 7f780ca00640 1 -- 172.21.15.167:0/165311614 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77f4032440 con 0x7f78080b3ea0 2024-05-11T12:12:42.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.774+0000 7f780ca00640 1 -- 172.21.15.167:0/165311614 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f4032810 con 0x7f78080b3ea0 2024-05-11T12:12:42.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.775+0000 7f7810800640 1 -- 172.21.15.167:0/165311614 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78080b3ea0 msgr2=0x7f7808161490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:42.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.775+0000 7f7810800640 1 --2- 172.21.15.167:0/165311614 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78080b3ea0 0x7f7808161490 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f77f40097a0 tx=0x7f77f402a180 comp rx=0 tx=0).stop 2024-05-11T12:12:42.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.775+0000 7f7810800640 1 -- 172.21.15.167:0/165311614 shutdown_connections 2024-05-11T12:12:42.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.775+0000 7f7810800640 1 --2- 172.21.15.167:0/165311614 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78080b3ea0 0x7f7808161490 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:42.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.775+0000 7f7810800640 1 --2- 172.21.15.167:0/165311614 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7808157750 0x7f780815d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:42.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.775+0000 7f7810800640 1 -- 172.21.15.167:0/165311614 >> 172.21.15.167:0/165311614 conn(0x7f78080cd8f0 msgr2=0x7f78080cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:42.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.775+0000 7f7810800640 1 -- 172.21.15.167:0/165311614 shutdown_connections 2024-05-11T12:12:42.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.775+0000 7f7810800640 1 -- 172.21.15.167:0/165311614 wait complete. 2024-05-11T12:12:42.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.777+0000 7f7810800640 1 Processor -- start 2024-05-11T12:12:42.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.777+0000 7f7810800640 1 -- start start 2024-05-11T12:12:42.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.778+0000 7f7810800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78080b3ea0 0x7f78081f7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:42.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.778+0000 7f7810800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7808157750 0x7f78080af870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:42.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.778+0000 7f7810800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78081619d0 0x7f78080afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:42.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.778+0000 7f7810800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f780806db70 con 0x7f78081619d0 2024-05-11T12:12:42.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.778+0000 7f7810800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7808157b30 con 0x7f78080b3ea0 2024-05-11T12:12:42.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.778+0000 7f7810800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f780806dcf0 con 0x7f7808157750 2024-05-11T12:12:42.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.778+0000 7f780e800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78081619d0 0x7f78080afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:42.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.778+0000 7f780e800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78081619d0 0x7f78080afdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45704/0 (socket says 172.21.15.167:45704) 2024-05-11T12:12:42.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.778+0000 7f780e800640 1 -- 172.21.15.167:0/1820723238 learned_addr learned my addr 172.21.15.167:0/1820723238 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:42.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.779+0000 7f780d400640 1 --2- 172.21.15.167:0/1820723238 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7808157750 0x7f78080af870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:42.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.779+0000 7f780de00640 1 --2- 172.21.15.167:0/1820723238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78080b3ea0 0x7f78081f7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:42.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.779+0000 7f780e800640 1 -- 172.21.15.167:0/1820723238 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7808157750 msgr2=0x7f78080af870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:42.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.779+0000 7f780e800640 1 --2- 172.21.15.167:0/1820723238 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7808157750 0x7f78080af870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:42.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.779+0000 7f780e800640 1 -- 172.21.15.167:0/1820723238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78080b3ea0 msgr2=0x7f78081f7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:42.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.779+0000 7f780e800640 1 --2- 172.21.15.167:0/1820723238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78080b3ea0 0x7f78081f7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:42.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.779+0000 7f780e800640 1 -- 172.21.15.167:0/1820723238 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f780815a9e0 con 0x7f78081619d0 2024-05-11T12:12:42.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.779+0000 7f780e800640 1 --2- 172.21.15.167:0/1820723238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78081619d0 0x7f78080afdb0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f780400c680 tx=0x7f780400cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:42.780 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.779+0000 7f7802000640 1 -- 172.21.15.167:0/1820723238 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7804018070 con 0x7f78081619d0 2024-05-11T12:12:42.780 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.779+0000 7f7802000640 1 -- 172.21.15.167:0/1820723238 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7804004d60 con 0x7f78081619d0 2024-05-11T12:12:42.780 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.779+0000 7f7802000640 1 -- 172.21.15.167:0/1820723238 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780400a880 con 0x7f78081619d0 2024-05-11T12:12:42.780 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.779+0000 7f7810800640 1 -- 172.21.15.167:0/1820723238 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f780815acc0 con 0x7f78081619d0 2024-05-11T12:12:42.780 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.780+0000 7f7810800640 1 -- 172.21.15.167:0/1820723238 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f780815b4e0 con 0x7f78081619d0 2024-05-11T12:12:42.780 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.781+0000 7f7810800640 1 -- 172.21.15.167:0/1820723238 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7808069590 con 0x7f78081619d0 2024-05-11T12:12:42.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.782+0000 7f7802000640 1 -- 172.21.15.167:0/1820723238 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f780400aa20 con 0x7f78081619d0 2024-05-11T12:12:42.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.783+0000 7f7802000640 1 --2- 172.21.15.167:0/1820723238 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f77e80762d0 0x7f77e8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:42.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.783+0000 7f7802000640 1 -- 172.21.15.167:0/1820723238 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f7804096de0 con 0x7f78081619d0 2024-05-11T12:12:42.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.783+0000 7f780de00640 1 --2- 172.21.15.167:0/1820723238 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f77e80762d0 0x7f77e8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:42.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.784+0000 7f780de00640 1 --2- 172.21.15.167:0/1820723238 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f77e80762d0 0x7f77e8078790 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f77fc006fd0 tx=0x7f77fc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:42.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:42.787+0000 7f7802000640 1 -- 172.21.15.167:0/1820723238 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7804060510 con 0x7f78081619d0 2024-05-11T12:12:43.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:43.034+0000 7f7810800640 1 -- 172.21.15.167:0/1820723238 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f78081604c0 con 0x7f78081619d0 2024-05-11T12:12:43.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:43.034+0000 7f7802000640 1 -- 172.21.15.167:0/1820723238 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7f780405fe10 con 0x7f78081619d0 2024-05-11T12:12:43.033 INFO:teuthology.orchestra.run.smithi167.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-11T12:12:43.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:43.037+0000 7f7810800640 1 -- 172.21.15.167:0/1820723238 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f77e80762d0 msgr2=0x7f77e8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:43.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:43.037+0000 7f7810800640 1 --2- 172.21.15.167:0/1820723238 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f77e80762d0 0x7f77e8078790 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f77fc006fd0 tx=0x7f77fc008040 comp rx=0 tx=0).stop 2024-05-11T12:12:43.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:43.037+0000 7f7810800640 1 -- 172.21.15.167:0/1820723238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78081619d0 msgr2=0x7f78080afdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:43.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:43.037+0000 7f7810800640 1 --2- 172.21.15.167:0/1820723238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78081619d0 0x7f78080afdb0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f780400c680 tx=0x7f780400cb40 comp rx=0 tx=0).stop 2024-05-11T12:12:43.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:43.037+0000 7f7810800640 1 -- 172.21.15.167:0/1820723238 shutdown_connections 2024-05-11T12:12:43.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:43.037+0000 7f7810800640 1 --2- 172.21.15.167:0/1820723238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78081619d0 0x7f78080afdb0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:43.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:43.037+0000 7f7810800640 1 --2- 172.21.15.167:0/1820723238 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7808157750 0x7f78080af870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:43.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:43.037+0000 7f7810800640 1 --2- 172.21.15.167:0/1820723238 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f77e80762d0 0x7f77e8078790 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:43.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:43.037+0000 7f7810800640 1 --2- 172.21.15.167:0/1820723238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78080b3ea0 0x7f78081f7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:43.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:43.037+0000 7f7810800640 1 -- 172.21.15.167:0/1820723238 >> 172.21.15.167:0/1820723238 conn(0x7f78080cd8f0 msgr2=0x7f78080cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:43.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:43.037+0000 7f7810800640 1 -- 172.21.15.167:0/1820723238 shutdown_connections 2024-05-11T12:12:43.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:43.038+0000 7f7810800640 1 -- 172.21.15.167:0/1820723238 wait complete. 2024-05-11T12:12:43.055 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-11T12:12:43.055 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-11T12:12:43.055 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi167.front.sepia.ceph.com 2024-05-11T12:12:43.055 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-11T12:12:43.055 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-11T12:12:43.055 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-11T12:12:43.055 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-11T12:12:43.055 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-11T12:12:43.055 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-11T12:12:43.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:12:43.056 DEBUG:teuthology.orchestra.run.smithi167:> ip addr 2024-05-11T12:12:43.107 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-11T12:12:43.107 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-11T12:12:43.107 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2024-05-11T12:12:43.107 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:12:43.107 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2024-05-11T12:12:43.107 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:12:43.107 INFO:teuthology.orchestra.run.smithi167.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:12:43.107 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e6 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:12:43.107 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f0 2024-05-11T12:12:43.107 INFO:teuthology.orchestra.run.smithi167.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:12:43.107 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e7 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:12:43.107 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f1 2024-05-11T12:12:43.108 INFO:teuthology.orchestra.run.smithi167.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:12:43.108 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:08 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:12:43.108 INFO:teuthology.orchestra.run.smithi167.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-11T12:12:43.108 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:09 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:12:43.108 INFO:teuthology.orchestra.run.smithi167.stdout: inet 172.21.15.167/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-11T12:12:43.108 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft 42506sec preferred_lft 42506sec 2024-05-11T12:12:43.108 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::ec4:7aff:fe88:7309/64 scope link 2024-05-11T12:12:43.108 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:12:43.108 INFO:teuthology.orchestra.run.smithi167.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-11T12:12:43.108 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 1a:a8:8d:b3:f4:32 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:12:43.108 INFO:teuthology.orchestra.run.smithi167.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-11T12:12:43.108 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:12:43.109 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::b017:48ff:fea0:7183/64 scope link 2024-05-11T12:12:43.109 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:12:43.109 INFO:teuthology.orchestra.run.smithi167.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-11T12:12:43.109 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 1a:a8:8d:b3:f4:32 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-11T12:12:43.109 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::18a8:8dff:feb3:f432/64 scope link 2024-05-11T12:12:43.109 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:12:43.109 INFO:teuthology.orchestra.run.smithi167.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-11T12:12:43.109 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 7e:79:1f:cb:d2:83 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-11T12:12:43.109 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::7c79:1fff:fecb:d283/64 scope link 2024-05-11T12:12:43.109 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:12:43.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:12:43.110 DEBUG:teuthology.orchestra.run.smithi167:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-11T12:12:43.186 INFO:teuthology.orchestra.run.smithi167.stdout:1 2024-05-11T12:12:43.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:12:43.190 DEBUG:teuthology.orchestra.run.smithi167:> route 2024-05-11T12:12:43.242 INFO:teuthology.orchestra.run.smithi167.stdout:Kernel IP routing table 2024-05-11T12:12:43.242 INFO:teuthology.orchestra.run.smithi167.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-11T12:12:43.242 INFO:teuthology.orchestra.run.smithi167.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-11T12:12:43.242 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-11T12:12:43.242 INFO:teuthology.orchestra.run.smithi167.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-11T12:12:43.243 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T12:12:43.243 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-05-11T12:12:43.243 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-11T12:12:43.243 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-11T12:12:43.243 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-11T12:12:43.243 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-05-11T12:12:43.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T12:12:43.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:12:43.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:12:43.447 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2024-05-11T12:12:43.489 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-11T12:12:43.489 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-11T12:12:43.489 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-11T12:12:43.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:12:43.490 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set brx.0 up 2024-05-11T12:12:43.617 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T12:12:43.617 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-11T12:12:43.664 INFO:teuthology.orchestra.run.smithi167.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-11T12:12:43.664 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T12:12:43.664 DEBUG:teuthology.orchestra.run.smithi167:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-11T12:12:43.712 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-05-11T12:12:43.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:12:43.713 DEBUG:teuthology.orchestra.run.smithi167:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:12:43.839 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:12:43.840 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-11T12:12:43.872 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:12:43.872 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-11T12:12:43.872 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.167:3300,172.21.15.204:3300,172.21.15.204:3301" 2024-05-11T12:12:43.872 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: trying mount with new device syntax: 0@5f32d344-7289-4c27-881b-1533419ab678.cephfs=/ 2024-05-11T12:12:43.872 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.167:3300/172.21.15.204:3300/172.21.15.204:3301" will pass to kernel 2024-05-11T12:12:43.874 INFO:teuthology.orchestra.run.smithi167.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-11T12:12:43.874 INFO:teuthology.orchestra.run.smithi167.stdout: You just mounted a file system that supports labels which does not 2024-05-11T12:12:43.874 INFO:teuthology.orchestra.run.smithi167.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-11T12:12:43.874 INFO:teuthology.orchestra.run.smithi167.stdout: applications will generate AVC messages and not be allowed access to 2024-05-11T12:12:43.874 INFO:teuthology.orchestra.run.smithi167.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-11T12:12:43.878 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-11T12:12:43.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:12:43.878 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-11T12:12:43.949 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-11T12:12:44.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-11T12:12:44.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:12:44.070 INFO:teuthology.orchestra.run.smithi167.stdout:client5448 2024-05-11T12:12:44.070 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:12:44.071 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:12:44.119 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-11T12:12:44.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.370+0000 7f8b59a00640 1 Processor -- start 2024-05-11T12:12:44.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.371+0000 7f8b59a00640 1 -- start start 2024-05-11T12:12:44.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.372+0000 7f8b59a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8b540718e0 0x7f8b54160e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:44.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.372+0000 7f8b59a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b541573a0 0x7f8b54157830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:44.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.372+0000 7f8b59a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b54161390 con 0x7f8b54071cc0 2024-05-11T12:12:44.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.372+0000 7f8b59a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f8b540d1d20 con 0x7f8b540718e0 2024-05-11T12:12:44.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.372+0000 7f8b59a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f8b54157d70 con 0x7f8b541573a0 2024-05-11T12:12:44.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.373+0000 7f8b4a000640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f8b54071cc0 0x7f8b540cd610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.167:46500/0 (socket says 172.21.15.167:46500) 2024-05-11T12:12:44.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.373+0000 7f8b4a000640 1 -- 172.21.15.167:0/873304492 learned_addr learned my addr 172.21.15.167:0/873304492 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:44.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.373+0000 7f8b52a00640 1 --2- 172.21.15.167:0/873304492 >> v2:172.21.15.204:3300/0 conn(0x7f8b540718e0 0x7f8b54160e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:44.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.373+0000 7f8b53400640 1 --2- 172.21.15.167:0/873304492 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b541573a0 0x7f8b54157830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:44.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.374+0000 7f8b52000640 1 -- 172.21.15.167:0/873304492 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1394916266 0 0) 0x7f8b54161390 con 0x7f8b54071cc0 2024-05-11T12:12:44.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.374+0000 7f8b52000640 1 -- 172.21.15.167:0/873304492 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8b38003520 con 0x7f8b54071cc0 2024-05-11T12:12:44.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.374+0000 7f8b53400640 1 -- 172.21.15.167:0/873304492 >> v2:172.21.15.204:3300/0 conn(0x7f8b540718e0 msgr2=0x7f8b54160e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:44.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.374+0000 7f8b53400640 1 --2- 172.21.15.167:0/873304492 >> v2:172.21.15.204:3300/0 conn(0x7f8b540718e0 0x7f8b54160e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:44.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.374+0000 7f8b53400640 1 -- 172.21.15.167:0/873304492 >> v1:172.21.15.204:6789/0 conn(0x7f8b54071cc0 legacy=0x7f8b540cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:44.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.374+0000 7f8b53400640 1 -- 172.21.15.167:0/873304492 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b540d0ec0 con 0x7f8b541573a0 2024-05-11T12:12:44.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.376+0000 7f8b53400640 1 --2- 172.21.15.167:0/873304492 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b541573a0 0x7f8b54157830 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c0097d0 tx=0x7f8b4c02a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af474cbc26e3236 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:44.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.376+0000 7f8b52000640 1 -- 172.21.15.167:0/873304492 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c037070 con 0x7f8b541573a0 2024-05-11T12:12:44.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.376+0000 7f8b52000640 1 -- 172.21.15.167:0/873304492 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b4c032440 con 0x7f8b541573a0 2024-05-11T12:12:44.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.376+0000 7f8b52000640 1 -- 172.21.15.167:0/873304492 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c032810 con 0x7f8b541573a0 2024-05-11T12:12:44.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.377+0000 7f8b59a00640 1 -- 172.21.15.167:0/873304492 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b541573a0 msgr2=0x7f8b54157830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:44.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.377+0000 7f8b59a00640 1 --2- 172.21.15.167:0/873304492 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b541573a0 0x7f8b54157830 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c0097d0 tx=0x7f8b4c02a1c0 comp rx=0 tx=0).stop 2024-05-11T12:12:44.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.377+0000 7f8b59a00640 1 -- 172.21.15.167:0/873304492 shutdown_connections 2024-05-11T12:12:44.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.377+0000 7f8b59a00640 1 --2- 172.21.15.167:0/873304492 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b541573a0 0x7f8b54157830 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:44.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.377+0000 7f8b59a00640 1 --2- 172.21.15.167:0/873304492 >> v2:172.21.15.204:3300/0 conn(0x7f8b540718e0 0x7f8b54160e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:44.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.377+0000 7f8b59a00640 1 -- 172.21.15.167:0/873304492 >> 172.21.15.167:0/873304492 conn(0x7f8b54072730 msgr2=0x7f8b540d2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:44.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.377+0000 7f8b59a00640 1 -- 172.21.15.167:0/873304492 shutdown_connections 2024-05-11T12:12:44.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.377+0000 7f8b59a00640 1 -- 172.21.15.167:0/873304492 wait complete. 2024-05-11T12:12:44.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.379+0000 7f8b59a00640 1 Processor -- start 2024-05-11T12:12:44.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.379+0000 7f8b59a00640 1 -- start start 2024-05-11T12:12:44.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.380+0000 7f8b59a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b540718e0 0x7f8b54148130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:44.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.380+0000 7f8b59a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b54071cc0 0x7f8b541497b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:44.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.380+0000 7f8b59a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b541573a0 0x7f8b5415dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:44.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.380+0000 7f8b59a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8b540d1760 con 0x7f8b540718e0 2024-05-11T12:12:44.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.380+0000 7f8b59a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8b540d1480 con 0x7f8b541573a0 2024-05-11T12:12:44.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.380+0000 7f8b59a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f8b540d1a40 con 0x7f8b54071cc0 2024-05-11T12:12:44.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.380+0000 7f8b52a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b540718e0 0x7f8b54148130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:44.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.380+0000 7f8b52a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b540718e0 0x7f8b54148130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45706/0 (socket says 172.21.15.167:45706) 2024-05-11T12:12:44.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.380+0000 7f8b52a00640 1 -- 172.21.15.167:0/1636464632 learned_addr learned my addr 172.21.15.167:0/1636464632 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:44.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.381+0000 7f8b4a000640 1 --2- 172.21.15.167:0/1636464632 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b54071cc0 0x7f8b541497b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:44.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.381+0000 7f8b52a00640 1 -- 172.21.15.167:0/1636464632 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b54071cc0 msgr2=0x7f8b541497b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:44.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.381+0000 7f8b52a00640 1 --2- 172.21.15.167:0/1636464632 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b54071cc0 0x7f8b541497b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:44.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.381+0000 7f8b52a00640 1 -- 172.21.15.167:0/1636464632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b541573a0 msgr2=0x7f8b5415dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:44.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.381+0000 7f8b53400640 1 --2- 172.21.15.167:0/1636464632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b541573a0 0x7f8b5415dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:44.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.381+0000 7f8b52a00640 1 --2- 172.21.15.167:0/1636464632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b541573a0 0x7f8b5415dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:44.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.381+0000 7f8b52a00640 1 -- 172.21.15.167:0/1636464632 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b54148710 con 0x7f8b540718e0 2024-05-11T12:12:44.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.381+0000 7f8b4a000640 1 --2- 172.21.15.167:0/1636464632 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b54071cc0 0x7f8b541497b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:44.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.381+0000 7f8b53400640 1 --2- 172.21.15.167:0/1636464632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b541573a0 0x7f8b5415dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:44.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.381+0000 7f8b52a00640 1 --2- 172.21.15.167:0/1636464632 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b540718e0 0x7f8b54148130 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8b4400efc0 tx=0x7f8b4400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:44.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.382+0000 7f8b4b400640 1 -- 172.21.15.167:0/1636464632 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b44018020 con 0x7f8b540718e0 2024-05-11T12:12:44.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.382+0000 7f8b4b400640 1 -- 172.21.15.167:0/1636464632 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b44014480 con 0x7f8b540718e0 2024-05-11T12:12:44.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.382+0000 7f8b59a00640 1 -- 172.21.15.167:0/1636464632 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b541489f0 con 0x7f8b540718e0 2024-05-11T12:12:44.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.382+0000 7f8b4b400640 1 -- 172.21.15.167:0/1636464632 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b44014850 con 0x7f8b540718e0 2024-05-11T12:12:44.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.382+0000 7f8b59a00640 1 -- 172.21.15.167:0/1636464632 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b54148fb0 con 0x7f8b540718e0 2024-05-11T12:12:44.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.383+0000 7f8b59a00640 1 -- 172.21.15.167:0/1636464632 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b54069590 con 0x7f8b540718e0 2024-05-11T12:12:44.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.384+0000 7f8b4b400640 1 -- 172.21.15.167:0/1636464632 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8b44005d10 con 0x7f8b540718e0 2024-05-11T12:12:44.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.386+0000 7f8b4b400640 1 --2- 172.21.15.167:0/1636464632 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f8b300762d0 0x7f8b30078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:44.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.386+0000 7f8b4b400640 1 -- 172.21.15.167:0/1636464632 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f8b440983c0 con 0x7f8b540718e0 2024-05-11T12:12:44.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.386+0000 7f8b4a000640 1 --2- 172.21.15.167:0/1636464632 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f8b300762d0 0x7f8b30078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:44.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.387+0000 7f8b4a000640 1 --2- 172.21.15.167:0/1636464632 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f8b300762d0 0x7f8b30078790 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c001220 tx=0x7f8b3c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:44.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.391+0000 7f8b4b400640 1 -- 172.21.15.167:0/1636464632 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8b44022070 con 0x7f8b540718e0 2024-05-11T12:12:44.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.640+0000 7f8b59a00640 1 -- 172.21.15.167:0/1636464632 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8b540d25e0 con 0x7f8b540718e0 2024-05-11T12:12:44.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.641+0000 7f8b4b400640 1 -- 172.21.15.167:0/1636464632 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7f8b44010040 con 0x7f8b540718e0 2024-05-11T12:12:44.640 INFO:teuthology.orchestra.run.smithi167.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-11T12:12:44.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.644+0000 7f8b59a00640 1 -- 172.21.15.167:0/1636464632 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f8b300762d0 msgr2=0x7f8b30078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:44.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.644+0000 7f8b59a00640 1 --2- 172.21.15.167:0/1636464632 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f8b300762d0 0x7f8b30078790 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c001220 tx=0x7f8b3c009040 comp rx=0 tx=0).stop 2024-05-11T12:12:44.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.644+0000 7f8b59a00640 1 -- 172.21.15.167:0/1636464632 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b540718e0 msgr2=0x7f8b54148130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:44.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.644+0000 7f8b59a00640 1 --2- 172.21.15.167:0/1636464632 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b540718e0 0x7f8b54148130 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8b4400efc0 tx=0x7f8b4400c490 comp rx=0 tx=0).stop 2024-05-11T12:12:44.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.644+0000 7f8b59a00640 1 -- 172.21.15.167:0/1636464632 shutdown_connections 2024-05-11T12:12:44.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.644+0000 7f8b59a00640 1 --2- 172.21.15.167:0/1636464632 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f8b300762d0 0x7f8b30078790 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:44.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.644+0000 7f8b59a00640 1 --2- 172.21.15.167:0/1636464632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b541573a0 0x7f8b5415dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:44.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.644+0000 7f8b59a00640 1 --2- 172.21.15.167:0/1636464632 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b54071cc0 0x7f8b541497b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:44.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.644+0000 7f8b59a00640 1 --2- 172.21.15.167:0/1636464632 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b540718e0 0x7f8b54148130 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:44.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.644+0000 7f8b59a00640 1 -- 172.21.15.167:0/1636464632 >> 172.21.15.167:0/1636464632 conn(0x7f8b54072730 msgr2=0x7f8b540b36e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:44.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.644+0000 7f8b59a00640 1 -- 172.21.15.167:0/1636464632 shutdown_connections 2024-05-11T12:12:44.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:44.645+0000 7f8b59a00640 1 -- 172.21.15.167:0/1636464632 wait complete. 2024-05-11T12:12:44.663 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-11T12:12:44.663 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-11T12:12:44.663 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi204.front.sepia.ceph.com 2024-05-11T12:12:44.663 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-11T12:12:44.663 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-11T12:12:44.663 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-11T12:12:44.663 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-11T12:12:44.663 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-11T12:12:44.663 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-11T12:12:44.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:12:44.664 DEBUG:teuthology.orchestra.run.smithi204:> ip addr 2024-05-11T12:12:44.721 INFO:teuthology.orchestra.run.smithi204.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-11T12:12:44.721 INFO:teuthology.orchestra.run.smithi204.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-11T12:12:44.721 INFO:teuthology.orchestra.run.smithi204.stdout: inet 127.0.0.1/8 scope host lo 2024-05-11T12:12:44.721 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:12:44.722 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 ::1/128 scope host 2024-05-11T12:12:44.722 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:12:44.722 INFO:teuthology.orchestra.run.smithi204.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:12:44.722 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2a brd ff:ff:ff:ff:ff:ff 2024-05-11T12:12:44.722 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f0 2024-05-11T12:12:44.722 INFO:teuthology.orchestra.run.smithi204.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:12:44.722 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2b brd ff:ff:ff:ff:ff:ff 2024-05-11T12:12:44.722 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f1 2024-05-11T12:12:44.722 INFO:teuthology.orchestra.run.smithi204.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:12:44.722 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:aa brd ff:ff:ff:ff:ff:ff 2024-05-11T12:12:44.722 INFO:teuthology.orchestra.run.smithi204.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-11T12:12:44.722 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:ab brd ff:ff:ff:ff:ff:ff 2024-05-11T12:12:44.723 INFO:teuthology.orchestra.run.smithi204.stdout: inet 172.21.15.204/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-11T12:12:44.723 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft 42500sec preferred_lft 42500sec 2024-05-11T12:12:44.723 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::ec4:7aff:fe88:77ab/64 scope link 2024-05-11T12:12:44.723 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:12:44.723 INFO:teuthology.orchestra.run.smithi204.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-11T12:12:44.723 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether e2:9e:5b:47:45:5d brd ff:ff:ff:ff:ff:ff 2024-05-11T12:12:44.723 INFO:teuthology.orchestra.run.smithi204.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-11T12:12:44.723 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:12:44.723 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::a498:7aff:fe50:48c3/64 scope link 2024-05-11T12:12:44.723 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:12:44.723 INFO:teuthology.orchestra.run.smithi204.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-11T12:12:44.723 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether fa:0a:7f:e4:30:35 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-11T12:12:44.723 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::f80a:7fff:fee4:3035/64 scope link 2024-05-11T12:12:44.723 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:12:44.723 INFO:teuthology.orchestra.run.smithi204.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-11T12:12:44.724 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether e2:9e:5b:47:45:5d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-11T12:12:44.724 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::e09e:5bff:fe47:455d/64 scope link 2024-05-11T12:12:44.724 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:12:44.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:12:44.725 DEBUG:teuthology.orchestra.run.smithi204:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-11T12:12:44.802 INFO:teuthology.orchestra.run.smithi204.stdout:1 2024-05-11T12:12:44.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:12:44.806 DEBUG:teuthology.orchestra.run.smithi204:> route 2024-05-11T12:12:44.857 INFO:teuthology.orchestra.run.smithi204.stdout:Kernel IP routing table 2024-05-11T12:12:44.857 INFO:teuthology.orchestra.run.smithi204.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-11T12:12:44.857 INFO:teuthology.orchestra.run.smithi204.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-11T12:12:44.857 INFO:teuthology.orchestra.run.smithi204.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-11T12:12:44.857 INFO:teuthology.orchestra.run.smithi204.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-11T12:12:44.857 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T12:12:44.857 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-05-11T12:12:44.858 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-11T12:12:44.858 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-11T12:12:44.858 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-11T12:12:44.858 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-05-11T12:12:44.963 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:12:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T12:12:45.056 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:12:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:12:45.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:12:45.065 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list 2024-05-11T12:12:45.118 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-11T12:12:45.118 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-11T12:12:45.119 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-11T12:12:45.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:12:45.119 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link set brx.0 up 2024-05-11T12:12:45.202 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T12:12:45.202 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-11T12:12:45.253 INFO:teuthology.orchestra.run.smithi204.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-11T12:12:45.253 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T12:12:45.253 DEBUG:teuthology.orchestra.run.smithi204:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-11T12:12:45.299 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-05-11T12:12:45.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:12:45.300 DEBUG:teuthology.orchestra.run.smithi204:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:12:45.474 INFO:teuthology.orchestra.run.smithi204.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:12:45.474 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-11T12:12:45.510 INFO:teuthology.orchestra.run.smithi204.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:12:45.510 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-11T12:12:45.510 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: resolved to: "172.21.15.167:3300,172.21.15.204:3300,172.21.15.204:3301" 2024-05-11T12:12:45.510 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: trying mount with new device syntax: 1@5f32d344-7289-4c27-881b-1533419ab678.cephfs=/ 2024-05-11T12:12:45.510 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.167:3300/172.21.15.204:3300/172.21.15.204:3301" will pass to kernel 2024-05-11T12:12:45.512 INFO:teuthology.orchestra.run.smithi204.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-05-11T12:12:45.512 INFO:teuthology.orchestra.run.smithi204.stdout: You just mounted a file system that supports labels which does not 2024-05-11T12:12:45.512 INFO:teuthology.orchestra.run.smithi204.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-11T12:12:45.512 INFO:teuthology.orchestra.run.smithi204.stdout: applications will generate AVC messages and not be allowed access to 2024-05-11T12:12:45.512 INFO:teuthology.orchestra.run.smithi204.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-11T12:12:45.517 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-11T12:12:45.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:12:45.517 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-11T12:12:45.591 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-11T12:12:45.695 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:12:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-11T12:12:45.702 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:12:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:12:45.711 INFO:teuthology.orchestra.run.smithi204.stdout:client4694 2024-05-11T12:12:45.711 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:12:45.711 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:12:45.759 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-11T12:12:45.759 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-11T12:12:45.887 DEBUG:teuthology.orchestra.run.smithi204:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-11T12:12:46.003 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-11T12:12:46.003 INFO:tasks.cephfs.mount:Terminating background process 2024-05-11T12:12:46.003 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:12:46.003 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:12:46.061 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:12:46.062 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:12:46.114 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-11T12:12:46.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.316+0000 7f3841000640 1 Processor -- start 2024-05-11T12:12:46.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.317+0000 7f3841000640 1 -- start start 2024-05-11T12:12:46.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.318+0000 7f3841000640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f383c073170 0x7f383c0b6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:46.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.318+0000 7f3841000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f383c070050 0x7f383c0ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:46.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.318+0000 7f3841000640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f383c156140 con 0x7f383c06f420 2024-05-11T12:12:46.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.318+0000 7f3841000640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f383c0b6e80 con 0x7f383c073170 2024-05-11T12:12:46.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.318+0000 7f3841000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f383c15bc60 con 0x7f383c070050 2024-05-11T12:12:46.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.319+0000 7f3839e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f383c06f420 0x7f383c072a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:58150/0 (socket says 172.21.15.167:58150) 2024-05-11T12:12:46.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.319+0000 7f3839e00640 1 -- 172.21.15.167:0/2245917265 learned_addr learned my addr 172.21.15.167:0/2245917265 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:46.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.319+0000 7f3839400640 1 --2- 172.21.15.167:0/2245917265 >> v2:172.21.15.167:3300/0 conn(0x7f383c073170 0x7f383c0b6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:46.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.319+0000 7f383a800640 1 --2- 172.21.15.167:0/2245917265 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f383c070050 0x7f383c0ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:46.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.319+0000 7f3838a00640 1 -- 172.21.15.167:0/2245917265 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 911138152 0 0) 0x7f382c001110 con 0x7f383c06f420 2024-05-11T12:12:46.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.319+0000 7f3838a00640 1 -- 172.21.15.167:0/2245917265 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3820005730 con 0x7f383c06f420 2024-05-11T12:12:46.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.319+0000 7f3839400640 1 -- 172.21.15.167:0/2245917265 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f383c070050 msgr2=0x7f383c0ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:46.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.319+0000 7f3839400640 1 --2- 172.21.15.167:0/2245917265 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f383c070050 0x7f383c0ceaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:46.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.319+0000 7f3839400640 1 -- 172.21.15.167:0/2245917265 >> v1:172.21.15.167:6789/0 conn(0x7f383c06f420 legacy=0x7f383c072a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:46.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.319+0000 7f3839400640 1 -- 172.21.15.167:0/2245917265 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f383c15bf20 con 0x7f383c073170 2024-05-11T12:12:46.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.320+0000 7f3839400640 1 --2- 172.21.15.167:0/2245917265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f383c073170 0x7f383c0b6940 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3824009dc0 tx=0x7f382402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93fa52b25eafa26e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:46.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.321+0000 7f3838a00640 1 -- 172.21.15.167:0/2245917265 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382402ae60 con 0x7f383c073170 2024-05-11T12:12:46.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.321+0000 7f3838a00640 1 -- 172.21.15.167:0/2245917265 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3824007740 con 0x7f383c073170 2024-05-11T12:12:46.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.321+0000 7f3838a00640 1 -- 172.21.15.167:0/2245917265 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3824007af0 con 0x7f383c073170 2024-05-11T12:12:46.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.321+0000 7f3841000640 1 -- 172.21.15.167:0/2245917265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f383c073170 msgr2=0x7f383c0b6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:46.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.321+0000 7f3841000640 1 --2- 172.21.15.167:0/2245917265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f383c073170 0x7f383c0b6940 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3824009dc0 tx=0x7f382402a680 comp rx=0 tx=0).stop 2024-05-11T12:12:46.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.321+0000 7f3841000640 1 -- 172.21.15.167:0/2245917265 shutdown_connections 2024-05-11T12:12:46.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.321+0000 7f3841000640 1 --2- 172.21.15.167:0/2245917265 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f383c070050 0x7f383c0ceaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:46.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.321+0000 7f3841000640 1 --2- 172.21.15.167:0/2245917265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f383c073170 0x7f383c0b6940 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:46.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.321+0000 7f3841000640 1 -- 172.21.15.167:0/2245917265 >> 172.21.15.167:0/2245917265 conn(0x7f383c06c450 msgr2=0x7f383c06c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:46.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.321+0000 7f3841000640 1 -- 172.21.15.167:0/2245917265 shutdown_connections 2024-05-11T12:12:46.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.321+0000 7f3841000640 1 -- 172.21.15.167:0/2245917265 wait complete. 2024-05-11T12:12:46.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.323+0000 7f3841000640 1 Processor -- start 2024-05-11T12:12:46.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.323+0000 7f3841000640 1 -- start start 2024-05-11T12:12:46.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.324+0000 7f3841000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f383c06f420 0x7f383c14f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:46.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.324+0000 7f3841000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f383c070050 0x7f383c153770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:46.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.324+0000 7f3841000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f383c073170 0x7f383c0cb120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:46.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.324+0000 7f3841000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f383c15c7c0 con 0x7f383c070050 2024-05-11T12:12:46.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.324+0000 7f3841000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f383c15c4e0 con 0x7f383c073170 2024-05-11T12:12:46.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.324+0000 7f3841000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f383c06e3c0 con 0x7f383c06f420 2024-05-11T12:12:46.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.324+0000 7f3839400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f383c070050 0x7f383c153770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:46.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.325+0000 7f3839400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f383c070050 0x7f383c153770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45730/0 (socket says 172.21.15.167:45730) 2024-05-11T12:12:46.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.325+0000 7f3839400640 1 -- 172.21.15.167:0/664931795 learned_addr learned my addr 172.21.15.167:0/664931795 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:46.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.325+0000 7f3839400640 1 -- 172.21.15.167:0/664931795 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f383c06f420 msgr2=0x7f383c14f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:46.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.325+0000 7f3839400640 1 --2- 172.21.15.167:0/664931795 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f383c06f420 0x7f383c14f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:46.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.325+0000 7f3839400640 1 -- 172.21.15.167:0/664931795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f383c073170 msgr2=0x7f383c0cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:46.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.325+0000 7f3839400640 1 --2- 172.21.15.167:0/664931795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f383c073170 0x7f383c0cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:46.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.325+0000 7f3839400640 1 -- 172.21.15.167:0/664931795 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f383c14fd70 con 0x7f383c070050 2024-05-11T12:12:46.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.325+0000 7f3839400640 1 --2- 172.21.15.167:0/664931795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f383c070050 0x7f383c153770 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f3824009ee0 tx=0x7f3824004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:46.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.326+0000 7f3832000640 1 -- 172.21.15.167:0/664931795 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382403f070 con 0x7f383c070050 2024-05-11T12:12:46.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.326+0000 7f3832000640 1 -- 172.21.15.167:0/664931795 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3824007e40 con 0x7f383c070050 2024-05-11T12:12:46.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.326+0000 7f3832000640 1 -- 172.21.15.167:0/664931795 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3824009450 con 0x7f383c070050 2024-05-11T12:12:46.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.326+0000 7f3841000640 1 -- 172.21.15.167:0/664931795 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f383c154070 con 0x7f383c070050 2024-05-11T12:12:46.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.326+0000 7f3841000640 1 -- 172.21.15.167:0/664931795 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f383c154630 con 0x7f383c070050 2024-05-11T12:12:46.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.327+0000 7f3841000640 1 -- 172.21.15.167:0/664931795 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f383c0694b0 con 0x7f383c070050 2024-05-11T12:12:46.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.328+0000 7f3832000640 1 -- 172.21.15.167:0/664931795 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f382403b410 con 0x7f383c070050 2024-05-11T12:12:46.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.329+0000 7f3832000640 1 --2- 172.21.15.167:0/664931795 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f38180762d0 0x7f3818078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:46.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.330+0000 7f3832000640 1 -- 172.21.15.167:0/664931795 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f38240b89a0 con 0x7f383c070050 2024-05-11T12:12:46.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.330+0000 7f3839e00640 1 --2- 172.21.15.167:0/664931795 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f38180762d0 0x7f3818078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:46.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.330+0000 7f3839e00640 1 --2- 172.21.15.167:0/664931795 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f38180762d0 0x7f3818078790 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f382c0013f0 tx=0x7f382c001240 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:46.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.335+0000 7f3832000640 1 -- 172.21.15.167:0/664931795 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f383c14fd70 con 0x7f383c070050 2024-05-11T12:12:46.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.557+0000 7f3841000640 1 -- 172.21.15.167:0/664931795 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f383c06fb70 con 0x7f383c070050 2024-05-11T12:12:46.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.558+0000 7f3832000640 1 -- 172.21.15.167:0/664931795 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v53) ==== 80+0+0 (secure 0 0 0) 0x7f3824082160 con 0x7f383c070050 2024-05-11T12:12:46.557 INFO:teuthology.orchestra.run.smithi167.stderr:listed 0 entries 2024-05-11T12:12:46.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.560+0000 7f3841000640 1 -- 172.21.15.167:0/664931795 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f38180762d0 msgr2=0x7f3818078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:46.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.560+0000 7f3841000640 1 --2- 172.21.15.167:0/664931795 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f38180762d0 0x7f3818078790 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f382c0013f0 tx=0x7f382c001240 comp rx=0 tx=0).stop 2024-05-11T12:12:46.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.560+0000 7f3841000640 1 -- 172.21.15.167:0/664931795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f383c070050 msgr2=0x7f383c153770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:46.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.560+0000 7f3841000640 1 --2- 172.21.15.167:0/664931795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f383c070050 0x7f383c153770 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f3824009ee0 tx=0x7f3824004a40 comp rx=0 tx=0).stop 2024-05-11T12:12:46.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.560+0000 7f3841000640 1 -- 172.21.15.167:0/664931795 shutdown_connections 2024-05-11T12:12:46.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.560+0000 7f3841000640 1 --2- 172.21.15.167:0/664931795 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f38180762d0 0x7f3818078790 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:46.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.560+0000 7f3841000640 1 --2- 172.21.15.167:0/664931795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f383c073170 0x7f383c0cb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:46.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.560+0000 7f3841000640 1 --2- 172.21.15.167:0/664931795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f383c070050 0x7f383c153770 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:46.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.560+0000 7f3841000640 1 --2- 172.21.15.167:0/664931795 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f383c06f420 0x7f383c14f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:46.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.560+0000 7f3841000640 1 -- 172.21.15.167:0/664931795 >> 172.21.15.167:0/664931795 conn(0x7f383c06c450 msgr2=0x7f383c159980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:46.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.561+0000 7f3841000640 1 -- 172.21.15.167:0/664931795 shutdown_connections 2024-05-11T12:12:46.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:46.561+0000 7f3841000640 1 -- 172.21.15.167:0/664931795 wait complete. 2024-05-11T12:12:46.578 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-11T12:12:46.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:12:46.579 DEBUG:teuthology.orchestra.run.smithi167:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-11T12:12:46.660 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2024-05-11T12:12:46.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:12:46.661 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-11T12:12:46.707 INFO:tasks.cephfs.mount:Terminating background process 2024-05-11T12:12:46.707 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:12:46.707 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:12:46.765 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:12:46.765 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:12:46.819 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-11T12:12:47.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.032+0000 7f5963000640 1 Processor -- start 2024-05-11T12:12:47.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.032+0000 7f5963000640 1 -- start start 2024-05-11T12:12:47.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.034+0000 7f5963000640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f595c15f290 0x7f595c06e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:47.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.034+0000 7f5963000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f595c0d1070 0x7f595c0ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:47.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.034+0000 7f5963000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f595c0708a0 0x7f595c070d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:47.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.034+0000 7f5963000640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f595c06df40 con 0x7f595c15f290 2024-05-11T12:12:47.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.034+0000 7f5963000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f595c0ce560 con 0x7f595c0708a0 2024-05-11T12:12:47.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.034+0000 7f5963000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f595c06d930 con 0x7f595c0d1070 2024-05-11T12:12:47.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.034+0000 7f595be00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f595c15f290 0x7f595c06e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:47.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.034+0000 7f595be00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f595c15f290 0x7f595c06e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45742/0 (socket says 172.21.15.167:45742) 2024-05-11T12:12:47.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.034+0000 7f595be00640 1 -- 172.21.15.167:0/2382178779 learned_addr learned my addr 172.21.15.167:0/2382178779 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:47.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.035+0000 7f595be00640 1 -- 172.21.15.167:0/2382178779 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f595c0d1070 msgr2=0x7f595c0ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:47.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.035+0000 7f595be00640 1 --2- 172.21.15.167:0/2382178779 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f595c0d1070 0x7f595c0ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:47.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.035+0000 7f595be00640 1 -- 172.21.15.167:0/2382178779 >> v2:172.21.15.204:3300/0 conn(0x7f595c0708a0 msgr2=0x7f595c070d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:47.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.035+0000 7f595be00640 1 --2- 172.21.15.167:0/2382178779 >> v2:172.21.15.204:3300/0 conn(0x7f595c0708a0 0x7f595c070d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:47.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.035+0000 7f595be00640 1 -- 172.21.15.167:0/2382178779 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f595c0ce7e0 con 0x7f595c15f290 2024-05-11T12:12:47.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.036+0000 7f595be00640 1 --2- 172.21.15.167:0/2382178779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f595c15f290 0x7f595c06e3d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5954009870 tx=0x7f595402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1acad17388097ae server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:47.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.036+0000 7f595aa00640 1 -- 172.21.15.167:0/2382178779 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5954037070 con 0x7f595c15f290 2024-05-11T12:12:47.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.036+0000 7f595aa00640 1 -- 172.21.15.167:0/2382178779 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5954032440 con 0x7f595c15f290 2024-05-11T12:12:47.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.036+0000 7f595aa00640 1 -- 172.21.15.167:0/2382178779 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5954032810 con 0x7f595c15f290 2024-05-11T12:12:47.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.037+0000 7f5963000640 1 -- 172.21.15.167:0/2382178779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f595c15f290 msgr2=0x7f595c06e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:47.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.037+0000 7f5963000640 1 --2- 172.21.15.167:0/2382178779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f595c15f290 0x7f595c06e3d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5954009870 tx=0x7f595402a210 comp rx=0 tx=0).stop 2024-05-11T12:12:47.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.037+0000 7f5963000640 1 -- 172.21.15.167:0/2382178779 shutdown_connections 2024-05-11T12:12:47.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.037+0000 7f5963000640 1 --2- 172.21.15.167:0/2382178779 >> v2:172.21.15.204:3300/0 conn(0x7f595c0708a0 0x7f595c070d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:47.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.037+0000 7f5963000640 1 --2- 172.21.15.167:0/2382178779 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f595c0d1070 0x7f595c0ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:47.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.037+0000 7f5963000640 1 --2- 172.21.15.167:0/2382178779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f595c15f290 0x7f595c06e3d0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:47.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.037+0000 7f5963000640 1 -- 172.21.15.167:0/2382178779 >> 172.21.15.167:0/2382178779 conn(0x7f595c06fb60 msgr2=0x7f595c070190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:47.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.037+0000 7f5963000640 1 -- 172.21.15.167:0/2382178779 shutdown_connections 2024-05-11T12:12:47.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.037+0000 7f5963000640 1 -- 172.21.15.167:0/2382178779 wait complete. 2024-05-11T12:12:47.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.039+0000 7f5963000640 1 Processor -- start 2024-05-11T12:12:47.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.039+0000 7f5963000640 1 -- start start 2024-05-11T12:12:47.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.040+0000 7f5963000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f595c0708a0 0x7f595c1f8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:47.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.040+0000 7f5963000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f595c0d1070 0x7f595c156ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:47.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.040+0000 7f5963000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f595c15f290 0x7f595c1f8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:47.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.040+0000 7f5963000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f595c15f670 con 0x7f595c0708a0 2024-05-11T12:12:47.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.040+0000 7f5963000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f595c06dbd0 con 0x7f595c0d1070 2024-05-11T12:12:47.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.040+0000 7f5963000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f595c06c450 con 0x7f595c15f290 2024-05-11T12:12:47.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.040+0000 7f595be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f595c0708a0 0x7f595c1f8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:47.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.040+0000 7f595be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f595c0708a0 0x7f595c1f8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45750/0 (socket says 172.21.15.167:45750) 2024-05-11T12:12:47.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.040+0000 7f595be00640 1 -- 172.21.15.167:0/3265908834 learned_addr learned my addr 172.21.15.167:0/3265908834 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:47.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.041+0000 7f595be00640 1 -- 172.21.15.167:0/3265908834 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f595c15f290 msgr2=0x7f595c1f8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:47.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.041+0000 7f595be00640 1 --2- 172.21.15.167:0/3265908834 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f595c15f290 0x7f595c1f8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:47.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.041+0000 7f595be00640 1 -- 172.21.15.167:0/3265908834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f595c0d1070 msgr2=0x7f595c156ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:47.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.041+0000 7f595be00640 1 --2- 172.21.15.167:0/3265908834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f595c0d1070 0x7f595c156ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:47.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.041+0000 7f595be00640 1 -- 172.21.15.167:0/3265908834 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f595c157580 con 0x7f595c0708a0 2024-05-11T12:12:47.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.041+0000 7f595be00640 1 --2- 172.21.15.167:0/3265908834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f595c0708a0 0x7f595c1f8090 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5954009870 tx=0x7f59540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:47.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.041+0000 7f5953e00640 1 -- 172.21.15.167:0/3265908834 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5954041070 con 0x7f595c0708a0 2024-05-11T12:12:47.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.042+0000 7f5953e00640 1 -- 172.21.15.167:0/3265908834 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f595403c870 con 0x7f595c0708a0 2024-05-11T12:12:47.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.042+0000 7f5953e00640 1 -- 172.21.15.167:0/3265908834 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5954037040 con 0x7f595c0708a0 2024-05-11T12:12:47.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.042+0000 7f5963000640 1 -- 172.21.15.167:0/3265908834 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f595c157860 con 0x7f595c0708a0 2024-05-11T12:12:47.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.042+0000 7f5963000640 1 -- 172.21.15.167:0/3265908834 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f595c157e20 con 0x7f595c0708a0 2024-05-11T12:12:47.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.043+0000 7f5963000640 1 -- 172.21.15.167:0/3265908834 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f595c0694b0 con 0x7f595c0708a0 2024-05-11T12:12:47.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.044+0000 7f5953e00640 1 -- 172.21.15.167:0/3265908834 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5954044070 con 0x7f595c0708a0 2024-05-11T12:12:47.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.046+0000 7f5953e00640 1 --2- 172.21.15.167:0/3265908834 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f59380762d0 0x7f5938078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:47.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.046+0000 7f5953e00640 1 -- 172.21.15.167:0/3265908834 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f59540b8300 con 0x7f595c0708a0 2024-05-11T12:12:47.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.046+0000 7f595b400640 1 --2- 172.21.15.167:0/3265908834 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f59380762d0 0x7f5938078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:47.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.047+0000 7f595b400640 1 --2- 172.21.15.167:0/3265908834 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f59380762d0 0x7f5938078790 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f595c0cce10 tx=0x7f5944006c90 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:47.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.051+0000 7f5953e00640 1 -- 172.21.15.167:0/3265908834 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5954081a30 con 0x7f595c0708a0 2024-05-11T12:12:47.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.265+0000 7f5963000640 1 -- 172.21.15.167:0/3265908834 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f595c161020 con 0x7f595c0708a0 2024-05-11T12:12:47.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.266+0000 7f5953e00640 1 -- 172.21.15.167:0/3265908834 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v53) ==== 80+0+0 (secure 0 0 0) 0x7f5954081330 con 0x7f595c0708a0 2024-05-11T12:12:47.266 INFO:teuthology.orchestra.run.smithi167.stderr:listed 0 entries 2024-05-11T12:12:47.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.268+0000 7f5963000640 1 -- 172.21.15.167:0/3265908834 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f59380762d0 msgr2=0x7f5938078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:47.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.269+0000 7f5963000640 1 --2- 172.21.15.167:0/3265908834 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f59380762d0 0x7f5938078790 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f595c0cce10 tx=0x7f5944006c90 comp rx=0 tx=0).stop 2024-05-11T12:12:47.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.269+0000 7f5963000640 1 -- 172.21.15.167:0/3265908834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f595c0708a0 msgr2=0x7f595c1f8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:47.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.269+0000 7f5963000640 1 --2- 172.21.15.167:0/3265908834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f595c0708a0 0x7f595c1f8090 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5954009870 tx=0x7f59540046c0 comp rx=0 tx=0).stop 2024-05-11T12:12:47.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.269+0000 7f5963000640 1 -- 172.21.15.167:0/3265908834 shutdown_connections 2024-05-11T12:12:47.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.269+0000 7f5963000640 1 --2- 172.21.15.167:0/3265908834 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f595c15f290 0x7f595c1f8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:47.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.269+0000 7f5963000640 1 --2- 172.21.15.167:0/3265908834 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f59380762d0 0x7f5938078790 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:47.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.269+0000 7f5963000640 1 --2- 172.21.15.167:0/3265908834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f595c0d1070 0x7f595c156ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:47.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.269+0000 7f5963000640 1 --2- 172.21.15.167:0/3265908834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f595c0708a0 0x7f595c1f8090 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:47.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.269+0000 7f5963000640 1 -- 172.21.15.167:0/3265908834 >> 172.21.15.167:0/3265908834 conn(0x7f595c06fb60 msgr2=0x7f595c15fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:47.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.269+0000 7f5963000640 1 -- 172.21.15.167:0/3265908834 shutdown_connections 2024-05-11T12:12:47.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.269+0000 7f5963000640 1 -- 172.21.15.167:0/3265908834 wait complete. 2024-05-11T12:12:47.288 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-11T12:12:47.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:12:47.288 DEBUG:teuthology.orchestra.run.smithi204:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-05-11T12:12:47.385 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi204.front.sepia.ceph.com 2024-05-11T12:12:47.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:12:47.385 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-11T12:12:47.436 INFO:tasks.cephfs.mount:Terminating background process 2024-05-11T12:12:47.436 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:12:47.436 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:12:47.491 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-11T12:12:47.492 INFO:tasks.cephfs.mount:Terminating background process 2024-05-11T12:12:47.492 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:12:47.492 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:12:47.549 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052713,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-11T12:12:47.549 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:12:47.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.752+0000 7f388e200640 1 Processor -- start 2024-05-11T12:12:47.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.753+0000 7f388e200640 1 -- start start 2024-05-11T12:12:47.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.754+0000 7f388e200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f388815f620 0x7f3888163d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:47.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.754+0000 7f388e200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f38880d18d0 0x7f38880cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:47.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.754+0000 7f388e200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f38880cdb50 0x7f38880b7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:47.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.754+0000 7f388e200640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f38880cdfc0 con 0x7f38880cdb50 2024-05-11T12:12:47.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.754+0000 7f388e200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f3888163870 con 0x7f38880d18d0 2024-05-11T12:12:47.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.754+0000 7f388e200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f388815fa00 con 0x7f388815f620 2024-05-11T12:12:47.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.754+0000 7f3887e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f38880cdb50 0x7f38880b7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:47.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.755+0000 7f3887e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f38880cdb50 0x7f38880b7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45754/0 (socket says 172.21.15.167:45754) 2024-05-11T12:12:47.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.755+0000 7f3887e00640 1 -- 172.21.15.167:0/2497819326 learned_addr learned my addr 172.21.15.167:0/2497819326 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:47.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.755+0000 7f3887400640 1 --2- 172.21.15.167:0/2497819326 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f388815f620 0x7f3888163d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:47.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.755+0000 7f3887e00640 1 -- 172.21.15.167:0/2497819326 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f388815f620 msgr2=0x7f3888163d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:47.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.755+0000 7f387fe00640 1 --2- 172.21.15.167:0/2497819326 >> v2:172.21.15.204:3300/0 conn(0x7f38880d18d0 0x7f38880cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:47.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.755+0000 7f3887e00640 1 --2- 172.21.15.167:0/2497819326 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f388815f620 0x7f3888163d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:47.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.755+0000 7f3887e00640 1 -- 172.21.15.167:0/2497819326 >> v2:172.21.15.204:3300/0 conn(0x7f38880d18d0 msgr2=0x7f38880cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:47.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.755+0000 7f3887e00640 1 --2- 172.21.15.167:0/2497819326 >> v2:172.21.15.204:3300/0 conn(0x7f38880d18d0 0x7f38880cd610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:47.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.755+0000 7f3887e00640 1 -- 172.21.15.167:0/2497819326 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38880b6200 con 0x7f38880cdb50 2024-05-11T12:12:47.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.755+0000 7f3887400640 1 --2- 172.21.15.167:0/2497819326 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f388815f620 0x7f3888163d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:47.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.756+0000 7f3887e00640 1 --2- 172.21.15.167:0/2497819326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38880cdb50 0x7f38880b7ad0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f38880d0630 tx=0x7f388002a600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b6f33fa75bee5c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:47.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.756+0000 7f3886a00640 1 -- 172.21.15.167:0/2497819326 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3880036070 con 0x7f38880cdb50 2024-05-11T12:12:47.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.756+0000 7f3886a00640 1 -- 172.21.15.167:0/2497819326 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3880031600 con 0x7f38880cdb50 2024-05-11T12:12:47.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.756+0000 7f3886a00640 1 -- 172.21.15.167:0/2497819326 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38800319d0 con 0x7f38880cdb50 2024-05-11T12:12:47.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.757+0000 7f388e200640 1 -- 172.21.15.167:0/2497819326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38880cdb50 msgr2=0x7f38880b7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:47.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.757+0000 7f388e200640 1 --2- 172.21.15.167:0/2497819326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38880cdb50 0x7f38880b7ad0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f38880d0630 tx=0x7f388002a600 comp rx=0 tx=0).stop 2024-05-11T12:12:47.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.757+0000 7f388e200640 1 -- 172.21.15.167:0/2497819326 shutdown_connections 2024-05-11T12:12:47.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.757+0000 7f388e200640 1 --2- 172.21.15.167:0/2497819326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38880cdb50 0x7f38880b7ad0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:47.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.757+0000 7f388e200640 1 --2- 172.21.15.167:0/2497819326 >> v2:172.21.15.204:3300/0 conn(0x7f38880d18d0 0x7f38880cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:47.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.757+0000 7f388e200640 1 --2- 172.21.15.167:0/2497819326 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f388815f620 0x7f3888163d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:47.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.757+0000 7f388e200640 1 -- 172.21.15.167:0/2497819326 >> 172.21.15.167:0/2497819326 conn(0x7f3888072730 msgr2=0x7f38880d2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:47.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.757+0000 7f388e200640 1 -- 172.21.15.167:0/2497819326 shutdown_connections 2024-05-11T12:12:47.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.757+0000 7f388e200640 1 -- 172.21.15.167:0/2497819326 wait complete. 2024-05-11T12:12:47.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.759+0000 7f388e200640 1 Processor -- start 2024-05-11T12:12:47.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.759+0000 7f388e200640 1 -- start start 2024-05-11T12:12:47.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.760+0000 7f388e200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38880cdb50 0x7f38880c7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:47.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.760+0000 7f388e200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38880d18d0 0x7f38881524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:47.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.760+0000 7f388e200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f388815f620 0x7f38880709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:47.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.760+0000 7f388e200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f38880d1d30 con 0x7f388815f620 2024-05-11T12:12:47.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.760+0000 7f388e200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3888163500 con 0x7f38880d18d0 2024-05-11T12:12:47.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.760+0000 7f388e200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f38880b5e90 con 0x7f38880cdb50 2024-05-11T12:12:47.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.760+0000 7f3887e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f388815f620 0x7f38880709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:47.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.760+0000 7f3887e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f388815f620 0x7f38880709b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45768/0 (socket says 172.21.15.167:45768) 2024-05-11T12:12:47.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.760+0000 7f3887e00640 1 -- 172.21.15.167:0/158009281 learned_addr learned my addr 172.21.15.167:0/158009281 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:47.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.761+0000 7f3887400640 1 --2- 172.21.15.167:0/158009281 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38880cdb50 0x7f38880c7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:47.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.761+0000 7f3887e00640 1 -- 172.21.15.167:0/158009281 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38880cdb50 msgr2=0x7f38880c7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:47.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.761+0000 7f3887e00640 1 --2- 172.21.15.167:0/158009281 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38880cdb50 0x7f38880c7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:47.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.761+0000 7f3887e00640 1 -- 172.21.15.167:0/158009281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38880d18d0 msgr2=0x7f38881524c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:47.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.761+0000 7f3887e00640 1 --2- 172.21.15.167:0/158009281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38880d18d0 0x7f38881524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:47.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.761+0000 7f3887e00640 1 -- 172.21.15.167:0/158009281 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38880711e0 con 0x7f388815f620 2024-05-11T12:12:47.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.761+0000 7f3887e00640 1 --2- 172.21.15.167:0/158009281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f388815f620 0x7f38880709b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3880009f40 tx=0x7f3880004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:47.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.761+0000 7f387f400640 1 -- 172.21.15.167:0/158009281 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3880036050 con 0x7f388815f620 2024-05-11T12:12:47.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.762+0000 7f387f400640 1 -- 172.21.15.167:0/158009281 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3880042070 con 0x7f388815f620 2024-05-11T12:12:47.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.762+0000 7f387f400640 1 -- 172.21.15.167:0/158009281 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3880009510 con 0x7f388815f620 2024-05-11T12:12:47.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.762+0000 7f388e200640 1 -- 172.21.15.167:0/158009281 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38880714c0 con 0x7f388815f620 2024-05-11T12:12:47.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.762+0000 7f388e200640 1 -- 172.21.15.167:0/158009281 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3888071a80 con 0x7f388815f620 2024-05-11T12:12:47.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.763+0000 7f388e200640 1 -- 172.21.15.167:0/158009281 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3888069590 con 0x7f388815f620 2024-05-11T12:12:47.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.764+0000 7f387f400640 1 -- 172.21.15.167:0/158009281 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3880007bd0 con 0x7f388815f620 2024-05-11T12:12:47.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.765+0000 7f387f400640 1 --2- 172.21.15.167:0/158009281 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f38640762d0 0x7f3864078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:47.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.765+0000 7f387f400640 1 -- 172.21.15.167:0/158009281 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f38800b8720 con 0x7f388815f620 2024-05-11T12:12:47.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.765+0000 7f3887400640 1 --2- 172.21.15.167:0/158009281 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f38640762d0 0x7f3864078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:47.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.766+0000 7f3887400640 1 --2- 172.21.15.167:0/158009281 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f38640762d0 0x7f3864078790 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f387800b840 tx=0x7f3878008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:47.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.769+0000 7f387f400640 1 -- 172.21.15.167:0/158009281 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f388002ae20 con 0x7f388815f620 2024-05-11T12:12:47.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.998+0000 7f388e200640 1 -- 172.21.15.167:0/158009281 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f38880c68d0 con 0x7f388815f620 2024-05-11T12:12:48.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:47.999+0000 7f387f400640 1 -- 172.21.15.167:0/158009281 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5193 (secure 0 0 0) 0x7f3880081ee0 con 0x7f388815f620 2024-05-11T12:12:48.000 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:12:48.000 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4634,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2640837270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":2640837270},{"type":"v1","addr":"172.21.15.204:6835","nonce":2640837270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4807,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3516271493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3516271493},{"type":"v1","addr":"172.21.15.204:6834","nonce":3516271493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4816,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2070289869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2070289869},{"type":"v1","addr":"172.21.15.167:6837","nonce":2070289869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:12:35.314711+0000","modified":"2024-05-11T12:12:38.382446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6835/4016299199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4016299199},{"type":"v1","addr":"172.21.15.167:6835","nonce":4016299199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5310,"qdb_cluster":[5310]},"id":3}]} 2024-05-11T12:12:48.000 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 26 2024-05-11T12:12:48.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.002+0000 7f388e200640 1 -- 172.21.15.167:0/158009281 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f38640762d0 msgr2=0x7f3864078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:48.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.002+0000 7f388e200640 1 --2- 172.21.15.167:0/158009281 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f38640762d0 0x7f3864078790 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f387800b840 tx=0x7f3878008040 comp rx=0 tx=0).stop 2024-05-11T12:12:48.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.002+0000 7f388e200640 1 -- 172.21.15.167:0/158009281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f388815f620 msgr2=0x7f38880709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:48.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.002+0000 7f388e200640 1 --2- 172.21.15.167:0/158009281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f388815f620 0x7f38880709b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3880009f40 tx=0x7f3880004a40 comp rx=0 tx=0).stop 2024-05-11T12:12:48.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.002+0000 7f388e200640 1 -- 172.21.15.167:0/158009281 shutdown_connections 2024-05-11T12:12:48.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.002+0000 7f388e200640 1 --2- 172.21.15.167:0/158009281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f388815f620 0x7f38880709b0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:48.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.002+0000 7f388e200640 1 --2- 172.21.15.167:0/158009281 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f38640762d0 0x7f3864078790 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:48.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.002+0000 7f388e200640 1 --2- 172.21.15.167:0/158009281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38880d18d0 0x7f38881524c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:48.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.002+0000 7f388e200640 1 --2- 172.21.15.167:0/158009281 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38880cdb50 0x7f38880c7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:48.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.002+0000 7f388e200640 1 -- 172.21.15.167:0/158009281 >> 172.21.15.167:0/158009281 conn(0x7f3888072730 msgr2=0x7f38880cfb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:48.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.003+0000 7f388e200640 1 -- 172.21.15.167:0/158009281 shutdown_connections 2024-05-11T12:12:48.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.003+0000 7f388e200640 1 -- 172.21.15.167:0/158009281 wait complete. 2024-05-11T12:12:48.021 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:12:48.219 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.219+0000 7f376b000640 1 Processor -- start 2024-05-11T12:12:48.219 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.219+0000 7f376b000640 1 -- start start 2024-05-11T12:12:48.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.221+0000 7f376b000640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3764157750 0x7f376415d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:48.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.221+0000 7f376b000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f376406d460 0x7f376406d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:48.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.221+0000 7f376b000640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f376415dc80 con 0x7f37640b3ea0 2024-05-11T12:12:48.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.221+0000 7f376b000640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f376406de10 con 0x7f3764157750 2024-05-11T12:12:48.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.221+0000 7f376b000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3764161000 con 0x7f376406d460 2024-05-11T12:12:48.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.221+0000 7f3763400640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f37640b3ea0 0x7f3764161490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:58156/0 (socket says 172.21.15.167:58156) 2024-05-11T12:12:48.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.221+0000 7f3763e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3764157750 0x7f376415d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:48.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.221+0000 7f3763400640 1 -- 172.21.15.167:0/534205273 learned_addr learned my addr 172.21.15.167:0/534205273 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:48.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.222+0000 7f3769000640 1 --2- 172.21.15.167:0/534205273 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f376406d460 0x7f376406d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:48.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.222+0000 7f3763e00640 1 -- 172.21.15.167:0/534205273 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f376406d460 msgr2=0x7f376406d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:48.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.222+0000 7f3763e00640 1 --2- 172.21.15.167:0/534205273 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f376406d460 0x7f376406d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:48.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.222+0000 7f3763e00640 1 -- 172.21.15.167:0/534205273 >> v1:172.21.15.167:6789/0 conn(0x7f37640b3ea0 legacy=0x7f3764161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:48.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.222+0000 7f3763e00640 1 -- 172.21.15.167:0/534205273 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376415df30 con 0x7f3764157750 2024-05-11T12:12:48.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.223+0000 7f3763e00640 1 --2- 172.21.15.167:0/534205273 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3764157750 0x7f376415d740 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f37540097a0 tx=0x7f375402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30ecf61de6713428 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:48.224 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.223+0000 7f3762a00640 1 -- 172.21.15.167:0/534205273 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3754037070 con 0x7f3764157750 2024-05-11T12:12:48.224 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.223+0000 7f3762a00640 1 -- 172.21.15.167:0/534205273 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3754032440 con 0x7f3764157750 2024-05-11T12:12:48.224 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.223+0000 7f3762a00640 1 -- 172.21.15.167:0/534205273 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3754032810 con 0x7f3764157750 2024-05-11T12:12:48.224 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.224+0000 7f376b000640 1 -- 172.21.15.167:0/534205273 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3764157750 msgr2=0x7f376415d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:48.224 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.224+0000 7f376b000640 1 --2- 172.21.15.167:0/534205273 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3764157750 0x7f376415d740 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f37540097a0 tx=0x7f375402a180 comp rx=0 tx=0).stop 2024-05-11T12:12:48.224 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.224+0000 7f376b000640 1 -- 172.21.15.167:0/534205273 shutdown_connections 2024-05-11T12:12:48.224 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.224+0000 7f376b000640 1 --2- 172.21.15.167:0/534205273 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f376406d460 0x7f376406d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:48.224 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.224+0000 7f376b000640 1 --2- 172.21.15.167:0/534205273 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3764157750 0x7f376415d740 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:48.224 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.224+0000 7f376b000640 1 -- 172.21.15.167:0/534205273 >> 172.21.15.167:0/534205273 conn(0x7f37640cd8f0 msgr2=0x7f37640cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:48.224 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.224+0000 7f376b000640 1 -- 172.21.15.167:0/534205273 shutdown_connections 2024-05-11T12:12:48.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.224+0000 7f376b000640 1 -- 172.21.15.167:0/534205273 wait complete. 2024-05-11T12:12:48.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.225+0000 7f376b000640 1 Processor -- start 2024-05-11T12:12:48.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.226+0000 7f376b000640 1 -- start start 2024-05-11T12:12:48.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.226+0000 7f376b000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f376406d460 0x7f37640b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:48.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.226+0000 7f376b000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37640b3ea0 0x7f376415c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:48.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.226+0000 7f376b000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3764157750 0x7f376415c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:48.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.226+0000 7f376b000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3764157b30 con 0x7f3764157750 2024-05-11T12:12:48.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.226+0000 7f376b000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f376415e4f0 con 0x7f376406d460 2024-05-11T12:12:48.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.226+0000 7f376b000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3764161ba0 con 0x7f37640b3ea0 2024-05-11T12:12:48.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.227+0000 7f3769000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3764157750 0x7f376415c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:48.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.227+0000 7f3769000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3764157750 0x7f376415c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45784/0 (socket says 172.21.15.167:45784) 2024-05-11T12:12:48.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.227+0000 7f3769000640 1 -- 172.21.15.167:0/4058354875 learned_addr learned my addr 172.21.15.167:0/4058354875 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:48.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.227+0000 7f3763400640 1 --2- 172.21.15.167:0/4058354875 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37640b3ea0 0x7f376415c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:48.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.227+0000 7f3769000640 1 -- 172.21.15.167:0/4058354875 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37640b3ea0 msgr2=0x7f376415c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:48.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.227+0000 7f3769000640 1 --2- 172.21.15.167:0/4058354875 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37640b3ea0 0x7f376415c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:48.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.227+0000 7f3769000640 1 -- 172.21.15.167:0/4058354875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f376406d460 msgr2=0x7f37640b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:48.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.227+0000 7f3769000640 1 --2- 172.21.15.167:0/4058354875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f376406d460 0x7f37640b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:48.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.227+0000 7f3769000640 1 -- 172.21.15.167:0/4058354875 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376413aa00 con 0x7f3764157750 2024-05-11T12:12:48.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.228+0000 7f3769000640 1 --2- 172.21.15.167:0/4058354875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3764157750 0x7f376415c690 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f375c007e70 tx=0x7f375c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:48.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.228+0000 7f375be00640 1 -- 172.21.15.167:0/4058354875 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c014620 con 0x7f3764157750 2024-05-11T12:12:48.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.228+0000 7f375be00640 1 -- 172.21.15.167:0/4058354875 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f375c019070 con 0x7f3764157750 2024-05-11T12:12:48.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.228+0000 7f375be00640 1 -- 172.21.15.167:0/4058354875 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c014a20 con 0x7f3764157750 2024-05-11T12:12:48.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.228+0000 7f376b000640 1 -- 172.21.15.167:0/4058354875 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f376413ace0 con 0x7f3764157750 2024-05-11T12:12:48.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.228+0000 7f376b000640 1 -- 172.21.15.167:0/4058354875 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37640ad520 con 0x7f3764157750 2024-05-11T12:12:48.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.229+0000 7f376b000640 1 -- 172.21.15.167:0/4058354875 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3764069590 con 0x7f3764157750 2024-05-11T12:12:48.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.230+0000 7f375be00640 1 -- 172.21.15.167:0/4058354875 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f375c014bc0 con 0x7f3764157750 2024-05-11T12:12:48.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.231+0000 7f375be00640 1 --2- 172.21.15.167:0/4058354875 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f37440762d0 0x7f3744078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:48.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.231+0000 7f375be00640 1 -- 172.21.15.167:0/4058354875 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f375c098130 con 0x7f3764157750 2024-05-11T12:12:48.232 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.231+0000 7f3763e00640 1 --2- 172.21.15.167:0/4058354875 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f37440762d0 0x7f3744078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:48.232 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.232+0000 7f3763e00640 1 --2- 172.21.15.167:0/4058354875 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f37440762d0 0x7f3744078790 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3754034000 tx=0x7f375402c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:48.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.235+0000 7f375be00640 1 -- 172.21.15.167:0/4058354875 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f375c061860 con 0x7f3764157750 2024-05-11T12:12:48.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.482+0000 7f376b000640 1 -- 172.21.15.167:0/4058354875 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f376406d960 con 0x7f3764157750 2024-05-11T12:12:48.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.483+0000 7f375be00640 1 -- 172.21.15.167:0/4058354875 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5193 (secure 0 0 0) 0x7f375c061160 con 0x7f3764157750 2024-05-11T12:12:48.483 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:12:48.483 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4634,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2640837270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":2640837270},{"type":"v1","addr":"172.21.15.204:6835","nonce":2640837270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4807,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3516271493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3516271493},{"type":"v1","addr":"172.21.15.204:6834","nonce":3516271493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4816,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2070289869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2070289869},{"type":"v1","addr":"172.21.15.167:6837","nonce":2070289869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:12:35.314711+0000","modified":"2024-05-11T12:12:38.382446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6835/4016299199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4016299199},{"type":"v1","addr":"172.21.15.167:6835","nonce":4016299199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5310,"qdb_cluster":[5310]},"id":3}]} 2024-05-11T12:12:48.484 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 26 2024-05-11T12:12:48.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.487+0000 7f376b000640 1 -- 172.21.15.167:0/4058354875 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f37440762d0 msgr2=0x7f3744078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:48.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.487+0000 7f376b000640 1 --2- 172.21.15.167:0/4058354875 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f37440762d0 0x7f3744078790 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3754034000 tx=0x7f375402c000 comp rx=0 tx=0).stop 2024-05-11T12:12:48.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.487+0000 7f376b000640 1 -- 172.21.15.167:0/4058354875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3764157750 msgr2=0x7f376415c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:48.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.487+0000 7f376b000640 1 --2- 172.21.15.167:0/4058354875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3764157750 0x7f376415c690 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f375c007e70 tx=0x7f375c00e4b0 comp rx=0 tx=0).stop 2024-05-11T12:12:48.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.487+0000 7f376b000640 1 -- 172.21.15.167:0/4058354875 shutdown_connections 2024-05-11T12:12:48.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.487+0000 7f376b000640 1 --2- 172.21.15.167:0/4058354875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3764157750 0x7f376415c690 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:48.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.487+0000 7f376b000640 1 --2- 172.21.15.167:0/4058354875 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37640b3ea0 0x7f376415c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:48.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.487+0000 7f376b000640 1 --2- 172.21.15.167:0/4058354875 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f37440762d0 0x7f3744078790 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:48.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.487+0000 7f376b000640 1 --2- 172.21.15.167:0/4058354875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f376406d460 0x7f37640b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:48.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.487+0000 7f376b000640 1 -- 172.21.15.167:0/4058354875 >> 172.21.15.167:0/4058354875 conn(0x7f37640cd8f0 msgr2=0x7f37640681c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:48.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.487+0000 7f376b000640 1 -- 172.21.15.167:0/4058354875 shutdown_connections 2024-05-11T12:12:48.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.487+0000 7f376b000640 1 -- 172.21.15.167:0/4058354875 wait complete. 2024-05-11T12:12:48.505 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:12:48.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.702+0000 7f6726800640 1 Processor -- start 2024-05-11T12:12:48.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.702+0000 7f6726800640 1 -- start start 2024-05-11T12:12:48.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.703+0000 7f6726800640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f67200ca930 0x7f672006ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:48.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.703+0000 7f6726800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f67200ced70 0x7f67200cfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:48.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.704+0000 7f6726800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6720163920 con 0x7f672015f120 2024-05-11T12:12:48.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.704+0000 7f6726800640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f67201636d0 con 0x7f67200ca930 2024-05-11T12:12:48.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.704+0000 7f6726800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f6720163c00 con 0x7f67200ced70 2024-05-11T12:12:48.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.704+0000 7f6724a00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f67200ca930 0x7f672006ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:48.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.704+0000 7f6724a00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f67200ca930 0x7f672006ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45800/0 (socket says 172.21.15.167:45800) 2024-05-11T12:12:48.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.704+0000 7f6724a00640 1 -- 172.21.15.167:0/1102753661 learned_addr learned my addr 172.21.15.167:0/1102753661 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:48.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.704+0000 7f6724a00640 1 -- 172.21.15.167:0/1102753661 >> v2:172.21.15.204:3300/0 conn(0x7f67200ced70 msgr2=0x7f67200cfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:48.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.704+0000 7f6724a00640 1 --2- 172.21.15.167:0/1102753661 >> v2:172.21.15.204:3300/0 conn(0x7f67200ced70 0x7f67200cfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:48.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.704+0000 7f6724a00640 1 -- 172.21.15.167:0/1102753661 >> v1:172.21.15.204:6789/0 conn(0x7f672015f120 legacy=0x7f67200ca220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:48.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.704+0000 7f6724a00640 1 -- 172.21.15.167:0/1102753661 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67200cf3f0 con 0x7f67200ca930 2024-05-11T12:12:48.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.705+0000 7f6724a00640 1 --2- 172.21.15.167:0/1102753661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67200ca930 0x7f672006ebb0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f67200ced00 tx=0x7f670c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=601638e8258cd5de server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:48.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.706+0000 7f671fe00640 1 -- 172.21.15.167:0/1102753661 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670c037070 con 0x7f67200ca930 2024-05-11T12:12:48.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.706+0000 7f671fe00640 1 -- 172.21.15.167:0/1102753661 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f670c032440 con 0x7f67200ca930 2024-05-11T12:12:48.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.706+0000 7f671fe00640 1 -- 172.21.15.167:0/1102753661 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670c032810 con 0x7f67200ca930 2024-05-11T12:12:48.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.706+0000 7f6726800640 1 -- 172.21.15.167:0/1102753661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67200ca930 msgr2=0x7f672006ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:48.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.706+0000 7f6726800640 1 --2- 172.21.15.167:0/1102753661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67200ca930 0x7f672006ebb0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f67200ced00 tx=0x7f670c02a140 comp rx=0 tx=0).stop 2024-05-11T12:12:48.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.706+0000 7f6726800640 1 -- 172.21.15.167:0/1102753661 shutdown_connections 2024-05-11T12:12:48.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.706+0000 7f6726800640 1 --2- 172.21.15.167:0/1102753661 >> v2:172.21.15.204:3300/0 conn(0x7f67200ced70 0x7f67200cfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:48.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.706+0000 7f6726800640 1 --2- 172.21.15.167:0/1102753661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67200ca930 0x7f672006ebb0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:48.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.706+0000 7f6726800640 1 -- 172.21.15.167:0/1102753661 >> 172.21.15.167:0/1102753661 conn(0x7f672006e090 msgr2=0x7f672006e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:48.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.706+0000 7f6726800640 1 -- 172.21.15.167:0/1102753661 shutdown_connections 2024-05-11T12:12:48.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.707+0000 7f6726800640 1 -- 172.21.15.167:0/1102753661 wait complete. 2024-05-11T12:12:48.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.708+0000 7f6726800640 1 Processor -- start 2024-05-11T12:12:48.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.708+0000 7f6726800640 1 -- start start 2024-05-11T12:12:48.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.709+0000 7f6726800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67200ca930 0x7f672015b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:48.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.709+0000 7f6726800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67200ced70 0x7f6720152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:48.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.709+0000 7f6726800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f672015f120 0x7f6720152f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:48.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.709+0000 7f6726800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f672015f500 con 0x7f672015f120 2024-05-11T12:12:48.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.709+0000 7f6726800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f67200cf9b0 con 0x7f67200ced70 2024-05-11T12:12:48.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.709+0000 7f6726800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f67200cbf20 con 0x7f67200ca930 2024-05-11T12:12:48.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.709+0000 7f6725e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f672015f120 0x7f6720152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:48.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.710+0000 7f6725e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f672015f120 0x7f6720152f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45816/0 (socket says 172.21.15.167:45816) 2024-05-11T12:12:48.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.710+0000 7f6725e00640 1 -- 172.21.15.167:0/3887904918 learned_addr learned my addr 172.21.15.167:0/3887904918 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:48.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.710+0000 7f6725400640 1 --2- 172.21.15.167:0/3887904918 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67200ca930 0x7f672015b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:48.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.710+0000 7f6724a00640 1 --2- 172.21.15.167:0/3887904918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67200ced70 0x7f6720152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:48.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.710+0000 7f6725e00640 1 -- 172.21.15.167:0/3887904918 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67200ca930 msgr2=0x7f672015b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:48.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.710+0000 7f6725e00640 1 --2- 172.21.15.167:0/3887904918 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67200ca930 0x7f672015b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:48.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.710+0000 7f6725e00640 1 -- 172.21.15.167:0/3887904918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67200ced70 msgr2=0x7f6720152a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:48.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.710+0000 7f6725e00640 1 --2- 172.21.15.167:0/3887904918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67200ced70 0x7f6720152a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:48.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.710+0000 7f6725e00640 1 -- 172.21.15.167:0/3887904918 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67200b5310 con 0x7f672015f120 2024-05-11T12:12:48.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.710+0000 7f6725400640 1 --2- 172.21.15.167:0/3887904918 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67200ca930 0x7f672015b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:48.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.710+0000 7f6725e00640 1 --2- 172.21.15.167:0/3887904918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f672015f120 0x7f6720152f80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f671800b9a0 tx=0x7f671800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:48.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.711+0000 7f671d600640 1 -- 172.21.15.167:0/3887904918 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6718019070 con 0x7f672015f120 2024-05-11T12:12:48.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.711+0000 7f671d600640 1 -- 172.21.15.167:0/3887904918 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f671800cbc0 con 0x7f672015f120 2024-05-11T12:12:48.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.711+0000 7f671d600640 1 -- 172.21.15.167:0/3887904918 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6718014440 con 0x7f672015f120 2024-05-11T12:12:48.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.711+0000 7f6726800640 1 -- 172.21.15.167:0/3887904918 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67200b55f0 con 0x7f672015f120 2024-05-11T12:12:48.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.711+0000 7f6726800640 1 -- 172.21.15.167:0/3887904918 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67200b5bb0 con 0x7f672015f120 2024-05-11T12:12:48.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.712+0000 7f6726800640 1 -- 172.21.15.167:0/3887904918 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67200cad10 con 0x7f672015f120 2024-05-11T12:12:48.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.713+0000 7f671d600640 1 -- 172.21.15.167:0/3887904918 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f671801c070 con 0x7f672015f120 2024-05-11T12:12:48.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.714+0000 7f671d600640 1 --2- 172.21.15.167:0/3887904918 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f67000762d0 0x7f6700078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:48.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.714+0000 7f671d600640 1 -- 172.21.15.167:0/3887904918 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f6718097560 con 0x7f672015f120 2024-05-11T12:12:48.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.715+0000 7f6725400640 1 --2- 172.21.15.167:0/3887904918 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f67000762d0 0x7f6700078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:48.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.715+0000 7f6725400640 1 --2- 172.21.15.167:0/3887904918 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f67000762d0 0x7f6700078790 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f671400a810 tx=0x7f6714008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:48.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.719+0000 7f671d600640 1 -- 172.21.15.167:0/3887904918 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f67200cad10 con 0x7f672015f120 2024-05-11T12:12:48.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.927+0000 7f6726800640 1 -- 172.21.15.167:0/3887904918 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f67200694b0 con 0x7f672015f120 2024-05-11T12:12:48.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.930+0000 7f671d600640 1 -- 172.21.15.167:0/3887904918 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) ==== 74+0+16990 (secure 0 0 0) 0x7f6718060c90 con 0x7f672015f120 2024-05-11T12:12:48.931 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:12:48.931 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":53,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:12:36.879705+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:12:33.261425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-11T12:12:34.290603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:12:48.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.933+0000 7f6726800640 1 -- 172.21.15.167:0/3887904918 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f67000762d0 msgr2=0x7f6700078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:48.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.933+0000 7f6726800640 1 --2- 172.21.15.167:0/3887904918 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f67000762d0 0x7f6700078790 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f671400a810 tx=0x7f6714008040 comp rx=0 tx=0).stop 2024-05-11T12:12:48.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.933+0000 7f6726800640 1 -- 172.21.15.167:0/3887904918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f672015f120 msgr2=0x7f6720152f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:48.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.933+0000 7f6726800640 1 --2- 172.21.15.167:0/3887904918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f672015f120 0x7f6720152f80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f671800b9a0 tx=0x7f671800be60 comp rx=0 tx=0).stop 2024-05-11T12:12:48.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.933+0000 7f6726800640 1 -- 172.21.15.167:0/3887904918 shutdown_connections 2024-05-11T12:12:48.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.933+0000 7f6726800640 1 --2- 172.21.15.167:0/3887904918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f672015f120 0x7f6720152f80 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:48.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.933+0000 7f6726800640 1 --2- 172.21.15.167:0/3887904918 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f67000762d0 0x7f6700078790 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:48.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.933+0000 7f6726800640 1 --2- 172.21.15.167:0/3887904918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67200ced70 0x7f6720152a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:48.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.933+0000 7f6726800640 1 --2- 172.21.15.167:0/3887904918 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67200ca930 0x7f672015b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:48.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.933+0000 7f6726800640 1 -- 172.21.15.167:0/3887904918 >> 172.21.15.167:0/3887904918 conn(0x7f672006e090 msgr2=0x7f67200b0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:48.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.933+0000 7f6726800640 1 -- 172.21.15.167:0/3887904918 shutdown_connections 2024-05-11T12:12:48.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:48.933+0000 7f6726800640 1 -- 172.21.15.167:0/3887904918 wait complete. 2024-05-11T12:12:48.953 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-11T12:12:48.953 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:12:49.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.155+0000 7f7fa6600640 1 Processor -- start 2024-05-11T12:12:49.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.155+0000 7f7fa6600640 1 -- start start 2024-05-11T12:12:49.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.156+0000 7f7fa6600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7fa00b3ea0 0x7f7fa0161490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:49.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.156+0000 7f7fa6600640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fa015df30 con 0x7f7fa01619d0 2024-05-11T12:12:49.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.156+0000 7f7fa6600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fa015e210 con 0x7f7fa0157750 2024-05-11T12:12:49.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.156+0000 7f7fa6600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f7fa0161000 con 0x7f7fa00b3ea0 2024-05-11T12:12:49.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.156+0000 7f7f9fe00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f7fa01619d0 0x7f7fa006d460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49294/0 (socket says 172.21.15.167:49294) 2024-05-11T12:12:49.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.157+0000 7f7f9fe00640 1 -- 172.21.15.167:0/223687764 learned_addr learned my addr 172.21.15.167:0/223687764 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:49.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.157+0000 7f7f9ea00640 1 --2- 172.21.15.167:0/223687764 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7fa00b3ea0 0x7f7fa0161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:49.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.157+0000 7f7f9e000640 1 -- 172.21.15.167:0/223687764 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2531439399 0 0) 0x7f7fa015df30 con 0x7f7fa01619d0 2024-05-11T12:12:49.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.157+0000 7f7f9e000640 1 -- 172.21.15.167:0/223687764 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f84003520 con 0x7f7fa01619d0 2024-05-11T12:12:49.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.157+0000 7f7f9e000640 1 -- 172.21.15.167:0/223687764 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1981794021 0 0) 0x7f7f84003520 con 0x7f7fa01619d0 2024-05-11T12:12:49.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.157+0000 7f7f9e000640 1 -- 172.21.15.167:0/223687764 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7fa00b3ea0 msgr2=0x7f7fa0161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:49.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.157+0000 7f7f9e000640 1 --2- 172.21.15.167:0/223687764 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7fa00b3ea0 0x7f7fa0161490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:49.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.157+0000 7f7f9e000640 1 -- 172.21.15.167:0/223687764 >> v1:172.21.15.204:6789/0 conn(0x7f7fa0157750 legacy=0x7f7fa015d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:49.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.157+0000 7f7f9e000640 1 -- 172.21.15.167:0/223687764 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fa015e4f0 con 0x7f7fa01619d0 2024-05-11T12:12:49.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.157+0000 7f7f9e000640 1 -- 172.21.15.167:0/223687764 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f7f980016f0 con 0x7f7fa01619d0 2024-05-11T12:12:49.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.157+0000 7f7f9e000640 1 -- 172.21.15.167:0/223687764 >> v1:172.21.15.167:6789/0 conn(0x7f7fa01619d0 legacy=0x7f7fa006d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:49.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.157+0000 7f7f9e000640 1 --2- 172.21.15.167:0/223687764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f84003a30 0x7f7f84003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:49.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.158+0000 7f7f9e000640 1 --2- 172.21.15.167:0/223687764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f84004800 0x7f7f84004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:49.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.158+0000 7f7f9e000640 1 --2- 172.21.15.167:0/223687764 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7fa00b3ea0 0x7f7f84005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:49.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.158+0000 7f7f9e000640 1 -- 172.21.15.167:0/223687764 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa0161000 con 0x7f7f84003a30 2024-05-11T12:12:49.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.158+0000 7f7f9e000640 1 -- 172.21.15.167:0/223687764 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7f8400baf0 con 0x7f7f84004800 2024-05-11T12:12:49.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.158+0000 7f7f9e000640 1 -- 172.21.15.167:0/223687764 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f7f8400bcf0 con 0x7f7fa00b3ea0 2024-05-11T12:12:49.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.158+0000 7f7f9f400640 1 --2- 172.21.15.167:0/223687764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f84003a30 0x7f7f84003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:49.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.158+0000 7f7f9f400640 1 -- 172.21.15.167:0/223687764 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7fa00b3ea0 msgr2=0x7f7f84005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:49.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.158+0000 7f7f9fe00640 1 --2- 172.21.15.167:0/223687764 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7fa00b3ea0 0x7f7f84005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:49.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.158+0000 7f7f9ea00640 1 --2- 172.21.15.167:0/223687764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f84004800 0x7f7f84004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:49.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.158+0000 7f7f9f400640 1 --2- 172.21.15.167:0/223687764 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7fa00b3ea0 0x7f7f84005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:49.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.158+0000 7f7f9f400640 1 -- 172.21.15.167:0/223687764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f84004800 msgr2=0x7f7f84004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:49.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.158+0000 7f7f9f400640 1 --2- 172.21.15.167:0/223687764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f84004800 0x7f7f84004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:49.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.158+0000 7f7f9f400640 1 -- 172.21.15.167:0/223687764 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f84003520 con 0x7f7f84003a30 2024-05-11T12:12:49.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.158+0000 7f7f9fe00640 1 --2- 172.21.15.167:0/223687764 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7fa00b3ea0 0x7f7f84005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:49.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.159+0000 7f7f9f400640 1 --2- 172.21.15.167:0/223687764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f84003a30 0x7f7f84003e50 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7f88001790 tx=0x7f7f84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f1d3d6c14e3bf70 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:49.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.160+0000 7f7f9e000640 1 -- 172.21.15.167:0/223687764 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f880013b0 con 0x7f7f84003a30 2024-05-11T12:12:49.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.160+0000 7f7f9e000640 1 -- 172.21.15.167:0/223687764 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f880052d0 con 0x7f7f84003a30 2024-05-11T12:12:49.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.160+0000 7f7f9e000640 1 -- 172.21.15.167:0/223687764 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f8802aae0 con 0x7f7f84003a30 2024-05-11T12:12:49.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.160+0000 7f7fa6600640 1 -- 172.21.15.167:0/223687764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f84003a30 msgr2=0x7f7f84003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:49.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.160+0000 7f7fa6600640 1 --2- 172.21.15.167:0/223687764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f84003a30 0x7f7f84003e50 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7f88001790 tx=0x7f7f84005830 comp rx=0 tx=0).stop 2024-05-11T12:12:49.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.160+0000 7f7f9fe00640 1 -- 172.21.15.167:0/223687764 reap_dead start 2024-05-11T12:12:49.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.160+0000 7f7fa6600640 1 -- 172.21.15.167:0/223687764 shutdown_connections 2024-05-11T12:12:49.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.160+0000 7f7fa6600640 1 -- 172.21.15.167:0/223687764 >> 172.21.15.167:0/223687764 conn(0x7f7fa00cd8f0 msgr2=0x7f7fa00cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:49.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.160+0000 7f7fa6600640 1 -- 172.21.15.167:0/223687764 shutdown_connections 2024-05-11T12:12:49.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.160+0000 7f7fa6600640 1 -- 172.21.15.167:0/223687764 wait complete. 2024-05-11T12:12:49.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.162+0000 7f7fa6600640 1 Processor -- start 2024-05-11T12:12:49.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.162+0000 7f7fa6600640 1 -- start start 2024-05-11T12:12:49.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.163+0000 7f7fa6600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f84003a30 0x7f7fa0146510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:49.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.163+0000 7f7fa6600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7fa01f9580 0x7f7fa013b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:49.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.163+0000 7f7fa6600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fa006cf00 0x7f7fa006d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:49.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.163+0000 7f7fa6600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa006dcf0 con 0x7f7fa006cf00 2024-05-11T12:12:49.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.163+0000 7f7fa6600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa006db70 con 0x7f7f84003a30 2024-05-11T12:12:49.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.163+0000 7f7fa6600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f7fa0161490 con 0x7f7fa01f9580 2024-05-11T12:12:49.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.164+0000 7f7f9fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fa006cf00 0x7f7fa006d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:49.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.164+0000 7f7f9fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fa006cf00 0x7f7fa006d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55968/0 (socket says 172.21.15.167:55968) 2024-05-11T12:12:49.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.164+0000 7f7f9fe00640 1 -- 172.21.15.167:0/355274039 learned_addr learned my addr 172.21.15.167:0/355274039 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:49.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.164+0000 7f7f9ea00640 1 --2- 172.21.15.167:0/355274039 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7fa01f9580 0x7f7fa013b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:49.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.164+0000 7f7f9f400640 1 --2- 172.21.15.167:0/355274039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f84003a30 0x7f7fa0146510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:49.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.164+0000 7f7f9fe00640 1 -- 172.21.15.167:0/355274039 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7fa01f9580 msgr2=0x7f7fa013b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:49.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.164+0000 7f7f9fe00640 1 --2- 172.21.15.167:0/355274039 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7fa01f9580 0x7f7fa013b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:49.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.164+0000 7f7f9fe00640 1 -- 172.21.15.167:0/355274039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f84003a30 msgr2=0x7f7fa0146510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:49.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.164+0000 7f7f9fe00640 1 --2- 172.21.15.167:0/355274039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f84003a30 0x7f7fa0146510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:49.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.164+0000 7f7f9fe00640 1 -- 172.21.15.167:0/355274039 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fa01598a0 con 0x7f7fa006cf00 2024-05-11T12:12:49.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.164+0000 7f7f9f400640 1 --2- 172.21.15.167:0/355274039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f84003a30 0x7f7fa0146510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:49.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.164+0000 7f7f9fe00640 1 --2- 172.21.15.167:0/355274039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fa006cf00 0x7f7fa006d3b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7fa0146f70 tx=0x7f7f9800d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:49.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.165+0000 7f7f97400640 1 -- 172.21.15.167:0/355274039 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f980075a0 con 0x7f7fa006cf00 2024-05-11T12:12:49.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.165+0000 7f7f97400640 1 -- 172.21.15.167:0/355274039 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f980047e0 con 0x7f7fa006cf00 2024-05-11T12:12:49.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.165+0000 7f7f97400640 1 -- 172.21.15.167:0/355274039 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f98004bd0 con 0x7f7fa006cf00 2024-05-11T12:12:49.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.165+0000 7f7fa6600640 1 -- 172.21.15.167:0/355274039 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fa013b540 con 0x7f7fa006cf00 2024-05-11T12:12:49.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.165+0000 7f7fa6600640 1 -- 172.21.15.167:0/355274039 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fa01f9a60 con 0x7f7fa006cf00 2024-05-11T12:12:49.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.166+0000 7f7fa6600640 1 -- 172.21.15.167:0/355274039 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fa0069590 con 0x7f7fa006cf00 2024-05-11T12:12:49.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.167+0000 7f7f97400640 1 -- 172.21.15.167:0/355274039 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7f9801b020 con 0x7f7fa006cf00 2024-05-11T12:12:49.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.169+0000 7f7f97400640 1 --2- 172.21.15.167:0/355274039 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f7f7c0762d0 0x7f7f7c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:49.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.169+0000 7f7f97400640 1 -- 172.21.15.167:0/355274039 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f7f98097350 con 0x7f7fa006cf00 2024-05-11T12:12:49.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.169+0000 7f7f9f400640 1 --2- 172.21.15.167:0/355274039 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f7f7c0762d0 0x7f7f7c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:49.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.170+0000 7f7f9f400640 1 --2- 172.21.15.167:0/355274039 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f7f7c0762d0 0x7f7f7c078790 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7f88032000 tx=0x7f7f8802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:49.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.173+0000 7f7f97400640 1 -- 172.21.15.167:0/355274039 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7f980604c0 con 0x7f7fa006cf00 2024-05-11T12:12:49.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.416+0000 7f7fa6600640 1 -- 172.21.15.167:0/355274039 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7fa01595e0 con 0x7f7fa006cf00 2024-05-11T12:12:49.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.417+0000 7f7f97400640 1 -- 172.21.15.167:0/355274039 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5193 (secure 0 0 0) 0x7f7f980647c0 con 0x7f7fa006cf00 2024-05-11T12:12:49.418 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:12:49.418 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4634,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2640837270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":2640837270},{"type":"v1","addr":"172.21.15.204:6835","nonce":2640837270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4807,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3516271493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3516271493},{"type":"v1","addr":"172.21.15.204:6834","nonce":3516271493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4816,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2070289869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2070289869},{"type":"v1","addr":"172.21.15.167:6837","nonce":2070289869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:12:35.314711+0000","modified":"2024-05-11T12:12:38.382446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6835/4016299199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4016299199},{"type":"v1","addr":"172.21.15.167:6835","nonce":4016299199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5310,"qdb_cluster":[5310]},"id":3}]} 2024-05-11T12:12:49.418 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 26 2024-05-11T12:12:49.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.420+0000 7f7fa6600640 1 -- 172.21.15.167:0/355274039 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f7f7c0762d0 msgr2=0x7f7f7c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:49.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.420+0000 7f7fa6600640 1 --2- 172.21.15.167:0/355274039 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f7f7c0762d0 0x7f7f7c078790 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7f88032000 tx=0x7f7f8802e000 comp rx=0 tx=0).stop 2024-05-11T12:12:49.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.420+0000 7f7fa6600640 1 -- 172.21.15.167:0/355274039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fa006cf00 msgr2=0x7f7fa006d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:49.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.420+0000 7f7fa6600640 1 --2- 172.21.15.167:0/355274039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fa006cf00 0x7f7fa006d3b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7fa0146f70 tx=0x7f7f9800d9d0 comp rx=0 tx=0).stop 2024-05-11T12:12:49.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.420+0000 7f7fa6600640 1 -- 172.21.15.167:0/355274039 shutdown_connections 2024-05-11T12:12:49.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.420+0000 7f7fa6600640 1 --2- 172.21.15.167:0/355274039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fa006cf00 0x7f7fa006d3b0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:49.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.420+0000 7f7fa6600640 1 --2- 172.21.15.167:0/355274039 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7fa01f9580 0x7f7fa013b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:49.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.420+0000 7f7fa6600640 1 --2- 172.21.15.167:0/355274039 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f7f7c0762d0 0x7f7f7c078790 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:49.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.420+0000 7f7fa6600640 1 --2- 172.21.15.167:0/355274039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f84003a30 0x7f7fa0146510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:49.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.420+0000 7f7fa6600640 1 -- 172.21.15.167:0/355274039 >> 172.21.15.167:0/355274039 conn(0x7f7fa00cd8f0 msgr2=0x7f7fa01e8b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:49.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.420+0000 7f7fa6600640 1 -- 172.21.15.167:0/355274039 shutdown_connections 2024-05-11T12:12:49.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.420+0000 7f7fa6600640 1 -- 172.21.15.167:0/355274039 wait complete. 2024-05-11T12:12:49.439 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:12:49.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.625+0000 7f4133e00640 1 Processor -- start 2024-05-11T12:12:49.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.625+0000 7f4133e00640 1 -- start start 2024-05-11T12:12:49.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.627+0000 7f4133e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41341632e0 0x7f41340d12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:49.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.627+0000 7f4133e00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4134156500 con 0x7f413406c660 2024-05-11T12:12:49.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.627+0000 7f4133e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41340d1810 con 0x7f4134156120 2024-05-11T12:12:49.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.627+0000 7f4133e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f413406d700 con 0x7f41341632e0 2024-05-11T12:12:49.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.627+0000 7f412be00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f413406c660 0x7f41340d2b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49310/0 (socket says 172.21.15.167:49310) 2024-05-11T12:12:49.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.627+0000 7f412be00640 1 -- 172.21.15.167:0/711350431 learned_addr learned my addr 172.21.15.167:0/711350431 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:49.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.628+0000 7f4133400640 1 --2- 172.21.15.167:0/711350431 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41341632e0 0x7f41340d12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:49.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.628+0000 7f4132000640 1 -- 172.21.15.167:0/711350431 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3559141349 0 0) 0x7f4134156500 con 0x7f413406c660 2024-05-11T12:12:49.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.628+0000 7f4132000640 1 -- 172.21.15.167:0/711350431 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4118003520 con 0x7f413406c660 2024-05-11T12:12:49.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.628+0000 7f4132000640 1 -- 172.21.15.167:0/711350431 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3291561533 0 0) 0x7f4118003520 con 0x7f413406c660 2024-05-11T12:12:49.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.628+0000 7f4132000640 1 -- 172.21.15.167:0/711350431 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41341632e0 msgr2=0x7f41340d12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:49.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.628+0000 7f4132000640 1 --2- 172.21.15.167:0/711350431 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41341632e0 0x7f41340d12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:49.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.628+0000 7f4132000640 1 -- 172.21.15.167:0/711350431 >> v1:172.21.15.204:6789/0 conn(0x7f4134156120 legacy=0x7f4134162bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:49.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.628+0000 7f4132000640 1 -- 172.21.15.167:0/711350431 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f413406caf0 con 0x7f413406c660 2024-05-11T12:12:49.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.628+0000 7f4132000640 1 -- 172.21.15.167:0/711350431 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f411c001710 con 0x7f413406c660 2024-05-11T12:12:49.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.628+0000 7f4132000640 1 -- 172.21.15.167:0/711350431 >> v1:172.21.15.167:6789/0 conn(0x7f413406c660 legacy=0x7f41340d2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:49.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.628+0000 7f4132000640 1 --2- 172.21.15.167:0/711350431 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41341632e0 0x7f4118003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:49.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.628+0000 7f4132000640 1 --2- 172.21.15.167:0/711350431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4118004000 0x7f4118004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:49.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.628+0000 7f4132000640 1 --2- 172.21.15.167:0/711350431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4118004d40 0x7f41180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:49.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.628+0000 7f4132000640 1 -- 172.21.15.167:0/711350431 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f413406d700 con 0x7f4118004d40 2024-05-11T12:12:49.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.628+0000 7f4132000640 1 -- 172.21.15.167:0/711350431 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f411800bb50 con 0x7f4118004000 2024-05-11T12:12:49.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.628+0000 7f4132000640 1 -- 172.21.15.167:0/711350431 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f411800bd50 con 0x7f41341632e0 2024-05-11T12:12:49.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.629+0000 7f4133400640 1 --2- 172.21.15.167:0/711350431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4118004d40 0x7f41180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:49.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.629+0000 7f4133400640 1 -- 172.21.15.167:0/711350431 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41341632e0 msgr2=0x7f4118003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:49.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.629+0000 7f4133400640 1 --2- 172.21.15.167:0/711350431 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41341632e0 0x7f4118003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:49.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.629+0000 7f4133400640 1 -- 172.21.15.167:0/711350431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4118004000 msgr2=0x7f4118004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:49.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.629+0000 7f4133400640 1 --2- 172.21.15.167:0/711350431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4118004000 0x7f4118004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:49.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.629+0000 7f4133400640 1 -- 172.21.15.167:0/711350431 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4118003520 con 0x7f4118004d40 2024-05-11T12:12:49.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.630+0000 7f4133400640 1 --2- 172.21.15.167:0/711350431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4118004d40 0x7f41180051c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f412c009f90 tx=0x7f4118005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce961ab38710f6c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:49.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.630+0000 7f4132000640 1 -- 172.21.15.167:0/711350431 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c009c10 con 0x7f4118004d40 2024-05-11T12:12:49.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.630+0000 7f4132000640 1 -- 172.21.15.167:0/711350431 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f412c036070 con 0x7f4118004d40 2024-05-11T12:12:49.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.630+0000 7f4132000640 1 -- 172.21.15.167:0/711350431 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c031880 con 0x7f4118004d40 2024-05-11T12:12:49.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.631+0000 7f4133e00640 1 -- 172.21.15.167:0/711350431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4118004d40 msgr2=0x7f41180051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:49.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.631+0000 7f4133e00640 1 --2- 172.21.15.167:0/711350431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4118004d40 0x7f41180051c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f412c009f90 tx=0x7f4118005830 comp rx=0 tx=0).stop 2024-05-11T12:12:49.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.631+0000 7f4133400640 1 -- 172.21.15.167:0/711350431 reap_dead start 2024-05-11T12:12:49.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.631+0000 7f4133e00640 1 -- 172.21.15.167:0/711350431 shutdown_connections 2024-05-11T12:12:49.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.631+0000 7f4133e00640 1 -- 172.21.15.167:0/711350431 >> 172.21.15.167:0/711350431 conn(0x7f41340b6af0 msgr2=0x7f41340b6f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:49.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.631+0000 7f4133e00640 1 -- 172.21.15.167:0/711350431 shutdown_connections 2024-05-11T12:12:49.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.631+0000 7f4133e00640 1 -- 172.21.15.167:0/711350431 wait complete. 2024-05-11T12:12:49.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.633+0000 7f4133e00640 1 Processor -- start 2024-05-11T12:12:49.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.633+0000 7f4133e00640 1 -- start start 2024-05-11T12:12:49.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.634+0000 7f4133e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4118004d40 0x7f4134155470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:49.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.634+0000 7f4133e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41340cd1a0 0x7f41340ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:49.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.634+0000 7f4133e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41340b99c0 0x7f413414af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:49.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.634+0000 7f4133e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f413406d390 con 0x7f41340cd1a0 2024-05-11T12:12:49.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.634+0000 7f4133e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f413406d0b0 con 0x7f4118004d40 2024-05-11T12:12:49.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.634+0000 7f4133e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f413406db90 con 0x7f41340b99c0 2024-05-11T12:12:49.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.634+0000 7f412be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41340cd1a0 0x7f41340ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:49.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.634+0000 7f412be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41340cd1a0 0x7f41340ce7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55990/0 (socket says 172.21.15.167:55990) 2024-05-11T12:12:49.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.634+0000 7f412be00640 1 -- 172.21.15.167:0/3214964652 learned_addr learned my addr 172.21.15.167:0/3214964652 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:49.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.634+0000 7f4133400640 1 --2- 172.21.15.167:0/3214964652 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41340b99c0 0x7f413414af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:49.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.634+0000 7f4132a00640 1 --2- 172.21.15.167:0/3214964652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4118004d40 0x7f4134155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:49.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.634+0000 7f412be00640 1 -- 172.21.15.167:0/3214964652 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41340b99c0 msgr2=0x7f413414af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:49.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.634+0000 7f412be00640 1 --2- 172.21.15.167:0/3214964652 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41340b99c0 0x7f413414af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:49.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.634+0000 7f412be00640 1 -- 172.21.15.167:0/3214964652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4118004d40 msgr2=0x7f4134155470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:49.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.634+0000 7f412be00640 1 --2- 172.21.15.167:0/3214964652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4118004d40 0x7f4134155470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:49.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.634+0000 7f412be00640 1 -- 172.21.15.167:0/3214964652 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41340cd710 con 0x7f41340cd1a0 2024-05-11T12:12:49.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.635+0000 7f4133400640 1 --2- 172.21.15.167:0/3214964652 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41340b99c0 0x7f413414af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:49.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.635+0000 7f412be00640 1 --2- 172.21.15.167:0/3214964652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41340cd1a0 0x7f41340ce7b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f411c00c730 tx=0x7f411c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:49.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.635+0000 7f412aa00640 1 -- 172.21.15.167:0/3214964652 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411c0072e0 con 0x7f41340cd1a0 2024-05-11T12:12:49.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.635+0000 7f412aa00640 1 -- 172.21.15.167:0/3214964652 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f411c0074a0 con 0x7f41340cd1a0 2024-05-11T12:12:49.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.635+0000 7f412aa00640 1 -- 172.21.15.167:0/3214964652 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411c004a20 con 0x7f41340cd1a0 2024-05-11T12:12:49.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.635+0000 7f4133e00640 1 -- 172.21.15.167:0/3214964652 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41340cd9f0 con 0x7f41340cd1a0 2024-05-11T12:12:49.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.635+0000 7f4133e00640 1 -- 172.21.15.167:0/3214964652 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41340cdfb0 con 0x7f41340cd1a0 2024-05-11T12:12:49.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.636+0000 7f4133e00640 1 -- 172.21.15.167:0/3214964652 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4134069bc0 con 0x7f41340cd1a0 2024-05-11T12:12:49.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.637+0000 7f412aa00640 1 -- 172.21.15.167:0/3214964652 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f411c011070 con 0x7f41340cd1a0 2024-05-11T12:12:49.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.639+0000 7f412aa00640 1 --2- 172.21.15.167:0/3214964652 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f41100762d0 0x7f4110078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:49.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.639+0000 7f412aa00640 1 -- 172.21.15.167:0/3214964652 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f411c0974b0 con 0x7f41340cd1a0 2024-05-11T12:12:49.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.639+0000 7f4132a00640 1 --2- 172.21.15.167:0/3214964652 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f41100762d0 0x7f4110078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:49.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.640+0000 7f4132a00640 1 --2- 172.21.15.167:0/3214964652 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f41100762d0 0x7f4110078790 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f413406e150 tx=0x7f4124009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:49.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.644+0000 7f412aa00640 1 -- 172.21.15.167:0/3214964652 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f411c060be0 con 0x7f41340cd1a0 2024-05-11T12:12:49.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.874+0000 7f4133e00640 1 -- 172.21.15.167:0/3214964652 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41340cfe70 con 0x7f41340cd1a0 2024-05-11T12:12:49.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.875+0000 7f412aa00640 1 -- 172.21.15.167:0/3214964652 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5193 (secure 0 0 0) 0x7f411c0604e0 con 0x7f41340cd1a0 2024-05-11T12:12:49.875 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:12:49.875 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4634,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2640837270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":2640837270},{"type":"v1","addr":"172.21.15.204:6835","nonce":2640837270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4807,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3516271493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3516271493},{"type":"v1","addr":"172.21.15.204:6834","nonce":3516271493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4816,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2070289869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2070289869},{"type":"v1","addr":"172.21.15.167:6837","nonce":2070289869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:12:35.314711+0000","modified":"2024-05-11T12:12:38.382446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6835/4016299199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4016299199},{"type":"v1","addr":"172.21.15.167:6835","nonce":4016299199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5310,"qdb_cluster":[5310]},"id":3}]} 2024-05-11T12:12:49.875 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 26 2024-05-11T12:12:49.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.878+0000 7f4133e00640 1 -- 172.21.15.167:0/3214964652 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f41100762d0 msgr2=0x7f4110078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:49.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.878+0000 7f4133e00640 1 --2- 172.21.15.167:0/3214964652 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f41100762d0 0x7f4110078790 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f413406e150 tx=0x7f4124009040 comp rx=0 tx=0).stop 2024-05-11T12:12:49.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.878+0000 7f4133e00640 1 -- 172.21.15.167:0/3214964652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41340cd1a0 msgr2=0x7f41340ce7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:49.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.878+0000 7f4133e00640 1 --2- 172.21.15.167:0/3214964652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41340cd1a0 0x7f41340ce7b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f411c00c730 tx=0x7f411c00cbf0 comp rx=0 tx=0).stop 2024-05-11T12:12:49.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.878+0000 7f4133e00640 1 -- 172.21.15.167:0/3214964652 shutdown_connections 2024-05-11T12:12:49.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.878+0000 7f4133e00640 1 --2- 172.21.15.167:0/3214964652 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41340b99c0 0x7f413414af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:49.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.878+0000 7f4133e00640 1 --2- 172.21.15.167:0/3214964652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41340cd1a0 0x7f41340ce7b0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:49.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.878+0000 7f4133e00640 1 --2- 172.21.15.167:0/3214964652 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f41100762d0 0x7f4110078790 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:49.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.878+0000 7f4133e00640 1 --2- 172.21.15.167:0/3214964652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4118004d40 0x7f4134155470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:49.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.878+0000 7f4133e00640 1 -- 172.21.15.167:0/3214964652 >> 172.21.15.167:0/3214964652 conn(0x7f41340b6af0 msgr2=0x7f41340d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:49.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.878+0000 7f4133e00640 1 -- 172.21.15.167:0/3214964652 shutdown_connections 2024-05-11T12:12:49.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:49.878+0000 7f4133e00640 1 -- 172.21.15.167:0/3214964652 wait complete. 2024-05-11T12:12:49.896 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:12:50.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.096+0000 7f3764a00640 1 Processor -- start 2024-05-11T12:12:50.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.096+0000 7f3764a00640 1 -- start start 2024-05-11T12:12:50.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.097+0000 7f3764a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3760145b00 0x7f376006f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:50.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.098+0000 7f3764a00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f376006c660 0x7f376006ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:50.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.098+0000 7f3764a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f376006cfb0 con 0x7f3760146770 2024-05-11T12:12:50.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.098+0000 7f3764a00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f376006fbc0 con 0x7f376006c660 2024-05-11T12:12:50.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.098+0000 7f3764a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f37600b4520 con 0x7f3760145b00 2024-05-11T12:12:50.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.098+0000 7f375f200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f376006c660 0x7f376006ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:50.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.098+0000 7f375f200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f376006c660 0x7f376006ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55992/0 (socket says 172.21.15.167:55992) 2024-05-11T12:12:50.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.098+0000 7f375f200640 1 -- 172.21.15.167:0/286856702 learned_addr learned my addr 172.21.15.167:0/286856702 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:50.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.098+0000 7f375f200640 1 -- 172.21.15.167:0/286856702 >> v2:172.21.15.204:3300/0 conn(0x7f3760145b00 msgr2=0x7f376006f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:50.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.098+0000 7f375e800640 1 --2- 172.21.15.167:0/286856702 >> v2:172.21.15.204:3300/0 conn(0x7f3760145b00 0x7f376006f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:50.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.098+0000 7f375f200640 1 --2- 172.21.15.167:0/286856702 >> v2:172.21.15.204:3300/0 conn(0x7f3760145b00 0x7f376006f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:50.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.098+0000 7f375f200640 1 -- 172.21.15.167:0/286856702 >> v1:172.21.15.204:6789/0 conn(0x7f3760146770 legacy=0x7f37600ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:50.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.098+0000 7f375f200640 1 -- 172.21.15.167:0/286856702 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37600b4800 con 0x7f376006c660 2024-05-11T12:12:50.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.100+0000 7f375f200640 1 --2- 172.21.15.167:0/286856702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f376006c660 0x7f376006ca70 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f37600701e0 tx=0x7f375802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caa03ee412843a3d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:50.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.100+0000 7f375de00640 1 -- 172.21.15.167:0/286856702 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3758037070 con 0x7f376006c660 2024-05-11T12:12:50.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.100+0000 7f375de00640 1 -- 172.21.15.167:0/286856702 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3758032440 con 0x7f376006c660 2024-05-11T12:12:50.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.100+0000 7f375de00640 1 -- 172.21.15.167:0/286856702 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3758032810 con 0x7f376006c660 2024-05-11T12:12:50.101 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.101+0000 7f3764a00640 1 -- 172.21.15.167:0/286856702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f376006c660 msgr2=0x7f376006ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:50.101 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.101+0000 7f3764a00640 1 --2- 172.21.15.167:0/286856702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f376006c660 0x7f376006ca70 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f37600701e0 tx=0x7f375802a140 comp rx=0 tx=0).stop 2024-05-11T12:12:50.101 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.101+0000 7f3764a00640 1 -- 172.21.15.167:0/286856702 shutdown_connections 2024-05-11T12:12:50.101 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.101+0000 7f3764a00640 1 --2- 172.21.15.167:0/286856702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f376006c660 0x7f376006ca70 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:50.101 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.101+0000 7f3764a00640 1 --2- 172.21.15.167:0/286856702 >> v2:172.21.15.204:3300/0 conn(0x7f3760145b00 0x7f376006f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:50.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.101+0000 7f3764a00640 1 -- 172.21.15.167:0/286856702 >> 172.21.15.167:0/286856702 conn(0x7f37600b3460 msgr2=0x7f37600d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:50.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.101+0000 7f3764a00640 1 -- 172.21.15.167:0/286856702 shutdown_connections 2024-05-11T12:12:50.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.101+0000 7f3764a00640 1 -- 172.21.15.167:0/286856702 wait complete. 2024-05-11T12:12:50.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.103+0000 7f3764a00640 1 Processor -- start 2024-05-11T12:12:50.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.103+0000 7f3764a00640 1 -- start start 2024-05-11T12:12:50.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.104+0000 7f3764a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f376006c660 0x7f37600b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:50.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.104+0000 7f3764a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3760145b00 0x7f3760144970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:50.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.104+0000 7f3764a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3760146770 0x7f376015c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:50.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.104+0000 7f3764a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3760145ee0 con 0x7f376006c660 2024-05-11T12:12:50.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.104+0000 7f3764a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f37600cf0e0 con 0x7f3760145b00 2024-05-11T12:12:50.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.104+0000 7f3764a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f376006f800 con 0x7f3760146770 2024-05-11T12:12:50.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.104+0000 7f375e800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f376006c660 0x7f37600b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:50.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.104+0000 7f375e800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f376006c660 0x7f37600b06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56008/0 (socket says 172.21.15.167:56008) 2024-05-11T12:12:50.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.104+0000 7f375e800640 1 -- 172.21.15.167:0/343683553 learned_addr learned my addr 172.21.15.167:0/343683553 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:50.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.104+0000 7f3755e00640 1 --2- 172.21.15.167:0/343683553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3760145b00 0x7f3760144970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:50.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.105+0000 7f375f200640 1 --2- 172.21.15.167:0/343683553 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3760146770 0x7f376015c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:50.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.105+0000 7f375e800640 1 -- 172.21.15.167:0/343683553 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3760146770 msgr2=0x7f376015c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:50.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.105+0000 7f375e800640 1 --2- 172.21.15.167:0/343683553 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3760146770 0x7f376015c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:50.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.105+0000 7f375e800640 1 -- 172.21.15.167:0/343683553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3760145b00 msgr2=0x7f3760144970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:50.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.105+0000 7f375e800640 1 --2- 172.21.15.167:0/343683553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3760145b00 0x7f3760144970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:50.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.105+0000 7f375e800640 1 -- 172.21.15.167:0/343683553 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3760144eb0 con 0x7f376006c660 2024-05-11T12:12:50.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.105+0000 7f375e800640 1 --2- 172.21.15.167:0/343683553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f376006c660 0x7f37600b06f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f375000ce10 tx=0x7f375000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:50.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.105+0000 7f3757400640 1 -- 172.21.15.167:0/343683553 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3750003b90 con 0x7f376006c660 2024-05-11T12:12:50.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.105+0000 7f3757400640 1 -- 172.21.15.167:0/343683553 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3750003d30 con 0x7f376006c660 2024-05-11T12:12:50.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.105+0000 7f3764a00640 1 -- 172.21.15.167:0/343683553 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f376015f380 con 0x7f376006c660 2024-05-11T12:12:50.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.105+0000 7f3757400640 1 -- 172.21.15.167:0/343683553 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3750004240 con 0x7f376006c660 2024-05-11T12:12:50.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.105+0000 7f3764a00640 1 -- 172.21.15.167:0/343683553 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37600b1340 con 0x7f376006c660 2024-05-11T12:12:50.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.107+0000 7f3764a00640 1 -- 172.21.15.167:0/343683553 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37600694b0 con 0x7f376006c660 2024-05-11T12:12:50.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.108+0000 7f3757400640 1 -- 172.21.15.167:0/343683553 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f37500043e0 con 0x7f376006c660 2024-05-11T12:12:50.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.109+0000 7f3757400640 1 --2- 172.21.15.167:0/343683553 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f373c076300 0x7f373c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:50.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.109+0000 7f3757400640 1 -- 172.21.15.167:0/343683553 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f3750098340 con 0x7f376006c660 2024-05-11T12:12:50.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.109+0000 7f3755e00640 1 --2- 172.21.15.167:0/343683553 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f373c076300 0x7f373c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:50.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.110+0000 7f3755e00640 1 --2- 172.21.15.167:0/343683553 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f373c076300 0x7f373c0787c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3748006fd0 tx=0x7f3748008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:50.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.114+0000 7f3757400640 1 -- 172.21.15.167:0/343683553 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3750061a70 con 0x7f376006c660 2024-05-11T12:12:50.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.323+0000 7f3764a00640 1 -- 172.21.15.167:0/343683553 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3760160b50 con 0x7f376006c660 2024-05-11T12:12:50.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.326+0000 7f3757400640 1 -- 172.21.15.167:0/343683553 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) ==== 74+0+16990 (secure 0 0 0) 0x7f3750061370 con 0x7f376006c660 2024-05-11T12:12:50.326 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:12:50.327 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":53,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:12:36.879705+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:12:33.261425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-11T12:12:34.290603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:12:50.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.328+0000 7f3764a00640 1 -- 172.21.15.167:0/343683553 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f373c076300 msgr2=0x7f373c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:50.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.328+0000 7f3764a00640 1 --2- 172.21.15.167:0/343683553 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f373c076300 0x7f373c0787c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3748006fd0 tx=0x7f3748008040 comp rx=0 tx=0).stop 2024-05-11T12:12:50.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.328+0000 7f3764a00640 1 -- 172.21.15.167:0/343683553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f376006c660 msgr2=0x7f37600b06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:50.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.328+0000 7f3764a00640 1 --2- 172.21.15.167:0/343683553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f376006c660 0x7f37600b06f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f375000ce10 tx=0x7f375000a660 comp rx=0 tx=0).stop 2024-05-11T12:12:50.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.329+0000 7f3764a00640 1 -- 172.21.15.167:0/343683553 shutdown_connections 2024-05-11T12:12:50.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.329+0000 7f3764a00640 1 --2- 172.21.15.167:0/343683553 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3760146770 0x7f376015c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:50.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.329+0000 7f3764a00640 1 --2- 172.21.15.167:0/343683553 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f373c076300 0x7f373c0787c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:50.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.329+0000 7f3764a00640 1 --2- 172.21.15.167:0/343683553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3760145b00 0x7f3760144970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:50.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.329+0000 7f3764a00640 1 --2- 172.21.15.167:0/343683553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f376006c660 0x7f37600b06f0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:50.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.329+0000 7f3764a00640 1 -- 172.21.15.167:0/343683553 >> 172.21.15.167:0/343683553 conn(0x7f37600b3460 msgr2=0x7f3760146060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:50.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.329+0000 7f3764a00640 1 -- 172.21.15.167:0/343683553 shutdown_connections 2024-05-11T12:12:50.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.329+0000 7f3764a00640 1 -- 172.21.15.167:0/343683553 wait complete. 2024-05-11T12:12:50.348 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-11T12:12:50.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.546+0000 7f41ba000640 1 Processor -- start 2024-05-11T12:12:50.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.546+0000 7f41ba000640 1 -- start start 2024-05-11T12:12:50.548 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.548+0000 7f41ba000640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f41b415f320 0x7f41b4162d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:50.548 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.548+0000 7f41ba000640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41b40b6460 con 0x7f41b4163980 2024-05-11T12:12:50.548 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.548+0000 7f41ba000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41b40b6740 con 0x7f41b415ef40 2024-05-11T12:12:50.548 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.548+0000 7f41ba000640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f41b40afb90 con 0x7f41b415f320 2024-05-11T12:12:50.548 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.548+0000 7f41b3e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f41b4163980 0x7f41b40b5ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49314/0 (socket says 172.21.15.167:49314) 2024-05-11T12:12:50.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.548+0000 7f41b3e00640 1 -- 172.21.15.167:0/3556279625 learned_addr learned my addr 172.21.15.167:0/3556279625 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:50.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.548+0000 7f41b9600640 1 --2- 172.21.15.167:0/3556279625 >> v2:172.21.15.167:3300/0 conn(0x7f41b415f320 0x7f41b4162d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:50.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.549+0000 7f41b3400640 1 -- 172.21.15.167:0/3556279625 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 839204553 0 0) 0x7f41b40b6460 con 0x7f41b4163980 2024-05-11T12:12:50.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.549+0000 7f41b3400640 1 -- 172.21.15.167:0/3556279625 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f419c003520 con 0x7f41b4163980 2024-05-11T12:12:50.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.549+0000 7f41b9600640 1 -- 172.21.15.167:0/3556279625 >> v1:172.21.15.204:6789/0 conn(0x7f41b415ef40 legacy=0x7f41b4071fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:50.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.549+0000 7f41b9600640 1 -- 172.21.15.167:0/3556279625 >> v1:172.21.15.167:6789/0 conn(0x7f41b4163980 legacy=0x7f41b40b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:50.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.549+0000 7f41b9600640 1 -- 172.21.15.167:0/3556279625 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41b40afe70 con 0x7f41b415f320 2024-05-11T12:12:50.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.550+0000 7f41b9600640 1 --2- 172.21.15.167:0/3556279625 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41b415f320 0x7f41b4162d70 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f41ac009d90 tx=0x7f41ac02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48c24c1bf51da553 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:50.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.550+0000 7f41b3400640 1 -- 172.21.15.167:0/3556279625 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ac007740 con 0x7f41b415f320 2024-05-11T12:12:50.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.550+0000 7f41b3400640 1 -- 172.21.15.167:0/3556279625 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41ac0078e0 con 0x7f41b415f320 2024-05-11T12:12:50.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.550+0000 7f41b3400640 1 -- 172.21.15.167:0/3556279625 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ac007cb0 con 0x7f41b415f320 2024-05-11T12:12:50.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.551+0000 7f41ba000640 1 -- 172.21.15.167:0/3556279625 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41b415f320 msgr2=0x7f41b4162d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:50.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.551+0000 7f41ba000640 1 --2- 172.21.15.167:0/3556279625 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41b415f320 0x7f41b4162d70 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f41ac009d90 tx=0x7f41ac02a6f0 comp rx=0 tx=0).stop 2024-05-11T12:12:50.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.551+0000 7f41ba000640 1 -- 172.21.15.167:0/3556279625 shutdown_connections 2024-05-11T12:12:50.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.551+0000 7f41ba000640 1 --2- 172.21.15.167:0/3556279625 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41b415f320 0x7f41b4162d70 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:50.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.551+0000 7f41ba000640 1 -- 172.21.15.167:0/3556279625 >> 172.21.15.167:0/3556279625 conn(0x7f41b406c450 msgr2=0x7f41b406c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:50.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.551+0000 7f41ba000640 1 -- 172.21.15.167:0/3556279625 shutdown_connections 2024-05-11T12:12:50.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.551+0000 7f41ba000640 1 -- 172.21.15.167:0/3556279625 wait complete. 2024-05-11T12:12:50.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.553+0000 7f41ba000640 1 Processor -- start 2024-05-11T12:12:50.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.553+0000 7f41ba000640 1 -- start start 2024-05-11T12:12:50.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.554+0000 7f41ba000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41b415ef40 0x7f41b41e8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:50.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.554+0000 7f41ba000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41b415f320 0x7f41b406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:50.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.554+0000 7f41ba000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41b4163980 0x7f41b406d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:50.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.554+0000 7f41ba000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f41b4163640 con 0x7f41b415f320 2024-05-11T12:12:50.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.554+0000 7f41ba000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f41b41634c0 con 0x7f41b415ef40 2024-05-11T12:12:50.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.554+0000 7f41ba000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f41b406f250 con 0x7f41b4163980 2024-05-11T12:12:50.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.554+0000 7f41b3e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41b415f320 0x7f41b406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:50.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.554+0000 7f41b3e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41b415f320 0x7f41b406d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56028/0 (socket says 172.21.15.167:56028) 2024-05-11T12:12:50.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.554+0000 7f41b3e00640 1 -- 172.21.15.167:0/3429404646 learned_addr learned my addr 172.21.15.167:0/3429404646 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:50.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.554+0000 7f41b8c00640 1 --2- 172.21.15.167:0/3429404646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41b415ef40 0x7f41b41e8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:50.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.555+0000 7f41b9600640 1 --2- 172.21.15.167:0/3429404646 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41b4163980 0x7f41b406d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:50.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.555+0000 7f41b3e00640 1 -- 172.21.15.167:0/3429404646 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41b4163980 msgr2=0x7f41b406d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:50.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.555+0000 7f41b3e00640 1 --2- 172.21.15.167:0/3429404646 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41b4163980 0x7f41b406d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:50.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.555+0000 7f41b3e00640 1 -- 172.21.15.167:0/3429404646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41b415ef40 msgr2=0x7f41b41e8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:50.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.555+0000 7f41b3e00640 1 --2- 172.21.15.167:0/3429404646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41b415ef40 0x7f41b41e8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:50.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.555+0000 7f41b3e00640 1 -- 172.21.15.167:0/3429404646 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41b41e94a0 con 0x7f41b415f320 2024-05-11T12:12:50.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.555+0000 7f41b3e00640 1 --2- 172.21.15.167:0/3429404646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41b415f320 0x7f41b406d0f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f41a000c3e0 tx=0x7f41a000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:50.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.555+0000 7f41b0c00640 1 -- 172.21.15.167:0/3429404646 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a0017070 con 0x7f41b415f320 2024-05-11T12:12:50.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.555+0000 7f41b0c00640 1 -- 172.21.15.167:0/3429404646 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41a0005ce0 con 0x7f41b415f320 2024-05-11T12:12:50.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.555+0000 7f41ba000640 1 -- 172.21.15.167:0/3429404646 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41b41e9a30 con 0x7f41b415f320 2024-05-11T12:12:50.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.555+0000 7f41b0c00640 1 -- 172.21.15.167:0/3429404646 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a00072f0 con 0x7f41b415f320 2024-05-11T12:12:50.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.555+0000 7f41ba000640 1 -- 172.21.15.167:0/3429404646 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41b4160cb0 con 0x7f41b415f320 2024-05-11T12:12:50.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.556+0000 7f41ba000640 1 -- 172.21.15.167:0/3429404646 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41b40694b0 con 0x7f41b415f320 2024-05-11T12:12:50.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.558+0000 7f41b0c00640 1 -- 172.21.15.167:0/3429404646 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f41a001a070 con 0x7f41b415f320 2024-05-11T12:12:50.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.558+0000 7f41b0c00640 1 --2- 172.21.15.167:0/3429404646 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f41940762d0 0x7f4194078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:50.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.559+0000 7f41b0c00640 1 -- 172.21.15.167:0/3429404646 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f41a0096b00 con 0x7f41b415f320 2024-05-11T12:12:50.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.559+0000 7f41b8c00640 1 --2- 172.21.15.167:0/3429404646 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f41940762d0 0x7f4194078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:50.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.559+0000 7f41b8c00640 1 --2- 172.21.15.167:0/3429404646 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f41940762d0 0x7f4194078790 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f41a8001680 tx=0x7f41a8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:50.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.563+0000 7f41b0c00640 1 -- 172.21.15.167:0/3429404646 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f41a0060380 con 0x7f41b415f320 2024-05-11T12:12:50.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.789+0000 7f41ba000640 1 -- 172.21.15.167:0/3429404646 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f41b4158d70 con 0x7f41b415f320 2024-05-11T12:12:50.790 INFO:tasks.ceph.mon.a.smithi167.stderr:2024-05-11T12:12:50.790+0000 7f3aac000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-11T12:12:50.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.794+0000 7f41b0c00640 1 -- 172.21.15.167:0/3429404646 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) ==== 161+0+0 (secure 0 0 0) 0x7f41a0096e20 con 0x7f41b415f320 2024-05-11T12:12:50.794 INFO:teuthology.orchestra.run.smithi167.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-11T12:12:50.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.797+0000 7f41ba000640 1 -- 172.21.15.167:0/3429404646 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f41940762d0 msgr2=0x7f4194078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:50.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.797+0000 7f41ba000640 1 --2- 172.21.15.167:0/3429404646 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f41940762d0 0x7f4194078790 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f41a8001680 tx=0x7f41a8009040 comp rx=0 tx=0).stop 2024-05-11T12:12:50.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.797+0000 7f41ba000640 1 -- 172.21.15.167:0/3429404646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41b415f320 msgr2=0x7f41b406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:50.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.797+0000 7f41ba000640 1 --2- 172.21.15.167:0/3429404646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41b415f320 0x7f41b406d0f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f41a000c3e0 tx=0x7f41a000c8a0 comp rx=0 tx=0).stop 2024-05-11T12:12:50.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.798+0000 7f41ba000640 1 -- 172.21.15.167:0/3429404646 shutdown_connections 2024-05-11T12:12:50.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.798+0000 7f41ba000640 1 --2- 172.21.15.167:0/3429404646 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41b4163980 0x7f41b406d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:50.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.798+0000 7f41ba000640 1 --2- 172.21.15.167:0/3429404646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41b415f320 0x7f41b406d0f0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:50.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.798+0000 7f41ba000640 1 --2- 172.21.15.167:0/3429404646 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f41940762d0 0x7f4194078790 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:50.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.798+0000 7f41ba000640 1 --2- 172.21.15.167:0/3429404646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41b415ef40 0x7f41b41e8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:50.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.798+0000 7f41ba000640 1 -- 172.21.15.167:0/3429404646 >> 172.21.15.167:0/3429404646 conn(0x7f41b406c450 msgr2=0x7f41b40b5660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:50.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.798+0000 7f41ba000640 1 -- 172.21.15.167:0/3429404646 shutdown_connections 2024-05-11T12:12:50.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.798+0000 7f41ba000640 1 -- 172.21.15.167:0/3429404646 wait complete. 2024-05-11T12:12:50.807 INFO:tasks.ceph.mds.a.smithi167.stderr: -2025> 2024-05-11T12:12:41.259+0000 7fcc07600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-11T12:12:50.819 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-11T12:12:50.861 INFO:tasks.ceph.mds.a.smithi167.stdout:starting mds.a at 2024-05-11T12:12:50.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.987+0000 7f427b200640 1 Processor -- start 2024-05-11T12:12:50.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.987+0000 7f427b200640 1 -- start start 2024-05-11T12:12:50.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.989+0000 7f427b200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f427415ac80 0x7f4274074260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:50.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.989+0000 7f427b200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4274161380 0x7f4274072950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:50.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.989+0000 7f427b200640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42740b65e0 con 0x7f42740b3d50 2024-05-11T12:12:50.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.989+0000 7f427b200640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f427406e3c0 con 0x7f4274161380 2024-05-11T12:12:50.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.989+0000 7f427b200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f427415b060 con 0x7f427415ac80 2024-05-11T12:12:50.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.989+0000 7f4279200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4274161380 0x7f4274072950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:50.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.989+0000 7f4273400640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f42740b3d50 0x7f4274160c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49324/0 (socket says 172.21.15.167:49324) 2024-05-11T12:12:50.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.989+0000 7f4273400640 1 -- 172.21.15.167:0/566774536 learned_addr learned my addr 172.21.15.167:0/566774536 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:50.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.989+0000 7f4273e00640 1 --2- 172.21.15.167:0/566774536 >> v2:172.21.15.204:3300/0 conn(0x7f427415ac80 0x7f4274074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:50.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.989+0000 7f4279200640 1 -- 172.21.15.167:0/566774536 >> v2:172.21.15.204:3300/0 conn(0x7f427415ac80 msgr2=0x7f4274074260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:50.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.989+0000 7f4279200640 1 --2- 172.21.15.167:0/566774536 >> v2:172.21.15.204:3300/0 conn(0x7f427415ac80 0x7f4274074260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:50.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.989+0000 7f4279200640 1 -- 172.21.15.167:0/566774536 >> v1:172.21.15.167:6789/0 conn(0x7f42740b3d50 legacy=0x7f4274160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:50.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.989+0000 7f4279200640 1 -- 172.21.15.167:0/566774536 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42740b68c0 con 0x7f4274161380 2024-05-11T12:12:50.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.990+0000 7f4279200640 1 --2- 172.21.15.167:0/566774536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4274161380 0x7f4274072950 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f426c009d90 tx=0x7f426c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b49bd5c873742be8 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:50.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.991+0000 7f4272a00640 1 -- 172.21.15.167:0/566774536 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426c007740 con 0x7f4274161380 2024-05-11T12:12:50.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.991+0000 7f4272a00640 1 -- 172.21.15.167:0/566774536 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f426c0078e0 con 0x7f4274161380 2024-05-11T12:12:50.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.991+0000 7f4272a00640 1 -- 172.21.15.167:0/566774536 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426c007cb0 con 0x7f4274161380 2024-05-11T12:12:50.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.991+0000 7f427b200640 1 -- 172.21.15.167:0/566774536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4274161380 msgr2=0x7f4274072950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:50.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.991+0000 7f427b200640 1 --2- 172.21.15.167:0/566774536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4274161380 0x7f4274072950 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f426c009d90 tx=0x7f426c02a6a0 comp rx=0 tx=0).stop 2024-05-11T12:12:50.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.992+0000 7f427b200640 1 -- 172.21.15.167:0/566774536 shutdown_connections 2024-05-11T12:12:50.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.992+0000 7f427b200640 1 --2- 172.21.15.167:0/566774536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4274161380 0x7f4274072950 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:50.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.992+0000 7f427b200640 1 --2- 172.21.15.167:0/566774536 >> v2:172.21.15.204:3300/0 conn(0x7f427415ac80 0x7f4274074260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:50.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.992+0000 7f427b200640 1 -- 172.21.15.167:0/566774536 >> 172.21.15.167:0/566774536 conn(0x7f427406c450 msgr2=0x7f427406c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:50.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.992+0000 7f427b200640 1 -- 172.21.15.167:0/566774536 shutdown_connections 2024-05-11T12:12:50.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.992+0000 7f427b200640 1 -- 172.21.15.167:0/566774536 wait complete. 2024-05-11T12:12:50.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.993+0000 7f427b200640 1 Processor -- start 2024-05-11T12:12:50.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.993+0000 7f427b200640 1 -- start start 2024-05-11T12:12:50.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.994+0000 7f427b200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42740b3d50 0x7f42740afd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:50.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.994+0000 7f427b200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f427415ac80 0x7f42740cfe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:50.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.994+0000 7f427b200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4274161380 0x7f4274145b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:50.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.994+0000 7f427b200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f42740b44f0 con 0x7f4274161380 2024-05-11T12:12:50.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.994+0000 7f427b200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f42740b41e0 con 0x7f42740b3d50 2024-05-11T12:12:50.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.994+0000 7f427b200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4274072e90 con 0x7f427415ac80 2024-05-11T12:12:50.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.995+0000 7f4279200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4274161380 0x7f4274145b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:50.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.995+0000 7f4279200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4274161380 0x7f4274145b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56072/0 (socket says 172.21.15.167:56072) 2024-05-11T12:12:50.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.995+0000 7f4279200640 1 -- 172.21.15.167:0/3381345967 learned_addr learned my addr 172.21.15.167:0/3381345967 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:50.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.995+0000 7f4279200640 1 -- 172.21.15.167:0/3381345967 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f427415ac80 msgr2=0x7f42740cfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:50.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.995+0000 7f4279200640 1 --2- 172.21.15.167:0/3381345967 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f427415ac80 0x7f42740cfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:50.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.995+0000 7f4279200640 1 -- 172.21.15.167:0/3381345967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42740b3d50 msgr2=0x7f42740afd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:50.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.995+0000 7f4279200640 1 --2- 172.21.15.167:0/3381345967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42740b3d50 0x7f42740afd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:50.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.995+0000 7f4279200640 1 -- 172.21.15.167:0/3381345967 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42740cedb0 con 0x7f4274161380 2024-05-11T12:12:50.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.995+0000 7f4279200640 1 --2- 172.21.15.167:0/3381345967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4274161380 0x7f4274145b70 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f426c009eb0 tx=0x7f426c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:50.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.996+0000 7f426be00640 1 -- 172.21.15.167:0/3381345967 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426c03f070 con 0x7f4274161380 2024-05-11T12:12:50.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.996+0000 7f426be00640 1 -- 172.21.15.167:0/3381345967 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f426c031bc0 con 0x7f4274161380 2024-05-11T12:12:50.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.996+0000 7f426be00640 1 -- 172.21.15.167:0/3381345967 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426c044460 con 0x7f4274161380 2024-05-11T12:12:50.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.996+0000 7f427b200640 1 -- 172.21.15.167:0/3381345967 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42740cf090 con 0x7f4274161380 2024-05-11T12:12:50.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.996+0000 7f427b200640 1 -- 172.21.15.167:0/3381345967 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42740cf650 con 0x7f4274161380 2024-05-11T12:12:50.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.997+0000 7f427b200640 1 -- 172.21.15.167:0/3381345967 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4274069520 con 0x7f4274161380 2024-05-11T12:12:50.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.998+0000 7f426be00640 1 -- 172.21.15.167:0/3381345967 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f426c007790 con 0x7f4274161380 2024-05-11T12:12:50.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.999+0000 7f426be00640 1 --2- 172.21.15.167:0/3381345967 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f42540762d0 0x7f4254078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:50.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.999+0000 7f426be00640 1 -- 172.21.15.167:0/3381345967 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6187+0+0 (secure 0 0 0) 0x7f426c0b8df0 con 0x7f4274161380 2024-05-11T12:12:51.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:50.999+0000 7f4273e00640 1 --2- 172.21.15.167:0/3381345967 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f42540762d0 0x7f4254078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:51.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:51.000+0000 7f4273e00640 1 --2- 172.21.15.167:0/3381345967 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f42540762d0 0x7f4254078790 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f42640098f0 tx=0x7f4264008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:51.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:51.004+0000 7f426be00640 1 -- 172.21.15.167:0/3381345967 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f426c04b030 con 0x7f4274161380 2024-05-11T12:12:51.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:51.235+0000 7f427b200640 1 -- 172.21.15.167:0/3381345967 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f427406f1c0 con 0x7f4274161380 2024-05-11T12:12:51.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:51.797+0000 7f426be00640 1 -- 172.21.15.167:0/3381345967 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) ==== 104+0+0 (secure 0 0 0) 0x7f426c082440 con 0x7f4274161380 2024-05-11T12:12:51.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:51.801+0000 7f427b200640 1 -- 172.21.15.167:0/3381345967 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f42540762d0 msgr2=0x7f4254078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:51.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:51.801+0000 7f427b200640 1 --2- 172.21.15.167:0/3381345967 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f42540762d0 0x7f4254078790 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f42640098f0 tx=0x7f4264008040 comp rx=0 tx=0).stop 2024-05-11T12:12:51.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:51.801+0000 7f427b200640 1 -- 172.21.15.167:0/3381345967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4274161380 msgr2=0x7f4274145b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:51.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:51.801+0000 7f427b200640 1 --2- 172.21.15.167:0/3381345967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4274161380 0x7f4274145b70 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f426c009eb0 tx=0x7f426c004a40 comp rx=0 tx=0).stop 2024-05-11T12:12:51.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:51.801+0000 7f427b200640 1 -- 172.21.15.167:0/3381345967 shutdown_connections 2024-05-11T12:12:51.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:51.801+0000 7f427b200640 1 --2- 172.21.15.167:0/3381345967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4274161380 0x7f4274145b70 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:51.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:51.801+0000 7f427b200640 1 --2- 172.21.15.167:0/3381345967 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f427415ac80 0x7f42740cfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:51.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:51.801+0000 7f427b200640 1 --2- 172.21.15.167:0/3381345967 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f42540762d0 0x7f4254078790 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:51.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:51.801+0000 7f427b200640 1 --2- 172.21.15.167:0/3381345967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42740b3d50 0x7f42740afd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:51.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:51.801+0000 7f427b200640 1 -- 172.21.15.167:0/3381345967 >> 172.21.15.167:0/3381345967 conn(0x7f427406c450 msgr2=0x7f427415f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:51.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:51.802+0000 7f427b200640 1 -- 172.21.15.167:0/3381345967 shutdown_connections 2024-05-11T12:12:51.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:51.802+0000 7f427b200640 1 -- 172.21.15.167:0/3381345967 wait complete. 2024-05-11T12:12:51.824 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-11T12:12:52.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.079+0000 7f3a62200640 1 Processor -- start 2024-05-11T12:12:52.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.079+0000 7f3a62200640 1 -- start start 2024-05-11T12:12:52.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.081+0000 7f3a62200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3a5c0c9a30 0x7f3a5c070000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:52.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.081+0000 7f3a62200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3a5c070540 0x7f3a5c0cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:52.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.081+0000 7f3a62200640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a5c0d11c0 con 0x7f3a5c146960 2024-05-11T12:12:52.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.081+0000 7f3a62200640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f3a5c070980 con 0x7f3a5c070540 2024-05-11T12:12:52.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.081+0000 7f3a62200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f3a5c0d2ad0 con 0x7f3a5c0c9a30 2024-05-11T12:12:52.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.081+0000 7f3a5be00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3a5c070540 0x7f3a5c0cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:52.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.081+0000 7f3a5b400640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f3a5c146960 0x7f3a5c161560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49336/0 (socket says 172.21.15.167:49336) 2024-05-11T12:12:52.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.081+0000 7f3a5b400640 1 -- 172.21.15.167:0/2668232180 learned_addr learned my addr 172.21.15.167:0/2668232180 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:52.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.082+0000 7f3a5aa00640 1 --2- 172.21.15.167:0/2668232180 >> v2:172.21.15.204:3300/0 conn(0x7f3a5c0c9a30 0x7f3a5c070000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:52.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.082+0000 7f3a5be00640 1 -- 172.21.15.167:0/2668232180 >> v2:172.21.15.204:3300/0 conn(0x7f3a5c0c9a30 msgr2=0x7f3a5c070000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:52.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.082+0000 7f3a5be00640 1 --2- 172.21.15.167:0/2668232180 >> v2:172.21.15.204:3300/0 conn(0x7f3a5c0c9a30 0x7f3a5c070000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:52.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.082+0000 7f3a5be00640 1 -- 172.21.15.167:0/2668232180 >> v1:172.21.15.167:6789/0 conn(0x7f3a5c146960 legacy=0x7f3a5c161560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:52.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.082+0000 7f3a5be00640 1 -- 172.21.15.167:0/2668232180 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a5c0d0c00 con 0x7f3a5c070540 2024-05-11T12:12:52.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.082+0000 7f3a5a000640 1 -- 172.21.15.167:0/2668232180 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3940326799 0 0) 0x7f3a5c0d11c0 con 0x7f3a5c146960 2024-05-11T12:12:52.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.082+0000 7f3a5aa00640 1 --2- 172.21.15.167:0/2668232180 >> v2:172.21.15.204:3300/0 conn(0x7f3a5c0c9a30 0x7f3a5c070000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:52.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.083+0000 7f3a5be00640 1 --2- 172.21.15.167:0/2668232180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a5c070540 0x7f3a5c0cd500 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c0712b0 tx=0x7f3a5402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aed578bd46809fce server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:52.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.083+0000 7f3a5a000640 1 -- 172.21.15.167:0/2668232180 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a54007740 con 0x7f3a5c070540 2024-05-11T12:12:52.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.083+0000 7f3a5a000640 1 -- 172.21.15.167:0/2668232180 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a540078e0 con 0x7f3a5c070540 2024-05-11T12:12:52.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.083+0000 7f3a5a000640 1 -- 172.21.15.167:0/2668232180 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a54007cb0 con 0x7f3a5c070540 2024-05-11T12:12:52.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.084+0000 7f3a62200640 1 -- 172.21.15.167:0/2668232180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a5c070540 msgr2=0x7f3a5c0cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:52.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.084+0000 7f3a62200640 1 --2- 172.21.15.167:0/2668232180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a5c070540 0x7f3a5c0cd500 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c0712b0 tx=0x7f3a5402a5f0 comp rx=0 tx=0).stop 2024-05-11T12:12:52.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.084+0000 7f3a62200640 1 -- 172.21.15.167:0/2668232180 shutdown_connections 2024-05-11T12:12:52.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.084+0000 7f3a62200640 1 --2- 172.21.15.167:0/2668232180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a5c070540 0x7f3a5c0cd500 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:52.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.084+0000 7f3a62200640 1 --2- 172.21.15.167:0/2668232180 >> v2:172.21.15.204:3300/0 conn(0x7f3a5c0c9a30 0x7f3a5c070000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:52.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.084+0000 7f3a62200640 1 -- 172.21.15.167:0/2668232180 >> 172.21.15.167:0/2668232180 conn(0x7f3a5c0b4ae0 msgr2=0x7f3a5c0b4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:52.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.084+0000 7f3a62200640 1 -- 172.21.15.167:0/2668232180 shutdown_connections 2024-05-11T12:12:52.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.084+0000 7f3a62200640 1 -- 172.21.15.167:0/2668232180 wait complete. 2024-05-11T12:12:52.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.086+0000 7f3a62200640 1 Processor -- start 2024-05-11T12:12:52.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.087+0000 7f3a62200640 1 -- start start 2024-05-11T12:12:52.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.088+0000 7f3a62200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a5c070540 0x7f3a5c13c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:52.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.088+0000 7f3a62200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3a5c0c9a30 0x7f3a5c06efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:52.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.088+0000 7f3a62200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a5c146960 0x7f3a5c15e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:52.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.088+0000 7f3a62200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5c070cf0 con 0x7f3a5c146960 2024-05-11T12:12:52.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.088+0000 7f3a62200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5c146d40 con 0x7f3a5c070540 2024-05-11T12:12:52.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.088+0000 7f3a62200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3a5c070e70 con 0x7f3a5c0c9a30 2024-05-11T12:12:52.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.088+0000 7f3a5be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a5c146960 0x7f3a5c15e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:52.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.088+0000 7f3a5be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a5c146960 0x7f3a5c15e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56096/0 (socket says 172.21.15.167:56096) 2024-05-11T12:12:52.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.088+0000 7f3a5be00640 1 -- 172.21.15.167:0/3850953419 learned_addr learned my addr 172.21.15.167:0/3850953419 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:52.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.088+0000 7f3a5b400640 1 --2- 172.21.15.167:0/3850953419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a5c070540 0x7f3a5c13c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:52.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.089+0000 7f3a5aa00640 1 --2- 172.21.15.167:0/3850953419 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3a5c0c9a30 0x7f3a5c06efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:52.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.089+0000 7f3a5be00640 1 -- 172.21.15.167:0/3850953419 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3a5c0c9a30 msgr2=0x7f3a5c06efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:52.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.089+0000 7f3a5be00640 1 --2- 172.21.15.167:0/3850953419 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3a5c0c9a30 0x7f3a5c06efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:52.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.089+0000 7f3a5be00640 1 -- 172.21.15.167:0/3850953419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a5c070540 msgr2=0x7f3a5c13c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:52.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.089+0000 7f3a5be00640 1 --2- 172.21.15.167:0/3850953419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a5c070540 0x7f3a5c13c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:52.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.089+0000 7f3a5be00640 1 -- 172.21.15.167:0/3850953419 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a5c141f60 con 0x7f3a5c146960 2024-05-11T12:12:52.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.089+0000 7f3a5be00640 1 --2- 172.21.15.167:0/3850953419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a5c146960 0x7f3a5c15e040 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3a5402abe0 tx=0x7f3a54004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:52.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.089+0000 7f3a53400640 1 -- 172.21.15.167:0/3850953419 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a54040070 con 0x7f3a5c146960 2024-05-11T12:12:52.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.089+0000 7f3a53400640 1 -- 172.21.15.167:0/3850953419 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a54031c40 con 0x7f3a5c146960 2024-05-11T12:12:52.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.089+0000 7f3a53400640 1 -- 172.21.15.167:0/3850953419 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a54003990 con 0x7f3a5c146960 2024-05-11T12:12:52.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.089+0000 7f3a62200640 1 -- 172.21.15.167:0/3850953419 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a5c142240 con 0x7f3a5c146960 2024-05-11T12:12:52.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.089+0000 7f3a62200640 1 -- 172.21.15.167:0/3850953419 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a5c142800 con 0x7f3a5c146960 2024-05-11T12:12:52.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.091+0000 7f3a62200640 1 -- 172.21.15.167:0/3850953419 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a5c0752e0 con 0x7f3a5c146960 2024-05-11T12:12:52.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.092+0000 7f3a53400640 1 -- 172.21.15.167:0/3850953419 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3a54039040 con 0x7f3a5c146960 2024-05-11T12:12:52.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.093+0000 7f3a53400640 1 --2- 172.21.15.167:0/3850953419 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3a3c0762d0 0x7f3a3c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:52.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.093+0000 7f3a53400640 1 -- 172.21.15.167:0/3850953419 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6187+0+0 (secure 0 0 0) 0x7f3a540b8c50 con 0x7f3a5c146960 2024-05-11T12:12:52.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.093+0000 7f3a5b400640 1 --2- 172.21.15.167:0/3850953419 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3a3c0762d0 0x7f3a3c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:52.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.094+0000 7f3a5b400640 1 --2- 172.21.15.167:0/3850953419 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3a3c0762d0 0x7f3a3c078790 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c000f60 tx=0x7f3a4c007c00 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:52.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.098+0000 7f3a53400640 1 -- 172.21.15.167:0/3850953419 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3a54082220 con 0x7f3a5c146960 2024-05-11T12:12:52.306 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:52.306+0000 7f3a62200640 1 -- 172.21.15.167:0/3850953419 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3a5c071580 con 0x7f3a5c146960 2024-05-11T12:12:53.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.001+0000 7f3a53400640 1 -- 172.21.15.167:0/3850953419 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v55) ==== 181+0+0 (secure 0 0 0) 0x7f3a5408a2a0 con 0x7f3a5c146960 2024-05-11T12:12:53.002 INFO:teuthology.orchestra.run.smithi167.stderr:pool 'cephfs_metadata' removed 2024-05-11T12:12:53.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.005+0000 7f3a62200640 1 -- 172.21.15.167:0/3850953419 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3a3c0762d0 msgr2=0x7f3a3c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:53.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.005+0000 7f3a62200640 1 --2- 172.21.15.167:0/3850953419 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3a3c0762d0 0x7f3a3c078790 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c000f60 tx=0x7f3a4c007c00 comp rx=0 tx=0).stop 2024-05-11T12:12:53.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.005+0000 7f3a62200640 1 -- 172.21.15.167:0/3850953419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a5c146960 msgr2=0x7f3a5c15e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:53.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.005+0000 7f3a62200640 1 --2- 172.21.15.167:0/3850953419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a5c146960 0x7f3a5c15e040 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3a5402abe0 tx=0x7f3a54004a40 comp rx=0 tx=0).stop 2024-05-11T12:12:53.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.005+0000 7f3a62200640 1 -- 172.21.15.167:0/3850953419 shutdown_connections 2024-05-11T12:12:53.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.005+0000 7f3a62200640 1 --2- 172.21.15.167:0/3850953419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a5c146960 0x7f3a5c15e040 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:53.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.005+0000 7f3a62200640 1 --2- 172.21.15.167:0/3850953419 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3a5c0c9a30 0x7f3a5c06efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:53.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.005+0000 7f3a62200640 1 --2- 172.21.15.167:0/3850953419 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3a3c0762d0 0x7f3a3c078790 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:53.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.005+0000 7f3a62200640 1 --2- 172.21.15.167:0/3850953419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a5c070540 0x7f3a5c13c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:53.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.005+0000 7f3a62200640 1 -- 172.21.15.167:0/3850953419 >> 172.21.15.167:0/3850953419 conn(0x7f3a5c0b4ae0 msgr2=0x7f3a5c0d2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:53.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.005+0000 7f3a62200640 1 -- 172.21.15.167:0/3850953419 shutdown_connections 2024-05-11T12:12:53.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.005+0000 7f3a62200640 1 -- 172.21.15.167:0/3850953419 wait complete. 2024-05-11T12:12:53.028 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-11T12:12:53.224 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.224+0000 7fb35f800640 1 Processor -- start 2024-05-11T12:12:53.224 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.224+0000 7fb35f800640 1 -- start start 2024-05-11T12:12:53.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.225+0000 7fb35f800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb35815cda0 0x7fb3580b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:53.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.225+0000 7fb35f800640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb358070140 0x7fb3580b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:53.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.225+0000 7fb35f800640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb35815f970 con 0x7fb3580b5eb0 2024-05-11T12:12:53.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.225+0000 7fb35f800640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fb35815fc50 con 0x7fb358070140 2024-05-11T12:12:53.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.225+0000 7fb35f800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fb35815ff30 con 0x7fb35815cda0 2024-05-11T12:12:53.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.226+0000 7fb357e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fb3580b5eb0 0x7fb35806fa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49348/0 (socket says 172.21.15.167:49348) 2024-05-11T12:12:53.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.226+0000 7fb357e00640 1 -- 172.21.15.167:0/3695477122 learned_addr learned my addr 172.21.15.167:0/3695477122 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:53.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.226+0000 7fb35d800640 1 --2- 172.21.15.167:0/3695477122 >> v2:172.21.15.167:3300/0 conn(0x7fb358070140 0x7fb3580b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:53.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.226+0000 7fb35ce00640 1 --2- 172.21.15.167:0/3695477122 >> v2:172.21.15.204:3300/0 conn(0x7fb35815cda0 0x7fb3580b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:53.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.226+0000 7fb357400640 1 -- 172.21.15.167:0/3695477122 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1190549464 0 0) 0x7fb35815f970 con 0x7fb3580b5eb0 2024-05-11T12:12:53.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.226+0000 7fb357400640 1 -- 172.21.15.167:0/3695477122 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb340003520 con 0x7fb3580b5eb0 2024-05-11T12:12:53.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.227+0000 7fb35d800640 1 -- 172.21.15.167:0/3695477122 >> v2:172.21.15.204:3300/0 conn(0x7fb35815cda0 msgr2=0x7fb3580b5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:53.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.227+0000 7fb35d800640 1 --2- 172.21.15.167:0/3695477122 >> v2:172.21.15.204:3300/0 conn(0x7fb35815cda0 0x7fb3580b5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:53.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.227+0000 7fb35d800640 1 -- 172.21.15.167:0/3695477122 >> v1:172.21.15.167:6789/0 conn(0x7fb3580b5eb0 legacy=0x7fb35806fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:53.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.227+0000 7fb35d800640 1 -- 172.21.15.167:0/3695477122 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb358160210 con 0x7fb358070140 2024-05-11T12:12:53.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.227+0000 7fb35ce00640 1 --2- 172.21.15.167:0/3695477122 >> v2:172.21.15.204:3300/0 conn(0x7fb35815cda0 0x7fb3580b5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:12:53.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.228+0000 7fb35d800640 1 --2- 172.21.15.167:0/3695477122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb358070140 0x7fb3580b3de0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb3580ae840 tx=0x7fb35002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ac1a8a455474a26 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:53.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.228+0000 7fb357400640 1 -- 172.21.15.167:0/3695477122 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb350007740 con 0x7fb358070140 2024-05-11T12:12:53.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.228+0000 7fb357400640 1 -- 172.21.15.167:0/3695477122 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3500078e0 con 0x7fb358070140 2024-05-11T12:12:53.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.228+0000 7fb357400640 1 -- 172.21.15.167:0/3695477122 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb350007cb0 con 0x7fb358070140 2024-05-11T12:12:53.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.229+0000 7fb35f800640 1 -- 172.21.15.167:0/3695477122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb358070140 msgr2=0x7fb3580b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:53.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.229+0000 7fb35f800640 1 --2- 172.21.15.167:0/3695477122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb358070140 0x7fb3580b3de0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb3580ae840 tx=0x7fb35002a680 comp rx=0 tx=0).stop 2024-05-11T12:12:53.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.229+0000 7fb35f800640 1 -- 172.21.15.167:0/3695477122 shutdown_connections 2024-05-11T12:12:53.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.229+0000 7fb35f800640 1 --2- 172.21.15.167:0/3695477122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb358070140 0x7fb3580b3de0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:53.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.229+0000 7fb35f800640 1 --2- 172.21.15.167:0/3695477122 >> v2:172.21.15.204:3300/0 conn(0x7fb35815cda0 0x7fb3580b5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:53.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.229+0000 7fb35f800640 1 -- 172.21.15.167:0/3695477122 >> 172.21.15.167:0/3695477122 conn(0x7fb3580d0c70 msgr2=0x7fb3580d1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:53.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.229+0000 7fb35f800640 1 -- 172.21.15.167:0/3695477122 shutdown_connections 2024-05-11T12:12:53.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.229+0000 7fb35f800640 1 -- 172.21.15.167:0/3695477122 wait complete. 2024-05-11T12:12:53.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.231+0000 7fb35f800640 1 Processor -- start 2024-05-11T12:12:53.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.231+0000 7fb35f800640 1 -- start start 2024-05-11T12:12:53.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.232+0000 7fb35f800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb358070140 0x7fb3581465c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:53.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.232+0000 7fb35f800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3580b5eb0 0x7fb358155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:53.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.232+0000 7fb35f800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb35815cda0 0x7fb3581562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:53.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.232+0000 7fb35f800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb3580af6c0 con 0x7fb3580b5eb0 2024-05-11T12:12:53.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.232+0000 7fb35f800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb35815d180 con 0x7fb35815cda0 2024-05-11T12:12:53.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.232+0000 7fb35f800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb3580b4320 con 0x7fb358070140 2024-05-11T12:12:53.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.232+0000 7fb357e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3580b5eb0 0x7fb358155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:53.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.232+0000 7fb357e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3580b5eb0 0x7fb358155db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56108/0 (socket says 172.21.15.167:56108) 2024-05-11T12:12:53.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.232+0000 7fb357e00640 1 -- 172.21.15.167:0/737658396 learned_addr learned my addr 172.21.15.167:0/737658396 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:53.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.232+0000 7fb357e00640 1 -- 172.21.15.167:0/737658396 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb358070140 msgr2=0x7fb3581465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:53.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.232+0000 7fb35ce00640 1 --2- 172.21.15.167:0/737658396 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb358070140 0x7fb3581465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:53.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.232+0000 7fb35d800640 1 --2- 172.21.15.167:0/737658396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb35815cda0 0x7fb3581562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:53.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.232+0000 7fb357e00640 1 --2- 172.21.15.167:0/737658396 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb358070140 0x7fb3581465c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:53.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.232+0000 7fb357e00640 1 -- 172.21.15.167:0/737658396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb35815cda0 msgr2=0x7fb3581562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:53.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.232+0000 7fb357e00640 1 --2- 172.21.15.167:0/737658396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb35815cda0 0x7fb3581562f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:53.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.232+0000 7fb357e00640 1 -- 172.21.15.167:0/737658396 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb358146b00 con 0x7fb3580b5eb0 2024-05-11T12:12:53.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.233+0000 7fb357e00640 1 --2- 172.21.15.167:0/737658396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3580b5eb0 0x7fb358155db0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fb34400c4d0 tx=0x7fb34400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:53.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.233+0000 7fb354c00640 1 -- 172.21.15.167:0/737658396 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb344017070 con 0x7fb3580b5eb0 2024-05-11T12:12:53.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.233+0000 7fb354c00640 1 -- 172.21.15.167:0/737658396 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb344005ce0 con 0x7fb3580b5eb0 2024-05-11T12:12:53.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.233+0000 7fb35f800640 1 -- 172.21.15.167:0/737658396 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb35815b4b0 con 0x7fb3580b5eb0 2024-05-11T12:12:53.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.233+0000 7fb354c00640 1 -- 172.21.15.167:0/737658396 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb344007290 con 0x7fb3580b5eb0 2024-05-11T12:12:53.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.233+0000 7fb35f800640 1 -- 172.21.15.167:0/737658396 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb35815ba70 con 0x7fb3580b5eb0 2024-05-11T12:12:53.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.235+0000 7fb35f800640 1 -- 172.21.15.167:0/737658396 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb35815c570 con 0x7fb3580b5eb0 2024-05-11T12:12:53.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.236+0000 7fb354c00640 1 -- 172.21.15.167:0/737658396 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb34401a070 con 0x7fb3580b5eb0 2024-05-11T12:12:53.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.237+0000 7fb354c00640 1 --2- 172.21.15.167:0/737658396 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb3380762d0 0x7fb338078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:53.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.237+0000 7fb354c00640 1 -- 172.21.15.167:0/737658396 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5749+0+0 (secure 0 0 0) 0x7fb344096880 con 0x7fb3580b5eb0 2024-05-11T12:12:53.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.237+0000 7fb35ce00640 1 --2- 172.21.15.167:0/737658396 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb3380762d0 0x7fb338078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:53.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.238+0000 7fb35ce00640 1 --2- 172.21.15.167:0/737658396 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb3380762d0 0x7fb338078790 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb34c006fd0 tx=0x7fb34c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:53.241 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.241+0000 7fb354c00640 1 -- 172.21.15.167:0/737658396 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb344060300 con 0x7fb3580b5eb0 2024-05-11T12:12:53.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:53.454+0000 7fb35f800640 1 -- 172.21.15.167:0/737658396 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb358069590 con 0x7fb3580b5eb0 2024-05-11T12:12:54.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.012+0000 7fb354c00640 1 -- 172.21.15.167:0/737658396 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v56) ==== 169+0+0 (secure 0 0 0) 0x7fb34405fc00 con 0x7fb3580b5eb0 2024-05-11T12:12:54.012 INFO:teuthology.orchestra.run.smithi167.stderr:pool 'cephfs_data' removed 2024-05-11T12:12:54.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.019+0000 7fb35f800640 1 -- 172.21.15.167:0/737658396 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb3380762d0 msgr2=0x7fb338078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:54.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.019+0000 7fb35f800640 1 --2- 172.21.15.167:0/737658396 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb3380762d0 0x7fb338078790 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb34c006fd0 tx=0x7fb34c008040 comp rx=0 tx=0).stop 2024-05-11T12:12:54.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.019+0000 7fb35f800640 1 -- 172.21.15.167:0/737658396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3580b5eb0 msgr2=0x7fb358155db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:54.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.019+0000 7fb35f800640 1 --2- 172.21.15.167:0/737658396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3580b5eb0 0x7fb358155db0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fb34400c4d0 tx=0x7fb34400c990 comp rx=0 tx=0).stop 2024-05-11T12:12:54.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.021+0000 7fb35f800640 1 -- 172.21.15.167:0/737658396 shutdown_connections 2024-05-11T12:12:54.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.021+0000 7fb35f800640 1 --2- 172.21.15.167:0/737658396 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb3380762d0 0x7fb338078790 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:54.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.021+0000 7fb35f800640 1 --2- 172.21.15.167:0/737658396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb35815cda0 0x7fb3581562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:54.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.021+0000 7fb35f800640 1 --2- 172.21.15.167:0/737658396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3580b5eb0 0x7fb358155db0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:54.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.021+0000 7fb35f800640 1 --2- 172.21.15.167:0/737658396 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb358070140 0x7fb3581465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:54.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.021+0000 7fb35f800640 1 -- 172.21.15.167:0/737658396 >> 172.21.15.167:0/737658396 conn(0x7fb3580d0c70 msgr2=0x7fb3580ccef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:54.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.022+0000 7fb35f800640 1 -- 172.21.15.167:0/737658396 shutdown_connections 2024-05-11T12:12:54.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.022+0000 7fb35f800640 1 -- 172.21.15.167:0/737658396 wait complete. 2024-05-11T12:12:54.046 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-11T12:12:54.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.227+0000 7f1523e00640 1 Processor -- start 2024-05-11T12:12:54.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.228+0000 7f1523e00640 1 -- start start 2024-05-11T12:12:54.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.229+0000 7f1523e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f152415f130 0x7f1524071e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:54.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.229+0000 7f1523e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f15240afb60 0x7f15240affc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:54.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.229+0000 7f1523e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15240b4b40 con 0x7f15240b5ba0 2024-05-11T12:12:54.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.229+0000 7f1523e00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f15240b0500 con 0x7f15240afb60 2024-05-11T12:12:54.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.229+0000 7f1523e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f15240b4de0 con 0x7f152415f130 2024-05-11T12:12:54.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.230+0000 7f1522000640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f15240afb60 0x7f15240affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:54.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.230+0000 7f1522000640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f15240afb60 0x7f15240affc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56116/0 (socket says 172.21.15.167:56116) 2024-05-11T12:12:54.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.230+0000 7f1522000640 1 -- 172.21.15.167:0/3262835043 learned_addr learned my addr 172.21.15.167:0/3262835043 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:54.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.230+0000 7f1522000640 1 -- 172.21.15.167:0/3262835043 >> v2:172.21.15.204:3300/0 conn(0x7f152415f130 msgr2=0x7f1524071e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:54.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.230+0000 7f1522000640 1 --2- 172.21.15.167:0/3262835043 >> v2:172.21.15.204:3300/0 conn(0x7f152415f130 0x7f1524071e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:54.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.230+0000 7f1522000640 1 -- 172.21.15.167:0/3262835043 >> v1:172.21.15.204:6789/0 conn(0x7f15240b5ba0 legacy=0x7f15240b6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:54.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.230+0000 7f1522000640 1 -- 172.21.15.167:0/3262835043 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15240b50c0 con 0x7f15240afb60 2024-05-11T12:12:54.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.231+0000 7f1522000640 1 --2- 172.21.15.167:0/3262835043 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15240afb60 0x7f15240affc0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1524074430 tx=0x7f151402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d77071dbd0c7eb12 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:54.232 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.231+0000 7f1521600640 1 -- 172.21.15.167:0/3262835043 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1514037070 con 0x7f15240afb60 2024-05-11T12:12:54.232 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.232+0000 7f1521600640 1 -- 172.21.15.167:0/3262835043 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1514032440 con 0x7f15240afb60 2024-05-11T12:12:54.232 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.232+0000 7f1521600640 1 -- 172.21.15.167:0/3262835043 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1514032810 con 0x7f15240afb60 2024-05-11T12:12:54.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.232+0000 7f1523e00640 1 -- 172.21.15.167:0/3262835043 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15240afb60 msgr2=0x7f15240affc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:54.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.232+0000 7f1523e00640 1 --2- 172.21.15.167:0/3262835043 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15240afb60 0x7f15240affc0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1524074430 tx=0x7f151402a1e0 comp rx=0 tx=0).stop 2024-05-11T12:12:54.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.232+0000 7f1523e00640 1 -- 172.21.15.167:0/3262835043 shutdown_connections 2024-05-11T12:12:54.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.232+0000 7f1523e00640 1 --2- 172.21.15.167:0/3262835043 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15240afb60 0x7f15240affc0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:54.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.232+0000 7f1523e00640 1 --2- 172.21.15.167:0/3262835043 >> v2:172.21.15.204:3300/0 conn(0x7f152415f130 0x7f1524071e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:54.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.232+0000 7f1523e00640 1 -- 172.21.15.167:0/3262835043 >> 172.21.15.167:0/3262835043 conn(0x7f15240b2a20 msgr2=0x7f15240b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:54.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.232+0000 7f1523e00640 1 -- 172.21.15.167:0/3262835043 shutdown_connections 2024-05-11T12:12:54.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.232+0000 7f1523e00640 1 -- 172.21.15.167:0/3262835043 wait complete. 2024-05-11T12:12:54.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.234+0000 7f1523e00640 1 Processor -- start 2024-05-11T12:12:54.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.234+0000 7f1523e00640 1 -- start start 2024-05-11T12:12:54.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.235+0000 7f1523e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15240afb60 0x7f152415bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:54.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.235+0000 7f1523e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15240b5ba0 0x7f15241eb000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:54.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.235+0000 7f1523e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f152415f130 0x7f152415d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:54.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.235+0000 7f1523e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1524072540 con 0x7f152415f130 2024-05-11T12:12:54.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.235+0000 7f1523e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f15240723c0 con 0x7f15240afb60 2024-05-11T12:12:54.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.235+0000 7f1523e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f152406d340 con 0x7f15240b5ba0 2024-05-11T12:12:54.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.236+0000 7f1523400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f152415f130 0x7f152415d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:54.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.236+0000 7f1523400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f152415f130 0x7f152415d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56128/0 (socket says 172.21.15.167:56128) 2024-05-11T12:12:54.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.236+0000 7f1523400640 1 -- 172.21.15.167:0/1435472835 learned_addr learned my addr 172.21.15.167:0/1435472835 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:54.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.236+0000 7f1522a00640 1 --2- 172.21.15.167:0/1435472835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15240afb60 0x7f152415bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:54.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.236+0000 7f1522000640 1 --2- 172.21.15.167:0/1435472835 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15240b5ba0 0x7f15241eb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:54.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.236+0000 7f1523400640 1 -- 172.21.15.167:0/1435472835 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15240b5ba0 msgr2=0x7f15241eb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:54.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.236+0000 7f1523400640 1 --2- 172.21.15.167:0/1435472835 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15240b5ba0 0x7f15241eb000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:54.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.236+0000 7f1523400640 1 -- 172.21.15.167:0/1435472835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15240afb60 msgr2=0x7f152415bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:54.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.236+0000 7f1523400640 1 --2- 172.21.15.167:0/1435472835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15240afb60 0x7f152415bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:54.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.236+0000 7f1523400640 1 -- 172.21.15.167:0/1435472835 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15241eb540 con 0x7f152415f130 2024-05-11T12:12:54.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.236+0000 7f1522a00640 1 --2- 172.21.15.167:0/1435472835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15240afb60 0x7f152415bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:54.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.236+0000 7f1523400640 1 --2- 172.21.15.167:0/1435472835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f152415f130 0x7f152415d5a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f151c00c780 tx=0x7f151c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:54.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.237+0000 7f151aa00640 1 -- 172.21.15.167:0/1435472835 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c017070 con 0x7f152415f130 2024-05-11T12:12:54.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.237+0000 7f151aa00640 1 -- 172.21.15.167:0/1435472835 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f151c003da0 con 0x7f152415f130 2024-05-11T12:12:54.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.237+0000 7f1523e00640 1 -- 172.21.15.167:0/1435472835 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15241e9f30 con 0x7f152415f130 2024-05-11T12:12:54.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.237+0000 7f151aa00640 1 -- 172.21.15.167:0/1435472835 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c0052b0 con 0x7f152415f130 2024-05-11T12:12:54.239 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.237+0000 7f1523e00640 1 -- 172.21.15.167:0/1435472835 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15241ea4f0 con 0x7f152415f130 2024-05-11T12:12:54.239 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.238+0000 7f1523e00640 1 -- 172.21.15.167:0/1435472835 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15240affc0 con 0x7f152415f130 2024-05-11T12:12:54.239 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.239+0000 7f151aa00640 1 -- 172.21.15.167:0/1435472835 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f151c01a070 con 0x7f152415f130 2024-05-11T12:12:54.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.240+0000 7f151aa00640 1 --2- 172.21.15.167:0/1435472835 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f15000762d0 0x7f1500078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:54.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.240+0000 7f151aa00640 1 -- 172.21.15.167:0/1435472835 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7f151c096b80 con 0x7f152415f130 2024-05-11T12:12:54.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.240+0000 7f1522a00640 1 --2- 172.21.15.167:0/1435472835 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f15000762d0 0x7f1500078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:54.241 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.241+0000 7f1522a00640 1 --2- 172.21.15.167:0/1435472835 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f15000762d0 0x7f1500078790 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f150c00a870 tx=0x7f150c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:54.245 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.244+0000 7f151aa00640 1 -- 172.21.15.167:0/1435472835 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f15240affc0 con 0x7f152415f130 2024-05-11T12:12:54.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.455+0000 7f1523e00640 1 -- 172.21.15.167:0/1435472835 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f15240694b0 con 0x7f152415f130 2024-05-11T12:12:54.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.466+0000 7f151aa00640 1 -- 172.21.15.167:0/1435472835 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) ==== 104+0+0 (secure 0 0 0) 0x7f151c0605b0 con 0x7f152415f130 2024-05-11T12:12:54.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.470+0000 7f1523e00640 1 -- 172.21.15.167:0/1435472835 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f15000762d0 msgr2=0x7f1500078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:54.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.470+0000 7f1523e00640 1 --2- 172.21.15.167:0/1435472835 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f15000762d0 0x7f1500078790 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f150c00a870 tx=0x7f150c008040 comp rx=0 tx=0).stop 2024-05-11T12:12:54.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.470+0000 7f1523e00640 1 -- 172.21.15.167:0/1435472835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f152415f130 msgr2=0x7f152415d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:54.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.470+0000 7f1523e00640 1 --2- 172.21.15.167:0/1435472835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f152415f130 0x7f152415d5a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f151c00c780 tx=0x7f151c00cc40 comp rx=0 tx=0).stop 2024-05-11T12:12:54.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.470+0000 7f1523e00640 1 -- 172.21.15.167:0/1435472835 shutdown_connections 2024-05-11T12:12:54.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.470+0000 7f1523e00640 1 --2- 172.21.15.167:0/1435472835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f152415f130 0x7f152415d5a0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:54.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.470+0000 7f1523e00640 1 --2- 172.21.15.167:0/1435472835 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15240b5ba0 0x7f15241eb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:54.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.470+0000 7f1523e00640 1 --2- 172.21.15.167:0/1435472835 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f15000762d0 0x7f1500078790 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:54.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.470+0000 7f1523e00640 1 --2- 172.21.15.167:0/1435472835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15240afb60 0x7f152415bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:54.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.470+0000 7f1523e00640 1 -- 172.21.15.167:0/1435472835 >> 172.21.15.167:0/1435472835 conn(0x7f15240b2a20 msgr2=0x7f1524161f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:54.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.470+0000 7f1523e00640 1 -- 172.21.15.167:0/1435472835 shutdown_connections 2024-05-11T12:12:54.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.471+0000 7f1523e00640 1 -- 172.21.15.167:0/1435472835 wait complete. 2024-05-11T12:12:54.490 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_evict_blocklisted' 2024-05-11T12:12:54.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.677+0000 7f29a6200640 1 Processor -- start 2024-05-11T12:12:54.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.677+0000 7f29a6200640 1 -- start start 2024-05-11T12:12:54.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.678+0000 7f29a6200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f29a013f290 0x7f29a00735c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:54.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.678+0000 7f29a6200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f29a00b50a0 0x7f29a0073b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:54.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.678+0000 7f29a6200640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29a015fb90 con 0x7f29a013eeb0 2024-05-11T12:12:54.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.679+0000 7f29a6200640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f29a015d5b0 con 0x7f29a00b50a0 2024-05-11T12:12:54.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.679+0000 7f29a6200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f29a015fe70 con 0x7f29a013f290 2024-05-11T12:12:54.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.679+0000 7f299f400640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f29a013eeb0 0x7f29a006f480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49358/0 (socket says 172.21.15.167:49358) 2024-05-11T12:12:54.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.679+0000 7f299f400640 1 -- 172.21.15.167:0/1708741673 learned_addr learned my addr 172.21.15.167:0/1708741673 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:54.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.679+0000 7f299fe00640 1 --2- 172.21.15.167:0/1708741673 >> v2:172.21.15.167:3300/0 conn(0x7f29a00b50a0 0x7f29a0073b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:54.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.679+0000 7f299ea00640 1 --2- 172.21.15.167:0/1708741673 >> v2:172.21.15.204:3300/0 conn(0x7f29a013f290 0x7f29a00735c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:54.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.679+0000 7f299e000640 1 -- 172.21.15.167:0/1708741673 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 494013197 0 0) 0x7f29a015fb90 con 0x7f29a013eeb0 2024-05-11T12:12:54.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.679+0000 7f299e000640 1 -- 172.21.15.167:0/1708741673 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2984003520 con 0x7f29a013eeb0 2024-05-11T12:12:54.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.679+0000 7f299fe00640 1 -- 172.21.15.167:0/1708741673 >> v2:172.21.15.204:3300/0 conn(0x7f29a013f290 msgr2=0x7f29a00735c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:54.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.679+0000 7f299fe00640 1 --2- 172.21.15.167:0/1708741673 >> v2:172.21.15.204:3300/0 conn(0x7f29a013f290 0x7f29a00735c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:54.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.679+0000 7f299fe00640 1 -- 172.21.15.167:0/1708741673 >> v1:172.21.15.167:6789/0 conn(0x7f29a013eeb0 legacy=0x7f29a006f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:54.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.679+0000 7f299fe00640 1 -- 172.21.15.167:0/1708741673 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29a0160150 con 0x7f29a00b50a0 2024-05-11T12:12:54.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.681+0000 7f299fe00640 1 --2- 172.21.15.167:0/1708741673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29a00b50a0 0x7f29a0073b00 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f29a0074520 tx=0x7f299802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d9020f70cbf4e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:54.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.681+0000 7f299e000640 1 -- 172.21.15.167:0/1708741673 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2998007740 con 0x7f29a00b50a0 2024-05-11T12:12:54.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.681+0000 7f299e000640 1 -- 172.21.15.167:0/1708741673 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29980078e0 con 0x7f29a00b50a0 2024-05-11T12:12:54.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.681+0000 7f299e000640 1 -- 172.21.15.167:0/1708741673 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2998007cb0 con 0x7f29a00b50a0 2024-05-11T12:12:54.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.682+0000 7f29a6200640 1 -- 172.21.15.167:0/1708741673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29a00b50a0 msgr2=0x7f29a0073b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:54.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.682+0000 7f29a6200640 1 --2- 172.21.15.167:0/1708741673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29a00b50a0 0x7f29a0073b00 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f29a0074520 tx=0x7f299802a5f0 comp rx=0 tx=0).stop 2024-05-11T12:12:54.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.682+0000 7f29a6200640 1 -- 172.21.15.167:0/1708741673 shutdown_connections 2024-05-11T12:12:54.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.682+0000 7f29a6200640 1 --2- 172.21.15.167:0/1708741673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29a00b50a0 0x7f29a0073b00 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:54.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.682+0000 7f29a6200640 1 --2- 172.21.15.167:0/1708741673 >> v2:172.21.15.204:3300/0 conn(0x7f29a013f290 0x7f29a00735c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:54.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.682+0000 7f29a6200640 1 -- 172.21.15.167:0/1708741673 >> 172.21.15.167:0/1708741673 conn(0x7f29a00cd8f0 msgr2=0x7f29a00cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:54.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.682+0000 7f29a6200640 1 -- 172.21.15.167:0/1708741673 shutdown_connections 2024-05-11T12:12:54.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.682+0000 7f29a6200640 1 -- 172.21.15.167:0/1708741673 wait complete. 2024-05-11T12:12:54.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.684+0000 7f29a6200640 1 Processor -- start 2024-05-11T12:12:54.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.684+0000 7f29a6200640 1 -- start start 2024-05-11T12:12:54.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.685+0000 7f29a6200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29a00b50a0 0x7f29a014bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:54.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.685+0000 7f29a6200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29a013eeb0 0x7f29a013af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:54.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.685+0000 7f29a6200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29a013f290 0x7f29a00d0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:54.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.685+0000 7f29a6200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f29a0074040 con 0x7f29a013f290 2024-05-11T12:12:54.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.685+0000 7f29a6200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f29a0160710 con 0x7f29a013eeb0 2024-05-11T12:12:54.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.685+0000 7f29a6200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f29a006fb90 con 0x7f29a00b50a0 2024-05-11T12:12:54.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.686+0000 7f299fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29a013f290 0x7f29a00d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:54.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.686+0000 7f299fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29a013f290 0x7f29a00d0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56150/0 (socket says 172.21.15.167:56150) 2024-05-11T12:12:54.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.686+0000 7f299fe00640 1 -- 172.21.15.167:0/1957263924 learned_addr learned my addr 172.21.15.167:0/1957263924 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:54.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.686+0000 7f299ea00640 1 --2- 172.21.15.167:0/1957263924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29a013eeb0 0x7f29a013af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:54.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.686+0000 7f299f400640 1 --2- 172.21.15.167:0/1957263924 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29a00b50a0 0x7f29a014bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:54.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.686+0000 7f299fe00640 1 -- 172.21.15.167:0/1957263924 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29a00b50a0 msgr2=0x7f29a014bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:54.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.686+0000 7f299fe00640 1 --2- 172.21.15.167:0/1957263924 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29a00b50a0 0x7f29a014bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:54.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.686+0000 7f299fe00640 1 -- 172.21.15.167:0/1957263924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29a013eeb0 msgr2=0x7f29a013af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:54.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.686+0000 7f299fe00640 1 --2- 172.21.15.167:0/1957263924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29a013eeb0 0x7f29a013af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:54.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.686+0000 7f299fe00640 1 -- 172.21.15.167:0/1957263924 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29a014fac0 con 0x7f29a013f290 2024-05-11T12:12:54.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.686+0000 7f299f400640 1 --2- 172.21.15.167:0/1957263924 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29a00b50a0 0x7f29a014bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:54.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.687+0000 7f299fe00640 1 --2- 172.21.15.167:0/1957263924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29a013f290 0x7f29a00d0150 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2998009e50 tx=0x7f2998004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:54.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.687+0000 7f2997400640 1 -- 172.21.15.167:0/1957263924 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2998040070 con 0x7f29a013f290 2024-05-11T12:12:54.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.687+0000 7f2997400640 1 -- 172.21.15.167:0/1957263924 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2998031bf0 con 0x7f29a013f290 2024-05-11T12:12:54.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.687+0000 7f2997400640 1 -- 172.21.15.167:0/1957263924 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2998009450 con 0x7f29a013f290 2024-05-11T12:12:54.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.687+0000 7f29a6200640 1 -- 172.21.15.167:0/1957263924 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29a013b950 con 0x7f29a013f290 2024-05-11T12:12:54.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.687+0000 7f29a6200640 1 -- 172.21.15.167:0/1957263924 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29a014fd80 con 0x7f29a013f290 2024-05-11T12:12:54.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.688+0000 7f29a6200640 1 -- 172.21.15.167:0/1957263924 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29a0076420 con 0x7f29a013f290 2024-05-11T12:12:54.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.689+0000 7f2997400640 1 -- 172.21.15.167:0/1957263924 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2998039040 con 0x7f29a013f290 2024-05-11T12:12:54.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.690+0000 7f2997400640 1 --2- 172.21.15.167:0/1957263924 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f29800762d0 0x7f2980078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:54.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.690+0000 7f2997400640 1 -- 172.21.15.167:0/1957263924 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7f29980b8dc0 con 0x7f29a013f290 2024-05-11T12:12:54.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.690+0000 7f299f400640 1 --2- 172.21.15.167:0/1957263924 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f29800762d0 0x7f2980078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:54.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.691+0000 7f299f400640 1 --2- 172.21.15.167:0/1957263924 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f29800762d0 0x7f2980078790 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2990001220 tx=0x7f2990009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:54.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.695+0000 7f2997400640 1 -- 172.21.15.167:0/1957263924 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f29a0076420 con 0x7f29a013f290 2024-05-11T12:12:54.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:54.986+0000 7f29a6200640 1 -- 172.21.15.167:0/1957263924 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_evict_blocklisted"]} v 0) -- 0x7f29a0069590 con 0x7f29a013f290 2024-05-11T12:12:55.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.015+0000 7f2997400640 1 -- 172.21.15.167:0/1957263924 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_evict_blocklisted"]}]=0 v139) ==== 153+0+0 (secure 0 0 0) 0x7f29980827f0 con 0x7f29a013f290 2024-05-11T12:12:55.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.018+0000 7f29a6200640 1 -- 172.21.15.167:0/1957263924 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f29800762d0 msgr2=0x7f2980078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:55.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.018+0000 7f29a6200640 1 --2- 172.21.15.167:0/1957263924 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f29800762d0 0x7f2980078790 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2990001220 tx=0x7f2990009040 comp rx=0 tx=0).stop 2024-05-11T12:12:55.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.018+0000 7f29a6200640 1 -- 172.21.15.167:0/1957263924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29a013f290 msgr2=0x7f29a00d0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:55.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.018+0000 7f29a6200640 1 --2- 172.21.15.167:0/1957263924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29a013f290 0x7f29a00d0150 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2998009e50 tx=0x7f2998004a40 comp rx=0 tx=0).stop 2024-05-11T12:12:55.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.018+0000 7f29a6200640 1 -- 172.21.15.167:0/1957263924 shutdown_connections 2024-05-11T12:12:55.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.018+0000 7f29a6200640 1 --2- 172.21.15.167:0/1957263924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29a013f290 0x7f29a00d0150 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:55.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.018+0000 7f29a6200640 1 --2- 172.21.15.167:0/1957263924 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f29800762d0 0x7f2980078790 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:55.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.018+0000 7f29a6200640 1 --2- 172.21.15.167:0/1957263924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29a013eeb0 0x7f29a013af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:55.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.018+0000 7f29a6200640 1 --2- 172.21.15.167:0/1957263924 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29a00b50a0 0x7f29a014bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:55.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.018+0000 7f29a6200640 1 -- 172.21.15.167:0/1957263924 >> 172.21.15.167:0/1957263924 conn(0x7f29a00cd8f0 msgr2=0x7f29a0071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:55.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.018+0000 7f29a6200640 1 -- 172.21.15.167:0/1957263924 shutdown_connections 2024-05-11T12:12:55.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.019+0000 7f29a6200640 1 -- 172.21.15.167:0/1957263924 wait complete. 2024-05-11T12:12:55.038 INFO:tasks.cephfs_test_runner:Check that mds evicts blocklisted client ... skipped 'Requires FUSE client to use mds_cluster.is_addr_blocklisted()' 2024-05-11T12:12:55.038 INFO:tasks.cephfs_test_runner:Starting test: test_session_reject (tasks.cephfs.test_sessionmap.TestSessionMap) 2024-05-11T12:12:55.038 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:12:55.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.232+0000 7f1683e00640 1 Processor -- start 2024-05-11T12:12:55.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.233+0000 7f1683e00640 1 -- start start 2024-05-11T12:12:55.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.234+0000 7f1683e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1684159350 0x7f16840cf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:55.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.234+0000 7f1683e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f168415c450 0x7f16840b4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:55.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.234+0000 7f1683e00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f168415cab0 con 0x7f16840cf7a0 2024-05-11T12:12:55.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.234+0000 7f1683e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f168415fff0 con 0x7f1684159350 2024-05-11T12:12:55.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.234+0000 7f1683e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f168415f230 con 0x7f168415c450 2024-05-11T12:12:55.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.234+0000 7f167be00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f16840cf7a0 0x7f168415bd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49362/0 (socket says 172.21.15.167:49362) 2024-05-11T12:12:55.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.234+0000 7f167be00640 1 -- 172.21.15.167:0/2588401221 learned_addr learned my addr 172.21.15.167:0/2588401221 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:55.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.235+0000 7f1682a00640 1 --2- 172.21.15.167:0/2588401221 >> v2:172.21.15.204:3300/0 conn(0x7f1684159350 0x7f16840cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:55.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.235+0000 7f1683400640 1 --2- 172.21.15.167:0/2588401221 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f168415c450 0x7f16840b4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:55.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.235+0000 7f1682000640 1 -- 172.21.15.167:0/2588401221 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2277307911 0 0) 0x7f168415cab0 con 0x7f16840cf7a0 2024-05-11T12:12:55.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.235+0000 7f1682000640 1 -- 172.21.15.167:0/2588401221 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1668003520 con 0x7f16840cf7a0 2024-05-11T12:12:55.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.235+0000 7f1682000640 1 -- 172.21.15.167:0/2588401221 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2114089393 0 0) 0x7f1668003520 con 0x7f16840cf7a0 2024-05-11T12:12:55.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.235+0000 7f1682000640 1 -- 172.21.15.167:0/2588401221 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f168415c450 msgr2=0x7f16840b4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:55.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.235+0000 7f1682000640 1 --2- 172.21.15.167:0/2588401221 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f168415c450 0x7f16840b4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:55.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.235+0000 7f1682000640 1 -- 172.21.15.167:0/2588401221 >> v2:172.21.15.204:3300/0 conn(0x7f1684159350 msgr2=0x7f16840cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:55.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.235+0000 7f1682000640 1 --2- 172.21.15.167:0/2588401221 >> v2:172.21.15.204:3300/0 conn(0x7f1684159350 0x7f16840cf260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:55.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.235+0000 7f1682000640 1 -- 172.21.15.167:0/2588401221 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f168415f510 con 0x7f16840cf7a0 2024-05-11T12:12:55.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.235+0000 7f1682000640 1 -- 172.21.15.167:0/2588401221 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f166c0016f0 con 0x7f16840cf7a0 2024-05-11T12:12:55.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.235+0000 7f1682000640 1 -- 172.21.15.167:0/2588401221 >> v1:172.21.15.167:6789/0 conn(0x7f16840cf7a0 legacy=0x7f168415bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:55.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.235+0000 7f1682000640 1 --2- 172.21.15.167:0/2588401221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1668003a00 0x7f1668003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:55.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.235+0000 7f1682000640 1 --2- 172.21.15.167:0/2588401221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1668004800 0x7f1668004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:55.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.236+0000 7f1682000640 1 --2- 172.21.15.167:0/2588401221 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f168415c450 0x7f16680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:55.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.236+0000 7f1682000640 1 -- 172.21.15.167:0/2588401221 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f168415fff0 con 0x7f1668004800 2024-05-11T12:12:55.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.236+0000 7f1682000640 1 -- 172.21.15.167:0/2588401221 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f168415f230 con 0x7f1668003a00 2024-05-11T12:12:55.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.236+0000 7f1682000640 1 -- 172.21.15.167:0/2588401221 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f166800dc50 con 0x7f168415c450 2024-05-11T12:12:55.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.236+0000 7f167be00640 1 --2- 172.21.15.167:0/2588401221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1668004800 0x7f1668004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:55.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.236+0000 7f1683400640 1 --2- 172.21.15.167:0/2588401221 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f168415c450 0x7f16680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:55.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.236+0000 7f167be00640 1 -- 172.21.15.167:0/2588401221 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f168415c450 msgr2=0x7f16680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:55.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.236+0000 7f167be00640 1 --2- 172.21.15.167:0/2588401221 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f168415c450 0x7f16680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:55.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.236+0000 7f1682a00640 1 --2- 172.21.15.167:0/2588401221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1668003a00 0x7f1668003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:55.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.236+0000 7f167be00640 1 -- 172.21.15.167:0/2588401221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1668003a00 msgr2=0x7f1668003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:55.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.236+0000 7f167be00640 1 --2- 172.21.15.167:0/2588401221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1668003a00 0x7f1668003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:55.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.236+0000 7f167be00640 1 -- 172.21.15.167:0/2588401221 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1668003520 con 0x7f1668004800 2024-05-11T12:12:55.239 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.237+0000 7f167be00640 1 --2- 172.21.15.167:0/2588401221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1668004800 0x7f1668004c80 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f166c00c290 tx=0x7f1668005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89e417ee14300dc8 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:55.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.238+0000 7f1682000640 1 -- 172.21.15.167:0/2588401221 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166c007d50 con 0x7f1668004800 2024-05-11T12:12:55.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.238+0000 7f1682000640 1 -- 172.21.15.167:0/2588401221 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f166c005af0 con 0x7f1668004800 2024-05-11T12:12:55.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.238+0000 7f1682000640 1 -- 172.21.15.167:0/2588401221 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166c007070 con 0x7f1668004800 2024-05-11T12:12:55.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.238+0000 7f1683e00640 1 -- 172.21.15.167:0/2588401221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1668004800 msgr2=0x7f1668004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:55.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.238+0000 7f1683e00640 1 --2- 172.21.15.167:0/2588401221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1668004800 0x7f1668004c80 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f166c00c290 tx=0x7f1668005830 comp rx=0 tx=0).stop 2024-05-11T12:12:55.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.238+0000 7f1683400640 1 -- 172.21.15.167:0/2588401221 reap_dead start 2024-05-11T12:12:55.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.238+0000 7f1683e00640 1 -- 172.21.15.167:0/2588401221 shutdown_connections 2024-05-11T12:12:55.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.238+0000 7f1683e00640 1 -- 172.21.15.167:0/2588401221 >> 172.21.15.167:0/2588401221 conn(0x7f16840b3460 msgr2=0x7f16840d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:55.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.238+0000 7f1683e00640 1 -- 172.21.15.167:0/2588401221 shutdown_connections 2024-05-11T12:12:55.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.238+0000 7f1683e00640 1 -- 172.21.15.167:0/2588401221 wait complete. 2024-05-11T12:12:55.241 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.240+0000 7f1683e00640 1 Processor -- start 2024-05-11T12:12:55.241 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.240+0000 7f1683e00640 1 -- start start 2024-05-11T12:12:55.241 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.241+0000 7f1683e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1668004800 0x7f1684149ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:55.241 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.241+0000 7f1683e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f16840736c0 0x7f168415cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:55.242 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.241+0000 7f1683e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1684073b40 0x7f168415cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:55.242 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.241+0000 7f1683e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f16840ce9d0 con 0x7f1668004800 2024-05-11T12:12:55.242 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.241+0000 7f1683e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f168415fad0 con 0x7f1684073b40 2024-05-11T12:12:55.242 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.241+0000 7f1683e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f16840ceb50 con 0x7f16840736c0 2024-05-11T12:12:55.242 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.241+0000 7f1682a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1668004800 0x7f1684149ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:55.242 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.241+0000 7f1682a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1668004800 0x7f1684149ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56164/0 (socket says 172.21.15.167:56164) 2024-05-11T12:12:55.242 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.241+0000 7f1682a00640 1 -- 172.21.15.167:0/2785085376 learned_addr learned my addr 172.21.15.167:0/2785085376 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:55.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.242+0000 7f1683400640 1 --2- 172.21.15.167:0/2785085376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1684073b40 0x7f168415cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:55.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.242+0000 7f1682a00640 1 -- 172.21.15.167:0/2785085376 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f16840736c0 msgr2=0x7f168415cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:55.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.242+0000 7f1682a00640 1 --2- 172.21.15.167:0/2785085376 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f16840736c0 0x7f168415cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:55.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.242+0000 7f1682a00640 1 -- 172.21.15.167:0/2785085376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1684073b40 msgr2=0x7f168415cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:55.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.242+0000 7f1682a00640 1 --2- 172.21.15.167:0/2785085376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1684073b40 0x7f168415cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:55.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.242+0000 7f1682a00640 1 -- 172.21.15.167:0/2785085376 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f168414a000 con 0x7f1668004800 2024-05-11T12:12:55.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.242+0000 7f1682a00640 1 --2- 172.21.15.167:0/2785085376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1668004800 0x7f1684149ac0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f167400f980 tx=0x7f167400fe40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:55.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.242+0000 7f167aa00640 1 -- 172.21.15.167:0/2785085376 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f167400b040 con 0x7f1668004800 2024-05-11T12:12:55.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.242+0000 7f167aa00640 1 -- 172.21.15.167:0/2785085376 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f167400dc50 con 0x7f1668004800 2024-05-11T12:12:55.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.242+0000 7f167aa00640 1 -- 172.21.15.167:0/2785085376 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1674012460 con 0x7f1668004800 2024-05-11T12:12:55.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.242+0000 7f1683e00640 1 -- 172.21.15.167:0/2785085376 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f168406ec60 con 0x7f1668004800 2024-05-11T12:12:55.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.242+0000 7f1683e00640 1 -- 172.21.15.167:0/2785085376 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f168406f220 con 0x7f1668004800 2024-05-11T12:12:55.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.244+0000 7f1683e00640 1 -- 172.21.15.167:0/2785085376 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16840694b0 con 0x7f1668004800 2024-05-11T12:12:55.245 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.245+0000 7f167aa00640 1 -- 172.21.15.167:0/2785085376 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1674012600 con 0x7f1668004800 2024-05-11T12:12:55.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.246+0000 7f167aa00640 1 --2- 172.21.15.167:0/2785085376 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f16600762d0 0x7f1660078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:55.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.246+0000 7f167aa00640 1 -- 172.21.15.167:0/2785085376 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7f16740976b0 con 0x7f1668004800 2024-05-11T12:12:55.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.246+0000 7f167be00640 1 --2- 172.21.15.167:0/2785085376 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f16600762d0 0x7f1660078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:55.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.247+0000 7f167be00640 1 --2- 172.21.15.167:0/2785085376 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f16600762d0 0x7f1660078790 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f166c007fd0 tx=0x7f166c009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:55.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.251+0000 7f167aa00640 1 -- 172.21.15.167:0/2785085376 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1674017070 con 0x7f1668004800 2024-05-11T12:12:55.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.461+0000 7f1683e00640 1 -- 172.21.15.167:0/2785085376 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f16840d0670 con 0x7f1668004800 2024-05-11T12:12:55.463 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.463+0000 7f167aa00640 1 -- 172.21.15.167:0/2785085376 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) ==== 74+0+13343 (secure 0 0 0) 0x7f1674061180 con 0x7f1668004800 2024-05-11T12:12:55.463 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:12:55.464 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":56,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:12:54.003256+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.167:6835/4016299199":"2024-05-12T12:12:50.791556+0000","172.21.15.167:6834/4016299199":"2024-05-12T12:12:50.791556+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:12:55.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.466+0000 7f1683e00640 1 -- 172.21.15.167:0/2785085376 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f16600762d0 msgr2=0x7f1660078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:55.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.466+0000 7f1683e00640 1 --2- 172.21.15.167:0/2785085376 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f16600762d0 0x7f1660078790 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f166c007fd0 tx=0x7f166c009030 comp rx=0 tx=0).stop 2024-05-11T12:12:55.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.466+0000 7f1683e00640 1 -- 172.21.15.167:0/2785085376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1668004800 msgr2=0x7f1684149ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:55.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.466+0000 7f1683e00640 1 --2- 172.21.15.167:0/2785085376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1668004800 0x7f1684149ac0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f167400f980 tx=0x7f167400fe40 comp rx=0 tx=0).stop 2024-05-11T12:12:55.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.466+0000 7f1683e00640 1 -- 172.21.15.167:0/2785085376 shutdown_connections 2024-05-11T12:12:55.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.466+0000 7f1683e00640 1 --2- 172.21.15.167:0/2785085376 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f16600762d0 0x7f1660078790 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:55.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.466+0000 7f1683e00640 1 --2- 172.21.15.167:0/2785085376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1684073b40 0x7f168415cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:55.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.466+0000 7f1683e00640 1 --2- 172.21.15.167:0/2785085376 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f16840736c0 0x7f168415cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:55.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.466+0000 7f1683e00640 1 --2- 172.21.15.167:0/2785085376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1668004800 0x7f1684149ac0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:55.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.466+0000 7f1683e00640 1 -- 172.21.15.167:0/2785085376 >> 172.21.15.167:0/2785085376 conn(0x7f16840b3460 msgr2=0x7f16840b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:55.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.466+0000 7f1683e00640 1 -- 172.21.15.167:0/2785085376 shutdown_connections 2024-05-11T12:12:55.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.466+0000 7f1683e00640 1 -- 172.21.15.167:0/2785085376 wait complete. 2024-05-11T12:12:55.485 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-11T12:10:33.260270+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-11T12:12:55.485 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-11T12:12:55.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.690+0000 7f5cfd200640 1 Processor -- start 2024-05-11T12:12:55.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.690+0000 7f5cfd200640 1 -- start start 2024-05-11T12:12:55.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.692+0000 7f5cfd200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5cf8073270 0x7f5cf815d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:55.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.692+0000 7f5cfd200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cf80ce8f0 0x7f5cf80ced80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:55.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.692+0000 7f5cfd200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cf815c6d0 con 0x7f5cf8073650 2024-05-11T12:12:55.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.692+0000 7f5cfd200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f5cf806ead0 con 0x7f5cf8073270 2024-05-11T12:12:55.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.692+0000 7f5cfd200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5cf8160b00 con 0x7f5cf80ce8f0 2024-05-11T12:12:55.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.692+0000 7f5cf6800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cf80ce8f0 0x7f5cf80ced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:55.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.692+0000 7f5cf5e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5cf8073270 0x7f5cf815d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:55.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.692+0000 7f5cf6800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cf80ce8f0 0x7f5cf80ced80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:38744/0 (socket says 172.21.15.167:38744) 2024-05-11T12:12:55.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.692+0000 7f5cf6800640 1 -- 172.21.15.167:0/2272344147 learned_addr learned my addr 172.21.15.167:0/2272344147 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:55.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.692+0000 7f5cefe00640 1 --1- 172.21.15.167:0/2272344147 >> v1:172.21.15.204:6789/0 conn(0x7f5cf8073650 0x7f5cf806e3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.167:49616/0 (socket says 172.21.15.167:49616) 2024-05-11T12:12:55.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.693+0000 7f5cf5e00640 1 -- 172.21.15.167:0/2272344147 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cf80ce8f0 msgr2=0x7f5cf80ced80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:55.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.693+0000 7f5cf5e00640 1 --2- 172.21.15.167:0/2272344147 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cf80ce8f0 0x7f5cf80ced80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:55.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.693+0000 7f5cf5e00640 1 -- 172.21.15.167:0/2272344147 >> v1:172.21.15.204:6789/0 conn(0x7f5cf8073650 legacy=0x7f5cf806e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:55.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.693+0000 7f5cf5e00640 1 -- 172.21.15.167:0/2272344147 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf80700b0 con 0x7f5cf8073270 2024-05-11T12:12:55.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.693+0000 7f5cf6800640 1 --2- 172.21.15.167:0/2272344147 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cf80ce8f0 0x7f5cf80ced80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:12:55.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.695+0000 7f5cf5e00640 1 --2- 172.21.15.167:0/2272344147 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cf8073270 0x7f5cf815d170 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5ce80097d0 tx=0x7f5ce802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b4dac57a3a53719 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:55.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.695+0000 7f5cf5400640 1 -- 172.21.15.167:0/2272344147 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8037070 con 0x7f5cf8073270 2024-05-11T12:12:55.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.695+0000 7f5cf5400640 1 -- 172.21.15.167:0/2272344147 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ce8032440 con 0x7f5cf8073270 2024-05-11T12:12:55.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.695+0000 7f5cf5400640 1 -- 172.21.15.167:0/2272344147 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8032810 con 0x7f5cf8073270 2024-05-11T12:12:55.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.696+0000 7f5cfd200640 1 -- 172.21.15.167:0/2272344147 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cf8073270 msgr2=0x7f5cf815d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:55.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.696+0000 7f5cfd200640 1 --2- 172.21.15.167:0/2272344147 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cf8073270 0x7f5cf815d170 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5ce80097d0 tx=0x7f5ce802a1c0 comp rx=0 tx=0).stop 2024-05-11T12:12:55.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.696+0000 7f5cfd200640 1 -- 172.21.15.167:0/2272344147 shutdown_connections 2024-05-11T12:12:55.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.696+0000 7f5cfd200640 1 --2- 172.21.15.167:0/2272344147 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cf80ce8f0 0x7f5cf80ced80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:55.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.696+0000 7f5cfd200640 1 --2- 172.21.15.167:0/2272344147 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cf8073270 0x7f5cf815d170 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:55.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.696+0000 7f5cfd200640 1 -- 172.21.15.167:0/2272344147 >> 172.21.15.167:0/2272344147 conn(0x7f5cf806c450 msgr2=0x7f5cf806c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:55.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.696+0000 7f5cfd200640 1 -- 172.21.15.167:0/2272344147 shutdown_connections 2024-05-11T12:12:55.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.696+0000 7f5cfd200640 1 -- 172.21.15.167:0/2272344147 wait complete. 2024-05-11T12:12:55.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.697+0000 7f5cfd200640 1 Processor -- start 2024-05-11T12:12:55.698 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.698+0000 7f5cfd200640 1 -- start start 2024-05-11T12:12:55.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.699+0000 7f5cfd200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cf8073270 0x7f5cf815b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:55.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.699+0000 7f5cfd200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cf8073650 0x7f5cf81f95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:55.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.699+0000 7f5cfd200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cf80ce8f0 0x7f5cf806d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:55.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.699+0000 7f5cfd200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf8070390 con 0x7f5cf8073650 2024-05-11T12:12:55.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.699+0000 7f5cfd200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf815cce0 con 0x7f5cf8073270 2024-05-11T12:12:55.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.699+0000 7f5cfd200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5cf8073b00 con 0x7f5cf80ce8f0 2024-05-11T12:12:55.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.699+0000 7f5cefe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cf8073650 0x7f5cf81f95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:55.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.699+0000 7f5cefe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cf8073650 0x7f5cf81f95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56172/0 (socket says 172.21.15.167:56172) 2024-05-11T12:12:55.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.699+0000 7f5cefe00640 1 -- 172.21.15.167:0/1714338378 learned_addr learned my addr 172.21.15.167:0/1714338378 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:55.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.699+0000 7f5cf5e00640 1 --2- 172.21.15.167:0/1714338378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cf8073270 0x7f5cf815b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:55.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.699+0000 7f5cefe00640 1 -- 172.21.15.167:0/1714338378 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cf80ce8f0 msgr2=0x7f5cf806d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:55.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.699+0000 7f5cefe00640 1 --2- 172.21.15.167:0/1714338378 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cf80ce8f0 0x7f5cf806d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:55.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.699+0000 7f5cefe00640 1 -- 172.21.15.167:0/1714338378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cf8073270 msgr2=0x7f5cf815b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:55.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.699+0000 7f5cefe00640 1 --2- 172.21.15.167:0/1714338378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cf8073270 0x7f5cf815b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:55.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.699+0000 7f5cefe00640 1 -- 172.21.15.167:0/1714338378 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf81fa000 con 0x7f5cf8073650 2024-05-11T12:12:55.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.700+0000 7f5cefe00640 1 --2- 172.21.15.167:0/1714338378 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cf8073650 0x7f5cf81f95d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5ce000c500 tx=0x7f5ce000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:55.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.700+0000 7f5cee000640 1 -- 172.21.15.167:0/1714338378 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0018070 con 0x7f5cf8073650 2024-05-11T12:12:55.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.700+0000 7f5cee000640 1 -- 172.21.15.167:0/1714338378 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ce0007500 con 0x7f5cf8073650 2024-05-11T12:12:55.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.700+0000 7f5cfd200640 1 -- 172.21.15.167:0/1714338378 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cf81fa2e0 con 0x7f5cf8073650 2024-05-11T12:12:55.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.700+0000 7f5cee000640 1 -- 172.21.15.167:0/1714338378 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce00049e0 con 0x7f5cf8073650 2024-05-11T12:12:55.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.700+0000 7f5cfd200640 1 -- 172.21.15.167:0/1714338378 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cf815e670 con 0x7f5cf8073650 2024-05-11T12:12:55.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.701+0000 7f5cfd200640 1 -- 172.21.15.167:0/1714338378 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cf8143110 con 0x7f5cf8073650 2024-05-11T12:12:55.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.702+0000 7f5cee000640 1 -- 172.21.15.167:0/1714338378 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5ce001b070 con 0x7f5cf8073650 2024-05-11T12:12:55.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.703+0000 7f5cee000640 1 --2- 172.21.15.167:0/1714338378 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5cd40762d0 0x7f5cd4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:55.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.704+0000 7f5cee000640 1 -- 172.21.15.167:0/1714338378 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7f5ce0096740 con 0x7f5cf8073650 2024-05-11T12:12:55.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.704+0000 7f5cf5e00640 1 --2- 172.21.15.167:0/1714338378 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5cd40762d0 0x7f5cd4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:55.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.705+0000 7f5cf5e00640 1 --2- 172.21.15.167:0/1714338378 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5cd40762d0 0x7f5cd4078790 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8034000 tx=0x7f5ce802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:55.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.708+0000 7f5cee000640 1 -- 172.21.15.167:0/1714338378 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5cf8143110 con 0x7f5cf8073650 2024-05-11T12:12:55.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.898+0000 7f5cfd200640 1 -- 172.21.15.167:0/1714338378 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5cf8141d00 con 0x7f5cf8073650 2024-05-11T12:12:55.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.899+0000 7f5cee000640 1 -- 172.21.15.167:0/1714338378 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) ==== 93+0+10 (secure 0 0 0) 0x7f5cf8141d00 con 0x7f5cf8073650 2024-05-11T12:12:55.900 INFO:teuthology.orchestra.run.smithi167.stdout:pg_num: 1 2024-05-11T12:12:55.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.902+0000 7f5cfd200640 1 -- 172.21.15.167:0/1714338378 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5cd40762d0 msgr2=0x7f5cd4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:55.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.902+0000 7f5cfd200640 1 --2- 172.21.15.167:0/1714338378 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5cd40762d0 0x7f5cd4078790 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8034000 tx=0x7f5ce802e000 comp rx=0 tx=0).stop 2024-05-11T12:12:55.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.902+0000 7f5cfd200640 1 -- 172.21.15.167:0/1714338378 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cf8073650 msgr2=0x7f5cf81f95d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:55.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.902+0000 7f5cfd200640 1 --2- 172.21.15.167:0/1714338378 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cf8073650 0x7f5cf81f95d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5ce000c500 tx=0x7f5ce000c9c0 comp rx=0 tx=0).stop 2024-05-11T12:12:55.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.902+0000 7f5cfd200640 1 -- 172.21.15.167:0/1714338378 shutdown_connections 2024-05-11T12:12:55.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.902+0000 7f5cfd200640 1 --2- 172.21.15.167:0/1714338378 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cf80ce8f0 0x7f5cf806d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:55.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.902+0000 7f5cfd200640 1 --2- 172.21.15.167:0/1714338378 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cf8073650 0x7f5cf81f95d0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:55.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.902+0000 7f5cfd200640 1 --2- 172.21.15.167:0/1714338378 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5cd40762d0 0x7f5cd4078790 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:55.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.902+0000 7f5cfd200640 1 --2- 172.21.15.167:0/1714338378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cf8073270 0x7f5cf815b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:55.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.902+0000 7f5cfd200640 1 -- 172.21.15.167:0/1714338378 >> 172.21.15.167:0/1714338378 conn(0x7f5cf806c450 msgr2=0x7f5cf80ae8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:55.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.902+0000 7f5cfd200640 1 -- 172.21.15.167:0/1714338378 shutdown_connections 2024-05-11T12:12:55.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:55.902+0000 7f5cfd200640 1 -- 172.21.15.167:0/1714338378 wait complete. 2024-05-11T12:12:55.919 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_reject' 2024-05-11T12:12:56.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.113+0000 7f04e4800640 1 Processor -- start 2024-05-11T12:12:56.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.113+0000 7f04e4800640 1 -- start start 2024-05-11T12:12:56.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.115+0000 7f04e4800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f04dc0b3ea0 0x7f04dc161490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:56.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.115+0000 7f04e4800640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f04dc1619d0 0x7f04dc06d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:56.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.115+0000 7f04e4800640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04dc15df30 con 0x7f04dc157750 2024-05-11T12:12:56.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.115+0000 7f04e4800640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f04dc161000 con 0x7f04dc1619d0 2024-05-11T12:12:56.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.115+0000 7f04e4800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f04dc15e210 con 0x7f04dc0b3ea0 2024-05-11T12:12:56.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.115+0000 7f04e1e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f04dc157750 0x7f04dc15d740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49370/0 (socket says 172.21.15.167:49370) 2024-05-11T12:12:56.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.115+0000 7f04e1e00640 1 -- 172.21.15.167:0/1908194783 learned_addr learned my addr 172.21.15.167:0/1908194783 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:56.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.115+0000 7f04e2800640 1 --2- 172.21.15.167:0/1908194783 >> v2:172.21.15.167:3300/0 conn(0x7f04dc1619d0 0x7f04dc06d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:56.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.115+0000 7f04e1400640 1 --2- 172.21.15.167:0/1908194783 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f04dc0b3ea0 0x7f04dc161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:56.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.115+0000 7f04e0a00640 1 -- 172.21.15.167:0/1908194783 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 725415187 0 0) 0x7f04dc15df30 con 0x7f04dc157750 2024-05-11T12:12:56.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.115+0000 7f04e0a00640 1 -- 172.21.15.167:0/1908194783 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f04c4003520 con 0x7f04dc157750 2024-05-11T12:12:56.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.116+0000 7f04e2800640 1 -- 172.21.15.167:0/1908194783 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f04dc0b3ea0 msgr2=0x7f04dc161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:56.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.116+0000 7f04e2800640 1 --2- 172.21.15.167:0/1908194783 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f04dc0b3ea0 0x7f04dc161490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:56.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.116+0000 7f04e2800640 1 -- 172.21.15.167:0/1908194783 >> v1:172.21.15.167:6789/0 conn(0x7f04dc157750 legacy=0x7f04dc15d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:56.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.116+0000 7f04e2800640 1 -- 172.21.15.167:0/1908194783 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04dc15e4f0 con 0x7f04dc1619d0 2024-05-11T12:12:56.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.117+0000 7f04e2800640 1 --2- 172.21.15.167:0/1908194783 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04dc1619d0 0x7f04dc06d460 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f04d8009d90 tx=0x7f04d802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbefa2ef73a97c4a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:56.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.117+0000 7f04e0a00640 1 -- 172.21.15.167:0/1908194783 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d8007740 con 0x7f04dc1619d0 2024-05-11T12:12:56.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.117+0000 7f04e0a00640 1 -- 172.21.15.167:0/1908194783 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04d80078e0 con 0x7f04dc1619d0 2024-05-11T12:12:56.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.117+0000 7f04e0a00640 1 -- 172.21.15.167:0/1908194783 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d8007cb0 con 0x7f04dc1619d0 2024-05-11T12:12:56.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.117+0000 7f04e4800640 1 -- 172.21.15.167:0/1908194783 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04dc1619d0 msgr2=0x7f04dc06d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:56.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.117+0000 7f04e4800640 1 --2- 172.21.15.167:0/1908194783 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04dc1619d0 0x7f04dc06d460 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f04d8009d90 tx=0x7f04d802a660 comp rx=0 tx=0).stop 2024-05-11T12:12:56.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.117+0000 7f04e4800640 1 -- 172.21.15.167:0/1908194783 shutdown_connections 2024-05-11T12:12:56.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.117+0000 7f04e4800640 1 --2- 172.21.15.167:0/1908194783 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04dc1619d0 0x7f04dc06d460 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:56.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.117+0000 7f04e4800640 1 --2- 172.21.15.167:0/1908194783 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f04dc0b3ea0 0x7f04dc161490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:56.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.118+0000 7f04e4800640 1 -- 172.21.15.167:0/1908194783 >> 172.21.15.167:0/1908194783 conn(0x7f04dc0cd8f0 msgr2=0x7f04dc0cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:56.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.118+0000 7f04e4800640 1 -- 172.21.15.167:0/1908194783 shutdown_connections 2024-05-11T12:12:56.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.118+0000 7f04e4800640 1 -- 172.21.15.167:0/1908194783 wait complete. 2024-05-11T12:12:56.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.119+0000 7f04e4800640 1 Processor -- start 2024-05-11T12:12:56.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.119+0000 7f04e4800640 1 -- start start 2024-05-11T12:12:56.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.120+0000 7f04e4800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f04dc0b3ea0 0x7f04dc1f5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:56.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.120+0000 7f04e4800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04dc157750 0x7f04dc1f2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:56.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.120+0000 7f04e4800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04dc1619d0 0x7f04dc158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:56.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.120+0000 7f04e4800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc06db20 con 0x7f04dc157750 2024-05-11T12:12:56.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.120+0000 7f04e4800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc06d9a0 con 0x7f04dc1619d0 2024-05-11T12:12:56.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.120+0000 7f04e4800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f04dc06dca0 con 0x7f04dc0b3ea0 2024-05-11T12:12:56.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.121+0000 7f04e1400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04dc157750 0x7f04dc1f2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:56.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.121+0000 7f04e1400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04dc157750 0x7f04dc1f2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56204/0 (socket says 172.21.15.167:56204) 2024-05-11T12:12:56.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.121+0000 7f04e1400640 1 -- 172.21.15.167:0/3190122784 learned_addr learned my addr 172.21.15.167:0/3190122784 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:56.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.121+0000 7f04e1400640 1 -- 172.21.15.167:0/3190122784 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f04dc0b3ea0 msgr2=0x7f04dc1f5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:56.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.121+0000 7f04e1400640 1 --2- 172.21.15.167:0/3190122784 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f04dc0b3ea0 0x7f04dc1f5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:56.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.121+0000 7f04e1400640 1 -- 172.21.15.167:0/3190122784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04dc1619d0 msgr2=0x7f04dc158ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:56.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.121+0000 7f04e1400640 1 --2- 172.21.15.167:0/3190122784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04dc1619d0 0x7f04dc158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:56.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.121+0000 7f04e1400640 1 -- 172.21.15.167:0/3190122784 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04dc1f3440 con 0x7f04dc157750 2024-05-11T12:12:56.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.121+0000 7f04e1400640 1 --2- 172.21.15.167:0/3190122784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04dc157750 0x7f04dc1f2f00 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f04cc00d800 tx=0x7f04cc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:56.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.122+0000 7f04d6000640 1 -- 172.21.15.167:0/3190122784 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc00b6c0 con 0x7f04dc157750 2024-05-11T12:12:56.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.122+0000 7f04d6000640 1 -- 172.21.15.167:0/3190122784 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04cc019070 con 0x7f04dc157750 2024-05-11T12:12:56.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.122+0000 7f04d6000640 1 -- 172.21.15.167:0/3190122784 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc00ba80 con 0x7f04dc157750 2024-05-11T12:12:56.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.122+0000 7f04e4800640 1 -- 172.21.15.167:0/3190122784 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04dc159990 con 0x7f04dc157750 2024-05-11T12:12:56.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.122+0000 7f04e4800640 1 -- 172.21.15.167:0/3190122784 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04dc159f50 con 0x7f04dc157750 2024-05-11T12:12:56.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.123+0000 7f04e4800640 1 -- 172.21.15.167:0/3190122784 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04dc069590 con 0x7f04dc157750 2024-05-11T12:12:56.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.124+0000 7f04d6000640 1 -- 172.21.15.167:0/3190122784 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f04cc00bc20 con 0x7f04dc157750 2024-05-11T12:12:56.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.125+0000 7f04d6000640 1 --2- 172.21.15.167:0/3190122784 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f04c00762d0 0x7f04c0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:56.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.125+0000 7f04d6000640 1 -- 172.21.15.167:0/3190122784 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7f04cc097950 con 0x7f04dc157750 2024-05-11T12:12:56.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.125+0000 7f04e1e00640 1 --2- 172.21.15.167:0/3190122784 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f04c00762d0 0x7f04c0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:56.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.126+0000 7f04e1e00640 1 --2- 172.21.15.167:0/3190122784 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f04c00762d0 0x7f04c0078790 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f04d0001220 tx=0x7f04d00078f0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:56.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.129+0000 7f04d6000640 1 -- 172.21.15.167:0/3190122784 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f04dc1f3440 con 0x7f04dc157750 2024-05-11T12:12:56.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:56.412+0000 7f04e4800640 1 -- 172.21.15.167:0/3190122784 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_reject"]} v 0) -- 0x7f04dc069e10 con 0x7f04dc157750 2024-05-11T12:12:57.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.027+0000 7f04d6000640 1 -- 172.21.15.167:0/3190122784 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_reject"]}]=0 v141) ==== 145+0+0 (secure 0 0 0) 0x7f04cc061410 con 0x7f04dc157750 2024-05-11T12:12:57.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.031+0000 7f04e4800640 1 -- 172.21.15.167:0/3190122784 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f04c00762d0 msgr2=0x7f04c0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:57.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.031+0000 7f04e4800640 1 --2- 172.21.15.167:0/3190122784 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f04c00762d0 0x7f04c0078790 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f04d0001220 tx=0x7f04d00078f0 comp rx=0 tx=0).stop 2024-05-11T12:12:57.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.031+0000 7f04e4800640 1 -- 172.21.15.167:0/3190122784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04dc157750 msgr2=0x7f04dc1f2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:57.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.031+0000 7f04e4800640 1 --2- 172.21.15.167:0/3190122784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04dc157750 0x7f04dc1f2f00 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f04cc00d800 tx=0x7f04cc00dcc0 comp rx=0 tx=0).stop 2024-05-11T12:12:57.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.031+0000 7f04e4800640 1 -- 172.21.15.167:0/3190122784 shutdown_connections 2024-05-11T12:12:57.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.031+0000 7f04e4800640 1 --2- 172.21.15.167:0/3190122784 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f04c00762d0 0x7f04c0078790 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:57.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.031+0000 7f04e4800640 1 --2- 172.21.15.167:0/3190122784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04dc1619d0 0x7f04dc158ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:57.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.031+0000 7f04e4800640 1 --2- 172.21.15.167:0/3190122784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04dc157750 0x7f04dc1f2f00 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:57.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.031+0000 7f04e4800640 1 --2- 172.21.15.167:0/3190122784 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f04dc0b3ea0 0x7f04dc1f5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:57.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.031+0000 7f04e4800640 1 -- 172.21.15.167:0/3190122784 >> 172.21.15.167:0/3190122784 conn(0x7f04dc0cd8f0 msgr2=0x7f04dc0cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:57.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.031+0000 7f04e4800640 1 -- 172.21.15.167:0/3190122784 shutdown_connections 2024-05-11T12:12:57.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.032+0000 7f04e4800640 1 -- 172.21.15.167:0/3190122784 wait complete. 2024-05-11T12:12:57.053 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-11T12:12:57.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.255+0000 7fe83ba00640 1 Processor -- start 2024-05-11T12:12:57.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.255+0000 7fe83ba00640 1 -- start start 2024-05-11T12:12:57.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.257+0000 7fe83ba00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe83415ac80 0x7fe834074260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:57.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.257+0000 7fe83ba00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe8340b3d50 0x7fe834160c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:57.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.257+0000 7fe83ba00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8340b65e0 con 0x7fe8341611b0 2024-05-11T12:12:57.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.257+0000 7fe83ba00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fe83406e3c0 con 0x7fe83415ac80 2024-05-11T12:12:57.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.257+0000 7fe83ba00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fe834161600 con 0x7fe8340b3d50 2024-05-11T12:12:57.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.257+0000 7fe839000640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe83415ac80 0x7fe834074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:57.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.257+0000 7fe839a00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fe8341611b0 0x7fe83415d680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49372/0 (socket says 172.21.15.167:49372) 2024-05-11T12:12:57.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.257+0000 7fe839a00640 1 -- 172.21.15.167:0/2266650674 learned_addr learned my addr 172.21.15.167:0/2266650674 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:57.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.258+0000 7fe839000640 1 -- 172.21.15.167:0/2266650674 >> v2:172.21.15.204:3300/0 conn(0x7fe8340b3d50 msgr2=0x7fe834160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:57.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.258+0000 7fe82be00640 1 --2- 172.21.15.167:0/2266650674 >> v2:172.21.15.204:3300/0 conn(0x7fe8340b3d50 0x7fe834160c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:57.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.258+0000 7fe833e00640 1 -- 172.21.15.167:0/2266650674 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3142314741 0 0) 0x7fe8340b65e0 con 0x7fe8341611b0 2024-05-11T12:12:57.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.258+0000 7fe839000640 1 --2- 172.21.15.167:0/2266650674 >> v2:172.21.15.204:3300/0 conn(0x7fe8340b3d50 0x7fe834160c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:57.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.258+0000 7fe839000640 1 -- 172.21.15.167:0/2266650674 >> v1:172.21.15.167:6789/0 conn(0x7fe8341611b0 legacy=0x7fe83415d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:57.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.258+0000 7fe839000640 1 -- 172.21.15.167:0/2266650674 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8340b68c0 con 0x7fe83415ac80 2024-05-11T12:12:57.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.258+0000 7fe82be00640 1 --2- 172.21.15.167:0/2266650674 >> v2:172.21.15.204:3300/0 conn(0x7fe8340b3d50 0x7fe834160c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:57.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.259+0000 7fe839000640 1 --2- 172.21.15.167:0/2266650674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe83415ac80 0x7fe834074260 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe824009d90 tx=0x7fe82402a6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=776774d5f4b30a95 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:57.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.259+0000 7fe833e00640 1 -- 172.21.15.167:0/2266650674 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe824007740 con 0x7fe83415ac80 2024-05-11T12:12:57.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.259+0000 7fe833e00640 1 -- 172.21.15.167:0/2266650674 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8240078e0 con 0x7fe83415ac80 2024-05-11T12:12:57.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.259+0000 7fe833e00640 1 -- 172.21.15.167:0/2266650674 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe824007cb0 con 0x7fe83415ac80 2024-05-11T12:12:57.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.259+0000 7fe83ba00640 1 -- 172.21.15.167:0/2266650674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe83415ac80 msgr2=0x7fe834074260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:57.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.259+0000 7fe83ba00640 1 --2- 172.21.15.167:0/2266650674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe83415ac80 0x7fe834074260 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe824009d90 tx=0x7fe82402a6b0 comp rx=0 tx=0).stop 2024-05-11T12:12:57.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.260+0000 7fe83ba00640 1 -- 172.21.15.167:0/2266650674 shutdown_connections 2024-05-11T12:12:57.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.260+0000 7fe83ba00640 1 --2- 172.21.15.167:0/2266650674 >> v2:172.21.15.204:3300/0 conn(0x7fe8340b3d50 0x7fe834160c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:57.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.260+0000 7fe83ba00640 1 --2- 172.21.15.167:0/2266650674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe83415ac80 0x7fe834074260 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:57.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.260+0000 7fe83ba00640 1 -- 172.21.15.167:0/2266650674 >> 172.21.15.167:0/2266650674 conn(0x7fe83406c450 msgr2=0x7fe83406c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:57.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.260+0000 7fe83ba00640 1 -- 172.21.15.167:0/2266650674 shutdown_connections 2024-05-11T12:12:57.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.260+0000 7fe83ba00640 1 -- 172.21.15.167:0/2266650674 wait complete. 2024-05-11T12:12:57.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.261+0000 7fe83ba00640 1 Processor -- start 2024-05-11T12:12:57.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.262+0000 7fe83ba00640 1 -- start start 2024-05-11T12:12:57.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.262+0000 7fe83ba00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8340b3d50 0x7fe83413c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:57.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.262+0000 7fe83ba00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe83415ac80 0x7fe83406f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:57.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.262+0000 7fe83ba00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8341611b0 0x7fe834159110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:57.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.262+0000 7fe83ba00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe8340b41e0 con 0x7fe83415ac80 2024-05-11T12:12:57.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.262+0000 7fe83ba00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe83415b060 con 0x7fe8341611b0 2024-05-11T12:12:57.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.262+0000 7fe83ba00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe8340b44f0 con 0x7fe8340b3d50 2024-05-11T12:12:57.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.263+0000 7fe82be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe83415ac80 0x7fe83406f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:57.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.263+0000 7fe82be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe83415ac80 0x7fe83406f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56212/0 (socket says 172.21.15.167:56212) 2024-05-11T12:12:57.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.263+0000 7fe82be00640 1 -- 172.21.15.167:0/2690549441 learned_addr learned my addr 172.21.15.167:0/2690549441 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:57.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.263+0000 7fe839000640 1 --2- 172.21.15.167:0/2690549441 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8340b3d50 0x7fe83413c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:57.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.263+0000 7fe82be00640 1 -- 172.21.15.167:0/2690549441 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8340b3d50 msgr2=0x7fe83413c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:57.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.263+0000 7fe82be00640 1 --2- 172.21.15.167:0/2690549441 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8340b3d50 0x7fe83413c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:57.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.263+0000 7fe839a00640 1 --2- 172.21.15.167:0/2690549441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8341611b0 0x7fe834159110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:57.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.263+0000 7fe82be00640 1 -- 172.21.15.167:0/2690549441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8341611b0 msgr2=0x7fe834159110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:57.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.263+0000 7fe82be00640 1 --2- 172.21.15.167:0/2690549441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8341611b0 0x7fe834159110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:57.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.263+0000 7fe82be00640 1 -- 172.21.15.167:0/2690549441 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe83406fbf0 con 0x7fe83415ac80 2024-05-11T12:12:57.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.264+0000 7fe82be00640 1 --2- 172.21.15.167:0/2690549441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe83415ac80 0x7fe83406f6b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe81c00c7f0 tx=0x7fe81c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:57.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.264+0000 7fe831600640 1 -- 172.21.15.167:0/2690549441 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81c015070 con 0x7fe83415ac80 2024-05-11T12:12:57.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.264+0000 7fe831600640 1 -- 172.21.15.167:0/2690549441 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe81c007a30 con 0x7fe83415ac80 2024-05-11T12:12:57.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.264+0000 7fe83ba00640 1 -- 172.21.15.167:0/2690549441 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe83413c8f0 con 0x7fe83415ac80 2024-05-11T12:12:57.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.264+0000 7fe831600640 1 -- 172.21.15.167:0/2690549441 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81c007e00 con 0x7fe83415ac80 2024-05-11T12:12:57.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.264+0000 7fe83ba00640 1 -- 172.21.15.167:0/2690549441 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe83413ceb0 con 0x7fe83415ac80 2024-05-11T12:12:57.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.265+0000 7fe83ba00640 1 -- 172.21.15.167:0/2690549441 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8340b47f0 con 0x7fe83415ac80 2024-05-11T12:12:57.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.266+0000 7fe831600640 1 -- 172.21.15.167:0/2690549441 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe81c018070 con 0x7fe83415ac80 2024-05-11T12:12:57.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.268+0000 7fe831600640 1 --2- 172.21.15.167:0/2690549441 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe810076300 0x7fe8100787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:57.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.268+0000 7fe831600640 1 -- 172.21.15.167:0/2690549441 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7fe81c097510 con 0x7fe83415ac80 2024-05-11T12:12:57.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.268+0000 7fe839000640 1 --2- 172.21.15.167:0/2690549441 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe810076300 0x7fe8100787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:57.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.269+0000 7fe839000640 1 --2- 172.21.15.167:0/2690549441 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe810076300 0x7fe8100787c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe824034000 tx=0x7fe82402f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:57.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.272+0000 7fe831600640 1 -- 172.21.15.167:0/2690549441 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe8340b47f0 con 0x7fe83415ac80 2024-05-11T12:12:57.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.476+0000 7fe83ba00640 1 -- 172.21.15.167:0/2690549441 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fe834069520 con 0x7fe83415ac80 2024-05-11T12:12:57.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.484+0000 7fe831600640 1 -- 172.21.15.167:0/2690549441 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7fe81c060eb0 con 0x7fe83415ac80 2024-05-11T12:12:57.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.486+0000 7fe83ba00640 1 -- 172.21.15.167:0/2690549441 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe810076300 msgr2=0x7fe8100787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:57.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.486+0000 7fe83ba00640 1 --2- 172.21.15.167:0/2690549441 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe810076300 0x7fe8100787c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe824034000 tx=0x7fe82402f000 comp rx=0 tx=0).stop 2024-05-11T12:12:57.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.486+0000 7fe83ba00640 1 -- 172.21.15.167:0/2690549441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe83415ac80 msgr2=0x7fe83406f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:57.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.486+0000 7fe83ba00640 1 --2- 172.21.15.167:0/2690549441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe83415ac80 0x7fe83406f6b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe81c00c7f0 tx=0x7fe81c00cbf0 comp rx=0 tx=0).stop 2024-05-11T12:12:57.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.487+0000 7fe83ba00640 1 -- 172.21.15.167:0/2690549441 shutdown_connections 2024-05-11T12:12:57.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.487+0000 7fe83ba00640 1 --2- 172.21.15.167:0/2690549441 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe810076300 0x7fe8100787c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:57.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.487+0000 7fe83ba00640 1 --2- 172.21.15.167:0/2690549441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8341611b0 0x7fe834159110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:57.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.487+0000 7fe83ba00640 1 --2- 172.21.15.167:0/2690549441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe83415ac80 0x7fe83406f6b0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:57.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.487+0000 7fe83ba00640 1 --2- 172.21.15.167:0/2690549441 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8340b3d50 0x7fe83413c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:57.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.487+0000 7fe83ba00640 1 -- 172.21.15.167:0/2690549441 >> 172.21.15.167:0/2690549441 conn(0x7fe83406c450 msgr2=0x7fe83415fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:57.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.487+0000 7fe83ba00640 1 -- 172.21.15.167:0/2690549441 shutdown_connections 2024-05-11T12:12:57.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:57.487+0000 7fe83ba00640 1 -- 172.21.15.167:0/2690549441 wait complete. 2024-05-11T12:12:57.505 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-11T12:12:57.505 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-11T12:12:57.596 DEBUG:teuthology.orchestra.run.smithi204:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-11T12:12:57.705 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-11T12:12:57.706 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:12:57.706 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:12:57.760 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-11T12:12:57.760 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:12:57.760 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:12:57.817 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052713,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-11T12:12:57.817 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:12:57.817 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:12:57.876 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-11T12:12:57.876 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:12:57.876 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:12:57.930 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052713,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-11T12:12:57.930 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-11T12:12:57.931 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:12:58.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.142+0000 7fa89f200640 1 Processor -- start 2024-05-11T12:12:58.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.142+0000 7fa89f200640 1 -- start start 2024-05-11T12:12:58.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.144+0000 7fa89f200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa8a006c660 0x7fa8a00d2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:58.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.144+0000 7fa89f200640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8a0156500 con 0x7fa8a0156120 2024-05-11T12:12:58.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.144+0000 7fa89f200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8a006caf0 con 0x7fa8a01632e0 2024-05-11T12:12:58.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.144+0000 7fa89f200640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fa8a006d700 con 0x7fa8a006c660 2024-05-11T12:12:58.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.144+0000 7fa89d400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa8a006c660 0x7fa8a00d2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:58.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.144+0000 7fa89de00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fa8a0156120 0x7fa8a0162bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49382/0 (socket says 172.21.15.167:49382) 2024-05-11T12:12:58.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.145+0000 7fa89de00640 1 -- 172.21.15.167:0/4051815422 learned_addr learned my addr 172.21.15.167:0/4051815422 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:58.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.145+0000 7fa89d400640 1 -- 172.21.15.167:0/4051815422 >> v1:172.21.15.204:6789/0 conn(0x7fa8a01632e0 legacy=0x7fa8a00d30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:58.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.145+0000 7fa89d400640 1 -- 172.21.15.167:0/4051815422 >> v1:172.21.15.167:6789/0 conn(0x7fa8a0156120 legacy=0x7fa8a0162bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:58.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.145+0000 7fa89ca00640 1 -- 172.21.15.167:0/4051815422 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4184610180 0 0) 0x7fa8a0156500 con 0x7fa8a0156120 2024-05-11T12:12:58.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.145+0000 7fa89d400640 1 -- 172.21.15.167:0/4051815422 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8a006cdd0 con 0x7fa8a006c660 2024-05-11T12:12:58.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.146+0000 7fa89d400640 1 --2- 172.21.15.167:0/4051815422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8a006c660 0x7fa8a00d2b70 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa8880097a0 tx=0x7fa88802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37fc98777109f5c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:58.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.146+0000 7fa89ca00640 1 -- 172.21.15.167:0/4051815422 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa888037070 con 0x7fa8a006c660 2024-05-11T12:12:58.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.147+0000 7fa89ca00640 1 -- 172.21.15.167:0/4051815422 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa888032440 con 0x7fa8a006c660 2024-05-11T12:12:58.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.147+0000 7fa89ca00640 1 -- 172.21.15.167:0/4051815422 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa888032810 con 0x7fa8a006c660 2024-05-11T12:12:58.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.147+0000 7fa89f200640 1 -- 172.21.15.167:0/4051815422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8a006c660 msgr2=0x7fa8a00d2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:58.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.147+0000 7fa89f200640 1 --2- 172.21.15.167:0/4051815422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8a006c660 0x7fa8a00d2b70 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa8880097a0 tx=0x7fa88802a180 comp rx=0 tx=0).stop 2024-05-11T12:12:58.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.147+0000 7fa89f200640 1 -- 172.21.15.167:0/4051815422 shutdown_connections 2024-05-11T12:12:58.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.147+0000 7fa89f200640 1 --2- 172.21.15.167:0/4051815422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8a006c660 0x7fa8a00d2b70 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:58.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.147+0000 7fa89f200640 1 -- 172.21.15.167:0/4051815422 >> 172.21.15.167:0/4051815422 conn(0x7fa8a00b6af0 msgr2=0x7fa8a00b6f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:58.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.147+0000 7fa89f200640 1 -- 172.21.15.167:0/4051815422 shutdown_connections 2024-05-11T12:12:58.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.147+0000 7fa89f200640 1 -- 172.21.15.167:0/4051815422 wait complete. 2024-05-11T12:12:58.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.149+0000 7fa89f200640 1 Processor -- start 2024-05-11T12:12:58.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.149+0000 7fa89f200640 1 -- start start 2024-05-11T12:12:58.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.150+0000 7fa89f200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8a0156120 0x7fa8a00cb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:58.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.150+0000 7fa89f200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8a01632e0 0x7fa8a0154a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:58.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.150+0000 7fa89f200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8a01552a0 0x7fa8a015b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:58.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.150+0000 7fa89f200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a006db90 con 0x7fa8a0156120 2024-05-11T12:12:58.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.150+0000 7fa89f200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a006d390 con 0x7fa8a01552a0 2024-05-11T12:12:58.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.150+0000 7fa89f200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa8a006dd10 con 0x7fa8a01632e0 2024-05-11T12:12:58.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.150+0000 7fa89de00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8a0156120 0x7fa8a00cb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:58.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.151+0000 7fa89de00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8a0156120 0x7fa8a00cb870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56236/0 (socket says 172.21.15.167:56236) 2024-05-11T12:12:58.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.151+0000 7fa89de00640 1 -- 172.21.15.167:0/2321504322 learned_addr learned my addr 172.21.15.167:0/2321504322 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:58.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.151+0000 7fa89de00640 1 -- 172.21.15.167:0/2321504322 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8a01632e0 msgr2=0x7fa8a0154a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:58.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.151+0000 7fa89de00640 1 --2- 172.21.15.167:0/2321504322 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8a01632e0 0x7fa8a0154a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:58.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.151+0000 7fa89de00640 1 -- 172.21.15.167:0/2321504322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8a01552a0 msgr2=0x7fa8a015b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:58.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.151+0000 7fa89de00640 1 --2- 172.21.15.167:0/2321504322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8a01552a0 0x7fa8a015b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:58.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.151+0000 7fa89de00640 1 -- 172.21.15.167:0/2321504322 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8a0154fc0 con 0x7fa8a0156120 2024-05-11T12:12:58.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.151+0000 7fa89de00640 1 --2- 172.21.15.167:0/2321504322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8a0156120 0x7fa8a00cb870 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa89000c4d0 tx=0x7fa89000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:58.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.151+0000 7fa896000640 1 -- 172.21.15.167:0/2321504322 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa890017070 con 0x7fa8a0156120 2024-05-11T12:12:58.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.151+0000 7fa896000640 1 -- 172.21.15.167:0/2321504322 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa890005d00 con 0x7fa8a0156120 2024-05-11T12:12:58.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.151+0000 7fa896000640 1 -- 172.21.15.167:0/2321504322 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8900072b0 con 0x7fa8a0156120 2024-05-11T12:12:58.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.152+0000 7fa89f200640 1 -- 172.21.15.167:0/2321504322 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8a01568a0 con 0x7fa8a0156120 2024-05-11T12:12:58.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.152+0000 7fa89f200640 1 -- 172.21.15.167:0/2321504322 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8a015aa90 con 0x7fa8a0156120 2024-05-11T12:12:58.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.153+0000 7fa89f200640 1 -- 172.21.15.167:0/2321504322 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8a0069bc0 con 0x7fa8a0156120 2024-05-11T12:12:58.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.154+0000 7fa896000640 1 -- 172.21.15.167:0/2321504322 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa89001a070 con 0x7fa8a0156120 2024-05-11T12:12:58.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.155+0000 7fa896000640 1 --2- 172.21.15.167:0/2321504322 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa87c076300 0x7fa87c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:58.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.155+0000 7fa896000640 1 -- 172.21.15.167:0/2321504322 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7fa890096850 con 0x7fa8a0156120 2024-05-11T12:12:58.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.155+0000 7fa89d400640 1 --2- 172.21.15.167:0/2321504322 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa87c076300 0x7fa87c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:58.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.156+0000 7fa89d400640 1 --2- 172.21.15.167:0/2321504322 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa87c076300 0x7fa87c0787c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa888034000 tx=0x7fa88802c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:58.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.160+0000 7fa896000640 1 -- 172.21.15.167:0/2321504322 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa8a0154fc0 con 0x7fa8a0156120 2024-05-11T12:12:58.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.412+0000 7fa89f200640 1 -- 172.21.15.167:0/2321504322 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa8a00d40a0 con 0x7fa8a0156120 2024-05-11T12:12:58.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.413+0000 7fa896000640 1 -- 172.21.15.167:0/2321504322 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+3881 (secure 0 0 0) 0x7fa890060280 con 0x7fa8a0156120 2024-05-11T12:12:58.413 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:12:58.413 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":28,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4634,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2640837270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":2640837270},{"type":"v1","addr":"172.21.15.204:6835","nonce":2640837270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4807,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3516271493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3516271493},{"type":"v1","addr":"172.21.15.204:6834","nonce":3516271493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4816,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2070289869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2070289869},{"type":"v1","addr":"172.21.15.167:6837","nonce":2070289869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5520,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2173441633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2173441633},{"type":"v1","addr":"172.21.15.167:6835","nonce":2173441633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[]} 2024-05-11T12:12:58.414 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 28 2024-05-11T12:12:58.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.416+0000 7fa89f200640 1 -- 172.21.15.167:0/2321504322 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa87c076300 msgr2=0x7fa87c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:58.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.416+0000 7fa89f200640 1 --2- 172.21.15.167:0/2321504322 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa87c076300 0x7fa87c0787c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa888034000 tx=0x7fa88802c000 comp rx=0 tx=0).stop 2024-05-11T12:12:58.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.416+0000 7fa89f200640 1 -- 172.21.15.167:0/2321504322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8a0156120 msgr2=0x7fa8a00cb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:58.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.416+0000 7fa89f200640 1 --2- 172.21.15.167:0/2321504322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8a0156120 0x7fa8a00cb870 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa89000c4d0 tx=0x7fa89000c990 comp rx=0 tx=0).stop 2024-05-11T12:12:58.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.416+0000 7fa89f200640 1 -- 172.21.15.167:0/2321504322 shutdown_connections 2024-05-11T12:12:58.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.416+0000 7fa89f200640 1 --2- 172.21.15.167:0/2321504322 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa87c076300 0x7fa87c0787c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:58.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.416+0000 7fa89f200640 1 --2- 172.21.15.167:0/2321504322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8a01552a0 0x7fa8a015b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:58.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.416+0000 7fa89f200640 1 --2- 172.21.15.167:0/2321504322 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8a01632e0 0x7fa8a0154a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:58.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.416+0000 7fa89f200640 1 --2- 172.21.15.167:0/2321504322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8a0156120 0x7fa8a00cb870 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:58.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.416+0000 7fa89f200640 1 -- 172.21.15.167:0/2321504322 >> 172.21.15.167:0/2321504322 conn(0x7fa8a00b6af0 msgr2=0x7fa8a015df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:58.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.416+0000 7fa89f200640 1 -- 172.21.15.167:0/2321504322 shutdown_connections 2024-05-11T12:12:58.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.416+0000 7fa89f200640 1 -- 172.21.15.167:0/2321504322 wait complete. 2024-05-11T12:12:58.437 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-11T12:12:58.437 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-11T12:12:58.437 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-11T12:12:58.437 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T12:12:58.437 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-11T12:12:58.438 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-11T12:12:58.438 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-11T12:12:58.438 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T12:12:58.438 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-11T12:12:58.438 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-11T12:12:58.438 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-11T12:12:58.438 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T12:12:58.439 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-11T12:12:58.439 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-11T12:12:58.439 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-11T12:12:58.439 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T12:12:58.454 INFO:tasks.ceph.mds.c:Stopped 2024-05-11T12:12:58.455 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-11T12:12:58.455 INFO:tasks.ceph.mds.a:Stopped 2024-05-11T12:12:58.456 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-11T12:12:58.460 INFO:tasks.ceph.mds.b:Stopped 2024-05-11T12:12:58.461 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-11T12:12:58.467 INFO:tasks.ceph.mds.d:Stopped 2024-05-11T12:12:58.467 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-11T12:12:58.502 INFO:tasks.ceph.mds.c:Started 2024-05-11T12:12:58.503 DEBUG:teuthology.parallel:result is None 2024-05-11T12:12:58.508 INFO:tasks.ceph.mds.a:Started 2024-05-11T12:12:58.508 DEBUG:teuthology.parallel:result is None 2024-05-11T12:12:58.514 INFO:tasks.ceph.mds.b:Started 2024-05-11T12:12:58.514 DEBUG:teuthology.parallel:result is None 2024-05-11T12:12:58.520 INFO:tasks.ceph.mds.d:Started 2024-05-11T12:12:58.521 DEBUG:teuthology.parallel:result is None 2024-05-11T12:12:58.521 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-11T12:12:58.655 INFO:tasks.ceph.mds.a.smithi167.stdout:starting mds.a at 2024-05-11T12:12:58.655 INFO:tasks.ceph.mds.c.smithi167.stdout:starting mds.c at 2024-05-11T12:12:58.670 INFO:tasks.ceph.mds.b.smithi204.stdout:starting mds.b at 2024-05-11T12:12:58.670 INFO:tasks.ceph.mds.d.smithi204.stdout:starting mds.d at 2024-05-11T12:12:58.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.702+0000 7fe65c000640 1 Processor -- start 2024-05-11T12:12:58.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.702+0000 7fe65c000640 1 -- start start 2024-05-11T12:12:58.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.704+0000 7fe65c000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe654145a00 0x7fe6540afe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:58.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.704+0000 7fe65c000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe6540ceb00 0x7fe6540cef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:58.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.704+0000 7fe65c000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6540aed90 con 0x7fe6540ae820 2024-05-11T12:12:58.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.704+0000 7fe65c000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fe6540af9a0 con 0x7fe6540ceb00 2024-05-11T12:12:58.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.704+0000 7fe65c000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe6540af070 con 0x7fe654145a00 2024-05-11T12:12:58.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.704+0000 7fe659600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe654145a00 0x7fe6540afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:58.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.704+0000 7fe65a000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe6540ceb00 0x7fe6540cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:58.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.704+0000 7fe65a000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe6540ceb00 0x7fe6540cef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.167:48342/0 (socket says 172.21.15.167:48342) 2024-05-11T12:12:58.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.704+0000 7fe65a000640 1 -- 172.21.15.167:0/684657260 learned_addr learned my addr 172.21.15.167:0/684657260 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:58.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.704+0000 7fe659600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe654145a00 0x7fe6540afe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:38806/0 (socket says 172.21.15.167:38806) 2024-05-11T12:12:58.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.705+0000 7fe65a000640 1 -- 172.21.15.167:0/684657260 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe654145a00 msgr2=0x7fe6540afe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:58.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.705+0000 7fe65a000640 1 --2- 172.21.15.167:0/684657260 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe654145a00 0x7fe6540afe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:58.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.705+0000 7fe65a000640 1 -- 172.21.15.167:0/684657260 >> v1:172.21.15.204:6789/0 conn(0x7fe6540ae820 legacy=0x7fe65406f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:58.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.705+0000 7fe65a000640 1 -- 172.21.15.167:0/684657260 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6540af350 con 0x7fe6540ceb00 2024-05-11T12:12:58.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.706+0000 7fe65a000640 1 --2- 172.21.15.167:0/684657260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6540ceb00 0x7fe6540cef90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe64c009e50 tx=0x7fe64c02a6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40b1994d94ca41dc server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:58.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.706+0000 7fe653e00640 1 -- 172.21.15.167:0/684657260 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64c036070 con 0x7fe6540ceb00 2024-05-11T12:12:58.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.706+0000 7fe653e00640 1 -- 172.21.15.167:0/684657260 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe64c031600 con 0x7fe6540ceb00 2024-05-11T12:12:58.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.706+0000 7fe653e00640 1 -- 172.21.15.167:0/684657260 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64c0319d0 con 0x7fe6540ceb00 2024-05-11T12:12:58.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.707+0000 7fe65c000640 1 -- 172.21.15.167:0/684657260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6540ceb00 msgr2=0x7fe6540cef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:58.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.707+0000 7fe65c000640 1 --2- 172.21.15.167:0/684657260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6540ceb00 0x7fe6540cef90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe64c009e50 tx=0x7fe64c02a6a0 comp rx=0 tx=0).stop 2024-05-11T12:12:58.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.707+0000 7fe65c000640 1 -- 172.21.15.167:0/684657260 shutdown_connections 2024-05-11T12:12:58.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.707+0000 7fe65c000640 1 --2- 172.21.15.167:0/684657260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6540ceb00 0x7fe6540cef90 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:58.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.707+0000 7fe65c000640 1 --2- 172.21.15.167:0/684657260 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe654145a00 0x7fe6540afe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:58.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.707+0000 7fe65c000640 1 -- 172.21.15.167:0/684657260 >> 172.21.15.167:0/684657260 conn(0x7fe65406c450 msgr2=0x7fe65406c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:58.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.707+0000 7fe65c000640 1 -- 172.21.15.167:0/684657260 shutdown_connections 2024-05-11T12:12:58.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.707+0000 7fe65c000640 1 -- 172.21.15.167:0/684657260 wait complete. 2024-05-11T12:12:58.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.709+0000 7fe65c000640 1 Processor -- start 2024-05-11T12:12:58.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.709+0000 7fe65c000640 1 -- start start 2024-05-11T12:12:58.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.710+0000 7fe65c000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6540ae820 0x7fe654143270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:58.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.710+0000 7fe65c000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6540ceb00 0x7fe654144bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:58.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.710+0000 7fe65c000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe654145a00 0x7fe6541407b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:58.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.710+0000 7fe65c000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe6540723b0 con 0x7fe6540ceb00 2024-05-11T12:12:58.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.710+0000 7fe65c000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe654072230 con 0x7fe654145a00 2024-05-11T12:12:58.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.710+0000 7fe65c000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe654072530 con 0x7fe6540ae820 2024-05-11T12:12:58.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.710+0000 7fe658c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6540ceb00 0x7fe654144bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:58.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.710+0000 7fe658c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6540ceb00 0x7fe654144bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56286/0 (socket says 172.21.15.167:56286) 2024-05-11T12:12:58.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.710+0000 7fe658c00640 1 -- 172.21.15.167:0/1530295665 learned_addr learned my addr 172.21.15.167:0/1530295665 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:58.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.710+0000 7fe659600640 1 --2- 172.21.15.167:0/1530295665 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6540ae820 0x7fe654143270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:58.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.710+0000 7fe65a000640 1 --2- 172.21.15.167:0/1530295665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe654145a00 0x7fe6541407b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:58.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.711+0000 7fe658c00640 1 -- 172.21.15.167:0/1530295665 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6540ae820 msgr2=0x7fe654143270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:58.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.711+0000 7fe658c00640 1 --2- 172.21.15.167:0/1530295665 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6540ae820 0x7fe654143270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:58.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.711+0000 7fe658c00640 1 -- 172.21.15.167:0/1530295665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe654145a00 msgr2=0x7fe6541407b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:58.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.711+0000 7fe658c00640 1 --2- 172.21.15.167:0/1530295665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe654145a00 0x7fe6541407b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:58.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.711+0000 7fe658c00640 1 -- 172.21.15.167:0/1530295665 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe654143b50 con 0x7fe6540ceb00 2024-05-11T12:12:58.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.711+0000 7fe658c00640 1 --2- 172.21.15.167:0/1530295665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6540ceb00 0x7fe654144bf0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe64000c450 tx=0x7fe64000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:58.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.711+0000 7fe651600640 1 -- 172.21.15.167:0/1530295665 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6400052d0 con 0x7fe6540ceb00 2024-05-11T12:12:58.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.712+0000 7fe651600640 1 -- 172.21.15.167:0/1530295665 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe640005470 con 0x7fe6540ceb00 2024-05-11T12:12:58.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.712+0000 7fe65c000640 1 -- 172.21.15.167:0/1530295665 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe654143e30 con 0x7fe6540ceb00 2024-05-11T12:12:58.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.712+0000 7fe651600640 1 -- 172.21.15.167:0/1530295665 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe640012870 con 0x7fe6540ceb00 2024-05-11T12:12:58.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.712+0000 7fe65c000640 1 -- 172.21.15.167:0/1530295665 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6541443f0 con 0x7fe6540ceb00 2024-05-11T12:12:58.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.713+0000 7fe65c000640 1 -- 172.21.15.167:0/1530295665 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6540694b0 con 0x7fe6540ceb00 2024-05-11T12:12:58.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.714+0000 7fe651600640 1 -- 172.21.15.167:0/1530295665 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe640012a10 con 0x7fe6540ceb00 2024-05-11T12:12:58.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.715+0000 7fe651600640 1 --2- 172.21.15.167:0/1530295665 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe6340762d0 0x7fe634078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:58.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.715+0000 7fe651600640 1 -- 172.21.15.167:0/1530295665 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7fe640096f10 con 0x7fe6540ceb00 2024-05-11T12:12:58.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.715+0000 7fe659600640 1 --2- 172.21.15.167:0/1530295665 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe6340762d0 0x7fe634078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:58.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.716+0000 7fe659600640 1 --2- 172.21.15.167:0/1530295665 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe6340762d0 0x7fe634078790 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe648009aa0 tx=0x7fe648009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:58.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.719+0000 7fe651600640 1 -- 172.21.15.167:0/1530295665 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe640060380 con 0x7fe6540ceb00 2024-05-11T12:12:58.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:58.941+0000 7fe65c000640 1 -- 172.21.15.167:0/1530295665 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fe654075bb0 con 0x7fe6540ceb00 2024-05-11T12:12:59.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.061+0000 7fe651600640 1 -- 172.21.15.167:0/1530295665 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v58) ==== 97+0+0 (secure 0 0 0) 0x7fe640064680 con 0x7fe6540ceb00 2024-05-11T12:12:59.061 INFO:teuthology.orchestra.run.smithi167.stderr: removed all blocklist entries 2024-05-11T12:12:59.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.065+0000 7fe65c000640 1 -- 172.21.15.167:0/1530295665 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe6340762d0 msgr2=0x7fe634078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:59.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.065+0000 7fe65c000640 1 --2- 172.21.15.167:0/1530295665 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe6340762d0 0x7fe634078790 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe648009aa0 tx=0x7fe648009190 comp rx=0 tx=0).stop 2024-05-11T12:12:59.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.065+0000 7fe65c000640 1 -- 172.21.15.167:0/1530295665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6540ceb00 msgr2=0x7fe654144bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:59.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.065+0000 7fe65c000640 1 --2- 172.21.15.167:0/1530295665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6540ceb00 0x7fe654144bf0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe64000c450 tx=0x7fe64000c910 comp rx=0 tx=0).stop 2024-05-11T12:12:59.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.065+0000 7fe65c000640 1 -- 172.21.15.167:0/1530295665 shutdown_connections 2024-05-11T12:12:59.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.065+0000 7fe65c000640 1 --2- 172.21.15.167:0/1530295665 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe6340762d0 0x7fe634078790 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:59.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.065+0000 7fe65c000640 1 --2- 172.21.15.167:0/1530295665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe654145a00 0x7fe6541407b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:59.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.065+0000 7fe65c000640 1 --2- 172.21.15.167:0/1530295665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6540ceb00 0x7fe654144bf0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:59.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.065+0000 7fe65c000640 1 --2- 172.21.15.167:0/1530295665 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6540ae820 0x7fe654143270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:59.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.065+0000 7fe65c000640 1 -- 172.21.15.167:0/1530295665 >> 172.21.15.167:0/1530295665 conn(0x7fe65406c450 msgr2=0x7fe6540b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:59.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.065+0000 7fe65c000640 1 -- 172.21.15.167:0/1530295665 shutdown_connections 2024-05-11T12:12:59.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.066+0000 7fe65c000640 1 -- 172.21.15.167:0/1530295665 wait complete. 2024-05-11T12:12:59.088 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-11T12:12:59.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.280+0000 7f6acfe00640 1 Processor -- start 2024-05-11T12:12:59.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.280+0000 7f6acfe00640 1 -- start start 2024-05-11T12:12:59.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.282+0000 7f6acfe00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6ad0159350 0x7f6ad00cf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:59.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.282+0000 7f6acfe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ad015c450 0x7f6ad00b4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:59.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.282+0000 7f6acfe00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ad015cab0 con 0x7f6ad00cf7a0 2024-05-11T12:12:59.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.282+0000 7f6acfe00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f6ad015fff0 con 0x7f6ad0159350 2024-05-11T12:12:59.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.282+0000 7f6acfe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6ad015f230 con 0x7f6ad015c450 2024-05-11T12:12:59.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.282+0000 7f6ac7e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f6ad00cf7a0 0x7f6ad015bd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:33266/0 (socket says 172.21.15.167:33266) 2024-05-11T12:12:59.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.282+0000 7f6ac7e00640 1 -- 172.21.15.167:0/3157398409 learned_addr learned my addr 172.21.15.167:0/3157398409 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:12:59.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.282+0000 7f6acea00640 1 --2- 172.21.15.167:0/3157398409 >> v2:172.21.15.204:3300/0 conn(0x7f6ad0159350 0x7f6ad00cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:59.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.283+0000 7f6acf400640 1 --2- 172.21.15.167:0/3157398409 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ad015c450 0x7f6ad00b4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:59.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.283+0000 7f6ace000640 1 -- 172.21.15.167:0/3157398409 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 576390010 0 0) 0x7f6ad015cab0 con 0x7f6ad00cf7a0 2024-05-11T12:12:59.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.283+0000 7f6ace000640 1 -- 172.21.15.167:0/3157398409 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ab4003520 con 0x7f6ad00cf7a0 2024-05-11T12:12:59.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.283+0000 7f6ace000640 1 -- 172.21.15.167:0/3157398409 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2134414345 0 0) 0x7f6ab4003520 con 0x7f6ad00cf7a0 2024-05-11T12:12:59.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.283+0000 7f6ace000640 1 -- 172.21.15.167:0/3157398409 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ad015c450 msgr2=0x7f6ad00b4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:59.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.283+0000 7f6ace000640 1 --2- 172.21.15.167:0/3157398409 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ad015c450 0x7f6ad00b4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:59.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.283+0000 7f6ace000640 1 -- 172.21.15.167:0/3157398409 >> v2:172.21.15.204:3300/0 conn(0x7f6ad0159350 msgr2=0x7f6ad00cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:59.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.283+0000 7f6ace000640 1 --2- 172.21.15.167:0/3157398409 >> v2:172.21.15.204:3300/0 conn(0x7f6ad0159350 0x7f6ad00cf260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:59.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.283+0000 7f6ace000640 1 -- 172.21.15.167:0/3157398409 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ad015f510 con 0x7f6ad00cf7a0 2024-05-11T12:12:59.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.283+0000 7f6ace000640 1 -- 172.21.15.167:0/3157398409 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f6ab80016f0 con 0x7f6ad00cf7a0 2024-05-11T12:12:59.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.283+0000 7f6ace000640 1 -- 172.21.15.167:0/3157398409 >> v1:172.21.15.167:6789/0 conn(0x7f6ad00cf7a0 legacy=0x7f6ad015bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:59.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.283+0000 7f6ace000640 1 --2- 172.21.15.167:0/3157398409 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ad015c450 0x7f6ab4003a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:59.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.283+0000 7f6ace000640 1 --2- 172.21.15.167:0/3157398409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ab4004020 0x7f6ab4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:59.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.283+0000 7f6ace000640 1 --2- 172.21.15.167:0/3157398409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ab4004d40 0x7f6ab40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:59.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.283+0000 7f6ace000640 1 -- 172.21.15.167:0/3157398409 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad015fff0 con 0x7f6ab4004020 2024-05-11T12:12:59.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.283+0000 7f6ace000640 1 -- 172.21.15.167:0/3157398409 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad015f230 con 0x7f6ab4004d40 2024-05-11T12:12:59.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.283+0000 7f6ace000640 1 -- 172.21.15.167:0/3157398409 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6ab400ccb0 con 0x7f6ad015c450 2024-05-11T12:12:59.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.284+0000 7f6ac7e00640 1 --2- 172.21.15.167:0/3157398409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ab4004020 0x7f6ab4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:59.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.284+0000 7f6acea00640 1 --2- 172.21.15.167:0/3157398409 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ad015c450 0x7f6ab4003a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:59.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.284+0000 7f6ac7e00640 1 -- 172.21.15.167:0/3157398409 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ad015c450 msgr2=0x7f6ab4003a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:59.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.284+0000 7f6acf400640 1 --2- 172.21.15.167:0/3157398409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ab4004d40 0x7f6ab40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:59.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.284+0000 7f6ac7e00640 1 --2- 172.21.15.167:0/3157398409 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ad015c450 0x7f6ab4003a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:59.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.284+0000 7f6ac7e00640 1 -- 172.21.15.167:0/3157398409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ab4004d40 msgr2=0x7f6ab40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:59.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.284+0000 7f6ac7e00640 1 --2- 172.21.15.167:0/3157398409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ab4004d40 0x7f6ab40051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:59.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.284+0000 7f6ac7e00640 1 -- 172.21.15.167:0/3157398409 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ab4003520 con 0x7f6ab4004020 2024-05-11T12:12:59.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.284+0000 7f6acea00640 1 --2- 172.21.15.167:0/3157398409 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ad015c450 0x7f6ab4003a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:59.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.285+0000 7f6ac7e00640 1 --2- 172.21.15.167:0/3157398409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ab4004020 0x7f6ab4004800 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f6ab800c210 tx=0x7f6ab4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3284780f63bb5a19 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:59.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.285+0000 7f6ace000640 1 -- 172.21.15.167:0/3157398409 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab8007ce0 con 0x7f6ab4004020 2024-05-11T12:12:59.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.285+0000 7f6ace000640 1 -- 172.21.15.167:0/3157398409 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ab8005af0 con 0x7f6ab4004020 2024-05-11T12:12:59.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.285+0000 7f6ace000640 1 -- 172.21.15.167:0/3157398409 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab8007070 con 0x7f6ab4004020 2024-05-11T12:12:59.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.286+0000 7f6acfe00640 1 -- 172.21.15.167:0/3157398409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ab4004020 msgr2=0x7f6ab4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:59.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.286+0000 7f6acfe00640 1 --2- 172.21.15.167:0/3157398409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ab4004020 0x7f6ab4004800 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f6ab800c210 tx=0x7f6ab4005830 comp rx=0 tx=0).stop 2024-05-11T12:12:59.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.286+0000 7f6acf400640 1 -- 172.21.15.167:0/3157398409 reap_dead start 2024-05-11T12:12:59.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.286+0000 7f6acfe00640 1 -- 172.21.15.167:0/3157398409 shutdown_connections 2024-05-11T12:12:59.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.286+0000 7f6acfe00640 1 -- 172.21.15.167:0/3157398409 >> 172.21.15.167:0/3157398409 conn(0x7f6ad00b3460 msgr2=0x7f6ad00d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:59.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.286+0000 7f6acfe00640 1 -- 172.21.15.167:0/3157398409 shutdown_connections 2024-05-11T12:12:59.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.286+0000 7f6acfe00640 1 -- 172.21.15.167:0/3157398409 wait complete. 2024-05-11T12:12:59.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.288+0000 7f6acfe00640 1 Processor -- start 2024-05-11T12:12:59.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.288+0000 7f6acfe00640 1 -- start start 2024-05-11T12:12:59.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.289+0000 7f6acfe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ab4004020 0x7f6ad014d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:59.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.289+0000 7f6acfe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ad0154470 0x7f6ad006d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:59.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.289+0000 7f6acfe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ad01536e0 0x7f6ad006ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:59.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.289+0000 7f6acfe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad00ce9d0 con 0x7f6ad0154470 2024-05-11T12:12:59.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.289+0000 7f6acfe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad015fad0 con 0x7f6ad01536e0 2024-05-11T12:12:59.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.289+0000 7f6acfe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6ad00ceb50 con 0x7f6ab4004020 2024-05-11T12:12:59.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.289+0000 7f6ac7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ad0154470 0x7f6ad006d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:59.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.289+0000 7f6ac7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ad0154470 0x7f6ad006d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43188/0 (socket says 172.21.15.167:43188) 2024-05-11T12:12:59.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.289+0000 7f6ac7e00640 1 -- 172.21.15.167:0/1854812416 learned_addr learned my addr 172.21.15.167:0/1854812416 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:59.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.290+0000 7f6acea00640 1 --2- 172.21.15.167:0/1854812416 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ab4004020 0x7f6ad014d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:59.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.290+0000 7f6ac7e00640 1 -- 172.21.15.167:0/1854812416 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ab4004020 msgr2=0x7f6ad014d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:59.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.290+0000 7f6acf400640 1 --2- 172.21.15.167:0/1854812416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ad01536e0 0x7f6ad006ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:59.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.290+0000 7f6ac7e00640 1 --2- 172.21.15.167:0/1854812416 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ab4004020 0x7f6ad014d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:59.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.290+0000 7f6ac7e00640 1 -- 172.21.15.167:0/1854812416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ad01536e0 msgr2=0x7f6ad006ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:59.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.290+0000 7f6ac7e00640 1 --2- 172.21.15.167:0/1854812416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ad01536e0 0x7f6ad006ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:59.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.290+0000 7f6ac7e00640 1 -- 172.21.15.167:0/1854812416 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ad01557e0 con 0x7f6ad0154470 2024-05-11T12:12:59.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.290+0000 7f6ac7e00640 1 --2- 172.21.15.167:0/1854812416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ad0154470 0x7f6ad006d8b0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f6ab802ad20 tx=0x7f6ad015bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:59.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.290+0000 7f6ac6a00640 1 -- 172.21.15.167:0/1854812416 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab8043070 con 0x7f6ad0154470 2024-05-11T12:12:59.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.290+0000 7f6ac6a00640 1 -- 172.21.15.167:0/1854812416 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ab8038040 con 0x7f6ad0154470 2024-05-11T12:12:59.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.291+0000 7f6ac6a00640 1 -- 172.21.15.167:0/1854812416 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab80072b0 con 0x7f6ad0154470 2024-05-11T12:12:59.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.291+0000 7f6acfe00640 1 -- 172.21.15.167:0/1854812416 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ad0154970 con 0x7f6ad0154470 2024-05-11T12:12:59.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.291+0000 7f6acfe00640 1 -- 172.21.15.167:0/1854812416 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ad0155d40 con 0x7f6ad0154470 2024-05-11T12:12:59.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.292+0000 7f6acfe00640 1 -- 172.21.15.167:0/1854812416 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ad00694b0 con 0x7f6ad0154470 2024-05-11T12:12:59.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.293+0000 7f6ac6a00640 1 -- 172.21.15.167:0/1854812416 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6ab8007450 con 0x7f6ad0154470 2024-05-11T12:12:59.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.294+0000 7f6ac6a00640 1 --2- 172.21.15.167:0/1854812416 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f6aac076300 0x7f6aac0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:59.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.294+0000 7f6ac6a00640 1 -- 172.21.15.167:0/1854812416 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5249+0+0 (secure 0 0 0) 0x7f6ab80b6fe0 con 0x7f6ad0154470 2024-05-11T12:12:59.295 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.294+0000 7f6acea00640 1 --2- 172.21.15.167:0/1854812416 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f6aac076300 0x7f6aac0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:59.295 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.295+0000 7f6acea00640 1 --2- 172.21.15.167:0/1854812416 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f6aac076300 0x7f6aac0787c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6ac0009a40 tx=0x7f6ac0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:59.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.299+0000 7f6ac6a00640 1 -- 172.21.15.167:0/1854812416 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6ad01557e0 con 0x7f6ad0154470 2024-05-11T12:12:59.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.586+0000 7f6acfe00640 1 -- 172.21.15.167:0/1854812416 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f6ad00d0670 con 0x7f6ad0154470 2024-05-11T12:12:59.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.587+0000 7f6ac6a00640 1 -- 172.21.15.167:0/1854812416 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) ==== 80+0+5304 (secure 0 0 0) 0x7f6ab80016b0 con 0x7f6ad0154470 2024-05-11T12:12:59.587 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:12:59.588 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T12:12:59.588 INFO:teuthology.orchestra.run.smithi167.stdout: "auth_dump": [ 2024-05-11T12:12:59.588 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:12:59.588 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "mds.a", 2024-05-11T12:12:59.588 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAEYD9msMGnFBAAamqokXQ+znxPHtHhhs611g==", 2024-05-11T12:12:59.588 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:12:59.588 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T12:12:59.588 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow *", 2024-05-11T12:12:59.588 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow *", 2024-05-11T12:12:59.588 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:12:59.588 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:12:59.588 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:12:59.588 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:12:59.588 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "mds.b", 2024-05-11T12:12:59.589 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAEYD9mqbCgMhAA5CNUf1cz77nUqLli2bTFcQ==", 2024-05-11T12:12:59.589 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:12:59.589 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T12:12:59.589 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow *", 2024-05-11T12:12:59.589 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow *", 2024-05-11T12:12:59.589 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:12:59.589 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:12:59.589 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:12:59.589 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:12:59.589 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "mds.c", 2024-05-11T12:12:59.589 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAEYD9mhylUIxAAqrIRYuNIHKfoI00zx+Gh1Q==", 2024-05-11T12:12:59.589 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:12:59.589 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T12:12:59.589 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow *", 2024-05-11T12:12:59.590 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow *", 2024-05-11T12:12:59.590 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:12:59.590 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:12:59.590 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:12:59.590 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:12:59.590 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "mds.d", 2024-05-11T12:12:59.590 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAFYD9mcUjpBRAAbx54iDHcqexfNXGKk/avlw==", 2024-05-11T12:12:59.590 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:12:59.590 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T12:12:59.590 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow *", 2024-05-11T12:12:59.590 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow *", 2024-05-11T12:12:59.590 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:12:59.591 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:12:59.591 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:12:59.591 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:12:59.591 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.0", 2024-05-11T12:12:59.591 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAJYD9mb71mBxAAKMSzkfppZVbEevwELKrqvw==", 2024-05-11T12:12:59.591 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:12:59.591 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T12:12:59.591 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T12:12:59.591 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:12:59.591 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:12:59.591 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:12:59.591 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:12:59.591 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.1", 2024-05-11T12:12:59.591 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAMYD9m/qx7EhAAUcZntottFNBwqjdc/CqFbw==", 2024-05-11T12:12:59.592 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:12:59.592 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T12:12:59.592 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T12:12:59.592 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:12:59.592 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:12:59.592 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:12:59.592 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:12:59.592 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.2", 2024-05-11T12:12:59.592 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAPYD9m5PKRKxAAFQuzh7dAN8PVEHBq+UdQvg==", 2024-05-11T12:12:59.592 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:12:59.592 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T12:12:59.592 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T12:12:59.592 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:12:59.592 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:12:59.593 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:12:59.593 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:12:59.593 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.3", 2024-05-11T12:12:59.593 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQASYD9mZAy+NhAA49k6QbSvtSxA/5t8bWbPuQ==", 2024-05-11T12:12:59.593 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:12:59.593 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T12:12:59.593 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T12:12:59.593 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:12:59.593 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:12:59.593 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:12:59.593 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:12:59.593 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.4", 2024-05-11T12:12:59.593 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAZYD9mEbsvIBAA3nkQXQTbIf7ubq1SJtUeIA==", 2024-05-11T12:12:59.593 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:12:59.593 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T12:12:59.594 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T12:12:59.594 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:12:59.594 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:12:59.594 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:12:59.594 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:12:59.594 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.5", 2024-05-11T12:12:59.594 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAcYD9mtdsHOxAAvNUSkMgQlDc2n1ak/lfRgg==", 2024-05-11T12:12:59.594 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:12:59.594 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T12:12:59.594 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T12:12:59.594 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:12:59.594 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:12:59.594 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:12:59.595 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:12:59.595 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.6", 2024-05-11T12:12:59.595 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAgYD9mr3u8CRAAm2ShwV0lJpIVudodmviusg==", 2024-05-11T12:12:59.595 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:12:59.595 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T12:12:59.595 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T12:12:59.595 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:12:59.595 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:12:59.595 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:12:59.595 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:12:59.595 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.7", 2024-05-11T12:12:59.595 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAjYD9m+0oyFRAAUVQW0kQmvtU8IE599nrWJQ==", 2024-05-11T12:12:59.595 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:12:59.595 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T12:12:59.596 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T12:12:59.596 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:12:59.596 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:12:59.596 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:12:59.596 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:12:59.596 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "client.0", 2024-05-11T12:12:59.596 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAFYD9mLK/lEhAAyERTYMl250KkBvIrAXWYSw==", 2024-05-11T12:12:59.596 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:12:59.596 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T12:12:59.596 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow r", 2024-05-11T12:12:59.596 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-11T12:12:59.596 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:12:59.596 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:12:59.596 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:12:59.597 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "client.1", 2024-05-11T12:12:59.597 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAFYD9m2oSjJhAAz53H1SvPStkxGq5V+2VFeA==", 2024-05-11T12:12:59.597 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:12:59.597 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T12:12:59.597 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow r", 2024-05-11T12:12:59.597 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow rw", 2024-05-11T12:12:59.597 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow rwx" 2024-05-11T12:12:59.597 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:12:59.597 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:12:59.597 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:12:59.597 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "client.2", 2024-05-11T12:12:59.597 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAFYD9mMzXnHBAAmJ//hwSofaZpk+6ykxVM/A==", 2024-05-11T12:12:59.597 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:12:59.597 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T12:12:59.597 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow r", 2024-05-11T12:12:59.597 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow rw", 2024-05-11T12:12:59.598 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow rwx" 2024-05-11T12:12:59.598 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:12:59.598 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:12:59.598 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:12:59.598 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "client.3", 2024-05-11T12:12:59.598 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAFYD9muOmsLxAA/tYFT4YNy3adENgVNLljdw==", 2024-05-11T12:12:59.598 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:12:59.598 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T12:12:59.598 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow r", 2024-05-11T12:12:59.598 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow rw", 2024-05-11T12:12:59.598 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow rwx" 2024-05-11T12:12:59.598 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:12:59.598 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:12:59.599 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:12:59.599 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "client.admin", 2024-05-11T12:12:59.599 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQADYD9mItegDBAALtuDP6qzAvbrwiROf94biw==", 2024-05-11T12:12:59.599 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:12:59.599 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow *", 2024-05-11T12:12:59.599 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow *", 2024-05-11T12:12:59.599 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow *", 2024-05-11T12:12:59.599 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:12:59.599 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:12:59.599 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:12:59.599 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:12:59.599 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "mgr.x", 2024-05-11T12:12:59.599 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAEYD9mAZTKCRAAJbY821O+4pUMC0DgxRa3og==", 2024-05-11T12:12:59.599 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:12:59.600 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow *", 2024-05-11T12:12:59.600 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile mgr", 2024-05-11T12:12:59.600 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:12:59.600 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:12:59.600 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:12:59.600 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:12:59.600 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "mgr.y", 2024-05-11T12:12:59.600 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQADYD9m2cZRORAAX9kYcT5/t4f5jQRZwXHHzw==", 2024-05-11T12:12:59.600 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:12:59.600 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow *", 2024-05-11T12:12:59.600 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile mgr", 2024-05-11T12:12:59.600 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:12:59.600 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:12:59.600 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:12:59.601 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-05-11T12:12:59.601 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T12:12:59.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.590+0000 7f6acfe00640 1 -- 172.21.15.167:0/1854812416 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f6aac076300 msgr2=0x7f6aac0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:59.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.590+0000 7f6acfe00640 1 --2- 172.21.15.167:0/1854812416 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f6aac076300 0x7f6aac0787c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6ac0009a40 tx=0x7f6ac0008040 comp rx=0 tx=0).stop 2024-05-11T12:12:59.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.590+0000 7f6acfe00640 1 -- 172.21.15.167:0/1854812416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ad0154470 msgr2=0x7f6ad006d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:59.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.590+0000 7f6acfe00640 1 --2- 172.21.15.167:0/1854812416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ad0154470 0x7f6ad006d8b0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f6ab802ad20 tx=0x7f6ad015bb10 comp rx=0 tx=0).stop 2024-05-11T12:12:59.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.590+0000 7f6acfe00640 1 -- 172.21.15.167:0/1854812416 shutdown_connections 2024-05-11T12:12:59.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.590+0000 7f6acfe00640 1 --2- 172.21.15.167:0/1854812416 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f6aac076300 0x7f6aac0787c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:59.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.590+0000 7f6acfe00640 1 --2- 172.21.15.167:0/1854812416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ad01536e0 0x7f6ad006ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:59.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.591+0000 7f6acfe00640 1 --2- 172.21.15.167:0/1854812416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ad0154470 0x7f6ad006d8b0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:59.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.591+0000 7f6acfe00640 1 --2- 172.21.15.167:0/1854812416 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ab4004020 0x7f6ad014d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:59.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.591+0000 7f6acfe00640 1 -- 172.21.15.167:0/1854812416 >> 172.21.15.167:0/1854812416 conn(0x7f6ad00b3460 msgr2=0x7f6ad00727d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:59.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.591+0000 7f6acfe00640 1 -- 172.21.15.167:0/1854812416 shutdown_connections 2024-05-11T12:12:59.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.591+0000 7f6acfe00640 1 -- 172.21.15.167:0/1854812416 wait complete. 2024-05-11T12:12:59.610 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-11T12:12:59.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.812+0000 7fc2d5a00640 1 Processor -- start 2024-05-11T12:12:59.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.813+0000 7fc2d5a00640 1 -- start start 2024-05-11T12:12:59.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.814+0000 7fc2d5a00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc2d0145b00 0x7fc2d006f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:59.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.814+0000 7fc2d5a00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2d006fbc0 con 0x7fc2d0146770 2024-05-11T12:12:59.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.814+0000 7fc2d5a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2d00b4520 con 0x7fc2d006c660 2024-05-11T12:12:59.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.814+0000 7fc2d5a00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fc2d00b4800 con 0x7fc2d0145b00 2024-05-11T12:12:59.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.815+0000 7fc2cfe00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc2d0145b00 0x7fc2d006f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:59.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.815+0000 7fc2cfe00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc2d0145b00 0x7fc2d006f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43194/0 (socket says 172.21.15.167:43194) 2024-05-11T12:12:59.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.815+0000 7fc2cf400640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fc2d0146770 0x7fc2d00ce9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:33278/0 (socket says 172.21.15.167:33278) 2024-05-11T12:12:59.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.815+0000 7fc2cfe00640 1 -- 172.21.15.167:0/552490072 learned_addr learned my addr 172.21.15.167:0/552490072 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:59.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.815+0000 7fc2cfe00640 1 -- 172.21.15.167:0/552490072 >> v1:172.21.15.204:6789/0 conn(0x7fc2d006c660 legacy=0x7fc2d006ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:12:59.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.815+0000 7fc2cfe00640 1 -- 172.21.15.167:0/552490072 >> v1:172.21.15.167:6789/0 conn(0x7fc2d0146770 legacy=0x7fc2d00ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:59.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.815+0000 7fc2cea00640 1 -- 172.21.15.167:0/552490072 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4223197207 0 0) 0x7fc2d006fbc0 con 0x7fc2d0146770 2024-05-11T12:12:59.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.815+0000 7fc2cfe00640 1 -- 172.21.15.167:0/552490072 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2d00b4ae0 con 0x7fc2d0145b00 2024-05-11T12:12:59.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.816+0000 7fc2cfe00640 1 --2- 172.21.15.167:0/552490072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2d0145b00 0x7fc2d006f2c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0009870 tx=0x7fc2c002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19bd1e19b7125d92 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:59.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.817+0000 7fc2cea00640 1 -- 172.21.15.167:0/552490072 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0037070 con 0x7fc2d0145b00 2024-05-11T12:12:59.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.817+0000 7fc2cea00640 1 -- 172.21.15.167:0/552490072 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2c0032440 con 0x7fc2d0145b00 2024-05-11T12:12:59.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.817+0000 7fc2cea00640 1 -- 172.21.15.167:0/552490072 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0032810 con 0x7fc2d0145b00 2024-05-11T12:12:59.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.817+0000 7fc2d5a00640 1 -- 172.21.15.167:0/552490072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2d0145b00 msgr2=0x7fc2d006f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:59.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.817+0000 7fc2d5a00640 1 --2- 172.21.15.167:0/552490072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2d0145b00 0x7fc2d006f2c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0009870 tx=0x7fc2c002a1c0 comp rx=0 tx=0).stop 2024-05-11T12:12:59.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.817+0000 7fc2d5a00640 1 -- 172.21.15.167:0/552490072 shutdown_connections 2024-05-11T12:12:59.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.817+0000 7fc2d5a00640 1 --2- 172.21.15.167:0/552490072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2d0145b00 0x7fc2d006f2c0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:59.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.817+0000 7fc2d5a00640 1 -- 172.21.15.167:0/552490072 >> 172.21.15.167:0/552490072 conn(0x7fc2d00b3460 msgr2=0x7fc2d00d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:59.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.818+0000 7fc2d5a00640 1 -- 172.21.15.167:0/552490072 shutdown_connections 2024-05-11T12:12:59.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.818+0000 7fc2d5a00640 1 -- 172.21.15.167:0/552490072 wait complete. 2024-05-11T12:12:59.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.819+0000 7fc2d5a00640 1 Processor -- start 2024-05-11T12:12:59.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.819+0000 7fc2d5a00640 1 -- start start 2024-05-11T12:12:59.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.820+0000 7fc2d5a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2d006c660 0x7fc2d0140b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:59.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.820+0000 7fc2d5a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2d0145b00 0x7fc2d0143590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:59.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.820+0000 7fc2d5a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc2d0146770 0x7fc2d006e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:59.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.820+0000 7fc2d5a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d006f800 con 0x7fc2d006c660 2024-05-11T12:12:59.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.820+0000 7fc2d5a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d0145ee0 con 0x7fc2d0145b00 2024-05-11T12:12:59.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.820+0000 7fc2d5a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc2d006f980 con 0x7fc2d0146770 2024-05-11T12:12:59.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.821+0000 7fc2cfe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2d006c660 0x7fc2d0140b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:59.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.821+0000 7fc2cfe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2d006c660 0x7fc2d0140b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43210/0 (socket says 172.21.15.167:43210) 2024-05-11T12:12:59.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.821+0000 7fc2cfe00640 1 -- 172.21.15.167:0/2806898327 learned_addr learned my addr 172.21.15.167:0/2806898327 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:12:59.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.821+0000 7fc2cf400640 1 --2- 172.21.15.167:0/2806898327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2d0145b00 0x7fc2d0143590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:59.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.821+0000 7fc2d5000640 1 --2- 172.21.15.167:0/2806898327 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc2d0146770 0x7fc2d006e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:59.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.821+0000 7fc2cfe00640 1 -- 172.21.15.167:0/2806898327 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc2d0146770 msgr2=0x7fc2d006e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:59.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.821+0000 7fc2cfe00640 1 --2- 172.21.15.167:0/2806898327 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc2d0146770 0x7fc2d006e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:59.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.821+0000 7fc2cfe00640 1 -- 172.21.15.167:0/2806898327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2d0145b00 msgr2=0x7fc2d0143590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:59.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.821+0000 7fc2cfe00640 1 --2- 172.21.15.167:0/2806898327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2d0145b00 0x7fc2d0143590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:59.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.821+0000 7fc2cfe00640 1 -- 172.21.15.167:0/2806898327 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2d0144050 con 0x7fc2d006c660 2024-05-11T12:12:59.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.821+0000 7fc2cf400640 1 --2- 172.21.15.167:0/2806898327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2d0145b00 0x7fc2d0143590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:12:59.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.821+0000 7fc2cfe00640 1 --2- 172.21.15.167:0/2806898327 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2d006c660 0x7fc2d0140b10 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0009870 tx=0x7fc2c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:59.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.822+0000 7fc2c7e00640 1 -- 172.21.15.167:0/2806898327 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0037040 con 0x7fc2d006c660 2024-05-11T12:12:59.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.822+0000 7fc2c7e00640 1 -- 172.21.15.167:0/2806898327 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2c0004250 con 0x7fc2d006c660 2024-05-11T12:12:59.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.822+0000 7fc2d5a00640 1 -- 172.21.15.167:0/2806898327 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2d0144330 con 0x7fc2d006c660 2024-05-11T12:12:59.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.822+0000 7fc2c7e00640 1 -- 172.21.15.167:0/2806898327 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c003c670 con 0x7fc2d006c660 2024-05-11T12:12:59.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.822+0000 7fc2d5a00640 1 -- 172.21.15.167:0/2806898327 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2d01448f0 con 0x7fc2d006c660 2024-05-11T12:12:59.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.823+0000 7fc2d5a00640 1 -- 172.21.15.167:0/2806898327 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2d0144b30 con 0x7fc2d006c660 2024-05-11T12:12:59.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.824+0000 7fc2c7e00640 1 -- 172.21.15.167:0/2806898327 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc2c003a070 con 0x7fc2d006c660 2024-05-11T12:12:59.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.826+0000 7fc2c7e00640 1 --2- 172.21.15.167:0/2806898327 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc2ac0762d0 0x7fc2ac078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:59.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.826+0000 7fc2c7e00640 1 -- 172.21.15.167:0/2806898327 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5249+0+0 (secure 0 0 0) 0x7fc2c00b7e10 con 0x7fc2d006c660 2024-05-11T12:12:59.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.826+0000 7fc2cf400640 1 --2- 172.21.15.167:0/2806898327 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc2ac0762d0 0x7fc2ac078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:59.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.827+0000 7fc2cf400640 1 --2- 172.21.15.167:0/2806898327 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc2ac0762d0 0x7fc2ac078790 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0161b70 tx=0x7fc2b8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:59.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:12:59.831+0000 7fc2c7e00640 1 -- 172.21.15.167:0/2806898327 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc2c00bb030 con 0x7fc2d006c660 2024-05-11T12:13:00.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.042+0000 7fc2d5a00640 1 -- 172.21.15.167:0/2806898327 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fc2d00b11e0 con 0x7fc2d006c660 2024-05-11T12:13:00.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.043+0000 7fc2c7e00640 1 -- 172.21.15.167:0/2806898327 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v60) ==== 84+0+65 (secure 0 0 0) 0x7fc2c0041020 con 0x7fc2d006c660 2024-05-11T12:13:00.044 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:13:00.044 INFO:teuthology.orchestra.run.smithi167.stdout:[ 2024-05-11T12:13:00.044 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:13:00.044 INFO:teuthology.orchestra.run.smithi167.stdout: "poolnum": 1, 2024-05-11T12:13:00.044 INFO:teuthology.orchestra.run.smithi167.stdout: "poolname": ".mgr" 2024-05-11T12:13:00.044 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:13:00.044 INFO:teuthology.orchestra.run.smithi167.stdout:] 2024-05-11T12:13:00.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.046+0000 7fc2d5a00640 1 -- 172.21.15.167:0/2806898327 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc2ac0762d0 msgr2=0x7fc2ac078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:00.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.046+0000 7fc2d5a00640 1 --2- 172.21.15.167:0/2806898327 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc2ac0762d0 0x7fc2ac078790 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0161b70 tx=0x7fc2b8009040 comp rx=0 tx=0).stop 2024-05-11T12:13:00.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.046+0000 7fc2d5a00640 1 -- 172.21.15.167:0/2806898327 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2d006c660 msgr2=0x7fc2d0140b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:00.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.046+0000 7fc2d5a00640 1 --2- 172.21.15.167:0/2806898327 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2d006c660 0x7fc2d0140b10 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0009870 tx=0x7fc2c00046c0 comp rx=0 tx=0).stop 2024-05-11T12:13:00.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.046+0000 7fc2d5a00640 1 -- 172.21.15.167:0/2806898327 shutdown_connections 2024-05-11T12:13:00.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.046+0000 7fc2d5a00640 1 --2- 172.21.15.167:0/2806898327 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc2d0146770 0x7fc2d006e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:00.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.046+0000 7fc2d5a00640 1 --2- 172.21.15.167:0/2806898327 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc2ac0762d0 0x7fc2ac078790 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:00.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.046+0000 7fc2d5a00640 1 --2- 172.21.15.167:0/2806898327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2d0145b00 0x7fc2d0143590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:00.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.046+0000 7fc2d5a00640 1 --2- 172.21.15.167:0/2806898327 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2d006c660 0x7fc2d0140b10 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:00.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.046+0000 7fc2d5a00640 1 -- 172.21.15.167:0/2806898327 >> 172.21.15.167:0/2806898327 conn(0x7fc2d00b3460 msgr2=0x7fc2d015f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:00.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.046+0000 7fc2d5a00640 1 -- 172.21.15.167:0/2806898327 shutdown_connections 2024-05-11T12:13:00.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.046+0000 7fc2d5a00640 1 -- 172.21.15.167:0/2806898327 wait complete. 2024-05-11T12:13:00.066 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-11T12:13:00.067 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-11T12:13:00.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.264+0000 7f6da3400640 1 Processor -- start 2024-05-11T12:13:00.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.265+0000 7f6da3400640 1 -- start start 2024-05-11T12:13:00.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.266+0000 7f6da3400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6d9c0d0180 0x7f6d9c0b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:00.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.266+0000 7f6da3400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d9c0b5090 0x7f6d9c1429f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:00.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.266+0000 7f6da3400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6d9c142f30 0x7f6d9c070000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:00.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.266+0000 7f6da3400640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f6d9c0b54f0 con 0x7f6d9c142f30 2024-05-11T12:13:00.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.266+0000 7f6da3400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f6d9c140480 con 0x7f6d9c0d0180 2024-05-11T12:13:00.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.266+0000 7f6da3400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6d9c161bd0 con 0x7f6d9c0b5090 2024-05-11T12:13:00.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.267+0000 7f6da1400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6d9c142f30 0x7f6d9c070000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:00.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.267+0000 7f6da1400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6d9c142f30 0x7f6d9c070000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43214/0 (socket says 172.21.15.167:43214) 2024-05-11T12:13:00.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.267+0000 7f6da1400640 1 -- 172.21.15.167:0/1506872376 learned_addr learned my addr 172.21.15.167:0/1506872376 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:00.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.267+0000 7f6d9be00640 1 --2- 172.21.15.167:0/1506872376 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d9c0b5090 0x7f6d9c1429f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:00.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.267+0000 7f6da0a00640 1 --2- 172.21.15.167:0/1506872376 >> v2:172.21.15.204:3300/0 conn(0x7f6d9c0d0180 0x7f6d9c0b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:00.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.267+0000 7f6da1400640 1 -- 172.21.15.167:0/1506872376 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d9c0b5090 msgr2=0x7f6d9c1429f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:00.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.267+0000 7f6da1400640 1 --2- 172.21.15.167:0/1506872376 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d9c0b5090 0x7f6d9c1429f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:00.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.267+0000 7f6da1400640 1 -- 172.21.15.167:0/1506872376 >> v2:172.21.15.204:3300/0 conn(0x7f6d9c0d0180 msgr2=0x7f6d9c0b4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:00.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.267+0000 7f6da1400640 1 --2- 172.21.15.167:0/1506872376 >> v2:172.21.15.204:3300/0 conn(0x7f6d9c0d0180 0x7f6d9c0b4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:00.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.267+0000 7f6da1400640 1 -- 172.21.15.167:0/1506872376 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d9c160e10 con 0x7f6d9c142f30 2024-05-11T12:13:00.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.268+0000 7f6da1400640 1 --2- 172.21.15.167:0/1506872376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d9c142f30 0x7f6d9c070000 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c072210 tx=0x7f6d9402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26cfcd830e3c1bfa server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:00.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.269+0000 7f6d9b400640 1 -- 172.21.15.167:0/1506872376 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d94037070 con 0x7f6d9c142f30 2024-05-11T12:13:00.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.269+0000 7f6d9b400640 1 -- 172.21.15.167:0/1506872376 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d94032440 con 0x7f6d9c142f30 2024-05-11T12:13:00.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.269+0000 7f6d9b400640 1 -- 172.21.15.167:0/1506872376 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d94032810 con 0x7f6d9c142f30 2024-05-11T12:13:00.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.269+0000 7f6da3400640 1 -- 172.21.15.167:0/1506872376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d9c142f30 msgr2=0x7f6d9c070000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:00.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.269+0000 7f6da3400640 1 --2- 172.21.15.167:0/1506872376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d9c142f30 0x7f6d9c070000 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c072210 tx=0x7f6d9402a1e0 comp rx=0 tx=0).stop 2024-05-11T12:13:00.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.269+0000 7f6da3400640 1 -- 172.21.15.167:0/1506872376 shutdown_connections 2024-05-11T12:13:00.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.269+0000 7f6da3400640 1 --2- 172.21.15.167:0/1506872376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d9c142f30 0x7f6d9c070000 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:00.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.269+0000 7f6da3400640 1 --2- 172.21.15.167:0/1506872376 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d9c0b5090 0x7f6d9c1429f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:00.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.269+0000 7f6da3400640 1 --2- 172.21.15.167:0/1506872376 >> v2:172.21.15.204:3300/0 conn(0x7f6d9c0d0180 0x7f6d9c0b4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:00.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.269+0000 7f6da3400640 1 -- 172.21.15.167:0/1506872376 >> 172.21.15.167:0/1506872376 conn(0x7f6d9c072870 msgr2=0x7f6d9c06c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:00.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.269+0000 7f6da3400640 1 -- 172.21.15.167:0/1506872376 shutdown_connections 2024-05-11T12:13:00.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.270+0000 7f6da3400640 1 -- 172.21.15.167:0/1506872376 wait complete. 2024-05-11T12:13:00.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.271+0000 7f6da3400640 1 Processor -- start 2024-05-11T12:13:00.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.271+0000 7f6da3400640 1 -- start start 2024-05-11T12:13:00.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.272+0000 7f6da3400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d9c0b5090 0x7f6d9c06f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:00.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.272+0000 7f6da3400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d9c0d0180 0x7f6d9c15ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:00.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.272+0000 7f6da3400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d9c142f30 0x7f6d9c06ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:00.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.272+0000 7f6da3400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9c1616b0 con 0x7f6d9c0b5090 2024-05-11T12:13:00.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.272+0000 7f6da3400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9c1613d0 con 0x7f6d9c0d0180 2024-05-11T12:13:00.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.272+0000 7f6da3400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6d9c0d18a0 con 0x7f6d9c142f30 2024-05-11T12:13:00.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.272+0000 7f6da0a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d9c0b5090 0x7f6d9c06f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:00.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.273+0000 7f6da0a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d9c0b5090 0x7f6d9c06f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43216/0 (socket says 172.21.15.167:43216) 2024-05-11T12:13:00.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.273+0000 7f6da0a00640 1 -- 172.21.15.167:0/3571404061 learned_addr learned my addr 172.21.15.167:0/3571404061 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:00.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.273+0000 7f6d9be00640 1 --2- 172.21.15.167:0/3571404061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d9c0d0180 0x7f6d9c15ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:00.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.273+0000 7f6da1400640 1 --2- 172.21.15.167:0/3571404061 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d9c142f30 0x7f6d9c06ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:00.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.273+0000 7f6da0a00640 1 -- 172.21.15.167:0/3571404061 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d9c142f30 msgr2=0x7f6d9c06ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:00.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.273+0000 7f6da0a00640 1 --2- 172.21.15.167:0/3571404061 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d9c142f30 0x7f6d9c06ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:00.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.273+0000 7f6da0a00640 1 -- 172.21.15.167:0/3571404061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d9c0d0180 msgr2=0x7f6d9c15ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:00.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.273+0000 7f6da0a00640 1 --2- 172.21.15.167:0/3571404061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d9c0d0180 0x7f6d9c15ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:00.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.273+0000 7f6da0a00640 1 -- 172.21.15.167:0/3571404061 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d9c15e270 con 0x7f6d9c0b5090 2024-05-11T12:13:00.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.273+0000 7f6da0a00640 1 --2- 172.21.15.167:0/3571404061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d9c0b5090 0x7f6d9c06f9a0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c00c7f0 tx=0x7f6d8c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:00.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.274+0000 7f6d98c00640 1 -- 172.21.15.167:0/3571404061 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c015070 con 0x7f6d9c0b5090 2024-05-11T12:13:00.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.274+0000 7f6d98c00640 1 -- 172.21.15.167:0/3571404061 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d8c004240 con 0x7f6d9c0b5090 2024-05-11T12:13:00.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.274+0000 7f6d98c00640 1 -- 172.21.15.167:0/3571404061 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c007620 con 0x7f6d9c0b5090 2024-05-11T12:13:00.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.274+0000 7f6da3400640 1 -- 172.21.15.167:0/3571404061 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d9c070310 con 0x7f6d9c0b5090 2024-05-11T12:13:00.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.274+0000 7f6da3400640 1 -- 172.21.15.167:0/3571404061 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d9c15e530 con 0x7f6d9c0b5090 2024-05-11T12:13:00.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.275+0000 7f6da3400640 1 -- 172.21.15.167:0/3571404061 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d9c069bc0 con 0x7f6d9c0b5090 2024-05-11T12:13:00.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.276+0000 7f6d98c00640 1 -- 172.21.15.167:0/3571404061 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6d8c0079d0 con 0x7f6d9c0b5090 2024-05-11T12:13:00.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.277+0000 7f6d98c00640 1 --2- 172.21.15.167:0/3571404061 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f6d780762d0 0x7f6d78078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:00.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.277+0000 7f6d98c00640 1 -- 172.21.15.167:0/3571404061 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5249+0+0 (secure 0 0 0) 0x7f6d8c0974f0 con 0x7f6d9c0b5090 2024-05-11T12:13:00.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.277+0000 7f6d9be00640 1 --2- 172.21.15.167:0/3571404061 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f6d780762d0 0x7f6d78078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:00.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.278+0000 7f6d9be00640 1 --2- 172.21.15.167:0/3571404061 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f6d780762d0 0x7f6d78078790 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6d84008ec0 tx=0x7f6d84008410 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:00.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.282+0000 7f6d98c00640 1 -- 172.21.15.167:0/3571404061 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6d8c061000 con 0x7f6d9c0b5090 2024-05-11T12:13:00.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:00.490+0000 7f6da3400640 1 -- 172.21.15.167:0/3571404061 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f6d9c0b4730 con 0x7f6d9c0b5090 2024-05-11T12:13:01.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.087+0000 7f6d98c00640 1 -- 172.21.15.167:0/3571404061 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v61) ==== 138+0+0 (secure 0 0 0) 0x7f6d8c060900 con 0x7f6d9c0b5090 2024-05-11T12:13:01.088 INFO:teuthology.orchestra.run.smithi167.stderr:pool 'cephfs_metadata' created 2024-05-11T12:13:01.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.091+0000 7f6da3400640 1 -- 172.21.15.167:0/3571404061 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f6d780762d0 msgr2=0x7f6d78078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:01.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.091+0000 7f6da3400640 1 --2- 172.21.15.167:0/3571404061 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f6d780762d0 0x7f6d78078790 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6d84008ec0 tx=0x7f6d84008410 comp rx=0 tx=0).stop 2024-05-11T12:13:01.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.091+0000 7f6da3400640 1 -- 172.21.15.167:0/3571404061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d9c0b5090 msgr2=0x7f6d9c06f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:01.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.091+0000 7f6da3400640 1 --2- 172.21.15.167:0/3571404061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d9c0b5090 0x7f6d9c06f9a0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c00c7f0 tx=0x7f6d8c00ccb0 comp rx=0 tx=0).stop 2024-05-11T12:13:01.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.092+0000 7f6da3400640 1 -- 172.21.15.167:0/3571404061 shutdown_connections 2024-05-11T12:13:01.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.092+0000 7f6da3400640 1 --2- 172.21.15.167:0/3571404061 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d9c142f30 0x7f6d9c06ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:01.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.092+0000 7f6da3400640 1 --2- 172.21.15.167:0/3571404061 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f6d780762d0 0x7f6d78078790 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:01.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.092+0000 7f6da3400640 1 --2- 172.21.15.167:0/3571404061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d9c0d0180 0x7f6d9c15ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:01.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.092+0000 7f6da3400640 1 --2- 172.21.15.167:0/3571404061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d9c0b5090 0x7f6d9c06f9a0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:01.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.092+0000 7f6da3400640 1 -- 172.21.15.167:0/3571404061 >> 172.21.15.167:0/3571404061 conn(0x7f6d9c072870 msgr2=0x7f6d9c0b64d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:01.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.092+0000 7f6da3400640 1 -- 172.21.15.167:0/3571404061 shutdown_connections 2024-05-11T12:13:01.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.092+0000 7f6da3400640 1 -- 172.21.15.167:0/3571404061 wait complete. 2024-05-11T12:13:01.120 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-11T12:13:01.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.318+0000 7fb613600640 1 Processor -- start 2024-05-11T12:13:01.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.318+0000 7fb613600640 1 -- start start 2024-05-11T12:13:01.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.319+0000 7fb613600640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb60c15e360 0x7fb60c142e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:01.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.319+0000 7fb613600640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb60c161a60 con 0x7fb60c072aa0 2024-05-11T12:13:01.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.319+0000 7fb613600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb60c143440 con 0x7fb60c06ea60 2024-05-11T12:13:01.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.319+0000 7fb613600640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fb60c15d620 con 0x7fb60c15e360 2024-05-11T12:13:01.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.320+0000 7fb610c00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fb60c072aa0 0x7fb60c06e350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:33294/0 (socket says 172.21.15.167:33294) 2024-05-11T12:13:01.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.320+0000 7fb610c00640 1 -- 172.21.15.167:0/4242133859 learned_addr learned my addr 172.21.15.167:0/4242133859 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:13:01.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.320+0000 7fb611600640 1 --2- 172.21.15.167:0/4242133859 >> v2:172.21.15.167:3300/0 conn(0x7fb60c15e360 0x7fb60c142e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:01.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.320+0000 7fb611600640 1 -- 172.21.15.167:0/4242133859 >> v1:172.21.15.204:6789/0 conn(0x7fb60c06ea60 legacy=0x7fb60c142750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:01.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.320+0000 7fb60b400640 1 -- 172.21.15.167:0/4242133859 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1959343908 0 0) 0x7fb60c161a60 con 0x7fb60c072aa0 2024-05-11T12:13:01.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.320+0000 7fb611600640 1 -- 172.21.15.167:0/4242133859 >> v1:172.21.15.167:6789/0 conn(0x7fb60c072aa0 legacy=0x7fb60c06e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:01.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.320+0000 7fb611600640 1 -- 172.21.15.167:0/4242133859 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb60c15d880 con 0x7fb60c15e360 2024-05-11T12:13:01.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.321+0000 7fb611600640 1 --2- 172.21.15.167:0/4242133859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb60c15e360 0x7fb60c142e60 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb604009d90 tx=0x7fb60402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aa077064f4c1ef0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:01.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.322+0000 7fb60b400640 1 -- 172.21.15.167:0/4242133859 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb604007740 con 0x7fb60c15e360 2024-05-11T12:13:01.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.322+0000 7fb60b400640 1 -- 172.21.15.167:0/4242133859 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6040078e0 con 0x7fb60c15e360 2024-05-11T12:13:01.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.322+0000 7fb60b400640 1 -- 172.21.15.167:0/4242133859 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb604007cb0 con 0x7fb60c15e360 2024-05-11T12:13:01.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.322+0000 7fb613600640 1 -- 172.21.15.167:0/4242133859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb60c15e360 msgr2=0x7fb60c142e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:01.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.322+0000 7fb613600640 1 --2- 172.21.15.167:0/4242133859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb60c15e360 0x7fb60c142e60 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb604009d90 tx=0x7fb60402a6f0 comp rx=0 tx=0).stop 2024-05-11T12:13:01.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.322+0000 7fb613600640 1 -- 172.21.15.167:0/4242133859 shutdown_connections 2024-05-11T12:13:01.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.322+0000 7fb613600640 1 --2- 172.21.15.167:0/4242133859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb60c15e360 0x7fb60c142e60 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:01.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.322+0000 7fb613600640 1 -- 172.21.15.167:0/4242133859 >> 172.21.15.167:0/4242133859 conn(0x7fb60c0b2f60 msgr2=0x7fb60c0b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:01.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.323+0000 7fb613600640 1 -- 172.21.15.167:0/4242133859 shutdown_connections 2024-05-11T12:13:01.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.323+0000 7fb613600640 1 -- 172.21.15.167:0/4242133859 wait complete. 2024-05-11T12:13:01.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.324+0000 7fb613600640 1 Processor -- start 2024-05-11T12:13:01.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.324+0000 7fb613600640 1 -- start start 2024-05-11T12:13:01.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.325+0000 7fb613600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb60c06ea60 0x7fb60c1e6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:01.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.325+0000 7fb613600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb60c072aa0 0x7fb60c0d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:01.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.325+0000 7fb613600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb60c15e360 0x7fb60c141840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:01.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.325+0000 7fb613600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb60c072e80 con 0x7fb60c06ea60 2024-05-11T12:13:01.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.325+0000 7fb613600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb60c15de40 con 0x7fb60c15e360 2024-05-11T12:13:01.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.325+0000 7fb613600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb60c073000 con 0x7fb60c072aa0 2024-05-11T12:13:01.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.326+0000 7fb610c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb60c06ea60 0x7fb60c1e6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:01.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.326+0000 7fb610c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb60c06ea60 0x7fb60c1e6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43242/0 (socket says 172.21.15.167:43242) 2024-05-11T12:13:01.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.326+0000 7fb610c00640 1 -- 172.21.15.167:0/647967483 learned_addr learned my addr 172.21.15.167:0/647967483 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:01.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.326+0000 7fb611600640 1 --2- 172.21.15.167:0/647967483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb60c15e360 0x7fb60c141840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:01.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.326+0000 7fb610c00640 1 -- 172.21.15.167:0/647967483 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb60c072aa0 msgr2=0x7fb60c0d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:01.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.326+0000 7fb610c00640 1 --2- 172.21.15.167:0/647967483 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb60c072aa0 0x7fb60c0d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:01.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.326+0000 7fb610c00640 1 -- 172.21.15.167:0/647967483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb60c15e360 msgr2=0x7fb60c141840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:01.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.326+0000 7fb610c00640 1 --2- 172.21.15.167:0/647967483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb60c15e360 0x7fb60c141840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:01.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.326+0000 7fb610c00640 1 -- 172.21.15.167:0/647967483 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb60c1e6f90 con 0x7fb60c06ea60 2024-05-11T12:13:01.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.326+0000 7fb611600640 1 --2- 172.21.15.167:0/647967483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb60c15e360 0x7fb60c141840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:01.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.326+0000 7fb610c00640 1 --2- 172.21.15.167:0/647967483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb60c06ea60 0x7fb60c1e6990 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb60000c450 tx=0x7fb60000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:01.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.327+0000 7fb608c00640 1 -- 172.21.15.167:0/647967483 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb600018070 con 0x7fb60c06ea60 2024-05-11T12:13:01.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.327+0000 7fb608c00640 1 -- 172.21.15.167:0/647967483 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6000074e0 con 0x7fb60c06ea60 2024-05-11T12:13:01.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.327+0000 7fb608c00640 1 -- 172.21.15.167:0/647967483 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb600004a20 con 0x7fb60c06ea60 2024-05-11T12:13:01.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.327+0000 7fb613600640 1 -- 172.21.15.167:0/647967483 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb60c1e7270 con 0x7fb60c06ea60 2024-05-11T12:13:01.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.327+0000 7fb613600640 1 -- 172.21.15.167:0/647967483 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb60c1e7830 con 0x7fb60c06ea60 2024-05-11T12:13:01.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.328+0000 7fb613600640 1 -- 172.21.15.167:0/647967483 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb60c069630 con 0x7fb60c06ea60 2024-05-11T12:13:01.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.329+0000 7fb608c00640 1 -- 172.21.15.167:0/647967483 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb600004bc0 con 0x7fb60c06ea60 2024-05-11T12:13:01.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.330+0000 7fb608c00640 1 --2- 172.21.15.167:0/647967483 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb5ec076300 0x7fb5ec0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:01.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.331+0000 7fb608c00640 1 -- 172.21.15.167:0/647967483 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5619+0+0 (secure 0 0 0) 0x7fb6000967e0 con 0x7fb60c06ea60 2024-05-11T12:13:01.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.331+0000 7fb60be00640 1 --2- 172.21.15.167:0/647967483 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb5ec076300 0x7fb5ec0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:01.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.331+0000 7fb60be00640 1 --2- 172.21.15.167:0/647967483 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb5ec076300 0x7fb5ec0787c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8000f60 tx=0x7fb5f80079b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:01.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.336+0000 7fb608c00640 1 -- 172.21.15.167:0/647967483 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb600060240 con 0x7fb60c06ea60 2024-05-11T12:13:01.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:01.549+0000 7fb613600640 1 -- 172.21.15.167:0/647967483 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fb60c06d7e0 con 0x7fb60c06ea60 2024-05-11T12:13:02.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.093+0000 7fb608c00640 1 -- 172.21.15.167:0/647967483 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v62) ==== 170+0+0 (secure 0 0 0) 0x7fb60005fb40 con 0x7fb60c06ea60 2024-05-11T12:13:02.094 INFO:teuthology.orchestra.run.smithi167.stderr:pool 'cephfs_data' created 2024-05-11T12:13:02.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.098+0000 7fb613600640 1 -- 172.21.15.167:0/647967483 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb5ec076300 msgr2=0x7fb5ec0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:02.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.098+0000 7fb613600640 1 --2- 172.21.15.167:0/647967483 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb5ec076300 0x7fb5ec0787c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8000f60 tx=0x7fb5f80079b0 comp rx=0 tx=0).stop 2024-05-11T12:13:02.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.098+0000 7fb613600640 1 -- 172.21.15.167:0/647967483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb60c06ea60 msgr2=0x7fb60c1e6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:02.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.098+0000 7fb613600640 1 --2- 172.21.15.167:0/647967483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb60c06ea60 0x7fb60c1e6990 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb60000c450 tx=0x7fb60000c910 comp rx=0 tx=0).stop 2024-05-11T12:13:02.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.098+0000 7fb613600640 1 -- 172.21.15.167:0/647967483 shutdown_connections 2024-05-11T12:13:02.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.098+0000 7fb613600640 1 --2- 172.21.15.167:0/647967483 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb5ec076300 0x7fb5ec0787c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:02.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.098+0000 7fb613600640 1 --2- 172.21.15.167:0/647967483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb60c15e360 0x7fb60c141840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:02.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.098+0000 7fb613600640 1 --2- 172.21.15.167:0/647967483 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb60c072aa0 0x7fb60c0d07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:02.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.098+0000 7fb613600640 1 --2- 172.21.15.167:0/647967483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb60c06ea60 0x7fb60c1e6990 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:02.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.098+0000 7fb613600640 1 -- 172.21.15.167:0/647967483 >> 172.21.15.167:0/647967483 conn(0x7fb60c0b2f60 msgr2=0x7fb60c0b1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:02.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.098+0000 7fb613600640 1 -- 172.21.15.167:0/647967483 shutdown_connections 2024-05-11T12:13:02.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.099+0000 7fb613600640 1 -- 172.21.15.167:0/647967483 wait complete. 2024-05-11T12:13:02.135 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-11T12:13:02.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.315+0000 7f14aa000640 1 Processor -- start 2024-05-11T12:13:02.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.315+0000 7f14aa000640 1 -- start start 2024-05-11T12:13:02.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.316+0000 7f14aa000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14a40d0660 0x7f14a41609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:02.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.316+0000 7f14aa000640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f14a4160ef0 0x7f14a406c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:02.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.316+0000 7f14aa000640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14a40cf5c0 con 0x7f14a413fe30 2024-05-11T12:13:02.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.316+0000 7f14aa000640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f14a40cf8a0 con 0x7f14a4160ef0 2024-05-11T12:13:02.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.316+0000 7f14aa000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f14a40cfb80 con 0x7f14a40d0660 2024-05-11T12:13:02.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.316+0000 7f14a8c00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f14a413fe30 0x7f14a415de40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:33302/0 (socket says 172.21.15.167:33302) 2024-05-11T12:13:02.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.317+0000 7f14a8c00640 1 -- 172.21.15.167:0/4021401178 learned_addr learned my addr 172.21.15.167:0/4021401178 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:13:02.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.317+0000 7f14a9600640 1 --2- 172.21.15.167:0/4021401178 >> v2:172.21.15.167:3300/0 conn(0x7f14a4160ef0 0x7f14a406c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:02.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.317+0000 7f149be00640 1 --2- 172.21.15.167:0/4021401178 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14a40d0660 0x7f14a41609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:02.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.317+0000 7f14a3e00640 1 -- 172.21.15.167:0/4021401178 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3521174731 0 0) 0x7f14a40cf5c0 con 0x7f14a413fe30 2024-05-11T12:13:02.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.317+0000 7f14a3e00640 1 -- 172.21.15.167:0/4021401178 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1488003520 con 0x7f14a413fe30 2024-05-11T12:13:02.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.317+0000 7f14a9600640 1 -- 172.21.15.167:0/4021401178 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14a40d0660 msgr2=0x7f14a41609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:02.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.317+0000 7f14a9600640 1 --2- 172.21.15.167:0/4021401178 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14a40d0660 0x7f14a41609b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:02.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.317+0000 7f14a9600640 1 -- 172.21.15.167:0/4021401178 >> v1:172.21.15.167:6789/0 conn(0x7f14a413fe30 legacy=0x7f14a415de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:02.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.317+0000 7f14a9600640 1 -- 172.21.15.167:0/4021401178 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14a40cfe60 con 0x7f14a4160ef0 2024-05-11T12:13:02.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.318+0000 7f14a9600640 1 --2- 172.21.15.167:0/4021401178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14a4160ef0 0x7f14a406c450 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f149c009d90 tx=0x7f149c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6be2f0dad0db6737 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:02.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.319+0000 7f14a3e00640 1 -- 172.21.15.167:0/4021401178 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149c007740 con 0x7f14a4160ef0 2024-05-11T12:13:02.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.319+0000 7f14a3e00640 1 -- 172.21.15.167:0/4021401178 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f149c0078e0 con 0x7f14a4160ef0 2024-05-11T12:13:02.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.319+0000 7f14a3e00640 1 -- 172.21.15.167:0/4021401178 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149c007cb0 con 0x7f14a4160ef0 2024-05-11T12:13:02.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.319+0000 7f14aa000640 1 -- 172.21.15.167:0/4021401178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14a4160ef0 msgr2=0x7f14a406c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:02.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.319+0000 7f14aa000640 1 --2- 172.21.15.167:0/4021401178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14a4160ef0 0x7f14a406c450 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f149c009d90 tx=0x7f149c02a660 comp rx=0 tx=0).stop 2024-05-11T12:13:02.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.319+0000 7f14aa000640 1 -- 172.21.15.167:0/4021401178 shutdown_connections 2024-05-11T12:13:02.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.319+0000 7f14aa000640 1 --2- 172.21.15.167:0/4021401178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14a4160ef0 0x7f14a406c450 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:02.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.319+0000 7f14aa000640 1 --2- 172.21.15.167:0/4021401178 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14a40d0660 0x7f14a41609b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:02.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.319+0000 7f14aa000640 1 -- 172.21.15.167:0/4021401178 >> 172.21.15.167:0/4021401178 conn(0x7f14a40b2a20 msgr2=0x7f14a40b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:02.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.319+0000 7f14aa000640 1 -- 172.21.15.167:0/4021401178 shutdown_connections 2024-05-11T12:13:02.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.319+0000 7f14aa000640 1 -- 172.21.15.167:0/4021401178 wait complete. 2024-05-11T12:13:02.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.321+0000 7f14aa000640 1 Processor -- start 2024-05-11T12:13:02.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.321+0000 7f14aa000640 1 -- start start 2024-05-11T12:13:02.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.322+0000 7f14aa000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14a40d0660 0x7f14a41556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:02.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.322+0000 7f14aa000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14a413fe30 0x7f14a41560f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:02.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.322+0000 7f14aa000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14a4160ef0 0x7f14a4072f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:02.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.322+0000 7f14aa000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f14a4161820 con 0x7f14a413fe30 2024-05-11T12:13:02.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.322+0000 7f14aa000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f14a41616a0 con 0x7f14a40d0660 2024-05-11T12:13:02.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.322+0000 7f14aa000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f14a40cf190 con 0x7f14a4160ef0 2024-05-11T12:13:02.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.322+0000 7f149be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14a413fe30 0x7f14a41560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:02.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.322+0000 7f149be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14a413fe30 0x7f14a41560f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43258/0 (socket says 172.21.15.167:43258) 2024-05-11T12:13:02.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.322+0000 7f149be00640 1 -- 172.21.15.167:0/3674156191 learned_addr learned my addr 172.21.15.167:0/3674156191 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:02.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.323+0000 7f14a9600640 1 --2- 172.21.15.167:0/3674156191 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14a4160ef0 0x7f14a4072f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:02.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.323+0000 7f14a8c00640 1 --2- 172.21.15.167:0/3674156191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14a40d0660 0x7f14a41556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:02.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.323+0000 7f149be00640 1 -- 172.21.15.167:0/3674156191 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14a4160ef0 msgr2=0x7f14a4072f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:02.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.323+0000 7f149be00640 1 --2- 172.21.15.167:0/3674156191 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14a4160ef0 0x7f14a4072f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:02.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.323+0000 7f149be00640 1 -- 172.21.15.167:0/3674156191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14a40d0660 msgr2=0x7f14a41556b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:02.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.323+0000 7f149be00640 1 --2- 172.21.15.167:0/3674156191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14a40d0660 0x7f14a41556b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:02.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.323+0000 7f149be00640 1 -- 172.21.15.167:0/3674156191 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14a4159830 con 0x7f14a413fe30 2024-05-11T12:13:02.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.323+0000 7f149be00640 1 --2- 172.21.15.167:0/3674156191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14a413fe30 0x7f14a41560f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f148c00d5c0 tx=0x7f148c00da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:02.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.324+0000 7f14a1600640 1 -- 172.21.15.167:0/3674156191 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f148c0125f0 con 0x7f14a413fe30 2024-05-11T12:13:02.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.324+0000 7f14a1600640 1 -- 172.21.15.167:0/3674156191 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f148c019070 con 0x7f14a413fe30 2024-05-11T12:13:02.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.324+0000 7f14a1600640 1 -- 172.21.15.167:0/3674156191 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f148c0129d0 con 0x7f14a413fe30 2024-05-11T12:13:02.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.324+0000 7f14aa000640 1 -- 172.21.15.167:0/3674156191 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14a4159b10 con 0x7f14a413fe30 2024-05-11T12:13:02.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.324+0000 7f14aa000640 1 -- 172.21.15.167:0/3674156191 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14a415a0d0 con 0x7f14a413fe30 2024-05-11T12:13:02.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.325+0000 7f14aa000640 1 -- 172.21.15.167:0/3674156191 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14a4161490 con 0x7f14a413fe30 2024-05-11T12:13:02.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.326+0000 7f14a1600640 1 -- 172.21.15.167:0/3674156191 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f148c012b70 con 0x7f14a413fe30 2024-05-11T12:13:02.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.327+0000 7f14a1600640 1 --2- 172.21.15.167:0/3674156191 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f14800762d0 0x7f1480078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:02.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.327+0000 7f14a1600640 1 -- 172.21.15.167:0/3674156191 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6001+0+0 (secure 0 0 0) 0x7f148c097b90 con 0x7f14a413fe30 2024-05-11T12:13:02.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.328+0000 7f14a8c00640 1 --2- 172.21.15.167:0/3674156191 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f14800762d0 0x7f1480078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:02.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.328+0000 7f14a8c00640 1 --2- 172.21.15.167:0/3674156191 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f14800762d0 0x7f1480078790 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1494001220 tx=0x7f1494009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:02.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.333+0000 7f14a1600640 1 -- 172.21.15.167:0/3674156191 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f14a4161490 con 0x7f14a413fe30 2024-05-11T12:13:02.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:02.577+0000 7f14aa000640 1 -- 172.21.15.167:0/3674156191 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f14a40694b0 con 0x7f14a413fe30 2024-05-11T12:13:03.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.148+0000 7f14a1600640 1 -- 172.21.15.167:0/3674156191 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v31) ==== 173+0+0 (secure 0 0 0) 0x7f148c0613b0 con 0x7f14a413fe30 2024-05-11T12:13:03.155 INFO:teuthology.orchestra.run.smithi167.stderr:new fs with metadata pool 9 and data pool 10 2024-05-11T12:13:03.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.158+0000 7f14aa000640 1 -- 172.21.15.167:0/3674156191 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f14800762d0 msgr2=0x7f1480078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:03.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.158+0000 7f14aa000640 1 --2- 172.21.15.167:0/3674156191 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f14800762d0 0x7f1480078790 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1494001220 tx=0x7f1494009040 comp rx=0 tx=0).stop 2024-05-11T12:13:03.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.158+0000 7f14aa000640 1 -- 172.21.15.167:0/3674156191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14a413fe30 msgr2=0x7f14a41560f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:03.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.158+0000 7f14aa000640 1 --2- 172.21.15.167:0/3674156191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14a413fe30 0x7f14a41560f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f148c00d5c0 tx=0x7f148c00da80 comp rx=0 tx=0).stop 2024-05-11T12:13:03.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.161+0000 7f14aa000640 1 -- 172.21.15.167:0/3674156191 shutdown_connections 2024-05-11T12:13:03.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.161+0000 7f14aa000640 1 --2- 172.21.15.167:0/3674156191 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14a4160ef0 0x7f14a4072f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:03.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.161+0000 7f14aa000640 1 --2- 172.21.15.167:0/3674156191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14a413fe30 0x7f14a41560f0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:03.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.161+0000 7f14aa000640 1 --2- 172.21.15.167:0/3674156191 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f14800762d0 0x7f1480078790 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:03.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.161+0000 7f14aa000640 1 --2- 172.21.15.167:0/3674156191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14a40d0660 0x7f14a41556b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:03.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.161+0000 7f14aa000640 1 -- 172.21.15.167:0/3674156191 >> 172.21.15.167:0/3674156191 conn(0x7f14a40b2a20 msgr2=0x7f14a40ae820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:03.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.162+0000 7f14aa000640 1 -- 172.21.15.167:0/3674156191 shutdown_connections 2024-05-11T12:13:03.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.162+0000 7f14aa000640 1 -- 172.21.15.167:0/3674156191 wait complete. 2024-05-11T12:13:03.193 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:13:03.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.364+0000 7ff624a00640 1 Processor -- start 2024-05-11T12:13:03.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.364+0000 7ff624a00640 1 -- start start 2024-05-11T12:13:03.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.366+0000 7ff624a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff62013eeb0 0x7ff62006f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:03.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.366+0000 7ff624a00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff6200b50a0 0x7ff620073cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:03.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.366+0000 7ff624a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff62015fb90 con 0x7ff62013f290 2024-05-11T12:13:03.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.366+0000 7ff624a00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7ff62015d5b0 con 0x7ff6200b50a0 2024-05-11T12:13:03.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.366+0000 7ff624a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff62015fe70 con 0x7ff62013eeb0 2024-05-11T12:13:03.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.366+0000 7ff61e800640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff6200b50a0 0x7ff620073cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:03.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.366+0000 7ff61e800640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff6200b50a0 0x7ff620073cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43266/0 (socket says 172.21.15.167:43266) 2024-05-11T12:13:03.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.366+0000 7ff61e800640 1 -- 172.21.15.167:0/2624902035 learned_addr learned my addr 172.21.15.167:0/2624902035 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:03.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.367+0000 7ff61de00640 1 --2- 172.21.15.167:0/2624902035 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff62013eeb0 0x7ff62006f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:03.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.367+0000 7ff61e800640 1 -- 172.21.15.167:0/2624902035 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff62013eeb0 msgr2=0x7ff62006f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:03.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.367+0000 7ff61e800640 1 --2- 172.21.15.167:0/2624902035 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff62013eeb0 0x7ff62006f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:03.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.367+0000 7ff61e800640 1 -- 172.21.15.167:0/2624902035 >> v1:172.21.15.204:6789/0 conn(0x7ff62013f290 legacy=0x7ff6200735c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:03.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.367+0000 7ff61e800640 1 -- 172.21.15.167:0/2624902035 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff620160150 con 0x7ff6200b50a0 2024-05-11T12:13:03.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.368+0000 7ff61e800640 1 --2- 172.21.15.167:0/2624902035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6200b50a0 0x7ff620073cd0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7ff618009e50 tx=0x7ff61802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9e074b9369fc80a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:03.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.368+0000 7ff61d400640 1 -- 172.21.15.167:0/2624902035 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff618036070 con 0x7ff6200b50a0 2024-05-11T12:13:03.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.368+0000 7ff61d400640 1 -- 172.21.15.167:0/2624902035 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff618031600 con 0x7ff6200b50a0 2024-05-11T12:13:03.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.368+0000 7ff61d400640 1 -- 172.21.15.167:0/2624902035 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6180319d0 con 0x7ff6200b50a0 2024-05-11T12:13:03.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.368+0000 7ff624a00640 1 -- 172.21.15.167:0/2624902035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6200b50a0 msgr2=0x7ff620073cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:03.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.368+0000 7ff624a00640 1 --2- 172.21.15.167:0/2624902035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6200b50a0 0x7ff620073cd0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7ff618009e50 tx=0x7ff61802a5f0 comp rx=0 tx=0).stop 2024-05-11T12:13:03.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.368+0000 7ff624a00640 1 -- 172.21.15.167:0/2624902035 shutdown_connections 2024-05-11T12:13:03.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.368+0000 7ff624a00640 1 --2- 172.21.15.167:0/2624902035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6200b50a0 0x7ff620073cd0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:03.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.368+0000 7ff624a00640 1 --2- 172.21.15.167:0/2624902035 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff62013eeb0 0x7ff62006f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:03.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.369+0000 7ff624a00640 1 -- 172.21.15.167:0/2624902035 >> 172.21.15.167:0/2624902035 conn(0x7ff6200cd8f0 msgr2=0x7ff6200cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:03.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.369+0000 7ff624a00640 1 -- 172.21.15.167:0/2624902035 shutdown_connections 2024-05-11T12:13:03.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.369+0000 7ff624a00640 1 -- 172.21.15.167:0/2624902035 wait complete. 2024-05-11T12:13:03.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.370+0000 7ff624a00640 1 Processor -- start 2024-05-11T12:13:03.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.371+0000 7ff624a00640 1 -- start start 2024-05-11T12:13:03.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.371+0000 7ff624a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6200b50a0 0x7ff6201386e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:03.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.371+0000 7ff624a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff62013eeb0 0x7ff62015e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:03.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.371+0000 7ff624a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff62013f290 0x7ff62015edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:03.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.371+0000 7ff624a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff62006f9c0 con 0x7ff62013f290 2024-05-11T12:13:03.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.371+0000 7ff624a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff620160710 con 0x7ff6200b50a0 2024-05-11T12:13:03.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.371+0000 7ff624a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff62006fb40 con 0x7ff62013eeb0 2024-05-11T12:13:03.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.372+0000 7ff61e800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff62013f290 0x7ff62015edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:03.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.372+0000 7ff61e800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff62013f290 0x7ff62015edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43278/0 (socket says 172.21.15.167:43278) 2024-05-11T12:13:03.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.372+0000 7ff61e800640 1 -- 172.21.15.167:0/2809120955 learned_addr learned my addr 172.21.15.167:0/2809120955 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:03.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.372+0000 7ff61e800640 1 -- 172.21.15.167:0/2809120955 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff62013eeb0 msgr2=0x7ff62015e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:03.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.372+0000 7ff61de00640 1 --2- 172.21.15.167:0/2809120955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6200b50a0 0x7ff6201386e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:03.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.372+0000 7ff615400640 1 --2- 172.21.15.167:0/2809120955 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff62013eeb0 0x7ff62015e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:03.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.372+0000 7ff61e800640 1 --2- 172.21.15.167:0/2809120955 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff62013eeb0 0x7ff62015e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:03.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.372+0000 7ff61e800640 1 -- 172.21.15.167:0/2809120955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6200b50a0 msgr2=0x7ff6201386e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:03.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.372+0000 7ff61e800640 1 --2- 172.21.15.167:0/2809120955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6200b50a0 0x7ff6201386e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:03.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.372+0000 7ff61e800640 1 -- 172.21.15.167:0/2809120955 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff62013c4a0 con 0x7ff62013f290 2024-05-11T12:13:03.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.373+0000 7ff61e800640 1 --2- 172.21.15.167:0/2809120955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff62013f290 0x7ff62015edb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ff618009e50 tx=0x7ff618004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:03.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.373+0000 7ff616a00640 1 -- 172.21.15.167:0/2809120955 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff618036050 con 0x7ff62013f290 2024-05-11T12:13:03.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.373+0000 7ff616a00640 1 -- 172.21.15.167:0/2809120955 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff618042070 con 0x7ff62013f290 2024-05-11T12:13:03.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.373+0000 7ff616a00640 1 -- 172.21.15.167:0/2809120955 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff618009510 con 0x7ff62013f290 2024-05-11T12:13:03.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.373+0000 7ff624a00640 1 -- 172.21.15.167:0/2809120955 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff62013c780 con 0x7ff62013f290 2024-05-11T12:13:03.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.373+0000 7ff624a00640 1 -- 172.21.15.167:0/2809120955 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff62013cd40 con 0x7ff62013f290 2024-05-11T12:13:03.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.374+0000 7ff624a00640 1 -- 172.21.15.167:0/2809120955 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff620069590 con 0x7ff62013f290 2024-05-11T12:13:03.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.375+0000 7ff616a00640 1 -- 172.21.15.167:0/2809120955 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff618007bd0 con 0x7ff62013f290 2024-05-11T12:13:03.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.377+0000 7ff616a00640 1 --2- 172.21.15.167:0/2809120955 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff5fc0762d0 0x7ff5fc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:03.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.377+0000 7ff616a00640 1 -- 172.21.15.167:0/2809120955 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6101+0+0 (secure 0 0 0) 0x7ff6180b8770 con 0x7ff62013f290 2024-05-11T12:13:03.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.377+0000 7ff61de00640 1 --2- 172.21.15.167:0/2809120955 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff5fc0762d0 0x7ff5fc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:03.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.378+0000 7ff61de00640 1 --2- 172.21.15.167:0/2809120955 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff5fc0762d0 0x7ff5fc078790 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff61000b7c0 tx=0x7ff610008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:03.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.381+0000 7ff616a00640 1 -- 172.21.15.167:0/2809120955 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff618081ea0 con 0x7ff62013f290 2024-05-11T12:13:03.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.583+0000 7ff624a00640 1 -- 172.21.15.167:0/2809120955 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff62006d830 con 0x7ff62013f290 2024-05-11T12:13:03.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.586+0000 7ff616a00640 1 -- 172.21.15.167:0/2809120955 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+16945 (secure 0 0 0) 0x7ff6180817a0 con 0x7ff62013f290 2024-05-11T12:13:03.587 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:13:03.587 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":63,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:13:02.579951+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:13:00.503965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-11T12:13:01.566330+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:13:03.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.588+0000 7ff624a00640 1 -- 172.21.15.167:0/2809120955 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff5fc0762d0 msgr2=0x7ff5fc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:03.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.588+0000 7ff624a00640 1 --2- 172.21.15.167:0/2809120955 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff5fc0762d0 0x7ff5fc078790 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff61000b7c0 tx=0x7ff610008040 comp rx=0 tx=0).stop 2024-05-11T12:13:03.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.588+0000 7ff624a00640 1 -- 172.21.15.167:0/2809120955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff62013f290 msgr2=0x7ff62015edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:03.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.588+0000 7ff624a00640 1 --2- 172.21.15.167:0/2809120955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff62013f290 0x7ff62015edb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ff618009e50 tx=0x7ff618004a40 comp rx=0 tx=0).stop 2024-05-11T12:13:03.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.588+0000 7ff624a00640 1 -- 172.21.15.167:0/2809120955 shutdown_connections 2024-05-11T12:13:03.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.589+0000 7ff624a00640 1 --2- 172.21.15.167:0/2809120955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff62013f290 0x7ff62015edb0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:03.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.589+0000 7ff624a00640 1 --2- 172.21.15.167:0/2809120955 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff62013eeb0 0x7ff62015e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:03.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.589+0000 7ff624a00640 1 --2- 172.21.15.167:0/2809120955 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff5fc0762d0 0x7ff5fc078790 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:03.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.589+0000 7ff624a00640 1 --2- 172.21.15.167:0/2809120955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6200b50a0 0x7ff6201386e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:03.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.589+0000 7ff624a00640 1 -- 172.21.15.167:0/2809120955 >> 172.21.15.167:0/2809120955 conn(0x7ff6200cd8f0 msgr2=0x7ff62006c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:03.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.589+0000 7ff624a00640 1 -- 172.21.15.167:0/2809120955 shutdown_connections 2024-05-11T12:13:03.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.589+0000 7ff624a00640 1 -- 172.21.15.167:0/2809120955 wait complete. 2024-05-11T12:13:03.607 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:13:03.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.796+0000 7f1f96200640 1 Processor -- start 2024-05-11T12:13:03.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.796+0000 7f1f96200640 1 -- start start 2024-05-11T12:13:03.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.797+0000 7f1f96200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1f90157750 0x7f1f9015d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:03.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.797+0000 7f1f96200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1f9006d460 0x7f1f9006d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:03.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.797+0000 7f1f96200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f9015dc80 con 0x7f1f900b3ea0 2024-05-11T12:13:03.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.797+0000 7f1f96200640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f1f90161000 con 0x7f1f9006d460 2024-05-11T12:13:03.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.797+0000 7f1f96200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f1f9006ddd0 con 0x7f1f90157750 2024-05-11T12:13:03.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.798+0000 7f1f8fe00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1f9006d460 0x7f1f9006d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:03.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.798+0000 7f1f8fe00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1f9006d460 0x7f1f9006d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43288/0 (socket says 172.21.15.167:43288) 2024-05-11T12:13:03.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.798+0000 7f1f8fe00640 1 -- 172.21.15.167:0/2872701880 learned_addr learned my addr 172.21.15.167:0/2872701880 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:03.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.798+0000 7f1f8f400640 1 --2- 172.21.15.167:0/2872701880 >> v2:172.21.15.204:3300/0 conn(0x7f1f90157750 0x7f1f9015d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:03.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.798+0000 7f1f8fe00640 1 -- 172.21.15.167:0/2872701880 >> v2:172.21.15.204:3300/0 conn(0x7f1f90157750 msgr2=0x7f1f9015d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:03.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.798+0000 7f1f8fe00640 1 --2- 172.21.15.167:0/2872701880 >> v2:172.21.15.204:3300/0 conn(0x7f1f90157750 0x7f1f9015d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:03.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.798+0000 7f1f8fe00640 1 -- 172.21.15.167:0/2872701880 >> v1:172.21.15.204:6789/0 conn(0x7f1f900b3ea0 legacy=0x7f1f90161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:03.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.798+0000 7f1f8fe00640 1 -- 172.21.15.167:0/2872701880 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f9015df30 con 0x7f1f9006d460 2024-05-11T12:13:03.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.799+0000 7f1f8fe00640 1 --2- 172.21.15.167:0/2872701880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f9006d460 0x7f1f9006d890 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f1f900ce840 tx=0x7f1f8802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8383dd1c7e7c2fb server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:03.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.799+0000 7f1f8ea00640 1 -- 172.21.15.167:0/2872701880 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f88036070 con 0x7f1f9006d460 2024-05-11T12:13:03.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.800+0000 7f1f8ea00640 1 -- 172.21.15.167:0/2872701880 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f88031600 con 0x7f1f9006d460 2024-05-11T12:13:03.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.800+0000 7f1f8ea00640 1 -- 172.21.15.167:0/2872701880 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f880319d0 con 0x7f1f9006d460 2024-05-11T12:13:03.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.800+0000 7f1f96200640 1 -- 172.21.15.167:0/2872701880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f9006d460 msgr2=0x7f1f9006d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:03.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.800+0000 7f1f96200640 1 --2- 172.21.15.167:0/2872701880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f9006d460 0x7f1f9006d890 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f1f900ce840 tx=0x7f1f8802a6a0 comp rx=0 tx=0).stop 2024-05-11T12:13:03.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.800+0000 7f1f96200640 1 -- 172.21.15.167:0/2872701880 shutdown_connections 2024-05-11T12:13:03.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.800+0000 7f1f96200640 1 --2- 172.21.15.167:0/2872701880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f9006d460 0x7f1f9006d890 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:03.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.800+0000 7f1f96200640 1 --2- 172.21.15.167:0/2872701880 >> v2:172.21.15.204:3300/0 conn(0x7f1f90157750 0x7f1f9015d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:03.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.800+0000 7f1f96200640 1 -- 172.21.15.167:0/2872701880 >> 172.21.15.167:0/2872701880 conn(0x7f1f900cd8f0 msgr2=0x7f1f900cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:03.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.800+0000 7f1f96200640 1 -- 172.21.15.167:0/2872701880 shutdown_connections 2024-05-11T12:13:03.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.800+0000 7f1f96200640 1 -- 172.21.15.167:0/2872701880 wait complete. 2024-05-11T12:13:03.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.802+0000 7f1f96200640 1 Processor -- start 2024-05-11T12:13:03.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.802+0000 7f1f96200640 1 -- start start 2024-05-11T12:13:03.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.803+0000 7f1f96200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f9006d460 0x7f1f90141ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:03.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.803+0000 7f1f96200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f900b3ea0 0x7f1f9015c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:03.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.803+0000 7f1f96200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f90157750 0x7f1f9015ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:03.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.803+0000 7f1f96200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1f90157b30 con 0x7f1f900b3ea0 2024-05-11T12:13:03.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.803+0000 7f1f96200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9015e4f0 con 0x7f1f9006d460 2024-05-11T12:13:03.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.803+0000 7f1f96200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1f90161ba0 con 0x7f1f90157750 2024-05-11T12:13:03.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.803+0000 7f1f87e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f900b3ea0 0x7f1f9015c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:03.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.803+0000 7f1f87e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f900b3ea0 0x7f1f9015c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43298/0 (socket says 172.21.15.167:43298) 2024-05-11T12:13:03.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.803+0000 7f1f87e00640 1 -- 172.21.15.167:0/635696264 learned_addr learned my addr 172.21.15.167:0/635696264 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:03.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.803+0000 7f1f8f400640 1 --2- 172.21.15.167:0/635696264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f9006d460 0x7f1f90141ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:03.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.803+0000 7f1f8fe00640 1 --2- 172.21.15.167:0/635696264 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f90157750 0x7f1f9015ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:03.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.804+0000 7f1f87e00640 1 -- 172.21.15.167:0/635696264 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f90157750 msgr2=0x7f1f9015ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:03.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.804+0000 7f1f87e00640 1 --2- 172.21.15.167:0/635696264 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f90157750 0x7f1f9015ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:03.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.804+0000 7f1f87e00640 1 -- 172.21.15.167:0/635696264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f9006d460 msgr2=0x7f1f90141ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:03.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.804+0000 7f1f87e00640 1 --2- 172.21.15.167:0/635696264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f9006d460 0x7f1f90141ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:03.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.804+0000 7f1f87e00640 1 -- 172.21.15.167:0/635696264 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f9015b000 con 0x7f1f900b3ea0 2024-05-11T12:13:03.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.804+0000 7f1f8f400640 1 --2- 172.21.15.167:0/635696264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f9006d460 0x7f1f90141ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:03.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.804+0000 7f1f8fe00640 1 --2- 172.21.15.167:0/635696264 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f90157750 0x7f1f9015ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:03.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.804+0000 7f1f87e00640 1 --2- 172.21.15.167:0/635696264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f900b3ea0 0x7f1f9015c770 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f1f7800c440 tx=0x7f1f7800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:03.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.804+0000 7f1f87400640 1 -- 172.21.15.167:0/635696264 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f78017070 con 0x7f1f900b3ea0 2024-05-11T12:13:03.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.804+0000 7f1f87400640 1 -- 172.21.15.167:0/635696264 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f780035f0 con 0x7f1f900b3ea0 2024-05-11T12:13:03.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.804+0000 7f1f96200640 1 -- 172.21.15.167:0/635696264 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f90157f90 con 0x7f1f900b3ea0 2024-05-11T12:13:03.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.804+0000 7f1f87400640 1 -- 172.21.15.167:0/635696264 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f780039e0 con 0x7f1f900b3ea0 2024-05-11T12:13:03.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.804+0000 7f1f96200640 1 -- 172.21.15.167:0/635696264 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f90140980 con 0x7f1f900b3ea0 2024-05-11T12:13:03.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.805+0000 7f1f96200640 1 -- 172.21.15.167:0/635696264 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f90069590 con 0x7f1f900b3ea0 2024-05-11T12:13:03.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.806+0000 7f1f87400640 1 -- 172.21.15.167:0/635696264 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1f7801a070 con 0x7f1f900b3ea0 2024-05-11T12:13:03.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.807+0000 7f1f87400640 1 --2- 172.21.15.167:0/635696264 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f1f6c0762d0 0x7f1f6c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:03.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.807+0000 7f1f87400640 1 -- 172.21.15.167:0/635696264 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6101+0+0 (secure 0 0 0) 0x7f1f78096bf0 con 0x7f1f900b3ea0 2024-05-11T12:13:03.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.808+0000 7f1f8f400640 1 --2- 172.21.15.167:0/635696264 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f1f6c0762d0 0x7f1f6c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:03.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.809+0000 7f1f8f400640 1 --2- 172.21.15.167:0/635696264 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f1f6c0762d0 0x7f1f6c078790 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1f9006d050 tx=0x7f1f80008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:03.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:03.812+0000 7f1f87400640 1 -- 172.21.15.167:0/635696264 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1f78060320 con 0x7f1f900b3ea0 2024-05-11T12:13:04.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.002+0000 7f1f96200640 1 -- 172.21.15.167:0/635696264 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1f90075b40 con 0x7f1f900b3ea0 2024-05-11T12:13:04.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.005+0000 7f1f87400640 1 -- 172.21.15.167:0/635696264 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+16945 (secure 0 0 0) 0x7f1f780010c0 con 0x7f1f900b3ea0 2024-05-11T12:13:04.006 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:13:04.006 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":63,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:13:02.579951+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:13:00.503965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-11T12:13:01.566330+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:13:04.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.008+0000 7f1f96200640 1 -- 172.21.15.167:0/635696264 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f1f6c0762d0 msgr2=0x7f1f6c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:04.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.008+0000 7f1f96200640 1 --2- 172.21.15.167:0/635696264 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f1f6c0762d0 0x7f1f6c078790 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1f9006d050 tx=0x7f1f80008040 comp rx=0 tx=0).stop 2024-05-11T12:13:04.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.008+0000 7f1f96200640 1 -- 172.21.15.167:0/635696264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f900b3ea0 msgr2=0x7f1f9015c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:04.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.008+0000 7f1f96200640 1 --2- 172.21.15.167:0/635696264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f900b3ea0 0x7f1f9015c770 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f1f7800c440 tx=0x7f1f7800c900 comp rx=0 tx=0).stop 2024-05-11T12:13:04.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.008+0000 7f1f96200640 1 -- 172.21.15.167:0/635696264 shutdown_connections 2024-05-11T12:13:04.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.008+0000 7f1f96200640 1 --2- 172.21.15.167:0/635696264 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f90157750 0x7f1f9015ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:04.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.008+0000 7f1f96200640 1 --2- 172.21.15.167:0/635696264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f900b3ea0 0x7f1f9015c770 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:04.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.008+0000 7f1f96200640 1 --2- 172.21.15.167:0/635696264 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f1f6c0762d0 0x7f1f6c078790 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:04.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.008+0000 7f1f96200640 1 --2- 172.21.15.167:0/635696264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f9006d460 0x7f1f90141ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:04.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.008+0000 7f1f96200640 1 -- 172.21.15.167:0/635696264 >> 172.21.15.167:0/635696264 conn(0x7f1f900cd8f0 msgr2=0x7f1f900681c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:04.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.008+0000 7f1f96200640 1 -- 172.21.15.167:0/635696264 shutdown_connections 2024-05-11T12:13:04.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.008+0000 7f1f96200640 1 -- 172.21.15.167:0/635696264 wait complete. 2024-05-11T12:13:04.027 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-11T12:13:04.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.217+0000 7f1e5be00640 1 Processor -- start 2024-05-11T12:13:04.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.217+0000 7f1e5be00640 1 -- start start 2024-05-11T12:13:04.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.219+0000 7f1e5be00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1e5c0b6a50 0x7f1e5c15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:04.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.219+0000 7f1e5be00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1e5c15da00 0x7f1e5c0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:04.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.219+0000 7f1e5be00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e5c0cff30 con 0x7f1e5c1468b0 2024-05-11T12:13:04.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.219+0000 7f1e5be00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f1e5c146c90 con 0x7f1e5c0b6a50 2024-05-11T12:13:04.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.219+0000 7f1e5be00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1e5c0d0210 con 0x7f1e5c15da00 2024-05-11T12:13:04.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.220+0000 7f1e53e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1e5c0b6a50 0x7f1e5c15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:04.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.220+0000 7f1e53e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1e5c0b6a50 0x7f1e5c15d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43308/0 (socket says 172.21.15.167:43308) 2024-05-11T12:13:04.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.220+0000 7f1e53e00640 1 -- 172.21.15.167:0/1006903080 learned_addr learned my addr 172.21.15.167:0/1006903080 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:04.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.220+0000 7f1e5b400640 1 --2- 172.21.15.167:0/1006903080 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1e5c15da00 0x7f1e5c0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:04.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.220+0000 7f1e53e00640 1 -- 172.21.15.167:0/1006903080 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1e5c15da00 msgr2=0x7f1e5c0b4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:04.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.220+0000 7f1e53e00640 1 --2- 172.21.15.167:0/1006903080 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1e5c15da00 0x7f1e5c0b4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:04.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.220+0000 7f1e53e00640 1 -- 172.21.15.167:0/1006903080 >> v1:172.21.15.204:6789/0 conn(0x7f1e5c1468b0 legacy=0x7f1e5c0cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:04.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.220+0000 7f1e53e00640 1 -- 172.21.15.167:0/1006903080 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e5c0d04f0 con 0x7f1e5c0b6a50 2024-05-11T12:13:04.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.221+0000 7f1e53e00640 1 --2- 172.21.15.167:0/1006903080 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e5c0b6a50 0x7f1e5c15d4c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f1e44009870 tx=0x7f1e4402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd7975752c3a8822 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:04.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.221+0000 7f1e5a000640 1 -- 172.21.15.167:0/1006903080 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e44037070 con 0x7f1e5c0b6a50 2024-05-11T12:13:04.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.221+0000 7f1e5a000640 1 -- 172.21.15.167:0/1006903080 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e44032440 con 0x7f1e5c0b6a50 2024-05-11T12:13:04.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.221+0000 7f1e5a000640 1 -- 172.21.15.167:0/1006903080 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e44032810 con 0x7f1e5c0b6a50 2024-05-11T12:13:04.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.222+0000 7f1e5be00640 1 -- 172.21.15.167:0/1006903080 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e5c0b6a50 msgr2=0x7f1e5c15d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:04.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.222+0000 7f1e5be00640 1 --2- 172.21.15.167:0/1006903080 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e5c0b6a50 0x7f1e5c15d4c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f1e44009870 tx=0x7f1e4402a1c0 comp rx=0 tx=0).stop 2024-05-11T12:13:04.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.222+0000 7f1e5be00640 1 -- 172.21.15.167:0/1006903080 shutdown_connections 2024-05-11T12:13:04.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.222+0000 7f1e5be00640 1 --2- 172.21.15.167:0/1006903080 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1e5c15da00 0x7f1e5c0b4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:04.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.222+0000 7f1e5be00640 1 --2- 172.21.15.167:0/1006903080 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e5c0b6a50 0x7f1e5c15d4c0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:04.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.222+0000 7f1e5be00640 1 -- 172.21.15.167:0/1006903080 >> 172.21.15.167:0/1006903080 conn(0x7f1e5c0b2f10 msgr2=0x7f1e5c06c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:04.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.222+0000 7f1e5be00640 1 -- 172.21.15.167:0/1006903080 shutdown_connections 2024-05-11T12:13:04.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.222+0000 7f1e5be00640 1 -- 172.21.15.167:0/1006903080 wait complete. 2024-05-11T12:13:04.224 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.224+0000 7f1e5be00640 1 Processor -- start 2024-05-11T12:13:04.224 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.224+0000 7f1e5be00640 1 -- start start 2024-05-11T12:13:04.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.225+0000 7f1e5be00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1e5c0b6a50 0x7f1e5c1e80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:04.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.225+0000 7f1e5be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e5c1468b0 0x7f1e5c1e6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:04.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.225+0000 7f1e5be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e5c15da00 0x7f1e5c1f6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:04.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.225+0000 7f1e5be00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5c0b53b0 con 0x7f1e5c15da00 2024-05-11T12:13:04.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.225+0000 7f1e5be00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5c15de70 con 0x7f1e5c1468b0 2024-05-11T12:13:04.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.225+0000 7f1e5be00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1e5c0b5530 con 0x7f1e5c0b6a50 2024-05-11T12:13:04.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.225+0000 7f1e5b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e5c15da00 0x7f1e5c1f6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:04.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.225+0000 7f1e5b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e5c15da00 0x7f1e5c1f6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43314/0 (socket says 172.21.15.167:43314) 2024-05-11T12:13:04.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.225+0000 7f1e5b400640 1 -- 172.21.15.167:0/1027569386 learned_addr learned my addr 172.21.15.167:0/1027569386 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:04.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.226+0000 7f1e5aa00640 1 --2- 172.21.15.167:0/1027569386 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1e5c0b6a50 0x7f1e5c1e80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:04.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.226+0000 7f1e53e00640 1 --2- 172.21.15.167:0/1027569386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e5c1468b0 0x7f1e5c1e6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:04.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.226+0000 7f1e5b400640 1 -- 172.21.15.167:0/1027569386 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1e5c0b6a50 msgr2=0x7f1e5c1e80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:04.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.226+0000 7f1e5b400640 1 --2- 172.21.15.167:0/1027569386 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1e5c0b6a50 0x7f1e5c1e80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:04.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.226+0000 7f1e5b400640 1 -- 172.21.15.167:0/1027569386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e5c1468b0 msgr2=0x7f1e5c1e6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:04.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.226+0000 7f1e5b400640 1 --2- 172.21.15.167:0/1027569386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e5c1468b0 0x7f1e5c1e6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:04.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.226+0000 7f1e5b400640 1 -- 172.21.15.167:0/1027569386 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e5c1e7000 con 0x7f1e5c15da00 2024-05-11T12:13:04.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.226+0000 7f1e5b400640 1 --2- 172.21.15.167:0/1027569386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e5c15da00 0x7f1e5c1f6630 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f1e54007cf0 tx=0x7f1e5400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:04.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.226+0000 7f1e52a00640 1 -- 172.21.15.167:0/1027569386 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e54014420 con 0x7f1e5c15da00 2024-05-11T12:13:04.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.226+0000 7f1e52a00640 1 -- 172.21.15.167:0/1027569386 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e54019070 con 0x7f1e5c15da00 2024-05-11T12:13:04.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.226+0000 7f1e52a00640 1 -- 172.21.15.167:0/1027569386 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e54014820 con 0x7f1e5c15da00 2024-05-11T12:13:04.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.226+0000 7f1e5be00640 1 -- 172.21.15.167:0/1027569386 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e5c1e72e0 con 0x7f1e5c15da00 2024-05-11T12:13:04.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.226+0000 7f1e5be00640 1 -- 172.21.15.167:0/1027569386 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e5c1e78a0 con 0x7f1e5c15da00 2024-05-11T12:13:04.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.228+0000 7f1e5be00640 1 -- 172.21.15.167:0/1027569386 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e5c0694b0 con 0x7f1e5c15da00 2024-05-11T12:13:04.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.229+0000 7f1e52a00640 1 -- 172.21.15.167:0/1027569386 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1e540149c0 con 0x7f1e5c15da00 2024-05-11T12:13:04.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.230+0000 7f1e52a00640 1 --2- 172.21.15.167:0/1027569386 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f1e380762d0 0x7f1e38078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:04.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.230+0000 7f1e52a00640 1 -- 172.21.15.167:0/1027569386 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f1e54097ea0 con 0x7f1e5c15da00 2024-05-11T12:13:04.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.230+0000 7f1e5aa00640 1 --2- 172.21.15.167:0/1027569386 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f1e380762d0 0x7f1e38078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:04.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.230+0000 7f1e5aa00640 1 --2- 172.21.15.167:0/1027569386 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f1e380762d0 0x7f1e38078790 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c003d90 tx=0x7f1e4c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:04.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.234+0000 7f1e52a00640 1 -- 172.21.15.167:0/1027569386 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1e540610b0 con 0x7f1e5c15da00 2024-05-11T12:13:04.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:04.481+0000 7f1e5be00640 1 -- 172.21.15.167:0/1027569386 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f1e5c1e8640 con 0x7f1e5c15da00 2024-05-11T12:13:05.075 INFO:tasks.ceph.mds.c.smithi167.stderr:2024-05-11T12:13:05.075+0000 7fbbf1400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-11T12:13:05.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.169+0000 7f1e52a00640 1 -- 172.21.15.167:0/1027569386 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v34) ==== 118+0+0 (secure 0 0 0) 0x7f1e540653b0 con 0x7f1e5c15da00 2024-05-11T12:13:05.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.173+0000 7f1e5be00640 1 -- 172.21.15.167:0/1027569386 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f1e380762d0 msgr2=0x7f1e38078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:05.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.173+0000 7f1e5be00640 1 --2- 172.21.15.167:0/1027569386 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f1e380762d0 0x7f1e38078790 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c003d90 tx=0x7f1e4c009040 comp rx=0 tx=0).stop 2024-05-11T12:13:05.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.173+0000 7f1e5be00640 1 -- 172.21.15.167:0/1027569386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e5c15da00 msgr2=0x7f1e5c1f6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:05.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.173+0000 7f1e5be00640 1 --2- 172.21.15.167:0/1027569386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e5c15da00 0x7f1e5c1f6630 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f1e54007cf0 tx=0x7f1e5400e4b0 comp rx=0 tx=0).stop 2024-05-11T12:13:05.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.173+0000 7f1e5be00640 1 -- 172.21.15.167:0/1027569386 shutdown_connections 2024-05-11T12:13:05.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.173+0000 7f1e5be00640 1 --2- 172.21.15.167:0/1027569386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e5c15da00 0x7f1e5c1f6630 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:05.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.173+0000 7f1e5be00640 1 --2- 172.21.15.167:0/1027569386 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f1e380762d0 0x7f1e38078790 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:05.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.173+0000 7f1e5be00640 1 --2- 172.21.15.167:0/1027569386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e5c1468b0 0x7f1e5c1e6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:05.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.173+0000 7f1e5be00640 1 --2- 172.21.15.167:0/1027569386 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1e5c0b6a50 0x7f1e5c1e80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:05.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.173+0000 7f1e5be00640 1 -- 172.21.15.167:0/1027569386 >> 172.21.15.167:0/1027569386 conn(0x7f1e5c0b2f10 msgr2=0x7f1e5c06ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:05.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.174+0000 7f1e5be00640 1 -- 172.21.15.167:0/1027569386 shutdown_connections 2024-05-11T12:13:05.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.174+0000 7f1e5be00640 1 -- 172.21.15.167:0/1027569386 wait complete. 2024-05-11T12:13:05.194 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-11T12:13:05.194 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-11T12:13:05.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.390+0000 7f5382600640 1 Processor -- start 2024-05-11T12:13:05.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.390+0000 7f5382600640 1 -- start start 2024-05-11T12:13:05.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.391+0000 7f5382600640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f537c1468b0 0x7f537c0cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:05.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.391+0000 7f5382600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f537c0b6a50 0x7f537c15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:05.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.391+0000 7f5382600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f537c15da00 0x7f537c160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:05.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.391+0000 7f5382600640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f537c146c90 con 0x7f537c1468b0 2024-05-11T12:13:05.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.391+0000 7f5382600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f537c0cff30 con 0x7f537c0b6a50 2024-05-11T12:13:05.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.391+0000 7f5382600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f537c0d0210 con 0x7f537c15da00 2024-05-11T12:13:05.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.392+0000 7f5381200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f537c1468b0 0x7f537c0cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:05.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.392+0000 7f5381200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f537c1468b0 0x7f537c0cd580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43316/0 (socket says 172.21.15.167:43316) 2024-05-11T12:13:05.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.392+0000 7f5381200640 1 -- 172.21.15.167:0/2114806704 learned_addr learned my addr 172.21.15.167:0/2114806704 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:05.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.392+0000 7f5381200640 1 -- 172.21.15.167:0/2114806704 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f537c15da00 msgr2=0x7f537c160ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:05.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.392+0000 7f5381c00640 1 --2- 172.21.15.167:0/2114806704 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f537c15da00 0x7f537c160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:05.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.392+0000 7f537be00640 1 --2- 172.21.15.167:0/2114806704 >> v2:172.21.15.204:3300/0 conn(0x7f537c0b6a50 0x7f537c15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:05.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.392+0000 7f5381200640 1 --2- 172.21.15.167:0/2114806704 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f537c15da00 0x7f537c160ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:05.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.392+0000 7f5381200640 1 -- 172.21.15.167:0/2114806704 >> v2:172.21.15.204:3300/0 conn(0x7f537c0b6a50 msgr2=0x7f537c15d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:05.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.392+0000 7f5381200640 1 --2- 172.21.15.167:0/2114806704 >> v2:172.21.15.204:3300/0 conn(0x7f537c0b6a50 0x7f537c15d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:05.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.392+0000 7f5381200640 1 -- 172.21.15.167:0/2114806704 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f537c0d04f0 con 0x7f537c1468b0 2024-05-11T12:13:05.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.392+0000 7f537be00640 1 --2- 172.21.15.167:0/2114806704 >> v2:172.21.15.204:3300/0 conn(0x7f537c0b6a50 0x7f537c15d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:05.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.393+0000 7f5381200640 1 --2- 172.21.15.167:0/2114806704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f537c1468b0 0x7f537c0cd580 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f537c0b3530 tx=0x7f536c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8513888af686ed3a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:05.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.393+0000 7f537b400640 1 -- 172.21.15.167:0/2114806704 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c037070 con 0x7f537c1468b0 2024-05-11T12:13:05.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.394+0000 7f537b400640 1 -- 172.21.15.167:0/2114806704 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f536c032440 con 0x7f537c1468b0 2024-05-11T12:13:05.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.394+0000 7f537b400640 1 -- 172.21.15.167:0/2114806704 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c032810 con 0x7f537c1468b0 2024-05-11T12:13:05.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.394+0000 7f5382600640 1 -- 172.21.15.167:0/2114806704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f537c1468b0 msgr2=0x7f537c0cd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:05.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.394+0000 7f5382600640 1 --2- 172.21.15.167:0/2114806704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f537c1468b0 0x7f537c0cd580 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f537c0b3530 tx=0x7f536c02a1e0 comp rx=0 tx=0).stop 2024-05-11T12:13:05.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.394+0000 7f5382600640 1 -- 172.21.15.167:0/2114806704 shutdown_connections 2024-05-11T12:13:05.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.394+0000 7f5382600640 1 --2- 172.21.15.167:0/2114806704 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f537c15da00 0x7f537c160ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:05.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.394+0000 7f5382600640 1 --2- 172.21.15.167:0/2114806704 >> v2:172.21.15.204:3300/0 conn(0x7f537c0b6a50 0x7f537c15d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:05.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.394+0000 7f5382600640 1 --2- 172.21.15.167:0/2114806704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f537c1468b0 0x7f537c0cd580 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:05.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.394+0000 7f5382600640 1 -- 172.21.15.167:0/2114806704 >> 172.21.15.167:0/2114806704 conn(0x7f537c0b2f10 msgr2=0x7f537c06c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:05.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.394+0000 7f5382600640 1 -- 172.21.15.167:0/2114806704 shutdown_connections 2024-05-11T12:13:05.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.394+0000 7f5382600640 1 -- 172.21.15.167:0/2114806704 wait complete. 2024-05-11T12:13:05.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.396+0000 7f5382600640 1 Processor -- start 2024-05-11T12:13:05.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.396+0000 7f5382600640 1 -- start start 2024-05-11T12:13:05.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.397+0000 7f5382600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f537c0b6a50 0x7f537c138e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:05.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.397+0000 7f5382600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f537c1468b0 0x7f537c13ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:05.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.397+0000 7f5382600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f537c15da00 0x7f537c13bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:05.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.397+0000 7f5382600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f537c0cdac0 con 0x7f537c15da00 2024-05-11T12:13:05.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.397+0000 7f5382600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f537c15de70 con 0x7f537c0b6a50 2024-05-11T12:13:05.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.397+0000 7f5382600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f537c0cdc40 con 0x7f537c1468b0 2024-05-11T12:13:05.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.397+0000 7f5381c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f537c15da00 0x7f537c13bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:05.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.397+0000 7f5381c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f537c15da00 0x7f537c13bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43318/0 (socket says 172.21.15.167:43318) 2024-05-11T12:13:05.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.397+0000 7f5381c00640 1 -- 172.21.15.167:0/955470499 learned_addr learned my addr 172.21.15.167:0/955470499 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:05.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.398+0000 7f5381200640 1 --2- 172.21.15.167:0/955470499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f537c0b6a50 0x7f537c138e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:05.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.398+0000 7f537be00640 1 --2- 172.21.15.167:0/955470499 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f537c1468b0 0x7f537c13ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:05.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.398+0000 7f5381c00640 1 -- 172.21.15.167:0/955470499 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f537c1468b0 msgr2=0x7f537c13ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:05.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.398+0000 7f5381c00640 1 --2- 172.21.15.167:0/955470499 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f537c1468b0 0x7f537c13ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:05.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.398+0000 7f5381c00640 1 -- 172.21.15.167:0/955470499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f537c0b6a50 msgr2=0x7f537c138e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:05.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.398+0000 7f5381c00640 1 --2- 172.21.15.167:0/955470499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f537c0b6a50 0x7f537c138e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:05.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.398+0000 7f5381c00640 1 -- 172.21.15.167:0/955470499 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f537c1437a0 con 0x7f537c15da00 2024-05-11T12:13:05.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.398+0000 7f537be00640 1 --2- 172.21.15.167:0/955470499 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f537c1468b0 0x7f537c13ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:05.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.398+0000 7f5381c00640 1 --2- 172.21.15.167:0/955470499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f537c15da00 0x7f537c13bd00 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f537400cc80 tx=0x7f537400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:05.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.398+0000 7f5378c00640 1 -- 172.21.15.167:0/955470499 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f537400ce30 con 0x7f537c15da00 2024-05-11T12:13:05.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.398+0000 7f5378c00640 1 -- 172.21.15.167:0/955470499 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5374003990 con 0x7f537c15da00 2024-05-11T12:13:05.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.398+0000 7f5378c00640 1 -- 172.21.15.167:0/955470499 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5374003d60 con 0x7f537c15da00 2024-05-11T12:13:05.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.398+0000 7f5382600640 1 -- 172.21.15.167:0/955470499 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f537c143a80 con 0x7f537c15da00 2024-05-11T12:13:05.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.399+0000 7f5382600640 1 -- 172.21.15.167:0/955470499 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f537c144040 con 0x7f537c15da00 2024-05-11T12:13:05.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.399+0000 7f5382600640 1 -- 172.21.15.167:0/955470499 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f537c0694b0 con 0x7f537c15da00 2024-05-11T12:13:05.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.401+0000 7f5378c00640 1 -- 172.21.15.167:0/955470499 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5374003f00 con 0x7f537c15da00 2024-05-11T12:13:05.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.402+0000 7f5378c00640 1 --2- 172.21.15.167:0/955470499 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f53600762d0 0x7f5360078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:05.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.402+0000 7f5378c00640 1 -- 172.21.15.167:0/955470499 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f5374097c30 con 0x7f537c15da00 2024-05-11T12:13:05.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.402+0000 7f5381200640 1 --2- 172.21.15.167:0/955470499 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f53600762d0 0x7f5360078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:05.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.403+0000 7f5381200640 1 --2- 172.21.15.167:0/955470499 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f53600762d0 0x7f5360078790 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f536c034000 tx=0x7f536c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:05.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.407+0000 7f5378c00640 1 -- 172.21.15.167:0/955470499 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5374014030 con 0x7f537c15da00 2024-05-11T12:13:05.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:05.650+0000 7f5382600640 1 -- 172.21.15.167:0/955470499 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f537c0b5460 con 0x7f537c15da00 2024-05-11T12:13:06.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.180+0000 7f5378c00640 1 -- 172.21.15.167:0/955470499 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v35) ==== 122+0+0 (secure 0 0 0) 0x7f5374061360 con 0x7f537c15da00 2024-05-11T12:13:06.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.183+0000 7f5382600640 1 -- 172.21.15.167:0/955470499 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f53600762d0 msgr2=0x7f5360078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:06.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.183+0000 7f5382600640 1 --2- 172.21.15.167:0/955470499 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f53600762d0 0x7f5360078790 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f536c034000 tx=0x7f536c02e000 comp rx=0 tx=0).stop 2024-05-11T12:13:06.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.183+0000 7f5382600640 1 -- 172.21.15.167:0/955470499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f537c15da00 msgr2=0x7f537c13bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:06.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.183+0000 7f5382600640 1 --2- 172.21.15.167:0/955470499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f537c15da00 0x7f537c13bd00 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f537400cc80 tx=0x7f537400a5c0 comp rx=0 tx=0).stop 2024-05-11T12:13:06.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.184+0000 7f5382600640 1 -- 172.21.15.167:0/955470499 shutdown_connections 2024-05-11T12:13:06.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.184+0000 7f5382600640 1 --2- 172.21.15.167:0/955470499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f537c15da00 0x7f537c13bd00 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:06.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.184+0000 7f5382600640 1 --2- 172.21.15.167:0/955470499 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f537c1468b0 0x7f537c13ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:06.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.184+0000 7f5382600640 1 --2- 172.21.15.167:0/955470499 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f53600762d0 0x7f5360078790 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:06.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.184+0000 7f5382600640 1 --2- 172.21.15.167:0/955470499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f537c0b6a50 0x7f537c138e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:06.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.184+0000 7f5382600640 1 -- 172.21.15.167:0/955470499 >> 172.21.15.167:0/955470499 conn(0x7f537c0b2f10 msgr2=0x7f537c15efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:06.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.184+0000 7f5382600640 1 -- 172.21.15.167:0/955470499 shutdown_connections 2024-05-11T12:13:06.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.184+0000 7f5382600640 1 -- 172.21.15.167:0/955470499 wait complete. 2024-05-11T12:13:06.207 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-11T12:13:06.207 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:13:06.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.406+0000 7fd053200640 1 Processor -- start 2024-05-11T12:13:06.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.406+0000 7fd053200640 1 -- start start 2024-05-11T12:13:06.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.407+0000 7fd053200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd054145b00 0x7fd05406f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:06.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.407+0000 7fd053200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd054146770 0x7fd0540ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:06.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.407+0000 7fd053200640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd05406fbc0 con 0x7fd0540cef10 2024-05-11T12:13:06.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.407+0000 7fd053200640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fd0540b4520 con 0x7fd054145b00 2024-05-11T12:13:06.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.407+0000 7fd053200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fd0540b4800 con 0x7fd054146770 2024-05-11T12:13:06.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.407+0000 7fd051e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd054145b00 0x7fd05406f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:06.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.408+0000 7fd052800640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fd0540cef10 0x7fd05406c660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:33310/0 (socket says 172.21.15.167:33310) 2024-05-11T12:13:06.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.408+0000 7fd052800640 1 -- 172.21.15.167:0/725116555 learned_addr learned my addr 172.21.15.167:0/725116555 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:13:06.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.408+0000 7fd051e00640 1 --2- 172.21.15.167:0/725116555 >> v2:172.21.15.167:3300/0 conn(0x7fd054145b00 0x7fd05406f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43322/0 (socket says 172.21.15.167:43322) 2024-05-11T12:13:06.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.408+0000 7fd04be00640 1 --2- 172.21.15.167:0/725116555 >> v2:172.21.15.204:3300/0 conn(0x7fd054146770 0x7fd0540ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:06.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.408+0000 7fd051e00640 1 -- 172.21.15.167:0/725116555 >> v2:172.21.15.204:3300/0 conn(0x7fd054146770 msgr2=0x7fd0540ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:06.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.408+0000 7fd051e00640 1 --2- 172.21.15.167:0/725116555 >> v2:172.21.15.204:3300/0 conn(0x7fd054146770 0x7fd0540ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:06.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.408+0000 7fd051e00640 1 -- 172.21.15.167:0/725116555 >> v1:172.21.15.167:6789/0 conn(0x7fd0540cef10 legacy=0x7fd05406c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:06.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.408+0000 7fd051e00640 1 -- 172.21.15.167:0/725116555 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0540b4ae0 con 0x7fd054145b00 2024-05-11T12:13:06.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.408+0000 7fd051400640 1 -- 172.21.15.167:0/725116555 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3660272103 0 0) 0x7fd05406fbc0 con 0x7fd0540cef10 2024-05-11T12:13:06.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.409+0000 7fd051e00640 1 --2- 172.21.15.167:0/725116555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd054145b00 0x7fd05406f2c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd054146d70 tx=0x7fd04402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a330e7cb32e4f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:06.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.409+0000 7fd051400640 1 -- 172.21.15.167:0/725116555 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd044037070 con 0x7fd054145b00 2024-05-11T12:13:06.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.409+0000 7fd051400640 1 -- 172.21.15.167:0/725116555 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd044032440 con 0x7fd054145b00 2024-05-11T12:13:06.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.409+0000 7fd051400640 1 -- 172.21.15.167:0/725116555 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04402ae60 con 0x7fd054145b00 2024-05-11T12:13:06.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.410+0000 7fd053200640 1 -- 172.21.15.167:0/725116555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd054145b00 msgr2=0x7fd05406f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:06.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.410+0000 7fd053200640 1 --2- 172.21.15.167:0/725116555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd054145b00 0x7fd05406f2c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd054146d70 tx=0x7fd04402a110 comp rx=0 tx=0).stop 2024-05-11T12:13:06.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.410+0000 7fd053200640 1 -- 172.21.15.167:0/725116555 shutdown_connections 2024-05-11T12:13:06.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.410+0000 7fd053200640 1 --2- 172.21.15.167:0/725116555 >> v2:172.21.15.204:3300/0 conn(0x7fd054146770 0x7fd0540ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:06.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.410+0000 7fd053200640 1 --2- 172.21.15.167:0/725116555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd054145b00 0x7fd05406f2c0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:06.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.410+0000 7fd053200640 1 -- 172.21.15.167:0/725116555 >> 172.21.15.167:0/725116555 conn(0x7fd0540b3460 msgr2=0x7fd0540d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:06.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.410+0000 7fd053200640 1 -- 172.21.15.167:0/725116555 shutdown_connections 2024-05-11T12:13:06.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.410+0000 7fd053200640 1 -- 172.21.15.167:0/725116555 wait complete. 2024-05-11T12:13:06.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.412+0000 7fd053200640 1 Processor -- start 2024-05-11T12:13:06.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.412+0000 7fd053200640 1 -- start start 2024-05-11T12:13:06.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.413+0000 7fd053200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0540cef10 0x7fd05413bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:06.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.413+0000 7fd053200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd054145b00 0x7fd05415c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:06.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.413+0000 7fd053200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd054146770 0x7fd05413afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:06.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.413+0000 7fd053200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd05406f980 con 0x7fd0540cef10 2024-05-11T12:13:06.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.413+0000 7fd053200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd05406f800 con 0x7fd054146770 2024-05-11T12:13:06.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.413+0000 7fd053200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd05406cd70 con 0x7fd054145b00 2024-05-11T12:13:06.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.413+0000 7fd051e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0540cef10 0x7fd05413bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:06.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.413+0000 7fd051e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0540cef10 0x7fd05413bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43332/0 (socket says 172.21.15.167:43332) 2024-05-11T12:13:06.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.413+0000 7fd051e00640 1 -- 172.21.15.167:0/793021986 learned_addr learned my addr 172.21.15.167:0/793021986 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:06.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.414+0000 7fd051e00640 1 -- 172.21.15.167:0/793021986 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd054145b00 msgr2=0x7fd05415c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:06.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.414+0000 7fd051e00640 1 --2- 172.21.15.167:0/793021986 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd054145b00 0x7fd05415c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:06.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.414+0000 7fd051e00640 1 -- 172.21.15.167:0/793021986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd054146770 msgr2=0x7fd05413afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:06.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.414+0000 7fd051e00640 1 --2- 172.21.15.167:0/793021986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd054146770 0x7fd05413afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:06.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.414+0000 7fd051e00640 1 -- 172.21.15.167:0/793021986 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd05415db20 con 0x7fd0540cef10 2024-05-11T12:13:06.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.414+0000 7fd051e00640 1 --2- 172.21.15.167:0/793021986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0540cef10 0x7fd05413bda0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fd044000c00 tx=0x7fd0440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:06.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.414+0000 7fd04a000640 1 -- 172.21.15.167:0/793021986 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd044037040 con 0x7fd0540cef10 2024-05-11T12:13:06.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.414+0000 7fd04a000640 1 -- 172.21.15.167:0/793021986 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd044042070 con 0x7fd0540cef10 2024-05-11T12:13:06.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.414+0000 7fd053200640 1 -- 172.21.15.167:0/793021986 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd05415de00 con 0x7fd0540cef10 2024-05-11T12:13:06.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.414+0000 7fd04a000640 1 -- 172.21.15.167:0/793021986 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd044030bb0 con 0x7fd0540cef10 2024-05-11T12:13:06.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.415+0000 7fd053200640 1 -- 172.21.15.167:0/793021986 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd05415e3c0 con 0x7fd0540cef10 2024-05-11T12:13:06.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.416+0000 7fd053200640 1 -- 172.21.15.167:0/793021986 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd05415e6f0 con 0x7fd0540cef10 2024-05-11T12:13:06.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.417+0000 7fd04a000640 1 -- 172.21.15.167:0/793021986 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd044047460 con 0x7fd0540cef10 2024-05-11T12:13:06.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.418+0000 7fd04a000640 1 --2- 172.21.15.167:0/793021986 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd0300762d0 0x7fd030078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:06.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.418+0000 7fd04a000640 1 -- 172.21.15.167:0/793021986 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7fd0440b8410 con 0x7fd0540cef10 2024-05-11T12:13:06.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.418+0000 7fd04be00640 1 --2- 172.21.15.167:0/793021986 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd0300762d0 0x7fd030078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:06.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.419+0000 7fd04be00640 1 --2- 172.21.15.167:0/793021986 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd0300762d0 0x7fd030078790 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd03c006e80 tx=0x7fd03c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:06.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.423+0000 7fd04a000640 1 -- 172.21.15.167:0/793021986 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd044032e20 con 0x7fd0540cef10 2024-05-11T12:13:06.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.672+0000 7fd053200640 1 -- 172.21.15.167:0/793021986 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0540d1000 con 0x7fd0540cef10 2024-05-11T12:13:06.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.673+0000 7fd04a000640 1 -- 172.21.15.167:0/793021986 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5185 (secure 0 0 0) 0x7fd0540d1000 con 0x7fd0540cef10 2024-05-11T12:13:06.673 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:13:06.674 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/2785478956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2785478956},{"type":"v1","addr":"172.21.15.204:6834","nonce":2785478956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":4742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/3478043182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":3478043182},{"type":"v1","addr":"172.21.15.204:6835","nonce":3478043182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/618833945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":618833945},{"type":"v1","addr":"172.21.15.167:6836","nonce":618833945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:13:02.579882+0000","modified":"2024-05-11T12:13:05.652972+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5610":{"gid":5610,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/42694429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":42694429},{"type":"v1","addr":"172.21.15.167:6837","nonce":42694429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5610,"qdb_cluster":[5610]},"id":4}]} 2024-05-11T12:13:06.674 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 35 2024-05-11T12:13:06.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.676+0000 7fd053200640 1 -- 172.21.15.167:0/793021986 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd0300762d0 msgr2=0x7fd030078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:06.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.676+0000 7fd053200640 1 --2- 172.21.15.167:0/793021986 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd0300762d0 0x7fd030078790 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd03c006e80 tx=0x7fd03c008040 comp rx=0 tx=0).stop 2024-05-11T12:13:06.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.676+0000 7fd053200640 1 -- 172.21.15.167:0/793021986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0540cef10 msgr2=0x7fd05413bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:06.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.676+0000 7fd053200640 1 --2- 172.21.15.167:0/793021986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0540cef10 0x7fd05413bda0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fd044000c00 tx=0x7fd0440046c0 comp rx=0 tx=0).stop 2024-05-11T12:13:06.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.676+0000 7fd053200640 1 -- 172.21.15.167:0/793021986 shutdown_connections 2024-05-11T12:13:06.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.676+0000 7fd053200640 1 --2- 172.21.15.167:0/793021986 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd0300762d0 0x7fd030078790 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:06.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.676+0000 7fd053200640 1 --2- 172.21.15.167:0/793021986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd054146770 0x7fd05413afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:06.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.676+0000 7fd053200640 1 --2- 172.21.15.167:0/793021986 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd054145b00 0x7fd05415c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:06.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.676+0000 7fd053200640 1 --2- 172.21.15.167:0/793021986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0540cef10 0x7fd05413bda0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:06.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.676+0000 7fd053200640 1 -- 172.21.15.167:0/793021986 >> 172.21.15.167:0/793021986 conn(0x7fd0540b3460 msgr2=0x7fd0540cd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:06.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.677+0000 7fd053200640 1 -- 172.21.15.167:0/793021986 shutdown_connections 2024-05-11T12:13:06.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.677+0000 7fd053200640 1 -- 172.21.15.167:0/793021986 wait complete. 2024-05-11T12:13:06.697 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:13:06.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.897+0000 7f34d3e00640 1 Processor -- start 2024-05-11T12:13:06.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.897+0000 7f34d3e00640 1 -- start start 2024-05-11T12:13:06.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.899+0000 7f34d3e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34d406c660 0x7f34d406ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:06.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.899+0000 7f34d3e00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34d406cfb0 con 0x7f34d4145b00 2024-05-11T12:13:06.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.899+0000 7f34d3e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34d406fbc0 con 0x7f34d4146770 2024-05-11T12:13:06.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.899+0000 7f34d3e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f34d40b4520 con 0x7f34d406c660 2024-05-11T12:13:06.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.899+0000 7f34d2a00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f34d4145b00 0x7f34d406f2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:33312/0 (socket says 172.21.15.167:33312) 2024-05-11T12:13:06.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.899+0000 7f34d2a00640 1 -- 172.21.15.167:0/2669063813 learned_addr learned my addr 172.21.15.167:0/2669063813 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:13:06.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.900+0000 7f34d1600640 1 -- 172.21.15.167:0/2669063813 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4200914753 0 0) 0x7f34c4001110 con 0x7f34d4145b00 2024-05-11T12:13:06.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.900+0000 7f34d1600640 1 -- 172.21.15.167:0/2669063813 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f34b8003520 con 0x7f34d4145b00 2024-05-11T12:13:06.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.900+0000 7f34d3400640 1 --2- 172.21.15.167:0/2669063813 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34d406c660 0x7f34d406ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:06.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.900+0000 7f34d1600640 1 -- 172.21.15.167:0/2669063813 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2369841037 0 0) 0x7f34b8003520 con 0x7f34d4145b00 2024-05-11T12:13:06.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.900+0000 7f34d1600640 1 -- 172.21.15.167:0/2669063813 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34d406c660 msgr2=0x7f34d406ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:06.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.900+0000 7f34d1600640 1 --2- 172.21.15.167:0/2669063813 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34d406c660 0x7f34d406ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:06.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.900+0000 7f34d1600640 1 -- 172.21.15.167:0/2669063813 >> v1:172.21.15.204:6789/0 conn(0x7f34d4146770 legacy=0x7f34d40ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:06.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.900+0000 7f34d1600640 1 -- 172.21.15.167:0/2669063813 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34d40b4800 con 0x7f34d4145b00 2024-05-11T12:13:06.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.900+0000 7f34d1600640 1 -- 172.21.15.167:0/2669063813 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f34c40018c0 con 0x7f34d4145b00 2024-05-11T12:13:06.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.900+0000 7f34d1600640 1 -- 172.21.15.167:0/2669063813 >> v1:172.21.15.167:6789/0 conn(0x7f34d4145b00 legacy=0x7f34d406f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:06.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.900+0000 7f34d1600640 1 --2- 172.21.15.167:0/2669063813 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34d406c660 0x7f34b8003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:06.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.900+0000 7f34d1600640 1 --2- 172.21.15.167:0/2669063813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f34b80040b0 0x7f34b8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:06.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.900+0000 7f34d1600640 1 --2- 172.21.15.167:0/2669063813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34b8004d90 0x7f34b8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:06.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.900+0000 7f34d1600640 1 -- 172.21.15.167:0/2669063813 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f34d40b4520 con 0x7f34b80040b0 2024-05-11T12:13:06.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.900+0000 7f34d1600640 1 -- 172.21.15.167:0/2669063813 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f34b800bca0 con 0x7f34b8004d90 2024-05-11T12:13:06.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.900+0000 7f34d1600640 1 -- 172.21.15.167:0/2669063813 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f34b800bea0 con 0x7f34d406c660 2024-05-11T12:13:06.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.901+0000 7f34d2000640 1 --2- 172.21.15.167:0/2669063813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f34b80040b0 0x7f34b8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:06.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.901+0000 7f34d3400640 1 --2- 172.21.15.167:0/2669063813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34b8004d90 0x7f34b8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:06.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.901+0000 7f34d2a00640 1 --2- 172.21.15.167:0/2669063813 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34d406c660 0x7f34b8003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:06.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.901+0000 7f34d2000640 1 -- 172.21.15.167:0/2669063813 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34d406c660 msgr2=0x7f34b8003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:06.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.901+0000 7f34d2000640 1 --2- 172.21.15.167:0/2669063813 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34d406c660 0x7f34b8003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:06.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.901+0000 7f34d2000640 1 -- 172.21.15.167:0/2669063813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34b8004d90 msgr2=0x7f34b8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:06.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.901+0000 7f34d2000640 1 --2- 172.21.15.167:0/2669063813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34b8004d90 0x7f34b8005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:06.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.901+0000 7f34d2000640 1 -- 172.21.15.167:0/2669063813 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b8003520 con 0x7f34b80040b0 2024-05-11T12:13:06.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.901+0000 7f34d2a00640 1 --2- 172.21.15.167:0/2669063813 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34d406c660 0x7f34b8003a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:06.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.902+0000 7f34d2000640 1 --2- 172.21.15.167:0/2669063813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f34b80040b0 0x7f34b8004800 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f34c0001760 tx=0x7f34b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3e1bd60a32c78cf server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:06.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.902+0000 7f34d1600640 1 -- 172.21.15.167:0/2669063813 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c00013b0 con 0x7f34b80040b0 2024-05-11T12:13:06.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.902+0000 7f34d1600640 1 -- 172.21.15.167:0/2669063813 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34c00052d0 con 0x7f34b80040b0 2024-05-11T12:13:06.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.902+0000 7f34d1600640 1 -- 172.21.15.167:0/2669063813 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c002aaa0 con 0x7f34b80040b0 2024-05-11T12:13:06.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.903+0000 7f34d3e00640 1 -- 172.21.15.167:0/2669063813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f34b80040b0 msgr2=0x7f34b8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:06.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.903+0000 7f34d3e00640 1 --2- 172.21.15.167:0/2669063813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f34b80040b0 0x7f34b8004800 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f34c0001760 tx=0x7f34b8005830 comp rx=0 tx=0).stop 2024-05-11T12:13:06.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.903+0000 7f34d3400640 1 -- 172.21.15.167:0/2669063813 reap_dead start 2024-05-11T12:13:06.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.903+0000 7f34d3e00640 1 -- 172.21.15.167:0/2669063813 shutdown_connections 2024-05-11T12:13:06.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.903+0000 7f34d3e00640 1 -- 172.21.15.167:0/2669063813 >> 172.21.15.167:0/2669063813 conn(0x7f34d40b3460 msgr2=0x7f34d40d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:06.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.903+0000 7f34d3e00640 1 -- 172.21.15.167:0/2669063813 shutdown_connections 2024-05-11T12:13:06.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.903+0000 7f34d3e00640 1 -- 172.21.15.167:0/2669063813 wait complete. 2024-05-11T12:13:06.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.905+0000 7f34d3e00640 1 Processor -- start 2024-05-11T12:13:06.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.905+0000 7f34d3e00640 1 -- start start 2024-05-11T12:13:06.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.906+0000 7f34d3e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34b80040b0 0x7f34d413de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:06.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.906+0000 7f34d3e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f34d415c570 0x7f34d41f8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:06.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.906+0000 7f34d3e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34d415bf70 0x7f34d41f8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:06.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.906+0000 7f34d3e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f34d4145ee0 con 0x7f34d415c570 2024-05-11T12:13:06.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.906+0000 7f34d3e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f34d40cf0e0 con 0x7f34b80040b0 2024-05-11T12:13:06.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.906+0000 7f34d3e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f34d406ca70 con 0x7f34d415bf70 2024-05-11T12:13:06.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.906+0000 7f34d2000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f34d415c570 0x7f34d41f8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:06.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.907+0000 7f34d2000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f34d415c570 0x7f34d41f8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43350/0 (socket says 172.21.15.167:43350) 2024-05-11T12:13:06.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.907+0000 7f34d2000640 1 -- 172.21.15.167:0/2296326542 learned_addr learned my addr 172.21.15.167:0/2296326542 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:06.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.907+0000 7f34d2000640 1 -- 172.21.15.167:0/2296326542 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34d415bf70 msgr2=0x7f34d41f8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:06.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.907+0000 7f34d2000640 1 --2- 172.21.15.167:0/2296326542 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34d415bf70 0x7f34d41f8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:06.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.907+0000 7f34d2000640 1 -- 172.21.15.167:0/2296326542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34b80040b0 msgr2=0x7f34d413de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:06.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.907+0000 7f34d2000640 1 --2- 172.21.15.167:0/2296326542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34b80040b0 0x7f34d413de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:06.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.907+0000 7f34d2000640 1 -- 172.21.15.167:0/2296326542 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34d413e6e0 con 0x7f34d415c570 2024-05-11T12:13:06.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.907+0000 7f34d2000640 1 --2- 172.21.15.167:0/2296326542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f34d415c570 0x7f34d41f8950 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f34c0005040 tx=0x7f34c0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:06.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.907+0000 7f34caa00640 1 -- 172.21.15.167:0/2296326542 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c0042070 con 0x7f34d415c570 2024-05-11T12:13:06.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.908+0000 7f34caa00640 1 -- 172.21.15.167:0/2296326542 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34c0005480 con 0x7f34d415c570 2024-05-11T12:13:06.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.908+0000 7f34caa00640 1 -- 172.21.15.167:0/2296326542 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c0037040 con 0x7f34d415c570 2024-05-11T12:13:06.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.908+0000 7f34d3e00640 1 -- 172.21.15.167:0/2296326542 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34d413e9c0 con 0x7f34d415c570 2024-05-11T12:13:06.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.908+0000 7f34d3e00640 1 -- 172.21.15.167:0/2296326542 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34d413e350 con 0x7f34d415c570 2024-05-11T12:13:06.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.909+0000 7f34d3e00640 1 -- 172.21.15.167:0/2296326542 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34d40b0500 con 0x7f34d415c570 2024-05-11T12:13:06.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.910+0000 7f34caa00640 1 -- 172.21.15.167:0/2296326542 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f34c0045070 con 0x7f34d415c570 2024-05-11T12:13:06.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.911+0000 7f34caa00640 1 --2- 172.21.15.167:0/2296326542 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f34b4076300 0x7f34b40787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:06.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.911+0000 7f34caa00640 1 -- 172.21.15.167:0/2296326542 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f34c00b6d50 con 0x7f34d415c570 2024-05-11T12:13:06.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.912+0000 7f34d2a00640 1 --2- 172.21.15.167:0/2296326542 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f34b4076300 0x7f34b40787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:06.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.912+0000 7f34d2a00640 1 --2- 172.21.15.167:0/2296326542 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f34b4076300 0x7f34b40787c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f34c4001eb0 tx=0x7f34c4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:06.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:06.916+0000 7f34caa00640 1 -- 172.21.15.167:0/2296326542 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f34c00ba030 con 0x7f34d415c570 2024-05-11T12:13:07.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.122+0000 7f34d3e00640 1 -- 172.21.15.167:0/2296326542 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f34d40694b0 con 0x7f34d415c570 2024-05-11T12:13:07.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.124+0000 7f34caa00640 1 -- 172.21.15.167:0/2296326542 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) ==== 74+0+16932 (secure 0 0 0) 0x7f34c007ff60 con 0x7f34d415c570 2024-05-11T12:13:07.125 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:13:07.125 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":64,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:13:04.144728+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:13:00.503965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-11T12:13:01.566330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:13:07.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.127+0000 7f34d3e00640 1 -- 172.21.15.167:0/2296326542 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f34b4076300 msgr2=0x7f34b40787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:07.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.127+0000 7f34d3e00640 1 --2- 172.21.15.167:0/2296326542 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f34b4076300 0x7f34b40787c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f34c4001eb0 tx=0x7f34c4009040 comp rx=0 tx=0).stop 2024-05-11T12:13:07.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.127+0000 7f34d3e00640 1 -- 172.21.15.167:0/2296326542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f34d415c570 msgr2=0x7f34d41f8950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:07.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.127+0000 7f34d3e00640 1 --2- 172.21.15.167:0/2296326542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f34d415c570 0x7f34d41f8950 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f34c0005040 tx=0x7f34c0001120 comp rx=0 tx=0).stop 2024-05-11T12:13:07.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.127+0000 7f34d3e00640 1 -- 172.21.15.167:0/2296326542 shutdown_connections 2024-05-11T12:13:07.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.127+0000 7f34d3e00640 1 --2- 172.21.15.167:0/2296326542 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34d415bf70 0x7f34d41f8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:07.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.127+0000 7f34d3e00640 1 --2- 172.21.15.167:0/2296326542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f34d415c570 0x7f34d41f8950 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:07.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.127+0000 7f34d3e00640 1 --2- 172.21.15.167:0/2296326542 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f34b4076300 0x7f34b40787c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:07.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.127+0000 7f34d3e00640 1 --2- 172.21.15.167:0/2296326542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34b80040b0 0x7f34d413de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:07.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.127+0000 7f34d3e00640 1 -- 172.21.15.167:0/2296326542 >> 172.21.15.167:0/2296326542 conn(0x7f34d40b3460 msgr2=0x7f34d4141460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:07.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.127+0000 7f34d3e00640 1 -- 172.21.15.167:0/2296326542 shutdown_connections 2024-05-11T12:13:07.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.127+0000 7f34d3e00640 1 -- 172.21.15.167:0/2296326542 wait complete. 2024-05-11T12:13:07.147 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-11T12:13:07.148 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T12:13:07.253 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T12:13:07.262 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T12:13:07.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.348+0000 7fccda600640 1 Processor -- start 2024-05-11T12:13:07.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.348+0000 7fccda600640 1 -- start start 2024-05-11T12:13:07.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.349+0000 7fccda600640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fccd40b3ea0 0x7fccd4161490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:07.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.349+0000 7fccda600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fccd41619d0 0x7fccd406d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:07.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.349+0000 7fccda600640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccd415df30 con 0x7fccd4157750 2024-05-11T12:13:07.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.349+0000 7fccda600640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fccd4161000 con 0x7fccd40b3ea0 2024-05-11T12:13:07.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.350+0000 7fccda600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fccd415e210 con 0x7fccd41619d0 2024-05-11T12:13:07.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.350+0000 7fccd3400640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fccd4157750 0x7fccd415d740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:33324/0 (socket says 172.21.15.167:33324) 2024-05-11T12:13:07.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.350+0000 7fccd3400640 1 -- 172.21.15.167:0/4032557822 learned_addr learned my addr 172.21.15.167:0/4032557822 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:13:07.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.350+0000 7fccd2a00640 1 --2- 172.21.15.167:0/4032557822 >> v2:172.21.15.167:3300/0 conn(0x7fccd40b3ea0 0x7fccd4161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:07.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.350+0000 7fccd3e00640 1 --2- 172.21.15.167:0/4032557822 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fccd41619d0 0x7fccd406d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:07.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.350+0000 7fccd2a00640 1 -- 172.21.15.167:0/4032557822 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fccd41619d0 msgr2=0x7fccd406d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:07.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.350+0000 7fccd2a00640 1 --2- 172.21.15.167:0/4032557822 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fccd41619d0 0x7fccd406d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:07.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.350+0000 7fccd2a00640 1 -- 172.21.15.167:0/4032557822 >> v1:172.21.15.167:6789/0 conn(0x7fccd4157750 legacy=0x7fccd415d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:07.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.350+0000 7fccd2000640 1 -- 172.21.15.167:0/4032557822 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1169708002 0 0) 0x7fccd415df30 con 0x7fccd4157750 2024-05-11T12:13:07.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.350+0000 7fccd2a00640 1 -- 172.21.15.167:0/4032557822 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd415e4f0 con 0x7fccd40b3ea0 2024-05-11T12:13:07.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.351+0000 7fccd2a00640 1 --2- 172.21.15.167:0/4032557822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fccd40b3ea0 0x7fccd4161490 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fccc00097a0 tx=0x7fccc002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b193e48a02b677e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:07.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.352+0000 7fccd2000640 1 -- 172.21.15.167:0/4032557822 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc0037070 con 0x7fccd40b3ea0 2024-05-11T12:13:07.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.352+0000 7fccd2000640 1 -- 172.21.15.167:0/4032557822 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccc0032440 con 0x7fccd40b3ea0 2024-05-11T12:13:07.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.352+0000 7fccd2000640 1 -- 172.21.15.167:0/4032557822 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc0032810 con 0x7fccd40b3ea0 2024-05-11T12:13:07.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.352+0000 7fccda600640 1 -- 172.21.15.167:0/4032557822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fccd40b3ea0 msgr2=0x7fccd4161490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:07.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.352+0000 7fccda600640 1 --2- 172.21.15.167:0/4032557822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fccd40b3ea0 0x7fccd4161490 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fccc00097a0 tx=0x7fccc002a180 comp rx=0 tx=0).stop 2024-05-11T12:13:07.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.352+0000 7fccda600640 1 -- 172.21.15.167:0/4032557822 shutdown_connections 2024-05-11T12:13:07.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.352+0000 7fccda600640 1 --2- 172.21.15.167:0/4032557822 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fccd41619d0 0x7fccd406d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:07.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.352+0000 7fccda600640 1 --2- 172.21.15.167:0/4032557822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fccd40b3ea0 0x7fccd4161490 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:07.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.352+0000 7fccda600640 1 -- 172.21.15.167:0/4032557822 >> 172.21.15.167:0/4032557822 conn(0x7fccd40cd8f0 msgr2=0x7fccd40cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:07.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.352+0000 7fccda600640 1 -- 172.21.15.167:0/4032557822 shutdown_connections 2024-05-11T12:13:07.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.352+0000 7fccda600640 1 -- 172.21.15.167:0/4032557822 wait complete. 2024-05-11T12:13:07.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.354+0000 7fccda600640 1 Processor -- start 2024-05-11T12:13:07.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.354+0000 7fccda600640 1 -- start start 2024-05-11T12:13:07.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.355+0000 7fccda600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fccd40b3ea0 0x7fccd41f8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:07.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.355+0000 7fccda600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccd4157750 0x7fccd40b1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:07.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.355+0000 7fccda600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fccd41619d0 0x7fccd40cd070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:07.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.355+0000 7fccda600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fccd406db20 con 0x7fccd40b3ea0 2024-05-11T12:13:07.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.355+0000 7fccda600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fccd406d9a0 con 0x7fccd4157750 2024-05-11T12:13:07.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.355+0000 7fccda600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fccd406dca0 con 0x7fccd41619d0 2024-05-11T12:13:07.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.355+0000 7fccd3400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fccd40b3ea0 0x7fccd41f8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:07.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.356+0000 7fccd3400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fccd40b3ea0 0x7fccd41f8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43362/0 (socket says 172.21.15.167:43362) 2024-05-11T12:13:07.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.356+0000 7fccd3400640 1 -- 172.21.15.167:0/2829457610 learned_addr learned my addr 172.21.15.167:0/2829457610 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:07.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.356+0000 7fccd3400640 1 -- 172.21.15.167:0/2829457610 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fccd41619d0 msgr2=0x7fccd40cd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:07.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.356+0000 7fccd3400640 1 --2- 172.21.15.167:0/2829457610 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fccd41619d0 0x7fccd40cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:07.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.356+0000 7fccd3400640 1 -- 172.21.15.167:0/2829457610 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccd4157750 msgr2=0x7fccd40b1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:07.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.356+0000 7fccd3400640 1 --2- 172.21.15.167:0/2829457610 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccd4157750 0x7fccd40b1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:07.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.356+0000 7fccd3400640 1 -- 172.21.15.167:0/2829457610 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd40cd5b0 con 0x7fccd40b3ea0 2024-05-11T12:13:07.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.356+0000 7fccd3400640 1 --2- 172.21.15.167:0/2829457610 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fccd40b3ea0 0x7fccd41f8f60 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fccc400c490 tx=0x7fccc400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:07.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.357+0000 7fcccb400640 1 -- 172.21.15.167:0/2829457610 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc4018070 con 0x7fccd40b3ea0 2024-05-11T12:13:07.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.357+0000 7fcccb400640 1 -- 172.21.15.167:0/2829457610 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccc4007500 con 0x7fccd40b3ea0 2024-05-11T12:13:07.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.357+0000 7fcccb400640 1 -- 172.21.15.167:0/2829457610 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc4004a20 con 0x7fccd40b3ea0 2024-05-11T12:13:07.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.357+0000 7fccda600640 1 -- 172.21.15.167:0/2829457610 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccd40b25f0 con 0x7fccd40b3ea0 2024-05-11T12:13:07.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.357+0000 7fccda600640 1 -- 172.21.15.167:0/2829457610 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccd41f8760 con 0x7fccd40b3ea0 2024-05-11T12:13:07.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.358+0000 7fccda600640 1 -- 172.21.15.167:0/2829457610 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccd41f8b90 con 0x7fccd40b3ea0 2024-05-11T12:13:07.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.359+0000 7fcccb400640 1 -- 172.21.15.167:0/2829457610 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fccc4004bc0 con 0x7fccd40b3ea0 2024-05-11T12:13:07.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.360+0000 7fcccb400640 1 --2- 172.21.15.167:0/2829457610 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fccb4076300 0x7fccb40787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:07.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.360+0000 7fcccb400640 1 -- 172.21.15.167:0/2829457610 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7fccc40969b0 con 0x7fccd40b3ea0 2024-05-11T12:13:07.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.361+0000 7fccd2a00640 1 --2- 172.21.15.167:0/2829457610 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fccb4076300 0x7fccb40787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:07.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.361+0000 7fccd2a00640 1 --2- 172.21.15.167:0/2829457610 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fccb4076300 0x7fccb40787c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fccc0034000 tx=0x7fccc002c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:07.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.365+0000 7fcccb400640 1 -- 172.21.15.167:0/2829457610 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fccc4060170 con 0x7fccd40b3ea0 2024-05-11T12:13:07.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.571+0000 7fccda600640 1 -- 172.21.15.167:0/2829457610 --> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fccd4069590 con 0x7fccb4076300 2024-05-11T12:13:07.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.591+0000 7fcccb400640 1 -- 172.21.15.167:0/2829457610 <== mgr.4106 v2:172.21.15.167:6832/578352559 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250460 (secure 0 0 0) 0x7fccd4069590 con 0x7fccb4076300 2024-05-11T12:13:07.596 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:13:07.598 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2024-05-11T12:13:07.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.597+0000 7fccda600640 1 -- 172.21.15.167:0/2829457610 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fccb4076300 msgr2=0x7fccb40787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:07.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.597+0000 7fccda600640 1 --2- 172.21.15.167:0/2829457610 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fccb4076300 0x7fccb40787c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fccc0034000 tx=0x7fccc002c000 comp rx=0 tx=0).stop 2024-05-11T12:13:07.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.597+0000 7fccda600640 1 -- 172.21.15.167:0/2829457610 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fccd40b3ea0 msgr2=0x7fccd41f8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:07.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.597+0000 7fccda600640 1 --2- 172.21.15.167:0/2829457610 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fccd40b3ea0 0x7fccd41f8f60 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fccc400c490 tx=0x7fccc400c950 comp rx=0 tx=0).stop 2024-05-11T12:13:07.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.597+0000 7fccda600640 1 -- 172.21.15.167:0/2829457610 shutdown_connections 2024-05-11T12:13:07.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.597+0000 7fccda600640 1 --2- 172.21.15.167:0/2829457610 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fccd41619d0 0x7fccd40cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:07.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.597+0000 7fccda600640 1 --2- 172.21.15.167:0/2829457610 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fccb4076300 0x7fccb40787c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:07.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.597+0000 7fccda600640 1 --2- 172.21.15.167:0/2829457610 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccd4157750 0x7fccd40b1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:07.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.597+0000 7fccda600640 1 --2- 172.21.15.167:0/2829457610 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fccd40b3ea0 0x7fccd41f8f60 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:07.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.597+0000 7fccda600640 1 -- 172.21.15.167:0/2829457610 >> 172.21.15.167:0/2829457610 conn(0x7fccd40cd8f0 msgr2=0x7fccd406ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:07.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.597+0000 7fccda600640 1 -- 172.21.15.167:0/2829457610 shutdown_connections 2024-05-11T12:13:07.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.597+0000 7fccda600640 1 -- 172.21.15.167:0/2829457610 wait complete. 2024-05-11T12:13:07.616 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":135,"stamp":"2024-05-11T12:13:07.279017+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590584,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":128,"num_write_kb":1638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91,"ondisk_log_size":91,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":93,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223108,"kb_used_data":6920,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751612,"statfs":{"total":773094113280,"available":772865650688,"internally_reserved":0,"allocated":7086080,"data_stored":3787685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001654"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.104414+0000","last_change":"2024-05-11T12:13:02.104414+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.104414+0000","last_undegraded":"2024-05-11T12:13:02.104414+0000","last_fullsized":"2024-05-11T12:13:02.104414+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.102208+0000","last_change":"2024-05-11T12:13:02.102208+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.102208+0000","last_undegraded":"2024-05-11T12:13:02.102208+0000","last_fullsized":"2024-05-11T12:13:02.102208+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.180687+0000","last_change":"2024-05-11T12:13:03.165899+0000","last_active":"2024-05-11T12:13:03.180687+0000","last_peered":"2024-05-11T12:13:03.180687+0000","last_clean":"2024-05-11T12:13:03.180687+0000","last_became_active":"2024-05-11T12:13:03.165494+0000","last_became_peered":"2024-05-11T12:13:03.165494+0000","last_unstale":"2024-05-11T12:13:03.180687+0000","last_undegraded":"2024-05-11T12:13:03.180687+0000","last_fullsized":"2024-05-11T12:13:03.180687+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:04:05.997706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.174472+0000","last_change":"2024-05-11T12:13:03.166762+0000","last_active":"2024-05-11T12:13:03.174472+0000","last_peered":"2024-05-11T12:13:03.174472+0000","last_clean":"2024-05-11T12:13:03.174472+0000","last_became_active":"2024-05-11T12:13:03.166457+0000","last_became_peered":"2024-05-11T12:13:03.166457+0000","last_unstale":"2024-05-11T12:13:03.174472+0000","last_undegraded":"2024-05-11T12:13:03.174472+0000","last_fullsized":"2024-05-11T12:13:03.174472+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:46:22.393593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.173955+0000","last_change":"2024-05-11T12:13:03.167675+0000","last_active":"2024-05-11T12:13:03.173955+0000","last_peered":"2024-05-11T12:13:03.173955+0000","last_clean":"2024-05-11T12:13:03.173955+0000","last_became_active":"2024-05-11T12:13:03.167381+0000","last_became_peered":"2024-05-11T12:13:03.167381+0000","last_unstale":"2024-05-11T12:13:03.173955+0000","last_undegraded":"2024-05-11T12:13:03.173955+0000","last_fullsized":"2024-05-11T12:13:03.173955+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:06:40.718612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.121564+0000","last_change":"2024-05-11T12:13:02.121564+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.121564+0000","last_undegraded":"2024-05-11T12:13:02.121564+0000","last_fullsized":"2024-05-11T12:13:02.121564+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.173969+0000","last_change":"2024-05-11T12:13:03.170170+0000","last_active":"2024-05-11T12:13:04.173969+0000","last_peered":"2024-05-11T12:13:04.173969+0000","last_clean":"2024-05-11T12:13:04.173969+0000","last_became_active":"2024-05-11T12:13:03.169888+0000","last_became_peered":"2024-05-11T12:13:03.169888+0000","last_unstale":"2024-05-11T12:13:04.173969+0000","last_undegraded":"2024-05-11T12:13:04.173969+0000","last_fullsized":"2024-05-11T12:13:04.173969+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:46:12.686412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.174546+0000","last_change":"2024-05-11T12:13:03.167147+0000","last_active":"2024-05-11T12:13:03.174546+0000","last_peered":"2024-05-11T12:13:03.174546+0000","last_clean":"2024-05-11T12:13:03.174546+0000","last_became_active":"2024-05-11T12:13:03.166882+0000","last_became_peered":"2024-05-11T12:13:03.166882+0000","last_unstale":"2024-05-11T12:13:03.174546+0000","last_undegraded":"2024-05-11T12:13:03.174546+0000","last_fullsized":"2024-05-11T12:13:03.174546+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:04:05.997706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.180451+0000","last_change":"2024-05-11T12:13:03.162128+0000","last_active":"2024-05-11T12:13:03.180451+0000","last_peered":"2024-05-11T12:13:03.180451+0000","last_clean":"2024-05-11T12:13:03.180451+0000","last_became_active":"2024-05-11T12:13:03.161841+0000","last_became_peered":"2024-05-11T12:13:03.161841+0000","last_unstale":"2024-05-11T12:13:03.180451+0000","last_undegraded":"2024-05-11T12:13:03.180451+0000","last_fullsized":"2024-05-11T12:13:03.180451+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:27:14.194818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.169523+0000","last_change":"2024-05-11T12:13:03.164933+0000","last_active":"2024-05-11T12:13:03.169523+0000","last_peered":"2024-05-11T12:13:03.169523+0000","last_clean":"2024-05-11T12:13:03.169523+0000","last_became_active":"2024-05-11T12:13:03.164680+0000","last_became_peered":"2024-05-11T12:13:03.164680+0000","last_unstale":"2024-05-11T12:13:03.169523+0000","last_undegraded":"2024-05-11T12:13:03.169523+0000","last_fullsized":"2024-05-11T12:13:03.169523+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:08:16.138838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.174853+0000","last_change":"2024-05-11T12:13:03.163978+0000","last_active":"2024-05-11T12:13:03.174853+0000","last_peered":"2024-05-11T12:13:03.174853+0000","last_clean":"2024-05-11T12:13:03.174853+0000","last_became_active":"2024-05-11T12:13:03.163688+0000","last_became_peered":"2024-05-11T12:13:03.163688+0000","last_unstale":"2024-05-11T12:13:03.174853+0000","last_undegraded":"2024-05-11T12:13:03.174853+0000","last_fullsized":"2024-05-11T12:13:03.174853+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:15:35.851039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.170759+0000","last_change":"2024-05-11T12:13:03.164295+0000","last_active":"2024-05-11T12:13:03.170759+0000","last_peered":"2024-05-11T12:13:03.170759+0000","last_clean":"2024-05-11T12:13:03.170759+0000","last_became_active":"2024-05-11T12:13:03.163948+0000","last_became_peered":"2024-05-11T12:13:03.163948+0000","last_unstale":"2024-05-11T12:13:03.170759+0000","last_undegraded":"2024-05-11T12:13:03.170759+0000","last_fullsized":"2024-05-11T12:13:03.170759+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:24:40.432373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.170701+0000","last_change":"2024-05-11T12:13:03.163777+0000","last_active":"2024-05-11T12:13:03.170701+0000","last_peered":"2024-05-11T12:13:03.170701+0000","last_clean":"2024-05-11T12:13:03.170701+0000","last_became_active":"2024-05-11T12:13:03.163472+0000","last_became_peered":"2024-05-11T12:13:03.163472+0000","last_unstale":"2024-05-11T12:13:03.170701+0000","last_undegraded":"2024-05-11T12:13:03.170701+0000","last_fullsized":"2024-05-11T12:13:03.170701+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:58:08.814828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.169784+0000","last_change":"2024-05-11T12:13:03.165418+0000","last_active":"2024-05-11T12:13:03.169784+0000","last_peered":"2024-05-11T12:13:03.169784+0000","last_clean":"2024-05-11T12:13:03.169784+0000","last_became_active":"2024-05-11T12:13:03.165020+0000","last_became_peered":"2024-05-11T12:13:03.165020+0000","last_unstale":"2024-05-11T12:13:03.169784+0000","last_undegraded":"2024-05-11T12:13:03.169784+0000","last_fullsized":"2024-05-11T12:13:03.169784+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:24:40.432373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.182452+0000","last_change":"2024-05-11T12:13:03.174279+0000","last_active":"2024-05-11T12:13:03.182452+0000","last_peered":"2024-05-11T12:13:03.182452+0000","last_clean":"2024-05-11T12:13:03.182452+0000","last_became_active":"2024-05-11T12:13:03.173960+0000","last_became_peered":"2024-05-11T12:13:03.173960+0000","last_unstale":"2024-05-11T12:13:03.182452+0000","last_undegraded":"2024-05-11T12:13:03.182452+0000","last_fullsized":"2024-05-11T12:13:03.182452+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:29:58.089646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.173798+0000","last_change":"2024-05-11T12:13:03.176759+0000","last_active":"2024-05-11T12:13:04.173798+0000","last_peered":"2024-05-11T12:13:04.173798+0000","last_clean":"2024-05-11T12:13:04.173798+0000","last_became_active":"2024-05-11T12:13:03.176496+0000","last_became_peered":"2024-05-11T12:13:03.176496+0000","last_unstale":"2024-05-11T12:13:04.173798+0000","last_undegraded":"2024-05-11T12:13:04.173798+0000","last_fullsized":"2024-05-11T12:13:04.173798+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:33:36.274987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.103891+0000","last_change":"2024-05-11T12:13:02.103891+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.103891+0000","last_undegraded":"2024-05-11T12:13:02.103891+0000","last_fullsized":"2024-05-11T12:13:02.103891+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.175933+0000","last_change":"2024-05-11T12:13:03.167581+0000","last_active":"2024-05-11T12:13:03.175933+0000","last_peered":"2024-05-11T12:13:03.175933+0000","last_clean":"2024-05-11T12:13:03.175933+0000","last_became_active":"2024-05-11T12:13:03.167285+0000","last_became_peered":"2024-05-11T12:13:03.167285+0000","last_unstale":"2024-05-11T12:13:03.175933+0000","last_undegraded":"2024-05-11T12:13:03.175933+0000","last_fullsized":"2024-05-11T12:13:03.175933+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:52:39.367286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.175530+0000","last_change":"2024-05-11T12:13:03.172989+0000","last_active":"2024-05-11T12:13:03.175530+0000","last_peered":"2024-05-11T12:13:03.175530+0000","last_clean":"2024-05-11T12:13:03.175530+0000","last_became_active":"2024-05-11T12:13:03.172704+0000","last_became_peered":"2024-05-11T12:13:03.172704+0000","last_unstale":"2024-05-11T12:13:03.175530+0000","last_undegraded":"2024-05-11T12:13:03.175530+0000","last_fullsized":"2024-05-11T12:13:03.175530+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:28:34.473489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.116797+0000","last_change":"2024-05-11T12:13:02.116797+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.116797+0000","last_undegraded":"2024-05-11T12:13:02.116797+0000","last_fullsized":"2024-05-11T12:13:02.116797+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.170678+0000","last_change":"2024-05-11T12:13:03.162655+0000","last_active":"2024-05-11T12:13:03.170678+0000","last_peered":"2024-05-11T12:13:03.170678+0000","last_clean":"2024-05-11T12:13:03.170678+0000","last_became_active":"2024-05-11T12:13:03.162259+0000","last_became_peered":"2024-05-11T12:13:03.162259+0000","last_unstale":"2024-05-11T12:13:03.170678+0000","last_undegraded":"2024-05-11T12:13:03.170678+0000","last_fullsized":"2024-05-11T12:13:03.170678+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:42:21.959847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.116334+0000","last_change":"2024-05-11T12:13:02.116334+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.116334+0000","last_undegraded":"2024-05-11T12:13:02.116334+0000","last_fullsized":"2024-05-11T12:13:02.116334+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.115725+0000","last_change":"2024-05-11T12:13:02.115725+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.115725+0000","last_undegraded":"2024-05-11T12:13:02.115725+0000","last_fullsized":"2024-05-11T12:13:02.115725+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.171059+0000","last_change":"2024-05-11T12:13:03.165438+0000","last_active":"2024-05-11T12:13:03.171059+0000","last_peered":"2024-05-11T12:13:03.171059+0000","last_clean":"2024-05-11T12:13:03.171059+0000","last_became_active":"2024-05-11T12:13:03.165118+0000","last_became_peered":"2024-05-11T12:13:03.165118+0000","last_unstale":"2024-05-11T12:13:03.171059+0000","last_undegraded":"2024-05-11T12:13:03.171059+0000","last_fullsized":"2024-05-11T12:13:03.171059+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:56:37.784627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.176455+0000","last_change":"2024-05-11T12:13:03.169517+0000","last_active":"2024-05-11T12:13:03.176455+0000","last_peered":"2024-05-11T12:13:03.176455+0000","last_clean":"2024-05-11T12:13:03.176455+0000","last_became_active":"2024-05-11T12:13:03.169235+0000","last_became_peered":"2024-05-11T12:13:03.169235+0000","last_unstale":"2024-05-11T12:13:03.176455+0000","last_undegraded":"2024-05-11T12:13:03.176455+0000","last_fullsized":"2024-05-11T12:13:03.176455+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:27:14.194818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.174469+0000","last_change":"2024-05-11T12:13:03.166968+0000","last_active":"2024-05-11T12:13:03.174469+0000","last_peered":"2024-05-11T12:13:03.174469+0000","last_clean":"2024-05-11T12:13:03.174469+0000","last_became_active":"2024-05-11T12:13:03.166704+0000","last_became_peered":"2024-05-11T12:13:03.166704+0000","last_unstale":"2024-05-11T12:13:03.174469+0000","last_undegraded":"2024-05-11T12:13:03.174469+0000","last_fullsized":"2024-05-11T12:13:03.174469+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:56:37.784627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.178640+0000","last_change":"2024-05-11T12:13:03.171885+0000","last_active":"2024-05-11T12:13:03.178640+0000","last_peered":"2024-05-11T12:13:03.178640+0000","last_clean":"2024-05-11T12:13:03.178640+0000","last_became_active":"2024-05-11T12:13:03.171590+0000","last_became_peered":"2024-05-11T12:13:03.171590+0000","last_unstale":"2024-05-11T12:13:03.178640+0000","last_undegraded":"2024-05-11T12:13:03.178640+0000","last_fullsized":"2024-05-11T12:13:03.178640+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:06:40.718612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.103639+0000","last_change":"2024-05-11T12:13:02.103639+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.103639+0000","last_undegraded":"2024-05-11T12:13:02.103639+0000","last_fullsized":"2024-05-11T12:13:02.103639+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.164436+0000","last_change":"2024-05-11T12:13:03.160607+0000","last_active":"2024-05-11T12:13:03.164436+0000","last_peered":"2024-05-11T12:13:03.164436+0000","last_clean":"2024-05-11T12:13:03.164436+0000","last_became_active":"2024-05-11T12:13:03.160281+0000","last_became_peered":"2024-05-11T12:13:03.160281+0000","last_unstale":"2024-05-11T12:13:03.164436+0000","last_undegraded":"2024-05-11T12:13:03.164436+0000","last_fullsized":"2024-05-11T12:13:03.164436+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:12:30.081833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.170126+0000","last_change":"2024-05-11T12:13:03.164270+0000","last_active":"2024-05-11T12:13:03.170126+0000","last_peered":"2024-05-11T12:13:03.170126+0000","last_clean":"2024-05-11T12:13:03.170126+0000","last_became_active":"2024-05-11T12:13:03.163977+0000","last_became_peered":"2024-05-11T12:13:03.163977+0000","last_unstale":"2024-05-11T12:13:03.170126+0000","last_undegraded":"2024-05-11T12:13:03.170126+0000","last_fullsized":"2024-05-11T12:13:03.170126+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:08:16.138838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.177851+0000","last_change":"2024-05-11T12:13:03.171792+0000","last_active":"2024-05-11T12:13:03.177851+0000","last_peered":"2024-05-11T12:13:03.177851+0000","last_clean":"2024-05-11T12:13:03.177851+0000","last_became_active":"2024-05-11T12:13:03.171502+0000","last_became_peered":"2024-05-11T12:13:03.171502+0000","last_unstale":"2024-05-11T12:13:03.177851+0000","last_undegraded":"2024-05-11T12:13:03.177851+0000","last_fullsized":"2024-05-11T12:13:03.177851+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:20:41.663583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.121306+0000","last_change":"2024-05-11T12:13:02.121306+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.121306+0000","last_undegraded":"2024-05-11T12:13:02.121306+0000","last_fullsized":"2024-05-11T12:13:02.121306+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.177562+0000","last_change":"2024-05-11T12:13:03.176021+0000","last_active":"2024-05-11T12:13:03.177562+0000","last_peered":"2024-05-11T12:13:03.177562+0000","last_clean":"2024-05-11T12:13:03.177562+0000","last_became_active":"2024-05-11T12:13:03.175708+0000","last_became_peered":"2024-05-11T12:13:03.175708+0000","last_unstale":"2024-05-11T12:13:03.177562+0000","last_undegraded":"2024-05-11T12:13:03.177562+0000","last_fullsized":"2024-05-11T12:13:03.177562+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:12:07.890717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.140559+0000","last_change":"2024-05-11T12:13:02.136461+0000","last_active":"2024-05-11T12:13:02.140559+0000","last_peered":"2024-05-11T12:13:02.140559+0000","last_clean":"2024-05-11T12:13:02.140559+0000","last_became_active":"2024-05-11T12:13:02.136210+0000","last_became_peered":"2024-05-11T12:13:02.136210+0000","last_unstale":"2024-05-11T12:13:02.140559+0000","last_undegraded":"2024-05-11T12:13:02.140559+0000","last_fullsized":"2024-05-11T12:13:02.140559+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:20:40.658655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.159893+0000","last_change":"2024-05-11T12:13:03.158314+0000","last_active":"2024-05-11T12:13:03.159893+0000","last_peered":"2024-05-11T12:13:03.159893+0000","last_clean":"2024-05-11T12:13:03.159893+0000","last_became_active":"2024-05-11T12:13:03.157991+0000","last_became_peered":"2024-05-11T12:13:03.157991+0000","last_unstale":"2024-05-11T12:13:03.159893+0000","last_undegraded":"2024-05-11T12:13:03.159893+0000","last_fullsized":"2024-05-11T12:13:03.159893+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:55:20.561101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.140990+0000","last_change":"2024-05-11T12:13:02.136740+0000","last_active":"2024-05-11T12:13:02.140990+0000","last_peered":"2024-05-11T12:13:02.140990+0000","last_clean":"2024-05-11T12:13:02.140990+0000","last_became_active":"2024-05-11T12:13:02.136446+0000","last_became_peered":"2024-05-11T12:13:02.136446+0000","last_unstale":"2024-05-11T12:13:02.140990+0000","last_undegraded":"2024-05-11T12:13:02.140990+0000","last_fullsized":"2024-05-11T12:13:02.140990+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:58:07.809901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.120695+0000","last_change":"2024-05-11T12:13:02.120695+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.120695+0000","last_undegraded":"2024-05-11T12:13:02.120695+0000","last_fullsized":"2024-05-11T12:13:02.120695+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.137480+0000","last_change":"2024-05-11T12:13:02.129658+0000","last_active":"2024-05-11T12:13:02.137480+0000","last_peered":"2024-05-11T12:13:02.137480+0000","last_clean":"2024-05-11T12:13:02.137480+0000","last_became_active":"2024-05-11T12:13:02.129326+0000","last_became_peered":"2024-05-11T12:13:02.129326+0000","last_unstale":"2024-05-11T12:13:02.137480+0000","last_undegraded":"2024-05-11T12:13:02.137480+0000","last_fullsized":"2024-05-11T12:13:02.137480+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:42:20.954919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.168978+0000","last_change":"2024-05-11T12:13:03.162149+0000","last_active":"2024-05-11T12:13:03.168978+0000","last_peered":"2024-05-11T12:13:03.168978+0000","last_clean":"2024-05-11T12:13:03.168978+0000","last_became_active":"2024-05-11T12:13:03.161890+0000","last_became_peered":"2024-05-11T12:13:03.161890+0000","last_unstale":"2024-05-11T12:13:03.168978+0000","last_undegraded":"2024-05-11T12:13:03.168978+0000","last_fullsized":"2024-05-11T12:13:03.168978+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:42:21.959847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.134444+0000","last_change":"2024-05-11T12:13:02.126473+0000","last_active":"2024-05-11T12:13:02.134444+0000","last_peered":"2024-05-11T12:13:02.134444+0000","last_clean":"2024-05-11T12:13:02.134444+0000","last_became_active":"2024-05-11T12:13:02.126154+0000","last_became_peered":"2024-05-11T12:13:02.126154+0000","last_unstale":"2024-05-11T12:13:02.134444+0000","last_undegraded":"2024-05-11T12:13:02.134444+0000","last_fullsized":"2024-05-11T12:13:02.134444+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:15:28.704713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.173923+0000","last_change":"2024-05-11T12:13:03.164511+0000","last_active":"2024-05-11T12:13:04.173923+0000","last_peered":"2024-05-11T12:13:04.173923+0000","last_clean":"2024-05-11T12:13:04.173923+0000","last_became_active":"2024-05-11T12:13:03.164197+0000","last_became_peered":"2024-05-11T12:13:03.164197+0000","last_unstale":"2024-05-11T12:13:04.173923+0000","last_undegraded":"2024-05-11T12:13:04.173923+0000","last_fullsized":"2024-05-11T12:13:04.173923+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:54:15.999941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.125903+0000","last_change":"2024-05-11T12:13:02.119338+0000","last_active":"2024-05-11T12:13:02.125903+0000","last_peered":"2024-05-11T12:13:02.125903+0000","last_clean":"2024-05-11T12:13:02.125903+0000","last_became_active":"2024-05-11T12:13:02.118127+0000","last_became_peered":"2024-05-11T12:13:02.118127+0000","last_unstale":"2024-05-11T12:13:02.125903+0000","last_undegraded":"2024-05-11T12:13:02.125903+0000","last_fullsized":"2024-05-11T12:13:02.125903+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:56:36.779700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.104889+0000","last_change":"2024-05-11T12:13:02.104889+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.104889+0000","last_undegraded":"2024-05-11T12:13:02.104889+0000","last_fullsized":"2024-05-11T12:13:02.104889+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.125968+0000","last_change":"2024-05-11T12:13:02.121903+0000","last_active":"2024-05-11T12:13:02.125968+0000","last_peered":"2024-05-11T12:13:02.125968+0000","last_clean":"2024-05-11T12:13:02.125968+0000","last_became_active":"2024-05-11T12:13:02.120293+0000","last_became_peered":"2024-05-11T12:13:02.120293+0000","last_unstale":"2024-05-11T12:13:02.125968+0000","last_undegraded":"2024-05-11T12:13:02.125968+0000","last_fullsized":"2024-05-11T12:13:02.125968+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:04:04.992779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.174363+0000","last_change":"2024-05-11T12:13:03.168572+0000","last_active":"2024-05-11T12:13:03.174363+0000","last_peered":"2024-05-11T12:13:03.174363+0000","last_clean":"2024-05-11T12:13:03.174363+0000","last_became_active":"2024-05-11T12:13:03.167854+0000","last_became_peered":"2024-05-11T12:13:03.167854+0000","last_unstale":"2024-05-11T12:13:03.174363+0000","last_undegraded":"2024-05-11T12:13:03.174363+0000","last_fullsized":"2024-05-11T12:13:03.174363+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:27:27.374412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.143860+0000","last_change":"2024-05-11T12:13:02.139509+0000","last_active":"2024-05-11T12:13:02.143860+0000","last_peered":"2024-05-11T12:13:02.143860+0000","last_clean":"2024-05-11T12:13:02.143860+0000","last_became_active":"2024-05-11T12:13:02.139201+0000","last_became_peered":"2024-05-11T12:13:02.139201+0000","last_unstale":"2024-05-11T12:13:02.143860+0000","last_undegraded":"2024-05-11T12:13:02.143860+0000","last_fullsized":"2024-05-11T12:13:02.143860+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:55:25.067740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.115715+0000","last_change":"2024-05-11T12:13:02.115715+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.115715+0000","last_undegraded":"2024-05-11T12:13:02.115715+0000","last_fullsized":"2024-05-11T12:13:02.115715+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.147186+0000","last_change":"2024-05-11T12:13:02.139692+0000","last_active":"2024-05-11T12:13:03.147186+0000","last_peered":"2024-05-11T12:13:03.147186+0000","last_clean":"2024-05-11T12:13:03.147186+0000","last_became_active":"2024-05-11T12:13:02.137935+0000","last_became_peered":"2024-05-11T12:13:02.137935+0000","last_unstale":"2024-05-11T12:13:03.147186+0000","last_undegraded":"2024-05-11T12:13:03.147186+0000","last_fullsized":"2024-05-11T12:13:03.147186+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:15:34.846112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.103086+0000","last_change":"2024-05-11T12:13:02.103086+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.103086+0000","last_undegraded":"2024-05-11T12:13:02.103086+0000","last_fullsized":"2024-05-11T12:13:02.103086+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.154876+0000","last_change":"2024-05-11T12:13:02.142438+0000","last_active":"2024-05-11T12:13:03.154876+0000","last_peered":"2024-05-11T12:13:03.154876+0000","last_clean":"2024-05-11T12:13:03.154876+0000","last_became_active":"2024-05-11T12:13:02.142125+0000","last_became_peered":"2024-05-11T12:13:02.142125+0000","last_unstale":"2024-05-11T12:13:03.154876+0000","last_undegraded":"2024-05-11T12:13:03.154876+0000","last_fullsized":"2024-05-11T12:13:03.154876+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:55:32.333055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.116293+0000","last_change":"2024-05-11T12:13:02.116293+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.116293+0000","last_undegraded":"2024-05-11T12:13:02.116293+0000","last_fullsized":"2024-05-11T12:13:02.116293+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.144040+0000","last_change":"2024-05-11T12:13:02.140895+0000","last_active":"2024-05-11T12:13:03.144040+0000","last_peered":"2024-05-11T12:13:03.144040+0000","last_clean":"2024-05-11T12:13:03.144040+0000","last_became_active":"2024-05-11T12:13:02.138733+0000","last_became_peered":"2024-05-11T12:13:02.138733+0000","last_unstale":"2024-05-11T12:13:03.144040+0000","last_undegraded":"2024-05-11T12:13:03.144040+0000","last_fullsized":"2024-05-11T12:13:03.144040+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:08:47.140652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.171236+0000","last_change":"2024-05-11T12:13:03.160320+0000","last_active":"2024-05-11T12:13:03.171236+0000","last_peered":"2024-05-11T12:13:03.171236+0000","last_clean":"2024-05-11T12:13:03.171236+0000","last_became_active":"2024-05-11T12:13:03.159984+0000","last_became_peered":"2024-05-11T12:13:03.159984+0000","last_unstale":"2024-05-11T12:13:03.171236+0000","last_undegraded":"2024-05-11T12:13:03.171236+0000","last_fullsized":"2024-05-11T12:13:03.171236+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:04:22.163536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"63'3","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.185645+0000","last_change":"2024-05-11T12:13:02.139752+0000","last_active":"2024-05-11T12:13:03.185645+0000","last_peered":"2024-05-11T12:13:03.185645+0000","last_clean":"2024-05-11T12:13:03.185645+0000","last_became_active":"2024-05-11T12:13:02.139431+0000","last_became_peered":"2024-05-11T12:13:02.139431+0000","last_unstale":"2024-05-11T12:13:03.185645+0000","last_undegraded":"2024-05-11T12:13:03.185645+0000","last_fullsized":"2024-05-11T12:13:03.185645+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:00:08.081754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.174833+0000","last_change":"2024-05-11T12:13:03.166818+0000","last_active":"2024-05-11T12:13:03.174833+0000","last_peered":"2024-05-11T12:13:03.174833+0000","last_clean":"2024-05-11T12:13:03.174833+0000","last_became_active":"2024-05-11T12:13:03.166560+0000","last_became_peered":"2024-05-11T12:13:03.166560+0000","last_unstale":"2024-05-11T12:13:03.174833+0000","last_undegraded":"2024-05-11T12:13:03.174833+0000","last_fullsized":"2024-05-11T12:13:03.174833+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:04:22.163536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.137897+0000","last_change":"2024-05-11T12:13:02.133552+0000","last_active":"2024-05-11T12:13:02.137897+0000","last_peered":"2024-05-11T12:13:02.137897+0000","last_clean":"2024-05-11T12:13:02.137897+0000","last_became_active":"2024-05-11T12:13:02.133236+0000","last_became_peered":"2024-05-11T12:13:02.133236+0000","last_unstale":"2024-05-11T12:13:02.137897+0000","last_undegraded":"2024-05-11T12:13:02.137897+0000","last_fullsized":"2024-05-11T12:13:02.137897+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:32:44.989980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.175555+0000","last_change":"2024-05-11T12:13:03.168373+0000","last_active":"2024-05-11T12:13:03.175555+0000","last_peered":"2024-05-11T12:13:03.175555+0000","last_clean":"2024-05-11T12:13:03.175555+0000","last_became_active":"2024-05-11T12:13:03.168105+0000","last_became_peered":"2024-05-11T12:13:03.168105+0000","last_unstale":"2024-05-11T12:13:03.175555+0000","last_undegraded":"2024-05-11T12:13:03.175555+0000","last_fullsized":"2024-05-11T12:13:03.175555+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:32:45.994907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"63'2","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.183854+0000","last_change":"2024-05-11T12:13:02.143192+0000","last_active":"2024-05-11T12:13:03.183854+0000","last_peered":"2024-05-11T12:13:03.183854+0000","last_clean":"2024-05-11T12:13:03.183854+0000","last_became_active":"2024-05-11T12:13:02.142901+0000","last_became_peered":"2024-05-11T12:13:02.142901+0000","last_unstale":"2024-05-11T12:13:03.183854+0000","last_undegraded":"2024-05-11T12:13:03.183854+0000","last_fullsized":"2024-05-11T12:13:03.183854+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:39:03.473134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.155659+0000","last_change":"2024-05-11T12:13:03.145731+0000","last_active":"2024-05-11T12:13:03.155659+0000","last_peered":"2024-05-11T12:13:03.155659+0000","last_clean":"2024-05-11T12:13:03.155659+0000","last_became_active":"2024-05-11T12:13:03.145419+0000","last_became_peered":"2024-05-11T12:13:03.145419+0000","last_unstale":"2024-05-11T12:13:03.155659+0000","last_undegraded":"2024-05-11T12:13:03.155659+0000","last_fullsized":"2024-05-11T12:13:03.155659+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:55:33.337983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.144327+0000","last_change":"2024-05-11T12:13:02.118817+0000","last_active":"2024-05-11T12:13:03.144327+0000","last_peered":"2024-05-11T12:13:03.144327+0000","last_clean":"2024-05-11T12:13:03.144327+0000","last_became_active":"2024-05-11T12:13:02.118040+0000","last_became_peered":"2024-05-11T12:13:02.118040+0000","last_unstale":"2024-05-11T12:13:03.144327+0000","last_undegraded":"2024-05-11T12:13:03.144327+0000","last_fullsized":"2024-05-11T12:13:03.144327+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:18:34.044830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.175410+0000","last_change":"2024-05-11T12:13:03.172604+0000","last_active":"2024-05-11T12:13:03.175410+0000","last_peered":"2024-05-11T12:13:03.175410+0000","last_clean":"2024-05-11T12:13:03.175410+0000","last_became_active":"2024-05-11T12:13:03.172198+0000","last_became_peered":"2024-05-11T12:13:03.172198+0000","last_unstale":"2024-05-11T12:13:03.175410+0000","last_undegraded":"2024-05-11T12:13:03.175410+0000","last_fullsized":"2024-05-11T12:13:03.175410+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:56:18.568549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.156874+0000","last_change":"2024-05-11T12:13:02.138604+0000","last_active":"2024-05-11T12:13:03.156874+0000","last_peered":"2024-05-11T12:13:03.156874+0000","last_clean":"2024-05-11T12:13:03.156874+0000","last_became_active":"2024-05-11T12:13:02.138227+0000","last_became_peered":"2024-05-11T12:13:02.138227+0000","last_unstale":"2024-05-11T12:13:03.156874+0000","last_undegraded":"2024-05-11T12:13:03.156874+0000","last_fullsized":"2024-05-11T12:13:03.156874+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:15:28.704713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.101871+0000","last_change":"2024-05-11T12:13:02.101871+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.101871+0000","last_undegraded":"2024-05-11T12:13:02.101871+0000","last_fullsized":"2024-05-11T12:13:02.101871+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"63'8","reported_seq":20,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.186047+0000","last_change":"2024-05-11T12:13:02.137326+0000","last_active":"2024-05-11T12:13:03.186047+0000","last_peered":"2024-05-11T12:13:03.186047+0000","last_clean":"2024-05-11T12:13:03.186047+0000","last_became_active":"2024-05-11T12:13:02.133763+0000","last_became_peered":"2024-05-11T12:13:02.133763+0000","last_unstale":"2024-05-11T12:13:03.186047+0000","last_undegraded":"2024-05-11T12:13:03.186047+0000","last_fullsized":"2024-05-11T12:13:03.186047+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:55:40.188834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.169808+0000","last_change":"2024-05-11T12:13:03.163731+0000","last_active":"2024-05-11T12:13:03.169808+0000","last_peered":"2024-05-11T12:13:03.169808+0000","last_clean":"2024-05-11T12:13:03.169808+0000","last_became_active":"2024-05-11T12:13:03.161280+0000","last_became_peered":"2024-05-11T12:13:03.161280+0000","last_unstale":"2024-05-11T12:13:03.169808+0000","last_undegraded":"2024-05-11T12:13:03.169808+0000","last_fullsized":"2024-05-11T12:13:03.169808+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:58:08.814828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"63'1","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.180608+0000","last_change":"2024-05-11T12:13:02.124463+0000","last_active":"2024-05-11T12:13:03.180608+0000","last_peered":"2024-05-11T12:13:03.180608+0000","last_clean":"2024-05-11T12:13:03.180608+0000","last_became_active":"2024-05-11T12:13:02.124136+0000","last_became_peered":"2024-05-11T12:13:02.124136+0000","last_unstale":"2024-05-11T12:13:03.180608+0000","last_undegraded":"2024-05-11T12:13:03.180608+0000","last_fullsized":"2024-05-11T12:13:03.180608+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:48:52.177182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.161587+0000","last_change":"2024-05-11T12:13:03.158635+0000","last_active":"2024-05-11T12:13:03.161587+0000","last_peered":"2024-05-11T12:13:03.161587+0000","last_clean":"2024-05-11T12:13:03.161587+0000","last_became_active":"2024-05-11T12:13:03.158311+0000","last_became_peered":"2024-05-11T12:13:03.158311+0000","last_unstale":"2024-05-11T12:13:03.161587+0000","last_undegraded":"2024-05-11T12:13:03.161587+0000","last_fullsized":"2024-05-11T12:13:03.161587+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:46:22.393593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"63'2","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.182624+0000","last_change":"2024-05-11T12:13:02.143315+0000","last_active":"2024-05-11T12:13:03.182624+0000","last_peered":"2024-05-11T12:13:03.182624+0000","last_clean":"2024-05-11T12:13:03.182624+0000","last_became_active":"2024-05-11T12:13:02.143012+0000","last_became_peered":"2024-05-11T12:13:02.143012+0000","last_unstale":"2024-05-11T12:13:03.182624+0000","last_undegraded":"2024-05-11T12:13:03.182624+0000","last_fullsized":"2024-05-11T12:13:03.182624+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:03:26.831522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.172645+0000","last_change":"2024-05-11T12:13:03.159923+0000","last_active":"2024-05-11T12:13:03.172645+0000","last_peered":"2024-05-11T12:13:03.172645+0000","last_clean":"2024-05-11T12:13:03.172645+0000","last_became_active":"2024-05-11T12:13:03.159119+0000","last_became_peered":"2024-05-11T12:13:03.159119+0000","last_unstale":"2024-05-11T12:13:03.172645+0000","last_undegraded":"2024-05-11T12:13:03.172645+0000","last_fullsized":"2024-05-11T12:13:03.172645+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:11:31.058373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"63'1","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.185684+0000","last_change":"2024-05-11T12:13:02.138872+0000","last_active":"2024-05-11T12:13:03.185684+0000","last_peered":"2024-05-11T12:13:03.185684+0000","last_clean":"2024-05-11T12:13:03.185684+0000","last_became_active":"2024-05-11T12:13:02.138589+0000","last_became_peered":"2024-05-11T12:13:02.138589+0000","last_unstale":"2024-05-11T12:13:03.185684+0000","last_undegraded":"2024-05-11T12:13:03.185684+0000","last_fullsized":"2024-05-11T12:13:03.185684+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:48:52.177182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.174205+0000","last_change":"2024-05-11T12:13:03.167395+0000","last_active":"2024-05-11T12:13:03.174205+0000","last_peered":"2024-05-11T12:13:03.174205+0000","last_clean":"2024-05-11T12:13:03.174205+0000","last_became_active":"2024-05-11T12:13:03.167111+0000","last_became_peered":"2024-05-11T12:13:03.167111+0000","last_unstale":"2024-05-11T12:13:03.174205+0000","last_undegraded":"2024-05-11T12:13:03.174205+0000","last_fullsized":"2024-05-11T12:13:03.174205+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:20:41.663583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.141325+0000","last_change":"2024-05-11T12:13:02.137581+0000","last_active":"2024-05-11T12:13:02.141325+0000","last_peered":"2024-05-11T12:13:02.141325+0000","last_clean":"2024-05-11T12:13:02.141325+0000","last_became_active":"2024-05-11T12:13:02.137260+0000","last_became_peered":"2024-05-11T12:13:02.137260+0000","last_unstale":"2024-05-11T12:13:02.141325+0000","last_undegraded":"2024-05-11T12:13:02.141325+0000","last_fullsized":"2024-05-11T12:13:02.141325+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:29:57.084718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.120434+0000","last_change":"2024-05-11T12:13:02.120434+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.120434+0000","last_undegraded":"2024-05-11T12:13:02.120434+0000","last_fullsized":"2024-05-11T12:13:02.120434+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.143793+0000","last_change":"2024-05-11T12:13:02.140367+0000","last_active":"2024-05-11T12:13:02.143793+0000","last_peered":"2024-05-11T12:13:02.143793+0000","last_clean":"2024-05-11T12:13:02.143793+0000","last_became_active":"2024-05-11T12:13:02.140069+0000","last_became_peered":"2024-05-11T12:13:02.140069+0000","last_unstale":"2024-05-11T12:13:02.143793+0000","last_undegraded":"2024-05-11T12:13:02.143793+0000","last_fullsized":"2024-05-11T12:13:02.143793+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:24:39.427445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.175442+0000","last_change":"2024-05-11T12:13:03.171936+0000","last_active":"2024-05-11T12:13:03.175442+0000","last_peered":"2024-05-11T12:13:03.175442+0000","last_clean":"2024-05-11T12:13:03.175442+0000","last_became_active":"2024-05-11T12:13:03.171669+0000","last_became_peered":"2024-05-11T12:13:03.171669+0000","last_unstale":"2024-05-11T12:13:03.175442+0000","last_undegraded":"2024-05-11T12:13:03.175442+0000","last_fullsized":"2024-05-11T12:13:03.175442+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:29:58.089646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.144512+0000","last_change":"2024-05-11T12:13:02.125827+0000","last_active":"2024-05-11T12:13:03.144512+0000","last_peered":"2024-05-11T12:13:03.144512+0000","last_clean":"2024-05-11T12:13:03.144512+0000","last_became_active":"2024-05-11T12:13:02.125520+0000","last_became_peered":"2024-05-11T12:13:02.125520+0000","last_unstale":"2024-05-11T12:13:03.144512+0000","last_undegraded":"2024-05-11T12:13:03.144512+0000","last_fullsized":"2024-05-11T12:13:03.144512+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:00:08.081754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.174385+0000","last_change":"2024-05-11T12:13:03.168791+0000","last_active":"2024-05-11T12:13:03.174385+0000","last_peered":"2024-05-11T12:13:03.174385+0000","last_clean":"2024-05-11T12:13:03.174385+0000","last_became_active":"2024-05-11T12:13:03.168255+0000","last_became_peered":"2024-05-11T12:13:03.168255+0000","last_unstale":"2024-05-11T12:13:03.174385+0000","last_undegraded":"2024-05-11T12:13:03.174385+0000","last_fullsized":"2024-05-11T12:13:03.174385+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:17:19.051848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.141028+0000","last_change":"2024-05-11T12:13:02.136770+0000","last_active":"2024-05-11T12:13:02.141028+0000","last_peered":"2024-05-11T12:13:02.141028+0000","last_clean":"2024-05-11T12:13:02.141028+0000","last_became_active":"2024-05-11T12:13:02.136465+0000","last_became_peered":"2024-05-11T12:13:02.136465+0000","last_unstale":"2024-05-11T12:13:02.141028+0000","last_undegraded":"2024-05-11T12:13:02.141028+0000","last_fullsized":"2024-05-11T12:13:02.141028+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:08:15.133910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.120491+0000","last_change":"2024-05-11T12:13:02.120491+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.120491+0000","last_undegraded":"2024-05-11T12:13:02.120491+0000","last_fullsized":"2024-05-11T12:13:02.120491+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"63'2","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.183009+0000","last_change":"2024-05-11T12:13:02.136984+0000","last_active":"2024-05-11T12:13:03.183009+0000","last_peered":"2024-05-11T12:13:03.183009+0000","last_clean":"2024-05-11T12:13:03.183009+0000","last_became_active":"2024-05-11T12:13:02.135837+0000","last_became_peered":"2024-05-11T12:13:02.135837+0000","last_unstale":"2024-05-11T12:13:03.183009+0000","last_undegraded":"2024-05-11T12:13:03.183009+0000","last_fullsized":"2024-05-11T12:13:03.183009+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:03:26.831522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.173813+0000","last_change":"2024-05-11T12:13:03.170696+0000","last_active":"2024-05-11T12:13:04.173813+0000","last_peered":"2024-05-11T12:13:04.173813+0000","last_clean":"2024-05-11T12:13:04.173813+0000","last_became_active":"2024-05-11T12:13:03.170411+0000","last_became_peered":"2024-05-11T12:13:03.170411+0000","last_unstale":"2024-05-11T12:13:04.173813+0000","last_undegraded":"2024-05-11T12:13:04.173813+0000","last_fullsized":"2024-05-11T12:13:04.173813+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:11:05.631699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.146050+0000","last_change":"2024-05-11T12:13:02.128082+0000","last_active":"2024-05-11T12:13:03.146050+0000","last_peered":"2024-05-11T12:13:03.146050+0000","last_clean":"2024-05-11T12:13:03.146050+0000","last_became_active":"2024-05-11T12:13:02.126292+0000","last_became_peered":"2024-05-11T12:13:02.126292+0000","last_unstale":"2024-05-11T12:13:03.146050+0000","last_undegraded":"2024-05-11T12:13:03.146050+0000","last_fullsized":"2024-05-11T12:13:03.146050+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:15:28.704713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.156288+0000","last_change":"2024-05-11T12:13:03.149098+0000","last_active":"2024-05-11T12:13:03.156288+0000","last_peered":"2024-05-11T12:13:03.156288+0000","last_clean":"2024-05-11T12:13:03.156288+0000","last_became_active":"2024-05-11T12:13:03.148358+0000","last_became_peered":"2024-05-11T12:13:03.148358+0000","last_unstale":"2024-05-11T12:13:03.156288+0000","last_undegraded":"2024-05-11T12:13:03.156288+0000","last_fullsized":"2024-05-11T12:13:03.156288+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:12:30.081833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.137459+0000","last_change":"2024-05-11T12:13:02.129624+0000","last_active":"2024-05-11T12:13:02.137459+0000","last_peered":"2024-05-11T12:13:02.137459+0000","last_clean":"2024-05-11T12:13:02.137459+0000","last_became_active":"2024-05-11T12:13:02.129286+0000","last_became_peered":"2024-05-11T12:13:02.129286+0000","last_unstale":"2024-05-11T12:13:02.137459+0000","last_undegraded":"2024-05-11T12:13:02.137459+0000","last_fullsized":"2024-05-11T12:13:02.137459+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:12:29.076905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.174228+0000","last_change":"2024-05-11T12:13:03.170501+0000","last_active":"2024-05-11T12:13:03.174228+0000","last_peered":"2024-05-11T12:13:03.174228+0000","last_clean":"2024-05-11T12:13:03.174228+0000","last_became_active":"2024-05-11T12:13:03.163918+0000","last_became_peered":"2024-05-11T12:13:03.163918+0000","last_unstale":"2024-05-11T12:13:03.174228+0000","last_undegraded":"2024-05-11T12:13:03.174228+0000","last_fullsized":"2024-05-11T12:13:03.174228+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:18:50.744350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.137839+0000","last_change":"2024-05-11T12:13:02.133532+0000","last_active":"2024-05-11T12:13:02.137839+0000","last_peered":"2024-05-11T12:13:02.137839+0000","last_clean":"2024-05-11T12:13:02.137839+0000","last_became_active":"2024-05-11T12:13:02.133209+0000","last_became_peered":"2024-05-11T12:13:02.133209+0000","last_unstale":"2024-05-11T12:13:02.137839+0000","last_undegraded":"2024-05-11T12:13:02.137839+0000","last_fullsized":"2024-05-11T12:13:02.137839+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:52:38.362358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.172256+0000","last_change":"2024-05-11T12:13:03.163919+0000","last_active":"2024-05-11T12:13:03.172256+0000","last_peered":"2024-05-11T12:13:03.172256+0000","last_clean":"2024-05-11T12:13:03.172256+0000","last_became_active":"2024-05-11T12:13:03.163595+0000","last_became_peered":"2024-05-11T12:13:03.163595+0000","last_unstale":"2024-05-11T12:13:03.172256+0000","last_undegraded":"2024-05-11T12:13:03.172256+0000","last_fullsized":"2024-05-11T12:13:03.172256+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:52:39.367286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.152331+0000","last_change":"2024-05-11T12:13:02.139859+0000","last_active":"2024-05-11T12:13:03.152331+0000","last_peered":"2024-05-11T12:13:03.152331+0000","last_clean":"2024-05-11T12:13:03.152331+0000","last_became_active":"2024-05-11T12:13:02.139265+0000","last_became_peered":"2024-05-11T12:13:02.139265+0000","last_unstale":"2024-05-11T12:13:03.152331+0000","last_undegraded":"2024-05-11T12:13:03.152331+0000","last_fullsized":"2024-05-11T12:13:03.152331+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:18:34.044830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.172282+0000","last_change":"2024-05-11T12:13:03.163034+0000","last_active":"2024-05-11T12:13:03.172282+0000","last_peered":"2024-05-11T12:13:03.172282+0000","last_clean":"2024-05-11T12:13:03.172282+0000","last_became_active":"2024-05-11T12:13:03.160775+0000","last_became_peered":"2024-05-11T12:13:03.160775+0000","last_unstale":"2024-05-11T12:13:03.172282+0000","last_undegraded":"2024-05-11T12:13:03.172282+0000","last_fullsized":"2024-05-11T12:13:03.172282+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:08:51.947443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.117231+0000","last_change":"2024-05-11T12:13:02.113452+0000","last_active":"2024-05-11T12:13:02.117231+0000","last_peered":"2024-05-11T12:13:02.117231+0000","last_clean":"2024-05-11T12:13:02.117231+0000","last_became_active":"2024-05-11T12:13:02.112863+0000","last_became_peered":"2024-05-11T12:13:02.112863+0000","last_unstale":"2024-05-11T12:13:02.117231+0000","last_undegraded":"2024-05-11T12:13:02.117231+0000","last_fullsized":"2024-05-11T12:13:02.117231+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:08:15.133910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.102609+0000","last_change":"2024-05-11T12:13:02.102609+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.102609+0000","last_undegraded":"2024-05-11T12:13:02.102609+0000","last_fullsized":"2024-05-11T12:13:02.102609+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.137700+0000","last_change":"2024-05-11T12:13:02.133595+0000","last_active":"2024-05-11T12:13:02.137700+0000","last_peered":"2024-05-11T12:13:02.137700+0000","last_clean":"2024-05-11T12:13:02.137700+0000","last_became_active":"2024-05-11T12:13:02.133168+0000","last_became_peered":"2024-05-11T12:13:02.133168+0000","last_unstale":"2024-05-11T12:13:02.137700+0000","last_undegraded":"2024-05-11T12:13:02.137700+0000","last_fullsized":"2024-05-11T12:13:02.137700+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:27:13.189890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.173843+0000","last_change":"2024-05-11T12:13:03.169252+0000","last_active":"2024-05-11T12:13:04.173843+0000","last_peered":"2024-05-11T12:13:04.173843+0000","last_clean":"2024-05-11T12:13:04.173843+0000","last_became_active":"2024-05-11T12:13:03.167540+0000","last_became_peered":"2024-05-11T12:13:03.167540+0000","last_unstale":"2024-05-11T12:13:04.173843+0000","last_undegraded":"2024-05-11T12:13:04.173843+0000","last_fullsized":"2024-05-11T12:13:04.173843+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:38:22.482431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.144252+0000","last_change":"2024-05-11T12:13:02.140835+0000","last_active":"2024-05-11T12:13:02.144252+0000","last_peered":"2024-05-11T12:13:02.144252+0000","last_clean":"2024-05-11T12:13:02.144252+0000","last_became_active":"2024-05-11T12:13:02.139971+0000","last_became_peered":"2024-05-11T12:13:02.139971+0000","last_unstale":"2024-05-11T12:13:02.144252+0000","last_undegraded":"2024-05-11T12:13:02.144252+0000","last_fullsized":"2024-05-11T12:13:02.144252+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:06:39.713684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.172847+0000","last_change":"2024-05-11T12:13:03.166247+0000","last_active":"2024-05-11T12:13:03.172847+0000","last_peered":"2024-05-11T12:13:03.172847+0000","last_clean":"2024-05-11T12:13:03.172847+0000","last_became_active":"2024-05-11T12:13:03.165118+0000","last_became_peered":"2024-05-11T12:13:03.165118+0000","last_unstale":"2024-05-11T12:13:03.172847+0000","last_undegraded":"2024-05-11T12:13:03.172847+0000","last_fullsized":"2024-05-11T12:13:03.172847+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:32:45.994907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.125060+0000","last_change":"2024-05-11T12:13:02.116771+0000","last_active":"2024-05-11T12:13:02.125060+0000","last_peered":"2024-05-11T12:13:02.125060+0000","last_clean":"2024-05-11T12:13:02.125060+0000","last_became_active":"2024-05-11T12:13:02.116028+0000","last_became_peered":"2024-05-11T12:13:02.116028+0000","last_unstale":"2024-05-11T12:13:02.125060+0000","last_undegraded":"2024-05-11T12:13:02.125060+0000","last_fullsized":"2024-05-11T12:13:02.125060+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:24:39.427445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":183,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.152992+0000","last_change":"2024-05-11T12:10:35.066673+0000","last_active":"2024-05-11T12:13:03.152992+0000","last_peered":"2024-05-11T12:13:03.152992+0000","last_clean":"2024-05-11T12:13:03.152992+0000","last_became_active":"2024-05-11T12:10:35.065942+0000","last_became_peered":"2024-05-11T12:10:35.065942+0000","last_unstale":"2024-05-11T12:13:03.152992+0000","last_undegraded":"2024-05-11T12:13:03.152992+0000","last_fullsized":"2024-05-11T12:13:03.152992+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_clean_scrub_stamp":"2024-05-11T12:10:33.601136+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:12:02.133182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19,"ondisk_log_size":19,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476769,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28320,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343520,"statfs":{"total":96636764160,"available":96607764480,"internally_reserved":0,"allocated":1331200,"data_stored":916169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476770,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27740,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344100,"statfs":{"total":96636764160,"available":96608358400,"internally_reserved":0,"allocated":737280,"data_stored":325801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476769,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":737280,"data_stored":325801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476771,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":737280,"data_stored":325801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509474,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27732,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344108,"statfs":{"total":96636764160,"available":96608366592,"internally_reserved":0,"allocated":724992,"data_stored":319142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509475,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":724992,"data_stored":319142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509474,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28332,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343508,"statfs":{"total":96636764160,"available":96607752192,"internally_reserved":0,"allocated":1331200,"data_stored":916169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542179,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":761856,"data_stored":339660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T12:13:07.620 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T12:13:07.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.861+0000 7f5e4a800640 1 Processor -- start 2024-05-11T12:13:07.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.861+0000 7f5e4a800640 1 -- start start 2024-05-11T12:13:07.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.863+0000 7f5e4a800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e44157750 0x7f5e4415d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:07.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.863+0000 7f5e4a800640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e4415dc80 con 0x7f5e4406d460 2024-05-11T12:13:07.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.863+0000 7f5e4a800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e4415df30 con 0x7f5e440b3ea0 2024-05-11T12:13:07.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.863+0000 7f5e4a800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5e44161000 con 0x7f5e44157750 2024-05-11T12:13:07.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.863+0000 7f5e43e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f5e4406d460 0x7f5e440cf1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:33336/0 (socket says 172.21.15.167:33336) 2024-05-11T12:13:07.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.863+0000 7f5e43e00640 1 -- 172.21.15.167:0/3778221337 learned_addr learned my addr 172.21.15.167:0/3778221337 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:13:07.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.864+0000 7f5e43400640 1 --2- 172.21.15.167:0/3778221337 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e44157750 0x7f5e4415d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:07.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.864+0000 7f5e42a00640 1 -- 172.21.15.167:0/3778221337 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1477692112 0 0) 0x7f5e4415dc80 con 0x7f5e4406d460 2024-05-11T12:13:07.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.864+0000 7f5e42a00640 1 -- 172.21.15.167:0/3778221337 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e28003520 con 0x7f5e4406d460 2024-05-11T12:13:07.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.864+0000 7f5e42a00640 1 -- 172.21.15.167:0/3778221337 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1967977033 0 0) 0x7f5e28003520 con 0x7f5e4406d460 2024-05-11T12:13:07.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.864+0000 7f5e42a00640 1 -- 172.21.15.167:0/3778221337 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e44157750 msgr2=0x7f5e4415d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:07.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.864+0000 7f5e42a00640 1 --2- 172.21.15.167:0/3778221337 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e44157750 0x7f5e4415d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:07.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.864+0000 7f5e42a00640 1 -- 172.21.15.167:0/3778221337 >> v1:172.21.15.204:6789/0 conn(0x7f5e440b3ea0 legacy=0x7f5e44161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:07.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.864+0000 7f5e42a00640 1 -- 172.21.15.167:0/3778221337 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e4415e210 con 0x7f5e4406d460 2024-05-11T12:13:07.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.864+0000 7f5e42a00640 1 -- 172.21.15.167:0/3778221337 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f5e3c001a70 con 0x7f5e4406d460 2024-05-11T12:13:07.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.864+0000 7f5e42a00640 1 -- 172.21.15.167:0/3778221337 >> v1:172.21.15.167:6789/0 conn(0x7f5e4406d460 legacy=0x7f5e440cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:07.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.864+0000 7f5e42a00640 1 --2- 172.21.15.167:0/3778221337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e28003a30 0x7f5e28003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:07.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.865+0000 7f5e42a00640 1 --2- 172.21.15.167:0/3778221337 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e44157750 0x7f5e28004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:07.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.865+0000 7f5e42a00640 1 --2- 172.21.15.167:0/3778221337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e28004d40 0x7f5e280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:07.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.865+0000 7f5e42a00640 1 -- 172.21.15.167:0/3778221337 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5e44161000 con 0x7f5e28003a30 2024-05-11T12:13:07.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.865+0000 7f5e42a00640 1 -- 172.21.15.167:0/3778221337 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2800baf0 con 0x7f5e28004d40 2024-05-11T12:13:07.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.865+0000 7f5e42a00640 1 -- 172.21.15.167:0/3778221337 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5e2800bcf0 con 0x7f5e44157750 2024-05-11T12:13:07.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.865+0000 7f5e43400640 1 --2- 172.21.15.167:0/3778221337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e28003a30 0x7f5e28003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:07.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.865+0000 7f5e3be00640 1 --2- 172.21.15.167:0/3778221337 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e44157750 0x7f5e28004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:07.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.865+0000 7f5e43400640 1 -- 172.21.15.167:0/3778221337 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e44157750 msgr2=0x7f5e28004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:07.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.865+0000 7f5e43400640 1 --2- 172.21.15.167:0/3778221337 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e44157750 0x7f5e28004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:07.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.865+0000 7f5e43400640 1 -- 172.21.15.167:0/3778221337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e28004d40 msgr2=0x7f5e280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:07.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.865+0000 7f5e43400640 1 --2- 172.21.15.167:0/3778221337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e28004d40 0x7f5e280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:07.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.865+0000 7f5e43400640 1 -- 172.21.15.167:0/3778221337 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e28003520 con 0x7f5e28003a30 2024-05-11T12:13:07.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.866+0000 7f5e43400640 1 --2- 172.21.15.167:0/3778221337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e28003a30 0x7f5e28003e50 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5e34009ac0 tx=0x7f5e28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=441ae431f28cd8e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:07.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.867+0000 7f5e42a00640 1 -- 172.21.15.167:0/3778221337 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e34036070 con 0x7f5e28003a30 2024-05-11T12:13:07.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.867+0000 7f5e42a00640 1 -- 172.21.15.167:0/3778221337 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e34031400 con 0x7f5e28003a30 2024-05-11T12:13:07.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.867+0000 7f5e42a00640 1 -- 172.21.15.167:0/3778221337 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e34009690 con 0x7f5e28003a30 2024-05-11T12:13:07.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.867+0000 7f5e4a800640 1 -- 172.21.15.167:0/3778221337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e28003a30 msgr2=0x7f5e28003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:07.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.867+0000 7f5e4a800640 1 --2- 172.21.15.167:0/3778221337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e28003a30 0x7f5e28003e50 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5e34009ac0 tx=0x7f5e28005830 comp rx=0 tx=0).stop 2024-05-11T12:13:07.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.867+0000 7f5e43e00640 1 -- 172.21.15.167:0/3778221337 reap_dead start 2024-05-11T12:13:07.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.867+0000 7f5e4a800640 1 -- 172.21.15.167:0/3778221337 shutdown_connections 2024-05-11T12:13:07.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.867+0000 7f5e4a800640 1 -- 172.21.15.167:0/3778221337 >> 172.21.15.167:0/3778221337 conn(0x7f5e440cd8f0 msgr2=0x7f5e440cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:07.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.867+0000 7f5e4a800640 1 -- 172.21.15.167:0/3778221337 shutdown_connections 2024-05-11T12:13:07.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.867+0000 7f5e4a800640 1 -- 172.21.15.167:0/3778221337 wait complete. 2024-05-11T12:13:07.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.869+0000 7f5e4a800640 1 Processor -- start 2024-05-11T12:13:07.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.869+0000 7f5e4a800640 1 -- start start 2024-05-11T12:13:07.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.870+0000 7f5e4a800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e28003a30 0x7f5e4413d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:07.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.870+0000 7f5e4a800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e440af200 0x7f5e4413ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:07.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.870+0000 7f5e4a800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e44072a20 0x7f5e44072ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:07.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.870+0000 7f5e4a800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5e44161ba0 con 0x7f5e440af200 2024-05-11T12:13:07.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.870+0000 7f5e4a800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5e44157b30 con 0x7f5e28003a30 2024-05-11T12:13:07.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.870+0000 7f5e4a800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5e440cf8d0 con 0x7f5e44072a20 2024-05-11T12:13:07.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.870+0000 7f5e3be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e440af200 0x7f5e4413ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:07.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.870+0000 7f5e3be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e440af200 0x7f5e4413ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43380/0 (socket says 172.21.15.167:43380) 2024-05-11T12:13:07.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.870+0000 7f5e3be00640 1 -- 172.21.15.167:0/143620286 learned_addr learned my addr 172.21.15.167:0/143620286 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:07.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.871+0000 7f5e43e00640 1 --2- 172.21.15.167:0/143620286 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e44072a20 0x7f5e44072ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:07.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.871+0000 7f5e3be00640 1 -- 172.21.15.167:0/143620286 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e44072a20 msgr2=0x7f5e44072ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:07.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.871+0000 7f5e3be00640 1 --2- 172.21.15.167:0/143620286 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e44072a20 0x7f5e44072ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:07.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.871+0000 7f5e3be00640 1 -- 172.21.15.167:0/143620286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e28003a30 msgr2=0x7f5e4413d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:07.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.871+0000 7f5e43400640 1 --2- 172.21.15.167:0/143620286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e28003a30 0x7f5e4413d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:07.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.871+0000 7f5e3be00640 1 --2- 172.21.15.167:0/143620286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e28003a30 0x7f5e4413d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:07.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.871+0000 7f5e3be00640 1 -- 172.21.15.167:0/143620286 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e440af680 con 0x7f5e440af200 2024-05-11T12:13:07.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.871+0000 7f5e3be00640 1 --2- 172.21.15.167:0/143620286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e440af200 0x7f5e4413ca20 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c00d7d0 tx=0x7f5e2c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:07.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.871+0000 7f5e3b400640 1 -- 172.21.15.167:0/143620286 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c005270 con 0x7f5e440af200 2024-05-11T12:13:07.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.871+0000 7f5e3b400640 1 -- 172.21.15.167:0/143620286 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e2c017070 con 0x7f5e440af200 2024-05-11T12:13:07.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.871+0000 7f5e3b400640 1 -- 172.21.15.167:0/143620286 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c00b6b0 con 0x7f5e440af200 2024-05-11T12:13:07.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.872+0000 7f5e4a800640 1 -- 172.21.15.167:0/143620286 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e440af930 con 0x7f5e440af200 2024-05-11T12:13:07.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.872+0000 7f5e4a800640 1 -- 172.21.15.167:0/143620286 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e440afef0 con 0x7f5e440af200 2024-05-11T12:13:07.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.873+0000 7f5e4a800640 1 -- 172.21.15.167:0/143620286 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e44069590 con 0x7f5e440af200 2024-05-11T12:13:07.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.874+0000 7f5e3b400640 1 -- 172.21.15.167:0/143620286 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5e2c00b850 con 0x7f5e440af200 2024-05-11T12:13:07.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.875+0000 7f5e3b400640 1 --2- 172.21.15.167:0/143620286 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5e200762d0 0x7f5e20078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:07.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.875+0000 7f5e3b400640 1 -- 172.21.15.167:0/143620286 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f5e2c097630 con 0x7f5e440af200 2024-05-11T12:13:07.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.876+0000 7f5e43400640 1 --2- 172.21.15.167:0/143620286 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5e200762d0 0x7f5e20078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:07.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.876+0000 7f5e43400640 1 --2- 172.21.15.167:0/143620286 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5e200762d0 0x7f5e20078790 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5e34033000 tx=0x7f5e34008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:07.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:07.880+0000 7f5e3b400640 1 -- 172.21.15.167:0/143620286 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5e2c060e00 con 0x7f5e440af200 2024-05-11T12:13:08.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.084+0000 7f5e4a800640 1 -- 172.21.15.167:0/143620286 --> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5e440b00f0 con 0x7f5e200762d0 2024-05-11T12:13:08.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.092+0000 7f5e3b400640 1 -- 172.21.15.167:0/143620286 <== mgr.4106 v2:172.21.15.167:6832/578352559 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250460 (secure 0 0 0) 0x7f5e440b00f0 con 0x7f5e200762d0 2024-05-11T12:13:08.095 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:13:08.096 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2024-05-11T12:13:08.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.097+0000 7f5e4a800640 1 -- 172.21.15.167:0/143620286 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5e200762d0 msgr2=0x7f5e20078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:08.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.097+0000 7f5e4a800640 1 --2- 172.21.15.167:0/143620286 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5e200762d0 0x7f5e20078790 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5e34033000 tx=0x7f5e34008030 comp rx=0 tx=0).stop 2024-05-11T12:13:08.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.097+0000 7f5e4a800640 1 -- 172.21.15.167:0/143620286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e440af200 msgr2=0x7f5e4413ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:08.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.097+0000 7f5e4a800640 1 --2- 172.21.15.167:0/143620286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e440af200 0x7f5e4413ca20 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c00d7d0 tx=0x7f5e2c00dc90 comp rx=0 tx=0).stop 2024-05-11T12:13:08.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.098+0000 7f5e4a800640 1 -- 172.21.15.167:0/143620286 shutdown_connections 2024-05-11T12:13:08.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.098+0000 7f5e4a800640 1 --2- 172.21.15.167:0/143620286 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e44072a20 0x7f5e44072ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:08.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.098+0000 7f5e4a800640 1 --2- 172.21.15.167:0/143620286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e440af200 0x7f5e4413ca20 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:08.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.098+0000 7f5e4a800640 1 --2- 172.21.15.167:0/143620286 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5e200762d0 0x7f5e20078790 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:08.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.098+0000 7f5e4a800640 1 --2- 172.21.15.167:0/143620286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e28003a30 0x7f5e4413d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:08.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.098+0000 7f5e4a800640 1 -- 172.21.15.167:0/143620286 >> 172.21.15.167:0/143620286 conn(0x7f5e440cd8f0 msgr2=0x7f5e440b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:08.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.098+0000 7f5e4a800640 1 -- 172.21.15.167:0/143620286 shutdown_connections 2024-05-11T12:13:08.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.098+0000 7f5e4a800640 1 -- 172.21.15.167:0/143620286 wait complete. 2024-05-11T12:13:08.118 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":135,"stamp":"2024-05-11T12:13:07.279017+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590584,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":128,"num_write_kb":1638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91,"ondisk_log_size":91,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":93,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223108,"kb_used_data":6920,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751612,"statfs":{"total":773094113280,"available":772865650688,"internally_reserved":0,"allocated":7086080,"data_stored":3787685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001654"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.104414+0000","last_change":"2024-05-11T12:13:02.104414+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.104414+0000","last_undegraded":"2024-05-11T12:13:02.104414+0000","last_fullsized":"2024-05-11T12:13:02.104414+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.102208+0000","last_change":"2024-05-11T12:13:02.102208+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.102208+0000","last_undegraded":"2024-05-11T12:13:02.102208+0000","last_fullsized":"2024-05-11T12:13:02.102208+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.180687+0000","last_change":"2024-05-11T12:13:03.165899+0000","last_active":"2024-05-11T12:13:03.180687+0000","last_peered":"2024-05-11T12:13:03.180687+0000","last_clean":"2024-05-11T12:13:03.180687+0000","last_became_active":"2024-05-11T12:13:03.165494+0000","last_became_peered":"2024-05-11T12:13:03.165494+0000","last_unstale":"2024-05-11T12:13:03.180687+0000","last_undegraded":"2024-05-11T12:13:03.180687+0000","last_fullsized":"2024-05-11T12:13:03.180687+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:04:05.997706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.174472+0000","last_change":"2024-05-11T12:13:03.166762+0000","last_active":"2024-05-11T12:13:03.174472+0000","last_peered":"2024-05-11T12:13:03.174472+0000","last_clean":"2024-05-11T12:13:03.174472+0000","last_became_active":"2024-05-11T12:13:03.166457+0000","last_became_peered":"2024-05-11T12:13:03.166457+0000","last_unstale":"2024-05-11T12:13:03.174472+0000","last_undegraded":"2024-05-11T12:13:03.174472+0000","last_fullsized":"2024-05-11T12:13:03.174472+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:46:22.393593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.173955+0000","last_change":"2024-05-11T12:13:03.167675+0000","last_active":"2024-05-11T12:13:03.173955+0000","last_peered":"2024-05-11T12:13:03.173955+0000","last_clean":"2024-05-11T12:13:03.173955+0000","last_became_active":"2024-05-11T12:13:03.167381+0000","last_became_peered":"2024-05-11T12:13:03.167381+0000","last_unstale":"2024-05-11T12:13:03.173955+0000","last_undegraded":"2024-05-11T12:13:03.173955+0000","last_fullsized":"2024-05-11T12:13:03.173955+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:06:40.718612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.121564+0000","last_change":"2024-05-11T12:13:02.121564+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.121564+0000","last_undegraded":"2024-05-11T12:13:02.121564+0000","last_fullsized":"2024-05-11T12:13:02.121564+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.173969+0000","last_change":"2024-05-11T12:13:03.170170+0000","last_active":"2024-05-11T12:13:04.173969+0000","last_peered":"2024-05-11T12:13:04.173969+0000","last_clean":"2024-05-11T12:13:04.173969+0000","last_became_active":"2024-05-11T12:13:03.169888+0000","last_became_peered":"2024-05-11T12:13:03.169888+0000","last_unstale":"2024-05-11T12:13:04.173969+0000","last_undegraded":"2024-05-11T12:13:04.173969+0000","last_fullsized":"2024-05-11T12:13:04.173969+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:46:12.686412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.174546+0000","last_change":"2024-05-11T12:13:03.167147+0000","last_active":"2024-05-11T12:13:03.174546+0000","last_peered":"2024-05-11T12:13:03.174546+0000","last_clean":"2024-05-11T12:13:03.174546+0000","last_became_active":"2024-05-11T12:13:03.166882+0000","last_became_peered":"2024-05-11T12:13:03.166882+0000","last_unstale":"2024-05-11T12:13:03.174546+0000","last_undegraded":"2024-05-11T12:13:03.174546+0000","last_fullsized":"2024-05-11T12:13:03.174546+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:04:05.997706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.180451+0000","last_change":"2024-05-11T12:13:03.162128+0000","last_active":"2024-05-11T12:13:03.180451+0000","last_peered":"2024-05-11T12:13:03.180451+0000","last_clean":"2024-05-11T12:13:03.180451+0000","last_became_active":"2024-05-11T12:13:03.161841+0000","last_became_peered":"2024-05-11T12:13:03.161841+0000","last_unstale":"2024-05-11T12:13:03.180451+0000","last_undegraded":"2024-05-11T12:13:03.180451+0000","last_fullsized":"2024-05-11T12:13:03.180451+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:27:14.194818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.169523+0000","last_change":"2024-05-11T12:13:03.164933+0000","last_active":"2024-05-11T12:13:03.169523+0000","last_peered":"2024-05-11T12:13:03.169523+0000","last_clean":"2024-05-11T12:13:03.169523+0000","last_became_active":"2024-05-11T12:13:03.164680+0000","last_became_peered":"2024-05-11T12:13:03.164680+0000","last_unstale":"2024-05-11T12:13:03.169523+0000","last_undegraded":"2024-05-11T12:13:03.169523+0000","last_fullsized":"2024-05-11T12:13:03.169523+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:08:16.138838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.174853+0000","last_change":"2024-05-11T12:13:03.163978+0000","last_active":"2024-05-11T12:13:03.174853+0000","last_peered":"2024-05-11T12:13:03.174853+0000","last_clean":"2024-05-11T12:13:03.174853+0000","last_became_active":"2024-05-11T12:13:03.163688+0000","last_became_peered":"2024-05-11T12:13:03.163688+0000","last_unstale":"2024-05-11T12:13:03.174853+0000","last_undegraded":"2024-05-11T12:13:03.174853+0000","last_fullsized":"2024-05-11T12:13:03.174853+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:15:35.851039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.170759+0000","last_change":"2024-05-11T12:13:03.164295+0000","last_active":"2024-05-11T12:13:03.170759+0000","last_peered":"2024-05-11T12:13:03.170759+0000","last_clean":"2024-05-11T12:13:03.170759+0000","last_became_active":"2024-05-11T12:13:03.163948+0000","last_became_peered":"2024-05-11T12:13:03.163948+0000","last_unstale":"2024-05-11T12:13:03.170759+0000","last_undegraded":"2024-05-11T12:13:03.170759+0000","last_fullsized":"2024-05-11T12:13:03.170759+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:24:40.432373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.170701+0000","last_change":"2024-05-11T12:13:03.163777+0000","last_active":"2024-05-11T12:13:03.170701+0000","last_peered":"2024-05-11T12:13:03.170701+0000","last_clean":"2024-05-11T12:13:03.170701+0000","last_became_active":"2024-05-11T12:13:03.163472+0000","last_became_peered":"2024-05-11T12:13:03.163472+0000","last_unstale":"2024-05-11T12:13:03.170701+0000","last_undegraded":"2024-05-11T12:13:03.170701+0000","last_fullsized":"2024-05-11T12:13:03.170701+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:58:08.814828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.169784+0000","last_change":"2024-05-11T12:13:03.165418+0000","last_active":"2024-05-11T12:13:03.169784+0000","last_peered":"2024-05-11T12:13:03.169784+0000","last_clean":"2024-05-11T12:13:03.169784+0000","last_became_active":"2024-05-11T12:13:03.165020+0000","last_became_peered":"2024-05-11T12:13:03.165020+0000","last_unstale":"2024-05-11T12:13:03.169784+0000","last_undegraded":"2024-05-11T12:13:03.169784+0000","last_fullsized":"2024-05-11T12:13:03.169784+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:24:40.432373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.182452+0000","last_change":"2024-05-11T12:13:03.174279+0000","last_active":"2024-05-11T12:13:03.182452+0000","last_peered":"2024-05-11T12:13:03.182452+0000","last_clean":"2024-05-11T12:13:03.182452+0000","last_became_active":"2024-05-11T12:13:03.173960+0000","last_became_peered":"2024-05-11T12:13:03.173960+0000","last_unstale":"2024-05-11T12:13:03.182452+0000","last_undegraded":"2024-05-11T12:13:03.182452+0000","last_fullsized":"2024-05-11T12:13:03.182452+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:29:58.089646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.173798+0000","last_change":"2024-05-11T12:13:03.176759+0000","last_active":"2024-05-11T12:13:04.173798+0000","last_peered":"2024-05-11T12:13:04.173798+0000","last_clean":"2024-05-11T12:13:04.173798+0000","last_became_active":"2024-05-11T12:13:03.176496+0000","last_became_peered":"2024-05-11T12:13:03.176496+0000","last_unstale":"2024-05-11T12:13:04.173798+0000","last_undegraded":"2024-05-11T12:13:04.173798+0000","last_fullsized":"2024-05-11T12:13:04.173798+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:33:36.274987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.103891+0000","last_change":"2024-05-11T12:13:02.103891+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.103891+0000","last_undegraded":"2024-05-11T12:13:02.103891+0000","last_fullsized":"2024-05-11T12:13:02.103891+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.175933+0000","last_change":"2024-05-11T12:13:03.167581+0000","last_active":"2024-05-11T12:13:03.175933+0000","last_peered":"2024-05-11T12:13:03.175933+0000","last_clean":"2024-05-11T12:13:03.175933+0000","last_became_active":"2024-05-11T12:13:03.167285+0000","last_became_peered":"2024-05-11T12:13:03.167285+0000","last_unstale":"2024-05-11T12:13:03.175933+0000","last_undegraded":"2024-05-11T12:13:03.175933+0000","last_fullsized":"2024-05-11T12:13:03.175933+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:52:39.367286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.175530+0000","last_change":"2024-05-11T12:13:03.172989+0000","last_active":"2024-05-11T12:13:03.175530+0000","last_peered":"2024-05-11T12:13:03.175530+0000","last_clean":"2024-05-11T12:13:03.175530+0000","last_became_active":"2024-05-11T12:13:03.172704+0000","last_became_peered":"2024-05-11T12:13:03.172704+0000","last_unstale":"2024-05-11T12:13:03.175530+0000","last_undegraded":"2024-05-11T12:13:03.175530+0000","last_fullsized":"2024-05-11T12:13:03.175530+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:28:34.473489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.116797+0000","last_change":"2024-05-11T12:13:02.116797+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.116797+0000","last_undegraded":"2024-05-11T12:13:02.116797+0000","last_fullsized":"2024-05-11T12:13:02.116797+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.170678+0000","last_change":"2024-05-11T12:13:03.162655+0000","last_active":"2024-05-11T12:13:03.170678+0000","last_peered":"2024-05-11T12:13:03.170678+0000","last_clean":"2024-05-11T12:13:03.170678+0000","last_became_active":"2024-05-11T12:13:03.162259+0000","last_became_peered":"2024-05-11T12:13:03.162259+0000","last_unstale":"2024-05-11T12:13:03.170678+0000","last_undegraded":"2024-05-11T12:13:03.170678+0000","last_fullsized":"2024-05-11T12:13:03.170678+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:42:21.959847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.116334+0000","last_change":"2024-05-11T12:13:02.116334+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.116334+0000","last_undegraded":"2024-05-11T12:13:02.116334+0000","last_fullsized":"2024-05-11T12:13:02.116334+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.115725+0000","last_change":"2024-05-11T12:13:02.115725+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.115725+0000","last_undegraded":"2024-05-11T12:13:02.115725+0000","last_fullsized":"2024-05-11T12:13:02.115725+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.171059+0000","last_change":"2024-05-11T12:13:03.165438+0000","last_active":"2024-05-11T12:13:03.171059+0000","last_peered":"2024-05-11T12:13:03.171059+0000","last_clean":"2024-05-11T12:13:03.171059+0000","last_became_active":"2024-05-11T12:13:03.165118+0000","last_became_peered":"2024-05-11T12:13:03.165118+0000","last_unstale":"2024-05-11T12:13:03.171059+0000","last_undegraded":"2024-05-11T12:13:03.171059+0000","last_fullsized":"2024-05-11T12:13:03.171059+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:56:37.784627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.176455+0000","last_change":"2024-05-11T12:13:03.169517+0000","last_active":"2024-05-11T12:13:03.176455+0000","last_peered":"2024-05-11T12:13:03.176455+0000","last_clean":"2024-05-11T12:13:03.176455+0000","last_became_active":"2024-05-11T12:13:03.169235+0000","last_became_peered":"2024-05-11T12:13:03.169235+0000","last_unstale":"2024-05-11T12:13:03.176455+0000","last_undegraded":"2024-05-11T12:13:03.176455+0000","last_fullsized":"2024-05-11T12:13:03.176455+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:27:14.194818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.174469+0000","last_change":"2024-05-11T12:13:03.166968+0000","last_active":"2024-05-11T12:13:03.174469+0000","last_peered":"2024-05-11T12:13:03.174469+0000","last_clean":"2024-05-11T12:13:03.174469+0000","last_became_active":"2024-05-11T12:13:03.166704+0000","last_became_peered":"2024-05-11T12:13:03.166704+0000","last_unstale":"2024-05-11T12:13:03.174469+0000","last_undegraded":"2024-05-11T12:13:03.174469+0000","last_fullsized":"2024-05-11T12:13:03.174469+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:56:37.784627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.178640+0000","last_change":"2024-05-11T12:13:03.171885+0000","last_active":"2024-05-11T12:13:03.178640+0000","last_peered":"2024-05-11T12:13:03.178640+0000","last_clean":"2024-05-11T12:13:03.178640+0000","last_became_active":"2024-05-11T12:13:03.171590+0000","last_became_peered":"2024-05-11T12:13:03.171590+0000","last_unstale":"2024-05-11T12:13:03.178640+0000","last_undegraded":"2024-05-11T12:13:03.178640+0000","last_fullsized":"2024-05-11T12:13:03.178640+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:06:40.718612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.103639+0000","last_change":"2024-05-11T12:13:02.103639+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.103639+0000","last_undegraded":"2024-05-11T12:13:02.103639+0000","last_fullsized":"2024-05-11T12:13:02.103639+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.164436+0000","last_change":"2024-05-11T12:13:03.160607+0000","last_active":"2024-05-11T12:13:03.164436+0000","last_peered":"2024-05-11T12:13:03.164436+0000","last_clean":"2024-05-11T12:13:03.164436+0000","last_became_active":"2024-05-11T12:13:03.160281+0000","last_became_peered":"2024-05-11T12:13:03.160281+0000","last_unstale":"2024-05-11T12:13:03.164436+0000","last_undegraded":"2024-05-11T12:13:03.164436+0000","last_fullsized":"2024-05-11T12:13:03.164436+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:12:30.081833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.170126+0000","last_change":"2024-05-11T12:13:03.164270+0000","last_active":"2024-05-11T12:13:03.170126+0000","last_peered":"2024-05-11T12:13:03.170126+0000","last_clean":"2024-05-11T12:13:03.170126+0000","last_became_active":"2024-05-11T12:13:03.163977+0000","last_became_peered":"2024-05-11T12:13:03.163977+0000","last_unstale":"2024-05-11T12:13:03.170126+0000","last_undegraded":"2024-05-11T12:13:03.170126+0000","last_fullsized":"2024-05-11T12:13:03.170126+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:08:16.138838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.177851+0000","last_change":"2024-05-11T12:13:03.171792+0000","last_active":"2024-05-11T12:13:03.177851+0000","last_peered":"2024-05-11T12:13:03.177851+0000","last_clean":"2024-05-11T12:13:03.177851+0000","last_became_active":"2024-05-11T12:13:03.171502+0000","last_became_peered":"2024-05-11T12:13:03.171502+0000","last_unstale":"2024-05-11T12:13:03.177851+0000","last_undegraded":"2024-05-11T12:13:03.177851+0000","last_fullsized":"2024-05-11T12:13:03.177851+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:20:41.663583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.121306+0000","last_change":"2024-05-11T12:13:02.121306+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.121306+0000","last_undegraded":"2024-05-11T12:13:02.121306+0000","last_fullsized":"2024-05-11T12:13:02.121306+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.177562+0000","last_change":"2024-05-11T12:13:03.176021+0000","last_active":"2024-05-11T12:13:03.177562+0000","last_peered":"2024-05-11T12:13:03.177562+0000","last_clean":"2024-05-11T12:13:03.177562+0000","last_became_active":"2024-05-11T12:13:03.175708+0000","last_became_peered":"2024-05-11T12:13:03.175708+0000","last_unstale":"2024-05-11T12:13:03.177562+0000","last_undegraded":"2024-05-11T12:13:03.177562+0000","last_fullsized":"2024-05-11T12:13:03.177562+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:12:07.890717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.140559+0000","last_change":"2024-05-11T12:13:02.136461+0000","last_active":"2024-05-11T12:13:02.140559+0000","last_peered":"2024-05-11T12:13:02.140559+0000","last_clean":"2024-05-11T12:13:02.140559+0000","last_became_active":"2024-05-11T12:13:02.136210+0000","last_became_peered":"2024-05-11T12:13:02.136210+0000","last_unstale":"2024-05-11T12:13:02.140559+0000","last_undegraded":"2024-05-11T12:13:02.140559+0000","last_fullsized":"2024-05-11T12:13:02.140559+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:20:40.658655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.159893+0000","last_change":"2024-05-11T12:13:03.158314+0000","last_active":"2024-05-11T12:13:03.159893+0000","last_peered":"2024-05-11T12:13:03.159893+0000","last_clean":"2024-05-11T12:13:03.159893+0000","last_became_active":"2024-05-11T12:13:03.157991+0000","last_became_peered":"2024-05-11T12:13:03.157991+0000","last_unstale":"2024-05-11T12:13:03.159893+0000","last_undegraded":"2024-05-11T12:13:03.159893+0000","last_fullsized":"2024-05-11T12:13:03.159893+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:55:20.561101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.140990+0000","last_change":"2024-05-11T12:13:02.136740+0000","last_active":"2024-05-11T12:13:02.140990+0000","last_peered":"2024-05-11T12:13:02.140990+0000","last_clean":"2024-05-11T12:13:02.140990+0000","last_became_active":"2024-05-11T12:13:02.136446+0000","last_became_peered":"2024-05-11T12:13:02.136446+0000","last_unstale":"2024-05-11T12:13:02.140990+0000","last_undegraded":"2024-05-11T12:13:02.140990+0000","last_fullsized":"2024-05-11T12:13:02.140990+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:58:07.809901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.120695+0000","last_change":"2024-05-11T12:13:02.120695+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.120695+0000","last_undegraded":"2024-05-11T12:13:02.120695+0000","last_fullsized":"2024-05-11T12:13:02.120695+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.137480+0000","last_change":"2024-05-11T12:13:02.129658+0000","last_active":"2024-05-11T12:13:02.137480+0000","last_peered":"2024-05-11T12:13:02.137480+0000","last_clean":"2024-05-11T12:13:02.137480+0000","last_became_active":"2024-05-11T12:13:02.129326+0000","last_became_peered":"2024-05-11T12:13:02.129326+0000","last_unstale":"2024-05-11T12:13:02.137480+0000","last_undegraded":"2024-05-11T12:13:02.137480+0000","last_fullsized":"2024-05-11T12:13:02.137480+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:42:20.954919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.168978+0000","last_change":"2024-05-11T12:13:03.162149+0000","last_active":"2024-05-11T12:13:03.168978+0000","last_peered":"2024-05-11T12:13:03.168978+0000","last_clean":"2024-05-11T12:13:03.168978+0000","last_became_active":"2024-05-11T12:13:03.161890+0000","last_became_peered":"2024-05-11T12:13:03.161890+0000","last_unstale":"2024-05-11T12:13:03.168978+0000","last_undegraded":"2024-05-11T12:13:03.168978+0000","last_fullsized":"2024-05-11T12:13:03.168978+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:42:21.959847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.134444+0000","last_change":"2024-05-11T12:13:02.126473+0000","last_active":"2024-05-11T12:13:02.134444+0000","last_peered":"2024-05-11T12:13:02.134444+0000","last_clean":"2024-05-11T12:13:02.134444+0000","last_became_active":"2024-05-11T12:13:02.126154+0000","last_became_peered":"2024-05-11T12:13:02.126154+0000","last_unstale":"2024-05-11T12:13:02.134444+0000","last_undegraded":"2024-05-11T12:13:02.134444+0000","last_fullsized":"2024-05-11T12:13:02.134444+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:15:28.704713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.173923+0000","last_change":"2024-05-11T12:13:03.164511+0000","last_active":"2024-05-11T12:13:04.173923+0000","last_peered":"2024-05-11T12:13:04.173923+0000","last_clean":"2024-05-11T12:13:04.173923+0000","last_became_active":"2024-05-11T12:13:03.164197+0000","last_became_peered":"2024-05-11T12:13:03.164197+0000","last_unstale":"2024-05-11T12:13:04.173923+0000","last_undegraded":"2024-05-11T12:13:04.173923+0000","last_fullsized":"2024-05-11T12:13:04.173923+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:54:15.999941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.125903+0000","last_change":"2024-05-11T12:13:02.119338+0000","last_active":"2024-05-11T12:13:02.125903+0000","last_peered":"2024-05-11T12:13:02.125903+0000","last_clean":"2024-05-11T12:13:02.125903+0000","last_became_active":"2024-05-11T12:13:02.118127+0000","last_became_peered":"2024-05-11T12:13:02.118127+0000","last_unstale":"2024-05-11T12:13:02.125903+0000","last_undegraded":"2024-05-11T12:13:02.125903+0000","last_fullsized":"2024-05-11T12:13:02.125903+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:56:36.779700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.104889+0000","last_change":"2024-05-11T12:13:02.104889+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.104889+0000","last_undegraded":"2024-05-11T12:13:02.104889+0000","last_fullsized":"2024-05-11T12:13:02.104889+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.125968+0000","last_change":"2024-05-11T12:13:02.121903+0000","last_active":"2024-05-11T12:13:02.125968+0000","last_peered":"2024-05-11T12:13:02.125968+0000","last_clean":"2024-05-11T12:13:02.125968+0000","last_became_active":"2024-05-11T12:13:02.120293+0000","last_became_peered":"2024-05-11T12:13:02.120293+0000","last_unstale":"2024-05-11T12:13:02.125968+0000","last_undegraded":"2024-05-11T12:13:02.125968+0000","last_fullsized":"2024-05-11T12:13:02.125968+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:04:04.992779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.174363+0000","last_change":"2024-05-11T12:13:03.168572+0000","last_active":"2024-05-11T12:13:03.174363+0000","last_peered":"2024-05-11T12:13:03.174363+0000","last_clean":"2024-05-11T12:13:03.174363+0000","last_became_active":"2024-05-11T12:13:03.167854+0000","last_became_peered":"2024-05-11T12:13:03.167854+0000","last_unstale":"2024-05-11T12:13:03.174363+0000","last_undegraded":"2024-05-11T12:13:03.174363+0000","last_fullsized":"2024-05-11T12:13:03.174363+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:27:27.374412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.143860+0000","last_change":"2024-05-11T12:13:02.139509+0000","last_active":"2024-05-11T12:13:02.143860+0000","last_peered":"2024-05-11T12:13:02.143860+0000","last_clean":"2024-05-11T12:13:02.143860+0000","last_became_active":"2024-05-11T12:13:02.139201+0000","last_became_peered":"2024-05-11T12:13:02.139201+0000","last_unstale":"2024-05-11T12:13:02.143860+0000","last_undegraded":"2024-05-11T12:13:02.143860+0000","last_fullsized":"2024-05-11T12:13:02.143860+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:55:25.067740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.115715+0000","last_change":"2024-05-11T12:13:02.115715+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.115715+0000","last_undegraded":"2024-05-11T12:13:02.115715+0000","last_fullsized":"2024-05-11T12:13:02.115715+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.147186+0000","last_change":"2024-05-11T12:13:02.139692+0000","last_active":"2024-05-11T12:13:03.147186+0000","last_peered":"2024-05-11T12:13:03.147186+0000","last_clean":"2024-05-11T12:13:03.147186+0000","last_became_active":"2024-05-11T12:13:02.137935+0000","last_became_peered":"2024-05-11T12:13:02.137935+0000","last_unstale":"2024-05-11T12:13:03.147186+0000","last_undegraded":"2024-05-11T12:13:03.147186+0000","last_fullsized":"2024-05-11T12:13:03.147186+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:15:34.846112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.103086+0000","last_change":"2024-05-11T12:13:02.103086+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.103086+0000","last_undegraded":"2024-05-11T12:13:02.103086+0000","last_fullsized":"2024-05-11T12:13:02.103086+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.154876+0000","last_change":"2024-05-11T12:13:02.142438+0000","last_active":"2024-05-11T12:13:03.154876+0000","last_peered":"2024-05-11T12:13:03.154876+0000","last_clean":"2024-05-11T12:13:03.154876+0000","last_became_active":"2024-05-11T12:13:02.142125+0000","last_became_peered":"2024-05-11T12:13:02.142125+0000","last_unstale":"2024-05-11T12:13:03.154876+0000","last_undegraded":"2024-05-11T12:13:03.154876+0000","last_fullsized":"2024-05-11T12:13:03.154876+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:55:32.333055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.116293+0000","last_change":"2024-05-11T12:13:02.116293+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.116293+0000","last_undegraded":"2024-05-11T12:13:02.116293+0000","last_fullsized":"2024-05-11T12:13:02.116293+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.144040+0000","last_change":"2024-05-11T12:13:02.140895+0000","last_active":"2024-05-11T12:13:03.144040+0000","last_peered":"2024-05-11T12:13:03.144040+0000","last_clean":"2024-05-11T12:13:03.144040+0000","last_became_active":"2024-05-11T12:13:02.138733+0000","last_became_peered":"2024-05-11T12:13:02.138733+0000","last_unstale":"2024-05-11T12:13:03.144040+0000","last_undegraded":"2024-05-11T12:13:03.144040+0000","last_fullsized":"2024-05-11T12:13:03.144040+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:08:47.140652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.171236+0000","last_change":"2024-05-11T12:13:03.160320+0000","last_active":"2024-05-11T12:13:03.171236+0000","last_peered":"2024-05-11T12:13:03.171236+0000","last_clean":"2024-05-11T12:13:03.171236+0000","last_became_active":"2024-05-11T12:13:03.159984+0000","last_became_peered":"2024-05-11T12:13:03.159984+0000","last_unstale":"2024-05-11T12:13:03.171236+0000","last_undegraded":"2024-05-11T12:13:03.171236+0000","last_fullsized":"2024-05-11T12:13:03.171236+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:04:22.163536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"63'3","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.185645+0000","last_change":"2024-05-11T12:13:02.139752+0000","last_active":"2024-05-11T12:13:03.185645+0000","last_peered":"2024-05-11T12:13:03.185645+0000","last_clean":"2024-05-11T12:13:03.185645+0000","last_became_active":"2024-05-11T12:13:02.139431+0000","last_became_peered":"2024-05-11T12:13:02.139431+0000","last_unstale":"2024-05-11T12:13:03.185645+0000","last_undegraded":"2024-05-11T12:13:03.185645+0000","last_fullsized":"2024-05-11T12:13:03.185645+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:00:08.081754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.174833+0000","last_change":"2024-05-11T12:13:03.166818+0000","last_active":"2024-05-11T12:13:03.174833+0000","last_peered":"2024-05-11T12:13:03.174833+0000","last_clean":"2024-05-11T12:13:03.174833+0000","last_became_active":"2024-05-11T12:13:03.166560+0000","last_became_peered":"2024-05-11T12:13:03.166560+0000","last_unstale":"2024-05-11T12:13:03.174833+0000","last_undegraded":"2024-05-11T12:13:03.174833+0000","last_fullsized":"2024-05-11T12:13:03.174833+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:04:22.163536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.137897+0000","last_change":"2024-05-11T12:13:02.133552+0000","last_active":"2024-05-11T12:13:02.137897+0000","last_peered":"2024-05-11T12:13:02.137897+0000","last_clean":"2024-05-11T12:13:02.137897+0000","last_became_active":"2024-05-11T12:13:02.133236+0000","last_became_peered":"2024-05-11T12:13:02.133236+0000","last_unstale":"2024-05-11T12:13:02.137897+0000","last_undegraded":"2024-05-11T12:13:02.137897+0000","last_fullsized":"2024-05-11T12:13:02.137897+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:32:44.989980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.175555+0000","last_change":"2024-05-11T12:13:03.168373+0000","last_active":"2024-05-11T12:13:03.175555+0000","last_peered":"2024-05-11T12:13:03.175555+0000","last_clean":"2024-05-11T12:13:03.175555+0000","last_became_active":"2024-05-11T12:13:03.168105+0000","last_became_peered":"2024-05-11T12:13:03.168105+0000","last_unstale":"2024-05-11T12:13:03.175555+0000","last_undegraded":"2024-05-11T12:13:03.175555+0000","last_fullsized":"2024-05-11T12:13:03.175555+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:32:45.994907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"63'2","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.183854+0000","last_change":"2024-05-11T12:13:02.143192+0000","last_active":"2024-05-11T12:13:03.183854+0000","last_peered":"2024-05-11T12:13:03.183854+0000","last_clean":"2024-05-11T12:13:03.183854+0000","last_became_active":"2024-05-11T12:13:02.142901+0000","last_became_peered":"2024-05-11T12:13:02.142901+0000","last_unstale":"2024-05-11T12:13:03.183854+0000","last_undegraded":"2024-05-11T12:13:03.183854+0000","last_fullsized":"2024-05-11T12:13:03.183854+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:39:03.473134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.155659+0000","last_change":"2024-05-11T12:13:03.145731+0000","last_active":"2024-05-11T12:13:03.155659+0000","last_peered":"2024-05-11T12:13:03.155659+0000","last_clean":"2024-05-11T12:13:03.155659+0000","last_became_active":"2024-05-11T12:13:03.145419+0000","last_became_peered":"2024-05-11T12:13:03.145419+0000","last_unstale":"2024-05-11T12:13:03.155659+0000","last_undegraded":"2024-05-11T12:13:03.155659+0000","last_fullsized":"2024-05-11T12:13:03.155659+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:55:33.337983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.144327+0000","last_change":"2024-05-11T12:13:02.118817+0000","last_active":"2024-05-11T12:13:03.144327+0000","last_peered":"2024-05-11T12:13:03.144327+0000","last_clean":"2024-05-11T12:13:03.144327+0000","last_became_active":"2024-05-11T12:13:02.118040+0000","last_became_peered":"2024-05-11T12:13:02.118040+0000","last_unstale":"2024-05-11T12:13:03.144327+0000","last_undegraded":"2024-05-11T12:13:03.144327+0000","last_fullsized":"2024-05-11T12:13:03.144327+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:18:34.044830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.175410+0000","last_change":"2024-05-11T12:13:03.172604+0000","last_active":"2024-05-11T12:13:03.175410+0000","last_peered":"2024-05-11T12:13:03.175410+0000","last_clean":"2024-05-11T12:13:03.175410+0000","last_became_active":"2024-05-11T12:13:03.172198+0000","last_became_peered":"2024-05-11T12:13:03.172198+0000","last_unstale":"2024-05-11T12:13:03.175410+0000","last_undegraded":"2024-05-11T12:13:03.175410+0000","last_fullsized":"2024-05-11T12:13:03.175410+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:56:18.568549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.156874+0000","last_change":"2024-05-11T12:13:02.138604+0000","last_active":"2024-05-11T12:13:03.156874+0000","last_peered":"2024-05-11T12:13:03.156874+0000","last_clean":"2024-05-11T12:13:03.156874+0000","last_became_active":"2024-05-11T12:13:02.138227+0000","last_became_peered":"2024-05-11T12:13:02.138227+0000","last_unstale":"2024-05-11T12:13:03.156874+0000","last_undegraded":"2024-05-11T12:13:03.156874+0000","last_fullsized":"2024-05-11T12:13:03.156874+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:15:28.704713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.101871+0000","last_change":"2024-05-11T12:13:02.101871+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.101871+0000","last_undegraded":"2024-05-11T12:13:02.101871+0000","last_fullsized":"2024-05-11T12:13:02.101871+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"63'8","reported_seq":20,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.186047+0000","last_change":"2024-05-11T12:13:02.137326+0000","last_active":"2024-05-11T12:13:03.186047+0000","last_peered":"2024-05-11T12:13:03.186047+0000","last_clean":"2024-05-11T12:13:03.186047+0000","last_became_active":"2024-05-11T12:13:02.133763+0000","last_became_peered":"2024-05-11T12:13:02.133763+0000","last_unstale":"2024-05-11T12:13:03.186047+0000","last_undegraded":"2024-05-11T12:13:03.186047+0000","last_fullsized":"2024-05-11T12:13:03.186047+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:55:40.188834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.169808+0000","last_change":"2024-05-11T12:13:03.163731+0000","last_active":"2024-05-11T12:13:03.169808+0000","last_peered":"2024-05-11T12:13:03.169808+0000","last_clean":"2024-05-11T12:13:03.169808+0000","last_became_active":"2024-05-11T12:13:03.161280+0000","last_became_peered":"2024-05-11T12:13:03.161280+0000","last_unstale":"2024-05-11T12:13:03.169808+0000","last_undegraded":"2024-05-11T12:13:03.169808+0000","last_fullsized":"2024-05-11T12:13:03.169808+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:58:08.814828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"63'1","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.180608+0000","last_change":"2024-05-11T12:13:02.124463+0000","last_active":"2024-05-11T12:13:03.180608+0000","last_peered":"2024-05-11T12:13:03.180608+0000","last_clean":"2024-05-11T12:13:03.180608+0000","last_became_active":"2024-05-11T12:13:02.124136+0000","last_became_peered":"2024-05-11T12:13:02.124136+0000","last_unstale":"2024-05-11T12:13:03.180608+0000","last_undegraded":"2024-05-11T12:13:03.180608+0000","last_fullsized":"2024-05-11T12:13:03.180608+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:48:52.177182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.161587+0000","last_change":"2024-05-11T12:13:03.158635+0000","last_active":"2024-05-11T12:13:03.161587+0000","last_peered":"2024-05-11T12:13:03.161587+0000","last_clean":"2024-05-11T12:13:03.161587+0000","last_became_active":"2024-05-11T12:13:03.158311+0000","last_became_peered":"2024-05-11T12:13:03.158311+0000","last_unstale":"2024-05-11T12:13:03.161587+0000","last_undegraded":"2024-05-11T12:13:03.161587+0000","last_fullsized":"2024-05-11T12:13:03.161587+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:46:22.393593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"63'2","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.182624+0000","last_change":"2024-05-11T12:13:02.143315+0000","last_active":"2024-05-11T12:13:03.182624+0000","last_peered":"2024-05-11T12:13:03.182624+0000","last_clean":"2024-05-11T12:13:03.182624+0000","last_became_active":"2024-05-11T12:13:02.143012+0000","last_became_peered":"2024-05-11T12:13:02.143012+0000","last_unstale":"2024-05-11T12:13:03.182624+0000","last_undegraded":"2024-05-11T12:13:03.182624+0000","last_fullsized":"2024-05-11T12:13:03.182624+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:03:26.831522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.172645+0000","last_change":"2024-05-11T12:13:03.159923+0000","last_active":"2024-05-11T12:13:03.172645+0000","last_peered":"2024-05-11T12:13:03.172645+0000","last_clean":"2024-05-11T12:13:03.172645+0000","last_became_active":"2024-05-11T12:13:03.159119+0000","last_became_peered":"2024-05-11T12:13:03.159119+0000","last_unstale":"2024-05-11T12:13:03.172645+0000","last_undegraded":"2024-05-11T12:13:03.172645+0000","last_fullsized":"2024-05-11T12:13:03.172645+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:11:31.058373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"63'1","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.185684+0000","last_change":"2024-05-11T12:13:02.138872+0000","last_active":"2024-05-11T12:13:03.185684+0000","last_peered":"2024-05-11T12:13:03.185684+0000","last_clean":"2024-05-11T12:13:03.185684+0000","last_became_active":"2024-05-11T12:13:02.138589+0000","last_became_peered":"2024-05-11T12:13:02.138589+0000","last_unstale":"2024-05-11T12:13:03.185684+0000","last_undegraded":"2024-05-11T12:13:03.185684+0000","last_fullsized":"2024-05-11T12:13:03.185684+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:48:52.177182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.174205+0000","last_change":"2024-05-11T12:13:03.167395+0000","last_active":"2024-05-11T12:13:03.174205+0000","last_peered":"2024-05-11T12:13:03.174205+0000","last_clean":"2024-05-11T12:13:03.174205+0000","last_became_active":"2024-05-11T12:13:03.167111+0000","last_became_peered":"2024-05-11T12:13:03.167111+0000","last_unstale":"2024-05-11T12:13:03.174205+0000","last_undegraded":"2024-05-11T12:13:03.174205+0000","last_fullsized":"2024-05-11T12:13:03.174205+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:20:41.663583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.141325+0000","last_change":"2024-05-11T12:13:02.137581+0000","last_active":"2024-05-11T12:13:02.141325+0000","last_peered":"2024-05-11T12:13:02.141325+0000","last_clean":"2024-05-11T12:13:02.141325+0000","last_became_active":"2024-05-11T12:13:02.137260+0000","last_became_peered":"2024-05-11T12:13:02.137260+0000","last_unstale":"2024-05-11T12:13:02.141325+0000","last_undegraded":"2024-05-11T12:13:02.141325+0000","last_fullsized":"2024-05-11T12:13:02.141325+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:29:57.084718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.120434+0000","last_change":"2024-05-11T12:13:02.120434+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.120434+0000","last_undegraded":"2024-05-11T12:13:02.120434+0000","last_fullsized":"2024-05-11T12:13:02.120434+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.143793+0000","last_change":"2024-05-11T12:13:02.140367+0000","last_active":"2024-05-11T12:13:02.143793+0000","last_peered":"2024-05-11T12:13:02.143793+0000","last_clean":"2024-05-11T12:13:02.143793+0000","last_became_active":"2024-05-11T12:13:02.140069+0000","last_became_peered":"2024-05-11T12:13:02.140069+0000","last_unstale":"2024-05-11T12:13:02.143793+0000","last_undegraded":"2024-05-11T12:13:02.143793+0000","last_fullsized":"2024-05-11T12:13:02.143793+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:24:39.427445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.175442+0000","last_change":"2024-05-11T12:13:03.171936+0000","last_active":"2024-05-11T12:13:03.175442+0000","last_peered":"2024-05-11T12:13:03.175442+0000","last_clean":"2024-05-11T12:13:03.175442+0000","last_became_active":"2024-05-11T12:13:03.171669+0000","last_became_peered":"2024-05-11T12:13:03.171669+0000","last_unstale":"2024-05-11T12:13:03.175442+0000","last_undegraded":"2024-05-11T12:13:03.175442+0000","last_fullsized":"2024-05-11T12:13:03.175442+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:29:58.089646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.144512+0000","last_change":"2024-05-11T12:13:02.125827+0000","last_active":"2024-05-11T12:13:03.144512+0000","last_peered":"2024-05-11T12:13:03.144512+0000","last_clean":"2024-05-11T12:13:03.144512+0000","last_became_active":"2024-05-11T12:13:02.125520+0000","last_became_peered":"2024-05-11T12:13:02.125520+0000","last_unstale":"2024-05-11T12:13:03.144512+0000","last_undegraded":"2024-05-11T12:13:03.144512+0000","last_fullsized":"2024-05-11T12:13:03.144512+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:00:08.081754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.174385+0000","last_change":"2024-05-11T12:13:03.168791+0000","last_active":"2024-05-11T12:13:03.174385+0000","last_peered":"2024-05-11T12:13:03.174385+0000","last_clean":"2024-05-11T12:13:03.174385+0000","last_became_active":"2024-05-11T12:13:03.168255+0000","last_became_peered":"2024-05-11T12:13:03.168255+0000","last_unstale":"2024-05-11T12:13:03.174385+0000","last_undegraded":"2024-05-11T12:13:03.174385+0000","last_fullsized":"2024-05-11T12:13:03.174385+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:17:19.051848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.141028+0000","last_change":"2024-05-11T12:13:02.136770+0000","last_active":"2024-05-11T12:13:02.141028+0000","last_peered":"2024-05-11T12:13:02.141028+0000","last_clean":"2024-05-11T12:13:02.141028+0000","last_became_active":"2024-05-11T12:13:02.136465+0000","last_became_peered":"2024-05-11T12:13:02.136465+0000","last_unstale":"2024-05-11T12:13:02.141028+0000","last_undegraded":"2024-05-11T12:13:02.141028+0000","last_fullsized":"2024-05-11T12:13:02.141028+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:08:15.133910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.120491+0000","last_change":"2024-05-11T12:13:02.120491+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.120491+0000","last_undegraded":"2024-05-11T12:13:02.120491+0000","last_fullsized":"2024-05-11T12:13:02.120491+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"63'2","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.183009+0000","last_change":"2024-05-11T12:13:02.136984+0000","last_active":"2024-05-11T12:13:03.183009+0000","last_peered":"2024-05-11T12:13:03.183009+0000","last_clean":"2024-05-11T12:13:03.183009+0000","last_became_active":"2024-05-11T12:13:02.135837+0000","last_became_peered":"2024-05-11T12:13:02.135837+0000","last_unstale":"2024-05-11T12:13:03.183009+0000","last_undegraded":"2024-05-11T12:13:03.183009+0000","last_fullsized":"2024-05-11T12:13:03.183009+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:03:26.831522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.173813+0000","last_change":"2024-05-11T12:13:03.170696+0000","last_active":"2024-05-11T12:13:04.173813+0000","last_peered":"2024-05-11T12:13:04.173813+0000","last_clean":"2024-05-11T12:13:04.173813+0000","last_became_active":"2024-05-11T12:13:03.170411+0000","last_became_peered":"2024-05-11T12:13:03.170411+0000","last_unstale":"2024-05-11T12:13:04.173813+0000","last_undegraded":"2024-05-11T12:13:04.173813+0000","last_fullsized":"2024-05-11T12:13:04.173813+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:11:05.631699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.146050+0000","last_change":"2024-05-11T12:13:02.128082+0000","last_active":"2024-05-11T12:13:03.146050+0000","last_peered":"2024-05-11T12:13:03.146050+0000","last_clean":"2024-05-11T12:13:03.146050+0000","last_became_active":"2024-05-11T12:13:02.126292+0000","last_became_peered":"2024-05-11T12:13:02.126292+0000","last_unstale":"2024-05-11T12:13:03.146050+0000","last_undegraded":"2024-05-11T12:13:03.146050+0000","last_fullsized":"2024-05-11T12:13:03.146050+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:15:28.704713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.156288+0000","last_change":"2024-05-11T12:13:03.149098+0000","last_active":"2024-05-11T12:13:03.156288+0000","last_peered":"2024-05-11T12:13:03.156288+0000","last_clean":"2024-05-11T12:13:03.156288+0000","last_became_active":"2024-05-11T12:13:03.148358+0000","last_became_peered":"2024-05-11T12:13:03.148358+0000","last_unstale":"2024-05-11T12:13:03.156288+0000","last_undegraded":"2024-05-11T12:13:03.156288+0000","last_fullsized":"2024-05-11T12:13:03.156288+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:12:30.081833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.137459+0000","last_change":"2024-05-11T12:13:02.129624+0000","last_active":"2024-05-11T12:13:02.137459+0000","last_peered":"2024-05-11T12:13:02.137459+0000","last_clean":"2024-05-11T12:13:02.137459+0000","last_became_active":"2024-05-11T12:13:02.129286+0000","last_became_peered":"2024-05-11T12:13:02.129286+0000","last_unstale":"2024-05-11T12:13:02.137459+0000","last_undegraded":"2024-05-11T12:13:02.137459+0000","last_fullsized":"2024-05-11T12:13:02.137459+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:12:29.076905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.174228+0000","last_change":"2024-05-11T12:13:03.170501+0000","last_active":"2024-05-11T12:13:03.174228+0000","last_peered":"2024-05-11T12:13:03.174228+0000","last_clean":"2024-05-11T12:13:03.174228+0000","last_became_active":"2024-05-11T12:13:03.163918+0000","last_became_peered":"2024-05-11T12:13:03.163918+0000","last_unstale":"2024-05-11T12:13:03.174228+0000","last_undegraded":"2024-05-11T12:13:03.174228+0000","last_fullsized":"2024-05-11T12:13:03.174228+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:18:50.744350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.137839+0000","last_change":"2024-05-11T12:13:02.133532+0000","last_active":"2024-05-11T12:13:02.137839+0000","last_peered":"2024-05-11T12:13:02.137839+0000","last_clean":"2024-05-11T12:13:02.137839+0000","last_became_active":"2024-05-11T12:13:02.133209+0000","last_became_peered":"2024-05-11T12:13:02.133209+0000","last_unstale":"2024-05-11T12:13:02.137839+0000","last_undegraded":"2024-05-11T12:13:02.137839+0000","last_fullsized":"2024-05-11T12:13:02.137839+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:52:38.362358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.172256+0000","last_change":"2024-05-11T12:13:03.163919+0000","last_active":"2024-05-11T12:13:03.172256+0000","last_peered":"2024-05-11T12:13:03.172256+0000","last_clean":"2024-05-11T12:13:03.172256+0000","last_became_active":"2024-05-11T12:13:03.163595+0000","last_became_peered":"2024-05-11T12:13:03.163595+0000","last_unstale":"2024-05-11T12:13:03.172256+0000","last_undegraded":"2024-05-11T12:13:03.172256+0000","last_fullsized":"2024-05-11T12:13:03.172256+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:52:39.367286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.152331+0000","last_change":"2024-05-11T12:13:02.139859+0000","last_active":"2024-05-11T12:13:03.152331+0000","last_peered":"2024-05-11T12:13:03.152331+0000","last_clean":"2024-05-11T12:13:03.152331+0000","last_became_active":"2024-05-11T12:13:02.139265+0000","last_became_peered":"2024-05-11T12:13:02.139265+0000","last_unstale":"2024-05-11T12:13:03.152331+0000","last_undegraded":"2024-05-11T12:13:03.152331+0000","last_fullsized":"2024-05-11T12:13:03.152331+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:18:34.044830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.172282+0000","last_change":"2024-05-11T12:13:03.163034+0000","last_active":"2024-05-11T12:13:03.172282+0000","last_peered":"2024-05-11T12:13:03.172282+0000","last_clean":"2024-05-11T12:13:03.172282+0000","last_became_active":"2024-05-11T12:13:03.160775+0000","last_became_peered":"2024-05-11T12:13:03.160775+0000","last_unstale":"2024-05-11T12:13:03.172282+0000","last_undegraded":"2024-05-11T12:13:03.172282+0000","last_fullsized":"2024-05-11T12:13:03.172282+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:08:51.947443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.117231+0000","last_change":"2024-05-11T12:13:02.113452+0000","last_active":"2024-05-11T12:13:02.117231+0000","last_peered":"2024-05-11T12:13:02.117231+0000","last_clean":"2024-05-11T12:13:02.117231+0000","last_became_active":"2024-05-11T12:13:02.112863+0000","last_became_peered":"2024-05-11T12:13:02.112863+0000","last_unstale":"2024-05-11T12:13:02.117231+0000","last_undegraded":"2024-05-11T12:13:02.117231+0000","last_fullsized":"2024-05-11T12:13:02.117231+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:08:15.133910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.102609+0000","last_change":"2024-05-11T12:13:02.102609+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.102609+0000","last_undegraded":"2024-05-11T12:13:02.102609+0000","last_fullsized":"2024-05-11T12:13:02.102609+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.137700+0000","last_change":"2024-05-11T12:13:02.133595+0000","last_active":"2024-05-11T12:13:02.137700+0000","last_peered":"2024-05-11T12:13:02.137700+0000","last_clean":"2024-05-11T12:13:02.137700+0000","last_became_active":"2024-05-11T12:13:02.133168+0000","last_became_peered":"2024-05-11T12:13:02.133168+0000","last_unstale":"2024-05-11T12:13:02.137700+0000","last_undegraded":"2024-05-11T12:13:02.137700+0000","last_fullsized":"2024-05-11T12:13:02.137700+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:27:13.189890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.173843+0000","last_change":"2024-05-11T12:13:03.169252+0000","last_active":"2024-05-11T12:13:04.173843+0000","last_peered":"2024-05-11T12:13:04.173843+0000","last_clean":"2024-05-11T12:13:04.173843+0000","last_became_active":"2024-05-11T12:13:03.167540+0000","last_became_peered":"2024-05-11T12:13:03.167540+0000","last_unstale":"2024-05-11T12:13:04.173843+0000","last_undegraded":"2024-05-11T12:13:04.173843+0000","last_fullsized":"2024-05-11T12:13:04.173843+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:38:22.482431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.144252+0000","last_change":"2024-05-11T12:13:02.140835+0000","last_active":"2024-05-11T12:13:02.144252+0000","last_peered":"2024-05-11T12:13:02.144252+0000","last_clean":"2024-05-11T12:13:02.144252+0000","last_became_active":"2024-05-11T12:13:02.139971+0000","last_became_peered":"2024-05-11T12:13:02.139971+0000","last_unstale":"2024-05-11T12:13:02.144252+0000","last_undegraded":"2024-05-11T12:13:02.144252+0000","last_fullsized":"2024-05-11T12:13:02.144252+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:06:39.713684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.172847+0000","last_change":"2024-05-11T12:13:03.166247+0000","last_active":"2024-05-11T12:13:03.172847+0000","last_peered":"2024-05-11T12:13:03.172847+0000","last_clean":"2024-05-11T12:13:03.172847+0000","last_became_active":"2024-05-11T12:13:03.165118+0000","last_became_peered":"2024-05-11T12:13:03.165118+0000","last_unstale":"2024-05-11T12:13:03.172847+0000","last_undegraded":"2024-05-11T12:13:03.172847+0000","last_fullsized":"2024-05-11T12:13:03.172847+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:32:45.994907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.125060+0000","last_change":"2024-05-11T12:13:02.116771+0000","last_active":"2024-05-11T12:13:02.125060+0000","last_peered":"2024-05-11T12:13:02.125060+0000","last_clean":"2024-05-11T12:13:02.125060+0000","last_became_active":"2024-05-11T12:13:02.116028+0000","last_became_peered":"2024-05-11T12:13:02.116028+0000","last_unstale":"2024-05-11T12:13:02.125060+0000","last_undegraded":"2024-05-11T12:13:02.125060+0000","last_fullsized":"2024-05-11T12:13:02.125060+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:24:39.427445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":183,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.152992+0000","last_change":"2024-05-11T12:10:35.066673+0000","last_active":"2024-05-11T12:13:03.152992+0000","last_peered":"2024-05-11T12:13:03.152992+0000","last_clean":"2024-05-11T12:13:03.152992+0000","last_became_active":"2024-05-11T12:10:35.065942+0000","last_became_peered":"2024-05-11T12:10:35.065942+0000","last_unstale":"2024-05-11T12:13:03.152992+0000","last_undegraded":"2024-05-11T12:13:03.152992+0000","last_fullsized":"2024-05-11T12:13:03.152992+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_clean_scrub_stamp":"2024-05-11T12:10:33.601136+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:12:02.133182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19,"ondisk_log_size":19,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476769,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28320,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343520,"statfs":{"total":96636764160,"available":96607764480,"internally_reserved":0,"allocated":1331200,"data_stored":916169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476770,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27740,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344100,"statfs":{"total":96636764160,"available":96608358400,"internally_reserved":0,"allocated":737280,"data_stored":325801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476769,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":737280,"data_stored":325801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476771,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":737280,"data_stored":325801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509474,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27732,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344108,"statfs":{"total":96636764160,"available":96608366592,"internally_reserved":0,"allocated":724992,"data_stored":319142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509475,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":724992,"data_stored":319142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509474,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28332,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343508,"statfs":{"total":96636764160,"available":96607752192,"internally_reserved":0,"allocated":1331200,"data_stored":916169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542179,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":761856,"data_stored":339660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T12:13:08.121 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T12:13:08.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.362+0000 7fc90b200640 1 Processor -- start 2024-05-11T12:13:08.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.362+0000 7fc90b200640 1 -- start start 2024-05-11T12:13:08.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.363+0000 7fc90b200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc90c1458f0 0x7fc90c06d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:08.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.363+0000 7fc90b200640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc90c071f30 con 0x7fc90c0b3a80 2024-05-11T12:13:08.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.363+0000 7fc90b200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc90c072210 con 0x7fc90c0b5220 2024-05-11T12:13:08.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.363+0000 7fc90b200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fc90c06db80 con 0x7fc90c1458f0 2024-05-11T12:13:08.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.364+0000 7fc909400640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fc90c0b3a80 0x7fc90c0b4970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:33344/0 (socket says 172.21.15.167:33344) 2024-05-11T12:13:08.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.364+0000 7fc909400640 1 -- 172.21.15.167:0/2570879388 learned_addr learned my addr 172.21.15.167:0/2570879388 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:13:08.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.364+0000 7fc908a00640 1 -- 172.21.15.167:0/2570879388 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 548732330 0 0) 0x7fc90c071f30 con 0x7fc90c0b3a80 2024-05-11T12:13:08.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.364+0000 7fc909e00640 1 --2- 172.21.15.167:0/2570879388 >> v2:172.21.15.204:3300/0 conn(0x7fc90c1458f0 0x7fc90c06d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:08.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.364+0000 7fc908a00640 1 -- 172.21.15.167:0/2570879388 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc8f0003520 con 0x7fc90c0b3a80 2024-05-11T12:13:08.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.365+0000 7fc908a00640 1 -- 172.21.15.167:0/2570879388 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3842248135 0 0) 0x7fc8f0003520 con 0x7fc90c0b3a80 2024-05-11T12:13:08.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.365+0000 7fc908a00640 1 -- 172.21.15.167:0/2570879388 >> v2:172.21.15.204:3300/0 conn(0x7fc90c1458f0 msgr2=0x7fc90c06d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:08.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.365+0000 7fc908a00640 1 --2- 172.21.15.167:0/2570879388 >> v2:172.21.15.204:3300/0 conn(0x7fc90c1458f0 0x7fc90c06d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:08.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.365+0000 7fc908a00640 1 -- 172.21.15.167:0/2570879388 >> v1:172.21.15.204:6789/0 conn(0x7fc90c0b5220 legacy=0x7fc90c072fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:08.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.365+0000 7fc908a00640 1 -- 172.21.15.167:0/2570879388 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc90c0724f0 con 0x7fc90c0b3a80 2024-05-11T12:13:08.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.365+0000 7fc908a00640 1 -- 172.21.15.167:0/2570879388 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7fc8f40016f0 con 0x7fc90c0b3a80 2024-05-11T12:13:08.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.365+0000 7fc908a00640 1 -- 172.21.15.167:0/2570879388 >> v1:172.21.15.167:6789/0 conn(0x7fc90c0b3a80 legacy=0x7fc90c0b4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:08.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.365+0000 7fc908a00640 1 --2- 172.21.15.167:0/2570879388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8f0003a00 0x7fc8f0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:08.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.365+0000 7fc908a00640 1 --2- 172.21.15.167:0/2570879388 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc8f0004800 0x7fc8f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:08.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.365+0000 7fc908a00640 1 --2- 172.21.15.167:0/2570879388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8f00051c0 0x7fc8f000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:08.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.365+0000 7fc908a00640 1 -- 172.21.15.167:0/2570879388 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f000c090 con 0x7fc8f0003a00 2024-05-11T12:13:08.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.365+0000 7fc908a00640 1 -- 172.21.15.167:0/2570879388 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f000c210 con 0x7fc8f00051c0 2024-05-11T12:13:08.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.365+0000 7fc908a00640 1 -- 172.21.15.167:0/2570879388 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc8f000c410 con 0x7fc8f0004800 2024-05-11T12:13:08.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.365+0000 7fc909e00640 1 --2- 172.21.15.167:0/2570879388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8f0003a00 0x7fc8f0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:08.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.365+0000 7fc909e00640 1 -- 172.21.15.167:0/2570879388 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc8f0004800 msgr2=0x7fc8f0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:08.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.366+0000 7fc909e00640 1 --2- 172.21.15.167:0/2570879388 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc8f0004800 0x7fc8f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:08.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.366+0000 7fc909e00640 1 -- 172.21.15.167:0/2570879388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8f00051c0 msgr2=0x7fc8f000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:08.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.366+0000 7fc909e00640 1 --2- 172.21.15.167:0/2570879388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8f00051c0 0x7fc8f000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:08.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.366+0000 7fc909e00640 1 -- 172.21.15.167:0/2570879388 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8f0003520 con 0x7fc8f0003a00 2024-05-11T12:13:08.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.366+0000 7fc90a800640 1 -- 172.21.15.167:0/2570879388 reap_dead start 2024-05-11T12:13:08.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.367+0000 7fc909e00640 1 --2- 172.21.15.167:0/2570879388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8f0003a00 0x7fc8f0003e00 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc009ac0 tx=0x7fc8f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=104200eef98ee46a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:08.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.367+0000 7fc908a00640 1 -- 172.21.15.167:0/2570879388 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc036070 con 0x7fc8f0003a00 2024-05-11T12:13:08.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.367+0000 7fc908a00640 1 -- 172.21.15.167:0/2570879388 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8fc031400 con 0x7fc8f0003a00 2024-05-11T12:13:08.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.367+0000 7fc908a00640 1 -- 172.21.15.167:0/2570879388 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc009690 con 0x7fc8f0003a00 2024-05-11T12:13:08.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.367+0000 7fc90b200640 1 -- 172.21.15.167:0/2570879388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8f0003a00 msgr2=0x7fc8f0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:08.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.367+0000 7fc90b200640 1 --2- 172.21.15.167:0/2570879388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8f0003a00 0x7fc8f0003e00 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc009ac0 tx=0x7fc8f0005830 comp rx=0 tx=0).stop 2024-05-11T12:13:08.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.368+0000 7fc90b200640 1 -- 172.21.15.167:0/2570879388 shutdown_connections 2024-05-11T12:13:08.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.368+0000 7fc90b200640 1 --2- 172.21.15.167:0/2570879388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8f0003a00 0x7fc8f0003e00 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:08.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.368+0000 7fc90b200640 1 -- 172.21.15.167:0/2570879388 >> 172.21.15.167:0/2570879388 conn(0x7fc90c0b3460 msgr2=0x7fc90c0d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:08.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.368+0000 7fc90b200640 1 -- 172.21.15.167:0/2570879388 shutdown_connections 2024-05-11T12:13:08.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.368+0000 7fc90b200640 1 -- 172.21.15.167:0/2570879388 wait complete. 2024-05-11T12:13:08.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.369+0000 7fc90b200640 1 Processor -- start 2024-05-11T12:13:08.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.369+0000 7fc90b200640 1 -- start start 2024-05-11T12:13:08.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.370+0000 7fc90b200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8f0003a00 0x7fc90c156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:08.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.370+0000 7fc90b200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc90c12f9b0 0x7fc90c12fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:08.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.370+0000 7fc90b200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc90c1366f0 0x7fc90c136b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:08.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.370+0000 7fc90b200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc90c06d640 con 0x7fc8f0003a00 2024-05-11T12:13:08.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.370+0000 7fc90b200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc90c072ab0 con 0x7fc90c12f9b0 2024-05-11T12:13:08.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.370+0000 7fc90b200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc90c06d7c0 con 0x7fc90c1366f0 2024-05-11T12:13:08.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.371+0000 7fc909e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8f0003a00 0x7fc90c156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:08.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.371+0000 7fc909e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8f0003a00 0x7fc90c156650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43396/0 (socket says 172.21.15.167:43396) 2024-05-11T12:13:08.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.371+0000 7fc909e00640 1 -- 172.21.15.167:0/2062078240 learned_addr learned my addr 172.21.15.167:0/2062078240 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:08.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.371+0000 7fc909e00640 1 -- 172.21.15.167:0/2062078240 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc90c1366f0 msgr2=0x7fc90c136b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:08.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.371+0000 7fc909e00640 1 --2- 172.21.15.167:0/2062078240 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc90c1366f0 0x7fc90c136b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:08.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.371+0000 7fc909e00640 1 -- 172.21.15.167:0/2062078240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc90c12f9b0 msgr2=0x7fc90c12fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:08.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.371+0000 7fc909e00640 1 --2- 172.21.15.167:0/2062078240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc90c12f9b0 0x7fc90c12fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:08.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.371+0000 7fc909e00640 1 -- 172.21.15.167:0/2062078240 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc90c130460 con 0x7fc8f0003a00 2024-05-11T12:13:08.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.371+0000 7fc909e00640 1 --2- 172.21.15.167:0/2062078240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8f0003a00 0x7fc90c156650 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc009be0 tx=0x7fc8fc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:08.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.372+0000 7fc902000640 1 -- 172.21.15.167:0/2062078240 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc03f020 con 0x7fc8f0003a00 2024-05-11T12:13:08.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.372+0000 7fc902000640 1 -- 172.21.15.167:0/2062078240 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8fc037040 con 0x7fc8f0003a00 2024-05-11T12:13:08.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.372+0000 7fc90b200640 1 -- 172.21.15.167:0/2062078240 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc90c130740 con 0x7fc8f0003a00 2024-05-11T12:13:08.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.372+0000 7fc902000640 1 -- 172.21.15.167:0/2062078240 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc036070 con 0x7fc8f0003a00 2024-05-11T12:13:08.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.372+0000 7fc90b200640 1 -- 172.21.15.167:0/2062078240 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc90c132810 con 0x7fc8f0003a00 2024-05-11T12:13:08.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.373+0000 7fc90b200640 1 -- 172.21.15.167:0/2062078240 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc90c0694b0 con 0x7fc8f0003a00 2024-05-11T12:13:08.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.374+0000 7fc902000640 1 -- 172.21.15.167:0/2062078240 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc8fc042020 con 0x7fc8f0003a00 2024-05-11T12:13:08.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.375+0000 7fc902000640 1 --2- 172.21.15.167:0/2062078240 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc8e8076300 0x7fc8e80787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:08.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.375+0000 7fc902000640 1 -- 172.21.15.167:0/2062078240 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7fc8fc0b75f0 con 0x7fc8f0003a00 2024-05-11T12:13:08.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.375+0000 7fc909400640 1 --2- 172.21.15.167:0/2062078240 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc8e8076300 0x7fc8e80787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:08.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.376+0000 7fc909400640 1 --2- 172.21.15.167:0/2062078240 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc8e8076300 0x7fc8e80787c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc90c13be40 tx=0x7fc8f4001510 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:08.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.380+0000 7fc902000640 1 -- 172.21.15.167:0/2062078240 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc8fc049070 con 0x7fc8f0003a00 2024-05-11T12:13:08.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.584+0000 7fc90b200640 1 -- 172.21.15.167:0/2062078240 --> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc90c0b5d80 con 0x7fc8e8076300 2024-05-11T12:13:08.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.604+0000 7fc902000640 1 -- 172.21.15.167:0/2062078240 <== mgr.4106 v2:172.21.15.167:6832/578352559 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250460 (secure 0 0 0) 0x7fc90c0b5d80 con 0x7fc8e8076300 2024-05-11T12:13:08.608 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:13:08.610 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2024-05-11T12:13:08.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.610+0000 7fc90b200640 1 -- 172.21.15.167:0/2062078240 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc8e8076300 msgr2=0x7fc8e80787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:08.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.610+0000 7fc90b200640 1 --2- 172.21.15.167:0/2062078240 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc8e8076300 0x7fc8e80787c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc90c13be40 tx=0x7fc8f4001510 comp rx=0 tx=0).stop 2024-05-11T12:13:08.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.610+0000 7fc90b200640 1 -- 172.21.15.167:0/2062078240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8f0003a00 msgr2=0x7fc90c156650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:08.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.610+0000 7fc90b200640 1 --2- 172.21.15.167:0/2062078240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8f0003a00 0x7fc90c156650 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc009be0 tx=0x7fc8fc004730 comp rx=0 tx=0).stop 2024-05-11T12:13:08.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.610+0000 7fc90b200640 1 -- 172.21.15.167:0/2062078240 shutdown_connections 2024-05-11T12:13:08.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.610+0000 7fc90b200640 1 --2- 172.21.15.167:0/2062078240 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc90c1366f0 0x7fc90c136b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:08.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.610+0000 7fc90b200640 1 --2- 172.21.15.167:0/2062078240 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc8e8076300 0x7fc8e80787c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:08.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.610+0000 7fc90b200640 1 --2- 172.21.15.167:0/2062078240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc90c12f9b0 0x7fc90c12fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:08.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.610+0000 7fc90b200640 1 --2- 172.21.15.167:0/2062078240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8f0003a00 0x7fc90c156650 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:08.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.610+0000 7fc90b200640 1 -- 172.21.15.167:0/2062078240 >> 172.21.15.167:0/2062078240 conn(0x7fc90c0b3460 msgr2=0x7fc90c0cf260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:08.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.610+0000 7fc90b200640 1 -- 172.21.15.167:0/2062078240 shutdown_connections 2024-05-11T12:13:08.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.610+0000 7fc90b200640 1 -- 172.21.15.167:0/2062078240 wait complete. 2024-05-11T12:13:08.629 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":135,"stamp":"2024-05-11T12:13:07.279017+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590584,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":128,"num_write_kb":1638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91,"ondisk_log_size":91,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":93,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223108,"kb_used_data":6920,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751612,"statfs":{"total":773094113280,"available":772865650688,"internally_reserved":0,"allocated":7086080,"data_stored":3787685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001654"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.104414+0000","last_change":"2024-05-11T12:13:02.104414+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.104414+0000","last_undegraded":"2024-05-11T12:13:02.104414+0000","last_fullsized":"2024-05-11T12:13:02.104414+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.102208+0000","last_change":"2024-05-11T12:13:02.102208+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.102208+0000","last_undegraded":"2024-05-11T12:13:02.102208+0000","last_fullsized":"2024-05-11T12:13:02.102208+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.180687+0000","last_change":"2024-05-11T12:13:03.165899+0000","last_active":"2024-05-11T12:13:03.180687+0000","last_peered":"2024-05-11T12:13:03.180687+0000","last_clean":"2024-05-11T12:13:03.180687+0000","last_became_active":"2024-05-11T12:13:03.165494+0000","last_became_peered":"2024-05-11T12:13:03.165494+0000","last_unstale":"2024-05-11T12:13:03.180687+0000","last_undegraded":"2024-05-11T12:13:03.180687+0000","last_fullsized":"2024-05-11T12:13:03.180687+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:04:05.997706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.174472+0000","last_change":"2024-05-11T12:13:03.166762+0000","last_active":"2024-05-11T12:13:03.174472+0000","last_peered":"2024-05-11T12:13:03.174472+0000","last_clean":"2024-05-11T12:13:03.174472+0000","last_became_active":"2024-05-11T12:13:03.166457+0000","last_became_peered":"2024-05-11T12:13:03.166457+0000","last_unstale":"2024-05-11T12:13:03.174472+0000","last_undegraded":"2024-05-11T12:13:03.174472+0000","last_fullsized":"2024-05-11T12:13:03.174472+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:46:22.393593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.173955+0000","last_change":"2024-05-11T12:13:03.167675+0000","last_active":"2024-05-11T12:13:03.173955+0000","last_peered":"2024-05-11T12:13:03.173955+0000","last_clean":"2024-05-11T12:13:03.173955+0000","last_became_active":"2024-05-11T12:13:03.167381+0000","last_became_peered":"2024-05-11T12:13:03.167381+0000","last_unstale":"2024-05-11T12:13:03.173955+0000","last_undegraded":"2024-05-11T12:13:03.173955+0000","last_fullsized":"2024-05-11T12:13:03.173955+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:06:40.718612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.121564+0000","last_change":"2024-05-11T12:13:02.121564+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.121564+0000","last_undegraded":"2024-05-11T12:13:02.121564+0000","last_fullsized":"2024-05-11T12:13:02.121564+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.173969+0000","last_change":"2024-05-11T12:13:03.170170+0000","last_active":"2024-05-11T12:13:04.173969+0000","last_peered":"2024-05-11T12:13:04.173969+0000","last_clean":"2024-05-11T12:13:04.173969+0000","last_became_active":"2024-05-11T12:13:03.169888+0000","last_became_peered":"2024-05-11T12:13:03.169888+0000","last_unstale":"2024-05-11T12:13:04.173969+0000","last_undegraded":"2024-05-11T12:13:04.173969+0000","last_fullsized":"2024-05-11T12:13:04.173969+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:46:12.686412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.174546+0000","last_change":"2024-05-11T12:13:03.167147+0000","last_active":"2024-05-11T12:13:03.174546+0000","last_peered":"2024-05-11T12:13:03.174546+0000","last_clean":"2024-05-11T12:13:03.174546+0000","last_became_active":"2024-05-11T12:13:03.166882+0000","last_became_peered":"2024-05-11T12:13:03.166882+0000","last_unstale":"2024-05-11T12:13:03.174546+0000","last_undegraded":"2024-05-11T12:13:03.174546+0000","last_fullsized":"2024-05-11T12:13:03.174546+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:04:05.997706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.180451+0000","last_change":"2024-05-11T12:13:03.162128+0000","last_active":"2024-05-11T12:13:03.180451+0000","last_peered":"2024-05-11T12:13:03.180451+0000","last_clean":"2024-05-11T12:13:03.180451+0000","last_became_active":"2024-05-11T12:13:03.161841+0000","last_became_peered":"2024-05-11T12:13:03.161841+0000","last_unstale":"2024-05-11T12:13:03.180451+0000","last_undegraded":"2024-05-11T12:13:03.180451+0000","last_fullsized":"2024-05-11T12:13:03.180451+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:27:14.194818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.169523+0000","last_change":"2024-05-11T12:13:03.164933+0000","last_active":"2024-05-11T12:13:03.169523+0000","last_peered":"2024-05-11T12:13:03.169523+0000","last_clean":"2024-05-11T12:13:03.169523+0000","last_became_active":"2024-05-11T12:13:03.164680+0000","last_became_peered":"2024-05-11T12:13:03.164680+0000","last_unstale":"2024-05-11T12:13:03.169523+0000","last_undegraded":"2024-05-11T12:13:03.169523+0000","last_fullsized":"2024-05-11T12:13:03.169523+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:08:16.138838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.174853+0000","last_change":"2024-05-11T12:13:03.163978+0000","last_active":"2024-05-11T12:13:03.174853+0000","last_peered":"2024-05-11T12:13:03.174853+0000","last_clean":"2024-05-11T12:13:03.174853+0000","last_became_active":"2024-05-11T12:13:03.163688+0000","last_became_peered":"2024-05-11T12:13:03.163688+0000","last_unstale":"2024-05-11T12:13:03.174853+0000","last_undegraded":"2024-05-11T12:13:03.174853+0000","last_fullsized":"2024-05-11T12:13:03.174853+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:15:35.851039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.170759+0000","last_change":"2024-05-11T12:13:03.164295+0000","last_active":"2024-05-11T12:13:03.170759+0000","last_peered":"2024-05-11T12:13:03.170759+0000","last_clean":"2024-05-11T12:13:03.170759+0000","last_became_active":"2024-05-11T12:13:03.163948+0000","last_became_peered":"2024-05-11T12:13:03.163948+0000","last_unstale":"2024-05-11T12:13:03.170759+0000","last_undegraded":"2024-05-11T12:13:03.170759+0000","last_fullsized":"2024-05-11T12:13:03.170759+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:24:40.432373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.170701+0000","last_change":"2024-05-11T12:13:03.163777+0000","last_active":"2024-05-11T12:13:03.170701+0000","last_peered":"2024-05-11T12:13:03.170701+0000","last_clean":"2024-05-11T12:13:03.170701+0000","last_became_active":"2024-05-11T12:13:03.163472+0000","last_became_peered":"2024-05-11T12:13:03.163472+0000","last_unstale":"2024-05-11T12:13:03.170701+0000","last_undegraded":"2024-05-11T12:13:03.170701+0000","last_fullsized":"2024-05-11T12:13:03.170701+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:58:08.814828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.169784+0000","last_change":"2024-05-11T12:13:03.165418+0000","last_active":"2024-05-11T12:13:03.169784+0000","last_peered":"2024-05-11T12:13:03.169784+0000","last_clean":"2024-05-11T12:13:03.169784+0000","last_became_active":"2024-05-11T12:13:03.165020+0000","last_became_peered":"2024-05-11T12:13:03.165020+0000","last_unstale":"2024-05-11T12:13:03.169784+0000","last_undegraded":"2024-05-11T12:13:03.169784+0000","last_fullsized":"2024-05-11T12:13:03.169784+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:24:40.432373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.182452+0000","last_change":"2024-05-11T12:13:03.174279+0000","last_active":"2024-05-11T12:13:03.182452+0000","last_peered":"2024-05-11T12:13:03.182452+0000","last_clean":"2024-05-11T12:13:03.182452+0000","last_became_active":"2024-05-11T12:13:03.173960+0000","last_became_peered":"2024-05-11T12:13:03.173960+0000","last_unstale":"2024-05-11T12:13:03.182452+0000","last_undegraded":"2024-05-11T12:13:03.182452+0000","last_fullsized":"2024-05-11T12:13:03.182452+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:29:58.089646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.173798+0000","last_change":"2024-05-11T12:13:03.176759+0000","last_active":"2024-05-11T12:13:04.173798+0000","last_peered":"2024-05-11T12:13:04.173798+0000","last_clean":"2024-05-11T12:13:04.173798+0000","last_became_active":"2024-05-11T12:13:03.176496+0000","last_became_peered":"2024-05-11T12:13:03.176496+0000","last_unstale":"2024-05-11T12:13:04.173798+0000","last_undegraded":"2024-05-11T12:13:04.173798+0000","last_fullsized":"2024-05-11T12:13:04.173798+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:33:36.274987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.103891+0000","last_change":"2024-05-11T12:13:02.103891+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.103891+0000","last_undegraded":"2024-05-11T12:13:02.103891+0000","last_fullsized":"2024-05-11T12:13:02.103891+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.175933+0000","last_change":"2024-05-11T12:13:03.167581+0000","last_active":"2024-05-11T12:13:03.175933+0000","last_peered":"2024-05-11T12:13:03.175933+0000","last_clean":"2024-05-11T12:13:03.175933+0000","last_became_active":"2024-05-11T12:13:03.167285+0000","last_became_peered":"2024-05-11T12:13:03.167285+0000","last_unstale":"2024-05-11T12:13:03.175933+0000","last_undegraded":"2024-05-11T12:13:03.175933+0000","last_fullsized":"2024-05-11T12:13:03.175933+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:52:39.367286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.175530+0000","last_change":"2024-05-11T12:13:03.172989+0000","last_active":"2024-05-11T12:13:03.175530+0000","last_peered":"2024-05-11T12:13:03.175530+0000","last_clean":"2024-05-11T12:13:03.175530+0000","last_became_active":"2024-05-11T12:13:03.172704+0000","last_became_peered":"2024-05-11T12:13:03.172704+0000","last_unstale":"2024-05-11T12:13:03.175530+0000","last_undegraded":"2024-05-11T12:13:03.175530+0000","last_fullsized":"2024-05-11T12:13:03.175530+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:28:34.473489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.116797+0000","last_change":"2024-05-11T12:13:02.116797+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.116797+0000","last_undegraded":"2024-05-11T12:13:02.116797+0000","last_fullsized":"2024-05-11T12:13:02.116797+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.170678+0000","last_change":"2024-05-11T12:13:03.162655+0000","last_active":"2024-05-11T12:13:03.170678+0000","last_peered":"2024-05-11T12:13:03.170678+0000","last_clean":"2024-05-11T12:13:03.170678+0000","last_became_active":"2024-05-11T12:13:03.162259+0000","last_became_peered":"2024-05-11T12:13:03.162259+0000","last_unstale":"2024-05-11T12:13:03.170678+0000","last_undegraded":"2024-05-11T12:13:03.170678+0000","last_fullsized":"2024-05-11T12:13:03.170678+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:42:21.959847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.116334+0000","last_change":"2024-05-11T12:13:02.116334+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.116334+0000","last_undegraded":"2024-05-11T12:13:02.116334+0000","last_fullsized":"2024-05-11T12:13:02.116334+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.115725+0000","last_change":"2024-05-11T12:13:02.115725+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.115725+0000","last_undegraded":"2024-05-11T12:13:02.115725+0000","last_fullsized":"2024-05-11T12:13:02.115725+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.171059+0000","last_change":"2024-05-11T12:13:03.165438+0000","last_active":"2024-05-11T12:13:03.171059+0000","last_peered":"2024-05-11T12:13:03.171059+0000","last_clean":"2024-05-11T12:13:03.171059+0000","last_became_active":"2024-05-11T12:13:03.165118+0000","last_became_peered":"2024-05-11T12:13:03.165118+0000","last_unstale":"2024-05-11T12:13:03.171059+0000","last_undegraded":"2024-05-11T12:13:03.171059+0000","last_fullsized":"2024-05-11T12:13:03.171059+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:56:37.784627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.176455+0000","last_change":"2024-05-11T12:13:03.169517+0000","last_active":"2024-05-11T12:13:03.176455+0000","last_peered":"2024-05-11T12:13:03.176455+0000","last_clean":"2024-05-11T12:13:03.176455+0000","last_became_active":"2024-05-11T12:13:03.169235+0000","last_became_peered":"2024-05-11T12:13:03.169235+0000","last_unstale":"2024-05-11T12:13:03.176455+0000","last_undegraded":"2024-05-11T12:13:03.176455+0000","last_fullsized":"2024-05-11T12:13:03.176455+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:27:14.194818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.174469+0000","last_change":"2024-05-11T12:13:03.166968+0000","last_active":"2024-05-11T12:13:03.174469+0000","last_peered":"2024-05-11T12:13:03.174469+0000","last_clean":"2024-05-11T12:13:03.174469+0000","last_became_active":"2024-05-11T12:13:03.166704+0000","last_became_peered":"2024-05-11T12:13:03.166704+0000","last_unstale":"2024-05-11T12:13:03.174469+0000","last_undegraded":"2024-05-11T12:13:03.174469+0000","last_fullsized":"2024-05-11T12:13:03.174469+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:56:37.784627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.178640+0000","last_change":"2024-05-11T12:13:03.171885+0000","last_active":"2024-05-11T12:13:03.178640+0000","last_peered":"2024-05-11T12:13:03.178640+0000","last_clean":"2024-05-11T12:13:03.178640+0000","last_became_active":"2024-05-11T12:13:03.171590+0000","last_became_peered":"2024-05-11T12:13:03.171590+0000","last_unstale":"2024-05-11T12:13:03.178640+0000","last_undegraded":"2024-05-11T12:13:03.178640+0000","last_fullsized":"2024-05-11T12:13:03.178640+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:06:40.718612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.103639+0000","last_change":"2024-05-11T12:13:02.103639+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.103639+0000","last_undegraded":"2024-05-11T12:13:02.103639+0000","last_fullsized":"2024-05-11T12:13:02.103639+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.164436+0000","last_change":"2024-05-11T12:13:03.160607+0000","last_active":"2024-05-11T12:13:03.164436+0000","last_peered":"2024-05-11T12:13:03.164436+0000","last_clean":"2024-05-11T12:13:03.164436+0000","last_became_active":"2024-05-11T12:13:03.160281+0000","last_became_peered":"2024-05-11T12:13:03.160281+0000","last_unstale":"2024-05-11T12:13:03.164436+0000","last_undegraded":"2024-05-11T12:13:03.164436+0000","last_fullsized":"2024-05-11T12:13:03.164436+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:12:30.081833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.170126+0000","last_change":"2024-05-11T12:13:03.164270+0000","last_active":"2024-05-11T12:13:03.170126+0000","last_peered":"2024-05-11T12:13:03.170126+0000","last_clean":"2024-05-11T12:13:03.170126+0000","last_became_active":"2024-05-11T12:13:03.163977+0000","last_became_peered":"2024-05-11T12:13:03.163977+0000","last_unstale":"2024-05-11T12:13:03.170126+0000","last_undegraded":"2024-05-11T12:13:03.170126+0000","last_fullsized":"2024-05-11T12:13:03.170126+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:08:16.138838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.177851+0000","last_change":"2024-05-11T12:13:03.171792+0000","last_active":"2024-05-11T12:13:03.177851+0000","last_peered":"2024-05-11T12:13:03.177851+0000","last_clean":"2024-05-11T12:13:03.177851+0000","last_became_active":"2024-05-11T12:13:03.171502+0000","last_became_peered":"2024-05-11T12:13:03.171502+0000","last_unstale":"2024-05-11T12:13:03.177851+0000","last_undegraded":"2024-05-11T12:13:03.177851+0000","last_fullsized":"2024-05-11T12:13:03.177851+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:20:41.663583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.121306+0000","last_change":"2024-05-11T12:13:02.121306+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.121306+0000","last_undegraded":"2024-05-11T12:13:02.121306+0000","last_fullsized":"2024-05-11T12:13:02.121306+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.177562+0000","last_change":"2024-05-11T12:13:03.176021+0000","last_active":"2024-05-11T12:13:03.177562+0000","last_peered":"2024-05-11T12:13:03.177562+0000","last_clean":"2024-05-11T12:13:03.177562+0000","last_became_active":"2024-05-11T12:13:03.175708+0000","last_became_peered":"2024-05-11T12:13:03.175708+0000","last_unstale":"2024-05-11T12:13:03.177562+0000","last_undegraded":"2024-05-11T12:13:03.177562+0000","last_fullsized":"2024-05-11T12:13:03.177562+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:12:07.890717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.140559+0000","last_change":"2024-05-11T12:13:02.136461+0000","last_active":"2024-05-11T12:13:02.140559+0000","last_peered":"2024-05-11T12:13:02.140559+0000","last_clean":"2024-05-11T12:13:02.140559+0000","last_became_active":"2024-05-11T12:13:02.136210+0000","last_became_peered":"2024-05-11T12:13:02.136210+0000","last_unstale":"2024-05-11T12:13:02.140559+0000","last_undegraded":"2024-05-11T12:13:02.140559+0000","last_fullsized":"2024-05-11T12:13:02.140559+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:20:40.658655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.159893+0000","last_change":"2024-05-11T12:13:03.158314+0000","last_active":"2024-05-11T12:13:03.159893+0000","last_peered":"2024-05-11T12:13:03.159893+0000","last_clean":"2024-05-11T12:13:03.159893+0000","last_became_active":"2024-05-11T12:13:03.157991+0000","last_became_peered":"2024-05-11T12:13:03.157991+0000","last_unstale":"2024-05-11T12:13:03.159893+0000","last_undegraded":"2024-05-11T12:13:03.159893+0000","last_fullsized":"2024-05-11T12:13:03.159893+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:55:20.561101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.140990+0000","last_change":"2024-05-11T12:13:02.136740+0000","last_active":"2024-05-11T12:13:02.140990+0000","last_peered":"2024-05-11T12:13:02.140990+0000","last_clean":"2024-05-11T12:13:02.140990+0000","last_became_active":"2024-05-11T12:13:02.136446+0000","last_became_peered":"2024-05-11T12:13:02.136446+0000","last_unstale":"2024-05-11T12:13:02.140990+0000","last_undegraded":"2024-05-11T12:13:02.140990+0000","last_fullsized":"2024-05-11T12:13:02.140990+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:58:07.809901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.120695+0000","last_change":"2024-05-11T12:13:02.120695+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.120695+0000","last_undegraded":"2024-05-11T12:13:02.120695+0000","last_fullsized":"2024-05-11T12:13:02.120695+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.137480+0000","last_change":"2024-05-11T12:13:02.129658+0000","last_active":"2024-05-11T12:13:02.137480+0000","last_peered":"2024-05-11T12:13:02.137480+0000","last_clean":"2024-05-11T12:13:02.137480+0000","last_became_active":"2024-05-11T12:13:02.129326+0000","last_became_peered":"2024-05-11T12:13:02.129326+0000","last_unstale":"2024-05-11T12:13:02.137480+0000","last_undegraded":"2024-05-11T12:13:02.137480+0000","last_fullsized":"2024-05-11T12:13:02.137480+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:42:20.954919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.168978+0000","last_change":"2024-05-11T12:13:03.162149+0000","last_active":"2024-05-11T12:13:03.168978+0000","last_peered":"2024-05-11T12:13:03.168978+0000","last_clean":"2024-05-11T12:13:03.168978+0000","last_became_active":"2024-05-11T12:13:03.161890+0000","last_became_peered":"2024-05-11T12:13:03.161890+0000","last_unstale":"2024-05-11T12:13:03.168978+0000","last_undegraded":"2024-05-11T12:13:03.168978+0000","last_fullsized":"2024-05-11T12:13:03.168978+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:42:21.959847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.134444+0000","last_change":"2024-05-11T12:13:02.126473+0000","last_active":"2024-05-11T12:13:02.134444+0000","last_peered":"2024-05-11T12:13:02.134444+0000","last_clean":"2024-05-11T12:13:02.134444+0000","last_became_active":"2024-05-11T12:13:02.126154+0000","last_became_peered":"2024-05-11T12:13:02.126154+0000","last_unstale":"2024-05-11T12:13:02.134444+0000","last_undegraded":"2024-05-11T12:13:02.134444+0000","last_fullsized":"2024-05-11T12:13:02.134444+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:15:28.704713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.173923+0000","last_change":"2024-05-11T12:13:03.164511+0000","last_active":"2024-05-11T12:13:04.173923+0000","last_peered":"2024-05-11T12:13:04.173923+0000","last_clean":"2024-05-11T12:13:04.173923+0000","last_became_active":"2024-05-11T12:13:03.164197+0000","last_became_peered":"2024-05-11T12:13:03.164197+0000","last_unstale":"2024-05-11T12:13:04.173923+0000","last_undegraded":"2024-05-11T12:13:04.173923+0000","last_fullsized":"2024-05-11T12:13:04.173923+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:54:15.999941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.125903+0000","last_change":"2024-05-11T12:13:02.119338+0000","last_active":"2024-05-11T12:13:02.125903+0000","last_peered":"2024-05-11T12:13:02.125903+0000","last_clean":"2024-05-11T12:13:02.125903+0000","last_became_active":"2024-05-11T12:13:02.118127+0000","last_became_peered":"2024-05-11T12:13:02.118127+0000","last_unstale":"2024-05-11T12:13:02.125903+0000","last_undegraded":"2024-05-11T12:13:02.125903+0000","last_fullsized":"2024-05-11T12:13:02.125903+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:56:36.779700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.104889+0000","last_change":"2024-05-11T12:13:02.104889+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.104889+0000","last_undegraded":"2024-05-11T12:13:02.104889+0000","last_fullsized":"2024-05-11T12:13:02.104889+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.125968+0000","last_change":"2024-05-11T12:13:02.121903+0000","last_active":"2024-05-11T12:13:02.125968+0000","last_peered":"2024-05-11T12:13:02.125968+0000","last_clean":"2024-05-11T12:13:02.125968+0000","last_became_active":"2024-05-11T12:13:02.120293+0000","last_became_peered":"2024-05-11T12:13:02.120293+0000","last_unstale":"2024-05-11T12:13:02.125968+0000","last_undegraded":"2024-05-11T12:13:02.125968+0000","last_fullsized":"2024-05-11T12:13:02.125968+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:04:04.992779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.174363+0000","last_change":"2024-05-11T12:13:03.168572+0000","last_active":"2024-05-11T12:13:03.174363+0000","last_peered":"2024-05-11T12:13:03.174363+0000","last_clean":"2024-05-11T12:13:03.174363+0000","last_became_active":"2024-05-11T12:13:03.167854+0000","last_became_peered":"2024-05-11T12:13:03.167854+0000","last_unstale":"2024-05-11T12:13:03.174363+0000","last_undegraded":"2024-05-11T12:13:03.174363+0000","last_fullsized":"2024-05-11T12:13:03.174363+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:27:27.374412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.143860+0000","last_change":"2024-05-11T12:13:02.139509+0000","last_active":"2024-05-11T12:13:02.143860+0000","last_peered":"2024-05-11T12:13:02.143860+0000","last_clean":"2024-05-11T12:13:02.143860+0000","last_became_active":"2024-05-11T12:13:02.139201+0000","last_became_peered":"2024-05-11T12:13:02.139201+0000","last_unstale":"2024-05-11T12:13:02.143860+0000","last_undegraded":"2024-05-11T12:13:02.143860+0000","last_fullsized":"2024-05-11T12:13:02.143860+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:55:25.067740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.115715+0000","last_change":"2024-05-11T12:13:02.115715+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.115715+0000","last_undegraded":"2024-05-11T12:13:02.115715+0000","last_fullsized":"2024-05-11T12:13:02.115715+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.147186+0000","last_change":"2024-05-11T12:13:02.139692+0000","last_active":"2024-05-11T12:13:03.147186+0000","last_peered":"2024-05-11T12:13:03.147186+0000","last_clean":"2024-05-11T12:13:03.147186+0000","last_became_active":"2024-05-11T12:13:02.137935+0000","last_became_peered":"2024-05-11T12:13:02.137935+0000","last_unstale":"2024-05-11T12:13:03.147186+0000","last_undegraded":"2024-05-11T12:13:03.147186+0000","last_fullsized":"2024-05-11T12:13:03.147186+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:15:34.846112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.103086+0000","last_change":"2024-05-11T12:13:02.103086+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.103086+0000","last_undegraded":"2024-05-11T12:13:02.103086+0000","last_fullsized":"2024-05-11T12:13:02.103086+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.154876+0000","last_change":"2024-05-11T12:13:02.142438+0000","last_active":"2024-05-11T12:13:03.154876+0000","last_peered":"2024-05-11T12:13:03.154876+0000","last_clean":"2024-05-11T12:13:03.154876+0000","last_became_active":"2024-05-11T12:13:02.142125+0000","last_became_peered":"2024-05-11T12:13:02.142125+0000","last_unstale":"2024-05-11T12:13:03.154876+0000","last_undegraded":"2024-05-11T12:13:03.154876+0000","last_fullsized":"2024-05-11T12:13:03.154876+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:55:32.333055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.116293+0000","last_change":"2024-05-11T12:13:02.116293+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.116293+0000","last_undegraded":"2024-05-11T12:13:02.116293+0000","last_fullsized":"2024-05-11T12:13:02.116293+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.144040+0000","last_change":"2024-05-11T12:13:02.140895+0000","last_active":"2024-05-11T12:13:03.144040+0000","last_peered":"2024-05-11T12:13:03.144040+0000","last_clean":"2024-05-11T12:13:03.144040+0000","last_became_active":"2024-05-11T12:13:02.138733+0000","last_became_peered":"2024-05-11T12:13:02.138733+0000","last_unstale":"2024-05-11T12:13:03.144040+0000","last_undegraded":"2024-05-11T12:13:03.144040+0000","last_fullsized":"2024-05-11T12:13:03.144040+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:08:47.140652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.171236+0000","last_change":"2024-05-11T12:13:03.160320+0000","last_active":"2024-05-11T12:13:03.171236+0000","last_peered":"2024-05-11T12:13:03.171236+0000","last_clean":"2024-05-11T12:13:03.171236+0000","last_became_active":"2024-05-11T12:13:03.159984+0000","last_became_peered":"2024-05-11T12:13:03.159984+0000","last_unstale":"2024-05-11T12:13:03.171236+0000","last_undegraded":"2024-05-11T12:13:03.171236+0000","last_fullsized":"2024-05-11T12:13:03.171236+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:04:22.163536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"63'3","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.185645+0000","last_change":"2024-05-11T12:13:02.139752+0000","last_active":"2024-05-11T12:13:03.185645+0000","last_peered":"2024-05-11T12:13:03.185645+0000","last_clean":"2024-05-11T12:13:03.185645+0000","last_became_active":"2024-05-11T12:13:02.139431+0000","last_became_peered":"2024-05-11T12:13:02.139431+0000","last_unstale":"2024-05-11T12:13:03.185645+0000","last_undegraded":"2024-05-11T12:13:03.185645+0000","last_fullsized":"2024-05-11T12:13:03.185645+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:00:08.081754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.174833+0000","last_change":"2024-05-11T12:13:03.166818+0000","last_active":"2024-05-11T12:13:03.174833+0000","last_peered":"2024-05-11T12:13:03.174833+0000","last_clean":"2024-05-11T12:13:03.174833+0000","last_became_active":"2024-05-11T12:13:03.166560+0000","last_became_peered":"2024-05-11T12:13:03.166560+0000","last_unstale":"2024-05-11T12:13:03.174833+0000","last_undegraded":"2024-05-11T12:13:03.174833+0000","last_fullsized":"2024-05-11T12:13:03.174833+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:04:22.163536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.137897+0000","last_change":"2024-05-11T12:13:02.133552+0000","last_active":"2024-05-11T12:13:02.137897+0000","last_peered":"2024-05-11T12:13:02.137897+0000","last_clean":"2024-05-11T12:13:02.137897+0000","last_became_active":"2024-05-11T12:13:02.133236+0000","last_became_peered":"2024-05-11T12:13:02.133236+0000","last_unstale":"2024-05-11T12:13:02.137897+0000","last_undegraded":"2024-05-11T12:13:02.137897+0000","last_fullsized":"2024-05-11T12:13:02.137897+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:32:44.989980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.175555+0000","last_change":"2024-05-11T12:13:03.168373+0000","last_active":"2024-05-11T12:13:03.175555+0000","last_peered":"2024-05-11T12:13:03.175555+0000","last_clean":"2024-05-11T12:13:03.175555+0000","last_became_active":"2024-05-11T12:13:03.168105+0000","last_became_peered":"2024-05-11T12:13:03.168105+0000","last_unstale":"2024-05-11T12:13:03.175555+0000","last_undegraded":"2024-05-11T12:13:03.175555+0000","last_fullsized":"2024-05-11T12:13:03.175555+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:32:45.994907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"63'2","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.183854+0000","last_change":"2024-05-11T12:13:02.143192+0000","last_active":"2024-05-11T12:13:03.183854+0000","last_peered":"2024-05-11T12:13:03.183854+0000","last_clean":"2024-05-11T12:13:03.183854+0000","last_became_active":"2024-05-11T12:13:02.142901+0000","last_became_peered":"2024-05-11T12:13:02.142901+0000","last_unstale":"2024-05-11T12:13:03.183854+0000","last_undegraded":"2024-05-11T12:13:03.183854+0000","last_fullsized":"2024-05-11T12:13:03.183854+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:39:03.473134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.155659+0000","last_change":"2024-05-11T12:13:03.145731+0000","last_active":"2024-05-11T12:13:03.155659+0000","last_peered":"2024-05-11T12:13:03.155659+0000","last_clean":"2024-05-11T12:13:03.155659+0000","last_became_active":"2024-05-11T12:13:03.145419+0000","last_became_peered":"2024-05-11T12:13:03.145419+0000","last_unstale":"2024-05-11T12:13:03.155659+0000","last_undegraded":"2024-05-11T12:13:03.155659+0000","last_fullsized":"2024-05-11T12:13:03.155659+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:55:33.337983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.144327+0000","last_change":"2024-05-11T12:13:02.118817+0000","last_active":"2024-05-11T12:13:03.144327+0000","last_peered":"2024-05-11T12:13:03.144327+0000","last_clean":"2024-05-11T12:13:03.144327+0000","last_became_active":"2024-05-11T12:13:02.118040+0000","last_became_peered":"2024-05-11T12:13:02.118040+0000","last_unstale":"2024-05-11T12:13:03.144327+0000","last_undegraded":"2024-05-11T12:13:03.144327+0000","last_fullsized":"2024-05-11T12:13:03.144327+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:18:34.044830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.175410+0000","last_change":"2024-05-11T12:13:03.172604+0000","last_active":"2024-05-11T12:13:03.175410+0000","last_peered":"2024-05-11T12:13:03.175410+0000","last_clean":"2024-05-11T12:13:03.175410+0000","last_became_active":"2024-05-11T12:13:03.172198+0000","last_became_peered":"2024-05-11T12:13:03.172198+0000","last_unstale":"2024-05-11T12:13:03.175410+0000","last_undegraded":"2024-05-11T12:13:03.175410+0000","last_fullsized":"2024-05-11T12:13:03.175410+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:56:18.568549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.156874+0000","last_change":"2024-05-11T12:13:02.138604+0000","last_active":"2024-05-11T12:13:03.156874+0000","last_peered":"2024-05-11T12:13:03.156874+0000","last_clean":"2024-05-11T12:13:03.156874+0000","last_became_active":"2024-05-11T12:13:02.138227+0000","last_became_peered":"2024-05-11T12:13:02.138227+0000","last_unstale":"2024-05-11T12:13:03.156874+0000","last_undegraded":"2024-05-11T12:13:03.156874+0000","last_fullsized":"2024-05-11T12:13:03.156874+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:15:28.704713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.101871+0000","last_change":"2024-05-11T12:13:02.101871+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.101871+0000","last_undegraded":"2024-05-11T12:13:02.101871+0000","last_fullsized":"2024-05-11T12:13:02.101871+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"63'8","reported_seq":20,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.186047+0000","last_change":"2024-05-11T12:13:02.137326+0000","last_active":"2024-05-11T12:13:03.186047+0000","last_peered":"2024-05-11T12:13:03.186047+0000","last_clean":"2024-05-11T12:13:03.186047+0000","last_became_active":"2024-05-11T12:13:02.133763+0000","last_became_peered":"2024-05-11T12:13:02.133763+0000","last_unstale":"2024-05-11T12:13:03.186047+0000","last_undegraded":"2024-05-11T12:13:03.186047+0000","last_fullsized":"2024-05-11T12:13:03.186047+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:55:40.188834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.169808+0000","last_change":"2024-05-11T12:13:03.163731+0000","last_active":"2024-05-11T12:13:03.169808+0000","last_peered":"2024-05-11T12:13:03.169808+0000","last_clean":"2024-05-11T12:13:03.169808+0000","last_became_active":"2024-05-11T12:13:03.161280+0000","last_became_peered":"2024-05-11T12:13:03.161280+0000","last_unstale":"2024-05-11T12:13:03.169808+0000","last_undegraded":"2024-05-11T12:13:03.169808+0000","last_fullsized":"2024-05-11T12:13:03.169808+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:58:08.814828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"63'1","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.180608+0000","last_change":"2024-05-11T12:13:02.124463+0000","last_active":"2024-05-11T12:13:03.180608+0000","last_peered":"2024-05-11T12:13:03.180608+0000","last_clean":"2024-05-11T12:13:03.180608+0000","last_became_active":"2024-05-11T12:13:02.124136+0000","last_became_peered":"2024-05-11T12:13:02.124136+0000","last_unstale":"2024-05-11T12:13:03.180608+0000","last_undegraded":"2024-05-11T12:13:03.180608+0000","last_fullsized":"2024-05-11T12:13:03.180608+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:48:52.177182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.161587+0000","last_change":"2024-05-11T12:13:03.158635+0000","last_active":"2024-05-11T12:13:03.161587+0000","last_peered":"2024-05-11T12:13:03.161587+0000","last_clean":"2024-05-11T12:13:03.161587+0000","last_became_active":"2024-05-11T12:13:03.158311+0000","last_became_peered":"2024-05-11T12:13:03.158311+0000","last_unstale":"2024-05-11T12:13:03.161587+0000","last_undegraded":"2024-05-11T12:13:03.161587+0000","last_fullsized":"2024-05-11T12:13:03.161587+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:46:22.393593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"63'2","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.182624+0000","last_change":"2024-05-11T12:13:02.143315+0000","last_active":"2024-05-11T12:13:03.182624+0000","last_peered":"2024-05-11T12:13:03.182624+0000","last_clean":"2024-05-11T12:13:03.182624+0000","last_became_active":"2024-05-11T12:13:02.143012+0000","last_became_peered":"2024-05-11T12:13:02.143012+0000","last_unstale":"2024-05-11T12:13:03.182624+0000","last_undegraded":"2024-05-11T12:13:03.182624+0000","last_fullsized":"2024-05-11T12:13:03.182624+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:03:26.831522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.172645+0000","last_change":"2024-05-11T12:13:03.159923+0000","last_active":"2024-05-11T12:13:03.172645+0000","last_peered":"2024-05-11T12:13:03.172645+0000","last_clean":"2024-05-11T12:13:03.172645+0000","last_became_active":"2024-05-11T12:13:03.159119+0000","last_became_peered":"2024-05-11T12:13:03.159119+0000","last_unstale":"2024-05-11T12:13:03.172645+0000","last_undegraded":"2024-05-11T12:13:03.172645+0000","last_fullsized":"2024-05-11T12:13:03.172645+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:11:31.058373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"63'1","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.185684+0000","last_change":"2024-05-11T12:13:02.138872+0000","last_active":"2024-05-11T12:13:03.185684+0000","last_peered":"2024-05-11T12:13:03.185684+0000","last_clean":"2024-05-11T12:13:03.185684+0000","last_became_active":"2024-05-11T12:13:02.138589+0000","last_became_peered":"2024-05-11T12:13:02.138589+0000","last_unstale":"2024-05-11T12:13:03.185684+0000","last_undegraded":"2024-05-11T12:13:03.185684+0000","last_fullsized":"2024-05-11T12:13:03.185684+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:48:52.177182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.174205+0000","last_change":"2024-05-11T12:13:03.167395+0000","last_active":"2024-05-11T12:13:03.174205+0000","last_peered":"2024-05-11T12:13:03.174205+0000","last_clean":"2024-05-11T12:13:03.174205+0000","last_became_active":"2024-05-11T12:13:03.167111+0000","last_became_peered":"2024-05-11T12:13:03.167111+0000","last_unstale":"2024-05-11T12:13:03.174205+0000","last_undegraded":"2024-05-11T12:13:03.174205+0000","last_fullsized":"2024-05-11T12:13:03.174205+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:20:41.663583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.141325+0000","last_change":"2024-05-11T12:13:02.137581+0000","last_active":"2024-05-11T12:13:02.141325+0000","last_peered":"2024-05-11T12:13:02.141325+0000","last_clean":"2024-05-11T12:13:02.141325+0000","last_became_active":"2024-05-11T12:13:02.137260+0000","last_became_peered":"2024-05-11T12:13:02.137260+0000","last_unstale":"2024-05-11T12:13:02.141325+0000","last_undegraded":"2024-05-11T12:13:02.141325+0000","last_fullsized":"2024-05-11T12:13:02.141325+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:29:57.084718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.120434+0000","last_change":"2024-05-11T12:13:02.120434+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.120434+0000","last_undegraded":"2024-05-11T12:13:02.120434+0000","last_fullsized":"2024-05-11T12:13:02.120434+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.143793+0000","last_change":"2024-05-11T12:13:02.140367+0000","last_active":"2024-05-11T12:13:02.143793+0000","last_peered":"2024-05-11T12:13:02.143793+0000","last_clean":"2024-05-11T12:13:02.143793+0000","last_became_active":"2024-05-11T12:13:02.140069+0000","last_became_peered":"2024-05-11T12:13:02.140069+0000","last_unstale":"2024-05-11T12:13:02.143793+0000","last_undegraded":"2024-05-11T12:13:02.143793+0000","last_fullsized":"2024-05-11T12:13:02.143793+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:24:39.427445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.175442+0000","last_change":"2024-05-11T12:13:03.171936+0000","last_active":"2024-05-11T12:13:03.175442+0000","last_peered":"2024-05-11T12:13:03.175442+0000","last_clean":"2024-05-11T12:13:03.175442+0000","last_became_active":"2024-05-11T12:13:03.171669+0000","last_became_peered":"2024-05-11T12:13:03.171669+0000","last_unstale":"2024-05-11T12:13:03.175442+0000","last_undegraded":"2024-05-11T12:13:03.175442+0000","last_fullsized":"2024-05-11T12:13:03.175442+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:29:58.089646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.144512+0000","last_change":"2024-05-11T12:13:02.125827+0000","last_active":"2024-05-11T12:13:03.144512+0000","last_peered":"2024-05-11T12:13:03.144512+0000","last_clean":"2024-05-11T12:13:03.144512+0000","last_became_active":"2024-05-11T12:13:02.125520+0000","last_became_peered":"2024-05-11T12:13:02.125520+0000","last_unstale":"2024-05-11T12:13:03.144512+0000","last_undegraded":"2024-05-11T12:13:03.144512+0000","last_fullsized":"2024-05-11T12:13:03.144512+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:00:08.081754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.174385+0000","last_change":"2024-05-11T12:13:03.168791+0000","last_active":"2024-05-11T12:13:03.174385+0000","last_peered":"2024-05-11T12:13:03.174385+0000","last_clean":"2024-05-11T12:13:03.174385+0000","last_became_active":"2024-05-11T12:13:03.168255+0000","last_became_peered":"2024-05-11T12:13:03.168255+0000","last_unstale":"2024-05-11T12:13:03.174385+0000","last_undegraded":"2024-05-11T12:13:03.174385+0000","last_fullsized":"2024-05-11T12:13:03.174385+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:17:19.051848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.141028+0000","last_change":"2024-05-11T12:13:02.136770+0000","last_active":"2024-05-11T12:13:02.141028+0000","last_peered":"2024-05-11T12:13:02.141028+0000","last_clean":"2024-05-11T12:13:02.141028+0000","last_became_active":"2024-05-11T12:13:02.136465+0000","last_became_peered":"2024-05-11T12:13:02.136465+0000","last_unstale":"2024-05-11T12:13:02.141028+0000","last_undegraded":"2024-05-11T12:13:02.141028+0000","last_fullsized":"2024-05-11T12:13:02.141028+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:08:15.133910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.120491+0000","last_change":"2024-05-11T12:13:02.120491+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.120491+0000","last_undegraded":"2024-05-11T12:13:02.120491+0000","last_fullsized":"2024-05-11T12:13:02.120491+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"63'2","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.183009+0000","last_change":"2024-05-11T12:13:02.136984+0000","last_active":"2024-05-11T12:13:03.183009+0000","last_peered":"2024-05-11T12:13:03.183009+0000","last_clean":"2024-05-11T12:13:03.183009+0000","last_became_active":"2024-05-11T12:13:02.135837+0000","last_became_peered":"2024-05-11T12:13:02.135837+0000","last_unstale":"2024-05-11T12:13:03.183009+0000","last_undegraded":"2024-05-11T12:13:03.183009+0000","last_fullsized":"2024-05-11T12:13:03.183009+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:03:26.831522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.173813+0000","last_change":"2024-05-11T12:13:03.170696+0000","last_active":"2024-05-11T12:13:04.173813+0000","last_peered":"2024-05-11T12:13:04.173813+0000","last_clean":"2024-05-11T12:13:04.173813+0000","last_became_active":"2024-05-11T12:13:03.170411+0000","last_became_peered":"2024-05-11T12:13:03.170411+0000","last_unstale":"2024-05-11T12:13:04.173813+0000","last_undegraded":"2024-05-11T12:13:04.173813+0000","last_fullsized":"2024-05-11T12:13:04.173813+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:11:05.631699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.146050+0000","last_change":"2024-05-11T12:13:02.128082+0000","last_active":"2024-05-11T12:13:03.146050+0000","last_peered":"2024-05-11T12:13:03.146050+0000","last_clean":"2024-05-11T12:13:03.146050+0000","last_became_active":"2024-05-11T12:13:02.126292+0000","last_became_peered":"2024-05-11T12:13:02.126292+0000","last_unstale":"2024-05-11T12:13:03.146050+0000","last_undegraded":"2024-05-11T12:13:03.146050+0000","last_fullsized":"2024-05-11T12:13:03.146050+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:15:28.704713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.156288+0000","last_change":"2024-05-11T12:13:03.149098+0000","last_active":"2024-05-11T12:13:03.156288+0000","last_peered":"2024-05-11T12:13:03.156288+0000","last_clean":"2024-05-11T12:13:03.156288+0000","last_became_active":"2024-05-11T12:13:03.148358+0000","last_became_peered":"2024-05-11T12:13:03.148358+0000","last_unstale":"2024-05-11T12:13:03.156288+0000","last_undegraded":"2024-05-11T12:13:03.156288+0000","last_fullsized":"2024-05-11T12:13:03.156288+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:12:30.081833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.137459+0000","last_change":"2024-05-11T12:13:02.129624+0000","last_active":"2024-05-11T12:13:02.137459+0000","last_peered":"2024-05-11T12:13:02.137459+0000","last_clean":"2024-05-11T12:13:02.137459+0000","last_became_active":"2024-05-11T12:13:02.129286+0000","last_became_peered":"2024-05-11T12:13:02.129286+0000","last_unstale":"2024-05-11T12:13:02.137459+0000","last_undegraded":"2024-05-11T12:13:02.137459+0000","last_fullsized":"2024-05-11T12:13:02.137459+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:12:29.076905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.174228+0000","last_change":"2024-05-11T12:13:03.170501+0000","last_active":"2024-05-11T12:13:03.174228+0000","last_peered":"2024-05-11T12:13:03.174228+0000","last_clean":"2024-05-11T12:13:03.174228+0000","last_became_active":"2024-05-11T12:13:03.163918+0000","last_became_peered":"2024-05-11T12:13:03.163918+0000","last_unstale":"2024-05-11T12:13:03.174228+0000","last_undegraded":"2024-05-11T12:13:03.174228+0000","last_fullsized":"2024-05-11T12:13:03.174228+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:18:50.744350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.137839+0000","last_change":"2024-05-11T12:13:02.133532+0000","last_active":"2024-05-11T12:13:02.137839+0000","last_peered":"2024-05-11T12:13:02.137839+0000","last_clean":"2024-05-11T12:13:02.137839+0000","last_became_active":"2024-05-11T12:13:02.133209+0000","last_became_peered":"2024-05-11T12:13:02.133209+0000","last_unstale":"2024-05-11T12:13:02.137839+0000","last_undegraded":"2024-05-11T12:13:02.137839+0000","last_fullsized":"2024-05-11T12:13:02.137839+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:52:38.362358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.172256+0000","last_change":"2024-05-11T12:13:03.163919+0000","last_active":"2024-05-11T12:13:03.172256+0000","last_peered":"2024-05-11T12:13:03.172256+0000","last_clean":"2024-05-11T12:13:03.172256+0000","last_became_active":"2024-05-11T12:13:03.163595+0000","last_became_peered":"2024-05-11T12:13:03.163595+0000","last_unstale":"2024-05-11T12:13:03.172256+0000","last_undegraded":"2024-05-11T12:13:03.172256+0000","last_fullsized":"2024-05-11T12:13:03.172256+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:52:39.367286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.152331+0000","last_change":"2024-05-11T12:13:02.139859+0000","last_active":"2024-05-11T12:13:03.152331+0000","last_peered":"2024-05-11T12:13:03.152331+0000","last_clean":"2024-05-11T12:13:03.152331+0000","last_became_active":"2024-05-11T12:13:02.139265+0000","last_became_peered":"2024-05-11T12:13:02.139265+0000","last_unstale":"2024-05-11T12:13:03.152331+0000","last_undegraded":"2024-05-11T12:13:03.152331+0000","last_fullsized":"2024-05-11T12:13:03.152331+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:18:34.044830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.172282+0000","last_change":"2024-05-11T12:13:03.163034+0000","last_active":"2024-05-11T12:13:03.172282+0000","last_peered":"2024-05-11T12:13:03.172282+0000","last_clean":"2024-05-11T12:13:03.172282+0000","last_became_active":"2024-05-11T12:13:03.160775+0000","last_became_peered":"2024-05-11T12:13:03.160775+0000","last_unstale":"2024-05-11T12:13:03.172282+0000","last_undegraded":"2024-05-11T12:13:03.172282+0000","last_fullsized":"2024-05-11T12:13:03.172282+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:08:51.947443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.117231+0000","last_change":"2024-05-11T12:13:02.113452+0000","last_active":"2024-05-11T12:13:02.117231+0000","last_peered":"2024-05-11T12:13:02.117231+0000","last_clean":"2024-05-11T12:13:02.117231+0000","last_became_active":"2024-05-11T12:13:02.112863+0000","last_became_peered":"2024-05-11T12:13:02.112863+0000","last_unstale":"2024-05-11T12:13:02.117231+0000","last_undegraded":"2024-05-11T12:13:02.117231+0000","last_fullsized":"2024-05-11T12:13:02.117231+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:08:15.133910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-05-11T12:13:02.102609+0000","last_change":"2024-05-11T12:13:02.102609+0000","last_active":"2024-05-11T12:13:02.088574+0000","last_peered":"2024-05-11T12:13:02.088574+0000","last_clean":"2024-05-11T12:13:02.088574+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:13:02.102609+0000","last_undegraded":"2024-05-11T12:13:02.102609+0000","last_fullsized":"2024-05-11T12:13:02.102609+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.137700+0000","last_change":"2024-05-11T12:13:02.133595+0000","last_active":"2024-05-11T12:13:02.137700+0000","last_peered":"2024-05-11T12:13:02.137700+0000","last_clean":"2024-05-11T12:13:02.137700+0000","last_became_active":"2024-05-11T12:13:02.133168+0000","last_became_peered":"2024-05-11T12:13:02.133168+0000","last_unstale":"2024-05-11T12:13:02.137700+0000","last_undegraded":"2024-05-11T12:13:02.137700+0000","last_fullsized":"2024-05-11T12:13:02.137700+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:27:13.189890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.173843+0000","last_change":"2024-05-11T12:13:03.169252+0000","last_active":"2024-05-11T12:13:04.173843+0000","last_peered":"2024-05-11T12:13:04.173843+0000","last_clean":"2024-05-11T12:13:04.173843+0000","last_became_active":"2024-05-11T12:13:03.167540+0000","last_became_peered":"2024-05-11T12:13:03.167540+0000","last_unstale":"2024-05-11T12:13:04.173843+0000","last_undegraded":"2024-05-11T12:13:04.173843+0000","last_fullsized":"2024-05-11T12:13:04.173843+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:38:22.482431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.144252+0000","last_change":"2024-05-11T12:13:02.140835+0000","last_active":"2024-05-11T12:13:02.144252+0000","last_peered":"2024-05-11T12:13:02.144252+0000","last_clean":"2024-05-11T12:13:02.144252+0000","last_became_active":"2024-05-11T12:13:02.139971+0000","last_became_peered":"2024-05-11T12:13:02.139971+0000","last_unstale":"2024-05-11T12:13:02.144252+0000","last_undegraded":"2024-05-11T12:13:02.144252+0000","last_fullsized":"2024-05-11T12:13:02.144252+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:06:39.713684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.172847+0000","last_change":"2024-05-11T12:13:03.166247+0000","last_active":"2024-05-11T12:13:03.172847+0000","last_peered":"2024-05-11T12:13:03.172847+0000","last_clean":"2024-05-11T12:13:03.172847+0000","last_became_active":"2024-05-11T12:13:03.165118+0000","last_became_peered":"2024-05-11T12:13:03.165118+0000","last_unstale":"2024-05-11T12:13:03.172847+0000","last_undegraded":"2024-05-11T12:13:03.172847+0000","last_fullsized":"2024-05-11T12:13:03.172847+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:32:45.994907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-11T12:13:02.125060+0000","last_change":"2024-05-11T12:13:02.116771+0000","last_active":"2024-05-11T12:13:02.125060+0000","last_peered":"2024-05-11T12:13:02.125060+0000","last_clean":"2024-05-11T12:13:02.125060+0000","last_became_active":"2024-05-11T12:13:02.116028+0000","last_became_peered":"2024-05-11T12:13:02.116028+0000","last_unstale":"2024-05-11T12:13:02.125060+0000","last_undegraded":"2024-05-11T12:13:02.125060+0000","last_fullsized":"2024-05-11T12:13:02.125060+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:24:39.427445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":183,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T12:13:03.152992+0000","last_change":"2024-05-11T12:10:35.066673+0000","last_active":"2024-05-11T12:13:03.152992+0000","last_peered":"2024-05-11T12:13:03.152992+0000","last_clean":"2024-05-11T12:13:03.152992+0000","last_became_active":"2024-05-11T12:10:35.065942+0000","last_became_peered":"2024-05-11T12:10:35.065942+0000","last_unstale":"2024-05-11T12:13:03.152992+0000","last_undegraded":"2024-05-11T12:13:03.152992+0000","last_fullsized":"2024-05-11T12:13:03.152992+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_clean_scrub_stamp":"2024-05-11T12:10:33.601136+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:12:02.133182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19,"ondisk_log_size":19,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476769,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28320,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343520,"statfs":{"total":96636764160,"available":96607764480,"internally_reserved":0,"allocated":1331200,"data_stored":916169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476770,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27740,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344100,"statfs":{"total":96636764160,"available":96608358400,"internally_reserved":0,"allocated":737280,"data_stored":325801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476769,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":737280,"data_stored":325801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476771,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":737280,"data_stored":325801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509474,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27732,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344108,"statfs":{"total":96636764160,"available":96608366592,"internally_reserved":0,"allocated":724992,"data_stored":319142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509475,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":724992,"data_stored":319142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509474,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28332,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343508,"statfs":{"total":96636764160,"available":96607752192,"internally_reserved":0,"allocated":1331200,"data_stored":916169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542179,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":761856,"data_stored":339660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T12:13:08.633 INFO:tasks.ceph.ceph_manager.ceph:PG 10.3f is not active+clean 2024-05-11T12:13:08.633 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:13:02.104414+0000', 'last_change': '2024-05-11T12:13:02.104414+0000', 'last_active': '2024-05-11T12:13:02.088574+0000', 'last_peered': '2024-05-11T12:13:02.088574+0000', 'last_clean': '2024-05-11T12:13:02.088574+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:13:02.104414+0000', 'last_undegraded': '2024-05-11T12:13:02.104414+0000', 'last_fullsized': '2024-05-11T12:13:02.104414+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'last_clean_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-11T12:13:08.633 INFO:tasks.ceph.ceph_manager.ceph:PG 10.3e is not active+clean 2024-05-11T12:13:08.633 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:13:02.102208+0000', 'last_change': '2024-05-11T12:13:02.102208+0000', 'last_active': '2024-05-11T12:13:02.088574+0000', 'last_peered': '2024-05-11T12:13:02.088574+0000', 'last_clean': '2024-05-11T12:13:02.088574+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:13:02.102208+0000', 'last_undegraded': '2024-05-11T12:13:02.102208+0000', 'last_fullsized': '2024-05-11T12:13:02.102208+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'last_clean_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-11T12:13:08.633 INFO:tasks.ceph.ceph_manager.ceph:PG 10.3a is not active+clean 2024-05-11T12:13:08.633 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:13:02.121564+0000', 'last_change': '2024-05-11T12:13:02.121564+0000', 'last_active': '2024-05-11T12:13:02.088574+0000', 'last_peered': '2024-05-11T12:13:02.088574+0000', 'last_clean': '2024-05-11T12:13:02.088574+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:13:02.121564+0000', 'last_undegraded': '2024-05-11T12:13:02.121564+0000', 'last_fullsized': '2024-05-11T12:13:02.121564+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'last_clean_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-11T12:13:08.633 INFO:tasks.ceph.ceph_manager.ceph:PG 10.2f is not active+clean 2024-05-11T12:13:08.633 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:13:02.103891+0000', 'last_change': '2024-05-11T12:13:02.103891+0000', 'last_active': '2024-05-11T12:13:02.088574+0000', 'last_peered': '2024-05-11T12:13:02.088574+0000', 'last_clean': '2024-05-11T12:13:02.088574+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:13:02.103891+0000', 'last_undegraded': '2024-05-11T12:13:02.103891+0000', 'last_fullsized': '2024-05-11T12:13:02.103891+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'last_clean_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-11T12:13:08.633 INFO:tasks.ceph.ceph_manager.ceph:PG 10.2c is not active+clean 2024-05-11T12:13:08.633 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:13:02.116797+0000', 'last_change': '2024-05-11T12:13:02.116797+0000', 'last_active': '2024-05-11T12:13:02.088574+0000', 'last_peered': '2024-05-11T12:13:02.088574+0000', 'last_clean': '2024-05-11T12:13:02.088574+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:13:02.116797+0000', 'last_undegraded': '2024-05-11T12:13:02.116797+0000', 'last_fullsized': '2024-05-11T12:13:02.116797+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'last_clean_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-11T12:13:08.633 INFO:tasks.ceph.ceph_manager.ceph:PG 10.2a is not active+clean 2024-05-11T12:13:08.634 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:13:02.116334+0000', 'last_change': '2024-05-11T12:13:02.116334+0000', 'last_active': '2024-05-11T12:13:02.088574+0000', 'last_peered': '2024-05-11T12:13:02.088574+0000', 'last_clean': '2024-05-11T12:13:02.088574+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:13:02.116334+0000', 'last_undegraded': '2024-05-11T12:13:02.116334+0000', 'last_fullsized': '2024-05-11T12:13:02.116334+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'last_clean_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-11T12:13:08.634 INFO:tasks.ceph.ceph_manager.ceph:PG 10.29 is not active+clean 2024-05-11T12:13:08.634 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:13:02.115725+0000', 'last_change': '2024-05-11T12:13:02.115725+0000', 'last_active': '2024-05-11T12:13:02.088574+0000', 'last_peered': '2024-05-11T12:13:02.088574+0000', 'last_clean': '2024-05-11T12:13:02.088574+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:13:02.115725+0000', 'last_undegraded': '2024-05-11T12:13:02.115725+0000', 'last_fullsized': '2024-05-11T12:13:02.115725+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'last_clean_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-11T12:13:08.634 INFO:tasks.ceph.ceph_manager.ceph:PG 10.24 is not active+clean 2024-05-11T12:13:08.634 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:13:02.103639+0000', 'last_change': '2024-05-11T12:13:02.103639+0000', 'last_active': '2024-05-11T12:13:02.088574+0000', 'last_peered': '2024-05-11T12:13:02.088574+0000', 'last_clean': '2024-05-11T12:13:02.088574+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:13:02.103639+0000', 'last_undegraded': '2024-05-11T12:13:02.103639+0000', 'last_fullsized': '2024-05-11T12:13:02.103639+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'last_clean_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-11T12:13:08.634 INFO:tasks.ceph.ceph_manager.ceph:PG 10.20 is not active+clean 2024-05-11T12:13:08.634 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:13:02.121306+0000', 'last_change': '2024-05-11T12:13:02.121306+0000', 'last_active': '2024-05-11T12:13:02.088574+0000', 'last_peered': '2024-05-11T12:13:02.088574+0000', 'last_clean': '2024-05-11T12:13:02.088574+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:13:02.121306+0000', 'last_undegraded': '2024-05-11T12:13:02.121306+0000', 'last_fullsized': '2024-05-11T12:13:02.121306+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'last_clean_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-11T12:13:08.634 INFO:tasks.ceph.ceph_manager.ceph:PG 10.1e is not active+clean 2024-05-11T12:13:08.634 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:13:02.120695+0000', 'last_change': '2024-05-11T12:13:02.120695+0000', 'last_active': '2024-05-11T12:13:02.088574+0000', 'last_peered': '2024-05-11T12:13:02.088574+0000', 'last_clean': '2024-05-11T12:13:02.088574+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:13:02.120695+0000', 'last_undegraded': '2024-05-11T12:13:02.120695+0000', 'last_fullsized': '2024-05-11T12:13:02.120695+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'last_clean_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-11T12:13:08.634 INFO:tasks.ceph.ceph_manager.ceph:PG 10.19 is not active+clean 2024-05-11T12:13:08.634 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:13:02.104889+0000', 'last_change': '2024-05-11T12:13:02.104889+0000', 'last_active': '2024-05-11T12:13:02.088574+0000', 'last_peered': '2024-05-11T12:13:02.088574+0000', 'last_clean': '2024-05-11T12:13:02.088574+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:13:02.104889+0000', 'last_undegraded': '2024-05-11T12:13:02.104889+0000', 'last_fullsized': '2024-05-11T12:13:02.104889+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'last_clean_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-11T12:13:08.635 INFO:tasks.ceph.ceph_manager.ceph:PG 10.1b is not active+clean 2024-05-11T12:13:08.635 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:13:02.115715+0000', 'last_change': '2024-05-11T12:13:02.115715+0000', 'last_active': '2024-05-11T12:13:02.088574+0000', 'last_peered': '2024-05-11T12:13:02.088574+0000', 'last_clean': '2024-05-11T12:13:02.088574+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:13:02.115715+0000', 'last_undegraded': '2024-05-11T12:13:02.115715+0000', 'last_fullsized': '2024-05-11T12:13:02.115715+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'last_clean_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-11T12:13:08.635 INFO:tasks.ceph.ceph_manager.ceph:PG 10.14 is not active+clean 2024-05-11T12:13:08.635 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:13:02.103086+0000', 'last_change': '2024-05-11T12:13:02.103086+0000', 'last_active': '2024-05-11T12:13:02.088574+0000', 'last_peered': '2024-05-11T12:13:02.088574+0000', 'last_clean': '2024-05-11T12:13:02.088574+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:13:02.103086+0000', 'last_undegraded': '2024-05-11T12:13:02.103086+0000', 'last_fullsized': '2024-05-11T12:13:02.103086+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'last_clean_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-11T12:13:08.635 INFO:tasks.ceph.ceph_manager.ceph:PG 10.15 is not active+clean 2024-05-11T12:13:08.635 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:13:02.116293+0000', 'last_change': '2024-05-11T12:13:02.116293+0000', 'last_active': '2024-05-11T12:13:02.088574+0000', 'last_peered': '2024-05-11T12:13:02.088574+0000', 'last_clean': '2024-05-11T12:13:02.088574+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:13:02.116293+0000', 'last_undegraded': '2024-05-11T12:13:02.116293+0000', 'last_fullsized': '2024-05-11T12:13:02.116293+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'last_clean_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-11T12:13:08.635 INFO:tasks.ceph.ceph_manager.ceph:PG 10.13 is not active+clean 2024-05-11T12:13:08.635 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:13:02.101871+0000', 'last_change': '2024-05-11T12:13:02.101871+0000', 'last_active': '2024-05-11T12:13:02.088574+0000', 'last_peered': '2024-05-11T12:13:02.088574+0000', 'last_clean': '2024-05-11T12:13:02.088574+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:13:02.101871+0000', 'last_undegraded': '2024-05-11T12:13:02.101871+0000', 'last_fullsized': '2024-05-11T12:13:02.101871+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'last_clean_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-11T12:13:08.635 INFO:tasks.ceph.ceph_manager.ceph:PG 10.8 is not active+clean 2024-05-11T12:13:08.635 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:13:02.120434+0000', 'last_change': '2024-05-11T12:13:02.120434+0000', 'last_active': '2024-05-11T12:13:02.088574+0000', 'last_peered': '2024-05-11T12:13:02.088574+0000', 'last_clean': '2024-05-11T12:13:02.088574+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:13:02.120434+0000', 'last_undegraded': '2024-05-11T12:13:02.120434+0000', 'last_fullsized': '2024-05-11T12:13:02.120434+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'last_clean_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-11T12:13:08.636 INFO:tasks.ceph.ceph_manager.ceph:PG 10.4 is not active+clean 2024-05-11T12:13:08.636 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:13:02.120491+0000', 'last_change': '2024-05-11T12:13:02.120491+0000', 'last_active': '2024-05-11T12:13:02.088574+0000', 'last_peered': '2024-05-11T12:13:02.088574+0000', 'last_clean': '2024-05-11T12:13:02.088574+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:13:02.120491+0000', 'last_undegraded': '2024-05-11T12:13:02.120491+0000', 'last_fullsized': '2024-05-11T12:13:02.120491+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'last_clean_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-11T12:13:08.636 INFO:tasks.ceph.ceph_manager.ceph:PG 10.2 is not active+clean 2024-05-11T12:13:08.636 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:13:02.102609+0000', 'last_change': '2024-05-11T12:13:02.102609+0000', 'last_active': '2024-05-11T12:13:02.088574+0000', 'last_peered': '2024-05-11T12:13:02.088574+0000', 'last_clean': '2024-05-11T12:13:02.088574+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:13:02.102609+0000', 'last_undegraded': '2024-05-11T12:13:02.102609+0000', 'last_fullsized': '2024-05-11T12:13:02.102609+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'last_clean_scrub_stamp': '2024-05-11T12:13:02.088574+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-11T12:13:08.636 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-11T12:13:08.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.875+0000 7f1751e00640 1 Processor -- start 2024-05-11T12:13:08.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.876+0000 7f1751e00640 1 -- start start 2024-05-11T12:13:08.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.877+0000 7f1751e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f174c15f620 0x7f174c163d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:08.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.877+0000 7f1751e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f174c0d18d0 0x7f174c0cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:08.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.877+0000 7f1751e00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f174c15fa00 con 0x7f174c0cdb50 2024-05-11T12:13:08.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.877+0000 7f1751e00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f174c0cdfc0 con 0x7f174c0d18d0 2024-05-11T12:13:08.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.877+0000 7f1751e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f174c163870 con 0x7f174c15f620 2024-05-11T12:13:08.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.877+0000 7f174a000640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f174c0d18d0 0x7f174c0cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:08.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.877+0000 7f174b400640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f174c0cdb50 0x7f174c0b7ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:48698/0 (socket says 172.21.15.167:48698) 2024-05-11T12:13:08.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.877+0000 7f174b400640 1 -- 172.21.15.167:0/992571104 learned_addr learned my addr 172.21.15.167:0/992571104 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:13:08.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.877+0000 7f174a000640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f174c0d18d0 0x7f174c0cd610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41400/0 (socket says 172.21.15.167:41400) 2024-05-11T12:13:08.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.878+0000 7f174aa00640 1 --2- 172.21.15.167:0/992571104 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f174c15f620 0x7f174c163d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:08.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.878+0000 7f174a000640 1 -- 172.21.15.167:0/992571104 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f174c15f620 msgr2=0x7f174c163d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:08.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.878+0000 7f174a000640 1 --2- 172.21.15.167:0/992571104 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f174c15f620 0x7f174c163d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:08.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.878+0000 7f174a000640 1 -- 172.21.15.167:0/992571104 >> v1:172.21.15.167:6789/0 conn(0x7f174c0cdb50 legacy=0x7f174c0b7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:08.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.878+0000 7f174a000640 1 -- 172.21.15.167:0/992571104 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f174c0b6200 con 0x7f174c0d18d0 2024-05-11T12:13:08.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.878+0000 7f1749600640 1 -- 172.21.15.167:0/992571104 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2492025792 0 0) 0x7f174c15fa00 con 0x7f174c0cdb50 2024-05-11T12:13:08.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.879+0000 7f174a000640 1 --2- 172.21.15.167:0/992571104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f174c0d18d0 0x7f174c0cd610 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f17380097a0 tx=0x7f173802a0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92866e342542d8df server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:08.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.879+0000 7f1749600640 1 -- 172.21.15.167:0/992571104 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1738037070 con 0x7f174c0d18d0 2024-05-11T12:13:08.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.879+0000 7f1749600640 1 -- 172.21.15.167:0/992571104 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f173802ae30 con 0x7f174c0d18d0 2024-05-11T12:13:08.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.879+0000 7f1749600640 1 -- 172.21.15.167:0/992571104 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1738032640 con 0x7f174c0d18d0 2024-05-11T12:13:08.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.880+0000 7f1751e00640 1 -- 172.21.15.167:0/992571104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f174c0d18d0 msgr2=0x7f174c0cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:08.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.880+0000 7f1751e00640 1 --2- 172.21.15.167:0/992571104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f174c0d18d0 0x7f174c0cd610 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f17380097a0 tx=0x7f173802a0e0 comp rx=0 tx=0).stop 2024-05-11T12:13:08.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.880+0000 7f1751e00640 1 -- 172.21.15.167:0/992571104 shutdown_connections 2024-05-11T12:13:08.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.880+0000 7f1751e00640 1 --2- 172.21.15.167:0/992571104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f174c0d18d0 0x7f174c0cd610 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:08.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.880+0000 7f1751e00640 1 --2- 172.21.15.167:0/992571104 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f174c15f620 0x7f174c163d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:08.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.880+0000 7f1751e00640 1 -- 172.21.15.167:0/992571104 >> 172.21.15.167:0/992571104 conn(0x7f174c072730 msgr2=0x7f174c0d2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:08.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.880+0000 7f1751e00640 1 -- 172.21.15.167:0/992571104 shutdown_connections 2024-05-11T12:13:08.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.880+0000 7f1751e00640 1 -- 172.21.15.167:0/992571104 wait complete. 2024-05-11T12:13:08.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.882+0000 7f1751e00640 1 Processor -- start 2024-05-11T12:13:08.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.882+0000 7f1751e00640 1 -- start start 2024-05-11T12:13:08.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.883+0000 7f1751e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f174c0cdb50 0x7f174c0c96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:08.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.883+0000 7f1751e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f174c0d18d0 0x7f174c1ea030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:08.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.883+0000 7f1751e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f174c15f620 0x7f174c1ea570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:08.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.883+0000 7f1751e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f174c0d1d30 con 0x7f174c0d18d0 2024-05-11T12:13:08.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.883+0000 7f1751e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f174c163500 con 0x7f174c0cdb50 2024-05-11T12:13:08.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.883+0000 7f1751e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f174c0b5e90 con 0x7f174c15f620 2024-05-11T12:13:08.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.883+0000 7f174a000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f174c0d18d0 0x7f174c1ea030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:08.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.883+0000 7f174a000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f174c0d18d0 0x7f174c1ea030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41414/0 (socket says 172.21.15.167:41414) 2024-05-11T12:13:08.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.883+0000 7f174a000640 1 -- 172.21.15.167:0/891045074 learned_addr learned my addr 172.21.15.167:0/891045074 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:08.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.883+0000 7f174a000640 1 -- 172.21.15.167:0/891045074 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f174c15f620 msgr2=0x7f174c1ea570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:08.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.883+0000 7f174a000640 1 --2- 172.21.15.167:0/891045074 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f174c15f620 0x7f174c1ea570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:08.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.883+0000 7f174a000640 1 -- 172.21.15.167:0/891045074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f174c0cdb50 msgr2=0x7f174c0c96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:08.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.883+0000 7f174a000640 1 --2- 172.21.15.167:0/891045074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f174c0cdb50 0x7f174c0c96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:08.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.883+0000 7f174a000640 1 -- 172.21.15.167:0/891045074 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f174c0c8c20 con 0x7f174c0d18d0 2024-05-11T12:13:08.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.884+0000 7f174a000640 1 --2- 172.21.15.167:0/891045074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f174c0d18d0 0x7f174c1ea030 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1738000c00 tx=0x7f17380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:08.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.884+0000 7f1743400640 1 -- 172.21.15.167:0/891045074 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1738041070 con 0x7f174c0d18d0 2024-05-11T12:13:08.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.884+0000 7f1743400640 1 -- 172.21.15.167:0/891045074 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f173803c670 con 0x7f174c0d18d0 2024-05-11T12:13:08.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.884+0000 7f1743400640 1 -- 172.21.15.167:0/891045074 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1738037040 con 0x7f174c0d18d0 2024-05-11T12:13:08.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.884+0000 7f1751e00640 1 -- 172.21.15.167:0/891045074 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f174c162a70 con 0x7f174c0d18d0 2024-05-11T12:13:08.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.884+0000 7f1751e00640 1 -- 172.21.15.167:0/891045074 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f174c0c8ef0 con 0x7f174c0d18d0 2024-05-11T12:13:08.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.886+0000 7f1751e00640 1 -- 172.21.15.167:0/891045074 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f174c0cfe60 con 0x7f174c0d18d0 2024-05-11T12:13:08.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.887+0000 7f1743400640 1 -- 172.21.15.167:0/891045074 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1738044070 con 0x7f174c0d18d0 2024-05-11T12:13:08.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.888+0000 7f1743400640 1 --2- 172.21.15.167:0/891045074 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f172c0762d0 0x7f172c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:08.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.888+0000 7f1743400640 1 -- 172.21.15.167:0/891045074 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f17380b7c10 con 0x7f174c0d18d0 2024-05-11T12:13:08.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.888+0000 7f174aa00640 1 --2- 172.21.15.167:0/891045074 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f172c0762d0 0x7f172c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:08.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.889+0000 7f174aa00640 1 --2- 172.21.15.167:0/891045074 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f172c0762d0 0x7f172c078790 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f173c006fd0 tx=0x7f173c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:08.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:08.892+0000 7f1743400640 1 -- 172.21.15.167:0/891045074 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f174c0cfe60 con 0x7f174c0d18d0 2024-05-11T12:13:09.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.160+0000 7f1751e00640 1 -- 172.21.15.167:0/891045074 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f174c0cd970 con 0x7f174c0d18d0 2024-05-11T12:13:09.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.160+0000 7f1743400640 1 -- 172.21.15.167:0/891045074 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1518 (secure 0 0 0) 0x7f174c0cd970 con 0x7f174c0d18d0 2024-05-11T12:13:09.161 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:13:09.161 INFO:teuthology.orchestra.run.smithi167.stdout:{"fsid":"5f32d344-7289-4c27-881b-1533419ab678","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":169,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":64,"num_osds":8,"num_up_osds":8,"osd_up_since":1715429432,"num_in_osds":8,"osd_in_since":1715429423,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":79},{"state_name":"creating+peering","count":18}],"num_pgs":97,"num_pools":3,"num_objects":14,"data_bytes":590584,"bytes_used":228462592,"bytes_avail":772865650688,"bytes_total":773094113280,"inactive_pgs_ratio":0.18556700646877289,"write_bytes_sec":1706,"read_op_per_sec":0,"write_op_per_sec":4},"fsmap":{"epoch":35,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"c","status":"up:active","gid":5610}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-05-11T12:12:51.275297+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-11T12:13:09.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.163+0000 7f1751e00640 1 -- 172.21.15.167:0/891045074 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f172c0762d0 msgr2=0x7f172c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:09.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.163+0000 7f1751e00640 1 --2- 172.21.15.167:0/891045074 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f172c0762d0 0x7f172c078790 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f173c006fd0 tx=0x7f173c008040 comp rx=0 tx=0).stop 2024-05-11T12:13:09.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.163+0000 7f1751e00640 1 -- 172.21.15.167:0/891045074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f174c0d18d0 msgr2=0x7f174c1ea030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:09.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.163+0000 7f1751e00640 1 --2- 172.21.15.167:0/891045074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f174c0d18d0 0x7f174c1ea030 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1738000c00 tx=0x7f17380046c0 comp rx=0 tx=0).stop 2024-05-11T12:13:09.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.163+0000 7f1751e00640 1 -- 172.21.15.167:0/891045074 shutdown_connections 2024-05-11T12:13:09.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.163+0000 7f1751e00640 1 --2- 172.21.15.167:0/891045074 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f174c15f620 0x7f174c1ea570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:09.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.163+0000 7f1751e00640 1 --2- 172.21.15.167:0/891045074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f174c0d18d0 0x7f174c1ea030 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:09.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.163+0000 7f1751e00640 1 --2- 172.21.15.167:0/891045074 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f172c0762d0 0x7f172c078790 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:09.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.163+0000 7f1751e00640 1 --2- 172.21.15.167:0/891045074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f174c0cdb50 0x7f174c0c96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:09.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.163+0000 7f1751e00640 1 -- 172.21.15.167:0/891045074 >> 172.21.15.167:0/891045074 conn(0x7f174c072730 msgr2=0x7f174c0d2da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:09.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.163+0000 7f1751e00640 1 -- 172.21.15.167:0/891045074 shutdown_connections 2024-05-11T12:13:09.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.163+0000 7f1751e00640 1 -- 172.21.15.167:0/891045074 wait complete. 2024-05-11T12:13:09.182 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-05-11T12:13:09.182 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T12:13:09.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.368+0000 7f071d400640 1 Processor -- start 2024-05-11T12:13:09.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.368+0000 7f071d400640 1 -- start start 2024-05-11T12:13:09.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.370+0000 7f071d400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f07180b31e0 0x7f07180cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:09.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.370+0000 7f071d400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f071815e8d0 0x7f07180b63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:09.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.370+0000 7f071d400640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0718154440 con 0x7f0718160af0 2024-05-11T12:13:09.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.370+0000 7f071d400640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f07180b7890 con 0x7f07180b31e0 2024-05-11T12:13:09.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.370+0000 7f071d400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f07180b5c60 con 0x7f071815e8d0 2024-05-11T12:13:09.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.370+0000 7f070fe00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f07180b31e0 0x7f07180cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:09.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.370+0000 7f0717e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f0718160af0 0x7f0718160ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:48712/0 (socket says 172.21.15.167:48712) 2024-05-11T12:13:09.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.370+0000 7f0717e00640 1 -- 172.21.15.167:0/3230715470 learned_addr learned my addr 172.21.15.167:0/3230715470 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:13:09.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.371+0000 7f070fe00640 1 -- 172.21.15.167:0/3230715470 >> v2:172.21.15.204:3300/0 conn(0x7f071815e8d0 msgr2=0x7f07180b63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:09.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.371+0000 7f071ca00640 1 --2- 172.21.15.167:0/3230715470 >> v2:172.21.15.204:3300/0 conn(0x7f071815e8d0 0x7f07180b63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:09.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.371+0000 7f070fe00640 1 --2- 172.21.15.167:0/3230715470 >> v2:172.21.15.204:3300/0 conn(0x7f071815e8d0 0x7f07180b63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:09.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.371+0000 7f070fe00640 1 -- 172.21.15.167:0/3230715470 >> v1:172.21.15.167:6789/0 conn(0x7f0718160af0 legacy=0x7f0718160ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:09.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.371+0000 7f070fe00640 1 -- 172.21.15.167:0/3230715470 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07180b3650 con 0x7f07180b31e0 2024-05-11T12:13:09.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.371+0000 7f0717400640 1 -- 172.21.15.167:0/3230715470 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3497058486 0 0) 0x7f0708001110 con 0x7f0718160af0 2024-05-11T12:13:09.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.372+0000 7f070fe00640 1 --2- 172.21.15.167:0/3230715470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07180b31e0 0x7f07180cf7d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f07180d0f70 tx=0x7f070002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=668898e309ba0c09 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:09.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.372+0000 7f0717400640 1 -- 172.21.15.167:0/3230715470 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0700037070 con 0x7f07180b31e0 2024-05-11T12:13:09.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.372+0000 7f0717400640 1 -- 172.21.15.167:0/3230715470 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0700032440 con 0x7f07180b31e0 2024-05-11T12:13:09.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.372+0000 7f0717400640 1 -- 172.21.15.167:0/3230715470 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f070002ae60 con 0x7f07180b31e0 2024-05-11T12:13:09.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.373+0000 7f071d400640 1 -- 172.21.15.167:0/3230715470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07180b31e0 msgr2=0x7f07180cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:09.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.373+0000 7f071d400640 1 --2- 172.21.15.167:0/3230715470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07180b31e0 0x7f07180cf7d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f07180d0f70 tx=0x7f070002a110 comp rx=0 tx=0).stop 2024-05-11T12:13:09.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.373+0000 7f071d400640 1 -- 172.21.15.167:0/3230715470 shutdown_connections 2024-05-11T12:13:09.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.373+0000 7f071d400640 1 --2- 172.21.15.167:0/3230715470 >> v2:172.21.15.204:3300/0 conn(0x7f071815e8d0 0x7f07180b63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:09.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.373+0000 7f071d400640 1 --2- 172.21.15.167:0/3230715470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07180b31e0 0x7f07180cf7d0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:09.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.373+0000 7f071d400640 1 -- 172.21.15.167:0/3230715470 >> 172.21.15.167:0/3230715470 conn(0x7f07180b8360 msgr2=0x7f07180b8770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:09.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.373+0000 7f071d400640 1 -- 172.21.15.167:0/3230715470 shutdown_connections 2024-05-11T12:13:09.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.373+0000 7f071d400640 1 -- 172.21.15.167:0/3230715470 wait complete. 2024-05-11T12:13:09.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.375+0000 7f071d400640 1 Processor -- start 2024-05-11T12:13:09.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.375+0000 7f071d400640 1 -- start start 2024-05-11T12:13:09.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.376+0000 7f071d400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f07180b31e0 0x7f0718157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:09.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.376+0000 7f071d400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f071815e8d0 0x7f071806ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:09.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.376+0000 7f071d400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0718160af0 0x7f071806e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:09.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.376+0000 7f071d400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f07180b5eb0 con 0x7f0718160af0 2024-05-11T12:13:09.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.376+0000 7f071d400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f07180b7dc0 con 0x7f07180b31e0 2024-05-11T12:13:09.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.376+0000 7f071d400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f071815ed60 con 0x7f071815e8d0 2024-05-11T12:13:09.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.377+0000 7f071ca00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0718160af0 0x7f071806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:09.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.377+0000 7f071ca00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0718160af0 0x7f071806e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41432/0 (socket says 172.21.15.167:41432) 2024-05-11T12:13:09.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.377+0000 7f071ca00640 1 -- 172.21.15.167:0/2927046497 learned_addr learned my addr 172.21.15.167:0/2927046497 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:09.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.377+0000 7f070fe00640 1 --2- 172.21.15.167:0/2927046497 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f071815e8d0 0x7f071806ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:09.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.377+0000 7f0717e00640 1 --2- 172.21.15.167:0/2927046497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f07180b31e0 0x7f0718157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:09.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.377+0000 7f071ca00640 1 -- 172.21.15.167:0/2927046497 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f071815e8d0 msgr2=0x7f071806ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:09.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.377+0000 7f071ca00640 1 --2- 172.21.15.167:0/2927046497 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f071815e8d0 0x7f071806ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:09.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.377+0000 7f071ca00640 1 -- 172.21.15.167:0/2927046497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f07180b31e0 msgr2=0x7f0718157e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:09.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.377+0000 7f071ca00640 1 --2- 172.21.15.167:0/2927046497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f07180b31e0 0x7f0718157e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:09.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.377+0000 7f071ca00640 1 -- 172.21.15.167:0/2927046497 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f071814fd90 con 0x7f0718160af0 2024-05-11T12:13:09.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.377+0000 7f070fe00640 1 --2- 172.21.15.167:0/2927046497 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f071815e8d0 0x7f071806ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:09.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.377+0000 7f0717e00640 1 --2- 172.21.15.167:0/2927046497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f07180b31e0 0x7f0718157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:09.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.377+0000 7f071ca00640 1 --2- 172.21.15.167:0/2927046497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0718160af0 0x7f071806e330 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f071000cd70 tx=0x7f071000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:09.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.378+0000 7f0714c00640 1 -- 172.21.15.167:0/2927046497 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0710003b90 con 0x7f0718160af0 2024-05-11T12:13:09.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.378+0000 7f0714c00640 1 -- 172.21.15.167:0/2927046497 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0710003d30 con 0x7f0718160af0 2024-05-11T12:13:09.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.378+0000 7f0714c00640 1 -- 172.21.15.167:0/2927046497 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0710004240 con 0x7f0718160af0 2024-05-11T12:13:09.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.378+0000 7f071d400640 1 -- 172.21.15.167:0/2927046497 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f071806e9d0 con 0x7f0718160af0 2024-05-11T12:13:09.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.378+0000 7f071d400640 1 -- 172.21.15.167:0/2927046497 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f071806ef90 con 0x7f0718160af0 2024-05-11T12:13:09.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.379+0000 7f071d400640 1 -- 172.21.15.167:0/2927046497 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07180d01b0 con 0x7f0718160af0 2024-05-11T12:13:09.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.380+0000 7f0714c00640 1 -- 172.21.15.167:0/2927046497 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f07100043e0 con 0x7f0718160af0 2024-05-11T12:13:09.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.381+0000 7f0714c00640 1 --2- 172.21.15.167:0/2927046497 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f06f4076300 0x7f06f40787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:09.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.381+0000 7f0714c00640 1 -- 172.21.15.167:0/2927046497 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f07100982a0 con 0x7f0718160af0 2024-05-11T12:13:09.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.382+0000 7f0717e00640 1 --2- 172.21.15.167:0/2927046497 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f06f4076300 0x7f06f40787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:09.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.382+0000 7f0717e00640 1 --2- 172.21.15.167:0/2927046497 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f06f4076300 0x7f06f40787c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0708000f60 tx=0x7f0708009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:09.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.386+0000 7f0714c00640 1 -- 172.21.15.167:0/2927046497 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f07180d01b0 con 0x7f0718160af0 2024-05-11T12:13:09.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.575+0000 7f071d400640 1 -- 172.21.15.167:0/2927046497 --> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f07180b3e10 con 0x7f06f4076300 2024-05-11T12:13:09.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.592+0000 7f0714c00640 1 -- 172.21.15.167:0/2927046497 <== mgr.4106 v2:172.21.15.167:6832/578352559 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252009 (secure 0 0 0) 0x7f07180b3e10 con 0x7f06f4076300 2024-05-11T12:13:09.596 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:13:09.597 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2024-05-11T12:13:09.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.598+0000 7f071d400640 1 -- 172.21.15.167:0/2927046497 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f06f4076300 msgr2=0x7f06f40787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:09.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.598+0000 7f071d400640 1 --2- 172.21.15.167:0/2927046497 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f06f4076300 0x7f06f40787c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0708000f60 tx=0x7f0708009040 comp rx=0 tx=0).stop 2024-05-11T12:13:09.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.598+0000 7f071d400640 1 -- 172.21.15.167:0/2927046497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0718160af0 msgr2=0x7f071806e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:09.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.598+0000 7f071d400640 1 --2- 172.21.15.167:0/2927046497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0718160af0 0x7f071806e330 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f071000cd70 tx=0x7f071000a660 comp rx=0 tx=0).stop 2024-05-11T12:13:09.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.598+0000 7f071d400640 1 -- 172.21.15.167:0/2927046497 shutdown_connections 2024-05-11T12:13:09.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.598+0000 7f071d400640 1 --2- 172.21.15.167:0/2927046497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0718160af0 0x7f071806e330 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:09.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.598+0000 7f071d400640 1 --2- 172.21.15.167:0/2927046497 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f071815e8d0 0x7f071806ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:09.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.598+0000 7f071d400640 1 --2- 172.21.15.167:0/2927046497 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f06f4076300 0x7f06f40787c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:09.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.598+0000 7f071d400640 1 --2- 172.21.15.167:0/2927046497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f07180b31e0 0x7f0718157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:09.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.598+0000 7f071d400640 1 -- 172.21.15.167:0/2927046497 >> 172.21.15.167:0/2927046497 conn(0x7f07180b8360 msgr2=0x7f0718070000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:09.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.598+0000 7f071d400640 1 -- 172.21.15.167:0/2927046497 shutdown_connections 2024-05-11T12:13:09.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:09.598+0000 7f071d400640 1 -- 172.21.15.167:0/2927046497 wait complete. 2024-05-11T12:13:09.617 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":136,"stamp":"2024-05-11T12:13:09.279454+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231684,"kb_used_data":7176,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754743036,"statfs":{"total":773094113280,"available":772856868864,"internally_reserved":0,"allocated":7348224,"data_stored":3902836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":229756465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.182772"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.305824+0000","last_change":"2024-05-11T12:13:03.162419+0000","last_active":"2024-05-11T12:13:04.305824+0000","last_peered":"2024-05-11T12:13:04.305824+0000","last_clean":"2024-05-11T12:13:04.305824+0000","last_became_active":"2024-05-11T12:13:03.162146+0000","last_became_peered":"2024-05-11T12:13:03.162146+0000","last_unstale":"2024-05-11T12:13:04.305824+0000","last_undegraded":"2024-05-11T12:13:04.305824+0000","last_fullsized":"2024-05-11T12:13:04.305824+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:58:08.814828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.305666+0000","last_change":"2024-05-11T12:13:03.168274+0000","last_active":"2024-05-11T12:13:04.305666+0000","last_peered":"2024-05-11T12:13:04.305666+0000","last_clean":"2024-05-11T12:13:04.305666+0000","last_became_active":"2024-05-11T12:13:03.168022+0000","last_became_peered":"2024-05-11T12:13:03.168022+0000","last_unstale":"2024-05-11T12:13:04.305666+0000","last_undegraded":"2024-05-11T12:13:04.305666+0000","last_fullsized":"2024-05-11T12:13:04.305666+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:42:21.959847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.162597+0000","last_change":"2024-05-11T12:13:03.165899+0000","last_active":"2024-05-11T12:13:04.162597+0000","last_peered":"2024-05-11T12:13:04.162597+0000","last_clean":"2024-05-11T12:13:04.162597+0000","last_became_active":"2024-05-11T12:13:03.165494+0000","last_became_peered":"2024-05-11T12:13:03.165494+0000","last_unstale":"2024-05-11T12:13:04.162597+0000","last_undegraded":"2024-05-11T12:13:04.162597+0000","last_fullsized":"2024-05-11T12:13:04.162597+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:06:51.473529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.304221+0000","last_change":"2024-05-11T12:13:03.166762+0000","last_active":"2024-05-11T12:13:04.304221+0000","last_peered":"2024-05-11T12:13:04.304221+0000","last_clean":"2024-05-11T12:13:04.304221+0000","last_became_active":"2024-05-11T12:13:03.166457+0000","last_became_peered":"2024-05-11T12:13:03.166457+0000","last_unstale":"2024-05-11T12:13:04.304221+0000","last_undegraded":"2024-05-11T12:13:04.304221+0000","last_fullsized":"2024-05-11T12:13:04.304221+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:12:07.890717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.163278+0000","last_change":"2024-05-11T12:13:03.167675+0000","last_active":"2024-05-11T12:13:04.163278+0000","last_peered":"2024-05-11T12:13:04.163278+0000","last_clean":"2024-05-11T12:13:04.163278+0000","last_became_active":"2024-05-11T12:13:03.167381+0000","last_became_peered":"2024-05-11T12:13:03.167381+0000","last_unstale":"2024-05-11T12:13:04.163278+0000","last_undegraded":"2024-05-11T12:13:04.163278+0000","last_fullsized":"2024-05-11T12:13:04.163278+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:18:35.049757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.303733+0000","last_change":"2024-05-11T12:13:03.165292+0000","last_active":"2024-05-11T12:13:04.303733+0000","last_peered":"2024-05-11T12:13:04.303733+0000","last_clean":"2024-05-11T12:13:04.303733+0000","last_became_active":"2024-05-11T12:13:03.164960+0000","last_became_peered":"2024-05-11T12:13:03.164960+0000","last_unstale":"2024-05-11T12:13:04.303733+0000","last_undegraded":"2024-05-11T12:13:04.303733+0000","last_fullsized":"2024-05-11T12:13:04.303733+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:05:24.561464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.173969+0000","last_change":"2024-05-11T12:13:03.170170+0000","last_active":"2024-05-11T12:13:04.173969+0000","last_peered":"2024-05-11T12:13:04.173969+0000","last_clean":"2024-05-11T12:13:04.173969+0000","last_became_active":"2024-05-11T12:13:03.169888+0000","last_became_peered":"2024-05-11T12:13:03.169888+0000","last_unstale":"2024-05-11T12:13:04.173969+0000","last_undegraded":"2024-05-11T12:13:04.173969+0000","last_fullsized":"2024-05-11T12:13:04.173969+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:46:12.686412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.168062+0000","last_change":"2024-05-11T12:13:03.167147+0000","last_active":"2024-05-11T12:13:04.168062+0000","last_peered":"2024-05-11T12:13:04.168062+0000","last_clean":"2024-05-11T12:13:04.168062+0000","last_became_active":"2024-05-11T12:13:03.166882+0000","last_became_peered":"2024-05-11T12:13:03.166882+0000","last_unstale":"2024-05-11T12:13:04.168062+0000","last_undegraded":"2024-05-11T12:13:04.168062+0000","last_fullsized":"2024-05-11T12:13:04.168062+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:06:51.473529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.164785+0000","last_change":"2024-05-11T12:13:03.162128+0000","last_active":"2024-05-11T12:13:04.164785+0000","last_peered":"2024-05-11T12:13:04.164785+0000","last_clean":"2024-05-11T12:13:04.164785+0000","last_became_active":"2024-05-11T12:13:03.161841+0000","last_became_peered":"2024-05-11T12:13:03.161841+0000","last_unstale":"2024-05-11T12:13:04.164785+0000","last_undegraded":"2024-05-11T12:13:04.164785+0000","last_fullsized":"2024-05-11T12:13:04.164785+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:06:18.759321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.164997+0000","last_change":"2024-05-11T12:13:03.164933+0000","last_active":"2024-05-11T12:13:04.164997+0000","last_peered":"2024-05-11T12:13:04.164997+0000","last_clean":"2024-05-11T12:13:04.164997+0000","last_became_active":"2024-05-11T12:13:03.164680+0000","last_became_peered":"2024-05-11T12:13:03.164680+0000","last_unstale":"2024-05-11T12:13:04.164997+0000","last_undegraded":"2024-05-11T12:13:04.164997+0000","last_fullsized":"2024-05-11T12:13:04.164997+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:47:23.761750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.303848+0000","last_change":"2024-05-11T12:13:03.163978+0000","last_active":"2024-05-11T12:13:04.303848+0000","last_peered":"2024-05-11T12:13:04.303848+0000","last_clean":"2024-05-11T12:13:04.303848+0000","last_became_active":"2024-05-11T12:13:03.163688+0000","last_became_peered":"2024-05-11T12:13:03.163688+0000","last_unstale":"2024-05-11T12:13:04.303848+0000","last_undegraded":"2024-05-11T12:13:04.303848+0000","last_fullsized":"2024-05-11T12:13:04.303848+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:18:50.744350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.168376+0000","last_change":"2024-05-11T12:13:03.164295+0000","last_active":"2024-05-11T12:13:04.168376+0000","last_peered":"2024-05-11T12:13:04.168376+0000","last_clean":"2024-05-11T12:13:04.168376+0000","last_became_active":"2024-05-11T12:13:03.163948+0000","last_became_peered":"2024-05-11T12:13:03.163948+0000","last_unstale":"2024-05-11T12:13:04.168376+0000","last_undegraded":"2024-05-11T12:13:04.168376+0000","last_fullsized":"2024-05-11T12:13:04.168376+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:04:07.281796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.168867+0000","last_change":"2024-05-11T12:13:03.163777+0000","last_active":"2024-05-11T12:13:04.168867+0000","last_peered":"2024-05-11T12:13:04.168867+0000","last_clean":"2024-05-11T12:13:04.168867+0000","last_became_active":"2024-05-11T12:13:03.163472+0000","last_became_peered":"2024-05-11T12:13:03.163472+0000","last_unstale":"2024-05-11T12:13:04.168867+0000","last_undegraded":"2024-05-11T12:13:04.168867+0000","last_fullsized":"2024-05-11T12:13:04.168867+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:07:22.610930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.163696+0000","last_change":"2024-05-11T12:13:03.165418+0000","last_active":"2024-05-11T12:13:04.163696+0000","last_peered":"2024-05-11T12:13:04.163696+0000","last_clean":"2024-05-11T12:13:04.163696+0000","last_became_active":"2024-05-11T12:13:03.165020+0000","last_became_peered":"2024-05-11T12:13:03.165020+0000","last_unstale":"2024-05-11T12:13:04.163696+0000","last_undegraded":"2024-05-11T12:13:04.163696+0000","last_fullsized":"2024-05-11T12:13:04.163696+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:03:05.647677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.161876+0000","last_change":"2024-05-11T12:13:03.174279+0000","last_active":"2024-05-11T12:13:04.161876+0000","last_peered":"2024-05-11T12:13:04.161876+0000","last_clean":"2024-05-11T12:13:04.161876+0000","last_became_active":"2024-05-11T12:13:03.173960+0000","last_became_peered":"2024-05-11T12:13:03.173960+0000","last_unstale":"2024-05-11T12:13:04.161876+0000","last_undegraded":"2024-05-11T12:13:04.161876+0000","last_fullsized":"2024-05-11T12:13:04.161876+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:55:26.072668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.173798+0000","last_change":"2024-05-11T12:13:03.176759+0000","last_active":"2024-05-11T12:13:04.173798+0000","last_peered":"2024-05-11T12:13:04.173798+0000","last_clean":"2024-05-11T12:13:04.173798+0000","last_became_active":"2024-05-11T12:13:03.176496+0000","last_became_peered":"2024-05-11T12:13:03.176496+0000","last_unstale":"2024-05-11T12:13:04.173798+0000","last_undegraded":"2024-05-11T12:13:04.173798+0000","last_fullsized":"2024-05-11T12:13:04.173798+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:33:36.274987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.305957+0000","last_change":"2024-05-11T12:13:03.161765+0000","last_active":"2024-05-11T12:13:04.305957+0000","last_peered":"2024-05-11T12:13:04.305957+0000","last_clean":"2024-05-11T12:13:04.305957+0000","last_became_active":"2024-05-11T12:13:03.159361+0000","last_became_peered":"2024-05-11T12:13:03.159361+0000","last_unstale":"2024-05-11T12:13:04.305957+0000","last_undegraded":"2024-05-11T12:13:04.305957+0000","last_fullsized":"2024-05-11T12:13:04.305957+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:08:16.138838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.304239+0000","last_change":"2024-05-11T12:13:03.167581+0000","last_active":"2024-05-11T12:13:04.304239+0000","last_peered":"2024-05-11T12:13:04.304239+0000","last_clean":"2024-05-11T12:13:04.304239+0000","last_became_active":"2024-05-11T12:13:03.167285+0000","last_became_peered":"2024-05-11T12:13:03.167285+0000","last_unstale":"2024-05-11T12:13:04.304239+0000","last_undegraded":"2024-05-11T12:13:04.304239+0000","last_fullsized":"2024-05-11T12:13:04.304239+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:04:57.052060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.164975+0000","last_change":"2024-05-11T12:13:03.172989+0000","last_active":"2024-05-11T12:13:04.164975+0000","last_peered":"2024-05-11T12:13:04.164975+0000","last_clean":"2024-05-11T12:13:04.164975+0000","last_became_active":"2024-05-11T12:13:03.172704+0000","last_became_peered":"2024-05-11T12:13:03.172704+0000","last_unstale":"2024-05-11T12:13:04.164975+0000","last_undegraded":"2024-05-11T12:13:04.164975+0000","last_fullsized":"2024-05-11T12:13:04.164975+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:55:33.337983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.159946+0000","last_change":"2024-05-11T12:13:03.162259+0000","last_active":"2024-05-11T12:13:04.159946+0000","last_peered":"2024-05-11T12:13:04.159946+0000","last_clean":"2024-05-11T12:13:04.159946+0000","last_became_active":"2024-05-11T12:13:03.159517+0000","last_became_peered":"2024-05-11T12:13:03.159517+0000","last_unstale":"2024-05-11T12:13:04.159946+0000","last_undegraded":"2024-05-11T12:13:04.159946+0000","last_fullsized":"2024-05-11T12:13:04.159946+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:10:38.278696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.169063+0000","last_change":"2024-05-11T12:13:03.162655+0000","last_active":"2024-05-11T12:13:04.169063+0000","last_peered":"2024-05-11T12:13:04.169063+0000","last_clean":"2024-05-11T12:13:04.169063+0000","last_became_active":"2024-05-11T12:13:03.162259+0000","last_became_peered":"2024-05-11T12:13:03.162259+0000","last_unstale":"2024-05-11T12:13:04.169063+0000","last_undegraded":"2024-05-11T12:13:04.169063+0000","last_fullsized":"2024-05-11T12:13:04.169063+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:06:18.759321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.159320+0000","last_change":"2024-05-11T12:13:03.166071+0000","last_active":"2024-05-11T12:13:04.159320+0000","last_peered":"2024-05-11T12:13:04.159320+0000","last_clean":"2024-05-11T12:13:04.159320+0000","last_became_active":"2024-05-11T12:13:03.165691+0000","last_became_peered":"2024-05-11T12:13:03.165691+0000","last_unstale":"2024-05-11T12:13:04.159320+0000","last_undegraded":"2024-05-11T12:13:04.159320+0000","last_fullsized":"2024-05-11T12:13:04.159320+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:57:28.721233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.159421+0000","last_change":"2024-05-11T12:13:03.161024+0000","last_active":"2024-05-11T12:13:04.159421+0000","last_peered":"2024-05-11T12:13:04.159421+0000","last_clean":"2024-05-11T12:13:04.159421+0000","last_became_active":"2024-05-11T12:13:03.160734+0000","last_became_peered":"2024-05-11T12:13:03.160734+0000","last_unstale":"2024-05-11T12:13:04.159421+0000","last_undegraded":"2024-05-11T12:13:04.159421+0000","last_fullsized":"2024-05-11T12:13:04.159421+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:23:31.280589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.164305+0000","last_change":"2024-05-11T12:13:03.165438+0000","last_active":"2024-05-11T12:13:04.164305+0000","last_peered":"2024-05-11T12:13:04.164305+0000","last_clean":"2024-05-11T12:13:04.164305+0000","last_became_active":"2024-05-11T12:13:03.165118+0000","last_became_peered":"2024-05-11T12:13:03.165118+0000","last_unstale":"2024-05-11T12:13:04.164305+0000","last_undegraded":"2024-05-11T12:13:04.164305+0000","last_fullsized":"2024-05-11T12:13:04.164305+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:00:30.103546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.170861+0000","last_change":"2024-05-11T12:13:03.169517+0000","last_active":"2024-05-11T12:13:04.170861+0000","last_peered":"2024-05-11T12:13:04.170861+0000","last_clean":"2024-05-11T12:13:04.170861+0000","last_became_active":"2024-05-11T12:13:03.169235+0000","last_became_peered":"2024-05-11T12:13:03.169235+0000","last_unstale":"2024-05-11T12:13:04.170861+0000","last_undegraded":"2024-05-11T12:13:04.170861+0000","last_fullsized":"2024-05-11T12:13:04.170861+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:05:24.561464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.169110+0000","last_change":"2024-05-11T12:13:03.166968+0000","last_active":"2024-05-11T12:13:04.169110+0000","last_peered":"2024-05-11T12:13:04.169110+0000","last_clean":"2024-05-11T12:13:04.169110+0000","last_became_active":"2024-05-11T12:13:03.166704+0000","last_became_peered":"2024-05-11T12:13:03.166704+0000","last_unstale":"2024-05-11T12:13:04.169110+0000","last_undegraded":"2024-05-11T12:13:04.169110+0000","last_fullsized":"2024-05-11T12:13:04.169110+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:47:23.761750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.303938+0000","last_change":"2024-05-11T12:13:03.171885+0000","last_active":"2024-05-11T12:13:04.303938+0000","last_peered":"2024-05-11T12:13:04.303938+0000","last_clean":"2024-05-11T12:13:04.303938+0000","last_became_active":"2024-05-11T12:13:03.171590+0000","last_became_peered":"2024-05-11T12:13:03.171590+0000","last_unstale":"2024-05-11T12:13:04.303938+0000","last_undegraded":"2024-05-11T12:13:04.303938+0000","last_fullsized":"2024-05-11T12:13:04.303938+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:28:34.473489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.305220+0000","last_change":"2024-05-11T12:13:03.166734+0000","last_active":"2024-05-11T12:13:04.305220+0000","last_peered":"2024-05-11T12:13:04.305220+0000","last_clean":"2024-05-11T12:13:04.305220+0000","last_became_active":"2024-05-11T12:13:03.160013+0000","last_became_peered":"2024-05-11T12:13:03.160013+0000","last_unstale":"2024-05-11T12:13:04.305220+0000","last_undegraded":"2024-05-11T12:13:04.305220+0000","last_fullsized":"2024-05-11T12:13:04.305220+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:15:29.709641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.164374+0000","last_change":"2024-05-11T12:13:03.160607+0000","last_active":"2024-05-11T12:13:04.164374+0000","last_peered":"2024-05-11T12:13:04.164374+0000","last_clean":"2024-05-11T12:13:04.164374+0000","last_became_active":"2024-05-11T12:13:03.160281+0000","last_became_peered":"2024-05-11T12:13:03.160281+0000","last_unstale":"2024-05-11T12:13:04.164374+0000","last_undegraded":"2024-05-11T12:13:04.164374+0000","last_fullsized":"2024-05-11T12:13:04.164374+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:07:22.610930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.168211+0000","last_change":"2024-05-11T12:13:03.164270+0000","last_active":"2024-05-11T12:13:04.168211+0000","last_peered":"2024-05-11T12:13:04.168211+0000","last_clean":"2024-05-11T12:13:04.168211+0000","last_became_active":"2024-05-11T12:13:03.163977+0000","last_became_peered":"2024-05-11T12:13:03.163977+0000","last_unstale":"2024-05-11T12:13:04.168211+0000","last_undegraded":"2024-05-11T12:13:04.168211+0000","last_fullsized":"2024-05-11T12:13:04.168211+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:03:05.647677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.303402+0000","last_change":"2024-05-11T12:13:03.171792+0000","last_active":"2024-05-11T12:13:04.303402+0000","last_peered":"2024-05-11T12:13:04.303402+0000","last_clean":"2024-05-11T12:13:04.303402+0000","last_became_active":"2024-05-11T12:13:03.171502+0000","last_became_peered":"2024-05-11T12:13:03.171502+0000","last_unstale":"2024-05-11T12:13:04.303402+0000","last_undegraded":"2024-05-11T12:13:04.303402+0000","last_fullsized":"2024-05-11T12:13:04.303402+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:27:27.374412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.303538+0000","last_change":"2024-05-11T12:13:03.165637+0000","last_active":"2024-05-11T12:13:04.303538+0000","last_peered":"2024-05-11T12:13:04.303538+0000","last_clean":"2024-05-11T12:13:04.303538+0000","last_became_active":"2024-05-11T12:13:03.165323+0000","last_became_peered":"2024-05-11T12:13:03.165323+0000","last_unstale":"2024-05-11T12:13:04.303538+0000","last_undegraded":"2024-05-11T12:13:04.303538+0000","last_fullsized":"2024-05-11T12:13:04.303538+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:47:23.761750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.165122+0000","last_change":"2024-05-11T12:13:03.176021+0000","last_active":"2024-05-11T12:13:04.165122+0000","last_peered":"2024-05-11T12:13:04.165122+0000","last_clean":"2024-05-11T12:13:04.165122+0000","last_became_active":"2024-05-11T12:13:03.175708+0000","last_became_peered":"2024-05-11T12:13:03.175708+0000","last_unstale":"2024-05-11T12:13:04.165122+0000","last_undegraded":"2024-05-11T12:13:04.165122+0000","last_fullsized":"2024-05-11T12:13:04.165122+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:12:30.081833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"63'4","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.305916+0000","last_change":"2024-05-11T12:13:02.136461+0000","last_active":"2024-05-11T12:13:04.305916+0000","last_peered":"2024-05-11T12:13:04.305916+0000","last_clean":"2024-05-11T12:13:04.305916+0000","last_became_active":"2024-05-11T12:13:02.136210+0000","last_became_peered":"2024-05-11T12:13:02.136210+0000","last_unstale":"2024-05-11T12:13:04.305916+0000","last_undegraded":"2024-05-11T12:13:04.305916+0000","last_fullsized":"2024-05-11T12:13:04.305916+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:28:33.468562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.304293+0000","last_change":"2024-05-11T12:13:03.158314+0000","last_active":"2024-05-11T12:13:04.304293+0000","last_peered":"2024-05-11T12:13:04.304293+0000","last_clean":"2024-05-11T12:13:04.304293+0000","last_became_active":"2024-05-11T12:13:03.157991+0000","last_became_peered":"2024-05-11T12:13:03.157991+0000","last_unstale":"2024-05-11T12:13:04.304293+0000","last_undegraded":"2024-05-11T12:13:04.304293+0000","last_fullsized":"2024-05-11T12:13:04.304293+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:45:52.844202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"63'1","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.304764+0000","last_change":"2024-05-11T12:13:02.136740+0000","last_active":"2024-05-11T12:13:04.304764+0000","last_peered":"2024-05-11T12:13:04.304764+0000","last_clean":"2024-05-11T12:13:04.304764+0000","last_became_active":"2024-05-11T12:13:02.136446+0000","last_became_peered":"2024-05-11T12:13:02.136446+0000","last_unstale":"2024-05-11T12:13:04.304764+0000","last_undegraded":"2024-05-11T12:13:04.304764+0000","last_fullsized":"2024-05-11T12:13:04.304764+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:06:50.468601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.305141+0000","last_change":"2024-05-11T12:13:03.165220+0000","last_active":"2024-05-11T12:13:04.305141+0000","last_peered":"2024-05-11T12:13:04.305141+0000","last_clean":"2024-05-11T12:13:04.305141+0000","last_became_active":"2024-05-11T12:13:03.164913+0000","last_became_peered":"2024-05-11T12:13:03.164913+0000","last_unstale":"2024-05-11T12:13:04.305141+0000","last_undegraded":"2024-05-11T12:13:04.305141+0000","last_fullsized":"2024-05-11T12:13:04.305141+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:55:45.751432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"63'2","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.304086+0000","last_change":"2024-05-11T12:13:02.129658+0000","last_active":"2024-05-11T12:13:04.304086+0000","last_peered":"2024-05-11T12:13:04.304086+0000","last_clean":"2024-05-11T12:13:04.304086+0000","last_became_active":"2024-05-11T12:13:02.129326+0000","last_became_peered":"2024-05-11T12:13:02.129326+0000","last_unstale":"2024-05-11T12:13:04.304086+0000","last_undegraded":"2024-05-11T12:13:04.304086+0000","last_fullsized":"2024-05-11T12:13:04.304086+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:29:57.084718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.164371+0000","last_change":"2024-05-11T12:13:03.162149+0000","last_active":"2024-05-11T12:13:04.164371+0000","last_peered":"2024-05-11T12:13:04.164371+0000","last_clean":"2024-05-11T12:13:04.164371+0000","last_became_active":"2024-05-11T12:13:03.161890+0000","last_became_peered":"2024-05-11T12:13:03.161890+0000","last_unstale":"2024-05-11T12:13:04.164371+0000","last_undegraded":"2024-05-11T12:13:04.164371+0000","last_fullsized":"2024-05-11T12:13:04.164371+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:04:07.281796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"63'2","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.304484+0000","last_change":"2024-05-11T12:13:02.126473+0000","last_active":"2024-05-11T12:13:04.304484+0000","last_peered":"2024-05-11T12:13:04.304484+0000","last_clean":"2024-05-11T12:13:04.304484+0000","last_became_active":"2024-05-11T12:13:02.126154+0000","last_became_peered":"2024-05-11T12:13:02.126154+0000","last_unstale":"2024-05-11T12:13:04.304484+0000","last_undegraded":"2024-05-11T12:13:04.304484+0000","last_fullsized":"2024-05-11T12:13:04.304484+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:49:40.991051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.173923+0000","last_change":"2024-05-11T12:13:03.164511+0000","last_active":"2024-05-11T12:13:04.173923+0000","last_peered":"2024-05-11T12:13:04.173923+0000","last_clean":"2024-05-11T12:13:04.173923+0000","last_became_active":"2024-05-11T12:13:03.164197+0000","last_became_peered":"2024-05-11T12:13:03.164197+0000","last_unstale":"2024-05-11T12:13:04.173923+0000","last_undegraded":"2024-05-11T12:13:04.173923+0000","last_fullsized":"2024-05-11T12:13:04.173923+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:54:15.999941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"63'1","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.159666+0000","last_change":"2024-05-11T12:13:02.119338+0000","last_active":"2024-05-11T12:13:04.159666+0000","last_peered":"2024-05-11T12:13:04.159666+0000","last_clean":"2024-05-11T12:13:04.159666+0000","last_became_active":"2024-05-11T12:13:02.118127+0000","last_became_peered":"2024-05-11T12:13:02.118127+0000","last_unstale":"2024-05-11T12:13:04.159666+0000","last_undegraded":"2024-05-11T12:13:04.159666+0000","last_fullsized":"2024-05-11T12:13:04.159666+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:07:21.606002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.303747+0000","last_change":"2024-05-11T12:13:03.169929+0000","last_active":"2024-05-11T12:13:04.303747+0000","last_peered":"2024-05-11T12:13:04.303747+0000","last_clean":"2024-05-11T12:13:04.303747+0000","last_became_active":"2024-05-11T12:13:03.158856+0000","last_became_peered":"2024-05-11T12:13:03.158856+0000","last_unstale":"2024-05-11T12:13:04.303747+0000","last_undegraded":"2024-05-11T12:13:04.303747+0000","last_fullsized":"2024-05-11T12:13:04.303747+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:48:53.182110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"63'1","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.159545+0000","last_change":"2024-05-11T12:13:02.121903+0000","last_active":"2024-05-11T12:13:04.159545+0000","last_peered":"2024-05-11T12:13:04.159545+0000","last_clean":"2024-05-11T12:13:04.159545+0000","last_became_active":"2024-05-11T12:13:02.120293+0000","last_became_peered":"2024-05-11T12:13:02.120293+0000","last_unstale":"2024-05-11T12:13:04.159545+0000","last_undegraded":"2024-05-11T12:13:04.159545+0000","last_fullsized":"2024-05-11T12:13:04.159545+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:00:29.098618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.162391+0000","last_change":"2024-05-11T12:13:03.168572+0000","last_active":"2024-05-11T12:13:04.162391+0000","last_peered":"2024-05-11T12:13:04.162391+0000","last_clean":"2024-05-11T12:13:04.162391+0000","last_became_active":"2024-05-11T12:13:03.167854+0000","last_became_peered":"2024-05-11T12:13:03.167854+0000","last_unstale":"2024-05-11T12:13:04.162391+0000","last_undegraded":"2024-05-11T12:13:04.162391+0000","last_fullsized":"2024-05-11T12:13:04.162391+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:05:06.248364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.159264+0000","last_change":"2024-05-11T12:13:02.139509+0000","last_active":"2024-05-11T12:13:04.159264+0000","last_peered":"2024-05-11T12:13:04.159264+0000","last_clean":"2024-05-11T12:13:04.159264+0000","last_became_active":"2024-05-11T12:13:02.139201+0000","last_became_peered":"2024-05-11T12:13:02.139201+0000","last_unstale":"2024-05-11T12:13:04.159264+0000","last_undegraded":"2024-05-11T12:13:04.159264+0000","last_fullsized":"2024-05-11T12:13:04.159264+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:49:40.991051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.159592+0000","last_change":"2024-05-11T12:13:03.163664+0000","last_active":"2024-05-11T12:13:04.159592+0000","last_peered":"2024-05-11T12:13:04.159592+0000","last_clean":"2024-05-11T12:13:04.159592+0000","last_became_active":"2024-05-11T12:13:03.163297+0000","last_became_peered":"2024-05-11T12:13:03.163297+0000","last_unstale":"2024-05-11T12:13:04.159592+0000","last_undegraded":"2024-05-11T12:13:04.159592+0000","last_fullsized":"2024-05-11T12:13:04.159592+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:21:48.140920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.162083+0000","last_change":"2024-05-11T12:13:02.139692+0000","last_active":"2024-05-11T12:13:04.162083+0000","last_peered":"2024-05-11T12:13:04.162083+0000","last_clean":"2024-05-11T12:13:04.162083+0000","last_became_active":"2024-05-11T12:13:02.137935+0000","last_became_peered":"2024-05-11T12:13:02.137935+0000","last_unstale":"2024-05-11T12:13:04.162083+0000","last_undegraded":"2024-05-11T12:13:04.162083+0000","last_fullsized":"2024-05-11T12:13:04.162083+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:15:34.846112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.305380+0000","last_change":"2024-05-11T12:13:03.171027+0000","last_active":"2024-05-11T12:13:04.305380+0000","last_peered":"2024-05-11T12:13:04.305380+0000","last_clean":"2024-05-11T12:13:04.305380+0000","last_became_active":"2024-05-11T12:13:03.170778+0000","last_became_peered":"2024-05-11T12:13:03.170778+0000","last_unstale":"2024-05-11T12:13:04.305380+0000","last_undegraded":"2024-05-11T12:13:04.305380+0000","last_fullsized":"2024-05-11T12:13:04.305380+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:12:30.081833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.165207+0000","last_change":"2024-05-11T12:13:02.142438+0000","last_active":"2024-05-11T12:13:04.165207+0000","last_peered":"2024-05-11T12:13:04.165207+0000","last_clean":"2024-05-11T12:13:04.165207+0000","last_became_active":"2024-05-11T12:13:02.142125+0000","last_became_peered":"2024-05-11T12:13:02.142125+0000","last_unstale":"2024-05-11T12:13:04.165207+0000","last_undegraded":"2024-05-11T12:13:04.165207+0000","last_fullsized":"2024-05-11T12:13:04.165207+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:55:32.333055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.159784+0000","last_change":"2024-05-11T12:13:03.149501+0000","last_active":"2024-05-11T12:13:04.159784+0000","last_peered":"2024-05-11T12:13:04.159784+0000","last_clean":"2024-05-11T12:13:04.159784+0000","last_became_active":"2024-05-11T12:13:03.149088+0000","last_became_peered":"2024-05-11T12:13:03.149088+0000","last_unstale":"2024-05-11T12:13:04.159784+0000","last_undegraded":"2024-05-11T12:13:04.159784+0000","last_fullsized":"2024-05-11T12:13:04.159784+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:03:01.682416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.304526+0000","last_change":"2024-05-11T12:13:02.140895+0000","last_active":"2024-05-11T12:13:04.304526+0000","last_peered":"2024-05-11T12:13:04.304526+0000","last_clean":"2024-05-11T12:13:04.304526+0000","last_became_active":"2024-05-11T12:13:02.138733+0000","last_became_peered":"2024-05-11T12:13:02.138733+0000","last_unstale":"2024-05-11T12:13:04.304526+0000","last_undegraded":"2024-05-11T12:13:04.304526+0000","last_fullsized":"2024-05-11T12:13:04.304526+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:08:47.140652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.165055+0000","last_change":"2024-05-11T12:13:03.160320+0000","last_active":"2024-05-11T12:13:04.165055+0000","last_peered":"2024-05-11T12:13:04.165055+0000","last_clean":"2024-05-11T12:13:04.165055+0000","last_became_active":"2024-05-11T12:13:03.159984+0000","last_became_peered":"2024-05-11T12:13:03.159984+0000","last_unstale":"2024-05-11T12:13:04.165055+0000","last_undegraded":"2024-05-11T12:13:04.165055+0000","last_fullsized":"2024-05-11T12:13:04.165055+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:15:29.709641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"63'3","reported_seq":17,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.162861+0000","last_change":"2024-05-11T12:13:02.139752+0000","last_active":"2024-05-11T12:13:04.162861+0000","last_peered":"2024-05-11T12:13:04.162861+0000","last_clean":"2024-05-11T12:13:04.162861+0000","last_became_active":"2024-05-11T12:13:02.139431+0000","last_became_peered":"2024-05-11T12:13:02.139431+0000","last_unstale":"2024-05-11T12:13:04.162861+0000","last_undegraded":"2024-05-11T12:13:04.162861+0000","last_fullsized":"2024-05-11T12:13:04.162861+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:00:08.081754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.304594+0000","last_change":"2024-05-11T12:13:03.166818+0000","last_active":"2024-05-11T12:13:04.304594+0000","last_peered":"2024-05-11T12:13:04.304594+0000","last_clean":"2024-05-11T12:13:04.304594+0000","last_became_active":"2024-05-11T12:13:03.166560+0000","last_became_peered":"2024-05-11T12:13:03.166560+0000","last_unstale":"2024-05-11T12:13:04.304594+0000","last_undegraded":"2024-05-11T12:13:04.304594+0000","last_fullsized":"2024-05-11T12:13:04.304594+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:57:14.643928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"63'6","reported_seq":20,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.305493+0000","last_change":"2024-05-11T12:13:02.133552+0000","last_active":"2024-05-11T12:13:04.305493+0000","last_peered":"2024-05-11T12:13:04.305493+0000","last_clean":"2024-05-11T12:13:04.305493+0000","last_became_active":"2024-05-11T12:13:02.133236+0000","last_became_peered":"2024-05-11T12:13:02.133236+0000","last_unstale":"2024-05-11T12:13:04.305493+0000","last_undegraded":"2024-05-11T12:13:04.305493+0000","last_fullsized":"2024-05-11T12:13:04.305493+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:17:18.046920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.303328+0000","last_change":"2024-05-11T12:13:03.168373+0000","last_active":"2024-05-11T12:13:04.303328+0000","last_peered":"2024-05-11T12:13:04.303328+0000","last_clean":"2024-05-11T12:13:04.303328+0000","last_became_active":"2024-05-11T12:13:03.168105+0000","last_became_peered":"2024-05-11T12:13:03.168105+0000","last_unstale":"2024-05-11T12:13:04.303328+0000","last_undegraded":"2024-05-11T12:13:04.303328+0000","last_fullsized":"2024-05-11T12:13:04.303328+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:56:18.568549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"63'2","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.164670+0000","last_change":"2024-05-11T12:13:02.143192+0000","last_active":"2024-05-11T12:13:04.164670+0000","last_peered":"2024-05-11T12:13:04.164670+0000","last_clean":"2024-05-11T12:13:04.164670+0000","last_became_active":"2024-05-11T12:13:02.142901+0000","last_became_peered":"2024-05-11T12:13:02.142901+0000","last_unstale":"2024-05-11T12:13:04.164670+0000","last_undegraded":"2024-05-11T12:13:04.164670+0000","last_fullsized":"2024-05-11T12:13:04.164670+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:39:03.473134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.168159+0000","last_change":"2024-05-11T12:13:03.145731+0000","last_active":"2024-05-11T12:13:04.168159+0000","last_peered":"2024-05-11T12:13:04.168159+0000","last_clean":"2024-05-11T12:13:04.168159+0000","last_became_active":"2024-05-11T12:13:03.145419+0000","last_became_peered":"2024-05-11T12:13:03.145419+0000","last_unstale":"2024-05-11T12:13:04.168159+0000","last_undegraded":"2024-05-11T12:13:04.168159+0000","last_fullsized":"2024-05-11T12:13:04.168159+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:49:41.995979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.167989+0000","last_change":"2024-05-11T12:13:02.118817+0000","last_active":"2024-05-11T12:13:04.167989+0000","last_peered":"2024-05-11T12:13:04.167989+0000","last_clean":"2024-05-11T12:13:04.167989+0000","last_became_active":"2024-05-11T12:13:02.118040+0000","last_became_peered":"2024-05-11T12:13:02.118040+0000","last_unstale":"2024-05-11T12:13:04.167989+0000","last_undegraded":"2024-05-11T12:13:04.167989+0000","last_fullsized":"2024-05-11T12:13:04.167989+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:18:34.044830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.162431+0000","last_change":"2024-05-11T12:13:03.172604+0000","last_active":"2024-05-11T12:13:04.162431+0000","last_peered":"2024-05-11T12:13:04.162431+0000","last_clean":"2024-05-11T12:13:04.162431+0000","last_became_active":"2024-05-11T12:13:03.172198+0000","last_became_peered":"2024-05-11T12:13:03.172198+0000","last_unstale":"2024-05-11T12:13:04.162431+0000","last_undegraded":"2024-05-11T12:13:04.162431+0000","last_fullsized":"2024-05-11T12:13:04.162431+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:57:14.643928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.162024+0000","last_change":"2024-05-11T12:13:02.138604+0000","last_active":"2024-05-11T12:13:04.162024+0000","last_peered":"2024-05-11T12:13:04.162024+0000","last_clean":"2024-05-11T12:13:04.162024+0000","last_became_active":"2024-05-11T12:13:02.138227+0000","last_became_peered":"2024-05-11T12:13:02.138227+0000","last_unstale":"2024-05-11T12:13:04.162024+0000","last_undegraded":"2024-05-11T12:13:04.162024+0000","last_fullsized":"2024-05-11T12:13:04.162024+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:15:28.704713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.304895+0000","last_change":"2024-05-11T12:13:03.169594+0000","last_active":"2024-05-11T12:13:04.304895+0000","last_peered":"2024-05-11T12:13:04.304895+0000","last_clean":"2024-05-11T12:13:04.304895+0000","last_became_active":"2024-05-11T12:13:03.160512+0000","last_became_peered":"2024-05-11T12:13:03.160512+0000","last_unstale":"2024-05-11T12:13:04.304895+0000","last_undegraded":"2024-05-11T12:13:04.304895+0000","last_fullsized":"2024-05-11T12:13:04.304895+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:55:33.337983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"63'8","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.303100+0000","last_change":"2024-05-11T12:13:02.137326+0000","last_active":"2024-05-11T12:13:04.303100+0000","last_peered":"2024-05-11T12:13:04.303100+0000","last_clean":"2024-05-11T12:13:04.303100+0000","last_became_active":"2024-05-11T12:13:02.133763+0000","last_became_peered":"2024-05-11T12:13:02.133763+0000","last_unstale":"2024-05-11T12:13:04.303100+0000","last_undegraded":"2024-05-11T12:13:04.303100+0000","last_fullsized":"2024-05-11T12:13:04.303100+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:55:40.188834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.163325+0000","last_change":"2024-05-11T12:13:03.163731+0000","last_active":"2024-05-11T12:13:04.163325+0000","last_peered":"2024-05-11T12:13:04.163325+0000","last_clean":"2024-05-11T12:13:04.163325+0000","last_became_active":"2024-05-11T12:13:03.161280+0000","last_became_peered":"2024-05-11T12:13:03.161280+0000","last_unstale":"2024-05-11T12:13:04.163325+0000","last_undegraded":"2024-05-11T12:13:04.163325+0000","last_fullsized":"2024-05-11T12:13:04.163325+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:49:41.995979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"63'1","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.170916+0000","last_change":"2024-05-11T12:13:02.124463+0000","last_active":"2024-05-11T12:13:04.170916+0000","last_peered":"2024-05-11T12:13:04.170916+0000","last_clean":"2024-05-11T12:13:04.170916+0000","last_became_active":"2024-05-11T12:13:02.124136+0000","last_became_peered":"2024-05-11T12:13:02.124136+0000","last_unstale":"2024-05-11T12:13:04.170916+0000","last_undegraded":"2024-05-11T12:13:04.170916+0000","last_fullsized":"2024-05-11T12:13:04.170916+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:48:52.177182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.165347+0000","last_change":"2024-05-11T12:13:03.158635+0000","last_active":"2024-05-11T12:13:04.165347+0000","last_peered":"2024-05-11T12:13:04.165347+0000","last_clean":"2024-05-11T12:13:04.165347+0000","last_became_active":"2024-05-11T12:13:03.158311+0000","last_became_peered":"2024-05-11T12:13:03.158311+0000","last_unstale":"2024-05-11T12:13:04.165347+0000","last_undegraded":"2024-05-11T12:13:04.165347+0000","last_fullsized":"2024-05-11T12:13:04.165347+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:42:21.959847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"63'2","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.165258+0000","last_change":"2024-05-11T12:13:02.143315+0000","last_active":"2024-05-11T12:13:04.165258+0000","last_peered":"2024-05-11T12:13:04.165258+0000","last_clean":"2024-05-11T12:13:04.165258+0000","last_became_active":"2024-05-11T12:13:02.143012+0000","last_became_peered":"2024-05-11T12:13:02.143012+0000","last_unstale":"2024-05-11T12:13:04.165258+0000","last_undegraded":"2024-05-11T12:13:04.165258+0000","last_fullsized":"2024-05-11T12:13:04.165258+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:03:26.831522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.304307+0000","last_change":"2024-05-11T12:13:03.159923+0000","last_active":"2024-05-11T12:13:04.304307+0000","last_peered":"2024-05-11T12:13:04.304307+0000","last_clean":"2024-05-11T12:13:04.304307+0000","last_became_active":"2024-05-11T12:13:03.159119+0000","last_became_peered":"2024-05-11T12:13:03.159119+0000","last_unstale":"2024-05-11T12:13:04.304307+0000","last_undegraded":"2024-05-11T12:13:04.304307+0000","last_fullsized":"2024-05-11T12:13:04.304307+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:05:06.248364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"63'1","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.163165+0000","last_change":"2024-05-11T12:13:02.138872+0000","last_active":"2024-05-11T12:13:04.163165+0000","last_peered":"2024-05-11T12:13:04.163165+0000","last_clean":"2024-05-11T12:13:04.163165+0000","last_became_active":"2024-05-11T12:13:02.138589+0000","last_became_peered":"2024-05-11T12:13:02.138589+0000","last_unstale":"2024-05-11T12:13:04.163165+0000","last_undegraded":"2024-05-11T12:13:04.163165+0000","last_fullsized":"2024-05-11T12:13:04.163165+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:48:52.177182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.163764+0000","last_change":"2024-05-11T12:13:03.167395+0000","last_active":"2024-05-11T12:13:04.163764+0000","last_peered":"2024-05-11T12:13:04.163764+0000","last_clean":"2024-05-11T12:13:04.163764+0000","last_became_active":"2024-05-11T12:13:03.167111+0000","last_became_peered":"2024-05-11T12:13:03.167111+0000","last_unstale":"2024-05-11T12:13:04.163764+0000","last_undegraded":"2024-05-11T12:13:04.163764+0000","last_fullsized":"2024-05-11T12:13:04.163764+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:48:53.182110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.159642+0000","last_change":"2024-05-11T12:13:02.137581+0000","last_active":"2024-05-11T12:13:04.159642+0000","last_peered":"2024-05-11T12:13:04.159642+0000","last_clean":"2024-05-11T12:13:04.159642+0000","last_became_active":"2024-05-11T12:13:02.137260+0000","last_became_peered":"2024-05-11T12:13:02.137260+0000","last_unstale":"2024-05-11T12:13:04.159642+0000","last_undegraded":"2024-05-11T12:13:04.159642+0000","last_fullsized":"2024-05-11T12:13:04.159642+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:06:17.754393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.303599+0000","last_change":"2024-05-11T12:13:03.165305+0000","last_active":"2024-05-11T12:13:04.303599+0000","last_peered":"2024-05-11T12:13:04.303599+0000","last_clean":"2024-05-11T12:13:04.303599+0000","last_became_active":"2024-05-11T12:13:03.164856+0000","last_became_peered":"2024-05-11T12:13:03.164856+0000","last_unstale":"2024-05-11T12:13:04.303599+0000","last_undegraded":"2024-05-11T12:13:04.303599+0000","last_fullsized":"2024-05-11T12:13:04.303599+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:22:54.995824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"63'2","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.303439+0000","last_change":"2024-05-11T12:13:02.140367+0000","last_active":"2024-05-11T12:13:04.303439+0000","last_peered":"2024-05-11T12:13:04.303439+0000","last_clean":"2024-05-11T12:13:04.303439+0000","last_became_active":"2024-05-11T12:13:02.140069+0000","last_became_peered":"2024-05-11T12:13:02.140069+0000","last_unstale":"2024-05-11T12:13:04.303439+0000","last_undegraded":"2024-05-11T12:13:04.303439+0000","last_fullsized":"2024-05-11T12:13:04.303439+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:27:13.189890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.168198+0000","last_change":"2024-05-11T12:13:03.171936+0000","last_active":"2024-05-11T12:13:04.168198+0000","last_peered":"2024-05-11T12:13:04.168198+0000","last_clean":"2024-05-11T12:13:04.168198+0000","last_became_active":"2024-05-11T12:13:03.171669+0000","last_became_peered":"2024-05-11T12:13:03.171669+0000","last_unstale":"2024-05-11T12:13:04.168198+0000","last_undegraded":"2024-05-11T12:13:04.168198+0000","last_fullsized":"2024-05-11T12:13:04.168198+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:00:30.103546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.168289+0000","last_change":"2024-05-11T12:13:02.125827+0000","last_active":"2024-05-11T12:13:04.168289+0000","last_peered":"2024-05-11T12:13:04.168289+0000","last_clean":"2024-05-11T12:13:04.168289+0000","last_became_active":"2024-05-11T12:13:02.125520+0000","last_became_peered":"2024-05-11T12:13:02.125520+0000","last_unstale":"2024-05-11T12:13:04.168289+0000","last_undegraded":"2024-05-11T12:13:04.168289+0000","last_fullsized":"2024-05-11T12:13:04.168289+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:00:08.081754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.162832+0000","last_change":"2024-05-11T12:13:03.168791+0000","last_active":"2024-05-11T12:13:04.162832+0000","last_peered":"2024-05-11T12:13:04.162832+0000","last_clean":"2024-05-11T12:13:04.162832+0000","last_became_active":"2024-05-11T12:13:03.168255+0000","last_became_peered":"2024-05-11T12:13:03.168255+0000","last_unstale":"2024-05-11T12:13:04.162832+0000","last_undegraded":"2024-05-11T12:13:04.162832+0000","last_fullsized":"2024-05-11T12:13:04.162832+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:39:04.478062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.303306+0000","last_change":"2024-05-11T12:13:02.136770+0000","last_active":"2024-05-11T12:13:04.303306+0000","last_peered":"2024-05-11T12:13:04.303306+0000","last_clean":"2024-05-11T12:13:04.303306+0000","last_became_active":"2024-05-11T12:13:02.136465+0000","last_became_peered":"2024-05-11T12:13:02.136465+0000","last_unstale":"2024-05-11T12:13:04.303306+0000","last_undegraded":"2024-05-11T12:13:04.303306+0000","last_fullsized":"2024-05-11T12:13:04.303306+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:04:04.992779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.304648+0000","last_change":"2024-05-11T12:13:03.160074+0000","last_active":"2024-05-11T12:13:04.304648+0000","last_peered":"2024-05-11T12:13:04.304648+0000","last_clean":"2024-05-11T12:13:04.304648+0000","last_became_active":"2024-05-11T12:13:03.158195+0000","last_became_peered":"2024-05-11T12:13:03.158195+0000","last_unstale":"2024-05-11T12:13:04.304648+0000","last_undegraded":"2024-05-11T12:13:04.304648+0000","last_fullsized":"2024-05-11T12:13:04.304648+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:39:18.725256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"63'2","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.171210+0000","last_change":"2024-05-11T12:13:02.136984+0000","last_active":"2024-05-11T12:13:04.171210+0000","last_peered":"2024-05-11T12:13:04.171210+0000","last_clean":"2024-05-11T12:13:04.171210+0000","last_became_active":"2024-05-11T12:13:02.135837+0000","last_became_peered":"2024-05-11T12:13:02.135837+0000","last_unstale":"2024-05-11T12:13:04.171210+0000","last_undegraded":"2024-05-11T12:13:04.171210+0000","last_fullsized":"2024-05-11T12:13:04.171210+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:03:26.831522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.173813+0000","last_change":"2024-05-11T12:13:03.170696+0000","last_active":"2024-05-11T12:13:04.173813+0000","last_peered":"2024-05-11T12:13:04.173813+0000","last_clean":"2024-05-11T12:13:04.173813+0000","last_became_active":"2024-05-11T12:13:03.170411+0000","last_became_peered":"2024-05-11T12:13:03.170411+0000","last_unstale":"2024-05-11T12:13:04.173813+0000","last_undegraded":"2024-05-11T12:13:04.173813+0000","last_fullsized":"2024-05-11T12:13:04.173813+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:11:05.631699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"64'1","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.169706+0000","last_change":"2024-05-11T12:13:02.128082+0000","last_active":"2024-05-11T12:13:04.169706+0000","last_peered":"2024-05-11T12:13:04.169706+0000","last_clean":"2024-05-11T12:13:04.169706+0000","last_became_active":"2024-05-11T12:13:02.126292+0000","last_became_peered":"2024-05-11T12:13:02.126292+0000","last_unstale":"2024-05-11T12:13:04.169706+0000","last_undegraded":"2024-05-11T12:13:04.169706+0000","last_fullsized":"2024-05-11T12:13:04.169706+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:15:28.704713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.169393+0000","last_change":"2024-05-11T12:13:03.149098+0000","last_active":"2024-05-11T12:13:04.169393+0000","last_peered":"2024-05-11T12:13:04.169393+0000","last_clean":"2024-05-11T12:13:04.169393+0000","last_became_active":"2024-05-11T12:13:03.148358+0000","last_became_peered":"2024-05-11T12:13:03.148358+0000","last_unstale":"2024-05-11T12:13:04.169393+0000","last_undegraded":"2024-05-11T12:13:04.169393+0000","last_fullsized":"2024-05-11T12:13:04.169393+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:22:54.995824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.304200+0000","last_change":"2024-05-11T12:13:02.129624+0000","last_active":"2024-05-11T12:13:04.304200+0000","last_peered":"2024-05-11T12:13:04.304200+0000","last_clean":"2024-05-11T12:13:04.304200+0000","last_became_active":"2024-05-11T12:13:02.129286+0000","last_became_peered":"2024-05-11T12:13:02.129286+0000","last_unstale":"2024-05-11T12:13:04.304200+0000","last_undegraded":"2024-05-11T12:13:04.304200+0000","last_fullsized":"2024-05-11T12:13:04.304200+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:55:25.067740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.164166+0000","last_change":"2024-05-11T12:13:03.170501+0000","last_active":"2024-05-11T12:13:04.164166+0000","last_peered":"2024-05-11T12:13:04.164166+0000","last_clean":"2024-05-11T12:13:04.164166+0000","last_became_active":"2024-05-11T12:13:03.163918+0000","last_became_peered":"2024-05-11T12:13:03.163918+0000","last_unstale":"2024-05-11T12:13:04.164166+0000","last_undegraded":"2024-05-11T12:13:04.164166+0000","last_fullsized":"2024-05-11T12:13:04.164166+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:03:27.836450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.305473+0000","last_change":"2024-05-11T12:13:02.133532+0000","last_active":"2024-05-11T12:13:04.305473+0000","last_peered":"2024-05-11T12:13:04.305473+0000","last_clean":"2024-05-11T12:13:04.305473+0000","last_became_active":"2024-05-11T12:13:02.133209+0000","last_became_peered":"2024-05-11T12:13:02.133209+0000","last_unstale":"2024-05-11T12:13:04.305473+0000","last_undegraded":"2024-05-11T12:13:04.305473+0000","last_fullsized":"2024-05-11T12:13:04.305473+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:56:17.563622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.161666+0000","last_change":"2024-05-11T12:13:03.163919+0000","last_active":"2024-05-11T12:13:04.161666+0000","last_peered":"2024-05-11T12:13:04.161666+0000","last_clean":"2024-05-11T12:13:04.161666+0000","last_became_active":"2024-05-11T12:13:03.163595+0000","last_became_peered":"2024-05-11T12:13:03.163595+0000","last_unstale":"2024-05-11T12:13:04.161666+0000","last_undegraded":"2024-05-11T12:13:04.161666+0000","last_fullsized":"2024-05-11T12:13:04.161666+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:45:52.844202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.164948+0000","last_change":"2024-05-11T12:13:02.139859+0000","last_active":"2024-05-11T12:13:04.164948+0000","last_peered":"2024-05-11T12:13:04.164948+0000","last_clean":"2024-05-11T12:13:04.164948+0000","last_became_active":"2024-05-11T12:13:02.139265+0000","last_became_peered":"2024-05-11T12:13:02.139265+0000","last_unstale":"2024-05-11T12:13:04.164948+0000","last_undegraded":"2024-05-11T12:13:04.164948+0000","last_fullsized":"2024-05-11T12:13:04.164948+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:18:34.044830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.303608+0000","last_change":"2024-05-11T12:13:03.163034+0000","last_active":"2024-05-11T12:13:04.303608+0000","last_peered":"2024-05-11T12:13:04.303608+0000","last_clean":"2024-05-11T12:13:04.303608+0000","last_became_active":"2024-05-11T12:13:03.160775+0000","last_became_peered":"2024-05-11T12:13:03.160775+0000","last_unstale":"2024-05-11T12:13:04.303608+0000","last_undegraded":"2024-05-11T12:13:04.303608+0000","last_fullsized":"2024-05-11T12:13:04.303608+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:17:19.051848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.159680+0000","last_change":"2024-05-11T12:13:02.113452+0000","last_active":"2024-05-11T12:13:04.159680+0000","last_peered":"2024-05-11T12:13:04.159680+0000","last_clean":"2024-05-11T12:13:04.159680+0000","last_became_active":"2024-05-11T12:13:02.112863+0000","last_became_peered":"2024-05-11T12:13:02.112863+0000","last_unstale":"2024-05-11T12:13:04.159680+0000","last_undegraded":"2024-05-11T12:13:04.159680+0000","last_fullsized":"2024-05-11T12:13:04.159680+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:04:06.276868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.304177+0000","last_change":"2024-05-11T12:13:03.162662+0000","last_active":"2024-05-11T12:13:04.304177+0000","last_peered":"2024-05-11T12:13:04.304177+0000","last_clean":"2024-05-11T12:13:04.304177+0000","last_became_active":"2024-05-11T12:13:03.162424+0000","last_became_peered":"2024-05-11T12:13:03.162424+0000","last_unstale":"2024-05-11T12:13:04.304177+0000","last_undegraded":"2024-05-11T12:13:04.304177+0000","last_fullsized":"2024-05-11T12:13:04.304177+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:03:27.836450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.159337+0000","last_change":"2024-05-11T12:13:02.133595+0000","last_active":"2024-05-11T12:13:04.159337+0000","last_peered":"2024-05-11T12:13:04.159337+0000","last_clean":"2024-05-11T12:13:04.159337+0000","last_became_active":"2024-05-11T12:13:02.133168+0000","last_became_peered":"2024-05-11T12:13:02.133168+0000","last_unstale":"2024-05-11T12:13:04.159337+0000","last_undegraded":"2024-05-11T12:13:04.159337+0000","last_fullsized":"2024-05-11T12:13:04.159337+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:03:04.642749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.173843+0000","last_change":"2024-05-11T12:13:03.169252+0000","last_active":"2024-05-11T12:13:04.173843+0000","last_peered":"2024-05-11T12:13:04.173843+0000","last_clean":"2024-05-11T12:13:04.173843+0000","last_became_active":"2024-05-11T12:13:03.167540+0000","last_became_peered":"2024-05-11T12:13:03.167540+0000","last_unstale":"2024-05-11T12:13:04.173843+0000","last_undegraded":"2024-05-11T12:13:04.173843+0000","last_fullsized":"2024-05-11T12:13:04.173843+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:38:22.482431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.304329+0000","last_change":"2024-05-11T12:13:02.140835+0000","last_active":"2024-05-11T12:13:04.304329+0000","last_peered":"2024-05-11T12:13:04.304329+0000","last_clean":"2024-05-11T12:13:04.304329+0000","last_became_active":"2024-05-11T12:13:02.139971+0000","last_became_peered":"2024-05-11T12:13:02.139971+0000","last_unstale":"2024-05-11T12:13:04.304329+0000","last_undegraded":"2024-05-11T12:13:04.304329+0000","last_fullsized":"2024-05-11T12:13:04.304329+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:18:49.739422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.163704+0000","last_change":"2024-05-11T12:13:03.166247+0000","last_active":"2024-05-11T12:13:04.163704+0000","last_peered":"2024-05-11T12:13:04.163704+0000","last_clean":"2024-05-11T12:13:04.163704+0000","last_became_active":"2024-05-11T12:13:03.165118+0000","last_became_peered":"2024-05-11T12:13:03.165118+0000","last_unstale":"2024-05-11T12:13:04.163704+0000","last_undegraded":"2024-05-11T12:13:04.163704+0000","last_fullsized":"2024-05-11T12:13:04.163704+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:00:09.086682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.159150+0000","last_change":"2024-05-11T12:13:02.116771+0000","last_active":"2024-05-11T12:13:04.159150+0000","last_peered":"2024-05-11T12:13:04.159150+0000","last_clean":"2024-05-11T12:13:04.159150+0000","last_became_active":"2024-05-11T12:13:02.116028+0000","last_became_peered":"2024-05-11T12:13:02.116028+0000","last_unstale":"2024-05-11T12:13:04.159150+0000","last_undegraded":"2024-05-11T12:13:04.159150+0000","last_fullsized":"2024-05-11T12:13:04.159150+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:06:50.468601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":185,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.165359+0000","last_change":"2024-05-11T12:10:35.066673+0000","last_active":"2024-05-11T12:13:04.165359+0000","last_peered":"2024-05-11T12:13:04.165359+0000","last_clean":"2024-05-11T12:13:04.165359+0000","last_became_active":"2024-05-11T12:10:35.065942+0000","last_became_peered":"2024-05-11T12:10:35.065942+0000","last_unstale":"2024-05-11T12:13:04.165359+0000","last_undegraded":"2024-05-11T12:13:04.165359+0000","last_fullsized":"2024-05-11T12:13:04.165359+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_clean_scrub_stamp":"2024-05-11T12:10:33.601136+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:12:02.133182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476770,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28368,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343472,"statfs":{"total":96636764160,"available":96607715328,"internally_reserved":0,"allocated":1372160,"data_stored":930804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476771,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27776,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344064,"statfs":{"total":96636764160,"available":96608321536,"internally_reserved":0,"allocated":765952,"data_stored":339694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476770,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":765952,"data_stored":339682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476772,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27776,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344064,"statfs":{"total":96636764160,"available":96608321536,"internally_reserved":0,"allocated":774144,"data_stored":341272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509475,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31924,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339916,"statfs":{"total":96636764160,"available":96604073984,"internally_reserved":0,"allocated":761856,"data_stored":339660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509476,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":790528,"data_stored":342036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509475,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32512,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339328,"statfs":{"total":96636764160,"available":96603471872,"internally_reserved":0,"allocated":1355776,"data_stored":930028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542179,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":761856,"data_stored":339660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T12:13:12.622 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T12:13:12.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.836+0000 7fe59de00640 1 Processor -- start 2024-05-11T12:13:12.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.837+0000 7fe59de00640 1 -- start start 2024-05-11T12:13:12.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.838+0000 7fe59de00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe598157750 0x7fe59815d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:12.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.838+0000 7fe59de00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe59806d460 0x7fe59806d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:12.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.838+0000 7fe59de00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe59815dc80 con 0x7fe5980b3ea0 2024-05-11T12:13:12.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.838+0000 7fe59de00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fe598161000 con 0x7fe59806d460 2024-05-11T12:13:12.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.838+0000 7fe59de00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fe59806ddd0 con 0x7fe598157750 2024-05-11T12:13:12.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.838+0000 7fe597400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe59806d460 0x7fe59806d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:12.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.839+0000 7fe597400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe59806d460 0x7fe59806d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41442/0 (socket says 172.21.15.167:41442) 2024-05-11T12:13:12.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.839+0000 7fe597400640 1 -- 172.21.15.167:0/283217819 learned_addr learned my addr 172.21.15.167:0/283217819 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:12.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.839+0000 7fe596a00640 1 --2- 172.21.15.167:0/283217819 >> v2:172.21.15.204:3300/0 conn(0x7fe598157750 0x7fe59815d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:12.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.839+0000 7fe597400640 1 -- 172.21.15.167:0/283217819 >> v2:172.21.15.204:3300/0 conn(0x7fe598157750 msgr2=0x7fe59815d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:12.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.839+0000 7fe597400640 1 --2- 172.21.15.167:0/283217819 >> v2:172.21.15.204:3300/0 conn(0x7fe598157750 0x7fe59815d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:12.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.839+0000 7fe597400640 1 -- 172.21.15.167:0/283217819 >> v1:172.21.15.204:6789/0 conn(0x7fe5980b3ea0 legacy=0x7fe598161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:12.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.839+0000 7fe597400640 1 -- 172.21.15.167:0/283217819 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe59815df30 con 0x7fe59806d460 2024-05-11T12:13:12.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.839+0000 7fe596a00640 1 --2- 172.21.15.167:0/283217819 >> v2:172.21.15.204:3300/0 conn(0x7fe598157750 0x7fe59815d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:12.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.840+0000 7fe597400640 1 --2- 172.21.15.167:0/283217819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe59806d460 0x7fe59806d890 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe5980ce840 tx=0x7fe59002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cd1919612168568 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:12.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.840+0000 7fe595600640 1 -- 172.21.15.167:0/283217819 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe590037070 con 0x7fe59806d460 2024-05-11T12:13:12.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.841+0000 7fe595600640 1 -- 172.21.15.167:0/283217819 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe590032440 con 0x7fe59806d460 2024-05-11T12:13:12.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.841+0000 7fe595600640 1 -- 172.21.15.167:0/283217819 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe590032810 con 0x7fe59806d460 2024-05-11T12:13:12.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.841+0000 7fe59de00640 1 -- 172.21.15.167:0/283217819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe59806d460 msgr2=0x7fe59806d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:12.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.841+0000 7fe59de00640 1 --2- 172.21.15.167:0/283217819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe59806d460 0x7fe59806d890 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe5980ce840 tx=0x7fe59002a1e0 comp rx=0 tx=0).stop 2024-05-11T12:13:12.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.841+0000 7fe59de00640 1 -- 172.21.15.167:0/283217819 shutdown_connections 2024-05-11T12:13:12.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.841+0000 7fe59de00640 1 --2- 172.21.15.167:0/283217819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe59806d460 0x7fe59806d890 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:12.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.841+0000 7fe59de00640 1 --2- 172.21.15.167:0/283217819 >> v2:172.21.15.204:3300/0 conn(0x7fe598157750 0x7fe59815d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:12.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.841+0000 7fe59de00640 1 -- 172.21.15.167:0/283217819 >> 172.21.15.167:0/283217819 conn(0x7fe5980cd8f0 msgr2=0x7fe5980cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:12.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.841+0000 7fe59de00640 1 -- 172.21.15.167:0/283217819 shutdown_connections 2024-05-11T12:13:12.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.841+0000 7fe59de00640 1 -- 172.21.15.167:0/283217819 wait complete. 2024-05-11T12:13:12.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.843+0000 7fe59de00640 1 Processor -- start 2024-05-11T12:13:12.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.843+0000 7fe59de00640 1 -- start start 2024-05-11T12:13:12.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.844+0000 7fe59de00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe59806d460 0x7fe59813bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:12.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.844+0000 7fe59de00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe5980b3ea0 0x7fe5980ad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:12.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.844+0000 7fe59de00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe598157750 0x7fe5981568a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:12.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.844+0000 7fe59de00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe598157b30 con 0x7fe59806d460 2024-05-11T12:13:12.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.844+0000 7fe59de00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe59815e4f0 con 0x7fe598157750 2024-05-11T12:13:12.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.844+0000 7fe59de00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe598161ba0 con 0x7fe5980b3ea0 2024-05-11T12:13:12.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.844+0000 7fe596a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe59806d460 0x7fe59813bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:12.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.845+0000 7fe596a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe59806d460 0x7fe59813bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41450/0 (socket says 172.21.15.167:41450) 2024-05-11T12:13:12.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.845+0000 7fe596a00640 1 -- 172.21.15.167:0/1762689766 learned_addr learned my addr 172.21.15.167:0/1762689766 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:12.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.845+0000 7fe596a00640 1 -- 172.21.15.167:0/1762689766 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe5980b3ea0 msgr2=0x7fe5980ad4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:12.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.845+0000 7fe596a00640 1 --2- 172.21.15.167:0/1762689766 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe5980b3ea0 0x7fe5980ad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:12.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.845+0000 7fe596a00640 1 -- 172.21.15.167:0/1762689766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe598157750 msgr2=0x7fe5981568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:12.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.845+0000 7fe596a00640 1 --2- 172.21.15.167:0/1762689766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe598157750 0x7fe5981568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:12.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.845+0000 7fe596a00640 1 -- 172.21.15.167:0/1762689766 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe59815c710 con 0x7fe59806d460 2024-05-11T12:13:12.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.845+0000 7fe596a00640 1 --2- 172.21.15.167:0/1762689766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe59806d460 0x7fe59813bac0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe58800c7f0 tx=0x7fe58800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:12.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.845+0000 7fe58f400640 1 -- 172.21.15.167:0/1762689766 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe588015070 con 0x7fe59806d460 2024-05-11T12:13:12.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.846+0000 7fe58f400640 1 -- 172.21.15.167:0/1762689766 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe588007a30 con 0x7fe59806d460 2024-05-11T12:13:12.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.846+0000 7fe58f400640 1 -- 172.21.15.167:0/1762689766 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe588007e00 con 0x7fe59806d460 2024-05-11T12:13:12.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.846+0000 7fe59de00640 1 -- 172.21.15.167:0/1762689766 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe59815bb90 con 0x7fe59806d460 2024-05-11T12:13:12.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.846+0000 7fe59de00640 1 -- 172.21.15.167:0/1762689766 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe59815c430 con 0x7fe59806d460 2024-05-11T12:13:12.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.847+0000 7fe59de00640 1 -- 172.21.15.167:0/1762689766 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe59815bf80 con 0x7fe59806d460 2024-05-11T12:13:12.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.848+0000 7fe58f400640 1 -- 172.21.15.167:0/1762689766 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe588018070 con 0x7fe59806d460 2024-05-11T12:13:12.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.849+0000 7fe58f400640 1 --2- 172.21.15.167:0/1762689766 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe5740762d0 0x7fe574078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:12.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.850+0000 7fe58f400640 1 -- 172.21.15.167:0/1762689766 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7fe588097780 con 0x7fe59806d460 2024-05-11T12:13:12.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.850+0000 7fe596000640 1 --2- 172.21.15.167:0/1762689766 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe5740762d0 0x7fe574078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:12.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.850+0000 7fe596000640 1 --2- 172.21.15.167:0/1762689766 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe5740762d0 0x7fe574078790 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe5981f6180 tx=0x7fe580007d10 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:12.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:12.854+0000 7fe58f400640 1 -- 172.21.15.167:0/1762689766 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe588060eb0 con 0x7fe59806d460 2024-05-11T12:13:13.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.065+0000 7fe59de00640 1 -- 172.21.15.167:0/1762689766 --> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe5980aa920 con 0x7fe5740762d0 2024-05-11T12:13:13.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.076+0000 7fe58f400640 1 -- 172.21.15.167:0/1762689766 <== mgr.4106 v2:172.21.15.167:6832/578352559 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252009 (secure 0 0 0) 0x7fe580003970 con 0x7fe5740762d0 2024-05-11T12:13:13.082 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:13:13.084 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2024-05-11T12:13:13.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.081+0000 7fe59de00640 1 -- 172.21.15.167:0/1762689766 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe5740762d0 msgr2=0x7fe574078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:13.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.081+0000 7fe59de00640 1 --2- 172.21.15.167:0/1762689766 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe5740762d0 0x7fe574078790 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe5981f6180 tx=0x7fe580007d10 comp rx=0 tx=0).stop 2024-05-11T12:13:13.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.081+0000 7fe59de00640 1 -- 172.21.15.167:0/1762689766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe59806d460 msgr2=0x7fe59813bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:13.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.081+0000 7fe59de00640 1 --2- 172.21.15.167:0/1762689766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe59806d460 0x7fe59813bac0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe58800c7f0 tx=0x7fe58800cbf0 comp rx=0 tx=0).stop 2024-05-11T12:13:13.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.081+0000 7fe59de00640 1 -- 172.21.15.167:0/1762689766 shutdown_connections 2024-05-11T12:13:13.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.081+0000 7fe59de00640 1 --2- 172.21.15.167:0/1762689766 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe5740762d0 0x7fe574078790 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:13.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.081+0000 7fe59de00640 1 --2- 172.21.15.167:0/1762689766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe598157750 0x7fe5981568a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:13.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.081+0000 7fe59de00640 1 --2- 172.21.15.167:0/1762689766 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe5980b3ea0 0x7fe5980ad4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:13.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.081+0000 7fe59de00640 1 --2- 172.21.15.167:0/1762689766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe59806d460 0x7fe59813bac0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:13.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.081+0000 7fe59de00640 1 -- 172.21.15.167:0/1762689766 >> 172.21.15.167:0/1762689766 conn(0x7fe5980cd8f0 msgr2=0x7fe5980681c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:13.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.081+0000 7fe59de00640 1 -- 172.21.15.167:0/1762689766 shutdown_connections 2024-05-11T12:13:13.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.081+0000 7fe59de00640 1 -- 172.21.15.167:0/1762689766 wait complete. 2024-05-11T12:13:13.100 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":137,"stamp":"2024-05-11T12:13:11.280122+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231684,"kb_used_data":7176,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754743036,"statfs":{"total":773094113280,"available":772856868864,"internally_reserved":0,"allocated":7348224,"data_stored":3902836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":229756465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.153914"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.305824+0000","last_change":"2024-05-11T12:13:03.162419+0000","last_active":"2024-05-11T12:13:04.305824+0000","last_peered":"2024-05-11T12:13:04.305824+0000","last_clean":"2024-05-11T12:13:04.305824+0000","last_became_active":"2024-05-11T12:13:03.162146+0000","last_became_peered":"2024-05-11T12:13:03.162146+0000","last_unstale":"2024-05-11T12:13:04.305824+0000","last_undegraded":"2024-05-11T12:13:04.305824+0000","last_fullsized":"2024-05-11T12:13:04.305824+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:58:08.814828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.305666+0000","last_change":"2024-05-11T12:13:03.168274+0000","last_active":"2024-05-11T12:13:04.305666+0000","last_peered":"2024-05-11T12:13:04.305666+0000","last_clean":"2024-05-11T12:13:04.305666+0000","last_became_active":"2024-05-11T12:13:03.168022+0000","last_became_peered":"2024-05-11T12:13:03.168022+0000","last_unstale":"2024-05-11T12:13:04.305666+0000","last_undegraded":"2024-05-11T12:13:04.305666+0000","last_fullsized":"2024-05-11T12:13:04.305666+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:42:21.959847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.162597+0000","last_change":"2024-05-11T12:13:03.165899+0000","last_active":"2024-05-11T12:13:04.162597+0000","last_peered":"2024-05-11T12:13:04.162597+0000","last_clean":"2024-05-11T12:13:04.162597+0000","last_became_active":"2024-05-11T12:13:03.165494+0000","last_became_peered":"2024-05-11T12:13:03.165494+0000","last_unstale":"2024-05-11T12:13:04.162597+0000","last_undegraded":"2024-05-11T12:13:04.162597+0000","last_fullsized":"2024-05-11T12:13:04.162597+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:06:51.473529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.304221+0000","last_change":"2024-05-11T12:13:03.166762+0000","last_active":"2024-05-11T12:13:04.304221+0000","last_peered":"2024-05-11T12:13:04.304221+0000","last_clean":"2024-05-11T12:13:04.304221+0000","last_became_active":"2024-05-11T12:13:03.166457+0000","last_became_peered":"2024-05-11T12:13:03.166457+0000","last_unstale":"2024-05-11T12:13:04.304221+0000","last_undegraded":"2024-05-11T12:13:04.304221+0000","last_fullsized":"2024-05-11T12:13:04.304221+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:12:07.890717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.163278+0000","last_change":"2024-05-11T12:13:03.167675+0000","last_active":"2024-05-11T12:13:04.163278+0000","last_peered":"2024-05-11T12:13:04.163278+0000","last_clean":"2024-05-11T12:13:04.163278+0000","last_became_active":"2024-05-11T12:13:03.167381+0000","last_became_peered":"2024-05-11T12:13:03.167381+0000","last_unstale":"2024-05-11T12:13:04.163278+0000","last_undegraded":"2024-05-11T12:13:04.163278+0000","last_fullsized":"2024-05-11T12:13:04.163278+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:18:35.049757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.303733+0000","last_change":"2024-05-11T12:13:03.165292+0000","last_active":"2024-05-11T12:13:04.303733+0000","last_peered":"2024-05-11T12:13:04.303733+0000","last_clean":"2024-05-11T12:13:04.303733+0000","last_became_active":"2024-05-11T12:13:03.164960+0000","last_became_peered":"2024-05-11T12:13:03.164960+0000","last_unstale":"2024-05-11T12:13:04.303733+0000","last_undegraded":"2024-05-11T12:13:04.303733+0000","last_fullsized":"2024-05-11T12:13:04.303733+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:05:24.561464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.173969+0000","last_change":"2024-05-11T12:13:03.170170+0000","last_active":"2024-05-11T12:13:04.173969+0000","last_peered":"2024-05-11T12:13:04.173969+0000","last_clean":"2024-05-11T12:13:04.173969+0000","last_became_active":"2024-05-11T12:13:03.169888+0000","last_became_peered":"2024-05-11T12:13:03.169888+0000","last_unstale":"2024-05-11T12:13:04.173969+0000","last_undegraded":"2024-05-11T12:13:04.173969+0000","last_fullsized":"2024-05-11T12:13:04.173969+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:46:12.686412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.168062+0000","last_change":"2024-05-11T12:13:03.167147+0000","last_active":"2024-05-11T12:13:04.168062+0000","last_peered":"2024-05-11T12:13:04.168062+0000","last_clean":"2024-05-11T12:13:04.168062+0000","last_became_active":"2024-05-11T12:13:03.166882+0000","last_became_peered":"2024-05-11T12:13:03.166882+0000","last_unstale":"2024-05-11T12:13:04.168062+0000","last_undegraded":"2024-05-11T12:13:04.168062+0000","last_fullsized":"2024-05-11T12:13:04.168062+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:06:51.473529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.164785+0000","last_change":"2024-05-11T12:13:03.162128+0000","last_active":"2024-05-11T12:13:04.164785+0000","last_peered":"2024-05-11T12:13:04.164785+0000","last_clean":"2024-05-11T12:13:04.164785+0000","last_became_active":"2024-05-11T12:13:03.161841+0000","last_became_peered":"2024-05-11T12:13:03.161841+0000","last_unstale":"2024-05-11T12:13:04.164785+0000","last_undegraded":"2024-05-11T12:13:04.164785+0000","last_fullsized":"2024-05-11T12:13:04.164785+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:06:18.759321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.164997+0000","last_change":"2024-05-11T12:13:03.164933+0000","last_active":"2024-05-11T12:13:04.164997+0000","last_peered":"2024-05-11T12:13:04.164997+0000","last_clean":"2024-05-11T12:13:04.164997+0000","last_became_active":"2024-05-11T12:13:03.164680+0000","last_became_peered":"2024-05-11T12:13:03.164680+0000","last_unstale":"2024-05-11T12:13:04.164997+0000","last_undegraded":"2024-05-11T12:13:04.164997+0000","last_fullsized":"2024-05-11T12:13:04.164997+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:47:23.761750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.303848+0000","last_change":"2024-05-11T12:13:03.163978+0000","last_active":"2024-05-11T12:13:04.303848+0000","last_peered":"2024-05-11T12:13:04.303848+0000","last_clean":"2024-05-11T12:13:04.303848+0000","last_became_active":"2024-05-11T12:13:03.163688+0000","last_became_peered":"2024-05-11T12:13:03.163688+0000","last_unstale":"2024-05-11T12:13:04.303848+0000","last_undegraded":"2024-05-11T12:13:04.303848+0000","last_fullsized":"2024-05-11T12:13:04.303848+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:18:50.744350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.168376+0000","last_change":"2024-05-11T12:13:03.164295+0000","last_active":"2024-05-11T12:13:04.168376+0000","last_peered":"2024-05-11T12:13:04.168376+0000","last_clean":"2024-05-11T12:13:04.168376+0000","last_became_active":"2024-05-11T12:13:03.163948+0000","last_became_peered":"2024-05-11T12:13:03.163948+0000","last_unstale":"2024-05-11T12:13:04.168376+0000","last_undegraded":"2024-05-11T12:13:04.168376+0000","last_fullsized":"2024-05-11T12:13:04.168376+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:04:07.281796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.168867+0000","last_change":"2024-05-11T12:13:03.163777+0000","last_active":"2024-05-11T12:13:04.168867+0000","last_peered":"2024-05-11T12:13:04.168867+0000","last_clean":"2024-05-11T12:13:04.168867+0000","last_became_active":"2024-05-11T12:13:03.163472+0000","last_became_peered":"2024-05-11T12:13:03.163472+0000","last_unstale":"2024-05-11T12:13:04.168867+0000","last_undegraded":"2024-05-11T12:13:04.168867+0000","last_fullsized":"2024-05-11T12:13:04.168867+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:07:22.610930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.163696+0000","last_change":"2024-05-11T12:13:03.165418+0000","last_active":"2024-05-11T12:13:04.163696+0000","last_peered":"2024-05-11T12:13:04.163696+0000","last_clean":"2024-05-11T12:13:04.163696+0000","last_became_active":"2024-05-11T12:13:03.165020+0000","last_became_peered":"2024-05-11T12:13:03.165020+0000","last_unstale":"2024-05-11T12:13:04.163696+0000","last_undegraded":"2024-05-11T12:13:04.163696+0000","last_fullsized":"2024-05-11T12:13:04.163696+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:03:05.647677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.161876+0000","last_change":"2024-05-11T12:13:03.174279+0000","last_active":"2024-05-11T12:13:04.161876+0000","last_peered":"2024-05-11T12:13:04.161876+0000","last_clean":"2024-05-11T12:13:04.161876+0000","last_became_active":"2024-05-11T12:13:03.173960+0000","last_became_peered":"2024-05-11T12:13:03.173960+0000","last_unstale":"2024-05-11T12:13:04.161876+0000","last_undegraded":"2024-05-11T12:13:04.161876+0000","last_fullsized":"2024-05-11T12:13:04.161876+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:55:26.072668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.173798+0000","last_change":"2024-05-11T12:13:03.176759+0000","last_active":"2024-05-11T12:13:04.173798+0000","last_peered":"2024-05-11T12:13:04.173798+0000","last_clean":"2024-05-11T12:13:04.173798+0000","last_became_active":"2024-05-11T12:13:03.176496+0000","last_became_peered":"2024-05-11T12:13:03.176496+0000","last_unstale":"2024-05-11T12:13:04.173798+0000","last_undegraded":"2024-05-11T12:13:04.173798+0000","last_fullsized":"2024-05-11T12:13:04.173798+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:33:36.274987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.305957+0000","last_change":"2024-05-11T12:13:03.161765+0000","last_active":"2024-05-11T12:13:04.305957+0000","last_peered":"2024-05-11T12:13:04.305957+0000","last_clean":"2024-05-11T12:13:04.305957+0000","last_became_active":"2024-05-11T12:13:03.159361+0000","last_became_peered":"2024-05-11T12:13:03.159361+0000","last_unstale":"2024-05-11T12:13:04.305957+0000","last_undegraded":"2024-05-11T12:13:04.305957+0000","last_fullsized":"2024-05-11T12:13:04.305957+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:08:16.138838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.304239+0000","last_change":"2024-05-11T12:13:03.167581+0000","last_active":"2024-05-11T12:13:04.304239+0000","last_peered":"2024-05-11T12:13:04.304239+0000","last_clean":"2024-05-11T12:13:04.304239+0000","last_became_active":"2024-05-11T12:13:03.167285+0000","last_became_peered":"2024-05-11T12:13:03.167285+0000","last_unstale":"2024-05-11T12:13:04.304239+0000","last_undegraded":"2024-05-11T12:13:04.304239+0000","last_fullsized":"2024-05-11T12:13:04.304239+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:04:57.052060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.164975+0000","last_change":"2024-05-11T12:13:03.172989+0000","last_active":"2024-05-11T12:13:04.164975+0000","last_peered":"2024-05-11T12:13:04.164975+0000","last_clean":"2024-05-11T12:13:04.164975+0000","last_became_active":"2024-05-11T12:13:03.172704+0000","last_became_peered":"2024-05-11T12:13:03.172704+0000","last_unstale":"2024-05-11T12:13:04.164975+0000","last_undegraded":"2024-05-11T12:13:04.164975+0000","last_fullsized":"2024-05-11T12:13:04.164975+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:55:33.337983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.159946+0000","last_change":"2024-05-11T12:13:03.162259+0000","last_active":"2024-05-11T12:13:04.159946+0000","last_peered":"2024-05-11T12:13:04.159946+0000","last_clean":"2024-05-11T12:13:04.159946+0000","last_became_active":"2024-05-11T12:13:03.159517+0000","last_became_peered":"2024-05-11T12:13:03.159517+0000","last_unstale":"2024-05-11T12:13:04.159946+0000","last_undegraded":"2024-05-11T12:13:04.159946+0000","last_fullsized":"2024-05-11T12:13:04.159946+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:10:38.278696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.169063+0000","last_change":"2024-05-11T12:13:03.162655+0000","last_active":"2024-05-11T12:13:04.169063+0000","last_peered":"2024-05-11T12:13:04.169063+0000","last_clean":"2024-05-11T12:13:04.169063+0000","last_became_active":"2024-05-11T12:13:03.162259+0000","last_became_peered":"2024-05-11T12:13:03.162259+0000","last_unstale":"2024-05-11T12:13:04.169063+0000","last_undegraded":"2024-05-11T12:13:04.169063+0000","last_fullsized":"2024-05-11T12:13:04.169063+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:06:18.759321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.159320+0000","last_change":"2024-05-11T12:13:03.166071+0000","last_active":"2024-05-11T12:13:04.159320+0000","last_peered":"2024-05-11T12:13:04.159320+0000","last_clean":"2024-05-11T12:13:04.159320+0000","last_became_active":"2024-05-11T12:13:03.165691+0000","last_became_peered":"2024-05-11T12:13:03.165691+0000","last_unstale":"2024-05-11T12:13:04.159320+0000","last_undegraded":"2024-05-11T12:13:04.159320+0000","last_fullsized":"2024-05-11T12:13:04.159320+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:57:28.721233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.159421+0000","last_change":"2024-05-11T12:13:03.161024+0000","last_active":"2024-05-11T12:13:04.159421+0000","last_peered":"2024-05-11T12:13:04.159421+0000","last_clean":"2024-05-11T12:13:04.159421+0000","last_became_active":"2024-05-11T12:13:03.160734+0000","last_became_peered":"2024-05-11T12:13:03.160734+0000","last_unstale":"2024-05-11T12:13:04.159421+0000","last_undegraded":"2024-05-11T12:13:04.159421+0000","last_fullsized":"2024-05-11T12:13:04.159421+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:23:31.280589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.164305+0000","last_change":"2024-05-11T12:13:03.165438+0000","last_active":"2024-05-11T12:13:04.164305+0000","last_peered":"2024-05-11T12:13:04.164305+0000","last_clean":"2024-05-11T12:13:04.164305+0000","last_became_active":"2024-05-11T12:13:03.165118+0000","last_became_peered":"2024-05-11T12:13:03.165118+0000","last_unstale":"2024-05-11T12:13:04.164305+0000","last_undegraded":"2024-05-11T12:13:04.164305+0000","last_fullsized":"2024-05-11T12:13:04.164305+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:00:30.103546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.170861+0000","last_change":"2024-05-11T12:13:03.169517+0000","last_active":"2024-05-11T12:13:04.170861+0000","last_peered":"2024-05-11T12:13:04.170861+0000","last_clean":"2024-05-11T12:13:04.170861+0000","last_became_active":"2024-05-11T12:13:03.169235+0000","last_became_peered":"2024-05-11T12:13:03.169235+0000","last_unstale":"2024-05-11T12:13:04.170861+0000","last_undegraded":"2024-05-11T12:13:04.170861+0000","last_fullsized":"2024-05-11T12:13:04.170861+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:05:24.561464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.169110+0000","last_change":"2024-05-11T12:13:03.166968+0000","last_active":"2024-05-11T12:13:04.169110+0000","last_peered":"2024-05-11T12:13:04.169110+0000","last_clean":"2024-05-11T12:13:04.169110+0000","last_became_active":"2024-05-11T12:13:03.166704+0000","last_became_peered":"2024-05-11T12:13:03.166704+0000","last_unstale":"2024-05-11T12:13:04.169110+0000","last_undegraded":"2024-05-11T12:13:04.169110+0000","last_fullsized":"2024-05-11T12:13:04.169110+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:47:23.761750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.303938+0000","last_change":"2024-05-11T12:13:03.171885+0000","last_active":"2024-05-11T12:13:04.303938+0000","last_peered":"2024-05-11T12:13:04.303938+0000","last_clean":"2024-05-11T12:13:04.303938+0000","last_became_active":"2024-05-11T12:13:03.171590+0000","last_became_peered":"2024-05-11T12:13:03.171590+0000","last_unstale":"2024-05-11T12:13:04.303938+0000","last_undegraded":"2024-05-11T12:13:04.303938+0000","last_fullsized":"2024-05-11T12:13:04.303938+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:28:34.473489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.305220+0000","last_change":"2024-05-11T12:13:03.166734+0000","last_active":"2024-05-11T12:13:04.305220+0000","last_peered":"2024-05-11T12:13:04.305220+0000","last_clean":"2024-05-11T12:13:04.305220+0000","last_became_active":"2024-05-11T12:13:03.160013+0000","last_became_peered":"2024-05-11T12:13:03.160013+0000","last_unstale":"2024-05-11T12:13:04.305220+0000","last_undegraded":"2024-05-11T12:13:04.305220+0000","last_fullsized":"2024-05-11T12:13:04.305220+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:15:29.709641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.164374+0000","last_change":"2024-05-11T12:13:03.160607+0000","last_active":"2024-05-11T12:13:04.164374+0000","last_peered":"2024-05-11T12:13:04.164374+0000","last_clean":"2024-05-11T12:13:04.164374+0000","last_became_active":"2024-05-11T12:13:03.160281+0000","last_became_peered":"2024-05-11T12:13:03.160281+0000","last_unstale":"2024-05-11T12:13:04.164374+0000","last_undegraded":"2024-05-11T12:13:04.164374+0000","last_fullsized":"2024-05-11T12:13:04.164374+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:07:22.610930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.168211+0000","last_change":"2024-05-11T12:13:03.164270+0000","last_active":"2024-05-11T12:13:04.168211+0000","last_peered":"2024-05-11T12:13:04.168211+0000","last_clean":"2024-05-11T12:13:04.168211+0000","last_became_active":"2024-05-11T12:13:03.163977+0000","last_became_peered":"2024-05-11T12:13:03.163977+0000","last_unstale":"2024-05-11T12:13:04.168211+0000","last_undegraded":"2024-05-11T12:13:04.168211+0000","last_fullsized":"2024-05-11T12:13:04.168211+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:03:05.647677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.303402+0000","last_change":"2024-05-11T12:13:03.171792+0000","last_active":"2024-05-11T12:13:04.303402+0000","last_peered":"2024-05-11T12:13:04.303402+0000","last_clean":"2024-05-11T12:13:04.303402+0000","last_became_active":"2024-05-11T12:13:03.171502+0000","last_became_peered":"2024-05-11T12:13:03.171502+0000","last_unstale":"2024-05-11T12:13:04.303402+0000","last_undegraded":"2024-05-11T12:13:04.303402+0000","last_fullsized":"2024-05-11T12:13:04.303402+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:27:27.374412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.303538+0000","last_change":"2024-05-11T12:13:03.165637+0000","last_active":"2024-05-11T12:13:04.303538+0000","last_peered":"2024-05-11T12:13:04.303538+0000","last_clean":"2024-05-11T12:13:04.303538+0000","last_became_active":"2024-05-11T12:13:03.165323+0000","last_became_peered":"2024-05-11T12:13:03.165323+0000","last_unstale":"2024-05-11T12:13:04.303538+0000","last_undegraded":"2024-05-11T12:13:04.303538+0000","last_fullsized":"2024-05-11T12:13:04.303538+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:47:23.761750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.165122+0000","last_change":"2024-05-11T12:13:03.176021+0000","last_active":"2024-05-11T12:13:04.165122+0000","last_peered":"2024-05-11T12:13:04.165122+0000","last_clean":"2024-05-11T12:13:04.165122+0000","last_became_active":"2024-05-11T12:13:03.175708+0000","last_became_peered":"2024-05-11T12:13:03.175708+0000","last_unstale":"2024-05-11T12:13:04.165122+0000","last_undegraded":"2024-05-11T12:13:04.165122+0000","last_fullsized":"2024-05-11T12:13:04.165122+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:12:30.081833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"63'4","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.305916+0000","last_change":"2024-05-11T12:13:02.136461+0000","last_active":"2024-05-11T12:13:04.305916+0000","last_peered":"2024-05-11T12:13:04.305916+0000","last_clean":"2024-05-11T12:13:04.305916+0000","last_became_active":"2024-05-11T12:13:02.136210+0000","last_became_peered":"2024-05-11T12:13:02.136210+0000","last_unstale":"2024-05-11T12:13:04.305916+0000","last_undegraded":"2024-05-11T12:13:04.305916+0000","last_fullsized":"2024-05-11T12:13:04.305916+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:28:33.468562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.304293+0000","last_change":"2024-05-11T12:13:03.158314+0000","last_active":"2024-05-11T12:13:04.304293+0000","last_peered":"2024-05-11T12:13:04.304293+0000","last_clean":"2024-05-11T12:13:04.304293+0000","last_became_active":"2024-05-11T12:13:03.157991+0000","last_became_peered":"2024-05-11T12:13:03.157991+0000","last_unstale":"2024-05-11T12:13:04.304293+0000","last_undegraded":"2024-05-11T12:13:04.304293+0000","last_fullsized":"2024-05-11T12:13:04.304293+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:45:52.844202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"63'1","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.304764+0000","last_change":"2024-05-11T12:13:02.136740+0000","last_active":"2024-05-11T12:13:04.304764+0000","last_peered":"2024-05-11T12:13:04.304764+0000","last_clean":"2024-05-11T12:13:04.304764+0000","last_became_active":"2024-05-11T12:13:02.136446+0000","last_became_peered":"2024-05-11T12:13:02.136446+0000","last_unstale":"2024-05-11T12:13:04.304764+0000","last_undegraded":"2024-05-11T12:13:04.304764+0000","last_fullsized":"2024-05-11T12:13:04.304764+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:06:50.468601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.305141+0000","last_change":"2024-05-11T12:13:03.165220+0000","last_active":"2024-05-11T12:13:04.305141+0000","last_peered":"2024-05-11T12:13:04.305141+0000","last_clean":"2024-05-11T12:13:04.305141+0000","last_became_active":"2024-05-11T12:13:03.164913+0000","last_became_peered":"2024-05-11T12:13:03.164913+0000","last_unstale":"2024-05-11T12:13:04.305141+0000","last_undegraded":"2024-05-11T12:13:04.305141+0000","last_fullsized":"2024-05-11T12:13:04.305141+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:55:45.751432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"63'2","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.304086+0000","last_change":"2024-05-11T12:13:02.129658+0000","last_active":"2024-05-11T12:13:04.304086+0000","last_peered":"2024-05-11T12:13:04.304086+0000","last_clean":"2024-05-11T12:13:04.304086+0000","last_became_active":"2024-05-11T12:13:02.129326+0000","last_became_peered":"2024-05-11T12:13:02.129326+0000","last_unstale":"2024-05-11T12:13:04.304086+0000","last_undegraded":"2024-05-11T12:13:04.304086+0000","last_fullsized":"2024-05-11T12:13:04.304086+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:29:57.084718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.164371+0000","last_change":"2024-05-11T12:13:03.162149+0000","last_active":"2024-05-11T12:13:04.164371+0000","last_peered":"2024-05-11T12:13:04.164371+0000","last_clean":"2024-05-11T12:13:04.164371+0000","last_became_active":"2024-05-11T12:13:03.161890+0000","last_became_peered":"2024-05-11T12:13:03.161890+0000","last_unstale":"2024-05-11T12:13:04.164371+0000","last_undegraded":"2024-05-11T12:13:04.164371+0000","last_fullsized":"2024-05-11T12:13:04.164371+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:04:07.281796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"63'2","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.304484+0000","last_change":"2024-05-11T12:13:02.126473+0000","last_active":"2024-05-11T12:13:04.304484+0000","last_peered":"2024-05-11T12:13:04.304484+0000","last_clean":"2024-05-11T12:13:04.304484+0000","last_became_active":"2024-05-11T12:13:02.126154+0000","last_became_peered":"2024-05-11T12:13:02.126154+0000","last_unstale":"2024-05-11T12:13:04.304484+0000","last_undegraded":"2024-05-11T12:13:04.304484+0000","last_fullsized":"2024-05-11T12:13:04.304484+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:49:40.991051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.173923+0000","last_change":"2024-05-11T12:13:03.164511+0000","last_active":"2024-05-11T12:13:04.173923+0000","last_peered":"2024-05-11T12:13:04.173923+0000","last_clean":"2024-05-11T12:13:04.173923+0000","last_became_active":"2024-05-11T12:13:03.164197+0000","last_became_peered":"2024-05-11T12:13:03.164197+0000","last_unstale":"2024-05-11T12:13:04.173923+0000","last_undegraded":"2024-05-11T12:13:04.173923+0000","last_fullsized":"2024-05-11T12:13:04.173923+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:54:15.999941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"63'1","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.159666+0000","last_change":"2024-05-11T12:13:02.119338+0000","last_active":"2024-05-11T12:13:04.159666+0000","last_peered":"2024-05-11T12:13:04.159666+0000","last_clean":"2024-05-11T12:13:04.159666+0000","last_became_active":"2024-05-11T12:13:02.118127+0000","last_became_peered":"2024-05-11T12:13:02.118127+0000","last_unstale":"2024-05-11T12:13:04.159666+0000","last_undegraded":"2024-05-11T12:13:04.159666+0000","last_fullsized":"2024-05-11T12:13:04.159666+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:07:21.606002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.303747+0000","last_change":"2024-05-11T12:13:03.169929+0000","last_active":"2024-05-11T12:13:04.303747+0000","last_peered":"2024-05-11T12:13:04.303747+0000","last_clean":"2024-05-11T12:13:04.303747+0000","last_became_active":"2024-05-11T12:13:03.158856+0000","last_became_peered":"2024-05-11T12:13:03.158856+0000","last_unstale":"2024-05-11T12:13:04.303747+0000","last_undegraded":"2024-05-11T12:13:04.303747+0000","last_fullsized":"2024-05-11T12:13:04.303747+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:48:53.182110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"63'1","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.159545+0000","last_change":"2024-05-11T12:13:02.121903+0000","last_active":"2024-05-11T12:13:04.159545+0000","last_peered":"2024-05-11T12:13:04.159545+0000","last_clean":"2024-05-11T12:13:04.159545+0000","last_became_active":"2024-05-11T12:13:02.120293+0000","last_became_peered":"2024-05-11T12:13:02.120293+0000","last_unstale":"2024-05-11T12:13:04.159545+0000","last_undegraded":"2024-05-11T12:13:04.159545+0000","last_fullsized":"2024-05-11T12:13:04.159545+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:00:29.098618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.162391+0000","last_change":"2024-05-11T12:13:03.168572+0000","last_active":"2024-05-11T12:13:04.162391+0000","last_peered":"2024-05-11T12:13:04.162391+0000","last_clean":"2024-05-11T12:13:04.162391+0000","last_became_active":"2024-05-11T12:13:03.167854+0000","last_became_peered":"2024-05-11T12:13:03.167854+0000","last_unstale":"2024-05-11T12:13:04.162391+0000","last_undegraded":"2024-05-11T12:13:04.162391+0000","last_fullsized":"2024-05-11T12:13:04.162391+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:05:06.248364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.159264+0000","last_change":"2024-05-11T12:13:02.139509+0000","last_active":"2024-05-11T12:13:04.159264+0000","last_peered":"2024-05-11T12:13:04.159264+0000","last_clean":"2024-05-11T12:13:04.159264+0000","last_became_active":"2024-05-11T12:13:02.139201+0000","last_became_peered":"2024-05-11T12:13:02.139201+0000","last_unstale":"2024-05-11T12:13:04.159264+0000","last_undegraded":"2024-05-11T12:13:04.159264+0000","last_fullsized":"2024-05-11T12:13:04.159264+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:49:40.991051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.159592+0000","last_change":"2024-05-11T12:13:03.163664+0000","last_active":"2024-05-11T12:13:04.159592+0000","last_peered":"2024-05-11T12:13:04.159592+0000","last_clean":"2024-05-11T12:13:04.159592+0000","last_became_active":"2024-05-11T12:13:03.163297+0000","last_became_peered":"2024-05-11T12:13:03.163297+0000","last_unstale":"2024-05-11T12:13:04.159592+0000","last_undegraded":"2024-05-11T12:13:04.159592+0000","last_fullsized":"2024-05-11T12:13:04.159592+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:21:48.140920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.162083+0000","last_change":"2024-05-11T12:13:02.139692+0000","last_active":"2024-05-11T12:13:04.162083+0000","last_peered":"2024-05-11T12:13:04.162083+0000","last_clean":"2024-05-11T12:13:04.162083+0000","last_became_active":"2024-05-11T12:13:02.137935+0000","last_became_peered":"2024-05-11T12:13:02.137935+0000","last_unstale":"2024-05-11T12:13:04.162083+0000","last_undegraded":"2024-05-11T12:13:04.162083+0000","last_fullsized":"2024-05-11T12:13:04.162083+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:15:34.846112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.305380+0000","last_change":"2024-05-11T12:13:03.171027+0000","last_active":"2024-05-11T12:13:04.305380+0000","last_peered":"2024-05-11T12:13:04.305380+0000","last_clean":"2024-05-11T12:13:04.305380+0000","last_became_active":"2024-05-11T12:13:03.170778+0000","last_became_peered":"2024-05-11T12:13:03.170778+0000","last_unstale":"2024-05-11T12:13:04.305380+0000","last_undegraded":"2024-05-11T12:13:04.305380+0000","last_fullsized":"2024-05-11T12:13:04.305380+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:12:30.081833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.165207+0000","last_change":"2024-05-11T12:13:02.142438+0000","last_active":"2024-05-11T12:13:04.165207+0000","last_peered":"2024-05-11T12:13:04.165207+0000","last_clean":"2024-05-11T12:13:04.165207+0000","last_became_active":"2024-05-11T12:13:02.142125+0000","last_became_peered":"2024-05-11T12:13:02.142125+0000","last_unstale":"2024-05-11T12:13:04.165207+0000","last_undegraded":"2024-05-11T12:13:04.165207+0000","last_fullsized":"2024-05-11T12:13:04.165207+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:55:32.333055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.159784+0000","last_change":"2024-05-11T12:13:03.149501+0000","last_active":"2024-05-11T12:13:04.159784+0000","last_peered":"2024-05-11T12:13:04.159784+0000","last_clean":"2024-05-11T12:13:04.159784+0000","last_became_active":"2024-05-11T12:13:03.149088+0000","last_became_peered":"2024-05-11T12:13:03.149088+0000","last_unstale":"2024-05-11T12:13:04.159784+0000","last_undegraded":"2024-05-11T12:13:04.159784+0000","last_fullsized":"2024-05-11T12:13:04.159784+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:03:01.682416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.304526+0000","last_change":"2024-05-11T12:13:02.140895+0000","last_active":"2024-05-11T12:13:04.304526+0000","last_peered":"2024-05-11T12:13:04.304526+0000","last_clean":"2024-05-11T12:13:04.304526+0000","last_became_active":"2024-05-11T12:13:02.138733+0000","last_became_peered":"2024-05-11T12:13:02.138733+0000","last_unstale":"2024-05-11T12:13:04.304526+0000","last_undegraded":"2024-05-11T12:13:04.304526+0000","last_fullsized":"2024-05-11T12:13:04.304526+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:08:47.140652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.165055+0000","last_change":"2024-05-11T12:13:03.160320+0000","last_active":"2024-05-11T12:13:04.165055+0000","last_peered":"2024-05-11T12:13:04.165055+0000","last_clean":"2024-05-11T12:13:04.165055+0000","last_became_active":"2024-05-11T12:13:03.159984+0000","last_became_peered":"2024-05-11T12:13:03.159984+0000","last_unstale":"2024-05-11T12:13:04.165055+0000","last_undegraded":"2024-05-11T12:13:04.165055+0000","last_fullsized":"2024-05-11T12:13:04.165055+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:15:29.709641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"63'3","reported_seq":17,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.162861+0000","last_change":"2024-05-11T12:13:02.139752+0000","last_active":"2024-05-11T12:13:04.162861+0000","last_peered":"2024-05-11T12:13:04.162861+0000","last_clean":"2024-05-11T12:13:04.162861+0000","last_became_active":"2024-05-11T12:13:02.139431+0000","last_became_peered":"2024-05-11T12:13:02.139431+0000","last_unstale":"2024-05-11T12:13:04.162861+0000","last_undegraded":"2024-05-11T12:13:04.162861+0000","last_fullsized":"2024-05-11T12:13:04.162861+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:00:08.081754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.304594+0000","last_change":"2024-05-11T12:13:03.166818+0000","last_active":"2024-05-11T12:13:04.304594+0000","last_peered":"2024-05-11T12:13:04.304594+0000","last_clean":"2024-05-11T12:13:04.304594+0000","last_became_active":"2024-05-11T12:13:03.166560+0000","last_became_peered":"2024-05-11T12:13:03.166560+0000","last_unstale":"2024-05-11T12:13:04.304594+0000","last_undegraded":"2024-05-11T12:13:04.304594+0000","last_fullsized":"2024-05-11T12:13:04.304594+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:57:14.643928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"63'6","reported_seq":20,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.305493+0000","last_change":"2024-05-11T12:13:02.133552+0000","last_active":"2024-05-11T12:13:04.305493+0000","last_peered":"2024-05-11T12:13:04.305493+0000","last_clean":"2024-05-11T12:13:04.305493+0000","last_became_active":"2024-05-11T12:13:02.133236+0000","last_became_peered":"2024-05-11T12:13:02.133236+0000","last_unstale":"2024-05-11T12:13:04.305493+0000","last_undegraded":"2024-05-11T12:13:04.305493+0000","last_fullsized":"2024-05-11T12:13:04.305493+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:17:18.046920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.303328+0000","last_change":"2024-05-11T12:13:03.168373+0000","last_active":"2024-05-11T12:13:04.303328+0000","last_peered":"2024-05-11T12:13:04.303328+0000","last_clean":"2024-05-11T12:13:04.303328+0000","last_became_active":"2024-05-11T12:13:03.168105+0000","last_became_peered":"2024-05-11T12:13:03.168105+0000","last_unstale":"2024-05-11T12:13:04.303328+0000","last_undegraded":"2024-05-11T12:13:04.303328+0000","last_fullsized":"2024-05-11T12:13:04.303328+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:56:18.568549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"63'2","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.164670+0000","last_change":"2024-05-11T12:13:02.143192+0000","last_active":"2024-05-11T12:13:04.164670+0000","last_peered":"2024-05-11T12:13:04.164670+0000","last_clean":"2024-05-11T12:13:04.164670+0000","last_became_active":"2024-05-11T12:13:02.142901+0000","last_became_peered":"2024-05-11T12:13:02.142901+0000","last_unstale":"2024-05-11T12:13:04.164670+0000","last_undegraded":"2024-05-11T12:13:04.164670+0000","last_fullsized":"2024-05-11T12:13:04.164670+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:39:03.473134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.168159+0000","last_change":"2024-05-11T12:13:03.145731+0000","last_active":"2024-05-11T12:13:04.168159+0000","last_peered":"2024-05-11T12:13:04.168159+0000","last_clean":"2024-05-11T12:13:04.168159+0000","last_became_active":"2024-05-11T12:13:03.145419+0000","last_became_peered":"2024-05-11T12:13:03.145419+0000","last_unstale":"2024-05-11T12:13:04.168159+0000","last_undegraded":"2024-05-11T12:13:04.168159+0000","last_fullsized":"2024-05-11T12:13:04.168159+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:49:41.995979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.167989+0000","last_change":"2024-05-11T12:13:02.118817+0000","last_active":"2024-05-11T12:13:04.167989+0000","last_peered":"2024-05-11T12:13:04.167989+0000","last_clean":"2024-05-11T12:13:04.167989+0000","last_became_active":"2024-05-11T12:13:02.118040+0000","last_became_peered":"2024-05-11T12:13:02.118040+0000","last_unstale":"2024-05-11T12:13:04.167989+0000","last_undegraded":"2024-05-11T12:13:04.167989+0000","last_fullsized":"2024-05-11T12:13:04.167989+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:18:34.044830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.162431+0000","last_change":"2024-05-11T12:13:03.172604+0000","last_active":"2024-05-11T12:13:04.162431+0000","last_peered":"2024-05-11T12:13:04.162431+0000","last_clean":"2024-05-11T12:13:04.162431+0000","last_became_active":"2024-05-11T12:13:03.172198+0000","last_became_peered":"2024-05-11T12:13:03.172198+0000","last_unstale":"2024-05-11T12:13:04.162431+0000","last_undegraded":"2024-05-11T12:13:04.162431+0000","last_fullsized":"2024-05-11T12:13:04.162431+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:57:14.643928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.162024+0000","last_change":"2024-05-11T12:13:02.138604+0000","last_active":"2024-05-11T12:13:04.162024+0000","last_peered":"2024-05-11T12:13:04.162024+0000","last_clean":"2024-05-11T12:13:04.162024+0000","last_became_active":"2024-05-11T12:13:02.138227+0000","last_became_peered":"2024-05-11T12:13:02.138227+0000","last_unstale":"2024-05-11T12:13:04.162024+0000","last_undegraded":"2024-05-11T12:13:04.162024+0000","last_fullsized":"2024-05-11T12:13:04.162024+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:15:28.704713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.304895+0000","last_change":"2024-05-11T12:13:03.169594+0000","last_active":"2024-05-11T12:13:04.304895+0000","last_peered":"2024-05-11T12:13:04.304895+0000","last_clean":"2024-05-11T12:13:04.304895+0000","last_became_active":"2024-05-11T12:13:03.160512+0000","last_became_peered":"2024-05-11T12:13:03.160512+0000","last_unstale":"2024-05-11T12:13:04.304895+0000","last_undegraded":"2024-05-11T12:13:04.304895+0000","last_fullsized":"2024-05-11T12:13:04.304895+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:55:33.337983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"63'8","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.303100+0000","last_change":"2024-05-11T12:13:02.137326+0000","last_active":"2024-05-11T12:13:04.303100+0000","last_peered":"2024-05-11T12:13:04.303100+0000","last_clean":"2024-05-11T12:13:04.303100+0000","last_became_active":"2024-05-11T12:13:02.133763+0000","last_became_peered":"2024-05-11T12:13:02.133763+0000","last_unstale":"2024-05-11T12:13:04.303100+0000","last_undegraded":"2024-05-11T12:13:04.303100+0000","last_fullsized":"2024-05-11T12:13:04.303100+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:55:40.188834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.163325+0000","last_change":"2024-05-11T12:13:03.163731+0000","last_active":"2024-05-11T12:13:04.163325+0000","last_peered":"2024-05-11T12:13:04.163325+0000","last_clean":"2024-05-11T12:13:04.163325+0000","last_became_active":"2024-05-11T12:13:03.161280+0000","last_became_peered":"2024-05-11T12:13:03.161280+0000","last_unstale":"2024-05-11T12:13:04.163325+0000","last_undegraded":"2024-05-11T12:13:04.163325+0000","last_fullsized":"2024-05-11T12:13:04.163325+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:49:41.995979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"63'1","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.170916+0000","last_change":"2024-05-11T12:13:02.124463+0000","last_active":"2024-05-11T12:13:04.170916+0000","last_peered":"2024-05-11T12:13:04.170916+0000","last_clean":"2024-05-11T12:13:04.170916+0000","last_became_active":"2024-05-11T12:13:02.124136+0000","last_became_peered":"2024-05-11T12:13:02.124136+0000","last_unstale":"2024-05-11T12:13:04.170916+0000","last_undegraded":"2024-05-11T12:13:04.170916+0000","last_fullsized":"2024-05-11T12:13:04.170916+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:48:52.177182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.165347+0000","last_change":"2024-05-11T12:13:03.158635+0000","last_active":"2024-05-11T12:13:04.165347+0000","last_peered":"2024-05-11T12:13:04.165347+0000","last_clean":"2024-05-11T12:13:04.165347+0000","last_became_active":"2024-05-11T12:13:03.158311+0000","last_became_peered":"2024-05-11T12:13:03.158311+0000","last_unstale":"2024-05-11T12:13:04.165347+0000","last_undegraded":"2024-05-11T12:13:04.165347+0000","last_fullsized":"2024-05-11T12:13:04.165347+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:42:21.959847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"63'2","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.165258+0000","last_change":"2024-05-11T12:13:02.143315+0000","last_active":"2024-05-11T12:13:04.165258+0000","last_peered":"2024-05-11T12:13:04.165258+0000","last_clean":"2024-05-11T12:13:04.165258+0000","last_became_active":"2024-05-11T12:13:02.143012+0000","last_became_peered":"2024-05-11T12:13:02.143012+0000","last_unstale":"2024-05-11T12:13:04.165258+0000","last_undegraded":"2024-05-11T12:13:04.165258+0000","last_fullsized":"2024-05-11T12:13:04.165258+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:03:26.831522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.304307+0000","last_change":"2024-05-11T12:13:03.159923+0000","last_active":"2024-05-11T12:13:04.304307+0000","last_peered":"2024-05-11T12:13:04.304307+0000","last_clean":"2024-05-11T12:13:04.304307+0000","last_became_active":"2024-05-11T12:13:03.159119+0000","last_became_peered":"2024-05-11T12:13:03.159119+0000","last_unstale":"2024-05-11T12:13:04.304307+0000","last_undegraded":"2024-05-11T12:13:04.304307+0000","last_fullsized":"2024-05-11T12:13:04.304307+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:05:06.248364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"63'1","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.163165+0000","last_change":"2024-05-11T12:13:02.138872+0000","last_active":"2024-05-11T12:13:04.163165+0000","last_peered":"2024-05-11T12:13:04.163165+0000","last_clean":"2024-05-11T12:13:04.163165+0000","last_became_active":"2024-05-11T12:13:02.138589+0000","last_became_peered":"2024-05-11T12:13:02.138589+0000","last_unstale":"2024-05-11T12:13:04.163165+0000","last_undegraded":"2024-05-11T12:13:04.163165+0000","last_fullsized":"2024-05-11T12:13:04.163165+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:48:52.177182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.163764+0000","last_change":"2024-05-11T12:13:03.167395+0000","last_active":"2024-05-11T12:13:04.163764+0000","last_peered":"2024-05-11T12:13:04.163764+0000","last_clean":"2024-05-11T12:13:04.163764+0000","last_became_active":"2024-05-11T12:13:03.167111+0000","last_became_peered":"2024-05-11T12:13:03.167111+0000","last_unstale":"2024-05-11T12:13:04.163764+0000","last_undegraded":"2024-05-11T12:13:04.163764+0000","last_fullsized":"2024-05-11T12:13:04.163764+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:48:53.182110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.159642+0000","last_change":"2024-05-11T12:13:02.137581+0000","last_active":"2024-05-11T12:13:04.159642+0000","last_peered":"2024-05-11T12:13:04.159642+0000","last_clean":"2024-05-11T12:13:04.159642+0000","last_became_active":"2024-05-11T12:13:02.137260+0000","last_became_peered":"2024-05-11T12:13:02.137260+0000","last_unstale":"2024-05-11T12:13:04.159642+0000","last_undegraded":"2024-05-11T12:13:04.159642+0000","last_fullsized":"2024-05-11T12:13:04.159642+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:06:17.754393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.303599+0000","last_change":"2024-05-11T12:13:03.165305+0000","last_active":"2024-05-11T12:13:04.303599+0000","last_peered":"2024-05-11T12:13:04.303599+0000","last_clean":"2024-05-11T12:13:04.303599+0000","last_became_active":"2024-05-11T12:13:03.164856+0000","last_became_peered":"2024-05-11T12:13:03.164856+0000","last_unstale":"2024-05-11T12:13:04.303599+0000","last_undegraded":"2024-05-11T12:13:04.303599+0000","last_fullsized":"2024-05-11T12:13:04.303599+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:22:54.995824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"63'2","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.303439+0000","last_change":"2024-05-11T12:13:02.140367+0000","last_active":"2024-05-11T12:13:04.303439+0000","last_peered":"2024-05-11T12:13:04.303439+0000","last_clean":"2024-05-11T12:13:04.303439+0000","last_became_active":"2024-05-11T12:13:02.140069+0000","last_became_peered":"2024-05-11T12:13:02.140069+0000","last_unstale":"2024-05-11T12:13:04.303439+0000","last_undegraded":"2024-05-11T12:13:04.303439+0000","last_fullsized":"2024-05-11T12:13:04.303439+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:27:13.189890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.168198+0000","last_change":"2024-05-11T12:13:03.171936+0000","last_active":"2024-05-11T12:13:04.168198+0000","last_peered":"2024-05-11T12:13:04.168198+0000","last_clean":"2024-05-11T12:13:04.168198+0000","last_became_active":"2024-05-11T12:13:03.171669+0000","last_became_peered":"2024-05-11T12:13:03.171669+0000","last_unstale":"2024-05-11T12:13:04.168198+0000","last_undegraded":"2024-05-11T12:13:04.168198+0000","last_fullsized":"2024-05-11T12:13:04.168198+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:00:30.103546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.168289+0000","last_change":"2024-05-11T12:13:02.125827+0000","last_active":"2024-05-11T12:13:04.168289+0000","last_peered":"2024-05-11T12:13:04.168289+0000","last_clean":"2024-05-11T12:13:04.168289+0000","last_became_active":"2024-05-11T12:13:02.125520+0000","last_became_peered":"2024-05-11T12:13:02.125520+0000","last_unstale":"2024-05-11T12:13:04.168289+0000","last_undegraded":"2024-05-11T12:13:04.168289+0000","last_fullsized":"2024-05-11T12:13:04.168289+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:00:08.081754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.162832+0000","last_change":"2024-05-11T12:13:03.168791+0000","last_active":"2024-05-11T12:13:04.162832+0000","last_peered":"2024-05-11T12:13:04.162832+0000","last_clean":"2024-05-11T12:13:04.162832+0000","last_became_active":"2024-05-11T12:13:03.168255+0000","last_became_peered":"2024-05-11T12:13:03.168255+0000","last_unstale":"2024-05-11T12:13:04.162832+0000","last_undegraded":"2024-05-11T12:13:04.162832+0000","last_fullsized":"2024-05-11T12:13:04.162832+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:39:04.478062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.303306+0000","last_change":"2024-05-11T12:13:02.136770+0000","last_active":"2024-05-11T12:13:04.303306+0000","last_peered":"2024-05-11T12:13:04.303306+0000","last_clean":"2024-05-11T12:13:04.303306+0000","last_became_active":"2024-05-11T12:13:02.136465+0000","last_became_peered":"2024-05-11T12:13:02.136465+0000","last_unstale":"2024-05-11T12:13:04.303306+0000","last_undegraded":"2024-05-11T12:13:04.303306+0000","last_fullsized":"2024-05-11T12:13:04.303306+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:04:04.992779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.304648+0000","last_change":"2024-05-11T12:13:03.160074+0000","last_active":"2024-05-11T12:13:04.304648+0000","last_peered":"2024-05-11T12:13:04.304648+0000","last_clean":"2024-05-11T12:13:04.304648+0000","last_became_active":"2024-05-11T12:13:03.158195+0000","last_became_peered":"2024-05-11T12:13:03.158195+0000","last_unstale":"2024-05-11T12:13:04.304648+0000","last_undegraded":"2024-05-11T12:13:04.304648+0000","last_fullsized":"2024-05-11T12:13:04.304648+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:39:18.725256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"63'2","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.171210+0000","last_change":"2024-05-11T12:13:02.136984+0000","last_active":"2024-05-11T12:13:04.171210+0000","last_peered":"2024-05-11T12:13:04.171210+0000","last_clean":"2024-05-11T12:13:04.171210+0000","last_became_active":"2024-05-11T12:13:02.135837+0000","last_became_peered":"2024-05-11T12:13:02.135837+0000","last_unstale":"2024-05-11T12:13:04.171210+0000","last_undegraded":"2024-05-11T12:13:04.171210+0000","last_fullsized":"2024-05-11T12:13:04.171210+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:03:26.831522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.173813+0000","last_change":"2024-05-11T12:13:03.170696+0000","last_active":"2024-05-11T12:13:04.173813+0000","last_peered":"2024-05-11T12:13:04.173813+0000","last_clean":"2024-05-11T12:13:04.173813+0000","last_became_active":"2024-05-11T12:13:03.170411+0000","last_became_peered":"2024-05-11T12:13:03.170411+0000","last_unstale":"2024-05-11T12:13:04.173813+0000","last_undegraded":"2024-05-11T12:13:04.173813+0000","last_fullsized":"2024-05-11T12:13:04.173813+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:11:05.631699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"64'1","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.169706+0000","last_change":"2024-05-11T12:13:02.128082+0000","last_active":"2024-05-11T12:13:04.169706+0000","last_peered":"2024-05-11T12:13:04.169706+0000","last_clean":"2024-05-11T12:13:04.169706+0000","last_became_active":"2024-05-11T12:13:02.126292+0000","last_became_peered":"2024-05-11T12:13:02.126292+0000","last_unstale":"2024-05-11T12:13:04.169706+0000","last_undegraded":"2024-05-11T12:13:04.169706+0000","last_fullsized":"2024-05-11T12:13:04.169706+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:15:28.704713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.169393+0000","last_change":"2024-05-11T12:13:03.149098+0000","last_active":"2024-05-11T12:13:04.169393+0000","last_peered":"2024-05-11T12:13:04.169393+0000","last_clean":"2024-05-11T12:13:04.169393+0000","last_became_active":"2024-05-11T12:13:03.148358+0000","last_became_peered":"2024-05-11T12:13:03.148358+0000","last_unstale":"2024-05-11T12:13:04.169393+0000","last_undegraded":"2024-05-11T12:13:04.169393+0000","last_fullsized":"2024-05-11T12:13:04.169393+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:22:54.995824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.304200+0000","last_change":"2024-05-11T12:13:02.129624+0000","last_active":"2024-05-11T12:13:04.304200+0000","last_peered":"2024-05-11T12:13:04.304200+0000","last_clean":"2024-05-11T12:13:04.304200+0000","last_became_active":"2024-05-11T12:13:02.129286+0000","last_became_peered":"2024-05-11T12:13:02.129286+0000","last_unstale":"2024-05-11T12:13:04.304200+0000","last_undegraded":"2024-05-11T12:13:04.304200+0000","last_fullsized":"2024-05-11T12:13:04.304200+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:55:25.067740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.164166+0000","last_change":"2024-05-11T12:13:03.170501+0000","last_active":"2024-05-11T12:13:04.164166+0000","last_peered":"2024-05-11T12:13:04.164166+0000","last_clean":"2024-05-11T12:13:04.164166+0000","last_became_active":"2024-05-11T12:13:03.163918+0000","last_became_peered":"2024-05-11T12:13:03.163918+0000","last_unstale":"2024-05-11T12:13:04.164166+0000","last_undegraded":"2024-05-11T12:13:04.164166+0000","last_fullsized":"2024-05-11T12:13:04.164166+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:03:27.836450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.305473+0000","last_change":"2024-05-11T12:13:02.133532+0000","last_active":"2024-05-11T12:13:04.305473+0000","last_peered":"2024-05-11T12:13:04.305473+0000","last_clean":"2024-05-11T12:13:04.305473+0000","last_became_active":"2024-05-11T12:13:02.133209+0000","last_became_peered":"2024-05-11T12:13:02.133209+0000","last_unstale":"2024-05-11T12:13:04.305473+0000","last_undegraded":"2024-05-11T12:13:04.305473+0000","last_fullsized":"2024-05-11T12:13:04.305473+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:56:17.563622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.161666+0000","last_change":"2024-05-11T12:13:03.163919+0000","last_active":"2024-05-11T12:13:04.161666+0000","last_peered":"2024-05-11T12:13:04.161666+0000","last_clean":"2024-05-11T12:13:04.161666+0000","last_became_active":"2024-05-11T12:13:03.163595+0000","last_became_peered":"2024-05-11T12:13:03.163595+0000","last_unstale":"2024-05-11T12:13:04.161666+0000","last_undegraded":"2024-05-11T12:13:04.161666+0000","last_fullsized":"2024-05-11T12:13:04.161666+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:45:52.844202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.164948+0000","last_change":"2024-05-11T12:13:02.139859+0000","last_active":"2024-05-11T12:13:04.164948+0000","last_peered":"2024-05-11T12:13:04.164948+0000","last_clean":"2024-05-11T12:13:04.164948+0000","last_became_active":"2024-05-11T12:13:02.139265+0000","last_became_peered":"2024-05-11T12:13:02.139265+0000","last_unstale":"2024-05-11T12:13:04.164948+0000","last_undegraded":"2024-05-11T12:13:04.164948+0000","last_fullsized":"2024-05-11T12:13:04.164948+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:18:34.044830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.303608+0000","last_change":"2024-05-11T12:13:03.163034+0000","last_active":"2024-05-11T12:13:04.303608+0000","last_peered":"2024-05-11T12:13:04.303608+0000","last_clean":"2024-05-11T12:13:04.303608+0000","last_became_active":"2024-05-11T12:13:03.160775+0000","last_became_peered":"2024-05-11T12:13:03.160775+0000","last_unstale":"2024-05-11T12:13:04.303608+0000","last_undegraded":"2024-05-11T12:13:04.303608+0000","last_fullsized":"2024-05-11T12:13:04.303608+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:17:19.051848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.159680+0000","last_change":"2024-05-11T12:13:02.113452+0000","last_active":"2024-05-11T12:13:04.159680+0000","last_peered":"2024-05-11T12:13:04.159680+0000","last_clean":"2024-05-11T12:13:04.159680+0000","last_became_active":"2024-05-11T12:13:02.112863+0000","last_became_peered":"2024-05-11T12:13:02.112863+0000","last_unstale":"2024-05-11T12:13:04.159680+0000","last_undegraded":"2024-05-11T12:13:04.159680+0000","last_fullsized":"2024-05-11T12:13:04.159680+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:04:06.276868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.304177+0000","last_change":"2024-05-11T12:13:03.162662+0000","last_active":"2024-05-11T12:13:04.304177+0000","last_peered":"2024-05-11T12:13:04.304177+0000","last_clean":"2024-05-11T12:13:04.304177+0000","last_became_active":"2024-05-11T12:13:03.162424+0000","last_became_peered":"2024-05-11T12:13:03.162424+0000","last_unstale":"2024-05-11T12:13:04.304177+0000","last_undegraded":"2024-05-11T12:13:04.304177+0000","last_fullsized":"2024-05-11T12:13:04.304177+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:03:27.836450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.159337+0000","last_change":"2024-05-11T12:13:02.133595+0000","last_active":"2024-05-11T12:13:04.159337+0000","last_peered":"2024-05-11T12:13:04.159337+0000","last_clean":"2024-05-11T12:13:04.159337+0000","last_became_active":"2024-05-11T12:13:02.133168+0000","last_became_peered":"2024-05-11T12:13:02.133168+0000","last_unstale":"2024-05-11T12:13:04.159337+0000","last_undegraded":"2024-05-11T12:13:04.159337+0000","last_fullsized":"2024-05-11T12:13:04.159337+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:03:04.642749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.173843+0000","last_change":"2024-05-11T12:13:03.169252+0000","last_active":"2024-05-11T12:13:04.173843+0000","last_peered":"2024-05-11T12:13:04.173843+0000","last_clean":"2024-05-11T12:13:04.173843+0000","last_became_active":"2024-05-11T12:13:03.167540+0000","last_became_peered":"2024-05-11T12:13:03.167540+0000","last_unstale":"2024-05-11T12:13:04.173843+0000","last_undegraded":"2024-05-11T12:13:04.173843+0000","last_fullsized":"2024-05-11T12:13:04.173843+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:38:22.482431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.304329+0000","last_change":"2024-05-11T12:13:02.140835+0000","last_active":"2024-05-11T12:13:04.304329+0000","last_peered":"2024-05-11T12:13:04.304329+0000","last_clean":"2024-05-11T12:13:04.304329+0000","last_became_active":"2024-05-11T12:13:02.139971+0000","last_became_peered":"2024-05-11T12:13:02.139971+0000","last_unstale":"2024-05-11T12:13:04.304329+0000","last_undegraded":"2024-05-11T12:13:04.304329+0000","last_fullsized":"2024-05-11T12:13:04.304329+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:18:49.739422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.163704+0000","last_change":"2024-05-11T12:13:03.166247+0000","last_active":"2024-05-11T12:13:04.163704+0000","last_peered":"2024-05-11T12:13:04.163704+0000","last_clean":"2024-05-11T12:13:04.163704+0000","last_became_active":"2024-05-11T12:13:03.165118+0000","last_became_peered":"2024-05-11T12:13:03.165118+0000","last_unstale":"2024-05-11T12:13:04.163704+0000","last_undegraded":"2024-05-11T12:13:04.163704+0000","last_fullsized":"2024-05-11T12:13:04.163704+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:02.088574+0000","last_clean_scrub_stamp":"2024-05-11T12:13:02.088574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:00:09.086682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.159150+0000","last_change":"2024-05-11T12:13:02.116771+0000","last_active":"2024-05-11T12:13:04.159150+0000","last_peered":"2024-05-11T12:13:04.159150+0000","last_clean":"2024-05-11T12:13:04.159150+0000","last_became_active":"2024-05-11T12:13:02.116028+0000","last_became_peered":"2024-05-11T12:13:02.116028+0000","last_unstale":"2024-05-11T12:13:04.159150+0000","last_undegraded":"2024-05-11T12:13:04.159150+0000","last_fullsized":"2024-05-11T12:13:04.159150+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:01.083647+0000","last_clean_scrub_stamp":"2024-05-11T12:13:01.083647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:06:50.468601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":185,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-11T12:13:04.165359+0000","last_change":"2024-05-11T12:10:35.066673+0000","last_active":"2024-05-11T12:13:04.165359+0000","last_peered":"2024-05-11T12:13:04.165359+0000","last_clean":"2024-05-11T12:13:04.165359+0000","last_became_active":"2024-05-11T12:10:35.065942+0000","last_became_peered":"2024-05-11T12:10:35.065942+0000","last_unstale":"2024-05-11T12:13:04.165359+0000","last_undegraded":"2024-05-11T12:13:04.165359+0000","last_fullsized":"2024-05-11T12:13:04.165359+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_clean_scrub_stamp":"2024-05-11T12:10:33.601136+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:12:02.133182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476770,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28368,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343472,"statfs":{"total":96636764160,"available":96607715328,"internally_reserved":0,"allocated":1372160,"data_stored":930804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476771,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27776,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344064,"statfs":{"total":96636764160,"available":96608321536,"internally_reserved":0,"allocated":765952,"data_stored":339694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476770,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":765952,"data_stored":339682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476772,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27776,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344064,"statfs":{"total":96636764160,"available":96608321536,"internally_reserved":0,"allocated":774144,"data_stored":341272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509475,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31924,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339916,"statfs":{"total":96636764160,"available":96604073984,"internally_reserved":0,"allocated":761856,"data_stored":339660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509476,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":790528,"data_stored":342036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509475,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32512,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339328,"statfs":{"total":96636764160,"available":96603471872,"internally_reserved":0,"allocated":1355776,"data_stored":930028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542179,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":761856,"data_stored":339660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T12:13:13.108 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-11T12:13:13.109 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-11T12:13:13.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.361+0000 7fbcd6800640 1 Processor -- start 2024-05-11T12:13:13.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.361+0000 7fbcd6800640 1 -- start start 2024-05-11T12:13:13.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.362+0000 7fbcd6800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fbcd00b3fb0 0x7fbcd00b84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:13.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.362+0000 7fbcd6800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbcd00d3fb0 0x7fbcd006cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:13.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.363+0000 7fbcd6800640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcd006dec0 con 0x7fbcd0162a40 2024-05-11T12:13:13.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.363+0000 7fbcd6800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fbcd00b89f0 con 0x7fbcd00b3fb0 2024-05-11T12:13:13.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.363+0000 7fbcd6800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbcd00d2b70 con 0x7fbcd00d3fb0 2024-05-11T12:13:13.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.363+0000 7fbcd5400640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fbcd0162a40 0x7fbcd00d38a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:48720/0 (socket says 172.21.15.167:48720) 2024-05-11T12:13:13.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.363+0000 7fbcd5400640 1 -- 172.21.15.167:0/2888296209 learned_addr learned my addr 172.21.15.167:0/2888296209 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:13:13.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.363+0000 7fbcd4a00640 1 --2- 172.21.15.167:0/2888296209 >> v2:172.21.15.204:3300/0 conn(0x7fbcd00b3fb0 0x7fbcd00b84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:13.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.363+0000 7fbccfe00640 1 -- 172.21.15.167:0/2888296209 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2356943918 0 0) 0x7fbcd006dec0 con 0x7fbcd0162a40 2024-05-11T12:13:13.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.363+0000 7fbcd5e00640 1 --2- 172.21.15.167:0/2888296209 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbcd00d3fb0 0x7fbcd006cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:13.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.363+0000 7fbccfe00640 1 -- 172.21.15.167:0/2888296209 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbcb4003520 con 0x7fbcd0162a40 2024-05-11T12:13:13.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.364+0000 7fbccfe00640 1 -- 172.21.15.167:0/2888296209 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1679477295 0 0) 0x7fbcb4003520 con 0x7fbcd0162a40 2024-05-11T12:13:13.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.364+0000 7fbccfe00640 1 -- 172.21.15.167:0/2888296209 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbcd00d3fb0 msgr2=0x7fbcd006cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:13.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.364+0000 7fbccfe00640 1 --2- 172.21.15.167:0/2888296209 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbcd00d3fb0 0x7fbcd006cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:13.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.364+0000 7fbccfe00640 1 -- 172.21.15.167:0/2888296209 >> v2:172.21.15.204:3300/0 conn(0x7fbcd00b3fb0 msgr2=0x7fbcd00b84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:13.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.364+0000 7fbccfe00640 1 --2- 172.21.15.167:0/2888296209 >> v2:172.21.15.204:3300/0 conn(0x7fbcd00b3fb0 0x7fbcd00b84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:13.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.364+0000 7fbccfe00640 1 -- 172.21.15.167:0/2888296209 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcd00d2dc0 con 0x7fbcd0162a40 2024-05-11T12:13:13.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.364+0000 7fbccfe00640 1 -- 172.21.15.167:0/2888296209 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7fbcc00016f0 con 0x7fbcd0162a40 2024-05-11T12:13:13.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.364+0000 7fbccfe00640 1 -- 172.21.15.167:0/2888296209 >> v1:172.21.15.167:6789/0 conn(0x7fbcd0162a40 legacy=0x7fbcd00d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:13.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.364+0000 7fbccfe00640 1 --2- 172.21.15.167:0/2888296209 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbcb4003a50 0x7fbcb4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:13.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.364+0000 7fbccfe00640 1 --2- 172.21.15.167:0/2888296209 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbcd00d3fb0 0x7fbcb4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:13.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.364+0000 7fbccfe00640 1 --2- 172.21.15.167:0/2888296209 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcb4004d40 0x7fbcb40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:13.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.364+0000 7fbccfe00640 1 -- 172.21.15.167:0/2888296209 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd00d2b70 con 0x7fbcb4004d40 2024-05-11T12:13:13.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.364+0000 7fbccfe00640 1 -- 172.21.15.167:0/2888296209 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb400b9c0 con 0x7fbcb4003a50 2024-05-11T12:13:13.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.364+0000 7fbccfe00640 1 -- 172.21.15.167:0/2888296209 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbcb400bbc0 con 0x7fbcd00d3fb0 2024-05-11T12:13:13.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.364+0000 7fbcd5e00640 1 --2- 172.21.15.167:0/2888296209 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcb4004d40 0x7fbcb40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:13.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.364+0000 7fbcd5400640 1 --2- 172.21.15.167:0/2888296209 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbcb4003a50 0x7fbcb4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:13.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.365+0000 7fbcd4a00640 1 --2- 172.21.15.167:0/2888296209 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbcd00d3fb0 0x7fbcb4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:13.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.365+0000 7fbcd5e00640 1 -- 172.21.15.167:0/2888296209 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbcd00d3fb0 msgr2=0x7fbcb4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:13.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.365+0000 7fbcd5e00640 1 --2- 172.21.15.167:0/2888296209 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbcd00d3fb0 0x7fbcb4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:13.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.365+0000 7fbcd5e00640 1 -- 172.21.15.167:0/2888296209 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbcb4003a50 msgr2=0x7fbcb4003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:13.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.365+0000 7fbcd5e00640 1 --2- 172.21.15.167:0/2888296209 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbcb4003a50 0x7fbcb4003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:13.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.365+0000 7fbcd5e00640 1 -- 172.21.15.167:0/2888296209 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcb4003520 con 0x7fbcb4004d40 2024-05-11T12:13:13.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.365+0000 7fbcd4a00640 1 --2- 172.21.15.167:0/2888296209 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbcd00d3fb0 0x7fbcb4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:13.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.366+0000 7fbcd5e00640 1 --2- 172.21.15.167:0/2888296209 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcb4004d40 0x7fbcb40051c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8009fc0 tx=0x7fbcb4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=232e703aeedd56b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:13.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.366+0000 7fbccfe00640 1 -- 172.21.15.167:0/2888296209 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc8009c10 con 0x7fbcb4004d40 2024-05-11T12:13:13.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.366+0000 7fbccfe00640 1 -- 172.21.15.167:0/2888296209 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcc8036070 con 0x7fbcb4004d40 2024-05-11T12:13:13.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.366+0000 7fbccfe00640 1 -- 172.21.15.167:0/2888296209 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc8031880 con 0x7fbcb4004d40 2024-05-11T12:13:13.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.367+0000 7fbcd6800640 1 -- 172.21.15.167:0/2888296209 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcb4004d40 msgr2=0x7fbcb40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:13.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.367+0000 7fbcd6800640 1 --2- 172.21.15.167:0/2888296209 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcb4004d40 0x7fbcb40051c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8009fc0 tx=0x7fbcb4005830 comp rx=0 tx=0).stop 2024-05-11T12:13:13.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.367+0000 7fbcd5e00640 1 -- 172.21.15.167:0/2888296209 reap_dead start 2024-05-11T12:13:13.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.367+0000 7fbcd6800640 1 -- 172.21.15.167:0/2888296209 shutdown_connections 2024-05-11T12:13:13.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.367+0000 7fbcd6800640 1 -- 172.21.15.167:0/2888296209 >> 172.21.15.167:0/2888296209 conn(0x7fbcd00d1c00 msgr2=0x7fbcd00d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:13.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.367+0000 7fbcd6800640 1 -- 172.21.15.167:0/2888296209 shutdown_connections 2024-05-11T12:13:13.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.367+0000 7fbcd6800640 1 -- 172.21.15.167:0/2888296209 wait complete. 2024-05-11T12:13:13.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.369+0000 7fbcd6800640 1 Processor -- start 2024-05-11T12:13:13.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.369+0000 7fbcd6800640 1 -- start start 2024-05-11T12:13:13.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.370+0000 7fbcd6800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcb4004d40 0x7fbcd00ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:13.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.370+0000 7fbcd6800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbcd01536b0 0x7fbcd00c9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:13.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.370+0000 7fbcd6800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbcd0158300 0x7fbcd00caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:13.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.370+0000 7fbcd6800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd00d4440 con 0x7fbcb4004d40 2024-05-11T12:13:13.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.370+0000 7fbcd6800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd00b3c60 con 0x7fbcd01536b0 2024-05-11T12:13:13.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.370+0000 7fbcd6800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbcd00d45c0 con 0x7fbcd0158300 2024-05-11T12:13:13.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.370+0000 7fbcd5400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcb4004d40 0x7fbcd00ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:13.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.370+0000 7fbcd5400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcb4004d40 0x7fbcd00ceab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41476/0 (socket says 172.21.15.167:41476) 2024-05-11T12:13:13.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.370+0000 7fbcd5400640 1 -- 172.21.15.167:0/3379530967 learned_addr learned my addr 172.21.15.167:0/3379530967 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:13.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.370+0000 7fbcd4a00640 1 --2- 172.21.15.167:0/3379530967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbcd01536b0 0x7fbcd00c9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:13.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.370+0000 7fbcd5400640 1 -- 172.21.15.167:0/3379530967 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbcd0158300 msgr2=0x7fbcd00caaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:13.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.370+0000 7fbcd5400640 1 --2- 172.21.15.167:0/3379530967 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbcd0158300 0x7fbcd00caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:13.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.370+0000 7fbcd5400640 1 -- 172.21.15.167:0/3379530967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbcd01536b0 msgr2=0x7fbcd00c9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:13.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.370+0000 7fbcd5400640 1 --2- 172.21.15.167:0/3379530967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbcd01536b0 0x7fbcd00c9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:13.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.370+0000 7fbcd5400640 1 -- 172.21.15.167:0/3379530967 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcd00c9e70 con 0x7fbcb4004d40 2024-05-11T12:13:13.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.371+0000 7fbcd5400640 1 --2- 172.21.15.167:0/3379530967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcb4004d40 0x7fbcd00ceab0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fbcc000e7c0 tx=0x7fbcc000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:13.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.371+0000 7fbccd600640 1 -- 172.21.15.167:0/3379530967 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc000ee70 con 0x7fbcb4004d40 2024-05-11T12:13:13.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.371+0000 7fbccd600640 1 -- 172.21.15.167:0/3379530967 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcc0018070 con 0x7fbcb4004d40 2024-05-11T12:13:13.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.371+0000 7fbcd6800640 1 -- 172.21.15.167:0/3379530967 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcd00ca0f0 con 0x7fbcb4004d40 2024-05-11T12:13:13.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.371+0000 7fbccd600640 1 -- 172.21.15.167:0/3379530967 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc000ca50 con 0x7fbcb4004d40 2024-05-11T12:13:13.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.371+0000 7fbcd6800640 1 -- 172.21.15.167:0/3379530967 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcd00cc3f0 con 0x7fbcb4004d40 2024-05-11T12:13:13.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.372+0000 7fbcd6800640 1 -- 172.21.15.167:0/3379530967 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcd00694b0 con 0x7fbcb4004d40 2024-05-11T12:13:13.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.374+0000 7fbccd600640 1 -- 172.21.15.167:0/3379530967 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbcc000cbf0 con 0x7fbcb4004d40 2024-05-11T12:13:13.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.375+0000 7fbccd600640 1 --2- 172.21.15.167:0/3379530967 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbcb00762d0 0x7fbcb0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:13.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.375+0000 7fbccd600640 1 -- 172.21.15.167:0/3379530967 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7fbcc0097560 con 0x7fbcb4004d40 2024-05-11T12:13:13.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.375+0000 7fbcd4a00640 1 --2- 172.21.15.167:0/3379530967 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbcb00762d0 0x7fbcb0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:13.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.376+0000 7fbcd4a00640 1 --2- 172.21.15.167:0/3379530967 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbcb00762d0 0x7fbcb0078790 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbcd01f7f50 tx=0x7fbcbc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:13.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.379+0000 7fbccd600640 1 -- 172.21.15.167:0/3379530967 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fbcd00c9e70 con 0x7fbcb4004d40 2024-05-11T12:13:13.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.665+0000 7fbcd6800640 1 -- 172.21.15.167:0/3379530967 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fbcd00759b0 con 0x7fbcb4004d40 2024-05-11T12:13:13.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.670+0000 7fbccd600640 1 -- 172.21.15.167:0/3379530967 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) ==== 190+0+147 (secure 0 0 0) 0x7fbcc0060d30 con 0x7fbcb4004d40 2024-05-11T12:13:13.670 INFO:teuthology.orchestra.run.smithi167.stdout:[client.0] 2024-05-11T12:13:13.670 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQAFYD9mLK/lEhAAyERTYMl250KkBvIrAXWYSw== 2024-05-11T12:13:13.670 INFO:teuthology.orchestra.run.smithi167.stdout: caps mds = "allow" 2024-05-11T12:13:13.670 INFO:teuthology.orchestra.run.smithi167.stdout: caps mon = "allow r" 2024-05-11T12:13:13.671 INFO:teuthology.orchestra.run.smithi167.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-11T12:13:13.671 INFO:teuthology.orchestra.run.smithi167.stderr:updated caps for client.0 2024-05-11T12:13:13.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.672+0000 7fbcd6800640 1 -- 172.21.15.167:0/3379530967 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbcb00762d0 msgr2=0x7fbcb0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:13.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.672+0000 7fbcd6800640 1 --2- 172.21.15.167:0/3379530967 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbcb00762d0 0x7fbcb0078790 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbcd01f7f50 tx=0x7fbcbc008040 comp rx=0 tx=0).stop 2024-05-11T12:13:13.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.673+0000 7fbcd6800640 1 -- 172.21.15.167:0/3379530967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcb4004d40 msgr2=0x7fbcd00ceab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:13.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.673+0000 7fbcd6800640 1 --2- 172.21.15.167:0/3379530967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcb4004d40 0x7fbcd00ceab0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fbcc000e7c0 tx=0x7fbcc000ec80 comp rx=0 tx=0).stop 2024-05-11T12:13:13.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.673+0000 7fbcd6800640 1 -- 172.21.15.167:0/3379530967 shutdown_connections 2024-05-11T12:13:13.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.673+0000 7fbcd6800640 1 --2- 172.21.15.167:0/3379530967 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbcd0158300 0x7fbcd00caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:13.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.673+0000 7fbcd6800640 1 --2- 172.21.15.167:0/3379530967 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbcb00762d0 0x7fbcb0078790 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:13.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.673+0000 7fbcd6800640 1 --2- 172.21.15.167:0/3379530967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbcd01536b0 0x7fbcd00c9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:13.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.673+0000 7fbcd6800640 1 --2- 172.21.15.167:0/3379530967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcb4004d40 0x7fbcd00ceab0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:13.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.673+0000 7fbcd6800640 1 -- 172.21.15.167:0/3379530967 >> 172.21.15.167:0/3379530967 conn(0x7fbcd00d1c00 msgr2=0x7fbcd00b60d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:13.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.673+0000 7fbcd6800640 1 -- 172.21.15.167:0/3379530967 shutdown_connections 2024-05-11T12:13:13.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.673+0000 7fbcd6800640 1 -- 172.21.15.167:0/3379530967 wait complete. 2024-05-11T12:13:13.693 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:13:13.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.883+0000 7fa72de00640 1 Processor -- start 2024-05-11T12:13:13.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.883+0000 7fa72de00640 1 -- start start 2024-05-11T12:13:13.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.885+0000 7fa72de00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa728145b00 0x7fa72806f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:13.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.885+0000 7fa72de00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa72806c660 0x7fa72806ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:13.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.885+0000 7fa72de00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa72806fbc0 con 0x7fa728146770 2024-05-11T12:13:13.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.885+0000 7fa72de00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fa72806cfd0 con 0x7fa72806c660 2024-05-11T12:13:13.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.885+0000 7fa72de00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa7280b4520 con 0x7fa728145b00 2024-05-11T12:13:13.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.886+0000 7fa72ca00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa728145b00 0x7fa72806f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:13.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.886+0000 7fa72d400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa72806c660 0x7fa72806ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:13.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.886+0000 7fa72ca00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa728145b00 0x7fa72806f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:50610/0 (socket says 172.21.15.167:50610) 2024-05-11T12:13:13.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.886+0000 7fa72ca00640 1 -- 172.21.15.167:0/2448112950 learned_addr learned my addr 172.21.15.167:0/2448112950 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:13.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.886+0000 7fa72d400640 1 -- 172.21.15.167:0/2448112950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa728145b00 msgr2=0x7fa72806f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:13.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.886+0000 7fa72d400640 1 --2- 172.21.15.167:0/2448112950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa728145b00 0x7fa72806f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:13.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.887+0000 7fa72d400640 1 -- 172.21.15.167:0/2448112950 >> v1:172.21.15.204:6789/0 conn(0x7fa728146770 legacy=0x7fa7280ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:13.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.887+0000 7fa72d400640 1 -- 172.21.15.167:0/2448112950 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7280b4800 con 0x7fa72806c660 2024-05-11T12:13:13.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.887+0000 7fa72ca00640 1 --2- 172.21.15.167:0/2448112950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa728145b00 0x7fa72806f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:13:13.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.888+0000 7fa72d400640 1 --2- 172.21.15.167:0/2448112950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa72806c660 0x7fa72806ca90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa7200097a0 tx=0x7fa72002a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59ddfecf1390f79d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:13.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.889+0000 7fa727400640 1 -- 172.21.15.167:0/2448112950 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa720037070 con 0x7fa72806c660 2024-05-11T12:13:13.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.889+0000 7fa727400640 1 -- 172.21.15.167:0/2448112950 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa72002ae50 con 0x7fa72806c660 2024-05-11T12:13:13.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.889+0000 7fa727400640 1 -- 172.21.15.167:0/2448112950 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa720032660 con 0x7fa72806c660 2024-05-11T12:13:13.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.890+0000 7fa72de00640 1 -- 172.21.15.167:0/2448112950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa72806c660 msgr2=0x7fa72806ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:13.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.890+0000 7fa72de00640 1 --2- 172.21.15.167:0/2448112950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa72806c660 0x7fa72806ca90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa7200097a0 tx=0x7fa72002a100 comp rx=0 tx=0).stop 2024-05-11T12:13:13.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.890+0000 7fa72de00640 1 -- 172.21.15.167:0/2448112950 shutdown_connections 2024-05-11T12:13:13.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.890+0000 7fa72de00640 1 --2- 172.21.15.167:0/2448112950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa72806c660 0x7fa72806ca90 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:13.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.890+0000 7fa72de00640 1 --2- 172.21.15.167:0/2448112950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa728145b00 0x7fa72806f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:13.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.890+0000 7fa72de00640 1 -- 172.21.15.167:0/2448112950 >> 172.21.15.167:0/2448112950 conn(0x7fa7280b3460 msgr2=0x7fa7280d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:13.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.890+0000 7fa72de00640 1 -- 172.21.15.167:0/2448112950 shutdown_connections 2024-05-11T12:13:13.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.890+0000 7fa72de00640 1 -- 172.21.15.167:0/2448112950 wait complete. 2024-05-11T12:13:13.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.892+0000 7fa72de00640 1 Processor -- start 2024-05-11T12:13:13.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.892+0000 7fa72de00640 1 -- start start 2024-05-11T12:13:13.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.893+0000 7fa72de00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa72806c660 0x7fa72815ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:13.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.893+0000 7fa72de00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa728145b00 0x7fa728144aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:13.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.893+0000 7fa72de00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa728146770 0x7fa7280b1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:13.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.893+0000 7fa72de00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa728145ee0 con 0x7fa728145b00 2024-05-11T12:13:13.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.893+0000 7fa72de00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa7280cf0e0 con 0x7fa728146770 2024-05-11T12:13:13.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.893+0000 7fa72de00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa72806f800 con 0x7fa72806c660 2024-05-11T12:13:13.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.893+0000 7fa727e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa728145b00 0x7fa728144aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:13.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.893+0000 7fa727e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa728145b00 0x7fa728144aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41482/0 (socket says 172.21.15.167:41482) 2024-05-11T12:13:13.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.893+0000 7fa727e00640 1 -- 172.21.15.167:0/2191514447 learned_addr learned my addr 172.21.15.167:0/2191514447 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:13.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.893+0000 7fa72ca00640 1 --2- 172.21.15.167:0/2191514447 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa72806c660 0x7fa72815ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:13.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.893+0000 7fa72d400640 1 --2- 172.21.15.167:0/2191514447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa728146770 0x7fa7280b1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:13.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.893+0000 7fa727e00640 1 -- 172.21.15.167:0/2191514447 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa72806c660 msgr2=0x7fa72815ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:13.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.893+0000 7fa727e00640 1 --2- 172.21.15.167:0/2191514447 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa72806c660 0x7fa72815ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:13.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.893+0000 7fa727e00640 1 -- 172.21.15.167:0/2191514447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa728146770 msgr2=0x7fa7280b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:13.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.893+0000 7fa727e00640 1 --2- 172.21.15.167:0/2191514447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa728146770 0x7fa7280b1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:13.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.893+0000 7fa727e00640 1 -- 172.21.15.167:0/2191514447 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa72813f500 con 0x7fa728145b00 2024-05-11T12:13:13.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.893+0000 7fa72d400640 1 --2- 172.21.15.167:0/2191514447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa728146770 0x7fa7280b1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:13.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.894+0000 7fa727e00640 1 --2- 172.21.15.167:0/2191514447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa728145b00 0x7fa728144aa0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fa71000c3d0 tx=0x7fa71000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:13.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.894+0000 7fa724c00640 1 -- 172.21.15.167:0/2191514447 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa710017070 con 0x7fa728145b00 2024-05-11T12:13:13.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.894+0000 7fa724c00640 1 -- 172.21.15.167:0/2191514447 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7100052d0 con 0x7fa728145b00 2024-05-11T12:13:13.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.894+0000 7fa724c00640 1 -- 172.21.15.167:0/2191514447 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa710003830 con 0x7fa728145b00 2024-05-11T12:13:13.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.894+0000 7fa72de00640 1 -- 172.21.15.167:0/2191514447 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa72813f7e0 con 0x7fa728145b00 2024-05-11T12:13:13.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.894+0000 7fa72de00640 1 -- 172.21.15.167:0/2191514447 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa72813fda0 con 0x7fa728145b00 2024-05-11T12:13:13.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.895+0000 7fa72de00640 1 -- 172.21.15.167:0/2191514447 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7280694b0 con 0x7fa728145b00 2024-05-11T12:13:13.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.896+0000 7fa724c00640 1 -- 172.21.15.167:0/2191514447 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa71001a070 con 0x7fa728145b00 2024-05-11T12:13:13.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.897+0000 7fa724c00640 1 --2- 172.21.15.167:0/2191514447 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa7040762d0 0x7fa704078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:13.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.898+0000 7fa724c00640 1 -- 172.21.15.167:0/2191514447 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7fa710096b10 con 0x7fa728145b00 2024-05-11T12:13:13.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.898+0000 7fa72ca00640 1 --2- 172.21.15.167:0/2191514447 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa7040762d0 0x7fa704078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:13.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.898+0000 7fa72ca00640 1 --2- 172.21.15.167:0/2191514447 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa7040762d0 0x7fa704078790 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa718009240 tx=0x7fa71800b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:13.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:13.902+0000 7fa724c00640 1 -- 172.21.15.167:0/2191514447 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa710060250 con 0x7fa728145b00 2024-05-11T12:13:14.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.163+0000 7fa72de00640 1 -- 172.21.15.167:0/2191514447 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa7280759b0 con 0x7fa728145b00 2024-05-11T12:13:14.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.165+0000 7fa724c00640 1 -- 172.21.15.167:0/2191514447 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5185 (secure 0 0 0) 0x7fa710068240 con 0x7fa728145b00 2024-05-11T12:13:14.165 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:13:14.165 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/2785478956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2785478956},{"type":"v1","addr":"172.21.15.204:6834","nonce":2785478956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":4742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/3478043182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":3478043182},{"type":"v1","addr":"172.21.15.204:6835","nonce":3478043182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/618833945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":618833945},{"type":"v1","addr":"172.21.15.167:6836","nonce":618833945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:13:02.579882+0000","modified":"2024-05-11T12:13:05.652972+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5610":{"gid":5610,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/42694429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":42694429},{"type":"v1","addr":"172.21.15.167:6837","nonce":42694429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5610,"qdb_cluster":[5610]},"id":4}]} 2024-05-11T12:13:14.165 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 35 2024-05-11T12:13:14.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.168+0000 7fa72de00640 1 -- 172.21.15.167:0/2191514447 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa7040762d0 msgr2=0x7fa704078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:14.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.168+0000 7fa72de00640 1 --2- 172.21.15.167:0/2191514447 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa7040762d0 0x7fa704078790 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa718009240 tx=0x7fa71800b040 comp rx=0 tx=0).stop 2024-05-11T12:13:14.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.168+0000 7fa72de00640 1 -- 172.21.15.167:0/2191514447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa728145b00 msgr2=0x7fa728144aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:14.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.168+0000 7fa72de00640 1 --2- 172.21.15.167:0/2191514447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa728145b00 0x7fa728144aa0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fa71000c3d0 tx=0x7fa71000c890 comp rx=0 tx=0).stop 2024-05-11T12:13:14.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.168+0000 7fa72de00640 1 -- 172.21.15.167:0/2191514447 shutdown_connections 2024-05-11T12:13:14.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.168+0000 7fa72de00640 1 --2- 172.21.15.167:0/2191514447 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa7040762d0 0x7fa704078790 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:14.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.168+0000 7fa72de00640 1 --2- 172.21.15.167:0/2191514447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa728146770 0x7fa7280b1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:14.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.168+0000 7fa72de00640 1 --2- 172.21.15.167:0/2191514447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa728145b00 0x7fa728144aa0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:14.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.168+0000 7fa72de00640 1 --2- 172.21.15.167:0/2191514447 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa72806c660 0x7fa72815ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:14.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.168+0000 7fa72de00640 1 -- 172.21.15.167:0/2191514447 >> 172.21.15.167:0/2191514447 conn(0x7fa7280b3460 msgr2=0x7fa728146060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:14.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.168+0000 7fa72de00640 1 -- 172.21.15.167:0/2191514447 shutdown_connections 2024-05-11T12:13:14.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.168+0000 7fa72de00640 1 -- 172.21.15.167:0/2191514447 wait complete. 2024-05-11T12:13:14.589 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 35, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-11T12:13:02.579882+0000', 'modified': '2024-05-11T12:13:05.652972+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5610}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5610': {'gid': 5610, 'name': 'c', 'rank': 0, 'incarnation': 31, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.167:6837/42694429', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6835', 'nonce': 42694429}, {'type': 'v1', 'addr': '172.21.15.167:6837', 'nonce': 42694429}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5610, 'qdb_cluster': [5610]} 2024-05-11T12:13:14.589 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-11T12:13:14.589 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-11T12:13:14.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.797+0000 7f8e6a400640 1 Processor -- start 2024-05-11T12:13:14.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.798+0000 7f8e6a400640 1 -- start start 2024-05-11T12:13:14.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.799+0000 7f8e6a400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8e640d4310 0x7f8e64169eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:14.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.799+0000 7f8e6a400640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e640d13f0 con 0x7f8e641625d0 2024-05-11T12:13:14.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.799+0000 7f8e6a400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e640d2e40 con 0x7f8e640d0f60 2024-05-11T12:13:14.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.799+0000 7f8e6a400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f8e6406df00 con 0x7f8e640d4310 2024-05-11T12:13:14.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.799+0000 7f8e69000640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f8e641625d0 0x7f8e640b8390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:48736/0 (socket says 172.21.15.167:48736) 2024-05-11T12:13:14.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.799+0000 7f8e69000640 1 -- 172.21.15.167:0/4212566882 learned_addr learned my addr 172.21.15.167:0/4212566882 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:13:14.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.800+0000 7f8e69a00640 1 --2- 172.21.15.167:0/4212566882 >> v2:172.21.15.204:3300/0 conn(0x7f8e640d4310 0x7f8e64169eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:14.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.800+0000 7f8e63400640 1 -- 172.21.15.167:0/4212566882 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1922377527 0 0) 0x7f8e640d13f0 con 0x7f8e641625d0 2024-05-11T12:13:14.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.800+0000 7f8e63400640 1 -- 172.21.15.167:0/4212566882 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e44003520 con 0x7f8e641625d0 2024-05-11T12:13:14.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.800+0000 7f8e63400640 1 -- 172.21.15.167:0/4212566882 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4041290804 0 0) 0x7f8e44003520 con 0x7f8e641625d0 2024-05-11T12:13:14.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.800+0000 7f8e63400640 1 -- 172.21.15.167:0/4212566882 >> v2:172.21.15.204:3300/0 conn(0x7f8e640d4310 msgr2=0x7f8e64169eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:14.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.800+0000 7f8e63400640 1 --2- 172.21.15.167:0/4212566882 >> v2:172.21.15.204:3300/0 conn(0x7f8e640d4310 0x7f8e64169eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:14.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.800+0000 7f8e63400640 1 -- 172.21.15.167:0/4212566882 >> v1:172.21.15.204:6789/0 conn(0x7f8e640d0f60 legacy=0x7f8e640d3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:14.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.800+0000 7f8e63400640 1 -- 172.21.15.167:0/4212566882 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e640d3120 con 0x7f8e641625d0 2024-05-11T12:13:14.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.800+0000 7f8e63400640 1 -- 172.21.15.167:0/4212566882 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f8e54001710 con 0x7f8e641625d0 2024-05-11T12:13:14.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.801+0000 7f8e63400640 1 -- 172.21.15.167:0/4212566882 >> v1:172.21.15.167:6789/0 conn(0x7f8e641625d0 legacy=0x7f8e640b8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:14.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.801+0000 7f8e63400640 1 --2- 172.21.15.167:0/4212566882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e44003b10 0x7f8e44003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:14.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.801+0000 7f8e63400640 1 --2- 172.21.15.167:0/4212566882 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8e44004800 0x7f8e44004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:14.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.801+0000 7f8e63400640 1 --2- 172.21.15.167:0/4212566882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e440052e0 0x7f8e4400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:14.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.801+0000 7f8e63400640 1 -- 172.21.15.167:0/4212566882 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4400c180 con 0x7f8e440052e0 2024-05-11T12:13:14.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.801+0000 7f8e63400640 1 -- 172.21.15.167:0/4212566882 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4400c330 con 0x7f8e44003b10 2024-05-11T12:13:14.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.801+0000 7f8e63400640 1 -- 172.21.15.167:0/4212566882 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f8e4400c530 con 0x7f8e44004800 2024-05-11T12:13:14.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.801+0000 7f8e69a00640 1 --2- 172.21.15.167:0/4212566882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e440052e0 0x7f8e4400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:14.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.801+0000 7f8e69000640 1 --2- 172.21.15.167:0/4212566882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e44003b10 0x7f8e44003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:14.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.801+0000 7f8e63e00640 1 --2- 172.21.15.167:0/4212566882 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8e44004800 0x7f8e44004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:14.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.801+0000 7f8e69a00640 1 -- 172.21.15.167:0/4212566882 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8e44004800 msgr2=0x7f8e44004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:14.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.801+0000 7f8e69a00640 1 --2- 172.21.15.167:0/4212566882 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8e44004800 0x7f8e44004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:14.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.801+0000 7f8e69a00640 1 -- 172.21.15.167:0/4212566882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e44003b10 msgr2=0x7f8e44003f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:14.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.801+0000 7f8e69a00640 1 --2- 172.21.15.167:0/4212566882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e44003b10 0x7f8e44003f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:14.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.801+0000 7f8e69a00640 1 -- 172.21.15.167:0/4212566882 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e44003520 con 0x7f8e440052e0 2024-05-11T12:13:14.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.801+0000 7f8e63e00640 1 --2- 172.21.15.167:0/4212566882 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8e44004800 0x7f8e44004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:14.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.802+0000 7f8e69a00640 1 -- 172.21.15.167:0/4212566882 reap_dead start 2024-05-11T12:13:14.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.802+0000 7f8e69a00640 1 --2- 172.21.15.167:0/4212566882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e440052e0 0x7f8e4400b9c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c009ac0 tx=0x7f8e44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bd7c66e36eaa820 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:14.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.803+0000 7f8e63400640 1 -- 172.21.15.167:0/4212566882 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c009690 con 0x7f8e440052e0 2024-05-11T12:13:14.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.803+0000 7f8e63400640 1 -- 172.21.15.167:0/4212566882 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e5c004020 con 0x7f8e440052e0 2024-05-11T12:13:14.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.803+0000 7f8e63400640 1 -- 172.21.15.167:0/4212566882 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c004410 con 0x7f8e440052e0 2024-05-11T12:13:14.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.803+0000 7f8e6a400640 1 -- 172.21.15.167:0/4212566882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e440052e0 msgr2=0x7f8e4400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:14.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.803+0000 7f8e6a400640 1 --2- 172.21.15.167:0/4212566882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e440052e0 0x7f8e4400b9c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c009ac0 tx=0x7f8e44005830 comp rx=0 tx=0).stop 2024-05-11T12:13:14.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.803+0000 7f8e6a400640 1 -- 172.21.15.167:0/4212566882 shutdown_connections 2024-05-11T12:13:14.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.803+0000 7f8e6a400640 1 --2- 172.21.15.167:0/4212566882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e440052e0 0x7f8e4400b9c0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:14.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.803+0000 7f8e6a400640 1 -- 172.21.15.167:0/4212566882 >> 172.21.15.167:0/4212566882 conn(0x7f8e640d50f0 msgr2=0x7f8e640d5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:14.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.803+0000 7f8e6a400640 1 -- 172.21.15.167:0/4212566882 shutdown_connections 2024-05-11T12:13:14.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.804+0000 7f8e6a400640 1 -- 172.21.15.167:0/4212566882 wait complete. 2024-05-11T12:13:14.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.805+0000 7f8e6a400640 1 Processor -- start 2024-05-11T12:13:14.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.805+0000 7f8e6a400640 1 -- start start 2024-05-11T12:13:14.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.806+0000 7f8e6a400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8e440052e0 0x7f8e640ce680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:14.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.806+0000 7f8e6a400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e641f59b0 0x7f8e64151420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:14.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.806+0000 7f8e6a400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e64160480 0x7f8e64160910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:14.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.806+0000 7f8e6a400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8e641629b0 con 0x7f8e641f59b0 2024-05-11T12:13:14.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.806+0000 7f8e6a400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8e640d36e0 con 0x7f8e64160480 2024-05-11T12:13:14.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.806+0000 7f8e6a400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f8e64162b30 con 0x7f8e440052e0 2024-05-11T12:13:14.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.807+0000 7f8e63e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e641f59b0 0x7f8e64151420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:14.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.807+0000 7f8e63e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e641f59b0 0x7f8e64151420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41506/0 (socket says 172.21.15.167:41506) 2024-05-11T12:13:14.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.807+0000 7f8e63e00640 1 -- 172.21.15.167:0/2307412463 learned_addr learned my addr 172.21.15.167:0/2307412463 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:14.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.807+0000 7f8e69000640 1 --2- 172.21.15.167:0/2307412463 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8e440052e0 0x7f8e640ce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:14.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.807+0000 7f8e63e00640 1 -- 172.21.15.167:0/2307412463 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8e440052e0 msgr2=0x7f8e640ce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:14.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.807+0000 7f8e63e00640 1 --2- 172.21.15.167:0/2307412463 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8e440052e0 0x7f8e640ce680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:14.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.807+0000 7f8e63e00640 1 -- 172.21.15.167:0/2307412463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e64160480 msgr2=0x7f8e64160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:14.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.807+0000 7f8e63e00640 1 --2- 172.21.15.167:0/2307412463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e64160480 0x7f8e64160910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:14.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.807+0000 7f8e63e00640 1 -- 172.21.15.167:0/2307412463 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e641f8330 con 0x7f8e641f59b0 2024-05-11T12:13:14.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.807+0000 7f8e69000640 1 --2- 172.21.15.167:0/2307412463 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8e440052e0 0x7f8e640ce680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:14.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.807+0000 7f8e63e00640 1 --2- 172.21.15.167:0/2307412463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e641f59b0 0x7f8e64151420 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c00dae0 tx=0x7f8e4c0020e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:14.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.808+0000 7f8e60c00640 1 -- 172.21.15.167:0/2307412463 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c003ba0 con 0x7f8e641f59b0 2024-05-11T12:13:14.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.808+0000 7f8e60c00640 1 -- 172.21.15.167:0/2307412463 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e4c003d40 con 0x7f8e641f59b0 2024-05-11T12:13:14.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.808+0000 7f8e60c00640 1 -- 172.21.15.167:0/2307412463 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c006d90 con 0x7f8e641f59b0 2024-05-11T12:13:14.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.808+0000 7f8e6a400640 1 -- 172.21.15.167:0/2307412463 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e641f8610 con 0x7f8e641f59b0 2024-05-11T12:13:14.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.808+0000 7f8e6a400640 1 -- 172.21.15.167:0/2307412463 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e641f8bd0 con 0x7f8e641f59b0 2024-05-11T12:13:14.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.810+0000 7f8e60c00640 1 -- 172.21.15.167:0/2307412463 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8e4c01a020 con 0x7f8e641f59b0 2024-05-11T12:13:14.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.811+0000 7f8e60c00640 1 --2- 172.21.15.167:0/2307412463 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f8e40076300 0x7f8e400787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:14.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.811+0000 7f8e6e7d9740 1 Processor -- start 2024-05-11T12:13:14.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.811+0000 7f8e60c00640 1 -- 172.21.15.167:0/2307412463 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f8e4c097360 con 0x7f8e641f59b0 2024-05-11T12:13:14.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.812+0000 7f8e69000640 1 --2- 172.21.15.167:0/2307412463 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f8e40076300 0x7f8e400787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:14.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.812+0000 7f8e6e7d9740 1 -- start start 2024-05-11T12:13:14.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.812+0000 7f8e69000640 1 --2- 172.21.15.167:0/2307412463 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f8e40076300 0x7f8e400787c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8e54001eb0 tx=0x7f8e54000f10 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:14.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.813+0000 7f8e6e7d9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c272cda10 0x556c272cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:14.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.813+0000 7f8e6e7d9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c272c88a0 0x556c272cc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:14.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.813+0000 7f8e6e7d9740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x556c272ccb50 0x556c272ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:14.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.813+0000 7f8e6e7d9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556c27231790 con 0x556c272c88a0 2024-05-11T12:13:14.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.813+0000 7f8e6e7d9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556c271d6f20 con 0x556c272cda10 2024-05-11T12:13:14.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.813+0000 7f8e6e7d9740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x556c2722d750 con 0x556c272ccb50 2024-05-11T12:13:14.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.813+0000 7f8e63e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c272c88a0 0x556c272cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:14.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.813+0000 7f8e63e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c272c88a0 0x556c272cc610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41508/0 (socket says 172.21.15.167:41508) 2024-05-11T12:13:14.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.813+0000 7f8e63e00640 1 -- 172.21.15.167:0/3091873198 learned_addr learned my addr 172.21.15.167:0/3091873198 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:14.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.813+0000 7f8e69000640 1 --2- 172.21.15.167:0/3091873198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c272cda10 0x556c272cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:14.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.813+0000 7f8e63e00640 1 -- 172.21.15.167:0/3091873198 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x556c272ccb50 msgr2=0x556c272ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:14.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.813+0000 7f8e63e00640 1 --2- 172.21.15.167:0/3091873198 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x556c272ccb50 0x556c272ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:14.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.813+0000 7f8e63e00640 1 -- 172.21.15.167:0/3091873198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c272cda10 msgr2=0x556c272cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:14.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.813+0000 7f8e63e00640 1 --2- 172.21.15.167:0/3091873198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c272cda10 0x556c272cddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:14.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.813+0000 7f8e63e00640 1 -- 172.21.15.167:0/3091873198 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c26f432d0 con 0x556c272c88a0 2024-05-11T12:13:14.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.814+0000 7f8e63e00640 1 --2- 172.21.15.167:0/3091873198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c272c88a0 0x556c272cc610 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c05c4a0 tx=0x7f8e4c05fc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28b549c9892a1dcd server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:14.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.814+0000 7f8e59600640 1 -- 172.21.15.167:0/3091873198 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c060420 con 0x556c272c88a0 2024-05-11T12:13:14.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.814+0000 7f8e59600640 1 -- 172.21.15.167:0/3091873198 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e4c0647d0 con 0x556c272c88a0 2024-05-11T12:13:14.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.814+0000 7f8e59600640 1 -- 172.21.15.167:0/3091873198 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c0691f0 con 0x556c272c88a0 2024-05-11T12:13:14.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.814+0000 7f8e6e7d9740 1 -- 172.21.15.167:0/3091873198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c272c88a0 msgr2=0x556c272cc610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:14.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.814+0000 7f8e6e7d9740 1 --2- 172.21.15.167:0/3091873198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c272c88a0 0x556c272cc610 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c05c4a0 tx=0x7f8e4c05fc50 comp rx=0 tx=0).stop 2024-05-11T12:13:14.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.815+0000 7f8e6e7d9740 1 -- 172.21.15.167:0/3091873198 shutdown_connections 2024-05-11T12:13:14.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.815+0000 7f8e6e7d9740 1 --2- 172.21.15.167:0/3091873198 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x556c272ccb50 0x556c272ccf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:14.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.815+0000 7f8e6e7d9740 1 --2- 172.21.15.167:0/3091873198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c272c88a0 0x556c272cc610 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:14.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.815+0000 7f8e6e7d9740 1 --2- 172.21.15.167:0/3091873198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c272cda10 0x556c272cddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:14.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.815+0000 7f8e6e7d9740 1 -- 172.21.15.167:0/3091873198 >> 172.21.15.167:0/3091873198 conn(0x556c272cab10 msgr2=0x556c273135d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:14.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.815+0000 7f8e6e7d9740 1 -- 172.21.15.167:0/3091873198 shutdown_connections 2024-05-11T12:13:14.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.815+0000 7f8e6e7d9740 1 -- 172.21.15.167:0/3091873198 wait complete. 2024-05-11T12:13:14.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.815+0000 7f8e6e7d9740 1 -- start start 2024-05-11T12:13:14.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.816+0000 7f8e6e7d9740 1 Processor -- start 2024-05-11T12:13:14.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.817+0000 7f8e6e7d9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c272ccb50 0x556c272edc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:14.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.817+0000 7f8e6e7d9740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x556c272cda10 0x556c27316f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:14.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.817+0000 7f8e6e7d9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c272ee1c0 0x556c27317470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:14.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.817+0000 7f8e6e7d9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556c272f95a0 con 0x556c272ee1c0 2024-05-11T12:13:14.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.817+0000 7f8e6e7d9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556c271ea9c0 con 0x556c272ccb50 2024-05-11T12:13:14.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.817+0000 7f8e6e7d9740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x556c272f8a20 con 0x556c272cda10 2024-05-11T12:13:14.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.817+0000 7f8e6e7d9740 10 client.0 _ll_register_callbacks cb 0x556c271b7120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:13:14.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.817+0000 7f8e69a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c272ee1c0 0x556c27317470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:14.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.817+0000 7f8e69a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c272ee1c0 0x556c27317470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41522/0 (socket says 172.21.15.167:41522) 2024-05-11T12:13:14.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.817+0000 7f8e69a00640 1 -- 172.21.15.167:0/1490701110 learned_addr learned my addr 172.21.15.167:0/1490701110 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:14.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.817+0000 7f8e69000640 1 --2- 172.21.15.167:0/1490701110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c272ccb50 0x556c272edc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:14.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.817+0000 7f8e63e00640 1 --2- 172.21.15.167:0/1490701110 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x556c272cda10 0x556c27316f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:14.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.817+0000 7f8e69a00640 1 -- 172.21.15.167:0/1490701110 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x556c272cda10 msgr2=0x556c27316f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:14.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.817+0000 7f8e69a00640 1 --2- 172.21.15.167:0/1490701110 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x556c272cda10 0x556c27316f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:14.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.817+0000 7f8e69a00640 1 -- 172.21.15.167:0/1490701110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c272ccb50 msgr2=0x556c272edc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:14.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.817+0000 7f8e69a00640 1 --2- 172.21.15.167:0/1490701110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c272ccb50 0x556c272edc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:14.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.817+0000 7f8e69a00640 1 -- 172.21.15.167:0/1490701110 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c2700d6b0 con 0x556c272ee1c0 2024-05-11T12:13:14.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.817+0000 7f8e69000640 1 --2- 172.21.15.167:0/1490701110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c272ccb50 0x556c272edc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:14.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.818+0000 7f8e69a00640 1 --2- 172.21.15.167:0/1490701110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c272ee1c0 0x556c27317470 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c02abe0 tx=0x7f8e5c004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a54561e5c729738e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:14.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.818+0000 7f8e52000640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-11T12:13:14.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.818+0000 7f8e52000640 1 -- 172.21.15.167:0/1490701110 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c003610 con 0x556c272ee1c0 2024-05-11T12:13:14.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.818+0000 7f8e52000640 1 -- 172.21.15.167:0/1490701110 <== mon.0 v2:172.21.15.167:3300/0 2 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f8e5c0037b0 con 0x556c272ee1c0 2024-05-11T12:13:14.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.818+0000 7f8e6e7d9740 1 -- 172.21.15.167:0/1490701110 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e4c0647d0 con 0x556c272ee1c0 2024-05-11T12:13:14.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.818+0000 7f8e52000640 1 -- 172.21.15.167:0/1490701110 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8e5c047050 con 0x556c272ee1c0 2024-05-11T12:13:14.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.818+0000 7f8e6e7d9740 10 client.5772 fetch_fsmap learned FSMap version 35 2024-05-11T12:13:14.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.818+0000 7f8e6e7d9740 1 -- 172.21.15.167:0/1490701110 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x556c27315b90 con 0x556c272ee1c0 2024-05-11T12:13:14.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.819+0000 7f8e52000640 1 -- 172.21.15.167:0/1490701110 <== mon.0 v2:172.21.15.167:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e4c0647d0 con 0x556c272ee1c0 2024-05-11T12:13:14.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.819+0000 7f8e52000640 1 -- 172.21.15.167:0/1490701110 <== mon.0 v2:172.21.15.167:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f8e5c04a020 con 0x556c272ee1c0 2024-05-11T12:13:14.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.819+0000 7f8e6e7d9740 10 client.5772 fetch_fsmap finished waiting for FSMap version 35 2024-05-11T12:13:14.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.819+0000 7f8e6e7d9740 10 client.5772 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-11T12:13:14.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.819+0000 7f8e6e7d9740 20 client.5772 populate_metadata read hostname 'smithi167' 2024-05-11T12:13:14.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.819+0000 7f8e6e7d9740 1 --2- 172.21.15.167:0/1490701110 >> [v2:172.21.15.167:6835/42694429,v1:172.21.15.167:6837/42694429] conn(0x556c27314ff0 0x556c273153d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:14.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.819+0000 7f8e6e7d9740 4 client.5772 mds_command: new command op to 5610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:13:14.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.819+0000 7f8e6e7d9740 1 -- 172.21.15.167:0/1490701110 --> [v2:172.21.15.167:6835/42694429,v1:172.21.15.167:6837/42694429] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c2723f3a0 con 0x556c27314ff0 2024-05-11T12:13:14.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.819+0000 7f8e69000640 1 --2- 172.21.15.167:0/1490701110 >> [v2:172.21.15.167:6835/42694429,v1:172.21.15.167:6837/42694429] conn(0x556c27314ff0 0x556c273153d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:14.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.820+0000 7f8e69000640 1 --2- 172.21.15.167:0/1490701110 >> [v2:172.21.15.167:6835/42694429,v1:172.21.15.167:6837/42694429] conn(0x556c27314ff0 0x556c273153d0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acc5377adc693c5 server_cookie=b7e11c43588b7fd1 in_seq=0 out_seq=0 2024-05-11T12:13:14.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.820+0000 7f8e52000640 10 client.5772 ms_handle_connect on v2:172.21.15.167:6835/42694429 2024-05-11T12:13:14.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.821+0000 7f8e52000640 1 -- 172.21.15.167:0/1490701110 <== mds.0 v2:172.21.15.167:6835/42694429 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556c2723f3a0 con 0x556c27314ff0 2024-05-11T12:13:14.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.821+0000 7f8e52000640 10 client.5772 handle_command_reply: tid=0 2024-05-11T12:13:14.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.821+0000 7f8e6e7d9740 1 client.5772 shutdown 2024-05-11T12:13:14.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.821+0000 7f8e6e7d9740 1 -- 172.21.15.167:0/1490701110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c272ee1c0 msgr2=0x556c27317470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:14.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.821+0000 7f8e6e7d9740 1 --2- 172.21.15.167:0/1490701110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c272ee1c0 0x556c27317470 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c02abe0 tx=0x7f8e5c004610 comp rx=0 tx=0).stop 2024-05-11T12:13:14.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.821+0000 7f8e6e7d9740 1 -- 172.21.15.167:0/1490701110 shutdown_connections 2024-05-11T12:13:14.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.821+0000 7f8e6e7d9740 1 --2- 172.21.15.167:0/1490701110 >> [v2:172.21.15.167:6835/42694429,v1:172.21.15.167:6837/42694429] conn(0x556c27314ff0 0x556c273153d0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:14.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.821+0000 7f8e6e7d9740 1 --2- 172.21.15.167:0/1490701110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c272ee1c0 0x556c27317470 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:14.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.821+0000 7f8e6e7d9740 1 --2- 172.21.15.167:0/1490701110 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x556c272cda10 0x556c27316f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:14.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.821+0000 7f8e6e7d9740 1 --2- 172.21.15.167:0/1490701110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c272ccb50 0x556c272edc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:14.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.821+0000 7f8e52000640 0 client.5772 ms_handle_reset on v2:172.21.15.167:6835/42694429 2024-05-11T12:13:14.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.821+0000 7f8e6e7d9740 1 -- 172.21.15.167:0/1490701110 >> 172.21.15.167:0/1490701110 conn(0x556c272c88a0 msgr2=0x556c272c7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:14.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.821+0000 7f8e6e7d9740 1 -- 172.21.15.167:0/1490701110 shutdown_connections 2024-05-11T12:13:14.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.821+0000 7f8e6e7d9740 1 -- 172.21.15.167:0/1490701110 wait complete. 2024-05-11T12:13:14.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.821+0000 7f8e6e7d9740 20 client.5772 trim_cache size 0 max 16384 2024-05-11T12:13:14.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.842+0000 7f8e6e7d9740 1 Processor -- start 2024-05-11T12:13:14.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.842+0000 7f8e6e7d9740 1 -- start start 2024-05-11T12:13:14.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.843+0000 7f8e6e7d9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c272ccb50 0x556c272f9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:14.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.843+0000 7f8e6e7d9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c272cda10 0x556c272f0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:14.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.843+0000 7f8e6e7d9740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x556c272ee1c0 0x556c27316d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:14.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.843+0000 7f8e6e7d9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8e64202480 con 0x556c272cda10 2024-05-11T12:13:14.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.843+0000 7f8e6e7d9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8e642014c0 con 0x556c272ccb50 2024-05-11T12:13:14.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.843+0000 7f8e6e7d9740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f8e64200820 con 0x556c272ee1c0 2024-05-11T12:13:14.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.844+0000 7f8e63e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c272cda10 0x556c272f0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:14.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.844+0000 7f8e63e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c272cda10 0x556c272f0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41532/0 (socket says 172.21.15.167:41532) 2024-05-11T12:13:14.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.844+0000 7f8e63e00640 1 -- 172.21.15.167:0/14673816 learned_addr learned my addr 172.21.15.167:0/14673816 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:14.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.844+0000 7f8e69a00640 1 --2- 172.21.15.167:0/14673816 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x556c272ee1c0 0x556c27316d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:14.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.844+0000 7f8e69000640 1 --2- 172.21.15.167:0/14673816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c272ccb50 0x556c272f9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:14.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.844+0000 7f8e63e00640 1 -- 172.21.15.167:0/14673816 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x556c272ee1c0 msgr2=0x556c27316d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:14.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.844+0000 7f8e63e00640 1 --2- 172.21.15.167:0/14673816 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x556c272ee1c0 0x556c27316d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:14.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.844+0000 7f8e63e00640 1 -- 172.21.15.167:0/14673816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c272ccb50 msgr2=0x556c272f9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:14.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.844+0000 7f8e63e00640 1 --2- 172.21.15.167:0/14673816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c272ccb50 0x556c272f9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:14.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.844+0000 7f8e63e00640 1 -- 172.21.15.167:0/14673816 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c273194b0 con 0x556c272cda10 2024-05-11T12:13:14.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.844+0000 7f8e63e00640 1 --2- 172.21.15.167:0/14673816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c272cda10 0x556c272f0e30 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c01b000 tx=0x7f8e4c063000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1820ed6b5302fdfa server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:14.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.845+0000 7f8e58c00640 1 -- 172.21.15.167:0/14673816 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c0647d0 con 0x556c272cda10 2024-05-11T12:13:14.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.845+0000 7f8e58c00640 1 -- 172.21.15.167:0/14673816 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e4c0691f0 con 0x556c272cda10 2024-05-11T12:13:14.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.845+0000 7f8e58c00640 1 -- 172.21.15.167:0/14673816 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c060420 con 0x556c272cda10 2024-05-11T12:13:14.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.845+0000 7f8e6e7d9740 1 -- 172.21.15.167:0/14673816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c272cda10 msgr2=0x556c272f0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:14.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.845+0000 7f8e6e7d9740 1 --2- 172.21.15.167:0/14673816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c272cda10 0x556c272f0e30 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c01b000 tx=0x7f8e4c063000 comp rx=0 tx=0).stop 2024-05-11T12:13:14.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.845+0000 7f8e6e7d9740 1 -- 172.21.15.167:0/14673816 shutdown_connections 2024-05-11T12:13:14.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.845+0000 7f8e6e7d9740 1 --2- 172.21.15.167:0/14673816 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x556c272ee1c0 0x556c27316d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:14.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.845+0000 7f8e6e7d9740 1 --2- 172.21.15.167:0/14673816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c272cda10 0x556c272f0e30 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:14.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.845+0000 7f8e6e7d9740 1 --2- 172.21.15.167:0/14673816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c272ccb50 0x556c272f9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:14.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.845+0000 7f8e6e7d9740 1 -- 172.21.15.167:0/14673816 >> 172.21.15.167:0/14673816 conn(0x556c272c88a0 msgr2=0x556c273153d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:14.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.846+0000 7f8e6e7d9740 1 -- 172.21.15.167:0/14673816 shutdown_connections 2024-05-11T12:13:14.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.846+0000 7f8e6e7d9740 1 -- 172.21.15.167:0/14673816 wait complete. 2024-05-11T12:13:14.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.846+0000 7f8e6e7d9740 1 -- start start 2024-05-11T12:13:14.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.847+0000 7f8e6e7d9740 1 Processor -- start 2024-05-11T12:13:14.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.847+0000 7f8e6e7d9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c272ccb50 0x556c27392670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:14.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.848+0000 7f8e6e7d9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c272ee1c0 0x556c272eab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:14.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.848+0000 7f8e6e7d9740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x556c272f8c80 0x556c272ef2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:14.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.848+0000 7f8e6e7d9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556c272ee710 con 0x556c272ccb50 2024-05-11T12:13:14.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.848+0000 7f8e6e7d9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8e64200e70 con 0x556c272ee1c0 2024-05-11T12:13:14.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.848+0000 7f8e6e7d9740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x556c272eed20 con 0x556c272f8c80 2024-05-11T12:13:14.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.848+0000 7f8e6e7d9740 10 client.0 _ll_register_callbacks cb 0x7f8e440036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:13:14.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.848+0000 7f8e69000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c272ccb50 0x556c27392670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:14.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.848+0000 7f8e69000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c272ccb50 0x556c27392670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41540/0 (socket says 172.21.15.167:41540) 2024-05-11T12:13:14.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.848+0000 7f8e69000640 1 -- 172.21.15.167:0/1430358843 learned_addr learned my addr 172.21.15.167:0/1430358843 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:14.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.848+0000 7f8e69000640 1 -- 172.21.15.167:0/1430358843 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x556c272f8c80 msgr2=0x556c272ef2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:14.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.848+0000 7f8e69000640 1 --2- 172.21.15.167:0/1430358843 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x556c272f8c80 0x556c272ef2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:14.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.848+0000 7f8e69000640 1 -- 172.21.15.167:0/1430358843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c272ee1c0 msgr2=0x556c272eab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:14.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.848+0000 7f8e63e00640 1 --2- 172.21.15.167:0/1430358843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c272ee1c0 0x556c272eab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:14.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.848+0000 7f8e69000640 1 --2- 172.21.15.167:0/1430358843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c272ee1c0 0x556c272eab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:14.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.848+0000 7f8e69000640 1 -- 172.21.15.167:0/1430358843 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c272ed0f0 con 0x556c272ccb50 2024-05-11T12:13:14.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.849+0000 7f8e69000640 1 --2- 172.21.15.167:0/1430358843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c272ccb50 0x556c27392670 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8e54006f20 tx=0x7f8e640b8140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a756a384b4bacaa3 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:14.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.849+0000 7f8e52000640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-11T12:13:14.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.849+0000 7f8e52000640 1 -- 172.21.15.167:0/1430358843 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e54029020 con 0x556c272ccb50 2024-05-11T12:13:14.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.849+0000 7f8e52000640 1 -- 172.21.15.167:0/1430358843 <== mon.0 v2:172.21.15.167:3300/0 2 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f8e54014630 con 0x556c272ccb50 2024-05-11T12:13:14.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.849+0000 7f8e6e7d9740 1 -- 172.21.15.167:0/1430358843 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e4c0691f0 con 0x556c272ccb50 2024-05-11T12:13:14.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.849+0000 7f8e52000640 1 -- 172.21.15.167:0/1430358843 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8e54010040 con 0x556c272ccb50 2024-05-11T12:13:14.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.849+0000 7f8e6e7d9740 10 client.5778 fetch_fsmap learned FSMap version 35 2024-05-11T12:13:14.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.849+0000 7f8e6e7d9740 1 -- 172.21.15.167:0/1430358843 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x556c27391e70 con 0x556c272ccb50 2024-05-11T12:13:14.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.849+0000 7f8e52000640 1 -- 172.21.15.167:0/1430358843 <== mon.0 v2:172.21.15.167:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e4c0691f0 con 0x556c272ccb50 2024-05-11T12:13:14.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.849+0000 7f8e52000640 1 -- 172.21.15.167:0/1430358843 <== mon.0 v2:172.21.15.167:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f8e54036020 con 0x556c272ccb50 2024-05-11T12:13:14.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.849+0000 7f8e6e7d9740 10 client.5778 fetch_fsmap finished waiting for FSMap version 35 2024-05-11T12:13:14.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.850+0000 7f8e6e7d9740 10 client.5778 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-11T12:13:14.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.850+0000 7f8e6e7d9740 20 client.5778 populate_metadata read hostname 'smithi167' 2024-05-11T12:13:14.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.850+0000 7f8e6e7d9740 1 --2- 172.21.15.167:0/1430358843 >> [v2:172.21.15.167:6835/42694429,v1:172.21.15.167:6837/42694429] conn(0x556c273366c0 0x556c27336aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:14.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.850+0000 7f8e6e7d9740 4 client.5778 mds_command: new command op to 5610 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-11T12:13:14.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.850+0000 7f8e6e7d9740 1 -- 172.21.15.167:0/1430358843 --> [v2:172.21.15.167:6835/42694429,v1:172.21.15.167:6837/42694429] -- command(tid 0: {"prefix": "status"}) -- 0x556c27239210 con 0x556c273366c0 2024-05-11T12:13:14.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.850+0000 7f8e63e00640 1 --2- 172.21.15.167:0/1430358843 >> [v2:172.21.15.167:6835/42694429,v1:172.21.15.167:6837/42694429] conn(0x556c273366c0 0x556c27336aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:14.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.850+0000 7f8e63e00640 1 --2- 172.21.15.167:0/1430358843 >> [v2:172.21.15.167:6835/42694429,v1:172.21.15.167:6837/42694429] conn(0x556c273366c0 0x556c27336aa0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dddd223eca759747 server_cookie=6ede33f3c0d258da in_seq=0 out_seq=0 2024-05-11T12:13:14.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.850+0000 7f8e52000640 10 client.5778 ms_handle_connect on v2:172.21.15.167:6835/42694429 2024-05-11T12:13:14.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.851+0000 7f8e52000640 1 -- 172.21.15.167:0/1430358843 <== mds.0 v2:172.21.15.167:6835/42694429 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x556c27239210 con 0x556c273366c0 2024-05-11T12:13:14.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.851+0000 7f8e52000640 10 client.5778 handle_command_reply: tid=0 2024-05-11T12:13:14.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.851+0000 7f8e6e7d9740 1 client.5778 shutdown 2024-05-11T12:13:14.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.851+0000 7f8e6e7d9740 1 -- 172.21.15.167:0/1430358843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c272ccb50 msgr2=0x556c27392670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:14.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.851+0000 7f8e6e7d9740 1 --2- 172.21.15.167:0/1430358843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c272ccb50 0x556c27392670 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8e54006f20 tx=0x7f8e640b8140 comp rx=0 tx=0).stop 2024-05-11T12:13:14.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.851+0000 7f8e6e7d9740 1 -- 172.21.15.167:0/1430358843 shutdown_connections 2024-05-11T12:13:14.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.851+0000 7f8e6e7d9740 1 --2- 172.21.15.167:0/1430358843 >> [v2:172.21.15.167:6835/42694429,v1:172.21.15.167:6837/42694429] conn(0x556c273366c0 0x556c27336aa0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:14.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.851+0000 7f8e6e7d9740 1 --2- 172.21.15.167:0/1430358843 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x556c272f8c80 0x556c272ef2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:14.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.851+0000 7f8e6e7d9740 1 --2- 172.21.15.167:0/1430358843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c272ee1c0 0x556c272eab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:14.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.851+0000 7f8e6e7d9740 1 --2- 172.21.15.167:0/1430358843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c272ccb50 0x556c27392670 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:14.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.851+0000 7f8e6e7d9740 1 -- 172.21.15.167:0/1430358843 >> 172.21.15.167:0/1430358843 conn(0x556c272c88a0 msgr2=0x556c272f0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:14.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.851+0000 7f8e52000640 0 client.5778 ms_handle_reset on v2:172.21.15.167:6835/42694429 2024-05-11T12:13:14.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.851+0000 7f8e6e7d9740 1 -- 172.21.15.167:0/1430358843 shutdown_connections 2024-05-11T12:13:14.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.851+0000 7f8e6e7d9740 1 -- 172.21.15.167:0/1430358843 wait complete. 2024-05-11T12:13:14.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.851+0000 7f8e6e7d9740 20 client.5778 trim_cache size 0 max 16384 2024-05-11T12:13:14.855 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T12:13:14.855 INFO:teuthology.orchestra.run.smithi167.stdout: "cluster_fsid": "5f32d344-7289-4c27-881b-1533419ab678", 2024-05-11T12:13:14.855 INFO:teuthology.orchestra.run.smithi167.stdout: "whoami": 0, 2024-05-11T12:13:14.855 INFO:teuthology.orchestra.run.smithi167.stdout: "id": 5610, 2024-05-11T12:13:14.855 INFO:teuthology.orchestra.run.smithi167.stdout: "want_state": "up:active", 2024-05-11T12:13:14.856 INFO:teuthology.orchestra.run.smithi167.stdout: "state": "up:active", 2024-05-11T12:13:14.856 INFO:teuthology.orchestra.run.smithi167.stdout: "fs_name": "cephfs", 2024-05-11T12:13:14.856 INFO:teuthology.orchestra.run.smithi167.stdout: "rank_uptime": 11.704380955, 2024-05-11T12:13:14.856 INFO:teuthology.orchestra.run.smithi167.stdout: "mdsmap_epoch": 35, 2024-05-11T12:13:14.856 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_epoch": 64, 2024-05-11T12:13:14.856 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_epoch_barrier": 64, 2024-05-11T12:13:14.856 INFO:teuthology.orchestra.run.smithi167.stdout: "uptime": 16.189518113999998 2024-05-11T12:13:14.856 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T12:13:14.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.852+0000 7f8e6a400640 1 -- 172.21.15.167:0/2307412463 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f8e40076300 msgr2=0x7f8e400787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:14.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.852+0000 7f8e6a400640 1 --2- 172.21.15.167:0/2307412463 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f8e40076300 0x7f8e400787c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8e54001eb0 tx=0x7f8e54000f10 comp rx=0 tx=0).stop 2024-05-11T12:13:14.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.852+0000 7f8e6a400640 1 -- 172.21.15.167:0/2307412463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e641f59b0 msgr2=0x7f8e64151420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:14.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.852+0000 7f8e6a400640 1 --2- 172.21.15.167:0/2307412463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e641f59b0 0x7f8e64151420 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c00dae0 tx=0x7f8e4c0020e0 comp rx=0 tx=0).stop 2024-05-11T12:13:14.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.852+0000 7f8e6a400640 1 -- 172.21.15.167:0/2307412463 shutdown_connections 2024-05-11T12:13:14.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.852+0000 7f8e6a400640 1 --2- 172.21.15.167:0/2307412463 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f8e40076300 0x7f8e400787c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:14.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.852+0000 7f8e6a400640 1 --2- 172.21.15.167:0/2307412463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e64160480 0x7f8e64160910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:14.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.852+0000 7f8e6a400640 1 --2- 172.21.15.167:0/2307412463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e641f59b0 0x7f8e64151420 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:14.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.852+0000 7f8e6a400640 1 --2- 172.21.15.167:0/2307412463 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8e440052e0 0x7f8e640ce680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:14.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.852+0000 7f8e6a400640 1 -- 172.21.15.167:0/2307412463 >> 172.21.15.167:0/2307412463 conn(0x7f8e640d50f0 msgr2=0x7f8e640cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:14.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.852+0000 7f8e6a400640 1 -- 172.21.15.167:0/2307412463 shutdown_connections 2024-05-11T12:13:14.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:14.852+0000 7f8e6a400640 1 -- 172.21.15.167:0/2307412463 wait complete. 2024-05-11T12:13:14.870 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-11T12:13:15.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.057+0000 7f8935200640 1 Processor -- start 2024-05-11T12:13:15.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.058+0000 7f8935200640 1 -- start start 2024-05-11T12:13:15.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.059+0000 7f8935200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8930157750 0x7f893015d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:15.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.059+0000 7f8935200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f89300b3ea0 0x7f8930161490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:15.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.059+0000 7f8935200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f893015dc80 con 0x7f89301619d0 2024-05-11T12:13:15.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.059+0000 7f8935200640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f8930161000 con 0x7f89300b3ea0 2024-05-11T12:13:15.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.059+0000 7f8935200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f893015df30 con 0x7f8930157750 2024-05-11T12:13:15.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.059+0000 7f892d400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f89300b3ea0 0x7f8930161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:15.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.060+0000 7f892d400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f89300b3ea0 0x7f8930161490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41552/0 (socket says 172.21.15.167:41552) 2024-05-11T12:13:15.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.060+0000 7f892d400640 1 -- 172.21.15.167:0/977306021 learned_addr learned my addr 172.21.15.167:0/977306021 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:15.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.060+0000 7f892de00640 1 --2- 172.21.15.167:0/977306021 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8930157750 0x7f893015d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:15.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.060+0000 7f892d400640 1 -- 172.21.15.167:0/977306021 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8930157750 msgr2=0x7f893015d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:15.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.060+0000 7f892d400640 1 --2- 172.21.15.167:0/977306021 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8930157750 0x7f893015d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:15.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.060+0000 7f892d400640 1 -- 172.21.15.167:0/977306021 >> v1:172.21.15.204:6789/0 conn(0x7f89301619d0 legacy=0x7f893006d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:15.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.060+0000 7f892d400640 1 -- 172.21.15.167:0/977306021 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f893015e210 con 0x7f89300b3ea0 2024-05-11T12:13:15.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.060+0000 7f892de00640 1 --2- 172.21.15.167:0/977306021 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8930157750 0x7f893015d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:15.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.061+0000 7f892d400640 1 --2- 172.21.15.167:0/977306021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89300b3ea0 0x7f8930161490 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f8918009870 tx=0x7f891802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81f2d20351aa8eb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:15.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.062+0000 7f892ca00640 1 -- 172.21.15.167:0/977306021 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8918037070 con 0x7f89300b3ea0 2024-05-11T12:13:15.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.062+0000 7f892ca00640 1 -- 172.21.15.167:0/977306021 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8918032440 con 0x7f89300b3ea0 2024-05-11T12:13:15.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.062+0000 7f892ca00640 1 -- 172.21.15.167:0/977306021 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8918032810 con 0x7f89300b3ea0 2024-05-11T12:13:15.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.062+0000 7f8935200640 1 -- 172.21.15.167:0/977306021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89300b3ea0 msgr2=0x7f8930161490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:15.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.062+0000 7f8935200640 1 --2- 172.21.15.167:0/977306021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89300b3ea0 0x7f8930161490 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f8918009870 tx=0x7f891802a1c0 comp rx=0 tx=0).stop 2024-05-11T12:13:15.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.062+0000 7f8935200640 1 -- 172.21.15.167:0/977306021 shutdown_connections 2024-05-11T12:13:15.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.062+0000 7f8935200640 1 --2- 172.21.15.167:0/977306021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89300b3ea0 0x7f8930161490 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:15.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.062+0000 7f8935200640 1 --2- 172.21.15.167:0/977306021 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8930157750 0x7f893015d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:15.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.062+0000 7f8935200640 1 -- 172.21.15.167:0/977306021 >> 172.21.15.167:0/977306021 conn(0x7f89300cd8f0 msgr2=0x7f89300cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:15.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.062+0000 7f8935200640 1 -- 172.21.15.167:0/977306021 shutdown_connections 2024-05-11T12:13:15.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.062+0000 7f8935200640 1 -- 172.21.15.167:0/977306021 wait complete. 2024-05-11T12:13:15.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.064+0000 7f8935200640 1 Processor -- start 2024-05-11T12:13:15.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.064+0000 7f8935200640 1 -- start start 2024-05-11T12:13:15.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.065+0000 7f8935200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f89300b3ea0 0x7f8930146830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:15.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.065+0000 7f8935200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8930157750 0x7f89300aae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:15.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.065+0000 7f8935200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89301619d0 0x7f89300ab360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:15.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.065+0000 7f8935200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f893006db70 con 0x7f89301619d0 2024-05-11T12:13:15.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.065+0000 7f8935200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8930157b30 con 0x7f8930157750 2024-05-11T12:13:15.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.065+0000 7f8935200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f893006dcf0 con 0x7f89300b3ea0 2024-05-11T12:13:15.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.066+0000 7f892e800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89301619d0 0x7f89300ab360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:15.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.066+0000 7f892e800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89301619d0 0x7f89300ab360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41558/0 (socket says 172.21.15.167:41558) 2024-05-11T12:13:15.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.066+0000 7f892e800640 1 -- 172.21.15.167:0/2176144266 learned_addr learned my addr 172.21.15.167:0/2176144266 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:15.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.066+0000 7f892de00640 1 --2- 172.21.15.167:0/2176144266 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f89300b3ea0 0x7f8930146830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:15.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.066+0000 7f892d400640 1 --2- 172.21.15.167:0/2176144266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8930157750 0x7f89300aae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:15.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.066+0000 7f892e800640 1 -- 172.21.15.167:0/2176144266 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f89300b3ea0 msgr2=0x7f8930146830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:15.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.066+0000 7f892e800640 1 --2- 172.21.15.167:0/2176144266 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f89300b3ea0 0x7f8930146830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:15.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.066+0000 7f892e800640 1 -- 172.21.15.167:0/2176144266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8930157750 msgr2=0x7f89300aae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:15.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.066+0000 7f892e800640 1 --2- 172.21.15.167:0/2176144266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8930157750 0x7f89300aae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:15.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.066+0000 7f892e800640 1 -- 172.21.15.167:0/2176144266 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8930158d30 con 0x7f89301619d0 2024-05-11T12:13:15.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.066+0000 7f892e800640 1 --2- 172.21.15.167:0/2176144266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89301619d0 0x7f89300ab360 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f892800b9e0 tx=0x7f892800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:15.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.067+0000 7f8926000640 1 -- 172.21.15.167:0/2176144266 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8928003b70 con 0x7f89301619d0 2024-05-11T12:13:15.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.067+0000 7f8926000640 1 -- 172.21.15.167:0/2176144266 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8928003d10 con 0x7f89301619d0 2024-05-11T12:13:15.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.067+0000 7f8926000640 1 -- 172.21.15.167:0/2176144266 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892800cb00 con 0x7f89301619d0 2024-05-11T12:13:15.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.067+0000 7f8935200640 1 -- 172.21.15.167:0/2176144266 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8930159010 con 0x7f89301619d0 2024-05-11T12:13:15.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.067+0000 7f8935200640 1 -- 172.21.15.167:0/2176144266 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89301595d0 con 0x7f89301619d0 2024-05-11T12:13:15.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.068+0000 7f8935200640 1 -- 172.21.15.167:0/2176144266 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8930069590 con 0x7f89301619d0 2024-05-11T12:13:15.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.069+0000 7f8926000640 1 -- 172.21.15.167:0/2176144266 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f892800cca0 con 0x7f89301619d0 2024-05-11T12:13:15.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.070+0000 7f8926000640 1 --2- 172.21.15.167:0/2176144266 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f890c076300 0x7f890c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:15.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.070+0000 7f8926000640 1 -- 172.21.15.167:0/2176144266 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f8928097500 con 0x7f89301619d0 2024-05-11T12:13:15.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.070+0000 7f892de00640 1 --2- 172.21.15.167:0/2176144266 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f890c076300 0x7f890c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:15.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.071+0000 7f892de00640 1 --2- 172.21.15.167:0/2176144266 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f890c076300 0x7f890c0787c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f892000a990 tx=0x7f8920008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:15.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.075+0000 7f8926000640 1 -- 172.21.15.167:0/2176144266 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8928060cc0 con 0x7f89301619d0 2024-05-11T12:13:15.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.329+0000 7f8935200640 1 -- 172.21.15.167:0/2176144266 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f89300b1d30 con 0x7f89301619d0 2024-05-11T12:13:15.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.329+0000 7f8926000640 1 -- 172.21.15.167:0/2176144266 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v35) ==== 53+0+73 (secure 0 0 0) 0x7f89280605c0 con 0x7f89301619d0 2024-05-11T12:13:15.330 INFO:teuthology.orchestra.run.smithi167.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-11T12:13:15.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.332+0000 7f8935200640 1 -- 172.21.15.167:0/2176144266 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f890c076300 msgr2=0x7f890c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:15.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.332+0000 7f8935200640 1 --2- 172.21.15.167:0/2176144266 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f890c076300 0x7f890c0787c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f892000a990 tx=0x7f8920008040 comp rx=0 tx=0).stop 2024-05-11T12:13:15.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.332+0000 7f8935200640 1 -- 172.21.15.167:0/2176144266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89301619d0 msgr2=0x7f89300ab360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:15.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.332+0000 7f8935200640 1 --2- 172.21.15.167:0/2176144266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89301619d0 0x7f89300ab360 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f892800b9e0 tx=0x7f892800bea0 comp rx=0 tx=0).stop 2024-05-11T12:13:15.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.332+0000 7f8935200640 1 -- 172.21.15.167:0/2176144266 shutdown_connections 2024-05-11T12:13:15.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.332+0000 7f8935200640 1 --2- 172.21.15.167:0/2176144266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89301619d0 0x7f89300ab360 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:15.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.332+0000 7f8935200640 1 --2- 172.21.15.167:0/2176144266 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f890c076300 0x7f890c0787c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:15.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.332+0000 7f8935200640 1 --2- 172.21.15.167:0/2176144266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8930157750 0x7f89300aae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:15.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.332+0000 7f8935200640 1 --2- 172.21.15.167:0/2176144266 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f89300b3ea0 0x7f8930146830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:15.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.332+0000 7f8935200640 1 -- 172.21.15.167:0/2176144266 >> 172.21.15.167:0/2176144266 conn(0x7f89300cd8f0 msgr2=0x7f89300cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:15.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.332+0000 7f8935200640 1 -- 172.21.15.167:0/2176144266 shutdown_connections 2024-05-11T12:13:15.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15.332+0000 7f8935200640 1 -- 172.21.15.167:0/2176144266 wait complete. 2024-05-11T12:13:15.352 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-11T12:13:15.352 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-11T12:13:15.352 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi167.front.sepia.ceph.com 2024-05-11T12:13:15.352 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-11T12:13:15.352 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-11T12:13:15.352 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-11T12:13:15.352 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-11T12:13:15.352 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-11T12:13:15.352 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-11T12:13:15.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:13:15.352 DEBUG:teuthology.orchestra.run.smithi167:> ip addr 2024-05-11T12:13:15.404 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-11T12:13:15.404 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-11T12:13:15.405 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2024-05-11T12:13:15.405 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:13:15.405 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2024-05-11T12:13:15.405 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:13:15.405 INFO:teuthology.orchestra.run.smithi167.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:13:15.405 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e6 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:13:15.405 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f0 2024-05-11T12:13:15.405 INFO:teuthology.orchestra.run.smithi167.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:13:15.405 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e7 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:13:15.405 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f1 2024-05-11T12:13:15.405 INFO:teuthology.orchestra.run.smithi167.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:13:15.405 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:08 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:13:15.405 INFO:teuthology.orchestra.run.smithi167.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-11T12:13:15.405 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:09 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:13:15.406 INFO:teuthology.orchestra.run.smithi167.stdout: inet 172.21.15.167/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-11T12:13:15.406 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft 42474sec preferred_lft 42474sec 2024-05-11T12:13:15.406 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::ec4:7aff:fe88:7309/64 scope link 2024-05-11T12:13:15.406 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:13:15.406 INFO:teuthology.orchestra.run.smithi167.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-11T12:13:15.406 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 1a:a8:8d:b3:f4:32 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:13:15.406 INFO:teuthology.orchestra.run.smithi167.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-11T12:13:15.406 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:13:15.406 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::b017:48ff:fea0:7183/64 scope link 2024-05-11T12:13:15.406 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:13:15.406 INFO:teuthology.orchestra.run.smithi167.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-11T12:13:15.406 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 1a:a8:8d:b3:f4:32 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-11T12:13:15.406 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::18a8:8dff:feb3:f432/64 scope link 2024-05-11T12:13:15.406 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:13:15.406 INFO:teuthology.orchestra.run.smithi167.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-11T12:13:15.407 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 7e:79:1f:cb:d2:83 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-11T12:13:15.407 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::7c79:1fff:fecb:d283/64 scope link 2024-05-11T12:13:15.407 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:13:15.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:13:15.408 DEBUG:teuthology.orchestra.run.smithi167:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-11T12:13:15.484 INFO:teuthology.orchestra.run.smithi167.stdout:1 2024-05-11T12:13:15.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:13:15.488 DEBUG:teuthology.orchestra.run.smithi167:> route 2024-05-11T12:13:15.537 INFO:teuthology.orchestra.run.smithi167.stdout:Kernel IP routing table 2024-05-11T12:13:15.537 INFO:teuthology.orchestra.run.smithi167.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-11T12:13:15.538 INFO:teuthology.orchestra.run.smithi167.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-11T12:13:15.538 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-11T12:13:15.538 INFO:teuthology.orchestra.run.smithi167.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-11T12:13:15.538 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T12:13:15.538 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-05-11T12:13:15.538 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-11T12:13:15.538 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-11T12:13:15.538 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-11T12:13:15.538 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-05-11T12:13:15.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T12:13:15.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:13:15.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:13:15.746 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2024-05-11T12:13:15.795 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-11T12:13:15.795 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-11T12:13:15.796 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-11T12:13:15.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:13:15.796 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set brx.0 up 2024-05-11T12:13:15.880 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T12:13:15.880 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-11T12:13:15.935 INFO:teuthology.orchestra.run.smithi167.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-11T12:13:15.935 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T12:13:15.935 DEBUG:teuthology.orchestra.run.smithi167:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-11T12:13:15.987 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-05-11T12:13:15.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:13:15.987 DEBUG:teuthology.orchestra.run.smithi167:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:13:16.114 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:13:16.114 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-11T12:13:16.148 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:13:16.149 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-11T12:13:16.149 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.167:3300,172.21.15.204:3300,172.21.15.204:3301" 2024-05-11T12:13:16.149 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: trying mount with new device syntax: 0@5f32d344-7289-4c27-881b-1533419ab678.cephfs=/ 2024-05-11T12:13:16.149 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.167:3300/172.21.15.204:3300/172.21.15.204:3301" will pass to kernel 2024-05-11T12:13:16.151 INFO:teuthology.orchestra.run.smithi167.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-11T12:13:16.151 INFO:teuthology.orchestra.run.smithi167.stdout: You just mounted a file system that supports labels which does not 2024-05-11T12:13:16.151 INFO:teuthology.orchestra.run.smithi167.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-11T12:13:16.152 INFO:teuthology.orchestra.run.smithi167.stdout: applications will generate AVC messages and not be allowed access to 2024-05-11T12:13:16.152 INFO:teuthology.orchestra.run.smithi167.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-11T12:13:16.156 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-11T12:13:16.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:13:16.156 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-11T12:13:16.225 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-11T12:13:16.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-11T12:13:16.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:13:16.351 INFO:teuthology.orchestra.run.smithi167.stdout:client4787 2024-05-11T12:13:16.352 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:13:16.352 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:13:16.399 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-11T12:13:16.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.648+0000 7f22bba00640 1 Processor -- start 2024-05-11T12:13:16.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.648+0000 7f22bba00640 1 -- start start 2024-05-11T12:13:16.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.650+0000 7f22bba00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f22b40b3ea0 0x7f22b4161490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:16.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.650+0000 7f22bba00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f22b41619d0 0x7f22b406d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:16.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.650+0000 7f22bba00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22b415df30 con 0x7f22b4157750 2024-05-11T12:13:16.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.650+0000 7f22bba00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f22b4161000 con 0x7f22b41619d0 2024-05-11T12:13:16.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.650+0000 7f22bba00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f22b415e210 con 0x7f22b40b3ea0 2024-05-11T12:13:16.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.650+0000 7f22b9000640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f22b4157750 0x7f22b415d740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:48752/0 (socket says 172.21.15.167:48752) 2024-05-11T12:13:16.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.650+0000 7f22b9000640 1 -- 172.21.15.167:0/3973208014 learned_addr learned my addr 172.21.15.167:0/3973208014 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:13:16.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.650+0000 7f22b9a00640 1 --2- 172.21.15.167:0/3973208014 >> v2:172.21.15.204:3300/0 conn(0x7f22b41619d0 0x7f22b406d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:16.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.650+0000 7f22b3400640 1 -- 172.21.15.167:0/3973208014 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3251243886 0 0) 0x7f22b415df30 con 0x7f22b4157750 2024-05-11T12:13:16.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.651+0000 7f22b3400640 1 -- 172.21.15.167:0/3973208014 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f229c003520 con 0x7f22b4157750 2024-05-11T12:13:16.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.651+0000 7f22b3e00640 1 --2- 172.21.15.167:0/3973208014 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f22b40b3ea0 0x7f22b4161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:16.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.651+0000 7f22b3400640 1 -- 172.21.15.167:0/3973208014 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1964206918 0 0) 0x7f229c003520 con 0x7f22b4157750 2024-05-11T12:13:16.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.651+0000 7f22b3400640 1 -- 172.21.15.167:0/3973208014 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f22b40b3ea0 msgr2=0x7f22b4161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:16.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.651+0000 7f22b3400640 1 --2- 172.21.15.167:0/3973208014 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f22b40b3ea0 0x7f22b4161490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:16.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.651+0000 7f22b3400640 1 -- 172.21.15.167:0/3973208014 >> v2:172.21.15.204:3300/0 conn(0x7f22b41619d0 msgr2=0x7f22b406d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:16.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.651+0000 7f22b3400640 1 --2- 172.21.15.167:0/3973208014 >> v2:172.21.15.204:3300/0 conn(0x7f22b41619d0 0x7f22b406d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:16.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.651+0000 7f22b3400640 1 -- 172.21.15.167:0/3973208014 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22b415e4f0 con 0x7f22b4157750 2024-05-11T12:13:16.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.651+0000 7f22b3400640 1 -- 172.21.15.167:0/3973208014 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f22a80016f0 con 0x7f22b4157750 2024-05-11T12:13:16.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.651+0000 7f22b9a00640 1 --2- 172.21.15.167:0/3973208014 >> v2:172.21.15.204:3300/0 conn(0x7f22b41619d0 0x7f22b406d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:13:16.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.651+0000 7f22b3400640 1 -- 172.21.15.167:0/3973208014 >> v1:172.21.15.167:6789/0 conn(0x7f22b4157750 legacy=0x7f22b415d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:16.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.651+0000 7f22b3400640 1 --2- 172.21.15.167:0/3973208014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f229c003a00 0x7f229c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:16.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.651+0000 7f22b3400640 1 --2- 172.21.15.167:0/3973208014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f229c004800 0x7f229c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:16.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.651+0000 7f22b3400640 1 --2- 172.21.15.167:0/3973208014 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f22b40b3ea0 0x7f229c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:16.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.651+0000 7f22b3400640 1 -- 172.21.15.167:0/3973208014 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f22b4161000 con 0x7f229c003a00 2024-05-11T12:13:16.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.651+0000 7f22b3400640 1 -- 172.21.15.167:0/3973208014 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f22b415e210 con 0x7f229c004800 2024-05-11T12:13:16.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.651+0000 7f22b3400640 1 -- 172.21.15.167:0/3973208014 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f229c00bb20 con 0x7f22b40b3ea0 2024-05-11T12:13:16.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.651+0000 7f22b9000640 1 --2- 172.21.15.167:0/3973208014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f229c003a00 0x7f229c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:16.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.652+0000 7f22b3e00640 1 --2- 172.21.15.167:0/3973208014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f229c004800 0x7f229c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:16.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.652+0000 7f22b9a00640 1 --2- 172.21.15.167:0/3973208014 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f22b40b3ea0 0x7f229c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:16.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.652+0000 7f22b9000640 1 -- 172.21.15.167:0/3973208014 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f22b40b3ea0 msgr2=0x7f229c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:16.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.652+0000 7f22b9000640 1 --2- 172.21.15.167:0/3973208014 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f22b40b3ea0 0x7f229c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:16.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.652+0000 7f22b9000640 1 -- 172.21.15.167:0/3973208014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f229c004800 msgr2=0x7f229c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:16.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.652+0000 7f22b9000640 1 --2- 172.21.15.167:0/3973208014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f229c004800 0x7f229c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:16.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.652+0000 7f22b9000640 1 -- 172.21.15.167:0/3973208014 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f229c003520 con 0x7f229c003a00 2024-05-11T12:13:16.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.652+0000 7f22b3e00640 1 --2- 172.21.15.167:0/3973208014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f229c004800 0x7f229c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:16.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.653+0000 7f22b9000640 1 --2- 172.21.15.167:0/3973208014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f229c003a00 0x7f229c003e20 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f22a8007fd0 tx=0x7f229c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b27c50c93548f95f server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:16.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.653+0000 7f22b3400640 1 -- 172.21.15.167:0/3973208014 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a8007d50 con 0x7f229c003a00 2024-05-11T12:13:16.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.653+0000 7f22b3400640 1 -- 172.21.15.167:0/3973208014 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22a8005af0 con 0x7f229c003a00 2024-05-11T12:13:16.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.653+0000 7f22b3400640 1 -- 172.21.15.167:0/3973208014 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a8007070 con 0x7f229c003a00 2024-05-11T12:13:16.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.654+0000 7f22bba00640 1 -- 172.21.15.167:0/3973208014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f229c003a00 msgr2=0x7f229c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:16.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.654+0000 7f22bba00640 1 --2- 172.21.15.167:0/3973208014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f229c003a00 0x7f229c003e20 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f22a8007fd0 tx=0x7f229c005830 comp rx=0 tx=0).stop 2024-05-11T12:13:16.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.654+0000 7f22b9a00640 1 -- 172.21.15.167:0/3973208014 reap_dead start 2024-05-11T12:13:16.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.654+0000 7f22bba00640 1 -- 172.21.15.167:0/3973208014 shutdown_connections 2024-05-11T12:13:16.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.654+0000 7f22bba00640 1 -- 172.21.15.167:0/3973208014 >> 172.21.15.167:0/3973208014 conn(0x7f22b40cd8f0 msgr2=0x7f22b40cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:16.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.654+0000 7f22bba00640 1 -- 172.21.15.167:0/3973208014 shutdown_connections 2024-05-11T12:13:16.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.654+0000 7f22bba00640 1 -- 172.21.15.167:0/3973208014 wait complete. 2024-05-11T12:13:16.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.656+0000 7f22bba00640 1 Processor -- start 2024-05-11T12:13:16.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.656+0000 7f22bba00640 1 -- start start 2024-05-11T12:13:16.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.657+0000 7f22bba00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f229c003a00 0x7f22b4135350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:16.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.657+0000 7f22bba00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22b406cc10 0x7f22b406d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:16.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.657+0000 7f22bba00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22b4157da0 0x7f22b4158220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:16.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.657+0000 7f22bba00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f22b4161610 con 0x7f22b406cc10 2024-05-11T12:13:16.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.657+0000 7f22bba00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f22b4161490 con 0x7f22b4157da0 2024-05-11T12:13:16.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.657+0000 7f22bba00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f22b4161790 con 0x7f229c003a00 2024-05-11T12:13:16.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.657+0000 7f22b3e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22b406cc10 0x7f22b406d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:16.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.657+0000 7f22b3e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22b406cc10 0x7f22b406d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41566/0 (socket says 172.21.15.167:41566) 2024-05-11T12:13:16.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.657+0000 7f22b3e00640 1 -- 172.21.15.167:0/1071796684 learned_addr learned my addr 172.21.15.167:0/1071796684 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:16.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.657+0000 7f22b9000640 1 --2- 172.21.15.167:0/1071796684 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f229c003a00 0x7f22b4135350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:16.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.657+0000 7f22b9a00640 1 --2- 172.21.15.167:0/1071796684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22b4157da0 0x7f22b4158220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:16.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.657+0000 7f22b3e00640 1 -- 172.21.15.167:0/1071796684 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f229c003a00 msgr2=0x7f22b4135350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:16.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.657+0000 7f22b3e00640 1 --2- 172.21.15.167:0/1071796684 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f229c003a00 0x7f22b4135350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:16.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.657+0000 7f22b3e00640 1 -- 172.21.15.167:0/1071796684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22b4157da0 msgr2=0x7f22b4158220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:16.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.657+0000 7f22b3e00640 1 --2- 172.21.15.167:0/1071796684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22b4157da0 0x7f22b4158220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:16.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.657+0000 7f22b3e00640 1 -- 172.21.15.167:0/1071796684 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22b4159f20 con 0x7f22b406cc10 2024-05-11T12:13:16.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.657+0000 7f22b9a00640 1 --2- 172.21.15.167:0/1071796684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22b4157da0 0x7f22b4158220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:16.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.658+0000 7f22b3e00640 1 --2- 172.21.15.167:0/1071796684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22b406cc10 0x7f22b406d090 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f22a000e810 tx=0x7f22a000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:16.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.658+0000 7f22b0c00640 1 -- 172.21.15.167:0/1071796684 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a0017070 con 0x7f22b406cc10 2024-05-11T12:13:16.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.658+0000 7f22b0c00640 1 -- 172.21.15.167:0/1071796684 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22a000c6d0 con 0x7f22b406cc10 2024-05-11T12:13:16.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.658+0000 7f22b0c00640 1 -- 172.21.15.167:0/1071796684 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a000f040 con 0x7f22b406cc10 2024-05-11T12:13:16.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.658+0000 7f22bba00640 1 -- 172.21.15.167:0/1071796684 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22b4135890 con 0x7f22b406cc10 2024-05-11T12:13:16.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.658+0000 7f22bba00640 1 -- 172.21.15.167:0/1071796684 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22b415a1c0 con 0x7f22b406cc10 2024-05-11T12:13:16.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.659+0000 7f22bba00640 1 -- 172.21.15.167:0/1071796684 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22b4069590 con 0x7f22b406cc10 2024-05-11T12:13:16.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.660+0000 7f22b0c00640 1 -- 172.21.15.167:0/1071796684 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f22a001a070 con 0x7f22b406cc10 2024-05-11T12:13:16.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.661+0000 7f22b0c00640 1 --2- 172.21.15.167:0/1071796684 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2294076300 0x7f22940787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:16.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.661+0000 7f22b0c00640 1 -- 172.21.15.167:0/1071796684 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f22a00972c0 con 0x7f22b406cc10 2024-05-11T12:13:16.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.662+0000 7f22b9000640 1 --2- 172.21.15.167:0/1071796684 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2294076300 0x7f22940787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:16.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.662+0000 7f22b9000640 1 --2- 172.21.15.167:0/1071796684 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2294076300 0x7f22940787c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f22a8032000 tx=0x7f22a802d000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:16.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.666+0000 7f22b0c00640 1 -- 172.21.15.167:0/1071796684 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f22a0060a80 con 0x7f22b406cc10 2024-05-11T12:13:16.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.914+0000 7f22bba00640 1 -- 172.21.15.167:0/1071796684 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f22b40cf910 con 0x7f22b406cc10 2024-05-11T12:13:16.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.915+0000 7f22b0c00640 1 -- 172.21.15.167:0/1071796684 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v35) ==== 53+0+73 (secure 0 0 0) 0x7f22a0060380 con 0x7f22b406cc10 2024-05-11T12:13:16.915 INFO:teuthology.orchestra.run.smithi167.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-11T12:13:16.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.918+0000 7f22bba00640 1 -- 172.21.15.167:0/1071796684 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2294076300 msgr2=0x7f22940787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:16.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.918+0000 7f22bba00640 1 --2- 172.21.15.167:0/1071796684 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2294076300 0x7f22940787c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f22a8032000 tx=0x7f22a802d000 comp rx=0 tx=0).stop 2024-05-11T12:13:16.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.918+0000 7f22bba00640 1 -- 172.21.15.167:0/1071796684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22b406cc10 msgr2=0x7f22b406d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:16.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.918+0000 7f22bba00640 1 --2- 172.21.15.167:0/1071796684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22b406cc10 0x7f22b406d090 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f22a000e810 tx=0x7f22a000ecd0 comp rx=0 tx=0).stop 2024-05-11T12:13:16.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.918+0000 7f22bba00640 1 -- 172.21.15.167:0/1071796684 shutdown_connections 2024-05-11T12:13:16.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.918+0000 7f22bba00640 1 --2- 172.21.15.167:0/1071796684 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2294076300 0x7f22940787c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:16.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.918+0000 7f22bba00640 1 --2- 172.21.15.167:0/1071796684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22b4157da0 0x7f22b4158220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:16.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.918+0000 7f22bba00640 1 --2- 172.21.15.167:0/1071796684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22b406cc10 0x7f22b406d090 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:16.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.918+0000 7f22bba00640 1 --2- 172.21.15.167:0/1071796684 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f229c003a00 0x7f22b4135350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:16.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.918+0000 7f22bba00640 1 -- 172.21.15.167:0/1071796684 >> 172.21.15.167:0/1071796684 conn(0x7f22b40cd8f0 msgr2=0x7f22b40cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:16.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.918+0000 7f22bba00640 1 -- 172.21.15.167:0/1071796684 shutdown_connections 2024-05-11T12:13:16.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:16.918+0000 7f22bba00640 1 -- 172.21.15.167:0/1071796684 wait complete. 2024-05-11T12:13:16.937 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-11T12:13:16.937 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-11T12:13:16.937 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi204.front.sepia.ceph.com 2024-05-11T12:13:16.937 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-11T12:13:16.937 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-11T12:13:16.937 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-11T12:13:16.937 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-11T12:13:16.937 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-11T12:13:16.937 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-11T12:13:16.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:13:16.937 DEBUG:teuthology.orchestra.run.smithi204:> ip addr 2024-05-11T12:13:16.995 INFO:teuthology.orchestra.run.smithi204.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-11T12:13:16.995 INFO:teuthology.orchestra.run.smithi204.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-11T12:13:16.995 INFO:teuthology.orchestra.run.smithi204.stdout: inet 127.0.0.1/8 scope host lo 2024-05-11T12:13:16.995 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:13:16.995 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 ::1/128 scope host 2024-05-11T12:13:16.995 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:13:16.995 INFO:teuthology.orchestra.run.smithi204.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:13:16.995 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2a brd ff:ff:ff:ff:ff:ff 2024-05-11T12:13:16.996 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f0 2024-05-11T12:13:16.996 INFO:teuthology.orchestra.run.smithi204.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:13:16.996 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2b brd ff:ff:ff:ff:ff:ff 2024-05-11T12:13:16.996 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f1 2024-05-11T12:13:16.996 INFO:teuthology.orchestra.run.smithi204.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:13:16.996 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:aa brd ff:ff:ff:ff:ff:ff 2024-05-11T12:13:16.996 INFO:teuthology.orchestra.run.smithi204.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-11T12:13:16.996 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:ab brd ff:ff:ff:ff:ff:ff 2024-05-11T12:13:16.996 INFO:teuthology.orchestra.run.smithi204.stdout: inet 172.21.15.204/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-11T12:13:16.996 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft 42468sec preferred_lft 42468sec 2024-05-11T12:13:16.996 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::ec4:7aff:fe88:77ab/64 scope link 2024-05-11T12:13:16.996 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:13:16.996 INFO:teuthology.orchestra.run.smithi204.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-11T12:13:16.996 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether e2:9e:5b:47:45:5d brd ff:ff:ff:ff:ff:ff 2024-05-11T12:13:16.996 INFO:teuthology.orchestra.run.smithi204.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-11T12:13:16.996 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:13:16.997 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::a498:7aff:fe50:48c3/64 scope link 2024-05-11T12:13:16.997 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:13:16.997 INFO:teuthology.orchestra.run.smithi204.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-11T12:13:16.997 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether fa:0a:7f:e4:30:35 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-11T12:13:16.997 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::f80a:7fff:fee4:3035/64 scope link 2024-05-11T12:13:16.997 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:13:16.997 INFO:teuthology.orchestra.run.smithi204.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-11T12:13:16.997 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether e2:9e:5b:47:45:5d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-11T12:13:16.997 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::e09e:5bff:fe47:455d/64 scope link 2024-05-11T12:13:16.997 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:13:16.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:13:16.998 DEBUG:teuthology.orchestra.run.smithi204:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-11T12:13:17.074 INFO:teuthology.orchestra.run.smithi204.stdout:1 2024-05-11T12:13:17.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:13:17.079 DEBUG:teuthology.orchestra.run.smithi204:> route 2024-05-11T12:13:17.133 INFO:teuthology.orchestra.run.smithi204.stdout:Kernel IP routing table 2024-05-11T12:13:17.134 INFO:teuthology.orchestra.run.smithi204.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-11T12:13:17.134 INFO:teuthology.orchestra.run.smithi204.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-11T12:13:17.134 INFO:teuthology.orchestra.run.smithi204.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-11T12:13:17.134 INFO:teuthology.orchestra.run.smithi204.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-11T12:13:17.134 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T12:13:17.134 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-05-11T12:13:17.134 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-11T12:13:17.134 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-11T12:13:17.134 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-11T12:13:17.135 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-05-11T12:13:17.234 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:13:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T12:13:17.333 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:13:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:13:17.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:13:17.342 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list 2024-05-11T12:13:17.390 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-11T12:13:17.390 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-11T12:13:17.391 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-11T12:13:17.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:13:17.391 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link set brx.0 up 2024-05-11T12:13:17.470 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T12:13:17.470 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-11T12:13:17.525 INFO:teuthology.orchestra.run.smithi204.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-11T12:13:17.525 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T12:13:17.525 DEBUG:teuthology.orchestra.run.smithi204:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-11T12:13:17.577 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-05-11T12:13:17.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:13:17.577 DEBUG:teuthology.orchestra.run.smithi204:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:13:17.702 INFO:teuthology.orchestra.run.smithi204.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:13:17.703 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-11T12:13:17.736 INFO:teuthology.orchestra.run.smithi204.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:13:17.736 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-11T12:13:17.737 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: resolved to: "172.21.15.167:3300,172.21.15.204:3300,172.21.15.204:3301" 2024-05-11T12:13:17.737 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: trying mount with new device syntax: 1@5f32d344-7289-4c27-881b-1533419ab678.cephfs=/ 2024-05-11T12:13:17.737 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.167:3300/172.21.15.204:3300/172.21.15.204:3301" will pass to kernel 2024-05-11T12:13:17.739 INFO:teuthology.orchestra.run.smithi204.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-05-11T12:13:17.739 INFO:teuthology.orchestra.run.smithi204.stdout: You just mounted a file system that supports labels which does not 2024-05-11T12:13:17.739 INFO:teuthology.orchestra.run.smithi204.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-11T12:13:17.740 INFO:teuthology.orchestra.run.smithi204.stdout: applications will generate AVC messages and not be allowed access to 2024-05-11T12:13:17.740 INFO:teuthology.orchestra.run.smithi204.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-11T12:13:17.744 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-11T12:13:17.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:13:17.745 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-11T12:13:17.819 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-11T12:13:17.921 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:13:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-11T12:13:17.927 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:13:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:13:17.936 INFO:teuthology.orchestra.run.smithi204.stdout:client4990 2024-05-11T12:13:17.936 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:13:17.936 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:13:17.985 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-11T12:13:17.985 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-11T12:13:18.089 DEBUG:teuthology.orchestra.run.smithi204:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-11T12:13:18.205 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-11T12:13:18.205 INFO:tasks.cephfs.mount:Terminating background process 2024-05-11T12:13:18.205 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:13:18.206 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:13:18.262 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:13:18.262 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:13:18.316 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-11T12:13:18.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.523+0000 7f4ed8200640 1 Processor -- start 2024-05-11T12:13:18.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.524+0000 7f4ed8200640 1 -- start start 2024-05-11T12:13:18.525 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.525+0000 7f4ed8200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ed0145a00 0x7f4ed00afe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:18.525 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.525+0000 7f4ed8200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4ed00ae820 0x7f4ed006f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:18.525 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.525+0000 7f4ed8200640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ed00aed90 con 0x7f4ed00ceb00 2024-05-11T12:13:18.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.525+0000 7f4ed8200640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f4ed00af9a0 con 0x7f4ed00ae820 2024-05-11T12:13:18.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.525+0000 7f4ed8200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4ed00af070 con 0x7f4ed0145a00 2024-05-11T12:13:18.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.526+0000 7f4ed4e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4ed00ae820 0x7f4ed006f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:18.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.526+0000 7f4ed4e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4ed00ae820 0x7f4ed006f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41582/0 (socket says 172.21.15.167:41582) 2024-05-11T12:13:18.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.526+0000 7f4ed4e00640 1 -- 172.21.15.167:0/1337898295 learned_addr learned my addr 172.21.15.167:0/1337898295 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:18.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.526+0000 7f4ed5800640 1 --2- 172.21.15.167:0/1337898295 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ed0145a00 0x7f4ed00afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:18.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.526+0000 7f4ed4e00640 1 -- 172.21.15.167:0/1337898295 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ed0145a00 msgr2=0x7f4ed00afe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:18.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.526+0000 7f4ed4e00640 1 --2- 172.21.15.167:0/1337898295 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ed0145a00 0x7f4ed00afe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:18.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.526+0000 7f4ed4e00640 1 -- 172.21.15.167:0/1337898295 >> v1:172.21.15.167:6789/0 conn(0x7f4ed00ceb00 legacy=0x7f4ed0072230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:18.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.526+0000 7f4ed4e00640 1 -- 172.21.15.167:0/1337898295 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ed00af350 con 0x7f4ed00ae820 2024-05-11T12:13:18.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.526+0000 7f4ecfe00640 1 -- 172.21.15.167:0/1337898295 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3116816934 0 0) 0x7f4ed00aed90 con 0x7f4ed00ceb00 2024-05-11T12:13:18.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.526+0000 7f4ed5800640 1 --2- 172.21.15.167:0/1337898295 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ed0145a00 0x7f4ed00afe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:18.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.527+0000 7f4ed4e00640 1 --2- 172.21.15.167:0/1337898295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ed00ae820 0x7f4ed006f8e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc009870 tx=0x7f4ebc02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e045eef39d816512 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:18.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.528+0000 7f4ecfe00640 1 -- 172.21.15.167:0/1337898295 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc037070 con 0x7f4ed00ae820 2024-05-11T12:13:18.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.528+0000 7f4ecfe00640 1 -- 172.21.15.167:0/1337898295 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ebc032440 con 0x7f4ed00ae820 2024-05-11T12:13:18.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.528+0000 7f4ecfe00640 1 -- 172.21.15.167:0/1337898295 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc032810 con 0x7f4ed00ae820 2024-05-11T12:13:18.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.528+0000 7f4ed8200640 1 -- 172.21.15.167:0/1337898295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ed00ae820 msgr2=0x7f4ed006f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:18.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.528+0000 7f4ed8200640 1 --2- 172.21.15.167:0/1337898295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ed00ae820 0x7f4ed006f8e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc009870 tx=0x7f4ebc02a1c0 comp rx=0 tx=0).stop 2024-05-11T12:13:18.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.528+0000 7f4ed8200640 1 -- 172.21.15.167:0/1337898295 shutdown_connections 2024-05-11T12:13:18.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.528+0000 7f4ed8200640 1 --2- 172.21.15.167:0/1337898295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ed00ae820 0x7f4ed006f8e0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:18.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.528+0000 7f4ed8200640 1 --2- 172.21.15.167:0/1337898295 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ed0145a00 0x7f4ed00afe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:18.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.528+0000 7f4ed8200640 1 -- 172.21.15.167:0/1337898295 >> 172.21.15.167:0/1337898295 conn(0x7f4ed006c450 msgr2=0x7f4ed006c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:18.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.528+0000 7f4ed8200640 1 -- 172.21.15.167:0/1337898295 shutdown_connections 2024-05-11T12:13:18.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.528+0000 7f4ed8200640 1 -- 172.21.15.167:0/1337898295 wait complete. 2024-05-11T12:13:18.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.530+0000 7f4ed8200640 1 Processor -- start 2024-05-11T12:13:18.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.530+0000 7f4ed8200640 1 -- start start 2024-05-11T12:13:18.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.531+0000 7f4ed8200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ed00ae820 0x7f4ed00c7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:18.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.531+0000 7f4ed8200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ed00ceb00 0x7f4ed01f6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:18.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.531+0000 7f4ed8200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ed0145a00 0x7f4ed01406d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:18.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.531+0000 7f4ed8200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed006ffa0 con 0x7f4ed00ceb00 2024-05-11T12:13:18.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.531+0000 7f4ed8200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed006fe20 con 0x7f4ed0145a00 2024-05-11T12:13:18.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.531+0000 7f4ed8200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4ed00cef90 con 0x7f4ed00ae820 2024-05-11T12:13:18.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.532+0000 7f4ed4e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ed00ceb00 0x7f4ed01f6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:18.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.532+0000 7f4ed4e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ed00ceb00 0x7f4ed01f6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41592/0 (socket says 172.21.15.167:41592) 2024-05-11T12:13:18.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.532+0000 7f4ed4e00640 1 -- 172.21.15.167:0/4275306335 learned_addr learned my addr 172.21.15.167:0/4275306335 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:18.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.532+0000 7f4ed4e00640 1 -- 172.21.15.167:0/4275306335 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ed00ae820 msgr2=0x7f4ed00c7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:18.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.532+0000 7f4ed4e00640 1 --2- 172.21.15.167:0/4275306335 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ed00ae820 0x7f4ed00c7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:18.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.532+0000 7f4ed4e00640 1 -- 172.21.15.167:0/4275306335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ed0145a00 msgr2=0x7f4ed01406d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:18.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.532+0000 7f4ed4e00640 1 --2- 172.21.15.167:0/4275306335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ed0145a00 0x7f4ed01406d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:18.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.532+0000 7f4ed4e00640 1 -- 172.21.15.167:0/4275306335 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ed01f6640 con 0x7f4ed00ceb00 2024-05-11T12:13:18.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.532+0000 7f4ed4e00640 1 --2- 172.21.15.167:0/4275306335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ed00ceb00 0x7f4ed01f6100 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc000c00 tx=0x7f4ebc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:18.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.533+0000 7f4ecd600640 1 -- 172.21.15.167:0/4275306335 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc037040 con 0x7f4ed00ceb00 2024-05-11T12:13:18.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.533+0000 7f4ecd600640 1 -- 172.21.15.167:0/4275306335 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ebc042070 con 0x7f4ed00ceb00 2024-05-11T12:13:18.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.533+0000 7f4ecd600640 1 -- 172.21.15.167:0/4275306335 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc030bd0 con 0x7f4ed00ceb00 2024-05-11T12:13:18.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.533+0000 7f4ed8200640 1 -- 172.21.15.167:0/4275306335 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ed00c6180 con 0x7f4ed00ceb00 2024-05-11T12:13:18.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.533+0000 7f4ed8200640 1 -- 172.21.15.167:0/4275306335 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ed00c6740 con 0x7f4ed00ceb00 2024-05-11T12:13:18.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.534+0000 7f4ed8200640 1 -- 172.21.15.167:0/4275306335 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ed00b7010 con 0x7f4ed00ceb00 2024-05-11T12:13:18.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.535+0000 7f4ecd600640 1 -- 172.21.15.167:0/4275306335 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4ebc02ad10 con 0x7f4ed00ceb00 2024-05-11T12:13:18.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.536+0000 7f4ecd600640 1 --2- 172.21.15.167:0/4275306335 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4eb00762d0 0x7f4eb0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:18.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.536+0000 7f4ecd600640 1 -- 172.21.15.167:0/4275306335 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f4ebc0b80b0 con 0x7f4ed00ceb00 2024-05-11T12:13:18.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.536+0000 7f4ed5800640 1 --2- 172.21.15.167:0/4275306335 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4eb00762d0 0x7f4eb0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:18.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.537+0000 7f4ed5800640 1 --2- 172.21.15.167:0/4275306335 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4eb00762d0 0x7f4eb0078790 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4006fd0 tx=0x7f4ec4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:18.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.540+0000 7f4ecd600640 1 -- 172.21.15.167:0/4275306335 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4ed00b7010 con 0x7f4ed00ceb00 2024-05-11T12:13:18.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.765+0000 7f4ed8200640 1 -- 172.21.15.167:0/4275306335 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4ed00694b0 con 0x7f4ed00ceb00 2024-05-11T12:13:18.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.765+0000 7f4ecd600640 1 -- 172.21.15.167:0/4275306335 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v64) ==== 80+0+0 (secure 0 0 0) 0x7f4ebc081870 con 0x7f4ed00ceb00 2024-05-11T12:13:18.766 INFO:teuthology.orchestra.run.smithi167.stderr:listed 0 entries 2024-05-11T12:13:18.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.768+0000 7f4ed8200640 1 -- 172.21.15.167:0/4275306335 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4eb00762d0 msgr2=0x7f4eb0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:18.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.768+0000 7f4ed8200640 1 --2- 172.21.15.167:0/4275306335 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4eb00762d0 0x7f4eb0078790 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4006fd0 tx=0x7f4ec4008040 comp rx=0 tx=0).stop 2024-05-11T12:13:18.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.768+0000 7f4ed8200640 1 -- 172.21.15.167:0/4275306335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ed00ceb00 msgr2=0x7f4ed01f6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:18.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.768+0000 7f4ed8200640 1 --2- 172.21.15.167:0/4275306335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ed00ceb00 0x7f4ed01f6100 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc000c00 tx=0x7f4ebc0046c0 comp rx=0 tx=0).stop 2024-05-11T12:13:18.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.769+0000 7f4ed8200640 1 -- 172.21.15.167:0/4275306335 shutdown_connections 2024-05-11T12:13:18.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.769+0000 7f4ed8200640 1 --2- 172.21.15.167:0/4275306335 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4eb00762d0 0x7f4eb0078790 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:18.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.769+0000 7f4ed8200640 1 --2- 172.21.15.167:0/4275306335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ed0145a00 0x7f4ed01406d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:18.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.769+0000 7f4ed8200640 1 --2- 172.21.15.167:0/4275306335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ed00ceb00 0x7f4ed01f6100 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:18.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.769+0000 7f4ed8200640 1 --2- 172.21.15.167:0/4275306335 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ed00ae820 0x7f4ed00c7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:18.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.769+0000 7f4ed8200640 1 -- 172.21.15.167:0/4275306335 >> 172.21.15.167:0/4275306335 conn(0x7f4ed006c450 msgr2=0x7f4ed006d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:18.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.769+0000 7f4ed8200640 1 -- 172.21.15.167:0/4275306335 shutdown_connections 2024-05-11T12:13:18.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:18.769+0000 7f4ed8200640 1 -- 172.21.15.167:0/4275306335 wait complete. 2024-05-11T12:13:18.788 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-11T12:13:18.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:13:18.789 DEBUG:teuthology.orchestra.run.smithi167:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-11T12:13:18.874 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2024-05-11T12:13:18.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:13:18.874 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-11T12:13:18.924 INFO:tasks.cephfs.mount:Terminating background process 2024-05-11T12:13:18.924 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:13:18.925 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:13:18.983 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:13:18.983 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:13:19.040 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-11T12:13:19.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.257+0000 7ff976a00640 1 Processor -- start 2024-05-11T12:13:19.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.258+0000 7ff976a00640 1 -- start start 2024-05-11T12:13:19.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.259+0000 7ff976a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff97015bb20 0x7ff97006ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:19.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.259+0000 7ff976a00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff97006f230 0x7ff97015f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:19.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.259+0000 7ff976a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9700b3ed0 con 0x7ff970155bf0 2024-05-11T12:13:19.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.259+0000 7ff976a00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7ff97006f680 con 0x7ff97006f230 2024-05-11T12:13:19.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.259+0000 7ff976a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7ff9700b4ae0 con 0x7ff97015bb20 2024-05-11T12:13:19.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.260+0000 7ff976000640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff97006f230 0x7ff97015f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:19.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.260+0000 7ff974c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff97015bb20 0x7ff97006ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:19.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.260+0000 7ff976000640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff97006f230 0x7ff97015f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39350/0 (socket says 172.21.15.167:39350) 2024-05-11T12:13:19.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.260+0000 7ff976000640 1 -- 172.21.15.167:0/948582885 learned_addr learned my addr 172.21.15.167:0/948582885 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:19.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.260+0000 7ff976000640 1 -- 172.21.15.167:0/948582885 >> v2:172.21.15.204:3300/0 conn(0x7ff97015bb20 msgr2=0x7ff97006ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:19.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.260+0000 7ff976000640 1 --2- 172.21.15.167:0/948582885 >> v2:172.21.15.204:3300/0 conn(0x7ff97015bb20 0x7ff97006ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:19.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.260+0000 7ff976000640 1 -- 172.21.15.167:0/948582885 >> v1:172.21.15.204:6789/0 conn(0x7ff970155bf0 legacy=0x7ff970073310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:19.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.260+0000 7ff976000640 1 -- 172.21.15.167:0/948582885 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9700b41b0 con 0x7ff97006f230 2024-05-11T12:13:19.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.261+0000 7ff976000640 1 --2- 172.21.15.167:0/948582885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff97006f230 0x7ff97015f240 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7ff970068e20 tx=0x7ff96802c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b04176847294d1c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:19.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.262+0000 7ff96fe00640 1 -- 172.21.15.167:0/948582885 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff968033640 con 0x7ff97006f230 2024-05-11T12:13:19.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.262+0000 7ff96fe00640 1 -- 172.21.15.167:0/948582885 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff968038070 con 0x7ff97006f230 2024-05-11T12:13:19.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.262+0000 7ff96fe00640 1 -- 172.21.15.167:0/948582885 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff968033a00 con 0x7ff97006f230 2024-05-11T12:13:19.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.262+0000 7ff976a00640 1 -- 172.21.15.167:0/948582885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff97006f230 msgr2=0x7ff97015f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:19.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.262+0000 7ff976a00640 1 --2- 172.21.15.167:0/948582885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff97006f230 0x7ff97015f240 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7ff970068e20 tx=0x7ff96802c8b0 comp rx=0 tx=0).stop 2024-05-11T12:13:19.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.263+0000 7ff976a00640 1 -- 172.21.15.167:0/948582885 shutdown_connections 2024-05-11T12:13:19.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.263+0000 7ff976a00640 1 --2- 172.21.15.167:0/948582885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff97006f230 0x7ff97015f240 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:19.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.263+0000 7ff976a00640 1 --2- 172.21.15.167:0/948582885 >> v2:172.21.15.204:3300/0 conn(0x7ff97015bb20 0x7ff97006ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:19.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.263+0000 7ff976a00640 1 -- 172.21.15.167:0/948582885 >> 172.21.15.167:0/948582885 conn(0x7ff97006c450 msgr2=0x7ff97006c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:19.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.263+0000 7ff976a00640 1 -- 172.21.15.167:0/948582885 shutdown_connections 2024-05-11T12:13:19.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.263+0000 7ff976a00640 1 -- 172.21.15.167:0/948582885 wait complete. 2024-05-11T12:13:19.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.264+0000 7ff976a00640 1 Processor -- start 2024-05-11T12:13:19.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.264+0000 7ff976a00640 1 -- start start 2024-05-11T12:13:19.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.265+0000 7ff976a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff97006f230 0x7ff9701f5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:19.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.265+0000 7ff976a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff970155bf0 0x7ff970152830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:19.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.265+0000 7ff976a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff97015bb20 0x7ff970154920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:19.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.265+0000 7ff976a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff970155fd0 con 0x7ff97006f230 2024-05-11T12:13:19.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.265+0000 7ff976a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff9700b4770 con 0x7ff97015bb20 2024-05-11T12:13:19.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.265+0000 7ff976a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff970073fd0 con 0x7ff970155bf0 2024-05-11T12:13:19.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.266+0000 7ff975600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff97006f230 0x7ff9701f5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:19.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.266+0000 7ff975600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff97006f230 0x7ff9701f5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39352/0 (socket says 172.21.15.167:39352) 2024-05-11T12:13:19.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.266+0000 7ff975600640 1 -- 172.21.15.167:0/244448935 learned_addr learned my addr 172.21.15.167:0/244448935 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:19.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.266+0000 7ff976000640 1 --2- 172.21.15.167:0/244448935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff97015bb20 0x7ff970154920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:19.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.266+0000 7ff975600640 1 -- 172.21.15.167:0/244448935 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff970155bf0 msgr2=0x7ff970152830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:19.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.266+0000 7ff975600640 1 --2- 172.21.15.167:0/244448935 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff970155bf0 0x7ff970152830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:19.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.266+0000 7ff975600640 1 -- 172.21.15.167:0/244448935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff97015bb20 msgr2=0x7ff970154920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:19.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.266+0000 7ff975600640 1 --2- 172.21.15.167:0/244448935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff97015bb20 0x7ff970154920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:19.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.266+0000 7ff975600640 1 -- 172.21.15.167:0/244448935 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9701f8b60 con 0x7ff97006f230 2024-05-11T12:13:19.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.267+0000 7ff975600640 1 --2- 172.21.15.167:0/244448935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff97006f230 0x7ff9701f5be0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ff96000c8a0 tx=0x7ff96000cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:19.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.267+0000 7ff96d600640 1 -- 172.21.15.167:0/244448935 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff960003b70 con 0x7ff97006f230 2024-05-11T12:13:19.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.267+0000 7ff96d600640 1 -- 172.21.15.167:0/244448935 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff960003d10 con 0x7ff97006f230 2024-05-11T12:13:19.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.267+0000 7ff96d600640 1 -- 172.21.15.167:0/244448935 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96000aa70 con 0x7ff97006f230 2024-05-11T12:13:19.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.267+0000 7ff976a00640 1 -- 172.21.15.167:0/244448935 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9701f8e40 con 0x7ff97006f230 2024-05-11T12:13:19.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.267+0000 7ff976a00640 1 -- 172.21.15.167:0/244448935 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9701f9400 con 0x7ff97006f230 2024-05-11T12:13:19.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.268+0000 7ff976a00640 1 -- 172.21.15.167:0/244448935 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9700694b0 con 0x7ff97006f230 2024-05-11T12:13:19.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.269+0000 7ff96d600640 1 -- 172.21.15.167:0/244448935 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff96000ac10 con 0x7ff97006f230 2024-05-11T12:13:19.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.271+0000 7ff96d600640 1 --2- 172.21.15.167:0/244448935 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff9500762d0 0x7ff950078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:19.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.271+0000 7ff96d600640 1 -- 172.21.15.167:0/244448935 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7ff960097750 con 0x7ff97006f230 2024-05-11T12:13:19.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.271+0000 7ff974c00640 1 --2- 172.21.15.167:0/244448935 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff9500762d0 0x7ff950078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:19.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.272+0000 7ff974c00640 1 --2- 172.21.15.167:0/244448935 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff9500762d0 0x7ff950078790 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff95c006a90 tx=0x7ff95c006a20 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:19.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.276+0000 7ff96d600640 1 -- 172.21.15.167:0/244448935 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff960060f10 con 0x7ff97006f230 2024-05-11T12:13:19.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.499+0000 7ff976a00640 1 -- 172.21.15.167:0/244448935 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff9700cd6a0 con 0x7ff97006f230 2024-05-11T12:13:19.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.500+0000 7ff96d600640 1 -- 172.21.15.167:0/244448935 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v64) ==== 80+0+0 (secure 0 0 0) 0x7ff960060810 con 0x7ff97006f230 2024-05-11T12:13:19.500 INFO:teuthology.orchestra.run.smithi167.stderr:listed 0 entries 2024-05-11T12:13:19.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.503+0000 7ff976a00640 1 -- 172.21.15.167:0/244448935 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff9500762d0 msgr2=0x7ff950078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:19.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.503+0000 7ff976a00640 1 --2- 172.21.15.167:0/244448935 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff9500762d0 0x7ff950078790 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff95c006a90 tx=0x7ff95c006a20 comp rx=0 tx=0).stop 2024-05-11T12:13:19.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.503+0000 7ff976a00640 1 -- 172.21.15.167:0/244448935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff97006f230 msgr2=0x7ff9701f5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:19.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.503+0000 7ff976a00640 1 --2- 172.21.15.167:0/244448935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff97006f230 0x7ff9701f5be0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ff96000c8a0 tx=0x7ff96000cd60 comp rx=0 tx=0).stop 2024-05-11T12:13:19.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.503+0000 7ff976a00640 1 -- 172.21.15.167:0/244448935 shutdown_connections 2024-05-11T12:13:19.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.503+0000 7ff976a00640 1 --2- 172.21.15.167:0/244448935 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff9500762d0 0x7ff950078790 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:19.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.503+0000 7ff976a00640 1 --2- 172.21.15.167:0/244448935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff97015bb20 0x7ff970154920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:19.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.503+0000 7ff976a00640 1 --2- 172.21.15.167:0/244448935 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff970155bf0 0x7ff970152830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:19.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.503+0000 7ff976a00640 1 --2- 172.21.15.167:0/244448935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff97006f230 0x7ff9701f5be0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:19.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.503+0000 7ff976a00640 1 -- 172.21.15.167:0/244448935 >> 172.21.15.167:0/244448935 conn(0x7ff97006c450 msgr2=0x7ff970071ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:19.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.503+0000 7ff976a00640 1 -- 172.21.15.167:0/244448935 shutdown_connections 2024-05-11T12:13:19.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:19.503+0000 7ff976a00640 1 -- 172.21.15.167:0/244448935 wait complete. 2024-05-11T12:13:19.523 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-11T12:13:19.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:13:19.524 DEBUG:teuthology.orchestra.run.smithi204:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-05-11T12:13:19.626 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi204.front.sepia.ceph.com 2024-05-11T12:13:19.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:13:19.626 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-11T12:13:19.683 INFO:tasks.cephfs.mount:Terminating background process 2024-05-11T12:13:19.683 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:13:19.683 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:13:19.739 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-11T12:13:19.739 INFO:tasks.cephfs.mount:Terminating background process 2024-05-11T12:13:19.739 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:13:19.739 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:13:19.795 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052713,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-11T12:13:19.796 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:13:20.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.007+0000 7f5646600640 1 Processor -- start 2024-05-11T12:13:20.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.007+0000 7f5646600640 1 -- start start 2024-05-11T12:13:20.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.009+0000 7f5646600640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5640157750 0x7f564015d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:20.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.009+0000 7f5646600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f564006d460 0x7f564006d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:20.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.009+0000 7f5646600640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f564015dc80 con 0x7f56400b3ea0 2024-05-11T12:13:20.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.009+0000 7f5646600640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f5640161000 con 0x7f5640157750 2024-05-11T12:13:20.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.009+0000 7f5646600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f564006ddd0 con 0x7f564006d460 2024-05-11T12:13:20.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.009+0000 7f563f400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5640157750 0x7f564015d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:20.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.009+0000 7f563ea00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f56400b3ea0 0x7f5640161490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:37316/0 (socket says 172.21.15.167:37316) 2024-05-11T12:13:20.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.009+0000 7f563f400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5640157750 0x7f564015d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39358/0 (socket says 172.21.15.167:39358) 2024-05-11T12:13:20.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.009+0000 7f563ea00640 1 -- 172.21.15.167:0/525551342 learned_addr learned my addr 172.21.15.167:0/525551342 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:13:20.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.010+0000 7f563fe00640 1 --2- 172.21.15.167:0/525551342 >> v2:172.21.15.204:3300/0 conn(0x7f564006d460 0x7f564006d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:20.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.010+0000 7f563f400640 1 -- 172.21.15.167:0/525551342 >> v2:172.21.15.204:3300/0 conn(0x7f564006d460 msgr2=0x7f564006d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:20.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.010+0000 7f563f400640 1 --2- 172.21.15.167:0/525551342 >> v2:172.21.15.204:3300/0 conn(0x7f564006d460 0x7f564006d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:20.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.010+0000 7f563f400640 1 -- 172.21.15.167:0/525551342 >> v1:172.21.15.167:6789/0 conn(0x7f56400b3ea0 legacy=0x7f5640161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:20.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.010+0000 7f563f400640 1 -- 172.21.15.167:0/525551342 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f564015df30 con 0x7f5640157750 2024-05-11T12:13:20.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.010+0000 7f563e000640 1 -- 172.21.15.167:0/525551342 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3000201876 0 0) 0x7f564015dc80 con 0x7f56400b3ea0 2024-05-11T12:13:20.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.011+0000 7f563f400640 1 --2- 172.21.15.167:0/525551342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5640157750 0x7f564015d740 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f56400b0aa0 tx=0x7f563002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb42fe0f4995e8e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:20.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.011+0000 7f563e000640 1 -- 172.21.15.167:0/525551342 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5630037070 con 0x7f5640157750 2024-05-11T12:13:20.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.011+0000 7f563e000640 1 -- 172.21.15.167:0/525551342 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5630032440 con 0x7f5640157750 2024-05-11T12:13:20.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.011+0000 7f563e000640 1 -- 172.21.15.167:0/525551342 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5630032810 con 0x7f5640157750 2024-05-11T12:13:20.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.012+0000 7f5646600640 1 -- 172.21.15.167:0/525551342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5640157750 msgr2=0x7f564015d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:20.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.012+0000 7f5646600640 1 --2- 172.21.15.167:0/525551342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5640157750 0x7f564015d740 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f56400b0aa0 tx=0x7f563002a1a0 comp rx=0 tx=0).stop 2024-05-11T12:13:20.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.012+0000 7f5646600640 1 -- 172.21.15.167:0/525551342 shutdown_connections 2024-05-11T12:13:20.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.012+0000 7f5646600640 1 --2- 172.21.15.167:0/525551342 >> v2:172.21.15.204:3300/0 conn(0x7f564006d460 0x7f564006d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:20.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.012+0000 7f5646600640 1 --2- 172.21.15.167:0/525551342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5640157750 0x7f564015d740 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:20.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.012+0000 7f5646600640 1 -- 172.21.15.167:0/525551342 >> 172.21.15.167:0/525551342 conn(0x7f56400cd8f0 msgr2=0x7f56400cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:20.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.012+0000 7f5646600640 1 -- 172.21.15.167:0/525551342 shutdown_connections 2024-05-11T12:13:20.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.012+0000 7f5646600640 1 -- 172.21.15.167:0/525551342 wait complete. 2024-05-11T12:13:20.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.014+0000 7f5646600640 1 Processor -- start 2024-05-11T12:13:20.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.014+0000 7f5646600640 1 -- start start 2024-05-11T12:13:20.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.015+0000 7f5646600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f564006d460 0x7f5640144550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:20.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.015+0000 7f5646600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f56400b3ea0 0x7f5640158d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:20.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.015+0000 7f5646600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5640157750 0x7f564015b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:20.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.015+0000 7f5646600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5640157b30 con 0x7f5640157750 2024-05-11T12:13:20.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.015+0000 7f5646600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f564015e4f0 con 0x7f564006d460 2024-05-11T12:13:20.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.015+0000 7f5646600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5640161ba0 con 0x7f56400b3ea0 2024-05-11T12:13:20.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.015+0000 7f563fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5640157750 0x7f564015b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:20.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.015+0000 7f563fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5640157750 0x7f564015b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39368/0 (socket says 172.21.15.167:39368) 2024-05-11T12:13:20.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.015+0000 7f563fe00640 1 -- 172.21.15.167:0/3796455316 learned_addr learned my addr 172.21.15.167:0/3796455316 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:20.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.016+0000 7f563ea00640 1 --2- 172.21.15.167:0/3796455316 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f56400b3ea0 0x7f5640158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:20.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.016+0000 7f563f400640 1 --2- 172.21.15.167:0/3796455316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f564006d460 0x7f5640144550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:20.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.016+0000 7f563fe00640 1 -- 172.21.15.167:0/3796455316 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f56400b3ea0 msgr2=0x7f5640158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:20.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.016+0000 7f563fe00640 1 --2- 172.21.15.167:0/3796455316 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f56400b3ea0 0x7f5640158d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:20.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.016+0000 7f563fe00640 1 -- 172.21.15.167:0/3796455316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f564006d460 msgr2=0x7f5640144550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:20.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.016+0000 7f563fe00640 1 --2- 172.21.15.167:0/3796455316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f564006d460 0x7f5640144550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:20.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.016+0000 7f563fe00640 1 -- 172.21.15.167:0/3796455316 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5640159700 con 0x7f5640157750 2024-05-11T12:13:20.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.016+0000 7f563f400640 1 --2- 172.21.15.167:0/3796455316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f564006d460 0x7f5640144550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:20.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.016+0000 7f563fe00640 1 --2- 172.21.15.167:0/3796455316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5640157750 0x7f564015b920 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f563800cc30 tx=0x7f563800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:20.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.016+0000 7f5637400640 1 -- 172.21.15.167:0/3796455316 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f563800ce40 con 0x7f5640157750 2024-05-11T12:13:20.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.016+0000 7f5637400640 1 -- 172.21.15.167:0/3796455316 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5638003990 con 0x7f5640157750 2024-05-11T12:13:20.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.016+0000 7f5646600640 1 -- 172.21.15.167:0/3796455316 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56401599e0 con 0x7f5640157750 2024-05-11T12:13:20.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.016+0000 7f5637400640 1 -- 172.21.15.167:0/3796455316 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5638003d40 con 0x7f5640157750 2024-05-11T12:13:20.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.017+0000 7f5646600640 1 -- 172.21.15.167:0/3796455316 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5640159fa0 con 0x7f5640157750 2024-05-11T12:13:20.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.018+0000 7f5646600640 1 -- 172.21.15.167:0/3796455316 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5640069590 con 0x7f5640157750 2024-05-11T12:13:20.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.019+0000 7f5637400640 1 -- 172.21.15.167:0/3796455316 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5638003ee0 con 0x7f5640157750 2024-05-11T12:13:20.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.020+0000 7f5637400640 1 --2- 172.21.15.167:0/3796455316 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f561c076300 0x7f561c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:20.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.020+0000 7f5637400640 1 -- 172.21.15.167:0/3796455316 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f5638097cd0 con 0x7f5640157750 2024-05-11T12:13:20.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.020+0000 7f563f400640 1 --2- 172.21.15.167:0/3796455316 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f561c076300 0x7f561c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:20.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.021+0000 7f563f400640 1 --2- 172.21.15.167:0/3796455316 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f561c076300 0x7f561c0787c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5630034000 tx=0x7f563002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:20.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.025+0000 7f5637400640 1 -- 172.21.15.167:0/3796455316 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5638014030 con 0x7f5640157750 2024-05-11T12:13:20.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.268+0000 7f5646600640 1 -- 172.21.15.167:0/3796455316 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5640144a90 con 0x7f5640157750 2024-05-11T12:13:20.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.269+0000 7f5637400640 1 -- 172.21.15.167:0/3796455316 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5185 (secure 0 0 0) 0x7f5638061400 con 0x7f5640157750 2024-05-11T12:13:20.269 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:13:20.270 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/2785478956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2785478956},{"type":"v1","addr":"172.21.15.204:6834","nonce":2785478956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":4742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/3478043182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":3478043182},{"type":"v1","addr":"172.21.15.204:6835","nonce":3478043182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/618833945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":618833945},{"type":"v1","addr":"172.21.15.167:6836","nonce":618833945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:13:02.579882+0000","modified":"2024-05-11T12:13:05.652972+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5610":{"gid":5610,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/42694429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":42694429},{"type":"v1","addr":"172.21.15.167:6837","nonce":42694429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5610,"qdb_cluster":[5610]},"id":4}]} 2024-05-11T12:13:20.270 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 35 2024-05-11T12:13:20.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.272+0000 7f5646600640 1 -- 172.21.15.167:0/3796455316 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f561c076300 msgr2=0x7f561c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:20.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.272+0000 7f5646600640 1 --2- 172.21.15.167:0/3796455316 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f561c076300 0x7f561c0787c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5630034000 tx=0x7f563002e000 comp rx=0 tx=0).stop 2024-05-11T12:13:20.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.272+0000 7f5646600640 1 -- 172.21.15.167:0/3796455316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5640157750 msgr2=0x7f564015b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:20.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.272+0000 7f5646600640 1 --2- 172.21.15.167:0/3796455316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5640157750 0x7f564015b920 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f563800cc30 tx=0x7f563800a5c0 comp rx=0 tx=0).stop 2024-05-11T12:13:20.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.272+0000 7f5646600640 1 -- 172.21.15.167:0/3796455316 shutdown_connections 2024-05-11T12:13:20.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.272+0000 7f5646600640 1 --2- 172.21.15.167:0/3796455316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5640157750 0x7f564015b920 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:20.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.272+0000 7f5646600640 1 --2- 172.21.15.167:0/3796455316 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f56400b3ea0 0x7f5640158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:20.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.272+0000 7f5646600640 1 --2- 172.21.15.167:0/3796455316 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f561c076300 0x7f561c0787c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:20.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.272+0000 7f5646600640 1 --2- 172.21.15.167:0/3796455316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f564006d460 0x7f5640144550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:20.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.272+0000 7f5646600640 1 -- 172.21.15.167:0/3796455316 >> 172.21.15.167:0/3796455316 conn(0x7f56400cd8f0 msgr2=0x7f56400cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:20.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.272+0000 7f5646600640 1 -- 172.21.15.167:0/3796455316 shutdown_connections 2024-05-11T12:13:20.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.272+0000 7f5646600640 1 -- 172.21.15.167:0/3796455316 wait complete. 2024-05-11T12:13:20.291 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:13:20.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.493+0000 7fe3df200640 1 Processor -- start 2024-05-11T12:13:20.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.493+0000 7fe3df200640 1 -- start start 2024-05-11T12:13:20.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.495+0000 7fe3df200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe3e015f460 0x7fe3e00b83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:20.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.495+0000 7fe3df200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3e00b88e0 0x7fe3e00d3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:20.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.495+0000 7fe3df200640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3e00cf990 con 0x7fe3e006cc40 2024-05-11T12:13:20.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.495+0000 7fe3df200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fe3e00cfc70 con 0x7fe3e015f460 2024-05-11T12:13:20.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.495+0000 7fe3df200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe3e00cff50 con 0x7fe3e00b88e0 2024-05-11T12:13:20.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.495+0000 7fe3de800640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fe3e006cc40 0x7fe3e00d0a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:37320/0 (socket says 172.21.15.167:37320) 2024-05-11T12:13:20.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.495+0000 7fe3de800640 1 -- 172.21.15.167:0/699213195 learned_addr learned my addr 172.21.15.167:0/699213195 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:13:20.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.495+0000 7fe3dde00640 1 --2- 172.21.15.167:0/699213195 >> v2:172.21.15.204:3300/0 conn(0x7fe3e015f460 0x7fe3e00b83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:20.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.495+0000 7fe3dd400640 1 --2- 172.21.15.167:0/699213195 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3e00b88e0 0x7fe3e00d3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:20.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.495+0000 7fe3dca00640 1 -- 172.21.15.167:0/699213195 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 618576579 0 0) 0x7fe3e00cf990 con 0x7fe3e006cc40 2024-05-11T12:13:20.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.495+0000 7fe3dca00640 1 -- 172.21.15.167:0/699213195 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe3c4003520 con 0x7fe3e006cc40 2024-05-11T12:13:20.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.496+0000 7fe3dca00640 1 -- 172.21.15.167:0/699213195 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3641586863 0 0) 0x7fe3c4003520 con 0x7fe3e006cc40 2024-05-11T12:13:20.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.496+0000 7fe3dca00640 1 -- 172.21.15.167:0/699213195 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3e00b88e0 msgr2=0x7fe3e00d3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:20.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.496+0000 7fe3dca00640 1 --2- 172.21.15.167:0/699213195 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3e00b88e0 0x7fe3e00d3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:20.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.496+0000 7fe3dca00640 1 -- 172.21.15.167:0/699213195 >> v2:172.21.15.204:3300/0 conn(0x7fe3e015f460 msgr2=0x7fe3e00b83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:20.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.496+0000 7fe3dca00640 1 --2- 172.21.15.167:0/699213195 >> v2:172.21.15.204:3300/0 conn(0x7fe3e015f460 0x7fe3e00b83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:20.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.496+0000 7fe3dca00640 1 -- 172.21.15.167:0/699213195 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3e00d0230 con 0x7fe3e006cc40 2024-05-11T12:13:20.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.496+0000 7fe3dca00640 1 -- 172.21.15.167:0/699213195 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7fe3d8001a70 con 0x7fe3e006cc40 2024-05-11T12:13:20.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.496+0000 7fe3dde00640 1 --2- 172.21.15.167:0/699213195 >> v2:172.21.15.204:3300/0 conn(0x7fe3e015f460 0x7fe3e00b83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:13:20.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.496+0000 7fe3dca00640 1 -- 172.21.15.167:0/699213195 >> v1:172.21.15.167:6789/0 conn(0x7fe3e006cc40 legacy=0x7fe3e00d0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:20.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.496+0000 7fe3dca00640 1 --2- 172.21.15.167:0/699213195 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3e00b88e0 0x7fe3c4003a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:20.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.496+0000 7fe3dca00640 1 --2- 172.21.15.167:0/699213195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3c4004020 0x7fe3c4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:20.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.496+0000 7fe3dca00640 1 --2- 172.21.15.167:0/699213195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3c4004d40 0x7fe3c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:20.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.496+0000 7fe3dca00640 1 -- 172.21.15.167:0/699213195 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e00cfc70 con 0x7fe3c4004020 2024-05-11T12:13:20.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.496+0000 7fe3dca00640 1 -- 172.21.15.167:0/699213195 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e00cff50 con 0x7fe3c4004d40 2024-05-11T12:13:20.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.496+0000 7fe3dca00640 1 -- 172.21.15.167:0/699213195 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe3c400bb20 con 0x7fe3e00b88e0 2024-05-11T12:13:20.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.496+0000 7fe3dd400640 1 --2- 172.21.15.167:0/699213195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3c4004020 0x7fe3c4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:20.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.497+0000 7fe3dd400640 1 -- 172.21.15.167:0/699213195 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3e00b88e0 msgr2=0x7fe3c4003a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:20.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.497+0000 7fe3dd400640 1 --2- 172.21.15.167:0/699213195 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3e00b88e0 0x7fe3c4003a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:20.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.497+0000 7fe3dd400640 1 -- 172.21.15.167:0/699213195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3c4004d40 msgr2=0x7fe3c40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:20.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.497+0000 7fe3de800640 1 --2- 172.21.15.167:0/699213195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3c4004d40 0x7fe3c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:20.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.497+0000 7fe3dd400640 1 --2- 172.21.15.167:0/699213195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3c4004d40 0x7fe3c40051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:20.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.497+0000 7fe3dd400640 1 -- 172.21.15.167:0/699213195 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3c4003520 con 0x7fe3c4004020 2024-05-11T12:13:20.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.498+0000 7fe3dd400640 1 --2- 172.21.15.167:0/699213195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3c4004020 0x7fe3c4004800 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc009a90 tx=0x7fe3c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7904b65664d8552 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:20.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.498+0000 7fe3dca00640 1 -- 172.21.15.167:0/699213195 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc036070 con 0x7fe3c4004020 2024-05-11T12:13:20.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.498+0000 7fe3dca00640 1 -- 172.21.15.167:0/699213195 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3cc031400 con 0x7fe3c4004020 2024-05-11T12:13:20.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.498+0000 7fe3dca00640 1 -- 172.21.15.167:0/699213195 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc009690 con 0x7fe3c4004020 2024-05-11T12:13:20.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.498+0000 7fe3df200640 1 -- 172.21.15.167:0/699213195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3c4004020 msgr2=0x7fe3c4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:20.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.498+0000 7fe3df200640 1 --2- 172.21.15.167:0/699213195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3c4004020 0x7fe3c4004800 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc009a90 tx=0x7fe3c4005830 comp rx=0 tx=0).stop 2024-05-11T12:13:20.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.498+0000 7fe3de800640 1 -- 172.21.15.167:0/699213195 reap_dead start 2024-05-11T12:13:20.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.499+0000 7fe3df200640 1 -- 172.21.15.167:0/699213195 shutdown_connections 2024-05-11T12:13:20.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.499+0000 7fe3df200640 1 -- 172.21.15.167:0/699213195 >> 172.21.15.167:0/699213195 conn(0x7fe3e00d1a50 msgr2=0x7fe3e00d1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:20.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.499+0000 7fe3df200640 1 -- 172.21.15.167:0/699213195 shutdown_connections 2024-05-11T12:13:20.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.499+0000 7fe3df200640 1 -- 172.21.15.167:0/699213195 wait complete. 2024-05-11T12:13:20.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.500+0000 7fe3df200640 1 Processor -- start 2024-05-11T12:13:20.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.500+0000 7fe3df200640 1 -- start start 2024-05-11T12:13:20.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.501+0000 7fe3df200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3c4004020 0x7fe3e0157bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:20.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.501+0000 7fe3df200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3e00ca2e0 0x7fe3e01f8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:20.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.501+0000 7fe3df200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3e00ce760 0x7fe3e00cec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:20.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.501+0000 7fe3df200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e006db20 con 0x7fe3c4004020 2024-05-11T12:13:20.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.501+0000 7fe3df200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e00cf620 con 0x7fe3e00ca2e0 2024-05-11T12:13:20.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.501+0000 7fe3df200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe3e006dca0 con 0x7fe3e00ce760 2024-05-11T12:13:20.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.502+0000 7fe3dde00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3c4004020 0x7fe3e0157bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:20.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.502+0000 7fe3dde00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3c4004020 0x7fe3e0157bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39372/0 (socket says 172.21.15.167:39372) 2024-05-11T12:13:20.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.502+0000 7fe3dde00640 1 -- 172.21.15.167:0/3726717946 learned_addr learned my addr 172.21.15.167:0/3726717946 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:20.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.502+0000 7fe3dd400640 1 --2- 172.21.15.167:0/3726717946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3e00ca2e0 0x7fe3e01f8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:20.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.502+0000 7fe3dde00640 1 -- 172.21.15.167:0/3726717946 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3e00ce760 msgr2=0x7fe3e00cec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:20.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.502+0000 7fe3dde00640 1 --2- 172.21.15.167:0/3726717946 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3e00ce760 0x7fe3e00cec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:20.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.502+0000 7fe3dde00640 1 -- 172.21.15.167:0/3726717946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3e00ca2e0 msgr2=0x7fe3e01f8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:20.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.502+0000 7fe3dde00640 1 --2- 172.21.15.167:0/3726717946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3e00ca2e0 0x7fe3e01f8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:20.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.502+0000 7fe3dde00640 1 -- 172.21.15.167:0/3726717946 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3e0158480 con 0x7fe3c4004020 2024-05-11T12:13:20.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.502+0000 7fe3dde00640 1 --2- 172.21.15.167:0/3726717946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3c4004020 0x7fe3e0157bc0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fe3d000e7e0 tx=0x7fe3d000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:20.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.503+0000 7fe3d6000640 1 -- 172.21.15.167:0/3726717946 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d000f040 con 0x7fe3c4004020 2024-05-11T12:13:20.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.503+0000 7fe3d6000640 1 -- 172.21.15.167:0/3726717946 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3d0004230 con 0x7fe3c4004020 2024-05-11T12:13:20.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.503+0000 7fe3d6000640 1 -- 172.21.15.167:0/3726717946 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d000c6d0 con 0x7fe3c4004020 2024-05-11T12:13:20.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.503+0000 7fe3df200640 1 -- 172.21.15.167:0/3726717946 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3e0158760 con 0x7fe3c4004020 2024-05-11T12:13:20.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.503+0000 7fe3df200640 1 -- 172.21.15.167:0/3726717946 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3e0158d20 con 0x7fe3c4004020 2024-05-11T12:13:20.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.504+0000 7fe3df200640 1 -- 172.21.15.167:0/3726717946 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3e00694b0 con 0x7fe3c4004020 2024-05-11T12:13:20.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.505+0000 7fe3d6000640 1 -- 172.21.15.167:0/3726717946 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe3d000c870 con 0x7fe3c4004020 2024-05-11T12:13:20.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.506+0000 7fe3d6000640 1 --2- 172.21.15.167:0/3726717946 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe3c0076300 0x7fe3c00787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:20.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.507+0000 7fe3d6000640 1 -- 172.21.15.167:0/3726717946 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7fe3d0097400 con 0x7fe3c4004020 2024-05-11T12:13:20.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.507+0000 7fe3dd400640 1 --2- 172.21.15.167:0/3726717946 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe3c0076300 0x7fe3c00787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:20.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.507+0000 7fe3dd400640 1 --2- 172.21.15.167:0/3726717946 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe3c0076300 0x7fe3c00787c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc033000 tx=0x7fe3cc008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:20.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.511+0000 7fe3d6000640 1 -- 172.21.15.167:0/3726717946 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe3d0017020 con 0x7fe3c4004020 2024-05-11T12:13:20.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.756+0000 7fe3df200640 1 -- 172.21.15.167:0/3726717946 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe3e00c8450 con 0x7fe3c4004020 2024-05-11T12:13:20.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.757+0000 7fe3d6000640 1 -- 172.21.15.167:0/3726717946 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5185 (secure 0 0 0) 0x7fe3d0060c60 con 0x7fe3c4004020 2024-05-11T12:13:20.758 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:13:20.758 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/2785478956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2785478956},{"type":"v1","addr":"172.21.15.204:6834","nonce":2785478956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":4742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/3478043182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":3478043182},{"type":"v1","addr":"172.21.15.204:6835","nonce":3478043182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/618833945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":618833945},{"type":"v1","addr":"172.21.15.167:6836","nonce":618833945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:13:02.579882+0000","modified":"2024-05-11T12:13:05.652972+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5610":{"gid":5610,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/42694429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":42694429},{"type":"v1","addr":"172.21.15.167:6837","nonce":42694429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5610,"qdb_cluster":[5610]},"id":4}]} 2024-05-11T12:13:20.758 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 35 2024-05-11T12:13:20.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.760+0000 7fe3df200640 1 -- 172.21.15.167:0/3726717946 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe3c0076300 msgr2=0x7fe3c00787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:20.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.760+0000 7fe3df200640 1 --2- 172.21.15.167:0/3726717946 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe3c0076300 0x7fe3c00787c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc033000 tx=0x7fe3cc008030 comp rx=0 tx=0).stop 2024-05-11T12:13:20.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.760+0000 7fe3df200640 1 -- 172.21.15.167:0/3726717946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3c4004020 msgr2=0x7fe3e0157bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:20.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.760+0000 7fe3df200640 1 --2- 172.21.15.167:0/3726717946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3c4004020 0x7fe3e0157bc0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fe3d000e7e0 tx=0x7fe3d000eca0 comp rx=0 tx=0).stop 2024-05-11T12:13:20.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.760+0000 7fe3df200640 1 -- 172.21.15.167:0/3726717946 shutdown_connections 2024-05-11T12:13:20.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.760+0000 7fe3df200640 1 --2- 172.21.15.167:0/3726717946 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3e00ce760 0x7fe3e00cec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:20.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.760+0000 7fe3df200640 1 --2- 172.21.15.167:0/3726717946 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe3c0076300 0x7fe3c00787c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:20.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.760+0000 7fe3df200640 1 --2- 172.21.15.167:0/3726717946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3e00ca2e0 0x7fe3e01f8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:20.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.760+0000 7fe3df200640 1 --2- 172.21.15.167:0/3726717946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3c4004020 0x7fe3e0157bc0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:20.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.760+0000 7fe3df200640 1 -- 172.21.15.167:0/3726717946 >> 172.21.15.167:0/3726717946 conn(0x7fe3e00d1a50 msgr2=0x7fe3e00ba470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:20.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.761+0000 7fe3df200640 1 -- 172.21.15.167:0/3726717946 shutdown_connections 2024-05-11T12:13:20.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.761+0000 7fe3df200640 1 -- 172.21.15.167:0/3726717946 wait complete. 2024-05-11T12:13:20.780 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:13:20.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.969+0000 7f091c600640 1 Processor -- start 2024-05-11T12:13:20.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.969+0000 7f091c600640 1 -- start start 2024-05-11T12:13:20.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.970+0000 7f091c600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0914157750 0x7f091415d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:20.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.970+0000 7f091c600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f09140b3ea0 0x7f0914161490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:20.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.970+0000 7f091c600640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f09141619d0 0x7f091406d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:20.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.970+0000 7f091c600640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f0914161000 con 0x7f09141619d0 2024-05-11T12:13:20.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.970+0000 7f091c600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f091415dc80 con 0x7f09140b3ea0 2024-05-11T12:13:20.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.970+0000 7f091c600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f091415df30 con 0x7f0914157750 2024-05-11T12:13:20.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.971+0000 7f091a600640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f09141619d0 0x7f091406d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:20.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.971+0000 7f091a600640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f09141619d0 0x7f091406d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39376/0 (socket says 172.21.15.167:39376) 2024-05-11T12:13:20.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.971+0000 7f091a600640 1 -- 172.21.15.167:0/260731840 learned_addr learned my addr 172.21.15.167:0/260731840 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:20.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.971+0000 7f0919c00640 1 --2- 172.21.15.167:0/260731840 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0914157750 0x7f091415d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:20.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.971+0000 7f0919200640 1 --2- 172.21.15.167:0/260731840 >> v2:172.21.15.204:3300/0 conn(0x7f09140b3ea0 0x7f0914161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:20.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.971+0000 7f091a600640 1 -- 172.21.15.167:0/260731840 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0914157750 msgr2=0x7f091415d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:20.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.971+0000 7f091a600640 1 --2- 172.21.15.167:0/260731840 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0914157750 0x7f091415d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:20.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.971+0000 7f091a600640 1 -- 172.21.15.167:0/260731840 >> v2:172.21.15.204:3300/0 conn(0x7f09140b3ea0 msgr2=0x7f0914161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:20.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.971+0000 7f091a600640 1 --2- 172.21.15.167:0/260731840 >> v2:172.21.15.204:3300/0 conn(0x7f09140b3ea0 0x7f0914161490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:20.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.971+0000 7f091a600640 1 -- 172.21.15.167:0/260731840 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f091415e210 con 0x7f09141619d0 2024-05-11T12:13:20.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.972+0000 7f091a600640 1 --2- 172.21.15.167:0/260731840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09141619d0 0x7f091406d460 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f091406e0a0 tx=0x7f090c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e31c608d2bda02 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:20.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.972+0000 7f0913e00640 1 -- 172.21.15.167:0/260731840 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090c036070 con 0x7f09141619d0 2024-05-11T12:13:20.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.973+0000 7f0913e00640 1 -- 172.21.15.167:0/260731840 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f090c031600 con 0x7f09141619d0 2024-05-11T12:13:20.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.973+0000 7f0913e00640 1 -- 172.21.15.167:0/260731840 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090c0319d0 con 0x7f09141619d0 2024-05-11T12:13:20.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.973+0000 7f091c600640 1 -- 172.21.15.167:0/260731840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09141619d0 msgr2=0x7f091406d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:20.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.973+0000 7f091c600640 1 --2- 172.21.15.167:0/260731840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09141619d0 0x7f091406d460 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f091406e0a0 tx=0x7f090c02a6a0 comp rx=0 tx=0).stop 2024-05-11T12:13:20.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.973+0000 7f091c600640 1 -- 172.21.15.167:0/260731840 shutdown_connections 2024-05-11T12:13:20.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.973+0000 7f091c600640 1 --2- 172.21.15.167:0/260731840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09141619d0 0x7f091406d460 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:20.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.973+0000 7f091c600640 1 --2- 172.21.15.167:0/260731840 >> v2:172.21.15.204:3300/0 conn(0x7f09140b3ea0 0x7f0914161490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:20.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.973+0000 7f091c600640 1 --2- 172.21.15.167:0/260731840 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0914157750 0x7f091415d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:20.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.973+0000 7f091c600640 1 -- 172.21.15.167:0/260731840 >> 172.21.15.167:0/260731840 conn(0x7f09140cd8f0 msgr2=0x7f09140cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:20.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.973+0000 7f091c600640 1 -- 172.21.15.167:0/260731840 shutdown_connections 2024-05-11T12:13:20.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.973+0000 7f091c600640 1 -- 172.21.15.167:0/260731840 wait complete. 2024-05-11T12:13:20.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.975+0000 7f091c600640 1 Processor -- start 2024-05-11T12:13:20.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.975+0000 7f091c600640 1 -- start start 2024-05-11T12:13:20.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.976+0000 7f091c600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09140b3ea0 0x7f0914144090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:20.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.976+0000 7f091c600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0914157750 0x7f091415c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:20.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.976+0000 7f091c600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09141619d0 0x7f091415cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:20.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.976+0000 7f091c600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f091406d9a0 con 0x7f09140b3ea0 2024-05-11T12:13:20.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.976+0000 7f091c600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0914157b30 con 0x7f09141619d0 2024-05-11T12:13:20.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.976+0000 7f091c600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f091406db20 con 0x7f0914157750 2024-05-11T12:13:20.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.976+0000 7f0919c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09140b3ea0 0x7f0914144090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:20.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.976+0000 7f0919c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09140b3ea0 0x7f0914144090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39390/0 (socket says 172.21.15.167:39390) 2024-05-11T12:13:20.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.976+0000 7f0919c00640 1 -- 172.21.15.167:0/1222829778 learned_addr learned my addr 172.21.15.167:0/1222829778 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:20.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.977+0000 7f0919c00640 1 -- 172.21.15.167:0/1222829778 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0914157750 msgr2=0x7f091415c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:20.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.977+0000 7f0919c00640 1 --2- 172.21.15.167:0/1222829778 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0914157750 0x7f091415c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:20.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.977+0000 7f091a600640 1 --2- 172.21.15.167:0/1222829778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09141619d0 0x7f091415cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:20.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.977+0000 7f0919c00640 1 -- 172.21.15.167:0/1222829778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09141619d0 msgr2=0x7f091415cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:20.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.977+0000 7f0919c00640 1 --2- 172.21.15.167:0/1222829778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09141619d0 0x7f091415cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:20.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.977+0000 7f0919c00640 1 -- 172.21.15.167:0/1222829778 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09141f61c0 con 0x7f09140b3ea0 2024-05-11T12:13:20.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.977+0000 7f0919c00640 1 --2- 172.21.15.167:0/1222829778 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09140b3ea0 0x7f0914144090 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f090800d5f0 tx=0x7f090800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:20.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.977+0000 7f0911600640 1 -- 172.21.15.167:0/1222829778 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09080125f0 con 0x7f09140b3ea0 2024-05-11T12:13:20.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.977+0000 7f0911600640 1 -- 172.21.15.167:0/1222829778 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0908019070 con 0x7f09140b3ea0 2024-05-11T12:13:20.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.977+0000 7f0911600640 1 -- 172.21.15.167:0/1222829778 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09080129d0 con 0x7f09140b3ea0 2024-05-11T12:13:20.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.977+0000 7f091c600640 1 -- 172.21.15.167:0/1222829778 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09141f64a0 con 0x7f09140b3ea0 2024-05-11T12:13:20.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.978+0000 7f091c600640 1 -- 172.21.15.167:0/1222829778 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09141f6a60 con 0x7f09140b3ea0 2024-05-11T12:13:20.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.979+0000 7f091c600640 1 -- 172.21.15.167:0/1222829778 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0914069590 con 0x7f09140b3ea0 2024-05-11T12:13:20.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.980+0000 7f0911600640 1 -- 172.21.15.167:0/1222829778 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0908012b70 con 0x7f09140b3ea0 2024-05-11T12:13:20.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.981+0000 7f0911600640 1 --2- 172.21.15.167:0/1222829778 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f08f40762d0 0x7f08f4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:20.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.981+0000 7f0911600640 1 -- 172.21.15.167:0/1222829778 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f0908097bd0 con 0x7f09140b3ea0 2024-05-11T12:13:20.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.982+0000 7f0919200640 1 --2- 172.21.15.167:0/1222829778 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f08f40762d0 0x7f08f4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:20.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.982+0000 7f0919200640 1 --2- 172.21.15.167:0/1222829778 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f08f40762d0 0x7f08f4078790 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f09141598a0 tx=0x7f0900008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:20.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:20.986+0000 7f0911600640 1 -- 172.21.15.167:0/1222829778 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0908061390 con 0x7f09140b3ea0 2024-05-11T12:13:21.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.193+0000 7f091c600640 1 -- 172.21.15.167:0/1222829778 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f09141f6e40 con 0x7f09140b3ea0 2024-05-11T12:13:21.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.196+0000 7f0911600640 1 -- 172.21.15.167:0/1222829778 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) ==== 74+0+16932 (secure 0 0 0) 0x7f0908060c90 con 0x7f09140b3ea0 2024-05-11T12:13:21.196 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:13:21.196 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":64,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:13:04.144728+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:13:00.503965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-11T12:13:01.566330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:13:21.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.198+0000 7f091c600640 1 -- 172.21.15.167:0/1222829778 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f08f40762d0 msgr2=0x7f08f4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:21.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.198+0000 7f091c600640 1 --2- 172.21.15.167:0/1222829778 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f08f40762d0 0x7f08f4078790 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f09141598a0 tx=0x7f0900008040 comp rx=0 tx=0).stop 2024-05-11T12:13:21.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.198+0000 7f091c600640 1 -- 172.21.15.167:0/1222829778 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09140b3ea0 msgr2=0x7f0914144090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:21.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.198+0000 7f091c600640 1 --2- 172.21.15.167:0/1222829778 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09140b3ea0 0x7f0914144090 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f090800d5f0 tx=0x7f090800dab0 comp rx=0 tx=0).stop 2024-05-11T12:13:21.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.199+0000 7f091c600640 1 -- 172.21.15.167:0/1222829778 shutdown_connections 2024-05-11T12:13:21.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.199+0000 7f091c600640 1 --2- 172.21.15.167:0/1222829778 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f08f40762d0 0x7f08f4078790 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:21.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.199+0000 7f091c600640 1 --2- 172.21.15.167:0/1222829778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09141619d0 0x7f091415cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:21.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.199+0000 7f091c600640 1 --2- 172.21.15.167:0/1222829778 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0914157750 0x7f091415c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:21.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.199+0000 7f091c600640 1 --2- 172.21.15.167:0/1222829778 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09140b3ea0 0x7f0914144090 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:21.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.199+0000 7f091c600640 1 -- 172.21.15.167:0/1222829778 >> 172.21.15.167:0/1222829778 conn(0x7f09140cd8f0 msgr2=0x7f09140cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:21.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.199+0000 7f091c600640 1 -- 172.21.15.167:0/1222829778 shutdown_connections 2024-05-11T12:13:21.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.199+0000 7f091c600640 1 -- 172.21.15.167:0/1222829778 wait complete. 2024-05-11T12:13:21.217 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-11T12:13:21.218 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:13:21.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.417+0000 7f7980000640 1 Processor -- start 2024-05-11T12:13:21.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.417+0000 7f7980000640 1 -- start start 2024-05-11T12:13:21.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.419+0000 7f7980000640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f79780cd610 0x7f7978163be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:21.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.419+0000 7f7980000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79780cd9f0 0x7f7978160230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:21.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.419+0000 7f7980000640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79780b6300 con 0x7f7978160770 2024-05-11T12:13:21.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.419+0000 7f7980000640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f79780b65e0 con 0x7f79780cd610 2024-05-11T12:13:21.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.419+0000 7f7980000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f7978163100 con 0x7f79780cd9f0 2024-05-11T12:13:21.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.419+0000 7f797d600640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f79780cd610 0x7f7978163be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:21.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.419+0000 7f797d600640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f79780cd610 0x7f7978163be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39396/0 (socket says 172.21.15.167:39396) 2024-05-11T12:13:21.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.419+0000 7f797e000640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f7978160770 0x7f79780d0ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:37336/0 (socket says 172.21.15.167:37336) 2024-05-11T12:13:21.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.419+0000 7f797d600640 1 -- 172.21.15.167:0/776349579 learned_addr learned my addr 172.21.15.167:0/776349579 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:21.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.420+0000 7f797cc00640 1 --2- 172.21.15.167:0/776349579 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79780cd9f0 0x7f7978160230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:21.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.420+0000 7f797d600640 1 -- 172.21.15.167:0/776349579 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79780cd9f0 msgr2=0x7f7978160230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:21.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.420+0000 7f797d600640 1 --2- 172.21.15.167:0/776349579 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79780cd9f0 0x7f7978160230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:21.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.420+0000 7f797d600640 1 -- 172.21.15.167:0/776349579 >> v1:172.21.15.167:6789/0 conn(0x7f7978160770 legacy=0x7f79780d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:21.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.420+0000 7f797d600640 1 -- 172.21.15.167:0/776349579 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79781633e0 con 0x7f79780cd610 2024-05-11T12:13:21.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.420+0000 7f7977e00640 1 -- 172.21.15.167:0/776349579 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4236238620 0 0) 0x7f79780b6300 con 0x7f7978160770 2024-05-11T12:13:21.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.421+0000 7f797d600640 1 --2- 172.21.15.167:0/776349579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79780cd610 0x7f7978163be0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f796c009870 tx=0x7f796c02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5a18e920caba9e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:21.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.421+0000 7f7977e00640 1 -- 172.21.15.167:0/776349579 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796c037070 con 0x7f79780cd610 2024-05-11T12:13:21.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.421+0000 7f7977e00640 1 -- 172.21.15.167:0/776349579 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f796c032440 con 0x7f79780cd610 2024-05-11T12:13:21.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.421+0000 7f7977e00640 1 -- 172.21.15.167:0/776349579 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796c032810 con 0x7f79780cd610 2024-05-11T12:13:21.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.422+0000 7f7980000640 1 -- 172.21.15.167:0/776349579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79780cd610 msgr2=0x7f7978163be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:21.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.422+0000 7f7980000640 1 --2- 172.21.15.167:0/776349579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79780cd610 0x7f7978163be0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f796c009870 tx=0x7f796c02a250 comp rx=0 tx=0).stop 2024-05-11T12:13:21.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.422+0000 7f7980000640 1 -- 172.21.15.167:0/776349579 shutdown_connections 2024-05-11T12:13:21.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.422+0000 7f7980000640 1 --2- 172.21.15.167:0/776349579 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79780cd9f0 0x7f7978160230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:21.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.422+0000 7f7980000640 1 --2- 172.21.15.167:0/776349579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79780cd610 0x7f7978163be0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:21.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.422+0000 7f7980000640 1 -- 172.21.15.167:0/776349579 >> 172.21.15.167:0/776349579 conn(0x7f7978072730 msgr2=0x7f79780d2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:21.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.422+0000 7f7980000640 1 -- 172.21.15.167:0/776349579 shutdown_connections 2024-05-11T12:13:21.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.422+0000 7f7980000640 1 -- 172.21.15.167:0/776349579 wait complete. 2024-05-11T12:13:21.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.424+0000 7f7980000640 1 Processor -- start 2024-05-11T12:13:21.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.424+0000 7f7980000640 1 -- start start 2024-05-11T12:13:21.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.425+0000 7f7980000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79780cd610 0x7f79781f8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:21.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.425+0000 7f7980000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79780cd9f0 0x7f79780c6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:21.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.425+0000 7f7980000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7978160770 0x7f797814b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:21.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.425+0000 7f7980000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f79780cde50 con 0x7f79780cd9f0 2024-05-11T12:13:21.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.425+0000 7f7980000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f79780d2a80 con 0x7f7978160770 2024-05-11T12:13:21.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.425+0000 7f7980000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f79780b5e90 con 0x7f79780cd610 2024-05-11T12:13:21.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.425+0000 7f797cc00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79780cd9f0 0x7f79780c6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:21.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.425+0000 7f797cc00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79780cd9f0 0x7f79780c6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39400/0 (socket says 172.21.15.167:39400) 2024-05-11T12:13:21.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.425+0000 7f797cc00640 1 -- 172.21.15.167:0/2047419971 learned_addr learned my addr 172.21.15.167:0/2047419971 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:21.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.426+0000 7f797d600640 1 --2- 172.21.15.167:0/2047419971 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79780cd610 0x7f79781f8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:21.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.426+0000 7f797e000640 1 --2- 172.21.15.167:0/2047419971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7978160770 0x7f797814b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:21.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.426+0000 7f797cc00640 1 -- 172.21.15.167:0/2047419971 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79780cd610 msgr2=0x7f79781f8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:21.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.426+0000 7f797cc00640 1 --2- 172.21.15.167:0/2047419971 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79780cd610 0x7f79781f8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:21.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.426+0000 7f797cc00640 1 -- 172.21.15.167:0/2047419971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7978160770 msgr2=0x7f797814b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:21.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.426+0000 7f797cc00640 1 --2- 172.21.15.167:0/2047419971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7978160770 0x7f797814b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:21.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.426+0000 7f797cc00640 1 -- 172.21.15.167:0/2047419971 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79781f93c0 con 0x7f79780cd9f0 2024-05-11T12:13:21.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.426+0000 7f797d600640 1 --2- 172.21.15.167:0/2047419971 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79780cd610 0x7f79781f8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:21.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.426+0000 7f797cc00640 1 --2- 172.21.15.167:0/2047419971 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79780cd9f0 0x7f79780c6b60 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f796400d800 tx=0x7f796400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:21.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.426+0000 7f7975600640 1 -- 172.21.15.167:0/2047419971 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796400b6c0 con 0x7f79780cd9f0 2024-05-11T12:13:21.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.426+0000 7f7975600640 1 -- 172.21.15.167:0/2047419971 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7964019070 con 0x7f79780cd9f0 2024-05-11T12:13:21.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.426+0000 7f7975600640 1 -- 172.21.15.167:0/2047419971 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796400ba80 con 0x7f79780cd9f0 2024-05-11T12:13:21.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.426+0000 7f7980000640 1 -- 172.21.15.167:0/2047419971 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79781f96a0 con 0x7f79780cd9f0 2024-05-11T12:13:21.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.427+0000 7f7980000640 1 -- 172.21.15.167:0/2047419971 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f797814b920 con 0x7f79780cd9f0 2024-05-11T12:13:21.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.428+0000 7f7980000640 1 -- 172.21.15.167:0/2047419971 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79781f9bc0 con 0x7f79780cd9f0 2024-05-11T12:13:21.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.429+0000 7f7975600640 1 -- 172.21.15.167:0/2047419971 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f796400bc20 con 0x7f79780cd9f0 2024-05-11T12:13:21.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.430+0000 7f7975600640 1 --2- 172.21.15.167:0/2047419971 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f79580762d0 0x7f7958078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:21.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.430+0000 7f7975600640 1 -- 172.21.15.167:0/2047419971 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f7964097ba0 con 0x7f79780cd9f0 2024-05-11T12:13:21.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.430+0000 7f797d600640 1 --2- 172.21.15.167:0/2047419971 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f79580762d0 0x7f7958078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:21.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.431+0000 7f797d600640 1 --2- 172.21.15.167:0/2047419971 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f79580762d0 0x7f7958078790 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f796c034000 tx=0x7f796c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:21.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.435+0000 7f7975600640 1 -- 172.21.15.167:0/2047419971 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f79781f9bc0 con 0x7f79780cd9f0 2024-05-11T12:13:21.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.680+0000 7f7980000640 1 -- 172.21.15.167:0/2047419971 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79781f56f0 con 0x7f79780cd9f0 2024-05-11T12:13:21.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.682+0000 7f7975600640 1 -- 172.21.15.167:0/2047419971 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5185 (secure 0 0 0) 0x7f7964061400 con 0x7f79780cd9f0 2024-05-11T12:13:21.682 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:13:21.682 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/2785478956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2785478956},{"type":"v1","addr":"172.21.15.204:6834","nonce":2785478956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":4742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/3478043182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":3478043182},{"type":"v1","addr":"172.21.15.204:6835","nonce":3478043182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/618833945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":618833945},{"type":"v1","addr":"172.21.15.167:6836","nonce":618833945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:13:02.579882+0000","modified":"2024-05-11T12:13:05.652972+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5610":{"gid":5610,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/42694429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":42694429},{"type":"v1","addr":"172.21.15.167:6837","nonce":42694429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5610,"qdb_cluster":[5610]},"id":4}]} 2024-05-11T12:13:21.682 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 35 2024-05-11T12:13:21.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.685+0000 7f7980000640 1 -- 172.21.15.167:0/2047419971 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f79580762d0 msgr2=0x7f7958078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:21.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.685+0000 7f7980000640 1 --2- 172.21.15.167:0/2047419971 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f79580762d0 0x7f7958078790 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f796c034000 tx=0x7f796c02e000 comp rx=0 tx=0).stop 2024-05-11T12:13:21.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.685+0000 7f7980000640 1 -- 172.21.15.167:0/2047419971 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79780cd9f0 msgr2=0x7f79780c6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:21.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.685+0000 7f7980000640 1 --2- 172.21.15.167:0/2047419971 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79780cd9f0 0x7f79780c6b60 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f796400d800 tx=0x7f796400dcc0 comp rx=0 tx=0).stop 2024-05-11T12:13:21.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.685+0000 7f7980000640 1 -- 172.21.15.167:0/2047419971 shutdown_connections 2024-05-11T12:13:21.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.685+0000 7f7980000640 1 --2- 172.21.15.167:0/2047419971 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f79580762d0 0x7f7958078790 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:21.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.685+0000 7f7980000640 1 --2- 172.21.15.167:0/2047419971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7978160770 0x7f797814b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:21.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.685+0000 7f7980000640 1 --2- 172.21.15.167:0/2047419971 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79780cd9f0 0x7f79780c6b60 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:21.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.685+0000 7f7980000640 1 --2- 172.21.15.167:0/2047419971 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79780cd610 0x7f79781f8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:21.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.685+0000 7f7980000640 1 -- 172.21.15.167:0/2047419971 >> 172.21.15.167:0/2047419971 conn(0x7f7978072730 msgr2=0x7f79780b33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:21.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.685+0000 7f7980000640 1 -- 172.21.15.167:0/2047419971 shutdown_connections 2024-05-11T12:13:21.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.685+0000 7f7980000640 1 -- 172.21.15.167:0/2047419971 wait complete. 2024-05-11T12:13:21.704 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:13:21.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.899+0000 7fd330e00640 1 Processor -- start 2024-05-11T12:13:21.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.899+0000 7fd330e00640 1 -- start start 2024-05-11T12:13:21.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.900+0000 7fd330e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd32c1458f0 0x7fd32c06d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:21.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.900+0000 7fd330e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd32c0b3a80 0x7fd32c0b4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:21.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.900+0000 7fd330e00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd32c071f30 con 0x7fd32c0b5220 2024-05-11T12:13:21.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.900+0000 7fd330e00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fd32c06db80 con 0x7fd32c1458f0 2024-05-11T12:13:21.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.900+0000 7fd330e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd32c072210 con 0x7fd32c0b3a80 2024-05-11T12:13:21.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.901+0000 7fd32b400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd32c1458f0 0x7fd32c06d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:21.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.901+0000 7fd32be00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fd32c0b5220 0x7fd32c072fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:37350/0 (socket says 172.21.15.167:37350) 2024-05-11T12:13:21.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.901+0000 7fd32b400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd32c1458f0 0x7fd32c06d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39416/0 (socket says 172.21.15.167:39416) 2024-05-11T12:13:21.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.901+0000 7fd32be00640 1 -- 172.21.15.167:0/1764619726 learned_addr learned my addr 172.21.15.167:0/1764619726 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:13:21.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.901+0000 7fd32aa00640 1 --2- 172.21.15.167:0/1764619726 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd32c0b3a80 0x7fd32c0b4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:21.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.901+0000 7fd32b400640 1 -- 172.21.15.167:0/1764619726 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd32c0b3a80 msgr2=0x7fd32c0b4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:21.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.901+0000 7fd32b400640 1 --2- 172.21.15.167:0/1764619726 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd32c0b3a80 0x7fd32c0b4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:21.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.901+0000 7fd32b400640 1 -- 172.21.15.167:0/1764619726 >> v1:172.21.15.167:6789/0 conn(0x7fd32c0b5220 legacy=0x7fd32c072fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:21.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.901+0000 7fd32b400640 1 -- 172.21.15.167:0/1764619726 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd32c0724f0 con 0x7fd32c1458f0 2024-05-11T12:13:21.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.901+0000 7fd32a000640 1 -- 172.21.15.167:0/1764619726 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2854445093 0 0) 0x7fd32c071f30 con 0x7fd32c0b5220 2024-05-11T12:13:21.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.902+0000 7fd32b400640 1 --2- 172.21.15.167:0/1764619726 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd32c1458f0 0x7fd32c06d640 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd31c0097a0 tx=0x7fd31c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36b0296e742040d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:21.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.903+0000 7fd32a000640 1 -- 172.21.15.167:0/1764619726 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31c037070 con 0x7fd32c1458f0 2024-05-11T12:13:21.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.903+0000 7fd32a000640 1 -- 172.21.15.167:0/1764619726 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd31c032440 con 0x7fd32c1458f0 2024-05-11T12:13:21.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.903+0000 7fd32a000640 1 -- 172.21.15.167:0/1764619726 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31c032810 con 0x7fd32c1458f0 2024-05-11T12:13:21.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.903+0000 7fd330e00640 1 -- 172.21.15.167:0/1764619726 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd32c1458f0 msgr2=0x7fd32c06d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:21.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.903+0000 7fd330e00640 1 --2- 172.21.15.167:0/1764619726 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd32c1458f0 0x7fd32c06d640 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd31c0097a0 tx=0x7fd31c02a180 comp rx=0 tx=0).stop 2024-05-11T12:13:21.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.903+0000 7fd330e00640 1 -- 172.21.15.167:0/1764619726 shutdown_connections 2024-05-11T12:13:21.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.903+0000 7fd330e00640 1 --2- 172.21.15.167:0/1764619726 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd32c0b3a80 0x7fd32c0b4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:21.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.903+0000 7fd330e00640 1 --2- 172.21.15.167:0/1764619726 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd32c1458f0 0x7fd32c06d640 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:21.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.903+0000 7fd330e00640 1 -- 172.21.15.167:0/1764619726 >> 172.21.15.167:0/1764619726 conn(0x7fd32c0b3460 msgr2=0x7fd32c0d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:21.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.904+0000 7fd330e00640 1 -- 172.21.15.167:0/1764619726 shutdown_connections 2024-05-11T12:13:21.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.904+0000 7fd330e00640 1 -- 172.21.15.167:0/1764619726 wait complete. 2024-05-11T12:13:21.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.905+0000 7fd330e00640 1 Processor -- start 2024-05-11T12:13:21.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.905+0000 7fd330e00640 1 -- start start 2024-05-11T12:13:21.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.906+0000 7fd330e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd32c0b3a80 0x7fd32c144ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:21.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.906+0000 7fd330e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd32c0b5220 0x7fd32c137050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:21.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.906+0000 7fd330e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd32c1458f0 0x7fd32c15fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:21.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.906+0000 7fd330e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd32c0b4eb0 con 0x7fd32c0b5220 2024-05-11T12:13:21.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.906+0000 7fd330e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd32c072ab0 con 0x7fd32c0b3a80 2024-05-11T12:13:21.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.906+0000 7fd330e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd32c0b5670 con 0x7fd32c1458f0 2024-05-11T12:13:21.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.906+0000 7fd32aa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd32c0b5220 0x7fd32c137050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:21.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.907+0000 7fd32aa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd32c0b5220 0x7fd32c137050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39420/0 (socket says 172.21.15.167:39420) 2024-05-11T12:13:21.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.907+0000 7fd32aa00640 1 -- 172.21.15.167:0/653442140 learned_addr learned my addr 172.21.15.167:0/653442140 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:21.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.907+0000 7fd32aa00640 1 -- 172.21.15.167:0/653442140 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd32c1458f0 msgr2=0x7fd32c15fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:21.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.907+0000 7fd32be00640 1 --2- 172.21.15.167:0/653442140 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd32c1458f0 0x7fd32c15fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:21.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.907+0000 7fd32b400640 1 --2- 172.21.15.167:0/653442140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd32c0b3a80 0x7fd32c144ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:21.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.907+0000 7fd32aa00640 1 --2- 172.21.15.167:0/653442140 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd32c1458f0 0x7fd32c15fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:21.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.907+0000 7fd32aa00640 1 -- 172.21.15.167:0/653442140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd32c0b3a80 msgr2=0x7fd32c144ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:21.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.907+0000 7fd32aa00640 1 --2- 172.21.15.167:0/653442140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd32c0b3a80 0x7fd32c144ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:21.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.907+0000 7fd32aa00640 1 -- 172.21.15.167:0/653442140 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd32c1450e0 con 0x7fd32c0b5220 2024-05-11T12:13:21.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.907+0000 7fd32aa00640 1 --2- 172.21.15.167:0/653442140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd32c0b5220 0x7fd32c137050 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd31800d800 tx=0x7fd31800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:21.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.907+0000 7fd323400640 1 -- 172.21.15.167:0/653442140 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31800b6c0 con 0x7fd32c0b5220 2024-05-11T12:13:21.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.908+0000 7fd323400640 1 -- 172.21.15.167:0/653442140 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd318019070 con 0x7fd32c0b5220 2024-05-11T12:13:21.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.908+0000 7fd330e00640 1 -- 172.21.15.167:0/653442140 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd32c1379c0 con 0x7fd32c0b5220 2024-05-11T12:13:21.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.908+0000 7fd323400640 1 -- 172.21.15.167:0/653442140 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31800ba80 con 0x7fd32c0b5220 2024-05-11T12:13:21.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.908+0000 7fd330e00640 1 -- 172.21.15.167:0/653442140 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd32c137f80 con 0x7fd32c0b5220 2024-05-11T12:13:21.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.909+0000 7fd330e00640 1 -- 172.21.15.167:0/653442140 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd32c0694b0 con 0x7fd32c0b5220 2024-05-11T12:13:21.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.910+0000 7fd323400640 1 -- 172.21.15.167:0/653442140 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd31800bc20 con 0x7fd32c0b5220 2024-05-11T12:13:21.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.911+0000 7fd323400640 1 --2- 172.21.15.167:0/653442140 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd30c076350 0x7fd30c078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:21.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.912+0000 7fd323400640 1 -- 172.21.15.167:0/653442140 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7fd318097c00 con 0x7fd32c0b5220 2024-05-11T12:13:21.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.912+0000 7fd32b400640 1 --2- 172.21.15.167:0/653442140 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd30c076350 0x7fd30c078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:21.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.913+0000 7fd32b400640 1 --2- 172.21.15.167:0/653442140 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd30c076350 0x7fd30c078810 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd31c034000 tx=0x7fd31c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:21.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:21.916+0000 7fd323400640 1 -- 172.21.15.167:0/653442140 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd3180613b0 con 0x7fd32c0b5220 2024-05-11T12:13:22.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.163+0000 7fd330e00640 1 -- 172.21.15.167:0/653442140 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd32c068890 con 0x7fd32c0b5220 2024-05-11T12:13:22.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.164+0000 7fd323400640 1 -- 172.21.15.167:0/653442140 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5185 (secure 0 0 0) 0x7fd318060cb0 con 0x7fd32c0b5220 2024-05-11T12:13:22.164 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:13:22.164 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/2785478956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2785478956},{"type":"v1","addr":"172.21.15.204:6834","nonce":2785478956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":4742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/3478043182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":3478043182},{"type":"v1","addr":"172.21.15.204:6835","nonce":3478043182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/618833945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":618833945},{"type":"v1","addr":"172.21.15.167:6836","nonce":618833945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:13:02.579882+0000","modified":"2024-05-11T12:13:05.652972+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5610":{"gid":5610,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/42694429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":42694429},{"type":"v1","addr":"172.21.15.167:6837","nonce":42694429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5610,"qdb_cluster":[5610]},"id":4}]} 2024-05-11T12:13:22.165 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 35 2024-05-11T12:13:22.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.167+0000 7fd330e00640 1 -- 172.21.15.167:0/653442140 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd30c076350 msgr2=0x7fd30c078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:22.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.167+0000 7fd330e00640 1 --2- 172.21.15.167:0/653442140 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd30c076350 0x7fd30c078810 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd31c034000 tx=0x7fd31c02e000 comp rx=0 tx=0).stop 2024-05-11T12:13:22.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.167+0000 7fd330e00640 1 -- 172.21.15.167:0/653442140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd32c0b5220 msgr2=0x7fd32c137050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:22.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.167+0000 7fd330e00640 1 --2- 172.21.15.167:0/653442140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd32c0b5220 0x7fd32c137050 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd31800d800 tx=0x7fd31800dcc0 comp rx=0 tx=0).stop 2024-05-11T12:13:22.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.167+0000 7fd330e00640 1 -- 172.21.15.167:0/653442140 shutdown_connections 2024-05-11T12:13:22.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.167+0000 7fd330e00640 1 --2- 172.21.15.167:0/653442140 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd32c1458f0 0x7fd32c15fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:22.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.167+0000 7fd330e00640 1 --2- 172.21.15.167:0/653442140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd32c0b5220 0x7fd32c137050 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:22.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.167+0000 7fd330e00640 1 --2- 172.21.15.167:0/653442140 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd30c076350 0x7fd30c078810 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:22.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.167+0000 7fd330e00640 1 --2- 172.21.15.167:0/653442140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd32c0b3a80 0x7fd32c144ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:22.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.167+0000 7fd330e00640 1 -- 172.21.15.167:0/653442140 >> 172.21.15.167:0/653442140 conn(0x7fd32c0b3460 msgr2=0x7fd32c0cf7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:22.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.167+0000 7fd330e00640 1 -- 172.21.15.167:0/653442140 shutdown_connections 2024-05-11T12:13:22.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.167+0000 7fd330e00640 1 -- 172.21.15.167:0/653442140 wait complete. 2024-05-11T12:13:22.186 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:13:22.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.388+0000 7f2baaa00640 1 Processor -- start 2024-05-11T12:13:22.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.388+0000 7f2baaa00640 1 -- start start 2024-05-11T12:13:22.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.389+0000 7f2baaa00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2ba40b3ea0 0x7f2ba4161490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:22.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.389+0000 7f2baaa00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2ba41619d0 0x7f2ba406d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:22.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.389+0000 7f2baaa00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ba415df30 con 0x7f2ba4157750 2024-05-11T12:13:22.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.389+0000 7f2baaa00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f2ba4161000 con 0x7f2ba41619d0 2024-05-11T12:13:22.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.389+0000 7f2baaa00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f2ba415e210 con 0x7f2ba40b3ea0 2024-05-11T12:13:22.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.390+0000 7f2ba8a00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2ba41619d0 0x7f2ba406d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:22.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.390+0000 7f2ba8a00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2ba41619d0 0x7f2ba406d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39422/0 (socket says 172.21.15.167:39422) 2024-05-11T12:13:22.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.390+0000 7f2ba8a00640 1 -- 172.21.15.167:0/2380978612 learned_addr learned my addr 172.21.15.167:0/2380978612 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:22.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.390+0000 7f2ba3400640 1 --2- 172.21.15.167:0/2380978612 >> v2:172.21.15.204:3300/0 conn(0x7f2ba40b3ea0 0x7f2ba4161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:22.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.390+0000 7f2ba8a00640 1 -- 172.21.15.167:0/2380978612 >> v2:172.21.15.204:3300/0 conn(0x7f2ba40b3ea0 msgr2=0x7f2ba4161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:22.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.390+0000 7f2ba8a00640 1 --2- 172.21.15.167:0/2380978612 >> v2:172.21.15.204:3300/0 conn(0x7f2ba40b3ea0 0x7f2ba4161490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:22.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.390+0000 7f2ba8a00640 1 -- 172.21.15.167:0/2380978612 >> v1:172.21.15.204:6789/0 conn(0x7f2ba4157750 legacy=0x7f2ba415d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:22.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.390+0000 7f2ba8a00640 1 -- 172.21.15.167:0/2380978612 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ba415e4f0 con 0x7f2ba41619d0 2024-05-11T12:13:22.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.391+0000 7f2ba8a00640 1 --2- 172.21.15.167:0/2380978612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ba41619d0 0x7f2ba406d460 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f2ba40702f0 tx=0x7f2b9c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4acbd10c6ae5266f server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:22.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.392+0000 7f2ba2a00640 1 -- 172.21.15.167:0/2380978612 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c036070 con 0x7f2ba41619d0 2024-05-11T12:13:22.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.392+0000 7f2ba2a00640 1 -- 172.21.15.167:0/2380978612 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b9c031600 con 0x7f2ba41619d0 2024-05-11T12:13:22.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.392+0000 7f2ba2a00640 1 -- 172.21.15.167:0/2380978612 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c0319d0 con 0x7f2ba41619d0 2024-05-11T12:13:22.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.392+0000 7f2baaa00640 1 -- 172.21.15.167:0/2380978612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ba41619d0 msgr2=0x7f2ba406d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:22.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.392+0000 7f2baaa00640 1 --2- 172.21.15.167:0/2380978612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ba41619d0 0x7f2ba406d460 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f2ba40702f0 tx=0x7f2b9c02a6a0 comp rx=0 tx=0).stop 2024-05-11T12:13:22.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.392+0000 7f2baaa00640 1 -- 172.21.15.167:0/2380978612 shutdown_connections 2024-05-11T12:13:22.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.392+0000 7f2baaa00640 1 --2- 172.21.15.167:0/2380978612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ba41619d0 0x7f2ba406d460 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:22.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.392+0000 7f2baaa00640 1 --2- 172.21.15.167:0/2380978612 >> v2:172.21.15.204:3300/0 conn(0x7f2ba40b3ea0 0x7f2ba4161490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:22.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.392+0000 7f2baaa00640 1 -- 172.21.15.167:0/2380978612 >> 172.21.15.167:0/2380978612 conn(0x7f2ba40cd8f0 msgr2=0x7f2ba40cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:22.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.392+0000 7f2baaa00640 1 -- 172.21.15.167:0/2380978612 shutdown_connections 2024-05-11T12:13:22.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.393+0000 7f2baaa00640 1 -- 172.21.15.167:0/2380978612 wait complete. 2024-05-11T12:13:22.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.394+0000 7f2baaa00640 1 Processor -- start 2024-05-11T12:13:22.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.394+0000 7f2baaa00640 1 -- start start 2024-05-11T12:13:22.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.395+0000 7f2baaa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ba40b3ea0 0x7f2ba415ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:22.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.395+0000 7f2baaa00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2ba4157750 0x7f2ba40aadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:22.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.395+0000 7f2baaa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ba41619d0 0x7f2ba41f3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:22.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.395+0000 7f2baaa00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba406db20 con 0x7f2ba41619d0 2024-05-11T12:13:22.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.395+0000 7f2baaa00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba406d9a0 con 0x7f2ba40b3ea0 2024-05-11T12:13:22.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.395+0000 7f2baaa00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2ba406dca0 con 0x7f2ba4157750 2024-05-11T12:13:22.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.396+0000 7f2ba8a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ba41619d0 0x7f2ba41f3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:22.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.396+0000 7f2ba8a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ba41619d0 0x7f2ba41f3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39426/0 (socket says 172.21.15.167:39426) 2024-05-11T12:13:22.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.396+0000 7f2ba8a00640 1 -- 172.21.15.167:0/1834808815 learned_addr learned my addr 172.21.15.167:0/1834808815 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:22.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.396+0000 7f2ba3e00640 1 --2- 172.21.15.167:0/1834808815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ba40b3ea0 0x7f2ba415ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:22.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.396+0000 7f2ba8a00640 1 -- 172.21.15.167:0/1834808815 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2ba4157750 msgr2=0x7f2ba40aadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:22.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.396+0000 7f2ba3400640 1 --2- 172.21.15.167:0/1834808815 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2ba4157750 0x7f2ba40aadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:22.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.396+0000 7f2ba8a00640 1 --2- 172.21.15.167:0/1834808815 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2ba4157750 0x7f2ba40aadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:22.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.396+0000 7f2ba8a00640 1 -- 172.21.15.167:0/1834808815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ba40b3ea0 msgr2=0x7f2ba415ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:22.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.396+0000 7f2ba8a00640 1 --2- 172.21.15.167:0/1834808815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ba40b3ea0 0x7f2ba415ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:22.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.396+0000 7f2ba8a00640 1 -- 172.21.15.167:0/1834808815 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ba40ab590 con 0x7f2ba41619d0 2024-05-11T12:13:22.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.396+0000 7f2ba3e00640 1 --2- 172.21.15.167:0/1834808815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ba40b3ea0 0x7f2ba415ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:22.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.397+0000 7f2ba8a00640 1 --2- 172.21.15.167:0/1834808815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ba41619d0 0x7f2ba41f3380 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c009f40 tx=0x7f2b9c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:22.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.397+0000 7f2b9be00640 1 -- 172.21.15.167:0/1834808815 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c036050 con 0x7f2ba41619d0 2024-05-11T12:13:22.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.397+0000 7f2b9be00640 1 -- 172.21.15.167:0/1834808815 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b9c042070 con 0x7f2ba41619d0 2024-05-11T12:13:22.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.397+0000 7f2b9be00640 1 -- 172.21.15.167:0/1834808815 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c009510 con 0x7f2ba41619d0 2024-05-11T12:13:22.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.397+0000 7f2baaa00640 1 -- 172.21.15.167:0/1834808815 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ba40abd90 con 0x7f2ba41619d0 2024-05-11T12:13:22.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.397+0000 7f2baaa00640 1 -- 172.21.15.167:0/1834808815 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ba40ac350 con 0x7f2ba41619d0 2024-05-11T12:13:22.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.398+0000 7f2baaa00640 1 -- 172.21.15.167:0/1834808815 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ba4069590 con 0x7f2ba41619d0 2024-05-11T12:13:22.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.399+0000 7f2b9be00640 1 -- 172.21.15.167:0/1834808815 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2b9c007bd0 con 0x7f2ba41619d0 2024-05-11T12:13:22.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.401+0000 7f2b9be00640 1 --2- 172.21.15.167:0/1834808815 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2b800762d0 0x7f2b80078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:22.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.401+0000 7f2b9be00640 1 -- 172.21.15.167:0/1834808815 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f2b9c0b8580 con 0x7f2ba41619d0 2024-05-11T12:13:22.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.401+0000 7f2ba3e00640 1 --2- 172.21.15.167:0/1834808815 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2b800762d0 0x7f2b80078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:22.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.402+0000 7f2ba3e00640 1 --2- 172.21.15.167:0/1834808815 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2b800762d0 0x7f2b80078790 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c003eb0 tx=0x7f2b8c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:22.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.405+0000 7f2b9be00640 1 -- 172.21.15.167:0/1834808815 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2b9c081de0 con 0x7f2ba41619d0 2024-05-11T12:13:22.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.600+0000 7f2baaa00640 1 -- 172.21.15.167:0/1834808815 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2ba406d460 con 0x7f2ba41619d0 2024-05-11T12:13:22.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.603+0000 7f2b9be00640 1 -- 172.21.15.167:0/1834808815 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) ==== 74+0+16932 (secure 0 0 0) 0x7f2b9c0816e0 con 0x7f2ba41619d0 2024-05-11T12:13:22.603 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:13:22.604 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":64,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:13:04.144728+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:13:00.503965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-11T12:13:01.566330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:13:22.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.606+0000 7f2baaa00640 1 -- 172.21.15.167:0/1834808815 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2b800762d0 msgr2=0x7f2b80078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:22.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.606+0000 7f2baaa00640 1 --2- 172.21.15.167:0/1834808815 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2b800762d0 0x7f2b80078790 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c003eb0 tx=0x7f2b8c009040 comp rx=0 tx=0).stop 2024-05-11T12:13:22.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.606+0000 7f2baaa00640 1 -- 172.21.15.167:0/1834808815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ba41619d0 msgr2=0x7f2ba41f3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:22.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.606+0000 7f2baaa00640 1 --2- 172.21.15.167:0/1834808815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ba41619d0 0x7f2ba41f3380 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c009f40 tx=0x7f2b9c004a40 comp rx=0 tx=0).stop 2024-05-11T12:13:22.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.606+0000 7f2baaa00640 1 -- 172.21.15.167:0/1834808815 shutdown_connections 2024-05-11T12:13:22.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.606+0000 7f2baaa00640 1 --2- 172.21.15.167:0/1834808815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ba41619d0 0x7f2ba41f3380 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:22.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.606+0000 7f2baaa00640 1 --2- 172.21.15.167:0/1834808815 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2ba4157750 0x7f2ba40aadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:22.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.606+0000 7f2baaa00640 1 --2- 172.21.15.167:0/1834808815 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2b800762d0 0x7f2b80078790 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:22.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.606+0000 7f2baaa00640 1 --2- 172.21.15.167:0/1834808815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ba40b3ea0 0x7f2ba415ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:22.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.606+0000 7f2baaa00640 1 -- 172.21.15.167:0/1834808815 >> 172.21.15.167:0/1834808815 conn(0x7f2ba40cd8f0 msgr2=0x7f2ba415f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:22.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.606+0000 7f2baaa00640 1 -- 172.21.15.167:0/1834808815 shutdown_connections 2024-05-11T12:13:22.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.606+0000 7f2baaa00640 1 -- 172.21.15.167:0/1834808815 wait complete. 2024-05-11T12:13:22.625 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-11T12:13:22.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.823+0000 7f0c3b000640 1 Processor -- start 2024-05-11T12:13:22.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.824+0000 7f0c3b000640 1 -- start start 2024-05-11T12:13:22.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.825+0000 7f0c3b000640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0c34161380 0x7f0c34074260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:22.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.825+0000 7f0c3b000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c340b3d50 0x7f0c34072950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:22.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.825+0000 7f0c3b000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c340b65e0 con 0x7f0c3415ac80 2024-05-11T12:13:22.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.825+0000 7f0c3b000640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f0c3406e3c0 con 0x7f0c34161380 2024-05-11T12:13:22.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.825+0000 7f0c3b000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0c3415b060 con 0x7f0c340b3d50 2024-05-11T12:13:22.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.826+0000 7f0c2be00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0c34161380 0x7f0c34074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:22.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.826+0000 7f0c2be00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0c34161380 0x7f0c34074260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39442/0 (socket says 172.21.15.167:39442) 2024-05-11T12:13:22.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.826+0000 7f0c2be00640 1 -- 172.21.15.167:0/3287775625 learned_addr learned my addr 172.21.15.167:0/3287775625 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:22.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.826+0000 7f0c39000640 1 --2- 172.21.15.167:0/3287775625 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c340b3d50 0x7f0c34072950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:22.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.826+0000 7f0c2be00640 1 -- 172.21.15.167:0/3287775625 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c340b3d50 msgr2=0x7f0c34072950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:22.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.826+0000 7f0c2be00640 1 --2- 172.21.15.167:0/3287775625 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c340b3d50 0x7f0c34072950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:22.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.826+0000 7f0c2be00640 1 -- 172.21.15.167:0/3287775625 >> v1:172.21.15.204:6789/0 conn(0x7f0c3415ac80 legacy=0x7f0c34160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:22.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.826+0000 7f0c2be00640 1 -- 172.21.15.167:0/3287775625 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c340b68c0 con 0x7f0c34161380 2024-05-11T12:13:22.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.827+0000 7f0c2be00640 1 --2- 172.21.15.167:0/3287775625 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c34161380 0x7f0c34074260 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c009870 tx=0x7f0c1c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f006a974cac1066 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:22.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.828+0000 7f0c33400640 1 -- 172.21.15.167:0/3287775625 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c037070 con 0x7f0c34161380 2024-05-11T12:13:22.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.828+0000 7f0c33400640 1 -- 172.21.15.167:0/3287775625 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c1c032440 con 0x7f0c34161380 2024-05-11T12:13:22.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.828+0000 7f0c33400640 1 -- 172.21.15.167:0/3287775625 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c032810 con 0x7f0c34161380 2024-05-11T12:13:22.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.828+0000 7f0c3b000640 1 -- 172.21.15.167:0/3287775625 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c34161380 msgr2=0x7f0c34074260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:22.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.828+0000 7f0c3b000640 1 --2- 172.21.15.167:0/3287775625 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c34161380 0x7f0c34074260 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c009870 tx=0x7f0c1c02a1c0 comp rx=0 tx=0).stop 2024-05-11T12:13:22.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.828+0000 7f0c3b000640 1 -- 172.21.15.167:0/3287775625 shutdown_connections 2024-05-11T12:13:22.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.828+0000 7f0c3b000640 1 --2- 172.21.15.167:0/3287775625 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c340b3d50 0x7f0c34072950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:22.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.828+0000 7f0c3b000640 1 --2- 172.21.15.167:0/3287775625 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c34161380 0x7f0c34074260 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:22.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.828+0000 7f0c3b000640 1 -- 172.21.15.167:0/3287775625 >> 172.21.15.167:0/3287775625 conn(0x7f0c3406c450 msgr2=0x7f0c3406c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:22.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.829+0000 7f0c3b000640 1 -- 172.21.15.167:0/3287775625 shutdown_connections 2024-05-11T12:13:22.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.829+0000 7f0c3b000640 1 -- 172.21.15.167:0/3287775625 wait complete. 2024-05-11T12:13:22.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.830+0000 7f0c3b000640 1 Processor -- start 2024-05-11T12:13:22.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.830+0000 7f0c3b000640 1 -- start start 2024-05-11T12:13:22.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.831+0000 7f0c3b000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c340b3d50 0x7f0c34145460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:22.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.831+0000 7f0c3b000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c3415ac80 0x7f0c34140800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:22.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.831+0000 7f0c3b000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c34161380 0x7f0c3415c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:22.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.831+0000 7f0c3b000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0c340b44f0 con 0x7f0c3415ac80 2024-05-11T12:13:22.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.831+0000 7f0c3b000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0c340b41a0 con 0x7f0c340b3d50 2024-05-11T12:13:22.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.831+0000 7f0c3b000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0c34072e90 con 0x7f0c34161380 2024-05-11T12:13:22.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.832+0000 7f0c2be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c3415ac80 0x7f0c34140800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:22.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.832+0000 7f0c2be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c3415ac80 0x7f0c34140800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39458/0 (socket says 172.21.15.167:39458) 2024-05-11T12:13:22.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.832+0000 7f0c2be00640 1 -- 172.21.15.167:0/4232281134 learned_addr learned my addr 172.21.15.167:0/4232281134 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:22.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.832+0000 7f0c39000640 1 --2- 172.21.15.167:0/4232281134 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c34161380 0x7f0c3415c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:22.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.832+0000 7f0c33e00640 1 --2- 172.21.15.167:0/4232281134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c340b3d50 0x7f0c34145460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:22.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.832+0000 7f0c2be00640 1 -- 172.21.15.167:0/4232281134 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c34161380 msgr2=0x7f0c3415c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:22.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.832+0000 7f0c2be00640 1 --2- 172.21.15.167:0/4232281134 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c34161380 0x7f0c3415c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:22.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.832+0000 7f0c2be00640 1 -- 172.21.15.167:0/4232281134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c340b3d50 msgr2=0x7f0c34145460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:22.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.832+0000 7f0c2be00640 1 --2- 172.21.15.167:0/4232281134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c340b3d50 0x7f0c34145460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:22.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.832+0000 7f0c2be00640 1 -- 172.21.15.167:0/4232281134 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c34142f40 con 0x7f0c3415ac80 2024-05-11T12:13:22.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.832+0000 7f0c33e00640 1 --2- 172.21.15.167:0/4232281134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c340b3d50 0x7f0c34145460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:22.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.832+0000 7f0c2be00640 1 --2- 172.21.15.167:0/4232281134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c3415ac80 0x7f0c34140800 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c0090b0 tx=0x7f0c1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:22.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.833+0000 7f0c30c00640 1 -- 172.21.15.167:0/4232281134 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c041020 con 0x7f0c3415ac80 2024-05-11T12:13:22.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.833+0000 7f0c30c00640 1 -- 172.21.15.167:0/4232281134 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c1c0306c0 con 0x7f0c3415ac80 2024-05-11T12:13:22.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.833+0000 7f0c3b000640 1 -- 172.21.15.167:0/4232281134 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c34159e00 con 0x7f0c3415ac80 2024-05-11T12:13:22.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.833+0000 7f0c30c00640 1 -- 172.21.15.167:0/4232281134 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c037040 con 0x7f0c3415ac80 2024-05-11T12:13:22.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.833+0000 7f0c3b000640 1 -- 172.21.15.167:0/4232281134 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c34141250 con 0x7f0c3415ac80 2024-05-11T12:13:22.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.834+0000 7f0c3b000640 1 -- 172.21.15.167:0/4232281134 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c340694b0 con 0x7f0c3415ac80 2024-05-11T12:13:22.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.835+0000 7f0c30c00640 1 -- 172.21.15.167:0/4232281134 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0c1c044020 con 0x7f0c3415ac80 2024-05-11T12:13:22.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.837+0000 7f0c30c00640 1 --2- 172.21.15.167:0/4232281134 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0c100762d0 0x7f0c10078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:22.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.837+0000 7f0c30c00640 1 -- 172.21.15.167:0/4232281134 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f0c1c0b7fb0 con 0x7f0c3415ac80 2024-05-11T12:13:22.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.837+0000 7f0c33e00640 1 --2- 172.21.15.167:0/4232281134 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0c100762d0 0x7f0c10078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:22.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.838+0000 7f0c33e00640 1 --2- 172.21.15.167:0/4232281134 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0c100762d0 0x7f0c10078790 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0c24003eb0 tx=0x7f0c24009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:22.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:22.841+0000 7f0c30c00640 1 -- 172.21.15.167:0/4232281134 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0c1c04b030 con 0x7f0c3415ac80 2024-05-11T12:13:23.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.092+0000 7f0c3b000640 1 -- 172.21.15.167:0/4232281134 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f0c341f2f40 con 0x7f0c3415ac80 2024-05-11T12:13:23.093 INFO:tasks.ceph.mon.a.smithi167.stderr:2024-05-11T12:13:23.093+0000 7f3aac000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-11T12:13:23.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.097+0000 7f0c30c00640 1 -- 172.21.15.167:0/4232281134 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v36) ==== 161+0+0 (secure 0 0 0) 0x7f0c1c081770 con 0x7f0c3415ac80 2024-05-11T12:13:23.098 INFO:teuthology.orchestra.run.smithi167.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-11T12:13:23.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.101+0000 7f0c3b000640 1 -- 172.21.15.167:0/4232281134 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0c100762d0 msgr2=0x7f0c10078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:23.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.101+0000 7f0c3b000640 1 --2- 172.21.15.167:0/4232281134 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0c100762d0 0x7f0c10078790 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0c24003eb0 tx=0x7f0c24009040 comp rx=0 tx=0).stop 2024-05-11T12:13:23.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.101+0000 7f0c3b000640 1 -- 172.21.15.167:0/4232281134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c3415ac80 msgr2=0x7f0c34140800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:23.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.101+0000 7f0c3b000640 1 --2- 172.21.15.167:0/4232281134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c3415ac80 0x7f0c34140800 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c0090b0 tx=0x7f0c1c0046c0 comp rx=0 tx=0).stop 2024-05-11T12:13:23.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.102+0000 7f0c3b000640 1 -- 172.21.15.167:0/4232281134 shutdown_connections 2024-05-11T12:13:23.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.102+0000 7f0c3b000640 1 --2- 172.21.15.167:0/4232281134 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c34161380 0x7f0c3415c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:23.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.102+0000 7f0c3b000640 1 --2- 172.21.15.167:0/4232281134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c3415ac80 0x7f0c34140800 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:23.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.102+0000 7f0c3b000640 1 --2- 172.21.15.167:0/4232281134 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0c100762d0 0x7f0c10078790 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:23.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.102+0000 7f0c3b000640 1 --2- 172.21.15.167:0/4232281134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c340b3d50 0x7f0c34145460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:23.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.102+0000 7f0c3b000640 1 -- 172.21.15.167:0/4232281134 >> 172.21.15.167:0/4232281134 conn(0x7f0c3406c450 msgr2=0x7f0c340681c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:23.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.103+0000 7f0c3b000640 1 -- 172.21.15.167:0/4232281134 shutdown_connections 2024-05-11T12:13:23.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.103+0000 7f0c3b000640 1 -- 172.21.15.167:0/4232281134 wait complete. 2024-05-11T12:13:23.111 INFO:tasks.ceph.mds.c.smithi167.stderr: -2322> 2024-05-11T12:13:05.075+0000 7fbbf1400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-11T12:13:23.120 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-11T12:13:23.180 INFO:tasks.ceph.mds.c.smithi167.stdout:starting mds.c at 2024-05-11T12:13:23.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.300+0000 7faf97e00640 1 Processor -- start 2024-05-11T12:13:23.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.300+0000 7faf97e00640 1 -- start start 2024-05-11T12:13:23.302 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.301+0000 7faf97e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7faf9813fe30 0x7faf9815de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:23.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.301+0000 7faf97e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7faf980d0660 0x7faf981609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:23.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.301+0000 7faf97e00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf9815e380 con 0x7faf98160ef0 2024-05-11T12:13:23.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.301+0000 7faf97e00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7faf98161340 con 0x7faf9813fe30 2024-05-11T12:13:23.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.301+0000 7faf97e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7faf980cf5c0 con 0x7faf980d0660 2024-05-11T12:13:23.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.302+0000 7faf96a00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7faf9813fe30 0x7faf9815de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:23.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.302+0000 7faf97400640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7faf98160ef0 0x7faf981424a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:37364/0 (socket says 172.21.15.167:37364) 2024-05-11T12:13:23.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.302+0000 7faf97400640 1 -- 172.21.15.167:0/3174553927 learned_addr learned my addr 172.21.15.167:0/3174553927 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:13:23.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.302+0000 7faf96000640 1 --2- 172.21.15.167:0/3174553927 >> v2:172.21.15.204:3300/0 conn(0x7faf980d0660 0x7faf981609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:23.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.302+0000 7faf96a00640 1 -- 172.21.15.167:0/3174553927 >> v2:172.21.15.204:3300/0 conn(0x7faf980d0660 msgr2=0x7faf981609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:23.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.302+0000 7faf96a00640 1 --2- 172.21.15.167:0/3174553927 >> v2:172.21.15.204:3300/0 conn(0x7faf980d0660 0x7faf981609b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:23.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.302+0000 7faf96a00640 1 -- 172.21.15.167:0/3174553927 >> v1:172.21.15.167:6789/0 conn(0x7faf98160ef0 legacy=0x7faf981424a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:23.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.302+0000 7faf96a00640 1 -- 172.21.15.167:0/3174553927 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf980cf8a0 con 0x7faf9813fe30 2024-05-11T12:13:23.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.302+0000 7faf96000640 1 --2- 172.21.15.167:0/3174553927 >> v2:172.21.15.204:3300/0 conn(0x7faf980d0660 0x7faf981609b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:23.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.302+0000 7faf95600640 1 -- 172.21.15.167:0/3174553927 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2222144138 0 0) 0x7faf90001490 con 0x7faf98160ef0 2024-05-11T12:13:23.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.303+0000 7faf96a00640 1 --2- 172.21.15.167:0/3174553927 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf9813fe30 0x7faf9815de40 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7faf980b4180 tx=0x7faf8802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fb6057c0f6d1024 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:23.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.304+0000 7faf95600640 1 -- 172.21.15.167:0/3174553927 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf88037070 con 0x7faf9813fe30 2024-05-11T12:13:23.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.304+0000 7faf95600640 1 -- 172.21.15.167:0/3174553927 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf88032440 con 0x7faf9813fe30 2024-05-11T12:13:23.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.304+0000 7faf95600640 1 -- 172.21.15.167:0/3174553927 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf88032810 con 0x7faf9813fe30 2024-05-11T12:13:23.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.305+0000 7faf97e00640 1 -- 172.21.15.167:0/3174553927 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf9813fe30 msgr2=0x7faf9815de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:23.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.305+0000 7faf97e00640 1 --2- 172.21.15.167:0/3174553927 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf9813fe30 0x7faf9815de40 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7faf980b4180 tx=0x7faf8802a1a0 comp rx=0 tx=0).stop 2024-05-11T12:13:23.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.305+0000 7faf97e00640 1 -- 172.21.15.167:0/3174553927 shutdown_connections 2024-05-11T12:13:23.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.305+0000 7faf97e00640 1 --2- 172.21.15.167:0/3174553927 >> v2:172.21.15.204:3300/0 conn(0x7faf980d0660 0x7faf981609b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:23.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.305+0000 7faf97e00640 1 --2- 172.21.15.167:0/3174553927 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf9813fe30 0x7faf9815de40 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:23.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.305+0000 7faf97e00640 1 -- 172.21.15.167:0/3174553927 >> 172.21.15.167:0/3174553927 conn(0x7faf980b2a20 msgr2=0x7faf980b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:23.306 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.305+0000 7faf97e00640 1 -- 172.21.15.167:0/3174553927 shutdown_connections 2024-05-11T12:13:23.306 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.305+0000 7faf97e00640 1 -- 172.21.15.167:0/3174553927 wait complete. 2024-05-11T12:13:23.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.307+0000 7faf97e00640 1 Processor -- start 2024-05-11T12:13:23.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.307+0000 7faf97e00640 1 -- start start 2024-05-11T12:13:23.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.308+0000 7faf97e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf980d0660 0x7faf9813e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:23.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.308+0000 7faf97e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf9813fe30 0x7faf981e8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:23.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.308+0000 7faf97e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faf98160ef0 0x7faf98072f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:23.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.308+0000 7faf97e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faf980d0140 con 0x7faf980d0660 2024-05-11T12:13:23.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.308+0000 7faf97e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7faf980cfe60 con 0x7faf9813fe30 2024-05-11T12:13:23.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.308+0000 7faf97e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7faf981616a0 con 0x7faf98160ef0 2024-05-11T12:13:23.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.308+0000 7faf96a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf980d0660 0x7faf9813e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:23.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.308+0000 7faf96a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf980d0660 0x7faf9813e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39498/0 (socket says 172.21.15.167:39498) 2024-05-11T12:13:23.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.308+0000 7faf96a00640 1 -- 172.21.15.167:0/698216400 learned_addr learned my addr 172.21.15.167:0/698216400 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:23.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.308+0000 7faf96a00640 1 -- 172.21.15.167:0/698216400 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faf98160ef0 msgr2=0x7faf98072f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:23.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.308+0000 7faf96a00640 1 --2- 172.21.15.167:0/698216400 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faf98160ef0 0x7faf98072f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:23.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.308+0000 7faf96a00640 1 -- 172.21.15.167:0/698216400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf9813fe30 msgr2=0x7faf981e8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:23.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.308+0000 7faf96a00640 1 --2- 172.21.15.167:0/698216400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf9813fe30 0x7faf981e8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:23.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.308+0000 7faf96a00640 1 -- 172.21.15.167:0/698216400 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf981e8c90 con 0x7faf980d0660 2024-05-11T12:13:23.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.309+0000 7faf96a00640 1 --2- 172.21.15.167:0/698216400 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf980d0660 0x7faf9813e7b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7faf88000c00 tx=0x7faf880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:23.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.309+0000 7faf8ea00640 1 -- 172.21.15.167:0/698216400 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf88037040 con 0x7faf980d0660 2024-05-11T12:13:23.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.309+0000 7faf8ea00640 1 -- 172.21.15.167:0/698216400 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf88042070 con 0x7faf980d0660 2024-05-11T12:13:23.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.309+0000 7faf8ea00640 1 -- 172.21.15.167:0/698216400 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf88030bd0 con 0x7faf980d0660 2024-05-11T12:13:23.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.309+0000 7faf97e00640 1 -- 172.21.15.167:0/698216400 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf981e8f70 con 0x7faf980d0660 2024-05-11T12:13:23.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.309+0000 7faf97e00640 1 -- 172.21.15.167:0/698216400 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf981e9530 con 0x7faf980d0660 2024-05-11T12:13:23.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.310+0000 7faf97e00640 1 -- 172.21.15.167:0/698216400 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf980694b0 con 0x7faf980d0660 2024-05-11T12:13:23.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.312+0000 7faf8ea00640 1 -- 172.21.15.167:0/698216400 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7faf88047460 con 0x7faf980d0660 2024-05-11T12:13:23.313 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.313+0000 7faf8ea00640 1 --2- 172.21.15.167:0/698216400 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7faf780762d0 0x7faf78078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:23.313 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.313+0000 7faf8ea00640 1 -- 172.21.15.167:0/698216400 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6187+0+0 (secure 0 0 0) 0x7faf880b84e0 con 0x7faf980d0660 2024-05-11T12:13:23.313 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.313+0000 7faf96000640 1 --2- 172.21.15.167:0/698216400 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7faf780762d0 0x7faf78078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:23.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.314+0000 7faf96000640 1 --2- 172.21.15.167:0/698216400 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7faf780762d0 0x7faf78078790 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7faf84006fd0 tx=0x7faf84008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:23.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.317+0000 7faf8ea00640 1 -- 172.21.15.167:0/698216400 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7faf88032e20 con 0x7faf980d0660 2024-05-11T12:13:23.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:23.567+0000 7faf97e00640 1 -- 172.21.15.167:0/698216400 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7faf980b41b0 con 0x7faf980d0660 2024-05-11T12:13:24.101 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.101+0000 7faf8ea00640 1 -- 172.21.15.167:0/698216400 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v37) ==== 104+0+0 (secure 0 0 0) 0x7faf88081c60 con 0x7faf980d0660 2024-05-11T12:13:24.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.105+0000 7faf97e00640 1 -- 172.21.15.167:0/698216400 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7faf780762d0 msgr2=0x7faf78078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:24.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.105+0000 7faf97e00640 1 --2- 172.21.15.167:0/698216400 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7faf780762d0 0x7faf78078790 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7faf84006fd0 tx=0x7faf84008040 comp rx=0 tx=0).stop 2024-05-11T12:13:24.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.105+0000 7faf97e00640 1 -- 172.21.15.167:0/698216400 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf980d0660 msgr2=0x7faf9813e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:24.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.105+0000 7faf97e00640 1 --2- 172.21.15.167:0/698216400 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf980d0660 0x7faf9813e7b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7faf88000c00 tx=0x7faf880046c0 comp rx=0 tx=0).stop 2024-05-11T12:13:24.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.105+0000 7faf97e00640 1 -- 172.21.15.167:0/698216400 shutdown_connections 2024-05-11T12:13:24.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.105+0000 7faf97e00640 1 --2- 172.21.15.167:0/698216400 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faf98160ef0 0x7faf98072f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:24.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.105+0000 7faf97e00640 1 --2- 172.21.15.167:0/698216400 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7faf780762d0 0x7faf78078790 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:24.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.105+0000 7faf97e00640 1 --2- 172.21.15.167:0/698216400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf9813fe30 0x7faf981e8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:24.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.105+0000 7faf97e00640 1 --2- 172.21.15.167:0/698216400 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf980d0660 0x7faf9813e7b0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:24.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.105+0000 7faf97e00640 1 -- 172.21.15.167:0/698216400 >> 172.21.15.167:0/698216400 conn(0x7faf980b2a20 msgr2=0x7faf9806e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:24.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.107+0000 7faf97e00640 1 -- 172.21.15.167:0/698216400 shutdown_connections 2024-05-11T12:13:24.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.107+0000 7faf97e00640 1 -- 172.21.15.167:0/698216400 wait complete. 2024-05-11T12:13:24.128 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-11T12:13:24.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.322+0000 7f1511200640 1 Processor -- start 2024-05-11T12:13:24.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.322+0000 7f1511200640 1 -- start start 2024-05-11T12:13:24.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.323+0000 7f1511200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f150c13ffd0 0x7f150c143170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:24.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.323+0000 7f1511200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f150c1428d0 0x7f150c06d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:24.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.323+0000 7f1511200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f150c1436b0 con 0x7f150c06d9a0 2024-05-11T12:13:24.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.323+0000 7f1511200640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f150c160c40 con 0x7f150c1428d0 2024-05-11T12:13:24.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.323+0000 7f1511200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f150c161aa0 con 0x7f150c13ffd0 2024-05-11T12:13:24.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.323+0000 7f1503e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f150c1428d0 0x7f150c06d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:24.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.323+0000 7f1503e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f150c1428d0 0x7f150c06d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39506/0 (socket says 172.21.15.167:39506) 2024-05-11T12:13:24.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.323+0000 7f1503e00640 1 -- 172.21.15.167:0/3244675935 learned_addr learned my addr 172.21.15.167:0/3244675935 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:24.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.324+0000 7f1503e00640 1 -- 172.21.15.167:0/3244675935 >> v2:172.21.15.204:3300/0 conn(0x7f150c13ffd0 msgr2=0x7f150c143170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:24.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.324+0000 7f150b400640 1 --2- 172.21.15.167:0/3244675935 >> v2:172.21.15.204:3300/0 conn(0x7f150c13ffd0 0x7f150c143170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:24.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.324+0000 7f1503e00640 1 --2- 172.21.15.167:0/3244675935 >> v2:172.21.15.204:3300/0 conn(0x7f150c13ffd0 0x7f150c143170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:24.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.324+0000 7f1503e00640 1 -- 172.21.15.167:0/3244675935 >> v1:172.21.15.204:6789/0 conn(0x7f150c06d9a0 legacy=0x7f150c06e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:24.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.324+0000 7f1503e00640 1 -- 172.21.15.167:0/3244675935 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f150c160e90 con 0x7f150c1428d0 2024-05-11T12:13:24.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.324+0000 7f150b400640 1 --2- 172.21.15.167:0/3244675935 >> v2:172.21.15.204:3300/0 conn(0x7f150c13ffd0 0x7f150c143170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:24.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.325+0000 7f1503e00640 1 --2- 172.21.15.167:0/3244675935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f150c1428d0 0x7f150c06d460 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f150c074320 tx=0x7f14f402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2486e9f998baf763 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:24.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.325+0000 7f150aa00640 1 -- 172.21.15.167:0/3244675935 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f4037070 con 0x7f150c1428d0 2024-05-11T12:13:24.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.325+0000 7f150aa00640 1 -- 172.21.15.167:0/3244675935 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14f4032440 con 0x7f150c1428d0 2024-05-11T12:13:24.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.325+0000 7f150aa00640 1 -- 172.21.15.167:0/3244675935 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f4032810 con 0x7f150c1428d0 2024-05-11T12:13:24.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.326+0000 7f1511200640 1 -- 172.21.15.167:0/3244675935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f150c1428d0 msgr2=0x7f150c06d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:24.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.326+0000 7f1511200640 1 --2- 172.21.15.167:0/3244675935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f150c1428d0 0x7f150c06d460 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f150c074320 tx=0x7f14f402a1e0 comp rx=0 tx=0).stop 2024-05-11T12:13:24.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.326+0000 7f1511200640 1 -- 172.21.15.167:0/3244675935 shutdown_connections 2024-05-11T12:13:24.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.326+0000 7f1511200640 1 --2- 172.21.15.167:0/3244675935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f150c1428d0 0x7f150c06d460 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:24.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.326+0000 7f1511200640 1 --2- 172.21.15.167:0/3244675935 >> v2:172.21.15.204:3300/0 conn(0x7f150c13ffd0 0x7f150c143170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:24.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.326+0000 7f1511200640 1 -- 172.21.15.167:0/3244675935 >> 172.21.15.167:0/3244675935 conn(0x7f150c0b32a0 msgr2=0x7f150c0d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:24.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.326+0000 7f1511200640 1 -- 172.21.15.167:0/3244675935 shutdown_connections 2024-05-11T12:13:24.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.326+0000 7f1511200640 1 -- 172.21.15.167:0/3244675935 wait complete. 2024-05-11T12:13:24.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.328+0000 7f1511200640 1 Processor -- start 2024-05-11T12:13:24.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.328+0000 7f1511200640 1 -- start start 2024-05-11T12:13:24.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.329+0000 7f1511200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f150c06d9a0 0x7f150c155940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:24.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.329+0000 7f1511200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f150c13ffd0 0x7f150c140d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:24.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.329+0000 7f1511200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f150c1428d0 0x7f150c141290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:24.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.329+0000 7f1511200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f150c161450 con 0x7f150c1428d0 2024-05-11T12:13:24.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.329+0000 7f1511200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f150c06dde0 con 0x7f150c06d9a0 2024-05-11T12:13:24.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.329+0000 7f1511200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f150c161730 con 0x7f150c13ffd0 2024-05-11T12:13:24.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.329+0000 7f150be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f150c1428d0 0x7f150c141290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:24.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.329+0000 7f150be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f150c1428d0 0x7f150c141290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39516/0 (socket says 172.21.15.167:39516) 2024-05-11T12:13:24.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.329+0000 7f150be00640 1 -- 172.21.15.167:0/2136549605 learned_addr learned my addr 172.21.15.167:0/2136549605 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:24.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.330+0000 7f150b400640 1 --2- 172.21.15.167:0/2136549605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f150c06d9a0 0x7f150c155940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:24.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.330+0000 7f1503e00640 1 --2- 172.21.15.167:0/2136549605 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f150c13ffd0 0x7f150c140d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:24.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.330+0000 7f150be00640 1 -- 172.21.15.167:0/2136549605 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f150c13ffd0 msgr2=0x7f150c140d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:24.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.330+0000 7f150be00640 1 --2- 172.21.15.167:0/2136549605 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f150c13ffd0 0x7f150c140d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:24.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.330+0000 7f150be00640 1 -- 172.21.15.167:0/2136549605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f150c06d9a0 msgr2=0x7f150c155940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:24.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.330+0000 7f150be00640 1 --2- 172.21.15.167:0/2136549605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f150c06d9a0 0x7f150c155940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:24.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.330+0000 7f150be00640 1 -- 172.21.15.167:0/2136549605 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f150c155e80 con 0x7f150c1428d0 2024-05-11T12:13:24.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.330+0000 7f150be00640 1 --2- 172.21.15.167:0/2136549605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f150c1428d0 0x7f150c141290 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f150400c780 tx=0x7f150400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:24.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.330+0000 7f1503400640 1 -- 172.21.15.167:0/2136549605 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1504004b60 con 0x7f150c1428d0 2024-05-11T12:13:24.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.330+0000 7f1503400640 1 -- 172.21.15.167:0/2136549605 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1504004d00 con 0x7f150c1428d0 2024-05-11T12:13:24.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.330+0000 7f1503400640 1 -- 172.21.15.167:0/2136549605 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150400aa30 con 0x7f150c1428d0 2024-05-11T12:13:24.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.330+0000 7f1511200640 1 -- 172.21.15.167:0/2136549605 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f150c142450 con 0x7f150c1428d0 2024-05-11T12:13:24.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.331+0000 7f1511200640 1 -- 172.21.15.167:0/2136549605 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f150c141bb0 con 0x7f150c1428d0 2024-05-11T12:13:24.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.332+0000 7f1511200640 1 -- 172.21.15.167:0/2136549605 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f150c0694b0 con 0x7f150c1428d0 2024-05-11T12:13:24.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.333+0000 7f1503400640 1 -- 172.21.15.167:0/2136549605 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f150400abd0 con 0x7f150c1428d0 2024-05-11T12:13:24.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.334+0000 7f1503400640 1 --2- 172.21.15.167:0/2136549605 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f14e80762d0 0x7f14e8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:24.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.334+0000 7f1503400640 1 -- 172.21.15.167:0/2136549605 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6187+0+0 (secure 0 0 0) 0x7f1504097920 con 0x7f150c1428d0 2024-05-11T12:13:24.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.334+0000 7f150b400640 1 --2- 172.21.15.167:0/2136549605 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f14e80762d0 0x7f14e8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:24.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.335+0000 7f150b400640 1 --2- 172.21.15.167:0/2136549605 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f14e80762d0 0x7f14e8078790 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f14fc00a990 tx=0x7f14fc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:24.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.339+0000 7f1503400640 1 -- 172.21.15.167:0/2136549605 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1504020070 con 0x7f150c1428d0 2024-05-11T12:13:24.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:24.551+0000 7f1511200640 1 -- 172.21.15.167:0/2136549605 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f150c13d380 con 0x7f150c1428d0 2024-05-11T12:13:25.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.429+0000 7f1503400640 1 -- 172.21.15.167:0/2136549605 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v66) ==== 181+0+0 (secure 0 0 0) 0x7f1504061130 con 0x7f150c1428d0 2024-05-11T12:13:25.429 INFO:teuthology.orchestra.run.smithi167.stderr:pool 'cephfs_metadata' removed 2024-05-11T12:13:25.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.433+0000 7f1511200640 1 -- 172.21.15.167:0/2136549605 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f14e80762d0 msgr2=0x7f14e8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:25.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.433+0000 7f1511200640 1 --2- 172.21.15.167:0/2136549605 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f14e80762d0 0x7f14e8078790 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f14fc00a990 tx=0x7f14fc008040 comp rx=0 tx=0).stop 2024-05-11T12:13:25.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.433+0000 7f1511200640 1 -- 172.21.15.167:0/2136549605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f150c1428d0 msgr2=0x7f150c141290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:25.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.433+0000 7f1511200640 1 --2- 172.21.15.167:0/2136549605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f150c1428d0 0x7f150c141290 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f150400c780 tx=0x7f150400cc40 comp rx=0 tx=0).stop 2024-05-11T12:13:25.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.433+0000 7f1511200640 1 -- 172.21.15.167:0/2136549605 shutdown_connections 2024-05-11T12:13:25.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.433+0000 7f1511200640 1 --2- 172.21.15.167:0/2136549605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f150c1428d0 0x7f150c141290 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:25.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.433+0000 7f1511200640 1 --2- 172.21.15.167:0/2136549605 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f150c13ffd0 0x7f150c140d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:25.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.433+0000 7f1511200640 1 --2- 172.21.15.167:0/2136549605 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f14e80762d0 0x7f14e8078790 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:25.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.433+0000 7f1511200640 1 --2- 172.21.15.167:0/2136549605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f150c06d9a0 0x7f150c155940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:25.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.433+0000 7f1511200640 1 -- 172.21.15.167:0/2136549605 >> 172.21.15.167:0/2136549605 conn(0x7f150c0b32a0 msgr2=0x7f150c06d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:25.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.436+0000 7f1511200640 1 -- 172.21.15.167:0/2136549605 shutdown_connections 2024-05-11T12:13:25.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.436+0000 7f1511200640 1 -- 172.21.15.167:0/2136549605 wait complete. 2024-05-11T12:13:25.463 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-11T12:13:25.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.658+0000 7f4722a00640 1 Processor -- start 2024-05-11T12:13:25.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.658+0000 7f4722a00640 1 -- start start 2024-05-11T12:13:25.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.659+0000 7f4722a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f471c15cb90 0x7f471c15fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:25.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.660+0000 7f4722a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f471c15cf70 0x7f471c06d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:25.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.660+0000 7f4722a00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f471c160340 con 0x7f471c06d9a0 2024-05-11T12:13:25.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.660+0000 7f4722a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f471c06de10 con 0x7f471c15cf70 2024-05-11T12:13:25.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.660+0000 7f4722a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f471c1600e0 con 0x7f471c15cb90 2024-05-11T12:13:25.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.660+0000 7f4722000640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f471c06d9a0 0x7f471c0b3b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:37374/0 (socket says 172.21.15.167:37374) 2024-05-11T12:13:25.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.660+0000 7f4722000640 1 -- 172.21.15.167:0/1206860293 learned_addr learned my addr 172.21.15.167:0/1206860293 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:13:25.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.660+0000 7f4720c00640 1 --2- 172.21.15.167:0/1206860293 >> v2:172.21.15.204:3300/0 conn(0x7f471c15cf70 0x7f471c06d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:25.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.660+0000 7f4721600640 1 --2- 172.21.15.167:0/1206860293 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f471c15cb90 0x7f471c15fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:25.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.661+0000 7f471be00640 1 -- 172.21.15.167:0/1206860293 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 396201885 0 0) 0x7f471c160340 con 0x7f471c06d9a0 2024-05-11T12:13:25.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.661+0000 7f471be00640 1 -- 172.21.15.167:0/1206860293 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4704003520 con 0x7f471c06d9a0 2024-05-11T12:13:25.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.661+0000 7f471be00640 1 -- 172.21.15.167:0/1206860293 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1780025153 0 0) 0x7f4704003520 con 0x7f471c06d9a0 2024-05-11T12:13:25.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.661+0000 7f471be00640 1 -- 172.21.15.167:0/1206860293 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f471c15cb90 msgr2=0x7f471c15fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:25.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.661+0000 7f471be00640 1 --2- 172.21.15.167:0/1206860293 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f471c15cb90 0x7f471c15fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:25.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.661+0000 7f471be00640 1 -- 172.21.15.167:0/1206860293 >> v2:172.21.15.204:3300/0 conn(0x7f471c15cf70 msgr2=0x7f471c06d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:25.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.661+0000 7f471be00640 1 --2- 172.21.15.167:0/1206860293 >> v2:172.21.15.204:3300/0 conn(0x7f471c15cf70 0x7f471c06d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:25.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.661+0000 7f471be00640 1 -- 172.21.15.167:0/1206860293 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f471c160620 con 0x7f471c06d9a0 2024-05-11T12:13:25.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.661+0000 7f471be00640 1 -- 172.21.15.167:0/1206860293 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f4714001a70 con 0x7f471c06d9a0 2024-05-11T12:13:25.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.661+0000 7f471be00640 1 -- 172.21.15.167:0/1206860293 >> v1:172.21.15.167:6789/0 conn(0x7f471c06d9a0 legacy=0x7f471c0b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:25.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.661+0000 7f471be00640 1 --2- 172.21.15.167:0/1206860293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4704003a30 0x7f4704003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:25.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.661+0000 7f471be00640 1 --2- 172.21.15.167:0/1206860293 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f471c15cb90 0x7f4704004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:25.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.661+0000 7f471be00640 1 --2- 172.21.15.167:0/1206860293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4704004d40 0x7f47040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:25.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.661+0000 7f471be00640 1 -- 172.21.15.167:0/1206860293 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f471c06de10 con 0x7f4704003a30 2024-05-11T12:13:25.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.661+0000 7f471be00640 1 -- 172.21.15.167:0/1206860293 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f471c1600e0 con 0x7f4704004d40 2024-05-11T12:13:25.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.661+0000 7f471be00640 1 -- 172.21.15.167:0/1206860293 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f470400b9f0 con 0x7f471c15cb90 2024-05-11T12:13:25.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.661+0000 7f4721600640 1 --2- 172.21.15.167:0/1206860293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4704003a30 0x7f4704003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:25.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.662+0000 7f4721600640 1 -- 172.21.15.167:0/1206860293 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f471c15cb90 msgr2=0x7f4704004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:25.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.662+0000 7f4721600640 1 --2- 172.21.15.167:0/1206860293 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f471c15cb90 0x7f4704004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:25.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.662+0000 7f4721600640 1 -- 172.21.15.167:0/1206860293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4704004d40 msgr2=0x7f47040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:25.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.662+0000 7f4721600640 1 --2- 172.21.15.167:0/1206860293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4704004d40 0x7f47040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:25.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.662+0000 7f4721600640 1 -- 172.21.15.167:0/1206860293 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4704003520 con 0x7f4704003a30 2024-05-11T12:13:25.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.663+0000 7f4721600640 1 --2- 172.21.15.167:0/1206860293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4704003a30 0x7f4704003e50 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f4710009ac0 tx=0x7f4704005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daafce26ad52fd15 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:25.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.663+0000 7f471be00640 1 -- 172.21.15.167:0/1206860293 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4710036070 con 0x7f4704003a30 2024-05-11T12:13:25.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.663+0000 7f471be00640 1 -- 172.21.15.167:0/1206860293 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4710031400 con 0x7f4704003a30 2024-05-11T12:13:25.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.663+0000 7f471be00640 1 -- 172.21.15.167:0/1206860293 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4710009690 con 0x7f4704003a30 2024-05-11T12:13:25.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.664+0000 7f4722a00640 1 -- 172.21.15.167:0/1206860293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4704003a30 msgr2=0x7f4704003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:25.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.664+0000 7f4722a00640 1 --2- 172.21.15.167:0/1206860293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4704003a30 0x7f4704003e50 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f4710009ac0 tx=0x7f4704005830 comp rx=0 tx=0).stop 2024-05-11T12:13:25.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.664+0000 7f4722000640 1 -- 172.21.15.167:0/1206860293 reap_dead start 2024-05-11T12:13:25.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.664+0000 7f4722a00640 1 -- 172.21.15.167:0/1206860293 shutdown_connections 2024-05-11T12:13:25.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.664+0000 7f4722a00640 1 -- 172.21.15.167:0/1206860293 >> 172.21.15.167:0/1206860293 conn(0x7f471c0b32a0 msgr2=0x7f471c0d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:25.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.664+0000 7f4722a00640 1 -- 172.21.15.167:0/1206860293 shutdown_connections 2024-05-11T12:13:25.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.664+0000 7f4722a00640 1 -- 172.21.15.167:0/1206860293 wait complete. 2024-05-11T12:13:25.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.666+0000 7f4722a00640 1 Processor -- start 2024-05-11T12:13:25.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.666+0000 7f4722a00640 1 -- start start 2024-05-11T12:13:25.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.667+0000 7f4722a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4704003a30 0x7f471c13dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:25.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.667+0000 7f4722a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f471c140ab0 0x7f471c156c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:25.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.667+0000 7f4722a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f471c15f230 0x7f471c0724d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:25.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.667+0000 7f4722a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f471c15d3f0 con 0x7f4704003a30 2024-05-11T12:13:25.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.667+0000 7f4722a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f471c072160 con 0x7f471c15f230 2024-05-11T12:13:25.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.667+0000 7f4722a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f471c15d570 con 0x7f471c140ab0 2024-05-11T12:13:25.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.667+0000 7f4721600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4704003a30 0x7f471c13dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:25.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.667+0000 7f4721600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4704003a30 0x7f471c13dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39534/0 (socket says 172.21.15.167:39534) 2024-05-11T12:13:25.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.667+0000 7f4721600640 1 -- 172.21.15.167:0/1629146793 learned_addr learned my addr 172.21.15.167:0/1629146793 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:25.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.667+0000 7f4720c00640 1 --2- 172.21.15.167:0/1629146793 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f471c140ab0 0x7f471c156c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:25.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.667+0000 7f4722000640 1 --2- 172.21.15.167:0/1629146793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f471c15f230 0x7f471c0724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:25.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.667+0000 7f4721600640 1 -- 172.21.15.167:0/1629146793 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f471c140ab0 msgr2=0x7f471c156c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:25.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.667+0000 7f4721600640 1 --2- 172.21.15.167:0/1629146793 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f471c140ab0 0x7f471c156c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:25.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.667+0000 7f4721600640 1 -- 172.21.15.167:0/1629146793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f471c15f230 msgr2=0x7f471c0724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:25.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.667+0000 7f4721600640 1 --2- 172.21.15.167:0/1629146793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f471c15f230 0x7f471c0724d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:25.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.667+0000 7f4721600640 1 -- 172.21.15.167:0/1629146793 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f471c13ee70 con 0x7f4704003a30 2024-05-11T12:13:25.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.667+0000 7f4720c00640 1 --2- 172.21.15.167:0/1629146793 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f471c140ab0 0x7f471c156c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:25.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.668+0000 7f4721600640 1 --2- 172.21.15.167:0/1629146793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4704003a30 0x7f471c13dff0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f4710009ac0 tx=0x7f4710004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:25.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.668+0000 7f4719600640 1 -- 172.21.15.167:0/1629146793 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4710040020 con 0x7f4704003a30 2024-05-11T12:13:25.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.668+0000 7f4719600640 1 -- 172.21.15.167:0/1629146793 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4710003bc0 con 0x7f4704003a30 2024-05-11T12:13:25.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.668+0000 7f4719600640 1 -- 172.21.15.167:0/1629146793 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4710036040 con 0x7f4704003a30 2024-05-11T12:13:25.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.668+0000 7f4722a00640 1 -- 172.21.15.167:0/1629146793 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f471c13e5d0 con 0x7f4704003a30 2024-05-11T12:13:25.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.668+0000 7f4722a00640 1 -- 172.21.15.167:0/1629146793 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f471c13eb90 con 0x7f4704003a30 2024-05-11T12:13:25.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.669+0000 7f4722a00640 1 -- 172.21.15.167:0/1629146793 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f471c0694b0 con 0x7f4704003a30 2024-05-11T12:13:25.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.671+0000 7f4719600640 1 -- 172.21.15.167:0/1629146793 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4710043020 con 0x7f4704003a30 2024-05-11T12:13:25.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.671+0000 7f4719600640 1 --2- 172.21.15.167:0/1629146793 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f46fc0762d0 0x7f46fc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:25.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.672+0000 7f4719600640 1 -- 172.21.15.167:0/1629146793 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5749+0+0 (secure 0 0 0) 0x7f47100b77c0 con 0x7f4704003a30 2024-05-11T12:13:25.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.672+0000 7f4720c00640 1 --2- 172.21.15.167:0/1629146793 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f46fc0762d0 0x7f46fc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:25.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.673+0000 7f4720c00640 1 --2- 172.21.15.167:0/1629146793 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f46fc0762d0 0x7f46fc078790 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f471c16a120 tx=0x7f4708008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:25.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.677+0000 7f4719600640 1 -- 172.21.15.167:0/1629146793 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f471004a070 con 0x7f4704003a30 2024-05-11T12:13:25.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:25.885+0000 7f4722a00640 1 -- 172.21.15.167:0/1629146793 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f471c15d720 con 0x7f4704003a30 2024-05-11T12:13:26.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.433+0000 7f4719600640 1 -- 172.21.15.167:0/1629146793 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v67) ==== 169+0+0 (secure 0 0 0) 0x7f47100044e0 con 0x7f4704003a30 2024-05-11T12:13:26.433 INFO:teuthology.orchestra.run.smithi167.stderr:pool 'cephfs_data' removed 2024-05-11T12:13:26.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.437+0000 7f4722a00640 1 -- 172.21.15.167:0/1629146793 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f46fc0762d0 msgr2=0x7f46fc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:26.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.437+0000 7f4722a00640 1 --2- 172.21.15.167:0/1629146793 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f46fc0762d0 0x7f46fc078790 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f471c16a120 tx=0x7f4708008040 comp rx=0 tx=0).stop 2024-05-11T12:13:26.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.437+0000 7f4722a00640 1 -- 172.21.15.167:0/1629146793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4704003a30 msgr2=0x7f471c13dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:26.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.437+0000 7f4722a00640 1 --2- 172.21.15.167:0/1629146793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4704003a30 0x7f471c13dff0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f4710009ac0 tx=0x7f4710004730 comp rx=0 tx=0).stop 2024-05-11T12:13:26.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.437+0000 7f4722a00640 1 -- 172.21.15.167:0/1629146793 shutdown_connections 2024-05-11T12:13:26.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.437+0000 7f4722a00640 1 --2- 172.21.15.167:0/1629146793 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f46fc0762d0 0x7f46fc078790 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:26.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.437+0000 7f4722a00640 1 --2- 172.21.15.167:0/1629146793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f471c15f230 0x7f471c0724d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:26.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.437+0000 7f4722a00640 1 --2- 172.21.15.167:0/1629146793 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f471c140ab0 0x7f471c156c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:26.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.437+0000 7f4722a00640 1 --2- 172.21.15.167:0/1629146793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4704003a30 0x7f471c13dff0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:26.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.437+0000 7f4722a00640 1 -- 172.21.15.167:0/1629146793 >> 172.21.15.167:0/1629146793 conn(0x7f471c0b32a0 msgr2=0x7f471c073770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:26.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.438+0000 7f4722a00640 1 -- 172.21.15.167:0/1629146793 shutdown_connections 2024-05-11T12:13:26.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.438+0000 7f4722a00640 1 -- 172.21.15.167:0/1629146793 wait complete. 2024-05-11T12:13:26.468 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-11T12:13:26.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.648+0000 7ff527800640 1 Processor -- start 2024-05-11T12:13:26.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.648+0000 7ff527800640 1 -- start start 2024-05-11T12:13:26.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.650+0000 7ff527800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff52015ac80 0x7ff520074260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:26.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.650+0000 7ff527800640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5200b65e0 con 0x7ff5200b3d50 2024-05-11T12:13:26.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.650+0000 7ff527800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5200b68c0 con 0x7ff520161380 2024-05-11T12:13:26.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.650+0000 7ff527800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7ff52006e3c0 con 0x7ff52015ac80 2024-05-11T12:13:26.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.650+0000 7ff51fe00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7ff5200b3d50 0x7ff520160c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:37378/0 (socket says 172.21.15.167:37378) 2024-05-11T12:13:26.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.650+0000 7ff51fe00640 1 -- 172.21.15.167:0/1025475661 learned_addr learned my addr 172.21.15.167:0/1025475661 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:13:26.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.651+0000 7ff524e00640 1 --2- 172.21.15.167:0/1025475661 >> v2:172.21.15.204:3300/0 conn(0x7ff52015ac80 0x7ff520074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:26.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.651+0000 7ff51f400640 1 -- 172.21.15.167:0/1025475661 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4256908436 0 0) 0x7ff5200b65e0 con 0x7ff5200b3d50 2024-05-11T12:13:26.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.651+0000 7ff51f400640 1 -- 172.21.15.167:0/1025475661 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff508003520 con 0x7ff5200b3d50 2024-05-11T12:13:26.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.651+0000 7ff51f400640 1 -- 172.21.15.167:0/1025475661 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 945007560 0 0) 0x7ff508003520 con 0x7ff5200b3d50 2024-05-11T12:13:26.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.651+0000 7ff51f400640 1 -- 172.21.15.167:0/1025475661 >> v2:172.21.15.204:3300/0 conn(0x7ff52015ac80 msgr2=0x7ff520074260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:26.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.651+0000 7ff51f400640 1 --2- 172.21.15.167:0/1025475661 >> v2:172.21.15.204:3300/0 conn(0x7ff52015ac80 0x7ff520074260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:26.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.651+0000 7ff51f400640 1 -- 172.21.15.167:0/1025475661 >> v1:172.21.15.204:6789/0 conn(0x7ff520161380 legacy=0x7ff520072950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:26.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.651+0000 7ff51f400640 1 -- 172.21.15.167:0/1025475661 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5200b6ba0 con 0x7ff5200b3d50 2024-05-11T12:13:26.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.651+0000 7ff51f400640 1 -- 172.21.15.167:0/1025475661 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7ff5100016f0 con 0x7ff5200b3d50 2024-05-11T12:13:26.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.652+0000 7ff51f400640 1 -- 172.21.15.167:0/1025475661 >> v1:172.21.15.167:6789/0 conn(0x7ff5200b3d50 legacy=0x7ff520160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:26.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.652+0000 7ff51f400640 1 --2- 172.21.15.167:0/1025475661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff508003aa0 0x7ff508003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:26.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.652+0000 7ff51f400640 1 --2- 172.21.15.167:0/1025475661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff508004800 0x7ff508004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:26.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.652+0000 7ff51f400640 1 --2- 172.21.15.167:0/1025475661 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff508005210 0x7ff50800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:26.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.652+0000 7ff51f400640 1 -- 172.21.15.167:0/1025475661 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff50800c130 con 0x7ff508004800 2024-05-11T12:13:26.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.652+0000 7ff51f400640 1 -- 172.21.15.167:0/1025475661 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff50800c2e0 con 0x7ff508003aa0 2024-05-11T12:13:26.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.652+0000 7ff51f400640 1 -- 172.21.15.167:0/1025475661 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff50800c4e0 con 0x7ff508005210 2024-05-11T12:13:26.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.652+0000 7ff51fe00640 1 --2- 172.21.15.167:0/1025475661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff508004800 0x7ff508004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:26.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.652+0000 7ff524e00640 1 --2- 172.21.15.167:0/1025475661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff508003aa0 0x7ff508003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:26.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.652+0000 7ff51fe00640 1 -- 172.21.15.167:0/1025475661 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff508005210 msgr2=0x7ff50800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:26.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.652+0000 7ff51fe00640 1 --2- 172.21.15.167:0/1025475661 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff508005210 0x7ff50800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:26.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.652+0000 7ff51fe00640 1 -- 172.21.15.167:0/1025475661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff508003aa0 msgr2=0x7ff508003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:26.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.652+0000 7ff51fe00640 1 --2- 172.21.15.167:0/1025475661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff508003aa0 0x7ff508003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:26.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.652+0000 7ff51fe00640 1 -- 172.21.15.167:0/1025475661 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff508003520 con 0x7ff508004800 2024-05-11T12:13:26.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.652+0000 7ff525800640 1 -- 172.21.15.167:0/1025475661 reap_dead start 2024-05-11T12:13:26.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.653+0000 7ff51fe00640 1 --2- 172.21.15.167:0/1025475661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff508004800 0x7ff508004c80 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff51000c160 tx=0x7ff508005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=130b2699b5fa4bb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:26.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.654+0000 7ff51f400640 1 -- 172.21.15.167:0/1025475661 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff510007c60 con 0x7ff508004800 2024-05-11T12:13:26.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.654+0000 7ff51f400640 1 -- 172.21.15.167:0/1025475661 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff510005af0 con 0x7ff508004800 2024-05-11T12:13:26.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.654+0000 7ff51f400640 1 -- 172.21.15.167:0/1025475661 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff510007070 con 0x7ff508004800 2024-05-11T12:13:26.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.654+0000 7ff527800640 1 -- 172.21.15.167:0/1025475661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff508004800 msgr2=0x7ff508004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:26.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.654+0000 7ff527800640 1 --2- 172.21.15.167:0/1025475661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff508004800 0x7ff508004c80 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff51000c160 tx=0x7ff508005830 comp rx=0 tx=0).stop 2024-05-11T12:13:26.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.654+0000 7ff527800640 1 -- 172.21.15.167:0/1025475661 shutdown_connections 2024-05-11T12:13:26.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.654+0000 7ff527800640 1 --2- 172.21.15.167:0/1025475661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff508004800 0x7ff508004c80 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:26.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.654+0000 7ff527800640 1 -- 172.21.15.167:0/1025475661 >> 172.21.15.167:0/1025475661 conn(0x7ff52006c450 msgr2=0x7ff52006c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:26.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.655+0000 7ff527800640 1 -- 172.21.15.167:0/1025475661 shutdown_connections 2024-05-11T12:13:26.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.655+0000 7ff527800640 1 -- 172.21.15.167:0/1025475661 wait complete. 2024-05-11T12:13:26.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.656+0000 7ff527800640 1 Processor -- start 2024-05-11T12:13:26.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.657+0000 7ff527800640 1 -- start start 2024-05-11T12:13:26.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.657+0000 7ff527800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff508004800 0x7ff52015b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:26.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.657+0000 7ff527800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff520153fb0 0x7ff52015b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:26.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.657+0000 7ff527800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff520158950 0x7ff520158dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:26.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.658+0000 7ff527800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff5200b44f0 con 0x7ff520158950 2024-05-11T12:13:26.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.658+0000 7ff527800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff52015b060 con 0x7ff520153fb0 2024-05-11T12:13:26.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.658+0000 7ff527800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff520074260 con 0x7ff508004800 2024-05-11T12:13:26.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.658+0000 7ff525800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff520158950 0x7ff520158dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:26.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.658+0000 7ff525800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff520158950 0x7ff520158dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39546/0 (socket says 172.21.15.167:39546) 2024-05-11T12:13:26.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.658+0000 7ff525800640 1 -- 172.21.15.167:0/2890698638 learned_addr learned my addr 172.21.15.167:0/2890698638 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:26.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.658+0000 7ff525800640 1 -- 172.21.15.167:0/2890698638 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff508004800 msgr2=0x7ff52015b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:26.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.658+0000 7ff525800640 1 --2- 172.21.15.167:0/2890698638 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff508004800 0x7ff52015b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:26.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.658+0000 7ff525800640 1 -- 172.21.15.167:0/2890698638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff520153fb0 msgr2=0x7ff52015b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:26.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.658+0000 7ff525800640 1 --2- 172.21.15.167:0/2890698638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff520153fb0 0x7ff52015b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:26.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.658+0000 7ff525800640 1 -- 172.21.15.167:0/2890698638 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff520154520 con 0x7ff520158950 2024-05-11T12:13:26.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.658+0000 7ff525800640 1 --2- 172.21.15.167:0/2890698638 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff520158950 0x7ff520158dd0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7ff51800c700 tx=0x7ff51800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:26.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.659+0000 7ff51cc00640 1 -- 172.21.15.167:0/2890698638 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff518005d40 con 0x7ff520158950 2024-05-11T12:13:26.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.659+0000 7ff51cc00640 1 -- 172.21.15.167:0/2890698638 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff51800a640 con 0x7ff520158950 2024-05-11T12:13:26.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.659+0000 7ff51cc00640 1 -- 172.21.15.167:0/2890698638 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51800aa30 con 0x7ff520158950 2024-05-11T12:13:26.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.659+0000 7ff527800640 1 -- 172.21.15.167:0/2890698638 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff520154800 con 0x7ff520158950 2024-05-11T12:13:26.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.659+0000 7ff527800640 1 -- 172.21.15.167:0/2890698638 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff520154dc0 con 0x7ff520158950 2024-05-11T12:13:26.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.660+0000 7ff527800640 1 -- 172.21.15.167:0/2890698638 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff520074590 con 0x7ff520158950 2024-05-11T12:13:26.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.661+0000 7ff51cc00640 1 -- 172.21.15.167:0/2890698638 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff518011070 con 0x7ff520158950 2024-05-11T12:13:26.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.662+0000 7ff51cc00640 1 --2- 172.21.15.167:0/2890698638 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff504076300 0x7ff5040787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:26.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.663+0000 7ff51cc00640 1 -- 172.21.15.167:0/2890698638 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5335+0+0 (secure 0 0 0) 0x7ff5180960d0 con 0x7ff520158950 2024-05-11T12:13:26.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.663+0000 7ff524e00640 1 --2- 172.21.15.167:0/2890698638 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff504076300 0x7ff5040787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:26.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.663+0000 7ff524e00640 1 --2- 172.21.15.167:0/2890698638 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff504076300 0x7ff5040787c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff514009990 tx=0x7ff514008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:26.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.667+0000 7ff51cc00640 1 -- 172.21.15.167:0/2890698638 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff520074590 con 0x7ff520158950 2024-05-11T12:13:26.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.880+0000 7ff527800640 1 -- 172.21.15.167:0/2890698638 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7ff520069520 con 0x7ff520158950 2024-05-11T12:13:26.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.886+0000 7ff51cc00640 1 -- 172.21.15.167:0/2890698638 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) ==== 104+0+0 (secure 0 0 0) 0x7ff51805fb00 con 0x7ff520158950 2024-05-11T12:13:26.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.889+0000 7ff527800640 1 -- 172.21.15.167:0/2890698638 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff504076300 msgr2=0x7ff5040787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:26.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.889+0000 7ff527800640 1 --2- 172.21.15.167:0/2890698638 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff504076300 0x7ff5040787c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff514009990 tx=0x7ff514008040 comp rx=0 tx=0).stop 2024-05-11T12:13:26.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.889+0000 7ff527800640 1 -- 172.21.15.167:0/2890698638 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff520158950 msgr2=0x7ff520158dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:26.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.889+0000 7ff527800640 1 --2- 172.21.15.167:0/2890698638 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff520158950 0x7ff520158dd0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7ff51800c700 tx=0x7ff51800cbc0 comp rx=0 tx=0).stop 2024-05-11T12:13:26.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.889+0000 7ff527800640 1 -- 172.21.15.167:0/2890698638 shutdown_connections 2024-05-11T12:13:26.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.889+0000 7ff527800640 1 --2- 172.21.15.167:0/2890698638 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff520158950 0x7ff520158dd0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:26.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.889+0000 7ff527800640 1 --2- 172.21.15.167:0/2890698638 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff504076300 0x7ff5040787c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:26.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.889+0000 7ff527800640 1 --2- 172.21.15.167:0/2890698638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff520153fb0 0x7ff52015b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:26.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.889+0000 7ff527800640 1 --2- 172.21.15.167:0/2890698638 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff508004800 0x7ff52015b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:26.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.889+0000 7ff527800640 1 -- 172.21.15.167:0/2890698638 >> 172.21.15.167:0/2890698638 conn(0x7ff52006c450 msgr2=0x7ff52006c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:26.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.889+0000 7ff527800640 1 -- 172.21.15.167:0/2890698638 shutdown_connections 2024-05-11T12:13:26.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:26.890+0000 7ff527800640 1 -- 172.21.15.167:0/2890698638 wait complete. 2024-05-11T12:13:26.908 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_reject' 2024-05-11T12:13:27.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.109+0000 7ff8fcc00640 1 Processor -- start 2024-05-11T12:13:27.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.110+0000 7ff8fcc00640 1 -- start start 2024-05-11T12:13:27.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.111+0000 7ff8fcc00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff8f8148a10 0x7ff8f80b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:27.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.111+0000 7ff8fcc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8f80cdc60 0x7ff8f80ce110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:27.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.111+0000 7ff8fcc00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8f8148df0 con 0x7ff8f80b4fa0 2024-05-11T12:13:27.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.112+0000 7ff8fcc00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7ff8f80b5500 con 0x7ff8f8148a10 2024-05-11T12:13:27.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.112+0000 7ff8fcc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff8f80d1440 con 0x7ff8f80cdc60 2024-05-11T12:13:27.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.112+0000 7ff8f7400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff8f8148a10 0x7ff8f80b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:27.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.112+0000 7ff8f7400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff8f8148a10 0x7ff8f80b4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39560/0 (socket says 172.21.15.167:39560) 2024-05-11T12:13:27.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.112+0000 7ff8f7400640 1 -- 172.21.15.167:0/691501948 learned_addr learned my addr 172.21.15.167:0/691501948 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:27.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.112+0000 7ff8f7400640 1 -- 172.21.15.167:0/691501948 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8f80cdc60 msgr2=0x7ff8f80ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:27.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.112+0000 7ff8f7e00640 1 --2- 172.21.15.167:0/691501948 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8f80cdc60 0x7ff8f80ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:27.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.112+0000 7ff8f7400640 1 --2- 172.21.15.167:0/691501948 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8f80cdc60 0x7ff8f80ce110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:27.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.112+0000 7ff8f7400640 1 -- 172.21.15.167:0/691501948 >> v1:172.21.15.204:6789/0 conn(0x7ff8f80b4fa0 legacy=0x7ff8f80b6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:27.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.112+0000 7ff8f7400640 1 -- 172.21.15.167:0/691501948 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f80d1720 con 0x7ff8f8148a10 2024-05-11T12:13:27.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.112+0000 7ff8f7e00640 1 --2- 172.21.15.167:0/691501948 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8f80cdc60 0x7ff8f80ce110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:27.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.113+0000 7ff8f7400640 1 --2- 172.21.15.167:0/691501948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8f8148a10 0x7ff8f80b4a60 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8009870 tx=0x7ff8e802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=246672c2f259ab4 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:27.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.114+0000 7ff8f6000640 1 -- 172.21.15.167:0/691501948 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e8037070 con 0x7ff8f8148a10 2024-05-11T12:13:27.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.114+0000 7ff8f6000640 1 -- 172.21.15.167:0/691501948 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8e8032440 con 0x7ff8f8148a10 2024-05-11T12:13:27.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.114+0000 7ff8f6000640 1 -- 172.21.15.167:0/691501948 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e8032810 con 0x7ff8f8148a10 2024-05-11T12:13:27.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.115+0000 7ff8fcc00640 1 -- 172.21.15.167:0/691501948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8f8148a10 msgr2=0x7ff8f80b4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:27.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.115+0000 7ff8fcc00640 1 --2- 172.21.15.167:0/691501948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8f8148a10 0x7ff8f80b4a60 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8009870 tx=0x7ff8e802a1c0 comp rx=0 tx=0).stop 2024-05-11T12:13:27.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.115+0000 7ff8fcc00640 1 -- 172.21.15.167:0/691501948 shutdown_connections 2024-05-11T12:13:27.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.115+0000 7ff8fcc00640 1 --2- 172.21.15.167:0/691501948 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8f80cdc60 0x7ff8f80ce110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:27.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.115+0000 7ff8fcc00640 1 --2- 172.21.15.167:0/691501948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8f8148a10 0x7ff8f80b4a60 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:27.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.115+0000 7ff8fcc00640 1 -- 172.21.15.167:0/691501948 >> 172.21.15.167:0/691501948 conn(0x7ff8f80d2e20 msgr2=0x7ff8f80d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:27.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.115+0000 7ff8fcc00640 1 -- 172.21.15.167:0/691501948 shutdown_connections 2024-05-11T12:13:27.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.115+0000 7ff8fcc00640 1 -- 172.21.15.167:0/691501948 wait complete. 2024-05-11T12:13:27.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.117+0000 7ff8fcc00640 1 Processor -- start 2024-05-11T12:13:27.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.117+0000 7ff8fcc00640 1 -- start start 2024-05-11T12:13:27.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.118+0000 7ff8fcc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8f80b4fa0 0x7ff8f815b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:27.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.118+0000 7ff8fcc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8f80cdc60 0x7ff8f815a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:27.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.118+0000 7ff8fcc00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8f8148a10 0x7ff8f80b2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:27.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.118+0000 7ff8fcc00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f8163b70 con 0x7ff8f8148a10 2024-05-11T12:13:27.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.118+0000 7ff8fcc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f80d1ce0 con 0x7ff8f80cdc60 2024-05-11T12:13:27.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.118+0000 7ff8fcc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff8f80b6930 con 0x7ff8f80b4fa0 2024-05-11T12:13:27.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.118+0000 7ff8f7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8f8148a10 0x7ff8f80b2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:27.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.118+0000 7ff8f7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8f8148a10 0x7ff8f80b2a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39576/0 (socket says 172.21.15.167:39576) 2024-05-11T12:13:27.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.118+0000 7ff8f7e00640 1 -- 172.21.15.167:0/653887448 learned_addr learned my addr 172.21.15.167:0/653887448 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:27.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.118+0000 7ff8f7400640 1 --2- 172.21.15.167:0/653887448 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8f80b4fa0 0x7ff8f815b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:27.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.118+0000 7ff8f6a00640 1 --2- 172.21.15.167:0/653887448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8f80cdc60 0x7ff8f815a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:27.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.118+0000 7ff8f7e00640 1 -- 172.21.15.167:0/653887448 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8f80b4fa0 msgr2=0x7ff8f815b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:27.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.118+0000 7ff8f7e00640 1 --2- 172.21.15.167:0/653887448 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8f80b4fa0 0x7ff8f815b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:27.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.118+0000 7ff8f7e00640 1 -- 172.21.15.167:0/653887448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8f80cdc60 msgr2=0x7ff8f815a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:27.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.118+0000 7ff8f7e00640 1 --2- 172.21.15.167:0/653887448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8f80cdc60 0x7ff8f815a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:27.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.118+0000 7ff8f7e00640 1 -- 172.21.15.167:0/653887448 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f81eb560 con 0x7ff8f8148a10 2024-05-11T12:13:27.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.118+0000 7ff8f6a00640 1 --2- 172.21.15.167:0/653887448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8f80cdc60 0x7ff8f815a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:27.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.119+0000 7ff8f7e00640 1 --2- 172.21.15.167:0/653887448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8f8148a10 0x7ff8f80b2a40 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7ff8f000c7f0 tx=0x7ff8f000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:27.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.119+0000 7ff8ef400640 1 -- 172.21.15.167:0/653887448 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0015070 con 0x7ff8f8148a10 2024-05-11T12:13:27.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.119+0000 7ff8ef400640 1 -- 172.21.15.167:0/653887448 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8f0007820 con 0x7ff8f8148a10 2024-05-11T12:13:27.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.119+0000 7ff8ef400640 1 -- 172.21.15.167:0/653887448 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0007c10 con 0x7ff8f8148a10 2024-05-11T12:13:27.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.119+0000 7ff8fcc00640 1 -- 172.21.15.167:0/653887448 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8f81594b0 con 0x7ff8f8148a10 2024-05-11T12:13:27.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.119+0000 7ff8fcc00640 1 -- 172.21.15.167:0/653887448 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8f8159a70 con 0x7ff8f8148a10 2024-05-11T12:13:27.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.120+0000 7ff8fcc00640 1 -- 172.21.15.167:0/653887448 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8f8069bc0 con 0x7ff8f8148a10 2024-05-11T12:13:27.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.122+0000 7ff8ef400640 1 -- 172.21.15.167:0/653887448 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff8f0018070 con 0x7ff8f8148a10 2024-05-11T12:13:27.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.123+0000 7ff8ef400640 1 --2- 172.21.15.167:0/653887448 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff8d40762d0 0x7ff8d4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:27.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.123+0000 7ff8ef400640 1 -- 172.21.15.167:0/653887448 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5335+0+0 (secure 0 0 0) 0x7ff8f00975e0 con 0x7ff8f8148a10 2024-05-11T12:13:27.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.123+0000 7ff8f7400640 1 --2- 172.21.15.167:0/653887448 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff8d40762d0 0x7ff8d4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:27.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.124+0000 7ff8f7400640 1 --2- 172.21.15.167:0/653887448 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff8d40762d0 0x7ff8d4078790 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8034000 tx=0x7ff8e802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:27.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.127+0000 7ff8ef400640 1 -- 172.21.15.167:0/653887448 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff8f00610b0 con 0x7ff8f8148a10 2024-05-11T12:13:27.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.414+0000 7ff8fcc00640 1 -- 172.21.15.167:0/653887448 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_reject"]} v 0) -- 0x7ff8f80cf7d0 con 0x7ff8f8148a10 2024-05-11T12:13:27.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.450+0000 7ff8ef400640 1 -- 172.21.15.167:0/653887448 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_reject"]}]=0 v170) ==== 142+0+0 (secure 0 0 0) 0x7ff8f00609b0 con 0x7ff8f8148a10 2024-05-11T12:13:27.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.453+0000 7ff8fcc00640 1 -- 172.21.15.167:0/653887448 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff8d40762d0 msgr2=0x7ff8d4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:27.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.453+0000 7ff8fcc00640 1 --2- 172.21.15.167:0/653887448 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff8d40762d0 0x7ff8d4078790 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8034000 tx=0x7ff8e802e000 comp rx=0 tx=0).stop 2024-05-11T12:13:27.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.453+0000 7ff8fcc00640 1 -- 172.21.15.167:0/653887448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8f8148a10 msgr2=0x7ff8f80b2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:27.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.453+0000 7ff8fcc00640 1 --2- 172.21.15.167:0/653887448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8f8148a10 0x7ff8f80b2a40 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7ff8f000c7f0 tx=0x7ff8f000ccb0 comp rx=0 tx=0).stop 2024-05-11T12:13:27.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.454+0000 7ff8fcc00640 1 -- 172.21.15.167:0/653887448 shutdown_connections 2024-05-11T12:13:27.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.454+0000 7ff8fcc00640 1 --2- 172.21.15.167:0/653887448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8f8148a10 0x7ff8f80b2a40 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:27.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.454+0000 7ff8fcc00640 1 --2- 172.21.15.167:0/653887448 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff8d40762d0 0x7ff8d4078790 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:27.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.454+0000 7ff8fcc00640 1 --2- 172.21.15.167:0/653887448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8f80cdc60 0x7ff8f815a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:27.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.454+0000 7ff8fcc00640 1 --2- 172.21.15.167:0/653887448 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8f80b4fa0 0x7ff8f815b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:27.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.454+0000 7ff8fcc00640 1 -- 172.21.15.167:0/653887448 >> 172.21.15.167:0/653887448 conn(0x7ff8f80d2e20 msgr2=0x7ff8f80b3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:27.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.454+0000 7ff8fcc00640 1 -- 172.21.15.167:0/653887448 shutdown_connections 2024-05-11T12:13:27.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.454+0000 7ff8fcc00640 1 -- 172.21.15.167:0/653887448 wait complete. 2024-05-11T12:13:27.472 INFO:tasks.cephfs_test_runner:test_session_reject (tasks.cephfs.test_sessionmap.TestSessionMap) ... skipped 'Requires FUSE client to inject client metadata' 2024-05-11T12:13:27.472 INFO:tasks.cephfs_test_runner:Starting test: test_tell_conn_close (tasks.cephfs.test_sessionmap.TestSessionMap) That when a `tell` command is sent using the python CLI, 2024-05-11T12:13:27.472 INFO:tasks.cephfs_test_runner:test_tell_conn_close (tasks.cephfs.test_sessionmap.TestSessionMap) 2024-05-11T12:13:27.473 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:13:27.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.673+0000 7f0f2a800640 1 Processor -- start 2024-05-11T12:13:27.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.673+0000 7f0f2a800640 1 -- start start 2024-05-11T12:13:27.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.675+0000 7f0f2a800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f2413eeb0 0x7f0f2406f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:27.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.675+0000 7f0f2a800640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0f240b50a0 0x7f0f24073cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:27.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.675+0000 7f0f2a800640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f2415fb90 con 0x7f0f2413f290 2024-05-11T12:13:27.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.675+0000 7f0f2a800640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f0f2415d5b0 con 0x7f0f240b50a0 2024-05-11T12:13:27.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.675+0000 7f0f2a800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0f2415fe70 con 0x7f0f2413eeb0 2024-05-11T12:13:27.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.675+0000 7f0f23e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0f240b50a0 0x7f0f24073cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:27.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.675+0000 7f0f1be00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f0f2413f290 0x7f0f240735c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:37380/0 (socket says 172.21.15.167:37380) 2024-05-11T12:13:27.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.675+0000 7f0f1be00640 1 -- 172.21.15.167:0/3878064802 learned_addr learned my addr 172.21.15.167:0/3878064802 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:13:27.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.675+0000 7f0f23400640 1 --2- 172.21.15.167:0/3878064802 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f2413eeb0 0x7f0f2406f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:27.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.675+0000 7f0f23e00640 1 -- 172.21.15.167:0/3878064802 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f2413eeb0 msgr2=0x7f0f2406f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:27.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.675+0000 7f0f23e00640 1 --2- 172.21.15.167:0/3878064802 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f2413eeb0 0x7f0f2406f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:27.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.675+0000 7f0f23e00640 1 -- 172.21.15.167:0/3878064802 >> v1:172.21.15.167:6789/0 conn(0x7f0f2413f290 legacy=0x7f0f240735c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:27.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.676+0000 7f0f23e00640 1 -- 172.21.15.167:0/3878064802 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f24160150 con 0x7f0f240b50a0 2024-05-11T12:13:27.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.676+0000 7f0f23400640 1 --2- 172.21.15.167:0/3878064802 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f2413eeb0 0x7f0f2406f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:27.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.676+0000 7f0f22a00640 1 -- 172.21.15.167:0/3878064802 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2398499867 0 0) 0x7f0f2415fb90 con 0x7f0f2413f290 2024-05-11T12:13:27.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.677+0000 7f0f23e00640 1 --2- 172.21.15.167:0/3878064802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f240b50a0 0x7f0f24073cd0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c009d90 tx=0x7f0f1c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d413dd83781bbc8a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:27.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.677+0000 7f0f22a00640 1 -- 172.21.15.167:0/3878064802 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c031600 con 0x7f0f240b50a0 2024-05-11T12:13:27.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.677+0000 7f0f22a00640 1 -- 172.21.15.167:0/3878064802 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f1c0317a0 con 0x7f0f240b50a0 2024-05-11T12:13:27.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.677+0000 7f0f22a00640 1 -- 172.21.15.167:0/3878064802 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c031b70 con 0x7f0f240b50a0 2024-05-11T12:13:27.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.678+0000 7f0f2a800640 1 -- 172.21.15.167:0/3878064802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f240b50a0 msgr2=0x7f0f24073cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:27.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.678+0000 7f0f2a800640 1 --2- 172.21.15.167:0/3878064802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f240b50a0 0x7f0f24073cd0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c009d90 tx=0x7f0f1c02a5d0 comp rx=0 tx=0).stop 2024-05-11T12:13:27.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.678+0000 7f0f2a800640 1 -- 172.21.15.167:0/3878064802 shutdown_connections 2024-05-11T12:13:27.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.678+0000 7f0f2a800640 1 --2- 172.21.15.167:0/3878064802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f240b50a0 0x7f0f24073cd0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:27.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.678+0000 7f0f2a800640 1 --2- 172.21.15.167:0/3878064802 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f2413eeb0 0x7f0f2406f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:27.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.678+0000 7f0f2a800640 1 -- 172.21.15.167:0/3878064802 >> 172.21.15.167:0/3878064802 conn(0x7f0f240cd8f0 msgr2=0x7f0f240cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:27.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.678+0000 7f0f2a800640 1 -- 172.21.15.167:0/3878064802 shutdown_connections 2024-05-11T12:13:27.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.678+0000 7f0f2a800640 1 -- 172.21.15.167:0/3878064802 wait complete. 2024-05-11T12:13:27.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.680+0000 7f0f2a800640 1 Processor -- start 2024-05-11T12:13:27.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.680+0000 7f0f2a800640 1 -- start start 2024-05-11T12:13:27.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.681+0000 7f0f2a800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f240b50a0 0x7f0f24139790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:27.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.681+0000 7f0f2a800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f2413eeb0 0x7f0f2413b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:27.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.681+0000 7f0f2a800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f2413f290 0x7f0f2413b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:27.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.681+0000 7f0f2a800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2406f9c0 con 0x7f0f2413f290 2024-05-11T12:13:27.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.681+0000 7f0f2a800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0f24160710 con 0x7f0f2413eeb0 2024-05-11T12:13:27.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.681+0000 7f0f2a800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0f2406fb40 con 0x7f0f240b50a0 2024-05-11T12:13:27.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.682+0000 7f0f23e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f2413f290 0x7f0f2413b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:27.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.682+0000 7f0f23e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f2413f290 0x7f0f2413b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39592/0 (socket says 172.21.15.167:39592) 2024-05-11T12:13:27.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.682+0000 7f0f23e00640 1 -- 172.21.15.167:0/399445820 learned_addr learned my addr 172.21.15.167:0/399445820 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:27.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.682+0000 7f0f23400640 1 --2- 172.21.15.167:0/399445820 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f240b50a0 0x7f0f24139790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:27.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.682+0000 7f0f23e00640 1 -- 172.21.15.167:0/399445820 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f240b50a0 msgr2=0x7f0f24139790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:27.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.682+0000 7f0f23e00640 1 --2- 172.21.15.167:0/399445820 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f240b50a0 0x7f0f24139790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:27.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.682+0000 7f0f23e00640 1 -- 172.21.15.167:0/399445820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f2413eeb0 msgr2=0x7f0f2413b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:27.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.682+0000 7f0f1be00640 1 --2- 172.21.15.167:0/399445820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f2413eeb0 0x7f0f2413b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:27.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.682+0000 7f0f23e00640 1 --2- 172.21.15.167:0/399445820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f2413eeb0 0x7f0f2413b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:27.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.682+0000 7f0f23e00640 1 -- 172.21.15.167:0/399445820 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f241f60d0 con 0x7f0f2413f290 2024-05-11T12:13:27.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.683+0000 7f0f23e00640 1 --2- 172.21.15.167:0/399445820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f2413f290 0x7f0f2413b840 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c009eb0 tx=0x7f0f1c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:27.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.683+0000 7f0f1b400640 1 -- 172.21.15.167:0/399445820 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c040070 con 0x7f0f2413f290 2024-05-11T12:13:27.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.683+0000 7f0f1b400640 1 -- 172.21.15.167:0/399445820 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f1c030b60 con 0x7f0f2413f290 2024-05-11T12:13:27.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.683+0000 7f0f1b400640 1 -- 172.21.15.167:0/399445820 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c045460 con 0x7f0f2413f290 2024-05-11T12:13:27.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.683+0000 7f0f2a800640 1 -- 172.21.15.167:0/399445820 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f241f63b0 con 0x7f0f2413f290 2024-05-11T12:13:27.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.683+0000 7f0f2a800640 1 -- 172.21.15.167:0/399445820 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f241f6970 con 0x7f0f2413f290 2024-05-11T12:13:27.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.684+0000 7f0f2a800640 1 -- 172.21.15.167:0/399445820 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f24069590 con 0x7f0f2413f290 2024-05-11T12:13:27.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.686+0000 7f0f1b400640 1 -- 172.21.15.167:0/399445820 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0f1c039040 con 0x7f0f2413f290 2024-05-11T12:13:27.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.687+0000 7f0f1b400640 1 --2- 172.21.15.167:0/399445820 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0f000762d0 0x7f0f00078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:27.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.687+0000 7f0f1b400640 1 -- 172.21.15.167:0/399445820 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5335+0+0 (secure 0 0 0) 0x7f0f1c0b8cc0 con 0x7f0f2413f290 2024-05-11T12:13:27.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.687+0000 7f0f23400640 1 --2- 172.21.15.167:0/399445820 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0f000762d0 0x7f0f00078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:27.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.687+0000 7f0f23400640 1 --2- 172.21.15.167:0/399445820 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0f000762d0 0x7f0f00078790 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0f14008850 tx=0x7f0f14008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:27.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.691+0000 7f0f1b400640 1 -- 172.21.15.167:0/399445820 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0f1c0bf080 con 0x7f0f2413f290 2024-05-11T12:13:27.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.905+0000 7f0f2a800640 1 -- 172.21.15.167:0/399445820 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0f240b6040 con 0x7f0f2413f290 2024-05-11T12:13:27.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.907+0000 7f0f1b400640 1 -- 172.21.15.167:0/399445820 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) ==== 74+0+13340 (secure 0 0 0) 0x7f0f1c082840 con 0x7f0f2413f290 2024-05-11T12:13:27.906 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:13:27.907 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":67,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:13:26.429509+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.167:6837/42694429":"2024-05-12T12:13:23.094604+0000","172.21.15.167:6835/42694429":"2024-05-12T12:13:23.094604+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:13:27.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.910+0000 7f0f2a800640 1 -- 172.21.15.167:0/399445820 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0f000762d0 msgr2=0x7f0f00078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:27.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.910+0000 7f0f2a800640 1 --2- 172.21.15.167:0/399445820 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0f000762d0 0x7f0f00078790 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0f14008850 tx=0x7f0f14008040 comp rx=0 tx=0).stop 2024-05-11T12:13:27.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.910+0000 7f0f2a800640 1 -- 172.21.15.167:0/399445820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f2413f290 msgr2=0x7f0f2413b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:27.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.910+0000 7f0f2a800640 1 --2- 172.21.15.167:0/399445820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f2413f290 0x7f0f2413b840 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c009eb0 tx=0x7f0f1c004a40 comp rx=0 tx=0).stop 2024-05-11T12:13:27.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.910+0000 7f0f2a800640 1 -- 172.21.15.167:0/399445820 shutdown_connections 2024-05-11T12:13:27.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.910+0000 7f0f2a800640 1 --2- 172.21.15.167:0/399445820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f2413f290 0x7f0f2413b840 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:27.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.910+0000 7f0f2a800640 1 --2- 172.21.15.167:0/399445820 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0f000762d0 0x7f0f00078790 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:27.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.910+0000 7f0f2a800640 1 --2- 172.21.15.167:0/399445820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f2413eeb0 0x7f0f2413b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:27.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.910+0000 7f0f2a800640 1 --2- 172.21.15.167:0/399445820 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f240b50a0 0x7f0f24139790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:27.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.910+0000 7f0f2a800640 1 -- 172.21.15.167:0/399445820 >> 172.21.15.167:0/399445820 conn(0x7f0f240cd8f0 msgr2=0x7f0f240cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:27.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.910+0000 7f0f2a800640 1 -- 172.21.15.167:0/399445820 shutdown_connections 2024-05-11T12:13:27.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:27.910+0000 7f0f2a800640 1 -- 172.21.15.167:0/399445820 wait complete. 2024-05-11T12:13:27.929 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-11T12:10:33.260270+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-11T12:13:27.929 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-11T12:13:28.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.134+0000 7f0a2b800640 1 Processor -- start 2024-05-11T12:13:28.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.134+0000 7f0a2b800640 1 -- start start 2024-05-11T12:13:28.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.136+0000 7f0a2b800640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0a24161380 0x7f0a24074260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:28.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.136+0000 7f0a2b800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a240b3d50 0x7f0a24072950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:28.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.136+0000 7f0a2b800640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a240b65e0 con 0x7f0a2415ac80 2024-05-11T12:13:28.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.136+0000 7f0a2b800640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f0a2406e3c0 con 0x7f0a24161380 2024-05-11T12:13:28.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.136+0000 7f0a2b800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0a2415b060 con 0x7f0a240b3d50 2024-05-11T12:13:28.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.136+0000 7f0a23e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0a24161380 0x7f0a24074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:28.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.136+0000 7f0a23e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0a24161380 0x7f0a24074260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39598/0 (socket says 172.21.15.167:39598) 2024-05-11T12:13:28.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.136+0000 7f0a28e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f0a2415ac80 0x7f0a24160c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:37388/0 (socket says 172.21.15.167:37388) 2024-05-11T12:13:28.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.136+0000 7f0a23e00640 1 -- 172.21.15.167:0/213654349 learned_addr learned my addr 172.21.15.167:0/213654349 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:28.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.137+0000 7f0a29800640 1 --2- 172.21.15.167:0/213654349 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a240b3d50 0x7f0a24072950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:28.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.137+0000 7f0a23e00640 1 -- 172.21.15.167:0/213654349 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a240b3d50 msgr2=0x7f0a24072950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:28.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.137+0000 7f0a23e00640 1 --2- 172.21.15.167:0/213654349 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a240b3d50 0x7f0a24072950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:28.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.137+0000 7f0a23e00640 1 -- 172.21.15.167:0/213654349 >> v1:172.21.15.167:6789/0 conn(0x7f0a2415ac80 legacy=0x7f0a24160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:28.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.137+0000 7f0a23e00640 1 -- 172.21.15.167:0/213654349 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a240b68c0 con 0x7f0a24161380 2024-05-11T12:13:28.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.137+0000 7f0a29800640 1 --2- 172.21.15.167:0/213654349 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a240b3d50 0x7f0a24072950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:28.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.137+0000 7f0a23400640 1 -- 172.21.15.167:0/213654349 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3921129839 0 0) 0x7f0a240b65e0 con 0x7f0a2415ac80 2024-05-11T12:13:28.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.138+0000 7f0a23e00640 1 --2- 172.21.15.167:0/213654349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a24161380 0x7f0a24074260 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f0a10009870 tx=0x7f0a1002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e3330c2ecc041ff server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:28.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.138+0000 7f0a23400640 1 -- 172.21.15.167:0/213654349 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a10037070 con 0x7f0a24161380 2024-05-11T12:13:28.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.138+0000 7f0a23400640 1 -- 172.21.15.167:0/213654349 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a10032440 con 0x7f0a24161380 2024-05-11T12:13:28.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.138+0000 7f0a23400640 1 -- 172.21.15.167:0/213654349 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a10032810 con 0x7f0a24161380 2024-05-11T12:13:28.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.139+0000 7f0a2b800640 1 -- 172.21.15.167:0/213654349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a24161380 msgr2=0x7f0a24074260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:28.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.139+0000 7f0a2b800640 1 --2- 172.21.15.167:0/213654349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a24161380 0x7f0a24074260 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f0a10009870 tx=0x7f0a1002a250 comp rx=0 tx=0).stop 2024-05-11T12:13:28.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.139+0000 7f0a2b800640 1 -- 172.21.15.167:0/213654349 shutdown_connections 2024-05-11T12:13:28.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.139+0000 7f0a2b800640 1 --2- 172.21.15.167:0/213654349 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a240b3d50 0x7f0a24072950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:28.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.139+0000 7f0a2b800640 1 --2- 172.21.15.167:0/213654349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a24161380 0x7f0a24074260 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:28.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.139+0000 7f0a2b800640 1 -- 172.21.15.167:0/213654349 >> 172.21.15.167:0/213654349 conn(0x7f0a2406c450 msgr2=0x7f0a2406c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:28.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.139+0000 7f0a2b800640 1 -- 172.21.15.167:0/213654349 shutdown_connections 2024-05-11T12:13:28.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.139+0000 7f0a2b800640 1 -- 172.21.15.167:0/213654349 wait complete. 2024-05-11T12:13:28.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.141+0000 7f0a2b800640 1 Processor -- start 2024-05-11T12:13:28.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.141+0000 7f0a2b800640 1 -- start start 2024-05-11T12:13:28.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.142+0000 7f0a2b800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a240b3d50 0x7f0a240c61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:28.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.142+0000 7f0a2b800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a2415ac80 0x7f0a241f5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:28.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.142+0000 7f0a2b800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a24161380 0x7f0a241f8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:28.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.142+0000 7f0a2b800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0a240b44f0 con 0x7f0a240b3d50 2024-05-11T12:13:28.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.142+0000 7f0a2b800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0a240b41a0 con 0x7f0a24161380 2024-05-11T12:13:28.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.142+0000 7f0a2b800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0a24072e90 con 0x7f0a2415ac80 2024-05-11T12:13:28.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.142+0000 7f0a28e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a240b3d50 0x7f0a240c61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:28.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.142+0000 7f0a28e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a240b3d50 0x7f0a240c61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39600/0 (socket says 172.21.15.167:39600) 2024-05-11T12:13:28.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.142+0000 7f0a28e00640 1 -- 172.21.15.167:0/3657818427 learned_addr learned my addr 172.21.15.167:0/3657818427 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:28.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.142+0000 7f0a28e00640 1 -- 172.21.15.167:0/3657818427 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a2415ac80 msgr2=0x7f0a241f5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:28.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.142+0000 7f0a28e00640 1 --2- 172.21.15.167:0/3657818427 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a2415ac80 0x7f0a241f5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:28.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.142+0000 7f0a28e00640 1 -- 172.21.15.167:0/3657818427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a24161380 msgr2=0x7f0a241f8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:28.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.142+0000 7f0a28e00640 1 --2- 172.21.15.167:0/3657818427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a24161380 0x7f0a241f8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:28.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.142+0000 7f0a28e00640 1 -- 172.21.15.167:0/3657818427 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a241f8a20 con 0x7f0a240b3d50 2024-05-11T12:13:28.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.143+0000 7f0a28e00640 1 --2- 172.21.15.167:0/3657818427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a240b3d50 0x7f0a240c61f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0a1800c4c0 tx=0x7f0a1800c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:28.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.143+0000 7f0a20c00640 1 -- 172.21.15.167:0/3657818427 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a18018070 con 0x7f0a240b3d50 2024-05-11T12:13:28.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.143+0000 7f0a20c00640 1 -- 172.21.15.167:0/3657818427 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a18007500 con 0x7f0a240b3d50 2024-05-11T12:13:28.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.143+0000 7f0a20c00640 1 -- 172.21.15.167:0/3657818427 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a18004a20 con 0x7f0a240b3d50 2024-05-11T12:13:28.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.143+0000 7f0a2b800640 1 -- 172.21.15.167:0/3657818427 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a241f8d00 con 0x7f0a240b3d50 2024-05-11T12:13:28.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.143+0000 7f0a2b800640 1 -- 172.21.15.167:0/3657818427 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a240c6730 con 0x7f0a240b3d50 2024-05-11T12:13:28.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.144+0000 7f0a2b800640 1 -- 172.21.15.167:0/3657818427 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a24069520 con 0x7f0a240b3d50 2024-05-11T12:13:28.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.145+0000 7f0a20c00640 1 -- 172.21.15.167:0/3657818427 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0a18004bc0 con 0x7f0a240b3d50 2024-05-11T12:13:28.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.146+0000 7f0a20c00640 1 --2- 172.21.15.167:0/3657818427 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0a04076300 0x7f0a040787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:28.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.146+0000 7f0a20c00640 1 -- 172.21.15.167:0/3657818427 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5335+0+0 (secure 0 0 0) 0x7f0a18096690 con 0x7f0a240b3d50 2024-05-11T12:13:28.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.147+0000 7f0a23e00640 1 --2- 172.21.15.167:0/3657818427 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0a04076300 0x7f0a040787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:28.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.147+0000 7f0a23e00640 1 --2- 172.21.15.167:0/3657818427 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0a04076300 0x7f0a040787c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0a10034000 tx=0x7f0a1002c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:28.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.151+0000 7f0a20c00640 1 -- 172.21.15.167:0/3657818427 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0a18060160 con 0x7f0a240b3d50 2024-05-11T12:13:28.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.354+0000 7f0a2b800640 1 -- 172.21.15.167:0/3657818427 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0a241f8ed0 con 0x7f0a240b3d50 2024-05-11T12:13:28.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.355+0000 7f0a20c00640 1 -- 172.21.15.167:0/3657818427 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v67) ==== 93+0+10 (secure 0 0 0) 0x7f0a18068150 con 0x7f0a240b3d50 2024-05-11T12:13:28.354 INFO:teuthology.orchestra.run.smithi167.stdout:pg_num: 1 2024-05-11T12:13:28.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.358+0000 7f0a2b800640 1 -- 172.21.15.167:0/3657818427 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0a04076300 msgr2=0x7f0a040787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:28.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.358+0000 7f0a2b800640 1 --2- 172.21.15.167:0/3657818427 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0a04076300 0x7f0a040787c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0a10034000 tx=0x7f0a1002c000 comp rx=0 tx=0).stop 2024-05-11T12:13:28.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.358+0000 7f0a2b800640 1 -- 172.21.15.167:0/3657818427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a240b3d50 msgr2=0x7f0a240c61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:28.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.358+0000 7f0a2b800640 1 --2- 172.21.15.167:0/3657818427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a240b3d50 0x7f0a240c61f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0a1800c4c0 tx=0x7f0a1800c980 comp rx=0 tx=0).stop 2024-05-11T12:13:28.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.358+0000 7f0a2b800640 1 -- 172.21.15.167:0/3657818427 shutdown_connections 2024-05-11T12:13:28.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.358+0000 7f0a2b800640 1 --2- 172.21.15.167:0/3657818427 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0a04076300 0x7f0a040787c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:28.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.358+0000 7f0a2b800640 1 --2- 172.21.15.167:0/3657818427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a24161380 0x7f0a241f8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:28.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.358+0000 7f0a2b800640 1 --2- 172.21.15.167:0/3657818427 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a2415ac80 0x7f0a241f5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:28.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.358+0000 7f0a2b800640 1 --2- 172.21.15.167:0/3657818427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a240b3d50 0x7f0a240c61f0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:28.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.358+0000 7f0a2b800640 1 -- 172.21.15.167:0/3657818427 >> 172.21.15.167:0/3657818427 conn(0x7f0a2406c450 msgr2=0x7f0a240732c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:28.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.359+0000 7f0a2b800640 1 -- 172.21.15.167:0/3657818427 shutdown_connections 2024-05-11T12:13:28.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.359+0000 7f0a2b800640 1 -- 172.21.15.167:0/3657818427 wait complete. 2024-05-11T12:13:28.377 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_conn_close' 2024-05-11T12:13:28.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.571+0000 7f01c1600640 1 Processor -- start 2024-05-11T12:13:28.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.571+0000 7f01c1600640 1 -- start start 2024-05-11T12:13:28.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.573+0000 7f01c1600640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f01bc0d4340 0x7f01bc0d2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:28.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.573+0000 7f01c1600640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01bc0d32a0 con 0x7f01bc06d4e0 2024-05-11T12:13:28.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.573+0000 7f01c1600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01bc0d3580 con 0x7f01bc162780 2024-05-11T12:13:28.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.573+0000 7f01c1600640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f01bc0d3070 con 0x7f01bc0d4340 2024-05-11T12:13:28.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.573+0000 7f01b3400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f01bc0d4340 0x7f01bc0d2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:28.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.573+0000 7f01b3400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f01bc0d4340 0x7f01bc0d2b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39610/0 (socket says 172.21.15.167:39610) 2024-05-11T12:13:28.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.573+0000 7f01b3400640 1 -- 172.21.15.167:0/1142390612 learned_addr learned my addr 172.21.15.167:0/1142390612 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:28.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.574+0000 7f01b3400640 1 -- 172.21.15.167:0/1142390612 >> v1:172.21.15.204:6789/0 conn(0x7f01bc162780 legacy=0x7f01bc06cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:28.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.574+0000 7f01b3400640 1 -- 172.21.15.167:0/1142390612 >> v1:172.21.15.167:6789/0 conn(0x7f01bc06d4e0 legacy=0x7f01bc0b8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:28.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.574+0000 7f01b3400640 1 -- 172.21.15.167:0/1142390612 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01bc0d3860 con 0x7f01bc0d4340 2024-05-11T12:13:28.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.575+0000 7f01b3400640 1 --2- 172.21.15.167:0/1142390612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01bc0d4340 0x7f01bc0d2b30 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f01a4009870 tx=0x7f01a402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f5826a66b442a7f server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:28.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.575+0000 7f01bb400640 1 -- 172.21.15.167:0/1142390612 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a4037070 con 0x7f01bc0d4340 2024-05-11T12:13:28.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.575+0000 7f01bb400640 1 -- 172.21.15.167:0/1142390612 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01a4032440 con 0x7f01bc0d4340 2024-05-11T12:13:28.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.575+0000 7f01bb400640 1 -- 172.21.15.167:0/1142390612 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a4032810 con 0x7f01bc0d4340 2024-05-11T12:13:28.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.576+0000 7f01c1600640 1 -- 172.21.15.167:0/1142390612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01bc0d4340 msgr2=0x7f01bc0d2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:28.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.576+0000 7f01c1600640 1 --2- 172.21.15.167:0/1142390612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01bc0d4340 0x7f01bc0d2b30 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f01a4009870 tx=0x7f01a402a1c0 comp rx=0 tx=0).stop 2024-05-11T12:13:28.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.576+0000 7f01c1600640 1 -- 172.21.15.167:0/1142390612 shutdown_connections 2024-05-11T12:13:28.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.576+0000 7f01c1600640 1 --2- 172.21.15.167:0/1142390612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01bc0d4340 0x7f01bc0d2b30 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:28.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.576+0000 7f01c1600640 1 -- 172.21.15.167:0/1142390612 >> 172.21.15.167:0/1142390612 conn(0x7f01bc0d1bc0 msgr2=0x7f01bc0d1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:28.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.576+0000 7f01c1600640 1 -- 172.21.15.167:0/1142390612 shutdown_connections 2024-05-11T12:13:28.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.576+0000 7f01c1600640 1 -- 172.21.15.167:0/1142390612 wait complete. 2024-05-11T12:13:28.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.578+0000 7f01c1600640 1 Processor -- start 2024-05-11T12:13:28.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.578+0000 7f01c1600640 1 -- start start 2024-05-11T12:13:28.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.579+0000 7f01c1600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01bc06d4e0 0x7f01bc153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:28.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.579+0000 7f01c1600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01bc162780 0x7f01bc0b5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:28.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.579+0000 7f01c1600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01bc0b5d90 0x7f01bc0cc030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:28.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.579+0000 7f01c1600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f01bc162b60 con 0x7f01bc06d4e0 2024-05-11T12:13:28.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.579+0000 7f01c1600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f01bc0d3e20 con 0x7f01bc0b5d90 2024-05-11T12:13:28.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.579+0000 7f01c1600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f01bc162ce0 con 0x7f01bc162780 2024-05-11T12:13:28.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.579+0000 7f01bbe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01bc06d4e0 0x7f01bc153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:28.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.579+0000 7f01bbe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01bc06d4e0 0x7f01bc153d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39618/0 (socket says 172.21.15.167:39618) 2024-05-11T12:13:28.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.579+0000 7f01bbe00640 1 -- 172.21.15.167:0/4073321150 learned_addr learned my addr 172.21.15.167:0/4073321150 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:28.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.579+0000 7f01b3400640 1 --2- 172.21.15.167:0/4073321150 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01bc162780 0x7f01bc0b5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:28.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.579+0000 7f01c0c00640 1 --2- 172.21.15.167:0/4073321150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01bc0b5d90 0x7f01bc0cc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:28.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.580+0000 7f01bbe00640 1 -- 172.21.15.167:0/4073321150 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01bc162780 msgr2=0x7f01bc0b5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:28.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.580+0000 7f01bbe00640 1 --2- 172.21.15.167:0/4073321150 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01bc162780 0x7f01bc0b5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:28.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.580+0000 7f01bbe00640 1 -- 172.21.15.167:0/4073321150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01bc0b5d90 msgr2=0x7f01bc0cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:28.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.580+0000 7f01bbe00640 1 --2- 172.21.15.167:0/4073321150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01bc0b5d90 0x7f01bc0cc030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:28.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.580+0000 7f01bbe00640 1 -- 172.21.15.167:0/4073321150 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01bc0c9cb0 con 0x7f01bc06d4e0 2024-05-11T12:13:28.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.580+0000 7f01c0c00640 1 --2- 172.21.15.167:0/4073321150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01bc0b5d90 0x7f01bc0cc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:28.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.580+0000 7f01bbe00640 1 --2- 172.21.15.167:0/4073321150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01bc06d4e0 0x7f01bc153d70 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f01ac00ba50 tx=0x7f01ac00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:28.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.580+0000 7f01b8c00640 1 -- 172.21.15.167:0/4073321150 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ac019070 con 0x7f01bc06d4e0 2024-05-11T12:13:28.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.580+0000 7f01b8c00640 1 -- 172.21.15.167:0/4073321150 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01ac00cc20 con 0x7f01bc06d4e0 2024-05-11T12:13:28.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.580+0000 7f01c1600640 1 -- 172.21.15.167:0/4073321150 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01bc0ccc40 con 0x7f01bc06d4e0 2024-05-11T12:13:28.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.580+0000 7f01b8c00640 1 -- 172.21.15.167:0/4073321150 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ac014680 con 0x7f01bc06d4e0 2024-05-11T12:13:28.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.581+0000 7f01c1600640 1 -- 172.21.15.167:0/4073321150 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01bc0ca770 con 0x7f01bc06d4e0 2024-05-11T12:13:28.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.582+0000 7f01c1600640 1 -- 172.21.15.167:0/4073321150 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01bc0694b0 con 0x7f01bc06d4e0 2024-05-11T12:13:28.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.583+0000 7f01b8c00640 1 -- 172.21.15.167:0/4073321150 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f01ac01c070 con 0x7f01bc06d4e0 2024-05-11T12:13:28.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.584+0000 7f01b8c00640 1 --2- 172.21.15.167:0/4073321150 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f01980762d0 0x7f0198078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:28.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.585+0000 7f01b8c00640 1 -- 172.21.15.167:0/4073321150 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5335+0+0 (secure 0 0 0) 0x7f01ac0971b0 con 0x7f01bc06d4e0 2024-05-11T12:13:28.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.585+0000 7f01b3400640 1 --2- 172.21.15.167:0/4073321150 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f01980762d0 0x7f0198078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:28.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.585+0000 7f01b3400640 1 --2- 172.21.15.167:0/4073321150 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f01980762d0 0x7f0198078790 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f01a4034000 tx=0x7f01a402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:28.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.591+0000 7f01b8c00640 1 -- 172.21.15.167:0/4073321150 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f01ac060590 con 0x7f01bc06d4e0 2024-05-11T12:13:28.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:28.861+0000 7f01c1600640 1 -- 172.21.15.167:0/4073321150 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_conn_close"]} v 0) -- 0x7f01bc0b6200 con 0x7f01bc06d4e0 2024-05-11T12:13:29.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.456+0000 7f01b8c00640 1 -- 172.21.15.167:0/4073321150 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_conn_close"]}]=0 v172) ==== 146+0+0 (secure 0 0 0) 0x7f01bc0b6200 con 0x7f01bc06d4e0 2024-05-11T12:13:29.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.460+0000 7f01c1600640 1 -- 172.21.15.167:0/4073321150 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f01980762d0 msgr2=0x7f0198078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:29.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.460+0000 7f01c1600640 1 --2- 172.21.15.167:0/4073321150 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f01980762d0 0x7f0198078790 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f01a4034000 tx=0x7f01a402e000 comp rx=0 tx=0).stop 2024-05-11T12:13:29.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.460+0000 7f01c1600640 1 -- 172.21.15.167:0/4073321150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01bc06d4e0 msgr2=0x7f01bc153d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:29.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.460+0000 7f01c1600640 1 --2- 172.21.15.167:0/4073321150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01bc06d4e0 0x7f01bc153d70 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f01ac00ba50 tx=0x7f01ac00bf10 comp rx=0 tx=0).stop 2024-05-11T12:13:29.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.460+0000 7f01c1600640 1 -- 172.21.15.167:0/4073321150 shutdown_connections 2024-05-11T12:13:29.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.460+0000 7f01c1600640 1 --2- 172.21.15.167:0/4073321150 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f01980762d0 0x7f0198078790 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:29.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.460+0000 7f01c1600640 1 --2- 172.21.15.167:0/4073321150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01bc0b5d90 0x7f01bc0cc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:29.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.460+0000 7f01c1600640 1 --2- 172.21.15.167:0/4073321150 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01bc162780 0x7f01bc0b5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:29.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.460+0000 7f01c1600640 1 --2- 172.21.15.167:0/4073321150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01bc06d4e0 0x7f01bc153d70 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:29.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.460+0000 7f01c1600640 1 -- 172.21.15.167:0/4073321150 >> 172.21.15.167:0/4073321150 conn(0x7f01bc0d1bc0 msgr2=0x7f01bc0ba440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:29.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.461+0000 7f01c1600640 1 -- 172.21.15.167:0/4073321150 shutdown_connections 2024-05-11T12:13:29.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.461+0000 7f01c1600640 1 -- 172.21.15.167:0/4073321150 wait complete. 2024-05-11T12:13:29.483 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-11T12:13:29.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.690+0000 7fd5d3200640 1 Processor -- start 2024-05-11T12:13:29.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.690+0000 7fd5d3200640 1 -- start start 2024-05-11T12:13:29.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.692+0000 7fd5d3200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5d41468b0 0x7fd5d40cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:29.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.692+0000 7fd5d3200640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5d40cff30 con 0x7fd5d415dbd0 2024-05-11T12:13:29.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.692+0000 7fd5d3200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5d40d0210 con 0x7fd5d40b6a50 2024-05-11T12:13:29.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.692+0000 7fd5d3200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd5d4146c90 con 0x7fd5d41468b0 2024-05-11T12:13:29.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.692+0000 7fd5d2800640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fd5d415dbd0 0x7fd5d40b4e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:52368/0 (socket says 172.21.15.167:52368) 2024-05-11T12:13:29.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.692+0000 7fd5d2800640 1 -- 172.21.15.167:0/4192378677 learned_addr learned my addr 172.21.15.167:0/4192378677 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:13:29.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.693+0000 7fd5d1e00640 1 --2- 172.21.15.167:0/4192378677 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5d41468b0 0x7fd5d40cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:29.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.693+0000 7fd5d0a00640 1 -- 172.21.15.167:0/4192378677 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3906098870 0 0) 0x7fd5d40cff30 con 0x7fd5d415dbd0 2024-05-11T12:13:29.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.693+0000 7fd5d0a00640 1 -- 172.21.15.167:0/4192378677 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5b8003520 con 0x7fd5d415dbd0 2024-05-11T12:13:29.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.693+0000 7fd5d0a00640 1 -- 172.21.15.167:0/4192378677 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 190644295 0 0) 0x7fd5b8003520 con 0x7fd5d415dbd0 2024-05-11T12:13:29.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.693+0000 7fd5d0a00640 1 -- 172.21.15.167:0/4192378677 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5d41468b0 msgr2=0x7fd5d40cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:29.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.693+0000 7fd5d0a00640 1 --2- 172.21.15.167:0/4192378677 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5d41468b0 0x7fd5d40cd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:29.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.693+0000 7fd5d0a00640 1 -- 172.21.15.167:0/4192378677 >> v1:172.21.15.204:6789/0 conn(0x7fd5d40b6a50 legacy=0x7fd5d415d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:29.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.693+0000 7fd5d0a00640 1 -- 172.21.15.167:0/4192378677 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5d40d04f0 con 0x7fd5d415dbd0 2024-05-11T12:13:29.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.693+0000 7fd5d0a00640 1 -- 172.21.15.167:0/4192378677 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7fd5cc001a70 con 0x7fd5d415dbd0 2024-05-11T12:13:29.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.693+0000 7fd5d0a00640 1 -- 172.21.15.167:0/4192378677 >> v1:172.21.15.167:6789/0 conn(0x7fd5d415dbd0 legacy=0x7fd5d40b4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:29.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.693+0000 7fd5d0a00640 1 --2- 172.21.15.167:0/4192378677 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5d41468b0 0x7fd5b8003a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:29.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.693+0000 7fd5d0a00640 1 --2- 172.21.15.167:0/4192378677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5b8004050 0x7fd5b8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:29.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.693+0000 7fd5d0a00640 1 --2- 172.21.15.167:0/4192378677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5b8004d40 0x7fd5b80051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:29.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.693+0000 7fd5d0a00640 1 -- 172.21.15.167:0/4192378677 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b800bb80 con 0x7fd5b8004d40 2024-05-11T12:13:29.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.693+0000 7fd5d0a00640 1 -- 172.21.15.167:0/4192378677 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b800bd30 con 0x7fd5b8004050 2024-05-11T12:13:29.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.694+0000 7fd5d0a00640 1 -- 172.21.15.167:0/4192378677 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd5b8010340 con 0x7fd5d41468b0 2024-05-11T12:13:29.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.694+0000 7fd5d2800640 1 --2- 172.21.15.167:0/4192378677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5b8004d40 0x7fd5b80051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:29.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.694+0000 7fd5d1e00640 1 --2- 172.21.15.167:0/4192378677 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5d41468b0 0x7fd5b8003a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:29.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.694+0000 7fd5d1400640 1 --2- 172.21.15.167:0/4192378677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5b8004050 0x7fd5b8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:29.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.694+0000 7fd5d2800640 1 -- 172.21.15.167:0/4192378677 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5d41468b0 msgr2=0x7fd5b8003a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:29.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.694+0000 7fd5d2800640 1 --2- 172.21.15.167:0/4192378677 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5d41468b0 0x7fd5b8003a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:29.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.694+0000 7fd5d2800640 1 -- 172.21.15.167:0/4192378677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5b8004050 msgr2=0x7fd5b8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:29.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.694+0000 7fd5d2800640 1 --2- 172.21.15.167:0/4192378677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5b8004050 0x7fd5b8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:29.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.694+0000 7fd5d2800640 1 -- 172.21.15.167:0/4192378677 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b8003520 con 0x7fd5b8004d40 2024-05-11T12:13:29.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.695+0000 7fd5d2800640 1 --2- 172.21.15.167:0/4192378677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5b8004d40 0x7fd5b80051a0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc00a8c0 tx=0x7fd5b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c8b2f14b9865fa server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:29.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.695+0000 7fd5d0a00640 1 -- 172.21.15.167:0/4192378677 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc00a4e0 con 0x7fd5b8004d40 2024-05-11T12:13:29.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.696+0000 7fd5d0a00640 1 -- 172.21.15.167:0/4192378677 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5cc02f070 con 0x7fd5b8004d40 2024-05-11T12:13:29.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.696+0000 7fd5d0a00640 1 -- 172.21.15.167:0/4192378677 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc02bc50 con 0x7fd5b8004d40 2024-05-11T12:13:29.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.696+0000 7fd5d3200640 1 -- 172.21.15.167:0/4192378677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5b8004d40 msgr2=0x7fd5b80051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:29.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.696+0000 7fd5d3200640 1 --2- 172.21.15.167:0/4192378677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5b8004d40 0x7fd5b80051a0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc00a8c0 tx=0x7fd5b8005830 comp rx=0 tx=0).stop 2024-05-11T12:13:29.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.696+0000 7fd5d2800640 1 -- 172.21.15.167:0/4192378677 reap_dead start 2024-05-11T12:13:29.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.696+0000 7fd5d3200640 1 -- 172.21.15.167:0/4192378677 shutdown_connections 2024-05-11T12:13:29.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.696+0000 7fd5d3200640 1 -- 172.21.15.167:0/4192378677 >> 172.21.15.167:0/4192378677 conn(0x7fd5d40b2f10 msgr2=0x7fd5d406c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:29.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.696+0000 7fd5d3200640 1 -- 172.21.15.167:0/4192378677 shutdown_connections 2024-05-11T12:13:29.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.696+0000 7fd5d3200640 1 -- 172.21.15.167:0/4192378677 wait complete. 2024-05-11T12:13:29.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.698+0000 7fd5d3200640 1 Processor -- start 2024-05-11T12:13:29.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.698+0000 7fd5d3200640 1 -- start start 2024-05-11T12:13:29.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.699+0000 7fd5d3200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5b8004d40 0x7fd5d41f02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:29.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.699+0000 7fd5d3200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5d4145be0 0x7fd5d41f42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:29.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.699+0000 7fd5d3200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5d415b950 0x7fd5d415bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:29.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.699+0000 7fd5d3200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d40cd700 con 0x7fd5d415b950 2024-05-11T12:13:29.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.699+0000 7fd5d3200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d40cd580 con 0x7fd5d4145be0 2024-05-11T12:13:29.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.699+0000 7fd5d3200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd5d40cd880 con 0x7fd5b8004d40 2024-05-11T12:13:29.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.699+0000 7fd5d2800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5d415b950 0x7fd5d415bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:29.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.699+0000 7fd5d2800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5d415b950 0x7fd5d415bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:47832/0 (socket says 172.21.15.167:47832) 2024-05-11T12:13:29.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.699+0000 7fd5d2800640 1 -- 172.21.15.167:0/2047259654 learned_addr learned my addr 172.21.15.167:0/2047259654 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:29.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.700+0000 7fd5d1e00640 1 --2- 172.21.15.167:0/2047259654 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5b8004d40 0x7fd5d41f02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:29.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.700+0000 7fd5d2800640 1 -- 172.21.15.167:0/2047259654 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5b8004d40 msgr2=0x7fd5d41f02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:29.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.700+0000 7fd5d1400640 1 --2- 172.21.15.167:0/2047259654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5d4145be0 0x7fd5d41f42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:29.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.700+0000 7fd5d2800640 1 --2- 172.21.15.167:0/2047259654 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5b8004d40 0x7fd5d41f02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:29.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.700+0000 7fd5d2800640 1 -- 172.21.15.167:0/2047259654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5d4145be0 msgr2=0x7fd5d41f42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:29.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.700+0000 7fd5d2800640 1 --2- 172.21.15.167:0/2047259654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5d4145be0 0x7fd5d41f42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:29.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.700+0000 7fd5d2800640 1 -- 172.21.15.167:0/2047259654 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5d41f4820 con 0x7fd5d415b950 2024-05-11T12:13:29.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.700+0000 7fd5d2800640 1 --2- 172.21.15.167:0/2047259654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5d415b950 0x7fd5d415bdd0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc02b500 tx=0x7fd5cc001d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:29.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.700+0000 7fd5ca000640 1 -- 172.21.15.167:0/2047259654 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc00a4e0 con 0x7fd5d415b950 2024-05-11T12:13:29.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.700+0000 7fd5ca000640 1 -- 172.21.15.167:0/2047259654 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5cc034470 con 0x7fd5d415b950 2024-05-11T12:13:29.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.700+0000 7fd5ca000640 1 -- 172.21.15.167:0/2047259654 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc002030 con 0x7fd5d415b950 2024-05-11T12:13:29.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.700+0000 7fd5d3200640 1 -- 172.21.15.167:0/2047259654 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5d41f0880 con 0x7fd5d415b950 2024-05-11T12:13:29.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.701+0000 7fd5d3200640 1 -- 172.21.15.167:0/2047259654 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5d41f0e40 con 0x7fd5d415b950 2024-05-11T12:13:29.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.702+0000 7fd5d3200640 1 -- 172.21.15.167:0/2047259654 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5d415c310 con 0x7fd5d415b950 2024-05-11T12:13:29.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.703+0000 7fd5ca000640 1 -- 172.21.15.167:0/2047259654 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd5cc034860 con 0x7fd5d415b950 2024-05-11T12:13:29.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.704+0000 7fd5ca000640 1 --2- 172.21.15.167:0/2047259654 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd5b40762d0 0x7fd5b4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:29.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.705+0000 7fd5ca000640 1 -- 172.21.15.167:0/2047259654 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5335+0+0 (secure 0 0 0) 0x7fd5cc0b7070 con 0x7fd5d415b950 2024-05-11T12:13:29.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.705+0000 7fd5d1e00640 1 --2- 172.21.15.167:0/2047259654 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd5b40762d0 0x7fd5b4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:29.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.705+0000 7fd5d1e00640 1 --2- 172.21.15.167:0/2047259654 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd5b40762d0 0x7fd5b4078790 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd5d406f750 tx=0x7fd5c4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:29.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.709+0000 7fd5ca000640 1 -- 172.21.15.167:0/2047259654 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd5cc047030 con 0x7fd5d415b950 2024-05-11T12:13:29.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.920+0000 7fd5d3200640 1 -- 172.21.15.167:0/2047259654 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fd5d40694b0 con 0x7fd5d415b950 2024-05-11T12:13:29.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.929+0000 7fd5ca000640 1 -- 172.21.15.167:0/2047259654 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7fd5d41f4820 con 0x7fd5d415b950 2024-05-11T12:13:29.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.931+0000 7fd5d3200640 1 -- 172.21.15.167:0/2047259654 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd5b40762d0 msgr2=0x7fd5b4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:29.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.931+0000 7fd5d3200640 1 --2- 172.21.15.167:0/2047259654 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd5b40762d0 0x7fd5b4078790 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd5d406f750 tx=0x7fd5c4008040 comp rx=0 tx=0).stop 2024-05-11T12:13:29.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.931+0000 7fd5d3200640 1 -- 172.21.15.167:0/2047259654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5d415b950 msgr2=0x7fd5d415bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:29.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.931+0000 7fd5d3200640 1 --2- 172.21.15.167:0/2047259654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5d415b950 0x7fd5d415bdd0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc02b500 tx=0x7fd5cc001d70 comp rx=0 tx=0).stop 2024-05-11T12:13:29.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.931+0000 7fd5d3200640 1 -- 172.21.15.167:0/2047259654 shutdown_connections 2024-05-11T12:13:29.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.931+0000 7fd5d3200640 1 --2- 172.21.15.167:0/2047259654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5d415b950 0x7fd5d415bdd0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:29.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.931+0000 7fd5d3200640 1 --2- 172.21.15.167:0/2047259654 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd5b40762d0 0x7fd5b4078790 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:29.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.931+0000 7fd5d3200640 1 --2- 172.21.15.167:0/2047259654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5d4145be0 0x7fd5d41f42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:29.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.931+0000 7fd5d3200640 1 --2- 172.21.15.167:0/2047259654 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5b8004d40 0x7fd5d41f02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:29.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.931+0000 7fd5d3200640 1 -- 172.21.15.167:0/2047259654 >> 172.21.15.167:0/2047259654 conn(0x7fd5d40b2f10 msgr2=0x7fd5d413fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:29.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.932+0000 7fd5d3200640 1 -- 172.21.15.167:0/2047259654 shutdown_connections 2024-05-11T12:13:29.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:29.932+0000 7fd5d3200640 1 -- 172.21.15.167:0/2047259654 wait complete. 2024-05-11T12:13:29.951 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-11T12:13:29.951 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-11T12:13:30.064 DEBUG:teuthology.orchestra.run.smithi204:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-11T12:13:30.176 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-11T12:13:30.177 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:13:30.177 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:13:30.236 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-11T12:13:30.236 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:13:30.236 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:13:30.294 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052713,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-11T12:13:30.294 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:13:30.295 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:13:30.350 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-11T12:13:30.350 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:13:30.350 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:13:30.408 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052713,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-11T12:13:30.408 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-11T12:13:30.408 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:13:30.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.612+0000 7f4ed5600640 1 Processor -- start 2024-05-11T12:13:30.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.612+0000 7f4ed5600640 1 -- start start 2024-05-11T12:13:30.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.614+0000 7f4ed5600640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4ed00cf970 0x7f4ed01601f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:30.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.614+0000 7f4ed5600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ed00d04a0 0x7f4ed00b55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:30.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.614+0000 7f4ed5600640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ed00b4560 con 0x7f4ed00afcd0 2024-05-11T12:13:30.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.614+0000 7f4ed5600640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f4ed00b5130 con 0x7f4ed00cf970 2024-05-11T12:13:30.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.614+0000 7f4ed5600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4ed00b4800 con 0x7f4ed00d04a0 2024-05-11T12:13:30.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.615+0000 7f4ecf400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4ed00cf970 0x7f4ed01601f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:30.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.615+0000 7f4ecf400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4ed00cf970 0x7f4ed01601f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:47836/0 (socket says 172.21.15.167:47836) 2024-05-11T12:13:30.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.615+0000 7f4ecf400640 1 -- 172.21.15.167:0/912696036 learned_addr learned my addr 172.21.15.167:0/912696036 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:30.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.615+0000 7f4ed4c00640 1 --2- 172.21.15.167:0/912696036 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ed00d04a0 0x7f4ed00b55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:30.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.615+0000 7f4ecf400640 1 -- 172.21.15.167:0/912696036 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ed00d04a0 msgr2=0x7f4ed00b55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:30.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.615+0000 7f4ecf400640 1 --2- 172.21.15.167:0/912696036 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ed00d04a0 0x7f4ed00b55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:30.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.615+0000 7f4ecf400640 1 -- 172.21.15.167:0/912696036 >> v1:172.21.15.167:6789/0 conn(0x7f4ed00afcd0 legacy=0x7f4ed00cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:30.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.615+0000 7f4ecf400640 1 -- 172.21.15.167:0/912696036 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ed00b4ae0 con 0x7f4ed00cf970 2024-05-11T12:13:30.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.616+0000 7f4ecf400640 1 --2- 172.21.15.167:0/912696036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ed00cf970 0x7f4ed01601f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f4ed00b52b0 tx=0x7f4ebc02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58bafd3155a1355 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:30.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.617+0000 7f4ecea00640 1 -- 172.21.15.167:0/912696036 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc037070 con 0x7f4ed00cf970 2024-05-11T12:13:30.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.617+0000 7f4ecea00640 1 -- 172.21.15.167:0/912696036 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ebc032440 con 0x7f4ed00cf970 2024-05-11T12:13:30.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.617+0000 7f4ecea00640 1 -- 172.21.15.167:0/912696036 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc032810 con 0x7f4ed00cf970 2024-05-11T12:13:30.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.617+0000 7f4ed5600640 1 -- 172.21.15.167:0/912696036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ed00cf970 msgr2=0x7f4ed01601f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:30.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.617+0000 7f4ed5600640 1 --2- 172.21.15.167:0/912696036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ed00cf970 0x7f4ed01601f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f4ed00b52b0 tx=0x7f4ebc02a250 comp rx=0 tx=0).stop 2024-05-11T12:13:30.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.618+0000 7f4ed5600640 1 -- 172.21.15.167:0/912696036 shutdown_connections 2024-05-11T12:13:30.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.618+0000 7f4ed5600640 1 --2- 172.21.15.167:0/912696036 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ed00d04a0 0x7f4ed00b55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:30.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.618+0000 7f4ed5600640 1 --2- 172.21.15.167:0/912696036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ed00cf970 0x7f4ed01601f0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:30.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.618+0000 7f4ed5600640 1 -- 172.21.15.167:0/912696036 >> 172.21.15.167:0/912696036 conn(0x7f4ed00b3460 msgr2=0x7f4ed00d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:30.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.618+0000 7f4ed5600640 1 -- 172.21.15.167:0/912696036 shutdown_connections 2024-05-11T12:13:30.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.618+0000 7f4ed5600640 1 -- 172.21.15.167:0/912696036 wait complete. 2024-05-11T12:13:30.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.619+0000 7f4ed5600640 1 Processor -- start 2024-05-11T12:13:30.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.620+0000 7f4ed5600640 1 -- start start 2024-05-11T12:13:30.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.620+0000 7f4ed5600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ed00afcd0 0x7f4ed0146f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:30.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.620+0000 7f4ed5600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ed00cf970 0x7f4ed015e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:30.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.620+0000 7f4ed5600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ed00d04a0 0x7f4ed015e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:30.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.620+0000 7f4ed5600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed00cff40 con 0x7f4ed00cf970 2024-05-11T12:13:30.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.621+0000 7f4ed5600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed00b00b0 con 0x7f4ed00d04a0 2024-05-11T12:13:30.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.621+0000 7f4ed5600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4ed00d00c0 con 0x7f4ed00afcd0 2024-05-11T12:13:30.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.621+0000 7f4ecf400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ed00cf970 0x7f4ed015e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:30.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.621+0000 7f4ecf400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ed00cf970 0x7f4ed015e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:47852/0 (socket says 172.21.15.167:47852) 2024-05-11T12:13:30.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.621+0000 7f4ecf400640 1 -- 172.21.15.167:0/2225332304 learned_addr learned my addr 172.21.15.167:0/2225332304 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:30.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.621+0000 7f4ecfe00640 1 --2- 172.21.15.167:0/2225332304 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ed00afcd0 0x7f4ed0146f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:30.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.621+0000 7f4ecf400640 1 -- 172.21.15.167:0/2225332304 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ed00afcd0 msgr2=0x7f4ed0146f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:30.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.621+0000 7f4ed4c00640 1 --2- 172.21.15.167:0/2225332304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ed00d04a0 0x7f4ed015e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:30.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.621+0000 7f4ecf400640 1 --2- 172.21.15.167:0/2225332304 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ed00afcd0 0x7f4ed0146f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:30.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.621+0000 7f4ecf400640 1 -- 172.21.15.167:0/2225332304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ed00d04a0 msgr2=0x7f4ed015e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:30.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.621+0000 7f4ecf400640 1 --2- 172.21.15.167:0/2225332304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ed00d04a0 0x7f4ed015e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:30.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.621+0000 7f4ecf400640 1 -- 172.21.15.167:0/2225332304 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ed015ca00 con 0x7f4ed00cf970 2024-05-11T12:13:30.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.622+0000 7f4ecf400640 1 --2- 172.21.15.167:0/2225332304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ed00cf970 0x7f4ed015e270 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc000c00 tx=0x7f4ebc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:30.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.622+0000 7f4ec7e00640 1 -- 172.21.15.167:0/2225332304 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc041020 con 0x7f4ed00cf970 2024-05-11T12:13:30.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.622+0000 7f4ec7e00640 1 -- 172.21.15.167:0/2225332304 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ebc0306c0 con 0x7f4ed00cf970 2024-05-11T12:13:30.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.622+0000 7f4ed5600640 1 -- 172.21.15.167:0/2225332304 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ed015cce0 con 0x7f4ed00cf970 2024-05-11T12:13:30.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.622+0000 7f4ec7e00640 1 -- 172.21.15.167:0/2225332304 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc037040 con 0x7f4ed00cf970 2024-05-11T12:13:30.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.622+0000 7f4ed5600640 1 -- 172.21.15.167:0/2225332304 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ed006c980 con 0x7f4ed00cf970 2024-05-11T12:13:30.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.623+0000 7f4ed5600640 1 -- 172.21.15.167:0/2225332304 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ed00694b0 con 0x7f4ed00cf970 2024-05-11T12:13:30.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.624+0000 7f4ec7e00640 1 -- 172.21.15.167:0/2225332304 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4ebc044020 con 0x7f4ed00cf970 2024-05-11T12:13:30.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.625+0000 7f4ec7e00640 1 --2- 172.21.15.167:0/2225332304 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4eb00762d0 0x7f4eb0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:30.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.625+0000 7f4ec7e00640 1 -- 172.21.15.167:0/2225332304 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5335+0+0 (secure 0 0 0) 0x7f4ebc0b7c80 con 0x7f4ed00cf970 2024-05-11T12:13:30.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.626+0000 7f4ecfe00640 1 --2- 172.21.15.167:0/2225332304 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4eb00762d0 0x7f4eb0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:30.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.626+0000 7f4ecfe00640 1 --2- 172.21.15.167:0/2225332304 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4eb00762d0 0x7f4eb0078790 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4ec00015a0 tx=0x7f4ec0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:30.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.630+0000 7f4ec7e00640 1 -- 172.21.15.167:0/2225332304 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4ebc04b030 con 0x7f4ed00cf970 2024-05-11T12:13:30.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.876+0000 7f4ed5600640 1 -- 172.21.15.167:0/2225332304 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ed00cf040 con 0x7f4ed00cf970 2024-05-11T12:13:30.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.877+0000 7f4ec7e00640 1 -- 172.21.15.167:0/2225332304 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+3878 (secure 0 0 0) 0x7f4ebc081740 con 0x7f4ed00cf970 2024-05-11T12:13:30.876 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:13:30.876 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":37,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/2785478956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2785478956},{"type":"v1","addr":"172.21.15.204:6834","nonce":2785478956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":4742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/3478043182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":3478043182},{"type":"v1","addr":"172.21.15.204:6835","nonce":3478043182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/618833945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":618833945},{"type":"v1","addr":"172.21.15.167:6836","nonce":618833945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5862,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3912732080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3912732080},{"type":"v1","addr":"172.21.15.167:6837","nonce":3912732080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[]} 2024-05-11T12:13:30.877 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 37 2024-05-11T12:13:30.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.879+0000 7f4ed5600640 1 -- 172.21.15.167:0/2225332304 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4eb00762d0 msgr2=0x7f4eb0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:30.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.879+0000 7f4ed5600640 1 --2- 172.21.15.167:0/2225332304 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4eb00762d0 0x7f4eb0078790 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4ec00015a0 tx=0x7f4ec0009040 comp rx=0 tx=0).stop 2024-05-11T12:13:30.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.879+0000 7f4ed5600640 1 -- 172.21.15.167:0/2225332304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ed00cf970 msgr2=0x7f4ed015e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:30.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.879+0000 7f4ed5600640 1 --2- 172.21.15.167:0/2225332304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ed00cf970 0x7f4ed015e270 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc000c00 tx=0x7f4ebc0046c0 comp rx=0 tx=0).stop 2024-05-11T12:13:30.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.880+0000 7f4ed5600640 1 -- 172.21.15.167:0/2225332304 shutdown_connections 2024-05-11T12:13:30.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.880+0000 7f4ed5600640 1 --2- 172.21.15.167:0/2225332304 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4eb00762d0 0x7f4eb0078790 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:30.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.880+0000 7f4ed5600640 1 --2- 172.21.15.167:0/2225332304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ed00d04a0 0x7f4ed015e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:30.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.880+0000 7f4ed5600640 1 --2- 172.21.15.167:0/2225332304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ed00cf970 0x7f4ed015e270 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:30.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.880+0000 7f4ed5600640 1 --2- 172.21.15.167:0/2225332304 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ed00afcd0 0x7f4ed0146f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:30.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.880+0000 7f4ed5600640 1 -- 172.21.15.167:0/2225332304 >> 172.21.15.167:0/2225332304 conn(0x7f4ed00b3460 msgr2=0x7f4ed00724d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:30.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.880+0000 7f4ed5600640 1 -- 172.21.15.167:0/2225332304 shutdown_connections 2024-05-11T12:13:30.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:30.880+0000 7f4ed5600640 1 -- 172.21.15.167:0/2225332304 wait complete. 2024-05-11T12:13:30.898 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-11T12:13:30.898 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-11T12:13:30.898 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-11T12:13:30.898 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T12:13:30.899 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-11T12:13:30.899 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-11T12:13:30.899 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-11T12:13:30.899 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T12:13:30.899 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-11T12:13:30.899 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-11T12:13:30.899 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-11T12:13:30.899 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T12:13:30.900 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-11T12:13:30.900 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-11T12:13:30.900 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-11T12:13:30.900 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T12:13:30.917 INFO:tasks.ceph.mds.a:Stopped 2024-05-11T12:13:30.917 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-11T12:13:30.920 INFO:tasks.ceph.mds.c:Stopped 2024-05-11T12:13:30.920 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-11T12:13:30.922 INFO:tasks.ceph.mds.b:Stopped 2024-05-11T12:13:30.922 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-11T12:13:30.928 INFO:tasks.ceph.mds.d:Stopped 2024-05-11T12:13:30.928 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-11T12:13:30.967 INFO:tasks.ceph.mds.a:Started 2024-05-11T12:13:30.967 DEBUG:teuthology.parallel:result is None 2024-05-11T12:13:30.973 INFO:tasks.ceph.mds.c:Started 2024-05-11T12:13:30.973 DEBUG:teuthology.parallel:result is None 2024-05-11T12:13:30.975 INFO:tasks.ceph.mds.b:Started 2024-05-11T12:13:30.975 DEBUG:teuthology.parallel:result is None 2024-05-11T12:13:30.981 INFO:tasks.ceph.mds.d:Started 2024-05-11T12:13:30.981 DEBUG:teuthology.parallel:result is None 2024-05-11T12:13:30.982 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-11T12:13:31.114 INFO:tasks.ceph.mds.c.smithi167.stdout:starting mds.c at 2024-05-11T12:13:31.116 INFO:tasks.ceph.mds.a.smithi167.stdout:starting mds.a at 2024-05-11T12:13:31.135 INFO:tasks.ceph.mds.b.smithi204.stdout:starting mds.b at 2024-05-11T12:13:31.137 INFO:tasks.ceph.mds.d.smithi204.stdout:starting mds.d at 2024-05-11T12:13:31.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.155+0000 7f075f200640 1 Processor -- start 2024-05-11T12:13:31.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.155+0000 7f075f200640 1 -- start start 2024-05-11T12:13:31.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.156+0000 7f075f200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f075815ac80 0x7f0758074260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:31.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.156+0000 7f075f200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f07580b3d50 0x7f0758160c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:31.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.156+0000 7f075f200640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07580b65e0 con 0x7f07581611b0 2024-05-11T12:13:31.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.156+0000 7f075f200640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f075806e3c0 con 0x7f075815ac80 2024-05-11T12:13:31.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.156+0000 7f075f200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f075815b060 con 0x7f07580b3d50 2024-05-11T12:13:31.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.157+0000 7f0757e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f075815ac80 0x7f0758074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:31.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.157+0000 7f0757e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f075815ac80 0x7f0758074260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:47888/0 (socket says 172.21.15.167:47888) 2024-05-11T12:13:31.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.157+0000 7f0757e00640 1 -- 172.21.15.167:0/1698608421 learned_addr learned my addr 172.21.15.167:0/1698608421 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:31.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.157+0000 7f074f400640 1 --2- 172.21.15.167:0/1698608421 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f07580b3d50 0x7f0758160c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:31.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.157+0000 7f0757e00640 1 -- 172.21.15.167:0/1698608421 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f07580b3d50 msgr2=0x7f0758160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:31.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.157+0000 7f0757e00640 1 --2- 172.21.15.167:0/1698608421 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f07580b3d50 0x7f0758160c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:31.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.157+0000 7f0757e00640 1 -- 172.21.15.167:0/1698608421 >> v1:172.21.15.167:6789/0 conn(0x7f07581611b0 legacy=0x7f075815d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:31.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.157+0000 7f0757e00640 1 -- 172.21.15.167:0/1698608421 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07580b68c0 con 0x7f075815ac80 2024-05-11T12:13:31.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.158+0000 7f0757e00640 1 --2- 172.21.15.167:0/1698608421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f075815ac80 0x7f0758074260 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f0748009870 tx=0x7f074802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e142c64648c92a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:31.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.158+0000 7f0757400640 1 -- 172.21.15.167:0/1698608421 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0748037070 con 0x7f075815ac80 2024-05-11T12:13:31.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.159+0000 7f0757400640 1 -- 172.21.15.167:0/1698608421 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0748032440 con 0x7f075815ac80 2024-05-11T12:13:31.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.159+0000 7f0757400640 1 -- 172.21.15.167:0/1698608421 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0748032810 con 0x7f075815ac80 2024-05-11T12:13:31.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.159+0000 7f075f200640 1 -- 172.21.15.167:0/1698608421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f075815ac80 msgr2=0x7f0758074260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:31.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.159+0000 7f075f200640 1 --2- 172.21.15.167:0/1698608421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f075815ac80 0x7f0758074260 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f0748009870 tx=0x7f074802a250 comp rx=0 tx=0).stop 2024-05-11T12:13:31.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.159+0000 7f075f200640 1 -- 172.21.15.167:0/1698608421 shutdown_connections 2024-05-11T12:13:31.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.159+0000 7f075f200640 1 --2- 172.21.15.167:0/1698608421 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f07580b3d50 0x7f0758160c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:31.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.159+0000 7f075f200640 1 --2- 172.21.15.167:0/1698608421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f075815ac80 0x7f0758074260 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:31.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.159+0000 7f075f200640 1 -- 172.21.15.167:0/1698608421 >> 172.21.15.167:0/1698608421 conn(0x7f075806c450 msgr2=0x7f075806c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:31.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.159+0000 7f075f200640 1 -- 172.21.15.167:0/1698608421 shutdown_connections 2024-05-11T12:13:31.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.159+0000 7f075f200640 1 -- 172.21.15.167:0/1698608421 wait complete. 2024-05-11T12:13:31.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.161+0000 7f075f200640 1 Processor -- start 2024-05-11T12:13:31.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.161+0000 7f075f200640 1 -- start start 2024-05-11T12:13:31.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.162+0000 7f075f200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f07580b3d50 0x7f07580c63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:31.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.162+0000 7f075f200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f075815ac80 0x7f075815b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:31.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.162+0000 7f075f200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f07581611b0 0x7f075815b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:31.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.162+0000 7f075f200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f07580b44f0 con 0x7f075815ac80 2024-05-11T12:13:31.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.162+0000 7f075f200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f07580b41b0 con 0x7f07580b3d50 2024-05-11T12:13:31.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.162+0000 7f075f200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0758073960 con 0x7f07581611b0 2024-05-11T12:13:31.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.162+0000 7f074f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f075815ac80 0x7f075815b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:31.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.162+0000 7f074f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f075815ac80 0x7f075815b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:47896/0 (socket says 172.21.15.167:47896) 2024-05-11T12:13:31.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.162+0000 7f074f400640 1 -- 172.21.15.167:0/3552855216 learned_addr learned my addr 172.21.15.167:0/3552855216 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:31.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.162+0000 7f0757e00640 1 --2- 172.21.15.167:0/3552855216 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f07580b3d50 0x7f07580c63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:31.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.162+0000 7f075d200640 1 --2- 172.21.15.167:0/3552855216 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f07581611b0 0x7f075815b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:31.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.163+0000 7f074f400640 1 -- 172.21.15.167:0/3552855216 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f07581611b0 msgr2=0x7f075815b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:31.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.163+0000 7f074f400640 1 --2- 172.21.15.167:0/3552855216 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f07581611b0 0x7f075815b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:31.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.163+0000 7f074f400640 1 -- 172.21.15.167:0/3552855216 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f07580b3d50 msgr2=0x7f07580c63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:31.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.163+0000 7f074f400640 1 --2- 172.21.15.167:0/3552855216 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f07580b3d50 0x7f07580c63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:31.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.163+0000 7f074f400640 1 -- 172.21.15.167:0/3552855216 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07580c6930 con 0x7f075815ac80 2024-05-11T12:13:31.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.163+0000 7f0757e00640 1 --2- 172.21.15.167:0/3552855216 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f07580b3d50 0x7f07580c63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:13:31.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.163+0000 7f074f400640 1 --2- 172.21.15.167:0/3552855216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f075815ac80 0x7f075815b1e0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f0740007f20 tx=0x7f074000e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:31.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.163+0000 7f0754c00640 1 -- 172.21.15.167:0/3552855216 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0740014620 con 0x7f075815ac80 2024-05-11T12:13:31.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.163+0000 7f0754c00640 1 -- 172.21.15.167:0/3552855216 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0740019070 con 0x7f075815ac80 2024-05-11T12:13:31.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.163+0000 7f075f200640 1 -- 172.21.15.167:0/3552855216 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0758136880 con 0x7f075815ac80 2024-05-11T12:13:31.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.163+0000 7f0754c00640 1 -- 172.21.15.167:0/3552855216 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0740014a00 con 0x7f075815ac80 2024-05-11T12:13:31.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.163+0000 7f075f200640 1 -- 172.21.15.167:0/3552855216 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0758136e40 con 0x7f075815ac80 2024-05-11T12:13:31.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.164+0000 7f075f200640 1 -- 172.21.15.167:0/3552855216 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07580694b0 con 0x7f075815ac80 2024-05-11T12:13:31.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.165+0000 7f0754c00640 1 -- 172.21.15.167:0/3552855216 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0740014ba0 con 0x7f075815ac80 2024-05-11T12:13:31.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.166+0000 7f0754c00640 1 --2- 172.21.15.167:0/3552855216 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0734076350 0x7f0734078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:31.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.167+0000 7f0754c00640 1 -- 172.21.15.167:0/3552855216 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5335+0+0 (secure 0 0 0) 0x7f0740097cb0 con 0x7f075815ac80 2024-05-11T12:13:31.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.167+0000 7f0757e00640 1 --2- 172.21.15.167:0/3552855216 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0734076350 0x7f0734078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:31.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.168+0000 7f0757e00640 1 --2- 172.21.15.167:0/3552855216 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0734076350 0x7f0734078810 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0748034000 tx=0x7f074802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:31.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.171+0000 7f0754c00640 1 -- 172.21.15.167:0/3552855216 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0740061860 con 0x7f075815ac80 2024-05-11T12:13:31.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.393+0000 7f075f200640 1 -- 172.21.15.167:0/3552855216 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f0758070120 con 0x7f075815ac80 2024-05-11T12:13:31.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.488+0000 7f0754c00640 1 -- 172.21.15.167:0/3552855216 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v69) ==== 97+0+0 (secure 0 0 0) 0x7f0740061160 con 0x7f075815ac80 2024-05-11T12:13:31.487 INFO:teuthology.orchestra.run.smithi167.stderr: removed all blocklist entries 2024-05-11T12:13:31.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.492+0000 7f075f200640 1 -- 172.21.15.167:0/3552855216 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0734076350 msgr2=0x7f0734078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:31.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.492+0000 7f075f200640 1 --2- 172.21.15.167:0/3552855216 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0734076350 0x7f0734078810 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0748034000 tx=0x7f074802e000 comp rx=0 tx=0).stop 2024-05-11T12:13:31.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.492+0000 7f075f200640 1 -- 172.21.15.167:0/3552855216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f075815ac80 msgr2=0x7f075815b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:31.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.492+0000 7f075f200640 1 --2- 172.21.15.167:0/3552855216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f075815ac80 0x7f075815b1e0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f0740007f20 tx=0x7f074000e570 comp rx=0 tx=0).stop 2024-05-11T12:13:31.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.493+0000 7f075f200640 1 -- 172.21.15.167:0/3552855216 shutdown_connections 2024-05-11T12:13:31.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.493+0000 7f075f200640 1 --2- 172.21.15.167:0/3552855216 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f07581611b0 0x7f075815b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:31.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.493+0000 7f075f200640 1 --2- 172.21.15.167:0/3552855216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f075815ac80 0x7f075815b1e0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:31.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.493+0000 7f075f200640 1 --2- 172.21.15.167:0/3552855216 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0734076350 0x7f0734078810 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:31.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.493+0000 7f075f200640 1 --2- 172.21.15.167:0/3552855216 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f07580b3d50 0x7f07580c63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:31.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.493+0000 7f075f200640 1 -- 172.21.15.167:0/3552855216 >> 172.21.15.167:0/3552855216 conn(0x7f075806c450 msgr2=0x7f07580b5ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:31.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.493+0000 7f075f200640 1 -- 172.21.15.167:0/3552855216 shutdown_connections 2024-05-11T12:13:31.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.493+0000 7f075f200640 1 -- 172.21.15.167:0/3552855216 wait complete. 2024-05-11T12:13:31.516 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-11T12:13:31.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.707+0000 7f56d6a00640 1 Processor -- start 2024-05-11T12:13:31.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.707+0000 7f56d6a00640 1 -- start start 2024-05-11T12:13:31.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.708+0000 7f56d6a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f56d013f290 0x7f56d00735c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:31.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.708+0000 7f56d6a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f56d00b50a0 0x7f56d0073b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:31.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.708+0000 7f56d6a00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56d015fb90 con 0x7f56d013eeb0 2024-05-11T12:13:31.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.708+0000 7f56d6a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f56d015d5b0 con 0x7f56d013f290 2024-05-11T12:13:31.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.708+0000 7f56d6a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f56d015fe70 con 0x7f56d00b50a0 2024-05-11T12:13:31.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.708+0000 7f56cfe00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f56d013eeb0 0x7f56d006f480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:52410/0 (socket says 172.21.15.167:52410) 2024-05-11T12:13:31.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.708+0000 7f56cfe00640 1 -- 172.21.15.167:0/1389743437 learned_addr learned my addr 172.21.15.167:0/1389743437 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:13:31.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.709+0000 7f56cea00640 1 -- 172.21.15.167:0/1389743437 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1718346089 0 0) 0x7f56d015fb90 con 0x7f56d013eeb0 2024-05-11T12:13:31.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.709+0000 7f56cea00640 1 -- 172.21.15.167:0/1389743437 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f56b4003520 con 0x7f56d013eeb0 2024-05-11T12:13:31.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.709+0000 7f56cf400640 1 --2- 172.21.15.167:0/1389743437 >> v2:172.21.15.204:3300/0 conn(0x7f56d013f290 0x7f56d00735c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:31.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.709+0000 7f56d4a00640 1 --2- 172.21.15.167:0/1389743437 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f56d00b50a0 0x7f56d0073b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:31.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.709+0000 7f56cea00640 1 -- 172.21.15.167:0/1389743437 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3381726114 0 0) 0x7f56b4003520 con 0x7f56d013eeb0 2024-05-11T12:13:31.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.709+0000 7f56cea00640 1 -- 172.21.15.167:0/1389743437 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f56d00b50a0 msgr2=0x7f56d0073b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:31.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.709+0000 7f56cea00640 1 --2- 172.21.15.167:0/1389743437 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f56d00b50a0 0x7f56d0073b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:31.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.709+0000 7f56cea00640 1 -- 172.21.15.167:0/1389743437 >> v2:172.21.15.204:3300/0 conn(0x7f56d013f290 msgr2=0x7f56d00735c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:31.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.709+0000 7f56cea00640 1 --2- 172.21.15.167:0/1389743437 >> v2:172.21.15.204:3300/0 conn(0x7f56d013f290 0x7f56d00735c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:31.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.709+0000 7f56cea00640 1 -- 172.21.15.167:0/1389743437 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56d0160150 con 0x7f56d013eeb0 2024-05-11T12:13:31.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.709+0000 7f56cea00640 1 -- 172.21.15.167:0/1389743437 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f56c00016f0 con 0x7f56d013eeb0 2024-05-11T12:13:31.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.709+0000 7f56cea00640 1 -- 172.21.15.167:0/1389743437 >> v1:172.21.15.167:6789/0 conn(0x7f56d013eeb0 legacy=0x7f56d006f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:31.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.709+0000 7f56cea00640 1 --2- 172.21.15.167:0/1389743437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56b4003a00 0x7f56b4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:31.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.709+0000 7f56cea00640 1 --2- 172.21.15.167:0/1389743437 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56b4004800 0x7f56b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:31.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.710+0000 7f56cea00640 1 --2- 172.21.15.167:0/1389743437 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f56d00b50a0 0x7f56b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:31.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.710+0000 7f56cea00640 1 -- 172.21.15.167:0/1389743437 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f56d015fe70 con 0x7f56b4004800 2024-05-11T12:13:31.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.710+0000 7f56cea00640 1 -- 172.21.15.167:0/1389743437 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f56b400baa0 con 0x7f56b4003a00 2024-05-11T12:13:31.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.710+0000 7f56cea00640 1 -- 172.21.15.167:0/1389743437 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f56b400bca0 con 0x7f56d00b50a0 2024-05-11T12:13:31.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.710+0000 7f56cf400640 1 --2- 172.21.15.167:0/1389743437 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56b4004800 0x7f56b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:31.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.710+0000 7f56cfe00640 1 --2- 172.21.15.167:0/1389743437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56b4003a00 0x7f56b4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:31.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.710+0000 7f56cf400640 1 -- 172.21.15.167:0/1389743437 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f56d00b50a0 msgr2=0x7f56b40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:31.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.710+0000 7f56cf400640 1 --2- 172.21.15.167:0/1389743437 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f56d00b50a0 0x7f56b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:31.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.710+0000 7f56cf400640 1 -- 172.21.15.167:0/1389743437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56b4003a00 msgr2=0x7f56b4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:31.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.710+0000 7f56cf400640 1 --2- 172.21.15.167:0/1389743437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56b4003a00 0x7f56b4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:31.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.710+0000 7f56cf400640 1 -- 172.21.15.167:0/1389743437 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56b4003520 con 0x7f56b4004800 2024-05-11T12:13:31.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.710+0000 7f56cfe00640 1 --2- 172.21.15.167:0/1389743437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56b4003a00 0x7f56b4003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:31.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.711+0000 7f56cf400640 1 --2- 172.21.15.167:0/1389743437 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56b4004800 0x7f56b4004c80 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f56bc009fc0 tx=0x7f56b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a154c75d0488ea8f server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:31.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.712+0000 7f56cea00640 1 -- 172.21.15.167:0/1389743437 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc009c10 con 0x7f56b4004800 2024-05-11T12:13:31.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.712+0000 7f56cea00640 1 -- 172.21.15.167:0/1389743437 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56bc036070 con 0x7f56b4004800 2024-05-11T12:13:31.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.712+0000 7f56cea00640 1 -- 172.21.15.167:0/1389743437 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc02d070 con 0x7f56b4004800 2024-05-11T12:13:31.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.712+0000 7f56d6a00640 1 -- 172.21.15.167:0/1389743437 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56b4004800 msgr2=0x7f56b4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:31.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.712+0000 7f56d6a00640 1 --2- 172.21.15.167:0/1389743437 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56b4004800 0x7f56b4004c80 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f56bc009fc0 tx=0x7f56b4005830 comp rx=0 tx=0).stop 2024-05-11T12:13:31.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.712+0000 7f56d4a00640 1 -- 172.21.15.167:0/1389743437 reap_dead start 2024-05-11T12:13:31.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.712+0000 7f56d6a00640 1 -- 172.21.15.167:0/1389743437 shutdown_connections 2024-05-11T12:13:31.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.712+0000 7f56d6a00640 1 -- 172.21.15.167:0/1389743437 >> 172.21.15.167:0/1389743437 conn(0x7f56d00cd8f0 msgr2=0x7f56d00cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:31.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.712+0000 7f56d6a00640 1 -- 172.21.15.167:0/1389743437 shutdown_connections 2024-05-11T12:13:31.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.712+0000 7f56d6a00640 1 -- 172.21.15.167:0/1389743437 wait complete. 2024-05-11T12:13:31.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.714+0000 7f56d6a00640 1 Processor -- start 2024-05-11T12:13:31.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.714+0000 7f56d6a00640 1 -- start start 2024-05-11T12:13:31.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.715+0000 7f56d6a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56b4004800 0x7f56d01385e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:31.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.715+0000 7f56d6a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56d013d620 0x7f56d013daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:31.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.715+0000 7f56d6a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f56d015e6c0 0x7f56d015eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:31.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.715+0000 7f56d6a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f56d006fb90 con 0x7f56b4004800 2024-05-11T12:13:31.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.715+0000 7f56d6a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f56d0160710 con 0x7f56d013d620 2024-05-11T12:13:31.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.715+0000 7f56d6a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f56d006fd10 con 0x7f56d015e6c0 2024-05-11T12:13:31.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.715+0000 7f56cfe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56b4004800 0x7f56d01385e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:31.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.716+0000 7f56cfe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56b4004800 0x7f56d01385e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:47906/0 (socket says 172.21.15.167:47906) 2024-05-11T12:13:31.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.716+0000 7f56cfe00640 1 -- 172.21.15.167:0/4185844918 learned_addr learned my addr 172.21.15.167:0/4185844918 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:31.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.716+0000 7f56cf400640 1 --2- 172.21.15.167:0/4185844918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56d013d620 0x7f56d013daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:31.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.716+0000 7f56d4a00640 1 --2- 172.21.15.167:0/4185844918 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f56d015e6c0 0x7f56d015eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:31.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.716+0000 7f56cfe00640 1 -- 172.21.15.167:0/4185844918 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f56d015e6c0 msgr2=0x7f56d015eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:31.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.716+0000 7f56cfe00640 1 --2- 172.21.15.167:0/4185844918 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f56d015e6c0 0x7f56d015eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:31.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.716+0000 7f56cfe00640 1 -- 172.21.15.167:0/4185844918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56d013d620 msgr2=0x7f56d013daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:31.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.716+0000 7f56cfe00640 1 --2- 172.21.15.167:0/4185844918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56d013d620 0x7f56d013daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:31.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.716+0000 7f56cfe00640 1 -- 172.21.15.167:0/4185844918 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56d015fb60 con 0x7f56b4004800 2024-05-11T12:13:31.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.716+0000 7f56cf400640 1 --2- 172.21.15.167:0/4185844918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56d013d620 0x7f56d013daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:31.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.716+0000 7f56cfe00640 1 --2- 172.21.15.167:0/4185844918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56b4004800 0x7f56d01385e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f56c000d830 tx=0x7f56c000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:31.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.717+0000 7f56c7e00640 1 -- 172.21.15.167:0/4185844918 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c00074e0 con 0x7f56b4004800 2024-05-11T12:13:31.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.717+0000 7f56c7e00640 1 -- 172.21.15.167:0/4185844918 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56c00047e0 con 0x7f56b4004800 2024-05-11T12:13:31.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.717+0000 7f56c7e00640 1 -- 172.21.15.167:0/4185844918 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c0004b90 con 0x7f56b4004800 2024-05-11T12:13:31.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.717+0000 7f56d6a00640 1 -- 172.21.15.167:0/4185844918 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56d015f2c0 con 0x7f56b4004800 2024-05-11T12:13:31.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.717+0000 7f56d6a00640 1 -- 172.21.15.167:0/4185844918 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56d015f880 con 0x7f56b4004800 2024-05-11T12:13:31.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.718+0000 7f56d6a00640 1 -- 172.21.15.167:0/4185844918 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56d0069590 con 0x7f56b4004800 2024-05-11T12:13:31.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.719+0000 7f56c7e00640 1 -- 172.21.15.167:0/4185844918 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f56c001b020 con 0x7f56b4004800 2024-05-11T12:13:31.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.720+0000 7f56c7e00640 1 --2- 172.21.15.167:0/4185844918 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f56b00762d0 0x7f56b0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:31.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.720+0000 7f56c7e00640 1 -- 172.21.15.167:0/4185844918 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5249+0+0 (secure 0 0 0) 0x7f56c0097410 con 0x7f56b4004800 2024-05-11T12:13:31.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.720+0000 7f56cf400640 1 --2- 172.21.15.167:0/4185844918 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f56b00762d0 0x7f56b0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:31.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.721+0000 7f56cf400640 1 --2- 172.21.15.167:0/4185844918 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f56b00762d0 0x7f56b0078790 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f56bc035000 tx=0x7f56bc02f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:31.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:31.725+0000 7f56c7e00640 1 -- 172.21.15.167:0/4185844918 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f56c0060f50 con 0x7f56b4004800 2024-05-11T12:13:32.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.004+0000 7f56d6a00640 1 -- 172.21.15.167:0/4185844918 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f56d0166f40 con 0x7f56b4004800 2024-05-11T12:13:32.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.005+0000 7f56c7e00640 1 -- 172.21.15.167:0/4185844918 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) ==== 80+0+5304 (secure 0 0 0) 0x7f56c0060850 con 0x7f56b4004800 2024-05-11T12:13:32.005 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:13:32.005 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T12:13:32.005 INFO:teuthology.orchestra.run.smithi167.stdout: "auth_dump": [ 2024-05-11T12:13:32.005 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:13:32.005 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "mds.a", 2024-05-11T12:13:32.005 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAEYD9msMGnFBAAamqokXQ+znxPHtHhhs611g==", 2024-05-11T12:13:32.005 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:13:32.005 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T12:13:32.005 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow *", 2024-05-11T12:13:32.005 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow *", 2024-05-11T12:13:32.005 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:13:32.005 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:13:32.006 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:13:32.006 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:13:32.006 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "mds.b", 2024-05-11T12:13:32.006 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAEYD9mqbCgMhAA5CNUf1cz77nUqLli2bTFcQ==", 2024-05-11T12:13:32.006 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:13:32.006 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T12:13:32.006 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow *", 2024-05-11T12:13:32.006 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow *", 2024-05-11T12:13:32.006 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:13:32.006 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:13:32.006 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:13:32.007 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:13:32.007 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "mds.c", 2024-05-11T12:13:32.007 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAEYD9mhylUIxAAqrIRYuNIHKfoI00zx+Gh1Q==", 2024-05-11T12:13:32.007 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:13:32.007 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T12:13:32.008 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow *", 2024-05-11T12:13:32.008 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow *", 2024-05-11T12:13:32.008 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:13:32.008 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:13:32.008 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:13:32.008 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:13:32.008 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "mds.d", 2024-05-11T12:13:32.009 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAFYD9mcUjpBRAAbx54iDHcqexfNXGKk/avlw==", 2024-05-11T12:13:32.009 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:13:32.009 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T12:13:32.009 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow *", 2024-05-11T12:13:32.009 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow *", 2024-05-11T12:13:32.009 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:13:32.009 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:13:32.009 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:13:32.010 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:13:32.010 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.0", 2024-05-11T12:13:32.010 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAJYD9mb71mBxAAKMSzkfppZVbEevwELKrqvw==", 2024-05-11T12:13:32.010 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:13:32.010 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T12:13:32.010 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T12:13:32.010 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:13:32.010 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:13:32.010 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:13:32.010 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:13:32.010 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.1", 2024-05-11T12:13:32.011 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAMYD9m/qx7EhAAUcZntottFNBwqjdc/CqFbw==", 2024-05-11T12:13:32.011 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:13:32.011 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T12:13:32.011 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T12:13:32.011 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:13:32.011 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:13:32.011 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:13:32.011 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:13:32.011 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.2", 2024-05-11T12:13:32.011 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAPYD9m5PKRKxAAFQuzh7dAN8PVEHBq+UdQvg==", 2024-05-11T12:13:32.011 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:13:32.011 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T12:13:32.011 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T12:13:32.011 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:13:32.012 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:13:32.012 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:13:32.012 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:13:32.012 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.3", 2024-05-11T12:13:32.012 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQASYD9mZAy+NhAA49k6QbSvtSxA/5t8bWbPuQ==", 2024-05-11T12:13:32.012 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:13:32.012 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T12:13:32.012 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T12:13:32.012 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:13:32.012 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:13:32.012 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:13:32.012 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:13:32.012 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.4", 2024-05-11T12:13:32.012 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAZYD9mEbsvIBAA3nkQXQTbIf7ubq1SJtUeIA==", 2024-05-11T12:13:32.012 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:13:32.013 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T12:13:32.013 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T12:13:32.013 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:13:32.013 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:13:32.013 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:13:32.013 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:13:32.013 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.5", 2024-05-11T12:13:32.013 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAcYD9mtdsHOxAAvNUSkMgQlDc2n1ak/lfRgg==", 2024-05-11T12:13:32.013 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:13:32.013 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T12:13:32.013 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T12:13:32.013 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:13:32.013 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:13:32.013 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:13:32.013 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:13:32.014 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.6", 2024-05-11T12:13:32.014 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAgYD9mr3u8CRAAm2ShwV0lJpIVudodmviusg==", 2024-05-11T12:13:32.014 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:13:32.014 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T12:13:32.014 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T12:13:32.014 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:13:32.014 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:13:32.014 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:13:32.014 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:13:32.014 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.7", 2024-05-11T12:13:32.014 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAjYD9m+0oyFRAAUVQW0kQmvtU8IE599nrWJQ==", 2024-05-11T12:13:32.014 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:13:32.014 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T12:13:32.015 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T12:13:32.015 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:13:32.015 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:13:32.015 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:13:32.015 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:13:32.015 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "client.0", 2024-05-11T12:13:32.015 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAFYD9mLK/lEhAAyERTYMl250KkBvIrAXWYSw==", 2024-05-11T12:13:32.015 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:13:32.015 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T12:13:32.015 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow r", 2024-05-11T12:13:32.015 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-11T12:13:32.015 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:13:32.015 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:13:32.015 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:13:32.016 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "client.1", 2024-05-11T12:13:32.016 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAFYD9m2oSjJhAAz53H1SvPStkxGq5V+2VFeA==", 2024-05-11T12:13:32.016 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:13:32.016 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T12:13:32.016 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow r", 2024-05-11T12:13:32.016 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow rw", 2024-05-11T12:13:32.016 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow rwx" 2024-05-11T12:13:32.016 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:13:32.016 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:13:32.016 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:13:32.016 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "client.2", 2024-05-11T12:13:32.016 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAFYD9mMzXnHBAAmJ//hwSofaZpk+6ykxVM/A==", 2024-05-11T12:13:32.016 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:13:32.016 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T12:13:32.016 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow r", 2024-05-11T12:13:32.016 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow rw", 2024-05-11T12:13:32.017 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow rwx" 2024-05-11T12:13:32.017 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:13:32.017 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:13:32.017 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:13:32.017 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "client.3", 2024-05-11T12:13:32.017 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAFYD9muOmsLxAA/tYFT4YNy3adENgVNLljdw==", 2024-05-11T12:13:32.017 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:13:32.017 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T12:13:32.017 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow r", 2024-05-11T12:13:32.017 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow rw", 2024-05-11T12:13:32.017 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow rwx" 2024-05-11T12:13:32.017 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:13:32.017 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:13:32.017 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:13:32.017 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "client.admin", 2024-05-11T12:13:32.018 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQADYD9mItegDBAALtuDP6qzAvbrwiROf94biw==", 2024-05-11T12:13:32.018 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:13:32.018 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow *", 2024-05-11T12:13:32.018 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow *", 2024-05-11T12:13:32.018 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow *", 2024-05-11T12:13:32.018 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:13:32.018 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:13:32.018 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:13:32.018 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:13:32.018 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "mgr.x", 2024-05-11T12:13:32.018 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAEYD9mAZTKCRAAJbY821O+4pUMC0DgxRa3og==", 2024-05-11T12:13:32.018 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:13:32.018 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow *", 2024-05-11T12:13:32.018 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile mgr", 2024-05-11T12:13:32.018 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:13:32.018 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:13:32.019 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:13:32.019 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:13:32.019 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "mgr.y", 2024-05-11T12:13:32.019 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQADYD9m2cZRORAAX9kYcT5/t4f5jQRZwXHHzw==", 2024-05-11T12:13:32.019 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:13:32.019 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow *", 2024-05-11T12:13:32.019 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile mgr", 2024-05-11T12:13:32.019 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:13:32.019 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:13:32.019 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:13:32.019 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-05-11T12:13:32.019 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T12:13:32.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.008+0000 7f56d6a00640 1 -- 172.21.15.167:0/4185844918 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f56b00762d0 msgr2=0x7f56b0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:32.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.008+0000 7f56d6a00640 1 --2- 172.21.15.167:0/4185844918 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f56b00762d0 0x7f56b0078790 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f56bc035000 tx=0x7f56bc02f000 comp rx=0 tx=0).stop 2024-05-11T12:13:32.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.008+0000 7f56d6a00640 1 -- 172.21.15.167:0/4185844918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56b4004800 msgr2=0x7f56d01385e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:32.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.008+0000 7f56d6a00640 1 --2- 172.21.15.167:0/4185844918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56b4004800 0x7f56d01385e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f56c000d830 tx=0x7f56c000dcf0 comp rx=0 tx=0).stop 2024-05-11T12:13:32.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.008+0000 7f56d6a00640 1 -- 172.21.15.167:0/4185844918 shutdown_connections 2024-05-11T12:13:32.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.008+0000 7f56d6a00640 1 --2- 172.21.15.167:0/4185844918 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f56d015e6c0 0x7f56d015eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:32.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.008+0000 7f56d6a00640 1 --2- 172.21.15.167:0/4185844918 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f56b00762d0 0x7f56b0078790 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:32.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.008+0000 7f56d6a00640 1 --2- 172.21.15.167:0/4185844918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56d013d620 0x7f56d013daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:32.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.008+0000 7f56d6a00640 1 --2- 172.21.15.167:0/4185844918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56b4004800 0x7f56d01385e0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:32.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.008+0000 7f56d6a00640 1 -- 172.21.15.167:0/4185844918 >> 172.21.15.167:0/4185844918 conn(0x7f56d00cd8f0 msgr2=0x7f56d006eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:32.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.008+0000 7f56d6a00640 1 -- 172.21.15.167:0/4185844918 shutdown_connections 2024-05-11T12:13:32.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.008+0000 7f56d6a00640 1 -- 172.21.15.167:0/4185844918 wait complete. 2024-05-11T12:13:32.025 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-11T12:13:32.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.226+0000 7fdac3e00640 1 Processor -- start 2024-05-11T12:13:32.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.226+0000 7fdac3e00640 1 -- start start 2024-05-11T12:13:32.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.227+0000 7fdac3e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fdac4145b00 0x7fdac406f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:32.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.227+0000 7fdac3e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdac4146770 0x7fdac40ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:32.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.228+0000 7fdac3e00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdac406fbc0 con 0x7fdac40cef10 2024-05-11T12:13:32.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.228+0000 7fdac3e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fdac40b4520 con 0x7fdac4145b00 2024-05-11T12:13:32.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.228+0000 7fdac3e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fdac40b4800 con 0x7fdac4146770 2024-05-11T12:13:32.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.228+0000 7fdac3400640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fdac40cef10 0x7fdac406c660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:52416/0 (socket says 172.21.15.167:52416) 2024-05-11T12:13:32.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.228+0000 7fdac3400640 1 -- 172.21.15.167:0/1447389140 learned_addr learned my addr 172.21.15.167:0/1447389140 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:13:32.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.228+0000 7fdac2000640 1 --2- 172.21.15.167:0/1447389140 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdac4146770 0x7fdac40ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:32.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.228+0000 7fdac2a00640 1 --2- 172.21.15.167:0/1447389140 >> v2:172.21.15.204:3300/0 conn(0x7fdac4145b00 0x7fdac406f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:32.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.228+0000 7fdac1600640 1 -- 172.21.15.167:0/1447389140 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3341484207 0 0) 0x7fdac406fbc0 con 0x7fdac40cef10 2024-05-11T12:13:32.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.228+0000 7fdac1600640 1 -- 172.21.15.167:0/1447389140 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdaa4003520 con 0x7fdac40cef10 2024-05-11T12:13:32.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.229+0000 7fdac1600640 1 -- 172.21.15.167:0/1447389140 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1935470746 0 0) 0x7fdaa4003520 con 0x7fdac40cef10 2024-05-11T12:13:32.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.229+0000 7fdac1600640 1 -- 172.21.15.167:0/1447389140 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdac4146770 msgr2=0x7fdac40ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:32.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.229+0000 7fdac1600640 1 --2- 172.21.15.167:0/1447389140 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdac4146770 0x7fdac40ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:32.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.229+0000 7fdac1600640 1 -- 172.21.15.167:0/1447389140 >> v2:172.21.15.204:3300/0 conn(0x7fdac4145b00 msgr2=0x7fdac406f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:32.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.229+0000 7fdac1600640 1 --2- 172.21.15.167:0/1447389140 >> v2:172.21.15.204:3300/0 conn(0x7fdac4145b00 0x7fdac406f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:32.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.229+0000 7fdac1600640 1 -- 172.21.15.167:0/1447389140 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdac40b4ae0 con 0x7fdac40cef10 2024-05-11T12:13:32.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.229+0000 7fdac1600640 1 -- 172.21.15.167:0/1447389140 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7fdabc001a70 con 0x7fdac40cef10 2024-05-11T12:13:32.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.229+0000 7fdac1600640 1 -- 172.21.15.167:0/1447389140 >> v1:172.21.15.167:6789/0 conn(0x7fdac40cef10 legacy=0x7fdac406c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:32.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.229+0000 7fdac1600640 1 --2- 172.21.15.167:0/1447389140 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdac4146770 0x7fdaa4003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:32.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.229+0000 7fdac1600640 1 --2- 172.21.15.167:0/1447389140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdaa4003fd0 0x7fdaa4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:32.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.229+0000 7fdac1600640 1 --2- 172.21.15.167:0/1447389140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdaa4004d40 0x7fdaa40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:32.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.229+0000 7fdac1600640 1 -- 172.21.15.167:0/1447389140 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdac40b4520 con 0x7fdaa4004d40 2024-05-11T12:13:32.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.229+0000 7fdac1600640 1 -- 172.21.15.167:0/1447389140 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdac40b4800 con 0x7fdaa4003fd0 2024-05-11T12:13:32.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.229+0000 7fdac1600640 1 -- 172.21.15.167:0/1447389140 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fdaa400baa0 con 0x7fdac4146770 2024-05-11T12:13:32.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.229+0000 7fdac3400640 1 --2- 172.21.15.167:0/1447389140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdaa4004d40 0x7fdaa40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:32.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.230+0000 7fdac2000640 1 --2- 172.21.15.167:0/1447389140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdaa4003fd0 0x7fdaa4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:32.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.230+0000 7fdac2a00640 1 --2- 172.21.15.167:0/1447389140 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdac4146770 0x7fdaa4003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:32.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.230+0000 7fdac3400640 1 -- 172.21.15.167:0/1447389140 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdac4146770 msgr2=0x7fdaa4003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:32.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.230+0000 7fdac3400640 1 --2- 172.21.15.167:0/1447389140 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdac4146770 0x7fdaa4003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:32.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.230+0000 7fdac3400640 1 -- 172.21.15.167:0/1447389140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdaa4003fd0 msgr2=0x7fdaa4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:32.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.230+0000 7fdac3400640 1 --2- 172.21.15.167:0/1447389140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdaa4003fd0 0x7fdaa4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:32.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.230+0000 7fdac3400640 1 -- 172.21.15.167:0/1447389140 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa4003520 con 0x7fdaa4004d40 2024-05-11T12:13:32.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.230+0000 7fdac2000640 1 --2- 172.21.15.167:0/1447389140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdaa4003fd0 0x7fdaa4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:13:32.232 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.231+0000 7fdac3400640 1 --2- 172.21.15.167:0/1447389140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdaa4004d40 0x7fdaa40051c0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fdabc00a850 tx=0x7fdaa4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c30d140f930f138b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:32.232 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.232+0000 7fdac1600640 1 -- 172.21.15.167:0/1447389140 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc00a4a0 con 0x7fdaa4004d40 2024-05-11T12:13:32.232 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.232+0000 7fdac1600640 1 -- 172.21.15.167:0/1447389140 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdabc02f070 con 0x7fdaa4004d40 2024-05-11T12:13:32.232 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.232+0000 7fdac1600640 1 -- 172.21.15.167:0/1447389140 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc02bc30 con 0x7fdaa4004d40 2024-05-11T12:13:32.232 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.232+0000 7fdac3e00640 1 -- 172.21.15.167:0/1447389140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdaa4004d40 msgr2=0x7fdaa40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:32.232 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.232+0000 7fdac3e00640 1 --2- 172.21.15.167:0/1447389140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdaa4004d40 0x7fdaa40051c0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fdabc00a850 tx=0x7fdaa4005830 comp rx=0 tx=0).stop 2024-05-11T12:13:32.232 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.232+0000 7fdac3400640 1 -- 172.21.15.167:0/1447389140 reap_dead start 2024-05-11T12:13:32.232 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.232+0000 7fdac3e00640 1 -- 172.21.15.167:0/1447389140 shutdown_connections 2024-05-11T12:13:32.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.232+0000 7fdac3e00640 1 -- 172.21.15.167:0/1447389140 >> 172.21.15.167:0/1447389140 conn(0x7fdac40b3460 msgr2=0x7fdac40d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:32.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.232+0000 7fdac3e00640 1 -- 172.21.15.167:0/1447389140 shutdown_connections 2024-05-11T12:13:32.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.233+0000 7fdac3e00640 1 -- 172.21.15.167:0/1447389140 wait complete. 2024-05-11T12:13:32.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.234+0000 7fdac3e00640 1 Processor -- start 2024-05-11T12:13:32.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.234+0000 7fdac3e00640 1 -- start start 2024-05-11T12:13:32.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.235+0000 7fdac3e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdaa4004d40 0x7fdac413d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:32.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.235+0000 7fdac3e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdac415e880 0x7fdac415ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:32.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.235+0000 7fdac3e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdac413ea10 0x7fdac413ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:32.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.235+0000 7fdac3e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdac406f980 con 0x7fdaa4004d40 2024-05-11T12:13:32.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.235+0000 7fdac3e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdac406f800 con 0x7fdac415e880 2024-05-11T12:13:32.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.235+0000 7fdac3e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fdac406cd70 con 0x7fdac413ea10 2024-05-11T12:13:32.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.236+0000 7fdac2a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdaa4004d40 0x7fdac413d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:32.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.236+0000 7fdac2a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdaa4004d40 0x7fdac413d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:47932/0 (socket says 172.21.15.167:47932) 2024-05-11T12:13:32.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.236+0000 7fdac2a00640 1 -- 172.21.15.167:0/2685434874 learned_addr learned my addr 172.21.15.167:0/2685434874 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:32.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.236+0000 7fdac2000640 1 --2- 172.21.15.167:0/2685434874 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdac415e880 0x7fdac415ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:32.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.236+0000 7fdac3400640 1 --2- 172.21.15.167:0/2685434874 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdac413ea10 0x7fdac413ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:32.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.236+0000 7fdac2a00640 1 -- 172.21.15.167:0/2685434874 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdac413ea10 msgr2=0x7fdac413ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:32.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.236+0000 7fdac2a00640 1 --2- 172.21.15.167:0/2685434874 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdac413ea10 0x7fdac413ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:32.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.236+0000 7fdac2a00640 1 -- 172.21.15.167:0/2685434874 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdac415e880 msgr2=0x7fdac415ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:32.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.236+0000 7fdac2a00640 1 --2- 172.21.15.167:0/2685434874 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdac415e880 0x7fdac415ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:32.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.236+0000 7fdac2a00640 1 -- 172.21.15.167:0/2685434874 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdac415d7e0 con 0x7fdaa4004d40 2024-05-11T12:13:32.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.237+0000 7fdac2a00640 1 --2- 172.21.15.167:0/2685434874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdaa4004d40 0x7fdac413d5c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fdab4011700 tx=0x7fdab4011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:32.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.237+0000 7fdabaa00640 1 -- 172.21.15.167:0/2685434874 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab400f9c0 con 0x7fdaa4004d40 2024-05-11T12:13:32.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.237+0000 7fdabaa00640 1 -- 172.21.15.167:0/2685434874 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdab400fb60 con 0x7fdaa4004d40 2024-05-11T12:13:32.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.237+0000 7fdac3e00640 1 -- 172.21.15.167:0/2685434874 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdac415dac0 con 0x7fdaa4004d40 2024-05-11T12:13:32.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.237+0000 7fdabaa00640 1 -- 172.21.15.167:0/2685434874 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab4015460 con 0x7fdaa4004d40 2024-05-11T12:13:32.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.237+0000 7fdac3e00640 1 -- 172.21.15.167:0/2685434874 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdac415e080 con 0x7fdaa4004d40 2024-05-11T12:13:32.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.238+0000 7fdac3e00640 1 -- 172.21.15.167:0/2685434874 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdac40694b0 con 0x7fdaa4004d40 2024-05-11T12:13:32.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.239+0000 7fdabaa00640 1 -- 172.21.15.167:0/2685434874 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fdab4015600 con 0x7fdaa4004d40 2024-05-11T12:13:32.239 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.240+0000 7fdabaa00640 1 --2- 172.21.15.167:0/2685434874 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fdaa00762d0 0x7fdaa0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:32.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.241+0000 7fdabaa00640 1 -- 172.21.15.167:0/2685434874 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5249+0+0 (secure 0 0 0) 0x7fdab4097db0 con 0x7fdaa4004d40 2024-05-11T12:13:32.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.241+0000 7fdac2000640 1 --2- 172.21.15.167:0/2685434874 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fdaa00762d0 0x7fdaa0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:32.241 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.241+0000 7fdac2000640 1 --2- 172.21.15.167:0/2685434874 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fdaa00762d0 0x7fdaa0078790 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fdac40b1b70 tx=0x7fdaac008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:32.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.245+0000 7fdabaa00640 1 -- 172.21.15.167:0/2685434874 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fdab4019030 con 0x7fdaa4004d40 2024-05-11T12:13:32.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.448+0000 7fdac3e00640 1 -- 172.21.15.167:0/2685434874 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fdac40b1cf0 con 0x7fdaa4004d40 2024-05-11T12:13:32.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.448+0000 7fdabaa00640 1 -- 172.21.15.167:0/2685434874 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v71) ==== 84+0+65 (secure 0 0 0) 0x7fdab4061960 con 0x7fdaa4004d40 2024-05-11T12:13:32.448 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:13:32.448 INFO:teuthology.orchestra.run.smithi167.stdout:[ 2024-05-11T12:13:32.448 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:13:32.448 INFO:teuthology.orchestra.run.smithi167.stdout: "poolnum": 1, 2024-05-11T12:13:32.448 INFO:teuthology.orchestra.run.smithi167.stdout: "poolname": ".mgr" 2024-05-11T12:13:32.448 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:13:32.448 INFO:teuthology.orchestra.run.smithi167.stdout:] 2024-05-11T12:13:32.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.451+0000 7fdac3e00640 1 -- 172.21.15.167:0/2685434874 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fdaa00762d0 msgr2=0x7fdaa0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:32.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.451+0000 7fdac3e00640 1 --2- 172.21.15.167:0/2685434874 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fdaa00762d0 0x7fdaa0078790 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fdac40b1b70 tx=0x7fdaac008040 comp rx=0 tx=0).stop 2024-05-11T12:13:32.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.451+0000 7fdac3e00640 1 -- 172.21.15.167:0/2685434874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdaa4004d40 msgr2=0x7fdac413d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:32.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.451+0000 7fdac3e00640 1 --2- 172.21.15.167:0/2685434874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdaa4004d40 0x7fdac413d5c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fdab4011700 tx=0x7fdab4011bc0 comp rx=0 tx=0).stop 2024-05-11T12:13:32.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.451+0000 7fdac3e00640 1 -- 172.21.15.167:0/2685434874 shutdown_connections 2024-05-11T12:13:32.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.451+0000 7fdac3e00640 1 --2- 172.21.15.167:0/2685434874 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdac413ea10 0x7fdac413ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:32.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.451+0000 7fdac3e00640 1 --2- 172.21.15.167:0/2685434874 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fdaa00762d0 0x7fdaa0078790 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:32.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.451+0000 7fdac3e00640 1 --2- 172.21.15.167:0/2685434874 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdac415e880 0x7fdac415ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:32.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.451+0000 7fdac3e00640 1 --2- 172.21.15.167:0/2685434874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdaa4004d40 0x7fdac413d5c0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:32.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.451+0000 7fdac3e00640 1 -- 172.21.15.167:0/2685434874 >> 172.21.15.167:0/2685434874 conn(0x7fdac40b3460 msgr2=0x7fdac40d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:32.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.451+0000 7fdac3e00640 1 -- 172.21.15.167:0/2685434874 shutdown_connections 2024-05-11T12:13:32.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.451+0000 7fdac3e00640 1 -- 172.21.15.167:0/2685434874 wait complete. 2024-05-11T12:13:32.469 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-11T12:13:32.469 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-11T12:13:32.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.664+0000 7f1a10a00640 1 Processor -- start 2024-05-11T12:13:32.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.665+0000 7f1a10a00640 1 -- start start 2024-05-11T12:13:32.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.666+0000 7f1a10a00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1a0c0cfd90 0x7f1a0c0d3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:32.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.666+0000 7f1a10a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a0c0d4310 0x7f1a0c06d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:32.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.666+0000 7f1a10a00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a0c15f640 con 0x7f1a0c15f260 2024-05-11T12:13:32.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.666+0000 7f1a10a00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f1a0c0d3940 con 0x7f1a0c0cfd90 2024-05-11T12:13:32.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.666+0000 7f1a10a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1a0c0d2d30 con 0x7f1a0c0d4310 2024-05-11T12:13:32.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.666+0000 7f1a09e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1a0c0cfd90 0x7f1a0c0d3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:32.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.666+0000 7f1a09e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1a0c0cfd90 0x7f1a0c0d3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:47936/0 (socket says 172.21.15.167:47936) 2024-05-11T12:13:32.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.666+0000 7f1a09e00640 1 -- 172.21.15.167:0/409995120 learned_addr learned my addr 172.21.15.167:0/409995120 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:32.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.667+0000 7f1a09e00640 1 -- 172.21.15.167:0/409995120 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a0c0d4310 msgr2=0x7f1a0c06d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:32.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.667+0000 7f1a09e00640 1 --2- 172.21.15.167:0/409995120 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a0c0d4310 0x7f1a0c06d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:32.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.667+0000 7f1a09e00640 1 -- 172.21.15.167:0/409995120 >> v1:172.21.15.167:6789/0 conn(0x7f1a0c15f260 legacy=0x7f1a0c1637e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:32.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.667+0000 7f1a09e00640 1 -- 172.21.15.167:0/409995120 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a0c0d3010 con 0x7f1a0c0cfd90 2024-05-11T12:13:32.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.668+0000 7f1a09e00640 1 --2- 172.21.15.167:0/409995120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a0c0cfd90 0x7f1a0c0d3dd0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f19f4009870 tx=0x7f19f402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=641ff16ac9c6f73b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:32.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.668+0000 7f1a09400640 1 -- 172.21.15.167:0/409995120 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f4037070 con 0x7f1a0c0cfd90 2024-05-11T12:13:32.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.668+0000 7f1a09400640 1 -- 172.21.15.167:0/409995120 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19f4032440 con 0x7f1a0c0cfd90 2024-05-11T12:13:32.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.668+0000 7f1a09400640 1 -- 172.21.15.167:0/409995120 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f4032810 con 0x7f1a0c0cfd90 2024-05-11T12:13:32.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.669+0000 7f1a10a00640 1 -- 172.21.15.167:0/409995120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a0c0cfd90 msgr2=0x7f1a0c0d3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:32.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.669+0000 7f1a10a00640 1 --2- 172.21.15.167:0/409995120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a0c0cfd90 0x7f1a0c0d3dd0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f19f4009870 tx=0x7f19f402a270 comp rx=0 tx=0).stop 2024-05-11T12:13:32.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.669+0000 7f1a10a00640 1 -- 172.21.15.167:0/409995120 shutdown_connections 2024-05-11T12:13:32.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.669+0000 7f1a10a00640 1 --2- 172.21.15.167:0/409995120 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a0c0d4310 0x7f1a0c06d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:32.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.669+0000 7f1a10a00640 1 --2- 172.21.15.167:0/409995120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a0c0cfd90 0x7f1a0c0d3dd0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:32.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.669+0000 7f1a10a00640 1 -- 172.21.15.167:0/409995120 >> 172.21.15.167:0/409995120 conn(0x7f1a0c0b6cd0 msgr2=0x7f1a0c0b70e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:32.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.669+0000 7f1a10a00640 1 -- 172.21.15.167:0/409995120 shutdown_connections 2024-05-11T12:13:32.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.669+0000 7f1a10a00640 1 -- 172.21.15.167:0/409995120 wait complete. 2024-05-11T12:13:32.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.671+0000 7f1a10a00640 1 Processor -- start 2024-05-11T12:13:32.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.671+0000 7f1a10a00640 1 -- start start 2024-05-11T12:13:32.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.672+0000 7f1a10a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a0c0cfd90 0x7f1a0c1582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:32.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.672+0000 7f1a10a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a0c0d4310 0x7f1a0c0cb820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:32.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.672+0000 7f1a10a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a0c15f260 0x7f1a0c1564f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:32.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.672+0000 7f1a10a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0c06d5a0 con 0x7f1a0c0d4310 2024-05-11T12:13:32.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.672+0000 7f1a10a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0c0d35d0 con 0x7f1a0c0cfd90 2024-05-11T12:13:32.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.672+0000 7f1a10a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1a0c06d720 con 0x7f1a0c15f260 2024-05-11T12:13:32.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.673+0000 7f1a09e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a0c0d4310 0x7f1a0c0cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:32.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.673+0000 7f1a09e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a0c0d4310 0x7f1a0c0cb820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:47938/0 (socket says 172.21.15.167:47938) 2024-05-11T12:13:32.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.673+0000 7f1a09e00640 1 -- 172.21.15.167:0/779952836 learned_addr learned my addr 172.21.15.167:0/779952836 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:32.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.673+0000 7f1a0a800640 1 --2- 172.21.15.167:0/779952836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a0c0cfd90 0x7f1a0c1582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:32.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.673+0000 7f1a09e00640 1 -- 172.21.15.167:0/779952836 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a0c15f260 msgr2=0x7f1a0c1564f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:32.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.673+0000 7f1a09e00640 1 --2- 172.21.15.167:0/779952836 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a0c15f260 0x7f1a0c1564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:32.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.673+0000 7f1a09e00640 1 -- 172.21.15.167:0/779952836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a0c0cfd90 msgr2=0x7f1a0c1582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:32.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.673+0000 7f1a09e00640 1 --2- 172.21.15.167:0/779952836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a0c0cfd90 0x7f1a0c1582b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:32.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.673+0000 7f1a09e00640 1 -- 172.21.15.167:0/779952836 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a0c157210 con 0x7f1a0c0d4310 2024-05-11T12:13:32.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.673+0000 7f1a0a800640 1 --2- 172.21.15.167:0/779952836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a0c0cfd90 0x7f1a0c1582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:32.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.673+0000 7f1a09e00640 1 --2- 172.21.15.167:0/779952836 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a0c0d4310 0x7f1a0c0cb820 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f19f4000c00 tx=0x7f19f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:32.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.674+0000 7f1a02a00640 1 -- 172.21.15.167:0/779952836 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f4041070 con 0x7f1a0c0d4310 2024-05-11T12:13:32.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.674+0000 7f1a02a00640 1 -- 172.21.15.167:0/779952836 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19f403c870 con 0x7f1a0c0d4310 2024-05-11T12:13:32.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.674+0000 7f1a02a00640 1 -- 172.21.15.167:0/779952836 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f4037040 con 0x7f1a0c0d4310 2024-05-11T12:13:32.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.674+0000 7f1a10a00640 1 -- 172.21.15.167:0/779952836 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a0c1574f0 con 0x7f1a0c0d4310 2024-05-11T12:13:32.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.674+0000 7f1a10a00640 1 -- 172.21.15.167:0/779952836 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a0c157ab0 con 0x7f1a0c0d4310 2024-05-11T12:13:32.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.675+0000 7f1a10a00640 1 -- 172.21.15.167:0/779952836 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a0c0694b0 con 0x7f1a0c0d4310 2024-05-11T12:13:32.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.676+0000 7f1a02a00640 1 -- 172.21.15.167:0/779952836 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f19f4044070 con 0x7f1a0c0d4310 2024-05-11T12:13:32.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.677+0000 7f1a02a00640 1 --2- 172.21.15.167:0/779952836 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f19e80762d0 0x7f19e8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:32.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.678+0000 7f1a02a00640 1 -- 172.21.15.167:0/779952836 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5249+0+0 (secure 0 0 0) 0x7f19f40b80a0 con 0x7f1a0c0d4310 2024-05-11T12:13:32.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.678+0000 7f1a0a800640 1 --2- 172.21.15.167:0/779952836 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f19e80762d0 0x7f19e8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:32.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.679+0000 7f1a0a800640 1 --2- 172.21.15.167:0/779952836 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f19e80762d0 0x7f19e8078790 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c06d160 tx=0x7f19fc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:32.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.683+0000 7f1a02a00640 1 -- 172.21.15.167:0/779952836 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f19f4081b20 con 0x7f1a0c0d4310 2024-05-11T12:13:32.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:32.890+0000 7f1a10a00640 1 -- 172.21.15.167:0/779952836 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f1a0c0759b0 con 0x7f1a0c0d4310 2024-05-11T12:13:33.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.502+0000 7f1a02a00640 1 -- 172.21.15.167:0/779952836 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v72) ==== 138+0+0 (secure 0 0 0) 0x7f19f4081420 con 0x7f1a0c0d4310 2024-05-11T12:13:33.502 INFO:teuthology.orchestra.run.smithi167.stderr:pool 'cephfs_metadata' created 2024-05-11T12:13:33.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.506+0000 7f1a10a00640 1 -- 172.21.15.167:0/779952836 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f19e80762d0 msgr2=0x7f19e8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:33.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.506+0000 7f1a10a00640 1 --2- 172.21.15.167:0/779952836 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f19e80762d0 0x7f19e8078790 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c06d160 tx=0x7f19fc009040 comp rx=0 tx=0).stop 2024-05-11T12:13:33.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.506+0000 7f1a10a00640 1 -- 172.21.15.167:0/779952836 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a0c0d4310 msgr2=0x7f1a0c0cb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:33.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.506+0000 7f1a10a00640 1 --2- 172.21.15.167:0/779952836 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a0c0d4310 0x7f1a0c0cb820 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f19f4000c00 tx=0x7f19f40046c0 comp rx=0 tx=0).stop 2024-05-11T12:13:33.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.507+0000 7f1a10a00640 1 -- 172.21.15.167:0/779952836 shutdown_connections 2024-05-11T12:13:33.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.507+0000 7f1a10a00640 1 --2- 172.21.15.167:0/779952836 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a0c15f260 0x7f1a0c1564f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:33.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.507+0000 7f1a10a00640 1 --2- 172.21.15.167:0/779952836 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a0c0d4310 0x7f1a0c0cb820 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:33.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.507+0000 7f1a10a00640 1 --2- 172.21.15.167:0/779952836 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f19e80762d0 0x7f19e8078790 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:33.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.507+0000 7f1a10a00640 1 --2- 172.21.15.167:0/779952836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a0c0cfd90 0x7f1a0c1582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:33.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.507+0000 7f1a10a00640 1 -- 172.21.15.167:0/779952836 >> 172.21.15.167:0/779952836 conn(0x7f1a0c0b6cd0 msgr2=0x7f1a0c0b9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:33.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.507+0000 7f1a10a00640 1 -- 172.21.15.167:0/779952836 shutdown_connections 2024-05-11T12:13:33.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.507+0000 7f1a10a00640 1 -- 172.21.15.167:0/779952836 wait complete. 2024-05-11T12:13:33.535 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-11T12:13:33.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.734+0000 7f5215e00640 1 Processor -- start 2024-05-11T12:13:33.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.734+0000 7f5215e00640 1 -- start start 2024-05-11T12:13:33.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.736+0000 7f5215e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5210072aa0 0x7f521006e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:33.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.736+0000 7f5215e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f521015e360 0x7f521006e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:33.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.736+0000 7f5215e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5210161a60 con 0x7f5210142750 2024-05-11T12:13:33.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.736+0000 7f5215e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f521015d620 con 0x7f5210072aa0 2024-05-11T12:13:33.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.736+0000 7f5215e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5210143440 con 0x7f521015e360 2024-05-11T12:13:33.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.737+0000 7f520ea00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5210072aa0 0x7f521006e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:33.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.737+0000 7f520e000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f521015e360 0x7f521006e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:33.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.737+0000 7f520e000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f521015e360 0x7f521006e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:47158/0 (socket says 172.21.15.167:47158) 2024-05-11T12:13:33.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.737+0000 7f520e000640 1 -- 172.21.15.167:0/351341913 learned_addr learned my addr 172.21.15.167:0/351341913 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:33.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.738+0000 7f520ea00640 1 -- 172.21.15.167:0/351341913 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f521015e360 msgr2=0x7f521006e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:33.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.738+0000 7f520ea00640 1 --2- 172.21.15.167:0/351341913 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f521015e360 0x7f521006e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:33.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.738+0000 7f520ea00640 1 -- 172.21.15.167:0/351341913 >> v1:172.21.15.204:6789/0 conn(0x7f5210142750 legacy=0x7f5210142c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:33.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.738+0000 7f520ea00640 1 -- 172.21.15.167:0/351341913 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f521015d880 con 0x7f5210072aa0 2024-05-11T12:13:33.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.738+0000 7f520e000640 1 --2- 172.21.15.167:0/351341913 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f521015e360 0x7f521006e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:13:33.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.739+0000 7f520ea00640 1 --2- 172.21.15.167:0/351341913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5210072aa0 0x7f521006e350 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f52000097a0 tx=0x7f520002a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccf14f72396bafa9 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:33.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.740+0000 7f520d600640 1 -- 172.21.15.167:0/351341913 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5200037070 con 0x7f5210072aa0 2024-05-11T12:13:33.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.740+0000 7f520d600640 1 -- 172.21.15.167:0/351341913 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f520002ae50 con 0x7f5210072aa0 2024-05-11T12:13:33.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.740+0000 7f520d600640 1 -- 172.21.15.167:0/351341913 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5200032660 con 0x7f5210072aa0 2024-05-11T12:13:33.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.740+0000 7f5215e00640 1 -- 172.21.15.167:0/351341913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5210072aa0 msgr2=0x7f521006e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:33.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.740+0000 7f5215e00640 1 --2- 172.21.15.167:0/351341913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5210072aa0 0x7f521006e350 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f52000097a0 tx=0x7f520002a100 comp rx=0 tx=0).stop 2024-05-11T12:13:33.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.740+0000 7f5215e00640 1 -- 172.21.15.167:0/351341913 shutdown_connections 2024-05-11T12:13:33.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.740+0000 7f5215e00640 1 --2- 172.21.15.167:0/351341913 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f521015e360 0x7f521006e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:33.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.740+0000 7f5215e00640 1 --2- 172.21.15.167:0/351341913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5210072aa0 0x7f521006e350 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:33.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.740+0000 7f5215e00640 1 -- 172.21.15.167:0/351341913 >> 172.21.15.167:0/351341913 conn(0x7f52100b2f60 msgr2=0x7f52100b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:33.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.741+0000 7f5215e00640 1 -- 172.21.15.167:0/351341913 shutdown_connections 2024-05-11T12:13:33.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.741+0000 7f5215e00640 1 -- 172.21.15.167:0/351341913 wait complete. 2024-05-11T12:13:33.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.742+0000 7f5215e00640 1 Processor -- start 2024-05-11T12:13:33.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.742+0000 7f5215e00640 1 -- start start 2024-05-11T12:13:33.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.743+0000 7f5215e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5210072aa0 0x7f52101405c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:33.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.743+0000 7f5215e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5210142750 0x7f52101398f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:33.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.743+0000 7f5215e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f521015e360 0x7f52100b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:33.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.743+0000 7f5215e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5210072e80 con 0x7f5210142750 2024-05-11T12:13:33.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.743+0000 7f5215e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f521015de40 con 0x7f5210072aa0 2024-05-11T12:13:33.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.743+0000 7f5215e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5210073000 con 0x7f521015e360 2024-05-11T12:13:33.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.744+0000 7f520e000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5210142750 0x7f52101398f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:33.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.744+0000 7f520e000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5210142750 0x7f52101398f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:47940/0 (socket says 172.21.15.167:47940) 2024-05-11T12:13:33.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.744+0000 7f520e000640 1 -- 172.21.15.167:0/1706196861 learned_addr learned my addr 172.21.15.167:0/1706196861 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:33.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.744+0000 7f520ea00640 1 --2- 172.21.15.167:0/1706196861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5210072aa0 0x7f52101405c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:33.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.744+0000 7f520f400640 1 --2- 172.21.15.167:0/1706196861 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f521015e360 0x7f52100b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:33.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.744+0000 7f520e000640 1 -- 172.21.15.167:0/1706196861 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f521015e360 msgr2=0x7f52100b4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:33.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.744+0000 7f520e000640 1 --2- 172.21.15.167:0/1706196861 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f521015e360 0x7f52100b4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:33.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.744+0000 7f520e000640 1 -- 172.21.15.167:0/1706196861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5210072aa0 msgr2=0x7f52101405c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:33.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.744+0000 7f520e000640 1 --2- 172.21.15.167:0/1706196861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5210072aa0 0x7f52101405c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:33.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.744+0000 7f520e000640 1 -- 172.21.15.167:0/1706196861 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5210138e00 con 0x7f5210142750 2024-05-11T12:13:33.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.745+0000 7f520e000640 1 --2- 172.21.15.167:0/1706196861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5210142750 0x7f52101398f0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f51fc00e9b0 tx=0x7f51fc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:33.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.745+0000 7f5207400640 1 -- 172.21.15.167:0/1706196861 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51fc017070 con 0x7f5210142750 2024-05-11T12:13:33.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.745+0000 7f5207400640 1 -- 172.21.15.167:0/1706196861 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51fc00ca00 con 0x7f5210142750 2024-05-11T12:13:33.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.745+0000 7f5215e00640 1 -- 172.21.15.167:0/1706196861 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52101390e0 con 0x7f5210142750 2024-05-11T12:13:33.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.745+0000 7f5207400640 1 -- 172.21.15.167:0/1706196861 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51fc00cdf0 con 0x7f5210142750 2024-05-11T12:13:33.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.745+0000 7f5215e00640 1 -- 172.21.15.167:0/1706196861 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52100b49f0 con 0x7f5210142750 2024-05-11T12:13:33.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.746+0000 7f5215e00640 1 -- 172.21.15.167:0/1706196861 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5210073ae0 con 0x7f5210142750 2024-05-11T12:13:33.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.747+0000 7f5207400640 1 -- 172.21.15.167:0/1706196861 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f51fc01a070 con 0x7f5210142750 2024-05-11T12:13:33.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.748+0000 7f5207400640 1 --2- 172.21.15.167:0/1706196861 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f51f00762d0 0x7f51f0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:33.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.749+0000 7f5207400640 1 -- 172.21.15.167:0/1706196861 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5619+0+0 (secure 0 0 0) 0x7f51fc0978c0 con 0x7f5210142750 2024-05-11T12:13:33.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.749+0000 7f520ea00640 1 --2- 172.21.15.167:0/1706196861 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f51f00762d0 0x7f51f0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:33.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.749+0000 7f520ea00640 1 --2- 172.21.15.167:0/1706196861 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f51f00762d0 0x7f51f0078790 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5200034000 tx=0x7f520002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:33.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.754+0000 7f5207400640 1 -- 172.21.15.167:0/1706196861 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5210073ae0 con 0x7f5210142750 2024-05-11T12:13:33.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:33.967+0000 7f5215e00640 1 -- 172.21.15.167:0/1706196861 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f5210069630 con 0x7f5210142750 2024-05-11T12:13:34.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.509+0000 7f5207400640 1 -- 172.21.15.167:0/1706196861 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v73) ==== 170+0+0 (secure 0 0 0) 0x7f51fc0611d0 con 0x7f5210142750 2024-05-11T12:13:34.509 INFO:teuthology.orchestra.run.smithi167.stderr:pool 'cephfs_data' created 2024-05-11T12:13:34.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.514+0000 7f5215e00640 1 -- 172.21.15.167:0/1706196861 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f51f00762d0 msgr2=0x7f51f0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:34.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.514+0000 7f5215e00640 1 --2- 172.21.15.167:0/1706196861 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f51f00762d0 0x7f51f0078790 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5200034000 tx=0x7f520002e000 comp rx=0 tx=0).stop 2024-05-11T12:13:34.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.514+0000 7f5215e00640 1 -- 172.21.15.167:0/1706196861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5210142750 msgr2=0x7f52101398f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:34.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.514+0000 7f5215e00640 1 --2- 172.21.15.167:0/1706196861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5210142750 0x7f52101398f0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f51fc00e9b0 tx=0x7f51fc00ee70 comp rx=0 tx=0).stop 2024-05-11T12:13:34.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.515+0000 7f5215e00640 1 -- 172.21.15.167:0/1706196861 shutdown_connections 2024-05-11T12:13:34.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.515+0000 7f5215e00640 1 --2- 172.21.15.167:0/1706196861 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f521015e360 0x7f52100b4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:34.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.515+0000 7f5215e00640 1 --2- 172.21.15.167:0/1706196861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5210142750 0x7f52101398f0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:34.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.515+0000 7f5215e00640 1 --2- 172.21.15.167:0/1706196861 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f51f00762d0 0x7f51f0078790 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:34.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.515+0000 7f5215e00640 1 --2- 172.21.15.167:0/1706196861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5210072aa0 0x7f52101405c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:34.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.515+0000 7f5215e00640 1 -- 172.21.15.167:0/1706196861 >> 172.21.15.167:0/1706196861 conn(0x7f52100b2f60 msgr2=0x7f5210160a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:34.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.515+0000 7f5215e00640 1 -- 172.21.15.167:0/1706196861 shutdown_connections 2024-05-11T12:13:34.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.516+0000 7f5215e00640 1 -- 172.21.15.167:0/1706196861 wait complete. 2024-05-11T12:13:34.550 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-11T12:13:34.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.727+0000 7fc628400640 1 Processor -- start 2024-05-11T12:13:34.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.728+0000 7fc628400640 1 -- start start 2024-05-11T12:13:34.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.729+0000 7fc628400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc6201468b0 0x7fc620072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:34.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.729+0000 7fc628400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc620164340 0x7fc62006eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:34.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.729+0000 7fc628400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc620167520 con 0x7fc620072690 2024-05-11T12:13:34.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.729+0000 7fc628400640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fc62006ea10 con 0x7fc620164340 2024-05-11T12:13:34.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.729+0000 7fc628400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fc620164810 con 0x7fc6201468b0 2024-05-11T12:13:34.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.730+0000 7fc626400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc620164340 0x7fc62006eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:34.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.730+0000 7fc626400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc620164340 0x7fc62006eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:47952/0 (socket says 172.21.15.167:47952) 2024-05-11T12:13:34.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.730+0000 7fc626400640 1 -- 172.21.15.167:0/2039787975 learned_addr learned my addr 172.21.15.167:0/2039787975 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:34.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.730+0000 7fc625a00640 1 --2- 172.21.15.167:0/2039787975 >> v2:172.21.15.204:3300/0 conn(0x7fc6201468b0 0x7fc620072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:34.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.730+0000 7fc626400640 1 -- 172.21.15.167:0/2039787975 >> v2:172.21.15.204:3300/0 conn(0x7fc6201468b0 msgr2=0x7fc620072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:34.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.730+0000 7fc626400640 1 --2- 172.21.15.167:0/2039787975 >> v2:172.21.15.204:3300/0 conn(0x7fc6201468b0 0x7fc620072150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:34.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.730+0000 7fc626400640 1 -- 172.21.15.167:0/2039787975 >> v1:172.21.15.204:6789/0 conn(0x7fc620072690 legacy=0x7fc620163c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:34.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.730+0000 7fc626400640 1 -- 172.21.15.167:0/2039787975 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc620164af0 con 0x7fc620164340 2024-05-11T12:13:34.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.730+0000 7fc625a00640 1 --2- 172.21.15.167:0/2039787975 >> v2:172.21.15.204:3300/0 conn(0x7fc6201468b0 0x7fc620072150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:34.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.731+0000 7fc626400640 1 --2- 172.21.15.167:0/2039787975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc620164340 0x7fc62006eea0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fc620068f70 tx=0x7fc61802a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da326e4ee109c024 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:34.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.731+0000 7fc61fe00640 1 -- 172.21.15.167:0/2039787975 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc618036070 con 0x7fc620164340 2024-05-11T12:13:34.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.732+0000 7fc61fe00640 1 -- 172.21.15.167:0/2039787975 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc618031600 con 0x7fc620164340 2024-05-11T12:13:34.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.732+0000 7fc61fe00640 1 -- 172.21.15.167:0/2039787975 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6180319d0 con 0x7fc620164340 2024-05-11T12:13:34.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.732+0000 7fc628400640 1 -- 172.21.15.167:0/2039787975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc620164340 msgr2=0x7fc62006eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:34.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.732+0000 7fc628400640 1 --2- 172.21.15.167:0/2039787975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc620164340 0x7fc62006eea0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fc620068f70 tx=0x7fc61802a570 comp rx=0 tx=0).stop 2024-05-11T12:13:34.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.732+0000 7fc628400640 1 -- 172.21.15.167:0/2039787975 shutdown_connections 2024-05-11T12:13:34.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.732+0000 7fc628400640 1 --2- 172.21.15.167:0/2039787975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc620164340 0x7fc62006eea0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:34.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.732+0000 7fc628400640 1 --2- 172.21.15.167:0/2039787975 >> v2:172.21.15.204:3300/0 conn(0x7fc6201468b0 0x7fc620072150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:34.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.732+0000 7fc628400640 1 -- 172.21.15.167:0/2039787975 >> 172.21.15.167:0/2039787975 conn(0x7fc62006c450 msgr2=0x7fc62006c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:34.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.732+0000 7fc628400640 1 -- 172.21.15.167:0/2039787975 shutdown_connections 2024-05-11T12:13:34.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.732+0000 7fc628400640 1 -- 172.21.15.167:0/2039787975 wait complete. 2024-05-11T12:13:34.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.734+0000 7fc628400640 1 Processor -- start 2024-05-11T12:13:34.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.734+0000 7fc628400640 1 -- start start 2024-05-11T12:13:34.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.735+0000 7fc628400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc620072690 0x7fc620139670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:34.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.735+0000 7fc628400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6201468b0 0x7fc62014c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:34.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.735+0000 7fc628400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc620164340 0x7fc6200cf920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:34.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.735+0000 7fc628400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc620146e20 con 0x7fc620164340 2024-05-11T12:13:34.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.735+0000 7fc628400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc62006e6a0 con 0x7fc6201468b0 2024-05-11T12:13:34.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.735+0000 7fc628400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc620146fa0 con 0x7fc620072690 2024-05-11T12:13:34.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.736+0000 7fc626400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc620164340 0x7fc6200cf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:34.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.736+0000 7fc626400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc620164340 0x7fc6200cf920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:47958/0 (socket says 172.21.15.167:47958) 2024-05-11T12:13:34.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.736+0000 7fc626400640 1 -- 172.21.15.167:0/1177290780 learned_addr learned my addr 172.21.15.167:0/1177290780 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:34.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.736+0000 7fc625a00640 1 --2- 172.21.15.167:0/1177290780 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc620072690 0x7fc620139670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:34.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.736+0000 7fc625000640 1 --2- 172.21.15.167:0/1177290780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6201468b0 0x7fc62014c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:34.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.736+0000 7fc626400640 1 -- 172.21.15.167:0/1177290780 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc620072690 msgr2=0x7fc620139670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:34.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.736+0000 7fc626400640 1 --2- 172.21.15.167:0/1177290780 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc620072690 0x7fc620139670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:34.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.736+0000 7fc626400640 1 -- 172.21.15.167:0/1177290780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6201468b0 msgr2=0x7fc62014c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:34.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.736+0000 7fc626400640 1 --2- 172.21.15.167:0/1177290780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6201468b0 0x7fc62014c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:34.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.736+0000 7fc626400640 1 -- 172.21.15.167:0/1177290780 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6201fa940 con 0x7fc620164340 2024-05-11T12:13:34.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.736+0000 7fc625000640 1 --2- 172.21.15.167:0/1177290780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6201468b0 0x7fc62014c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:34.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.737+0000 7fc626400640 1 --2- 172.21.15.167:0/1177290780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc620164340 0x7fc6200cf920 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc618009f40 tx=0x7fc618004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:34.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.737+0000 7fc61d600640 1 -- 172.21.15.167:0/1177290780 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc618036050 con 0x7fc620164340 2024-05-11T12:13:34.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.737+0000 7fc61d600640 1 -- 172.21.15.167:0/1177290780 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc618042070 con 0x7fc620164340 2024-05-11T12:13:34.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.737+0000 7fc628400640 1 -- 172.21.15.167:0/1177290780 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6201fac20 con 0x7fc620164340 2024-05-11T12:13:34.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.737+0000 7fc61d600640 1 -- 172.21.15.167:0/1177290780 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc618007bf0 con 0x7fc620164340 2024-05-11T12:13:34.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.737+0000 7fc628400640 1 -- 172.21.15.167:0/1177290780 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6201fb1e0 con 0x7fc620164340 2024-05-11T12:13:34.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.738+0000 7fc628400640 1 -- 172.21.15.167:0/1177290780 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc620069520 con 0x7fc620164340 2024-05-11T12:13:34.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.740+0000 7fc61d600640 1 -- 172.21.15.167:0/1177290780 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc618047460 con 0x7fc620164340 2024-05-11T12:13:34.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.741+0000 7fc61d600640 1 --2- 172.21.15.167:0/1177290780 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc6040762d0 0x7fc604078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:34.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.741+0000 7fc61d600640 1 -- 172.21.15.167:0/1177290780 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6001+0+0 (secure 0 0 0) 0x7fc6180b92d0 con 0x7fc620164340 2024-05-11T12:13:34.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.742+0000 7fc625a00640 1 --2- 172.21.15.167:0/1177290780 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc6040762d0 0x7fc604078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:34.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.742+0000 7fc625a00640 1 --2- 172.21.15.167:0/1177290780 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc6040762d0 0x7fc604078790 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc61400a990 tx=0x7fc614008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:34.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.745+0000 7fc61d600640 1 -- 172.21.15.167:0/1177290780 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc618082ae0 con 0x7fc620164340 2024-05-11T12:13:34.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:34.992+0000 7fc628400640 1 -- 172.21.15.167:0/1177290780 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fc6200702f0 con 0x7fc620164340 2024-05-11T12:13:35.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.536+0000 7fc61d600640 1 -- 172.21.15.167:0/1177290780 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v40) ==== 174+0+0 (secure 0 0 0) 0x7fc6180823e0 con 0x7fc620164340 2024-05-11T12:13:35.535 INFO:teuthology.orchestra.run.smithi167.stderr:new fs with metadata pool 11 and data pool 12 2024-05-11T12:13:35.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.541+0000 7fc628400640 1 -- 172.21.15.167:0/1177290780 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc6040762d0 msgr2=0x7fc604078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:35.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.541+0000 7fc628400640 1 --2- 172.21.15.167:0/1177290780 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc6040762d0 0x7fc604078790 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc61400a990 tx=0x7fc614008040 comp rx=0 tx=0).stop 2024-05-11T12:13:35.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.541+0000 7fc628400640 1 -- 172.21.15.167:0/1177290780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc620164340 msgr2=0x7fc6200cf920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:35.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.541+0000 7fc628400640 1 --2- 172.21.15.167:0/1177290780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc620164340 0x7fc6200cf920 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc618009f40 tx=0x7fc618004a40 comp rx=0 tx=0).stop 2024-05-11T12:13:35.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.541+0000 7fc628400640 1 -- 172.21.15.167:0/1177290780 shutdown_connections 2024-05-11T12:13:35.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.541+0000 7fc628400640 1 --2- 172.21.15.167:0/1177290780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc620164340 0x7fc6200cf920 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:35.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.541+0000 7fc628400640 1 --2- 172.21.15.167:0/1177290780 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc6040762d0 0x7fc604078790 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:35.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.541+0000 7fc628400640 1 --2- 172.21.15.167:0/1177290780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6201468b0 0x7fc62014c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:35.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.541+0000 7fc628400640 1 --2- 172.21.15.167:0/1177290780 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc620072690 0x7fc620139670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:35.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.541+0000 7fc628400640 1 -- 172.21.15.167:0/1177290780 >> 172.21.15.167:0/1177290780 conn(0x7fc62006c450 msgr2=0x7fc6201655c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:35.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.542+0000 7fc628400640 1 -- 172.21.15.167:0/1177290780 shutdown_connections 2024-05-11T12:13:35.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.543+0000 7fc628400640 1 -- 172.21.15.167:0/1177290780 wait complete. 2024-05-11T12:13:35.583 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:13:35.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.756+0000 7fdceb600640 1 Processor -- start 2024-05-11T12:13:35.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.756+0000 7fdceb600640 1 -- start start 2024-05-11T12:13:35.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.758+0000 7fdceb600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fdce413eeb0 0x7fdce406f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:35.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.758+0000 7fdceb600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdce413f290 0x7fdce406f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:35.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.758+0000 7fdceb600640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdce415fb90 con 0x7fdce40b50a0 2024-05-11T12:13:35.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.758+0000 7fdceb600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fdce415d5b0 con 0x7fdce413eeb0 2024-05-11T12:13:35.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.758+0000 7fdceb600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fdce415fe70 con 0x7fdce413f290 2024-05-11T12:13:35.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.758+0000 7fdce9600640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fdce40b50a0 0x7fdce40735c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:52444/0 (socket says 172.21.15.167:52444) 2024-05-11T12:13:35.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.758+0000 7fdce9600640 1 -- 172.21.15.167:0/3531528087 learned_addr learned my addr 172.21.15.167:0/3531528087 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:13:35.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.758+0000 7fdce8c00640 1 --2- 172.21.15.167:0/3531528087 >> v2:172.21.15.204:3300/0 conn(0x7fdce413eeb0 0x7fdce406f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:35.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.758+0000 7fdce3400640 1 -- 172.21.15.167:0/3531528087 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1711322470 0 0) 0x7fdce415fb90 con 0x7fdce40b50a0 2024-05-11T12:13:35.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.759+0000 7fdce3e00640 1 --2- 172.21.15.167:0/3531528087 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdce413f290 0x7fdce406f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:35.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.759+0000 7fdce3400640 1 -- 172.21.15.167:0/3531528087 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdccc003520 con 0x7fdce40b50a0 2024-05-11T12:13:35.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.759+0000 7fdce3400640 1 -- 172.21.15.167:0/3531528087 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1273667357 0 0) 0x7fdccc003520 con 0x7fdce40b50a0 2024-05-11T12:13:35.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.759+0000 7fdce3400640 1 -- 172.21.15.167:0/3531528087 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdce413f290 msgr2=0x7fdce406f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:35.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.759+0000 7fdce3400640 1 --2- 172.21.15.167:0/3531528087 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdce413f290 0x7fdce406f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:35.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.759+0000 7fdce3400640 1 -- 172.21.15.167:0/3531528087 >> v2:172.21.15.204:3300/0 conn(0x7fdce413eeb0 msgr2=0x7fdce406f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:35.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.759+0000 7fdce3400640 1 --2- 172.21.15.167:0/3531528087 >> v2:172.21.15.204:3300/0 conn(0x7fdce413eeb0 0x7fdce406f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:35.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.759+0000 7fdce3400640 1 -- 172.21.15.167:0/3531528087 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdce4160150 con 0x7fdce40b50a0 2024-05-11T12:13:35.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.759+0000 7fdce3400640 1 -- 172.21.15.167:0/3531528087 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7fdcdc001a70 con 0x7fdce40b50a0 2024-05-11T12:13:35.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.759+0000 7fdce8c00640 1 --2- 172.21.15.167:0/3531528087 >> v2:172.21.15.204:3300/0 conn(0x7fdce413eeb0 0x7fdce406f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:13:35.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.759+0000 7fdce3400640 1 -- 172.21.15.167:0/3531528087 >> v1:172.21.15.167:6789/0 conn(0x7fdce40b50a0 legacy=0x7fdce40735c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:35.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.759+0000 7fdce3400640 1 --2- 172.21.15.167:0/3531528087 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdce413f290 0x7fdccc003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:35.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.759+0000 7fdce3400640 1 --2- 172.21.15.167:0/3531528087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdccc004080 0x7fdccc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:35.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.759+0000 7fdce3400640 1 --2- 172.21.15.167:0/3531528087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdccc004d40 0x7fdccc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:35.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.759+0000 7fdce3400640 1 -- 172.21.15.167:0/3531528087 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdce415fe70 con 0x7fdccc004080 2024-05-11T12:13:35.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.759+0000 7fdce3400640 1 -- 172.21.15.167:0/3531528087 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdccc00bbb0 con 0x7fdccc004d40 2024-05-11T12:13:35.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.759+0000 7fdce3400640 1 -- 172.21.15.167:0/3531528087 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fdccc00bdb0 con 0x7fdce413f290 2024-05-11T12:13:35.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.759+0000 7fdce3e00640 1 --2- 172.21.15.167:0/3531528087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdccc004080 0x7fdccc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:35.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.760+0000 7fdce3e00640 1 -- 172.21.15.167:0/3531528087 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdce413f290 msgr2=0x7fdccc003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:35.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.760+0000 7fdce3e00640 1 --2- 172.21.15.167:0/3531528087 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdce413f290 0x7fdccc003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:35.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.760+0000 7fdce3e00640 1 -- 172.21.15.167:0/3531528087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdccc004d40 msgr2=0x7fdccc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:35.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.760+0000 7fdce3e00640 1 --2- 172.21.15.167:0/3531528087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdccc004d40 0x7fdccc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:35.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.760+0000 7fdce3e00640 1 -- 172.21.15.167:0/3531528087 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdccc003520 con 0x7fdccc004080 2024-05-11T12:13:35.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.761+0000 7fdce3e00640 1 --2- 172.21.15.167:0/3531528087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdccc004080 0x7fdccc004800 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fdcd000b5c0 tx=0x7fdccc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a56c1ab223031c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:35.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.761+0000 7fdce3400640 1 -- 172.21.15.167:0/3531528087 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd003a070 con 0x7fdccc004080 2024-05-11T12:13:35.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.761+0000 7fdce3400640 1 -- 172.21.15.167:0/3531528087 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcd0035410 con 0x7fdccc004080 2024-05-11T12:13:35.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.761+0000 7fdce3400640 1 -- 172.21.15.167:0/3531528087 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd000b600 con 0x7fdccc004080 2024-05-11T12:13:35.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.762+0000 7fdceb600640 1 -- 172.21.15.167:0/3531528087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdccc004080 msgr2=0x7fdccc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:35.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.762+0000 7fdceb600640 1 --2- 172.21.15.167:0/3531528087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdccc004080 0x7fdccc004800 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fdcd000b5c0 tx=0x7fdccc005830 comp rx=0 tx=0).stop 2024-05-11T12:13:35.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.762+0000 7fdce9600640 1 -- 172.21.15.167:0/3531528087 reap_dead start 2024-05-11T12:13:35.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.762+0000 7fdceb600640 1 -- 172.21.15.167:0/3531528087 shutdown_connections 2024-05-11T12:13:35.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.762+0000 7fdceb600640 1 -- 172.21.15.167:0/3531528087 >> 172.21.15.167:0/3531528087 conn(0x7fdce40cd8f0 msgr2=0x7fdce40cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:35.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.762+0000 7fdceb600640 1 -- 172.21.15.167:0/3531528087 shutdown_connections 2024-05-11T12:13:35.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.762+0000 7fdceb600640 1 -- 172.21.15.167:0/3531528087 wait complete. 2024-05-11T12:13:35.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.764+0000 7fdceb600640 1 Processor -- start 2024-05-11T12:13:35.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.764+0000 7fdceb600640 1 -- start start 2024-05-11T12:13:35.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.765+0000 7fdceb600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdccc004080 0x7fdce41f4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:35.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.765+0000 7fdceb600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdce406c450 0x7fdce406c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:35.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.765+0000 7fdceb600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdce4159dd0 0x7fdce415a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:35.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.765+0000 7fdceb600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdce4073cd0 con 0x7fdce406c450 2024-05-11T12:13:35.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.765+0000 7fdceb600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdce4160710 con 0x7fdccc004080 2024-05-11T12:13:35.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.765+0000 7fdceb600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fdce4073e50 con 0x7fdce4159dd0 2024-05-11T12:13:35.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.765+0000 7fdce3e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdce406c450 0x7fdce406c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:35.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.765+0000 7fdce3e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdce406c450 0x7fdce406c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:47988/0 (socket says 172.21.15.167:47988) 2024-05-11T12:13:35.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.765+0000 7fdce3e00640 1 -- 172.21.15.167:0/1517810928 learned_addr learned my addr 172.21.15.167:0/1517810928 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:35.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.765+0000 7fdce8c00640 1 --2- 172.21.15.167:0/1517810928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdccc004080 0x7fdce41f4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:35.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.765+0000 7fdce3e00640 1 -- 172.21.15.167:0/1517810928 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdce4159dd0 msgr2=0x7fdce415a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:35.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.765+0000 7fdce3e00640 1 --2- 172.21.15.167:0/1517810928 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdce4159dd0 0x7fdce415a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:35.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.765+0000 7fdce3e00640 1 -- 172.21.15.167:0/1517810928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdccc004080 msgr2=0x7fdce41f4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:35.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.765+0000 7fdce3e00640 1 --2- 172.21.15.167:0/1517810928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdccc004080 0x7fdce41f4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:35.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.765+0000 7fdce3e00640 1 -- 172.21.15.167:0/1517810928 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdce415a9e0 con 0x7fdce406c450 2024-05-11T12:13:35.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.766+0000 7fdce3e00640 1 --2- 172.21.15.167:0/1517810928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdce406c450 0x7fdce406c8d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fdcd002cb10 tx=0x7fdcd0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:35.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.766+0000 7fdce0c00640 1 -- 172.21.15.167:0/1517810928 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd003a070 con 0x7fdce406c450 2024-05-11T12:13:35.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.766+0000 7fdce0c00640 1 -- 172.21.15.167:0/1517810928 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcd0004020 con 0x7fdce406c450 2024-05-11T12:13:35.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.766+0000 7fdce0c00640 1 -- 172.21.15.167:0/1517810928 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd000b600 con 0x7fdce406c450 2024-05-11T12:13:35.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.766+0000 7fdceb600640 1 -- 172.21.15.167:0/1517810928 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdce4158b90 con 0x7fdce406c450 2024-05-11T12:13:35.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.766+0000 7fdceb600640 1 -- 172.21.15.167:0/1517810928 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdce4159110 con 0x7fdce406c450 2024-05-11T12:13:35.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.767+0000 7fdceb600640 1 -- 172.21.15.167:0/1517810928 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdce4069590 con 0x7fdce406c450 2024-05-11T12:13:35.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.768+0000 7fdce0c00640 1 -- 172.21.15.167:0/1517810928 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fdcd003e070 con 0x7fdce406c450 2024-05-11T12:13:35.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.769+0000 7fdce0c00640 1 --2- 172.21.15.167:0/1517810928 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fdcc40762d0 0x7fdcc4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:35.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.769+0000 7fdce0c00640 1 -- 172.21.15.167:0/1517810928 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7fdcd00b7750 con 0x7fdce406c450 2024-05-11T12:13:35.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.770+0000 7fdce8c00640 1 --2- 172.21.15.167:0/1517810928 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fdcc40762d0 0x7fdcc4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:35.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.770+0000 7fdce8c00640 1 --2- 172.21.15.167:0/1517810928 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fdcc40762d0 0x7fdcc4078790 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8006a90 tx=0x7fdcd8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:35.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.775+0000 7fdce0c00640 1 -- 172.21.15.167:0/1517810928 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fdcd0080830 con 0x7fdce406c450 2024-05-11T12:13:35.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.968+0000 7fdceb600640 1 -- 172.21.15.167:0/1517810928 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdce4075b40 con 0x7fdce406c450 2024-05-11T12:13:35.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.972+0000 7fdce0c00640 1 -- 172.21.15.167:0/1517810928 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+17006 (secure 0 0 0) 0x7fdcd0087e20 con 0x7fdce406c450 2024-05-11T12:13:35.971 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:13:35.971 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":74,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:13:34.993990+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:13:32.903032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-11T12:13:33.980753+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:13:35.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.974+0000 7fdceb600640 1 -- 172.21.15.167:0/1517810928 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fdcc40762d0 msgr2=0x7fdcc4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:35.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.974+0000 7fdceb600640 1 --2- 172.21.15.167:0/1517810928 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fdcc40762d0 0x7fdcc4078790 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8006a90 tx=0x7fdcd8008040 comp rx=0 tx=0).stop 2024-05-11T12:13:35.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.974+0000 7fdceb600640 1 -- 172.21.15.167:0/1517810928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdce406c450 msgr2=0x7fdce406c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:35.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.974+0000 7fdceb600640 1 --2- 172.21.15.167:0/1517810928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdce406c450 0x7fdce406c8d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fdcd002cb10 tx=0x7fdcd0004730 comp rx=0 tx=0).stop 2024-05-11T12:13:35.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.974+0000 7fdceb600640 1 -- 172.21.15.167:0/1517810928 shutdown_connections 2024-05-11T12:13:35.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.974+0000 7fdceb600640 1 --2- 172.21.15.167:0/1517810928 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdce4159dd0 0x7fdce415a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:35.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.974+0000 7fdceb600640 1 --2- 172.21.15.167:0/1517810928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdce406c450 0x7fdce406c8d0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:35.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.974+0000 7fdceb600640 1 --2- 172.21.15.167:0/1517810928 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fdcc40762d0 0x7fdcc4078790 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:35.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.974+0000 7fdceb600640 1 --2- 172.21.15.167:0/1517810928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdccc004080 0x7fdce41f4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:35.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.974+0000 7fdceb600640 1 -- 172.21.15.167:0/1517810928 >> 172.21.15.167:0/1517810928 conn(0x7fdce40cd8f0 msgr2=0x7fdce406f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:35.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.975+0000 7fdceb600640 1 -- 172.21.15.167:0/1517810928 shutdown_connections 2024-05-11T12:13:35.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:35.975+0000 7fdceb600640 1 -- 172.21.15.167:0/1517810928 wait complete. 2024-05-11T12:13:35.992 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:13:36.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.191+0000 7f4a47e00640 1 Processor -- start 2024-05-11T12:13:36.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.192+0000 7f4a47e00640 1 -- start start 2024-05-11T12:13:36.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.193+0000 7f4a47e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4a4806c6f0 0x7f4a4806cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:36.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.193+0000 7f4a47e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a480b87f0 0x7f4a480b8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:36.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.193+0000 7f4a47e00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a4806d0e0 con 0x7f4a480d0af0 2024-05-11T12:13:36.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.193+0000 7f4a47e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f4a4806d390 con 0x7f4a4806c6f0 2024-05-11T12:13:36.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.193+0000 7f4a47e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4a4806d670 con 0x7f4a480b87f0 2024-05-11T12:13:36.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.194+0000 7f4a46a00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f4a480d0af0 0x7f4a480d0ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:52452/0 (socket says 172.21.15.167:52452) 2024-05-11T12:13:36.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.194+0000 7f4a46a00640 1 -- 172.21.15.167:0/81362679 learned_addr learned my addr 172.21.15.167:0/81362679 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:13:36.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.194+0000 7f4a46000640 1 --2- 172.21.15.167:0/81362679 >> v2:172.21.15.204:3300/0 conn(0x7f4a4806c6f0 0x7f4a4806cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:36.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.194+0000 7f4a47400640 1 --2- 172.21.15.167:0/81362679 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a480b87f0 0x7f4a480b8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:36.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.194+0000 7f4a45600640 1 -- 172.21.15.167:0/81362679 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1970329477 0 0) 0x7f4a4806d0e0 con 0x7f4a480d0af0 2024-05-11T12:13:36.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.194+0000 7f4a45600640 1 -- 172.21.15.167:0/81362679 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a2c003520 con 0x7f4a480d0af0 2024-05-11T12:13:36.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.195+0000 7f4a45600640 1 -- 172.21.15.167:0/81362679 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3018343363 0 0) 0x7f4a2c003520 con 0x7f4a480d0af0 2024-05-11T12:13:36.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.195+0000 7f4a45600640 1 -- 172.21.15.167:0/81362679 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a480b87f0 msgr2=0x7f4a480b8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:36.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.195+0000 7f4a45600640 1 --2- 172.21.15.167:0/81362679 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a480b87f0 0x7f4a480b8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:36.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.195+0000 7f4a45600640 1 -- 172.21.15.167:0/81362679 >> v2:172.21.15.204:3300/0 conn(0x7f4a4806c6f0 msgr2=0x7f4a4806cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:36.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.195+0000 7f4a45600640 1 --2- 172.21.15.167:0/81362679 >> v2:172.21.15.204:3300/0 conn(0x7f4a4806c6f0 0x7f4a4806cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:36.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.195+0000 7f4a45600640 1 -- 172.21.15.167:0/81362679 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a4806d950 con 0x7f4a480d0af0 2024-05-11T12:13:36.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.195+0000 7f4a45600640 1 -- 172.21.15.167:0/81362679 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f4a380016f0 con 0x7f4a480d0af0 2024-05-11T12:13:36.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.195+0000 7f4a45600640 1 -- 172.21.15.167:0/81362679 >> v1:172.21.15.167:6789/0 conn(0x7f4a480d0af0 legacy=0x7f4a480d0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:36.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.195+0000 7f4a45600640 1 --2- 172.21.15.167:0/81362679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a2c003a00 0x7f4a2c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:36.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.195+0000 7f4a45600640 1 --2- 172.21.15.167:0/81362679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a2c004800 0x7f4a2c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:36.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.195+0000 7f4a45600640 1 --2- 172.21.15.167:0/81362679 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a480b87f0 0x7f4a2c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:36.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.195+0000 7f4a45600640 1 -- 172.21.15.167:0/81362679 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4806d390 con 0x7f4a2c003a00 2024-05-11T12:13:36.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.195+0000 7f4a45600640 1 -- 172.21.15.167:0/81362679 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4806d670 con 0x7f4a2c004800 2024-05-11T12:13:36.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.195+0000 7f4a45600640 1 -- 172.21.15.167:0/81362679 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4a2c00b9c0 con 0x7f4a480b87f0 2024-05-11T12:13:36.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.195+0000 7f4a46a00640 1 --2- 172.21.15.167:0/81362679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a2c003a00 0x7f4a2c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:36.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.196+0000 7f4a46a00640 1 -- 172.21.15.167:0/81362679 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a480b87f0 msgr2=0x7f4a2c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:36.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.196+0000 7f4a47400640 1 --2- 172.21.15.167:0/81362679 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a480b87f0 0x7f4a2c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:36.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.196+0000 7f4a46a00640 1 --2- 172.21.15.167:0/81362679 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a480b87f0 0x7f4a2c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:36.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.196+0000 7f4a46a00640 1 -- 172.21.15.167:0/81362679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a2c004800 msgr2=0x7f4a2c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:36.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.196+0000 7f4a46a00640 1 --2- 172.21.15.167:0/81362679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a2c004800 0x7f4a2c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:36.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.196+0000 7f4a46a00640 1 -- 172.21.15.167:0/81362679 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a2c003520 con 0x7f4a2c003a00 2024-05-11T12:13:36.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.197+0000 7f4a46a00640 1 --2- 172.21.15.167:0/81362679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a2c003a00 0x7f4a2c003e20 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f4a3800c240 tx=0x7f4a2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bf88ab264d21f80 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:36.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.197+0000 7f4a45600640 1 -- 172.21.15.167:0/81362679 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a38007d50 con 0x7f4a2c003a00 2024-05-11T12:13:36.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.197+0000 7f4a45600640 1 -- 172.21.15.167:0/81362679 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a38005af0 con 0x7f4a2c003a00 2024-05-11T12:13:36.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.197+0000 7f4a45600640 1 -- 172.21.15.167:0/81362679 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a38007070 con 0x7f4a2c003a00 2024-05-11T12:13:36.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.198+0000 7f4a47e00640 1 -- 172.21.15.167:0/81362679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a2c003a00 msgr2=0x7f4a2c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:36.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.198+0000 7f4a47e00640 1 --2- 172.21.15.167:0/81362679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a2c003a00 0x7f4a2c003e20 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f4a3800c240 tx=0x7f4a2c005830 comp rx=0 tx=0).stop 2024-05-11T12:13:36.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.198+0000 7f4a47400640 1 -- 172.21.15.167:0/81362679 reap_dead start 2024-05-11T12:13:36.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.198+0000 7f4a47e00640 1 -- 172.21.15.167:0/81362679 shutdown_connections 2024-05-11T12:13:36.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.198+0000 7f4a47e00640 1 -- 172.21.15.167:0/81362679 >> 172.21.15.167:0/81362679 conn(0x7f4a480d1a40 msgr2=0x7f4a480d1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:36.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.198+0000 7f4a47e00640 1 -- 172.21.15.167:0/81362679 shutdown_connections 2024-05-11T12:13:36.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.198+0000 7f4a47e00640 1 -- 172.21.15.167:0/81362679 wait complete. 2024-05-11T12:13:36.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.200+0000 7f4a47e00640 1 Processor -- start 2024-05-11T12:13:36.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.200+0000 7f4a47e00640 1 -- start start 2024-05-11T12:13:36.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.201+0000 7f4a47e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a2c003a00 0x7f4a4815d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:36.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.201+0000 7f4a47e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a480cdf30 0x7f4a480ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:36.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.201+0000 7f4a47e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a48156c40 0x7f4a481570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:36.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.201+0000 7f4a47e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4815f470 con 0x7f4a2c003a00 2024-05-11T12:13:36.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.201+0000 7f4a47e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4a480b8260 con 0x7f4a48156c40 2024-05-11T12:13:36.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.201+0000 7f4a47e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4a4815f5f0 con 0x7f4a480cdf30 2024-05-11T12:13:36.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.201+0000 7f4a46a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a2c003a00 0x7f4a4815d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:36.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.201+0000 7f4a46a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a2c003a00 0x7f4a4815d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:47998/0 (socket says 172.21.15.167:47998) 2024-05-11T12:13:36.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.201+0000 7f4a46a00640 1 -- 172.21.15.167:0/3274570571 learned_addr learned my addr 172.21.15.167:0/3274570571 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:36.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.201+0000 7f4a46a00640 1 -- 172.21.15.167:0/3274570571 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a480cdf30 msgr2=0x7f4a480ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:36.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.201+0000 7f4a46a00640 1 --2- 172.21.15.167:0/3274570571 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a480cdf30 0x7f4a480ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:36.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.201+0000 7f4a46a00640 1 -- 172.21.15.167:0/3274570571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a48156c40 msgr2=0x7f4a481570f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:36.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.201+0000 7f4a46a00640 1 --2- 172.21.15.167:0/3274570571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a48156c40 0x7f4a481570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:36.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.201+0000 7f4a46a00640 1 -- 172.21.15.167:0/3274570571 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a48157970 con 0x7f4a2c003a00 2024-05-11T12:13:36.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.201+0000 7f4a46a00640 1 --2- 172.21.15.167:0/3274570571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a2c003a00 0x7f4a4815d8e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f4a3800c240 tx=0x7f4a38001a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:36.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.202+0000 7f4a3ea00640 1 -- 172.21.15.167:0/3274570571 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a38037040 con 0x7f4a2c003a00 2024-05-11T12:13:36.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.202+0000 7f4a3ea00640 1 -- 172.21.15.167:0/3274570571 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a38041070 con 0x7f4a2c003a00 2024-05-11T12:13:36.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.202+0000 7f4a47e00640 1 -- 172.21.15.167:0/3274570571 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a48157c50 con 0x7f4a2c003a00 2024-05-11T12:13:36.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.202+0000 7f4a3ea00640 1 -- 172.21.15.167:0/3274570571 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a38007d50 con 0x7f4a2c003a00 2024-05-11T12:13:36.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.202+0000 7f4a47e00640 1 -- 172.21.15.167:0/3274570571 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a48158210 con 0x7f4a2c003a00 2024-05-11T12:13:36.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.203+0000 7f4a47e00640 1 -- 172.21.15.167:0/3274570571 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a480694b0 con 0x7f4a2c003a00 2024-05-11T12:13:36.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.204+0000 7f4a3ea00640 1 -- 172.21.15.167:0/3274570571 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4a38004a20 con 0x7f4a2c003a00 2024-05-11T12:13:36.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.205+0000 7f4a3ea00640 1 --2- 172.21.15.167:0/3274570571 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4a28076300 0x7f4a280787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:36.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.205+0000 7f4a3ea00640 1 -- 172.21.15.167:0/3274570571 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7f4a380b7390 con 0x7f4a2c003a00 2024-05-11T12:13:36.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.205+0000 7f4a46000640 1 --2- 172.21.15.167:0/3274570571 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4a28076300 0x7f4a280787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:36.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.206+0000 7f4a46000640 1 --2- 172.21.15.167:0/3274570571 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4a28076300 0x7f4a280787c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4a34006e80 tx=0x7f4a34006dd0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:36.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.209+0000 7f4a3ea00640 1 -- 172.21.15.167:0/3274570571 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4a3804b020 con 0x7f4a2c003a00 2024-05-11T12:13:36.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.405+0000 7f4a47e00640 1 -- 172.21.15.167:0/3274570571 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4a480b5ee0 con 0x7f4a2c003a00 2024-05-11T12:13:36.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.407+0000 7f4a3ea00640 1 -- 172.21.15.167:0/3274570571 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+17006 (secure 0 0 0) 0x7f4a380016b0 con 0x7f4a2c003a00 2024-05-11T12:13:36.407 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:13:36.407 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":74,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:13:34.993990+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:13:32.903032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-11T12:13:33.980753+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:13:36.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.410+0000 7f4a47e00640 1 -- 172.21.15.167:0/3274570571 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4a28076300 msgr2=0x7f4a280787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:36.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.410+0000 7f4a47e00640 1 --2- 172.21.15.167:0/3274570571 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4a28076300 0x7f4a280787c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4a34006e80 tx=0x7f4a34006dd0 comp rx=0 tx=0).stop 2024-05-11T12:13:36.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.410+0000 7f4a47e00640 1 -- 172.21.15.167:0/3274570571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a2c003a00 msgr2=0x7f4a4815d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:36.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.410+0000 7f4a47e00640 1 --2- 172.21.15.167:0/3274570571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a2c003a00 0x7f4a4815d8e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f4a3800c240 tx=0x7f4a38001a30 comp rx=0 tx=0).stop 2024-05-11T12:13:36.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.410+0000 7f4a47e00640 1 -- 172.21.15.167:0/3274570571 shutdown_connections 2024-05-11T12:13:36.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.410+0000 7f4a47e00640 1 --2- 172.21.15.167:0/3274570571 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4a28076300 0x7f4a280787c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:36.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.410+0000 7f4a47e00640 1 --2- 172.21.15.167:0/3274570571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a48156c40 0x7f4a481570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:36.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.410+0000 7f4a47e00640 1 --2- 172.21.15.167:0/3274570571 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a480cdf30 0x7f4a480ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:36.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.410+0000 7f4a47e00640 1 --2- 172.21.15.167:0/3274570571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a2c003a00 0x7f4a4815d8e0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:36.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.410+0000 7f4a47e00640 1 -- 172.21.15.167:0/3274570571 >> 172.21.15.167:0/3274570571 conn(0x7f4a480d1a40 msgr2=0x7f4a480b8e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:36.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.410+0000 7f4a47e00640 1 -- 172.21.15.167:0/3274570571 shutdown_connections 2024-05-11T12:13:36.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.410+0000 7f4a47e00640 1 -- 172.21.15.167:0/3274570571 wait complete. 2024-05-11T12:13:36.427 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-11T12:13:36.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.625+0000 7fbbc3200640 1 Processor -- start 2024-05-11T12:13:36.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.625+0000 7fbbc3200640 1 -- start start 2024-05-11T12:13:36.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.627+0000 7fbbc3200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbbc40d4240 0x7fbbc40b4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:36.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.627+0000 7fbbc3200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbbc40b3ef0 0x7fbbc40d0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:36.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.627+0000 7fbbc3200640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbc40d1320 con 0x7fbbc41626e0 2024-05-11T12:13:36.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.627+0000 7fbbc3200640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fbbc40d1600 con 0x7fbbc40d4240 2024-05-11T12:13:36.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.627+0000 7fbbc3200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbbc40cf5a0 con 0x7fbbc40b3ef0 2024-05-11T12:13:36.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.627+0000 7fbbc1e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fbbc41626e0 0x7fbbc40d3b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:52462/0 (socket says 172.21.15.167:52462) 2024-05-11T12:13:36.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.627+0000 7fbbc1e00640 1 -- 172.21.15.167:0/1945793138 learned_addr learned my addr 172.21.15.167:0/1945793138 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:13:36.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.627+0000 7fbbbbe00640 1 --2- 172.21.15.167:0/1945793138 >> v2:172.21.15.167:3300/0 conn(0x7fbbc40d4240 0x7fbbc40b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:36.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.627+0000 7fbbc1400640 1 -- 172.21.15.167:0/1945793138 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1839132367 0 0) 0x7fbbc40d1320 con 0x7fbbc41626e0 2024-05-11T12:13:36.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.628+0000 7fbbc1400640 1 -- 172.21.15.167:0/1945793138 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbba8003520 con 0x7fbbc41626e0 2024-05-11T12:13:36.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.628+0000 7fbbc2800640 1 --2- 172.21.15.167:0/1945793138 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbbc40b3ef0 0x7fbbc40d0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:36.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.628+0000 7fbbbbe00640 1 -- 172.21.15.167:0/1945793138 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbbc40b3ef0 msgr2=0x7fbbc40d0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:36.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.628+0000 7fbbbbe00640 1 --2- 172.21.15.167:0/1945793138 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbbc40b3ef0 0x7fbbc40d0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:36.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.628+0000 7fbbbbe00640 1 -- 172.21.15.167:0/1945793138 >> v1:172.21.15.167:6789/0 conn(0x7fbbc41626e0 legacy=0x7fbbc40d3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:36.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.628+0000 7fbbbbe00640 1 -- 172.21.15.167:0/1945793138 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbc40cf880 con 0x7fbbc40d4240 2024-05-11T12:13:36.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.628+0000 7fbbc2800640 1 --2- 172.21.15.167:0/1945793138 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbbc40b3ef0 0x7fbbc40d0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:36.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.629+0000 7fbbbbe00640 1 --2- 172.21.15.167:0/1945793138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbc40d4240 0x7fbbc40b4440 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fbbac0097a0 tx=0x7fbbac02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ffe4580dd117cc2 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:36.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.629+0000 7fbbc1400640 1 -- 172.21.15.167:0/1945793138 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac037070 con 0x7fbbc40d4240 2024-05-11T12:13:36.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.629+0000 7fbbc1400640 1 -- 172.21.15.167:0/1945793138 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbac032440 con 0x7fbbc40d4240 2024-05-11T12:13:36.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.629+0000 7fbbc1400640 1 -- 172.21.15.167:0/1945793138 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac032810 con 0x7fbbc40d4240 2024-05-11T12:13:36.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.630+0000 7fbbc3200640 1 -- 172.21.15.167:0/1945793138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbc40d4240 msgr2=0x7fbbc40b4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:36.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.630+0000 7fbbc3200640 1 --2- 172.21.15.167:0/1945793138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbc40d4240 0x7fbbc40b4440 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fbbac0097a0 tx=0x7fbbac02a180 comp rx=0 tx=0).stop 2024-05-11T12:13:36.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.630+0000 7fbbc3200640 1 -- 172.21.15.167:0/1945793138 shutdown_connections 2024-05-11T12:13:36.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.630+0000 7fbbc3200640 1 --2- 172.21.15.167:0/1945793138 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbbc40b3ef0 0x7fbbc40d0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:36.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.630+0000 7fbbc3200640 1 --2- 172.21.15.167:0/1945793138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbc40d4240 0x7fbbc40b4440 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:36.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.630+0000 7fbbc3200640 1 -- 172.21.15.167:0/1945793138 >> 172.21.15.167:0/1945793138 conn(0x7fbbc40b6a40 msgr2=0x7fbbc40b6e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:36.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.630+0000 7fbbc3200640 1 -- 172.21.15.167:0/1945793138 shutdown_connections 2024-05-11T12:13:36.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.630+0000 7fbbc3200640 1 -- 172.21.15.167:0/1945793138 wait complete. 2024-05-11T12:13:36.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.632+0000 7fbbc3200640 1 Processor -- start 2024-05-11T12:13:36.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.632+0000 7fbbc3200640 1 -- start start 2024-05-11T12:13:36.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.633+0000 7fbbc3200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbc40b3ef0 0x7fbbc40cc020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:36.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.633+0000 7fbbc3200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbc40d4240 0x7fbbc414f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:36.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.633+0000 7fbbc3200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbbc41626e0 0x7fbbc415c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:36.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.633+0000 7fbbc3200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc406e220 con 0x7fbbc40d4240 2024-05-11T12:13:36.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.633+0000 7fbbc3200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc4162ac0 con 0x7fbbc40b3ef0 2024-05-11T12:13:36.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.633+0000 7fbbc3200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbbc40cc700 con 0x7fbbc41626e0 2024-05-11T12:13:36.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.633+0000 7fbbbbe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbc40d4240 0x7fbbc414f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:36.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.633+0000 7fbbbbe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbc40d4240 0x7fbbc414f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:48016/0 (socket says 172.21.15.167:48016) 2024-05-11T12:13:36.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.633+0000 7fbbbbe00640 1 -- 172.21.15.167:0/3118498675 learned_addr learned my addr 172.21.15.167:0/3118498675 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:36.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.633+0000 7fbbc2800640 1 --2- 172.21.15.167:0/3118498675 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbbc41626e0 0x7fbbc415c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:36.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.634+0000 7fbbc1e00640 1 --2- 172.21.15.167:0/3118498675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbc40b3ef0 0x7fbbc40cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:36.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.634+0000 7fbbbbe00640 1 -- 172.21.15.167:0/3118498675 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbbc41626e0 msgr2=0x7fbbc415c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:36.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.634+0000 7fbbbbe00640 1 --2- 172.21.15.167:0/3118498675 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbbc41626e0 0x7fbbc415c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:36.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.634+0000 7fbbbbe00640 1 -- 172.21.15.167:0/3118498675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbc40b3ef0 msgr2=0x7fbbc40cc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:36.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.634+0000 7fbbbbe00640 1 --2- 172.21.15.167:0/3118498675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbc40b3ef0 0x7fbbc40cc020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:36.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.634+0000 7fbbbbe00640 1 -- 172.21.15.167:0/3118498675 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbc40cc9e0 con 0x7fbbc40d4240 2024-05-11T12:13:36.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.634+0000 7fbbc1e00640 1 --2- 172.21.15.167:0/3118498675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbc40b3ef0 0x7fbbc40cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:36.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.634+0000 7fbbbbe00640 1 --2- 172.21.15.167:0/3118498675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbc40d4240 0x7fbbc414f8e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fbbac035040 tx=0x7fbbac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:36.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.634+0000 7fbbba000640 1 -- 172.21.15.167:0/3118498675 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac037040 con 0x7fbbc40d4240 2024-05-11T12:13:36.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.634+0000 7fbbba000640 1 -- 172.21.15.167:0/3118498675 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbac030500 con 0x7fbbc40d4240 2024-05-11T12:13:36.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.634+0000 7fbbc3200640 1 -- 172.21.15.167:0/3118498675 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbc40cccc0 con 0x7fbbc40d4240 2024-05-11T12:13:36.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.634+0000 7fbbba000640 1 -- 172.21.15.167:0/3118498675 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac030910 con 0x7fbbc40d4240 2024-05-11T12:13:36.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.634+0000 7fbbc3200640 1 -- 172.21.15.167:0/3118498675 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbc414ff60 con 0x7fbbc40d4240 2024-05-11T12:13:36.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.636+0000 7fbbc3200640 1 -- 172.21.15.167:0/3118498675 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbc4069bc0 con 0x7fbbc40d4240 2024-05-11T12:13:36.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.637+0000 7fbbba000640 1 -- 172.21.15.167:0/3118498675 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbbac044020 con 0x7fbbc40d4240 2024-05-11T12:13:36.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.638+0000 7fbbba000640 1 --2- 172.21.15.167:0/3118498675 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbba0076300 0x7fbba00787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:36.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.638+0000 7fbbba000640 1 -- 172.21.15.167:0/3118498675 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7fbbac0b7a30 con 0x7fbbc40d4240 2024-05-11T12:13:36.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.638+0000 7fbbc1e00640 1 --2- 172.21.15.167:0/3118498675 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbba0076300 0x7fbba00787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:36.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.639+0000 7fbbc1e00640 1 --2- 172.21.15.167:0/3118498675 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbba0076300 0x7fbba00787c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4001240 tx=0x7fbbb4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:36.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.642+0000 7fbbba000640 1 -- 172.21.15.167:0/3118498675 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fbbac0811f0 con 0x7fbbc40d4240 2024-05-11T12:13:36.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:36.869+0000 7fbbc3200640 1 -- 172.21.15.167:0/3118498675 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fbbc40ba700 con 0x7fbbc40d4240 2024-05-11T12:13:37.388 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T12:13:37.396 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T12:13:37.497 INFO:tasks.ceph.mds.a.smithi167.stderr:2024-05-11T12:13:37.497+0000 7f497e000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-11T12:13:37.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.581+0000 7fbbba000640 1 -- 172.21.15.167:0/3118498675 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v43) ==== 118+0+0 (secure 0 0 0) 0x7fbbac080af0 con 0x7fbbc40d4240 2024-05-11T12:13:37.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.584+0000 7fbbc3200640 1 -- 172.21.15.167:0/3118498675 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbba0076300 msgr2=0x7fbba00787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:37.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.584+0000 7fbbc3200640 1 --2- 172.21.15.167:0/3118498675 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbba0076300 0x7fbba00787c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4001240 tx=0x7fbbb4009040 comp rx=0 tx=0).stop 2024-05-11T12:13:37.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.585+0000 7fbbc3200640 1 -- 172.21.15.167:0/3118498675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbc40d4240 msgr2=0x7fbbc414f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:37.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.585+0000 7fbbc3200640 1 --2- 172.21.15.167:0/3118498675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbc40d4240 0x7fbbc414f8e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fbbac035040 tx=0x7fbbac0046c0 comp rx=0 tx=0).stop 2024-05-11T12:13:37.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.585+0000 7fbbc3200640 1 -- 172.21.15.167:0/3118498675 shutdown_connections 2024-05-11T12:13:37.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.585+0000 7fbbc3200640 1 --2- 172.21.15.167:0/3118498675 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbbc41626e0 0x7fbbc415c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:37.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.585+0000 7fbbc3200640 1 --2- 172.21.15.167:0/3118498675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbc40d4240 0x7fbbc414f8e0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:37.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.585+0000 7fbbc3200640 1 --2- 172.21.15.167:0/3118498675 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbba0076300 0x7fbba00787c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:37.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.585+0000 7fbbc3200640 1 --2- 172.21.15.167:0/3118498675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbc40b3ef0 0x7fbbc40cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:37.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.585+0000 7fbbc3200640 1 -- 172.21.15.167:0/3118498675 >> 172.21.15.167:0/3118498675 conn(0x7fbbc40b6a40 msgr2=0x7fbbc40b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:37.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.585+0000 7fbbc3200640 1 -- 172.21.15.167:0/3118498675 shutdown_connections 2024-05-11T12:13:37.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.585+0000 7fbbc3200640 1 -- 172.21.15.167:0/3118498675 wait complete. 2024-05-11T12:13:37.606 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-11T12:13:37.606 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-11T12:13:37.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.809+0000 7fa6df800640 1 Processor -- start 2024-05-11T12:13:37.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.809+0000 7fa6df800640 1 -- start start 2024-05-11T12:13:37.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.811+0000 7fa6df800640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa6d80ae820 0x7fa6d80afe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:37.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.811+0000 7fa6df800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa6d80ceb00 0x7fa6d80cef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:37.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.811+0000 7fa6df800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6d80aed90 con 0x7fa6d8145a00 2024-05-11T12:13:37.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.811+0000 7fa6df800640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fa6d80af9a0 con 0x7fa6d80ae820 2024-05-11T12:13:37.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.811+0000 7fa6df800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fa6d80af070 con 0x7fa6d80ceb00 2024-05-11T12:13:37.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.811+0000 7fa6d7e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa6d80ae820 0x7fa6d80afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:37.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.811+0000 7fa6d7e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa6d80ae820 0x7fa6d80afe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:48020/0 (socket says 172.21.15.167:48020) 2024-05-11T12:13:37.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.812+0000 7fa6d7e00640 1 -- 172.21.15.167:0/1755870551 learned_addr learned my addr 172.21.15.167:0/1755870551 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:37.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.812+0000 7fa6d7e00640 1 -- 172.21.15.167:0/1755870551 >> v2:172.21.15.204:3300/0 conn(0x7fa6d80ceb00 msgr2=0x7fa6d80cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:37.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.812+0000 7fa6dd800640 1 --2- 172.21.15.167:0/1755870551 >> v2:172.21.15.204:3300/0 conn(0x7fa6d80ceb00 0x7fa6d80cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:37.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.812+0000 7fa6d7e00640 1 --2- 172.21.15.167:0/1755870551 >> v2:172.21.15.204:3300/0 conn(0x7fa6d80ceb00 0x7fa6d80cef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:37.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.812+0000 7fa6d7e00640 1 -- 172.21.15.167:0/1755870551 >> v1:172.21.15.204:6789/0 conn(0x7fa6d8145a00 legacy=0x7fa6d806f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:37.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.812+0000 7fa6d7e00640 1 -- 172.21.15.167:0/1755870551 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6d80af350 con 0x7fa6d80ae820 2024-05-11T12:13:37.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.813+0000 7fa6d7e00640 1 --2- 172.21.15.167:0/1755870551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6d80ae820 0x7fa6d80afe30 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa6d80b06d0 tx=0x7fa6c402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=741bd2bfcce4ead8 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:37.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.813+0000 7fa6d7400640 1 -- 172.21.15.167:0/1755870551 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4037070 con 0x7fa6d80ae820 2024-05-11T12:13:37.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.813+0000 7fa6d7400640 1 -- 172.21.15.167:0/1755870551 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6c4032440 con 0x7fa6d80ae820 2024-05-11T12:13:37.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.813+0000 7fa6d7400640 1 -- 172.21.15.167:0/1755870551 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4032810 con 0x7fa6d80ae820 2024-05-11T12:13:37.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.814+0000 7fa6df800640 1 -- 172.21.15.167:0/1755870551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6d80ae820 msgr2=0x7fa6d80afe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:37.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.814+0000 7fa6df800640 1 --2- 172.21.15.167:0/1755870551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6d80ae820 0x7fa6d80afe30 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa6d80b06d0 tx=0x7fa6c402a140 comp rx=0 tx=0).stop 2024-05-11T12:13:37.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.814+0000 7fa6df800640 1 -- 172.21.15.167:0/1755870551 shutdown_connections 2024-05-11T12:13:37.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.814+0000 7fa6df800640 1 --2- 172.21.15.167:0/1755870551 >> v2:172.21.15.204:3300/0 conn(0x7fa6d80ceb00 0x7fa6d80cef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:37.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.814+0000 7fa6df800640 1 --2- 172.21.15.167:0/1755870551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6d80ae820 0x7fa6d80afe30 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:37.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.814+0000 7fa6df800640 1 -- 172.21.15.167:0/1755870551 >> 172.21.15.167:0/1755870551 conn(0x7fa6d806c450 msgr2=0x7fa6d806c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:37.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.814+0000 7fa6df800640 1 -- 172.21.15.167:0/1755870551 shutdown_connections 2024-05-11T12:13:37.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.814+0000 7fa6df800640 1 -- 172.21.15.167:0/1755870551 wait complete. 2024-05-11T12:13:37.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.816+0000 7fa6df800640 1 Processor -- start 2024-05-11T12:13:37.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.816+0000 7fa6df800640 1 -- start start 2024-05-11T12:13:37.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.817+0000 7fa6df800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6d80ae820 0x7fa6d81411b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:37.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.817+0000 7fa6df800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6d80ceb00 0x7fa6d81f81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:37.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.817+0000 7fa6df800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6d8145a00 0x7fa6d81f8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:37.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.817+0000 7fa6df800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d80723b0 con 0x7fa6d80ae820 2024-05-11T12:13:37.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.817+0000 7fa6df800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d8072230 con 0x7fa6d8145a00 2024-05-11T12:13:37.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.817+0000 7fa6df800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa6d8072530 con 0x7fa6d80ceb00 2024-05-11T12:13:37.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.817+0000 7fa6dce00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6d80ae820 0x7fa6d81411b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:37.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.817+0000 7fa6dce00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6d80ae820 0x7fa6d81411b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:48028/0 (socket says 172.21.15.167:48028) 2024-05-11T12:13:37.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.817+0000 7fa6dce00640 1 -- 172.21.15.167:0/3530718477 learned_addr learned my addr 172.21.15.167:0/3530718477 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:37.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.818+0000 7fa6d7e00640 1 --2- 172.21.15.167:0/3530718477 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6d80ceb00 0x7fa6d81f81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:37.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.818+0000 7fa6dce00640 1 -- 172.21.15.167:0/3530718477 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6d80ceb00 msgr2=0x7fa6d81f81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:37.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.818+0000 7fa6dce00640 1 --2- 172.21.15.167:0/3530718477 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6d80ceb00 0x7fa6d81f81d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:37.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.818+0000 7fa6dce00640 1 -- 172.21.15.167:0/3530718477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6d8145a00 msgr2=0x7fa6d81f8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:37.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.818+0000 7fa6dd800640 1 --2- 172.21.15.167:0/3530718477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6d8145a00 0x7fa6d81f8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:37.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.818+0000 7fa6dce00640 1 --2- 172.21.15.167:0/3530718477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6d8145a00 0x7fa6d81f8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:37.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.818+0000 7fa6dce00640 1 -- 172.21.15.167:0/3530718477 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6d8142a60 con 0x7fa6d80ae820 2024-05-11T12:13:37.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.818+0000 7fa6dce00640 1 --2- 172.21.15.167:0/3530718477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6d80ae820 0x7fa6d81411b0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc00b3e0 tx=0x7fa6cc00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:37.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.818+0000 7fa6d4c00640 1 -- 172.21.15.167:0/3530718477 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc015020 con 0x7fa6d80ae820 2024-05-11T12:13:37.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.818+0000 7fa6d4c00640 1 -- 172.21.15.167:0/3530718477 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6cc011850 con 0x7fa6d80ae820 2024-05-11T12:13:37.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.818+0000 7fa6d4c00640 1 -- 172.21.15.167:0/3530718477 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc011c40 con 0x7fa6d80ae820 2024-05-11T12:13:37.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.818+0000 7fa6df800640 1 -- 172.21.15.167:0/3530718477 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6d81416f0 con 0x7fa6d80ae820 2024-05-11T12:13:37.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.819+0000 7fa6df800640 1 -- 172.21.15.167:0/3530718477 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6d8141ca0 con 0x7fa6d80ae820 2024-05-11T12:13:37.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.820+0000 7fa6df800640 1 -- 172.21.15.167:0/3530718477 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6d8069520 con 0x7fa6d80ae820 2024-05-11T12:13:37.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.821+0000 7fa6d4c00640 1 -- 172.21.15.167:0/3530718477 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa6cc018020 con 0x7fa6d80ae820 2024-05-11T12:13:37.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.822+0000 7fa6d4c00640 1 --2- 172.21.15.167:0/3530718477 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa6b80762d0 0x7fa6b8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:37.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.822+0000 7fa6d4c00640 1 -- 172.21.15.167:0/3530718477 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7fa6cc097690 con 0x7fa6d80ae820 2024-05-11T12:13:37.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.823+0000 7fa6d7e00640 1 --2- 172.21.15.167:0/3530718477 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa6b80762d0 0x7fa6b8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:37.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.823+0000 7fa6d7e00640 1 --2- 172.21.15.167:0/3530718477 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa6b80762d0 0x7fa6b8078790 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4034000 tx=0x7fa6c402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:37.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:37.827+0000 7fa6d4c00640 1 -- 172.21.15.167:0/3530718477 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa6cc01f020 con 0x7fa6d80ae820 2024-05-11T12:13:38.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.071+0000 7fa6df800640 1 -- 172.21.15.167:0/3530718477 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fa6d8075bb0 con 0x7fa6d80ae820 2024-05-11T12:13:38.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.591+0000 7fa6d4c00640 1 -- 172.21.15.167:0/3530718477 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v44) ==== 122+0+0 (secure 0 0 0) 0x7fa6cc060e60 con 0x7fa6d80ae820 2024-05-11T12:13:38.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.596+0000 7fa6df800640 1 -- 172.21.15.167:0/3530718477 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa6b80762d0 msgr2=0x7fa6b8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:38.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.596+0000 7fa6df800640 1 --2- 172.21.15.167:0/3530718477 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa6b80762d0 0x7fa6b8078790 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4034000 tx=0x7fa6c402e000 comp rx=0 tx=0).stop 2024-05-11T12:13:38.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.596+0000 7fa6df800640 1 -- 172.21.15.167:0/3530718477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6d80ae820 msgr2=0x7fa6d81411b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:38.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.596+0000 7fa6df800640 1 --2- 172.21.15.167:0/3530718477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6d80ae820 0x7fa6d81411b0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc00b3e0 tx=0x7fa6cc00b8a0 comp rx=0 tx=0).stop 2024-05-11T12:13:38.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.596+0000 7fa6df800640 1 -- 172.21.15.167:0/3530718477 shutdown_connections 2024-05-11T12:13:38.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.596+0000 7fa6df800640 1 --2- 172.21.15.167:0/3530718477 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa6b80762d0 0x7fa6b8078790 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:38.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.596+0000 7fa6df800640 1 --2- 172.21.15.167:0/3530718477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6d8145a00 0x7fa6d81f8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:38.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.596+0000 7fa6df800640 1 --2- 172.21.15.167:0/3530718477 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6d80ceb00 0x7fa6d81f81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:38.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.596+0000 7fa6df800640 1 --2- 172.21.15.167:0/3530718477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6d80ae820 0x7fa6d81411b0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:38.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.596+0000 7fa6df800640 1 -- 172.21.15.167:0/3530718477 >> 172.21.15.167:0/3530718477 conn(0x7fa6d806c450 msgr2=0x7fa6d80b3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:38.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.597+0000 7fa6df800640 1 -- 172.21.15.167:0/3530718477 shutdown_connections 2024-05-11T12:13:38.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.597+0000 7fa6df800640 1 -- 172.21.15.167:0/3530718477 wait complete. 2024-05-11T12:13:38.618 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-11T12:13:38.619 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:13:38.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.823+0000 7f45da400640 1 Processor -- start 2024-05-11T12:13:38.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.823+0000 7f45da400640 1 -- start start 2024-05-11T12:13:38.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.825+0000 7f45da400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f45d4157750 0x7f45d415d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:38.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.825+0000 7f45da400640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45d415dc80 con 0x7f45d406d460 2024-05-11T12:13:38.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.825+0000 7f45da400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45d415df30 con 0x7f45d40b3ea0 2024-05-11T12:13:38.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.825+0000 7f45da400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f45d4161000 con 0x7f45d4157750 2024-05-11T12:13:38.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.825+0000 7f45d3e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f45d406d460 0x7f45d40cf1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:42332/0 (socket says 172.21.15.167:42332) 2024-05-11T12:13:38.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.825+0000 7f45d3e00640 1 -- 172.21.15.167:0/481666072 learned_addr learned my addr 172.21.15.167:0/481666072 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:13:38.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.826+0000 7f45d3400640 1 --2- 172.21.15.167:0/481666072 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f45d4157750 0x7f45d415d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:38.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.826+0000 7f45d2000640 1 -- 172.21.15.167:0/481666072 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4270629499 0 0) 0x7f45d415dc80 con 0x7f45d406d460 2024-05-11T12:13:38.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.826+0000 7f45d2000640 1 -- 172.21.15.167:0/481666072 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f45b8003520 con 0x7f45d406d460 2024-05-11T12:13:38.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.826+0000 7f45d2000640 1 -- 172.21.15.167:0/481666072 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1085184867 0 0) 0x7f45b8003520 con 0x7f45d406d460 2024-05-11T12:13:38.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.826+0000 7f45d2000640 1 -- 172.21.15.167:0/481666072 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f45d4157750 msgr2=0x7f45d415d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:38.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.826+0000 7f45d2000640 1 --2- 172.21.15.167:0/481666072 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f45d4157750 0x7f45d415d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:38.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.826+0000 7f45d2000640 1 -- 172.21.15.167:0/481666072 >> v1:172.21.15.204:6789/0 conn(0x7f45d40b3ea0 legacy=0x7f45d4161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:38.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.826+0000 7f45d2000640 1 -- 172.21.15.167:0/481666072 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d415e210 con 0x7f45d406d460 2024-05-11T12:13:38.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.826+0000 7f45d2000640 1 -- 172.21.15.167:0/481666072 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f45cc001a70 con 0x7f45d406d460 2024-05-11T12:13:38.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.826+0000 7f45d2000640 1 -- 172.21.15.167:0/481666072 >> v1:172.21.15.167:6789/0 conn(0x7f45d406d460 legacy=0x7f45d40cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:38.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.826+0000 7f45d2000640 1 --2- 172.21.15.167:0/481666072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45b8003a80 0x7f45b8003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:38.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.826+0000 7f45d2000640 1 --2- 172.21.15.167:0/481666072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45b8004800 0x7f45b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:38.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.826+0000 7f45d2000640 1 --2- 172.21.15.167:0/481666072 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f45d4157750 0x7f45b800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:38.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.826+0000 7f45d2000640 1 -- 172.21.15.167:0/481666072 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f45d4161000 con 0x7f45b8003a80 2024-05-11T12:13:38.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.826+0000 7f45d2000640 1 -- 172.21.15.167:0/481666072 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f45b80054f0 con 0x7f45b8004800 2024-05-11T12:13:38.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.826+0000 7f45d2000640 1 -- 172.21.15.167:0/481666072 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f45b800bf00 con 0x7f45d4157750 2024-05-11T12:13:38.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.827+0000 7f45d3400640 1 --2- 172.21.15.167:0/481666072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45b8003a80 0x7f45b8003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:38.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.827+0000 7f45d2a00640 1 --2- 172.21.15.167:0/481666072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45b8004800 0x7f45b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:38.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.827+0000 7f45d3e00640 1 --2- 172.21.15.167:0/481666072 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f45d4157750 0x7f45b800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:38.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.827+0000 7f45d3400640 1 -- 172.21.15.167:0/481666072 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f45d4157750 msgr2=0x7f45b800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:38.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.827+0000 7f45d3400640 1 --2- 172.21.15.167:0/481666072 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f45d4157750 0x7f45b800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:38.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.827+0000 7f45d3400640 1 -- 172.21.15.167:0/481666072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45b8004800 msgr2=0x7f45b8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:38.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.827+0000 7f45d3400640 1 --2- 172.21.15.167:0/481666072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45b8004800 0x7f45b8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:38.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.827+0000 7f45d3400640 1 -- 172.21.15.167:0/481666072 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45b8003520 con 0x7f45b8003a80 2024-05-11T12:13:38.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.828+0000 7f45d3400640 1 --2- 172.21.15.167:0/481666072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45b8003a80 0x7f45b8003ea0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f45bc00ba30 tx=0x7f45b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d7553e70dda174a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:38.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.829+0000 7f45d2000640 1 -- 172.21.15.167:0/481666072 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45bc030070 con 0x7f45b8003a80 2024-05-11T12:13:38.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.829+0000 7f45d2000640 1 -- 172.21.15.167:0/481666072 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45bc02ade0 con 0x7f45b8003a80 2024-05-11T12:13:38.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.829+0000 7f45d2000640 1 -- 172.21.15.167:0/481666072 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45bc00b600 con 0x7f45b8003a80 2024-05-11T12:13:38.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.829+0000 7f45da400640 1 -- 172.21.15.167:0/481666072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45b8003a80 msgr2=0x7f45b8003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:38.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.829+0000 7f45da400640 1 --2- 172.21.15.167:0/481666072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45b8003a80 0x7f45b8003ea0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f45bc00ba30 tx=0x7f45b8005830 comp rx=0 tx=0).stop 2024-05-11T12:13:38.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.829+0000 7f45d3e00640 1 -- 172.21.15.167:0/481666072 reap_dead start 2024-05-11T12:13:38.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.830+0000 7f45da400640 1 -- 172.21.15.167:0/481666072 shutdown_connections 2024-05-11T12:13:38.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.830+0000 7f45da400640 1 -- 172.21.15.167:0/481666072 >> 172.21.15.167:0/481666072 conn(0x7f45d40cd8f0 msgr2=0x7f45d40cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:38.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.830+0000 7f45da400640 1 -- 172.21.15.167:0/481666072 shutdown_connections 2024-05-11T12:13:38.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.830+0000 7f45da400640 1 -- 172.21.15.167:0/481666072 wait complete. 2024-05-11T12:13:38.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.831+0000 7f45da400640 1 Processor -- start 2024-05-11T12:13:38.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.832+0000 7f45da400640 1 -- start start 2024-05-11T12:13:38.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.832+0000 7f45da400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f45b8003a80 0x7f45d41421f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:38.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.833+0000 7f45da400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45d40b1930 0x7f45d40b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:38.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.833+0000 7f45da400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45d40733e0 0x7f45d4073860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:38.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.833+0000 7f45da400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f45d4161ba0 con 0x7f45d40733e0 2024-05-11T12:13:38.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.833+0000 7f45da400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f45d4157b30 con 0x7f45d40b1930 2024-05-11T12:13:38.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.833+0000 7f45da400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f45d40cf8d0 con 0x7f45b8003a80 2024-05-11T12:13:38.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.833+0000 7f45d3e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45d40733e0 0x7f45d4073860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:38.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.833+0000 7f45d3e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45d40733e0 0x7f45d4073860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41200/0 (socket says 172.21.15.167:41200) 2024-05-11T12:13:38.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.833+0000 7f45d3e00640 1 -- 172.21.15.167:0/2651581433 learned_addr learned my addr 172.21.15.167:0/2651581433 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:38.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.833+0000 7f45d3400640 1 --2- 172.21.15.167:0/2651581433 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f45b8003a80 0x7f45d41421f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:38.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.833+0000 7f45d2a00640 1 --2- 172.21.15.167:0/2651581433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45d40b1930 0x7f45d40b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:38.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.833+0000 7f45d3e00640 1 -- 172.21.15.167:0/2651581433 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f45b8003a80 msgr2=0x7f45d41421f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:38.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.833+0000 7f45d3e00640 1 --2- 172.21.15.167:0/2651581433 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f45b8003a80 0x7f45d41421f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:38.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.833+0000 7f45d3e00640 1 -- 172.21.15.167:0/2651581433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45d40b1930 msgr2=0x7f45d40b1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:38.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.833+0000 7f45d3e00640 1 --2- 172.21.15.167:0/2651581433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45d40b1930 0x7f45d40b1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:38.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.833+0000 7f45d3e00640 1 -- 172.21.15.167:0/2651581433 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d41441c0 con 0x7f45d40733e0 2024-05-11T12:13:38.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.834+0000 7f45d3e00640 1 --2- 172.21.15.167:0/2651581433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45d40733e0 0x7f45d4073860 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f45d40ce840 tx=0x7f45cc00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:38.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.834+0000 7f45cb400640 1 -- 172.21.15.167:0/2651581433 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45cc00de90 con 0x7f45d40733e0 2024-05-11T12:13:38.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.834+0000 7f45cb400640 1 -- 172.21.15.167:0/2651581433 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45cc00bb80 con 0x7f45d40733e0 2024-05-11T12:13:38.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.834+0000 7f45cb400640 1 -- 172.21.15.167:0/2651581433 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45cc007070 con 0x7f45d40733e0 2024-05-11T12:13:38.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.834+0000 7f45da400640 1 -- 172.21.15.167:0/2651581433 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45d41444a0 con 0x7f45d40733e0 2024-05-11T12:13:38.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.834+0000 7f45da400640 1 -- 172.21.15.167:0/2651581433 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45d4144a60 con 0x7f45d40733e0 2024-05-11T12:13:38.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.835+0000 7f45da400640 1 -- 172.21.15.167:0/2651581433 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45d4144c60 con 0x7f45d40733e0 2024-05-11T12:13:38.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.836+0000 7f45cb400640 1 -- 172.21.15.167:0/2651581433 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f45cc01b020 con 0x7f45d40733e0 2024-05-11T12:13:38.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.837+0000 7f45cb400640 1 --2- 172.21.15.167:0/2651581433 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f45b00764a0 0x7f45b0078960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:38.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.837+0000 7f45cb400640 1 -- 172.21.15.167:0/2651581433 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7f45cc097a70 con 0x7f45d40733e0 2024-05-11T12:13:38.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.838+0000 7f45d3400640 1 --2- 172.21.15.167:0/2651581433 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f45b00764a0 0x7f45b0078960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:38.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.838+0000 7f45d3400640 1 --2- 172.21.15.167:0/2651581433 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f45b00764a0 0x7f45b0078960 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f45bc00ba00 tx=0x7f45bc030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:38.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:38.842+0000 7f45cb400640 1 -- 172.21.15.167:0/2651581433 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f45d4144c60 con 0x7f45d40733e0 2024-05-11T12:13:39.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.087+0000 7f45da400640 1 -- 172.21.15.167:0/2651581433 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f45d4069590 con 0x7f45d40733e0 2024-05-11T12:13:39.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.088+0000 7f45cb400640 1 -- 172.21.15.167:0/2651581433 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5195 (secure 0 0 0) 0x7f45cc0611a0 con 0x7f45d40733e0 2024-05-11T12:13:39.088 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:13:39.088 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5020,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3075930607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3075930607},{"type":"v1","addr":"172.21.15.167:6836","nonce":3075930607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":5032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3056051531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3056051531},{"type":"v1","addr":"172.21.15.204:6834","nonce":3056051531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5035,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2900478909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":2900478909},{"type":"v1","addr":"172.21.15.204:6835","nonce":2900478909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:13:34.993949+0000","modified":"2024-05-11T12:13:38.072880+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5952},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5952":{"gid":5952,"name":"a","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/1306616214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1306616214},{"type":"v1","addr":"172.21.15.167:6837","nonce":1306616214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5952,"qdb_cluster":[5952]},"id":5}]} 2024-05-11T12:13:39.088 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 44 2024-05-11T12:13:39.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.091+0000 7f45da400640 1 -- 172.21.15.167:0/2651581433 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f45b00764a0 msgr2=0x7f45b0078960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:39.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.091+0000 7f45da400640 1 --2- 172.21.15.167:0/2651581433 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f45b00764a0 0x7f45b0078960 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f45bc00ba00 tx=0x7f45bc030000 comp rx=0 tx=0).stop 2024-05-11T12:13:39.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.091+0000 7f45da400640 1 -- 172.21.15.167:0/2651581433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45d40733e0 msgr2=0x7f45d4073860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:39.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.091+0000 7f45da400640 1 --2- 172.21.15.167:0/2651581433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45d40733e0 0x7f45d4073860 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f45d40ce840 tx=0x7f45cc00b4a0 comp rx=0 tx=0).stop 2024-05-11T12:13:39.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.091+0000 7f45da400640 1 -- 172.21.15.167:0/2651581433 shutdown_connections 2024-05-11T12:13:39.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.091+0000 7f45da400640 1 --2- 172.21.15.167:0/2651581433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45d40733e0 0x7f45d4073860 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:39.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.091+0000 7f45da400640 1 --2- 172.21.15.167:0/2651581433 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f45b00764a0 0x7f45b0078960 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:39.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.091+0000 7f45da400640 1 --2- 172.21.15.167:0/2651581433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45d40b1930 0x7f45d40b1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:39.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.091+0000 7f45da400640 1 --2- 172.21.15.167:0/2651581433 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f45b8003a80 0x7f45d41421f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:39.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.091+0000 7f45da400640 1 -- 172.21.15.167:0/2651581433 >> 172.21.15.167:0/2651581433 conn(0x7f45d40cd8f0 msgr2=0x7f45d415cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:39.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.091+0000 7f45da400640 1 -- 172.21.15.167:0/2651581433 shutdown_connections 2024-05-11T12:13:39.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.091+0000 7f45da400640 1 -- 172.21.15.167:0/2651581433 wait complete. 2024-05-11T12:13:39.108 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:13:39.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.308+0000 7f253ca00640 1 Processor -- start 2024-05-11T12:13:39.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.308+0000 7f253ca00640 1 -- start start 2024-05-11T12:13:39.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.309+0000 7f253ca00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f253815f1a0 0x7f25380aff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:39.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.309+0000 7f253ca00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f25380736a0 0x7f25380b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:39.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.309+0000 7f253ca00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25380b0c20 con 0x7f25380b0470 2024-05-11T12:13:39.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.309+0000 7f253ca00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f25380b0940 con 0x7f25380736a0 2024-05-11T12:13:39.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.309+0000 7f253ca00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f253815f9d0 con 0x7f253815f1a0 2024-05-11T12:13:39.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.310+0000 7f2537200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f25380736a0 0x7f25380b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:39.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.310+0000 7f252fe00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f25380b0470 0x7f2538072f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:42340/0 (socket says 172.21.15.167:42340) 2024-05-11T12:13:39.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.310+0000 7f252fe00640 1 -- 172.21.15.167:0/889041663 learned_addr learned my addr 172.21.15.167:0/889041663 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:13:39.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.310+0000 7f2536800640 1 --2- 172.21.15.167:0/889041663 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f253815f1a0 0x7f25380aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:39.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.310+0000 7f2537200640 1 -- 172.21.15.167:0/889041663 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f253815f1a0 msgr2=0x7f25380aff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:39.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.310+0000 7f2537200640 1 --2- 172.21.15.167:0/889041663 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f253815f1a0 0x7f25380aff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:39.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.310+0000 7f2537200640 1 -- 172.21.15.167:0/889041663 >> v1:172.21.15.167:6789/0 conn(0x7f25380b0470 legacy=0x7f2538072f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:39.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.310+0000 7f2537200640 1 -- 172.21.15.167:0/889041663 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f253815fcb0 con 0x7f25380736a0 2024-05-11T12:13:39.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.311+0000 7f2537200640 1 --2- 172.21.15.167:0/889041663 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25380736a0 0x7f25380b4d60 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f2530009d90 tx=0x7f253002a540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c9a1bad40b73ea6 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:39.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.312+0000 7f2535e00640 1 -- 172.21.15.167:0/889041663 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2530007740 con 0x7f25380736a0 2024-05-11T12:13:39.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.312+0000 7f2535e00640 1 -- 172.21.15.167:0/889041663 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25300078e0 con 0x7f25380736a0 2024-05-11T12:13:39.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.312+0000 7f2535e00640 1 -- 172.21.15.167:0/889041663 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2530007cb0 con 0x7f25380736a0 2024-05-11T12:13:39.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.312+0000 7f253ca00640 1 -- 172.21.15.167:0/889041663 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25380736a0 msgr2=0x7f25380b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:39.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.312+0000 7f253ca00640 1 --2- 172.21.15.167:0/889041663 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25380736a0 0x7f25380b4d60 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f2530009d90 tx=0x7f253002a540 comp rx=0 tx=0).stop 2024-05-11T12:13:39.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.312+0000 7f253ca00640 1 -- 172.21.15.167:0/889041663 shutdown_connections 2024-05-11T12:13:39.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.312+0000 7f253ca00640 1 --2- 172.21.15.167:0/889041663 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25380736a0 0x7f25380b4d60 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:39.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.312+0000 7f253ca00640 1 --2- 172.21.15.167:0/889041663 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f253815f1a0 0x7f25380aff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:39.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.312+0000 7f253ca00640 1 -- 172.21.15.167:0/889041663 >> 172.21.15.167:0/889041663 conn(0x7f25380b3a60 msgr2=0x7f25380b3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:39.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.312+0000 7f253ca00640 1 -- 172.21.15.167:0/889041663 shutdown_connections 2024-05-11T12:13:39.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.312+0000 7f253ca00640 1 -- 172.21.15.167:0/889041663 wait complete. 2024-05-11T12:13:39.313 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.314+0000 7f253ca00640 1 Processor -- start 2024-05-11T12:13:39.313 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.314+0000 7f253ca00640 1 -- start start 2024-05-11T12:13:39.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.315+0000 7f253ca00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25380736a0 0x7f2538155bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:39.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.315+0000 7f253ca00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25380b0470 0x7f253814cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:39.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.315+0000 7f253ca00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f253815f1a0 0x7f253814d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:39.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.315+0000 7f253ca00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f25380b6de0 con 0x7f25380736a0 2024-05-11T12:13:39.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.315+0000 7f253ca00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f25380b6c60 con 0x7f253815f1a0 2024-05-11T12:13:39.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.315+0000 7f253ca00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f253815f580 con 0x7f25380b0470 2024-05-11T12:13:39.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.315+0000 7f2536800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25380736a0 0x7f2538155bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:39.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.316+0000 7f2536800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25380736a0 0x7f2538155bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41210/0 (socket says 172.21.15.167:41210) 2024-05-11T12:13:39.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.316+0000 7f2536800640 1 -- 172.21.15.167:0/887250845 learned_addr learned my addr 172.21.15.167:0/887250845 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:39.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.316+0000 7f252fe00640 1 --2- 172.21.15.167:0/887250845 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25380b0470 0x7f253814cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:39.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.316+0000 7f2536800640 1 -- 172.21.15.167:0/887250845 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25380b0470 msgr2=0x7f253814cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:39.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.316+0000 7f2536800640 1 --2- 172.21.15.167:0/887250845 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25380b0470 0x7f253814cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:39.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.316+0000 7f2536800640 1 -- 172.21.15.167:0/887250845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f253815f1a0 msgr2=0x7f253814d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:39.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.316+0000 7f2536800640 1 --2- 172.21.15.167:0/887250845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f253815f1a0 0x7f253814d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:39.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.316+0000 7f2536800640 1 -- 172.21.15.167:0/887250845 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f253814da20 con 0x7f25380736a0 2024-05-11T12:13:39.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.316+0000 7f2536800640 1 --2- 172.21.15.167:0/887250845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25380736a0 0x7f2538155bf0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f252800d620 tx=0x7f252800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:39.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.317+0000 7f252ea00640 1 -- 172.21.15.167:0/887250845 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25280125f0 con 0x7f25380736a0 2024-05-11T12:13:39.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.317+0000 7f252ea00640 1 -- 172.21.15.167:0/887250845 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2528019070 con 0x7f25380736a0 2024-05-11T12:13:39.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.317+0000 7f252ea00640 1 -- 172.21.15.167:0/887250845 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25280129d0 con 0x7f25380736a0 2024-05-11T12:13:39.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.317+0000 7f253ca00640 1 -- 172.21.15.167:0/887250845 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2538161950 con 0x7f25380736a0 2024-05-11T12:13:39.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.317+0000 7f253ca00640 1 -- 172.21.15.167:0/887250845 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2538161f10 con 0x7f25380736a0 2024-05-11T12:13:39.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.318+0000 7f253ca00640 1 -- 172.21.15.167:0/887250845 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2538069bc0 con 0x7f25380736a0 2024-05-11T12:13:39.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.319+0000 7f252ea00640 1 -- 172.21.15.167:0/887250845 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2528012b70 con 0x7f25380736a0 2024-05-11T12:13:39.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.320+0000 7f252ea00640 1 --2- 172.21.15.167:0/887250845 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f25140762d0 0x7f2514078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:39.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.320+0000 7f252ea00640 1 -- 172.21.15.167:0/887250845 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7f2528097b50 con 0x7f25380736a0 2024-05-11T12:13:39.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.320+0000 7f252fe00640 1 --2- 172.21.15.167:0/887250845 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f25140762d0 0x7f2514078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:39.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.321+0000 7f252fe00640 1 --2- 172.21.15.167:0/887250845 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f25140762d0 0x7f2514078790 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f25380b3a30 tx=0x7f2520009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:39.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.324+0000 7f252ea00640 1 -- 172.21.15.167:0/887250845 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2528061310 con 0x7f25380736a0 2024-05-11T12:13:39.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.531+0000 7f253ca00640 1 -- 172.21.15.167:0/887250845 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f25380b63a0 con 0x7f25380736a0 2024-05-11T12:13:39.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.534+0000 7f252ea00640 1 -- 172.21.15.167:0/887250845 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+16993 (secure 0 0 0) 0x7f2528060c10 con 0x7f25380736a0 2024-05-11T12:13:39.534 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:13:39.534 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":75,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:13:36.554508+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:13:32.903032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-11T12:13:33.980753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:13:39.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.536+0000 7f253ca00640 1 -- 172.21.15.167:0/887250845 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f25140762d0 msgr2=0x7f2514078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:39.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.536+0000 7f253ca00640 1 --2- 172.21.15.167:0/887250845 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f25140762d0 0x7f2514078790 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f25380b3a30 tx=0x7f2520009040 comp rx=0 tx=0).stop 2024-05-11T12:13:39.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.536+0000 7f253ca00640 1 -- 172.21.15.167:0/887250845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25380736a0 msgr2=0x7f2538155bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:39.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.537+0000 7f253ca00640 1 --2- 172.21.15.167:0/887250845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25380736a0 0x7f2538155bf0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f252800d620 tx=0x7f252800dae0 comp rx=0 tx=0).stop 2024-05-11T12:13:39.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.537+0000 7f253ca00640 1 -- 172.21.15.167:0/887250845 shutdown_connections 2024-05-11T12:13:39.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.537+0000 7f253ca00640 1 --2- 172.21.15.167:0/887250845 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f25140762d0 0x7f2514078790 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:39.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.537+0000 7f253ca00640 1 --2- 172.21.15.167:0/887250845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f253815f1a0 0x7f253814d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:39.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.537+0000 7f253ca00640 1 --2- 172.21.15.167:0/887250845 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25380b0470 0x7f253814cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:39.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.537+0000 7f253ca00640 1 --2- 172.21.15.167:0/887250845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25380736a0 0x7f2538155bf0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:39.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.537+0000 7f253ca00640 1 -- 172.21.15.167:0/887250845 >> 172.21.15.167:0/887250845 conn(0x7f25380b3a60 msgr2=0x7f25380d1d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:39.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.537+0000 7f253ca00640 1 -- 172.21.15.167:0/887250845 shutdown_connections 2024-05-11T12:13:39.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.537+0000 7f253ca00640 1 -- 172.21.15.167:0/887250845 wait complete. 2024-05-11T12:13:39.554 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-11T12:13:39.554 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T12:13:39.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.756+0000 7f6dd3e00640 1 Processor -- start 2024-05-11T12:13:39.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.756+0000 7f6dd3e00640 1 -- start start 2024-05-11T12:13:39.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.758+0000 7f6dd3e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6dcc0b3ea0 0x7f6dcc161490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:39.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.758+0000 7f6dd3e00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6dcc15df30 con 0x7f6dcc157750 2024-05-11T12:13:39.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.758+0000 7f6dd3e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6dcc15e210 con 0x7f6dcc1619d0 2024-05-11T12:13:39.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.758+0000 7f6dd3e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f6dcc161000 con 0x7f6dcc0b3ea0 2024-05-11T12:13:39.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.758+0000 7f6dd1400640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f6dcc157750 0x7f6dcc15d740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:42350/0 (socket says 172.21.15.167:42350) 2024-05-11T12:13:39.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.758+0000 7f6dd1400640 1 -- 172.21.15.167:0/1324895002 learned_addr learned my addr 172.21.15.167:0/1324895002 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:13:39.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.758+0000 7f6dd0a00640 1 --2- 172.21.15.167:0/1324895002 >> v2:172.21.15.204:3300/0 conn(0x7f6dcc0b3ea0 0x7f6dcc161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:39.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.759+0000 7f6dcbe00640 1 -- 172.21.15.167:0/1324895002 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1055689255 0 0) 0x7f6dcc15df30 con 0x7f6dcc157750 2024-05-11T12:13:39.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.759+0000 7f6dcbe00640 1 -- 172.21.15.167:0/1324895002 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6db4003520 con 0x7f6dcc157750 2024-05-11T12:13:39.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.759+0000 7f6dcbe00640 1 -- 172.21.15.167:0/1324895002 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1711234179 0 0) 0x7f6db4003520 con 0x7f6dcc157750 2024-05-11T12:13:39.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.759+0000 7f6dcbe00640 1 -- 172.21.15.167:0/1324895002 >> v2:172.21.15.204:3300/0 conn(0x7f6dcc0b3ea0 msgr2=0x7f6dcc161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:39.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.759+0000 7f6dcbe00640 1 --2- 172.21.15.167:0/1324895002 >> v2:172.21.15.204:3300/0 conn(0x7f6dcc0b3ea0 0x7f6dcc161490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:39.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.759+0000 7f6dcbe00640 1 -- 172.21.15.167:0/1324895002 >> v1:172.21.15.204:6789/0 conn(0x7f6dcc1619d0 legacy=0x7f6dcc06d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:39.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.759+0000 7f6dcbe00640 1 -- 172.21.15.167:0/1324895002 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dcc15e4f0 con 0x7f6dcc157750 2024-05-11T12:13:39.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.759+0000 7f6dcbe00640 1 -- 172.21.15.167:0/1324895002 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f6dc00016f0 con 0x7f6dcc157750 2024-05-11T12:13:39.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.759+0000 7f6dcbe00640 1 -- 172.21.15.167:0/1324895002 >> v1:172.21.15.167:6789/0 conn(0x7f6dcc157750 legacy=0x7f6dcc15d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:39.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.759+0000 7f6dcbe00640 1 --2- 172.21.15.167:0/1324895002 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6db4003aa0 0x7f6db4003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:39.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.759+0000 7f6dcbe00640 1 --2- 172.21.15.167:0/1324895002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6db4004800 0x7f6db4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:39.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.759+0000 7f6dcbe00640 1 --2- 172.21.15.167:0/1324895002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6db4005210 0x7f6db400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:39.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.759+0000 7f6dcbe00640 1 -- 172.21.15.167:0/1324895002 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6dcc161000 con 0x7f6db4004800 2024-05-11T12:13:39.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.759+0000 7f6dcbe00640 1 -- 172.21.15.167:0/1324895002 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6db400c160 con 0x7f6db4005210 2024-05-11T12:13:39.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.759+0000 7f6dcbe00640 1 -- 172.21.15.167:0/1324895002 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6db400c360 con 0x7f6db4003aa0 2024-05-11T12:13:39.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.760+0000 7f6dd0a00640 1 --2- 172.21.15.167:0/1324895002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6db4004800 0x7f6db4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:39.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.760+0000 7f6dd0a00640 1 -- 172.21.15.167:0/1324895002 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6db4003aa0 msgr2=0x7f6db4003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:39.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.760+0000 7f6dd0a00640 1 --2- 172.21.15.167:0/1324895002 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6db4003aa0 0x7f6db4003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:39.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.760+0000 7f6dd0a00640 1 -- 172.21.15.167:0/1324895002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6db4005210 msgr2=0x7f6db400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:39.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.760+0000 7f6dd1e00640 1 --2- 172.21.15.167:0/1324895002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6db4005210 0x7f6db400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:39.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.760+0000 7f6dd0a00640 1 --2- 172.21.15.167:0/1324895002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6db4005210 0x7f6db400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:39.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.760+0000 7f6dd0a00640 1 -- 172.21.15.167:0/1324895002 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6db4003520 con 0x7f6db4004800 2024-05-11T12:13:39.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.760+0000 7f6dd1e00640 1 -- 172.21.15.167:0/1324895002 reap_dead start 2024-05-11T12:13:39.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.761+0000 7f6dd0a00640 1 --2- 172.21.15.167:0/1324895002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6db4004800 0x7f6db4004c80 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f6db8009a90 tx=0x7f6db4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64c4f81debe880a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:39.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.761+0000 7f6dcbe00640 1 -- 172.21.15.167:0/1324895002 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db8036070 con 0x7f6db4004800 2024-05-11T12:13:39.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.761+0000 7f6dcbe00640 1 -- 172.21.15.167:0/1324895002 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6db8031400 con 0x7f6db4004800 2024-05-11T12:13:39.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.761+0000 7f6dcbe00640 1 -- 172.21.15.167:0/1324895002 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db8009690 con 0x7f6db4004800 2024-05-11T12:13:39.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.762+0000 7f6dd3e00640 1 -- 172.21.15.167:0/1324895002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6db4004800 msgr2=0x7f6db4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:39.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.762+0000 7f6dd3e00640 1 --2- 172.21.15.167:0/1324895002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6db4004800 0x7f6db4004c80 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f6db8009a90 tx=0x7f6db4005830 comp rx=0 tx=0).stop 2024-05-11T12:13:39.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.762+0000 7f6dd3e00640 1 -- 172.21.15.167:0/1324895002 shutdown_connections 2024-05-11T12:13:39.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.762+0000 7f6dd3e00640 1 --2- 172.21.15.167:0/1324895002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6db4004800 0x7f6db4004c80 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:39.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.762+0000 7f6dd3e00640 1 -- 172.21.15.167:0/1324895002 >> 172.21.15.167:0/1324895002 conn(0x7f6dcc0cd8f0 msgr2=0x7f6dcc0cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:39.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.762+0000 7f6dd3e00640 1 -- 172.21.15.167:0/1324895002 shutdown_connections 2024-05-11T12:13:39.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.762+0000 7f6dd3e00640 1 -- 172.21.15.167:0/1324895002 wait complete. 2024-05-11T12:13:39.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.764+0000 7f6dd3e00640 1 Processor -- start 2024-05-11T12:13:39.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.764+0000 7f6dd3e00640 1 -- start start 2024-05-11T12:13:39.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.765+0000 7f6dd3e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6db4004800 0x7f6dcc15fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:39.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.765+0000 7f6dd3e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6dcc1fd1a0 0x7f6dcc144050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:39.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.765+0000 7f6dd3e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6dcc1fd620 0x7f6dcc144590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:39.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.765+0000 7f6dd3e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6dcc161610 con 0x7f6dcc1fd620 2024-05-11T12:13:39.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.765+0000 7f6dd3e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6dcc161490 con 0x7f6dcc1fd1a0 2024-05-11T12:13:39.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.765+0000 7f6dd3e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6dcc161790 con 0x7f6db4004800 2024-05-11T12:13:39.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.765+0000 7f6dd1e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6dcc1fd620 0x7f6dcc144590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:39.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.765+0000 7f6dd1e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6dcc1fd620 0x7f6dcc144590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41230/0 (socket says 172.21.15.167:41230) 2024-05-11T12:13:39.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.765+0000 7f6dd1e00640 1 -- 172.21.15.167:0/2330496197 learned_addr learned my addr 172.21.15.167:0/2330496197 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:39.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.765+0000 7f6dd1400640 1 --2- 172.21.15.167:0/2330496197 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6db4004800 0x7f6dcc15fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:39.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.765+0000 7f6dd0a00640 1 --2- 172.21.15.167:0/2330496197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6dcc1fd1a0 0x7f6dcc144050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:39.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.765+0000 7f6dd1e00640 1 -- 172.21.15.167:0/2330496197 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6db4004800 msgr2=0x7f6dcc15fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:39.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.765+0000 7f6dd1e00640 1 --2- 172.21.15.167:0/2330496197 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6db4004800 0x7f6dcc15fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:39.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.765+0000 7f6dd1e00640 1 -- 172.21.15.167:0/2330496197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6dcc1fd1a0 msgr2=0x7f6dcc144050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:39.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.765+0000 7f6dd1e00640 1 --2- 172.21.15.167:0/2330496197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6dcc1fd1a0 0x7f6dcc144050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:39.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.766+0000 7f6dd1e00640 1 -- 172.21.15.167:0/2330496197 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dcc15ae00 con 0x7f6dcc1fd620 2024-05-11T12:13:39.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.766+0000 7f6dd1400640 1 --2- 172.21.15.167:0/2330496197 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6db4004800 0x7f6dcc15fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:39.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.766+0000 7f6dd1e00640 1 --2- 172.21.15.167:0/2330496197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6dcc1fd620 0x7f6dcc144590 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc161db0 tx=0x7f6dc400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:39.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.766+0000 7f6dc9600640 1 -- 172.21.15.167:0/2330496197 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4018070 con 0x7f6dcc1fd620 2024-05-11T12:13:39.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.766+0000 7f6dc9600640 1 -- 172.21.15.167:0/2330496197 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dc4003b70 con 0x7f6dcc1fd620 2024-05-11T12:13:39.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.766+0000 7f6dc9600640 1 -- 172.21.15.167:0/2330496197 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4005070 con 0x7f6dcc1fd620 2024-05-11T12:13:39.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.766+0000 7f6dd3e00640 1 -- 172.21.15.167:0/2330496197 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dcc15b0e0 con 0x7f6dcc1fd620 2024-05-11T12:13:39.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.766+0000 7f6dd3e00640 1 -- 172.21.15.167:0/2330496197 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dcc15b6a0 con 0x7f6dcc1fd620 2024-05-11T12:13:39.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.768+0000 7f6dd3e00640 1 -- 172.21.15.167:0/2330496197 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dcc069590 con 0x7f6dcc1fd620 2024-05-11T12:13:39.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.769+0000 7f6dc9600640 1 -- 172.21.15.167:0/2330496197 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6dc401b070 con 0x7f6dcc1fd620 2024-05-11T12:13:39.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.770+0000 7f6dc9600640 1 --2- 172.21.15.167:0/2330496197 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f6dac0762d0 0x7f6dac078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:39.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.770+0000 7f6dc9600640 1 -- 172.21.15.167:0/2330496197 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7f6dc4096ec0 con 0x7f6dcc1fd620 2024-05-11T12:13:39.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.770+0000 7f6dd1400640 1 --2- 172.21.15.167:0/2330496197 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f6dac0762d0 0x7f6dac078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:39.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.771+0000 7f6dd1400640 1 --2- 172.21.15.167:0/2330496197 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f6dac0762d0 0x7f6dac078790 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc160820 tx=0x7f6dc0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:39.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.774+0000 7f6dc9600640 1 -- 172.21.15.167:0/2330496197 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6dc4060680 con 0x7f6dcc1fd620 2024-05-11T12:13:39.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.967+0000 7f6dd3e00640 1 -- 172.21.15.167:0/2330496197 --> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6dcc0cdf20 con 0x7f6dac0762d0 2024-05-11T12:13:39.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.983+0000 7f6dc9600640 1 -- 172.21.15.167:0/2330496197 <== mgr.4106 v2:172.21.15.167:6832/578352559 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252069 (secure 0 0 0) 0x7f6dcc0cdf20 con 0x7f6dac0762d0 2024-05-11T12:13:39.985 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:13:39.988 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2024-05-11T12:13:39.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.989+0000 7f6dd3e00640 1 -- 172.21.15.167:0/2330496197 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f6dac0762d0 msgr2=0x7f6dac078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:39.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.989+0000 7f6dd3e00640 1 --2- 172.21.15.167:0/2330496197 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f6dac0762d0 0x7f6dac078790 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc160820 tx=0x7f6dc0009040 comp rx=0 tx=0).stop 2024-05-11T12:13:39.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.990+0000 7f6dd3e00640 1 -- 172.21.15.167:0/2330496197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6dcc1fd620 msgr2=0x7f6dcc144590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:39.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.990+0000 7f6dd3e00640 1 --2- 172.21.15.167:0/2330496197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6dcc1fd620 0x7f6dcc144590 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc161db0 tx=0x7f6dc400dc00 comp rx=0 tx=0).stop 2024-05-11T12:13:39.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.990+0000 7f6dd3e00640 1 -- 172.21.15.167:0/2330496197 shutdown_connections 2024-05-11T12:13:39.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.990+0000 7f6dd3e00640 1 --2- 172.21.15.167:0/2330496197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6dcc1fd620 0x7f6dcc144590 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:39.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.990+0000 7f6dd3e00640 1 --2- 172.21.15.167:0/2330496197 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f6dac0762d0 0x7f6dac078790 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:39.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.990+0000 7f6dd3e00640 1 --2- 172.21.15.167:0/2330496197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6dcc1fd1a0 0x7f6dcc144050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:39.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.990+0000 7f6dd3e00640 1 --2- 172.21.15.167:0/2330496197 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6db4004800 0x7f6dcc15fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:39.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.990+0000 7f6dd3e00640 1 -- 172.21.15.167:0/2330496197 >> 172.21.15.167:0/2330496197 conn(0x7f6dcc0cd8f0 msgr2=0x7f6dcc0b4360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:39.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.990+0000 7f6dd3e00640 1 -- 172.21.15.167:0/2330496197 shutdown_connections 2024-05-11T12:13:39.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:39.990+0000 7f6dd3e00640 1 -- 172.21.15.167:0/2330496197 wait complete. 2024-05-11T12:13:40.006 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":162,"stamp":"2024-05-11T12:13:39.289103+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":171,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":232800,"kb_used_data":8272,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754741920,"statfs":{"total":773094113280,"available":772855726080,"internally_reserved":0,"allocated":8470528,"data_stored":4429548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":229756465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.785018"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.612718+0000","last_change":"2024-05-11T12:13:35.590959+0000","last_active":"2024-05-11T12:13:36.612718+0000","last_peered":"2024-05-11T12:13:36.612718+0000","last_clean":"2024-05-11T12:13:36.612718+0000","last_became_active":"2024-05-11T12:13:35.590652+0000","last_became_peered":"2024-05-11T12:13:35.590652+0000","last_unstale":"2024-05-11T12:13:36.612718+0000","last_undegraded":"2024-05-11T12:13:36.612718+0000","last_fullsized":"2024-05-11T12:13:36.612718+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:59:10.432575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.570899+0000","last_change":"2024-05-11T12:13:35.579673+0000","last_active":"2024-05-11T12:13:36.570899+0000","last_peered":"2024-05-11T12:13:36.570899+0000","last_clean":"2024-05-11T12:13:36.570899+0000","last_became_active":"2024-05-11T12:13:35.579365+0000","last_became_peered":"2024-05-11T12:13:35.579365+0000","last_unstale":"2024-05-11T12:13:36.570899+0000","last_undegraded":"2024-05-11T12:13:36.570899+0000","last_fullsized":"2024-05-11T12:13:36.570899+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:07:56.484940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.611904+0000","last_change":"2024-05-11T12:13:35.575102+0000","last_active":"2024-05-11T12:13:36.611904+0000","last_peered":"2024-05-11T12:13:36.611904+0000","last_clean":"2024-05-11T12:13:36.611904+0000","last_became_active":"2024-05-11T12:13:35.574842+0000","last_became_peered":"2024-05-11T12:13:35.574842+0000","last_unstale":"2024-05-11T12:13:36.611904+0000","last_undegraded":"2024-05-11T12:13:36.611904+0000","last_fullsized":"2024-05-11T12:13:36.611904+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:53:41.942115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.576746+0000","last_change":"2024-05-11T12:13:35.580885+0000","last_active":"2024-05-11T12:13:36.576746+0000","last_peered":"2024-05-11T12:13:36.576746+0000","last_clean":"2024-05-11T12:13:36.576746+0000","last_became_active":"2024-05-11T12:13:35.580556+0000","last_became_peered":"2024-05-11T12:13:35.580556+0000","last_unstale":"2024-05-11T12:13:36.576746+0000","last_undegraded":"2024-05-11T12:13:36.576746+0000","last_fullsized":"2024-05-11T12:13:36.576746+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:30:09.663375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T12:13:35.591326+0000","last_change":"2024-05-11T12:13:35.584342+0000","last_active":"2024-05-11T12:13:35.591326+0000","last_peered":"2024-05-11T12:13:35.591326+0000","last_clean":"2024-05-11T12:13:35.591326+0000","last_became_active":"2024-05-11T12:13:35.584082+0000","last_became_peered":"2024-05-11T12:13:35.584082+0000","last_unstale":"2024-05-11T12:13:35.591326+0000","last_undegraded":"2024-05-11T12:13:35.591326+0000","last_fullsized":"2024-05-11T12:13:35.591326+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:49:00.732158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.611075+0000","last_change":"2024-05-11T12:13:35.586701+0000","last_active":"2024-05-11T12:13:36.611075+0000","last_peered":"2024-05-11T12:13:36.611075+0000","last_clean":"2024-05-11T12:13:36.611075+0000","last_became_active":"2024-05-11T12:13:35.585360+0000","last_became_peered":"2024-05-11T12:13:35.585360+0000","last_unstale":"2024-05-11T12:13:36.611075+0000","last_undegraded":"2024-05-11T12:13:36.611075+0000","last_fullsized":"2024-05-11T12:13:36.611075+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:07:56.484940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.610245+0000","last_change":"2024-05-11T12:13:35.582888+0000","last_active":"2024-05-11T12:13:36.610245+0000","last_peered":"2024-05-11T12:13:36.610245+0000","last_clean":"2024-05-11T12:13:36.610245+0000","last_became_active":"2024-05-11T12:13:35.581729+0000","last_became_peered":"2024-05-11T12:13:35.581729+0000","last_unstale":"2024-05-11T12:13:36.610245+0000","last_undegraded":"2024-05-11T12:13:36.610245+0000","last_fullsized":"2024-05-11T12:13:36.610245+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:53:27.837561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T12:13:35.587082+0000","last_change":"2024-05-11T12:13:35.578615+0000","last_active":"2024-05-11T12:13:35.587082+0000","last_peered":"2024-05-11T12:13:35.587082+0000","last_clean":"2024-05-11T12:13:35.587082+0000","last_became_active":"2024-05-11T12:13:35.578267+0000","last_became_peered":"2024-05-11T12:13:35.578267+0000","last_unstale":"2024-05-11T12:13:35.587082+0000","last_undegraded":"2024-05-11T12:13:35.587082+0000","last_fullsized":"2024-05-11T12:13:35.587082+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:59:48.432764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.576580+0000","last_change":"2024-05-11T12:13:35.586537+0000","last_active":"2024-05-11T12:13:36.576580+0000","last_peered":"2024-05-11T12:13:36.576580+0000","last_clean":"2024-05-11T12:13:36.576580+0000","last_became_active":"2024-05-11T12:13:35.586257+0000","last_became_peered":"2024-05-11T12:13:35.586257+0000","last_unstale":"2024-05-11T12:13:36.576580+0000","last_undegraded":"2024-05-11T12:13:36.576580+0000","last_fullsized":"2024-05-11T12:13:36.576580+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:56:18.165463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.575915+0000","last_change":"2024-05-11T12:13:35.574940+0000","last_active":"2024-05-11T12:13:36.575915+0000","last_peered":"2024-05-11T12:13:36.575915+0000","last_clean":"2024-05-11T12:13:36.575915+0000","last_became_active":"2024-05-11T12:13:35.574635+0000","last_became_peered":"2024-05-11T12:13:35.574635+0000","last_unstale":"2024-05-11T12:13:36.575915+0000","last_undegraded":"2024-05-11T12:13:36.575915+0000","last_fullsized":"2024-05-11T12:13:36.575915+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:22:57.600314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.571076+0000","last_change":"2024-05-11T12:13:35.580946+0000","last_active":"2024-05-11T12:13:36.571076+0000","last_peered":"2024-05-11T12:13:36.571076+0000","last_clean":"2024-05-11T12:13:36.571076+0000","last_became_active":"2024-05-11T12:13:35.579856+0000","last_became_peered":"2024-05-11T12:13:35.579856+0000","last_unstale":"2024-05-11T12:13:36.571076+0000","last_undegraded":"2024-05-11T12:13:36.571076+0000","last_fullsized":"2024-05-11T12:13:36.571076+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:52:33.630801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.576963+0000","last_change":"2024-05-11T12:13:35.563837+0000","last_active":"2024-05-11T12:13:36.576963+0000","last_peered":"2024-05-11T12:13:36.576963+0000","last_clean":"2024-05-11T12:13:36.576963+0000","last_became_active":"2024-05-11T12:13:35.563138+0000","last_became_peered":"2024-05-11T12:13:35.563138+0000","last_unstale":"2024-05-11T12:13:36.576963+0000","last_undegraded":"2024-05-11T12:13:36.576963+0000","last_fullsized":"2024-05-11T12:13:36.576963+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:14:02.793555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T12:13:35.587129+0000","last_change":"2024-05-11T12:13:35.580551+0000","last_active":"2024-05-11T12:13:35.587129+0000","last_peered":"2024-05-11T12:13:35.587129+0000","last_clean":"2024-05-11T12:13:35.587129+0000","last_became_active":"2024-05-11T12:13:35.578638+0000","last_became_peered":"2024-05-11T12:13:35.578638+0000","last_unstale":"2024-05-11T12:13:35.587129+0000","last_undegraded":"2024-05-11T12:13:35.587129+0000","last_fullsized":"2024-05-11T12:13:35.587129+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:37:48.454996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.611729+0000","last_change":"2024-05-11T12:13:35.585657+0000","last_active":"2024-05-11T12:13:36.611729+0000","last_peered":"2024-05-11T12:13:36.611729+0000","last_clean":"2024-05-11T12:13:36.611729+0000","last_became_active":"2024-05-11T12:13:35.585029+0000","last_became_peered":"2024-05-11T12:13:35.585029+0000","last_unstale":"2024-05-11T12:13:36.611729+0000","last_undegraded":"2024-05-11T12:13:36.611729+0000","last_fullsized":"2024-05-11T12:13:36.611729+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:22:46.485528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.570467+0000","last_change":"2024-05-11T12:13:35.563300+0000","last_active":"2024-05-11T12:13:36.570467+0000","last_peered":"2024-05-11T12:13:36.570467+0000","last_clean":"2024-05-11T12:13:36.570467+0000","last_became_active":"2024-05-11T12:13:35.561620+0000","last_became_peered":"2024-05-11T12:13:35.561620+0000","last_unstale":"2024-05-11T12:13:36.570467+0000","last_undegraded":"2024-05-11T12:13:36.570467+0000","last_fullsized":"2024-05-11T12:13:36.570467+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:16:26.769737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.610983+0000","last_change":"2024-05-11T12:13:35.584780+0000","last_active":"2024-05-11T12:13:36.610983+0000","last_peered":"2024-05-11T12:13:36.610983+0000","last_clean":"2024-05-11T12:13:36.610983+0000","last_became_active":"2024-05-11T12:13:35.583787+0000","last_became_peered":"2024-05-11T12:13:35.583787+0000","last_unstale":"2024-05-11T12:13:36.610983+0000","last_undegraded":"2024-05-11T12:13:36.610983+0000","last_fullsized":"2024-05-11T12:13:36.610983+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:16:26.769737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T12:13:35.590730+0000","last_change":"2024-05-11T12:13:35.583403+0000","last_active":"2024-05-11T12:13:35.590730+0000","last_peered":"2024-05-11T12:13:35.590730+0000","last_clean":"2024-05-11T12:13:35.590730+0000","last_became_active":"2024-05-11T12:13:35.583146+0000","last_became_peered":"2024-05-11T12:13:35.583146+0000","last_unstale":"2024-05-11T12:13:35.590730+0000","last_undegraded":"2024-05-11T12:13:35.590730+0000","last_fullsized":"2024-05-11T12:13:35.590730+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:03:08.848832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.612290+0000","last_change":"2024-05-11T12:13:35.589753+0000","last_active":"2024-05-11T12:13:36.612290+0000","last_peered":"2024-05-11T12:13:36.612290+0000","last_clean":"2024-05-11T12:13:36.612290+0000","last_became_active":"2024-05-11T12:13:35.589349+0000","last_became_peered":"2024-05-11T12:13:35.589349+0000","last_unstale":"2024-05-11T12:13:36.612290+0000","last_undegraded":"2024-05-11T12:13:36.612290+0000","last_fullsized":"2024-05-11T12:13:36.612290+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:00:39.269552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.612161+0000","last_change":"2024-05-11T12:13:35.584149+0000","last_active":"2024-05-11T12:13:36.612161+0000","last_peered":"2024-05-11T12:13:36.612161+0000","last_clean":"2024-05-11T12:13:36.612161+0000","last_became_active":"2024-05-11T12:13:35.583826+0000","last_became_peered":"2024-05-11T12:13:35.583826+0000","last_unstale":"2024-05-11T12:13:36.612161+0000","last_undegraded":"2024-05-11T12:13:36.612161+0000","last_fullsized":"2024-05-11T12:13:36.612161+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:52:33.630801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T12:13:35.585431+0000","last_change":"2024-05-11T12:13:35.574491+0000","last_active":"2024-05-11T12:13:35.585431+0000","last_peered":"2024-05-11T12:13:35.585431+0000","last_clean":"2024-05-11T12:13:35.585431+0000","last_became_active":"2024-05-11T12:13:35.573770+0000","last_became_peered":"2024-05-11T12:13:35.573770+0000","last_unstale":"2024-05-11T12:13:35.585431+0000","last_undegraded":"2024-05-11T12:13:35.585431+0000","last_fullsized":"2024-05-11T12:13:35.585431+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:57:14.890262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.574897+0000","last_change":"2024-05-11T12:13:35.577440+0000","last_active":"2024-05-11T12:13:36.574897+0000","last_peered":"2024-05-11T12:13:36.574897+0000","last_clean":"2024-05-11T12:13:36.574897+0000","last_became_active":"2024-05-11T12:13:35.576906+0000","last_became_peered":"2024-05-11T12:13:35.576906+0000","last_unstale":"2024-05-11T12:13:36.574897+0000","last_undegraded":"2024-05-11T12:13:36.574897+0000","last_fullsized":"2024-05-11T12:13:36.574897+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:47:56.175781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T12:13:35.585495+0000","last_change":"2024-05-11T12:13:35.573224+0000","last_active":"2024-05-11T12:13:35.585495+0000","last_peered":"2024-05-11T12:13:35.585495+0000","last_clean":"2024-05-11T12:13:35.585495+0000","last_became_active":"2024-05-11T12:13:35.572950+0000","last_became_peered":"2024-05-11T12:13:35.572950+0000","last_unstale":"2024-05-11T12:13:35.585495+0000","last_undegraded":"2024-05-11T12:13:35.585495+0000","last_fullsized":"2024-05-11T12:13:35.585495+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:04:54.339323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.574183+0000","last_change":"2024-05-11T12:13:35.578862+0000","last_active":"2024-05-11T12:13:36.574183+0000","last_peered":"2024-05-11T12:13:36.574183+0000","last_clean":"2024-05-11T12:13:36.574183+0000","last_became_active":"2024-05-11T12:13:35.578596+0000","last_became_peered":"2024-05-11T12:13:35.578596+0000","last_unstale":"2024-05-11T12:13:36.574183+0000","last_undegraded":"2024-05-11T12:13:36.574183+0000","last_fullsized":"2024-05-11T12:13:36.574183+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:06:51.173352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.570380+0000","last_change":"2024-05-11T12:13:35.559838+0000","last_active":"2024-05-11T12:13:36.570380+0000","last_peered":"2024-05-11T12:13:36.570380+0000","last_clean":"2024-05-11T12:13:36.570380+0000","last_became_active":"2024-05-11T12:13:35.559521+0000","last_became_peered":"2024-05-11T12:13:35.559521+0000","last_unstale":"2024-05-11T12:13:36.570380+0000","last_undegraded":"2024-05-11T12:13:36.570380+0000","last_fullsized":"2024-05-11T12:13:36.570380+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:48:24.466731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.571121+0000","last_change":"2024-05-11T12:13:35.567028+0000","last_active":"2024-05-11T12:13:36.571121+0000","last_peered":"2024-05-11T12:13:36.571121+0000","last_clean":"2024-05-11T12:13:36.571121+0000","last_became_active":"2024-05-11T12:13:35.566753+0000","last_became_peered":"2024-05-11T12:13:35.566753+0000","last_unstale":"2024-05-11T12:13:36.571121+0000","last_undegraded":"2024-05-11T12:13:36.571121+0000","last_fullsized":"2024-05-11T12:13:36.571121+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:30:40.375149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.576293+0000","last_change":"2024-05-11T12:13:35.566601+0000","last_active":"2024-05-11T12:13:36.576293+0000","last_peered":"2024-05-11T12:13:36.576293+0000","last_clean":"2024-05-11T12:13:36.576293+0000","last_became_active":"2024-05-11T12:13:35.566239+0000","last_became_peered":"2024-05-11T12:13:35.566239+0000","last_unstale":"2024-05-11T12:13:36.576293+0000","last_undegraded":"2024-05-11T12:13:36.576293+0000","last_fullsized":"2024-05-11T12:13:36.576293+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:12:13.340132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.612443+0000","last_change":"2024-05-11T12:13:35.585177+0000","last_active":"2024-05-11T12:13:36.612443+0000","last_peered":"2024-05-11T12:13:36.612443+0000","last_clean":"2024-05-11T12:13:36.612443+0000","last_became_active":"2024-05-11T12:13:35.584719+0000","last_became_peered":"2024-05-11T12:13:35.584719+0000","last_unstale":"2024-05-11T12:13:36.612443+0000","last_undegraded":"2024-05-11T12:13:36.612443+0000","last_fullsized":"2024-05-11T12:13:36.612443+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:14:41.103392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T12:13:35.585455+0000","last_change":"2024-05-11T12:13:35.574715+0000","last_active":"2024-05-11T12:13:35.585455+0000","last_peered":"2024-05-11T12:13:35.585455+0000","last_clean":"2024-05-11T12:13:35.585455+0000","last_became_active":"2024-05-11T12:13:35.574079+0000","last_became_peered":"2024-05-11T12:13:35.574079+0000","last_unstale":"2024-05-11T12:13:35.585455+0000","last_undegraded":"2024-05-11T12:13:35.585455+0000","last_fullsized":"2024-05-11T12:13:35.585455+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:42:31.620827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.570722+0000","last_change":"2024-05-11T12:13:35.563312+0000","last_active":"2024-05-11T12:13:36.570722+0000","last_peered":"2024-05-11T12:13:36.570722+0000","last_clean":"2024-05-11T12:13:36.570722+0000","last_became_active":"2024-05-11T12:13:35.563032+0000","last_became_peered":"2024-05-11T12:13:35.563032+0000","last_unstale":"2024-05-11T12:13:36.570722+0000","last_undegraded":"2024-05-11T12:13:36.570722+0000","last_fullsized":"2024-05-11T12:13:36.570722+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:59:10.432575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.611812+0000","last_change":"2024-05-11T12:13:35.581294+0000","last_active":"2024-05-11T12:13:36.611812+0000","last_peered":"2024-05-11T12:13:36.611812+0000","last_clean":"2024-05-11T12:13:36.611812+0000","last_became_active":"2024-05-11T12:13:35.580366+0000","last_became_peered":"2024-05-11T12:13:35.580366+0000","last_unstale":"2024-05-11T12:13:36.611812+0000","last_undegraded":"2024-05-11T12:13:36.611812+0000","last_fullsized":"2024-05-11T12:13:36.611812+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:15:48.806971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.610102+0000","last_change":"2024-05-11T12:13:35.576683+0000","last_active":"2024-05-11T12:13:36.610102+0000","last_peered":"2024-05-11T12:13:36.610102+0000","last_clean":"2024-05-11T12:13:36.610102+0000","last_became_active":"2024-05-11T12:13:35.576409+0000","last_became_peered":"2024-05-11T12:13:35.576409+0000","last_unstale":"2024-05-11T12:13:36.610102+0000","last_undegraded":"2024-05-11T12:13:36.610102+0000","last_fullsized":"2024-05-11T12:13:36.610102+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:58:11.648486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.610947+0000","last_change":"2024-05-11T12:13:35.590578+0000","last_active":"2024-05-11T12:13:36.610947+0000","last_peered":"2024-05-11T12:13:36.610947+0000","last_clean":"2024-05-11T12:13:36.610947+0000","last_became_active":"2024-05-11T12:13:35.590115+0000","last_became_peered":"2024-05-11T12:13:35.590115+0000","last_unstale":"2024-05-11T12:13:36.610947+0000","last_undegraded":"2024-05-11T12:13:36.610947+0000","last_fullsized":"2024-05-11T12:13:36.610947+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:48:24.466731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.609724+0000","last_change":"2024-05-11T12:13:35.587193+0000","last_active":"2024-05-11T12:13:36.609724+0000","last_peered":"2024-05-11T12:13:36.609724+0000","last_clean":"2024-05-11T12:13:36.609724+0000","last_became_active":"2024-05-11T12:13:35.586683+0000","last_became_peered":"2024-05-11T12:13:35.586683+0000","last_unstale":"2024-05-11T12:13:36.609724+0000","last_undegraded":"2024-05-11T12:13:36.609724+0000","last_fullsized":"2024-05-11T12:13:36.609724+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:11:10.692727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"74'4","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T12:13:35.595247+0000","last_change":"2024-05-11T12:13:34.529830+0000","last_active":"2024-05-11T12:13:35.595247+0000","last_peered":"2024-05-11T12:13:35.595247+0000","last_clean":"2024-05-11T12:13:35.595247+0000","last_became_active":"2024-05-11T12:13:34.529271+0000","last_became_peered":"2024-05-11T12:13:34.529271+0000","last_unstale":"2024-05-11T12:13:35.595247+0000","last_undegraded":"2024-05-11T12:13:35.595247+0000","last_fullsized":"2024-05-11T12:13:35.595247+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:56:13.129398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.570614+0000","last_change":"2024-05-11T12:13:35.557326+0000","last_active":"2024-05-11T12:13:36.570614+0000","last_peered":"2024-05-11T12:13:36.570614+0000","last_clean":"2024-05-11T12:13:36.570614+0000","last_became_active":"2024-05-11T12:13:35.557021+0000","last_became_peered":"2024-05-11T12:13:35.557021+0000","last_unstale":"2024-05-11T12:13:36.570614+0000","last_undegraded":"2024-05-11T12:13:36.570614+0000","last_fullsized":"2024-05-11T12:13:36.570614+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:15:48.806971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"74'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.612602+0000","last_change":"2024-05-11T12:13:34.552817+0000","last_active":"2024-05-11T12:13:36.612602+0000","last_peered":"2024-05-11T12:13:36.612602+0000","last_clean":"2024-05-11T12:13:36.612602+0000","last_became_active":"2024-05-11T12:13:34.551902+0000","last_became_peered":"2024-05-11T12:13:34.551902+0000","last_unstale":"2024-05-11T12:13:36.612602+0000","last_undegraded":"2024-05-11T12:13:36.612602+0000","last_fullsized":"2024-05-11T12:13:36.612602+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:29:20.677448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.575434+0000","last_change":"2024-05-11T12:13:35.588607+0000","last_active":"2024-05-11T12:13:36.575434+0000","last_peered":"2024-05-11T12:13:36.575434+0000","last_clean":"2024-05-11T12:13:36.575434+0000","last_became_active":"2024-05-11T12:13:35.587899+0000","last_became_peered":"2024-05-11T12:13:35.587899+0000","last_unstale":"2024-05-11T12:13:36.575434+0000","last_undegraded":"2024-05-11T12:13:36.575434+0000","last_fullsized":"2024-05-11T12:13:36.575434+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:23:27.409855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"74'2","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.612320+0000","last_change":"2024-05-11T12:13:34.552747+0000","last_active":"2024-05-11T12:13:36.612320+0000","last_peered":"2024-05-11T12:13:36.612320+0000","last_clean":"2024-05-11T12:13:36.612320+0000","last_became_active":"2024-05-11T12:13:34.552384+0000","last_became_peered":"2024-05-11T12:13:34.552384+0000","last_unstale":"2024-05-11T12:13:36.612320+0000","last_undegraded":"2024-05-11T12:13:36.612320+0000","last_fullsized":"2024-05-11T12:13:36.612320+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:48:51.249018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.575536+0000","last_change":"2024-05-11T12:13:35.577190+0000","last_active":"2024-05-11T12:13:36.575536+0000","last_peered":"2024-05-11T12:13:36.575536+0000","last_clean":"2024-05-11T12:13:36.575536+0000","last_became_active":"2024-05-11T12:13:35.575641+0000","last_became_peered":"2024-05-11T12:13:35.575641+0000","last_unstale":"2024-05-11T12:13:36.575536+0000","last_undegraded":"2024-05-11T12:13:36.575536+0000","last_fullsized":"2024-05-11T12:13:36.575536+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:05:56.975495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"74'2","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.610195+0000","last_change":"2024-05-11T12:13:34.546506+0000","last_active":"2024-05-11T12:13:36.610195+0000","last_peered":"2024-05-11T12:13:36.610195+0000","last_clean":"2024-05-11T12:13:36.610195+0000","last_became_active":"2024-05-11T12:13:34.544853+0000","last_became_peered":"2024-05-11T12:13:34.544853+0000","last_unstale":"2024-05-11T12:13:36.610195+0000","last_undegraded":"2024-05-11T12:13:36.610195+0000","last_fullsized":"2024-05-11T12:13:36.610195+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:23:26.395424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.570878+0000","last_change":"2024-05-11T12:13:35.580300+0000","last_active":"2024-05-11T12:13:36.570878+0000","last_peered":"2024-05-11T12:13:36.570878+0000","last_clean":"2024-05-11T12:13:36.570878+0000","last_became_active":"2024-05-11T12:13:35.580028+0000","last_became_peered":"2024-05-11T12:13:35.580028+0000","last_unstale":"2024-05-11T12:13:36.570878+0000","last_undegraded":"2024-05-11T12:13:36.570878+0000","last_fullsized":"2024-05-11T12:13:36.570878+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:25:10.295427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"74'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.575835+0000","last_change":"2024-05-11T12:13:34.543147+0000","last_active":"2024-05-11T12:13:36.575835+0000","last_peered":"2024-05-11T12:13:36.575835+0000","last_clean":"2024-05-11T12:13:36.575835+0000","last_became_active":"2024-05-11T12:13:34.542853+0000","last_became_peered":"2024-05-11T12:13:34.542853+0000","last_unstale":"2024-05-11T12:13:36.575835+0000","last_undegraded":"2024-05-11T12:13:36.575835+0000","last_fullsized":"2024-05-11T12:13:36.575835+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:57:09.184227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.571369+0000","last_change":"2024-05-11T12:13:35.579836+0000","last_active":"2024-05-11T12:13:36.571369+0000","last_peered":"2024-05-11T12:13:36.571369+0000","last_clean":"2024-05-11T12:13:36.571369+0000","last_became_active":"2024-05-11T12:13:35.579438+0000","last_became_peered":"2024-05-11T12:13:35.579438+0000","last_unstale":"2024-05-11T12:13:36.571369+0000","last_undegraded":"2024-05-11T12:13:36.571369+0000","last_fullsized":"2024-05-11T12:13:36.571369+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:02:53.925038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"74'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.574065+0000","last_change":"2024-05-11T12:13:34.527739+0000","last_active":"2024-05-11T12:13:36.574065+0000","last_peered":"2024-05-11T12:13:36.574065+0000","last_clean":"2024-05-11T12:13:36.574065+0000","last_became_active":"2024-05-11T12:13:34.526853+0000","last_became_peered":"2024-05-11T12:13:34.526853+0000","last_unstale":"2024-05-11T12:13:36.574065+0000","last_undegraded":"2024-05-11T12:13:36.574065+0000","last_fullsized":"2024-05-11T12:13:36.574065+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:53:26.823130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.577054+0000","last_change":"2024-05-11T12:13:35.587403+0000","last_active":"2024-05-11T12:13:36.577054+0000","last_peered":"2024-05-11T12:13:36.577054+0000","last_clean":"2024-05-11T12:13:36.577054+0000","last_became_active":"2024-05-11T12:13:35.587125+0000","last_became_peered":"2024-05-11T12:13:35.587125+0000","last_unstale":"2024-05-11T12:13:36.577054+0000","last_undegraded":"2024-05-11T12:13:36.577054+0000","last_fullsized":"2024-05-11T12:13:36.577054+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:58:01.135264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.570561+0000","last_change":"2024-05-11T12:13:34.543098+0000","last_active":"2024-05-11T12:13:36.570561+0000","last_peered":"2024-05-11T12:13:36.570561+0000","last_clean":"2024-05-11T12:13:36.570561+0000","last_became_active":"2024-05-11T12:13:34.542360+0000","last_became_peered":"2024-05-11T12:13:34.542360+0000","last_unstale":"2024-05-11T12:13:36.570561+0000","last_undegraded":"2024-05-11T12:13:36.570561+0000","last_fullsized":"2024-05-11T12:13:36.570561+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:23:00.798976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.611169+0000","last_change":"2024-05-11T12:13:35.589538+0000","last_active":"2024-05-11T12:13:36.611169+0000","last_peered":"2024-05-11T12:13:36.611169+0000","last_clean":"2024-05-11T12:13:36.611169+0000","last_became_active":"2024-05-11T12:13:35.589064+0000","last_became_peered":"2024-05-11T12:13:35.589064+0000","last_unstale":"2024-05-11T12:13:36.611169+0000","last_undegraded":"2024-05-11T12:13:36.611169+0000","last_fullsized":"2024-05-11T12:13:36.611169+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:29:21.691879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T12:13:35.568494+0000","last_change":"2024-05-11T12:13:34.545745+0000","last_active":"2024-05-11T12:13:35.568494+0000","last_peered":"2024-05-11T12:13:35.568494+0000","last_clean":"2024-05-11T12:13:35.568494+0000","last_became_active":"2024-05-11T12:13:34.545455+0000","last_became_peered":"2024-05-11T12:13:34.545455+0000","last_unstale":"2024-05-11T12:13:35.568494+0000","last_undegraded":"2024-05-11T12:13:35.568494+0000","last_fullsized":"2024-05-11T12:13:35.568494+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:39:06.776230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.570642+0000","last_change":"2024-05-11T12:13:35.575951+0000","last_active":"2024-05-11T12:13:36.570642+0000","last_peered":"2024-05-11T12:13:36.570642+0000","last_clean":"2024-05-11T12:13:36.570642+0000","last_became_active":"2024-05-11T12:13:35.575694+0000","last_became_peered":"2024-05-11T12:13:35.575694+0000","last_unstale":"2024-05-11T12:13:36.570642+0000","last_undegraded":"2024-05-11T12:13:36.570642+0000","last_fullsized":"2024-05-11T12:13:36.570642+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:14:41.103392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.613028+0000","last_change":"2024-05-11T12:13:34.553421+0000","last_active":"2024-05-11T12:13:36.613028+0000","last_peered":"2024-05-11T12:13:36.613028+0000","last_clean":"2024-05-11T12:13:36.613028+0000","last_became_active":"2024-05-11T12:13:34.551745+0000","last_became_peered":"2024-05-11T12:13:34.551745+0000","last_unstale":"2024-05-11T12:13:36.613028+0000","last_undegraded":"2024-05-11T12:13:36.613028+0000","last_fullsized":"2024-05-11T12:13:36.613028+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:48:40.873393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.574270+0000","last_change":"2024-05-11T12:13:35.579319+0000","last_active":"2024-05-11T12:13:36.574270+0000","last_peered":"2024-05-11T12:13:36.574270+0000","last_clean":"2024-05-11T12:13:36.574270+0000","last_became_active":"2024-05-11T12:13:35.577954+0000","last_became_peered":"2024-05-11T12:13:35.577954+0000","last_unstale":"2024-05-11T12:13:36.574270+0000","last_undegraded":"2024-05-11T12:13:36.574270+0000","last_fullsized":"2024-05-11T12:13:36.574270+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:42:41.648383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.576562+0000","last_change":"2024-05-11T12:13:34.550391+0000","last_active":"2024-05-11T12:13:36.576562+0000","last_peered":"2024-05-11T12:13:36.576562+0000","last_clean":"2024-05-11T12:13:36.576562+0000","last_became_active":"2024-05-11T12:13:34.549397+0000","last_became_peered":"2024-05-11T12:13:34.549397+0000","last_unstale":"2024-05-11T12:13:36.576562+0000","last_undegraded":"2024-05-11T12:13:36.576562+0000","last_fullsized":"2024-05-11T12:13:36.576562+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:48:40.873393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.611601+0000","last_change":"2024-05-11T12:13:35.563098+0000","last_active":"2024-05-11T12:13:36.611601+0000","last_peered":"2024-05-11T12:13:36.611601+0000","last_clean":"2024-05-11T12:13:36.611601+0000","last_became_active":"2024-05-11T12:13:35.561915+0000","last_became_peered":"2024-05-11T12:13:35.561915+0000","last_unstale":"2024-05-11T12:13:36.611601+0000","last_undegraded":"2024-05-11T12:13:36.611601+0000","last_fullsized":"2024-05-11T12:13:36.611601+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:50:15.033452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"74'3","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T12:13:35.585147+0000","last_change":"2024-05-11T12:13:34.545422+0000","last_active":"2024-05-11T12:13:35.585147+0000","last_peered":"2024-05-11T12:13:35.585147+0000","last_clean":"2024-05-11T12:13:35.585147+0000","last_became_active":"2024-05-11T12:13:34.545097+0000","last_became_peered":"2024-05-11T12:13:34.545097+0000","last_unstale":"2024-05-11T12:13:35.585147+0000","last_undegraded":"2024-05-11T12:13:35.585147+0000","last_fullsized":"2024-05-11T12:13:35.585147+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:51:57.156752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.610640+0000","last_change":"2024-05-11T12:13:35.567723+0000","last_active":"2024-05-11T12:13:36.610640+0000","last_peered":"2024-05-11T12:13:36.610640+0000","last_clean":"2024-05-11T12:13:36.610640+0000","last_became_active":"2024-05-11T12:13:35.567449+0000","last_became_peered":"2024-05-11T12:13:35.567449+0000","last_unstale":"2024-05-11T12:13:36.610640+0000","last_undegraded":"2024-05-11T12:13:36.610640+0000","last_fullsized":"2024-05-11T12:13:36.610640+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:53:59.844302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"74'6","reported_seq":20,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.610500+0000","last_change":"2024-05-11T12:13:34.547145+0000","last_active":"2024-05-11T12:13:36.610500+0000","last_peered":"2024-05-11T12:13:36.610500+0000","last_clean":"2024-05-11T12:13:36.610500+0000","last_became_active":"2024-05-11T12:13:34.546831+0000","last_became_peered":"2024-05-11T12:13:34.546831+0000","last_unstale":"2024-05-11T12:13:36.610500+0000","last_undegraded":"2024-05-11T12:13:36.610500+0000","last_fullsized":"2024-05-11T12:13:36.610500+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:05:55.961064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.577047+0000","last_change":"2024-05-11T12:13:35.559118+0000","last_active":"2024-05-11T12:13:36.577047+0000","last_peered":"2024-05-11T12:13:36.577047+0000","last_clean":"2024-05-11T12:13:36.577047+0000","last_became_active":"2024-05-11T12:13:35.558012+0000","last_became_peered":"2024-05-11T12:13:35.558012+0000","last_unstale":"2024-05-11T12:13:36.577047+0000","last_undegraded":"2024-05-11T12:13:36.577047+0000","last_fullsized":"2024-05-11T12:13:36.577047+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:06:33.862467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"74'2","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.576179+0000","last_change":"2024-05-11T12:13:34.543802+0000","last_active":"2024-05-11T12:13:36.576179+0000","last_peered":"2024-05-11T12:13:36.576179+0000","last_clean":"2024-05-11T12:13:36.576179+0000","last_became_active":"2024-05-11T12:13:34.543498+0000","last_became_peered":"2024-05-11T12:13:34.543498+0000","last_unstale":"2024-05-11T12:13:36.576179+0000","last_undegraded":"2024-05-11T12:13:36.576179+0000","last_fullsized":"2024-05-11T12:13:36.576179+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:23:00.798976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.610590+0000","last_change":"2024-05-11T12:13:35.565054+0000","last_active":"2024-05-11T12:13:36.610590+0000","last_peered":"2024-05-11T12:13:36.610590+0000","last_clean":"2024-05-11T12:13:36.610590+0000","last_became_active":"2024-05-11T12:13:35.564748+0000","last_became_peered":"2024-05-11T12:13:35.564748+0000","last_unstale":"2024-05-11T12:13:36.610590+0000","last_undegraded":"2024-05-11T12:13:36.610590+0000","last_fullsized":"2024-05-11T12:13:36.610590+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:23:01.813407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.611776+0000","last_change":"2024-05-11T12:13:34.553121+0000","last_active":"2024-05-11T12:13:36.611776+0000","last_peered":"2024-05-11T12:13:36.611776+0000","last_clean":"2024-05-11T12:13:36.611776+0000","last_became_active":"2024-05-11T12:13:34.552859+0000","last_became_peered":"2024-05-11T12:13:34.552859+0000","last_unstale":"2024-05-11T12:13:36.611776+0000","last_undegraded":"2024-05-11T12:13:36.611776+0000","last_fullsized":"2024-05-11T12:13:36.611776+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:45:44.981395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.610931+0000","last_change":"2024-05-11T12:13:35.588305+0000","last_active":"2024-05-11T12:13:36.610931+0000","last_peered":"2024-05-11T12:13:36.610931+0000","last_clean":"2024-05-11T12:13:36.610931+0000","last_became_active":"2024-05-11T12:13:35.587031+0000","last_became_peered":"2024-05-11T12:13:35.587031+0000","last_unstale":"2024-05-11T12:13:36.610931+0000","last_undegraded":"2024-05-11T12:13:36.610931+0000","last_fullsized":"2024-05-11T12:13:36.610931+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:48:41.887824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T12:13:35.568788+0000","last_change":"2024-05-11T12:13:34.532223+0000","last_active":"2024-05-11T12:13:35.568788+0000","last_peered":"2024-05-11T12:13:35.568788+0000","last_clean":"2024-05-11T12:13:35.568788+0000","last_became_active":"2024-05-11T12:13:34.531957+0000","last_became_peered":"2024-05-11T12:13:34.531957+0000","last_unstale":"2024-05-11T12:13:35.568788+0000","last_undegraded":"2024-05-11T12:13:35.568788+0000","last_fullsized":"2024-05-11T12:13:35.568788+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:27:21.412461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.611167+0000","last_change":"2024-05-11T12:13:35.573744+0000","last_active":"2024-05-11T12:13:36.611167+0000","last_peered":"2024-05-11T12:13:36.611167+0000","last_clean":"2024-05-11T12:13:36.611167+0000","last_became_active":"2024-05-11T12:13:35.571944+0000","last_became_peered":"2024-05-11T12:13:35.571944+0000","last_unstale":"2024-05-11T12:13:36.611167+0000","last_undegraded":"2024-05-11T12:13:36.611167+0000","last_fullsized":"2024-05-11T12:13:36.611167+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:34:40.157185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"74'8","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.570490+0000","last_change":"2024-05-11T12:13:34.549604+0000","last_active":"2024-05-11T12:13:36.570490+0000","last_peered":"2024-05-11T12:13:36.570490+0000","last_clean":"2024-05-11T12:13:36.570490+0000","last_became_active":"2024-05-11T12:13:34.549283+0000","last_became_peered":"2024-05-11T12:13:34.549283+0000","last_unstale":"2024-05-11T12:13:36.570490+0000","last_undegraded":"2024-05-11T12:13:36.570490+0000","last_fullsized":"2024-05-11T12:13:36.570490+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:45:44.981395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.610687+0000","last_change":"2024-05-11T12:13:35.579632+0000","last_active":"2024-05-11T12:13:36.610687+0000","last_peered":"2024-05-11T12:13:36.610687+0000","last_clean":"2024-05-11T12:13:36.610687+0000","last_became_active":"2024-05-11T12:13:35.579306+0000","last_became_peered":"2024-05-11T12:13:35.579306+0000","last_unstale":"2024-05-11T12:13:36.610687+0000","last_undegraded":"2024-05-11T12:13:36.610687+0000","last_fullsized":"2024-05-11T12:13:36.610687+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:25:10.295427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"74'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.613057+0000","last_change":"2024-05-11T12:13:34.555461+0000","last_active":"2024-05-11T12:13:36.613057+0000","last_peered":"2024-05-11T12:13:36.613057+0000","last_clean":"2024-05-11T12:13:36.613057+0000","last_became_active":"2024-05-11T12:13:34.555021+0000","last_became_peered":"2024-05-11T12:13:34.555021+0000","last_unstale":"2024-05-11T12:13:36.613057+0000","last_undegraded":"2024-05-11T12:13:36.613057+0000","last_fullsized":"2024-05-11T12:13:36.613057+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:34:39.142754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T12:13:35.587147+0000","last_change":"2024-05-11T12:13:35.580694+0000","last_active":"2024-05-11T12:13:35.587147+0000","last_peered":"2024-05-11T12:13:35.587147+0000","last_clean":"2024-05-11T12:13:35.587147+0000","last_became_active":"2024-05-11T12:13:35.580237+0000","last_became_peered":"2024-05-11T12:13:35.580237+0000","last_unstale":"2024-05-11T12:13:35.587147+0000","last_undegraded":"2024-05-11T12:13:35.587147+0000","last_fullsized":"2024-05-11T12:13:35.587147+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:50:57.189033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"74'2","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.577051+0000","last_change":"2024-05-11T12:13:34.544507+0000","last_active":"2024-05-11T12:13:36.577051+0000","last_peered":"2024-05-11T12:13:36.577051+0000","last_clean":"2024-05-11T12:13:36.577051+0000","last_became_active":"2024-05-11T12:13:34.544222+0000","last_became_peered":"2024-05-11T12:13:34.544222+0000","last_unstale":"2024-05-11T12:13:36.577051+0000","last_undegraded":"2024-05-11T12:13:36.577051+0000","last_fullsized":"2024-05-11T12:13:36.577051+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:04:37.397306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T12:13:35.590614+0000","last_change":"2024-05-11T12:13:35.581881+0000","last_active":"2024-05-11T12:13:35.590614+0000","last_peered":"2024-05-11T12:13:35.590614+0000","last_clean":"2024-05-11T12:13:35.590614+0000","last_became_active":"2024-05-11T12:13:35.581591+0000","last_became_peered":"2024-05-11T12:13:35.581591+0000","last_unstale":"2024-05-11T12:13:35.590614+0000","last_undegraded":"2024-05-11T12:13:35.590614+0000","last_fullsized":"2024-05-11T12:13:35.590614+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:20:22.619157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"74'1","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T12:13:35.593646+0000","last_change":"2024-05-11T12:13:34.543092+0000","last_active":"2024-05-11T12:13:35.593646+0000","last_peered":"2024-05-11T12:13:35.593646+0000","last_clean":"2024-05-11T12:13:35.593646+0000","last_became_active":"2024-05-11T12:13:34.542481+0000","last_became_peered":"2024-05-11T12:13:34.542481+0000","last_unstale":"2024-05-11T12:13:35.593646+0000","last_undegraded":"2024-05-11T12:13:35.593646+0000","last_fullsized":"2024-05-11T12:13:35.593646+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:21:50.635577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.612765+0000","last_change":"2024-05-11T12:13:35.588411+0000","last_active":"2024-05-11T12:13:36.612765+0000","last_peered":"2024-05-11T12:13:36.612765+0000","last_clean":"2024-05-11T12:13:36.612765+0000","last_became_active":"2024-05-11T12:13:35.587858+0000","last_became_peered":"2024-05-11T12:13:35.587858+0000","last_unstale":"2024-05-11T12:13:36.612765+0000","last_undegraded":"2024-05-11T12:13:36.612765+0000","last_fullsized":"2024-05-11T12:13:36.612765+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:30:40.375149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.611062+0000","last_change":"2024-05-11T12:13:34.542352+0000","last_active":"2024-05-11T12:13:36.611062+0000","last_peered":"2024-05-11T12:13:36.611062+0000","last_clean":"2024-05-11T12:13:36.611062+0000","last_became_active":"2024-05-11T12:13:34.542051+0000","last_became_peered":"2024-05-11T12:13:34.542051+0000","last_unstale":"2024-05-11T12:13:36.611062+0000","last_undegraded":"2024-05-11T12:13:36.611062+0000","last_fullsized":"2024-05-11T12:13:36.611062+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:06:50.158921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.570907+0000","last_change":"2024-05-11T12:13:35.559349+0000","last_active":"2024-05-11T12:13:36.570907+0000","last_peered":"2024-05-11T12:13:36.570907+0000","last_clean":"2024-05-11T12:13:36.570907+0000","last_became_active":"2024-05-11T12:13:35.558903+0000","last_became_peered":"2024-05-11T12:13:35.558903+0000","last_unstale":"2024-05-11T12:13:36.570907+0000","last_undegraded":"2024-05-11T12:13:36.570907+0000","last_fullsized":"2024-05-11T12:13:36.570907+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:50:15.033452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"74'2","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.612820+0000","last_change":"2024-05-11T12:13:34.537344+0000","last_active":"2024-05-11T12:13:36.612820+0000","last_peered":"2024-05-11T12:13:36.612820+0000","last_clean":"2024-05-11T12:13:36.612820+0000","last_became_active":"2024-05-11T12:13:34.537010+0000","last_became_peered":"2024-05-11T12:13:34.537010+0000","last_unstale":"2024-05-11T12:13:36.612820+0000","last_undegraded":"2024-05-11T12:13:36.612820+0000","last_fullsized":"2024-05-11T12:13:36.612820+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:14:01.779124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.571021+0000","last_change":"2024-05-11T12:13:35.590045+0000","last_active":"2024-05-11T12:13:36.571021+0000","last_peered":"2024-05-11T12:13:36.571021+0000","last_clean":"2024-05-11T12:13:36.571021+0000","last_became_active":"2024-05-11T12:13:35.589779+0000","last_became_peered":"2024-05-11T12:13:35.589779+0000","last_unstale":"2024-05-11T12:13:36.571021+0000","last_undegraded":"2024-05-11T12:13:36.571021+0000","last_fullsized":"2024-05-11T12:13:36.571021+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:22:46.485528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.611129+0000","last_change":"2024-05-11T12:13:34.543746+0000","last_active":"2024-05-11T12:13:36.611129+0000","last_peered":"2024-05-11T12:13:36.611129+0000","last_clean":"2024-05-11T12:13:36.611129+0000","last_became_active":"2024-05-11T12:13:34.543433+0000","last_became_peered":"2024-05-11T12:13:34.543433+0000","last_unstale":"2024-05-11T12:13:36.611129+0000","last_undegraded":"2024-05-11T12:13:36.611129+0000","last_fullsized":"2024-05-11T12:13:36.611129+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:30:08.648944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.611610+0000","last_change":"2024-05-11T12:13:35.572254+0000","last_active":"2024-05-11T12:13:36.611610+0000","last_peered":"2024-05-11T12:13:36.611610+0000","last_clean":"2024-05-11T12:13:36.611610+0000","last_became_active":"2024-05-11T12:13:35.571618+0000","last_became_peered":"2024-05-11T12:13:35.571618+0000","last_unstale":"2024-05-11T12:13:36.611610+0000","last_undegraded":"2024-05-11T12:13:36.611610+0000","last_fullsized":"2024-05-11T12:13:36.611610+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:39:45.796956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.610505+0000","last_change":"2024-05-11T12:13:34.549729+0000","last_active":"2024-05-11T12:13:36.610505+0000","last_peered":"2024-05-11T12:13:36.610505+0000","last_clean":"2024-05-11T12:13:36.610505+0000","last_became_active":"2024-05-11T12:13:34.549419+0000","last_became_peered":"2024-05-11T12:13:34.549419+0000","last_unstale":"2024-05-11T12:13:36.610505+0000","last_undegraded":"2024-05-11T12:13:36.610505+0000","last_fullsized":"2024-05-11T12:13:36.610505+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:12:12.325701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.574154+0000","last_change":"2024-05-11T12:13:35.581713+0000","last_active":"2024-05-11T12:13:36.574154+0000","last_peered":"2024-05-11T12:13:36.574154+0000","last_clean":"2024-05-11T12:13:36.574154+0000","last_became_active":"2024-05-11T12:13:35.580208+0000","last_became_peered":"2024-05-11T12:13:35.580208+0000","last_unstale":"2024-05-11T12:13:36.574154+0000","last_undegraded":"2024-05-11T12:13:36.574154+0000","last_fullsized":"2024-05-11T12:13:36.574154+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:07:55.024960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"74'2","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.575037+0000","last_change":"2024-05-11T12:13:34.540349+0000","last_active":"2024-05-11T12:13:36.575037+0000","last_peered":"2024-05-11T12:13:36.575037+0000","last_clean":"2024-05-11T12:13:36.575037+0000","last_became_active":"2024-05-11T12:13:34.540020+0000","last_became_peered":"2024-05-11T12:13:34.540020+0000","last_unstale":"2024-05-11T12:13:36.575037+0000","last_undegraded":"2024-05-11T12:13:36.575037+0000","last_fullsized":"2024-05-11T12:13:36.575037+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:45:44.981395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.610049+0000","last_change":"2024-05-11T12:13:35.565377+0000","last_active":"2024-05-11T12:13:36.610049+0000","last_peered":"2024-05-11T12:13:36.610049+0000","last_clean":"2024-05-11T12:13:36.610049+0000","last_became_active":"2024-05-11T12:13:35.565039+0000","last_became_peered":"2024-05-11T12:13:35.565039+0000","last_unstale":"2024-05-11T12:13:36.610049+0000","last_undegraded":"2024-05-11T12:13:36.610049+0000","last_fullsized":"2024-05-11T12:13:36.610049+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:53:41.942115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.571037+0000","last_change":"2024-05-11T12:13:34.541270+0000","last_active":"2024-05-11T12:13:36.571037+0000","last_peered":"2024-05-11T12:13:36.571037+0000","last_clean":"2024-05-11T12:13:36.571037+0000","last_became_active":"2024-05-11T12:13:34.541007+0000","last_became_peered":"2024-05-11T12:13:34.541007+0000","last_unstale":"2024-05-11T12:13:36.571037+0000","last_undegraded":"2024-05-11T12:13:36.571037+0000","last_fullsized":"2024-05-11T12:13:36.571037+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:53:26.823130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.575864+0000","last_change":"2024-05-11T12:13:35.576440+0000","last_active":"2024-05-11T12:13:36.575864+0000","last_peered":"2024-05-11T12:13:36.575864+0000","last_clean":"2024-05-11T12:13:36.575864+0000","last_became_active":"2024-05-11T12:13:35.576150+0000","last_became_peered":"2024-05-11T12:13:35.576150+0000","last_unstale":"2024-05-11T12:13:36.575864+0000","last_undegraded":"2024-05-11T12:13:36.575864+0000","last_fullsized":"2024-05-11T12:13:36.575864+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:39:51.139286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.612659+0000","last_change":"2024-05-11T12:13:34.554691+0000","last_active":"2024-05-11T12:13:36.612659+0000","last_peered":"2024-05-11T12:13:36.612659+0000","last_clean":"2024-05-11T12:13:36.612659+0000","last_became_active":"2024-05-11T12:13:34.554398+0000","last_became_peered":"2024-05-11T12:13:34.554398+0000","last_unstale":"2024-05-11T12:13:36.612659+0000","last_undegraded":"2024-05-11T12:13:36.612659+0000","last_fullsized":"2024-05-11T12:13:36.612659+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:46:16.678499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.570798+0000","last_change":"2024-05-11T12:13:35.576606+0000","last_active":"2024-05-11T12:13:36.570798+0000","last_peered":"2024-05-11T12:13:36.570798+0000","last_clean":"2024-05-11T12:13:36.570798+0000","last_became_active":"2024-05-11T12:13:35.576339+0000","last_became_peered":"2024-05-11T12:13:35.576339+0000","last_unstale":"2024-05-11T12:13:36.570798+0000","last_undegraded":"2024-05-11T12:13:36.570798+0000","last_fullsized":"2024-05-11T12:13:36.570798+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:00:15.439630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.612240+0000","last_change":"2024-05-11T12:13:34.552074+0000","last_active":"2024-05-11T12:13:36.612240+0000","last_peered":"2024-05-11T12:13:36.612240+0000","last_clean":"2024-05-11T12:13:36.612240+0000","last_became_active":"2024-05-11T12:13:34.551763+0000","last_became_peered":"2024-05-11T12:13:34.551763+0000","last_unstale":"2024-05-11T12:13:36.612240+0000","last_undegraded":"2024-05-11T12:13:36.612240+0000","last_fullsized":"2024-05-11T12:13:36.612240+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:39:44.782525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.610222+0000","last_change":"2024-05-11T12:13:35.565177+0000","last_active":"2024-05-11T12:13:36.610222+0000","last_peered":"2024-05-11T12:13:36.610222+0000","last_clean":"2024-05-11T12:13:36.610222+0000","last_became_active":"2024-05-11T12:13:35.563208+0000","last_became_peered":"2024-05-11T12:13:35.563208+0000","last_unstale":"2024-05-11T12:13:36.610222+0000","last_undegraded":"2024-05-11T12:13:36.610222+0000","last_fullsized":"2024-05-11T12:13:36.610222+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:00:39.269552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.610006+0000","last_change":"2024-05-11T12:13:34.546169+0000","last_active":"2024-05-11T12:13:36.610006+0000","last_peered":"2024-05-11T12:13:36.610006+0000","last_clean":"2024-05-11T12:13:36.610006+0000","last_became_active":"2024-05-11T12:13:34.543615+0000","last_became_peered":"2024-05-11T12:13:34.543615+0000","last_unstale":"2024-05-11T12:13:36.610006+0000","last_undegraded":"2024-05-11T12:13:36.610006+0000","last_fullsized":"2024-05-11T12:13:36.610006+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:06:50.158921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.610689+0000","last_change":"2024-05-11T12:13:35.581994+0000","last_active":"2024-05-11T12:13:36.610689+0000","last_peered":"2024-05-11T12:13:36.610689+0000","last_clean":"2024-05-11T12:13:36.610689+0000","last_became_active":"2024-05-11T12:13:35.581533+0000","last_became_peered":"2024-05-11T12:13:35.581533+0000","last_unstale":"2024-05-11T12:13:36.610689+0000","last_undegraded":"2024-05-11T12:13:36.610689+0000","last_fullsized":"2024-05-11T12:13:36.610689+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:45:45.995826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T12:13:35.569081+0000","last_change":"2024-05-11T12:13:34.528956+0000","last_active":"2024-05-11T12:13:35.569081+0000","last_peered":"2024-05-11T12:13:35.569081+0000","last_clean":"2024-05-11T12:13:35.569081+0000","last_became_active":"2024-05-11T12:13:34.527362+0000","last_became_peered":"2024-05-11T12:13:34.527362+0000","last_unstale":"2024-05-11T12:13:35.569081+0000","last_undegraded":"2024-05-11T12:13:35.569081+0000","last_fullsized":"2024-05-11T12:13:35.569081+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:29:44.656678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.612558+0000","last_change":"2024-05-11T12:13:35.571076+0000","last_active":"2024-05-11T12:13:36.612558+0000","last_peered":"2024-05-11T12:13:36.612558+0000","last_clean":"2024-05-11T12:13:36.612558+0000","last_became_active":"2024-05-11T12:13:35.570117+0000","last_became_peered":"2024-05-11T12:13:35.570117+0000","last_unstale":"2024-05-11T12:13:36.612558+0000","last_undegraded":"2024-05-11T12:13:36.612558+0000","last_fullsized":"2024-05-11T12:13:36.612558+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:23:01.813407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.610453+0000","last_change":"2024-05-11T12:13:34.543983+0000","last_active":"2024-05-11T12:13:36.610453+0000","last_peered":"2024-05-11T12:13:36.610453+0000","last_clean":"2024-05-11T12:13:36.610453+0000","last_became_active":"2024-05-11T12:13:34.543600+0000","last_became_peered":"2024-05-11T12:13:34.543600+0000","last_unstale":"2024-05-11T12:13:36.610453+0000","last_undegraded":"2024-05-11T12:13:36.610453+0000","last_fullsized":"2024-05-11T12:13:36.610453+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:47:55.161350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.612447+0000","last_change":"2024-05-11T12:13:35.582866+0000","last_active":"2024-05-11T12:13:36.612447+0000","last_peered":"2024-05-11T12:13:36.612447+0000","last_clean":"2024-05-11T12:13:36.612447+0000","last_became_active":"2024-05-11T12:13:35.582528+0000","last_became_peered":"2024-05-11T12:13:35.582528+0000","last_unstale":"2024-05-11T12:13:36.612447+0000","last_undegraded":"2024-05-11T12:13:36.612447+0000","last_fullsized":"2024-05-11T12:13:36.612447+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:53:27.837561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.609992+0000","last_change":"2024-05-11T12:13:34.544436+0000","last_active":"2024-05-11T12:13:36.609992+0000","last_peered":"2024-05-11T12:13:36.609992+0000","last_clean":"2024-05-11T12:13:36.609992+0000","last_became_active":"2024-05-11T12:13:34.542958+0000","last_became_peered":"2024-05-11T12:13:34.542958+0000","last_unstale":"2024-05-11T12:13:36.609992+0000","last_undegraded":"2024-05-11T12:13:36.609992+0000","last_fullsized":"2024-05-11T12:13:36.609992+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:47:55.161350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.612751+0000","last_change":"2024-05-11T12:13:35.587071+0000","last_active":"2024-05-11T12:13:36.612751+0000","last_peered":"2024-05-11T12:13:36.612751+0000","last_clean":"2024-05-11T12:13:36.612751+0000","last_became_active":"2024-05-11T12:13:35.586766+0000","last_became_peered":"2024-05-11T12:13:35.586766+0000","last_unstale":"2024-05-11T12:13:36.612751+0000","last_undegraded":"2024-05-11T12:13:36.612751+0000","last_fullsized":"2024-05-11T12:13:36.612751+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:00:15.439630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.611886+0000","last_change":"2024-05-11T12:13:34.547212+0000","last_active":"2024-05-11T12:13:36.611886+0000","last_peered":"2024-05-11T12:13:36.611886+0000","last_clean":"2024-05-11T12:13:36.611886+0000","last_became_active":"2024-05-11T12:13:34.546679+0000","last_became_peered":"2024-05-11T12:13:34.546679+0000","last_unstale":"2024-05-11T12:13:36.611886+0000","last_undegraded":"2024-05-11T12:13:36.611886+0000","last_fullsized":"2024-05-11T12:13:36.611886+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:06:32.848036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":206,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.611088+0000","last_change":"2024-05-11T12:10:35.066673+0000","last_active":"2024-05-11T12:13:36.611088+0000","last_peered":"2024-05-11T12:13:36.611088+0000","last_clean":"2024-05-11T12:13:36.611088+0000","last_became_active":"2024-05-11T12:10:35.065942+0000","last_became_peered":"2024-05-11T12:10:35.065942+0000","last_unstale":"2024-05-11T12:13:36.611088+0000","last_undegraded":"2024-05-11T12:13:36.611088+0000","last_fullsized":"2024-05-11T12:13:36.611088+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_clean_scrub_stamp":"2024-05-11T12:10:33.601136+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:12:02.133182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476776,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28492,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343348,"statfs":{"total":96636764160,"available":96607588352,"internally_reserved":0,"allocated":1503232,"data_stored":997562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476777,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27924,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":917504,"data_stored":408784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476776,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27908,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343932,"statfs":{"total":96636764160,"available":96608186368,"internally_reserved":0,"allocated":909312,"data_stored":407206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476778,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":925696,"data_stored":408888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509481,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32068,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339772,"statfs":{"total":96636764160,"available":96603926528,"internally_reserved":0,"allocated":909312,"data_stored":407754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509482,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27908,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343932,"statfs":{"total":96636764160,"available":96608186368,"internally_reserved":0,"allocated":901120,"data_stored":400592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509481,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32652,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339188,"statfs":{"total":96636764160,"available":96603328512,"internally_reserved":0,"allocated":1499136,"data_stored":997540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542185,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27916,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343924,"statfs":{"total":96636764160,"available":96608178176,"internally_reserved":0,"allocated":905216,"data_stored":401222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T12:13:40.010 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T12:13:40.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.247+0000 7faa10c00640 1 Processor -- start 2024-05-11T12:13:40.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.247+0000 7faa10c00640 1 -- start start 2024-05-11T12:13:40.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.249+0000 7faa10c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7faa0c073d50 0x7faa0c0cff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:40.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.249+0000 7faa10c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faa0c0ce9d0 0x7faa0c0b4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:40.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.249+0000 7faa10c00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa0c0ceec0 con 0x7faa0c167080 2024-05-11T12:13:40.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.249+0000 7faa10c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7faa0c0cf1a0 con 0x7faa0c073d50 2024-05-11T12:13:40.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.249+0000 7faa10c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7faa0c0cf480 con 0x7faa0c0ce9d0 2024-05-11T12:13:40.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.249+0000 7faa0be00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7faa0c167080 0x7faa0c072540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:42364/0 (socket says 172.21.15.167:42364) 2024-05-11T12:13:40.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.249+0000 7faa0be00640 1 -- 172.21.15.167:0/4263512204 learned_addr learned my addr 172.21.15.167:0/4263512204 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:13:40.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.250+0000 7faa0aa00640 1 -- 172.21.15.167:0/4263512204 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 521101301 0 0) 0x7faa0c0ceec0 con 0x7faa0c167080 2024-05-11T12:13:40.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.250+0000 7faa0aa00640 1 -- 172.21.15.167:0/4263512204 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa9ec003520 con 0x7faa0c167080 2024-05-11T12:13:40.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.250+0000 7faa03e00640 1 --2- 172.21.15.167:0/4263512204 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faa0c0ce9d0 0x7faa0c0b4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:40.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.250+0000 7faa0b400640 1 --2- 172.21.15.167:0/4263512204 >> v2:172.21.15.204:3300/0 conn(0x7faa0c073d50 0x7faa0c0cff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:40.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.250+0000 7faa0aa00640 1 -- 172.21.15.167:0/4263512204 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3788099239 0 0) 0x7fa9ec003520 con 0x7faa0c167080 2024-05-11T12:13:40.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.250+0000 7faa0aa00640 1 -- 172.21.15.167:0/4263512204 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faa0c0ce9d0 msgr2=0x7faa0c0b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:40.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.250+0000 7faa0aa00640 1 --2- 172.21.15.167:0/4263512204 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faa0c0ce9d0 0x7faa0c0b4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:40.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.250+0000 7faa0aa00640 1 -- 172.21.15.167:0/4263512204 >> v2:172.21.15.204:3300/0 conn(0x7faa0c073d50 msgr2=0x7faa0c0cff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:40.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.250+0000 7faa0aa00640 1 --2- 172.21.15.167:0/4263512204 >> v2:172.21.15.204:3300/0 conn(0x7faa0c073d50 0x7faa0c0cff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:40.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.250+0000 7faa0aa00640 1 -- 172.21.15.167:0/4263512204 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa0c0cf760 con 0x7faa0c167080 2024-05-11T12:13:40.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.250+0000 7faa0aa00640 1 -- 172.21.15.167:0/4263512204 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7faa04001a70 con 0x7faa0c167080 2024-05-11T12:13:40.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.250+0000 7faa0aa00640 1 -- 172.21.15.167:0/4263512204 >> v1:172.21.15.167:6789/0 conn(0x7faa0c167080 legacy=0x7faa0c072540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:40.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.250+0000 7faa0aa00640 1 --2- 172.21.15.167:0/4263512204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9ec003a00 0x7fa9ec003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:40.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.250+0000 7faa0aa00640 1 --2- 172.21.15.167:0/4263512204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9ec004800 0x7fa9ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:40.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.250+0000 7faa0aa00640 1 --2- 172.21.15.167:0/4263512204 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faa0c0ce9d0 0x7fa9ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:40.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.250+0000 7faa0aa00640 1 -- 172.21.15.167:0/4263512204 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faa0c0cf1a0 con 0x7fa9ec003a00 2024-05-11T12:13:40.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.250+0000 7faa0aa00640 1 -- 172.21.15.167:0/4263512204 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7faa0c0cf480 con 0x7fa9ec004800 2024-05-11T12:13:40.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.250+0000 7faa0aa00640 1 -- 172.21.15.167:0/4263512204 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa9ec00bb70 con 0x7faa0c0ce9d0 2024-05-11T12:13:40.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.251+0000 7faa0b400640 1 --2- 172.21.15.167:0/4263512204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9ec003a00 0x7fa9ec003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:40.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.251+0000 7faa03e00640 1 --2- 172.21.15.167:0/4263512204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9ec004800 0x7fa9ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:40.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.251+0000 7faa0b400640 1 -- 172.21.15.167:0/4263512204 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faa0c0ce9d0 msgr2=0x7fa9ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:40.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.251+0000 7faa0b400640 1 --2- 172.21.15.167:0/4263512204 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faa0c0ce9d0 0x7fa9ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:40.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.251+0000 7faa0b400640 1 -- 172.21.15.167:0/4263512204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9ec004800 msgr2=0x7fa9ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:40.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.251+0000 7faa0b400640 1 --2- 172.21.15.167:0/4263512204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9ec004800 0x7fa9ec004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:40.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.251+0000 7faa0b400640 1 -- 172.21.15.167:0/4263512204 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9ec003520 con 0x7fa9ec003a00 2024-05-11T12:13:40.253 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.252+0000 7faa0b400640 1 --2- 172.21.15.167:0/4263512204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9ec003a00 0x7fa9ec003e20 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc00ba30 tx=0x7fa9ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f5ba27d74b0236 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:40.253 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.252+0000 7faa0aa00640 1 -- 172.21.15.167:0/4263512204 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc030070 con 0x7fa9ec003a00 2024-05-11T12:13:40.253 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.252+0000 7faa0aa00640 1 -- 172.21.15.167:0/4263512204 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9fc02ae10 con 0x7fa9ec003a00 2024-05-11T12:13:40.253 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.252+0000 7faa0aa00640 1 -- 172.21.15.167:0/4263512204 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc00b600 con 0x7fa9ec003a00 2024-05-11T12:13:40.253 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.253+0000 7faa10c00640 1 -- 172.21.15.167:0/4263512204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9ec003a00 msgr2=0x7fa9ec003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:40.253 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.253+0000 7faa10c00640 1 --2- 172.21.15.167:0/4263512204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9ec003a00 0x7fa9ec003e20 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc00ba30 tx=0x7fa9ec005830 comp rx=0 tx=0).stop 2024-05-11T12:13:40.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.253+0000 7faa0be00640 1 -- 172.21.15.167:0/4263512204 reap_dead start 2024-05-11T12:13:40.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.253+0000 7faa10c00640 1 -- 172.21.15.167:0/4263512204 shutdown_connections 2024-05-11T12:13:40.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.253+0000 7faa10c00640 1 -- 172.21.15.167:0/4263512204 >> 172.21.15.167:0/4263512204 conn(0x7faa0c0b3460 msgr2=0x7faa0c0d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:40.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.253+0000 7faa10c00640 1 -- 172.21.15.167:0/4263512204 shutdown_connections 2024-05-11T12:13:40.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.253+0000 7faa10c00640 1 -- 172.21.15.167:0/4263512204 wait complete. 2024-05-11T12:13:40.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.255+0000 7faa10c00640 1 Processor -- start 2024-05-11T12:13:40.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.255+0000 7faa10c00640 1 -- start start 2024-05-11T12:13:40.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.256+0000 7faa10c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9ec003a00 0x7faa0c1637f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:40.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.256+0000 7faa10c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa0c0aa410 0x7faa0c163d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:40.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.256+0000 7faa10c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa0c156350 0x7faa0c1f9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:40.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.256+0000 7faa10c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faa0c167530 con 0x7faa0c0aa410 2024-05-11T12:13:40.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.256+0000 7faa10c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7faa0c074130 con 0x7faa0c156350 2024-05-11T12:13:40.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.256+0000 7faa10c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7faa0c1676b0 con 0x7fa9ec003a00 2024-05-11T12:13:40.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.256+0000 7faa03e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa0c0aa410 0x7faa0c163d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:40.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.256+0000 7faa03e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa0c0aa410 0x7faa0c163d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41256/0 (socket says 172.21.15.167:41256) 2024-05-11T12:13:40.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.256+0000 7faa03e00640 1 -- 172.21.15.167:0/685176716 learned_addr learned my addr 172.21.15.167:0/685176716 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:40.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.256+0000 7faa0b400640 1 --2- 172.21.15.167:0/685176716 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9ec003a00 0x7faa0c1637f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:40.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.256+0000 7faa0be00640 1 --2- 172.21.15.167:0/685176716 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa0c156350 0x7faa0c1f9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:40.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.256+0000 7faa03e00640 1 -- 172.21.15.167:0/685176716 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9ec003a00 msgr2=0x7faa0c1637f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:40.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.256+0000 7faa03e00640 1 --2- 172.21.15.167:0/685176716 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9ec003a00 0x7faa0c1637f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:40.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.256+0000 7faa03e00640 1 -- 172.21.15.167:0/685176716 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa0c156350 msgr2=0x7faa0c1f9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:40.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.256+0000 7faa03e00640 1 --2- 172.21.15.167:0/685176716 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa0c156350 0x7faa0c1f9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:40.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.256+0000 7faa03e00640 1 -- 172.21.15.167:0/685176716 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa0c0aa8a0 con 0x7faa0c0aa410 2024-05-11T12:13:40.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.257+0000 7faa03e00640 1 --2- 172.21.15.167:0/685176716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa0c0aa410 0x7faa0c163d30 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fa9f400e7e0 tx=0x7fa9f400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:40.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.257+0000 7faa03400640 1 -- 172.21.15.167:0/685176716 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f4017070 con 0x7faa0c0aa410 2024-05-11T12:13:40.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.257+0000 7faa03400640 1 -- 172.21.15.167:0/685176716 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9f400c6d0 con 0x7faa0c0aa410 2024-05-11T12:13:40.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.257+0000 7faa03400640 1 -- 172.21.15.167:0/685176716 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f400f040 con 0x7faa0c0aa410 2024-05-11T12:13:40.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.257+0000 7faa10c00640 1 -- 172.21.15.167:0/685176716 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa0c0afdd0 con 0x7faa0c0aa410 2024-05-11T12:13:40.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.257+0000 7faa10c00640 1 -- 172.21.15.167:0/685176716 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa0c155980 con 0x7faa0c0aa410 2024-05-11T12:13:40.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.258+0000 7faa10c00640 1 -- 172.21.15.167:0/685176716 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa0c0694b0 con 0x7faa0c0aa410 2024-05-11T12:13:40.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.259+0000 7faa03400640 1 -- 172.21.15.167:0/685176716 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa9f401a070 con 0x7faa0c0aa410 2024-05-11T12:13:40.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.260+0000 7faa03400640 1 --2- 172.21.15.167:0/685176716 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa9e8076300 0x7fa9e80787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:40.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.260+0000 7faa03400640 1 -- 172.21.15.167:0/685176716 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7fa9f4097450 con 0x7faa0c0aa410 2024-05-11T12:13:40.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.261+0000 7faa0b400640 1 --2- 172.21.15.167:0/685176716 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa9e8076300 0x7fa9e80787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:40.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.261+0000 7faa0b400640 1 --2- 172.21.15.167:0/685176716 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa9e8076300 0x7fa9e80787c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc00ba00 tx=0x7fa9fc030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:40.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.265+0000 7faa03400640 1 -- 172.21.15.167:0/685176716 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa9f4060b80 con 0x7faa0c0aa410 2024-05-11T12:13:40.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.454+0000 7faa10c00640 1 -- 172.21.15.167:0/685176716 --> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faa0c164880 con 0x7fa9e8076300 2024-05-11T12:13:40.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.461+0000 7faa03400640 1 -- 172.21.15.167:0/685176716 <== mgr.4106 v2:172.21.15.167:6832/578352559 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252069 (secure 0 0 0) 0x7faa0c164880 con 0x7fa9e8076300 2024-05-11T12:13:40.464 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:13:40.466 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2024-05-11T12:13:40.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.467+0000 7faa10c00640 1 -- 172.21.15.167:0/685176716 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa9e8076300 msgr2=0x7fa9e80787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:40.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.467+0000 7faa10c00640 1 --2- 172.21.15.167:0/685176716 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa9e8076300 0x7fa9e80787c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc00ba00 tx=0x7fa9fc030000 comp rx=0 tx=0).stop 2024-05-11T12:13:40.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.467+0000 7faa10c00640 1 -- 172.21.15.167:0/685176716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa0c0aa410 msgr2=0x7faa0c163d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:40.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.467+0000 7faa10c00640 1 --2- 172.21.15.167:0/685176716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa0c0aa410 0x7faa0c163d30 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fa9f400e7e0 tx=0x7fa9f400eca0 comp rx=0 tx=0).stop 2024-05-11T12:13:40.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.468+0000 7faa10c00640 1 -- 172.21.15.167:0/685176716 shutdown_connections 2024-05-11T12:13:40.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.468+0000 7faa10c00640 1 --2- 172.21.15.167:0/685176716 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa9e8076300 0x7fa9e80787c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:40.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.468+0000 7faa10c00640 1 --2- 172.21.15.167:0/685176716 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa0c156350 0x7faa0c1f9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:40.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.468+0000 7faa10c00640 1 --2- 172.21.15.167:0/685176716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa0c0aa410 0x7faa0c163d30 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:40.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.468+0000 7faa10c00640 1 --2- 172.21.15.167:0/685176716 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9ec003a00 0x7faa0c1637f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:40.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.468+0000 7faa10c00640 1 -- 172.21.15.167:0/685176716 >> 172.21.15.167:0/685176716 conn(0x7faa0c0b3460 msgr2=0x7faa0c0b05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:40.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.468+0000 7faa10c00640 1 -- 172.21.15.167:0/685176716 shutdown_connections 2024-05-11T12:13:40.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.468+0000 7faa10c00640 1 -- 172.21.15.167:0/685176716 wait complete. 2024-05-11T12:13:40.485 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":162,"stamp":"2024-05-11T12:13:39.289103+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":171,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":232800,"kb_used_data":8272,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754741920,"statfs":{"total":773094113280,"available":772855726080,"internally_reserved":0,"allocated":8470528,"data_stored":4429548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":229756465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.785018"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.612718+0000","last_change":"2024-05-11T12:13:35.590959+0000","last_active":"2024-05-11T12:13:36.612718+0000","last_peered":"2024-05-11T12:13:36.612718+0000","last_clean":"2024-05-11T12:13:36.612718+0000","last_became_active":"2024-05-11T12:13:35.590652+0000","last_became_peered":"2024-05-11T12:13:35.590652+0000","last_unstale":"2024-05-11T12:13:36.612718+0000","last_undegraded":"2024-05-11T12:13:36.612718+0000","last_fullsized":"2024-05-11T12:13:36.612718+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:59:10.432575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.570899+0000","last_change":"2024-05-11T12:13:35.579673+0000","last_active":"2024-05-11T12:13:36.570899+0000","last_peered":"2024-05-11T12:13:36.570899+0000","last_clean":"2024-05-11T12:13:36.570899+0000","last_became_active":"2024-05-11T12:13:35.579365+0000","last_became_peered":"2024-05-11T12:13:35.579365+0000","last_unstale":"2024-05-11T12:13:36.570899+0000","last_undegraded":"2024-05-11T12:13:36.570899+0000","last_fullsized":"2024-05-11T12:13:36.570899+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:07:56.484940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.611904+0000","last_change":"2024-05-11T12:13:35.575102+0000","last_active":"2024-05-11T12:13:36.611904+0000","last_peered":"2024-05-11T12:13:36.611904+0000","last_clean":"2024-05-11T12:13:36.611904+0000","last_became_active":"2024-05-11T12:13:35.574842+0000","last_became_peered":"2024-05-11T12:13:35.574842+0000","last_unstale":"2024-05-11T12:13:36.611904+0000","last_undegraded":"2024-05-11T12:13:36.611904+0000","last_fullsized":"2024-05-11T12:13:36.611904+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:53:41.942115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.576746+0000","last_change":"2024-05-11T12:13:35.580885+0000","last_active":"2024-05-11T12:13:36.576746+0000","last_peered":"2024-05-11T12:13:36.576746+0000","last_clean":"2024-05-11T12:13:36.576746+0000","last_became_active":"2024-05-11T12:13:35.580556+0000","last_became_peered":"2024-05-11T12:13:35.580556+0000","last_unstale":"2024-05-11T12:13:36.576746+0000","last_undegraded":"2024-05-11T12:13:36.576746+0000","last_fullsized":"2024-05-11T12:13:36.576746+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:30:09.663375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T12:13:35.591326+0000","last_change":"2024-05-11T12:13:35.584342+0000","last_active":"2024-05-11T12:13:35.591326+0000","last_peered":"2024-05-11T12:13:35.591326+0000","last_clean":"2024-05-11T12:13:35.591326+0000","last_became_active":"2024-05-11T12:13:35.584082+0000","last_became_peered":"2024-05-11T12:13:35.584082+0000","last_unstale":"2024-05-11T12:13:35.591326+0000","last_undegraded":"2024-05-11T12:13:35.591326+0000","last_fullsized":"2024-05-11T12:13:35.591326+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:49:00.732158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.611075+0000","last_change":"2024-05-11T12:13:35.586701+0000","last_active":"2024-05-11T12:13:36.611075+0000","last_peered":"2024-05-11T12:13:36.611075+0000","last_clean":"2024-05-11T12:13:36.611075+0000","last_became_active":"2024-05-11T12:13:35.585360+0000","last_became_peered":"2024-05-11T12:13:35.585360+0000","last_unstale":"2024-05-11T12:13:36.611075+0000","last_undegraded":"2024-05-11T12:13:36.611075+0000","last_fullsized":"2024-05-11T12:13:36.611075+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:07:56.484940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.610245+0000","last_change":"2024-05-11T12:13:35.582888+0000","last_active":"2024-05-11T12:13:36.610245+0000","last_peered":"2024-05-11T12:13:36.610245+0000","last_clean":"2024-05-11T12:13:36.610245+0000","last_became_active":"2024-05-11T12:13:35.581729+0000","last_became_peered":"2024-05-11T12:13:35.581729+0000","last_unstale":"2024-05-11T12:13:36.610245+0000","last_undegraded":"2024-05-11T12:13:36.610245+0000","last_fullsized":"2024-05-11T12:13:36.610245+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:53:27.837561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T12:13:35.587082+0000","last_change":"2024-05-11T12:13:35.578615+0000","last_active":"2024-05-11T12:13:35.587082+0000","last_peered":"2024-05-11T12:13:35.587082+0000","last_clean":"2024-05-11T12:13:35.587082+0000","last_became_active":"2024-05-11T12:13:35.578267+0000","last_became_peered":"2024-05-11T12:13:35.578267+0000","last_unstale":"2024-05-11T12:13:35.587082+0000","last_undegraded":"2024-05-11T12:13:35.587082+0000","last_fullsized":"2024-05-11T12:13:35.587082+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:59:48.432764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.576580+0000","last_change":"2024-05-11T12:13:35.586537+0000","last_active":"2024-05-11T12:13:36.576580+0000","last_peered":"2024-05-11T12:13:36.576580+0000","last_clean":"2024-05-11T12:13:36.576580+0000","last_became_active":"2024-05-11T12:13:35.586257+0000","last_became_peered":"2024-05-11T12:13:35.586257+0000","last_unstale":"2024-05-11T12:13:36.576580+0000","last_undegraded":"2024-05-11T12:13:36.576580+0000","last_fullsized":"2024-05-11T12:13:36.576580+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:56:18.165463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.575915+0000","last_change":"2024-05-11T12:13:35.574940+0000","last_active":"2024-05-11T12:13:36.575915+0000","last_peered":"2024-05-11T12:13:36.575915+0000","last_clean":"2024-05-11T12:13:36.575915+0000","last_became_active":"2024-05-11T12:13:35.574635+0000","last_became_peered":"2024-05-11T12:13:35.574635+0000","last_unstale":"2024-05-11T12:13:36.575915+0000","last_undegraded":"2024-05-11T12:13:36.575915+0000","last_fullsized":"2024-05-11T12:13:36.575915+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:22:57.600314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.571076+0000","last_change":"2024-05-11T12:13:35.580946+0000","last_active":"2024-05-11T12:13:36.571076+0000","last_peered":"2024-05-11T12:13:36.571076+0000","last_clean":"2024-05-11T12:13:36.571076+0000","last_became_active":"2024-05-11T12:13:35.579856+0000","last_became_peered":"2024-05-11T12:13:35.579856+0000","last_unstale":"2024-05-11T12:13:36.571076+0000","last_undegraded":"2024-05-11T12:13:36.571076+0000","last_fullsized":"2024-05-11T12:13:36.571076+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:52:33.630801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.576963+0000","last_change":"2024-05-11T12:13:35.563837+0000","last_active":"2024-05-11T12:13:36.576963+0000","last_peered":"2024-05-11T12:13:36.576963+0000","last_clean":"2024-05-11T12:13:36.576963+0000","last_became_active":"2024-05-11T12:13:35.563138+0000","last_became_peered":"2024-05-11T12:13:35.563138+0000","last_unstale":"2024-05-11T12:13:36.576963+0000","last_undegraded":"2024-05-11T12:13:36.576963+0000","last_fullsized":"2024-05-11T12:13:36.576963+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:14:02.793555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T12:13:35.587129+0000","last_change":"2024-05-11T12:13:35.580551+0000","last_active":"2024-05-11T12:13:35.587129+0000","last_peered":"2024-05-11T12:13:35.587129+0000","last_clean":"2024-05-11T12:13:35.587129+0000","last_became_active":"2024-05-11T12:13:35.578638+0000","last_became_peered":"2024-05-11T12:13:35.578638+0000","last_unstale":"2024-05-11T12:13:35.587129+0000","last_undegraded":"2024-05-11T12:13:35.587129+0000","last_fullsized":"2024-05-11T12:13:35.587129+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:37:48.454996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.611729+0000","last_change":"2024-05-11T12:13:35.585657+0000","last_active":"2024-05-11T12:13:36.611729+0000","last_peered":"2024-05-11T12:13:36.611729+0000","last_clean":"2024-05-11T12:13:36.611729+0000","last_became_active":"2024-05-11T12:13:35.585029+0000","last_became_peered":"2024-05-11T12:13:35.585029+0000","last_unstale":"2024-05-11T12:13:36.611729+0000","last_undegraded":"2024-05-11T12:13:36.611729+0000","last_fullsized":"2024-05-11T12:13:36.611729+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:22:46.485528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.570467+0000","last_change":"2024-05-11T12:13:35.563300+0000","last_active":"2024-05-11T12:13:36.570467+0000","last_peered":"2024-05-11T12:13:36.570467+0000","last_clean":"2024-05-11T12:13:36.570467+0000","last_became_active":"2024-05-11T12:13:35.561620+0000","last_became_peered":"2024-05-11T12:13:35.561620+0000","last_unstale":"2024-05-11T12:13:36.570467+0000","last_undegraded":"2024-05-11T12:13:36.570467+0000","last_fullsized":"2024-05-11T12:13:36.570467+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:16:26.769737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.610983+0000","last_change":"2024-05-11T12:13:35.584780+0000","last_active":"2024-05-11T12:13:36.610983+0000","last_peered":"2024-05-11T12:13:36.610983+0000","last_clean":"2024-05-11T12:13:36.610983+0000","last_became_active":"2024-05-11T12:13:35.583787+0000","last_became_peered":"2024-05-11T12:13:35.583787+0000","last_unstale":"2024-05-11T12:13:36.610983+0000","last_undegraded":"2024-05-11T12:13:36.610983+0000","last_fullsized":"2024-05-11T12:13:36.610983+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:16:26.769737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T12:13:35.590730+0000","last_change":"2024-05-11T12:13:35.583403+0000","last_active":"2024-05-11T12:13:35.590730+0000","last_peered":"2024-05-11T12:13:35.590730+0000","last_clean":"2024-05-11T12:13:35.590730+0000","last_became_active":"2024-05-11T12:13:35.583146+0000","last_became_peered":"2024-05-11T12:13:35.583146+0000","last_unstale":"2024-05-11T12:13:35.590730+0000","last_undegraded":"2024-05-11T12:13:35.590730+0000","last_fullsized":"2024-05-11T12:13:35.590730+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:03:08.848832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.612290+0000","last_change":"2024-05-11T12:13:35.589753+0000","last_active":"2024-05-11T12:13:36.612290+0000","last_peered":"2024-05-11T12:13:36.612290+0000","last_clean":"2024-05-11T12:13:36.612290+0000","last_became_active":"2024-05-11T12:13:35.589349+0000","last_became_peered":"2024-05-11T12:13:35.589349+0000","last_unstale":"2024-05-11T12:13:36.612290+0000","last_undegraded":"2024-05-11T12:13:36.612290+0000","last_fullsized":"2024-05-11T12:13:36.612290+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:00:39.269552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.612161+0000","last_change":"2024-05-11T12:13:35.584149+0000","last_active":"2024-05-11T12:13:36.612161+0000","last_peered":"2024-05-11T12:13:36.612161+0000","last_clean":"2024-05-11T12:13:36.612161+0000","last_became_active":"2024-05-11T12:13:35.583826+0000","last_became_peered":"2024-05-11T12:13:35.583826+0000","last_unstale":"2024-05-11T12:13:36.612161+0000","last_undegraded":"2024-05-11T12:13:36.612161+0000","last_fullsized":"2024-05-11T12:13:36.612161+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:52:33.630801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T12:13:35.585431+0000","last_change":"2024-05-11T12:13:35.574491+0000","last_active":"2024-05-11T12:13:35.585431+0000","last_peered":"2024-05-11T12:13:35.585431+0000","last_clean":"2024-05-11T12:13:35.585431+0000","last_became_active":"2024-05-11T12:13:35.573770+0000","last_became_peered":"2024-05-11T12:13:35.573770+0000","last_unstale":"2024-05-11T12:13:35.585431+0000","last_undegraded":"2024-05-11T12:13:35.585431+0000","last_fullsized":"2024-05-11T12:13:35.585431+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:57:14.890262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.574897+0000","last_change":"2024-05-11T12:13:35.577440+0000","last_active":"2024-05-11T12:13:36.574897+0000","last_peered":"2024-05-11T12:13:36.574897+0000","last_clean":"2024-05-11T12:13:36.574897+0000","last_became_active":"2024-05-11T12:13:35.576906+0000","last_became_peered":"2024-05-11T12:13:35.576906+0000","last_unstale":"2024-05-11T12:13:36.574897+0000","last_undegraded":"2024-05-11T12:13:36.574897+0000","last_fullsized":"2024-05-11T12:13:36.574897+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:47:56.175781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T12:13:35.585495+0000","last_change":"2024-05-11T12:13:35.573224+0000","last_active":"2024-05-11T12:13:35.585495+0000","last_peered":"2024-05-11T12:13:35.585495+0000","last_clean":"2024-05-11T12:13:35.585495+0000","last_became_active":"2024-05-11T12:13:35.572950+0000","last_became_peered":"2024-05-11T12:13:35.572950+0000","last_unstale":"2024-05-11T12:13:35.585495+0000","last_undegraded":"2024-05-11T12:13:35.585495+0000","last_fullsized":"2024-05-11T12:13:35.585495+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:04:54.339323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.574183+0000","last_change":"2024-05-11T12:13:35.578862+0000","last_active":"2024-05-11T12:13:36.574183+0000","last_peered":"2024-05-11T12:13:36.574183+0000","last_clean":"2024-05-11T12:13:36.574183+0000","last_became_active":"2024-05-11T12:13:35.578596+0000","last_became_peered":"2024-05-11T12:13:35.578596+0000","last_unstale":"2024-05-11T12:13:36.574183+0000","last_undegraded":"2024-05-11T12:13:36.574183+0000","last_fullsized":"2024-05-11T12:13:36.574183+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:06:51.173352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.570380+0000","last_change":"2024-05-11T12:13:35.559838+0000","last_active":"2024-05-11T12:13:36.570380+0000","last_peered":"2024-05-11T12:13:36.570380+0000","last_clean":"2024-05-11T12:13:36.570380+0000","last_became_active":"2024-05-11T12:13:35.559521+0000","last_became_peered":"2024-05-11T12:13:35.559521+0000","last_unstale":"2024-05-11T12:13:36.570380+0000","last_undegraded":"2024-05-11T12:13:36.570380+0000","last_fullsized":"2024-05-11T12:13:36.570380+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:48:24.466731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.571121+0000","last_change":"2024-05-11T12:13:35.567028+0000","last_active":"2024-05-11T12:13:36.571121+0000","last_peered":"2024-05-11T12:13:36.571121+0000","last_clean":"2024-05-11T12:13:36.571121+0000","last_became_active":"2024-05-11T12:13:35.566753+0000","last_became_peered":"2024-05-11T12:13:35.566753+0000","last_unstale":"2024-05-11T12:13:36.571121+0000","last_undegraded":"2024-05-11T12:13:36.571121+0000","last_fullsized":"2024-05-11T12:13:36.571121+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:30:40.375149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.576293+0000","last_change":"2024-05-11T12:13:35.566601+0000","last_active":"2024-05-11T12:13:36.576293+0000","last_peered":"2024-05-11T12:13:36.576293+0000","last_clean":"2024-05-11T12:13:36.576293+0000","last_became_active":"2024-05-11T12:13:35.566239+0000","last_became_peered":"2024-05-11T12:13:35.566239+0000","last_unstale":"2024-05-11T12:13:36.576293+0000","last_undegraded":"2024-05-11T12:13:36.576293+0000","last_fullsized":"2024-05-11T12:13:36.576293+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:12:13.340132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.612443+0000","last_change":"2024-05-11T12:13:35.585177+0000","last_active":"2024-05-11T12:13:36.612443+0000","last_peered":"2024-05-11T12:13:36.612443+0000","last_clean":"2024-05-11T12:13:36.612443+0000","last_became_active":"2024-05-11T12:13:35.584719+0000","last_became_peered":"2024-05-11T12:13:35.584719+0000","last_unstale":"2024-05-11T12:13:36.612443+0000","last_undegraded":"2024-05-11T12:13:36.612443+0000","last_fullsized":"2024-05-11T12:13:36.612443+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:14:41.103392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T12:13:35.585455+0000","last_change":"2024-05-11T12:13:35.574715+0000","last_active":"2024-05-11T12:13:35.585455+0000","last_peered":"2024-05-11T12:13:35.585455+0000","last_clean":"2024-05-11T12:13:35.585455+0000","last_became_active":"2024-05-11T12:13:35.574079+0000","last_became_peered":"2024-05-11T12:13:35.574079+0000","last_unstale":"2024-05-11T12:13:35.585455+0000","last_undegraded":"2024-05-11T12:13:35.585455+0000","last_fullsized":"2024-05-11T12:13:35.585455+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:42:31.620827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.570722+0000","last_change":"2024-05-11T12:13:35.563312+0000","last_active":"2024-05-11T12:13:36.570722+0000","last_peered":"2024-05-11T12:13:36.570722+0000","last_clean":"2024-05-11T12:13:36.570722+0000","last_became_active":"2024-05-11T12:13:35.563032+0000","last_became_peered":"2024-05-11T12:13:35.563032+0000","last_unstale":"2024-05-11T12:13:36.570722+0000","last_undegraded":"2024-05-11T12:13:36.570722+0000","last_fullsized":"2024-05-11T12:13:36.570722+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:59:10.432575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.611812+0000","last_change":"2024-05-11T12:13:35.581294+0000","last_active":"2024-05-11T12:13:36.611812+0000","last_peered":"2024-05-11T12:13:36.611812+0000","last_clean":"2024-05-11T12:13:36.611812+0000","last_became_active":"2024-05-11T12:13:35.580366+0000","last_became_peered":"2024-05-11T12:13:35.580366+0000","last_unstale":"2024-05-11T12:13:36.611812+0000","last_undegraded":"2024-05-11T12:13:36.611812+0000","last_fullsized":"2024-05-11T12:13:36.611812+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:15:48.806971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.610102+0000","last_change":"2024-05-11T12:13:35.576683+0000","last_active":"2024-05-11T12:13:36.610102+0000","last_peered":"2024-05-11T12:13:36.610102+0000","last_clean":"2024-05-11T12:13:36.610102+0000","last_became_active":"2024-05-11T12:13:35.576409+0000","last_became_peered":"2024-05-11T12:13:35.576409+0000","last_unstale":"2024-05-11T12:13:36.610102+0000","last_undegraded":"2024-05-11T12:13:36.610102+0000","last_fullsized":"2024-05-11T12:13:36.610102+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:58:11.648486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.610947+0000","last_change":"2024-05-11T12:13:35.590578+0000","last_active":"2024-05-11T12:13:36.610947+0000","last_peered":"2024-05-11T12:13:36.610947+0000","last_clean":"2024-05-11T12:13:36.610947+0000","last_became_active":"2024-05-11T12:13:35.590115+0000","last_became_peered":"2024-05-11T12:13:35.590115+0000","last_unstale":"2024-05-11T12:13:36.610947+0000","last_undegraded":"2024-05-11T12:13:36.610947+0000","last_fullsized":"2024-05-11T12:13:36.610947+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:48:24.466731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.609724+0000","last_change":"2024-05-11T12:13:35.587193+0000","last_active":"2024-05-11T12:13:36.609724+0000","last_peered":"2024-05-11T12:13:36.609724+0000","last_clean":"2024-05-11T12:13:36.609724+0000","last_became_active":"2024-05-11T12:13:35.586683+0000","last_became_peered":"2024-05-11T12:13:35.586683+0000","last_unstale":"2024-05-11T12:13:36.609724+0000","last_undegraded":"2024-05-11T12:13:36.609724+0000","last_fullsized":"2024-05-11T12:13:36.609724+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:11:10.692727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"74'4","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T12:13:35.595247+0000","last_change":"2024-05-11T12:13:34.529830+0000","last_active":"2024-05-11T12:13:35.595247+0000","last_peered":"2024-05-11T12:13:35.595247+0000","last_clean":"2024-05-11T12:13:35.595247+0000","last_became_active":"2024-05-11T12:13:34.529271+0000","last_became_peered":"2024-05-11T12:13:34.529271+0000","last_unstale":"2024-05-11T12:13:35.595247+0000","last_undegraded":"2024-05-11T12:13:35.595247+0000","last_fullsized":"2024-05-11T12:13:35.595247+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:56:13.129398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.570614+0000","last_change":"2024-05-11T12:13:35.557326+0000","last_active":"2024-05-11T12:13:36.570614+0000","last_peered":"2024-05-11T12:13:36.570614+0000","last_clean":"2024-05-11T12:13:36.570614+0000","last_became_active":"2024-05-11T12:13:35.557021+0000","last_became_peered":"2024-05-11T12:13:35.557021+0000","last_unstale":"2024-05-11T12:13:36.570614+0000","last_undegraded":"2024-05-11T12:13:36.570614+0000","last_fullsized":"2024-05-11T12:13:36.570614+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:15:48.806971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"74'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.612602+0000","last_change":"2024-05-11T12:13:34.552817+0000","last_active":"2024-05-11T12:13:36.612602+0000","last_peered":"2024-05-11T12:13:36.612602+0000","last_clean":"2024-05-11T12:13:36.612602+0000","last_became_active":"2024-05-11T12:13:34.551902+0000","last_became_peered":"2024-05-11T12:13:34.551902+0000","last_unstale":"2024-05-11T12:13:36.612602+0000","last_undegraded":"2024-05-11T12:13:36.612602+0000","last_fullsized":"2024-05-11T12:13:36.612602+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:29:20.677448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.575434+0000","last_change":"2024-05-11T12:13:35.588607+0000","last_active":"2024-05-11T12:13:36.575434+0000","last_peered":"2024-05-11T12:13:36.575434+0000","last_clean":"2024-05-11T12:13:36.575434+0000","last_became_active":"2024-05-11T12:13:35.587899+0000","last_became_peered":"2024-05-11T12:13:35.587899+0000","last_unstale":"2024-05-11T12:13:36.575434+0000","last_undegraded":"2024-05-11T12:13:36.575434+0000","last_fullsized":"2024-05-11T12:13:36.575434+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:23:27.409855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"74'2","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.612320+0000","last_change":"2024-05-11T12:13:34.552747+0000","last_active":"2024-05-11T12:13:36.612320+0000","last_peered":"2024-05-11T12:13:36.612320+0000","last_clean":"2024-05-11T12:13:36.612320+0000","last_became_active":"2024-05-11T12:13:34.552384+0000","last_became_peered":"2024-05-11T12:13:34.552384+0000","last_unstale":"2024-05-11T12:13:36.612320+0000","last_undegraded":"2024-05-11T12:13:36.612320+0000","last_fullsized":"2024-05-11T12:13:36.612320+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:48:51.249018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.575536+0000","last_change":"2024-05-11T12:13:35.577190+0000","last_active":"2024-05-11T12:13:36.575536+0000","last_peered":"2024-05-11T12:13:36.575536+0000","last_clean":"2024-05-11T12:13:36.575536+0000","last_became_active":"2024-05-11T12:13:35.575641+0000","last_became_peered":"2024-05-11T12:13:35.575641+0000","last_unstale":"2024-05-11T12:13:36.575536+0000","last_undegraded":"2024-05-11T12:13:36.575536+0000","last_fullsized":"2024-05-11T12:13:36.575536+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:05:56.975495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"74'2","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.610195+0000","last_change":"2024-05-11T12:13:34.546506+0000","last_active":"2024-05-11T12:13:36.610195+0000","last_peered":"2024-05-11T12:13:36.610195+0000","last_clean":"2024-05-11T12:13:36.610195+0000","last_became_active":"2024-05-11T12:13:34.544853+0000","last_became_peered":"2024-05-11T12:13:34.544853+0000","last_unstale":"2024-05-11T12:13:36.610195+0000","last_undegraded":"2024-05-11T12:13:36.610195+0000","last_fullsized":"2024-05-11T12:13:36.610195+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:23:26.395424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.570878+0000","last_change":"2024-05-11T12:13:35.580300+0000","last_active":"2024-05-11T12:13:36.570878+0000","last_peered":"2024-05-11T12:13:36.570878+0000","last_clean":"2024-05-11T12:13:36.570878+0000","last_became_active":"2024-05-11T12:13:35.580028+0000","last_became_peered":"2024-05-11T12:13:35.580028+0000","last_unstale":"2024-05-11T12:13:36.570878+0000","last_undegraded":"2024-05-11T12:13:36.570878+0000","last_fullsized":"2024-05-11T12:13:36.570878+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:25:10.295427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"74'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.575835+0000","last_change":"2024-05-11T12:13:34.543147+0000","last_active":"2024-05-11T12:13:36.575835+0000","last_peered":"2024-05-11T12:13:36.575835+0000","last_clean":"2024-05-11T12:13:36.575835+0000","last_became_active":"2024-05-11T12:13:34.542853+0000","last_became_peered":"2024-05-11T12:13:34.542853+0000","last_unstale":"2024-05-11T12:13:36.575835+0000","last_undegraded":"2024-05-11T12:13:36.575835+0000","last_fullsized":"2024-05-11T12:13:36.575835+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:57:09.184227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.571369+0000","last_change":"2024-05-11T12:13:35.579836+0000","last_active":"2024-05-11T12:13:36.571369+0000","last_peered":"2024-05-11T12:13:36.571369+0000","last_clean":"2024-05-11T12:13:36.571369+0000","last_became_active":"2024-05-11T12:13:35.579438+0000","last_became_peered":"2024-05-11T12:13:35.579438+0000","last_unstale":"2024-05-11T12:13:36.571369+0000","last_undegraded":"2024-05-11T12:13:36.571369+0000","last_fullsized":"2024-05-11T12:13:36.571369+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:02:53.925038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"74'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.574065+0000","last_change":"2024-05-11T12:13:34.527739+0000","last_active":"2024-05-11T12:13:36.574065+0000","last_peered":"2024-05-11T12:13:36.574065+0000","last_clean":"2024-05-11T12:13:36.574065+0000","last_became_active":"2024-05-11T12:13:34.526853+0000","last_became_peered":"2024-05-11T12:13:34.526853+0000","last_unstale":"2024-05-11T12:13:36.574065+0000","last_undegraded":"2024-05-11T12:13:36.574065+0000","last_fullsized":"2024-05-11T12:13:36.574065+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:53:26.823130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.577054+0000","last_change":"2024-05-11T12:13:35.587403+0000","last_active":"2024-05-11T12:13:36.577054+0000","last_peered":"2024-05-11T12:13:36.577054+0000","last_clean":"2024-05-11T12:13:36.577054+0000","last_became_active":"2024-05-11T12:13:35.587125+0000","last_became_peered":"2024-05-11T12:13:35.587125+0000","last_unstale":"2024-05-11T12:13:36.577054+0000","last_undegraded":"2024-05-11T12:13:36.577054+0000","last_fullsized":"2024-05-11T12:13:36.577054+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:58:01.135264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.570561+0000","last_change":"2024-05-11T12:13:34.543098+0000","last_active":"2024-05-11T12:13:36.570561+0000","last_peered":"2024-05-11T12:13:36.570561+0000","last_clean":"2024-05-11T12:13:36.570561+0000","last_became_active":"2024-05-11T12:13:34.542360+0000","last_became_peered":"2024-05-11T12:13:34.542360+0000","last_unstale":"2024-05-11T12:13:36.570561+0000","last_undegraded":"2024-05-11T12:13:36.570561+0000","last_fullsized":"2024-05-11T12:13:36.570561+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:23:00.798976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.611169+0000","last_change":"2024-05-11T12:13:35.589538+0000","last_active":"2024-05-11T12:13:36.611169+0000","last_peered":"2024-05-11T12:13:36.611169+0000","last_clean":"2024-05-11T12:13:36.611169+0000","last_became_active":"2024-05-11T12:13:35.589064+0000","last_became_peered":"2024-05-11T12:13:35.589064+0000","last_unstale":"2024-05-11T12:13:36.611169+0000","last_undegraded":"2024-05-11T12:13:36.611169+0000","last_fullsized":"2024-05-11T12:13:36.611169+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:29:21.691879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T12:13:35.568494+0000","last_change":"2024-05-11T12:13:34.545745+0000","last_active":"2024-05-11T12:13:35.568494+0000","last_peered":"2024-05-11T12:13:35.568494+0000","last_clean":"2024-05-11T12:13:35.568494+0000","last_became_active":"2024-05-11T12:13:34.545455+0000","last_became_peered":"2024-05-11T12:13:34.545455+0000","last_unstale":"2024-05-11T12:13:35.568494+0000","last_undegraded":"2024-05-11T12:13:35.568494+0000","last_fullsized":"2024-05-11T12:13:35.568494+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:39:06.776230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.570642+0000","last_change":"2024-05-11T12:13:35.575951+0000","last_active":"2024-05-11T12:13:36.570642+0000","last_peered":"2024-05-11T12:13:36.570642+0000","last_clean":"2024-05-11T12:13:36.570642+0000","last_became_active":"2024-05-11T12:13:35.575694+0000","last_became_peered":"2024-05-11T12:13:35.575694+0000","last_unstale":"2024-05-11T12:13:36.570642+0000","last_undegraded":"2024-05-11T12:13:36.570642+0000","last_fullsized":"2024-05-11T12:13:36.570642+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:14:41.103392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.613028+0000","last_change":"2024-05-11T12:13:34.553421+0000","last_active":"2024-05-11T12:13:36.613028+0000","last_peered":"2024-05-11T12:13:36.613028+0000","last_clean":"2024-05-11T12:13:36.613028+0000","last_became_active":"2024-05-11T12:13:34.551745+0000","last_became_peered":"2024-05-11T12:13:34.551745+0000","last_unstale":"2024-05-11T12:13:36.613028+0000","last_undegraded":"2024-05-11T12:13:36.613028+0000","last_fullsized":"2024-05-11T12:13:36.613028+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:48:40.873393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.574270+0000","last_change":"2024-05-11T12:13:35.579319+0000","last_active":"2024-05-11T12:13:36.574270+0000","last_peered":"2024-05-11T12:13:36.574270+0000","last_clean":"2024-05-11T12:13:36.574270+0000","last_became_active":"2024-05-11T12:13:35.577954+0000","last_became_peered":"2024-05-11T12:13:35.577954+0000","last_unstale":"2024-05-11T12:13:36.574270+0000","last_undegraded":"2024-05-11T12:13:36.574270+0000","last_fullsized":"2024-05-11T12:13:36.574270+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:42:41.648383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.576562+0000","last_change":"2024-05-11T12:13:34.550391+0000","last_active":"2024-05-11T12:13:36.576562+0000","last_peered":"2024-05-11T12:13:36.576562+0000","last_clean":"2024-05-11T12:13:36.576562+0000","last_became_active":"2024-05-11T12:13:34.549397+0000","last_became_peered":"2024-05-11T12:13:34.549397+0000","last_unstale":"2024-05-11T12:13:36.576562+0000","last_undegraded":"2024-05-11T12:13:36.576562+0000","last_fullsized":"2024-05-11T12:13:36.576562+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:48:40.873393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.611601+0000","last_change":"2024-05-11T12:13:35.563098+0000","last_active":"2024-05-11T12:13:36.611601+0000","last_peered":"2024-05-11T12:13:36.611601+0000","last_clean":"2024-05-11T12:13:36.611601+0000","last_became_active":"2024-05-11T12:13:35.561915+0000","last_became_peered":"2024-05-11T12:13:35.561915+0000","last_unstale":"2024-05-11T12:13:36.611601+0000","last_undegraded":"2024-05-11T12:13:36.611601+0000","last_fullsized":"2024-05-11T12:13:36.611601+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:50:15.033452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"74'3","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T12:13:35.585147+0000","last_change":"2024-05-11T12:13:34.545422+0000","last_active":"2024-05-11T12:13:35.585147+0000","last_peered":"2024-05-11T12:13:35.585147+0000","last_clean":"2024-05-11T12:13:35.585147+0000","last_became_active":"2024-05-11T12:13:34.545097+0000","last_became_peered":"2024-05-11T12:13:34.545097+0000","last_unstale":"2024-05-11T12:13:35.585147+0000","last_undegraded":"2024-05-11T12:13:35.585147+0000","last_fullsized":"2024-05-11T12:13:35.585147+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:51:57.156752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.610640+0000","last_change":"2024-05-11T12:13:35.567723+0000","last_active":"2024-05-11T12:13:36.610640+0000","last_peered":"2024-05-11T12:13:36.610640+0000","last_clean":"2024-05-11T12:13:36.610640+0000","last_became_active":"2024-05-11T12:13:35.567449+0000","last_became_peered":"2024-05-11T12:13:35.567449+0000","last_unstale":"2024-05-11T12:13:36.610640+0000","last_undegraded":"2024-05-11T12:13:36.610640+0000","last_fullsized":"2024-05-11T12:13:36.610640+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:53:59.844302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"74'6","reported_seq":20,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.610500+0000","last_change":"2024-05-11T12:13:34.547145+0000","last_active":"2024-05-11T12:13:36.610500+0000","last_peered":"2024-05-11T12:13:36.610500+0000","last_clean":"2024-05-11T12:13:36.610500+0000","last_became_active":"2024-05-11T12:13:34.546831+0000","last_became_peered":"2024-05-11T12:13:34.546831+0000","last_unstale":"2024-05-11T12:13:36.610500+0000","last_undegraded":"2024-05-11T12:13:36.610500+0000","last_fullsized":"2024-05-11T12:13:36.610500+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:05:55.961064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.577047+0000","last_change":"2024-05-11T12:13:35.559118+0000","last_active":"2024-05-11T12:13:36.577047+0000","last_peered":"2024-05-11T12:13:36.577047+0000","last_clean":"2024-05-11T12:13:36.577047+0000","last_became_active":"2024-05-11T12:13:35.558012+0000","last_became_peered":"2024-05-11T12:13:35.558012+0000","last_unstale":"2024-05-11T12:13:36.577047+0000","last_undegraded":"2024-05-11T12:13:36.577047+0000","last_fullsized":"2024-05-11T12:13:36.577047+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:06:33.862467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"74'2","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.576179+0000","last_change":"2024-05-11T12:13:34.543802+0000","last_active":"2024-05-11T12:13:36.576179+0000","last_peered":"2024-05-11T12:13:36.576179+0000","last_clean":"2024-05-11T12:13:36.576179+0000","last_became_active":"2024-05-11T12:13:34.543498+0000","last_became_peered":"2024-05-11T12:13:34.543498+0000","last_unstale":"2024-05-11T12:13:36.576179+0000","last_undegraded":"2024-05-11T12:13:36.576179+0000","last_fullsized":"2024-05-11T12:13:36.576179+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:23:00.798976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.610590+0000","last_change":"2024-05-11T12:13:35.565054+0000","last_active":"2024-05-11T12:13:36.610590+0000","last_peered":"2024-05-11T12:13:36.610590+0000","last_clean":"2024-05-11T12:13:36.610590+0000","last_became_active":"2024-05-11T12:13:35.564748+0000","last_became_peered":"2024-05-11T12:13:35.564748+0000","last_unstale":"2024-05-11T12:13:36.610590+0000","last_undegraded":"2024-05-11T12:13:36.610590+0000","last_fullsized":"2024-05-11T12:13:36.610590+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:23:01.813407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.611776+0000","last_change":"2024-05-11T12:13:34.553121+0000","last_active":"2024-05-11T12:13:36.611776+0000","last_peered":"2024-05-11T12:13:36.611776+0000","last_clean":"2024-05-11T12:13:36.611776+0000","last_became_active":"2024-05-11T12:13:34.552859+0000","last_became_peered":"2024-05-11T12:13:34.552859+0000","last_unstale":"2024-05-11T12:13:36.611776+0000","last_undegraded":"2024-05-11T12:13:36.611776+0000","last_fullsized":"2024-05-11T12:13:36.611776+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:45:44.981395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.610931+0000","last_change":"2024-05-11T12:13:35.588305+0000","last_active":"2024-05-11T12:13:36.610931+0000","last_peered":"2024-05-11T12:13:36.610931+0000","last_clean":"2024-05-11T12:13:36.610931+0000","last_became_active":"2024-05-11T12:13:35.587031+0000","last_became_peered":"2024-05-11T12:13:35.587031+0000","last_unstale":"2024-05-11T12:13:36.610931+0000","last_undegraded":"2024-05-11T12:13:36.610931+0000","last_fullsized":"2024-05-11T12:13:36.610931+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:48:41.887824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T12:13:35.568788+0000","last_change":"2024-05-11T12:13:34.532223+0000","last_active":"2024-05-11T12:13:35.568788+0000","last_peered":"2024-05-11T12:13:35.568788+0000","last_clean":"2024-05-11T12:13:35.568788+0000","last_became_active":"2024-05-11T12:13:34.531957+0000","last_became_peered":"2024-05-11T12:13:34.531957+0000","last_unstale":"2024-05-11T12:13:35.568788+0000","last_undegraded":"2024-05-11T12:13:35.568788+0000","last_fullsized":"2024-05-11T12:13:35.568788+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:27:21.412461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.611167+0000","last_change":"2024-05-11T12:13:35.573744+0000","last_active":"2024-05-11T12:13:36.611167+0000","last_peered":"2024-05-11T12:13:36.611167+0000","last_clean":"2024-05-11T12:13:36.611167+0000","last_became_active":"2024-05-11T12:13:35.571944+0000","last_became_peered":"2024-05-11T12:13:35.571944+0000","last_unstale":"2024-05-11T12:13:36.611167+0000","last_undegraded":"2024-05-11T12:13:36.611167+0000","last_fullsized":"2024-05-11T12:13:36.611167+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:34:40.157185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"74'8","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.570490+0000","last_change":"2024-05-11T12:13:34.549604+0000","last_active":"2024-05-11T12:13:36.570490+0000","last_peered":"2024-05-11T12:13:36.570490+0000","last_clean":"2024-05-11T12:13:36.570490+0000","last_became_active":"2024-05-11T12:13:34.549283+0000","last_became_peered":"2024-05-11T12:13:34.549283+0000","last_unstale":"2024-05-11T12:13:36.570490+0000","last_undegraded":"2024-05-11T12:13:36.570490+0000","last_fullsized":"2024-05-11T12:13:36.570490+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:45:44.981395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.610687+0000","last_change":"2024-05-11T12:13:35.579632+0000","last_active":"2024-05-11T12:13:36.610687+0000","last_peered":"2024-05-11T12:13:36.610687+0000","last_clean":"2024-05-11T12:13:36.610687+0000","last_became_active":"2024-05-11T12:13:35.579306+0000","last_became_peered":"2024-05-11T12:13:35.579306+0000","last_unstale":"2024-05-11T12:13:36.610687+0000","last_undegraded":"2024-05-11T12:13:36.610687+0000","last_fullsized":"2024-05-11T12:13:36.610687+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:25:10.295427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"74'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.613057+0000","last_change":"2024-05-11T12:13:34.555461+0000","last_active":"2024-05-11T12:13:36.613057+0000","last_peered":"2024-05-11T12:13:36.613057+0000","last_clean":"2024-05-11T12:13:36.613057+0000","last_became_active":"2024-05-11T12:13:34.555021+0000","last_became_peered":"2024-05-11T12:13:34.555021+0000","last_unstale":"2024-05-11T12:13:36.613057+0000","last_undegraded":"2024-05-11T12:13:36.613057+0000","last_fullsized":"2024-05-11T12:13:36.613057+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:34:39.142754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T12:13:35.587147+0000","last_change":"2024-05-11T12:13:35.580694+0000","last_active":"2024-05-11T12:13:35.587147+0000","last_peered":"2024-05-11T12:13:35.587147+0000","last_clean":"2024-05-11T12:13:35.587147+0000","last_became_active":"2024-05-11T12:13:35.580237+0000","last_became_peered":"2024-05-11T12:13:35.580237+0000","last_unstale":"2024-05-11T12:13:35.587147+0000","last_undegraded":"2024-05-11T12:13:35.587147+0000","last_fullsized":"2024-05-11T12:13:35.587147+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:50:57.189033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"74'2","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.577051+0000","last_change":"2024-05-11T12:13:34.544507+0000","last_active":"2024-05-11T12:13:36.577051+0000","last_peered":"2024-05-11T12:13:36.577051+0000","last_clean":"2024-05-11T12:13:36.577051+0000","last_became_active":"2024-05-11T12:13:34.544222+0000","last_became_peered":"2024-05-11T12:13:34.544222+0000","last_unstale":"2024-05-11T12:13:36.577051+0000","last_undegraded":"2024-05-11T12:13:36.577051+0000","last_fullsized":"2024-05-11T12:13:36.577051+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:04:37.397306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T12:13:35.590614+0000","last_change":"2024-05-11T12:13:35.581881+0000","last_active":"2024-05-11T12:13:35.590614+0000","last_peered":"2024-05-11T12:13:35.590614+0000","last_clean":"2024-05-11T12:13:35.590614+0000","last_became_active":"2024-05-11T12:13:35.581591+0000","last_became_peered":"2024-05-11T12:13:35.581591+0000","last_unstale":"2024-05-11T12:13:35.590614+0000","last_undegraded":"2024-05-11T12:13:35.590614+0000","last_fullsized":"2024-05-11T12:13:35.590614+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:20:22.619157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"74'1","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T12:13:35.593646+0000","last_change":"2024-05-11T12:13:34.543092+0000","last_active":"2024-05-11T12:13:35.593646+0000","last_peered":"2024-05-11T12:13:35.593646+0000","last_clean":"2024-05-11T12:13:35.593646+0000","last_became_active":"2024-05-11T12:13:34.542481+0000","last_became_peered":"2024-05-11T12:13:34.542481+0000","last_unstale":"2024-05-11T12:13:35.593646+0000","last_undegraded":"2024-05-11T12:13:35.593646+0000","last_fullsized":"2024-05-11T12:13:35.593646+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:21:50.635577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.612765+0000","last_change":"2024-05-11T12:13:35.588411+0000","last_active":"2024-05-11T12:13:36.612765+0000","last_peered":"2024-05-11T12:13:36.612765+0000","last_clean":"2024-05-11T12:13:36.612765+0000","last_became_active":"2024-05-11T12:13:35.587858+0000","last_became_peered":"2024-05-11T12:13:35.587858+0000","last_unstale":"2024-05-11T12:13:36.612765+0000","last_undegraded":"2024-05-11T12:13:36.612765+0000","last_fullsized":"2024-05-11T12:13:36.612765+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:30:40.375149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.611062+0000","last_change":"2024-05-11T12:13:34.542352+0000","last_active":"2024-05-11T12:13:36.611062+0000","last_peered":"2024-05-11T12:13:36.611062+0000","last_clean":"2024-05-11T12:13:36.611062+0000","last_became_active":"2024-05-11T12:13:34.542051+0000","last_became_peered":"2024-05-11T12:13:34.542051+0000","last_unstale":"2024-05-11T12:13:36.611062+0000","last_undegraded":"2024-05-11T12:13:36.611062+0000","last_fullsized":"2024-05-11T12:13:36.611062+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:06:50.158921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.570907+0000","last_change":"2024-05-11T12:13:35.559349+0000","last_active":"2024-05-11T12:13:36.570907+0000","last_peered":"2024-05-11T12:13:36.570907+0000","last_clean":"2024-05-11T12:13:36.570907+0000","last_became_active":"2024-05-11T12:13:35.558903+0000","last_became_peered":"2024-05-11T12:13:35.558903+0000","last_unstale":"2024-05-11T12:13:36.570907+0000","last_undegraded":"2024-05-11T12:13:36.570907+0000","last_fullsized":"2024-05-11T12:13:36.570907+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:50:15.033452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"74'2","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.612820+0000","last_change":"2024-05-11T12:13:34.537344+0000","last_active":"2024-05-11T12:13:36.612820+0000","last_peered":"2024-05-11T12:13:36.612820+0000","last_clean":"2024-05-11T12:13:36.612820+0000","last_became_active":"2024-05-11T12:13:34.537010+0000","last_became_peered":"2024-05-11T12:13:34.537010+0000","last_unstale":"2024-05-11T12:13:36.612820+0000","last_undegraded":"2024-05-11T12:13:36.612820+0000","last_fullsized":"2024-05-11T12:13:36.612820+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:14:01.779124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.571021+0000","last_change":"2024-05-11T12:13:35.590045+0000","last_active":"2024-05-11T12:13:36.571021+0000","last_peered":"2024-05-11T12:13:36.571021+0000","last_clean":"2024-05-11T12:13:36.571021+0000","last_became_active":"2024-05-11T12:13:35.589779+0000","last_became_peered":"2024-05-11T12:13:35.589779+0000","last_unstale":"2024-05-11T12:13:36.571021+0000","last_undegraded":"2024-05-11T12:13:36.571021+0000","last_fullsized":"2024-05-11T12:13:36.571021+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:22:46.485528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.611129+0000","last_change":"2024-05-11T12:13:34.543746+0000","last_active":"2024-05-11T12:13:36.611129+0000","last_peered":"2024-05-11T12:13:36.611129+0000","last_clean":"2024-05-11T12:13:36.611129+0000","last_became_active":"2024-05-11T12:13:34.543433+0000","last_became_peered":"2024-05-11T12:13:34.543433+0000","last_unstale":"2024-05-11T12:13:36.611129+0000","last_undegraded":"2024-05-11T12:13:36.611129+0000","last_fullsized":"2024-05-11T12:13:36.611129+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:30:08.648944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.611610+0000","last_change":"2024-05-11T12:13:35.572254+0000","last_active":"2024-05-11T12:13:36.611610+0000","last_peered":"2024-05-11T12:13:36.611610+0000","last_clean":"2024-05-11T12:13:36.611610+0000","last_became_active":"2024-05-11T12:13:35.571618+0000","last_became_peered":"2024-05-11T12:13:35.571618+0000","last_unstale":"2024-05-11T12:13:36.611610+0000","last_undegraded":"2024-05-11T12:13:36.611610+0000","last_fullsized":"2024-05-11T12:13:36.611610+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:39:45.796956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.610505+0000","last_change":"2024-05-11T12:13:34.549729+0000","last_active":"2024-05-11T12:13:36.610505+0000","last_peered":"2024-05-11T12:13:36.610505+0000","last_clean":"2024-05-11T12:13:36.610505+0000","last_became_active":"2024-05-11T12:13:34.549419+0000","last_became_peered":"2024-05-11T12:13:34.549419+0000","last_unstale":"2024-05-11T12:13:36.610505+0000","last_undegraded":"2024-05-11T12:13:36.610505+0000","last_fullsized":"2024-05-11T12:13:36.610505+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:12:12.325701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.574154+0000","last_change":"2024-05-11T12:13:35.581713+0000","last_active":"2024-05-11T12:13:36.574154+0000","last_peered":"2024-05-11T12:13:36.574154+0000","last_clean":"2024-05-11T12:13:36.574154+0000","last_became_active":"2024-05-11T12:13:35.580208+0000","last_became_peered":"2024-05-11T12:13:35.580208+0000","last_unstale":"2024-05-11T12:13:36.574154+0000","last_undegraded":"2024-05-11T12:13:36.574154+0000","last_fullsized":"2024-05-11T12:13:36.574154+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:07:55.024960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"74'2","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.575037+0000","last_change":"2024-05-11T12:13:34.540349+0000","last_active":"2024-05-11T12:13:36.575037+0000","last_peered":"2024-05-11T12:13:36.575037+0000","last_clean":"2024-05-11T12:13:36.575037+0000","last_became_active":"2024-05-11T12:13:34.540020+0000","last_became_peered":"2024-05-11T12:13:34.540020+0000","last_unstale":"2024-05-11T12:13:36.575037+0000","last_undegraded":"2024-05-11T12:13:36.575037+0000","last_fullsized":"2024-05-11T12:13:36.575037+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:45:44.981395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.610049+0000","last_change":"2024-05-11T12:13:35.565377+0000","last_active":"2024-05-11T12:13:36.610049+0000","last_peered":"2024-05-11T12:13:36.610049+0000","last_clean":"2024-05-11T12:13:36.610049+0000","last_became_active":"2024-05-11T12:13:35.565039+0000","last_became_peered":"2024-05-11T12:13:35.565039+0000","last_unstale":"2024-05-11T12:13:36.610049+0000","last_undegraded":"2024-05-11T12:13:36.610049+0000","last_fullsized":"2024-05-11T12:13:36.610049+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:53:41.942115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.571037+0000","last_change":"2024-05-11T12:13:34.541270+0000","last_active":"2024-05-11T12:13:36.571037+0000","last_peered":"2024-05-11T12:13:36.571037+0000","last_clean":"2024-05-11T12:13:36.571037+0000","last_became_active":"2024-05-11T12:13:34.541007+0000","last_became_peered":"2024-05-11T12:13:34.541007+0000","last_unstale":"2024-05-11T12:13:36.571037+0000","last_undegraded":"2024-05-11T12:13:36.571037+0000","last_fullsized":"2024-05-11T12:13:36.571037+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:53:26.823130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.575864+0000","last_change":"2024-05-11T12:13:35.576440+0000","last_active":"2024-05-11T12:13:36.575864+0000","last_peered":"2024-05-11T12:13:36.575864+0000","last_clean":"2024-05-11T12:13:36.575864+0000","last_became_active":"2024-05-11T12:13:35.576150+0000","last_became_peered":"2024-05-11T12:13:35.576150+0000","last_unstale":"2024-05-11T12:13:36.575864+0000","last_undegraded":"2024-05-11T12:13:36.575864+0000","last_fullsized":"2024-05-11T12:13:36.575864+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:39:51.139286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.612659+0000","last_change":"2024-05-11T12:13:34.554691+0000","last_active":"2024-05-11T12:13:36.612659+0000","last_peered":"2024-05-11T12:13:36.612659+0000","last_clean":"2024-05-11T12:13:36.612659+0000","last_became_active":"2024-05-11T12:13:34.554398+0000","last_became_peered":"2024-05-11T12:13:34.554398+0000","last_unstale":"2024-05-11T12:13:36.612659+0000","last_undegraded":"2024-05-11T12:13:36.612659+0000","last_fullsized":"2024-05-11T12:13:36.612659+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:46:16.678499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.570798+0000","last_change":"2024-05-11T12:13:35.576606+0000","last_active":"2024-05-11T12:13:36.570798+0000","last_peered":"2024-05-11T12:13:36.570798+0000","last_clean":"2024-05-11T12:13:36.570798+0000","last_became_active":"2024-05-11T12:13:35.576339+0000","last_became_peered":"2024-05-11T12:13:35.576339+0000","last_unstale":"2024-05-11T12:13:36.570798+0000","last_undegraded":"2024-05-11T12:13:36.570798+0000","last_fullsized":"2024-05-11T12:13:36.570798+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:00:15.439630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.612240+0000","last_change":"2024-05-11T12:13:34.552074+0000","last_active":"2024-05-11T12:13:36.612240+0000","last_peered":"2024-05-11T12:13:36.612240+0000","last_clean":"2024-05-11T12:13:36.612240+0000","last_became_active":"2024-05-11T12:13:34.551763+0000","last_became_peered":"2024-05-11T12:13:34.551763+0000","last_unstale":"2024-05-11T12:13:36.612240+0000","last_undegraded":"2024-05-11T12:13:36.612240+0000","last_fullsized":"2024-05-11T12:13:36.612240+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:39:44.782525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.610222+0000","last_change":"2024-05-11T12:13:35.565177+0000","last_active":"2024-05-11T12:13:36.610222+0000","last_peered":"2024-05-11T12:13:36.610222+0000","last_clean":"2024-05-11T12:13:36.610222+0000","last_became_active":"2024-05-11T12:13:35.563208+0000","last_became_peered":"2024-05-11T12:13:35.563208+0000","last_unstale":"2024-05-11T12:13:36.610222+0000","last_undegraded":"2024-05-11T12:13:36.610222+0000","last_fullsized":"2024-05-11T12:13:36.610222+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:00:39.269552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.610006+0000","last_change":"2024-05-11T12:13:34.546169+0000","last_active":"2024-05-11T12:13:36.610006+0000","last_peered":"2024-05-11T12:13:36.610006+0000","last_clean":"2024-05-11T12:13:36.610006+0000","last_became_active":"2024-05-11T12:13:34.543615+0000","last_became_peered":"2024-05-11T12:13:34.543615+0000","last_unstale":"2024-05-11T12:13:36.610006+0000","last_undegraded":"2024-05-11T12:13:36.610006+0000","last_fullsized":"2024-05-11T12:13:36.610006+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:06:50.158921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.610689+0000","last_change":"2024-05-11T12:13:35.581994+0000","last_active":"2024-05-11T12:13:36.610689+0000","last_peered":"2024-05-11T12:13:36.610689+0000","last_clean":"2024-05-11T12:13:36.610689+0000","last_became_active":"2024-05-11T12:13:35.581533+0000","last_became_peered":"2024-05-11T12:13:35.581533+0000","last_unstale":"2024-05-11T12:13:36.610689+0000","last_undegraded":"2024-05-11T12:13:36.610689+0000","last_fullsized":"2024-05-11T12:13:36.610689+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:45:45.995826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T12:13:35.569081+0000","last_change":"2024-05-11T12:13:34.528956+0000","last_active":"2024-05-11T12:13:35.569081+0000","last_peered":"2024-05-11T12:13:35.569081+0000","last_clean":"2024-05-11T12:13:35.569081+0000","last_became_active":"2024-05-11T12:13:34.527362+0000","last_became_peered":"2024-05-11T12:13:34.527362+0000","last_unstale":"2024-05-11T12:13:35.569081+0000","last_undegraded":"2024-05-11T12:13:35.569081+0000","last_fullsized":"2024-05-11T12:13:35.569081+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:29:44.656678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.612558+0000","last_change":"2024-05-11T12:13:35.571076+0000","last_active":"2024-05-11T12:13:36.612558+0000","last_peered":"2024-05-11T12:13:36.612558+0000","last_clean":"2024-05-11T12:13:36.612558+0000","last_became_active":"2024-05-11T12:13:35.570117+0000","last_became_peered":"2024-05-11T12:13:35.570117+0000","last_unstale":"2024-05-11T12:13:36.612558+0000","last_undegraded":"2024-05-11T12:13:36.612558+0000","last_fullsized":"2024-05-11T12:13:36.612558+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:23:01.813407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.610453+0000","last_change":"2024-05-11T12:13:34.543983+0000","last_active":"2024-05-11T12:13:36.610453+0000","last_peered":"2024-05-11T12:13:36.610453+0000","last_clean":"2024-05-11T12:13:36.610453+0000","last_became_active":"2024-05-11T12:13:34.543600+0000","last_became_peered":"2024-05-11T12:13:34.543600+0000","last_unstale":"2024-05-11T12:13:36.610453+0000","last_undegraded":"2024-05-11T12:13:36.610453+0000","last_fullsized":"2024-05-11T12:13:36.610453+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:47:55.161350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.612447+0000","last_change":"2024-05-11T12:13:35.582866+0000","last_active":"2024-05-11T12:13:36.612447+0000","last_peered":"2024-05-11T12:13:36.612447+0000","last_clean":"2024-05-11T12:13:36.612447+0000","last_became_active":"2024-05-11T12:13:35.582528+0000","last_became_peered":"2024-05-11T12:13:35.582528+0000","last_unstale":"2024-05-11T12:13:36.612447+0000","last_undegraded":"2024-05-11T12:13:36.612447+0000","last_fullsized":"2024-05-11T12:13:36.612447+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:53:27.837561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.609992+0000","last_change":"2024-05-11T12:13:34.544436+0000","last_active":"2024-05-11T12:13:36.609992+0000","last_peered":"2024-05-11T12:13:36.609992+0000","last_clean":"2024-05-11T12:13:36.609992+0000","last_became_active":"2024-05-11T12:13:34.542958+0000","last_became_peered":"2024-05-11T12:13:34.542958+0000","last_unstale":"2024-05-11T12:13:36.609992+0000","last_undegraded":"2024-05-11T12:13:36.609992+0000","last_fullsized":"2024-05-11T12:13:36.609992+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:47:55.161350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.612751+0000","last_change":"2024-05-11T12:13:35.587071+0000","last_active":"2024-05-11T12:13:36.612751+0000","last_peered":"2024-05-11T12:13:36.612751+0000","last_clean":"2024-05-11T12:13:36.612751+0000","last_became_active":"2024-05-11T12:13:35.586766+0000","last_became_peered":"2024-05-11T12:13:35.586766+0000","last_unstale":"2024-05-11T12:13:36.612751+0000","last_undegraded":"2024-05-11T12:13:36.612751+0000","last_fullsized":"2024-05-11T12:13:36.612751+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:34.502605+0000","last_clean_scrub_stamp":"2024-05-11T12:13:34.502605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:00:15.439630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.611886+0000","last_change":"2024-05-11T12:13:34.547212+0000","last_active":"2024-05-11T12:13:36.611886+0000","last_peered":"2024-05-11T12:13:36.611886+0000","last_clean":"2024-05-11T12:13:36.611886+0000","last_became_active":"2024-05-11T12:13:34.546679+0000","last_became_peered":"2024-05-11T12:13:34.546679+0000","last_unstale":"2024-05-11T12:13:36.611886+0000","last_undegraded":"2024-05-11T12:13:36.611886+0000","last_fullsized":"2024-05-11T12:13:36.611886+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:13:33.488174+0000","last_clean_scrub_stamp":"2024-05-11T12:13:33.488174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:06:32.848036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":206,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-11T12:13:36.611088+0000","last_change":"2024-05-11T12:10:35.066673+0000","last_active":"2024-05-11T12:13:36.611088+0000","last_peered":"2024-05-11T12:13:36.611088+0000","last_clean":"2024-05-11T12:13:36.611088+0000","last_became_active":"2024-05-11T12:10:35.065942+0000","last_became_peered":"2024-05-11T12:10:35.065942+0000","last_unstale":"2024-05-11T12:13:36.611088+0000","last_undegraded":"2024-05-11T12:13:36.611088+0000","last_fullsized":"2024-05-11T12:13:36.611088+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_clean_scrub_stamp":"2024-05-11T12:10:33.601136+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:12:02.133182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476776,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28492,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343348,"statfs":{"total":96636764160,"available":96607588352,"internally_reserved":0,"allocated":1503232,"data_stored":997562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476777,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27924,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":917504,"data_stored":408784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476776,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27908,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343932,"statfs":{"total":96636764160,"available":96608186368,"internally_reserved":0,"allocated":909312,"data_stored":407206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476778,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":925696,"data_stored":408888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509481,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32068,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339772,"statfs":{"total":96636764160,"available":96603926528,"internally_reserved":0,"allocated":909312,"data_stored":407754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509482,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27908,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343932,"statfs":{"total":96636764160,"available":96608186368,"internally_reserved":0,"allocated":901120,"data_stored":400592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509481,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32652,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339188,"statfs":{"total":96636764160,"available":96603328512,"internally_reserved":0,"allocated":1499136,"data_stored":997540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542185,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27916,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343924,"statfs":{"total":96636764160,"available":96608178176,"internally_reserved":0,"allocated":905216,"data_stored":401222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T12:13:40.489 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-11T12:13:40.489 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-11T12:13:40.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.736+0000 7f8362400640 1 Processor -- start 2024-05-11T12:13:40.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.736+0000 7f8362400640 1 -- start start 2024-05-11T12:13:40.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.738+0000 7f8362400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f835c156120 0x7f835c0d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:40.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.738+0000 7f8362400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f835c0b3b00 0x7f835c06c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:40.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.738+0000 7f8362400640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f835c0cdf40 con 0x7f835c06c990 2024-05-11T12:13:40.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.738+0000 7f8362400640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f835c0d10d0 con 0x7f835c0b3b00 2024-05-11T12:13:40.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.738+0000 7f8362400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f835c15f6c0 con 0x7f835c156120 2024-05-11T12:13:40.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.738+0000 7f8361a00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f835c06c990 0x7f835c072810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:42366/0 (socket says 172.21.15.167:42366) 2024-05-11T12:13:40.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.738+0000 7f8361a00640 1 -- 172.21.15.167:0/2748552618 learned_addr learned my addr 172.21.15.167:0/2748552618 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:13:40.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.738+0000 7f835be00640 1 --2- 172.21.15.167:0/2748552618 >> v2:172.21.15.167:3300/0 conn(0x7f835c0b3b00 0x7f835c06c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:40.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.739+0000 7f8361000640 1 --2- 172.21.15.167:0/2748552618 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f835c156120 0x7f835c0d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:40.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.739+0000 7f835be00640 1 -- 172.21.15.167:0/2748552618 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f835c156120 msgr2=0x7f835c0d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:40.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.739+0000 7f835be00640 1 --2- 172.21.15.167:0/2748552618 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f835c156120 0x7f835c0d0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:40.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.739+0000 7f835b400640 1 -- 172.21.15.167:0/2748552618 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1350158715 0 0) 0x7f835c0cdf40 con 0x7f835c06c990 2024-05-11T12:13:40.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.739+0000 7f835be00640 1 -- 172.21.15.167:0/2748552618 >> v1:172.21.15.167:6789/0 conn(0x7f835c06c990 legacy=0x7f835c072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:40.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.739+0000 7f835be00640 1 -- 172.21.15.167:0/2748552618 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f835c0ce220 con 0x7f835c0b3b00 2024-05-11T12:13:40.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.740+0000 7f835be00640 1 --2- 172.21.15.167:0/2748552618 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f835c0b3b00 0x7f835c06c450 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f83480097a0 tx=0x7f834802a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0ef92473d1da1da server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:40.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.740+0000 7f835b400640 1 -- 172.21.15.167:0/2748552618 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8348037070 con 0x7f835c0b3b00 2024-05-11T12:13:40.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.740+0000 7f835b400640 1 -- 172.21.15.167:0/2748552618 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f834802adb0 con 0x7f835c0b3b00 2024-05-11T12:13:40.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.740+0000 7f835b400640 1 -- 172.21.15.167:0/2748552618 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8348032660 con 0x7f835c0b3b00 2024-05-11T12:13:40.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.741+0000 7f8362400640 1 -- 172.21.15.167:0/2748552618 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f835c0b3b00 msgr2=0x7f835c06c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:40.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.741+0000 7f8362400640 1 --2- 172.21.15.167:0/2748552618 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f835c0b3b00 0x7f835c06c450 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f83480097a0 tx=0x7f834802a060 comp rx=0 tx=0).stop 2024-05-11T12:13:40.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.741+0000 7f8362400640 1 -- 172.21.15.167:0/2748552618 shutdown_connections 2024-05-11T12:13:40.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.741+0000 7f8362400640 1 --2- 172.21.15.167:0/2748552618 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f835c0b3b00 0x7f835c06c450 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:40.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.741+0000 7f8362400640 1 --2- 172.21.15.167:0/2748552618 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f835c156120 0x7f835c0d0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:40.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.741+0000 7f8362400640 1 -- 172.21.15.167:0/2748552618 >> 172.21.15.167:0/2748552618 conn(0x7f835c06ff50 msgr2=0x7f835c070360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:40.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.741+0000 7f8362400640 1 -- 172.21.15.167:0/2748552618 shutdown_connections 2024-05-11T12:13:40.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.741+0000 7f8362400640 1 -- 172.21.15.167:0/2748552618 wait complete. 2024-05-11T12:13:40.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.742+0000 7f8362400640 1 Processor -- start 2024-05-11T12:13:40.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.743+0000 7f8362400640 1 -- start start 2024-05-11T12:13:40.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.743+0000 7f8362400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f835c06c990 0x7f835c144ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:40.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.743+0000 7f8362400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f835c0b3b00 0x7f835c0b49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:40.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.744+0000 7f8362400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f835c156120 0x7f835c150320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:40.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.744+0000 7f8362400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f835c072f20 con 0x7f835c0b3b00 2024-05-11T12:13:40.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.744+0000 7f8362400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f835c0d0dc0 con 0x7f835c156120 2024-05-11T12:13:40.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.744+0000 7f8362400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f835c0730a0 con 0x7f835c06c990 2024-05-11T12:13:40.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.744+0000 7f835be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f835c0b3b00 0x7f835c0b49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:40.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.744+0000 7f835be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f835c0b3b00 0x7f835c0b49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41276/0 (socket says 172.21.15.167:41276) 2024-05-11T12:13:40.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.744+0000 7f835be00640 1 -- 172.21.15.167:0/487642249 learned_addr learned my addr 172.21.15.167:0/487642249 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:40.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.744+0000 7f8361a00640 1 --2- 172.21.15.167:0/487642249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f835c156120 0x7f835c150320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:40.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.744+0000 7f8361000640 1 --2- 172.21.15.167:0/487642249 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f835c06c990 0x7f835c144ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:40.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.744+0000 7f835be00640 1 -- 172.21.15.167:0/487642249 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f835c06c990 msgr2=0x7f835c144ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:40.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.744+0000 7f835be00640 1 --2- 172.21.15.167:0/487642249 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f835c06c990 0x7f835c144ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:40.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.744+0000 7f835be00640 1 -- 172.21.15.167:0/487642249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f835c156120 msgr2=0x7f835c150320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:40.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.744+0000 7f835be00640 1 --2- 172.21.15.167:0/487642249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f835c156120 0x7f835c150320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:40.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.744+0000 7f835be00640 1 -- 172.21.15.167:0/487642249 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f835c145530 con 0x7f835c0b3b00 2024-05-11T12:13:40.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.745+0000 7f835be00640 1 --2- 172.21.15.167:0/487642249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f835c0b3b00 0x7f835c0b49f0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f8348000c00 tx=0x7f83480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:40.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.745+0000 7f8358c00640 1 -- 172.21.15.167:0/487642249 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8348037040 con 0x7f835c0b3b00 2024-05-11T12:13:40.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.745+0000 7f8358c00640 1 -- 172.21.15.167:0/487642249 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8348004250 con 0x7f835c0b3b00 2024-05-11T12:13:40.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.745+0000 7f8362400640 1 -- 172.21.15.167:0/487642249 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f835c1566a0 con 0x7f835c0b3b00 2024-05-11T12:13:40.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.745+0000 7f8358c00640 1 -- 172.21.15.167:0/487642249 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8348003610 con 0x7f835c0b3b00 2024-05-11T12:13:40.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.745+0000 7f8362400640 1 -- 172.21.15.167:0/487642249 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f835c156c60 con 0x7f835c0b3b00 2024-05-11T12:13:40.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.746+0000 7f8362400640 1 -- 172.21.15.167:0/487642249 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f835c0694b0 con 0x7f835c0b3b00 2024-05-11T12:13:40.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.747+0000 7f8358c00640 1 -- 172.21.15.167:0/487642249 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f834802ac70 con 0x7f835c0b3b00 2024-05-11T12:13:40.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.748+0000 7f8358c00640 1 --2- 172.21.15.167:0/487642249 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f833c0762d0 0x7f833c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:40.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.748+0000 7f8358c00640 1 -- 172.21.15.167:0/487642249 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7f83480b8590 con 0x7f835c0b3b00 2024-05-11T12:13:40.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.748+0000 7f8361000640 1 --2- 172.21.15.167:0/487642249 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f833c0762d0 0x7f833c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:40.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.749+0000 7f8361000640 1 --2- 172.21.15.167:0/487642249 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f833c0762d0 0x7f833c078790 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8350006fd0 tx=0x7f8350008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:40.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:40.752+0000 7f8358c00640 1 -- 172.21.15.167:0/487642249 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8348081d50 con 0x7f835c0b3b00 2024-05-11T12:13:41.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.035+0000 7f8362400640 1 -- 172.21.15.167:0/487642249 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f835c0736c0 con 0x7f835c0b3b00 2024-05-11T12:13:41.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.041+0000 7f8358c00640 1 -- 172.21.15.167:0/487642249 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) ==== 190+0+147 (secure 0 0 0) 0x7f8348081650 con 0x7f835c0b3b00 2024-05-11T12:13:41.040 INFO:teuthology.orchestra.run.smithi167.stdout:[client.0] 2024-05-11T12:13:41.041 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQAFYD9mLK/lEhAAyERTYMl250KkBvIrAXWYSw== 2024-05-11T12:13:41.041 INFO:teuthology.orchestra.run.smithi167.stdout: caps mds = "allow" 2024-05-11T12:13:41.041 INFO:teuthology.orchestra.run.smithi167.stdout: caps mon = "allow r" 2024-05-11T12:13:41.041 INFO:teuthology.orchestra.run.smithi167.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-11T12:13:41.041 INFO:teuthology.orchestra.run.smithi167.stderr:updated caps for client.0 2024-05-11T12:13:41.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.043+0000 7f8362400640 1 -- 172.21.15.167:0/487642249 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f833c0762d0 msgr2=0x7f833c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:41.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.044+0000 7f8362400640 1 --2- 172.21.15.167:0/487642249 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f833c0762d0 0x7f833c078790 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8350006fd0 tx=0x7f8350008040 comp rx=0 tx=0).stop 2024-05-11T12:13:41.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.044+0000 7f8362400640 1 -- 172.21.15.167:0/487642249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f835c0b3b00 msgr2=0x7f835c0b49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:41.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.044+0000 7f8362400640 1 --2- 172.21.15.167:0/487642249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f835c0b3b00 0x7f835c0b49f0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f8348000c00 tx=0x7f83480046c0 comp rx=0 tx=0).stop 2024-05-11T12:13:41.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.044+0000 7f8362400640 1 -- 172.21.15.167:0/487642249 shutdown_connections 2024-05-11T12:13:41.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.044+0000 7f8362400640 1 --2- 172.21.15.167:0/487642249 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f833c0762d0 0x7f833c078790 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:41.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.044+0000 7f8362400640 1 --2- 172.21.15.167:0/487642249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f835c156120 0x7f835c150320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:41.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.044+0000 7f8362400640 1 --2- 172.21.15.167:0/487642249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f835c0b3b00 0x7f835c0b49f0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:41.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.044+0000 7f8362400640 1 --2- 172.21.15.167:0/487642249 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f835c06c990 0x7f835c144ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:41.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.044+0000 7f8362400640 1 -- 172.21.15.167:0/487642249 >> 172.21.15.167:0/487642249 conn(0x7f835c06ff50 msgr2=0x7f835c070330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:41.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.044+0000 7f8362400640 1 -- 172.21.15.167:0/487642249 shutdown_connections 2024-05-11T12:13:41.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.044+0000 7f8362400640 1 -- 172.21.15.167:0/487642249 wait complete. 2024-05-11T12:13:41.060 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:13:41.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.256+0000 7f943b400640 1 Processor -- start 2024-05-11T12:13:41.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.256+0000 7f943b400640 1 -- start start 2024-05-11T12:13:41.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.258+0000 7f943b400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9434157750 0x7f943415d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:41.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.258+0000 7f943b400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f94340b3ea0 0x7f9434161490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:41.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.258+0000 7f943b400640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f943415dc80 con 0x7f94341619d0 2024-05-11T12:13:41.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.258+0000 7f943b400640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f9434161000 con 0x7f94340b3ea0 2024-05-11T12:13:41.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.258+0000 7f943b400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f943415df30 con 0x7f9434157750 2024-05-11T12:13:41.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.258+0000 7f9439400640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f94341619d0 0x7f943406d460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:42374/0 (socket says 172.21.15.167:42374) 2024-05-11T12:13:41.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.258+0000 7f9439400640 1 -- 172.21.15.167:0/503355496 learned_addr learned my addr 172.21.15.167:0/503355496 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:13:41.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.258+0000 7f9433e00640 1 --2- 172.21.15.167:0/503355496 >> v2:172.21.15.167:3300/0 conn(0x7f94340b3ea0 0x7f9434161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:41.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.258+0000 7f9438a00640 1 --2- 172.21.15.167:0/503355496 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9434157750 0x7f943415d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:41.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.259+0000 7f9433e00640 1 -- 172.21.15.167:0/503355496 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9434157750 msgr2=0x7f943415d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:41.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.259+0000 7f9433e00640 1 --2- 172.21.15.167:0/503355496 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9434157750 0x7f943415d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:41.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.259+0000 7f9433e00640 1 -- 172.21.15.167:0/503355496 >> v1:172.21.15.167:6789/0 conn(0x7f94341619d0 legacy=0x7f943406d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:41.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.259+0000 7f9433e00640 1 -- 172.21.15.167:0/503355496 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f943415e210 con 0x7f94340b3ea0 2024-05-11T12:13:41.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.260+0000 7f9433e00640 1 --2- 172.21.15.167:0/503355496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94340b3ea0 0x7f9434161490 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f94200097a0 tx=0x7f942002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56e24bc1ea2f8bcc server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:41.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.260+0000 7f9433400640 1 -- 172.21.15.167:0/503355496 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9420037070 con 0x7f94340b3ea0 2024-05-11T12:13:41.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.260+0000 7f9433400640 1 -- 172.21.15.167:0/503355496 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9420032440 con 0x7f94340b3ea0 2024-05-11T12:13:41.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.260+0000 7f9433400640 1 -- 172.21.15.167:0/503355496 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9420032810 con 0x7f94340b3ea0 2024-05-11T12:13:41.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.261+0000 7f943b400640 1 -- 172.21.15.167:0/503355496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94340b3ea0 msgr2=0x7f9434161490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:41.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.261+0000 7f943b400640 1 --2- 172.21.15.167:0/503355496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94340b3ea0 0x7f9434161490 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f94200097a0 tx=0x7f942002a180 comp rx=0 tx=0).stop 2024-05-11T12:13:41.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.261+0000 7f943b400640 1 -- 172.21.15.167:0/503355496 shutdown_connections 2024-05-11T12:13:41.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.261+0000 7f943b400640 1 --2- 172.21.15.167:0/503355496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94340b3ea0 0x7f9434161490 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:41.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.261+0000 7f943b400640 1 --2- 172.21.15.167:0/503355496 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9434157750 0x7f943415d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:41.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.261+0000 7f943b400640 1 -- 172.21.15.167:0/503355496 >> 172.21.15.167:0/503355496 conn(0x7f94340cd8f0 msgr2=0x7f94340cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:41.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.261+0000 7f943b400640 1 -- 172.21.15.167:0/503355496 shutdown_connections 2024-05-11T12:13:41.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.261+0000 7f943b400640 1 -- 172.21.15.167:0/503355496 wait complete. 2024-05-11T12:13:41.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.263+0000 7f943b400640 1 Processor -- start 2024-05-11T12:13:41.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.263+0000 7f943b400640 1 -- start start 2024-05-11T12:13:41.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.264+0000 7f943b400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94340b3ea0 0x7f9434157d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:41.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.264+0000 7f943b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9434157750 0x7f94341456c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:41.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.264+0000 7f943b400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94341619d0 0x7f943413fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:41.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.264+0000 7f943b400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f943406db70 con 0x7f9434157750 2024-05-11T12:13:41.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.264+0000 7f943b400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9434157b30 con 0x7f94340b3ea0 2024-05-11T12:13:41.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.264+0000 7f943b400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f943406dcf0 con 0x7f94341619d0 2024-05-11T12:13:41.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.264+0000 7f9433e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9434157750 0x7f94341456c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:41.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.264+0000 7f9433e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9434157750 0x7f94341456c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41286/0 (socket says 172.21.15.167:41286) 2024-05-11T12:13:41.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.264+0000 7f9433e00640 1 -- 172.21.15.167:0/2092975621 learned_addr learned my addr 172.21.15.167:0/2092975621 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:41.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.265+0000 7f9439400640 1 --2- 172.21.15.167:0/2092975621 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94341619d0 0x7f943413fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:41.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.265+0000 7f9433e00640 1 -- 172.21.15.167:0/2092975621 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94341619d0 msgr2=0x7f943413fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:41.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.265+0000 7f9438a00640 1 --2- 172.21.15.167:0/2092975621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94340b3ea0 0x7f9434157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:41.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.265+0000 7f9433e00640 1 --2- 172.21.15.167:0/2092975621 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94341619d0 0x7f943413fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:41.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.265+0000 7f9433e00640 1 -- 172.21.15.167:0/2092975621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94340b3ea0 msgr2=0x7f9434157d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:41.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.265+0000 7f9433e00640 1 --2- 172.21.15.167:0/2092975621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94340b3ea0 0x7f9434157d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:41.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.265+0000 7f9433e00640 1 -- 172.21.15.167:0/2092975621 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94340723f0 con 0x7f9434157750 2024-05-11T12:13:41.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.265+0000 7f9433e00640 1 --2- 172.21.15.167:0/2092975621 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9434157750 0x7f94341456c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f9420035040 tx=0x7f94200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:41.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.265+0000 7f9430c00640 1 -- 172.21.15.167:0/2092975621 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9420037040 con 0x7f9434157750 2024-05-11T12:13:41.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.265+0000 7f9430c00640 1 -- 172.21.15.167:0/2092975621 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9420030770 con 0x7f9434157750 2024-05-11T12:13:41.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.265+0000 7f943b400640 1 -- 172.21.15.167:0/2092975621 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94340726d0 con 0x7f9434157750 2024-05-11T12:13:41.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.265+0000 7f9430c00640 1 -- 172.21.15.167:0/2092975621 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9420030b80 con 0x7f9434157750 2024-05-11T12:13:41.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.265+0000 7f943b400640 1 -- 172.21.15.167:0/2092975621 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9434072c90 con 0x7f9434157750 2024-05-11T12:13:41.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.266+0000 7f943b400640 1 -- 172.21.15.167:0/2092975621 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f943406feb0 con 0x7f9434157750 2024-05-11T12:13:41.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.268+0000 7f9430c00640 1 -- 172.21.15.167:0/2092975621 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9420044020 con 0x7f9434157750 2024-05-11T12:13:41.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.269+0000 7f9430c00640 1 --2- 172.21.15.167:0/2092975621 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f94140762d0 0x7f9414078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:41.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.269+0000 7f9430c00640 1 -- 172.21.15.167:0/2092975621 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7f94200b7d30 con 0x7f9434157750 2024-05-11T12:13:41.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.269+0000 7f9438a00640 1 --2- 172.21.15.167:0/2092975621 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f94140762d0 0x7f9414078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:41.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.270+0000 7f9438a00640 1 --2- 172.21.15.167:0/2092975621 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f94140762d0 0x7f9414078790 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f9428006fd0 tx=0x7f9428008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:41.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.273+0000 7f9430c00640 1 -- 172.21.15.167:0/2092975621 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f943406feb0 con 0x7f9434157750 2024-05-11T12:13:41.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.522+0000 7f943b400640 1 -- 172.21.15.167:0/2092975621 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9434069590 con 0x7f9434157750 2024-05-11T12:13:41.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.523+0000 7f9430c00640 1 -- 172.21.15.167:0/2092975621 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5195 (secure 0 0 0) 0x7f9420032e20 con 0x7f9434157750 2024-05-11T12:13:41.522 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:13:41.522 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5020,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3075930607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3075930607},{"type":"v1","addr":"172.21.15.167:6836","nonce":3075930607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":5032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3056051531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3056051531},{"type":"v1","addr":"172.21.15.204:6834","nonce":3056051531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5035,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2900478909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":2900478909},{"type":"v1","addr":"172.21.15.204:6835","nonce":2900478909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:13:34.993949+0000","modified":"2024-05-11T12:13:38.072880+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5952},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5952":{"gid":5952,"name":"a","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/1306616214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1306616214},{"type":"v1","addr":"172.21.15.167:6837","nonce":1306616214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5952,"qdb_cluster":[5952]},"id":5}]} 2024-05-11T12:13:41.523 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 44 2024-05-11T12:13:41.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.526+0000 7f943b400640 1 -- 172.21.15.167:0/2092975621 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f94140762d0 msgr2=0x7f9414078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:41.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.526+0000 7f943b400640 1 --2- 172.21.15.167:0/2092975621 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f94140762d0 0x7f9414078790 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f9428006fd0 tx=0x7f9428008040 comp rx=0 tx=0).stop 2024-05-11T12:13:41.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.526+0000 7f943b400640 1 -- 172.21.15.167:0/2092975621 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9434157750 msgr2=0x7f94341456c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:41.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.526+0000 7f943b400640 1 --2- 172.21.15.167:0/2092975621 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9434157750 0x7f94341456c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f9420035040 tx=0x7f94200046c0 comp rx=0 tx=0).stop 2024-05-11T12:13:41.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.526+0000 7f943b400640 1 -- 172.21.15.167:0/2092975621 shutdown_connections 2024-05-11T12:13:41.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.526+0000 7f943b400640 1 --2- 172.21.15.167:0/2092975621 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94341619d0 0x7f943413fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:41.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.526+0000 7f943b400640 1 --2- 172.21.15.167:0/2092975621 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9434157750 0x7f94341456c0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:41.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.526+0000 7f943b400640 1 --2- 172.21.15.167:0/2092975621 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f94140762d0 0x7f9414078790 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:41.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.526+0000 7f943b400640 1 --2- 172.21.15.167:0/2092975621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94340b3ea0 0x7f9434157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:41.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.526+0000 7f943b400640 1 -- 172.21.15.167:0/2092975621 >> 172.21.15.167:0/2092975621 conn(0x7f94340cd8f0 msgr2=0x7f94340cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:41.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.526+0000 7f943b400640 1 -- 172.21.15.167:0/2092975621 shutdown_connections 2024-05-11T12:13:41.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:41.526+0000 7f943b400640 1 -- 172.21.15.167:0/2092975621 wait complete. 2024-05-11T12:13:41.947 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 44, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-11T12:13:34.993949+0000', 'modified': '2024-05-11T12:13:38.072880+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5952}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5952': {'gid': 5952, 'name': 'a', 'rank': 0, 'incarnation': 40, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.167:6837/1306616214', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6835', 'nonce': 1306616214}, {'type': 'v1', 'addr': '172.21.15.167:6837', 'nonce': 1306616214}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5952, 'qdb_cluster': [5952]} 2024-05-11T12:13:41.947 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-11T12:13:41.947 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-11T12:13:42.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.158+0000 7f00aee00640 1 Processor -- start 2024-05-11T12:13:42.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.158+0000 7f00aee00640 1 -- start start 2024-05-11T12:13:42.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.160+0000 7f00aee00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f00a8163980 0x7f00a80b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:42.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.160+0000 7f00aee00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00a80b6020 con 0x7f00a815ef40 2024-05-11T12:13:42.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.160+0000 7f00aee00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00a80b6460 con 0x7f00a815f320 2024-05-11T12:13:42.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.160+0000 7f00aee00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f00a80b6740 con 0x7f00a8163980 2024-05-11T12:13:42.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.161+0000 7f00ada00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f00a815ef40 0x7f00a8071fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:42382/0 (socket says 172.21.15.167:42382) 2024-05-11T12:13:42.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.161+0000 7f00ada00640 1 -- 172.21.15.167:0/2905600900 learned_addr learned my addr 172.21.15.167:0/2905600900 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:13:42.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.161+0000 7f00ad000640 1 --2- 172.21.15.167:0/2905600900 >> v2:172.21.15.204:3300/0 conn(0x7f00a8163980 0x7f00a80b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:42.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.161+0000 7f00a7e00640 1 -- 172.21.15.167:0/2905600900 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2981218677 0 0) 0x7f00a80b6020 con 0x7f00a815ef40 2024-05-11T12:13:42.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.161+0000 7f00a7e00640 1 -- 172.21.15.167:0/2905600900 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0090003520 con 0x7f00a815ef40 2024-05-11T12:13:42.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.162+0000 7f00a7e00640 1 -- 172.21.15.167:0/2905600900 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 203398667 0 0) 0x7f0090003520 con 0x7f00a815ef40 2024-05-11T12:13:42.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.162+0000 7f00a7e00640 1 -- 172.21.15.167:0/2905600900 >> v2:172.21.15.204:3300/0 conn(0x7f00a8163980 msgr2=0x7f00a80b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:42.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.162+0000 7f00a7e00640 1 --2- 172.21.15.167:0/2905600900 >> v2:172.21.15.204:3300/0 conn(0x7f00a8163980 0x7f00a80b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:42.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.162+0000 7f00a7e00640 1 -- 172.21.15.167:0/2905600900 >> v1:172.21.15.204:6789/0 conn(0x7f00a815f320 legacy=0x7f00a8162d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:42.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.162+0000 7f00a7e00640 1 -- 172.21.15.167:0/2905600900 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00a80afb90 con 0x7f00a815ef40 2024-05-11T12:13:42.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.162+0000 7f00a7e00640 1 -- 172.21.15.167:0/2905600900 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f009c0016f0 con 0x7f00a815ef40 2024-05-11T12:13:42.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.162+0000 7f00a7e00640 1 -- 172.21.15.167:0/2905600900 >> v1:172.21.15.167:6789/0 conn(0x7f00a815ef40 legacy=0x7f00a8071fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:42.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.162+0000 7f00a7e00640 1 --2- 172.21.15.167:0/2905600900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0090003ac0 0x7f0090003ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:42.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.162+0000 7f00a7e00640 1 --2- 172.21.15.167:0/2905600900 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0090004800 0x7f0090004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:42.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.162+0000 7f00a7e00640 1 --2- 172.21.15.167:0/2905600900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0090005230 0x7f009000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:42.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.162+0000 7f00a7e00640 1 -- 172.21.15.167:0/2905600900 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f00a80b6740 con 0x7f0090003ac0 2024-05-11T12:13:42.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.162+0000 7f00a7e00640 1 -- 172.21.15.167:0/2905600900 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f009000c190 con 0x7f0090005230 2024-05-11T12:13:42.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.162+0000 7f00a7e00640 1 -- 172.21.15.167:0/2905600900 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f009000c390 con 0x7f0090004800 2024-05-11T12:13:42.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.162+0000 7f00ada00640 1 --2- 172.21.15.167:0/2905600900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0090003ac0 0x7f0090003ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:42.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.162+0000 7f00ad000640 1 --2- 172.21.15.167:0/2905600900 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0090004800 0x7f0090004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:42.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.162+0000 7f00ada00640 1 -- 172.21.15.167:0/2905600900 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0090004800 msgr2=0x7f0090004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:42.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.162+0000 7f00ada00640 1 --2- 172.21.15.167:0/2905600900 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0090004800 0x7f0090004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:42.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.162+0000 7f00ada00640 1 -- 172.21.15.167:0/2905600900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0090005230 msgr2=0x7f009000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:42.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.162+0000 7f00ada00640 1 --2- 172.21.15.167:0/2905600900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0090005230 0x7f009000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:42.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.163+0000 7f00ada00640 1 -- 172.21.15.167:0/2905600900 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0090003520 con 0x7f0090003ac0 2024-05-11T12:13:42.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.163+0000 7f00ae400640 1 -- 172.21.15.167:0/2905600900 reap_dead start 2024-05-11T12:13:42.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.163+0000 7f00ad000640 1 --2- 172.21.15.167:0/2905600900 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0090004800 0x7f0090004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:42.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.164+0000 7f00ada00640 1 --2- 172.21.15.167:0/2905600900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0090003ac0 0x7f0090003ee0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f009c00c240 tx=0x7f0090005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee305ca89968c8aa server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:42.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.164+0000 7f00a7e00640 1 -- 172.21.15.167:0/2905600900 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009c007d70 con 0x7f0090003ac0 2024-05-11T12:13:42.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.164+0000 7f00a7e00640 1 -- 172.21.15.167:0/2905600900 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f009c005ad0 con 0x7f0090003ac0 2024-05-11T12:13:42.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.164+0000 7f00a7e00640 1 -- 172.21.15.167:0/2905600900 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009c007070 con 0x7f0090003ac0 2024-05-11T12:13:42.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.164+0000 7f00aee00640 1 -- 172.21.15.167:0/2905600900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0090003ac0 msgr2=0x7f0090003ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:42.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.164+0000 7f00aee00640 1 --2- 172.21.15.167:0/2905600900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0090003ac0 0x7f0090003ee0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f009c00c240 tx=0x7f0090005830 comp rx=0 tx=0).stop 2024-05-11T12:13:42.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.165+0000 7f00aee00640 1 -- 172.21.15.167:0/2905600900 shutdown_connections 2024-05-11T12:13:42.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.165+0000 7f00aee00640 1 --2- 172.21.15.167:0/2905600900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0090003ac0 0x7f0090003ee0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:42.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.165+0000 7f00aee00640 1 -- 172.21.15.167:0/2905600900 >> 172.21.15.167:0/2905600900 conn(0x7f00a806c450 msgr2=0x7f00a806c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:42.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.165+0000 7f00aee00640 1 -- 172.21.15.167:0/2905600900 shutdown_connections 2024-05-11T12:13:42.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.165+0000 7f00aee00640 1 -- 172.21.15.167:0/2905600900 wait complete. 2024-05-11T12:13:42.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.166+0000 7f00aee00640 1 Processor -- start 2024-05-11T12:13:42.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.167+0000 7f00aee00640 1 -- start start 2024-05-11T12:13:42.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.168+0000 7f00aee00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0090003ac0 0x7f00a813ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:42.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.168+0000 7f00aee00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00a8072720 0x7f00a8072b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:42.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.168+0000 7f00aee00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00a8142a00 0x7f00a8142e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:42.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.168+0000 7f00aee00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f00a8072120 con 0x7f00a8072720 2024-05-11T12:13:42.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.168+0000 7f00aee00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f00a8071fa0 con 0x7f0090003ac0 2024-05-11T12:13:42.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.168+0000 7f00aee00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f00a80722a0 con 0x7f00a8142a00 2024-05-11T12:13:42.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.168+0000 7f00ad000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00a8072720 0x7f00a8072b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:42.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.168+0000 7f00ad000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00a8072720 0x7f00a8072b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41300/0 (socket says 172.21.15.167:41300) 2024-05-11T12:13:42.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.168+0000 7f00ad000640 1 -- 172.21.15.167:0/2415904861 learned_addr learned my addr 172.21.15.167:0/2415904861 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:42.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.168+0000 7f00ae400640 1 --2- 172.21.15.167:0/2415904861 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00a8142a00 0x7f00a8142e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:42.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.168+0000 7f00ad000640 1 -- 172.21.15.167:0/2415904861 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00a8142a00 msgr2=0x7f00a8142e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:42.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.168+0000 7f00ada00640 1 --2- 172.21.15.167:0/2415904861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0090003ac0 0x7f00a813ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:42.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.168+0000 7f00ad000640 1 --2- 172.21.15.167:0/2415904861 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00a8142a00 0x7f00a8142e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:42.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.168+0000 7f00ad000640 1 -- 172.21.15.167:0/2415904861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0090003ac0 msgr2=0x7f00a813ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:42.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.168+0000 7f00ad000640 1 --2- 172.21.15.167:0/2415904861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0090003ac0 0x7f00a813ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:42.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.168+0000 7f00ad000640 1 -- 172.21.15.167:0/2415904861 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00a81436f0 con 0x7f00a8072720 2024-05-11T12:13:42.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.169+0000 7f00ad000640 1 --2- 172.21.15.167:0/2415904861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00a8072720 0x7f00a8072b80 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f00940107c0 tx=0x7f0094010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:42.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.169+0000 7f00a5600640 1 -- 172.21.15.167:0/2415904861 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009400f040 con 0x7f00a8072720 2024-05-11T12:13:42.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.169+0000 7f00a5600640 1 -- 172.21.15.167:0/2415904861 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f009400eb00 con 0x7f00a8072720 2024-05-11T12:13:42.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.169+0000 7f00a5600640 1 -- 172.21.15.167:0/2415904861 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0094010e70 con 0x7f00a8072720 2024-05-11T12:13:42.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.169+0000 7f00aee00640 1 -- 172.21.15.167:0/2415904861 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00a8140af0 con 0x7f00a8072720 2024-05-11T12:13:42.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.169+0000 7f00aee00640 1 -- 172.21.15.167:0/2415904861 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00a81410b0 con 0x7f00a8072720 2024-05-11T12:13:42.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.172+0000 7f00a5600640 1 -- 172.21.15.167:0/2415904861 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f009401c400 con 0x7f00a8072720 2024-05-11T12:13:42.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.173+0000 7f00a5600640 1 --2- 172.21.15.167:0/2415904861 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0088076300 0x7f00880787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:42.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.173+0000 7f00a5600640 1 -- 172.21.15.167:0/2415904861 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7f0094097c60 con 0x7f00a8072720 2024-05-11T12:13:42.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.173+0000 7f00ada00640 1 --2- 172.21.15.167:0/2415904861 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0088076300 0x7f00880787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:42.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.173+0000 7f00b33ef740 1 Processor -- start 2024-05-11T12:13:42.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.173+0000 7f00b33ef740 1 -- start start 2024-05-11T12:13:42.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.174+0000 7f00ada00640 1 --2- 172.21.15.167:0/2415904861 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0088076300 0x7f00880787c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f009c032000 tx=0x7f009c009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:42.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.174+0000 7f00b33ef740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x56320a0b8a10 0x56320a0b8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:42.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.174+0000 7f00b33ef740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56320a0b38a0 0x56320a0b7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:42.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.174+0000 7f00b33ef740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56320a0b7b50 0x56320a0b7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:42.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.174+0000 7f00b33ef740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56320a01c790 con 0x56320a0b38a0 2024-05-11T12:13:42.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.174+0000 7f00b33ef740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x563209fc1f20 con 0x56320a0b7b50 2024-05-11T12:13:42.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.174+0000 7f00b33ef740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x56320a018750 con 0x56320a0b8a10 2024-05-11T12:13:42.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.175+0000 7f00ad000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56320a0b38a0 0x56320a0b7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:42.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.175+0000 7f00ad000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56320a0b38a0 0x56320a0b7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41316/0 (socket says 172.21.15.167:41316) 2024-05-11T12:13:42.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.175+0000 7f00ad000640 1 -- 172.21.15.167:0/3776421531 learned_addr learned my addr 172.21.15.167:0/3776421531 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:42.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.175+0000 7f00ae400640 1 --2- 172.21.15.167:0/3776421531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56320a0b7b50 0x56320a0b7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:42.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.175+0000 7f00ada00640 1 --2- 172.21.15.167:0/3776421531 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x56320a0b8a10 0x56320a0b8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:42.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.175+0000 7f00ad000640 1 -- 172.21.15.167:0/3776421531 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x56320a0b8a10 msgr2=0x56320a0b8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:42.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.175+0000 7f00ad000640 1 --2- 172.21.15.167:0/3776421531 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x56320a0b8a10 0x56320a0b8df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:42.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.175+0000 7f00ad000640 1 -- 172.21.15.167:0/3776421531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56320a0b7b50 msgr2=0x56320a0b7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:42.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.175+0000 7f00ad000640 1 --2- 172.21.15.167:0/3776421531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56320a0b7b50 0x56320a0b7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:42.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.175+0000 7f00ad000640 1 -- 172.21.15.167:0/3776421531 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563209d2e2d0 con 0x56320a0b38a0 2024-05-11T12:13:42.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.175+0000 7f00ae400640 1 --2- 172.21.15.167:0/3776421531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56320a0b7b50 0x56320a0b7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:42.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.175+0000 7f00ad000640 1 --2- 172.21.15.167:0/3776421531 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56320a0b38a0 0x56320a0b7610 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f009405d280 tx=0x7f0094060860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12542eb7603dbbff server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:42.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.176+0000 7f009a000640 1 -- 172.21.15.167:0/3776421531 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0094021040 con 0x56320a0b38a0 2024-05-11T12:13:42.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.176+0000 7f009a000640 1 -- 172.21.15.167:0/3776421531 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00940693b0 con 0x56320a0b38a0 2024-05-11T12:13:42.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.176+0000 7f009a000640 1 -- 172.21.15.167:0/3776421531 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0094065040 con 0x56320a0b38a0 2024-05-11T12:13:42.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.176+0000 7f00b33ef740 1 -- 172.21.15.167:0/3776421531 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56320a0b38a0 msgr2=0x56320a0b7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:42.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.176+0000 7f00b33ef740 1 --2- 172.21.15.167:0/3776421531 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56320a0b38a0 0x56320a0b7610 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f009405d280 tx=0x7f0094060860 comp rx=0 tx=0).stop 2024-05-11T12:13:42.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.176+0000 7f00b33ef740 1 -- 172.21.15.167:0/3776421531 shutdown_connections 2024-05-11T12:13:42.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.176+0000 7f00b33ef740 1 --2- 172.21.15.167:0/3776421531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56320a0b7b50 0x56320a0b7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:42.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.176+0000 7f00b33ef740 1 --2- 172.21.15.167:0/3776421531 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56320a0b38a0 0x56320a0b7610 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:42.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.176+0000 7f00b33ef740 1 --2- 172.21.15.167:0/3776421531 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x56320a0b8a10 0x56320a0b8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:42.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.176+0000 7f00b33ef740 1 -- 172.21.15.167:0/3776421531 >> 172.21.15.167:0/3776421531 conn(0x56320a0b5b10 msgr2=0x56320a0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:42.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.177+0000 7f00b33ef740 1 -- 172.21.15.167:0/3776421531 shutdown_connections 2024-05-11T12:13:42.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.177+0000 7f00b33ef740 1 -- 172.21.15.167:0/3776421531 wait complete. 2024-05-11T12:13:42.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.177+0000 7f00b33ef740 1 -- start start 2024-05-11T12:13:42.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.178+0000 7f00b33ef740 1 Processor -- start 2024-05-11T12:13:42.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.179+0000 7f00b33ef740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56320a0b7b50 0x56320a0d66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:42.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.179+0000 7f00b33ef740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56320a0b8a10 0x56320a105f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:42.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.179+0000 7f00b33ef740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x56320a101090 0x56320a101510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:42.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.179+0000 7f00b33ef740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56320a0e45a0 con 0x56320a0b8a10 2024-05-11T12:13:42.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.179+0000 7f00b33ef740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x563209fd59c0 con 0x56320a0b7b50 2024-05-11T12:13:42.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.179+0000 7f00b33ef740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x56320a0e3a20 con 0x56320a101090 2024-05-11T12:13:42.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.179+0000 7f00b33ef740 10 client.0 _ll_register_callbacks cb 0x563209fa2120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:13:42.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.179+0000 7f00ad000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56320a0b8a10 0x56320a105f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:42.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.179+0000 7f00ad000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56320a0b8a10 0x56320a105f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41330/0 (socket says 172.21.15.167:41330) 2024-05-11T12:13:42.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.179+0000 7f00ad000640 1 -- 172.21.15.167:0/3501843782 learned_addr learned my addr 172.21.15.167:0/3501843782 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:42.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.179+0000 7f00ada00640 1 --2- 172.21.15.167:0/3501843782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56320a0b7b50 0x56320a0d66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:42.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.179+0000 7f00ad000640 1 -- 172.21.15.167:0/3501843782 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x56320a101090 msgr2=0x56320a101510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:42.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.179+0000 7f00ad000640 1 --2- 172.21.15.167:0/3501843782 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x56320a101090 0x56320a101510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:42.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.179+0000 7f00ad000640 1 -- 172.21.15.167:0/3501843782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56320a0b7b50 msgr2=0x56320a0d66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:42.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.179+0000 7f00ad000640 1 --2- 172.21.15.167:0/3501843782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56320a0b7b50 0x56320a0d66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:42.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.179+0000 7f00ad000640 1 -- 172.21.15.167:0/3501843782 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563209df86b0 con 0x56320a0b8a10 2024-05-11T12:13:42.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.179+0000 7f00ada00640 1 --2- 172.21.15.167:0/3501843782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56320a0b7b50 0x56320a0d66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:42.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.180+0000 7f00ad000640 1 --2- 172.21.15.167:0/3501843782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56320a0b8a10 0x56320a105f80 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f0094010790 tx=0x7f0094013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de0953acf4a1d6b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:42.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.180+0000 7f008ea00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-11T12:13:42.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.180+0000 7f008ea00640 1 -- 172.21.15.167:0/3501843782 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009407f060 con 0x56320a0b8a10 2024-05-11T12:13:42.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.180+0000 7f008ea00640 1 -- 172.21.15.167:0/3501843782 <== mon.0 v2:172.21.15.167:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7f009405fef0 con 0x56320a0b8a10 2024-05-11T12:13:42.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.180+0000 7f00b33ef740 1 -- 172.21.15.167:0/3501843782 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00940693b0 con 0x56320a0b8a10 2024-05-11T12:13:42.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.180+0000 7f008ea00640 1 -- 172.21.15.167:0/3501843782 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f0094021040 con 0x56320a0b8a10 2024-05-11T12:13:42.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.180+0000 7f00b33ef740 10 client.6087 fetch_fsmap learned FSMap version 44 2024-05-11T12:13:42.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.180+0000 7f00b33ef740 1 -- 172.21.15.167:0/3501843782 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) -- 0x56320a106d20 con 0x56320a0b8a10 2024-05-11T12:13:42.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.180+0000 7f008ea00640 1 -- 172.21.15.167:0/3501843782 <== mon.0 v2:172.21.15.167:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00940693b0 con 0x56320a0b8a10 2024-05-11T12:13:42.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.180+0000 7f008ea00640 1 -- 172.21.15.167:0/3501843782 <== mon.0 v2:172.21.15.167:3300/0 5 ==== fsmap(e 44) ==== 3494+0+0 (secure 0 0 0) 0x7f0094082080 con 0x56320a0b8a10 2024-05-11T12:13:42.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.180+0000 7f00b33ef740 10 client.6087 fetch_fsmap finished waiting for FSMap version 44 2024-05-11T12:13:42.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.180+0000 7f00b33ef740 10 client.6087 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-11T12:13:42.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.180+0000 7f00b33ef740 20 client.6087 populate_metadata read hostname 'smithi167' 2024-05-11T12:13:42.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.180+0000 7f00b33ef740 1 --2- 172.21.15.167:0/3501843782 >> [v2:172.21.15.167:6835/1306616214,v1:172.21.15.167:6837/1306616214] conn(0x56320a0d8be0 0x56320a0d8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:42.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.180+0000 7f00b33ef740 4 client.6087 mds_command: new command op to 5952 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:13:42.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.181+0000 7f00b33ef740 1 -- 172.21.15.167:0/3501843782 --> [v2:172.21.15.167:6835/1306616214,v1:172.21.15.167:6837/1306616214] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56320a02a3a0 con 0x56320a0d8be0 2024-05-11T12:13:42.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.181+0000 7f00ada00640 1 --2- 172.21.15.167:0/3501843782 >> [v2:172.21.15.167:6835/1306616214,v1:172.21.15.167:6837/1306616214] conn(0x56320a0d8be0 0x56320a0d8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:42.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.181+0000 7f00ada00640 1 --2- 172.21.15.167:0/3501843782 >> [v2:172.21.15.167:6835/1306616214,v1:172.21.15.167:6837/1306616214] conn(0x56320a0d8be0 0x56320a0d8fc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d99f87c48a9a3c04 server_cookie=40e7c09e5a8fd0d8 in_seq=0 out_seq=0 2024-05-11T12:13:42.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.181+0000 7f008ea00640 10 client.6087 ms_handle_connect on v2:172.21.15.167:6835/1306616214 2024-05-11T12:13:42.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.182+0000 7f008ea00640 1 -- 172.21.15.167:0/3501843782 <== mds.0 v2:172.21.15.167:6835/1306616214 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56320a02a3a0 con 0x56320a0d8be0 2024-05-11T12:13:42.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.182+0000 7f008ea00640 10 client.6087 handle_command_reply: tid=0 2024-05-11T12:13:42.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.182+0000 7f00b33ef740 1 client.6087 shutdown 2024-05-11T12:13:42.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.183+0000 7f00b33ef740 1 -- 172.21.15.167:0/3501843782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56320a0b8a10 msgr2=0x56320a105f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:42.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.183+0000 7f00b33ef740 1 --2- 172.21.15.167:0/3501843782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56320a0b8a10 0x56320a105f80 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f0094010790 tx=0x7f0094013000 comp rx=0 tx=0).stop 2024-05-11T12:13:42.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.183+0000 7f00b33ef740 1 -- 172.21.15.167:0/3501843782 shutdown_connections 2024-05-11T12:13:42.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.183+0000 7f00b33ef740 1 --2- 172.21.15.167:0/3501843782 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x56320a101090 0x56320a101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:42.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.183+0000 7f00b33ef740 1 --2- 172.21.15.167:0/3501843782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56320a0b8a10 0x56320a105f80 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:42.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.183+0000 7f00b33ef740 1 --2- 172.21.15.167:0/3501843782 >> [v2:172.21.15.167:6835/1306616214,v1:172.21.15.167:6837/1306616214] conn(0x56320a0d8be0 0x56320a0d8fc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:42.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.183+0000 7f00b33ef740 1 --2- 172.21.15.167:0/3501843782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56320a0b7b50 0x56320a0d66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:42.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.183+0000 7f00b33ef740 1 -- 172.21.15.167:0/3501843782 >> 172.21.15.167:0/3501843782 conn(0x56320a0b38a0 msgr2=0x56320a0b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:42.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.183+0000 7f008ea00640 0 client.6087 ms_handle_reset on v2:172.21.15.167:6835/1306616214 2024-05-11T12:13:42.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.183+0000 7f00b33ef740 1 -- 172.21.15.167:0/3501843782 shutdown_connections 2024-05-11T12:13:42.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.183+0000 7f00b33ef740 1 -- 172.21.15.167:0/3501843782 wait complete. 2024-05-11T12:13:42.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.183+0000 7f00b33ef740 20 client.6087 trim_cache size 0 max 16384 2024-05-11T12:13:42.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.203+0000 7f00b33ef740 1 Processor -- start 2024-05-11T12:13:42.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.203+0000 7f00b33ef740 1 -- start start 2024-05-11T12:13:42.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.204+0000 7f00b33ef740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56320a0b38a0 0x56320a104490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:42.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.204+0000 7f00b33ef740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56320a0b7b50 0x56320a172af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:42.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.204+0000 7f00b33ef740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x56320a0b8a10 0x56320a0dbba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:42.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.204+0000 7f00b33ef740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f00a82016b0 con 0x56320a0b7b50 2024-05-11T12:13:42.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.204+0000 7f00b33ef740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f00a81425c0 con 0x56320a0b38a0 2024-05-11T12:13:42.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.204+0000 7f00b33ef740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f00a8201060 con 0x56320a0b8a10 2024-05-11T12:13:42.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.204+0000 7f00ad000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56320a0b7b50 0x56320a172af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:42.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.204+0000 7f00ad000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56320a0b7b50 0x56320a172af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41336/0 (socket says 172.21.15.167:41336) 2024-05-11T12:13:42.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.204+0000 7f00ad000640 1 -- 172.21.15.167:0/4266665288 learned_addr learned my addr 172.21.15.167:0/4266665288 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:42.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.205+0000 7f00ad000640 1 -- 172.21.15.167:0/4266665288 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x56320a0b8a10 msgr2=0x56320a0dbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:42.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.205+0000 7f00ad000640 1 --2- 172.21.15.167:0/4266665288 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x56320a0b8a10 0x56320a0dbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:42.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.205+0000 7f00ad000640 1 -- 172.21.15.167:0/4266665288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56320a0b38a0 msgr2=0x56320a104490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:42.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.205+0000 7f00ad000640 1 --2- 172.21.15.167:0/4266665288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56320a0b38a0 0x56320a104490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:42.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.205+0000 7f00ad000640 1 -- 172.21.15.167:0/4266665288 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56320a0ae2e0 con 0x56320a0b7b50 2024-05-11T12:13:42.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.205+0000 7f00ad000640 1 --2- 172.21.15.167:0/4266665288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56320a0b7b50 0x56320a172af0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f009405d230 tx=0x7f0094092040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab8bd55aa49bb8d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:42.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.205+0000 7f0099600640 1 -- 172.21.15.167:0/4266665288 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0094065040 con 0x56320a0b7b50 2024-05-11T12:13:42.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.205+0000 7f0099600640 1 -- 172.21.15.167:0/4266665288 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0094069a60 con 0x56320a0b7b50 2024-05-11T12:13:42.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.205+0000 7f0099600640 1 -- 172.21.15.167:0/4266665288 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009405fef0 con 0x56320a0b7b50 2024-05-11T12:13:42.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.206+0000 7f00b33ef740 1 -- 172.21.15.167:0/4266665288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56320a0b7b50 msgr2=0x56320a172af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:42.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.206+0000 7f00b33ef740 1 --2- 172.21.15.167:0/4266665288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56320a0b7b50 0x56320a172af0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f009405d230 tx=0x7f0094092040 comp rx=0 tx=0).stop 2024-05-11T12:13:42.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.206+0000 7f00b33ef740 1 -- 172.21.15.167:0/4266665288 shutdown_connections 2024-05-11T12:13:42.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.206+0000 7f00b33ef740 1 --2- 172.21.15.167:0/4266665288 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x56320a0b8a10 0x56320a0dbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:42.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.206+0000 7f00b33ef740 1 --2- 172.21.15.167:0/4266665288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56320a0b7b50 0x56320a172af0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:42.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.206+0000 7f00b33ef740 1 --2- 172.21.15.167:0/4266665288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56320a0b38a0 0x56320a104490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:42.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.206+0000 7f00b33ef740 1 -- 172.21.15.167:0/4266665288 >> 172.21.15.167:0/4266665288 conn(0x56320a0b6a20 msgr2=0x56320a0d6560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:42.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.206+0000 7f00b33ef740 1 -- 172.21.15.167:0/4266665288 shutdown_connections 2024-05-11T12:13:42.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.206+0000 7f00b33ef740 1 -- 172.21.15.167:0/4266665288 wait complete. 2024-05-11T12:13:42.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.207+0000 7f00b33ef740 1 -- start start 2024-05-11T12:13:42.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.207+0000 7f00b33ef740 1 Processor -- start 2024-05-11T12:13:42.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.208+0000 7f00b33ef740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56320a0b38a0 0x56320a105d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:42.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.208+0000 7f00b33ef740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x56320a0b7b50 0x56320a0d4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:42.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.208+0000 7f00b33ef740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56320a0b8a10 0x56320a0d3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:42.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.208+0000 7f00b33ef740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56320a0e45a0 con 0x56320a0b38a0 2024-05-11T12:13:42.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.208+0000 7f00b33ef740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56320a0e3a20 con 0x56320a0b8a10 2024-05-11T12:13:42.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.208+0000 7f00b33ef740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x563209fd59c0 con 0x56320a0b7b50 2024-05-11T12:13:42.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.208+0000 7f00b33ef740 10 client.0 _ll_register_callbacks cb 0x7f0094060ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:13:42.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.208+0000 7f00ada00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56320a0b38a0 0x56320a105d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:42.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.208+0000 7f00ada00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56320a0b38a0 0x56320a105d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41350/0 (socket says 172.21.15.167:41350) 2024-05-11T12:13:42.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.208+0000 7f00ada00640 1 -- 172.21.15.167:0/3324130928 learned_addr learned my addr 172.21.15.167:0/3324130928 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:42.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.209+0000 7f00ada00640 1 -- 172.21.15.167:0/3324130928 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x56320a0b7b50 msgr2=0x56320a0d4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:42.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.209+0000 7f00ad000640 1 --2- 172.21.15.167:0/3324130928 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x56320a0b7b50 0x56320a0d4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:42.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.209+0000 7f00ada00640 1 --2- 172.21.15.167:0/3324130928 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x56320a0b7b50 0x56320a0d4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:42.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.209+0000 7f00ada00640 1 -- 172.21.15.167:0/3324130928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56320a0b8a10 msgr2=0x56320a0d3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:42.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.209+0000 7f00ada00640 1 --2- 172.21.15.167:0/3324130928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56320a0b8a10 0x56320a0d3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:42.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.209+0000 7f00ada00640 1 -- 172.21.15.167:0/3324130928 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56320a1009c0 con 0x56320a0b38a0 2024-05-11T12:13:42.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.209+0000 7f00ada00640 1 --2- 172.21.15.167:0/3324130928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56320a0b38a0 0x56320a105d70 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f009c008fe0 tx=0x7f00a80b4360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e61a298966d4dac3 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:42.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.209+0000 7f008ea00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-11T12:13:42.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.209+0000 7f008ea00640 1 -- 172.21.15.167:0/3324130928 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009c054060 con 0x56320a0b38a0 2024-05-11T12:13:42.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.209+0000 7f008ea00640 1 -- 172.21.15.167:0/3324130928 <== mon.0 v2:172.21.15.167:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7f009c0074c0 con 0x56320a0b38a0 2024-05-11T12:13:42.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.209+0000 7f00b33ef740 1 -- 172.21.15.167:0/3324130928 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0094069a60 con 0x56320a0b38a0 2024-05-11T12:13:42.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.209+0000 7f008ea00640 1 -- 172.21.15.167:0/3324130928 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f009c05a050 con 0x56320a0b38a0 2024-05-11T12:13:42.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.210+0000 7f00b33ef740 10 client.6093 fetch_fsmap learned FSMap version 44 2024-05-11T12:13:42.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.210+0000 7f00b33ef740 1 -- 172.21.15.167:0/3324130928 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) -- 0x56320a0d53a0 con 0x56320a0b38a0 2024-05-11T12:13:42.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.210+0000 7f008ea00640 1 -- 172.21.15.167:0/3324130928 <== mon.0 v2:172.21.15.167:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0094069a60 con 0x56320a0b38a0 2024-05-11T12:13:42.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.210+0000 7f008ea00640 1 -- 172.21.15.167:0/3324130928 <== mon.0 v2:172.21.15.167:3300/0 5 ==== fsmap(e 44) ==== 3494+0+0 (secure 0 0 0) 0x7f009c05d020 con 0x56320a0b38a0 2024-05-11T12:13:42.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.210+0000 7f00b33ef740 10 client.6093 fetch_fsmap finished waiting for FSMap version 44 2024-05-11T12:13:42.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.210+0000 7f00b33ef740 10 client.6093 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-11T12:13:42.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.210+0000 7f00b33ef740 20 client.6093 populate_metadata read hostname 'smithi167' 2024-05-11T12:13:42.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.210+0000 7f00b33ef740 1 --2- 172.21.15.167:0/3324130928 >> [v2:172.21.15.167:6835/1306616214,v1:172.21.15.167:6837/1306616214] conn(0x56320a101090 0x56320a0d37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:42.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.210+0000 7f00b33ef740 4 client.6093 mds_command: new command op to 5952 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-11T12:13:42.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.210+0000 7f00b33ef740 1 -- 172.21.15.167:0/3324130928 --> [v2:172.21.15.167:6835/1306616214,v1:172.21.15.167:6837/1306616214] -- command(tid 0: {"prefix": "status"}) -- 0x56320a024210 con 0x56320a101090 2024-05-11T12:13:42.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.210+0000 7f00ad000640 1 --2- 172.21.15.167:0/3324130928 >> [v2:172.21.15.167:6835/1306616214,v1:172.21.15.167:6837/1306616214] conn(0x56320a101090 0x56320a0d37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:42.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.211+0000 7f00ad000640 1 --2- 172.21.15.167:0/3324130928 >> [v2:172.21.15.167:6835/1306616214,v1:172.21.15.167:6837/1306616214] conn(0x56320a101090 0x56320a0d37b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=150fe4b338a04da5 server_cookie=dd48bb0c2ec0ea96 in_seq=0 out_seq=0 2024-05-11T12:13:42.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.211+0000 7f008ea00640 10 client.6093 ms_handle_connect on v2:172.21.15.167:6835/1306616214 2024-05-11T12:13:42.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.211+0000 7f008ea00640 1 -- 172.21.15.167:0/3324130928 <== mds.0 v2:172.21.15.167:6835/1306616214 1 ==== command_reply(tid 0: 0 ) ==== 8+0+318 (crc 0 0 0) 0x56320a024210 con 0x56320a101090 2024-05-11T12:13:42.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.211+0000 7f008ea00640 10 client.6093 handle_command_reply: tid=0 2024-05-11T12:13:42.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.211+0000 7f00b33ef740 1 client.6093 shutdown 2024-05-11T12:13:42.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.211+0000 7f00b33ef740 1 -- 172.21.15.167:0/3324130928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56320a0b38a0 msgr2=0x56320a105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:42.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.211+0000 7f00b33ef740 1 --2- 172.21.15.167:0/3324130928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56320a0b38a0 0x56320a105d70 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f009c008fe0 tx=0x7f00a80b4360 comp rx=0 tx=0).stop 2024-05-11T12:13:42.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.211+0000 7f00b33ef740 1 -- 172.21.15.167:0/3324130928 shutdown_connections 2024-05-11T12:13:42.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.211+0000 7f00b33ef740 1 --2- 172.21.15.167:0/3324130928 >> [v2:172.21.15.167:6835/1306616214,v1:172.21.15.167:6837/1306616214] conn(0x56320a101090 0x56320a0d37b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:42.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.211+0000 7f00b33ef740 1 --2- 172.21.15.167:0/3324130928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56320a0b8a10 0x56320a0d3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:42.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.211+0000 7f00b33ef740 1 --2- 172.21.15.167:0/3324130928 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x56320a0b7b50 0x56320a0d4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:42.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.211+0000 7f00b33ef740 1 --2- 172.21.15.167:0/3324130928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56320a0b38a0 0x56320a105d70 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:42.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.211+0000 7f008ea00640 0 client.6093 ms_handle_reset on v2:172.21.15.167:6835/1306616214 2024-05-11T12:13:42.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.211+0000 7f00b33ef740 1 -- 172.21.15.167:0/3324130928 >> 172.21.15.167:0/3324130928 conn(0x56320a0b6a20 msgr2=0x56320a104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:42.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.212+0000 7f00b33ef740 1 -- 172.21.15.167:0/3324130928 shutdown_connections 2024-05-11T12:13:42.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.212+0000 7f00b33ef740 1 -- 172.21.15.167:0/3324130928 wait complete. 2024-05-11T12:13:42.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.212+0000 7f00b33ef740 20 client.6093 trim_cache size 0 max 16384 2024-05-11T12:13:42.214 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T12:13:42.214 INFO:teuthology.orchestra.run.smithi167.stdout: "cluster_fsid": "5f32d344-7289-4c27-881b-1533419ab678", 2024-05-11T12:13:42.215 INFO:teuthology.orchestra.run.smithi167.stdout: "whoami": 0, 2024-05-11T12:13:42.215 INFO:teuthology.orchestra.run.smithi167.stdout: "id": 5952, 2024-05-11T12:13:42.215 INFO:teuthology.orchestra.run.smithi167.stdout: "want_state": "up:active", 2024-05-11T12:13:42.215 INFO:teuthology.orchestra.run.smithi167.stdout: "state": "up:active", 2024-05-11T12:13:42.215 INFO:teuthology.orchestra.run.smithi167.stdout: "fs_name": "cephfs", 2024-05-11T12:13:42.215 INFO:teuthology.orchestra.run.smithi167.stdout: "rank_uptime": 6.673744213, 2024-05-11T12:13:42.215 INFO:teuthology.orchestra.run.smithi167.stdout: "mdsmap_epoch": 44, 2024-05-11T12:13:42.215 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_epoch": 75, 2024-05-11T12:13:42.215 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_epoch_barrier": 75, 2024-05-11T12:13:42.215 INFO:teuthology.orchestra.run.smithi167.stdout: "uptime": 11.089968016 2024-05-11T12:13:42.215 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T12:13:42.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.212+0000 7f00aee00640 1 -- 172.21.15.167:0/2415904861 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0088076300 msgr2=0x7f00880787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:42.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.212+0000 7f00aee00640 1 --2- 172.21.15.167:0/2415904861 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0088076300 0x7f00880787c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f009c032000 tx=0x7f009c009030 comp rx=0 tx=0).stop 2024-05-11T12:13:42.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.212+0000 7f00aee00640 1 -- 172.21.15.167:0/2415904861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00a8072720 msgr2=0x7f00a8072b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:42.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.212+0000 7f00aee00640 1 --2- 172.21.15.167:0/2415904861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00a8072720 0x7f00a8072b80 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f00940107c0 tx=0x7f0094010c80 comp rx=0 tx=0).stop 2024-05-11T12:13:42.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.212+0000 7f00aee00640 1 -- 172.21.15.167:0/2415904861 shutdown_connections 2024-05-11T12:13:42.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.212+0000 7f00aee00640 1 --2- 172.21.15.167:0/2415904861 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00a8142a00 0x7f00a8142e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:42.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.212+0000 7f00aee00640 1 --2- 172.21.15.167:0/2415904861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00a8072720 0x7f00a8072b80 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:42.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.212+0000 7f00aee00640 1 --2- 172.21.15.167:0/2415904861 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0088076300 0x7f00880787c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:42.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.212+0000 7f00aee00640 1 --2- 172.21.15.167:0/2415904861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0090003ac0 0x7f00a813ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:42.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.212+0000 7f00aee00640 1 -- 172.21.15.167:0/2415904861 >> 172.21.15.167:0/2415904861 conn(0x7f00a806c450 msgr2=0x7f00a80b4ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:42.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.213+0000 7f00aee00640 1 -- 172.21.15.167:0/2415904861 shutdown_connections 2024-05-11T12:13:42.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.213+0000 7f00aee00640 1 -- 172.21.15.167:0/2415904861 wait complete. 2024-05-11T12:13:42.229 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-11T12:13:42.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.419+0000 7f002e200640 1 Processor -- start 2024-05-11T12:13:42.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.419+0000 7f002e200640 1 -- start start 2024-05-11T12:13:42.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.420+0000 7f002e200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00281458f0 0x7f002806d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:42.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.420+0000 7f002e200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f00280b3a80 0x7f00280b4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:42.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.420+0000 7f002e200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f00280b5220 0x7f00280b56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:42.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.420+0000 7f002e200640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f00280b5bf0 con 0x7f00280b5220 2024-05-11T12:13:42.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.420+0000 7f002e200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f002806db80 con 0x7f00280b3a80 2024-05-11T12:13:42.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.420+0000 7f002e200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0028071f30 con 0x7f00281458f0 2024-05-11T12:13:42.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.421+0000 7f002d800640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f00280b5220 0x7f00280b56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:42.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.421+0000 7f002d800640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f00280b5220 0x7f00280b56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41364/0 (socket says 172.21.15.167:41364) 2024-05-11T12:13:42.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.421+0000 7f002d800640 1 -- 172.21.15.167:0/2715863230 learned_addr learned my addr 172.21.15.167:0/2715863230 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:42.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.421+0000 7f002ce00640 1 --2- 172.21.15.167:0/2715863230 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00281458f0 0x7f002806d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:42.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.421+0000 7f001fe00640 1 --2- 172.21.15.167:0/2715863230 >> v2:172.21.15.204:3300/0 conn(0x7f00280b3a80 0x7f00280b4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:42.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.421+0000 7f002d800640 1 -- 172.21.15.167:0/2715863230 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00281458f0 msgr2=0x7f002806d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:42.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.421+0000 7f002d800640 1 --2- 172.21.15.167:0/2715863230 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00281458f0 0x7f002806d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:42.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.421+0000 7f002d800640 1 -- 172.21.15.167:0/2715863230 >> v2:172.21.15.204:3300/0 conn(0x7f00280b3a80 msgr2=0x7f00280b4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:42.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.421+0000 7f002d800640 1 --2- 172.21.15.167:0/2715863230 >> v2:172.21.15.204:3300/0 conn(0x7f00280b3a80 0x7f00280b4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:42.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.421+0000 7f002d800640 1 -- 172.21.15.167:0/2715863230 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0028072210 con 0x7f00280b5220 2024-05-11T12:13:42.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.421+0000 7f001fe00640 1 --2- 172.21.15.167:0/2715863230 >> v2:172.21.15.204:3300/0 conn(0x7f00280b3a80 0x7f00280b4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:42.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.422+0000 7f002d800640 1 --2- 172.21.15.167:0/2715863230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00280b5220 0x7f00280b56b0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f0028070150 tx=0x7f002002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a912d49d825487a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:42.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.423+0000 7f0027e00640 1 -- 172.21.15.167:0/2715863230 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0020036070 con 0x7f00280b5220 2024-05-11T12:13:42.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.423+0000 7f0027e00640 1 -- 172.21.15.167:0/2715863230 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0020031600 con 0x7f00280b5220 2024-05-11T12:13:42.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.423+0000 7f0027e00640 1 -- 172.21.15.167:0/2715863230 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00200319d0 con 0x7f00280b5220 2024-05-11T12:13:42.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.423+0000 7f002e200640 1 -- 172.21.15.167:0/2715863230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00280b5220 msgr2=0x7f00280b56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:42.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.423+0000 7f002e200640 1 --2- 172.21.15.167:0/2715863230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00280b5220 0x7f00280b56b0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f0028070150 tx=0x7f002002a6a0 comp rx=0 tx=0).stop 2024-05-11T12:13:42.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.423+0000 7f002e200640 1 -- 172.21.15.167:0/2715863230 shutdown_connections 2024-05-11T12:13:42.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.424+0000 7f002e200640 1 --2- 172.21.15.167:0/2715863230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00280b5220 0x7f00280b56b0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:42.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.424+0000 7f002e200640 1 --2- 172.21.15.167:0/2715863230 >> v2:172.21.15.204:3300/0 conn(0x7f00280b3a80 0x7f00280b4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:42.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.424+0000 7f002e200640 1 --2- 172.21.15.167:0/2715863230 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00281458f0 0x7f002806d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:42.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.424+0000 7f002e200640 1 -- 172.21.15.167:0/2715863230 >> 172.21.15.167:0/2715863230 conn(0x7f00280b3460 msgr2=0x7f00280d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:42.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.424+0000 7f002e200640 1 -- 172.21.15.167:0/2715863230 shutdown_connections 2024-05-11T12:13:42.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.424+0000 7f002e200640 1 -- 172.21.15.167:0/2715863230 wait complete. 2024-05-11T12:13:42.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.425+0000 7f002e200640 1 Processor -- start 2024-05-11T12:13:42.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.425+0000 7f002e200640 1 -- start start 2024-05-11T12:13:42.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.426+0000 7f002e200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00280b3a80 0x7f002815b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:42.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.426+0000 7f002e200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00280b5220 0x7f0028159ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:42.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.426+0000 7f002e200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00281458f0 0x7f002813bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:42.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.426+0000 7f002e200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0028072ab0 con 0x7f00280b5220 2024-05-11T12:13:42.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.426+0000 7f002e200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f00280727d0 con 0x7f00281458f0 2024-05-11T12:13:42.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.426+0000 7f002e200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f00280b4eb0 con 0x7f00280b3a80 2024-05-11T12:13:42.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.427+0000 7f001fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00280b5220 0x7f0028159ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:42.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.427+0000 7f001fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00280b5220 0x7f0028159ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41372/0 (socket says 172.21.15.167:41372) 2024-05-11T12:13:42.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.427+0000 7f001fe00640 1 -- 172.21.15.167:0/2975854873 learned_addr learned my addr 172.21.15.167:0/2975854873 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:42.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.427+0000 7f002d800640 1 --2- 172.21.15.167:0/2975854873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00281458f0 0x7f002813bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:42.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.427+0000 7f002ce00640 1 --2- 172.21.15.167:0/2975854873 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00280b3a80 0x7f002815b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:42.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.427+0000 7f001fe00640 1 -- 172.21.15.167:0/2975854873 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00280b3a80 msgr2=0x7f002815b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:42.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.427+0000 7f001fe00640 1 --2- 172.21.15.167:0/2975854873 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00280b3a80 0x7f002815b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:42.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.427+0000 7f001fe00640 1 -- 172.21.15.167:0/2975854873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00281458f0 msgr2=0x7f002813bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:42.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.427+0000 7f001fe00640 1 --2- 172.21.15.167:0/2975854873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00281458f0 0x7f002813bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:42.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.427+0000 7f001fe00640 1 -- 172.21.15.167:0/2975854873 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f002815b190 con 0x7f00280b5220 2024-05-11T12:13:42.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.427+0000 7f002d800640 1 --2- 172.21.15.167:0/2975854873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00281458f0 0x7f002813bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:42.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.428+0000 7f001fe00640 1 --2- 172.21.15.167:0/2975854873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00280b5220 0x7f0028159ea0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f001000c7c0 tx=0x7f001000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:42.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.428+0000 7f0025600640 1 -- 172.21.15.167:0/2975854873 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0010015070 con 0x7f00280b5220 2024-05-11T12:13:42.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.428+0000 7f0025600640 1 -- 172.21.15.167:0/2975854873 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0010007a30 con 0x7f00280b5220 2024-05-11T12:13:42.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.428+0000 7f0025600640 1 -- 172.21.15.167:0/2975854873 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0010007e00 con 0x7f00280b5220 2024-05-11T12:13:42.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.428+0000 7f002e200640 1 -- 172.21.15.167:0/2975854873 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f002815a610 con 0x7f00280b5220 2024-05-11T12:13:42.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.428+0000 7f002e200640 1 -- 172.21.15.167:0/2975854873 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f002815abd0 con 0x7f00280b5220 2024-05-11T12:13:42.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.429+0000 7f002e200640 1 -- 172.21.15.167:0/2975854873 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00280694b0 con 0x7f00280b5220 2024-05-11T12:13:42.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.430+0000 7f0025600640 1 -- 172.21.15.167:0/2975854873 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0010018070 con 0x7f00280b5220 2024-05-11T12:13:42.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.431+0000 7f0025600640 1 --2- 172.21.15.167:0/2975854873 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f00040762d0 0x7f0004078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:42.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.431+0000 7f0025600640 1 -- 172.21.15.167:0/2975854873 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7f0010097740 con 0x7f00280b5220 2024-05-11T12:13:42.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.432+0000 7f002ce00640 1 --2- 172.21.15.167:0/2975854873 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f00040762d0 0x7f0004078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:42.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.432+0000 7f002ce00640 1 --2- 172.21.15.167:0/2975854873 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f00040762d0 0x7f0004078790 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0018006fd0 tx=0x7f0018008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:42.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.436+0000 7f0025600640 1 -- 172.21.15.167:0/2975854873 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0010060e70 con 0x7f00280b5220 2024-05-11T12:13:42.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.679+0000 7f002e200640 1 -- 172.21.15.167:0/2975854873 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f00280ceb80 con 0x7f00280b5220 2024-05-11T12:13:42.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.680+0000 7f0025600640 1 -- 172.21.15.167:0/2975854873 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v44) ==== 53+0+73 (secure 0 0 0) 0x7f0010060770 con 0x7f00280b5220 2024-05-11T12:13:42.679 INFO:teuthology.orchestra.run.smithi167.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-11T12:13:42.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.682+0000 7f002e200640 1 -- 172.21.15.167:0/2975854873 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f00040762d0 msgr2=0x7f0004078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:42.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.682+0000 7f002e200640 1 --2- 172.21.15.167:0/2975854873 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f00040762d0 0x7f0004078790 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0018006fd0 tx=0x7f0018008040 comp rx=0 tx=0).stop 2024-05-11T12:13:42.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.682+0000 7f002e200640 1 -- 172.21.15.167:0/2975854873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00280b5220 msgr2=0x7f0028159ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:42.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.682+0000 7f002e200640 1 --2- 172.21.15.167:0/2975854873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00280b5220 0x7f0028159ea0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f001000c7c0 tx=0x7f001000cc80 comp rx=0 tx=0).stop 2024-05-11T12:13:42.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.683+0000 7f002e200640 1 -- 172.21.15.167:0/2975854873 shutdown_connections 2024-05-11T12:13:42.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.683+0000 7f002e200640 1 --2- 172.21.15.167:0/2975854873 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f00040762d0 0x7f0004078790 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:42.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.683+0000 7f002e200640 1 --2- 172.21.15.167:0/2975854873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00281458f0 0x7f002813bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:42.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.683+0000 7f002e200640 1 --2- 172.21.15.167:0/2975854873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00280b5220 0x7f0028159ea0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:42.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.683+0000 7f002e200640 1 --2- 172.21.15.167:0/2975854873 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00280b3a80 0x7f002815b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:42.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.683+0000 7f002e200640 1 -- 172.21.15.167:0/2975854873 >> 172.21.15.167:0/2975854873 conn(0x7f00280b3460 msgr2=0x7f00280cf140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:42.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.683+0000 7f002e200640 1 -- 172.21.15.167:0/2975854873 shutdown_connections 2024-05-11T12:13:42.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42.683+0000 7f002e200640 1 -- 172.21.15.167:0/2975854873 wait complete. 2024-05-11T12:13:42.701 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-11T12:13:42.701 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-11T12:13:42.701 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi167.front.sepia.ceph.com 2024-05-11T12:13:42.701 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-11T12:13:42.701 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-11T12:13:42.701 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-11T12:13:42.701 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-11T12:13:42.701 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-11T12:13:42.701 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-11T12:13:42.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:13:42.702 DEBUG:teuthology.orchestra.run.smithi167:> ip addr 2024-05-11T12:13:42.750 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-11T12:13:42.750 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-11T12:13:42.750 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2024-05-11T12:13:42.750 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:13:42.750 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2024-05-11T12:13:42.750 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:13:42.750 INFO:teuthology.orchestra.run.smithi167.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:13:42.750 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e6 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:13:42.751 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f0 2024-05-11T12:13:42.751 INFO:teuthology.orchestra.run.smithi167.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:13:42.751 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e7 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:13:42.751 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f1 2024-05-11T12:13:42.751 INFO:teuthology.orchestra.run.smithi167.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:13:42.751 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:08 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:13:42.751 INFO:teuthology.orchestra.run.smithi167.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-11T12:13:42.751 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:09 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:13:42.751 INFO:teuthology.orchestra.run.smithi167.stdout: inet 172.21.15.167/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-11T12:13:42.751 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft 42446sec preferred_lft 42446sec 2024-05-11T12:13:42.751 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::ec4:7aff:fe88:7309/64 scope link 2024-05-11T12:13:42.751 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:13:42.751 INFO:teuthology.orchestra.run.smithi167.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-11T12:13:42.751 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 1a:a8:8d:b3:f4:32 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:13:42.752 INFO:teuthology.orchestra.run.smithi167.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-11T12:13:42.752 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:13:42.752 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::b017:48ff:fea0:7183/64 scope link 2024-05-11T12:13:42.752 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:13:42.752 INFO:teuthology.orchestra.run.smithi167.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-11T12:13:42.752 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 1a:a8:8d:b3:f4:32 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-11T12:13:42.752 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::18a8:8dff:feb3:f432/64 scope link 2024-05-11T12:13:42.752 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:13:42.752 INFO:teuthology.orchestra.run.smithi167.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-11T12:13:42.752 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 7e:79:1f:cb:d2:83 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-11T12:13:42.752 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::7c79:1fff:fecb:d283/64 scope link 2024-05-11T12:13:42.752 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:13:42.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:13:42.753 DEBUG:teuthology.orchestra.run.smithi167:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-11T12:13:42.830 INFO:teuthology.orchestra.run.smithi167.stdout:1 2024-05-11T12:13:42.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:13:42.834 DEBUG:teuthology.orchestra.run.smithi167:> route 2024-05-11T12:13:42.883 INFO:teuthology.orchestra.run.smithi167.stdout:Kernel IP routing table 2024-05-11T12:13:42.883 INFO:teuthology.orchestra.run.smithi167.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-11T12:13:42.883 INFO:teuthology.orchestra.run.smithi167.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-11T12:13:42.883 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-11T12:13:42.883 INFO:teuthology.orchestra.run.smithi167.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-11T12:13:42.884 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T12:13:42.884 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-05-11T12:13:42.884 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-11T12:13:42.884 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-11T12:13:42.884 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-11T12:13:42.884 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-05-11T12:13:42.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T12:13:43.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:13:43.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:13:43.092 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2024-05-11T12:13:43.139 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-11T12:13:43.139 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-11T12:13:43.140 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-11T12:13:43.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:13:43.140 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set brx.0 up 2024-05-11T12:13:43.219 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T12:13:43.219 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-11T12:13:43.271 INFO:teuthology.orchestra.run.smithi167.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-11T12:13:43.272 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T12:13:43.272 DEBUG:teuthology.orchestra.run.smithi167:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-11T12:13:43.324 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-05-11T12:13:43.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:13:43.324 DEBUG:teuthology.orchestra.run.smithi167:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:13:43.450 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:13:43.450 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-11T12:13:43.485 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:13:43.485 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-11T12:13:43.485 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.167:3300,172.21.15.204:3300,172.21.15.204:3301" 2024-05-11T12:13:43.485 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: trying mount with new device syntax: 0@5f32d344-7289-4c27-881b-1533419ab678.cephfs=/ 2024-05-11T12:13:43.485 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.167:3300/172.21.15.204:3300/172.21.15.204:3301" will pass to kernel 2024-05-11T12:13:43.488 INFO:teuthology.orchestra.run.smithi167.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-11T12:13:43.488 INFO:teuthology.orchestra.run.smithi167.stdout: You just mounted a file system that supports labels which does not 2024-05-11T12:13:43.488 INFO:teuthology.orchestra.run.smithi167.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-11T12:13:43.488 INFO:teuthology.orchestra.run.smithi167.stdout: applications will generate AVC messages and not be allowed access to 2024-05-11T12:13:43.488 INFO:teuthology.orchestra.run.smithi167.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-11T12:13:43.491 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-11T12:13:43.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:13:43.492 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-11T12:13:43.569 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-11T12:13:43.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-11T12:13:43.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:13:43.698 INFO:teuthology.orchestra.run.smithi167.stdout:client6102 2024-05-11T12:13:43.698 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:13:43.698 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:13:43.746 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-11T12:13:43.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:43.993+0000 7fe151e00640 1 Processor -- start 2024-05-11T12:13:43.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:43.993+0000 7fe151e00640 1 -- start start 2024-05-11T12:13:43.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:43.995+0000 7fe151e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe14c0af1f0 0x7fe14c0cf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:43.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:43.995+0000 7fe151e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe14c0af5d0 0x7fe14c073ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:43.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:43.995+0000 7fe151e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe14c0cf700 con 0x7fe14c06d460 2024-05-11T12:13:43.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:43.995+0000 7fe151e00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fe14c1608d0 con 0x7fe14c0af1f0 2024-05-11T12:13:43.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:43.995+0000 7fe151e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fe14c06fe10 con 0x7fe14c0af5d0 2024-05-11T12:13:43.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:43.995+0000 7fe14aa00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe14c0af1f0 0x7fe14c0cf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:43.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:43.995+0000 7fe14aa00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe14c0af1f0 0x7fe14c0cf1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41380/0 (socket says 172.21.15.167:41380) 2024-05-11T12:13:43.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:43.995+0000 7fe14aa00640 1 -- 172.21.15.167:0/1196567254 learned_addr learned my addr 172.21.15.167:0/1196567254 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:43.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:43.996+0000 7fe14a000640 1 --2- 172.21.15.167:0/1196567254 >> v2:172.21.15.204:3300/0 conn(0x7fe14c0af5d0 0x7fe14c073ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:43.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:43.996+0000 7fe14aa00640 1 -- 172.21.15.167:0/1196567254 >> v2:172.21.15.204:3300/0 conn(0x7fe14c0af5d0 msgr2=0x7fe14c073ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:43.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:43.996+0000 7fe14aa00640 1 --2- 172.21.15.167:0/1196567254 >> v2:172.21.15.204:3300/0 conn(0x7fe14c0af5d0 0x7fe14c073ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:43.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:43.996+0000 7fe14aa00640 1 -- 172.21.15.167:0/1196567254 >> v1:172.21.15.204:6789/0 conn(0x7fe14c06d460 legacy=0x7fe14c06e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:43.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:43.996+0000 7fe14aa00640 1 -- 172.21.15.167:0/1196567254 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe14c15fcc0 con 0x7fe14c0af1f0 2024-05-11T12:13:43.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:43.997+0000 7fe14aa00640 1 --2- 172.21.15.167:0/1196567254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe14c0af1f0 0x7fe14c0cf1c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fe14c0cebc0 tx=0x7fe13c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e706689b0f7574d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:43.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:43.997+0000 7fe149600640 1 -- 172.21.15.167:0/1196567254 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13c036070 con 0x7fe14c0af1f0 2024-05-11T12:13:43.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:43.997+0000 7fe149600640 1 -- 172.21.15.167:0/1196567254 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe13c031600 con 0x7fe14c0af1f0 2024-05-11T12:13:43.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:43.997+0000 7fe149600640 1 -- 172.21.15.167:0/1196567254 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13c0319d0 con 0x7fe14c0af1f0 2024-05-11T12:13:43.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:43.998+0000 7fe151e00640 1 -- 172.21.15.167:0/1196567254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe14c0af1f0 msgr2=0x7fe14c0cf1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:43.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:43.998+0000 7fe151e00640 1 --2- 172.21.15.167:0/1196567254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe14c0af1f0 0x7fe14c0cf1c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fe14c0cebc0 tx=0x7fe13c02a610 comp rx=0 tx=0).stop 2024-05-11T12:13:43.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:43.998+0000 7fe151e00640 1 -- 172.21.15.167:0/1196567254 shutdown_connections 2024-05-11T12:13:43.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:43.998+0000 7fe151e00640 1 --2- 172.21.15.167:0/1196567254 >> v2:172.21.15.204:3300/0 conn(0x7fe14c0af5d0 0x7fe14c073ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:43.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:43.998+0000 7fe151e00640 1 --2- 172.21.15.167:0/1196567254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe14c0af1f0 0x7fe14c0cf1c0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:43.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:43.998+0000 7fe151e00640 1 -- 172.21.15.167:0/1196567254 >> 172.21.15.167:0/1196567254 conn(0x7fe14c0cd8f0 msgr2=0x7fe14c0cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:43.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:43.998+0000 7fe151e00640 1 -- 172.21.15.167:0/1196567254 shutdown_connections 2024-05-11T12:13:43.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:43.998+0000 7fe151e00640 1 -- 172.21.15.167:0/1196567254 wait complete. 2024-05-11T12:13:43.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:44.000+0000 7fe151e00640 1 Processor -- start 2024-05-11T12:13:43.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:44.001+0000 7fe151e00640 1 -- start start 2024-05-11T12:13:44.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:44.002+0000 7fe151e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe14c06d460 0x7fe14c15c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:44.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:44.002+0000 7fe151e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe14c0af1f0 0x7fe14c073d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:44.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:44.002+0000 7fe151e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe14c0af5d0 0x7fe14c149760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:44.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:44.002+0000 7fe151e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c160560 con 0x7fe14c0af5d0 2024-05-11T12:13:44.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:44.002+0000 7fe151e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c160280 con 0x7fe14c0af1f0 2024-05-11T12:13:44.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:44.002+0000 7fe151e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe14c15f920 con 0x7fe14c06d460 2024-05-11T12:13:44.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:44.002+0000 7fe14b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe14c0af5d0 0x7fe14c149760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:44.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:44.002+0000 7fe14b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe14c0af5d0 0x7fe14c149760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41384/0 (socket says 172.21.15.167:41384) 2024-05-11T12:13:44.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:44.002+0000 7fe14b400640 1 -- 172.21.15.167:0/2619672783 learned_addr learned my addr 172.21.15.167:0/2619672783 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:44.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:44.003+0000 7fe14a000640 1 --2- 172.21.15.167:0/2619672783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe14c0af1f0 0x7fe14c073d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:44.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:44.003+0000 7fe14aa00640 1 --2- 172.21.15.167:0/2619672783 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe14c06d460 0x7fe14c15c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:44.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:44.003+0000 7fe14b400640 1 -- 172.21.15.167:0/2619672783 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe14c06d460 msgr2=0x7fe14c15c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:44.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:44.003+0000 7fe14b400640 1 --2- 172.21.15.167:0/2619672783 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe14c06d460 0x7fe14c15c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:44.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:44.003+0000 7fe14b400640 1 -- 172.21.15.167:0/2619672783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe14c0af1f0 msgr2=0x7fe14c073d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:44.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:44.003+0000 7fe14b400640 1 --2- 172.21.15.167:0/2619672783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe14c0af1f0 0x7fe14c073d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:44.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:44.003+0000 7fe14b400640 1 -- 172.21.15.167:0/2619672783 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe14c149ca0 con 0x7fe14c0af5d0 2024-05-11T12:13:44.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:44.003+0000 7fe14aa00640 1 --2- 172.21.15.167:0/2619672783 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe14c06d460 0x7fe14c15c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:44.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:44.003+0000 7fe14b400640 1 --2- 172.21.15.167:0/2619672783 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe14c0af5d0 0x7fe14c149760 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fe14400c3c0 tx=0x7fe14400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:44.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:44.003+0000 7fe143400640 1 -- 172.21.15.167:0/2619672783 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe144017070 con 0x7fe14c0af5d0 2024-05-11T12:13:44.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:44.003+0000 7fe143400640 1 -- 172.21.15.167:0/2619672783 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1440035f0 con 0x7fe14c0af5d0 2024-05-11T12:13:44.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:44.003+0000 7fe151e00640 1 -- 172.21.15.167:0/2619672783 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe14c072c70 con 0x7fe14c0af5d0 2024-05-11T12:13:44.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:44.003+0000 7fe143400640 1 -- 172.21.15.167:0/2619672783 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1440039e0 con 0x7fe14c0af5d0 2024-05-11T12:13:44.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:44.004+0000 7fe151e00640 1 -- 172.21.15.167:0/2619672783 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe14c073230 con 0x7fe14c0af5d0 2024-05-11T12:13:44.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:44.005+0000 7fe151e00640 1 -- 172.21.15.167:0/2619672783 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe14c069590 con 0x7fe14c0af5d0 2024-05-11T12:13:44.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:44.006+0000 7fe143400640 1 -- 172.21.15.167:0/2619672783 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe14401a070 con 0x7fe14c0af5d0 2024-05-11T12:13:44.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:44.007+0000 7fe143400640 1 --2- 172.21.15.167:0/2619672783 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe128076300 0x7fe1280787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:44.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:44.008+0000 7fe143400640 1 -- 172.21.15.167:0/2619672783 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7fe144096b30 con 0x7fe14c0af5d0 2024-05-11T12:13:44.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:44.008+0000 7fe14aa00640 1 --2- 172.21.15.167:0/2619672783 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe128076300 0x7fe1280787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:44.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:44.008+0000 7fe14aa00640 1 --2- 172.21.15.167:0/2619672783 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe128076300 0x7fe1280787c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe13c033000 tx=0x7fe13c009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:44.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:44.012+0000 7fe143400640 1 -- 172.21.15.167:0/2619672783 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe14c149ca0 con 0x7fe14c0af5d0 2024-05-11T12:13:44.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:44.262+0000 7fe151e00640 1 -- 172.21.15.167:0/2619672783 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe14c160ed0 con 0x7fe14c0af5d0 2024-05-11T12:13:44.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:44.263+0000 7fe143400640 1 -- 172.21.15.167:0/2619672783 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v44) ==== 53+0+73 (secure 0 0 0) 0x7fe144060260 con 0x7fe14c0af5d0 2024-05-11T12:13:44.262 INFO:teuthology.orchestra.run.smithi167.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-11T12:13:44.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:44.265+0000 7fe151e00640 1 -- 172.21.15.167:0/2619672783 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe128076300 msgr2=0x7fe1280787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:44.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:44.265+0000 7fe151e00640 1 --2- 172.21.15.167:0/2619672783 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe128076300 0x7fe1280787c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe13c033000 tx=0x7fe13c009000 comp rx=0 tx=0).stop 2024-05-11T12:13:44.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:44.266+0000 7fe151e00640 1 -- 172.21.15.167:0/2619672783 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe14c0af5d0 msgr2=0x7fe14c149760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:44.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:44.266+0000 7fe151e00640 1 --2- 172.21.15.167:0/2619672783 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe14c0af5d0 0x7fe14c149760 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fe14400c3c0 tx=0x7fe14400c880 comp rx=0 tx=0).stop 2024-05-11T12:13:44.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:44.266+0000 7fe151e00640 1 -- 172.21.15.167:0/2619672783 shutdown_connections 2024-05-11T12:13:44.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:44.266+0000 7fe151e00640 1 --2- 172.21.15.167:0/2619672783 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe14c0af5d0 0x7fe14c149760 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:44.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:44.266+0000 7fe151e00640 1 --2- 172.21.15.167:0/2619672783 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe128076300 0x7fe1280787c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:44.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:44.266+0000 7fe151e00640 1 --2- 172.21.15.167:0/2619672783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe14c0af1f0 0x7fe14c073d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:44.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:44.266+0000 7fe151e00640 1 --2- 172.21.15.167:0/2619672783 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe14c06d460 0x7fe14c15c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:44.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:44.266+0000 7fe151e00640 1 -- 172.21.15.167:0/2619672783 >> 172.21.15.167:0/2619672783 conn(0x7fe14c0cd8f0 msgr2=0x7fe14c0b0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:44.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:44.266+0000 7fe151e00640 1 -- 172.21.15.167:0/2619672783 shutdown_connections 2024-05-11T12:13:44.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:44.266+0000 7fe151e00640 1 -- 172.21.15.167:0/2619672783 wait complete. 2024-05-11T12:13:44.285 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-11T12:13:44.285 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-11T12:13:44.285 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi204.front.sepia.ceph.com 2024-05-11T12:13:44.285 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-11T12:13:44.285 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-11T12:13:44.285 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-11T12:13:44.285 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-11T12:13:44.286 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-11T12:13:44.286 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-11T12:13:44.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:13:44.286 DEBUG:teuthology.orchestra.run.smithi204:> ip addr 2024-05-11T12:13:44.345 INFO:teuthology.orchestra.run.smithi204.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-11T12:13:44.345 INFO:teuthology.orchestra.run.smithi204.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-11T12:13:44.345 INFO:teuthology.orchestra.run.smithi204.stdout: inet 127.0.0.1/8 scope host lo 2024-05-11T12:13:44.345 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:13:44.345 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 ::1/128 scope host 2024-05-11T12:13:44.345 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:13:44.345 INFO:teuthology.orchestra.run.smithi204.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:13:44.345 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2a brd ff:ff:ff:ff:ff:ff 2024-05-11T12:13:44.345 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f0 2024-05-11T12:13:44.345 INFO:teuthology.orchestra.run.smithi204.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:13:44.346 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2b brd ff:ff:ff:ff:ff:ff 2024-05-11T12:13:44.346 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f1 2024-05-11T12:13:44.346 INFO:teuthology.orchestra.run.smithi204.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:13:44.346 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:aa brd ff:ff:ff:ff:ff:ff 2024-05-11T12:13:44.346 INFO:teuthology.orchestra.run.smithi204.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-11T12:13:44.346 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:ab brd ff:ff:ff:ff:ff:ff 2024-05-11T12:13:44.346 INFO:teuthology.orchestra.run.smithi204.stdout: inet 172.21.15.204/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-11T12:13:44.346 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft 42440sec preferred_lft 42440sec 2024-05-11T12:13:44.346 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::ec4:7aff:fe88:77ab/64 scope link 2024-05-11T12:13:44.346 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:13:44.346 INFO:teuthology.orchestra.run.smithi204.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-11T12:13:44.346 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether e2:9e:5b:47:45:5d brd ff:ff:ff:ff:ff:ff 2024-05-11T12:13:44.346 INFO:teuthology.orchestra.run.smithi204.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-11T12:13:44.347 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:13:44.347 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::a498:7aff:fe50:48c3/64 scope link 2024-05-11T12:13:44.347 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:13:44.347 INFO:teuthology.orchestra.run.smithi204.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-11T12:13:44.347 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether fa:0a:7f:e4:30:35 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-11T12:13:44.347 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::f80a:7fff:fee4:3035/64 scope link 2024-05-11T12:13:44.347 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:13:44.347 INFO:teuthology.orchestra.run.smithi204.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-11T12:13:44.347 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether e2:9e:5b:47:45:5d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-11T12:13:44.347 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::e09e:5bff:fe47:455d/64 scope link 2024-05-11T12:13:44.347 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:13:44.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:13:44.348 DEBUG:teuthology.orchestra.run.smithi204:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-11T12:13:44.426 INFO:teuthology.orchestra.run.smithi204.stdout:1 2024-05-11T12:13:44.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:13:44.431 DEBUG:teuthology.orchestra.run.smithi204:> route 2024-05-11T12:13:44.483 INFO:teuthology.orchestra.run.smithi204.stdout:Kernel IP routing table 2024-05-11T12:13:44.483 INFO:teuthology.orchestra.run.smithi204.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-11T12:13:44.483 INFO:teuthology.orchestra.run.smithi204.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-11T12:13:44.483 INFO:teuthology.orchestra.run.smithi204.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-11T12:13:44.483 INFO:teuthology.orchestra.run.smithi204.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-11T12:13:44.484 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T12:13:44.484 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-05-11T12:13:44.484 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-11T12:13:44.484 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-11T12:13:44.484 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-11T12:13:44.484 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-05-11T12:13:44.586 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:13:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T12:13:44.682 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:13:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:13:44.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:13:44.691 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list 2024-05-11T12:13:44.736 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-11T12:13:44.737 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-11T12:13:44.737 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-11T12:13:44.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:13:44.738 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link set brx.0 up 2024-05-11T12:13:44.869 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T12:13:44.869 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-11T12:13:44.923 INFO:teuthology.orchestra.run.smithi204.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-11T12:13:44.923 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T12:13:44.923 DEBUG:teuthology.orchestra.run.smithi204:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-11T12:13:44.973 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-05-11T12:13:44.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:13:44.973 DEBUG:teuthology.orchestra.run.smithi204:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:13:45.102 INFO:teuthology.orchestra.run.smithi204.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:13:45.102 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-11T12:13:45.135 INFO:teuthology.orchestra.run.smithi204.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:13:45.135 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-11T12:13:45.136 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: resolved to: "172.21.15.167:3300,172.21.15.204:3300,172.21.15.204:3301" 2024-05-11T12:13:45.136 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: trying mount with new device syntax: 1@5f32d344-7289-4c27-881b-1533419ab678.cephfs=/ 2024-05-11T12:13:45.136 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.167:3300/172.21.15.204:3300/172.21.15.204:3301" will pass to kernel 2024-05-11T12:13:45.138 INFO:teuthology.orchestra.run.smithi204.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-05-11T12:13:45.138 INFO:teuthology.orchestra.run.smithi204.stdout: You just mounted a file system that supports labels which does not 2024-05-11T12:13:45.138 INFO:teuthology.orchestra.run.smithi204.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-11T12:13:45.139 INFO:teuthology.orchestra.run.smithi204.stdout: applications will generate AVC messages and not be allowed access to 2024-05-11T12:13:45.139 INFO:teuthology.orchestra.run.smithi204.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-11T12:13:45.143 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-11T12:13:45.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:13:45.143 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-11T12:13:45.212 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-11T12:13:45.314 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:13:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-11T12:13:45.319 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:13:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:13:45.328 INFO:teuthology.orchestra.run.smithi204.stdout:client5086 2024-05-11T12:13:45.329 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:13:45.329 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:13:45.377 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds ms_async_reap_threshold 1 2024-05-11T12:13:45.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.587+0000 7f46d3e00640 1 Processor -- start 2024-05-11T12:13:45.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.587+0000 7f46d3e00640 1 -- start start 2024-05-11T12:13:45.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.589+0000 7f46d3e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f46d41610c0 0x7f46d4167d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:45.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.589+0000 7f46d3e00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46d40cf5c0 con 0x7f46d413fe30 2024-05-11T12:13:45.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.589+0000 7f46d3e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46d40cf8a0 con 0x7f46d40d0660 2024-05-11T12:13:45.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.589+0000 7f46d3e00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f46d40cfb80 con 0x7f46d41610c0 2024-05-11T12:13:45.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.589+0000 7f46d2a00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f46d413fe30 0x7f46d415de40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:42394/0 (socket says 172.21.15.167:42394) 2024-05-11T12:13:45.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.589+0000 7f46d2a00640 1 -- 172.21.15.167:0/452432807 learned_addr learned my addr 172.21.15.167:0/452432807 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:13:45.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.589+0000 7f46d3400640 1 --2- 172.21.15.167:0/452432807 >> v2:172.21.15.167:3300/0 conn(0x7f46d41610c0 0x7f46d4167d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:45.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.589+0000 7f46d1600640 1 -- 172.21.15.167:0/452432807 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1760044467 0 0) 0x7f46d40cf5c0 con 0x7f46d413fe30 2024-05-11T12:13:45.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.589+0000 7f46d1600640 1 -- 172.21.15.167:0/452432807 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46b8003520 con 0x7f46d413fe30 2024-05-11T12:13:45.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.589+0000 7f46d3400640 1 -- 172.21.15.167:0/452432807 >> v1:172.21.15.204:6789/0 conn(0x7f46d40d0660 legacy=0x7f46d41609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:45.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.589+0000 7f46d3400640 1 -- 172.21.15.167:0/452432807 >> v1:172.21.15.167:6789/0 conn(0x7f46d413fe30 legacy=0x7f46d415de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:45.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.590+0000 7f46d3400640 1 -- 172.21.15.167:0/452432807 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d40cfe60 con 0x7f46d41610c0 2024-05-11T12:13:45.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.591+0000 7f46d3400640 1 --2- 172.21.15.167:0/452432807 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46d41610c0 0x7f46d4167d40 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f46cc009d90 tx=0x7f46cc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1b384e6baea69d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:45.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.591+0000 7f46d1600640 1 -- 172.21.15.167:0/452432807 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46cc007740 con 0x7f46d41610c0 2024-05-11T12:13:45.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.591+0000 7f46d1600640 1 -- 172.21.15.167:0/452432807 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46cc0078e0 con 0x7f46d41610c0 2024-05-11T12:13:45.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.591+0000 7f46d1600640 1 -- 172.21.15.167:0/452432807 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46cc007cb0 con 0x7f46d41610c0 2024-05-11T12:13:45.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.591+0000 7f46d3e00640 1 -- 172.21.15.167:0/452432807 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46d41610c0 msgr2=0x7f46d4167d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:45.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.591+0000 7f46d3e00640 1 --2- 172.21.15.167:0/452432807 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46d41610c0 0x7f46d4167d40 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f46cc009d90 tx=0x7f46cc02a660 comp rx=0 tx=0).stop 2024-05-11T12:13:45.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.591+0000 7f46d3e00640 1 -- 172.21.15.167:0/452432807 shutdown_connections 2024-05-11T12:13:45.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.591+0000 7f46d3e00640 1 --2- 172.21.15.167:0/452432807 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46d41610c0 0x7f46d4167d40 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:45.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.591+0000 7f46d3e00640 1 -- 172.21.15.167:0/452432807 >> 172.21.15.167:0/452432807 conn(0x7f46d40b2a20 msgr2=0x7f46d40b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:45.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.592+0000 7f46d3e00640 1 -- 172.21.15.167:0/452432807 shutdown_connections 2024-05-11T12:13:45.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.592+0000 7f46d3e00640 1 -- 172.21.15.167:0/452432807 wait complete. 2024-05-11T12:13:45.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.593+0000 7f46d3e00640 1 Processor -- start 2024-05-11T12:13:45.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.593+0000 7f46d3e00640 1 -- start start 2024-05-11T12:13:45.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.594+0000 7f46d3e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46d40d0660 0x7f46d4155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:45.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.594+0000 7f46d3e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46d413fe30 0x7f46d41e7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:45.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.594+0000 7f46d3e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46d41610c0 0x7f46d41e7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:45.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.594+0000 7f46d3e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f46d4161820 con 0x7f46d41610c0 2024-05-11T12:13:45.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.594+0000 7f46d3e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f46d41616a0 con 0x7f46d413fe30 2024-05-11T12:13:45.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.594+0000 7f46d3e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f46d40cf190 con 0x7f46d40d0660 2024-05-11T12:13:45.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.595+0000 7f46d3400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46d41610c0 0x7f46d41e7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:45.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.595+0000 7f46d3400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46d41610c0 0x7f46d41e7d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41410/0 (socket says 172.21.15.167:41410) 2024-05-11T12:13:45.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.595+0000 7f46d3400640 1 -- 172.21.15.167:0/2323510950 learned_addr learned my addr 172.21.15.167:0/2323510950 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:45.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.595+0000 7f46d2a00640 1 --2- 172.21.15.167:0/2323510950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46d40d0660 0x7f46d4155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:45.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.595+0000 7f46d3400640 1 -- 172.21.15.167:0/2323510950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46d40d0660 msgr2=0x7f46d4155fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:45.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.595+0000 7f46d3400640 1 --2- 172.21.15.167:0/2323510950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46d40d0660 0x7f46d4155fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:45.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.595+0000 7f46d3400640 1 -- 172.21.15.167:0/2323510950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46d413fe30 msgr2=0x7f46d41e7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:45.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.595+0000 7f46d2000640 1 --2- 172.21.15.167:0/2323510950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46d413fe30 0x7f46d41e7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:45.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.595+0000 7f46d3400640 1 --2- 172.21.15.167:0/2323510950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46d413fe30 0x7f46d41e7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:45.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.595+0000 7f46d3400640 1 -- 172.21.15.167:0/2323510950 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d415c500 con 0x7f46d41610c0 2024-05-11T12:13:45.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.595+0000 7f46d3400640 1 --2- 172.21.15.167:0/2323510950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46d41610c0 0x7f46d41e7d80 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f46cc009eb0 tx=0x7f46cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:45.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.596+0000 7f46caa00640 1 -- 172.21.15.167:0/2323510950 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46cc040070 con 0x7f46d41610c0 2024-05-11T12:13:45.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.596+0000 7f46caa00640 1 -- 172.21.15.167:0/2323510950 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46cc031bc0 con 0x7f46d41610c0 2024-05-11T12:13:45.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.596+0000 7f46caa00640 1 -- 172.21.15.167:0/2323510950 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46cc009450 con 0x7f46d41610c0 2024-05-11T12:13:45.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.596+0000 7f46d3e00640 1 -- 172.21.15.167:0/2323510950 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46d415c7e0 con 0x7f46d41610c0 2024-05-11T12:13:45.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.596+0000 7f46d3e00640 1 -- 172.21.15.167:0/2323510950 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46d413b3b0 con 0x7f46d41610c0 2024-05-11T12:13:45.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.597+0000 7f46d3e00640 1 -- 172.21.15.167:0/2323510950 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46d40694b0 con 0x7f46d41610c0 2024-05-11T12:13:45.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.598+0000 7f46caa00640 1 -- 172.21.15.167:0/2323510950 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f46cc039040 con 0x7f46d41610c0 2024-05-11T12:13:45.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.599+0000 7f46caa00640 1 --2- 172.21.15.167:0/2323510950 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f46b40762d0 0x7f46b4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:45.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.599+0000 7f46caa00640 1 -- 172.21.15.167:0/2323510950 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7f46cc0b8870 con 0x7f46d41610c0 2024-05-11T12:13:45.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.600+0000 7f46d2a00640 1 --2- 172.21.15.167:0/2323510950 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f46b40762d0 0x7f46b4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:45.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.600+0000 7f46d2a00640 1 --2- 172.21.15.167:0/2323510950 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f46b40762d0 0x7f46b4078790 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f46c4001220 tx=0x7f46c4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:45.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.604+0000 7f46caa00640 1 -- 172.21.15.167:0/2323510950 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f46cc081950 con 0x7f46d41610c0 2024-05-11T12:13:45.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.811+0000 7f46d3e00640 1 -- 172.21.15.167:0/2323510950 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command([{prefix=config set, name=ms_async_reap_threshold}] v 0) -- 0x7f46d413b580 con 0x7f46d41610c0 2024-05-11T12:13:45.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.816+0000 7f46caa00640 1 -- 172.21.15.167:0/2323510950 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{prefix=config set, name=ms_async_reap_threshold}]=0 v19)=0 v19) ==== 121+0+0 (secure 0 0 0) 0x7f46cc085c50 con 0x7f46d41610c0 2024-05-11T12:13:45.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.820+0000 7f46d3e00640 1 -- 172.21.15.167:0/2323510950 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f46b40762d0 msgr2=0x7f46b4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:45.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.820+0000 7f46d3e00640 1 --2- 172.21.15.167:0/2323510950 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f46b40762d0 0x7f46b4078790 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f46c4001220 tx=0x7f46c4009040 comp rx=0 tx=0).stop 2024-05-11T12:13:45.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.820+0000 7f46d3e00640 1 -- 172.21.15.167:0/2323510950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46d41610c0 msgr2=0x7f46d41e7d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:45.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.820+0000 7f46d3e00640 1 --2- 172.21.15.167:0/2323510950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46d41610c0 0x7f46d41e7d80 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f46cc009eb0 tx=0x7f46cc004a40 comp rx=0 tx=0).stop 2024-05-11T12:13:45.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.820+0000 7f46d3e00640 1 -- 172.21.15.167:0/2323510950 shutdown_connections 2024-05-11T12:13:45.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.820+0000 7f46d3e00640 1 --2- 172.21.15.167:0/2323510950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46d41610c0 0x7f46d41e7d80 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:45.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.820+0000 7f46d3e00640 1 --2- 172.21.15.167:0/2323510950 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f46b40762d0 0x7f46b4078790 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:45.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.820+0000 7f46d3e00640 1 --2- 172.21.15.167:0/2323510950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46d413fe30 0x7f46d41e7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:45.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.820+0000 7f46d3e00640 1 --2- 172.21.15.167:0/2323510950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46d40d0660 0x7f46d4155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:45.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.820+0000 7f46d3e00640 1 -- 172.21.15.167:0/2323510950 >> 172.21.15.167:0/2323510950 conn(0x7f46d40b2a20 msgr2=0x7f46d40b6890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:45.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.820+0000 7f46d3e00640 1 -- 172.21.15.167:0/2323510950 shutdown_connections 2024-05-11T12:13:45.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:45.820+0000 7f46d3e00640 1 -- 172.21.15.167:0/2323510950 wait complete. 2024-05-11T12:13:45.840 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:13:45.840 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:13:45.887 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:13:45.887 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:13:45.983 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-11T12:13:46.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.190+0000 7fe9ab800640 1 Processor -- start 2024-05-11T12:13:46.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.190+0000 7fe9ab800640 1 -- start start 2024-05-11T12:13:46.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.192+0000 7fe9ab800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe9a4147ff0 0x7fe9a40d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:46.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.192+0000 7fe9ab800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe9a4148cf0 0x7fe9a40b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:46.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.192+0000 7fe9ab800640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9a40d1430 con 0x7fe9a40d2e80 2024-05-11T12:13:46.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.192+0000 7fe9ab800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fe9a40d2040 con 0x7fe9a4148cf0 2024-05-11T12:13:46.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.192+0000 7fe9ab800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe9a40d1710 con 0x7fe9a4147ff0 2024-05-11T12:13:46.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.192+0000 7fe9a9800640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fe9a40d2e80 0x7fe9a415f950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:42408/0 (socket says 172.21.15.167:42408) 2024-05-11T12:13:46.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.192+0000 7fe9a9800640 1 -- 172.21.15.167:0/2214337480 learned_addr learned my addr 172.21.15.167:0/2214337480 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:13:46.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.193+0000 7fe9a3e00640 1 --2- 172.21.15.167:0/2214337480 >> v2:172.21.15.204:3300/0 conn(0x7fe9a4148cf0 0x7fe9a40b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:46.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.193+0000 7fe9a8e00640 1 --2- 172.21.15.167:0/2214337480 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe9a4147ff0 0x7fe9a40d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:46.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.193+0000 7fe9a3400640 1 -- 172.21.15.167:0/2214337480 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 114148779 0 0) 0x7fe9a40d1430 con 0x7fe9a40d2e80 2024-05-11T12:13:46.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.193+0000 7fe9a3400640 1 -- 172.21.15.167:0/2214337480 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe984003520 con 0x7fe9a40d2e80 2024-05-11T12:13:46.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.193+0000 7fe9a3400640 1 -- 172.21.15.167:0/2214337480 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1712223429 0 0) 0x7fe984003520 con 0x7fe9a40d2e80 2024-05-11T12:13:46.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.193+0000 7fe9a3400640 1 -- 172.21.15.167:0/2214337480 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe9a4147ff0 msgr2=0x7fe9a40d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:46.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.193+0000 7fe9a3400640 1 --2- 172.21.15.167:0/2214337480 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe9a4147ff0 0x7fe9a40d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:46.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.193+0000 7fe9a3400640 1 -- 172.21.15.167:0/2214337480 >> v2:172.21.15.204:3300/0 conn(0x7fe9a4148cf0 msgr2=0x7fe9a40b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:46.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.193+0000 7fe9a3400640 1 --2- 172.21.15.167:0/2214337480 >> v2:172.21.15.204:3300/0 conn(0x7fe9a4148cf0 0x7fe9a40b5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:46.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.193+0000 7fe9a3400640 1 -- 172.21.15.167:0/2214337480 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9a40d19f0 con 0x7fe9a40d2e80 2024-05-11T12:13:46.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.193+0000 7fe9a3400640 1 -- 172.21.15.167:0/2214337480 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7fe99c0016f0 con 0x7fe9a40d2e80 2024-05-11T12:13:46.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.193+0000 7fe9a3400640 1 -- 172.21.15.167:0/2214337480 >> v1:172.21.15.167:6789/0 conn(0x7fe9a40d2e80 legacy=0x7fe9a415f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:46.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.193+0000 7fe9a3400640 1 --2- 172.21.15.167:0/2214337480 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe9a4147ff0 0x7fe984003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:46.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.193+0000 7fe9a3400640 1 --2- 172.21.15.167:0/2214337480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe984003fd0 0x7fe984004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:46.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.194+0000 7fe9a3400640 1 --2- 172.21.15.167:0/2214337480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe984004d40 0x7fe9840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:46.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.194+0000 7fe9a3400640 1 -- 172.21.15.167:0/2214337480 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a40d2040 con 0x7fe984004d40 2024-05-11T12:13:46.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.194+0000 7fe9a3400640 1 -- 172.21.15.167:0/2214337480 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a40d1710 con 0x7fe984003fd0 2024-05-11T12:13:46.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.194+0000 7fe9a3400640 1 -- 172.21.15.167:0/2214337480 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe98400ba10 con 0x7fe9a4147ff0 2024-05-11T12:13:46.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.194+0000 7fe9a9800640 1 --2- 172.21.15.167:0/2214337480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe984004d40 0x7fe9840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:46.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.194+0000 7fe9a3e00640 1 --2- 172.21.15.167:0/2214337480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe984003fd0 0x7fe984004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:46.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.194+0000 7fe9a8e00640 1 --2- 172.21.15.167:0/2214337480 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe9a4147ff0 0x7fe984003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:46.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.194+0000 7fe9a9800640 1 -- 172.21.15.167:0/2214337480 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe9a4147ff0 msgr2=0x7fe984003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:46.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.194+0000 7fe9a9800640 1 --2- 172.21.15.167:0/2214337480 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe9a4147ff0 0x7fe984003a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:46.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.194+0000 7fe9a9800640 1 -- 172.21.15.167:0/2214337480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe984003fd0 msgr2=0x7fe984004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:46.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.194+0000 7fe9a9800640 1 --2- 172.21.15.167:0/2214337480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe984003fd0 0x7fe984004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:46.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.194+0000 7fe9a9800640 1 -- 172.21.15.167:0/2214337480 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe984003520 con 0x7fe984004d40 2024-05-11T12:13:46.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.195+0000 7fe9a9800640 1 --2- 172.21.15.167:0/2214337480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe984004d40 0x7fe9840051c0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fe99c00c240 tx=0x7fe984005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84372fc6f568f7b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:46.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.196+0000 7fe9a3400640 1 -- 172.21.15.167:0/2214337480 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99c007d30 con 0x7fe984004d40 2024-05-11T12:13:46.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.196+0000 7fe9a3400640 1 -- 172.21.15.167:0/2214337480 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe99c005af0 con 0x7fe984004d40 2024-05-11T12:13:46.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.196+0000 7fe9a3400640 1 -- 172.21.15.167:0/2214337480 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99c007070 con 0x7fe984004d40 2024-05-11T12:13:46.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.196+0000 7fe9ab800640 1 -- 172.21.15.167:0/2214337480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe984004d40 msgr2=0x7fe9840051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:46.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.196+0000 7fe9ab800640 1 --2- 172.21.15.167:0/2214337480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe984004d40 0x7fe9840051c0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fe99c00c240 tx=0x7fe984005830 comp rx=0 tx=0).stop 2024-05-11T12:13:46.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.196+0000 7fe9a9800640 1 -- 172.21.15.167:0/2214337480 reap_dead start 2024-05-11T12:13:46.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.196+0000 7fe9ab800640 1 -- 172.21.15.167:0/2214337480 shutdown_connections 2024-05-11T12:13:46.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.196+0000 7fe9ab800640 1 -- 172.21.15.167:0/2214337480 >> 172.21.15.167:0/2214337480 conn(0x7fe9a40b8300 msgr2=0x7fe9a40b86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:46.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.196+0000 7fe9ab800640 1 -- 172.21.15.167:0/2214337480 shutdown_connections 2024-05-11T12:13:46.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.196+0000 7fe9ab800640 1 -- 172.21.15.167:0/2214337480 wait complete. 2024-05-11T12:13:46.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.198+0000 7fe9ab800640 1 Processor -- start 2024-05-11T12:13:46.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.198+0000 7fe9ab800640 1 -- start start 2024-05-11T12:13:46.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.199+0000 7fe9ab800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe984004d40 0x7fe9a415e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:46.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.199+0000 7fe9ab800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe9a415ea70 0x7fe9a40cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:46.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.199+0000 7fe9ab800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9a413b4c0 0x7fe9a413b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:46.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.199+0000 7fe9ab800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a40b6320 con 0x7fe9a413b4c0 2024-05-11T12:13:46.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.199+0000 7fe9ab800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a41483d0 con 0x7fe984004d40 2024-05-11T12:13:46.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.199+0000 7fe9ab800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe9a40b64a0 con 0x7fe9a415ea70 2024-05-11T12:13:46.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.200+0000 7fe9a9800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9a413b4c0 0x7fe9a413b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:46.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.200+0000 7fe9a9800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9a413b4c0 0x7fe9a413b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41440/0 (socket says 172.21.15.167:41440) 2024-05-11T12:13:46.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.200+0000 7fe9a9800640 1 -- 172.21.15.167:0/3696294155 learned_addr learned my addr 172.21.15.167:0/3696294155 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:46.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.200+0000 7fe9a3e00640 1 --2- 172.21.15.167:0/3696294155 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe9a415ea70 0x7fe9a40cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:46.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.200+0000 7fe9a8e00640 1 --2- 172.21.15.167:0/3696294155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe984004d40 0x7fe9a415e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:46.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.200+0000 7fe9a9800640 1 -- 172.21.15.167:0/3696294155 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe9a415ea70 msgr2=0x7fe9a40cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:46.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.200+0000 7fe9a9800640 1 --2- 172.21.15.167:0/3696294155 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe9a415ea70 0x7fe9a40cf7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:46.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.200+0000 7fe9a9800640 1 -- 172.21.15.167:0/3696294155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe984004d40 msgr2=0x7fe9a415e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:46.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.200+0000 7fe9a9800640 1 --2- 172.21.15.167:0/3696294155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe984004d40 0x7fe9a415e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:46.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.200+0000 7fe9a9800640 1 -- 172.21.15.167:0/3696294155 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9a415e760 con 0x7fe9a413b4c0 2024-05-11T12:13:46.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.200+0000 7fe9a9800640 1 --2- 172.21.15.167:0/3696294155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9a413b4c0 0x7fe9a413b950 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fe99c00c330 tx=0x7fe9a40d1600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:46.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.201+0000 7fe9a0c00640 1 -- 172.21.15.167:0/3696294155 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99c037040 con 0x7fe9a413b4c0 2024-05-11T12:13:46.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.201+0000 7fe9a0c00640 1 -- 172.21.15.167:0/3696294155 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe99c041070 con 0x7fe9a413b4c0 2024-05-11T12:13:46.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.201+0000 7fe9a0c00640 1 -- 172.21.15.167:0/3696294155 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99c0016f0 con 0x7fe9a413b4c0 2024-05-11T12:13:46.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.201+0000 7fe9ab800640 1 -- 172.21.15.167:0/3696294155 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9a413a0c0 con 0x7fe9a413b4c0 2024-05-11T12:13:46.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.201+0000 7fe9ab800640 1 -- 172.21.15.167:0/3696294155 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9a413a680 con 0x7fe9a413b4c0 2024-05-11T12:13:46.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.202+0000 7fe9ab800640 1 -- 172.21.15.167:0/3696294155 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9a4143730 con 0x7fe9a413b4c0 2024-05-11T12:13:46.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.203+0000 7fe9a0c00640 1 -- 172.21.15.167:0/3696294155 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe99c0072b0 con 0x7fe9a413b4c0 2024-05-11T12:13:46.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.204+0000 7fe9a0c00640 1 --2- 172.21.15.167:0/3696294155 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe9800762d0 0x7fe980078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:46.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.204+0000 7fe9a0c00640 1 -- 172.21.15.167:0/3696294155 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7fe99c0b7610 con 0x7fe9a413b4c0 2024-05-11T12:13:46.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.204+0000 7fe9a8e00640 1 --2- 172.21.15.167:0/3696294155 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe9800762d0 0x7fe980078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:46.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.205+0000 7fe9a8e00640 1 --2- 172.21.15.167:0/3696294155 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe9800762d0 0x7fe980078790 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe994009910 tx=0x7fe994008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:46.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.209+0000 7fe9a0c00640 1 -- 172.21.15.167:0/3696294155 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe9a4143730 con 0x7fe9a413b4c0 2024-05-11T12:13:46.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.431+0000 7fe9ab800640 1 -- 172.21.15.167:0/3696294155 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe9a40694b0 con 0x7fe9a413b4c0 2024-05-11T12:13:46.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.432+0000 7fe9a0c00640 1 -- 172.21.15.167:0/3696294155 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v75) ==== 80+0+0 (secure 0 0 0) 0x7fe99c080d40 con 0x7fe9a413b4c0 2024-05-11T12:13:46.431 INFO:teuthology.orchestra.run.smithi167.stderr:listed 0 entries 2024-05-11T12:13:46.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.434+0000 7fe9ab800640 1 -- 172.21.15.167:0/3696294155 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe9800762d0 msgr2=0x7fe980078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:46.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.434+0000 7fe9ab800640 1 --2- 172.21.15.167:0/3696294155 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe9800762d0 0x7fe980078790 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe994009910 tx=0x7fe994008040 comp rx=0 tx=0).stop 2024-05-11T12:13:46.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.434+0000 7fe9ab800640 1 -- 172.21.15.167:0/3696294155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9a413b4c0 msgr2=0x7fe9a413b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:46.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.434+0000 7fe9ab800640 1 --2- 172.21.15.167:0/3696294155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9a413b4c0 0x7fe9a413b950 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fe99c00c330 tx=0x7fe9a40d1600 comp rx=0 tx=0).stop 2024-05-11T12:13:46.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.434+0000 7fe9ab800640 1 -- 172.21.15.167:0/3696294155 shutdown_connections 2024-05-11T12:13:46.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.435+0000 7fe9ab800640 1 --2- 172.21.15.167:0/3696294155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9a413b4c0 0x7fe9a413b950 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:46.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.435+0000 7fe9ab800640 1 --2- 172.21.15.167:0/3696294155 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe9a415ea70 0x7fe9a40cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:46.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.435+0000 7fe9ab800640 1 --2- 172.21.15.167:0/3696294155 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe9800762d0 0x7fe980078790 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:46.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.435+0000 7fe9ab800640 1 --2- 172.21.15.167:0/3696294155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe984004d40 0x7fe9a415e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:46.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.435+0000 7fe9ab800640 1 -- 172.21.15.167:0/3696294155 >> 172.21.15.167:0/3696294155 conn(0x7fe9a40b8300 msgr2=0x7fe9a40b40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:46.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.435+0000 7fe9ab800640 1 -- 172.21.15.167:0/3696294155 shutdown_connections 2024-05-11T12:13:46.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.435+0000 7fe9ab800640 1 -- 172.21.15.167:0/3696294155 wait complete. 2024-05-11T12:13:46.453 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-11T12:13:46.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:13:46.453 DEBUG:teuthology.orchestra.run.smithi167:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-11T12:13:46.546 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2024-05-11T12:13:46.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:13:46.546 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-11T12:13:46.596 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:13:46.596 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:13:46.654 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:13:46.654 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:13:46.706 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-11T12:13:46.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.917+0000 7f4cd6600640 1 Processor -- start 2024-05-11T12:13:46.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.917+0000 7f4cd6600640 1 -- start start 2024-05-11T12:13:46.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.918+0000 7f4cd6600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4cd01625d0 0x7f4cd00d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:46.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.918+0000 7f4cd6600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4cd00b8390 0x7f4cd00b8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:46.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.918+0000 7f4cd6600640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4cd00d3c00 0x7f4cd00d4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:46.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.918+0000 7f4cd6600640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f4cd00d45b0 con 0x7f4cd00d3c00 2024-05-11T12:13:46.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.918+0000 7f4cd6600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f4cd00d2e40 con 0x7f4cd01625d0 2024-05-11T12:13:46.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.918+0000 7f4cd6600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4cd00d3120 con 0x7f4cd00b8390 2024-05-11T12:13:46.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.919+0000 7f4cd5c00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4cd00d3c00 0x7f4cd00d4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:46.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.919+0000 7f4cd5c00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4cd00d3c00 0x7f4cd00d4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41454/0 (socket says 172.21.15.167:41454) 2024-05-11T12:13:46.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.919+0000 7f4cd5c00640 1 -- 172.21.15.167:0/4050479393 learned_addr learned my addr 172.21.15.167:0/4050479393 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:46.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.919+0000 7f4ccfe00640 1 --2- 172.21.15.167:0/4050479393 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4cd00b8390 0x7f4cd00b8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:46.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.919+0000 7f4cd5200640 1 --2- 172.21.15.167:0/4050479393 >> v2:172.21.15.204:3300/0 conn(0x7f4cd01625d0 0x7f4cd00d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:46.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.919+0000 7f4cd5c00640 1 -- 172.21.15.167:0/4050479393 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4cd00b8390 msgr2=0x7f4cd00b8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:46.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.919+0000 7f4cd5c00640 1 --2- 172.21.15.167:0/4050479393 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4cd00b8390 0x7f4cd00b8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:46.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.919+0000 7f4cd5c00640 1 -- 172.21.15.167:0/4050479393 >> v2:172.21.15.204:3300/0 conn(0x7f4cd01625d0 msgr2=0x7f4cd00d0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:46.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.919+0000 7f4cd5c00640 1 --2- 172.21.15.167:0/4050479393 >> v2:172.21.15.204:3300/0 conn(0x7f4cd01625d0 0x7f4cd00d0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:46.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.919+0000 7f4cd5c00640 1 -- 172.21.15.167:0/4050479393 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cd00d3400 con 0x7f4cd00d3c00 2024-05-11T12:13:46.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.921+0000 7f4cd5c00640 1 --2- 172.21.15.167:0/4050479393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cd00d3c00 0x7f4cd00d4070 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f4cd00b7c10 tx=0x7f4cc802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ced68fc11598b571 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:46.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.921+0000 7f4ccf400640 1 -- 172.21.15.167:0/4050479393 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc8037070 con 0x7f4cd00d3c00 2024-05-11T12:13:46.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.921+0000 7f4ccf400640 1 -- 172.21.15.167:0/4050479393 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cc8032440 con 0x7f4cd00d3c00 2024-05-11T12:13:46.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.921+0000 7f4ccf400640 1 -- 172.21.15.167:0/4050479393 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc8032810 con 0x7f4cd00d3c00 2024-05-11T12:13:46.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.922+0000 7f4cd6600640 1 -- 172.21.15.167:0/4050479393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cd00d3c00 msgr2=0x7f4cd00d4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:46.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.922+0000 7f4cd6600640 1 --2- 172.21.15.167:0/4050479393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cd00d3c00 0x7f4cd00d4070 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f4cd00b7c10 tx=0x7f4cc802a1e0 comp rx=0 tx=0).stop 2024-05-11T12:13:46.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.922+0000 7f4cd6600640 1 -- 172.21.15.167:0/4050479393 shutdown_connections 2024-05-11T12:13:46.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.922+0000 7f4cd6600640 1 --2- 172.21.15.167:0/4050479393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cd00d3c00 0x7f4cd00d4070 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:46.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.922+0000 7f4cd6600640 1 --2- 172.21.15.167:0/4050479393 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4cd00b8390 0x7f4cd00b8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:46.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.922+0000 7f4cd6600640 1 --2- 172.21.15.167:0/4050479393 >> v2:172.21.15.204:3300/0 conn(0x7f4cd01625d0 0x7f4cd00d0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:46.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.922+0000 7f4cd6600640 1 -- 172.21.15.167:0/4050479393 >> 172.21.15.167:0/4050479393 conn(0x7f4cd00d50f0 msgr2=0x7f4cd00d5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:46.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.922+0000 7f4cd6600640 1 -- 172.21.15.167:0/4050479393 shutdown_connections 2024-05-11T12:13:46.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.922+0000 7f4cd6600640 1 -- 172.21.15.167:0/4050479393 wait complete. 2024-05-11T12:13:46.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.924+0000 7f4cd6600640 1 Processor -- start 2024-05-11T12:13:46.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.924+0000 7f4cd6600640 1 -- start start 2024-05-11T12:13:46.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.925+0000 7f4cd6600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4cd00b8390 0x7f4cd0149380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:46.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.925+0000 7f4cd6600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4cd00d3c00 0x7f4cd00cc250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:46.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.925+0000 7f4cd6600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cd01625d0 0x7f4cd014be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:46.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.925+0000 7f4cd6600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd0162b30 con 0x7f4cd01625d0 2024-05-11T12:13:46.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.925+0000 7f4cd6600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd01629b0 con 0x7f4cd00b8390 2024-05-11T12:13:46.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.925+0000 7f4cd6600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4cd0162d50 con 0x7f4cd00d3c00 2024-05-11T12:13:46.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.925+0000 7f4cd5c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cd01625d0 0x7f4cd014be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:46.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.925+0000 7f4cd5c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cd01625d0 0x7f4cd014be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41464/0 (socket says 172.21.15.167:41464) 2024-05-11T12:13:46.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.925+0000 7f4cd5c00640 1 -- 172.21.15.167:0/3227389521 learned_addr learned my addr 172.21.15.167:0/3227389521 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:46.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.926+0000 7f4ccfe00640 1 --2- 172.21.15.167:0/3227389521 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4cd00d3c00 0x7f4cd00cc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:46.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.926+0000 7f4cd5200640 1 --2- 172.21.15.167:0/3227389521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4cd00b8390 0x7f4cd0149380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:46.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.926+0000 7f4cd5c00640 1 -- 172.21.15.167:0/3227389521 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4cd00d3c00 msgr2=0x7f4cd00cc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:46.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.926+0000 7f4cd5c00640 1 --2- 172.21.15.167:0/3227389521 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4cd00d3c00 0x7f4cd00cc250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:46.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.926+0000 7f4cd5c00640 1 -- 172.21.15.167:0/3227389521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4cd00b8390 msgr2=0x7f4cd0149380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:46.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.926+0000 7f4cd5c00640 1 --2- 172.21.15.167:0/3227389521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4cd00b8390 0x7f4cd0149380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:46.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.926+0000 7f4cd5c00640 1 -- 172.21.15.167:0/3227389521 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cd014c360 con 0x7f4cd01625d0 2024-05-11T12:13:46.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.926+0000 7f4cd5c00640 1 --2- 172.21.15.167:0/3227389521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cd01625d0 0x7f4cd014be20 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f4cc80090b0 tx=0x7f4cc80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:46.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.926+0000 7f4cccc00640 1 -- 172.21.15.167:0/3227389521 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc8037040 con 0x7f4cd01625d0 2024-05-11T12:13:46.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.926+0000 7f4cccc00640 1 -- 172.21.15.167:0/3227389521 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cc8042070 con 0x7f4cd01625d0 2024-05-11T12:13:46.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.926+0000 7f4cccc00640 1 -- 172.21.15.167:0/3227389521 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc8030920 con 0x7f4cd01625d0 2024-05-11T12:13:46.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.926+0000 7f4cd6600640 1 -- 172.21.15.167:0/3227389521 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cd00cdd30 con 0x7f4cd01625d0 2024-05-11T12:13:46.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.927+0000 7f4cd6600640 1 -- 172.21.15.167:0/3227389521 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cd00ce2b0 con 0x7f4cd01625d0 2024-05-11T12:13:46.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.928+0000 7f4cd6600640 1 -- 172.21.15.167:0/3227389521 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cd00694b0 con 0x7f4cd01625d0 2024-05-11T12:13:46.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.929+0000 7f4cccc00640 1 -- 172.21.15.167:0/3227389521 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4cc8030ac0 con 0x7f4cd01625d0 2024-05-11T12:13:46.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.930+0000 7f4cccc00640 1 --2- 172.21.15.167:0/3227389521 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4cac076300 0x7f4cac0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:46.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.930+0000 7f4cccc00640 1 -- 172.21.15.167:0/3227389521 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7f4cc80b7ff0 con 0x7f4cd01625d0 2024-05-11T12:13:46.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.930+0000 7f4cd5200640 1 --2- 172.21.15.167:0/3227389521 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4cac076300 0x7f4cac0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:46.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.931+0000 7f4cd5200640 1 --2- 172.21.15.167:0/3227389521 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4cac076300 0x7f4cac0787c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0009e90 tx=0x7f4cc000b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:46.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:46.935+0000 7f4cccc00640 1 -- 172.21.15.167:0/3227389521 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4cd014c360 con 0x7f4cd01625d0 2024-05-11T12:13:47.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.157+0000 7f4cd6600640 1 -- 172.21.15.167:0/3227389521 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4cd00d4080 con 0x7f4cd01625d0 2024-05-11T12:13:47.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.157+0000 7f4cccc00640 1 -- 172.21.15.167:0/3227389521 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v75) ==== 80+0+0 (secure 0 0 0) 0x7f4cc8081810 con 0x7f4cd01625d0 2024-05-11T12:13:47.158 INFO:teuthology.orchestra.run.smithi167.stderr:listed 0 entries 2024-05-11T12:13:47.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.160+0000 7f4cd6600640 1 -- 172.21.15.167:0/3227389521 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4cac076300 msgr2=0x7f4cac0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:47.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.160+0000 7f4cd6600640 1 --2- 172.21.15.167:0/3227389521 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4cac076300 0x7f4cac0787c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0009e90 tx=0x7f4cc000b040 comp rx=0 tx=0).stop 2024-05-11T12:13:47.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.160+0000 7f4cd6600640 1 -- 172.21.15.167:0/3227389521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cd01625d0 msgr2=0x7f4cd014be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:47.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.160+0000 7f4cd6600640 1 --2- 172.21.15.167:0/3227389521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cd01625d0 0x7f4cd014be20 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f4cc80090b0 tx=0x7f4cc80046c0 comp rx=0 tx=0).stop 2024-05-11T12:13:47.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.160+0000 7f4cd6600640 1 -- 172.21.15.167:0/3227389521 shutdown_connections 2024-05-11T12:13:47.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.160+0000 7f4cd6600640 1 --2- 172.21.15.167:0/3227389521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cd01625d0 0x7f4cd014be20 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:47.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.160+0000 7f4cd6600640 1 --2- 172.21.15.167:0/3227389521 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4cd00d3c00 0x7f4cd00cc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:47.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.160+0000 7f4cd6600640 1 --2- 172.21.15.167:0/3227389521 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4cac076300 0x7f4cac0787c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:47.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.160+0000 7f4cd6600640 1 --2- 172.21.15.167:0/3227389521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4cd00b8390 0x7f4cd0149380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:47.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.160+0000 7f4cd6600640 1 -- 172.21.15.167:0/3227389521 >> 172.21.15.167:0/3227389521 conn(0x7f4cd00d50f0 msgr2=0x7f4cd006ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:47.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.160+0000 7f4cd6600640 1 -- 172.21.15.167:0/3227389521 shutdown_connections 2024-05-11T12:13:47.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.160+0000 7f4cd6600640 1 -- 172.21.15.167:0/3227389521 wait complete. 2024-05-11T12:13:47.180 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-11T12:13:47.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:13:47.180 DEBUG:teuthology.orchestra.run.smithi204:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-05-11T12:13:47.276 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi204.front.sepia.ceph.com 2024-05-11T12:13:47.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:13:47.277 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-11T12:13:47.328 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:13:47.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.539+0000 7fb192200640 1 Processor -- start 2024-05-11T12:13:47.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.540+0000 7fb192200640 1 -- start start 2024-05-11T12:13:47.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.541+0000 7fb192200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb18c0cb6c0 0x7fb18c06dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:47.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.541+0000 7fb192200640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb18c0d1190 con 0x7fb18c1598d0 2024-05-11T12:13:47.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.541+0000 7fb192200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb18c06d500 con 0x7fb18c0d2cf0 2024-05-11T12:13:47.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.541+0000 7fb192200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fb18c06d7e0 con 0x7fb18c0cb6c0 2024-05-11T12:13:47.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.541+0000 7fb18b400640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fb18c1598d0 0x7fb18c0d0640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:42414/0 (socket says 172.21.15.167:42414) 2024-05-11T12:13:47.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.541+0000 7fb18b400640 1 -- 172.21.15.167:0/1468262265 learned_addr learned my addr 172.21.15.167:0/1468262265 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:13:47.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.542+0000 7fb18be00640 1 --2- 172.21.15.167:0/1468262265 >> v2:172.21.15.204:3300/0 conn(0x7fb18c0cb6c0 0x7fb18c06dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:47.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.542+0000 7fb18a000640 1 -- 172.21.15.167:0/1468262265 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 208888043 0 0) 0x7fb18c0d1190 con 0x7fb18c1598d0 2024-05-11T12:13:47.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.542+0000 7fb18a000640 1 -- 172.21.15.167:0/1468262265 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb170003520 con 0x7fb18c1598d0 2024-05-11T12:13:47.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.542+0000 7fb18a000640 1 -- 172.21.15.167:0/1468262265 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2936314076 0 0) 0x7fb170003520 con 0x7fb18c1598d0 2024-05-11T12:13:47.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.542+0000 7fb18a000640 1 -- 172.21.15.167:0/1468262265 >> v2:172.21.15.204:3300/0 conn(0x7fb18c0cb6c0 msgr2=0x7fb18c06dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:47.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.542+0000 7fb18a000640 1 --2- 172.21.15.167:0/1468262265 >> v2:172.21.15.204:3300/0 conn(0x7fb18c0cb6c0 0x7fb18c06dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:47.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.542+0000 7fb18a000640 1 -- 172.21.15.167:0/1468262265 >> v1:172.21.15.204:6789/0 conn(0x7fb18c0d2cf0 legacy=0x7fb18c0cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:47.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.542+0000 7fb18a000640 1 -- 172.21.15.167:0/1468262265 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb18c06dac0 con 0x7fb18c1598d0 2024-05-11T12:13:47.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.542+0000 7fb18a000640 1 -- 172.21.15.167:0/1468262265 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7fb17c001a70 con 0x7fb18c1598d0 2024-05-11T12:13:47.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.542+0000 7fb18a000640 1 -- 172.21.15.167:0/1468262265 >> v1:172.21.15.167:6789/0 conn(0x7fb18c1598d0 legacy=0x7fb18c0d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:47.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.543+0000 7fb18a000640 1 --2- 172.21.15.167:0/1468262265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb170003a50 0x7fb170003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:47.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.543+0000 7fb18a000640 1 --2- 172.21.15.167:0/1468262265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb170004800 0x7fb170004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:47.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.543+0000 7fb18a000640 1 --2- 172.21.15.167:0/1468262265 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1700051c0 0x7fb17000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:47.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.543+0000 7fb18a000640 1 -- 172.21.15.167:0/1468262265 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb18c06d7e0 con 0x7fb170003a50 2024-05-11T12:13:47.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.543+0000 7fb18a000640 1 -- 172.21.15.167:0/1468262265 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb17000c160 con 0x7fb170004800 2024-05-11T12:13:47.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.543+0000 7fb18a000640 1 -- 172.21.15.167:0/1468262265 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb17000c360 con 0x7fb1700051c0 2024-05-11T12:13:47.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.543+0000 7fb18b400640 1 --2- 172.21.15.167:0/1468262265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb170003a50 0x7fb170003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:47.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.543+0000 7fb18aa00640 1 --2- 172.21.15.167:0/1468262265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb170004800 0x7fb170004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:47.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.543+0000 7fb18be00640 1 --2- 172.21.15.167:0/1468262265 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1700051c0 0x7fb17000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:47.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.543+0000 7fb18b400640 1 -- 172.21.15.167:0/1468262265 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1700051c0 msgr2=0x7fb17000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:47.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.543+0000 7fb18b400640 1 --2- 172.21.15.167:0/1468262265 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1700051c0 0x7fb17000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:47.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.543+0000 7fb18b400640 1 -- 172.21.15.167:0/1468262265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb170004800 msgr2=0x7fb170004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:47.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.543+0000 7fb18b400640 1 --2- 172.21.15.167:0/1468262265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb170004800 0x7fb170004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:47.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.543+0000 7fb18b400640 1 -- 172.21.15.167:0/1468262265 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb170003520 con 0x7fb170003a50 2024-05-11T12:13:47.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.543+0000 7fb18be00640 1 -- 172.21.15.167:0/1468262265 reap_dead start 2024-05-11T12:13:47.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.544+0000 7fb18b400640 1 --2- 172.21.15.167:0/1468262265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb170003a50 0x7fb170003e70 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fb17c00a8c0 tx=0x7fb170005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf7f428b79d97071 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:47.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.545+0000 7fb18a000640 1 -- 172.21.15.167:0/1468262265 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17c00a4e0 con 0x7fb170003a50 2024-05-11T12:13:47.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.545+0000 7fb18a000640 1 -- 172.21.15.167:0/1468262265 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb17c02f070 con 0x7fb170003a50 2024-05-11T12:13:47.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.545+0000 7fb18a000640 1 -- 172.21.15.167:0/1468262265 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17c02bc10 con 0x7fb170003a50 2024-05-11T12:13:47.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.545+0000 7fb192200640 1 -- 172.21.15.167:0/1468262265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb170003a50 msgr2=0x7fb170003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:47.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.545+0000 7fb192200640 1 --2- 172.21.15.167:0/1468262265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb170003a50 0x7fb170003e70 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fb17c00a8c0 tx=0x7fb170005830 comp rx=0 tx=0).stop 2024-05-11T12:13:47.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.545+0000 7fb192200640 1 -- 172.21.15.167:0/1468262265 shutdown_connections 2024-05-11T12:13:47.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.545+0000 7fb192200640 1 --2- 172.21.15.167:0/1468262265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb170003a50 0x7fb170003e70 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:47.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.545+0000 7fb192200640 1 -- 172.21.15.167:0/1468262265 >> 172.21.15.167:0/1468262265 conn(0x7fb18c0d1ba0 msgr2=0x7fb18c0d1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:47.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.545+0000 7fb192200640 1 -- 172.21.15.167:0/1468262265 shutdown_connections 2024-05-11T12:13:47.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.546+0000 7fb192200640 1 -- 172.21.15.167:0/1468262265 wait complete. 2024-05-11T12:13:47.548 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.547+0000 7fb192200640 1 Processor -- start 2024-05-11T12:13:47.548 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.547+0000 7fb192200640 1 -- start start 2024-05-11T12:13:47.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.548+0000 7fb192200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb170003a50 0x7fb18c146bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:47.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.548+0000 7fb192200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb18c0cdac0 0x7fb18c0cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:47.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.548+0000 7fb192200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb18c156f90 0x7fb18c0b99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:47.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.548+0000 7fb192200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb18c06d1b0 con 0x7fb18c0cdac0 2024-05-11T12:13:47.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.548+0000 7fb192200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb18c159cb0 con 0x7fb170003a50 2024-05-11T12:13:47.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.548+0000 7fb192200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb18c0cafb0 con 0x7fb18c156f90 2024-05-11T12:13:47.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.549+0000 7fb18aa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb18c0cdac0 0x7fb18c0cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:47.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.549+0000 7fb18aa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb18c0cdac0 0x7fb18c0cc1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41484/0 (socket says 172.21.15.167:41484) 2024-05-11T12:13:47.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.549+0000 7fb18aa00640 1 -- 172.21.15.167:0/3554192478 learned_addr learned my addr 172.21.15.167:0/3554192478 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:47.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.549+0000 7fb18b400640 1 --2- 172.21.15.167:0/3554192478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb170003a50 0x7fb18c146bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:47.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.549+0000 7fb18be00640 1 --2- 172.21.15.167:0/3554192478 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb18c156f90 0x7fb18c0b99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:47.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.549+0000 7fb18aa00640 1 -- 172.21.15.167:0/3554192478 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb18c156f90 msgr2=0x7fb18c0b99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:47.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.549+0000 7fb18aa00640 1 --2- 172.21.15.167:0/3554192478 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb18c156f90 0x7fb18c0b99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:47.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.549+0000 7fb18aa00640 1 -- 172.21.15.167:0/3554192478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb170003a50 msgr2=0x7fb18c146bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:47.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.549+0000 7fb18aa00640 1 --2- 172.21.15.167:0/3554192478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb170003a50 0x7fb18c146bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:47.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.549+0000 7fb18aa00640 1 -- 172.21.15.167:0/3554192478 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb18c0cca20 con 0x7fb18c0cdac0 2024-05-11T12:13:47.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.549+0000 7fb18b400640 1 --2- 172.21.15.167:0/3554192478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb170003a50 0x7fb18c146bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:47.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.549+0000 7fb18be00640 1 --2- 172.21.15.167:0/3554192478 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb18c156f90 0x7fb18c0b99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:47.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.549+0000 7fb18aa00640 1 --2- 172.21.15.167:0/3554192478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb18c0cdac0 0x7fb18c0cc1f0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fb17800d4f0 tx=0x7fb17800d9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:47.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.550+0000 7fb183400640 1 -- 172.21.15.167:0/3554192478 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb178005280 con 0x7fb18c0cdac0 2024-05-11T12:13:47.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.550+0000 7fb183400640 1 -- 172.21.15.167:0/3554192478 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb178005420 con 0x7fb18c0cdac0 2024-05-11T12:13:47.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.550+0000 7fb183400640 1 -- 172.21.15.167:0/3554192478 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb178007250 con 0x7fb18c0cdac0 2024-05-11T12:13:47.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.550+0000 7fb192200640 1 -- 172.21.15.167:0/3554192478 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb18c0ccd00 con 0x7fb18c0cdac0 2024-05-11T12:13:47.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.550+0000 7fb192200640 1 -- 172.21.15.167:0/3554192478 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb18c0cd2c0 con 0x7fb18c0cdac0 2024-05-11T12:13:47.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.551+0000 7fb192200640 1 -- 172.21.15.167:0/3554192478 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb18c069590 con 0x7fb18c0cdac0 2024-05-11T12:13:47.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.552+0000 7fb183400640 1 -- 172.21.15.167:0/3554192478 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb17801a030 con 0x7fb18c0cdac0 2024-05-11T12:13:47.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.553+0000 7fb183400640 1 --2- 172.21.15.167:0/3554192478 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb16c076300 0x7fb16c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:47.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.553+0000 7fb183400640 1 -- 172.21.15.167:0/3554192478 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7fb178096d30 con 0x7fb18c0cdac0 2024-05-11T12:13:47.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.553+0000 7fb18b400640 1 --2- 172.21.15.167:0/3554192478 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb16c076300 0x7fb16c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:47.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.554+0000 7fb18b400640 1 --2- 172.21.15.167:0/3554192478 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb16c076300 0x7fb16c0787c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb17c033000 tx=0x7fb17c007000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:47.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.558+0000 7fb183400640 1 -- 172.21.15.167:0/3554192478 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb178021020 con 0x7fb18c0cdac0 2024-05-11T12:13:47.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.803+0000 7fb192200640 1 -- 172.21.15.167:0/3554192478 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb18c1579c0 con 0x7fb18c0cdac0 2024-05-11T12:13:47.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.804+0000 7fb183400640 1 -- 172.21.15.167:0/3554192478 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5195 (secure 0 0 0) 0x7fb1780604f0 con 0x7fb18c0cdac0 2024-05-11T12:13:47.805 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:13:47.805 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5020,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3075930607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3075930607},{"type":"v1","addr":"172.21.15.167:6836","nonce":3075930607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":5032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3056051531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3056051531},{"type":"v1","addr":"172.21.15.204:6834","nonce":3056051531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5035,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2900478909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":2900478909},{"type":"v1","addr":"172.21.15.204:6835","nonce":2900478909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:13:34.993949+0000","modified":"2024-05-11T12:13:38.072880+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5952},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5952":{"gid":5952,"name":"a","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/1306616214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1306616214},{"type":"v1","addr":"172.21.15.167:6837","nonce":1306616214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5952,"qdb_cluster":[5952]},"id":5}]} 2024-05-11T12:13:47.805 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 44 2024-05-11T12:13:47.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.809+0000 7fb192200640 1 -- 172.21.15.167:0/3554192478 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb16c076300 msgr2=0x7fb16c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:47.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.809+0000 7fb192200640 1 --2- 172.21.15.167:0/3554192478 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb16c076300 0x7fb16c0787c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb17c033000 tx=0x7fb17c007000 comp rx=0 tx=0).stop 2024-05-11T12:13:47.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.809+0000 7fb192200640 1 -- 172.21.15.167:0/3554192478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb18c0cdac0 msgr2=0x7fb18c0cc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:47.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.809+0000 7fb192200640 1 --2- 172.21.15.167:0/3554192478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb18c0cdac0 0x7fb18c0cc1f0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fb17800d4f0 tx=0x7fb17800d9b0 comp rx=0 tx=0).stop 2024-05-11T12:13:47.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.809+0000 7fb192200640 1 -- 172.21.15.167:0/3554192478 shutdown_connections 2024-05-11T12:13:47.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.809+0000 7fb192200640 1 --2- 172.21.15.167:0/3554192478 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb18c156f90 0x7fb18c0b99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:47.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.809+0000 7fb192200640 1 --2- 172.21.15.167:0/3554192478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb18c0cdac0 0x7fb18c0cc1f0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:47.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.809+0000 7fb192200640 1 --2- 172.21.15.167:0/3554192478 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb16c076300 0x7fb16c0787c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:47.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.809+0000 7fb192200640 1 --2- 172.21.15.167:0/3554192478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb170003a50 0x7fb18c146bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:47.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.809+0000 7fb192200640 1 -- 172.21.15.167:0/3554192478 >> 172.21.15.167:0/3554192478 conn(0x7fb18c0d1ba0 msgr2=0x7fb18c1609d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:47.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.809+0000 7fb192200640 1 -- 172.21.15.167:0/3554192478 shutdown_connections 2024-05-11T12:13:47.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:47.809+0000 7fb192200640 1 -- 172.21.15.167:0/3554192478 wait complete. 2024-05-11T12:13:47.827 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json perf dump 2024-05-11T12:13:48.038 INFO:teuthology.orchestra.run.smithi167.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":23,"msgr_send_messages":25,"msgr_recv_bytes":6215,"msgr_send_bytes":13671,"msgr_created_connections":7,"msgr_active_connections":2,"msgr_running_total_time":0.022900864,"msgr_running_send_time":0.001640621,"msgr_running_recv_time":0.015709157,"msgr_running_fast_dispatch_time":0.001736825,"msgr_send_messages_queue_lat":{"avgcount":24,"sum":0.296538751,"avgtime":0.012355781},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"AsyncMessenger::Worker-1":{"msgr_recv_messages":88,"msgr_send_messages":80,"msgr_recv_bytes":131181,"msgr_send_bytes":50511,"msgr_created_connections":7,"msgr_active_connections":3,"msgr_running_total_time":0.027221550,"msgr_running_send_time":0.009257158,"msgr_running_recv_time":0.040523185,"msgr_running_fast_dispatch_time":0.000628931,"msgr_send_messages_queue_lat":{"avgcount":79,"sum":0.108806917,"avgtime":0.001377302},"msgr_handle_ack_lat":{"avgcount":58,"sum":0.000021452,"avgtime":0.000000369},"msgr_recv_encrypted_bytes":120432,"msgr_send_encrypted_bytes":10288},"AsyncMessenger::Worker-2":{"msgr_recv_messages":26,"msgr_send_messages":28,"msgr_recv_bytes":7293,"msgr_send_bytes":110203,"msgr_created_connections":6,"msgr_active_connections":3,"msgr_running_total_time":0.014011146,"msgr_running_send_time":0.002113709,"msgr_running_recv_time":0.011117022,"msgr_running_fast_dispatch_time":0.000521552,"msgr_send_messages_queue_lat":{"avgcount":27,"sum":0.202264497,"avgtime":0.007491277},"msgr_handle_ack_lat":{"avgcount":12,"sum":0.000003600,"avgtime":0.000000300},"msgr_recv_encrypted_bytes":1728,"msgr_send_encrypted_bytes":96096},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":38,"sum":0.004935829,"avgtime":0.000129890}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":2,"sum":0.000028579,"avgtime":0.000014289}},"mds":{"request":29,"reply":29,"reply_latency":{"avgcount":29,"sum":0.012307906,"avgtime":0.000424410},"slow_reply":0,"forward":0,"dir_fetch_complete":0,"dir_fetch_keys":0,"dir_commit":12,"dir_split":0,"dir_merge":0,"inodes":11,"inodes_top":0,"inodes_bottom":11,"inodes_pin_tail":0,"inodes_pinned":10,"inodes_expired":0,"inodes_with_caps":1,"caps":2,"subtrees":2,"traverse":55,"traverse_hit":53,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":0,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":0,"handle_client_caps_dirty":0,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_quiesce_path":0,"ireq_quiesce_inode":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evlrg":0,"evadd":8,"evex":0,"evtrm":0,"ev":8,"evexg":0,"evexd":0,"segadd":2,"segex":0,"segtrm":0,"seg":2,"segmjr":2,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4197029,"rdpos":0,"jlat":{"avgcount":5,"sum":0.019663414,"avgtime":0.003932682},"replayed":0},"mds_mem":{"ino":13,"ino+":13,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":11,"dn+":11,"dn-":0,"cap":0,"cap+":2,"cap-":2,"rss":40704,"heap":231788},"mds_server":{"dispatch_client_request":29,"dispatch_server_request":0,"handle_client_request":29,"handle_client_session":6,"handle_peer_request":0,"req_create_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_getattr_latency":{"avgcount":26,"sum":0.010614534,"avgtime":0.000408251},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":2,"sum":0.000445241,"avgtime":0.000222620},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":1,"sum":0.001248131,"avgtime":0.001248131},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":0,"session_add":2,"session_remove":2,"sessions_open":0,"sessions_stale":0,"total_load":28,"average_load":14,"avg_session_uptime":1,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":217253,"buffer_anon_items":85,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5424,"osdmap_items":76,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":53784,"mds_co_items":218,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":43,"op_send_bytes":13845,"op_resend":28,"op_reply":43,"op_latency":{"avgcount":43,"sum":1.388976541,"avgtime":0.032301780},"op_inflight":0,"oplen_avg":{"avgcount":43,"sum":67},"op":43,"op_r":0,"op_w":43,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":11,"osdop_read":0,"osdop_write":5,"osdop_writefull":7,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":22,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":75,"map_full":0,"map_inc":1,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":15,"omap_rd":0,"omap_del":1},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":0,"omap_total_updates":0,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":133,"get_sum":128827,"get_or_fail_fail":0,"get_or_fail_success":133,"take":0,"take_sum":0,"put":133,"put_sum":128827,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":43,"take_sum":13845,"put":43,"put_sum":13845,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":43,"take_sum":43,"put":43,"put_sum":43,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x56187b98c500":{"val":0,"max":3758096384,"get_started":0,"get":8,"get_sum":2725,"get_or_fail_fail":0,"get_or_fail_success":8,"take":0,"take_sum":0,"put":5,"put_sum":2725,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-05-11T12:13:48.039 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 2, "msgr_created_connections": 7, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 6215, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 23, "msgr_running_fast_dispatch_time": 0.001736825, "msgr_running_recv_time": 0.015709157, "msgr_running_send_time": 0.001640621, "msgr_running_total_time": 0.022900864, "msgr_send_bytes": 13671, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 25, "msgr_send_messages_queue_lat": { "avgcount": 24, "avgtime": 0.012355781, "sum": 0.296538751 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 3, "msgr_created_connections": 7, "msgr_handle_ack_lat": { "avgcount": 58, "avgtime": 3.69e-07, "sum": 2.1452e-05 }, "msgr_recv_bytes": 131181, "msgr_recv_encrypted_bytes": 120432, "msgr_recv_messages": 88, "msgr_running_fast_dispatch_time": 0.000628931, "msgr_running_recv_time": 0.040523185, "msgr_running_send_time": 0.009257158, "msgr_running_total_time": 0.02722155, "msgr_send_bytes": 50511, "msgr_send_encrypted_bytes": 10288, "msgr_send_messages": 80, "msgr_send_messages_queue_lat": { "avgcount": 79, "avgtime": 0.001377302, "sum": 0.108806917 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 3, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 12, "avgtime": 3e-07, "sum": 3.6e-06 }, "msgr_recv_bytes": 7293, "msgr_recv_encrypted_bytes": 1728, "msgr_recv_messages": 26, "msgr_running_fast_dispatch_time": 0.000521552, "msgr_running_recv_time": 0.011117022, "msgr_running_send_time": 0.002113709, "msgr_running_total_time": 0.014011146, "msgr_send_bytes": 110203, "msgr_send_encrypted_bytes": 96096, "msgr_send_messages": 28, "msgr_send_messages_queue_lat": { "avgcount": 27, "avgtime": 0.007491277, "sum": 0.202264497 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 38, "avgtime": 0.00012989, "sum": 0.004935829 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 2, "avgtime": 1.4289e-05, "sum": 2.8579e-05 }, "queue_len": 0 }, "mds": { "caps": 2, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 12, "dir_fetch_complete": 0, "dir_fetch_keys": 0, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 0, "handle_client_caps_dirty": 0, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 11, "inodes_bottom": 11, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 10, "inodes_top": 0, "inodes_with_caps": 1, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 29, "reply_latency": { "avgcount": 29, "avgtime": 0.00042441, "sum": 0.012307906 }, "request": 29, "root_rbytes": 0, "root_rfiles": 0, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 55, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 53, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "ireq_quiesce_inode": 0, "ireq_quiesce_path": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 8, "evadd": 8, "evex": 0, "evexd": 0, "evexg": 0, "evlrg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 5, "avgtime": 0.003932682, "sum": 0.019663414 }, "rdpos": 0, "replayed": 0, "seg": 2, "segadd": 2, "segex": 0, "segexd": 0, "segexg": 0, "segmjr": 2, "segtrm": 0, "wrpos": 4197029 }, "mds_mem": { "cap": 0, "cap+": 2, "cap-": 2, "dir": 12, "dir+": 12, "dir-": 0, "dn": 11, "dn+": 11, "dn-": 0, "heap": 231788, "ino": 13, "ino+": 13, "ino-": 0, "rss": 40704 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 29, "dispatch_server_request": 0, "handle_client_request": 29, "handle_client_session": 6, "handle_peer_request": 0, "req_create_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getattr_latency": { "avgcount": 26, "avgtime": 0.000408251, "sum": 0.010614534 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 2, "avgtime": 0.00022262, "sum": 0.000445241 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 1, "avgtime": 0.001248131, "sum": 0.001248131 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 14, "avg_session_uptime": 1, "mdthresh_evicted": 0, "session_add": 2, "session_count": 0, "session_remove": 2, "sessions_open": 0, "sessions_stale": 0, "total_load": 28 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 217253, "buffer_anon_items": 85, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 53784, "mds_co_items": 218, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5424, "osdmap_items": 76, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 75, "map_full": 0, "map_inc": 1, "omap_del": 1, "omap_rd": 0, "omap_wr": 15, "op": 43, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 43, "avgtime": 0.03230178, "sum": 1.388976541 }, "op_pg": 0, "op_r": 0, "op_reply": 43, "op_resend": 28, "op_rmw": 0, "op_send": 43, "op_send_bytes": 13845, "op_w": 43, "oplen_avg": { "avgcount": 43, "sum": 67 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 11, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 22, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 5, "osdop_writefull": 7, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 0, "omap_total_objs": 1, "omap_total_removes": 0, "omap_total_updates": 0 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 133, "get_or_fail_fail": 0, "get_or_fail_success": 133, "get_started": 0, "get_sum": 128827, "max": 104857600, "put": 133, "put_sum": 128827, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 43, "put_sum": 13845, "take": 43, "take_sum": 13845, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 43, "put_sum": 43, "take": 43, "take_sum": 43, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x56187b98c500": { "get": 8, "get_or_fail_fail": 0, "get_or_fail_success": 8, "get_started": 0, "get_sum": 2725, "max": 3758096384, "put": 5, "put_sum": 2725, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-05-11T12:13:48.039 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 session ls 2024-05-11T12:13:48.245 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.245+0000 7fd9b6400640 1 Processor -- start 2024-05-11T12:13:48.245 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.245+0000 7fd9b6400640 1 -- start start 2024-05-11T12:13:48.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.246+0000 7fd9b6400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9b015ac80 0x7fd9b0074260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:48.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.247+0000 7fd9b6400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd9b0161380 0x7fd9b0072950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:48.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.247+0000 7fd9b6400640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9b00b65e0 con 0x7fd9b00b3d50 2024-05-11T12:13:48.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.247+0000 7fd9b6400640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fd9b006e3c0 con 0x7fd9b0161380 2024-05-11T12:13:48.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.247+0000 7fd9b6400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd9b015b060 con 0x7fd9b015ac80 2024-05-11T12:13:48.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.247+0000 7fd9afe00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd9b0161380 0x7fd9b0072950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:48.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.247+0000 7fd9a7e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fd9b00b3d50 0x7fd9b0160c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:42430/0 (socket says 172.21.15.167:42430) 2024-05-11T12:13:48.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.247+0000 7fd9a7e00640 1 -- 172.21.15.167:0/1670803724 learned_addr learned my addr 172.21.15.167:0/1670803724 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:13:48.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.247+0000 7fd9af400640 1 --2- 172.21.15.167:0/1670803724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9b015ac80 0x7fd9b0074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:48.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.247+0000 7fd9afe00640 1 -- 172.21.15.167:0/1670803724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9b015ac80 msgr2=0x7fd9b0074260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:48.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.247+0000 7fd9afe00640 1 --2- 172.21.15.167:0/1670803724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9b015ac80 0x7fd9b0074260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:48.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.247+0000 7fd9afe00640 1 -- 172.21.15.167:0/1670803724 >> v1:172.21.15.167:6789/0 conn(0x7fd9b00b3d50 legacy=0x7fd9b0160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:48.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.248+0000 7fd9aea00640 1 -- 172.21.15.167:0/1670803724 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4208743162 0 0) 0x7fd9b00b65e0 con 0x7fd9b00b3d50 2024-05-11T12:13:48.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.248+0000 7fd9afe00640 1 -- 172.21.15.167:0/1670803724 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9b00b68c0 con 0x7fd9b0161380 2024-05-11T12:13:48.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.249+0000 7fd9afe00640 1 --2- 172.21.15.167:0/1670803724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9b0161380 0x7fd9b0072950 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8009d90 tx=0x7fd9a802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b0326782b99882a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:48.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.249+0000 7fd9aea00640 1 -- 172.21.15.167:0/1670803724 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a8007740 con 0x7fd9b0161380 2024-05-11T12:13:48.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.249+0000 7fd9aea00640 1 -- 172.21.15.167:0/1670803724 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9a80078e0 con 0x7fd9b0161380 2024-05-11T12:13:48.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.249+0000 7fd9aea00640 1 -- 172.21.15.167:0/1670803724 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a8007cb0 con 0x7fd9b0161380 2024-05-11T12:13:48.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.250+0000 7fd9b6400640 1 -- 172.21.15.167:0/1670803724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9b0161380 msgr2=0x7fd9b0072950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:48.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.250+0000 7fd9b6400640 1 --2- 172.21.15.167:0/1670803724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9b0161380 0x7fd9b0072950 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8009d90 tx=0x7fd9a802a660 comp rx=0 tx=0).stop 2024-05-11T12:13:48.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.250+0000 7fd9b6400640 1 -- 172.21.15.167:0/1670803724 shutdown_connections 2024-05-11T12:13:48.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.250+0000 7fd9b6400640 1 --2- 172.21.15.167:0/1670803724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9b0161380 0x7fd9b0072950 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:48.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.250+0000 7fd9b6400640 1 --2- 172.21.15.167:0/1670803724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9b015ac80 0x7fd9b0074260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:48.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.250+0000 7fd9b6400640 1 -- 172.21.15.167:0/1670803724 >> 172.21.15.167:0/1670803724 conn(0x7fd9b006c450 msgr2=0x7fd9b006c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:48.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.250+0000 7fd9b6400640 1 -- 172.21.15.167:0/1670803724 shutdown_connections 2024-05-11T12:13:48.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.250+0000 7fd9b6400640 1 -- 172.21.15.167:0/1670803724 wait complete. 2024-05-11T12:13:48.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.252+0000 7fd9b6400640 1 Processor -- start 2024-05-11T12:13:48.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.252+0000 7fd9b6400640 1 -- start start 2024-05-11T12:13:48.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.253+0000 7fd9b6400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9b00b3d50 0x7fd9b0139620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:48.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.253+0000 7fd9b6400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9b015ac80 0x7fd9b00ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:48.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.253+0000 7fd9b6400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9b0161380 0x7fd9b01e7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:48.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.254+0000 7fd9b6400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b00b44f0 con 0x7fd9b0161380 2024-05-11T12:13:48.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.254+0000 7fd9b6400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b00b41b0 con 0x7fd9b015ac80 2024-05-11T12:13:48.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.254+0000 7fd9b6400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd9b0072e90 con 0x7fd9b00b3d50 2024-05-11T12:13:48.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.254+0000 7fd9afe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9b0161380 0x7fd9b01e7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:48.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.254+0000 7fd9afe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9b0161380 0x7fd9b01e7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41504/0 (socket says 172.21.15.167:41504) 2024-05-11T12:13:48.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.254+0000 7fd9afe00640 1 -- 172.21.15.167:0/3477220263 learned_addr learned my addr 172.21.15.167:0/3477220263 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:48.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.254+0000 7fd9a7e00640 1 --2- 172.21.15.167:0/3477220263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9b015ac80 0x7fd9b00ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:48.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.254+0000 7fd9af400640 1 --2- 172.21.15.167:0/3477220263 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9b00b3d50 0x7fd9b0139620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:48.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.254+0000 7fd9afe00640 1 -- 172.21.15.167:0/3477220263 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9b00b3d50 msgr2=0x7fd9b0139620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:48.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.254+0000 7fd9afe00640 1 --2- 172.21.15.167:0/3477220263 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9b00b3d50 0x7fd9b0139620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:48.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.254+0000 7fd9afe00640 1 -- 172.21.15.167:0/3477220263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9b015ac80 msgr2=0x7fd9b00ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:48.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.254+0000 7fd9afe00640 1 --2- 172.21.15.167:0/3477220263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9b015ac80 0x7fd9b00ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:48.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.254+0000 7fd9afe00640 1 -- 172.21.15.167:0/3477220263 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9b00cf370 con 0x7fd9b0161380 2024-05-11T12:13:48.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.254+0000 7fd9a7e00640 1 --2- 172.21.15.167:0/3477220263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9b015ac80 0x7fd9b00ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:48.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.255+0000 7fd9afe00640 1 --2- 172.21.15.167:0/3477220263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9b0161380 0x7fd9b01e7730 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd9a802a6a0 tx=0x7fd9a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:48.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.255+0000 7fd9a7400640 1 -- 172.21.15.167:0/3477220263 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a803f070 con 0x7fd9b0161380 2024-05-11T12:13:48.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.255+0000 7fd9a7400640 1 -- 172.21.15.167:0/3477220263 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9a8003b90 con 0x7fd9b0161380 2024-05-11T12:13:48.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.255+0000 7fd9a7400640 1 -- 172.21.15.167:0/3477220263 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a8031940 con 0x7fd9b0161380 2024-05-11T12:13:48.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.255+0000 7fd9b6400640 1 -- 172.21.15.167:0/3477220263 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9b00cf650 con 0x7fd9b0161380 2024-05-11T12:13:48.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.255+0000 7fd9b6400640 1 -- 172.21.15.167:0/3477220263 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9b00cfc10 con 0x7fd9b0161380 2024-05-11T12:13:48.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.258+0000 7fd9a7400640 1 -- 172.21.15.167:0/3477220263 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd9a8031ae0 con 0x7fd9b0161380 2024-05-11T12:13:48.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.259+0000 7fd9a7400640 1 --2- 172.21.15.167:0/3477220263 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd98c0762d0 0x7fd98c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:48.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.260+0000 7fd9a7400640 1 -- 172.21.15.167:0/3477220263 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7fd9a80b89a0 con 0x7fd9b0161380 2024-05-11T12:13:48.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.260+0000 7fd9b9174740 1 Processor -- start 2024-05-11T12:13:48.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.260+0000 7fd9af400640 1 --2- 172.21.15.167:0/3477220263 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd98c0762d0 0x7fd98c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:48.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.260+0000 7fd9b9174740 1 -- start start 2024-05-11T12:13:48.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.261+0000 7fd9af400640 1 --2- 172.21.15.167:0/3477220263 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd98c0762d0 0x7fd98c078790 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0006fd0 tx=0x7fd9a0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:48.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.261+0000 7fd9b9174740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x561dc7989770 0x561dc7989b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:48.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.261+0000 7fd9b9174740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc79a7010 0x561dc79a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:48.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.261+0000 7fd9b9174740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561dc795f450 0x561dc79a56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:48.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.261+0000 7fd9b9174740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561dc78c3d50 con 0x561dc79a7010 2024-05-11T12:13:48.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.261+0000 7fd9b9174740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x561dc78690c0 con 0x561dc795f450 2024-05-11T12:13:48.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.261+0000 7fd9b9174740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x561dc78bfd10 con 0x561dc7989770 2024-05-11T12:13:48.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.261+0000 7fd9a7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc79a7010 0x561dc79a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:48.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.261+0000 7fd9a7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc79a7010 0x561dc79a7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41510/0 (socket says 172.21.15.167:41510) 2024-05-11T12:13:48.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.261+0000 7fd9a7e00640 1 -- 172.21.15.167:0/805969782 learned_addr learned my addr 172.21.15.167:0/805969782 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:48.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.262+0000 7fd9afe00640 1 --2- 172.21.15.167:0/805969782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561dc795f450 0x561dc79a56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:48.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.262+0000 7fd9af400640 1 --2- 172.21.15.167:0/805969782 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x561dc7989770 0x561dc7989b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:48.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.262+0000 7fd9a7e00640 1 -- 172.21.15.167:0/805969782 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x561dc7989770 msgr2=0x561dc7989b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:48.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.262+0000 7fd9a7e00640 1 --2- 172.21.15.167:0/805969782 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x561dc7989770 0x561dc7989b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:48.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.262+0000 7fd9a7e00640 1 -- 172.21.15.167:0/805969782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561dc795f450 msgr2=0x561dc79a56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:48.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.262+0000 7fd9a7e00640 1 --2- 172.21.15.167:0/805969782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561dc795f450 0x561dc79a56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:48.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.262+0000 7fd9a7e00640 1 -- 172.21.15.167:0/805969782 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dc75d53f0 con 0x561dc79a7010 2024-05-11T12:13:48.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.262+0000 7fd9afe00640 1 --2- 172.21.15.167:0/805969782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561dc795f450 0x561dc79a56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:48.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.262+0000 7fd9a7e00640 1 --2- 172.21.15.167:0/805969782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc79a7010 0x561dc79a7560 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fd998001880 tx=0x7fd998001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=985ef77da41b6c43 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:48.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.262+0000 7fd99fe00640 1 -- 172.21.15.167:0/805969782 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9980074e0 con 0x561dc79a7010 2024-05-11T12:13:48.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.262+0000 7fd99fe00640 1 -- 172.21.15.167:0/805969782 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd998007680 con 0x561dc79a7010 2024-05-11T12:13:48.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.262+0000 7fd99fe00640 1 -- 172.21.15.167:0/805969782 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99800d9f0 con 0x561dc79a7010 2024-05-11T12:13:48.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.263+0000 7fd9b9174740 1 -- 172.21.15.167:0/805969782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc79a7010 msgr2=0x561dc79a7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:48.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.263+0000 7fd9b9174740 1 --2- 172.21.15.167:0/805969782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc79a7010 0x561dc79a7560 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fd998001880 tx=0x7fd998001d40 comp rx=0 tx=0).stop 2024-05-11T12:13:48.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.263+0000 7fd9b9174740 1 -- 172.21.15.167:0/805969782 shutdown_connections 2024-05-11T12:13:48.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.263+0000 7fd9b9174740 1 --2- 172.21.15.167:0/805969782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561dc795f450 0x561dc79a56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:48.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.263+0000 7fd9b9174740 1 --2- 172.21.15.167:0/805969782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc79a7010 0x561dc79a7560 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:48.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.263+0000 7fd9b9174740 1 --2- 172.21.15.167:0/805969782 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x561dc7989770 0x561dc7989b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:48.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.263+0000 7fd9b9174740 1 -- 172.21.15.167:0/805969782 >> 172.21.15.167:0/805969782 conn(0x561dc795bd80 msgr2=0x561dc79a4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:48.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.263+0000 7fd9b9174740 1 -- 172.21.15.167:0/805969782 shutdown_connections 2024-05-11T12:13:48.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.263+0000 7fd9b9174740 1 -- 172.21.15.167:0/805969782 wait complete. 2024-05-11T12:13:48.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.264+0000 7fd9b9174740 1 -- start start 2024-05-11T12:13:48.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.264+0000 7fd9b9174740 1 Processor -- start 2024-05-11T12:13:48.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.265+0000 7fd9b9174740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x561dc7989770 0x561dc798be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:48.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.265+0000 7fd9b9174740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561dc79a7010 0x561dc79a5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:48.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.265+0000 7fd9b9174740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc7985bc0 0x561dc7986040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:48.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.265+0000 7fd9b9174740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561dc79601f0 con 0x561dc7985bc0 2024-05-11T12:13:48.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.265+0000 7fd9b9174740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x561dc787cb50 con 0x561dc79a7010 2024-05-11T12:13:48.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.265+0000 7fd9b9174740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x561dc7959cb0 con 0x561dc7989770 2024-05-11T12:13:48.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.265+0000 7fd9b9174740 10 client.0 _ll_register_callbacks cb 0x561dc7846420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:13:48.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.266+0000 7fd9afe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc7985bc0 0x561dc7986040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:48.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.266+0000 7fd9afe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc7985bc0 0x561dc7986040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41524/0 (socket says 172.21.15.167:41524) 2024-05-11T12:13:48.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.266+0000 7fd9afe00640 1 -- 172.21.15.167:0/2210929091 learned_addr learned my addr 172.21.15.167:0/2210929091 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:48.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.266+0000 7fd9af400640 1 --2- 172.21.15.167:0/2210929091 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x561dc7989770 0x561dc798be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:48.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.266+0000 7fd9a7e00640 1 --2- 172.21.15.167:0/2210929091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561dc79a7010 0x561dc79a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:48.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.266+0000 7fd9afe00640 1 -- 172.21.15.167:0/2210929091 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x561dc7989770 msgr2=0x561dc798be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:48.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.266+0000 7fd9afe00640 1 --2- 172.21.15.167:0/2210929091 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x561dc7989770 0x561dc798be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:48.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.266+0000 7fd9afe00640 1 -- 172.21.15.167:0/2210929091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561dc79a7010 msgr2=0x561dc79a5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:48.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.266+0000 7fd9afe00640 1 --2- 172.21.15.167:0/2210929091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561dc79a7010 0x561dc79a5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:48.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.266+0000 7fd9afe00640 1 -- 172.21.15.167:0/2210929091 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dc769f7d0 con 0x561dc7985bc0 2024-05-11T12:13:48.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.266+0000 7fd9afe00640 1 --2- 172.21.15.167:0/2210929091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc7985bc0 0x561dc7986040 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8081760 tx=0x7fd9a80079a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f499bbe95fbdf16 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:48.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.266+0000 7fd99cc00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-11T12:13:48.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.267+0000 7fd99cc00640 1 -- 172.21.15.167:0/2210929091 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a8048030 con 0x561dc7985bc0 2024-05-11T12:13:48.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.267+0000 7fd99cc00640 1 -- 172.21.15.167:0/2210929091 <== mon.0 v2:172.21.15.167:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7fd9a808a3a0 con 0x561dc7985bc0 2024-05-11T12:13:48.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.267+0000 7fd9b9174740 1 -- 172.21.15.167:0/2210929091 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd998007680 con 0x561dc7985bc0 2024-05-11T12:13:48.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.267+0000 7fd99cc00640 1 -- 172.21.15.167:0/2210929091 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fd9a80b6050 con 0x561dc7985bc0 2024-05-11T12:13:48.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.267+0000 7fd9b9174740 10 client.6150 fetch_fsmap learned FSMap version 44 2024-05-11T12:13:48.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.267+0000 7fd9b9174740 1 -- 172.21.15.167:0/2210929091 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) -- 0x561dc799db60 con 0x561dc7985bc0 2024-05-11T12:13:48.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.267+0000 7fd99cc00640 1 -- 172.21.15.167:0/2210929091 <== mon.0 v2:172.21.15.167:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd998007680 con 0x561dc7985bc0 2024-05-11T12:13:48.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.267+0000 7fd99cc00640 1 -- 172.21.15.167:0/2210929091 <== mon.0 v2:172.21.15.167:3300/0 5 ==== fsmap(e 44) ==== 3494+0+0 (secure 0 0 0) 0x7fd9a80880c0 con 0x561dc7985bc0 2024-05-11T12:13:48.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.267+0000 7fd9b9174740 10 client.6150 fetch_fsmap finished waiting for FSMap version 44 2024-05-11T12:13:48.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.267+0000 7fd9b9174740 10 client.6150 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-11T12:13:48.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.267+0000 7fd9b9174740 20 client.6150 populate_metadata read hostname 'smithi167' 2024-05-11T12:13:48.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.267+0000 7fd9b9174740 1 --2- 172.21.15.167:0/2210929091 >> [v2:172.21.15.167:6835/1306616214,v1:172.21.15.167:6837/1306616214] conn(0x561dc799bb00 0x561dc799bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:48.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.267+0000 7fd9b9174740 4 client.6150 mds_command: new command op to 5952 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:13:48.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.267+0000 7fd9b9174740 1 -- 172.21.15.167:0/2210929091 --> [v2:172.21.15.167:6835/1306616214,v1:172.21.15.167:6837/1306616214] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561dc78d1960 con 0x561dc799bb00 2024-05-11T12:13:48.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.268+0000 7fd9af400640 1 --2- 172.21.15.167:0/2210929091 >> [v2:172.21.15.167:6835/1306616214,v1:172.21.15.167:6837/1306616214] conn(0x561dc799bb00 0x561dc799bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:48.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.268+0000 7fd9af400640 1 --2- 172.21.15.167:0/2210929091 >> [v2:172.21.15.167:6835/1306616214,v1:172.21.15.167:6837/1306616214] conn(0x561dc799bb00 0x561dc799bee0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=426324e3ec8ba74 server_cookie=cfbcf3a1bd4e3bc in_seq=0 out_seq=0 2024-05-11T12:13:48.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.268+0000 7fd99cc00640 10 client.6150 ms_handle_connect on v2:172.21.15.167:6835/1306616214 2024-05-11T12:13:48.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.269+0000 7fd99cc00640 1 -- 172.21.15.167:0/2210929091 <== mds.0 v2:172.21.15.167:6835/1306616214 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561dc78d1960 con 0x561dc799bb00 2024-05-11T12:13:48.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.269+0000 7fd99cc00640 10 client.6150 handle_command_reply: tid=0 2024-05-11T12:13:48.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.269+0000 7fd9b9174740 1 client.6150 shutdown 2024-05-11T12:13:48.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.269+0000 7fd9b9174740 1 -- 172.21.15.167:0/2210929091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc7985bc0 msgr2=0x561dc7986040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:48.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.269+0000 7fd9b9174740 1 --2- 172.21.15.167:0/2210929091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc7985bc0 0x561dc7986040 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8081760 tx=0x7fd9a80079a0 comp rx=0 tx=0).stop 2024-05-11T12:13:48.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.269+0000 7fd9b9174740 1 -- 172.21.15.167:0/2210929091 shutdown_connections 2024-05-11T12:13:48.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.269+0000 7fd9b9174740 1 --2- 172.21.15.167:0/2210929091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc7985bc0 0x561dc7986040 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:48.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.269+0000 7fd9b9174740 1 --2- 172.21.15.167:0/2210929091 >> [v2:172.21.15.167:6835/1306616214,v1:172.21.15.167:6837/1306616214] conn(0x561dc799bb00 0x561dc799bee0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:48.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.269+0000 7fd9b9174740 1 --2- 172.21.15.167:0/2210929091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561dc79a7010 0x561dc79a5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:48.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.269+0000 7fd9b9174740 1 --2- 172.21.15.167:0/2210929091 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x561dc7989770 0x561dc798be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:48.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.269+0000 7fd9b9174740 1 -- 172.21.15.167:0/2210929091 >> 172.21.15.167:0/2210929091 conn(0x561dc795f450 msgr2=0x561dc79a3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:48.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.269+0000 7fd99cc00640 0 client.6150 ms_handle_reset on v2:172.21.15.167:6835/1306616214 2024-05-11T12:13:48.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.270+0000 7fd9b9174740 1 -- 172.21.15.167:0/2210929091 shutdown_connections 2024-05-11T12:13:48.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.270+0000 7fd9b9174740 1 -- 172.21.15.167:0/2210929091 wait complete. 2024-05-11T12:13:48.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.270+0000 7fd9b9174740 20 client.6150 trim_cache size 0 max 16384 2024-05-11T12:13:48.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.290+0000 7fd9b9174740 1 Processor -- start 2024-05-11T12:13:48.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.290+0000 7fd9b9174740 1 -- start start 2024-05-11T12:13:48.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.291+0000 7fd9b9174740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x561dc7985bc0 0x561dc799c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:48.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.291+0000 7fd9b9174740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561dc7989770 0x561dc79a76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:48.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.291+0000 7fd9b9174740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc79a7010 0x561dc79a7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:48.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.291+0000 7fd9b9174740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b0201da0 con 0x561dc79a7010 2024-05-11T12:13:48.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.291+0000 7fd9b9174740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b0200dd0 con 0x561dc7989770 2024-05-11T12:13:48.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.291+0000 7fd9b9174740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd9b01f5c50 con 0x561dc7985bc0 2024-05-11T12:13:48.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.291+0000 7fd9afe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc79a7010 0x561dc79a7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:48.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.291+0000 7fd9afe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc79a7010 0x561dc79a7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41526/0 (socket says 172.21.15.167:41526) 2024-05-11T12:13:48.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.291+0000 7fd9afe00640 1 -- 172.21.15.167:0/3386479552 learned_addr learned my addr 172.21.15.167:0/3386479552 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:48.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.291+0000 7fd9a7e00640 1 --2- 172.21.15.167:0/3386479552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561dc7989770 0x561dc79a76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:48.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.291+0000 7fd9af400640 1 --2- 172.21.15.167:0/3386479552 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x561dc7985bc0 0x561dc799c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:48.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.292+0000 7fd9afe00640 1 -- 172.21.15.167:0/3386479552 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x561dc7985bc0 msgr2=0x561dc799c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:48.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.292+0000 7fd9afe00640 1 --2- 172.21.15.167:0/3386479552 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x561dc7985bc0 0x561dc799c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:48.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.292+0000 7fd9afe00640 1 -- 172.21.15.167:0/3386479552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561dc7989770 msgr2=0x561dc79a76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:48.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.292+0000 7fd9afe00640 1 --2- 172.21.15.167:0/3386479552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561dc7989770 0x561dc79a76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:48.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.292+0000 7fd9afe00640 1 -- 172.21.15.167:0/3386479552 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dc799c250 con 0x561dc79a7010 2024-05-11T12:13:48.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.292+0000 7fd9a7e00640 1 --2- 172.21.15.167:0/3386479552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561dc7989770 0x561dc79a76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:48.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.292+0000 7fd9af400640 1 --2- 172.21.15.167:0/3386479552 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x561dc7985bc0 0x561dc799c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:48.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.292+0000 7fd9afe00640 1 --2- 172.21.15.167:0/3386479552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc79a7010 0x561dc79a7c30 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fd9a80819f0 tx=0x7fd9a8088050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6352360047674a1f server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:48.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.292+0000 7fd99f400640 1 -- 172.21.15.167:0/3386479552 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a8048050 con 0x561dc79a7010 2024-05-11T12:13:48.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.292+0000 7fd99f400640 1 -- 172.21.15.167:0/3386479552 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9a80a3170 con 0x561dc79a7010 2024-05-11T12:13:48.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.292+0000 7fd99f400640 1 -- 172.21.15.167:0/3386479552 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a80813b0 con 0x561dc79a7010 2024-05-11T12:13:48.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.293+0000 7fd9b9174740 1 -- 172.21.15.167:0/3386479552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc79a7010 msgr2=0x561dc79a7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:48.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.293+0000 7fd9b9174740 1 --2- 172.21.15.167:0/3386479552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc79a7010 0x561dc79a7c30 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fd9a80819f0 tx=0x7fd9a8088050 comp rx=0 tx=0).stop 2024-05-11T12:13:48.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.293+0000 7fd9b9174740 1 -- 172.21.15.167:0/3386479552 shutdown_connections 2024-05-11T12:13:48.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.293+0000 7fd9b9174740 1 --2- 172.21.15.167:0/3386479552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc79a7010 0x561dc79a7c30 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:48.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.293+0000 7fd9b9174740 1 --2- 172.21.15.167:0/3386479552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561dc7989770 0x561dc79a76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:48.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.293+0000 7fd9b9174740 1 --2- 172.21.15.167:0/3386479552 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x561dc7985bc0 0x561dc799c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:48.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.293+0000 7fd9b9174740 1 -- 172.21.15.167:0/3386479552 >> 172.21.15.167:0/3386479552 conn(0x561dc795f450 msgr2=0x561dc7985420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:48.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.293+0000 7fd9b9174740 1 -- 172.21.15.167:0/3386479552 shutdown_connections 2024-05-11T12:13:48.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.293+0000 7fd9b9174740 1 -- 172.21.15.167:0/3386479552 wait complete. 2024-05-11T12:13:48.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.294+0000 7fd9b9174740 1 -- start start 2024-05-11T12:13:48.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.294+0000 7fd9b9174740 1 Processor -- start 2024-05-11T12:13:48.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.295+0000 7fd9b9174740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc7985bc0 0x561dc79a5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:48.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.295+0000 7fd9b9174740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x561dc7989770 0x561dc79a5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:48.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.295+0000 7fd9b9174740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561dc79a7010 0x561dc79a74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:48.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.295+0000 7fd9b9174740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561dc79a8a40 con 0x561dc7985bc0 2024-05-11T12:13:48.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.295+0000 7fd9b9174740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b01f62a0 con 0x561dc79a7010 2024-05-11T12:13:48.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.295+0000 7fd9b9174740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x561dc7986160 con 0x561dc7989770 2024-05-11T12:13:48.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.295+0000 7fd9b9174740 10 client.0 _ll_register_callbacks cb 0x7fd9a807e0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:13:48.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.295+0000 7fd9af400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc7985bc0 0x561dc79a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:48.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.295+0000 7fd9af400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc7985bc0 0x561dc79a5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41542/0 (socket says 172.21.15.167:41542) 2024-05-11T12:13:48.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.295+0000 7fd9af400640 1 -- 172.21.15.167:0/3713076148 learned_addr learned my addr 172.21.15.167:0/3713076148 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:48.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.295+0000 7fd9afe00640 1 --2- 172.21.15.167:0/3713076148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561dc79a7010 0x561dc79a74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:48.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.295+0000 7fd9a7e00640 1 --2- 172.21.15.167:0/3713076148 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x561dc7989770 0x561dc79a5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:48.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.295+0000 7fd9af400640 1 -- 172.21.15.167:0/3713076148 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x561dc7989770 msgr2=0x561dc79a5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:48.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.295+0000 7fd9af400640 1 --2- 172.21.15.167:0/3713076148 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x561dc7989770 0x561dc79a5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:48.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.296+0000 7fd9af400640 1 -- 172.21.15.167:0/3713076148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561dc79a7010 msgr2=0x561dc79a74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:48.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.296+0000 7fd9af400640 1 --2- 172.21.15.167:0/3713076148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561dc79a7010 0x561dc79a74c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:48.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.296+0000 7fd9af400640 1 -- 172.21.15.167:0/3713076148 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dc795d670 con 0x561dc7985bc0 2024-05-11T12:13:48.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.296+0000 7fd9af400640 1 --2- 172.21.15.167:0/3713076148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc7985bc0 0x561dc79a5710 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0009e50 tx=0x7fd9a0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c96dba30600202a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:48.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.296+0000 7fd99cc00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-11T12:13:48.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.296+0000 7fd99cc00640 1 -- 172.21.15.167:0/3713076148 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a000f040 con 0x561dc7985bc0 2024-05-11T12:13:48.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.296+0000 7fd99cc00640 1 -- 172.21.15.167:0/3713076148 <== mon.0 v2:172.21.15.167:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7fd9a000ece0 con 0x561dc7985bc0 2024-05-11T12:13:48.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.296+0000 7fd9b9174740 1 -- 172.21.15.167:0/3713076148 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9a80a3170 con 0x561dc7985bc0 2024-05-11T12:13:48.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.296+0000 7fd99cc00640 1 -- 172.21.15.167:0/3713076148 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fd9a0036050 con 0x561dc7985bc0 2024-05-11T12:13:48.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.296+0000 7fd9b9174740 10 client.6156 fetch_fsmap learned FSMap version 44 2024-05-11T12:13:48.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.296+0000 7fd9b9174740 1 -- 172.21.15.167:0/3713076148 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) -- 0x561dc7a2be40 con 0x561dc7985bc0 2024-05-11T12:13:48.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.297+0000 7fd99cc00640 1 -- 172.21.15.167:0/3713076148 <== mon.0 v2:172.21.15.167:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9a80a3170 con 0x561dc7985bc0 2024-05-11T12:13:48.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.297+0000 7fd99cc00640 1 -- 172.21.15.167:0/3713076148 <== mon.0 v2:172.21.15.167:3300/0 5 ==== fsmap(e 44) ==== 3494+0+0 (secure 0 0 0) 0x7fd9a0039020 con 0x561dc7985bc0 2024-05-11T12:13:48.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.297+0000 7fd9b9174740 10 client.6156 fetch_fsmap finished waiting for FSMap version 44 2024-05-11T12:13:48.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.297+0000 7fd9b9174740 10 client.6156 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-11T12:13:48.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.297+0000 7fd9b9174740 20 client.6156 populate_metadata read hostname 'smithi167' 2024-05-11T12:13:48.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.297+0000 7fd9b9174740 1 --2- 172.21.15.167:0/3713076148 >> [v2:172.21.15.167:6835/1306616214,v1:172.21.15.167:6837/1306616214] conn(0x561dc7988900 0x561dc79a7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:48.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.297+0000 7fd9b9174740 4 client.6156 mds_command: new command op to 5952 tid=0 multi_id=0 [{"prefix": "session ls"}] 2024-05-11T12:13:48.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.297+0000 7fd9b9174740 1 -- 172.21.15.167:0/3713076148 --> [v2:172.21.15.167:6835/1306616214,v1:172.21.15.167:6837/1306616214] -- command(tid 0: {"prefix": "session ls"}) -- 0x561dc78cb7d0 con 0x561dc7988900 2024-05-11T12:13:48.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.297+0000 7fd9a7e00640 1 --2- 172.21.15.167:0/3713076148 >> [v2:172.21.15.167:6835/1306616214,v1:172.21.15.167:6837/1306616214] conn(0x561dc7988900 0x561dc79a7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:48.302 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.298+0000 7fd9a7e00640 1 --2- 172.21.15.167:0/3713076148 >> [v2:172.21.15.167:6835/1306616214,v1:172.21.15.167:6837/1306616214] conn(0x561dc7988900 0x561dc79a7a00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=277f2dedff44bc9c server_cookie=cd4ef386bf739b76 in_seq=0 out_seq=0 2024-05-11T12:13:48.302 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.298+0000 7fd99cc00640 10 client.6156 ms_handle_connect on v2:172.21.15.167:6835/1306616214 2024-05-11T12:13:48.302 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.298+0000 7fd99cc00640 1 -- 172.21.15.167:0/3713076148 <== mds.0 v2:172.21.15.167:6835/1306616214 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561dc78cb7d0 con 0x561dc7988900 2024-05-11T12:13:48.302 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.298+0000 7fd99cc00640 10 client.6156 handle_command_reply: tid=0 2024-05-11T12:13:48.302 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.298+0000 7fd9b9174740 1 client.6156 shutdown 2024-05-11T12:13:48.302 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.298+0000 7fd9b9174740 1 -- 172.21.15.167:0/3713076148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc7985bc0 msgr2=0x561dc79a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:48.302 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.298+0000 7fd9b9174740 1 --2- 172.21.15.167:0/3713076148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc7985bc0 0x561dc79a5710 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0009e50 tx=0x7fd9a0004730 comp rx=0 tx=0).stop 2024-05-11T12:13:48.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.298+0000 7fd9b9174740 1 -- 172.21.15.167:0/3713076148 shutdown_connections 2024-05-11T12:13:48.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.298+0000 7fd9b9174740 1 --2- 172.21.15.167:0/3713076148 >> [v2:172.21.15.167:6835/1306616214,v1:172.21.15.167:6837/1306616214] conn(0x561dc7988900 0x561dc79a7a00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:48.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.298+0000 7fd9b9174740 1 --2- 172.21.15.167:0/3713076148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561dc79a7010 0x561dc79a74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:48.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.298+0000 7fd9b9174740 1 --2- 172.21.15.167:0/3713076148 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x561dc7989770 0x561dc79a5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:48.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.298+0000 7fd9b9174740 1 --2- 172.21.15.167:0/3713076148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc7985bc0 0x561dc79a5710 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:48.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.298+0000 7fd99cc00640 0 client.6156 ms_handle_reset on v2:172.21.15.167:6835/1306616214 2024-05-11T12:13:48.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.298+0000 7fd9b9174740 1 -- 172.21.15.167:0/3713076148 >> 172.21.15.167:0/3713076148 conn(0x561dc795f450 msgr2=0x561dc795e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:48.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.299+0000 7fd9b9174740 1 -- 172.21.15.167:0/3713076148 shutdown_connections 2024-05-11T12:13:48.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.299+0000 7fd9b9174740 1 -- 172.21.15.167:0/3713076148 wait complete. 2024-05-11T12:13:48.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.299+0000 7fd9b9174740 20 client.6156 trim_cache size 0 max 16384 2024-05-11T12:13:48.303 INFO:teuthology.orchestra.run.smithi167.stdout:[] 2024-05-11T12:13:48.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.299+0000 7fd9b6400640 1 -- 172.21.15.167:0/3477220263 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd98c0762d0 msgr2=0x7fd98c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:48.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.299+0000 7fd9b6400640 1 --2- 172.21.15.167:0/3477220263 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd98c0762d0 0x7fd98c078790 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0006fd0 tx=0x7fd9a0008040 comp rx=0 tx=0).stop 2024-05-11T12:13:48.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.299+0000 7fd9b6400640 1 -- 172.21.15.167:0/3477220263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9b0161380 msgr2=0x7fd9b01e7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:48.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.299+0000 7fd9b6400640 1 --2- 172.21.15.167:0/3477220263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9b0161380 0x7fd9b01e7730 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd9a802a6a0 tx=0x7fd9a8004a40 comp rx=0 tx=0).stop 2024-05-11T12:13:48.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.299+0000 7fd9b6400640 1 -- 172.21.15.167:0/3477220263 shutdown_connections 2024-05-11T12:13:48.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.299+0000 7fd9b6400640 1 --2- 172.21.15.167:0/3477220263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9b0161380 0x7fd9b01e7730 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:48.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.299+0000 7fd9b6400640 1 --2- 172.21.15.167:0/3477220263 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd98c0762d0 0x7fd98c078790 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:48.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.299+0000 7fd9b6400640 1 --2- 172.21.15.167:0/3477220263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9b015ac80 0x7fd9b00ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:48.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.299+0000 7fd9b6400640 1 --2- 172.21.15.167:0/3477220263 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9b00b3d50 0x7fd9b0139620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:48.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.299+0000 7fd9b6400640 1 -- 172.21.15.167:0/3477220263 >> 172.21.15.167:0/3477220263 conn(0x7fd9b006c450 msgr2=0x7fd9b00aeee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:48.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.300+0000 7fd9b6400640 1 -- 172.21.15.167:0/3477220263 shutdown_connections 2024-05-11T12:13:48.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:48.300+0000 7fd9b6400640 1 -- 172.21.15.167:0/3477220263 wait complete. 2024-05-11T12:13:48.319 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json perf dump 2024-05-11T12:13:48.542 INFO:teuthology.orchestra.run.smithi167.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":23,"msgr_send_messages":25,"msgr_recv_bytes":6215,"msgr_send_bytes":13671,"msgr_created_connections":7,"msgr_active_connections":2,"msgr_running_total_time":0.023210335,"msgr_running_send_time":0.001640621,"msgr_running_recv_time":0.015709157,"msgr_running_fast_dispatch_time":0.001736825,"msgr_send_messages_queue_lat":{"avgcount":24,"sum":0.296538751,"avgtime":0.012355781},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"AsyncMessenger::Worker-1":{"msgr_recv_messages":90,"msgr_send_messages":82,"msgr_recv_bytes":131471,"msgr_send_bytes":74358,"msgr_created_connections":9,"msgr_active_connections":3,"msgr_running_total_time":0.028586839,"msgr_running_send_time":0.009399761,"msgr_running_recv_time":0.041538099,"msgr_running_fast_dispatch_time":0.000628931,"msgr_send_messages_queue_lat":{"avgcount":81,"sum":0.108849861,"avgtime":0.001343825},"msgr_handle_ack_lat":{"avgcount":60,"sum":0.000021657,"avgtime":0.000000360},"msgr_recv_encrypted_bytes":120432,"msgr_send_encrypted_bytes":10288},"AsyncMessenger::Worker-2":{"msgr_recv_messages":26,"msgr_send_messages":28,"msgr_recv_bytes":7293,"msgr_send_bytes":110203,"msgr_created_connections":6,"msgr_active_connections":3,"msgr_running_total_time":0.014011146,"msgr_running_send_time":0.002113709,"msgr_running_recv_time":0.011117022,"msgr_running_fast_dispatch_time":0.000521552,"msgr_send_messages_queue_lat":{"avgcount":27,"sum":0.202264497,"avgtime":0.007491277},"msgr_handle_ack_lat":{"avgcount":12,"sum":0.000003600,"avgtime":0.000000300},"msgr_recv_encrypted_bytes":1728,"msgr_send_encrypted_bytes":96096},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":38,"sum":0.004935829,"avgtime":0.000129890}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":2,"sum":0.000028579,"avgtime":0.000014289}},"mds":{"request":29,"reply":29,"reply_latency":{"avgcount":29,"sum":0.012307906,"avgtime":0.000424410},"slow_reply":0,"forward":0,"dir_fetch_complete":0,"dir_fetch_keys":0,"dir_commit":12,"dir_split":0,"dir_merge":0,"inodes":11,"inodes_top":0,"inodes_bottom":11,"inodes_pin_tail":0,"inodes_pinned":10,"inodes_expired":0,"inodes_with_caps":1,"caps":2,"subtrees":2,"traverse":55,"traverse_hit":53,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":0,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":0,"handle_client_caps_dirty":0,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_quiesce_path":0,"ireq_quiesce_inode":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evlrg":0,"evadd":8,"evex":0,"evtrm":0,"ev":8,"evexg":0,"evexd":0,"segadd":2,"segex":0,"segtrm":0,"seg":2,"segmjr":2,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4197029,"rdpos":0,"jlat":{"avgcount":5,"sum":0.019663414,"avgtime":0.003932682},"replayed":0},"mds_mem":{"ino":13,"ino+":13,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":11,"dn+":11,"dn-":0,"cap":0,"cap+":2,"cap-":2,"rss":40704,"heap":231788},"mds_server":{"dispatch_client_request":29,"dispatch_server_request":0,"handle_client_request":29,"handle_client_session":6,"handle_peer_request":0,"req_create_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_getattr_latency":{"avgcount":26,"sum":0.010614534,"avgtime":0.000408251},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":2,"sum":0.000445241,"avgtime":0.000222620},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":1,"sum":0.001248131,"avgtime":0.001248131},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":0,"session_add":2,"session_remove":2,"sessions_open":0,"sessions_stale":0,"total_load":28,"average_load":14,"avg_session_uptime":1,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":217253,"buffer_anon_items":85,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5424,"osdmap_items":76,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":53784,"mds_co_items":218,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":43,"op_send_bytes":13845,"op_resend":28,"op_reply":43,"op_latency":{"avgcount":43,"sum":1.388976541,"avgtime":0.032301780},"op_inflight":0,"oplen_avg":{"avgcount":43,"sum":67},"op":43,"op_r":0,"op_w":43,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":11,"osdop_read":0,"osdop_write":5,"osdop_writefull":7,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":22,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":75,"map_full":0,"map_inc":1,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":15,"omap_rd":0,"omap_del":1},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":0,"omap_total_updates":0,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":135,"get_sum":128937,"get_or_fail_fail":0,"get_or_fail_success":135,"take":0,"take_sum":0,"put":135,"put_sum":128937,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":43,"take_sum":13845,"put":43,"put_sum":13845,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":43,"take_sum":43,"put":43,"put_sum":43,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x56187b98c500":{"val":0,"max":3758096384,"get_started":0,"get":8,"get_sum":2725,"get_or_fail_fail":0,"get_or_fail_success":8,"take":0,"take_sum":0,"put":5,"put_sum":2725,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-05-11T12:13:48.543 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 2, "msgr_created_connections": 7, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 6215, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 23, "msgr_running_fast_dispatch_time": 0.001736825, "msgr_running_recv_time": 0.015709157, "msgr_running_send_time": 0.001640621, "msgr_running_total_time": 0.023210335, "msgr_send_bytes": 13671, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 25, "msgr_send_messages_queue_lat": { "avgcount": 24, "avgtime": 0.012355781, "sum": 0.296538751 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 3, "msgr_created_connections": 9, "msgr_handle_ack_lat": { "avgcount": 60, "avgtime": 3.6e-07, "sum": 2.1657e-05 }, "msgr_recv_bytes": 131471, "msgr_recv_encrypted_bytes": 120432, "msgr_recv_messages": 90, "msgr_running_fast_dispatch_time": 0.000628931, "msgr_running_recv_time": 0.041538099, "msgr_running_send_time": 0.009399761, "msgr_running_total_time": 0.028586839, "msgr_send_bytes": 74358, "msgr_send_encrypted_bytes": 10288, "msgr_send_messages": 82, "msgr_send_messages_queue_lat": { "avgcount": 81, "avgtime": 0.001343825, "sum": 0.108849861 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 3, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 12, "avgtime": 3e-07, "sum": 3.6e-06 }, "msgr_recv_bytes": 7293, "msgr_recv_encrypted_bytes": 1728, "msgr_recv_messages": 26, "msgr_running_fast_dispatch_time": 0.000521552, "msgr_running_recv_time": 0.011117022, "msgr_running_send_time": 0.002113709, "msgr_running_total_time": 0.014011146, "msgr_send_bytes": 110203, "msgr_send_encrypted_bytes": 96096, "msgr_send_messages": 28, "msgr_send_messages_queue_lat": { "avgcount": 27, "avgtime": 0.007491277, "sum": 0.202264497 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 38, "avgtime": 0.00012989, "sum": 0.004935829 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 2, "avgtime": 1.4289e-05, "sum": 2.8579e-05 }, "queue_len": 0 }, "mds": { "caps": 2, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 12, "dir_fetch_complete": 0, "dir_fetch_keys": 0, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 0, "handle_client_caps_dirty": 0, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 11, "inodes_bottom": 11, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 10, "inodes_top": 0, "inodes_with_caps": 1, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 29, "reply_latency": { "avgcount": 29, "avgtime": 0.00042441, "sum": 0.012307906 }, "request": 29, "root_rbytes": 0, "root_rfiles": 0, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 55, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 53, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "ireq_quiesce_inode": 0, "ireq_quiesce_path": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 8, "evadd": 8, "evex": 0, "evexd": 0, "evexg": 0, "evlrg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 5, "avgtime": 0.003932682, "sum": 0.019663414 }, "rdpos": 0, "replayed": 0, "seg": 2, "segadd": 2, "segex": 0, "segexd": 0, "segexg": 0, "segmjr": 2, "segtrm": 0, "wrpos": 4197029 }, "mds_mem": { "cap": 0, "cap+": 2, "cap-": 2, "dir": 12, "dir+": 12, "dir-": 0, "dn": 11, "dn+": 11, "dn-": 0, "heap": 231788, "ino": 13, "ino+": 13, "ino-": 0, "rss": 40704 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 29, "dispatch_server_request": 0, "handle_client_request": 29, "handle_client_session": 6, "handle_peer_request": 0, "req_create_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getattr_latency": { "avgcount": 26, "avgtime": 0.000408251, "sum": 0.010614534 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 2, "avgtime": 0.00022262, "sum": 0.000445241 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 1, "avgtime": 0.001248131, "sum": 0.001248131 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 14, "avg_session_uptime": 1, "mdthresh_evicted": 0, "session_add": 2, "session_count": 0, "session_remove": 2, "sessions_open": 0, "sessions_stale": 0, "total_load": 28 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 217253, "buffer_anon_items": 85, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 53784, "mds_co_items": 218, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5424, "osdmap_items": 76, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 75, "map_full": 0, "map_inc": 1, "omap_del": 1, "omap_rd": 0, "omap_wr": 15, "op": 43, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 43, "avgtime": 0.03230178, "sum": 1.388976541 }, "op_pg": 0, "op_r": 0, "op_reply": 43, "op_resend": 28, "op_rmw": 0, "op_send": 43, "op_send_bytes": 13845, "op_w": 43, "oplen_avg": { "avgcount": 43, "sum": 67 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 11, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 22, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 5, "osdop_writefull": 7, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 0, "omap_total_objs": 1, "omap_total_removes": 0, "omap_total_updates": 0 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 135, "get_or_fail_fail": 0, "get_or_fail_success": 135, "get_started": 0, "get_sum": 128937, "max": 104857600, "put": 135, "put_sum": 128937, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 43, "put_sum": 13845, "take": 43, "take_sum": 13845, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 43, "put_sum": 43, "take": 43, "take_sum": 43, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x56187b98c500": { "get": 8, "get_or_fail_fail": 0, "get_or_fail_success": 8, "get_started": 0, "get_sum": 2725, "max": 3758096384, "put": 5, "put_sum": 2725, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-05-11T12:13:48.543 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-11T12:13:48.543 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-11T12:13:48.544 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-11T12:13:48.664 DEBUG:teuthology.orchestra.run.smithi204:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-11T12:13:48.773 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-11T12:13:48.773 INFO:tasks.cephfs.mount:Terminating background process 2024-05-11T12:13:48.774 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:13:48.774 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:13:48.831 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-11T12:13:48.832 INFO:tasks.cephfs.mount:Terminating background process 2024-05-11T12:13:48.832 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:13:48.832 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:13:48.891 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052713,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-11T12:13:48.891 INFO:tasks.cephfs.mount:Terminating background process 2024-05-11T12:13:48.891 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:13:48.891 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:13:48.949 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-11T12:13:48.950 INFO:tasks.cephfs.mount:Terminating background process 2024-05-11T12:13:48.950 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:13:48.950 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:13:49.006 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052713,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-11T12:13:49.006 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:13:49.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.214+0000 7f3e4ac00640 1 Processor -- start 2024-05-11T12:13:49.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.214+0000 7f3e4ac00640 1 -- start start 2024-05-11T12:13:49.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.216+0000 7f3e4ac00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3e4413f290 0x7f3e440735c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:49.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.216+0000 7f3e4ac00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3e440b50a0 0x7f3e44073b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:49.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.216+0000 7f3e4ac00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e4415fb90 con 0x7f3e4413eeb0 2024-05-11T12:13:49.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.216+0000 7f3e4ac00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f3e4415d5b0 con 0x7f3e440b50a0 2024-05-11T12:13:49.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.216+0000 7f3e4ac00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f3e4415fe70 con 0x7f3e4413f290 2024-05-11T12:13:49.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.216+0000 7f3e48c00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3e440b50a0 0x7f3e44073b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:49.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.216+0000 7f3e48c00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3e440b50a0 0x7f3e44073b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54134/0 (socket says 172.21.15.167:54134) 2024-05-11T12:13:49.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.216+0000 7f3e48c00640 1 -- 172.21.15.167:0/3262601357 learned_addr learned my addr 172.21.15.167:0/3262601357 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:49.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.216+0000 7f3e43400640 1 --2- 172.21.15.167:0/3262601357 >> v2:172.21.15.204:3300/0 conn(0x7f3e4413f290 0x7f3e440735c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:49.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.217+0000 7f3e48c00640 1 -- 172.21.15.167:0/3262601357 >> v2:172.21.15.204:3300/0 conn(0x7f3e4413f290 msgr2=0x7f3e440735c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:49.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.217+0000 7f3e48c00640 1 --2- 172.21.15.167:0/3262601357 >> v2:172.21.15.204:3300/0 conn(0x7f3e4413f290 0x7f3e440735c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:49.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.217+0000 7f3e48c00640 1 -- 172.21.15.167:0/3262601357 >> v1:172.21.15.204:6789/0 conn(0x7f3e4413eeb0 legacy=0x7f3e4406f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:49.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.217+0000 7f3e48c00640 1 -- 172.21.15.167:0/3262601357 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e44160150 con 0x7f3e440b50a0 2024-05-11T12:13:49.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.218+0000 7f3e48c00640 1 --2- 172.21.15.167:0/3262601357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e440b50a0 0x7f3e44073b00 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f3e44074520 tx=0x7f3e3c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b814fe1574193cba server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:49.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.218+0000 7f3e42a00640 1 -- 172.21.15.167:0/3262601357 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c036070 con 0x7f3e440b50a0 2024-05-11T12:13:49.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.218+0000 7f3e42a00640 1 -- 172.21.15.167:0/3262601357 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e3c031600 con 0x7f3e440b50a0 2024-05-11T12:13:49.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.218+0000 7f3e42a00640 1 -- 172.21.15.167:0/3262601357 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c0319d0 con 0x7f3e440b50a0 2024-05-11T12:13:49.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.219+0000 7f3e4ac00640 1 -- 172.21.15.167:0/3262601357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e440b50a0 msgr2=0x7f3e44073b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:49.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.219+0000 7f3e4ac00640 1 --2- 172.21.15.167:0/3262601357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e440b50a0 0x7f3e44073b00 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f3e44074520 tx=0x7f3e3c02a610 comp rx=0 tx=0).stop 2024-05-11T12:13:49.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.219+0000 7f3e4ac00640 1 -- 172.21.15.167:0/3262601357 shutdown_connections 2024-05-11T12:13:49.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.219+0000 7f3e4ac00640 1 --2- 172.21.15.167:0/3262601357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e440b50a0 0x7f3e44073b00 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:49.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.219+0000 7f3e4ac00640 1 --2- 172.21.15.167:0/3262601357 >> v2:172.21.15.204:3300/0 conn(0x7f3e4413f290 0x7f3e440735c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:49.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.219+0000 7f3e4ac00640 1 -- 172.21.15.167:0/3262601357 >> 172.21.15.167:0/3262601357 conn(0x7f3e440cd8f0 msgr2=0x7f3e440cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:49.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.219+0000 7f3e4ac00640 1 -- 172.21.15.167:0/3262601357 shutdown_connections 2024-05-11T12:13:49.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.219+0000 7f3e4ac00640 1 -- 172.21.15.167:0/3262601357 wait complete. 2024-05-11T12:13:49.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.220+0000 7f3e4ac00640 1 Processor -- start 2024-05-11T12:13:49.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.221+0000 7f3e4ac00640 1 -- start start 2024-05-11T12:13:49.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.221+0000 7f3e4ac00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e440b50a0 0x7f3e441f4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:49.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.222+0000 7f3e4ac00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e4413eeb0 0x7f3e4413d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:49.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.222+0000 7f3e4ac00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e4413f290 0x7f3e4413da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:49.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.222+0000 7f3e4ac00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3e44074040 con 0x7f3e4413eeb0 2024-05-11T12:13:49.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.222+0000 7f3e4ac00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3e44160710 con 0x7f3e440b50a0 2024-05-11T12:13:49.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.222+0000 7f3e4ac00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3e4406fb90 con 0x7f3e4413f290 2024-05-11T12:13:49.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.222+0000 7f3e43400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e4413eeb0 0x7f3e4413d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:49.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.222+0000 7f3e43400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e4413eeb0 0x7f3e4413d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54144/0 (socket says 172.21.15.167:54144) 2024-05-11T12:13:49.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.222+0000 7f3e43400640 1 -- 172.21.15.167:0/511700446 learned_addr learned my addr 172.21.15.167:0/511700446 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:49.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.222+0000 7f3e43400640 1 -- 172.21.15.167:0/511700446 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e4413f290 msgr2=0x7f3e4413da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:49.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.222+0000 7f3e43e00640 1 --2- 172.21.15.167:0/511700446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e440b50a0 0x7f3e441f4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:49.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.222+0000 7f3e43400640 1 --2- 172.21.15.167:0/511700446 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e4413f290 0x7f3e4413da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:49.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.222+0000 7f3e43400640 1 -- 172.21.15.167:0/511700446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e440b50a0 msgr2=0x7f3e441f4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:49.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.222+0000 7f3e43400640 1 --2- 172.21.15.167:0/511700446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e440b50a0 0x7f3e441f4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:49.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.222+0000 7f3e43400640 1 -- 172.21.15.167:0/511700446 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e440b0ba0 con 0x7f3e4413eeb0 2024-05-11T12:13:49.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.223+0000 7f3e43400640 1 --2- 172.21.15.167:0/511700446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e4413eeb0 0x7f3e4413d4f0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f3e3000d5f0 tx=0x7f3e3000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:49.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.223+0000 7f3e3be00640 1 -- 172.21.15.167:0/511700446 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e300125f0 con 0x7f3e4413eeb0 2024-05-11T12:13:49.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.223+0000 7f3e3be00640 1 -- 172.21.15.167:0/511700446 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e30019070 con 0x7f3e4413eeb0 2024-05-11T12:13:49.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.223+0000 7f3e3be00640 1 -- 172.21.15.167:0/511700446 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e300129d0 con 0x7f3e4413eeb0 2024-05-11T12:13:49.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.223+0000 7f3e4ac00640 1 -- 172.21.15.167:0/511700446 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e440b0e80 con 0x7f3e4413eeb0 2024-05-11T12:13:49.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.223+0000 7f3e4ac00640 1 -- 172.21.15.167:0/511700446 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e4415f370 con 0x7f3e4413eeb0 2024-05-11T12:13:49.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.224+0000 7f3e4ac00640 1 -- 172.21.15.167:0/511700446 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e44069590 con 0x7f3e4413eeb0 2024-05-11T12:13:49.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.225+0000 7f3e3be00640 1 -- 172.21.15.167:0/511700446 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3e30012b70 con 0x7f3e4413eeb0 2024-05-11T12:13:49.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.226+0000 7f3e3be00640 1 --2- 172.21.15.167:0/511700446 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3e240762d0 0x7f3e24078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:49.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.226+0000 7f3e3be00640 1 -- 172.21.15.167:0/511700446 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7f3e30097bf0 con 0x7f3e4413eeb0 2024-05-11T12:13:49.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.227+0000 7f3e43e00640 1 --2- 172.21.15.167:0/511700446 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3e240762d0 0x7f3e24078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:49.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.227+0000 7f3e43e00640 1 --2- 172.21.15.167:0/511700446 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3e240762d0 0x7f3e24078790 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3e340015d0 tx=0x7f3e34009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:49.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.232+0000 7f3e3be00640 1 -- 172.21.15.167:0/511700446 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3e30061320 con 0x7f3e4413eeb0 2024-05-11T12:13:49.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.474+0000 7f3e4ac00640 1 -- 172.21.15.167:0/511700446 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3e440cd3d0 con 0x7f3e4413eeb0 2024-05-11T12:13:49.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.475+0000 7f3e3be00640 1 -- 172.21.15.167:0/511700446 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5195 (secure 0 0 0) 0x7f3e30060c20 con 0x7f3e4413eeb0 2024-05-11T12:13:49.476 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:13:49.476 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5020,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3075930607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3075930607},{"type":"v1","addr":"172.21.15.167:6836","nonce":3075930607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":5032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3056051531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3056051531},{"type":"v1","addr":"172.21.15.204:6834","nonce":3056051531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5035,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2900478909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":2900478909},{"type":"v1","addr":"172.21.15.204:6835","nonce":2900478909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:13:34.993949+0000","modified":"2024-05-11T12:13:38.072880+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5952},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5952":{"gid":5952,"name":"a","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/1306616214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1306616214},{"type":"v1","addr":"172.21.15.167:6837","nonce":1306616214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5952,"qdb_cluster":[5952]},"id":5}]} 2024-05-11T12:13:49.476 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 44 2024-05-11T12:13:49.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.479+0000 7f3e4ac00640 1 -- 172.21.15.167:0/511700446 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3e240762d0 msgr2=0x7f3e24078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:49.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.479+0000 7f3e4ac00640 1 --2- 172.21.15.167:0/511700446 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3e240762d0 0x7f3e24078790 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3e340015d0 tx=0x7f3e34009040 comp rx=0 tx=0).stop 2024-05-11T12:13:49.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.479+0000 7f3e4ac00640 1 -- 172.21.15.167:0/511700446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e4413eeb0 msgr2=0x7f3e4413d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:49.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.479+0000 7f3e4ac00640 1 --2- 172.21.15.167:0/511700446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e4413eeb0 0x7f3e4413d4f0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f3e3000d5f0 tx=0x7f3e3000dab0 comp rx=0 tx=0).stop 2024-05-11T12:13:49.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.479+0000 7f3e4ac00640 1 -- 172.21.15.167:0/511700446 shutdown_connections 2024-05-11T12:13:49.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.479+0000 7f3e4ac00640 1 --2- 172.21.15.167:0/511700446 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e4413f290 0x7f3e4413da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:49.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.479+0000 7f3e4ac00640 1 --2- 172.21.15.167:0/511700446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e4413eeb0 0x7f3e4413d4f0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:49.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.479+0000 7f3e4ac00640 1 --2- 172.21.15.167:0/511700446 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3e240762d0 0x7f3e24078790 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:49.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.479+0000 7f3e4ac00640 1 --2- 172.21.15.167:0/511700446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e440b50a0 0x7f3e441f4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:49.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.479+0000 7f3e4ac00640 1 -- 172.21.15.167:0/511700446 >> 172.21.15.167:0/511700446 conn(0x7f3e440cd8f0 msgr2=0x7f3e440b5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:49.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.479+0000 7f3e4ac00640 1 -- 172.21.15.167:0/511700446 shutdown_connections 2024-05-11T12:13:49.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.479+0000 7f3e4ac00640 1 -- 172.21.15.167:0/511700446 wait complete. 2024-05-11T12:13:49.497 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:13:49.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.686+0000 7fe9bf800640 1 Processor -- start 2024-05-11T12:13:49.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.686+0000 7fe9bf800640 1 -- start start 2024-05-11T12:13:49.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.688+0000 7fe9bf800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe9b8157750 0x7fe9b815d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:49.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.688+0000 7fe9bf800640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe9b806d460 0x7fe9b806d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:49.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.688+0000 7fe9bf800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9b815dc80 con 0x7fe9b80b3ea0 2024-05-11T12:13:49.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.688+0000 7fe9bf800640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fe9b8161000 con 0x7fe9b806d460 2024-05-11T12:13:49.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.688+0000 7fe9bf800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fe9b806ddd0 con 0x7fe9b8157750 2024-05-11T12:13:49.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.688+0000 7fe9bd800640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe9b806d460 0x7fe9b806d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:49.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.688+0000 7fe9bd800640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe9b806d460 0x7fe9b806d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54154/0 (socket says 172.21.15.167:54154) 2024-05-11T12:13:49.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.688+0000 7fe9bd800640 1 -- 172.21.15.167:0/3187444035 learned_addr learned my addr 172.21.15.167:0/3187444035 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:49.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.688+0000 7fe9bce00640 1 --2- 172.21.15.167:0/3187444035 >> v2:172.21.15.204:3300/0 conn(0x7fe9b8157750 0x7fe9b815d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:49.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.689+0000 7fe9bd800640 1 -- 172.21.15.167:0/3187444035 >> v2:172.21.15.204:3300/0 conn(0x7fe9b8157750 msgr2=0x7fe9b815d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:49.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.689+0000 7fe9bd800640 1 --2- 172.21.15.167:0/3187444035 >> v2:172.21.15.204:3300/0 conn(0x7fe9b8157750 0x7fe9b815d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:49.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.689+0000 7fe9bd800640 1 -- 172.21.15.167:0/3187444035 >> v1:172.21.15.204:6789/0 conn(0x7fe9b80b3ea0 legacy=0x7fe9b8161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:49.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.689+0000 7fe9bd800640 1 -- 172.21.15.167:0/3187444035 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b815df30 con 0x7fe9b806d460 2024-05-11T12:13:49.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.690+0000 7fe9bd800640 1 --2- 172.21.15.167:0/3187444035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9b806d460 0x7fe9b806d890 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fe9b80ce840 tx=0x7fe9b002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf494759eab51321 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:49.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.690+0000 7fe9b7400640 1 -- 172.21.15.167:0/3187444035 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b0036070 con 0x7fe9b806d460 2024-05-11T12:13:49.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.690+0000 7fe9b7400640 1 -- 172.21.15.167:0/3187444035 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9b0031600 con 0x7fe9b806d460 2024-05-11T12:13:49.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.690+0000 7fe9b7400640 1 -- 172.21.15.167:0/3187444035 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b00319d0 con 0x7fe9b806d460 2024-05-11T12:13:49.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.691+0000 7fe9bf800640 1 -- 172.21.15.167:0/3187444035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9b806d460 msgr2=0x7fe9b806d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:49.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.691+0000 7fe9bf800640 1 --2- 172.21.15.167:0/3187444035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9b806d460 0x7fe9b806d890 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fe9b80ce840 tx=0x7fe9b002a6a0 comp rx=0 tx=0).stop 2024-05-11T12:13:49.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.691+0000 7fe9bf800640 1 -- 172.21.15.167:0/3187444035 shutdown_connections 2024-05-11T12:13:49.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.691+0000 7fe9bf800640 1 --2- 172.21.15.167:0/3187444035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9b806d460 0x7fe9b806d890 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:49.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.691+0000 7fe9bf800640 1 --2- 172.21.15.167:0/3187444035 >> v2:172.21.15.204:3300/0 conn(0x7fe9b8157750 0x7fe9b815d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:49.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.691+0000 7fe9bf800640 1 -- 172.21.15.167:0/3187444035 >> 172.21.15.167:0/3187444035 conn(0x7fe9b80cd8f0 msgr2=0x7fe9b80cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:49.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.691+0000 7fe9bf800640 1 -- 172.21.15.167:0/3187444035 shutdown_connections 2024-05-11T12:13:49.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.691+0000 7fe9bf800640 1 -- 172.21.15.167:0/3187444035 wait complete. 2024-05-11T12:13:49.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.693+0000 7fe9bf800640 1 Processor -- start 2024-05-11T12:13:49.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.693+0000 7fe9bf800640 1 -- start start 2024-05-11T12:13:49.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.694+0000 7fe9bf800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9b806d460 0x7fe9b8072cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:49.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.694+0000 7fe9bf800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9b80b3ea0 0x7fe9b80b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:49.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.694+0000 7fe9bf800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe9b8157750 0x7fe9b81f86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:49.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.694+0000 7fe9bf800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b8157b30 con 0x7fe9b80b3ea0 2024-05-11T12:13:49.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.694+0000 7fe9bf800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b815e4f0 con 0x7fe9b806d460 2024-05-11T12:13:49.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.694+0000 7fe9bf800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe9b8161ba0 con 0x7fe9b8157750 2024-05-11T12:13:49.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.694+0000 7fe9b7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9b80b3ea0 0x7fe9b80b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:49.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.694+0000 7fe9b7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9b80b3ea0 0x7fe9b80b1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54156/0 (socket says 172.21.15.167:54156) 2024-05-11T12:13:49.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.694+0000 7fe9b7e00640 1 -- 172.21.15.167:0/155495020 learned_addr learned my addr 172.21.15.167:0/155495020 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:49.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.695+0000 7fe9bce00640 1 --2- 172.21.15.167:0/155495020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9b806d460 0x7fe9b8072cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:49.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.695+0000 7fe9bd800640 1 --2- 172.21.15.167:0/155495020 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe9b8157750 0x7fe9b81f86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:49.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.695+0000 7fe9b7e00640 1 -- 172.21.15.167:0/155495020 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe9b8157750 msgr2=0x7fe9b81f86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:49.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.695+0000 7fe9b7e00640 1 --2- 172.21.15.167:0/155495020 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe9b8157750 0x7fe9b81f86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:49.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.695+0000 7fe9b7e00640 1 -- 172.21.15.167:0/155495020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9b806d460 msgr2=0x7fe9b8072cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:49.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.695+0000 7fe9b7e00640 1 --2- 172.21.15.167:0/155495020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9b806d460 0x7fe9b8072cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:49.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.695+0000 7fe9b7e00640 1 -- 172.21.15.167:0/155495020 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b81f9350 con 0x7fe9b80b3ea0 2024-05-11T12:13:49.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.695+0000 7fe9bd800640 1 --2- 172.21.15.167:0/155495020 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe9b8157750 0x7fe9b81f86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:49.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.695+0000 7fe9bce00640 1 --2- 172.21.15.167:0/155495020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9b806d460 0x7fe9b8072cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:49.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.695+0000 7fe9b7e00640 1 --2- 172.21.15.167:0/155495020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9b80b3ea0 0x7fe9b80b1cd0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe9a400c430 tx=0x7fe9a400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:49.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.695+0000 7fe9b4c00640 1 -- 172.21.15.167:0/155495020 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4011790 con 0x7fe9b80b3ea0 2024-05-11T12:13:49.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.695+0000 7fe9b4c00640 1 -- 172.21.15.167:0/155495020 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9a4017070 con 0x7fe9b80b3ea0 2024-05-11T12:13:49.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.695+0000 7fe9b4c00640 1 -- 172.21.15.167:0/155495020 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4011bb0 con 0x7fe9b80b3ea0 2024-05-11T12:13:49.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.695+0000 7fe9bf800640 1 -- 172.21.15.167:0/155495020 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9b80721f0 con 0x7fe9b80b3ea0 2024-05-11T12:13:49.698 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.696+0000 7fe9bf800640 1 -- 172.21.15.167:0/155495020 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9b80727b0 con 0x7fe9b80b3ea0 2024-05-11T12:13:49.698 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.697+0000 7fe9bf800640 1 -- 172.21.15.167:0/155495020 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9b8069590 con 0x7fe9b80b3ea0 2024-05-11T12:13:49.698 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.698+0000 7fe9b4c00640 1 -- 172.21.15.167:0/155495020 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe9a401c460 con 0x7fe9b80b3ea0 2024-05-11T12:13:49.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.699+0000 7fe9b4c00640 1 --2- 172.21.15.167:0/155495020 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe9980762d0 0x7fe998078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:49.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.699+0000 7fe9b4c00640 1 -- 172.21.15.167:0/155495020 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7fe9a40976b0 con 0x7fe9b80b3ea0 2024-05-11T12:13:49.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.699+0000 7fe9bce00640 1 --2- 172.21.15.167:0/155495020 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe9980762d0 0x7fe998078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:49.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.699+0000 7fe9bce00640 1 --2- 172.21.15.167:0/155495020 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe9980762d0 0x7fe998078790 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac006fd0 tx=0x7fe9ac008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:49.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.703+0000 7fe9b4c00640 1 -- 172.21.15.167:0/155495020 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe9a4060de0 con 0x7fe9b80b3ea0 2024-05-11T12:13:49.940 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.940+0000 7fe9bf800640 1 -- 172.21.15.167:0/155495020 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe9b806d920 con 0x7fe9b80b3ea0 2024-05-11T12:13:49.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.941+0000 7fe9b4c00640 1 -- 172.21.15.167:0/155495020 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5195 (secure 0 0 0) 0x7fe9a40606e0 con 0x7fe9b80b3ea0 2024-05-11T12:13:49.941 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:13:49.941 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5020,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3075930607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3075930607},{"type":"v1","addr":"172.21.15.167:6836","nonce":3075930607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":5032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3056051531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3056051531},{"type":"v1","addr":"172.21.15.204:6834","nonce":3056051531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5035,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2900478909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":2900478909},{"type":"v1","addr":"172.21.15.204:6835","nonce":2900478909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:13:34.993949+0000","modified":"2024-05-11T12:13:38.072880+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5952},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5952":{"gid":5952,"name":"a","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/1306616214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1306616214},{"type":"v1","addr":"172.21.15.167:6837","nonce":1306616214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5952,"qdb_cluster":[5952]},"id":5}]} 2024-05-11T12:13:49.942 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 44 2024-05-11T12:13:49.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.943+0000 7fe9bf800640 1 -- 172.21.15.167:0/155495020 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe9980762d0 msgr2=0x7fe998078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:49.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.943+0000 7fe9bf800640 1 --2- 172.21.15.167:0/155495020 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe9980762d0 0x7fe998078790 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac006fd0 tx=0x7fe9ac008040 comp rx=0 tx=0).stop 2024-05-11T12:13:49.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.943+0000 7fe9bf800640 1 -- 172.21.15.167:0/155495020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9b80b3ea0 msgr2=0x7fe9b80b1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:49.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.943+0000 7fe9bf800640 1 --2- 172.21.15.167:0/155495020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9b80b3ea0 0x7fe9b80b1cd0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe9a400c430 tx=0x7fe9a400c8f0 comp rx=0 tx=0).stop 2024-05-11T12:13:49.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.943+0000 7fe9bf800640 1 -- 172.21.15.167:0/155495020 shutdown_connections 2024-05-11T12:13:49.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.943+0000 7fe9bf800640 1 --2- 172.21.15.167:0/155495020 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe9b8157750 0x7fe9b81f86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:49.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.943+0000 7fe9bf800640 1 --2- 172.21.15.167:0/155495020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9b80b3ea0 0x7fe9b80b1cd0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:49.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.943+0000 7fe9bf800640 1 --2- 172.21.15.167:0/155495020 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe9980762d0 0x7fe998078790 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:49.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.943+0000 7fe9bf800640 1 --2- 172.21.15.167:0/155495020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9b806d460 0x7fe9b8072cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:49.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.943+0000 7fe9bf800640 1 -- 172.21.15.167:0/155495020 >> 172.21.15.167:0/155495020 conn(0x7fe9b80cd8f0 msgr2=0x7fe9b80681c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:49.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.943+0000 7fe9bf800640 1 -- 172.21.15.167:0/155495020 shutdown_connections 2024-05-11T12:13:49.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:49.944+0000 7fe9bf800640 1 -- 172.21.15.167:0/155495020 wait complete. 2024-05-11T12:13:49.963 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:13:50.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.160+0000 7f2a05a00640 1 Processor -- start 2024-05-11T12:13:50.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.160+0000 7f2a05a00640 1 -- start start 2024-05-11T12:13:50.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.162+0000 7f2a05a00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2a000afcd0 0x7f2a001601f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:50.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.162+0000 7f2a05a00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a000b4560 con 0x7f2a000d04a0 2024-05-11T12:13:50.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.162+0000 7f2a05a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a000b4800 con 0x7f2a000cf970 2024-05-11T12:13:50.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.162+0000 7f2a05a00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f2a000b5130 con 0x7f2a000afcd0 2024-05-11T12:13:50.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.162+0000 7f29ffe00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2a000afcd0 0x7f2a001601f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:50.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.162+0000 7f29f7400640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f2a000d04a0 0x7f2a000cf260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:52722/0 (socket says 172.21.15.167:52722) 2024-05-11T12:13:50.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.162+0000 7f29f7400640 1 -- 172.21.15.167:0/2329953342 learned_addr learned my addr 172.21.15.167:0/2329953342 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:13:50.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.162+0000 7f29ff400640 1 -- 172.21.15.167:0/2329953342 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 630480950 0 0) 0x7f2a000b4560 con 0x7f2a000d04a0 2024-05-11T12:13:50.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.163+0000 7f29ff400640 1 -- 172.21.15.167:0/2329953342 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f29e4003520 con 0x7f2a000d04a0 2024-05-11T12:13:50.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.163+0000 7f29ffe00640 1 -- 172.21.15.167:0/2329953342 >> v1:172.21.15.204:6789/0 conn(0x7f2a000cf970 legacy=0x7f2a000721d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:50.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.163+0000 7f29ffe00640 1 -- 172.21.15.167:0/2329953342 >> v1:172.21.15.167:6789/0 conn(0x7f2a000d04a0 legacy=0x7f2a000cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:50.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.163+0000 7f29ffe00640 1 -- 172.21.15.167:0/2329953342 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a000b4ae0 con 0x7f2a000afcd0 2024-05-11T12:13:50.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.164+0000 7f29ffe00640 1 --2- 172.21.15.167:0/2329953342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a000afcd0 0x7f2a001601f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f29f00097a0 tx=0x7f29f002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b78081664d5fa48c server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:50.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.164+0000 7f29ff400640 1 -- 172.21.15.167:0/2329953342 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f0037070 con 0x7f2a000afcd0 2024-05-11T12:13:50.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.164+0000 7f29ff400640 1 -- 172.21.15.167:0/2329953342 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29f0032440 con 0x7f2a000afcd0 2024-05-11T12:13:50.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.164+0000 7f29ff400640 1 -- 172.21.15.167:0/2329953342 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f0032810 con 0x7f2a000afcd0 2024-05-11T12:13:50.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.165+0000 7f2a05a00640 1 -- 172.21.15.167:0/2329953342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a000afcd0 msgr2=0x7f2a001601f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:50.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.165+0000 7f2a05a00640 1 --2- 172.21.15.167:0/2329953342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a000afcd0 0x7f2a001601f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f29f00097a0 tx=0x7f29f002a180 comp rx=0 tx=0).stop 2024-05-11T12:13:50.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.165+0000 7f2a05a00640 1 -- 172.21.15.167:0/2329953342 shutdown_connections 2024-05-11T12:13:50.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.165+0000 7f2a05a00640 1 --2- 172.21.15.167:0/2329953342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a000afcd0 0x7f2a001601f0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:50.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.165+0000 7f2a05a00640 1 -- 172.21.15.167:0/2329953342 >> 172.21.15.167:0/2329953342 conn(0x7f2a000b3460 msgr2=0x7f2a000d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:50.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.165+0000 7f2a05a00640 1 -- 172.21.15.167:0/2329953342 shutdown_connections 2024-05-11T12:13:50.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.165+0000 7f2a05a00640 1 -- 172.21.15.167:0/2329953342 wait complete. 2024-05-11T12:13:50.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.166+0000 7f2a05a00640 1 Processor -- start 2024-05-11T12:13:50.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.167+0000 7f2a05a00640 1 -- start start 2024-05-11T12:13:50.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.167+0000 7f2a05a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a000afcd0 0x7f2a00151c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:50.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.167+0000 7f2a05a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a000cf970 0x7f2a0006ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:50.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.168+0000 7f2a05a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2a000d04a0 0x7f2a00141490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:50.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.168+0000 7f2a05a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2a000cff40 con 0x7f2a000cf970 2024-05-11T12:13:50.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.168+0000 7f2a05a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2a000b00b0 con 0x7f2a000afcd0 2024-05-11T12:13:50.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.168+0000 7f2a05a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2a000d00c0 con 0x7f2a000d04a0 2024-05-11T12:13:50.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.168+0000 7f29f7400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a000cf970 0x7f2a0006ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:50.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.168+0000 7f29f7400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a000cf970 0x7f2a0006ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54166/0 (socket says 172.21.15.167:54166) 2024-05-11T12:13:50.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.168+0000 7f29f7400640 1 -- 172.21.15.167:0/841957889 learned_addr learned my addr 172.21.15.167:0/841957889 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:50.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.168+0000 7f29ffe00640 1 --2- 172.21.15.167:0/841957889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a000afcd0 0x7f2a00151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:50.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.168+0000 7f2a05000640 1 --2- 172.21.15.167:0/841957889 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2a000d04a0 0x7f2a00141490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:50.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.168+0000 7f29f7400640 1 -- 172.21.15.167:0/841957889 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2a000d04a0 msgr2=0x7f2a00141490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:50.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.168+0000 7f29f7400640 1 --2- 172.21.15.167:0/841957889 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2a000d04a0 0x7f2a00141490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:50.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.168+0000 7f29f7400640 1 -- 172.21.15.167:0/841957889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a000afcd0 msgr2=0x7f2a00151c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:50.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.168+0000 7f29f7400640 1 --2- 172.21.15.167:0/841957889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a000afcd0 0x7f2a00151c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:50.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.168+0000 7f29f7400640 1 -- 172.21.15.167:0/841957889 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a0006d600 con 0x7f2a000cf970 2024-05-11T12:13:50.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.168+0000 7f29ffe00640 1 --2- 172.21.15.167:0/841957889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a000afcd0 0x7f2a00151c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:50.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.169+0000 7f29f7400640 1 --2- 172.21.15.167:0/841957889 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a000cf970 0x7f2a0006ce20 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f29e800c400 tx=0x7f29e800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:50.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.169+0000 7f29fcc00640 1 -- 172.21.15.167:0/841957889 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e8017070 con 0x7f2a000cf970 2024-05-11T12:13:50.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.169+0000 7f29fcc00640 1 -- 172.21.15.167:0/841957889 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29e8005ce0 con 0x7f2a000cf970 2024-05-11T12:13:50.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.169+0000 7f29fcc00640 1 -- 172.21.15.167:0/841957889 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e80072d0 con 0x7f2a000cf970 2024-05-11T12:13:50.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.169+0000 7f2a05a00640 1 -- 172.21.15.167:0/841957889 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a00153140 con 0x7f2a000cf970 2024-05-11T12:13:50.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.169+0000 7f2a05a00640 1 -- 172.21.15.167:0/841957889 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a00152380 con 0x7f2a000cf970 2024-05-11T12:13:50.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.170+0000 7f2a05a00640 1 -- 172.21.15.167:0/841957889 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a000694b0 con 0x7f2a000cf970 2024-05-11T12:13:50.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.171+0000 7f29fcc00640 1 -- 172.21.15.167:0/841957889 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f29e801a070 con 0x7f2a000cf970 2024-05-11T12:13:50.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.172+0000 7f29fcc00640 1 --2- 172.21.15.167:0/841957889 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f29dc0762d0 0x7f29dc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:50.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.172+0000 7f29fcc00640 1 -- 172.21.15.167:0/841957889 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7f29e8096ae0 con 0x7f2a000cf970 2024-05-11T12:13:50.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.173+0000 7f29ffe00640 1 --2- 172.21.15.167:0/841957889 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f29dc0762d0 0x7f29dc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:50.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.173+0000 7f29ffe00640 1 --2- 172.21.15.167:0/841957889 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f29dc0762d0 0x7f29dc078790 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f29f0034000 tx=0x7f29f002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:50.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.177+0000 7f29fcc00640 1 -- 172.21.15.167:0/841957889 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2a0006d600 con 0x7f2a000cf970 2024-05-11T12:13:50.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.390+0000 7f2a05a00640 1 -- 172.21.15.167:0/841957889 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a0015f240 con 0x7f2a000cf970 2024-05-11T12:13:50.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.393+0000 7f29fcc00640 1 -- 172.21.15.167:0/841957889 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+16993 (secure 0 0 0) 0x7f29e805fc80 con 0x7f2a000cf970 2024-05-11T12:13:50.394 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:13:50.394 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":75,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:13:36.554508+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:13:32.903032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-11T12:13:33.980753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:13:50.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.396+0000 7f2a05a00640 1 -- 172.21.15.167:0/841957889 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f29dc0762d0 msgr2=0x7f29dc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:50.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.396+0000 7f2a05a00640 1 --2- 172.21.15.167:0/841957889 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f29dc0762d0 0x7f29dc078790 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f29f0034000 tx=0x7f29f002e000 comp rx=0 tx=0).stop 2024-05-11T12:13:50.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.396+0000 7f2a05a00640 1 -- 172.21.15.167:0/841957889 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a000cf970 msgr2=0x7f2a0006ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:50.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.396+0000 7f2a05a00640 1 --2- 172.21.15.167:0/841957889 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a000cf970 0x7f2a0006ce20 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f29e800c400 tx=0x7f29e800c8c0 comp rx=0 tx=0).stop 2024-05-11T12:13:50.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.396+0000 7f2a05a00640 1 -- 172.21.15.167:0/841957889 shutdown_connections 2024-05-11T12:13:50.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.396+0000 7f2a05a00640 1 --2- 172.21.15.167:0/841957889 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2a000d04a0 0x7f2a00141490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:50.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.396+0000 7f2a05a00640 1 --2- 172.21.15.167:0/841957889 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a000cf970 0x7f2a0006ce20 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:50.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.396+0000 7f2a05a00640 1 --2- 172.21.15.167:0/841957889 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f29dc0762d0 0x7f29dc078790 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:50.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.396+0000 7f2a05a00640 1 --2- 172.21.15.167:0/841957889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a000afcd0 0x7f2a00151c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:50.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.396+0000 7f2a05a00640 1 -- 172.21.15.167:0/841957889 >> 172.21.15.167:0/841957889 conn(0x7f2a000b3460 msgr2=0x7f2a000b0760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:50.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.396+0000 7f2a05a00640 1 -- 172.21.15.167:0/841957889 shutdown_connections 2024-05-11T12:13:50.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.397+0000 7f2a05a00640 1 -- 172.21.15.167:0/841957889 wait complete. 2024-05-11T12:13:50.417 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-11T12:13:50.417 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:13:50.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.612+0000 7f2e09600640 1 Processor -- start 2024-05-11T12:13:50.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.613+0000 7f2e09600640 1 -- start start 2024-05-11T12:13:50.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.614+0000 7f2e09600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e040d04a0 0x7f2e041601f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:50.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.614+0000 7f2e09600640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e040b4560 con 0x7f2e040afcd0 2024-05-11T12:13:50.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.614+0000 7f2e09600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e040b4800 con 0x7f2e040cf970 2024-05-11T12:13:50.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.614+0000 7f2e09600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2e040b5130 con 0x7f2e040d04a0 2024-05-11T12:13:50.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.614+0000 7f2e03e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f2e040afcd0 0x7f2e040cf260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:52730/0 (socket says 172.21.15.167:52730) 2024-05-11T12:13:50.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.615+0000 7f2e03e00640 1 -- 172.21.15.167:0/2001875900 learned_addr learned my addr 172.21.15.167:0/2001875900 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:13:50.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.615+0000 7f2e08c00640 1 --2- 172.21.15.167:0/2001875900 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e040d04a0 0x7f2e041601f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:50.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.615+0000 7f2e03400640 1 -- 172.21.15.167:0/2001875900 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2477089584 0 0) 0x7f2e040b4560 con 0x7f2e040afcd0 2024-05-11T12:13:50.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.615+0000 7f2e03400640 1 -- 172.21.15.167:0/2001875900 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2de8003520 con 0x7f2e040afcd0 2024-05-11T12:13:50.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.615+0000 7f2e03400640 1 -- 172.21.15.167:0/2001875900 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3578679365 0 0) 0x7f2de8003520 con 0x7f2e040afcd0 2024-05-11T12:13:50.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.615+0000 7f2e03400640 1 -- 172.21.15.167:0/2001875900 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e040d04a0 msgr2=0x7f2e041601f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:50.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.615+0000 7f2e03400640 1 --2- 172.21.15.167:0/2001875900 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e040d04a0 0x7f2e041601f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:50.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.615+0000 7f2e03400640 1 -- 172.21.15.167:0/2001875900 >> v1:172.21.15.204:6789/0 conn(0x7f2e040cf970 legacy=0x7f2e040721d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:50.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.615+0000 7f2e03400640 1 -- 172.21.15.167:0/2001875900 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e040b4ae0 con 0x7f2e040afcd0 2024-05-11T12:13:50.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.615+0000 7f2e03400640 1 -- 172.21.15.167:0/2001875900 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f2df40016f0 con 0x7f2e040afcd0 2024-05-11T12:13:50.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.616+0000 7f2e03400640 1 -- 172.21.15.167:0/2001875900 >> v1:172.21.15.167:6789/0 conn(0x7f2e040afcd0 legacy=0x7f2e040cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:50.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.616+0000 7f2e03400640 1 --2- 172.21.15.167:0/2001875900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2de8003a30 0x7f2de8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:50.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.616+0000 7f2e03400640 1 --2- 172.21.15.167:0/2001875900 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e040d04a0 0x7f2de8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:50.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.616+0000 7f2e03400640 1 --2- 172.21.15.167:0/2001875900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2de8004d40 0x7f2de80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:50.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.616+0000 7f2e03400640 1 -- 172.21.15.167:0/2001875900 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2e040b5130 con 0x7f2de8003a30 2024-05-11T12:13:50.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.616+0000 7f2e03400640 1 -- 172.21.15.167:0/2001875900 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2de800baf0 con 0x7f2de8004d40 2024-05-11T12:13:50.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.616+0000 7f2e03400640 1 -- 172.21.15.167:0/2001875900 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2de800bcf0 con 0x7f2e040d04a0 2024-05-11T12:13:50.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.616+0000 7f2e03e00640 1 --2- 172.21.15.167:0/2001875900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2de8003a30 0x7f2de8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:50.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.616+0000 7f2dfb400640 1 --2- 172.21.15.167:0/2001875900 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e040d04a0 0x7f2de8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:50.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.616+0000 7f2e08c00640 1 --2- 172.21.15.167:0/2001875900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2de8004d40 0x7f2de80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:50.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.616+0000 7f2e03e00640 1 -- 172.21.15.167:0/2001875900 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e040d04a0 msgr2=0x7f2de8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:50.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.616+0000 7f2e03e00640 1 --2- 172.21.15.167:0/2001875900 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e040d04a0 0x7f2de8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:50.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.616+0000 7f2e03e00640 1 -- 172.21.15.167:0/2001875900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2de8004d40 msgr2=0x7f2de80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:50.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.616+0000 7f2e03e00640 1 --2- 172.21.15.167:0/2001875900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2de8004d40 0x7f2de80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:50.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.617+0000 7f2e03e00640 1 -- 172.21.15.167:0/2001875900 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2de8003520 con 0x7f2de8003a30 2024-05-11T12:13:50.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.618+0000 7f2e03e00640 1 --2- 172.21.15.167:0/2001875900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2de8003a30 0x7f2de8003e50 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f2df400c240 tx=0x7f2de8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78bb8b5f5e89ec31 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:50.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.618+0000 7f2e03400640 1 -- 172.21.15.167:0/2001875900 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df4007d60 con 0x7f2de8003a30 2024-05-11T12:13:50.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.618+0000 7f2e03400640 1 -- 172.21.15.167:0/2001875900 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2df4005ad0 con 0x7f2de8003a30 2024-05-11T12:13:50.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.618+0000 7f2e03400640 1 -- 172.21.15.167:0/2001875900 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df4007070 con 0x7f2de8003a30 2024-05-11T12:13:50.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.618+0000 7f2e09600640 1 -- 172.21.15.167:0/2001875900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2de8003a30 msgr2=0x7f2de8003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:50.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.618+0000 7f2e09600640 1 --2- 172.21.15.167:0/2001875900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2de8003a30 0x7f2de8003e50 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f2df400c240 tx=0x7f2de8005830 comp rx=0 tx=0).stop 2024-05-11T12:13:50.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.618+0000 7f2e08c00640 1 -- 172.21.15.167:0/2001875900 reap_dead start 2024-05-11T12:13:50.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.618+0000 7f2e09600640 1 -- 172.21.15.167:0/2001875900 shutdown_connections 2024-05-11T12:13:50.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.618+0000 7f2e09600640 1 -- 172.21.15.167:0/2001875900 >> 172.21.15.167:0/2001875900 conn(0x7f2e040b3460 msgr2=0x7f2e040d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:50.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.619+0000 7f2e09600640 1 -- 172.21.15.167:0/2001875900 shutdown_connections 2024-05-11T12:13:50.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.619+0000 7f2e09600640 1 -- 172.21.15.167:0/2001875900 wait complete. 2024-05-11T12:13:50.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.620+0000 7f2e09600640 1 Processor -- start 2024-05-11T12:13:50.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.620+0000 7f2e09600640 1 -- start start 2024-05-11T12:13:50.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.621+0000 7f2e09600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2de8003a30 0x7f2e0414d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:50.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.621+0000 7f2e09600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e0415a750 0x7f2e0415cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:50.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.621+0000 7f2e09600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e0415abb0 0x7f2e0415d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:50.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.621+0000 7f2e09600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2e040cff40 con 0x7f2de8003a30 2024-05-11T12:13:50.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.621+0000 7f2e09600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2e040b00b0 con 0x7f2e0415abb0 2024-05-11T12:13:50.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.621+0000 7f2e09600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2e040d00c0 con 0x7f2e0415a750 2024-05-11T12:13:50.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.622+0000 7f2e03e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2de8003a30 0x7f2e0414d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:50.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.622+0000 7f2e03e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2de8003a30 0x7f2e0414d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54184/0 (socket says 172.21.15.167:54184) 2024-05-11T12:13:50.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.622+0000 7f2e03e00640 1 -- 172.21.15.167:0/219716735 learned_addr learned my addr 172.21.15.167:0/219716735 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:50.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.622+0000 7f2e08c00640 1 --2- 172.21.15.167:0/219716735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e0415abb0 0x7f2e0415d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:50.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.622+0000 7f2dfb400640 1 --2- 172.21.15.167:0/219716735 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e0415a750 0x7f2e0415cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:50.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.622+0000 7f2e03e00640 1 -- 172.21.15.167:0/219716735 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e0415a750 msgr2=0x7f2e0415cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:50.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.622+0000 7f2e03e00640 1 --2- 172.21.15.167:0/219716735 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e0415a750 0x7f2e0415cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:50.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.622+0000 7f2e03e00640 1 -- 172.21.15.167:0/219716735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e0415abb0 msgr2=0x7f2e0415d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:50.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.622+0000 7f2e03e00640 1 --2- 172.21.15.167:0/219716735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e0415abb0 0x7f2e0415d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:50.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.622+0000 7f2e03e00640 1 -- 172.21.15.167:0/219716735 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e04162690 con 0x7f2de8003a30 2024-05-11T12:13:50.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.622+0000 7f2dfb400640 1 --2- 172.21.15.167:0/219716735 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e0415a750 0x7f2e0415cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:50.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.622+0000 7f2e08c00640 1 --2- 172.21.15.167:0/219716735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e0415abb0 0x7f2e0415d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:50.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.622+0000 7f2e03e00640 1 --2- 172.21.15.167:0/219716735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2de8003a30 0x7f2e0414d4a0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f2df400c270 tx=0x7f2e040b4cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:50.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.623+0000 7f2e00c00640 1 -- 172.21.15.167:0/219716735 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df4043070 con 0x7f2de8003a30 2024-05-11T12:13:50.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.623+0000 7f2e00c00640 1 -- 172.21.15.167:0/219716735 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2df4038040 con 0x7f2de8003a30 2024-05-11T12:13:50.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.623+0000 7f2e00c00640 1 -- 172.21.15.167:0/219716735 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df4007290 con 0x7f2de8003a30 2024-05-11T12:13:50.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.623+0000 7f2e09600640 1 -- 172.21.15.167:0/219716735 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e04162970 con 0x7f2de8003a30 2024-05-11T12:13:50.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.623+0000 7f2e09600640 1 -- 172.21.15.167:0/219716735 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e04162f30 con 0x7f2de8003a30 2024-05-11T12:13:50.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.624+0000 7f2e09600640 1 -- 172.21.15.167:0/219716735 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e040ce9d0 con 0x7f2de8003a30 2024-05-11T12:13:50.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.625+0000 7f2e00c00640 1 -- 172.21.15.167:0/219716735 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2df4007430 con 0x7f2de8003a30 2024-05-11T12:13:50.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.626+0000 7f2e00c00640 1 --2- 172.21.15.167:0/219716735 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2de0076300 0x7f2de00787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:50.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.626+0000 7f2e00c00640 1 -- 172.21.15.167:0/219716735 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7f2df40b7290 con 0x7f2de8003a30 2024-05-11T12:13:50.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.627+0000 7f2dfb400640 1 --2- 172.21.15.167:0/219716735 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2de0076300 0x7f2de00787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:50.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.627+0000 7f2dfb400640 1 --2- 172.21.15.167:0/219716735 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2de0076300 0x7f2de00787c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2dec000c00 tx=0x7f2dec009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:50.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.631+0000 7f2e00c00640 1 -- 172.21.15.167:0/219716735 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2e040ce9d0 con 0x7f2de8003a30 2024-05-11T12:13:50.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.877+0000 7f2e09600640 1 -- 172.21.15.167:0/219716735 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2e040cee20 con 0x7f2de8003a30 2024-05-11T12:13:50.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.878+0000 7f2e00c00640 1 -- 172.21.15.167:0/219716735 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5195 (secure 0 0 0) 0x7f2e040cee20 con 0x7f2de8003a30 2024-05-11T12:13:50.879 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:13:50.879 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5020,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3075930607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3075930607},{"type":"v1","addr":"172.21.15.167:6836","nonce":3075930607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":5032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3056051531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3056051531},{"type":"v1","addr":"172.21.15.204:6834","nonce":3056051531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5035,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2900478909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":2900478909},{"type":"v1","addr":"172.21.15.204:6835","nonce":2900478909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:13:34.993949+0000","modified":"2024-05-11T12:13:38.072880+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5952},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5952":{"gid":5952,"name":"a","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/1306616214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1306616214},{"type":"v1","addr":"172.21.15.167:6837","nonce":1306616214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5952,"qdb_cluster":[5952]},"id":5}]} 2024-05-11T12:13:50.879 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 44 2024-05-11T12:13:50.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.881+0000 7f2e09600640 1 -- 172.21.15.167:0/219716735 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2de0076300 msgr2=0x7f2de00787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:50.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.881+0000 7f2e09600640 1 --2- 172.21.15.167:0/219716735 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2de0076300 0x7f2de00787c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2dec000c00 tx=0x7f2dec009040 comp rx=0 tx=0).stop 2024-05-11T12:13:50.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.881+0000 7f2e09600640 1 -- 172.21.15.167:0/219716735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2de8003a30 msgr2=0x7f2e0414d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:50.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.881+0000 7f2e09600640 1 --2- 172.21.15.167:0/219716735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2de8003a30 0x7f2e0414d4a0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f2df400c270 tx=0x7f2e040b4cb0 comp rx=0 tx=0).stop 2024-05-11T12:13:50.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.881+0000 7f2e09600640 1 -- 172.21.15.167:0/219716735 shutdown_connections 2024-05-11T12:13:50.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.881+0000 7f2e09600640 1 --2- 172.21.15.167:0/219716735 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2de0076300 0x7f2de00787c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:50.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.881+0000 7f2e09600640 1 --2- 172.21.15.167:0/219716735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e0415abb0 0x7f2e0415d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:50.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.881+0000 7f2e09600640 1 --2- 172.21.15.167:0/219716735 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e0415a750 0x7f2e0415cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:50.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.881+0000 7f2e09600640 1 --2- 172.21.15.167:0/219716735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2de8003a30 0x7f2e0414d4a0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:50.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.881+0000 7f2e09600640 1 -- 172.21.15.167:0/219716735 >> 172.21.15.167:0/219716735 conn(0x7f2e040b3460 msgr2=0x7f2e04072f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:50.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.881+0000 7f2e09600640 1 -- 172.21.15.167:0/219716735 shutdown_connections 2024-05-11T12:13:50.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:50.881+0000 7f2e09600640 1 -- 172.21.15.167:0/219716735 wait complete. 2024-05-11T12:13:50.900 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:13:51.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.103+0000 7f0a99600640 1 Processor -- start 2024-05-11T12:13:51.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.103+0000 7f0a99600640 1 -- start start 2024-05-11T12:13:51.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.105+0000 7f0a99600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a94157750 0x7f0a9415d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:51.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.105+0000 7f0a99600640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a9415dc80 con 0x7f0a9406d460 2024-05-11T12:13:51.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.105+0000 7f0a99600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a9415df30 con 0x7f0a940b3ea0 2024-05-11T12:13:51.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.105+0000 7f0a99600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0a94161000 con 0x7f0a94157750 2024-05-11T12:13:51.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.105+0000 7f0a93400640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f0a9406d460 0x7f0a940cf1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:52732/0 (socket says 172.21.15.167:52732) 2024-05-11T12:13:51.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.105+0000 7f0a93400640 1 -- 172.21.15.167:0/3723107390 learned_addr learned my addr 172.21.15.167:0/3723107390 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:13:51.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.105+0000 7f0a92a00640 1 --2- 172.21.15.167:0/3723107390 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a94157750 0x7f0a9415d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:51.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.106+0000 7f0a91600640 1 -- 172.21.15.167:0/3723107390 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 884994174 0 0) 0x7f0a9415dc80 con 0x7f0a9406d460 2024-05-11T12:13:51.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.106+0000 7f0a91600640 1 -- 172.21.15.167:0/3723107390 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0a78003520 con 0x7f0a9406d460 2024-05-11T12:13:51.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.106+0000 7f0a91600640 1 -- 172.21.15.167:0/3723107390 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4061996384 0 0) 0x7f0a78003520 con 0x7f0a9406d460 2024-05-11T12:13:51.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.106+0000 7f0a91600640 1 -- 172.21.15.167:0/3723107390 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a94157750 msgr2=0x7f0a9415d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:51.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.106+0000 7f0a91600640 1 --2- 172.21.15.167:0/3723107390 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a94157750 0x7f0a9415d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:51.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.106+0000 7f0a91600640 1 -- 172.21.15.167:0/3723107390 >> v1:172.21.15.204:6789/0 conn(0x7f0a940b3ea0 legacy=0x7f0a94161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:51.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.106+0000 7f0a91600640 1 -- 172.21.15.167:0/3723107390 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a9415e210 con 0x7f0a9406d460 2024-05-11T12:13:51.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.106+0000 7f0a91600640 1 -- 172.21.15.167:0/3723107390 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f0a8c001a70 con 0x7f0a9406d460 2024-05-11T12:13:51.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.106+0000 7f0a91600640 1 -- 172.21.15.167:0/3723107390 >> v1:172.21.15.167:6789/0 conn(0x7f0a9406d460 legacy=0x7f0a940cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:51.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.106+0000 7f0a91600640 1 --2- 172.21.15.167:0/3723107390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a78003a80 0x7f0a78003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:51.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.106+0000 7f0a91600640 1 --2- 172.21.15.167:0/3723107390 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a94157750 0x7f0a78004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:51.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.106+0000 7f0a91600640 1 --2- 172.21.15.167:0/3723107390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a78004d40 0x7f0a780051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:51.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.106+0000 7f0a91600640 1 -- 172.21.15.167:0/3723107390 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0a94161000 con 0x7f0a78003a80 2024-05-11T12:13:51.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.106+0000 7f0a91600640 1 -- 172.21.15.167:0/3723107390 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7800bb50 con 0x7f0a78004d40 2024-05-11T12:13:51.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.106+0000 7f0a91600640 1 -- 172.21.15.167:0/3723107390 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0a7800bd50 con 0x7f0a94157750 2024-05-11T12:13:51.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.106+0000 7f0a92a00640 1 --2- 172.21.15.167:0/3723107390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a78003a80 0x7f0a78003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:51.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.107+0000 7f0a92000640 1 --2- 172.21.15.167:0/3723107390 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a94157750 0x7f0a78004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:51.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.107+0000 7f0a92a00640 1 -- 172.21.15.167:0/3723107390 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a94157750 msgr2=0x7f0a78004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:51.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.107+0000 7f0a93400640 1 --2- 172.21.15.167:0/3723107390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a78004d40 0x7f0a780051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:51.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.107+0000 7f0a92a00640 1 --2- 172.21.15.167:0/3723107390 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a94157750 0x7f0a78004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:51.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.107+0000 7f0a92a00640 1 -- 172.21.15.167:0/3723107390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a78004d40 msgr2=0x7f0a780051a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:51.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.107+0000 7f0a92a00640 1 --2- 172.21.15.167:0/3723107390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a78004d40 0x7f0a780051a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:51.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.107+0000 7f0a92a00640 1 -- 172.21.15.167:0/3723107390 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a78003520 con 0x7f0a78003a80 2024-05-11T12:13:51.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.107+0000 7f0a92000640 1 --2- 172.21.15.167:0/3723107390 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a94157750 0x7f0a78004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:51.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.107+0000 7f0a93400640 1 --2- 172.21.15.167:0/3723107390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a78004d40 0x7f0a780051a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:51.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.108+0000 7f0a92a00640 1 --2- 172.21.15.167:0/3723107390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a78003a80 0x7f0a78003ea0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f0a84009ac0 tx=0x7f0a78005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c18b7a0c51ee333 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:51.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.108+0000 7f0a91600640 1 -- 172.21.15.167:0/3723107390 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a84036070 con 0x7f0a78003a80 2024-05-11T12:13:51.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.108+0000 7f0a91600640 1 -- 172.21.15.167:0/3723107390 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a84031400 con 0x7f0a78003a80 2024-05-11T12:13:51.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.108+0000 7f0a91600640 1 -- 172.21.15.167:0/3723107390 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a84009690 con 0x7f0a78003a80 2024-05-11T12:13:51.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.109+0000 7f0a99600640 1 -- 172.21.15.167:0/3723107390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a78003a80 msgr2=0x7f0a78003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:51.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.109+0000 7f0a99600640 1 --2- 172.21.15.167:0/3723107390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a78003a80 0x7f0a78003ea0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f0a84009ac0 tx=0x7f0a78005830 comp rx=0 tx=0).stop 2024-05-11T12:13:51.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.109+0000 7f0a93400640 1 -- 172.21.15.167:0/3723107390 reap_dead start 2024-05-11T12:13:51.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.109+0000 7f0a99600640 1 -- 172.21.15.167:0/3723107390 shutdown_connections 2024-05-11T12:13:51.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.109+0000 7f0a99600640 1 -- 172.21.15.167:0/3723107390 >> 172.21.15.167:0/3723107390 conn(0x7f0a940cd8f0 msgr2=0x7f0a940cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:51.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.109+0000 7f0a99600640 1 -- 172.21.15.167:0/3723107390 shutdown_connections 2024-05-11T12:13:51.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.109+0000 7f0a99600640 1 -- 172.21.15.167:0/3723107390 wait complete. 2024-05-11T12:13:51.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.111+0000 7f0a99600640 1 Processor -- start 2024-05-11T12:13:51.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.111+0000 7f0a99600640 1 -- start start 2024-05-11T12:13:51.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.112+0000 7f0a99600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a78003a80 0x7f0a940af230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:51.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.112+0000 7f0a99600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a9415b0d0 0x7f0a9413a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:51.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.112+0000 7f0a99600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a9415b720 0x7f0a9415bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:51.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.112+0000 7f0a99600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0a94161ba0 con 0x7f0a78003a80 2024-05-11T12:13:51.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.112+0000 7f0a99600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0a94157b30 con 0x7f0a9415b720 2024-05-11T12:13:51.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.112+0000 7f0a99600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0a940cf8d0 con 0x7f0a9415b0d0 2024-05-11T12:13:51.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.112+0000 7f0a92a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a78003a80 0x7f0a940af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:51.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.112+0000 7f0a92a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a78003a80 0x7f0a940af230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54198/0 (socket says 172.21.15.167:54198) 2024-05-11T12:13:51.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.112+0000 7f0a92a00640 1 -- 172.21.15.167:0/3599184068 learned_addr learned my addr 172.21.15.167:0/3599184068 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:51.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.112+0000 7f0a92000640 1 --2- 172.21.15.167:0/3599184068 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a9415b0d0 0x7f0a9413a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:51.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.112+0000 7f0a93400640 1 --2- 172.21.15.167:0/3599184068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a9415b720 0x7f0a9415bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:51.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.112+0000 7f0a92a00640 1 -- 172.21.15.167:0/3599184068 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a9415b0d0 msgr2=0x7f0a9413a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:51.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.113+0000 7f0a92a00640 1 --2- 172.21.15.167:0/3599184068 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a9415b0d0 0x7f0a9413a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:51.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.113+0000 7f0a92a00640 1 -- 172.21.15.167:0/3599184068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a9415b720 msgr2=0x7f0a9415bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:51.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.113+0000 7f0a92a00640 1 --2- 172.21.15.167:0/3599184068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a9415b720 0x7f0a9415bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:51.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.113+0000 7f0a92a00640 1 -- 172.21.15.167:0/3599184068 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a940afa60 con 0x7f0a78003a80 2024-05-11T12:13:51.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.113+0000 7f0a92a00640 1 --2- 172.21.15.167:0/3599184068 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a78003a80 0x7f0a940af230 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f0a84034040 tx=0x7f0a84004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:51.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.113+0000 7f0a8aa00640 1 -- 172.21.15.167:0/3599184068 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a84040020 con 0x7f0a78003a80 2024-05-11T12:13:51.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.113+0000 7f0a8aa00640 1 -- 172.21.15.167:0/3599184068 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a84003bc0 con 0x7f0a78003a80 2024-05-11T12:13:51.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.113+0000 7f0a8aa00640 1 -- 172.21.15.167:0/3599184068 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a84036040 con 0x7f0a78003a80 2024-05-11T12:13:51.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.113+0000 7f0a99600640 1 -- 172.21.15.167:0/3599184068 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a940afd40 con 0x7f0a78003a80 2024-05-11T12:13:51.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.113+0000 7f0a99600640 1 -- 172.21.15.167:0/3599184068 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a94072420 con 0x7f0a78003a80 2024-05-11T12:13:51.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.115+0000 7f0a99600640 1 -- 172.21.15.167:0/3599184068 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a94069590 con 0x7f0a78003a80 2024-05-11T12:13:51.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.116+0000 7f0a8aa00640 1 -- 172.21.15.167:0/3599184068 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0a84043020 con 0x7f0a78003a80 2024-05-11T12:13:51.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.117+0000 7f0a8aa00640 1 --2- 172.21.15.167:0/3599184068 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0a740762d0 0x7f0a74078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:51.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.117+0000 7f0a8aa00640 1 -- 172.21.15.167:0/3599184068 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7f0a840b78e0 con 0x7f0a78003a80 2024-05-11T12:13:51.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.117+0000 7f0a92000640 1 --2- 172.21.15.167:0/3599184068 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0a740762d0 0x7f0a74078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:51.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.118+0000 7f0a92000640 1 --2- 172.21.15.167:0/3599184068 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0a740762d0 0x7f0a74078790 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0a9406f600 tx=0x7f0a80009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:51.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.121+0000 7f0a8aa00640 1 -- 172.21.15.167:0/3599184068 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0a8404a070 con 0x7f0a78003a80 2024-05-11T12:13:51.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.369+0000 7f0a99600640 1 -- 172.21.15.167:0/3599184068 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a94160370 con 0x7f0a78003a80 2024-05-11T12:13:51.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.370+0000 7f0a8aa00640 1 -- 172.21.15.167:0/3599184068 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5195 (secure 0 0 0) 0x7f0a840044e0 con 0x7f0a78003a80 2024-05-11T12:13:51.371 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:13:51.371 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5020,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3075930607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3075930607},{"type":"v1","addr":"172.21.15.167:6836","nonce":3075930607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":5032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3056051531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3056051531},{"type":"v1","addr":"172.21.15.204:6834","nonce":3056051531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5035,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2900478909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":2900478909},{"type":"v1","addr":"172.21.15.204:6835","nonce":2900478909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:13:34.993949+0000","modified":"2024-05-11T12:13:38.072880+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5952},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5952":{"gid":5952,"name":"a","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/1306616214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1306616214},{"type":"v1","addr":"172.21.15.167:6837","nonce":1306616214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5952,"qdb_cluster":[5952]},"id":5}]} 2024-05-11T12:13:51.371 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 44 2024-05-11T12:13:51.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.373+0000 7f0a99600640 1 -- 172.21.15.167:0/3599184068 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0a740762d0 msgr2=0x7f0a74078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:51.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.373+0000 7f0a99600640 1 --2- 172.21.15.167:0/3599184068 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0a740762d0 0x7f0a74078790 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0a9406f600 tx=0x7f0a80009040 comp rx=0 tx=0).stop 2024-05-11T12:13:51.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.373+0000 7f0a99600640 1 -- 172.21.15.167:0/3599184068 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a78003a80 msgr2=0x7f0a940af230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:51.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.373+0000 7f0a99600640 1 --2- 172.21.15.167:0/3599184068 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a78003a80 0x7f0a940af230 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f0a84034040 tx=0x7f0a84004730 comp rx=0 tx=0).stop 2024-05-11T12:13:51.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.373+0000 7f0a99600640 1 -- 172.21.15.167:0/3599184068 shutdown_connections 2024-05-11T12:13:51.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.373+0000 7f0a99600640 1 --2- 172.21.15.167:0/3599184068 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0a740762d0 0x7f0a74078790 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:51.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.373+0000 7f0a99600640 1 --2- 172.21.15.167:0/3599184068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a9415b720 0x7f0a9415bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:51.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.373+0000 7f0a99600640 1 --2- 172.21.15.167:0/3599184068 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a9415b0d0 0x7f0a9413a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:51.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.373+0000 7f0a99600640 1 --2- 172.21.15.167:0/3599184068 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a78003a80 0x7f0a940af230 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:51.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.373+0000 7f0a99600640 1 -- 172.21.15.167:0/3599184068 >> 172.21.15.167:0/3599184068 conn(0x7f0a940cd8f0 msgr2=0x7f0a9406d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:51.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.373+0000 7f0a99600640 1 -- 172.21.15.167:0/3599184068 shutdown_connections 2024-05-11T12:13:51.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.373+0000 7f0a99600640 1 -- 172.21.15.167:0/3599184068 wait complete. 2024-05-11T12:13:51.394 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:13:51.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.578+0000 7f1a59400640 1 Processor -- start 2024-05-11T12:13:51.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.578+0000 7f1a59400640 1 -- start start 2024-05-11T12:13:51.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.580+0000 7f1a59400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1a5406d460 0x7f1a5406d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:51.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.580+0000 7f1a59400640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a5415df30 con 0x7f1a540b3ea0 2024-05-11T12:13:51.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.580+0000 7f1a59400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a5415e210 con 0x7f1a54157750 2024-05-11T12:13:51.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.580+0000 7f1a59400640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f1a5406ddf0 con 0x7f1a5406d460 2024-05-11T12:13:51.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.580+0000 7f1a53200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1a5406d460 0x7f1a5406d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:51.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.580+0000 7f1a51e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f1a540b3ea0 0x7f1a54161490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:52742/0 (socket says 172.21.15.167:52742) 2024-05-11T12:13:51.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.580+0000 7f1a51e00640 1 -- 172.21.15.167:0/703469408 learned_addr learned my addr 172.21.15.167:0/703469408 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:13:51.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.581+0000 7f1a53200640 1 -- 172.21.15.167:0/703469408 >> v1:172.21.15.204:6789/0 conn(0x7f1a54157750 legacy=0x7f1a5415d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:51.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.581+0000 7f1a53200640 1 -- 172.21.15.167:0/703469408 >> v1:172.21.15.167:6789/0 conn(0x7f1a540b3ea0 legacy=0x7f1a54161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:51.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.581+0000 7f1a53200640 1 -- 172.21.15.167:0/703469408 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a5415e4f0 con 0x7f1a5406d460 2024-05-11T12:13:51.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.582+0000 7f1a53200640 1 --2- 172.21.15.167:0/703469408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a5406d460 0x7f1a5406d8b0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c009d90 tx=0x7f1a4c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f834bfed3d79d3e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:51.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.582+0000 7f1a51400640 1 -- 172.21.15.167:0/703469408 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c007740 con 0x7f1a5406d460 2024-05-11T12:13:51.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.582+0000 7f1a51400640 1 -- 172.21.15.167:0/703469408 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a4c0078e0 con 0x7f1a5406d460 2024-05-11T12:13:51.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.582+0000 7f1a51400640 1 -- 172.21.15.167:0/703469408 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c007cb0 con 0x7f1a5406d460 2024-05-11T12:13:51.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.583+0000 7f1a59400640 1 -- 172.21.15.167:0/703469408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a5406d460 msgr2=0x7f1a5406d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:51.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.583+0000 7f1a59400640 1 --2- 172.21.15.167:0/703469408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a5406d460 0x7f1a5406d8b0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c009d90 tx=0x7f1a4c02a6f0 comp rx=0 tx=0).stop 2024-05-11T12:13:51.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.583+0000 7f1a59400640 1 -- 172.21.15.167:0/703469408 shutdown_connections 2024-05-11T12:13:51.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.583+0000 7f1a59400640 1 --2- 172.21.15.167:0/703469408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a5406d460 0x7f1a5406d8b0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:51.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.583+0000 7f1a59400640 1 -- 172.21.15.167:0/703469408 >> 172.21.15.167:0/703469408 conn(0x7f1a540cd8f0 msgr2=0x7f1a540cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:51.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.583+0000 7f1a59400640 1 -- 172.21.15.167:0/703469408 shutdown_connections 2024-05-11T12:13:51.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.583+0000 7f1a59400640 1 -- 172.21.15.167:0/703469408 wait complete. 2024-05-11T12:13:51.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.585+0000 7f1a59400640 1 Processor -- start 2024-05-11T12:13:51.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.585+0000 7f1a59400640 1 -- start start 2024-05-11T12:13:51.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.586+0000 7f1a59400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a5406d460 0x7f1a54144ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:51.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.586+0000 7f1a59400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a540b3ea0 0x7f1a54141c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:51.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.586+0000 7f1a59400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a54157750 0x7f1a54159390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:51.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.586+0000 7f1a59400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1a54161ba0 con 0x7f1a5406d460 2024-05-11T12:13:51.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.586+0000 7f1a59400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1a54161000 con 0x7f1a540b3ea0 2024-05-11T12:13:51.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.586+0000 7f1a59400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1a540cf1c0 con 0x7f1a54157750 2024-05-11T12:13:51.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.586+0000 7f1a52800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a5406d460 0x7f1a54144ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:51.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.586+0000 7f1a52800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a5406d460 0x7f1a54144ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54218/0 (socket says 172.21.15.167:54218) 2024-05-11T12:13:51.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.586+0000 7f1a52800640 1 -- 172.21.15.167:0/804298644 learned_addr learned my addr 172.21.15.167:0/804298644 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:51.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.587+0000 7f1a53200640 1 --2- 172.21.15.167:0/804298644 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a54157750 0x7f1a54159390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:51.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.587+0000 7f1a52800640 1 -- 172.21.15.167:0/804298644 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a54157750 msgr2=0x7f1a54159390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:51.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.587+0000 7f1a52800640 1 --2- 172.21.15.167:0/804298644 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a54157750 0x7f1a54159390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:51.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.587+0000 7f1a52800640 1 -- 172.21.15.167:0/804298644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a540b3ea0 msgr2=0x7f1a54141c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:51.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.587+0000 7f1a52800640 1 --2- 172.21.15.167:0/804298644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a540b3ea0 0x7f1a54141c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:51.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.587+0000 7f1a52800640 1 -- 172.21.15.167:0/804298644 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a541582f0 con 0x7f1a5406d460 2024-05-11T12:13:51.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.587+0000 7f1a52800640 1 --2- 172.21.15.167:0/804298644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a5406d460 0x7f1a54144ac0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f1a4400c400 tx=0x7f1a4400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:51.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.587+0000 7f1a4aa00640 1 -- 172.21.15.167:0/804298644 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a44017070 con 0x7f1a5406d460 2024-05-11T12:13:51.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.587+0000 7f1a4aa00640 1 -- 172.21.15.167:0/804298644 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a440052d0 con 0x7f1a5406d460 2024-05-11T12:13:51.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.587+0000 7f1a4aa00640 1 -- 172.21.15.167:0/804298644 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a44003830 con 0x7f1a5406d460 2024-05-11T12:13:51.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.587+0000 7f1a59400640 1 -- 172.21.15.167:0/804298644 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a541585d0 con 0x7f1a5406d460 2024-05-11T12:13:51.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.588+0000 7f1a59400640 1 -- 172.21.15.167:0/804298644 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a54158b90 con 0x7f1a5406d460 2024-05-11T12:13:51.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.589+0000 7f1a59400640 1 -- 172.21.15.167:0/804298644 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a54069590 con 0x7f1a5406d460 2024-05-11T12:13:51.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.590+0000 7f1a4aa00640 1 -- 172.21.15.167:0/804298644 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1a4401a070 con 0x7f1a5406d460 2024-05-11T12:13:51.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.591+0000 7f1a4aa00640 1 --2- 172.21.15.167:0/804298644 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f1a34076300 0x7f1a340787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:51.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.591+0000 7f1a4aa00640 1 -- 172.21.15.167:0/804298644 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7f1a44096af0 con 0x7f1a5406d460 2024-05-11T12:13:51.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.591+0000 7f1a51e00640 1 --2- 172.21.15.167:0/804298644 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f1a34076300 0x7f1a340787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:51.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.592+0000 7f1a51e00640 1 --2- 172.21.15.167:0/804298644 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f1a34076300 0x7f1a340787c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1a40005880 tx=0x7f1a40007a20 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:51.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.596+0000 7f1a4aa00640 1 -- 172.21.15.167:0/804298644 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1a440602b0 con 0x7f1a5406d460 2024-05-11T12:13:51.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.805+0000 7f1a59400640 1 -- 172.21.15.167:0/804298644 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1a5415d720 con 0x7f1a5406d460 2024-05-11T12:13:51.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.808+0000 7f1a4aa00640 1 -- 172.21.15.167:0/804298644 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+16993 (secure 0 0 0) 0x7f1a440682a0 con 0x7f1a5406d460 2024-05-11T12:13:51.809 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:13:51.809 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":75,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:13:36.554508+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:13:32.903032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-11T12:13:33.980753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:13:51.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.811+0000 7f1a59400640 1 -- 172.21.15.167:0/804298644 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f1a34076300 msgr2=0x7f1a340787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:51.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.811+0000 7f1a59400640 1 --2- 172.21.15.167:0/804298644 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f1a34076300 0x7f1a340787c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1a40005880 tx=0x7f1a40007a20 comp rx=0 tx=0).stop 2024-05-11T12:13:51.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.811+0000 7f1a59400640 1 -- 172.21.15.167:0/804298644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a5406d460 msgr2=0x7f1a54144ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:51.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.811+0000 7f1a59400640 1 --2- 172.21.15.167:0/804298644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a5406d460 0x7f1a54144ac0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f1a4400c400 tx=0x7f1a4400c8c0 comp rx=0 tx=0).stop 2024-05-11T12:13:51.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.811+0000 7f1a59400640 1 -- 172.21.15.167:0/804298644 shutdown_connections 2024-05-11T12:13:51.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.811+0000 7f1a59400640 1 --2- 172.21.15.167:0/804298644 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a54157750 0x7f1a54159390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:51.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.811+0000 7f1a59400640 1 --2- 172.21.15.167:0/804298644 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f1a34076300 0x7f1a340787c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:51.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.811+0000 7f1a59400640 1 --2- 172.21.15.167:0/804298644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a540b3ea0 0x7f1a54141c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:51.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.811+0000 7f1a59400640 1 --2- 172.21.15.167:0/804298644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a5406d460 0x7f1a54144ac0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:51.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.811+0000 7f1a59400640 1 -- 172.21.15.167:0/804298644 >> 172.21.15.167:0/804298644 conn(0x7f1a540cd8f0 msgr2=0x7f1a540b1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:51.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.812+0000 7f1a59400640 1 -- 172.21.15.167:0/804298644 shutdown_connections 2024-05-11T12:13:51.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:51.812+0000 7f1a59400640 1 -- 172.21.15.167:0/804298644 wait complete. 2024-05-11T12:13:51.831 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-11T12:13:52.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.017+0000 7fdce6600640 1 Processor -- start 2024-05-11T12:13:52.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.018+0000 7fdce6600640 1 -- start start 2024-05-11T12:13:52.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.019+0000 7fdce6600640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fdce013fc30 0x7fdce0160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:52.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.019+0000 7fdce6600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdce0140010 0x7fdce0161020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:52.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.019+0000 7fdce6600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdce00d0220 con 0x7fdce00cf580 2024-05-11T12:13:52.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.019+0000 7fdce6600640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fdce01619d0 con 0x7fdce013fc30 2024-05-11T12:13:52.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.019+0000 7fdce6600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fdce00d0ac0 con 0x7fdce0140010 2024-05-11T12:13:52.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.019+0000 7fdce5200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fdce013fc30 0x7fdce0160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:52.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.020+0000 7fdce5200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fdce013fc30 0x7fdce0160ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54234/0 (socket says 172.21.15.167:54234) 2024-05-11T12:13:52.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.020+0000 7fdce5200640 1 -- 172.21.15.167:0/3663575241 learned_addr learned my addr 172.21.15.167:0/3663575241 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:52.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.020+0000 7fdce5200640 1 -- 172.21.15.167:0/3663575241 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdce0140010 msgr2=0x7fdce0161020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:52.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.020+0000 7fdce5200640 1 --2- 172.21.15.167:0/3663575241 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdce0140010 0x7fdce0161020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:52.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.020+0000 7fdce5200640 1 -- 172.21.15.167:0/3663575241 >> v1:172.21.15.204:6789/0 conn(0x7fdce00cf580 legacy=0x7fdce00cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:52.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.020+0000 7fdce5200640 1 -- 172.21.15.167:0/3663575241 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdce00d0500 con 0x7fdce013fc30 2024-05-11T12:13:52.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.021+0000 7fdce5200640 1 --2- 172.21.15.167:0/3663575241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdce013fc30 0x7fdce0160ae0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4009870 tx=0x7fdcd402a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fd59f3f4442661 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:52.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.021+0000 7fdcdf400640 1 -- 172.21.15.167:0/3663575241 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd4037070 con 0x7fdce013fc30 2024-05-11T12:13:52.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.021+0000 7fdcdf400640 1 -- 172.21.15.167:0/3663575241 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcd402ae00 con 0x7fdce013fc30 2024-05-11T12:13:52.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.021+0000 7fdcdf400640 1 -- 172.21.15.167:0/3663575241 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd4032660 con 0x7fdce013fc30 2024-05-11T12:13:52.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.022+0000 7fdce6600640 1 -- 172.21.15.167:0/3663575241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdce013fc30 msgr2=0x7fdce0160ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:52.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.022+0000 7fdce6600640 1 --2- 172.21.15.167:0/3663575241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdce013fc30 0x7fdce0160ae0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4009870 tx=0x7fdcd402a0b0 comp rx=0 tx=0).stop 2024-05-11T12:13:52.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.022+0000 7fdce6600640 1 -- 172.21.15.167:0/3663575241 shutdown_connections 2024-05-11T12:13:52.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.022+0000 7fdce6600640 1 --2- 172.21.15.167:0/3663575241 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdce0140010 0x7fdce0161020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:52.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.022+0000 7fdce6600640 1 --2- 172.21.15.167:0/3663575241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdce013fc30 0x7fdce0160ae0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:52.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.022+0000 7fdce6600640 1 -- 172.21.15.167:0/3663575241 >> 172.21.15.167:0/3663575241 conn(0x7fdce006c450 msgr2=0x7fdce006c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:52.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.022+0000 7fdce6600640 1 -- 172.21.15.167:0/3663575241 shutdown_connections 2024-05-11T12:13:52.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.022+0000 7fdce6600640 1 -- 172.21.15.167:0/3663575241 wait complete. 2024-05-11T12:13:52.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.024+0000 7fdce6600640 1 Processor -- start 2024-05-11T12:13:52.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.024+0000 7fdce6600640 1 -- start start 2024-05-11T12:13:52.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.025+0000 7fdce6600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdce00cf580 0x7fdce01f4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:52.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.025+0000 7fdce6600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdce013fc30 0x7fdce00b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:52.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.025+0000 7fdce6600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdce0140010 0x7fdce00b62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:52.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.025+0000 7fdce6600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdce006eab0 con 0x7fdce00cf580 2024-05-11T12:13:52.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.025+0000 7fdce6600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdce006e590 con 0x7fdce0140010 2024-05-11T12:13:52.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.025+0000 7fdce6600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fdce006ec30 con 0x7fdce013fc30 2024-05-11T12:13:52.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.025+0000 7fdce5200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdce00cf580 0x7fdce01f4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:52.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.025+0000 7fdce5200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdce00cf580 0x7fdce01f4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54248/0 (socket says 172.21.15.167:54248) 2024-05-11T12:13:52.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.025+0000 7fdce5200640 1 -- 172.21.15.167:0/203443032 learned_addr learned my addr 172.21.15.167:0/203443032 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:52.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.026+0000 7fdce5200640 1 -- 172.21.15.167:0/203443032 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdce013fc30 msgr2=0x7fdce00b5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:52.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.026+0000 7fdce5200640 1 --2- 172.21.15.167:0/203443032 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdce013fc30 0x7fdce00b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:52.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.026+0000 7fdce5200640 1 -- 172.21.15.167:0/203443032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdce0140010 msgr2=0x7fdce00b62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:52.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.026+0000 7fdce5c00640 1 --2- 172.21.15.167:0/203443032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdce0140010 0x7fdce00b62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:52.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.026+0000 7fdce5200640 1 --2- 172.21.15.167:0/203443032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdce0140010 0x7fdce00b62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:52.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.026+0000 7fdce5200640 1 -- 172.21.15.167:0/203443032 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdce01571b0 con 0x7fdce00cf580 2024-05-11T12:13:52.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.026+0000 7fdce5200640 1 --2- 172.21.15.167:0/203443032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdce00cf580 0x7fdce01f4930 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4009870 tx=0x7fdcd40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:52.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.026+0000 7fdcdcc00640 1 -- 172.21.15.167:0/203443032 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd4042070 con 0x7fdce00cf580 2024-05-11T12:13:52.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.026+0000 7fdcdcc00640 1 -- 172.21.15.167:0/203443032 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcd403e670 con 0x7fdce00cf580 2024-05-11T12:13:52.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.026+0000 7fdcdcc00640 1 -- 172.21.15.167:0/203443032 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd4037040 con 0x7fdce00cf580 2024-05-11T12:13:52.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.026+0000 7fdce6600640 1 -- 172.21.15.167:0/203443032 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdce0157490 con 0x7fdce00cf580 2024-05-11T12:13:52.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.027+0000 7fdce6600640 1 -- 172.21.15.167:0/203443032 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdce015c340 con 0x7fdce00cf580 2024-05-11T12:13:52.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.028+0000 7fdce6600640 1 -- 172.21.15.167:0/203443032 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdce00694b0 con 0x7fdce00cf580 2024-05-11T12:13:52.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.029+0000 7fdcdcc00640 1 -- 172.21.15.167:0/203443032 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fdcd403a040 con 0x7fdce00cf580 2024-05-11T12:13:52.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.030+0000 7fdcdcc00640 1 --2- 172.21.15.167:0/203443032 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fdcc00762d0 0x7fdcc0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:52.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.030+0000 7fdcdcc00640 1 -- 172.21.15.167:0/203443032 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7fdcd40b7920 con 0x7fdce00cf580 2024-05-11T12:13:52.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.030+0000 7fdcdfe00640 1 --2- 172.21.15.167:0/203443032 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fdcc00762d0 0x7fdcc0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:52.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.031+0000 7fdcdfe00640 1 --2- 172.21.15.167:0/203443032 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fdcc00762d0 0x7fdcc0078790 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fdce01f8690 tx=0x7fdccc006bf0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:52.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.034+0000 7fdcdcc00640 1 -- 172.21.15.167:0/203443032 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fdcd4080fc0 con 0x7fdce00cf580 2024-05-11T12:13:52.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.278+0000 7fdce6600640 1 -- 172.21.15.167:0/203443032 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fdce00ce040 con 0x7fdce00cf580 2024-05-11T12:13:52.280 INFO:tasks.ceph.mon.a.smithi167.stderr:2024-05-11T12:13:52.279+0000 7f3aac000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-11T12:13:52.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.283+0000 7fdcdcc00640 1 -- 172.21.15.167:0/203443032 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v45) ==== 161+0+0 (secure 0 0 0) 0x7fdcd40808c0 con 0x7fdce00cf580 2024-05-11T12:13:52.284 INFO:teuthology.orchestra.run.smithi167.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-11T12:13:52.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.287+0000 7fdce6600640 1 -- 172.21.15.167:0/203443032 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fdcc00762d0 msgr2=0x7fdcc0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:52.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.287+0000 7fdce6600640 1 --2- 172.21.15.167:0/203443032 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fdcc00762d0 0x7fdcc0078790 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fdce01f8690 tx=0x7fdccc006bf0 comp rx=0 tx=0).stop 2024-05-11T12:13:52.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.287+0000 7fdce6600640 1 -- 172.21.15.167:0/203443032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdce00cf580 msgr2=0x7fdce01f4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:52.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.287+0000 7fdce6600640 1 --2- 172.21.15.167:0/203443032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdce00cf580 0x7fdce01f4930 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4009870 tx=0x7fdcd40046c0 comp rx=0 tx=0).stop 2024-05-11T12:13:52.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.287+0000 7fdce6600640 1 -- 172.21.15.167:0/203443032 shutdown_connections 2024-05-11T12:13:52.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.287+0000 7fdce6600640 1 --2- 172.21.15.167:0/203443032 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fdcc00762d0 0x7fdcc0078790 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:52.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.287+0000 7fdce6600640 1 --2- 172.21.15.167:0/203443032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdce0140010 0x7fdce00b62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:52.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.287+0000 7fdce6600640 1 --2- 172.21.15.167:0/203443032 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdce013fc30 0x7fdce00b5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:52.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.287+0000 7fdce6600640 1 --2- 172.21.15.167:0/203443032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdce00cf580 0x7fdce01f4930 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:52.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.287+0000 7fdce6600640 1 -- 172.21.15.167:0/203443032 >> 172.21.15.167:0/203443032 conn(0x7fdce006c450 msgr2=0x7fdce0137cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:52.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.287+0000 7fdce6600640 1 -- 172.21.15.167:0/203443032 shutdown_connections 2024-05-11T12:13:52.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.287+0000 7fdce6600640 1 -- 172.21.15.167:0/203443032 wait complete. 2024-05-11T12:13:52.306 INFO:tasks.ceph.mds.a.smithi167.stderr: -2301> 2024-05-11T12:13:37.497+0000 7f497e000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-11T12:13:52.307 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-11T12:13:52.372 INFO:tasks.ceph.mds.a.smithi167.stdout:starting mds.a at 2024-05-11T12:13:52.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.487+0000 7fc464c00640 1 Processor -- start 2024-05-11T12:13:52.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.487+0000 7fc464c00640 1 -- start start 2024-05-11T12:13:52.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.488+0000 7fc464c00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc46015ac80 0x7fc460074260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:52.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.488+0000 7fc464c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc4600b3d50 0x7fc460160c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:52.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.488+0000 7fc464c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4600b65e0 con 0x7fc4601611b0 2024-05-11T12:13:52.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.488+0000 7fc464c00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fc46006e3c0 con 0x7fc46015ac80 2024-05-11T12:13:52.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.488+0000 7fc464c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc46015b060 con 0x7fc4600b3d50 2024-05-11T12:13:52.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.489+0000 7fc45de00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc46015ac80 0x7fc460074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:52.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.489+0000 7fc45de00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc46015ac80 0x7fc460074260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54268/0 (socket says 172.21.15.167:54268) 2024-05-11T12:13:52.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.489+0000 7fc45de00640 1 -- 172.21.15.167:0/2058488564 learned_addr learned my addr 172.21.15.167:0/2058488564 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:52.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.489+0000 7fc457e00640 1 --2- 172.21.15.167:0/2058488564 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc4600b3d50 0x7fc460160c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:52.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.489+0000 7fc45de00640 1 -- 172.21.15.167:0/2058488564 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc4600b3d50 msgr2=0x7fc460160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:52.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.489+0000 7fc45de00640 1 --2- 172.21.15.167:0/2058488564 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc4600b3d50 0x7fc460160c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:52.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.489+0000 7fc45de00640 1 -- 172.21.15.167:0/2058488564 >> v1:172.21.15.204:6789/0 conn(0x7fc4601611b0 legacy=0x7fc46015d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:52.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.489+0000 7fc45de00640 1 -- 172.21.15.167:0/2058488564 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4600b68c0 con 0x7fc46015ac80 2024-05-11T12:13:52.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.490+0000 7fc45de00640 1 --2- 172.21.15.167:0/2058488564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc46015ac80 0x7fc460074260 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fc450009870 tx=0x7fc45002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a89bbb0a6dc310a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:52.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.491+0000 7fc45d400640 1 -- 172.21.15.167:0/2058488564 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc450037070 con 0x7fc46015ac80 2024-05-11T12:13:52.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.491+0000 7fc45d400640 1 -- 172.21.15.167:0/2058488564 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc450032440 con 0x7fc46015ac80 2024-05-11T12:13:52.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.491+0000 7fc45d400640 1 -- 172.21.15.167:0/2058488564 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc450032810 con 0x7fc46015ac80 2024-05-11T12:13:52.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.491+0000 7fc464c00640 1 -- 172.21.15.167:0/2058488564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc46015ac80 msgr2=0x7fc460074260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:52.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.491+0000 7fc464c00640 1 --2- 172.21.15.167:0/2058488564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc46015ac80 0x7fc460074260 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fc450009870 tx=0x7fc45002a1c0 comp rx=0 tx=0).stop 2024-05-11T12:13:52.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.491+0000 7fc464c00640 1 -- 172.21.15.167:0/2058488564 shutdown_connections 2024-05-11T12:13:52.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.491+0000 7fc464c00640 1 --2- 172.21.15.167:0/2058488564 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc4600b3d50 0x7fc460160c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:52.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.491+0000 7fc464c00640 1 --2- 172.21.15.167:0/2058488564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc46015ac80 0x7fc460074260 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:52.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.491+0000 7fc464c00640 1 -- 172.21.15.167:0/2058488564 >> 172.21.15.167:0/2058488564 conn(0x7fc46006c450 msgr2=0x7fc46006c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:52.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.492+0000 7fc464c00640 1 -- 172.21.15.167:0/2058488564 shutdown_connections 2024-05-11T12:13:52.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.492+0000 7fc464c00640 1 -- 172.21.15.167:0/2058488564 wait complete. 2024-05-11T12:13:52.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.493+0000 7fc464c00640 1 Processor -- start 2024-05-11T12:13:52.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.494+0000 7fc464c00640 1 -- start start 2024-05-11T12:13:52.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.495+0000 7fc464c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4600b3d50 0x7fc460156120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:52.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.495+0000 7fc464c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc46015ac80 0x7fc4601476b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:52.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.495+0000 7fc464c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4601611b0 0x7fc460157aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:52.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.495+0000 7fc464c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc4600b44f0 con 0x7fc4600b3d50 2024-05-11T12:13:52.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.495+0000 7fc464c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc4600b41b0 con 0x7fc4601611b0 2024-05-11T12:13:52.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.495+0000 7fc464c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc460073960 con 0x7fc46015ac80 2024-05-11T12:13:52.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.495+0000 7fc45de00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4600b3d50 0x7fc460156120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:52.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.495+0000 7fc45de00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4600b3d50 0x7fc460156120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54272/0 (socket says 172.21.15.167:54272) 2024-05-11T12:13:52.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.495+0000 7fc45de00640 1 -- 172.21.15.167:0/2271042920 learned_addr learned my addr 172.21.15.167:0/2271042920 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:52.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.495+0000 7fc45e800640 1 --2- 172.21.15.167:0/2271042920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4601611b0 0x7fc460157aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:52.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.495+0000 7fc45de00640 1 -- 172.21.15.167:0/2271042920 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc46015ac80 msgr2=0x7fc4601476b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:52.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.495+0000 7fc457e00640 1 --2- 172.21.15.167:0/2271042920 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc46015ac80 0x7fc4601476b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:52.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.495+0000 7fc45de00640 1 --2- 172.21.15.167:0/2271042920 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc46015ac80 0x7fc4601476b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:52.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.495+0000 7fc45de00640 1 -- 172.21.15.167:0/2271042920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4601611b0 msgr2=0x7fc460157aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:52.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.495+0000 7fc45de00640 1 --2- 172.21.15.167:0/2271042920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4601611b0 0x7fc460157aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:52.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.495+0000 7fc45de00640 1 -- 172.21.15.167:0/2271042920 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4601539f0 con 0x7fc4600b3d50 2024-05-11T12:13:52.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.496+0000 7fc45de00640 1 --2- 172.21.15.167:0/2271042920 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4600b3d50 0x7fc460156120 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fc450000c00 tx=0x7fc4500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:52.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.496+0000 7fc456000640 1 -- 172.21.15.167:0/2271042920 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc450041070 con 0x7fc4600b3d50 2024-05-11T12:13:52.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.496+0000 7fc456000640 1 -- 172.21.15.167:0/2271042920 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc45003c870 con 0x7fc4600b3d50 2024-05-11T12:13:52.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.496+0000 7fc456000640 1 -- 172.21.15.167:0/2271042920 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc450037040 con 0x7fc4600b3d50 2024-05-11T12:13:52.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.496+0000 7fc464c00640 1 -- 172.21.15.167:0/2271042920 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc460153cd0 con 0x7fc4600b3d50 2024-05-11T12:13:52.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.496+0000 7fc464c00640 1 -- 172.21.15.167:0/2271042920 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc460154290 con 0x7fc4600b3d50 2024-05-11T12:13:52.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.498+0000 7fc464c00640 1 -- 172.21.15.167:0/2271042920 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4600b5630 con 0x7fc4600b3d50 2024-05-11T12:13:52.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.499+0000 7fc456000640 1 -- 172.21.15.167:0/2271042920 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc450044070 con 0x7fc4600b3d50 2024-05-11T12:13:52.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.500+0000 7fc456000640 1 --2- 172.21.15.167:0/2271042920 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc43c0762d0 0x7fc43c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:52.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.500+0000 7fc456000640 1 -- 172.21.15.167:0/2271042920 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7fc4500b83f0 con 0x7fc4600b3d50 2024-05-11T12:13:52.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.500+0000 7fc457e00640 1 --2- 172.21.15.167:0/2271042920 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc43c0762d0 0x7fc43c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:52.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.501+0000 7fc457e00640 1 --2- 172.21.15.167:0/2271042920 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc43c0762d0 0x7fc43c078790 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc448006fd0 tx=0x7fc448008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:52.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.505+0000 7fc456000640 1 -- 172.21.15.167:0/2271042920 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc4600b5630 con 0x7fc4600b3d50 2024-05-11T12:13:52.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:52.748+0000 7fc464c00640 1 -- 172.21.15.167:0/2271042920 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fc460073d10 con 0x7fc4600b3d50 2024-05-11T12:13:53.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.287+0000 7fc456000640 1 -- 172.21.15.167:0/2271042920 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v46) ==== 104+0+0 (secure 0 0 0) 0x7fc460073d10 con 0x7fc4600b3d50 2024-05-11T12:13:53.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.291+0000 7fc464c00640 1 -- 172.21.15.167:0/2271042920 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc43c0762d0 msgr2=0x7fc43c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:53.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.291+0000 7fc464c00640 1 --2- 172.21.15.167:0/2271042920 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc43c0762d0 0x7fc43c078790 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc448006fd0 tx=0x7fc448008040 comp rx=0 tx=0).stop 2024-05-11T12:13:53.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.291+0000 7fc464c00640 1 -- 172.21.15.167:0/2271042920 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4600b3d50 msgr2=0x7fc460156120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:53.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.291+0000 7fc464c00640 1 --2- 172.21.15.167:0/2271042920 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4600b3d50 0x7fc460156120 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fc450000c00 tx=0x7fc4500046c0 comp rx=0 tx=0).stop 2024-05-11T12:13:53.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.291+0000 7fc464c00640 1 -- 172.21.15.167:0/2271042920 shutdown_connections 2024-05-11T12:13:53.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.291+0000 7fc464c00640 1 --2- 172.21.15.167:0/2271042920 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc43c0762d0 0x7fc43c078790 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:53.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.291+0000 7fc464c00640 1 --2- 172.21.15.167:0/2271042920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4601611b0 0x7fc460157aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:53.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.291+0000 7fc464c00640 1 --2- 172.21.15.167:0/2271042920 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc46015ac80 0x7fc4601476b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:53.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.291+0000 7fc464c00640 1 --2- 172.21.15.167:0/2271042920 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4600b3d50 0x7fc460156120 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:53.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.291+0000 7fc464c00640 1 -- 172.21.15.167:0/2271042920 >> 172.21.15.167:0/2271042920 conn(0x7fc46006c450 msgr2=0x7fc46006c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:53.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.291+0000 7fc464c00640 1 -- 172.21.15.167:0/2271042920 shutdown_connections 2024-05-11T12:13:53.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.291+0000 7fc464c00640 1 -- 172.21.15.167:0/2271042920 wait complete. 2024-05-11T12:13:53.312 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-11T12:13:53.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.511+0000 7f121fe00640 1 Processor -- start 2024-05-11T12:13:53.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.512+0000 7f121fe00640 1 -- start start 2024-05-11T12:13:53.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.513+0000 7f121fe00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f122015f260 0x7f12201637e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:53.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.513+0000 7f121fe00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f12200cfd90 0x7f12200d3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:53.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.513+0000 7f121fe00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1220163d20 con 0x7f12200d4310 2024-05-11T12:13:53.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.513+0000 7f121fe00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f12200d3940 con 0x7f12200cfd90 2024-05-11T12:13:53.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.513+0000 7f121fe00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f122015f640 con 0x7f122015f260 2024-05-11T12:13:53.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.513+0000 7f121e000640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f12200cfd90 0x7f12200d3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:53.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.513+0000 7f121f400640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f12200d4310 0x7f122006d060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:52756/0 (socket says 172.21.15.167:52756) 2024-05-11T12:13:53.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.514+0000 7f121f400640 1 -- 172.21.15.167:0/4187833034 learned_addr learned my addr 172.21.15.167:0/4187833034 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:13:53.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.514+0000 7f121ea00640 1 --2- 172.21.15.167:0/4187833034 >> v2:172.21.15.204:3300/0 conn(0x7f122015f260 0x7f12201637e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:53.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.514+0000 7f121e000640 1 -- 172.21.15.167:0/4187833034 >> v2:172.21.15.204:3300/0 conn(0x7f122015f260 msgr2=0x7f12201637e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:53.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.514+0000 7f121e000640 1 --2- 172.21.15.167:0/4187833034 >> v2:172.21.15.204:3300/0 conn(0x7f122015f260 0x7f12201637e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:53.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.514+0000 7f121e000640 1 -- 172.21.15.167:0/4187833034 >> v1:172.21.15.167:6789/0 conn(0x7f12200d4310 legacy=0x7f122006d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:53.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.514+0000 7f121e000640 1 -- 172.21.15.167:0/4187833034 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12200d2d30 con 0x7f12200cfd90 2024-05-11T12:13:53.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.514+0000 7f121ea00640 1 --2- 172.21.15.167:0/4187833034 >> v2:172.21.15.204:3300/0 conn(0x7f122015f260 0x7f12201637e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:53.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.514+0000 7f121d600640 1 -- 172.21.15.167:0/4187833034 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3929945907 0 0) 0x7f1218001110 con 0x7f12200d4310 2024-05-11T12:13:53.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.515+0000 7f121e000640 1 --2- 172.21.15.167:0/4187833034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12200cfd90 0x7f12200d3dd0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f122008d4e0 tx=0x7f120802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e3c16169ce9a11 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:53.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.515+0000 7f121d600640 1 -- 172.21.15.167:0/4187833034 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1208037070 con 0x7f12200cfd90 2024-05-11T12:13:53.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.515+0000 7f121d600640 1 -- 172.21.15.167:0/4187833034 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1208032440 con 0x7f12200cfd90 2024-05-11T12:13:53.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.515+0000 7f121d600640 1 -- 172.21.15.167:0/4187833034 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1208032810 con 0x7f12200cfd90 2024-05-11T12:13:53.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.516+0000 7f121fe00640 1 -- 172.21.15.167:0/4187833034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12200cfd90 msgr2=0x7f12200d3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:53.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.516+0000 7f121fe00640 1 --2- 172.21.15.167:0/4187833034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12200cfd90 0x7f12200d3dd0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f122008d4e0 tx=0x7f120802a1a0 comp rx=0 tx=0).stop 2024-05-11T12:13:53.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.516+0000 7f121fe00640 1 -- 172.21.15.167:0/4187833034 shutdown_connections 2024-05-11T12:13:53.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.516+0000 7f121fe00640 1 --2- 172.21.15.167:0/4187833034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12200cfd90 0x7f12200d3dd0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:53.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.516+0000 7f121fe00640 1 --2- 172.21.15.167:0/4187833034 >> v2:172.21.15.204:3300/0 conn(0x7f122015f260 0x7f12201637e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:53.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.516+0000 7f121fe00640 1 -- 172.21.15.167:0/4187833034 >> 172.21.15.167:0/4187833034 conn(0x7f12200b6cd0 msgr2=0x7f12200b70e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:53.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.516+0000 7f121fe00640 1 -- 172.21.15.167:0/4187833034 shutdown_connections 2024-05-11T12:13:53.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.516+0000 7f121fe00640 1 -- 172.21.15.167:0/4187833034 wait complete. 2024-05-11T12:13:53.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.518+0000 7f121fe00640 1 Processor -- start 2024-05-11T12:13:53.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.518+0000 7f121fe00640 1 -- start start 2024-05-11T12:13:53.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.519+0000 7f121fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12200cfd90 0x7f12200ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:53.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.519+0000 7f121fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12200d4310 0x7f12200cb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:53.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.519+0000 7f121fe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f122015f260 0x7f12200d19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:53.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.519+0000 7f121fe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f12200d35d0 con 0x7f12200d4310 2024-05-11T12:13:53.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.519+0000 7f121fe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f12200d32f0 con 0x7f12200cfd90 2024-05-11T12:13:53.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.519+0000 7f121fe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f122006d770 con 0x7f122015f260 2024-05-11T12:13:53.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.519+0000 7f121e000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12200d4310 0x7f12200cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:53.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.519+0000 7f121e000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12200d4310 0x7f12200cb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54292/0 (socket says 172.21.15.167:54292) 2024-05-11T12:13:53.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.519+0000 7f121e000640 1 -- 172.21.15.167:0/3576282441 learned_addr learned my addr 172.21.15.167:0/3576282441 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:53.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.520+0000 7f121e000640 1 -- 172.21.15.167:0/3576282441 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f122015f260 msgr2=0x7f12200d19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:53.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.520+0000 7f121e000640 1 --2- 172.21.15.167:0/3576282441 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f122015f260 0x7f12200d19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:53.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.520+0000 7f121e000640 1 -- 172.21.15.167:0/3576282441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12200cfd90 msgr2=0x7f12200ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:53.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.520+0000 7f121e000640 1 --2- 172.21.15.167:0/3576282441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12200cfd90 0x7f12200ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:53.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.520+0000 7f121e000640 1 -- 172.21.15.167:0/3576282441 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1220148c70 con 0x7f12200d4310 2024-05-11T12:13:53.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.520+0000 7f121e000640 1 --2- 172.21.15.167:0/3576282441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12200d4310 0x7f12200cb040 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f1208000c00 tx=0x7f12080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:53.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.520+0000 7f1217400640 1 -- 172.21.15.167:0/3576282441 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1208037050 con 0x7f12200d4310 2024-05-11T12:13:53.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.520+0000 7f1217400640 1 -- 172.21.15.167:0/3576282441 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1208038040 con 0x7f12200d4310 2024-05-11T12:13:53.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.520+0000 7f121fe00640 1 -- 172.21.15.167:0/3576282441 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1220148f50 con 0x7f12200d4310 2024-05-11T12:13:53.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.520+0000 7f1217400640 1 -- 172.21.15.167:0/3576282441 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1208004470 con 0x7f12200d4310 2024-05-11T12:13:53.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.520+0000 7f121fe00640 1 -- 172.21.15.167:0/3576282441 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1220149510 con 0x7f12200d4310 2024-05-11T12:13:53.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.521+0000 7f121fe00640 1 -- 172.21.15.167:0/3576282441 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12200694b0 con 0x7f12200d4310 2024-05-11T12:13:53.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.522+0000 7f1217400640 1 -- 172.21.15.167:0/3576282441 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1208030980 con 0x7f12200d4310 2024-05-11T12:13:53.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.523+0000 7f1217400640 1 --2- 172.21.15.167:0/3576282441 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f11fc0762d0 0x7f11fc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:53.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.523+0000 7f1217400640 1 -- 172.21.15.167:0/3576282441 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7f12080b8480 con 0x7f12200d4310 2024-05-11T12:13:53.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.524+0000 7f121ea00640 1 --2- 172.21.15.167:0/3576282441 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f11fc0762d0 0x7f11fc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:53.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.524+0000 7f121ea00640 1 --2- 172.21.15.167:0/3576282441 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f11fc0762d0 0x7f11fc078790 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1210008e00 tx=0x7f1210008330 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:53.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.528+0000 7f1217400640 1 -- 172.21.15.167:0/3576282441 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1208081b60 con 0x7f12200d4310 2024-05-11T12:13:53.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:53.722+0000 7f121fe00640 1 -- 172.21.15.167:0/3576282441 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f12200d3dd0 con 0x7f12200d4310 2024-05-11T12:13:54.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.296+0000 7f1217400640 1 -- 172.21.15.167:0/3576282441 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v77) ==== 181+0+0 (secure 0 0 0) 0x7f1208081460 con 0x7f12200d4310 2024-05-11T12:13:54.297 INFO:teuthology.orchestra.run.smithi167.stderr:pool 'cephfs_metadata' removed 2024-05-11T12:13:54.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.300+0000 7f121fe00640 1 -- 172.21.15.167:0/3576282441 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f11fc0762d0 msgr2=0x7f11fc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:54.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.300+0000 7f121fe00640 1 --2- 172.21.15.167:0/3576282441 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f11fc0762d0 0x7f11fc078790 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1210008e00 tx=0x7f1210008330 comp rx=0 tx=0).stop 2024-05-11T12:13:54.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.300+0000 7f121fe00640 1 -- 172.21.15.167:0/3576282441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12200d4310 msgr2=0x7f12200cb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:54.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.300+0000 7f121fe00640 1 --2- 172.21.15.167:0/3576282441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12200d4310 0x7f12200cb040 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f1208000c00 tx=0x7f12080046c0 comp rx=0 tx=0).stop 2024-05-11T12:13:54.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.301+0000 7f121fe00640 1 -- 172.21.15.167:0/3576282441 shutdown_connections 2024-05-11T12:13:54.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.301+0000 7f121fe00640 1 --2- 172.21.15.167:0/3576282441 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f122015f260 0x7f12200d19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:54.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.301+0000 7f121fe00640 1 --2- 172.21.15.167:0/3576282441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12200d4310 0x7f12200cb040 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:54.302 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.301+0000 7f121fe00640 1 --2- 172.21.15.167:0/3576282441 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f11fc0762d0 0x7f11fc078790 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:54.302 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.301+0000 7f121fe00640 1 --2- 172.21.15.167:0/3576282441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12200cfd90 0x7f12200ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:54.302 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.301+0000 7f121fe00640 1 -- 172.21.15.167:0/3576282441 >> 172.21.15.167:0/3576282441 conn(0x7f12200b6cd0 msgr2=0x7f12200ba970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:54.302 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.302+0000 7f121fe00640 1 -- 172.21.15.167:0/3576282441 shutdown_connections 2024-05-11T12:13:54.302 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.302+0000 7f121fe00640 1 -- 172.21.15.167:0/3576282441 wait complete. 2024-05-11T12:13:54.326 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-11T12:13:54.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.534+0000 7fc09c200640 1 Processor -- start 2024-05-11T12:13:54.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.534+0000 7fc09c200640 1 -- start start 2024-05-11T12:13:54.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.535+0000 7fc09c200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc0941439a0 0x7fc0940cfee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:54.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.536+0000 7fc09c200640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0940d0420 con 0x7fc0940cf820 2024-05-11T12:13:54.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.536+0000 7fc09c200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0940b3f30 con 0x7fc0940b59f0 2024-05-11T12:13:54.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.536+0000 7fc09c200640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fc0940b3ce0 con 0x7fc0941439a0 2024-05-11T12:13:54.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.536+0000 7fc098e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fc0940cf820 0x7fc0940b52e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:52766/0 (socket says 172.21.15.167:52766) 2024-05-11T12:13:54.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.536+0000 7fc098e00640 1 -- 172.21.15.167:0/1226166470 learned_addr learned my addr 172.21.15.167:0/1226166470 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:13:54.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.536+0000 7fc099800640 1 --2- 172.21.15.167:0/1226166470 >> v2:172.21.15.167:3300/0 conn(0x7fc0941439a0 0x7fc0940cfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:54.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.536+0000 7fc099800640 1 -- 172.21.15.167:0/1226166470 >> v1:172.21.15.204:6789/0 conn(0x7fc0940b59f0 legacy=0x7fc094073220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:54.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.536+0000 7fc099800640 1 -- 172.21.15.167:0/1226166470 >> v1:172.21.15.167:6789/0 conn(0x7fc0940cf820 legacy=0x7fc0940b52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:54.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.536+0000 7fc093e00640 1 -- 172.21.15.167:0/1226166470 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 907918434 0 0) 0x7fc0940d0420 con 0x7fc0940cf820 2024-05-11T12:13:54.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.536+0000 7fc099800640 1 -- 172.21.15.167:0/1226166470 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0940b4210 con 0x7fc0941439a0 2024-05-11T12:13:54.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.538+0000 7fc099800640 1 --2- 172.21.15.167:0/1226166470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0941439a0 0x7fc0940cfee0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fc0880097a0 tx=0x7fc08802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10d555c20890353 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:54.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.538+0000 7fc093e00640 1 -- 172.21.15.167:0/1226166470 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc088037070 con 0x7fc0941439a0 2024-05-11T12:13:54.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.538+0000 7fc093e00640 1 -- 172.21.15.167:0/1226166470 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc088032440 con 0x7fc0941439a0 2024-05-11T12:13:54.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.538+0000 7fc093e00640 1 -- 172.21.15.167:0/1226166470 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc088032810 con 0x7fc0941439a0 2024-05-11T12:13:54.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.538+0000 7fc09c200640 1 -- 172.21.15.167:0/1226166470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0941439a0 msgr2=0x7fc0940cfee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:54.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.538+0000 7fc09c200640 1 --2- 172.21.15.167:0/1226166470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0941439a0 0x7fc0940cfee0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fc0880097a0 tx=0x7fc08802a180 comp rx=0 tx=0).stop 2024-05-11T12:13:54.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.539+0000 7fc09c200640 1 -- 172.21.15.167:0/1226166470 shutdown_connections 2024-05-11T12:13:54.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.539+0000 7fc09c200640 1 --2- 172.21.15.167:0/1226166470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0941439a0 0x7fc0940cfee0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:54.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.539+0000 7fc09c200640 1 -- 172.21.15.167:0/1226166470 >> 172.21.15.167:0/1226166470 conn(0x7fc0940d0c70 msgr2=0x7fc0940d1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:54.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.539+0000 7fc09c200640 1 -- 172.21.15.167:0/1226166470 shutdown_connections 2024-05-11T12:13:54.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.539+0000 7fc09c200640 1 -- 172.21.15.167:0/1226166470 wait complete. 2024-05-11T12:13:54.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.540+0000 7fc09c200640 1 Processor -- start 2024-05-11T12:13:54.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.541+0000 7fc09c200640 1 -- start start 2024-05-11T12:13:54.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.541+0000 7fc09c200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0940b59f0 0x7fc094139790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:54.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.541+0000 7fc09c200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0940cf820 0x7fc09413d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:54.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.541+0000 7fc09c200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0941439a0 0x7fc0940b1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:54.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.542+0000 7fc09c200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc0940b4ab0 con 0x7fc0940b59f0 2024-05-11T12:13:54.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.542+0000 7fc09c200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc0940b47d0 con 0x7fc0941439a0 2024-05-11T12:13:54.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.542+0000 7fc09c200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc094073930 con 0x7fc0940cf820 2024-05-11T12:13:54.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.542+0000 7fc099800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0940b59f0 0x7fc094139790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:54.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.542+0000 7fc099800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0940b59f0 0x7fc094139790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54314/0 (socket says 172.21.15.167:54314) 2024-05-11T12:13:54.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.542+0000 7fc099800640 1 -- 172.21.15.167:0/633543577 learned_addr learned my addr 172.21.15.167:0/633543577 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:54.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.542+0000 7fc099800640 1 -- 172.21.15.167:0/633543577 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0940cf820 msgr2=0x7fc09413d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:54.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.542+0000 7fc099800640 1 --2- 172.21.15.167:0/633543577 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0940cf820 0x7fc09413d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:54.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.542+0000 7fc099800640 1 -- 172.21.15.167:0/633543577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0941439a0 msgr2=0x7fc0940b1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:54.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.542+0000 7fc099800640 1 --2- 172.21.15.167:0/633543577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0941439a0 0x7fc0940b1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:54.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.542+0000 7fc099800640 1 -- 172.21.15.167:0/633543577 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc09413ddf0 con 0x7fc0940b59f0 2024-05-11T12:13:54.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.542+0000 7fc099800640 1 --2- 172.21.15.167:0/633543577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0940b59f0 0x7fc094139790 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fc0880097a0 tx=0x7fc0880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:54.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.543+0000 7fc091600640 1 -- 172.21.15.167:0/633543577 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc088041070 con 0x7fc0940b59f0 2024-05-11T12:13:54.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.543+0000 7fc091600640 1 -- 172.21.15.167:0/633543577 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc08803c870 con 0x7fc0940b59f0 2024-05-11T12:13:54.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.543+0000 7fc09c200640 1 -- 172.21.15.167:0/633543577 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc09413e0d0 con 0x7fc0940b59f0 2024-05-11T12:13:54.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.543+0000 7fc091600640 1 -- 172.21.15.167:0/633543577 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc088037040 con 0x7fc0940b59f0 2024-05-11T12:13:54.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.543+0000 7fc09c200640 1 -- 172.21.15.167:0/633543577 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc09413e690 con 0x7fc0940b59f0 2024-05-11T12:13:54.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.544+0000 7fc09c200640 1 -- 172.21.15.167:0/633543577 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc094069590 con 0x7fc0940b59f0 2024-05-11T12:13:54.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.545+0000 7fc091600640 1 -- 172.21.15.167:0/633543577 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc088044070 con 0x7fc0940b59f0 2024-05-11T12:13:54.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.547+0000 7fc091600640 1 --2- 172.21.15.167:0/633543577 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc0740762d0 0x7fc074078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:54.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.547+0000 7fc091600640 1 -- 172.21.15.167:0/633543577 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5749+0+0 (secure 0 0 0) 0x7fc0880b8270 con 0x7fc0940b59f0 2024-05-11T12:13:54.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.547+0000 7fc098e00640 1 --2- 172.21.15.167:0/633543577 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc0740762d0 0x7fc074078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:54.548 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.548+0000 7fc098e00640 1 --2- 172.21.15.167:0/633543577 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc0740762d0 0x7fc074078790 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc09415f3d0 tx=0x7fc0800078d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:54.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.551+0000 7fc091600640 1 -- 172.21.15.167:0/633543577 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc088081b00 con 0x7fc0940b59f0 2024-05-11T12:13:54.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:54.763+0000 7fc09c200640 1 -- 172.21.15.167:0/633543577 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc0941602f0 con 0x7fc0940b59f0 2024-05-11T12:13:55.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.300+0000 7fc091600640 1 -- 172.21.15.167:0/633543577 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v78) ==== 169+0+0 (secure 0 0 0) 0x7fc088081400 con 0x7fc0940b59f0 2024-05-11T12:13:55.300 INFO:teuthology.orchestra.run.smithi167.stderr:pool 'cephfs_data' removed 2024-05-11T12:13:55.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.304+0000 7fc09c200640 1 -- 172.21.15.167:0/633543577 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc0740762d0 msgr2=0x7fc074078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:55.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.304+0000 7fc09c200640 1 --2- 172.21.15.167:0/633543577 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc0740762d0 0x7fc074078790 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc09415f3d0 tx=0x7fc0800078d0 comp rx=0 tx=0).stop 2024-05-11T12:13:55.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.304+0000 7fc09c200640 1 -- 172.21.15.167:0/633543577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0940b59f0 msgr2=0x7fc094139790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:55.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.304+0000 7fc09c200640 1 --2- 172.21.15.167:0/633543577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0940b59f0 0x7fc094139790 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fc0880097a0 tx=0x7fc0880046c0 comp rx=0 tx=0).stop 2024-05-11T12:13:55.306 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.305+0000 7fc09c200640 1 -- 172.21.15.167:0/633543577 shutdown_connections 2024-05-11T12:13:55.306 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.305+0000 7fc09c200640 1 --2- 172.21.15.167:0/633543577 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc0740762d0 0x7fc074078790 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:55.306 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.305+0000 7fc09c200640 1 --2- 172.21.15.167:0/633543577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0941439a0 0x7fc0940b1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:55.306 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.305+0000 7fc09c200640 1 --2- 172.21.15.167:0/633543577 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0940cf820 0x7fc09413d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:55.306 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.305+0000 7fc09c200640 1 --2- 172.21.15.167:0/633543577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0940b59f0 0x7fc094139790 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:55.306 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.305+0000 7fc09c200640 1 -- 172.21.15.167:0/633543577 >> 172.21.15.167:0/633543577 conn(0x7fc0940d0c70 msgr2=0x7fc094073d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:55.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.306+0000 7fc09c200640 1 -- 172.21.15.167:0/633543577 shutdown_connections 2024-05-11T12:13:55.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.306+0000 7fc09c200640 1 -- 172.21.15.167:0/633543577 wait complete. 2024-05-11T12:13:55.338 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds ms_async_reap_threshold 2024-05-11T12:13:55.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.518+0000 7efbfd600640 1 Processor -- start 2024-05-11T12:13:55.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.518+0000 7efbfd600640 1 -- start start 2024-05-11T12:13:55.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.520+0000 7efbfd600640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7efbf8073170 0x7efbf80b6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:55.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.520+0000 7efbfd600640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efbf8156140 con 0x7efbf8070050 2024-05-11T12:13:55.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.520+0000 7efbfd600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efbf815bc60 con 0x7efbf806f420 2024-05-11T12:13:55.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.520+0000 7efbfd600640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7efbf80b6e80 con 0x7efbf8073170 2024-05-11T12:13:55.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.520+0000 7efbf7400640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7efbf8070050 0x7efbf80ceaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:52774/0 (socket says 172.21.15.167:52774) 2024-05-11T12:13:55.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.520+0000 7efbf7400640 1 -- 172.21.15.167:0/1790207339 learned_addr learned my addr 172.21.15.167:0/1790207339 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:13:55.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.520+0000 7efbf6000640 1 --2- 172.21.15.167:0/1790207339 >> v2:172.21.15.167:3300/0 conn(0x7efbf8073170 0x7efbf80b6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:55.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.521+0000 7efbf5600640 1 -- 172.21.15.167:0/1790207339 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 973012487 0 0) 0x7efbf0001490 con 0x7efbf8070050 2024-05-11T12:13:55.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.521+0000 7efbf5600640 1 -- 172.21.15.167:0/1790207339 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efbdc003520 con 0x7efbf8070050 2024-05-11T12:13:55.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.521+0000 7efbf6000640 1 -- 172.21.15.167:0/1790207339 >> v1:172.21.15.204:6789/0 conn(0x7efbf806f420 legacy=0x7efbf8072a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:55.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.521+0000 7efbf6000640 1 -- 172.21.15.167:0/1790207339 >> v1:172.21.15.167:6789/0 conn(0x7efbf8070050 legacy=0x7efbf80ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:55.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.521+0000 7efbf6000640 1 -- 172.21.15.167:0/1790207339 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efbf815bf20 con 0x7efbf8073170 2024-05-11T12:13:55.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.522+0000 7efbf6000640 1 --2- 172.21.15.167:0/1790207339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efbf8073170 0x7efbf80b6940 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7efbe40097a0 tx=0x7efbe402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceff72eb8da91f64 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:55.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.522+0000 7efbf5600640 1 -- 172.21.15.167:0/1790207339 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbe4037070 con 0x7efbf8073170 2024-05-11T12:13:55.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.522+0000 7efbf5600640 1 -- 172.21.15.167:0/1790207339 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efbe4032440 con 0x7efbf8073170 2024-05-11T12:13:55.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.522+0000 7efbf5600640 1 -- 172.21.15.167:0/1790207339 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbe4032810 con 0x7efbf8073170 2024-05-11T12:13:55.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.523+0000 7efbfd600640 1 -- 172.21.15.167:0/1790207339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efbf8073170 msgr2=0x7efbf80b6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:55.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.523+0000 7efbfd600640 1 --2- 172.21.15.167:0/1790207339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efbf8073170 0x7efbf80b6940 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7efbe40097a0 tx=0x7efbe402a180 comp rx=0 tx=0).stop 2024-05-11T12:13:55.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.523+0000 7efbfd600640 1 -- 172.21.15.167:0/1790207339 shutdown_connections 2024-05-11T12:13:55.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.523+0000 7efbfd600640 1 --2- 172.21.15.167:0/1790207339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efbf8073170 0x7efbf80b6940 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:55.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.523+0000 7efbfd600640 1 -- 172.21.15.167:0/1790207339 >> 172.21.15.167:0/1790207339 conn(0x7efbf806c450 msgr2=0x7efbf806c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:55.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.523+0000 7efbfd600640 1 -- 172.21.15.167:0/1790207339 shutdown_connections 2024-05-11T12:13:55.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.523+0000 7efbfd600640 1 -- 172.21.15.167:0/1790207339 wait complete. 2024-05-11T12:13:55.525 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.525+0000 7efbfd600640 1 Processor -- start 2024-05-11T12:13:55.525 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.525+0000 7efbfd600640 1 -- start start 2024-05-11T12:13:55.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.526+0000 7efbfd600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efbf806f420 0x7efbf81544c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:55.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.526+0000 7efbfd600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efbf8070050 0x7efbf80cb120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:55.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.526+0000 7efbfd600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efbf8073170 0x7efbf80cb660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:55.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.526+0000 7efbfd600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efbf815c7c0 con 0x7efbf806f420 2024-05-11T12:13:55.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.526+0000 7efbfd600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7efbf815c4e0 con 0x7efbf8073170 2024-05-11T12:13:55.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.526+0000 7efbfd600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7efbf806e3c0 con 0x7efbf8070050 2024-05-11T12:13:55.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.526+0000 7efbf6a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efbf806f420 0x7efbf81544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:55.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.526+0000 7efbf6a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efbf806f420 0x7efbf81544c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54330/0 (socket says 172.21.15.167:54330) 2024-05-11T12:13:55.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.526+0000 7efbf6a00640 1 -- 172.21.15.167:0/1390229679 learned_addr learned my addr 172.21.15.167:0/1390229679 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:55.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.526+0000 7efbf6a00640 1 -- 172.21.15.167:0/1390229679 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efbf8070050 msgr2=0x7efbf80cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:55.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.526+0000 7efbf6a00640 1 --2- 172.21.15.167:0/1390229679 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efbf8070050 0x7efbf80cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:55.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.527+0000 7efbf6a00640 1 -- 172.21.15.167:0/1390229679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efbf8073170 msgr2=0x7efbf80cb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:55.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.527+0000 7efbf6a00640 1 --2- 172.21.15.167:0/1390229679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efbf8073170 0x7efbf80cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:55.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.527+0000 7efbf6a00640 1 -- 172.21.15.167:0/1390229679 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efbf8157e50 con 0x7efbf806f420 2024-05-11T12:13:55.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.527+0000 7efbf6a00640 1 --2- 172.21.15.167:0/1390229679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efbf806f420 0x7efbf81544c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7efbe800c430 tx=0x7efbe800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:55.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.527+0000 7efbeea00640 1 -- 172.21.15.167:0/1390229679 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbe8011790 con 0x7efbf806f420 2024-05-11T12:13:55.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.527+0000 7efbeea00640 1 -- 172.21.15.167:0/1390229679 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efbe8017070 con 0x7efbf806f420 2024-05-11T12:13:55.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.527+0000 7efbfd600640 1 -- 172.21.15.167:0/1390229679 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efbf8158130 con 0x7efbf806f420 2024-05-11T12:13:55.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.527+0000 7efbeea00640 1 -- 172.21.15.167:0/1390229679 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbe8011bb0 con 0x7efbf806f420 2024-05-11T12:13:55.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.527+0000 7efbfd600640 1 -- 172.21.15.167:0/1390229679 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efbf81586f0 con 0x7efbf806f420 2024-05-11T12:13:55.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.529+0000 7efbfd600640 1 -- 172.21.15.167:0/1390229679 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efbf8069520 con 0x7efbf806f420 2024-05-11T12:13:55.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.530+0000 7efbeea00640 1 -- 172.21.15.167:0/1390229679 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7efbe801c460 con 0x7efbf806f420 2024-05-11T12:13:55.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.531+0000 7efbeea00640 1 --2- 172.21.15.167:0/1390229679 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7efbd80762d0 0x7efbd8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:55.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.531+0000 7efbeea00640 1 -- 172.21.15.167:0/1390229679 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5335+0+0 (secure 0 0 0) 0x7efbe8097350 con 0x7efbf806f420 2024-05-11T12:13:55.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.531+0000 7efbf6000640 1 --2- 172.21.15.167:0/1390229679 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7efbd80762d0 0x7efbd8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:55.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.532+0000 7efbf6000640 1 --2- 172.21.15.167:0/1390229679 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7efbd80762d0 0x7efbd8078790 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7efbe4034000 tx=0x7efbe402c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:55.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.535+0000 7efbeea00640 1 -- 172.21.15.167:0/1390229679 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7efbe8060e20 con 0x7efbf806f420 2024-05-11T12:13:55.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.735+0000 7efbfd600640 1 -- 172.21.15.167:0/1390229679 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "ms_async_reap_threshold"} v 0) -- 0x7efbf80b62d0 con 0x7efbf806f420 2024-05-11T12:13:55.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.740+0000 7efbeea00640 1 -- 172.21.15.167:0/1390229679 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "ms_async_reap_threshold"}]=0 v20) ==== 106+0+0 (secure 0 0 0) 0x7efbe8060720 con 0x7efbf806f420 2024-05-11T12:13:55.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.743+0000 7efbfd600640 1 -- 172.21.15.167:0/1390229679 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7efbd80762d0 msgr2=0x7efbd8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:55.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.743+0000 7efbfd600640 1 --2- 172.21.15.167:0/1390229679 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7efbd80762d0 0x7efbd8078790 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7efbe4034000 tx=0x7efbe402c000 comp rx=0 tx=0).stop 2024-05-11T12:13:55.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.743+0000 7efbfd600640 1 -- 172.21.15.167:0/1390229679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efbf806f420 msgr2=0x7efbf81544c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:55.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.743+0000 7efbfd600640 1 --2- 172.21.15.167:0/1390229679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efbf806f420 0x7efbf81544c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7efbe800c430 tx=0x7efbe800c8f0 comp rx=0 tx=0).stop 2024-05-11T12:13:55.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.744+0000 7efbfd600640 1 -- 172.21.15.167:0/1390229679 shutdown_connections 2024-05-11T12:13:55.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.744+0000 7efbfd600640 1 --2- 172.21.15.167:0/1390229679 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7efbd80762d0 0x7efbd8078790 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:55.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.744+0000 7efbfd600640 1 --2- 172.21.15.167:0/1390229679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efbf8073170 0x7efbf80cb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:55.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.744+0000 7efbfd600640 1 --2- 172.21.15.167:0/1390229679 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efbf8070050 0x7efbf80cb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:55.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.744+0000 7efbfd600640 1 --2- 172.21.15.167:0/1390229679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efbf806f420 0x7efbf81544c0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:55.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.744+0000 7efbfd600640 1 -- 172.21.15.167:0/1390229679 >> 172.21.15.167:0/1390229679 conn(0x7efbf806c450 msgr2=0x7efbf8159880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:55.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.744+0000 7efbfd600640 1 -- 172.21.15.167:0/1390229679 shutdown_connections 2024-05-11T12:13:55.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.744+0000 7efbfd600640 1 -- 172.21.15.167:0/1390229679 wait complete. 2024-05-11T12:13:55.761 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-11T12:13:55.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.957+0000 7fa7ede00640 1 Processor -- start 2024-05-11T12:13:55.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.957+0000 7fa7ede00640 1 -- start start 2024-05-11T12:13:55.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.959+0000 7fa7ede00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7e80ae820 0x7fa7e80afe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:55.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.959+0000 7fa7ede00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa7e80ceb00 0x7fa7e80cef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:55.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.959+0000 7fa7ede00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7e80aed90 con 0x7fa7e8145a00 2024-05-11T12:13:55.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.959+0000 7fa7ede00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fa7e80af9a0 con 0x7fa7e80ceb00 2024-05-11T12:13:55.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.959+0000 7fa7ede00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa7e80af070 con 0x7fa7e80ae820 2024-05-11T12:13:55.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.959+0000 7fa7e6000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7e80ae820 0x7fa7e80afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:55.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.959+0000 7fa7e7400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa7e80ceb00 0x7fa7e80cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:55.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.959+0000 7fa7e7400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa7e80ceb00 0x7fa7e80cef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.167:41884/0 (socket says 172.21.15.167:41884) 2024-05-11T12:13:55.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.959+0000 7fa7e6000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7e80ae820 0x7fa7e80afe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:46126/0 (socket says 172.21.15.167:46126) 2024-05-11T12:13:55.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.959+0000 7fa7e7400640 1 -- 172.21.15.167:0/3606208793 learned_addr learned my addr 172.21.15.167:0/3606208793 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:55.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.960+0000 7fa7e6000640 1 -- 172.21.15.167:0/3606208793 >> v2:172.21.15.204:3300/0 conn(0x7fa7e80ceb00 msgr2=0x7fa7e80cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:55.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.960+0000 7fa7e6000640 1 --2- 172.21.15.167:0/3606208793 >> v2:172.21.15.204:3300/0 conn(0x7fa7e80ceb00 0x7fa7e80cef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:55.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.960+0000 7fa7e6000640 1 -- 172.21.15.167:0/3606208793 >> v1:172.21.15.204:6789/0 conn(0x7fa7e8145a00 legacy=0x7fa7e806f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:55.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.960+0000 7fa7e6000640 1 -- 172.21.15.167:0/3606208793 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7e80af350 con 0x7fa7e80ae820 2024-05-11T12:13:55.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.962+0000 7fa7e6000640 1 --2- 172.21.15.167:0/3606208793 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7e80ae820 0x7fa7e80afe30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa7d40097a0 tx=0x7fa7d402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e97e23550aa8e10 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:55.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.962+0000 7fa7e5600640 1 -- 172.21.15.167:0/3606208793 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d4037070 con 0x7fa7e80ae820 2024-05-11T12:13:55.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.962+0000 7fa7e5600640 1 -- 172.21.15.167:0/3606208793 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7d402ae50 con 0x7fa7e80ae820 2024-05-11T12:13:55.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.962+0000 7fa7e5600640 1 -- 172.21.15.167:0/3606208793 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d4032660 con 0x7fa7e80ae820 2024-05-11T12:13:55.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.963+0000 7fa7ede00640 1 -- 172.21.15.167:0/3606208793 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7e80ae820 msgr2=0x7fa7e80afe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:55.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.963+0000 7fa7ede00640 1 --2- 172.21.15.167:0/3606208793 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7e80ae820 0x7fa7e80afe30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa7d40097a0 tx=0x7fa7d402a100 comp rx=0 tx=0).stop 2024-05-11T12:13:55.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.963+0000 7fa7ede00640 1 -- 172.21.15.167:0/3606208793 shutdown_connections 2024-05-11T12:13:55.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.963+0000 7fa7ede00640 1 --2- 172.21.15.167:0/3606208793 >> v2:172.21.15.204:3300/0 conn(0x7fa7e80ceb00 0x7fa7e80cef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:55.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.963+0000 7fa7ede00640 1 --2- 172.21.15.167:0/3606208793 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7e80ae820 0x7fa7e80afe30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:55.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.963+0000 7fa7ede00640 1 -- 172.21.15.167:0/3606208793 >> 172.21.15.167:0/3606208793 conn(0x7fa7e806c450 msgr2=0x7fa7e806c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:55.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.963+0000 7fa7ede00640 1 -- 172.21.15.167:0/3606208793 shutdown_connections 2024-05-11T12:13:55.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.963+0000 7fa7ede00640 1 -- 172.21.15.167:0/3606208793 wait complete. 2024-05-11T12:13:55.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.965+0000 7fa7ede00640 1 Processor -- start 2024-05-11T12:13:55.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.965+0000 7fa7ede00640 1 -- start start 2024-05-11T12:13:55.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.966+0000 7fa7ede00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7e80ae820 0x7fa7e8140610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:55.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.966+0000 7fa7ede00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7e80ceb00 0x7fa7e8148370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:55.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.966+0000 7fa7ede00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7e8145a00 0x7fa7e80c64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:55.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.966+0000 7fa7ede00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e80723b0 con 0x7fa7e80ceb00 2024-05-11T12:13:55.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.966+0000 7fa7ede00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e8072230 con 0x7fa7e80ae820 2024-05-11T12:13:55.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.966+0000 7fa7ede00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa7e8072530 con 0x7fa7e8145a00 2024-05-11T12:13:55.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.966+0000 7fa7e6000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7e80ceb00 0x7fa7e8148370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:55.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.967+0000 7fa7e6000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7e80ceb00 0x7fa7e8148370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54346/0 (socket says 172.21.15.167:54346) 2024-05-11T12:13:55.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.967+0000 7fa7e6000640 1 -- 172.21.15.167:0/1964744807 learned_addr learned my addr 172.21.15.167:0/1964744807 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:55.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.967+0000 7fa7e6a00640 1 --2- 172.21.15.167:0/1964744807 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7e80ae820 0x7fa7e8140610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:55.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.967+0000 7fa7e7400640 1 --2- 172.21.15.167:0/1964744807 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7e8145a00 0x7fa7e80c64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:55.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.967+0000 7fa7e6000640 1 -- 172.21.15.167:0/1964744807 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7e8145a00 msgr2=0x7fa7e80c64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:55.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.967+0000 7fa7e6000640 1 --2- 172.21.15.167:0/1964744807 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7e8145a00 0x7fa7e80c64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:55.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.967+0000 7fa7e6000640 1 -- 172.21.15.167:0/1964744807 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7e80ae820 msgr2=0x7fa7e8140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:55.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.967+0000 7fa7e6000640 1 --2- 172.21.15.167:0/1964744807 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7e80ae820 0x7fa7e8140610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:55.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.967+0000 7fa7e6000640 1 -- 172.21.15.167:0/1964744807 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7e81f4e50 con 0x7fa7e80ceb00 2024-05-11T12:13:55.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.967+0000 7fa7e6a00640 1 --2- 172.21.15.167:0/1964744807 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7e80ae820 0x7fa7e8140610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:55.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.967+0000 7fa7e6000640 1 --2- 172.21.15.167:0/1964744807 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7e80ceb00 0x7fa7e8148370 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4035040 tx=0x7fa7d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:55.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.968+0000 7fa7df400640 1 -- 172.21.15.167:0/1964744807 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d4041020 con 0x7fa7e80ceb00 2024-05-11T12:13:55.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.968+0000 7fa7df400640 1 -- 172.21.15.167:0/1964744807 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7d402ac50 con 0x7fa7e80ceb00 2024-05-11T12:13:55.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.968+0000 7fa7df400640 1 -- 172.21.15.167:0/1964744807 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d4037040 con 0x7fa7e80ceb00 2024-05-11T12:13:55.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.968+0000 7fa7ede00640 1 -- 172.21.15.167:0/1964744807 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7e81f5130 con 0x7fa7e80ceb00 2024-05-11T12:13:55.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.968+0000 7fa7ede00640 1 -- 172.21.15.167:0/1964744807 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7e81f56f0 con 0x7fa7e80ceb00 2024-05-11T12:13:55.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.969+0000 7fa7ede00640 1 -- 172.21.15.167:0/1964744807 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7e8069520 con 0x7fa7e80ceb00 2024-05-11T12:13:55.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.970+0000 7fa7df400640 1 -- 172.21.15.167:0/1964744807 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa7d4044020 con 0x7fa7e80ceb00 2024-05-11T12:13:55.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.971+0000 7fa7df400640 1 --2- 172.21.15.167:0/1964744807 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa7c80762d0 0x7fa7c8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:55.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.971+0000 7fa7df400640 1 -- 172.21.15.167:0/1964744807 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5335+0+0 (secure 0 0 0) 0x7fa7d40b7850 con 0x7fa7e80ceb00 2024-05-11T12:13:55.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.971+0000 7fa7e6a00640 1 --2- 172.21.15.167:0/1964744807 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa7c80762d0 0x7fa7c8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:55.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.972+0000 7fa7e6a00640 1 --2- 172.21.15.167:0/1964744807 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa7c80762d0 0x7fa7c8078790 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa7d80016a0 tx=0x7fa7d8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:55.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:55.975+0000 7fa7df400640 1 -- 172.21.15.167:0/1964744807 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa7d404b070 con 0x7fa7e80ceb00 2024-05-11T12:13:56.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.185+0000 7fa7ede00640 1 -- 172.21.15.167:0/1964744807 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fa7e80b6ad0 con 0x7fa7e80ceb00 2024-05-11T12:13:56.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.191+0000 7fa7df400640 1 -- 172.21.15.167:0/1964744807 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) ==== 104+0+0 (secure 0 0 0) 0x7fa7d402ae20 con 0x7fa7e80ceb00 2024-05-11T12:13:56.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.194+0000 7fa7ede00640 1 -- 172.21.15.167:0/1964744807 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa7c80762d0 msgr2=0x7fa7c8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:56.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.194+0000 7fa7ede00640 1 --2- 172.21.15.167:0/1964744807 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa7c80762d0 0x7fa7c8078790 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa7d80016a0 tx=0x7fa7d8009040 comp rx=0 tx=0).stop 2024-05-11T12:13:56.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.194+0000 7fa7ede00640 1 -- 172.21.15.167:0/1964744807 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7e80ceb00 msgr2=0x7fa7e8148370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:56.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.194+0000 7fa7ede00640 1 --2- 172.21.15.167:0/1964744807 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7e80ceb00 0x7fa7e8148370 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4035040 tx=0x7fa7d40046c0 comp rx=0 tx=0).stop 2024-05-11T12:13:56.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.194+0000 7fa7ede00640 1 -- 172.21.15.167:0/1964744807 shutdown_connections 2024-05-11T12:13:56.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.194+0000 7fa7ede00640 1 --2- 172.21.15.167:0/1964744807 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7e8145a00 0x7fa7e80c64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:56.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.194+0000 7fa7ede00640 1 --2- 172.21.15.167:0/1964744807 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7e80ceb00 0x7fa7e8148370 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:56.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.194+0000 7fa7ede00640 1 --2- 172.21.15.167:0/1964744807 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa7c80762d0 0x7fa7c8078790 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:56.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.194+0000 7fa7ede00640 1 --2- 172.21.15.167:0/1964744807 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7e80ae820 0x7fa7e8140610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:56.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.194+0000 7fa7ede00640 1 -- 172.21.15.167:0/1964744807 >> 172.21.15.167:0/1964744807 conn(0x7fa7e806c450 msgr2=0x7fa7e80afe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:56.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.194+0000 7fa7ede00640 1 -- 172.21.15.167:0/1964744807 shutdown_connections 2024-05-11T12:13:56.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.194+0000 7fa7ede00640 1 -- 172.21.15.167:0/1964744807 wait complete. 2024-05-11T12:13:56.211 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_conn_close' 2024-05-11T12:13:56.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.408+0000 7fa05f200640 1 Processor -- start 2024-05-11T12:13:56.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.409+0000 7fa05f200640 1 -- start start 2024-05-11T12:13:56.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.410+0000 7fa05f200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa0580d2cf0 0x7fa0580cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:56.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.410+0000 7fa05f200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa0580cb4f0 0x7fa05806dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:56.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.410+0000 7fa05f200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0580d1190 con 0x7fa0581598d0 2024-05-11T12:13:56.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.410+0000 7fa05f200640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fa0580cb920 con 0x7fa0580cb4f0 2024-05-11T12:13:56.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.410+0000 7fa05f200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fa05806d500 con 0x7fa0580d2cf0 2024-05-11T12:13:56.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.410+0000 7fa05d200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa0580cb4f0 0x7fa05806dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:56.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.410+0000 7fa05d200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa0580cb4f0 0x7fa05806dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54350/0 (socket says 172.21.15.167:54350) 2024-05-11T12:13:56.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.410+0000 7fa05d200640 1 -- 172.21.15.167:0/2167430127 learned_addr learned my addr 172.21.15.167:0/2167430127 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:56.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.411+0000 7fa057400640 1 --2- 172.21.15.167:0/2167430127 >> v2:172.21.15.204:3300/0 conn(0x7fa0580d2cf0 0x7fa0580cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:56.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.411+0000 7fa05d200640 1 -- 172.21.15.167:0/2167430127 >> v2:172.21.15.204:3300/0 conn(0x7fa0580d2cf0 msgr2=0x7fa0580cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:56.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.411+0000 7fa05d200640 1 --2- 172.21.15.167:0/2167430127 >> v2:172.21.15.204:3300/0 conn(0x7fa0580d2cf0 0x7fa0580cafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:56.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.411+0000 7fa05d200640 1 -- 172.21.15.167:0/2167430127 >> v1:172.21.15.204:6789/0 conn(0x7fa0581598d0 legacy=0x7fa0580d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:56.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.411+0000 7fa05d200640 1 -- 172.21.15.167:0/2167430127 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa05806d7e0 con 0x7fa0580cb4f0 2024-05-11T12:13:56.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.412+0000 7fa05d200640 1 --2- 172.21.15.167:0/2167430127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0580cb4f0 0x7fa05806dfe0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fa0580b4c50 tx=0x7fa05002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce6f788cb6ec225a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:56.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.412+0000 7fa056a00640 1 -- 172.21.15.167:0/2167430127 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa050037070 con 0x7fa0580cb4f0 2024-05-11T12:13:56.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.412+0000 7fa056a00640 1 -- 172.21.15.167:0/2167430127 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa050032440 con 0x7fa0580cb4f0 2024-05-11T12:13:56.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.412+0000 7fa056a00640 1 -- 172.21.15.167:0/2167430127 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa050032810 con 0x7fa0580cb4f0 2024-05-11T12:13:56.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.413+0000 7fa05f200640 1 -- 172.21.15.167:0/2167430127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0580cb4f0 msgr2=0x7fa05806dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:56.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.413+0000 7fa05f200640 1 --2- 172.21.15.167:0/2167430127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0580cb4f0 0x7fa05806dfe0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fa0580b4c50 tx=0x7fa05002a1e0 comp rx=0 tx=0).stop 2024-05-11T12:13:56.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.413+0000 7fa05f200640 1 -- 172.21.15.167:0/2167430127 shutdown_connections 2024-05-11T12:13:56.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.413+0000 7fa05f200640 1 --2- 172.21.15.167:0/2167430127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0580cb4f0 0x7fa05806dfe0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:56.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.413+0000 7fa05f200640 1 --2- 172.21.15.167:0/2167430127 >> v2:172.21.15.204:3300/0 conn(0x7fa0580d2cf0 0x7fa0580cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:56.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.413+0000 7fa05f200640 1 -- 172.21.15.167:0/2167430127 >> 172.21.15.167:0/2167430127 conn(0x7fa0580d1ba0 msgr2=0x7fa0580d1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:56.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.413+0000 7fa05f200640 1 -- 172.21.15.167:0/2167430127 shutdown_connections 2024-05-11T12:13:56.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.413+0000 7fa05f200640 1 -- 172.21.15.167:0/2167430127 wait complete. 2024-05-11T12:13:56.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.415+0000 7fa05f200640 1 Processor -- start 2024-05-11T12:13:56.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.415+0000 7fa05f200640 1 -- start start 2024-05-11T12:13:56.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.416+0000 7fa05f200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa0580cb4f0 0x7fa058143dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:56.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.416+0000 7fa05f200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa0580d2cf0 0x7fa0580b8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:56.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.416+0000 7fa05f200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0581598d0 0x7fa058161af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:56.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.416+0000 7fa05f200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa058159cb0 con 0x7fa0581598d0 2024-05-11T12:13:56.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.416+0000 7fa05f200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa0581607c0 con 0x7fa0580d2cf0 2024-05-11T12:13:56.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.416+0000 7fa05f200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa05806d1b0 con 0x7fa0580cb4f0 2024-05-11T12:13:56.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.416+0000 7fa05d200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0581598d0 0x7fa058161af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:56.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.416+0000 7fa05d200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0581598d0 0x7fa058161af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54354/0 (socket says 172.21.15.167:54354) 2024-05-11T12:13:56.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.416+0000 7fa05d200640 1 -- 172.21.15.167:0/387064215 learned_addr learned my addr 172.21.15.167:0/387064215 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:56.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.416+0000 7fa057e00640 1 --2- 172.21.15.167:0/387064215 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa0580cb4f0 0x7fa058143dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:56.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.417+0000 7fa05d200640 1 -- 172.21.15.167:0/387064215 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa0580cb4f0 msgr2=0x7fa058143dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:56.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.417+0000 7fa05d200640 1 --2- 172.21.15.167:0/387064215 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa0580cb4f0 0x7fa058143dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:56.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.417+0000 7fa05d200640 1 -- 172.21.15.167:0/387064215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa0580d2cf0 msgr2=0x7fa0580b8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:56.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.417+0000 7fa05d200640 1 --2- 172.21.15.167:0/387064215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa0580d2cf0 0x7fa0580b8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:56.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.417+0000 7fa05d200640 1 -- 172.21.15.167:0/387064215 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa058141430 con 0x7fa0581598d0 2024-05-11T12:13:56.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.417+0000 7fa057e00640 1 --2- 172.21.15.167:0/387064215 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa0580cb4f0 0x7fa058143dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:56.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.417+0000 7fa05d200640 1 --2- 172.21.15.167:0/387064215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0581598d0 0x7fa058161af0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fa05002a220 tx=0x7fa0500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:56.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.417+0000 7fa04fe00640 1 -- 172.21.15.167:0/387064215 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa050041020 con 0x7fa0581598d0 2024-05-11T12:13:56.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.417+0000 7fa04fe00640 1 -- 172.21.15.167:0/387064215 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0500306c0 con 0x7fa0581598d0 2024-05-11T12:13:56.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.417+0000 7fa05f200640 1 -- 172.21.15.167:0/387064215 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0580b90f0 con 0x7fa0581598d0 2024-05-11T12:13:56.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.417+0000 7fa04fe00640 1 -- 172.21.15.167:0/387064215 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa050037040 con 0x7fa0581598d0 2024-05-11T12:13:56.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.417+0000 7fa05f200640 1 -- 172.21.15.167:0/387064215 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa058153980 con 0x7fa0581598d0 2024-05-11T12:13:56.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.418+0000 7fa05f200640 1 -- 172.21.15.167:0/387064215 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa058069590 con 0x7fa0581598d0 2024-05-11T12:13:56.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.420+0000 7fa04fe00640 1 -- 172.21.15.167:0/387064215 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa050044020 con 0x7fa0581598d0 2024-05-11T12:13:56.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.421+0000 7fa04fe00640 1 --2- 172.21.15.167:0/387064215 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa0340762d0 0x7fa034078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:56.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.421+0000 7fa04fe00640 1 -- 172.21.15.167:0/387064215 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5335+0+0 (secure 0 0 0) 0x7fa0500b7d00 con 0x7fa0581598d0 2024-05-11T12:13:56.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.421+0000 7fa057e00640 1 --2- 172.21.15.167:0/387064215 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa0340762d0 0x7fa034078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:56.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.422+0000 7fa057e00640 1 --2- 172.21.15.167:0/387064215 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa0340762d0 0x7fa034078790 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa058151960 tx=0x7fa048009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:56.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.425+0000 7fa04fe00640 1 -- 172.21.15.167:0/387064215 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa05004b030 con 0x7fa0581598d0 2024-05-11T12:13:56.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:56.704+0000 7fa05f200640 1 -- 172.21.15.167:0/387064215 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_conn_close"]} v 0) -- 0x7fa0580cbfb0 con 0x7fa0581598d0 2024-05-11T12:13:57.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.313+0000 7fa04fe00640 1 -- 172.21.15.167:0/387064215 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_conn_close"]}]=0 v197) ==== 143+0+0 (secure 0 0 0) 0x7fa050081730 con 0x7fa0581598d0 2024-05-11T12:13:57.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.317+0000 7fa05f200640 1 -- 172.21.15.167:0/387064215 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa0340762d0 msgr2=0x7fa034078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:57.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.317+0000 7fa05f200640 1 --2- 172.21.15.167:0/387064215 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa0340762d0 0x7fa034078790 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa058151960 tx=0x7fa048009040 comp rx=0 tx=0).stop 2024-05-11T12:13:57.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.317+0000 7fa05f200640 1 -- 172.21.15.167:0/387064215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0581598d0 msgr2=0x7fa058161af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:57.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.317+0000 7fa05f200640 1 --2- 172.21.15.167:0/387064215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0581598d0 0x7fa058161af0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fa05002a220 tx=0x7fa0500046c0 comp rx=0 tx=0).stop 2024-05-11T12:13:57.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.317+0000 7fa05f200640 1 -- 172.21.15.167:0/387064215 shutdown_connections 2024-05-11T12:13:57.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.317+0000 7fa05f200640 1 --2- 172.21.15.167:0/387064215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0581598d0 0x7fa058161af0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:57.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.317+0000 7fa05f200640 1 --2- 172.21.15.167:0/387064215 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa0340762d0 0x7fa034078790 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:57.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.317+0000 7fa05f200640 1 --2- 172.21.15.167:0/387064215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa0580d2cf0 0x7fa0580b8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:57.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.317+0000 7fa05f200640 1 --2- 172.21.15.167:0/387064215 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa0580cb4f0 0x7fa058143dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:57.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.317+0000 7fa05f200640 1 -- 172.21.15.167:0/387064215 >> 172.21.15.167:0/387064215 conn(0x7fa0580d1ba0 msgr2=0x7fa0581610a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:57.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.318+0000 7fa05f200640 1 -- 172.21.15.167:0/387064215 shutdown_connections 2024-05-11T12:13:57.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.318+0000 7fa05f200640 1 -- 172.21.15.167:0/387064215 wait complete. 2024-05-11T12:13:57.339 INFO:tasks.cephfs_test_runner:That when a `tell` command is sent using the python CLI, ... ok 2024-05-11T12:13:57.339 INFO:tasks.cephfs_test_runner:Starting test: test_tell_session_drop (tasks.cephfs.test_sessionmap.TestSessionMap) That when a `tell` command is sent using the python CLI, 2024-05-11T12:13:57.339 INFO:tasks.cephfs_test_runner:test_tell_session_drop (tasks.cephfs.test_sessionmap.TestSessionMap) 2024-05-11T12:13:57.339 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:13:57.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.529+0000 7f507e200640 1 Processor -- start 2024-05-11T12:13:57.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.529+0000 7f507e200640 1 -- start start 2024-05-11T12:13:57.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.531+0000 7f507e200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5078157750 0x7f507815d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:57.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.531+0000 7f507e200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f50780b3ea0 0x7f5078161490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:57.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.531+0000 7f507e200640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f507815dc80 con 0x7f50781619d0 2024-05-11T12:13:57.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.531+0000 7f507e200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f5078161000 con 0x7f50780b3ea0 2024-05-11T12:13:57.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.531+0000 7f507e200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f507815df30 con 0x7f5078157750 2024-05-11T12:13:57.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.531+0000 7f5077e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f50781619d0 0x7f507806d460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:52784/0 (socket says 172.21.15.167:52784) 2024-05-11T12:13:57.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.531+0000 7f5077e00640 1 -- 172.21.15.167:0/3644312095 learned_addr learned my addr 172.21.15.167:0/3644312095 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:13:57.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.532+0000 7f5077400640 1 --2- 172.21.15.167:0/3644312095 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5078157750 0x7f507815d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:57.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.532+0000 7f506fe00640 1 --2- 172.21.15.167:0/3644312095 >> v2:172.21.15.204:3300/0 conn(0x7f50780b3ea0 0x7f5078161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:57.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.532+0000 7f5076a00640 1 -- 172.21.15.167:0/3644312095 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1001357368 0 0) 0x7f507815dc80 con 0x7f50781619d0 2024-05-11T12:13:57.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.532+0000 7f5076a00640 1 -- 172.21.15.167:0/3644312095 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f505c003520 con 0x7f50781619d0 2024-05-11T12:13:57.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.532+0000 7f5076a00640 1 -- 172.21.15.167:0/3644312095 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3066215938 0 0) 0x7f505c003520 con 0x7f50781619d0 2024-05-11T12:13:57.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.532+0000 7f5076a00640 1 -- 172.21.15.167:0/3644312095 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5078157750 msgr2=0x7f507815d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:57.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.532+0000 7f5076a00640 1 --2- 172.21.15.167:0/3644312095 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5078157750 0x7f507815d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:57.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.532+0000 7f5076a00640 1 -- 172.21.15.167:0/3644312095 >> v2:172.21.15.204:3300/0 conn(0x7f50780b3ea0 msgr2=0x7f5078161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:57.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.532+0000 7f5076a00640 1 --2- 172.21.15.167:0/3644312095 >> v2:172.21.15.204:3300/0 conn(0x7f50780b3ea0 0x7f5078161490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:57.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.532+0000 7f5076a00640 1 -- 172.21.15.167:0/3644312095 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f507815e210 con 0x7f50781619d0 2024-05-11T12:13:57.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.532+0000 7f5076a00640 1 -- 172.21.15.167:0/3644312095 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f5070001a70 con 0x7f50781619d0 2024-05-11T12:13:57.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.532+0000 7f5076a00640 1 -- 172.21.15.167:0/3644312095 >> v1:172.21.15.167:6789/0 conn(0x7f50781619d0 legacy=0x7f507806d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:57.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.532+0000 7f5076a00640 1 --2- 172.21.15.167:0/3644312095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f505c003a50 0x7f505c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:57.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.532+0000 7f5076a00640 1 --2- 172.21.15.167:0/3644312095 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5078157750 0x7f505c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:57.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.532+0000 7f5076a00640 1 --2- 172.21.15.167:0/3644312095 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f505c004d40 0x7f505c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:57.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.532+0000 7f5076a00640 1 -- 172.21.15.167:0/3644312095 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5078161000 con 0x7f505c003a50 2024-05-11T12:13:57.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.532+0000 7f5076a00640 1 -- 172.21.15.167:0/3644312095 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f507815df30 con 0x7f505c004d40 2024-05-11T12:13:57.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.532+0000 7f5076a00640 1 -- 172.21.15.167:0/3644312095 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f505c00b9f0 con 0x7f5078157750 2024-05-11T12:13:57.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.533+0000 7f5077400640 1 --2- 172.21.15.167:0/3644312095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f505c003a50 0x7f505c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:57.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.533+0000 7f506fe00640 1 --2- 172.21.15.167:0/3644312095 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5078157750 0x7f505c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:57.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.533+0000 7f5077400640 1 -- 172.21.15.167:0/3644312095 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5078157750 msgr2=0x7f505c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:57.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.533+0000 7f5077400640 1 --2- 172.21.15.167:0/3644312095 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5078157750 0x7f505c004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:57.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.533+0000 7f5077400640 1 -- 172.21.15.167:0/3644312095 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f505c004d40 msgr2=0x7f505c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:57.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.533+0000 7f5077400640 1 --2- 172.21.15.167:0/3644312095 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f505c004d40 0x7f505c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:57.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.533+0000 7f5077400640 1 -- 172.21.15.167:0/3644312095 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f505c003520 con 0x7f505c003a50 2024-05-11T12:13:57.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.534+0000 7f5077400640 1 --2- 172.21.15.167:0/3644312095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f505c003a50 0x7f505c003e70 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f5068009ac0 tx=0x7f505c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=893e125eef5a44c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:57.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.534+0000 7f5076a00640 1 -- 172.21.15.167:0/3644312095 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5068036070 con 0x7f505c003a50 2024-05-11T12:13:57.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.534+0000 7f5076a00640 1 -- 172.21.15.167:0/3644312095 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5068031400 con 0x7f505c003a50 2024-05-11T12:13:57.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.534+0000 7f5076a00640 1 -- 172.21.15.167:0/3644312095 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5068009690 con 0x7f505c003a50 2024-05-11T12:13:57.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.535+0000 7f507e200640 1 -- 172.21.15.167:0/3644312095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f505c003a50 msgr2=0x7f505c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:57.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.535+0000 7f507e200640 1 --2- 172.21.15.167:0/3644312095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f505c003a50 0x7f505c003e70 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f5068009ac0 tx=0x7f505c005830 comp rx=0 tx=0).stop 2024-05-11T12:13:57.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.535+0000 7f5077e00640 1 -- 172.21.15.167:0/3644312095 reap_dead start 2024-05-11T12:13:57.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.535+0000 7f507e200640 1 -- 172.21.15.167:0/3644312095 shutdown_connections 2024-05-11T12:13:57.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.535+0000 7f507e200640 1 -- 172.21.15.167:0/3644312095 >> 172.21.15.167:0/3644312095 conn(0x7f50780cd8f0 msgr2=0x7f50780cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:57.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.535+0000 7f507e200640 1 -- 172.21.15.167:0/3644312095 shutdown_connections 2024-05-11T12:13:57.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.535+0000 7f507e200640 1 -- 172.21.15.167:0/3644312095 wait complete. 2024-05-11T12:13:57.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.537+0000 7f507e200640 1 Processor -- start 2024-05-11T12:13:57.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.537+0000 7f507e200640 1 -- start start 2024-05-11T12:13:57.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.538+0000 7f507e200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f505c003a50 0x7f50781e6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:57.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.538+0000 7f507e200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5078073d20 0x7f5078140c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:57.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.538+0000 7f507e200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50781e82e0 0x7f50781e8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:57.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.538+0000 7f507e200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f507806db70 con 0x7f5078073d20 2024-05-11T12:13:57.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.538+0000 7f507e200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5078157b30 con 0x7f505c003a50 2024-05-11T12:13:57.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.538+0000 7f507e200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f507806dcf0 con 0x7f50781e82e0 2024-05-11T12:13:57.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.538+0000 7f506fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5078073d20 0x7f5078140c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:57.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.538+0000 7f506fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5078073d20 0x7f5078140c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54374/0 (socket says 172.21.15.167:54374) 2024-05-11T12:13:57.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.538+0000 7f506fe00640 1 -- 172.21.15.167:0/1405458627 learned_addr learned my addr 172.21.15.167:0/1405458627 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:57.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.538+0000 7f5077e00640 1 --2- 172.21.15.167:0/1405458627 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50781e82e0 0x7f50781e8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:57.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.538+0000 7f506fe00640 1 -- 172.21.15.167:0/1405458627 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50781e82e0 msgr2=0x7f50781e8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:57.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.538+0000 7f5077400640 1 --2- 172.21.15.167:0/1405458627 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f505c003a50 0x7f50781e6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:57.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.538+0000 7f506fe00640 1 --2- 172.21.15.167:0/1405458627 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50781e82e0 0x7f50781e8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:57.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.538+0000 7f506fe00640 1 -- 172.21.15.167:0/1405458627 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f505c003a50 msgr2=0x7f50781e6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:57.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.538+0000 7f506fe00640 1 --2- 172.21.15.167:0/1405458627 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f505c003a50 0x7f50781e6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:57.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.538+0000 7f506fe00640 1 -- 172.21.15.167:0/1405458627 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5078073060 con 0x7f5078073d20 2024-05-11T12:13:57.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.539+0000 7f506fe00640 1 --2- 172.21.15.167:0/1405458627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5078073d20 0x7f5078140c20 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f50600108e0 tx=0x7f5060010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:57.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.539+0000 7f506f400640 1 -- 172.21.15.167:0/1405458627 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506000f040 con 0x7f5078073d20 2024-05-11T12:13:57.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.539+0000 7f506f400640 1 -- 172.21.15.167:0/1405458627 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f506000eb60 con 0x7f5078073d20 2024-05-11T12:13:57.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.539+0000 7f506f400640 1 -- 172.21.15.167:0/1405458627 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506001c400 con 0x7f5078073d20 2024-05-11T12:13:57.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.539+0000 7f507e200640 1 -- 172.21.15.167:0/1405458627 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50781e7210 con 0x7f5078073d20 2024-05-11T12:13:57.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.539+0000 7f507e200640 1 -- 172.21.15.167:0/1405458627 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50781e77d0 con 0x7f5078073d20 2024-05-11T12:13:57.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.540+0000 7f507e200640 1 -- 172.21.15.167:0/1405458627 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5078069590 con 0x7f5078073d20 2024-05-11T12:13:57.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.541+0000 7f506f400640 1 -- 172.21.15.167:0/1405458627 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f506001c5a0 con 0x7f5078073d20 2024-05-11T12:13:57.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.542+0000 7f506f400640 1 --2- 172.21.15.167:0/1405458627 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f50540762d0 0x7f5054078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:57.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.542+0000 7f506f400640 1 -- 172.21.15.167:0/1405458627 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5335+0+0 (secure 0 0 0) 0x7f5060097b80 con 0x7f5078073d20 2024-05-11T12:13:57.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.542+0000 7f5077400640 1 --2- 172.21.15.167:0/1405458627 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f50540762d0 0x7f5054078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:57.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.543+0000 7f5077400640 1 --2- 172.21.15.167:0/1405458627 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f50540762d0 0x7f5054078790 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5068033000 tx=0x7f5068008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:57.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.547+0000 7f506f400640 1 -- 172.21.15.167:0/1405458627 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5060018030 con 0x7f5078073d20 2024-05-11T12:13:57.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.740+0000 7f507e200640 1 -- 172.21.15.167:0/1405458627 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f50781e79d0 con 0x7f5078073d20 2024-05-11T12:13:57.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.742+0000 7f506f400640 1 -- 172.21.15.167:0/1405458627 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+13344 (secure 0 0 0) 0x7f5060061640 con 0x7f5078073d20 2024-05-11T12:13:57.743 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:13:57.743 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":78,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:13:55.297778+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.167:6837/1306616214":"2024-05-12T12:13:52.280695+0000","172.21.15.167:6835/1306616214":"2024-05-12T12:13:52.280695+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:13:57.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.745+0000 7f507e200640 1 -- 172.21.15.167:0/1405458627 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f50540762d0 msgr2=0x7f5054078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:57.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.745+0000 7f507e200640 1 --2- 172.21.15.167:0/1405458627 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f50540762d0 0x7f5054078790 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5068033000 tx=0x7f5068008030 comp rx=0 tx=0).stop 2024-05-11T12:13:57.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.745+0000 7f507e200640 1 -- 172.21.15.167:0/1405458627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5078073d20 msgr2=0x7f5078140c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:57.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.745+0000 7f507e200640 1 --2- 172.21.15.167:0/1405458627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5078073d20 0x7f5078140c20 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f50600108e0 tx=0x7f5060010da0 comp rx=0 tx=0).stop 2024-05-11T12:13:57.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.745+0000 7f507e200640 1 -- 172.21.15.167:0/1405458627 shutdown_connections 2024-05-11T12:13:57.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.745+0000 7f507e200640 1 --2- 172.21.15.167:0/1405458627 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50781e82e0 0x7f50781e8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:57.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.745+0000 7f507e200640 1 --2- 172.21.15.167:0/1405458627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5078073d20 0x7f5078140c20 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:57.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.745+0000 7f507e200640 1 --2- 172.21.15.167:0/1405458627 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f50540762d0 0x7f5054078790 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:57.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.745+0000 7f507e200640 1 --2- 172.21.15.167:0/1405458627 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f505c003a50 0x7f50781e6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:57.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.745+0000 7f507e200640 1 -- 172.21.15.167:0/1405458627 >> 172.21.15.167:0/1405458627 conn(0x7f50780cd8f0 msgr2=0x7f50780cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:57.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.745+0000 7f507e200640 1 -- 172.21.15.167:0/1405458627 shutdown_connections 2024-05-11T12:13:57.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.745+0000 7f507e200640 1 -- 172.21.15.167:0/1405458627 wait complete. 2024-05-11T12:13:57.763 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-11T12:10:33.260270+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-11T12:13:57.763 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-11T12:13:57.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.962+0000 7fe45da00640 1 Processor -- start 2024-05-11T12:13:57.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.962+0000 7fe45da00640 1 -- start start 2024-05-11T12:13:57.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.964+0000 7fe45da00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe4580d0660 0x7fe4581609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:57.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.964+0000 7fe45da00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe458160ef0 0x7fe45806c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:57.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.964+0000 7fe45da00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4580cf5c0 con 0x7fe45813fe30 2024-05-11T12:13:57.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.964+0000 7fe45da00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fe458161340 con 0x7fe4580d0660 2024-05-11T12:13:57.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.964+0000 7fe45da00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fe4580cf8a0 con 0x7fe458160ef0 2024-05-11T12:13:57.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.964+0000 7fe457e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fe45813fe30 0x7fe45815de40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:52800/0 (socket says 172.21.15.167:52800) 2024-05-11T12:13:57.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.964+0000 7fe457400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe4580d0660 0x7fe4581609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:57.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.964+0000 7fe457e00640 1 -- 172.21.15.167:0/743804792 learned_addr learned my addr 172.21.15.167:0/743804792 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:13:57.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.964+0000 7fe45d000640 1 --2- 172.21.15.167:0/743804792 >> v2:172.21.15.204:3300/0 conn(0x7fe458160ef0 0x7fe45806c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:57.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.965+0000 7fe457400640 1 -- 172.21.15.167:0/743804792 >> v2:172.21.15.204:3300/0 conn(0x7fe458160ef0 msgr2=0x7fe45806c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:57.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.965+0000 7fe457400640 1 --2- 172.21.15.167:0/743804792 >> v2:172.21.15.204:3300/0 conn(0x7fe458160ef0 0x7fe45806c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:57.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.965+0000 7fe457400640 1 -- 172.21.15.167:0/743804792 >> v1:172.21.15.167:6789/0 conn(0x7fe45813fe30 legacy=0x7fe45815de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:57.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.965+0000 7fe457400640 1 -- 172.21.15.167:0/743804792 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4580cfb80 con 0x7fe4580d0660 2024-05-11T12:13:57.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.965+0000 7fe456a00640 1 -- 172.21.15.167:0/743804792 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2375432025 0 0) 0x7fe4580cf5c0 con 0x7fe45813fe30 2024-05-11T12:13:57.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.966+0000 7fe457400640 1 --2- 172.21.15.167:0/743804792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4580d0660 0x7fe4581609b0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fe4580b3df0 tx=0x7fe44402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd301a7ecd7a7b8e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:57.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.966+0000 7fe456a00640 1 -- 172.21.15.167:0/743804792 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe444037070 con 0x7fe4580d0660 2024-05-11T12:13:57.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.966+0000 7fe456a00640 1 -- 172.21.15.167:0/743804792 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe444032440 con 0x7fe4580d0660 2024-05-11T12:13:57.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.966+0000 7fe456a00640 1 -- 172.21.15.167:0/743804792 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe444032810 con 0x7fe4580d0660 2024-05-11T12:13:57.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.967+0000 7fe45da00640 1 -- 172.21.15.167:0/743804792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4580d0660 msgr2=0x7fe4581609b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:57.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.967+0000 7fe45da00640 1 --2- 172.21.15.167:0/743804792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4580d0660 0x7fe4581609b0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fe4580b3df0 tx=0x7fe44402a1a0 comp rx=0 tx=0).stop 2024-05-11T12:13:57.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.967+0000 7fe45da00640 1 -- 172.21.15.167:0/743804792 shutdown_connections 2024-05-11T12:13:57.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.967+0000 7fe45da00640 1 --2- 172.21.15.167:0/743804792 >> v2:172.21.15.204:3300/0 conn(0x7fe458160ef0 0x7fe45806c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:57.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.967+0000 7fe45da00640 1 --2- 172.21.15.167:0/743804792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4580d0660 0x7fe4581609b0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:57.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.967+0000 7fe45da00640 1 -- 172.21.15.167:0/743804792 >> 172.21.15.167:0/743804792 conn(0x7fe4580b2a20 msgr2=0x7fe4580b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:57.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.967+0000 7fe45da00640 1 -- 172.21.15.167:0/743804792 shutdown_connections 2024-05-11T12:13:57.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.967+0000 7fe45da00640 1 -- 172.21.15.167:0/743804792 wait complete. 2024-05-11T12:13:57.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.968+0000 7fe45da00640 1 Processor -- start 2024-05-11T12:13:57.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.968+0000 7fe45da00640 1 -- start start 2024-05-11T12:13:57.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.969+0000 7fe45da00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4580d0660 0x7fe4580b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:57.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.969+0000 7fe45da00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe45813fe30 0x7fe4581e8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:57.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.969+0000 7fe45da00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe458160ef0 0x7fe45815ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:57.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.969+0000 7fe45da00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe4581616a0 con 0x7fe458160ef0 2024-05-11T12:13:57.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.969+0000 7fe45da00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe4580d0140 con 0x7fe4580d0660 2024-05-11T12:13:57.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.969+0000 7fe45da00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe458161820 con 0x7fe45813fe30 2024-05-11T12:13:57.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.970+0000 7fe45d000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe458160ef0 0x7fe45815ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:57.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.970+0000 7fe45d000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe458160ef0 0x7fe45815ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54392/0 (socket says 172.21.15.167:54392) 2024-05-11T12:13:57.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.970+0000 7fe45d000640 1 -- 172.21.15.167:0/1474248579 learned_addr learned my addr 172.21.15.167:0/1474248579 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:57.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.970+0000 7fe457e00640 1 --2- 172.21.15.167:0/1474248579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4580d0660 0x7fe4580b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:57.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.970+0000 7fe457400640 1 --2- 172.21.15.167:0/1474248579 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe45813fe30 0x7fe4581e8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:57.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.970+0000 7fe45d000640 1 -- 172.21.15.167:0/1474248579 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe45813fe30 msgr2=0x7fe4581e8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:57.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.970+0000 7fe45d000640 1 --2- 172.21.15.167:0/1474248579 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe45813fe30 0x7fe4581e8cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:57.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.970+0000 7fe45d000640 1 -- 172.21.15.167:0/1474248579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4580d0660 msgr2=0x7fe4580b1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:57.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.970+0000 7fe45d000640 1 --2- 172.21.15.167:0/1474248579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4580d0660 0x7fe4580b1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:57.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.970+0000 7fe45d000640 1 -- 172.21.15.167:0/1474248579 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4581f5080 con 0x7fe458160ef0 2024-05-11T12:13:57.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.970+0000 7fe45d000640 1 --2- 172.21.15.167:0/1474248579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe458160ef0 0x7fe45815ec60 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fe450007e60 tx=0x7fe45000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:57.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.971+0000 7fe44fe00640 1 -- 172.21.15.167:0/1474248579 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe450014420 con 0x7fe458160ef0 2024-05-11T12:13:57.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.971+0000 7fe44fe00640 1 -- 172.21.15.167:0/1474248579 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe450019070 con 0x7fe458160ef0 2024-05-11T12:13:57.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.971+0000 7fe45da00640 1 -- 172.21.15.167:0/1474248579 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4581e9900 con 0x7fe458160ef0 2024-05-11T12:13:57.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.971+0000 7fe44fe00640 1 -- 172.21.15.167:0/1474248579 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe450014820 con 0x7fe458160ef0 2024-05-11T12:13:57.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.971+0000 7fe45da00640 1 -- 172.21.15.167:0/1474248579 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4581e9ec0 con 0x7fe458160ef0 2024-05-11T12:13:57.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.972+0000 7fe45da00640 1 -- 172.21.15.167:0/1474248579 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4580694b0 con 0x7fe458160ef0 2024-05-11T12:13:57.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.973+0000 7fe44fe00640 1 -- 172.21.15.167:0/1474248579 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe4500149c0 con 0x7fe458160ef0 2024-05-11T12:13:57.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.974+0000 7fe44fe00640 1 --2- 172.21.15.167:0/1474248579 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe4380762d0 0x7fe438078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:57.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.974+0000 7fe44fe00640 1 -- 172.21.15.167:0/1474248579 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5335+0+0 (secure 0 0 0) 0x7fe450097e20 con 0x7fe458160ef0 2024-05-11T12:13:57.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.974+0000 7fe457e00640 1 --2- 172.21.15.167:0/1474248579 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe4380762d0 0x7fe438078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:57.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.975+0000 7fe457e00640 1 --2- 172.21.15.167:0/1474248579 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe4380762d0 0x7fe438078790 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe448000f60 tx=0x7fe448009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:57.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:57.978+0000 7fe44fe00640 1 -- 172.21.15.167:0/1474248579 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe450061170 con 0x7fe458160ef0 2024-05-11T12:13:58.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.169+0000 7fe45da00640 1 -- 172.21.15.167:0/1474248579 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe4580b41b0 con 0x7fe458160ef0 2024-05-11T12:13:58.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.169+0000 7fe44fe00640 1 -- 172.21.15.167:0/1474248579 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) ==== 93+0+10 (secure 0 0 0) 0x7fe450065470 con 0x7fe458160ef0 2024-05-11T12:13:58.170 INFO:teuthology.orchestra.run.smithi167.stdout:pg_num: 1 2024-05-11T12:13:58.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.172+0000 7fe45da00640 1 -- 172.21.15.167:0/1474248579 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe4380762d0 msgr2=0x7fe438078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:58.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.172+0000 7fe45da00640 1 --2- 172.21.15.167:0/1474248579 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe4380762d0 0x7fe438078790 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe448000f60 tx=0x7fe448009040 comp rx=0 tx=0).stop 2024-05-11T12:13:58.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.172+0000 7fe45da00640 1 -- 172.21.15.167:0/1474248579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe458160ef0 msgr2=0x7fe45815ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:58.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.172+0000 7fe45da00640 1 --2- 172.21.15.167:0/1474248579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe458160ef0 0x7fe45815ec60 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fe450007e60 tx=0x7fe45000e4b0 comp rx=0 tx=0).stop 2024-05-11T12:13:58.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.172+0000 7fe45da00640 1 -- 172.21.15.167:0/1474248579 shutdown_connections 2024-05-11T12:13:58.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.172+0000 7fe45da00640 1 --2- 172.21.15.167:0/1474248579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe458160ef0 0x7fe45815ec60 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:58.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.172+0000 7fe45da00640 1 --2- 172.21.15.167:0/1474248579 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe45813fe30 0x7fe4581e8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:58.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.172+0000 7fe45da00640 1 --2- 172.21.15.167:0/1474248579 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe4380762d0 0x7fe438078790 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:58.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.172+0000 7fe45da00640 1 --2- 172.21.15.167:0/1474248579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4580d0660 0x7fe4580b1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:58.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.172+0000 7fe45da00640 1 -- 172.21.15.167:0/1474248579 >> 172.21.15.167:0/1474248579 conn(0x7fe4580b2a20 msgr2=0x7fe4580b11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:58.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.172+0000 7fe45da00640 1 -- 172.21.15.167:0/1474248579 shutdown_connections 2024-05-11T12:13:58.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.172+0000 7fe45da00640 1 -- 172.21.15.167:0/1474248579 wait complete. 2024-05-11T12:13:58.189 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_session_drop' 2024-05-11T12:13:58.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.384+0000 7f22a3200640 1 Processor -- start 2024-05-11T12:13:58.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.384+0000 7f22a3200640 1 -- start start 2024-05-11T12:13:58.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.385+0000 7f22a3200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f22a413fd40 0x7f22a415de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:58.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.385+0000 7f22a3200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f22a4140120 0x7f22a4142690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:58.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.385+0000 7f22a3200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f22a40cf1c0 0x7f22a4142bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:58.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.385+0000 7f22a3200640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f22a41434c0 con 0x7f22a413fd40 2024-05-11T12:13:58.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.385+0000 7f22a3200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f22a415e390 con 0x7f22a40cf1c0 2024-05-11T12:13:58.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.385+0000 7f22a3200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f22a40cf780 con 0x7f22a4140120 2024-05-11T12:13:58.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.385+0000 7f22a1e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f22a413fd40 0x7f22a415de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:58.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.385+0000 7f22a1e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f22a413fd40 0x7f22a415de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54394/0 (socket says 172.21.15.167:54394) 2024-05-11T12:13:58.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.385+0000 7f22a1e00640 1 -- 172.21.15.167:0/1367933065 learned_addr learned my addr 172.21.15.167:0/1367933065 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:58.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.386+0000 7f22a2800640 1 --2- 172.21.15.167:0/1367933065 >> v2:172.21.15.204:3300/0 conn(0x7f22a40cf1c0 0x7f22a4142bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:58.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.386+0000 7f22a1e00640 1 -- 172.21.15.167:0/1367933065 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f22a4140120 msgr2=0x7f22a4142690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:58.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.386+0000 7f229be00640 1 --2- 172.21.15.167:0/1367933065 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f22a4140120 0x7f22a4142690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:58.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.386+0000 7f22a1e00640 1 --2- 172.21.15.167:0/1367933065 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f22a4140120 0x7f22a4142690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:58.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.386+0000 7f22a1e00640 1 -- 172.21.15.167:0/1367933065 >> v2:172.21.15.204:3300/0 conn(0x7f22a40cf1c0 msgr2=0x7f22a4142bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:58.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.386+0000 7f22a1e00640 1 --2- 172.21.15.167:0/1367933065 >> v2:172.21.15.204:3300/0 conn(0x7f22a40cf1c0 0x7f22a4142bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:58.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.386+0000 7f22a1e00640 1 -- 172.21.15.167:0/1367933065 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a40cfa60 con 0x7f22a413fd40 2024-05-11T12:13:58.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.386+0000 7f22a2800640 1 --2- 172.21.15.167:0/1367933065 >> v2:172.21.15.204:3300/0 conn(0x7f22a40cf1c0 0x7f22a4142bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:58.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.386+0000 7f229be00640 1 --2- 172.21.15.167:0/1367933065 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f22a4140120 0x7f22a4142690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:58.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.387+0000 7f22a1e00640 1 --2- 172.21.15.167:0/1367933065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22a413fd40 0x7f22a415de50 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f22a4070460 tx=0x7f229402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca9abca43f45d3b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:58.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.387+0000 7f22a1400640 1 -- 172.21.15.167:0/1367933065 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2294037070 con 0x7f22a413fd40 2024-05-11T12:13:58.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.387+0000 7f22a1400640 1 -- 172.21.15.167:0/1367933065 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2294032440 con 0x7f22a413fd40 2024-05-11T12:13:58.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.387+0000 7f22a1400640 1 -- 172.21.15.167:0/1367933065 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2294032810 con 0x7f22a413fd40 2024-05-11T12:13:58.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.388+0000 7f22a3200640 1 -- 172.21.15.167:0/1367933065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22a413fd40 msgr2=0x7f22a415de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:58.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.388+0000 7f22a3200640 1 --2- 172.21.15.167:0/1367933065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22a413fd40 0x7f22a415de50 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f22a4070460 tx=0x7f229402a1e0 comp rx=0 tx=0).stop 2024-05-11T12:13:58.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.388+0000 7f22a3200640 1 -- 172.21.15.167:0/1367933065 shutdown_connections 2024-05-11T12:13:58.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.388+0000 7f22a3200640 1 --2- 172.21.15.167:0/1367933065 >> v2:172.21.15.204:3300/0 conn(0x7f22a40cf1c0 0x7f22a4142bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:58.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.388+0000 7f22a3200640 1 --2- 172.21.15.167:0/1367933065 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f22a4140120 0x7f22a4142690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:58.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.388+0000 7f22a3200640 1 --2- 172.21.15.167:0/1367933065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22a413fd40 0x7f22a415de50 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:58.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.388+0000 7f22a3200640 1 -- 172.21.15.167:0/1367933065 >> 172.21.15.167:0/1367933065 conn(0x7f22a40b2840 msgr2=0x7f22a40b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:58.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.388+0000 7f22a3200640 1 -- 172.21.15.167:0/1367933065 shutdown_connections 2024-05-11T12:13:58.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.388+0000 7f22a3200640 1 -- 172.21.15.167:0/1367933065 wait complete. 2024-05-11T12:13:58.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.390+0000 7f22a3200640 1 Processor -- start 2024-05-11T12:13:58.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.390+0000 7f22a3200640 1 -- start start 2024-05-11T12:13:58.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.391+0000 7f22a3200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22a40cf1c0 0x7f22a413b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:58.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.391+0000 7f22a3200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22a413fd40 0x7f22a413cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:58.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.391+0000 7f22a3200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f22a4140120 0x7f22a40ae820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:58.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.391+0000 7f22a3200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f22a40d0300 con 0x7f22a40cf1c0 2024-05-11T12:13:58.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.391+0000 7f22a3200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f22a40d0020 con 0x7f22a413fd40 2024-05-11T12:13:58.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.391+0000 7f22a3200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f22a41609c0 con 0x7f22a4140120 2024-05-11T12:13:58.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.391+0000 7f22a1e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22a40cf1c0 0x7f22a413b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:58.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.391+0000 7f22a1e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22a40cf1c0 0x7f22a413b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54398/0 (socket says 172.21.15.167:54398) 2024-05-11T12:13:58.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.391+0000 7f22a1e00640 1 -- 172.21.15.167:0/1593625766 learned_addr learned my addr 172.21.15.167:0/1593625766 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:58.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.391+0000 7f22a1e00640 1 -- 172.21.15.167:0/1593625766 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f22a4140120 msgr2=0x7f22a40ae820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:58.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.391+0000 7f22a1e00640 1 --2- 172.21.15.167:0/1593625766 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f22a4140120 0x7f22a40ae820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:58.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.391+0000 7f22a1e00640 1 -- 172.21.15.167:0/1593625766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22a413fd40 msgr2=0x7f22a413cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:58.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.391+0000 7f22a1e00640 1 --2- 172.21.15.167:0/1593625766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22a413fd40 0x7f22a413cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:58.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.391+0000 7f22a1e00640 1 -- 172.21.15.167:0/1593625766 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a413e9a0 con 0x7f22a40cf1c0 2024-05-11T12:13:58.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.391+0000 7f22a1e00640 1 --2- 172.21.15.167:0/1593625766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22a40cf1c0 0x7f22a413b490 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f229402a220 tx=0x7f22940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:58.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.392+0000 7f229a000640 1 -- 172.21.15.167:0/1593625766 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2294037040 con 0x7f22a40cf1c0 2024-05-11T12:13:58.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.392+0000 7f229a000640 1 -- 172.21.15.167:0/1593625766 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2294042070 con 0x7f22a40cf1c0 2024-05-11T12:13:58.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.392+0000 7f22a3200640 1 -- 172.21.15.167:0/1593625766 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22a413ec80 con 0x7f22a40cf1c0 2024-05-11T12:13:58.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.392+0000 7f229a000640 1 -- 172.21.15.167:0/1593625766 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2294030bd0 con 0x7f22a40cf1c0 2024-05-11T12:13:58.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.392+0000 7f22a3200640 1 -- 172.21.15.167:0/1593625766 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22a4140c50 con 0x7f22a40cf1c0 2024-05-11T12:13:58.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.393+0000 7f22a3200640 1 -- 172.21.15.167:0/1593625766 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22a40694b0 con 0x7f22a40cf1c0 2024-05-11T12:13:58.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.394+0000 7f229a000640 1 -- 172.21.15.167:0/1593625766 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2294047460 con 0x7f22a40cf1c0 2024-05-11T12:13:58.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.395+0000 7f229a000640 1 --2- 172.21.15.167:0/1593625766 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f22800762d0 0x7f2280078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:58.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.395+0000 7f229a000640 1 -- 172.21.15.167:0/1593625766 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5335+0+0 (secure 0 0 0) 0x7f22940b8140 con 0x7f22a40cf1c0 2024-05-11T12:13:58.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.395+0000 7f229be00640 1 --2- 172.21.15.167:0/1593625766 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f22800762d0 0x7f2280078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:58.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.396+0000 7f229be00640 1 --2- 172.21.15.167:0/1593625766 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f22800762d0 0x7f2280078790 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f228c006b10 tx=0x7f228c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:58.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.400+0000 7f229a000640 1 -- 172.21.15.167:0/1593625766 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2294081c00 con 0x7f22a40cf1c0 2024-05-11T12:13:58.687 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:58.687+0000 7f22a3200640 1 -- 172.21.15.167:0/1593625766 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_session_drop"]} v 0) -- 0x7f22a40b0590 con 0x7f22a40cf1c0 2024-05-11T12:13:59.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.326+0000 7f229a000640 1 -- 172.21.15.167:0/1593625766 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_session_drop"]}]=0 v199) ==== 148+0+0 (secure 0 0 0) 0x7f2294081500 con 0x7f22a40cf1c0 2024-05-11T12:13:59.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.330+0000 7f22a3200640 1 -- 172.21.15.167:0/1593625766 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f22800762d0 msgr2=0x7f2280078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:59.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.330+0000 7f22a3200640 1 --2- 172.21.15.167:0/1593625766 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f22800762d0 0x7f2280078790 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f228c006b10 tx=0x7f228c008040 comp rx=0 tx=0).stop 2024-05-11T12:13:59.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.330+0000 7f22a3200640 1 -- 172.21.15.167:0/1593625766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22a40cf1c0 msgr2=0x7f22a413b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:59.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.330+0000 7f22a3200640 1 --2- 172.21.15.167:0/1593625766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22a40cf1c0 0x7f22a413b490 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f229402a220 tx=0x7f22940046c0 comp rx=0 tx=0).stop 2024-05-11T12:13:59.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.330+0000 7f22a3200640 1 -- 172.21.15.167:0/1593625766 shutdown_connections 2024-05-11T12:13:59.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.330+0000 7f22a3200640 1 --2- 172.21.15.167:0/1593625766 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f22a4140120 0x7f22a40ae820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:59.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.330+0000 7f22a3200640 1 --2- 172.21.15.167:0/1593625766 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f22800762d0 0x7f2280078790 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:59.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.330+0000 7f22a3200640 1 --2- 172.21.15.167:0/1593625766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22a413fd40 0x7f22a413cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:59.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.330+0000 7f22a3200640 1 --2- 172.21.15.167:0/1593625766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22a40cf1c0 0x7f22a413b490 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:59.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.330+0000 7f22a3200640 1 -- 172.21.15.167:0/1593625766 >> 172.21.15.167:0/1593625766 conn(0x7f22a40b2840 msgr2=0x7f22a40b48a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:59.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.330+0000 7f22a3200640 1 -- 172.21.15.167:0/1593625766 shutdown_connections 2024-05-11T12:13:59.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.331+0000 7f22a3200640 1 -- 172.21.15.167:0/1593625766 wait complete. 2024-05-11T12:13:59.354 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-11T12:13:59.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.557+0000 7f957d400640 1 Processor -- start 2024-05-11T12:13:59.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.558+0000 7f957d400640 1 -- start start 2024-05-11T12:13:59.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.559+0000 7f957d400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f957806cef0 0x7f957806d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:59.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.559+0000 7f957d400640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95780d2c50 con 0x7f957815f0d0 2024-05-11T12:13:59.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.559+0000 7f957d400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95780d2f30 con 0x7f95780cfda0 2024-05-11T12:13:59.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.559+0000 7f957d400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f957806d8a0 con 0x7f957806cef0 2024-05-11T12:13:59.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.559+0000 7f9577e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f957815f0d0 0x7f95780d0ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:40330/0 (socket says 172.21.15.167:40330) 2024-05-11T12:13:59.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.559+0000 7f9577e00640 1 -- 172.21.15.167:0/3915675045 learned_addr learned my addr 172.21.15.167:0/3915675045 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:13:59.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.560+0000 7f9576a00640 1 -- 172.21.15.167:0/3915675045 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 106884406 0 0) 0x7f95780d2c50 con 0x7f957815f0d0 2024-05-11T12:13:59.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.560+0000 7f957ca00640 1 --2- 172.21.15.167:0/3915675045 >> v2:172.21.15.204:3300/0 conn(0x7f957806cef0 0x7f957806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:59.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.560+0000 7f9576a00640 1 -- 172.21.15.167:0/3915675045 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f955c003520 con 0x7f957815f0d0 2024-05-11T12:13:59.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.560+0000 7f9576a00640 1 -- 172.21.15.167:0/3915675045 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 885966647 0 0) 0x7f955c003520 con 0x7f957815f0d0 2024-05-11T12:13:59.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.560+0000 7f9576a00640 1 -- 172.21.15.167:0/3915675045 >> v2:172.21.15.204:3300/0 conn(0x7f957806cef0 msgr2=0x7f957806d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:59.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.560+0000 7f9576a00640 1 --2- 172.21.15.167:0/3915675045 >> v2:172.21.15.204:3300/0 conn(0x7f957806cef0 0x7f957806d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:59.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.560+0000 7f9576a00640 1 -- 172.21.15.167:0/3915675045 >> v1:172.21.15.204:6789/0 conn(0x7f95780cfda0 legacy=0x7f95780d3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:59.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.560+0000 7f9576a00640 1 -- 172.21.15.167:0/3915675045 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95780d3210 con 0x7f957815f0d0 2024-05-11T12:13:59.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.560+0000 7f9576a00640 1 -- 172.21.15.167:0/3915675045 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f95680016f0 con 0x7f957815f0d0 2024-05-11T12:13:59.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.560+0000 7f9576a00640 1 -- 172.21.15.167:0/3915675045 >> v1:172.21.15.167:6789/0 conn(0x7f957815f0d0 legacy=0x7f95780d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:59.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.560+0000 7f9576a00640 1 --2- 172.21.15.167:0/3915675045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f955c003aa0 0x7f955c003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:59.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.560+0000 7f9576a00640 1 --2- 172.21.15.167:0/3915675045 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f955c004800 0x7f955c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:59.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.561+0000 7f9576a00640 1 --2- 172.21.15.167:0/3915675045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f955c0052e0 0x7f955c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:59.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.561+0000 7f9576a00640 1 -- 172.21.15.167:0/3915675045 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f957806d8a0 con 0x7f955c003aa0 2024-05-11T12:13:59.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.561+0000 7f9576a00640 1 -- 172.21.15.167:0/3915675045 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f955c00c200 con 0x7f955c0052e0 2024-05-11T12:13:59.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.561+0000 7f9576a00640 1 -- 172.21.15.167:0/3915675045 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f955c00c400 con 0x7f955c004800 2024-05-11T12:13:59.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.561+0000 7f957ca00640 1 --2- 172.21.15.167:0/3915675045 >> v2:172.21.15.204:3300/0 conn(0x7f957806cef0 0x7f957806d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:13:59.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.561+0000 7f9577e00640 1 --2- 172.21.15.167:0/3915675045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f955c003aa0 0x7f955c003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:59.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.561+0000 7f9577e00640 1 -- 172.21.15.167:0/3915675045 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f955c004800 msgr2=0x7f955c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:59.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.561+0000 7f9577e00640 1 --2- 172.21.15.167:0/3915675045 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f955c004800 0x7f955c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:59.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.561+0000 7f9577e00640 1 -- 172.21.15.167:0/3915675045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f955c0052e0 msgr2=0x7f955c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:13:59.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.561+0000 7f9577e00640 1 --2- 172.21.15.167:0/3915675045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f955c0052e0 0x7f955c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:59.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.561+0000 7f957ca00640 1 -- 172.21.15.167:0/3915675045 reap_dead start 2024-05-11T12:13:59.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.561+0000 7f9577e00640 1 -- 172.21.15.167:0/3915675045 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f955c003520 con 0x7f955c003aa0 2024-05-11T12:13:59.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.562+0000 7f9577e00640 1 --2- 172.21.15.167:0/3915675045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f955c003aa0 0x7f955c003ec0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f956800c210 tx=0x7f955c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97100dc261b0e838 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:59.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.563+0000 7f9576a00640 1 -- 172.21.15.167:0/3915675045 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9568007ce0 con 0x7f955c003aa0 2024-05-11T12:13:59.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.563+0000 7f9576a00640 1 -- 172.21.15.167:0/3915675045 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9568005ad0 con 0x7f955c003aa0 2024-05-11T12:13:59.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.563+0000 7f9576a00640 1 -- 172.21.15.167:0/3915675045 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9568007070 con 0x7f955c003aa0 2024-05-11T12:13:59.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.563+0000 7f957d400640 1 -- 172.21.15.167:0/3915675045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f955c003aa0 msgr2=0x7f955c003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:59.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.563+0000 7f957d400640 1 --2- 172.21.15.167:0/3915675045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f955c003aa0 0x7f955c003ec0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f956800c210 tx=0x7f955c005830 comp rx=0 tx=0).stop 2024-05-11T12:13:59.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.563+0000 7f957d400640 1 -- 172.21.15.167:0/3915675045 shutdown_connections 2024-05-11T12:13:59.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.563+0000 7f957d400640 1 --2- 172.21.15.167:0/3915675045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f955c003aa0 0x7f955c003ec0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:59.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.563+0000 7f957d400640 1 -- 172.21.15.167:0/3915675045 >> 172.21.15.167:0/3915675045 conn(0x7f95780b6af0 msgr2=0x7f95780b6f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:59.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.563+0000 7f957d400640 1 -- 172.21.15.167:0/3915675045 shutdown_connections 2024-05-11T12:13:59.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.563+0000 7f957d400640 1 -- 172.21.15.167:0/3915675045 wait complete. 2024-05-11T12:13:59.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.565+0000 7f957d400640 1 Processor -- start 2024-05-11T12:13:59.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.565+0000 7f957d400640 1 -- start start 2024-05-11T12:13:59.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.566+0000 7f957d400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f955c003aa0 0x7f95781561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:59.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.566+0000 7f957d400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95780cc0b0 0x7f9578161e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:59.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.566+0000 7f957d400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95781623d0 0x7f95780d1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:59.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.566+0000 7f957d400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f957815f4b0 con 0x7f95781623d0 2024-05-11T12:13:59.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.566+0000 7f957d400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f95780d37d0 con 0x7f95780cc0b0 2024-05-11T12:13:59.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.566+0000 7f957d400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f957806d360 con 0x7f955c003aa0 2024-05-11T12:13:59.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.566+0000 7f957ca00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95781623d0 0x7f95780d1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:59.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.566+0000 7f957ca00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95781623d0 0x7f95780d1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45162/0 (socket says 172.21.15.167:45162) 2024-05-11T12:13:59.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.566+0000 7f957ca00640 1 -- 172.21.15.167:0/3648658427 learned_addr learned my addr 172.21.15.167:0/3648658427 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:13:59.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.567+0000 7f9577e00640 1 --2- 172.21.15.167:0/3648658427 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f955c003aa0 0x7f95781561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:59.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.567+0000 7f957ca00640 1 -- 172.21.15.167:0/3648658427 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f955c003aa0 msgr2=0x7f95781561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:59.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.567+0000 7f957ca00640 1 --2- 172.21.15.167:0/3648658427 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f955c003aa0 0x7f95781561b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:59.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.567+0000 7f9577400640 1 --2- 172.21.15.167:0/3648658427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95780cc0b0 0x7f9578161e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:59.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.567+0000 7f957ca00640 1 -- 172.21.15.167:0/3648658427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95780cc0b0 msgr2=0x7f9578161e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:59.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.567+0000 7f957ca00640 1 --2- 172.21.15.167:0/3648658427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95780cc0b0 0x7f9578161e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:59.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.567+0000 7f957ca00640 1 -- 172.21.15.167:0/3648658427 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95780ce250 con 0x7f95781623d0 2024-05-11T12:13:59.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.567+0000 7f9577400640 1 --2- 172.21.15.167:0/3648658427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95780cc0b0 0x7f9578161e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:13:59.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.567+0000 7f957ca00640 1 --2- 172.21.15.167:0/3648658427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95781623d0 0x7f95780d1f00 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f9570007c40 tx=0x7f957000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:59.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.568+0000 7f956fe00640 1 -- 172.21.15.167:0/3648658427 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957001a070 con 0x7f95781623d0 2024-05-11T12:13:59.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.568+0000 7f956fe00640 1 -- 172.21.15.167:0/3648658427 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f957000dcd0 con 0x7f95781623d0 2024-05-11T12:13:59.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.568+0000 7f956fe00640 1 -- 172.21.15.167:0/3648658427 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9570003bd0 con 0x7f95781623d0 2024-05-11T12:13:59.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.568+0000 7f957d400640 1 -- 172.21.15.167:0/3648658427 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95780ce530 con 0x7f95781623d0 2024-05-11T12:13:59.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.568+0000 7f957d400640 1 -- 172.21.15.167:0/3648658427 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9578160390 con 0x7f95781623d0 2024-05-11T12:13:59.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.569+0000 7f957d400640 1 -- 172.21.15.167:0/3648658427 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95780694b0 con 0x7f95781623d0 2024-05-11T12:13:59.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.570+0000 7f956fe00640 1 -- 172.21.15.167:0/3648658427 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f957001d070 con 0x7f95781623d0 2024-05-11T12:13:59.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.571+0000 7f956fe00640 1 --2- 172.21.15.167:0/3648658427 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f9554076300 0x7f95540787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:59.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.571+0000 7f956fe00640 1 -- 172.21.15.167:0/3648658427 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5335+0+0 (secure 0 0 0) 0x7f95700971f0 con 0x7f95781623d0 2024-05-11T12:13:59.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.571+0000 7f9577e00640 1 --2- 172.21.15.167:0/3648658427 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f9554076300 0x7f95540787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:59.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.572+0000 7f9577e00640 1 --2- 172.21.15.167:0/3648658427 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f9554076300 0x7f95540787c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9568032000 tx=0x7f9568009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:59.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.575+0000 7f956fe00640 1 -- 172.21.15.167:0/3648658427 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9570010070 con 0x7f95781623d0 2024-05-11T12:13:59.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.776+0000 7f957d400640 1 -- 172.21.15.167:0/3648658427 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f95780c8580 con 0x7f95781623d0 2024-05-11T12:13:59.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.781+0000 7f956fe00640 1 -- 172.21.15.167:0/3648658427 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) ==== 122+0+0 (secure 0 0 0) 0x7f9570060cb0 con 0x7f95781623d0 2024-05-11T12:13:59.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.784+0000 7f957d400640 1 -- 172.21.15.167:0/3648658427 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f9554076300 msgr2=0x7f95540787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:59.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.784+0000 7f957d400640 1 --2- 172.21.15.167:0/3648658427 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f9554076300 0x7f95540787c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9568032000 tx=0x7f9568009030 comp rx=0 tx=0).stop 2024-05-11T12:13:59.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.784+0000 7f957d400640 1 -- 172.21.15.167:0/3648658427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95781623d0 msgr2=0x7f95780d1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:59.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.784+0000 7f957d400640 1 --2- 172.21.15.167:0/3648658427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95781623d0 0x7f95780d1f00 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f9570007c40 tx=0x7f957000d4a0 comp rx=0 tx=0).stop 2024-05-11T12:13:59.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.784+0000 7f957d400640 1 -- 172.21.15.167:0/3648658427 shutdown_connections 2024-05-11T12:13:59.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.784+0000 7f957d400640 1 --2- 172.21.15.167:0/3648658427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95781623d0 0x7f95780d1f00 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:59.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.784+0000 7f957d400640 1 --2- 172.21.15.167:0/3648658427 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f9554076300 0x7f95540787c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:59.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.784+0000 7f957d400640 1 --2- 172.21.15.167:0/3648658427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95780cc0b0 0x7f9578161e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:59.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.784+0000 7f957d400640 1 --2- 172.21.15.167:0/3648658427 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f955c003aa0 0x7f95781561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:59.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.784+0000 7f957d400640 1 -- 172.21.15.167:0/3648658427 >> 172.21.15.167:0/3648658427 conn(0x7f95780b6af0 msgr2=0x7f95780d3bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:59.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.784+0000 7f957d400640 1 -- 172.21.15.167:0/3648658427 shutdown_connections 2024-05-11T12:13:59.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:13:59.784+0000 7f957d400640 1 -- 172.21.15.167:0/3648658427 wait complete. 2024-05-11T12:13:59.802 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-11T12:13:59.802 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-11T12:13:59.895 DEBUG:teuthology.orchestra.run.smithi204:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-11T12:14:00.004 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-11T12:14:00.004 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:14:00.004 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:14:00.063 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-11T12:14:00.063 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:14:00.063 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:14:00.121 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052713,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-11T12:14:00.121 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:14:00.121 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:14:00.181 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-11T12:14:00.181 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:14:00.181 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:14:00.239 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052713,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-11T12:14:00.239 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-11T12:14:00.239 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:14:00.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.445+0000 7fbff4400640 1 Processor -- start 2024-05-11T12:14:00.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.446+0000 7fbff4400640 1 -- start start 2024-05-11T12:14:00.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.447+0000 7fbff4400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbfec13eeb0 0x7fbfec06f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:00.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.447+0000 7fbff4400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fbfec13f290 0x7fbfec06f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:00.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.447+0000 7fbff4400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbfec0b50a0 0x7fbfec0735c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:00.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.447+0000 7fbff4400640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fbfec15fb90 con 0x7fbfec0b50a0 2024-05-11T12:14:00.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.447+0000 7fbff4400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fbfec15fe70 con 0x7fbfec13f290 2024-05-11T12:14:00.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.447+0000 7fbff4400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbfec160150 con 0x7fbfec13eeb0 2024-05-11T12:14:00.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.448+0000 7fbff2400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbfec0b50a0 0x7fbfec0735c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:00.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.448+0000 7fbff2400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbfec0b50a0 0x7fbfec0735c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45178/0 (socket says 172.21.15.167:45178) 2024-05-11T12:14:00.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.448+0000 7fbff2400640 1 -- 172.21.15.167:0/2993951871 learned_addr learned my addr 172.21.15.167:0/2993951871 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:00.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.448+0000 7fbff1a00640 1 --2- 172.21.15.167:0/2993951871 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbfec13eeb0 0x7fbfec06f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:00.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.448+0000 7fbff1000640 1 --2- 172.21.15.167:0/2993951871 >> v2:172.21.15.204:3300/0 conn(0x7fbfec13f290 0x7fbfec06f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:00.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.448+0000 7fbff2400640 1 -- 172.21.15.167:0/2993951871 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbfec13eeb0 msgr2=0x7fbfec06f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:00.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.448+0000 7fbff2400640 1 --2- 172.21.15.167:0/2993951871 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbfec13eeb0 0x7fbfec06f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:00.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.448+0000 7fbff2400640 1 -- 172.21.15.167:0/2993951871 >> v2:172.21.15.204:3300/0 conn(0x7fbfec13f290 msgr2=0x7fbfec06f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:00.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.448+0000 7fbff2400640 1 --2- 172.21.15.167:0/2993951871 >> v2:172.21.15.204:3300/0 conn(0x7fbfec13f290 0x7fbfec06f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:00.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.448+0000 7fbff2400640 1 -- 172.21.15.167:0/2993951871 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfec160430 con 0x7fbfec0b50a0 2024-05-11T12:14:00.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.449+0000 7fbff2400640 1 --2- 172.21.15.167:0/2993951871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbfec0b50a0 0x7fbfec0735c0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fbfec0cebc0 tx=0x7fbfe402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eebb494ad8a857b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:00.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.450+0000 7fbfebe00640 1 -- 172.21.15.167:0/2993951871 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe4036070 con 0x7fbfec0b50a0 2024-05-11T12:14:00.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.450+0000 7fbfebe00640 1 -- 172.21.15.167:0/2993951871 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfe4031600 con 0x7fbfec0b50a0 2024-05-11T12:14:00.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.450+0000 7fbfebe00640 1 -- 172.21.15.167:0/2993951871 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe40319d0 con 0x7fbfec0b50a0 2024-05-11T12:14:00.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.450+0000 7fbff4400640 1 -- 172.21.15.167:0/2993951871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbfec0b50a0 msgr2=0x7fbfec0735c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:00.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.450+0000 7fbff4400640 1 --2- 172.21.15.167:0/2993951871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbfec0b50a0 0x7fbfec0735c0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fbfec0cebc0 tx=0x7fbfe402a610 comp rx=0 tx=0).stop 2024-05-11T12:14:00.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.451+0000 7fbff4400640 1 -- 172.21.15.167:0/2993951871 shutdown_connections 2024-05-11T12:14:00.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.451+0000 7fbff4400640 1 --2- 172.21.15.167:0/2993951871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbfec0b50a0 0x7fbfec0735c0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:00.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.451+0000 7fbff4400640 1 --2- 172.21.15.167:0/2993951871 >> v2:172.21.15.204:3300/0 conn(0x7fbfec13f290 0x7fbfec06f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:00.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.451+0000 7fbff4400640 1 --2- 172.21.15.167:0/2993951871 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbfec13eeb0 0x7fbfec06f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:00.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.451+0000 7fbff4400640 1 -- 172.21.15.167:0/2993951871 >> 172.21.15.167:0/2993951871 conn(0x7fbfec0cd8f0 msgr2=0x7fbfec0cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:00.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.451+0000 7fbff4400640 1 -- 172.21.15.167:0/2993951871 shutdown_connections 2024-05-11T12:14:00.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.451+0000 7fbff4400640 1 -- 172.21.15.167:0/2993951871 wait complete. 2024-05-11T12:14:00.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.452+0000 7fbff4400640 1 Processor -- start 2024-05-11T12:14:00.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.453+0000 7fbff4400640 1 -- start start 2024-05-11T12:14:00.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.453+0000 7fbff4400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbfec0b50a0 0x7fbfec14d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:00.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.454+0000 7fbff4400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbfec13eeb0 0x7fbfec0cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:00.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.454+0000 7fbff4400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbfec13f290 0x7fbfec0cf230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:00.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.454+0000 7fbff4400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbfec073c80 con 0x7fbfec13eeb0 2024-05-11T12:14:00.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.454+0000 7fbff4400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbfec073b00 con 0x7fbfec13f290 2024-05-11T12:14:00.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.454+0000 7fbff4400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbfec073e00 con 0x7fbfec0b50a0 2024-05-11T12:14:00.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.454+0000 7fbff1000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbfec13eeb0 0x7fbfec0cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:00.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.454+0000 7fbff1000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbfec13eeb0 0x7fbfec0cecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45188/0 (socket says 172.21.15.167:45188) 2024-05-11T12:14:00.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.454+0000 7fbff1000640 1 -- 172.21.15.167:0/1743153310 learned_addr learned my addr 172.21.15.167:0/1743153310 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:00.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.454+0000 7fbff1a00640 1 --2- 172.21.15.167:0/1743153310 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbfec0b50a0 0x7fbfec14d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:00.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.454+0000 7fbff2400640 1 --2- 172.21.15.167:0/1743153310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbfec13f290 0x7fbfec0cf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:00.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.454+0000 7fbff1000640 1 -- 172.21.15.167:0/1743153310 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbfec0b50a0 msgr2=0x7fbfec14d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:00.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.454+0000 7fbff1000640 1 --2- 172.21.15.167:0/1743153310 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbfec0b50a0 0x7fbfec14d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:00.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.454+0000 7fbff1000640 1 -- 172.21.15.167:0/1743153310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbfec13f290 msgr2=0x7fbfec0cf230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:00.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.454+0000 7fbff1000640 1 --2- 172.21.15.167:0/1743153310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbfec13f290 0x7fbfec0cf230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:00.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.454+0000 7fbff1000640 1 -- 172.21.15.167:0/1743153310 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfec14d5e0 con 0x7fbfec13eeb0 2024-05-11T12:14:00.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.454+0000 7fbff1a00640 1 --2- 172.21.15.167:0/1743153310 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbfec0b50a0 0x7fbfec14d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:00.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.454+0000 7fbff2400640 1 --2- 172.21.15.167:0/1743153310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbfec13f290 0x7fbfec0cf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:00.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.455+0000 7fbff1000640 1 --2- 172.21.15.167:0/1743153310 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbfec13eeb0 0x7fbfec0cecf0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fbfd800d5f0 tx=0x7fbfd800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:00.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.455+0000 7fbfe9600640 1 -- 172.21.15.167:0/1743153310 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd80125f0 con 0x7fbfec13eeb0 2024-05-11T12:14:00.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.455+0000 7fbfe9600640 1 -- 172.21.15.167:0/1743153310 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfd8019070 con 0x7fbfec13eeb0 2024-05-11T12:14:00.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.455+0000 7fbfe9600640 1 -- 172.21.15.167:0/1743153310 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd80129d0 con 0x7fbfec13eeb0 2024-05-11T12:14:00.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.455+0000 7fbff4400640 1 -- 172.21.15.167:0/1743153310 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfec150910 con 0x7fbfec13eeb0 2024-05-11T12:14:00.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.455+0000 7fbff4400640 1 -- 172.21.15.167:0/1743153310 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfec15cd40 con 0x7fbfec13eeb0 2024-05-11T12:14:00.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.456+0000 7fbff4400640 1 -- 172.21.15.167:0/1743153310 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfec15c850 con 0x7fbfec13eeb0 2024-05-11T12:14:00.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.457+0000 7fbfe9600640 1 -- 172.21.15.167:0/1743153310 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbfd8012b70 con 0x7fbfec13eeb0 2024-05-11T12:14:00.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.459+0000 7fbfe9600640 1 --2- 172.21.15.167:0/1743153310 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbfcc0762d0 0x7fbfcc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:00.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.459+0000 7fbfe9600640 1 -- 172.21.15.167:0/1743153310 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5335+0+0 (secure 0 0 0) 0x7fbfd8097920 con 0x7fbfec13eeb0 2024-05-11T12:14:00.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.459+0000 7fbff1a00640 1 --2- 172.21.15.167:0/1743153310 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbfcc0762d0 0x7fbfcc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:00.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.460+0000 7fbff1a00640 1 --2- 172.21.15.167:0/1743153310 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbfcc0762d0 0x7fbfcc078790 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0006fd0 tx=0x7fbfe0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:00.464 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.464+0000 7fbfe9600640 1 -- 172.21.15.167:0/1743153310 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fbfec14d5e0 con 0x7fbfec13eeb0 2024-05-11T12:14:00.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.712+0000 7fbff4400640 1 -- 172.21.15.167:0/1743153310 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbfec069590 con 0x7fbfec13eeb0 2024-05-11T12:14:00.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.713+0000 7fbfe9600640 1 -- 172.21.15.167:0/1743153310 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+3878 (secure 0 0 0) 0x7fbfd80612c0 con 0x7fbfec13eeb0 2024-05-11T12:14:00.714 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:00.714 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":46,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5020,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3075930607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3075930607},{"type":"v1","addr":"172.21.15.167:6836","nonce":3075930607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":5032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3056051531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3056051531},{"type":"v1","addr":"172.21.15.204:6834","nonce":3056051531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5035,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2900478909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":2900478909},{"type":"v1","addr":"172.21.15.204:6835","nonce":2900478909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6210,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/322761472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":322761472},{"type":"v1","addr":"172.21.15.167:6837","nonce":322761472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46}],"filesystems":[]} 2024-05-11T12:14:00.714 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 46 2024-05-11T12:14:00.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.716+0000 7fbff4400640 1 -- 172.21.15.167:0/1743153310 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbfcc0762d0 msgr2=0x7fbfcc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:00.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.716+0000 7fbff4400640 1 --2- 172.21.15.167:0/1743153310 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbfcc0762d0 0x7fbfcc078790 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0006fd0 tx=0x7fbfe0008040 comp rx=0 tx=0).stop 2024-05-11T12:14:00.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.716+0000 7fbff4400640 1 -- 172.21.15.167:0/1743153310 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbfec13eeb0 msgr2=0x7fbfec0cecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:00.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.716+0000 7fbff4400640 1 --2- 172.21.15.167:0/1743153310 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbfec13eeb0 0x7fbfec0cecf0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fbfd800d5f0 tx=0x7fbfd800dab0 comp rx=0 tx=0).stop 2024-05-11T12:14:00.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.716+0000 7fbff4400640 1 -- 172.21.15.167:0/1743153310 shutdown_connections 2024-05-11T12:14:00.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.716+0000 7fbff4400640 1 --2- 172.21.15.167:0/1743153310 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbfcc0762d0 0x7fbfcc078790 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:00.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.716+0000 7fbff4400640 1 --2- 172.21.15.167:0/1743153310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbfec13f290 0x7fbfec0cf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:00.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.716+0000 7fbff4400640 1 --2- 172.21.15.167:0/1743153310 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbfec13eeb0 0x7fbfec0cecf0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:00.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.716+0000 7fbff4400640 1 --2- 172.21.15.167:0/1743153310 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbfec0b50a0 0x7fbfec14d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:00.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.716+0000 7fbff4400640 1 -- 172.21.15.167:0/1743153310 >> 172.21.15.167:0/1743153310 conn(0x7fbfec0cd8f0 msgr2=0x7fbfec0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:00.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.716+0000 7fbff4400640 1 -- 172.21.15.167:0/1743153310 shutdown_connections 2024-05-11T12:14:00.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.716+0000 7fbff4400640 1 -- 172.21.15.167:0/1743153310 wait complete. 2024-05-11T12:14:00.735 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-11T12:14:00.735 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-11T12:14:00.735 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-11T12:14:00.735 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T12:14:00.735 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-11T12:14:00.735 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-11T12:14:00.735 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-11T12:14:00.736 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T12:14:00.736 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-11T12:14:00.736 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-11T12:14:00.736 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-11T12:14:00.736 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T12:14:00.736 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-11T12:14:00.736 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-11T12:14:00.737 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-11T12:14:00.737 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T12:14:00.756 INFO:tasks.ceph.mds.c:Stopped 2024-05-11T12:14:00.756 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-11T12:14:00.757 INFO:tasks.ceph.mds.a:Stopped 2024-05-11T12:14:00.757 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-11T12:14:00.758 INFO:tasks.ceph.mds.b:Stopped 2024-05-11T12:14:00.758 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-11T12:14:00.760 INFO:tasks.ceph.mds.d:Stopped 2024-05-11T12:14:00.760 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-11T12:14:00.804 INFO:tasks.ceph.mds.c:Started 2024-05-11T12:14:00.804 DEBUG:teuthology.parallel:result is None 2024-05-11T12:14:00.807 INFO:tasks.ceph.mds.b:Started 2024-05-11T12:14:00.807 DEBUG:teuthology.parallel:result is None 2024-05-11T12:14:00.810 INFO:tasks.ceph.mds.a:Started 2024-05-11T12:14:00.810 DEBUG:teuthology.parallel:result is None 2024-05-11T12:14:00.813 INFO:tasks.ceph.mds.d:Started 2024-05-11T12:14:00.813 DEBUG:teuthology.parallel:result is None 2024-05-11T12:14:00.814 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-11T12:14:00.953 INFO:tasks.ceph.mds.c.smithi167.stdout:starting mds.c at 2024-05-11T12:14:00.954 INFO:tasks.ceph.mds.a.smithi167.stdout:starting mds.a at 2024-05-11T12:14:00.966 INFO:tasks.ceph.mds.b.smithi204.stdout:starting mds.b at 2024-05-11T12:14:00.968 INFO:tasks.ceph.mds.d.smithi204.stdout:starting mds.d at 2024-05-11T12:14:00.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.997+0000 7ff4b0200640 1 Processor -- start 2024-05-11T12:14:00.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.997+0000 7ff4b0200640 1 -- start start 2024-05-11T12:14:00.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.998+0000 7ff4b0200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff4a81462f0 0x7ff4a80b6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:00.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.998+0000 7ff4b0200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff4a8161930 0x7ff4a80d2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:00.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.998+0000 7ff4b0200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff4a80b7280 0x7ff4a80d33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:00.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.998+0000 7ff4b0200640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7ff4a80d1290 con 0x7ff4a80b7280 2024-05-11T12:14:00.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.998+0000 7ff4b0200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7ff4a80b7740 con 0x7ff4a81462f0 2024-05-11T12:14:00.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.998+0000 7ff4b0200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff4a80b7a20 con 0x7ff4a8161930 2024-05-11T12:14:00.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.998+0000 7ff4ae200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff4a80b7280 0x7ff4a80d33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:01.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.998+0000 7ff4ae200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff4a80b7280 0x7ff4a80d33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45216/0 (socket says 172.21.15.167:45216) 2024-05-11T12:14:01.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.999+0000 7ff4ae200640 1 -- 172.21.15.167:0/429513789 learned_addr learned my addr 172.21.15.167:0/429513789 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:01.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.998+0000 7ff4ad800640 1 --2- 172.21.15.167:0/429513789 >> v2:172.21.15.204:3300/0 conn(0x7ff4a81462f0 0x7ff4a80b6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:01.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.999+0000 7ff4ace00640 1 --2- 172.21.15.167:0/429513789 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff4a8161930 0x7ff4a80d2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:01.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.999+0000 7ff4ae200640 1 -- 172.21.15.167:0/429513789 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff4a8161930 msgr2=0x7ff4a80d2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:01.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.999+0000 7ff4ae200640 1 --2- 172.21.15.167:0/429513789 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff4a8161930 0x7ff4a80d2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:01.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.999+0000 7ff4ae200640 1 -- 172.21.15.167:0/429513789 >> v2:172.21.15.204:3300/0 conn(0x7ff4a81462f0 msgr2=0x7ff4a80b6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:01.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.999+0000 7ff4ae200640 1 --2- 172.21.15.167:0/429513789 >> v2:172.21.15.204:3300/0 conn(0x7ff4a81462f0 0x7ff4a80b6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:01.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:00.999+0000 7ff4ae200640 1 -- 172.21.15.167:0/429513789 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4a80b7d00 con 0x7ff4a80b7280 2024-05-11T12:14:01.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.000+0000 7ff4ae200640 1 --2- 172.21.15.167:0/429513789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4a80b7280 0x7ff4a80d33c0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7ff4a80d3d80 tx=0x7ff4a002a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4cb1bfa720ab0ac server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:01.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.000+0000 7ff4a7e00640 1 -- 172.21.15.167:0/429513789 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0036070 con 0x7ff4a80b7280 2024-05-11T12:14:01.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.000+0000 7ff4a7e00640 1 -- 172.21.15.167:0/429513789 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4a0031600 con 0x7ff4a80b7280 2024-05-11T12:14:01.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.000+0000 7ff4a7e00640 1 -- 172.21.15.167:0/429513789 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a00319d0 con 0x7ff4a80b7280 2024-05-11T12:14:01.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.001+0000 7ff4b0200640 1 -- 172.21.15.167:0/429513789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4a80b7280 msgr2=0x7ff4a80d33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:01.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.001+0000 7ff4b0200640 1 --2- 172.21.15.167:0/429513789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4a80b7280 0x7ff4a80d33c0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7ff4a80d3d80 tx=0x7ff4a002a690 comp rx=0 tx=0).stop 2024-05-11T12:14:01.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.001+0000 7ff4b0200640 1 -- 172.21.15.167:0/429513789 shutdown_connections 2024-05-11T12:14:01.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.001+0000 7ff4b0200640 1 --2- 172.21.15.167:0/429513789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4a80b7280 0x7ff4a80d33c0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:01.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.001+0000 7ff4b0200640 1 --2- 172.21.15.167:0/429513789 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff4a8161930 0x7ff4a80d2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:01.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.001+0000 7ff4b0200640 1 --2- 172.21.15.167:0/429513789 >> v2:172.21.15.204:3300/0 conn(0x7ff4a81462f0 0x7ff4a80b6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:01.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.001+0000 7ff4b0200640 1 -- 172.21.15.167:0/429513789 >> 172.21.15.167:0/429513789 conn(0x7ff4a80b8300 msgr2=0x7ff4a80b86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:01.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.001+0000 7ff4b0200640 1 -- 172.21.15.167:0/429513789 shutdown_connections 2024-05-11T12:14:01.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.001+0000 7ff4b0200640 1 -- 172.21.15.167:0/429513789 wait complete. 2024-05-11T12:14:01.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.003+0000 7ff4b0200640 1 Processor -- start 2024-05-11T12:14:01.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.003+0000 7ff4b0200640 1 -- start start 2024-05-11T12:14:01.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.004+0000 7ff4b0200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff4a80b7280 0x7ff4a815ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:01.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.004+0000 7ff4b0200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4a81462f0 0x7ff4a813fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:01.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.004+0000 7ff4b0200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4a8161930 0x7ff4a8140190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:01.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.004+0000 7ff4b0200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a8146850 con 0x7ff4a8161930 2024-05-11T12:14:01.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.004+0000 7ff4b0200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a81466d0 con 0x7ff4a81462f0 2024-05-11T12:14:01.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.004+0000 7ff4b0200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff4a8070b30 con 0x7ff4a80b7280 2024-05-11T12:14:01.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.004+0000 7ff4ae200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4a8161930 0x7ff4a8140190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:01.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.004+0000 7ff4ae200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4a8161930 0x7ff4a8140190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45226/0 (socket says 172.21.15.167:45226) 2024-05-11T12:14:01.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.004+0000 7ff4ae200640 1 -- 172.21.15.167:0/2890684808 learned_addr learned my addr 172.21.15.167:0/2890684808 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:01.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.004+0000 7ff4ace00640 1 --2- 172.21.15.167:0/2890684808 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4a81462f0 0x7ff4a813fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:01.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.004+0000 7ff4ad800640 1 --2- 172.21.15.167:0/2890684808 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff4a80b7280 0x7ff4a815ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:01.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.005+0000 7ff4ae200640 1 -- 172.21.15.167:0/2890684808 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff4a80b7280 msgr2=0x7ff4a815ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:01.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.005+0000 7ff4ae200640 1 --2- 172.21.15.167:0/2890684808 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff4a80b7280 0x7ff4a815ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:01.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.005+0000 7ff4ae200640 1 -- 172.21.15.167:0/2890684808 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4a81462f0 msgr2=0x7ff4a813fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:01.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.005+0000 7ff4ae200640 1 --2- 172.21.15.167:0/2890684808 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4a81462f0 0x7ff4a813fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:01.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.005+0000 7ff4ae200640 1 -- 172.21.15.167:0/2890684808 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4a81f85e0 con 0x7ff4a8161930 2024-05-11T12:14:01.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.005+0000 7ff4ae200640 1 --2- 172.21.15.167:0/2890684808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4a8161930 0x7ff4a8140190 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7ff4a002ac80 tx=0x7ff4a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:01.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.005+0000 7ff4a5600640 1 -- 172.21.15.167:0/2890684808 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a003f020 con 0x7ff4a8161930 2024-05-11T12:14:01.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.005+0000 7ff4a5600640 1 -- 172.21.15.167:0/2890684808 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4a0037040 con 0x7ff4a8161930 2024-05-11T12:14:01.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.005+0000 7ff4a5600640 1 -- 172.21.15.167:0/2890684808 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0036070 con 0x7ff4a8161930 2024-05-11T12:14:01.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.005+0000 7ff4b0200640 1 -- 172.21.15.167:0/2890684808 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4a81f8880 con 0x7ff4a8161930 2024-05-11T12:14:01.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.006+0000 7ff4b0200640 1 -- 172.21.15.167:0/2890684808 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4a81f8e40 con 0x7ff4a8161930 2024-05-11T12:14:01.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.007+0000 7ff4b0200640 1 -- 172.21.15.167:0/2890684808 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4a81f9040 con 0x7ff4a8161930 2024-05-11T12:14:01.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.008+0000 7ff4a5600640 1 -- 172.21.15.167:0/2890684808 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff4a0042020 con 0x7ff4a8161930 2024-05-11T12:14:01.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.009+0000 7ff4a5600640 1 --2- 172.21.15.167:0/2890684808 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff4880762d0 0x7ff488078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:01.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.009+0000 7ff4a5600640 1 -- 172.21.15.167:0/2890684808 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5335+0+0 (secure 0 0 0) 0x7ff4a00b7670 con 0x7ff4a8161930 2024-05-11T12:14:01.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.009+0000 7ff4ad800640 1 --2- 172.21.15.167:0/2890684808 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff4880762d0 0x7ff488078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:01.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.010+0000 7ff4ad800640 1 --2- 172.21.15.167:0/2890684808 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff4880762d0 0x7ff488078790 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff4a81454e0 tx=0x7ff49c009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:01.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.013+0000 7ff4a5600640 1 -- 172.21.15.167:0/2890684808 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff4a81f9040 con 0x7ff4a8161930 2024-05-11T12:14:01.232 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.232+0000 7ff4b0200640 1 -- 172.21.15.167:0/2890684808 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7ff4a80694b0 con 0x7ff4a8161930 2024-05-11T12:14:01.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.353+0000 7ff4a5600640 1 -- 172.21.15.167:0/2890684808 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v80) ==== 97+0+0 (secure 0 0 0) 0x7ff4a0049030 con 0x7ff4a8161930 2024-05-11T12:14:01.353 INFO:teuthology.orchestra.run.smithi167.stderr: removed all blocklist entries 2024-05-11T12:14:01.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.356+0000 7ff4b0200640 1 -- 172.21.15.167:0/2890684808 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff4880762d0 msgr2=0x7ff488078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:01.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.356+0000 7ff4b0200640 1 --2- 172.21.15.167:0/2890684808 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff4880762d0 0x7ff488078790 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff4a81454e0 tx=0x7ff49c009190 comp rx=0 tx=0).stop 2024-05-11T12:14:01.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.356+0000 7ff4b0200640 1 -- 172.21.15.167:0/2890684808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4a8161930 msgr2=0x7ff4a8140190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:01.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.356+0000 7ff4b0200640 1 --2- 172.21.15.167:0/2890684808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4a8161930 0x7ff4a8140190 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7ff4a002ac80 tx=0x7ff4a0004a40 comp rx=0 tx=0).stop 2024-05-11T12:14:01.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.356+0000 7ff4b0200640 1 -- 172.21.15.167:0/2890684808 shutdown_connections 2024-05-11T12:14:01.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.356+0000 7ff4b0200640 1 --2- 172.21.15.167:0/2890684808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4a8161930 0x7ff4a8140190 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:01.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.356+0000 7ff4b0200640 1 --2- 172.21.15.167:0/2890684808 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff4880762d0 0x7ff488078790 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:01.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.356+0000 7ff4b0200640 1 --2- 172.21.15.167:0/2890684808 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4a81462f0 0x7ff4a813fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:01.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.356+0000 7ff4b0200640 1 --2- 172.21.15.167:0/2890684808 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff4a80b7280 0x7ff4a815ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:01.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.356+0000 7ff4b0200640 1 -- 172.21.15.167:0/2890684808 >> 172.21.15.167:0/2890684808 conn(0x7ff4a80b8300 msgr2=0x7ff4a80d1690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:01.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.357+0000 7ff4b0200640 1 -- 172.21.15.167:0/2890684808 shutdown_connections 2024-05-11T12:14:01.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.357+0000 7ff4b0200640 1 -- 172.21.15.167:0/2890684808 wait complete. 2024-05-11T12:14:01.379 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-11T12:14:01.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.573+0000 7f978b200640 1 Processor -- start 2024-05-11T12:14:01.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.573+0000 7f978b200640 1 -- start start 2024-05-11T12:14:01.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.574+0000 7f978b200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f978c145b00 0x7f978c06f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:01.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.575+0000 7f978b200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f978c06c660 0x7f978c06ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:01.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.575+0000 7f978b200640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f978c06cfb0 con 0x7f978c146770 2024-05-11T12:14:01.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.575+0000 7f978b200640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f978c06fbc0 con 0x7f978c06c660 2024-05-11T12:14:01.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.575+0000 7f978b200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f978c0b4520 con 0x7f978c145b00 2024-05-11T12:14:01.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.575+0000 7f9781400640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f978c146770 0x7f978c0ce9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:40356/0 (socket says 172.21.15.167:40356) 2024-05-11T12:14:01.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.575+0000 7f9781400640 1 -- 172.21.15.167:0/3041693886 learned_addr learned my addr 172.21.15.167:0/3041693886 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:14:01.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.575+0000 7f978a800640 1 --2- 172.21.15.167:0/3041693886 >> v2:172.21.15.167:3300/0 conn(0x7f978c06c660 0x7f978c06ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:01.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.575+0000 7f9789e00640 1 --2- 172.21.15.167:0/3041693886 >> v2:172.21.15.204:3300/0 conn(0x7f978c145b00 0x7f978c06f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:01.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.576+0000 7f9789400640 1 -- 172.21.15.167:0/3041693886 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3964373512 0 0) 0x7f9774001110 con 0x7f978c146770 2024-05-11T12:14:01.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.576+0000 7f9789400640 1 -- 172.21.15.167:0/3041693886 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9770003520 con 0x7f978c146770 2024-05-11T12:14:01.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.576+0000 7f978a800640 1 -- 172.21.15.167:0/3041693886 >> v2:172.21.15.204:3300/0 conn(0x7f978c145b00 msgr2=0x7f978c06f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:01.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.576+0000 7f978a800640 1 --2- 172.21.15.167:0/3041693886 >> v2:172.21.15.204:3300/0 conn(0x7f978c145b00 0x7f978c06f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:01.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.576+0000 7f978a800640 1 -- 172.21.15.167:0/3041693886 >> v1:172.21.15.167:6789/0 conn(0x7f978c146770 legacy=0x7f978c0ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:01.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.576+0000 7f978a800640 1 -- 172.21.15.167:0/3041693886 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f978c0b4800 con 0x7f978c06c660 2024-05-11T12:14:01.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.577+0000 7f978a800640 1 --2- 172.21.15.167:0/3041693886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f978c06c660 0x7f978c06ca70 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f978c0701e0 tx=0x7f978402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f62f86337da3b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:01.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.577+0000 7f9789400640 1 -- 172.21.15.167:0/3041693886 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9784007740 con 0x7f978c06c660 2024-05-11T12:14:01.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.577+0000 7f9789400640 1 -- 172.21.15.167:0/3041693886 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97840078e0 con 0x7f978c06c660 2024-05-11T12:14:01.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.577+0000 7f9789400640 1 -- 172.21.15.167:0/3041693886 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9784007cb0 con 0x7f978c06c660 2024-05-11T12:14:01.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.578+0000 7f978b200640 1 -- 172.21.15.167:0/3041693886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f978c06c660 msgr2=0x7f978c06ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:01.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.578+0000 7f978b200640 1 --2- 172.21.15.167:0/3041693886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f978c06c660 0x7f978c06ca70 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f978c0701e0 tx=0x7f978402a5f0 comp rx=0 tx=0).stop 2024-05-11T12:14:01.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.578+0000 7f978b200640 1 -- 172.21.15.167:0/3041693886 shutdown_connections 2024-05-11T12:14:01.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.578+0000 7f978b200640 1 --2- 172.21.15.167:0/3041693886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f978c06c660 0x7f978c06ca70 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:01.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.578+0000 7f978b200640 1 --2- 172.21.15.167:0/3041693886 >> v2:172.21.15.204:3300/0 conn(0x7f978c145b00 0x7f978c06f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:01.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.578+0000 7f978b200640 1 -- 172.21.15.167:0/3041693886 >> 172.21.15.167:0/3041693886 conn(0x7f978c0b3460 msgr2=0x7f978c0d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:01.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.578+0000 7f978b200640 1 -- 172.21.15.167:0/3041693886 shutdown_connections 2024-05-11T12:14:01.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.578+0000 7f978b200640 1 -- 172.21.15.167:0/3041693886 wait complete. 2024-05-11T12:14:01.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.579+0000 7f978b200640 1 Processor -- start 2024-05-11T12:14:01.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.580+0000 7f978b200640 1 -- start start 2024-05-11T12:14:01.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.580+0000 7f978b200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f978c06c660 0x7f978c140e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:01.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.580+0000 7f978b200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f978c145b00 0x7f978c0ab770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:01.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.581+0000 7f978b200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f978c146770 0x7f978c1f65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:01.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.581+0000 7f978b200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f978c145ee0 con 0x7f978c146770 2024-05-11T12:14:01.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.581+0000 7f978b200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f978c0cf0e0 con 0x7f978c06c660 2024-05-11T12:14:01.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.581+0000 7f978b200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f978c06f800 con 0x7f978c145b00 2024-05-11T12:14:01.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.581+0000 7f978a800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f978c146770 0x7f978c1f65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:01.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.581+0000 7f978a800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f978c146770 0x7f978c1f65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45246/0 (socket says 172.21.15.167:45246) 2024-05-11T12:14:01.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.581+0000 7f978a800640 1 -- 172.21.15.167:0/988820472 learned_addr learned my addr 172.21.15.167:0/988820472 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:01.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.581+0000 7f9789e00640 1 --2- 172.21.15.167:0/988820472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f978c06c660 0x7f978c140e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:01.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.581+0000 7f978a800640 1 -- 172.21.15.167:0/988820472 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f978c145b00 msgr2=0x7f978c0ab770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:01.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.581+0000 7f978a800640 1 --2- 172.21.15.167:0/988820472 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f978c145b00 0x7f978c0ab770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:01.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.581+0000 7f978a800640 1 -- 172.21.15.167:0/988820472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f978c06c660 msgr2=0x7f978c140e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:01.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.581+0000 7f978a800640 1 --2- 172.21.15.167:0/988820472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f978c06c660 0x7f978c140e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:01.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.581+0000 7f978a800640 1 -- 172.21.15.167:0/988820472 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f978c1413a0 con 0x7f978c146770 2024-05-11T12:14:01.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.582+0000 7f978a800640 1 --2- 172.21.15.167:0/988820472 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f978c146770 0x7f978c1f65d0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f978402abe0 tx=0x7f9784004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:01.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.582+0000 7f9782a00640 1 -- 172.21.15.167:0/988820472 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9784040070 con 0x7f978c146770 2024-05-11T12:14:01.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.582+0000 7f9782a00640 1 -- 172.21.15.167:0/988820472 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9784031bc0 con 0x7f978c146770 2024-05-11T12:14:01.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.582+0000 7f978b200640 1 -- 172.21.15.167:0/988820472 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f978c0ac1f0 con 0x7f978c146770 2024-05-11T12:14:01.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.582+0000 7f9782a00640 1 -- 172.21.15.167:0/988820472 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9784009450 con 0x7f978c146770 2024-05-11T12:14:01.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.582+0000 7f978b200640 1 -- 172.21.15.167:0/988820472 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f978c0ac7b0 con 0x7f978c146770 2024-05-11T12:14:01.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.583+0000 7f978b200640 1 -- 172.21.15.167:0/988820472 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f978c0694b0 con 0x7f978c146770 2024-05-11T12:14:01.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.584+0000 7f9782a00640 1 -- 172.21.15.167:0/988820472 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9784039040 con 0x7f978c146770 2024-05-11T12:14:01.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.586+0000 7f9782a00640 1 --2- 172.21.15.167:0/988820472 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f97680762d0 0x7f9768078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:01.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.587+0000 7f9782a00640 1 -- 172.21.15.167:0/988820472 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5249+0+0 (secure 0 0 0) 0x7f97840b8ca0 con 0x7f978c146770 2024-05-11T12:14:01.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.587+0000 7f9789e00640 1 --2- 172.21.15.167:0/988820472 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f97680762d0 0x7f9768078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:01.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.587+0000 7f9789e00640 1 --2- 172.21.15.167:0/988820472 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f97680762d0 0x7f9768078790 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f977c006fd0 tx=0x7f977c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:01.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.591+0000 7f9782a00640 1 -- 172.21.15.167:0/988820472 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f97840827b0 con 0x7f978c146770 2024-05-11T12:14:01.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.874+0000 7f978b200640 1 -- 172.21.15.167:0/988820472 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f978c15f840 con 0x7f978c146770 2024-05-11T12:14:01.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.875+0000 7f9782a00640 1 -- 172.21.15.167:0/988820472 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) ==== 80+0+5304 (secure 0 0 0) 0x7f97840820b0 con 0x7f978c146770 2024-05-11T12:14:01.875 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:01.876 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T12:14:01.876 INFO:teuthology.orchestra.run.smithi167.stdout: "auth_dump": [ 2024-05-11T12:14:01.876 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:14:01.876 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "mds.a", 2024-05-11T12:14:01.876 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAEYD9msMGnFBAAamqokXQ+znxPHtHhhs611g==", 2024-05-11T12:14:01.876 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:14:01.876 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T12:14:01.876 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow *", 2024-05-11T12:14:01.876 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow *", 2024-05-11T12:14:01.876 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:14:01.876 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:14:01.876 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:14:01.876 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:14:01.876 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "mds.b", 2024-05-11T12:14:01.877 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAEYD9mqbCgMhAA5CNUf1cz77nUqLli2bTFcQ==", 2024-05-11T12:14:01.877 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:14:01.877 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T12:14:01.877 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow *", 2024-05-11T12:14:01.877 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow *", 2024-05-11T12:14:01.877 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:14:01.877 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:14:01.877 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:14:01.877 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:14:01.877 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "mds.c", 2024-05-11T12:14:01.877 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAEYD9mhylUIxAAqrIRYuNIHKfoI00zx+Gh1Q==", 2024-05-11T12:14:01.877 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:14:01.877 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T12:14:01.877 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow *", 2024-05-11T12:14:01.877 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow *", 2024-05-11T12:14:01.878 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:14:01.878 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:14:01.878 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:14:01.878 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:14:01.878 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "mds.d", 2024-05-11T12:14:01.878 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAFYD9mcUjpBRAAbx54iDHcqexfNXGKk/avlw==", 2024-05-11T12:14:01.878 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:14:01.878 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T12:14:01.878 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow *", 2024-05-11T12:14:01.878 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow *", 2024-05-11T12:14:01.878 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:14:01.878 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:14:01.878 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:14:01.878 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:14:01.879 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.0", 2024-05-11T12:14:01.879 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAJYD9mb71mBxAAKMSzkfppZVbEevwELKrqvw==", 2024-05-11T12:14:01.879 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:14:01.879 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T12:14:01.879 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T12:14:01.879 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:14:01.879 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:14:01.879 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:14:01.879 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:14:01.879 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.1", 2024-05-11T12:14:01.879 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAMYD9m/qx7EhAAUcZntottFNBwqjdc/CqFbw==", 2024-05-11T12:14:01.879 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:14:01.879 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T12:14:01.879 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T12:14:01.880 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:14:01.880 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:14:01.880 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:14:01.880 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:14:01.880 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.2", 2024-05-11T12:14:01.880 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAPYD9m5PKRKxAAFQuzh7dAN8PVEHBq+UdQvg==", 2024-05-11T12:14:01.880 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:14:01.880 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T12:14:01.880 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T12:14:01.880 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:14:01.880 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:14:01.880 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:14:01.880 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:14:01.880 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.3", 2024-05-11T12:14:01.881 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQASYD9mZAy+NhAA49k6QbSvtSxA/5t8bWbPuQ==", 2024-05-11T12:14:01.881 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:14:01.881 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T12:14:01.881 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T12:14:01.881 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:14:01.881 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:14:01.881 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:14:01.881 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:14:01.881 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.4", 2024-05-11T12:14:01.881 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAZYD9mEbsvIBAA3nkQXQTbIf7ubq1SJtUeIA==", 2024-05-11T12:14:01.881 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:14:01.881 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T12:14:01.881 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T12:14:01.881 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:14:01.882 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:14:01.882 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:14:01.882 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:14:01.882 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.5", 2024-05-11T12:14:01.882 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAcYD9mtdsHOxAAvNUSkMgQlDc2n1ak/lfRgg==", 2024-05-11T12:14:01.882 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:14:01.882 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T12:14:01.882 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T12:14:01.882 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:14:01.882 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:14:01.882 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:14:01.882 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:14:01.882 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.6", 2024-05-11T12:14:01.882 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAgYD9mr3u8CRAAm2ShwV0lJpIVudodmviusg==", 2024-05-11T12:14:01.883 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:14:01.883 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T12:14:01.883 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T12:14:01.883 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:14:01.883 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:14:01.883 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:14:01.883 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:14:01.883 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.7", 2024-05-11T12:14:01.883 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAjYD9m+0oyFRAAUVQW0kQmvtU8IE599nrWJQ==", 2024-05-11T12:14:01.883 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:14:01.883 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T12:14:01.883 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T12:14:01.883 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:14:01.883 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:14:01.883 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:14:01.884 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:14:01.884 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "client.0", 2024-05-11T12:14:01.884 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAFYD9mLK/lEhAAyERTYMl250KkBvIrAXWYSw==", 2024-05-11T12:14:01.884 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:14:01.884 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T12:14:01.884 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow r", 2024-05-11T12:14:01.884 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-11T12:14:01.884 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:14:01.884 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:14:01.884 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:14:01.884 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "client.1", 2024-05-11T12:14:01.884 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAFYD9m2oSjJhAAz53H1SvPStkxGq5V+2VFeA==", 2024-05-11T12:14:01.884 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:14:01.884 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T12:14:01.884 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow r", 2024-05-11T12:14:01.884 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow rw", 2024-05-11T12:14:01.885 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow rwx" 2024-05-11T12:14:01.885 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:14:01.885 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:14:01.885 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:14:01.885 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "client.2", 2024-05-11T12:14:01.885 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAFYD9mMzXnHBAAmJ//hwSofaZpk+6ykxVM/A==", 2024-05-11T12:14:01.885 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:14:01.885 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T12:14:01.885 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow r", 2024-05-11T12:14:01.885 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow rw", 2024-05-11T12:14:01.885 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow rwx" 2024-05-11T12:14:01.885 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:14:01.885 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:14:01.885 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:14:01.885 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "client.3", 2024-05-11T12:14:01.885 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAFYD9muOmsLxAA/tYFT4YNy3adENgVNLljdw==", 2024-05-11T12:14:01.886 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:14:01.886 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T12:14:01.886 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow r", 2024-05-11T12:14:01.886 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow rw", 2024-05-11T12:14:01.886 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow rwx" 2024-05-11T12:14:01.886 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:14:01.886 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:14:01.886 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:14:01.886 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "client.admin", 2024-05-11T12:14:01.886 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQADYD9mItegDBAALtuDP6qzAvbrwiROf94biw==", 2024-05-11T12:14:01.886 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:14:01.886 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow *", 2024-05-11T12:14:01.886 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow *", 2024-05-11T12:14:01.886 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow *", 2024-05-11T12:14:01.887 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:14:01.887 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:14:01.887 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:14:01.887 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:14:01.887 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "mgr.x", 2024-05-11T12:14:01.887 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAEYD9mAZTKCRAAJbY821O+4pUMC0DgxRa3og==", 2024-05-11T12:14:01.887 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:14:01.887 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow *", 2024-05-11T12:14:01.887 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile mgr", 2024-05-11T12:14:01.887 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:14:01.887 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:14:01.887 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:14:01.887 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:14:01.887 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "mgr.y", 2024-05-11T12:14:01.887 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQADYD9m2cZRORAAX9kYcT5/t4f5jQRZwXHHzw==", 2024-05-11T12:14:01.887 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:14:01.888 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow *", 2024-05-11T12:14:01.888 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile mgr", 2024-05-11T12:14:01.888 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:14:01.888 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:14:01.888 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:14:01.888 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-05-11T12:14:01.888 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T12:14:01.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.878+0000 7f978b200640 1 -- 172.21.15.167:0/988820472 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f97680762d0 msgr2=0x7f9768078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:01.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.878+0000 7f978b200640 1 --2- 172.21.15.167:0/988820472 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f97680762d0 0x7f9768078790 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f977c006fd0 tx=0x7f977c008040 comp rx=0 tx=0).stop 2024-05-11T12:14:01.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.878+0000 7f978b200640 1 -- 172.21.15.167:0/988820472 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f978c146770 msgr2=0x7f978c1f65d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:01.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.878+0000 7f978b200640 1 --2- 172.21.15.167:0/988820472 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f978c146770 0x7f978c1f65d0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f978402abe0 tx=0x7f9784004a40 comp rx=0 tx=0).stop 2024-05-11T12:14:01.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.878+0000 7f978b200640 1 -- 172.21.15.167:0/988820472 shutdown_connections 2024-05-11T12:14:01.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.878+0000 7f978b200640 1 --2- 172.21.15.167:0/988820472 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f978c146770 0x7f978c1f65d0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:01.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.878+0000 7f978b200640 1 --2- 172.21.15.167:0/988820472 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f978c145b00 0x7f978c0ab770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:01.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.878+0000 7f978b200640 1 --2- 172.21.15.167:0/988820472 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f97680762d0 0x7f9768078790 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:01.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.878+0000 7f978b200640 1 --2- 172.21.15.167:0/988820472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f978c06c660 0x7f978c140e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:01.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.878+0000 7f978b200640 1 -- 172.21.15.167:0/988820472 >> 172.21.15.167:0/988820472 conn(0x7f978c0b3460 msgr2=0x7f978c0ce9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:01.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.879+0000 7f978b200640 1 -- 172.21.15.167:0/988820472 shutdown_connections 2024-05-11T12:14:01.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:01.879+0000 7f978b200640 1 -- 172.21.15.167:0/988820472 wait complete. 2024-05-11T12:14:01.897 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-11T12:14:02.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.085+0000 7f012d800640 1 Processor -- start 2024-05-11T12:14:02.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.085+0000 7f012d800640 1 -- start start 2024-05-11T12:14:02.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.086+0000 7f012d800640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f012815f460 0x7f01280b83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:02.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.087+0000 7f012d800640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01280cf990 con 0x7f01280b88e0 2024-05-11T12:14:02.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.087+0000 7f012d800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01280cfc70 con 0x7f01280d42c0 2024-05-11T12:14:02.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.087+0000 7f012d800640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f01280cff50 con 0x7f012815f460 2024-05-11T12:14:02.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.087+0000 7f0127e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f012815f460 0x7f01280b83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:02.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.087+0000 7f0127400640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f01280b88e0 0x7f01280d3bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:40368/0 (socket says 172.21.15.167:40368) 2024-05-11T12:14:02.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.087+0000 7f0127400640 1 -- 172.21.15.167:0/1677591969 learned_addr learned my addr 172.21.15.167:0/1677591969 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:14:02.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.087+0000 7f0126a00640 1 -- 172.21.15.167:0/1677591969 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 461109139 0 0) 0x7f01280cf990 con 0x7f01280b88e0 2024-05-11T12:14:02.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.087+0000 7f0126a00640 1 -- 172.21.15.167:0/1677591969 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f010c003520 con 0x7f01280b88e0 2024-05-11T12:14:02.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.088+0000 7f0127e00640 1 -- 172.21.15.167:0/1677591969 >> v1:172.21.15.204:6789/0 conn(0x7f01280d42c0 legacy=0x7f01280d0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:02.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.088+0000 7f0127e00640 1 -- 172.21.15.167:0/1677591969 >> v1:172.21.15.167:6789/0 conn(0x7f01280b88e0 legacy=0x7f01280d3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:02.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.088+0000 7f0127e00640 1 -- 172.21.15.167:0/1677591969 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01280d0230 con 0x7f012815f460 2024-05-11T12:14:02.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.089+0000 7f0127e00640 1 --2- 172.21.15.167:0/1677591969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f012815f460 0x7f01280b83a0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f01180097a0 tx=0x7f011802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f3430c5da0d258c server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:02.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.089+0000 7f0126a00640 1 -- 172.21.15.167:0/1677591969 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0118037070 con 0x7f012815f460 2024-05-11T12:14:02.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.089+0000 7f0126a00640 1 -- 172.21.15.167:0/1677591969 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0118032440 con 0x7f012815f460 2024-05-11T12:14:02.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.089+0000 7f0126a00640 1 -- 172.21.15.167:0/1677591969 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01180327f0 con 0x7f012815f460 2024-05-11T12:14:02.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.089+0000 7f012d800640 1 -- 172.21.15.167:0/1677591969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f012815f460 msgr2=0x7f01280b83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:02.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.089+0000 7f012d800640 1 --2- 172.21.15.167:0/1677591969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f012815f460 0x7f01280b83a0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f01180097a0 tx=0x7f011802a180 comp rx=0 tx=0).stop 2024-05-11T12:14:02.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.090+0000 7f012d800640 1 -- 172.21.15.167:0/1677591969 shutdown_connections 2024-05-11T12:14:02.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.090+0000 7f012d800640 1 --2- 172.21.15.167:0/1677591969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f012815f460 0x7f01280b83a0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:02.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.090+0000 7f012d800640 1 -- 172.21.15.167:0/1677591969 >> 172.21.15.167:0/1677591969 conn(0x7f01280d1a50 msgr2=0x7f01280d1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:02.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.090+0000 7f012d800640 1 -- 172.21.15.167:0/1677591969 shutdown_connections 2024-05-11T12:14:02.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.090+0000 7f012d800640 1 -- 172.21.15.167:0/1677591969 wait complete. 2024-05-11T12:14:02.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.092+0000 7f012d800640 1 Processor -- start 2024-05-11T12:14:02.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.092+0000 7f012d800640 1 -- start start 2024-05-11T12:14:02.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.093+0000 7f012d800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01280b88e0 0x7f0128160920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:02.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.093+0000 7f012d800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01280d42c0 0x7f01280d35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:02.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.093+0000 7f012d800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f012815f460 0x7f0128150520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:02.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.093+0000 7f012d800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f012806cc40 con 0x7f01280b88e0 2024-05-11T12:14:02.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.093+0000 7f012d800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f01280cf620 con 0x7f01280d42c0 2024-05-11T12:14:02.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.093+0000 7f012d800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f012806cdc0 con 0x7f012815f460 2024-05-11T12:14:02.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.093+0000 7f0127e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01280b88e0 0x7f0128160920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:02.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.093+0000 7f0127e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01280b88e0 0x7f0128160920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45266/0 (socket says 172.21.15.167:45266) 2024-05-11T12:14:02.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.093+0000 7f0127e00640 1 -- 172.21.15.167:0/2107173730 learned_addr learned my addr 172.21.15.167:0/2107173730 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:02.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.093+0000 7f0127400640 1 --2- 172.21.15.167:0/2107173730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01280d42c0 0x7f01280d35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:02.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.093+0000 7f0127e00640 1 -- 172.21.15.167:0/2107173730 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f012815f460 msgr2=0x7f0128150520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:02.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.093+0000 7f0127e00640 1 --2- 172.21.15.167:0/2107173730 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f012815f460 0x7f0128150520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:02.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.093+0000 7f0127e00640 1 -- 172.21.15.167:0/2107173730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01280d42c0 msgr2=0x7f01280d35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:02.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.093+0000 7f0127e00640 1 --2- 172.21.15.167:0/2107173730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01280d42c0 0x7f01280d35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:02.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.093+0000 7f0127e00640 1 -- 172.21.15.167:0/2107173730 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0128150a60 con 0x7f01280b88e0 2024-05-11T12:14:02.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.093+0000 7f0127400640 1 --2- 172.21.15.167:0/2107173730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01280d42c0 0x7f01280d35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:02.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.094+0000 7f0127e00640 1 --2- 172.21.15.167:0/2107173730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01280b88e0 0x7f0128160920 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f0118035040 tx=0x7f01180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:02.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.094+0000 7f011fe00640 1 -- 172.21.15.167:0/2107173730 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0118037040 con 0x7f01280b88e0 2024-05-11T12:14:02.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.094+0000 7f011fe00640 1 -- 172.21.15.167:0/2107173730 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0118004230 con 0x7f01280b88e0 2024-05-11T12:14:02.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.094+0000 7f011fe00640 1 -- 172.21.15.167:0/2107173730 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0118003610 con 0x7f01280b88e0 2024-05-11T12:14:02.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.094+0000 7f012d800640 1 -- 172.21.15.167:0/2107173730 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0128160f00 con 0x7f01280b88e0 2024-05-11T12:14:02.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.094+0000 7f012d800640 1 -- 172.21.15.167:0/2107173730 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01281614c0 con 0x7f01280b88e0 2024-05-11T12:14:02.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.095+0000 7f012d800640 1 -- 172.21.15.167:0/2107173730 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01280694b0 con 0x7f01280b88e0 2024-05-11T12:14:02.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.096+0000 7f011fe00640 1 -- 172.21.15.167:0/2107173730 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f01180037b0 con 0x7f01280b88e0 2024-05-11T12:14:02.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.097+0000 7f011fe00640 1 --2- 172.21.15.167:0/2107173730 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f01080762d0 0x7f0108078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:02.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.098+0000 7f011fe00640 1 -- 172.21.15.167:0/2107173730 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5249+0+0 (secure 0 0 0) 0x7f01180b85d0 con 0x7f01280b88e0 2024-05-11T12:14:02.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.098+0000 7f0127400640 1 --2- 172.21.15.167:0/2107173730 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f01080762d0 0x7f0108078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:02.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.098+0000 7f0127400640 1 --2- 172.21.15.167:0/2107173730 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f01080762d0 0x7f0108078790 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0114001220 tx=0x7f0114009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:02.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.102+0000 7f011fe00640 1 -- 172.21.15.167:0/2107173730 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0128150a60 con 0x7f01280b88e0 2024-05-11T12:14:02.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.310+0000 7f012d800640 1 -- 172.21.15.167:0/2107173730 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f01280b5c20 con 0x7f01280b88e0 2024-05-11T12:14:02.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.311+0000 7f011fe00640 1 -- 172.21.15.167:0/2107173730 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7f0118082050 con 0x7f01280b88e0 2024-05-11T12:14:02.311 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:02.311 INFO:teuthology.orchestra.run.smithi167.stdout:[ 2024-05-11T12:14:02.311 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:14:02.311 INFO:teuthology.orchestra.run.smithi167.stdout: "poolnum": 1, 2024-05-11T12:14:02.311 INFO:teuthology.orchestra.run.smithi167.stdout: "poolname": ".mgr" 2024-05-11T12:14:02.311 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:14:02.311 INFO:teuthology.orchestra.run.smithi167.stdout:] 2024-05-11T12:14:02.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.313+0000 7f012d800640 1 -- 172.21.15.167:0/2107173730 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f01080762d0 msgr2=0x7f0108078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:02.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.313+0000 7f012d800640 1 --2- 172.21.15.167:0/2107173730 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f01080762d0 0x7f0108078790 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0114001220 tx=0x7f0114009040 comp rx=0 tx=0).stop 2024-05-11T12:14:02.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.314+0000 7f012d800640 1 -- 172.21.15.167:0/2107173730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01280b88e0 msgr2=0x7f0128160920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:02.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.314+0000 7f012d800640 1 --2- 172.21.15.167:0/2107173730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01280b88e0 0x7f0128160920 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f0118035040 tx=0x7f01180046c0 comp rx=0 tx=0).stop 2024-05-11T12:14:02.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.314+0000 7f012d800640 1 -- 172.21.15.167:0/2107173730 shutdown_connections 2024-05-11T12:14:02.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.314+0000 7f012d800640 1 --2- 172.21.15.167:0/2107173730 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f012815f460 0x7f0128150520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:02.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.314+0000 7f012d800640 1 --2- 172.21.15.167:0/2107173730 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f01080762d0 0x7f0108078790 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:02.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.314+0000 7f012d800640 1 --2- 172.21.15.167:0/2107173730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01280d42c0 0x7f01280d35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:02.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.314+0000 7f012d800640 1 --2- 172.21.15.167:0/2107173730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01280b88e0 0x7f0128160920 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:02.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.314+0000 7f012d800640 1 -- 172.21.15.167:0/2107173730 >> 172.21.15.167:0/2107173730 conn(0x7f01280d1a50 msgr2=0x7f01280ba5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:02.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.314+0000 7f012d800640 1 -- 172.21.15.167:0/2107173730 shutdown_connections 2024-05-11T12:14:02.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.314+0000 7f012d800640 1 -- 172.21.15.167:0/2107173730 wait complete. 2024-05-11T12:14:02.331 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-11T12:14:02.331 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-11T12:14:02.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.531+0000 7f624a400640 1 Processor -- start 2024-05-11T12:14:02.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.531+0000 7f624a400640 1 -- start start 2024-05-11T12:14:02.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.533+0000 7f624a400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f62441439a0 0x7f62440cfee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:02.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.533+0000 7f624a400640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62440d0420 con 0x7f62440b59f0 2024-05-11T12:14:02.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.533+0000 7f624a400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62440b3f30 con 0x7f62440cf820 2024-05-11T12:14:02.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.533+0000 7f624a400640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f62440b3ce0 con 0x7f62441439a0 2024-05-11T12:14:02.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.533+0000 7f6243400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f62441439a0 0x7f62440cfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:02.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.533+0000 7f6243e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f62440b59f0 0x7f6244073220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:40378/0 (socket says 172.21.15.167:40378) 2024-05-11T12:14:02.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.533+0000 7f6243e00640 1 -- 172.21.15.167:0/3548449518 learned_addr learned my addr 172.21.15.167:0/3548449518 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:14:02.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.534+0000 7f6243400640 1 -- 172.21.15.167:0/3548449518 >> v1:172.21.15.204:6789/0 conn(0x7f62440cf820 legacy=0x7f62440b52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:02.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.534+0000 7f6243400640 1 -- 172.21.15.167:0/3548449518 >> v1:172.21.15.167:6789/0 conn(0x7f62440b59f0 legacy=0x7f6244073220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:02.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.534+0000 7f6243400640 1 -- 172.21.15.167:0/3548449518 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62440b4210 con 0x7f62441439a0 2024-05-11T12:14:02.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.534+0000 7f6242a00640 1 -- 172.21.15.167:0/3548449518 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2884811059 0 0) 0x7f62440d0420 con 0x7f62440b59f0 2024-05-11T12:14:02.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.535+0000 7f6243400640 1 --2- 172.21.15.167:0/3548449518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62441439a0 0x7f62440cfee0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f62340097a0 tx=0x7f623402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e985bce2f054844a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:02.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.535+0000 7f6242a00640 1 -- 172.21.15.167:0/3548449518 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6234037070 con 0x7f62441439a0 2024-05-11T12:14:02.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.535+0000 7f6242a00640 1 -- 172.21.15.167:0/3548449518 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6234032440 con 0x7f62441439a0 2024-05-11T12:14:02.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.535+0000 7f6242a00640 1 -- 172.21.15.167:0/3548449518 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6234032810 con 0x7f62441439a0 2024-05-11T12:14:02.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.536+0000 7f624a400640 1 -- 172.21.15.167:0/3548449518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62441439a0 msgr2=0x7f62440cfee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:02.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.536+0000 7f624a400640 1 --2- 172.21.15.167:0/3548449518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62441439a0 0x7f62440cfee0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f62340097a0 tx=0x7f623402a180 comp rx=0 tx=0).stop 2024-05-11T12:14:02.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.536+0000 7f624a400640 1 -- 172.21.15.167:0/3548449518 shutdown_connections 2024-05-11T12:14:02.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.536+0000 7f624a400640 1 --2- 172.21.15.167:0/3548449518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62441439a0 0x7f62440cfee0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:02.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.536+0000 7f624a400640 1 -- 172.21.15.167:0/3548449518 >> 172.21.15.167:0/3548449518 conn(0x7f62440d0c70 msgr2=0x7f62440d1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:02.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.536+0000 7f624a400640 1 -- 172.21.15.167:0/3548449518 shutdown_connections 2024-05-11T12:14:02.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.536+0000 7f624a400640 1 -- 172.21.15.167:0/3548449518 wait complete. 2024-05-11T12:14:02.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.538+0000 7f624a400640 1 Processor -- start 2024-05-11T12:14:02.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.538+0000 7f624a400640 1 -- start start 2024-05-11T12:14:02.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.539+0000 7f624a400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62440b59f0 0x7f6244159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:02.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.539+0000 7f624a400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62440cf820 0x7f6244158ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:02.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.539+0000 7f624a400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62441439a0 0x7f62440c5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:02.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.539+0000 7f624a400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f62440b4ab0 con 0x7f62441439a0 2024-05-11T12:14:02.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.539+0000 7f624a400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f62440b47d0 con 0x7f62440b59f0 2024-05-11T12:14:02.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.539+0000 7f624a400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6244073930 con 0x7f62440cf820 2024-05-11T12:14:02.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.539+0000 7f6243e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62441439a0 0x7f62440c5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:02.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.539+0000 7f6243e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62441439a0 0x7f62440c5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45292/0 (socket says 172.21.15.167:45292) 2024-05-11T12:14:02.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.539+0000 7f6243e00640 1 -- 172.21.15.167:0/3929872986 learned_addr learned my addr 172.21.15.167:0/3929872986 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:02.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.540+0000 7f6243400640 1 --2- 172.21.15.167:0/3929872986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62440b59f0 0x7f6244159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:02.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.540+0000 7f6243e00640 1 -- 172.21.15.167:0/3929872986 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62440cf820 msgr2=0x7f6244158ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:02.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.540+0000 7f6243e00640 1 --2- 172.21.15.167:0/3929872986 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62440cf820 0x7f6244158ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:02.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.540+0000 7f6243e00640 1 -- 172.21.15.167:0/3929872986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62440b59f0 msgr2=0x7f6244159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:02.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.540+0000 7f6243e00640 1 --2- 172.21.15.167:0/3929872986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62440b59f0 0x7f6244159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:02.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.540+0000 7f6243e00640 1 -- 172.21.15.167:0/3929872986 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f624406f420 con 0x7f62441439a0 2024-05-11T12:14:02.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.540+0000 7f6243400640 1 --2- 172.21.15.167:0/3929872986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62440b59f0 0x7f6244159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:02.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.540+0000 7f6243e00640 1 --2- 172.21.15.167:0/3929872986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62441439a0 0x7f62440c5c50 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f623c00c770 tx=0x7f623c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:02.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.540+0000 7f623b400640 1 -- 172.21.15.167:0/3929872986 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623c018070 con 0x7f62441439a0 2024-05-11T12:14:02.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.540+0000 7f623b400640 1 -- 172.21.15.167:0/3929872986 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f623c00a5e0 con 0x7f62441439a0 2024-05-11T12:14:02.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.540+0000 7f623b400640 1 -- 172.21.15.167:0/3929872986 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623c00a9d0 con 0x7f62441439a0 2024-05-11T12:14:02.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.540+0000 7f624a400640 1 -- 172.21.15.167:0/3929872986 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f624406fc20 con 0x7f62441439a0 2024-05-11T12:14:02.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.541+0000 7f624a400640 1 -- 172.21.15.167:0/3929872986 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62440701e0 con 0x7f62441439a0 2024-05-11T12:14:02.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.542+0000 7f624a400640 1 -- 172.21.15.167:0/3929872986 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62440b6040 con 0x7f62441439a0 2024-05-11T12:14:02.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.543+0000 7f623b400640 1 -- 172.21.15.167:0/3929872986 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f623c01b070 con 0x7f62441439a0 2024-05-11T12:14:02.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.544+0000 7f623b400640 1 --2- 172.21.15.167:0/3929872986 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f62200762d0 0x7f6220078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:02.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.544+0000 7f623b400640 1 -- 172.21.15.167:0/3929872986 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5249+0+0 (secure 0 0 0) 0x7f623c0969b0 con 0x7f62441439a0 2024-05-11T12:14:02.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.544+0000 7f6243400640 1 --2- 172.21.15.167:0/3929872986 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f62200762d0 0x7f6220078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:02.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.545+0000 7f6243400640 1 --2- 172.21.15.167:0/3929872986 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f62200762d0 0x7f6220078790 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6234034000 tx=0x7f623402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:02.548 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.548+0000 7f623b400640 1 -- 172.21.15.167:0/3929872986 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f62440b6040 con 0x7f62441439a0 2024-05-11T12:14:02.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:02.757+0000 7f624a400640 1 -- 172.21.15.167:0/3929872986 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f62440b1a90 con 0x7f62441439a0 2024-05-11T12:14:03.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.371+0000 7f623b400640 1 -- 172.21.15.167:0/3929872986 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7f623c0604c0 con 0x7f62441439a0 2024-05-11T12:14:03.372 INFO:teuthology.orchestra.run.smithi167.stderr:pool 'cephfs_metadata' created 2024-05-11T12:14:03.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.375+0000 7f624a400640 1 -- 172.21.15.167:0/3929872986 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f62200762d0 msgr2=0x7f6220078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:03.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.375+0000 7f624a400640 1 --2- 172.21.15.167:0/3929872986 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f62200762d0 0x7f6220078790 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6234034000 tx=0x7f623402e000 comp rx=0 tx=0).stop 2024-05-11T12:14:03.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.375+0000 7f624a400640 1 -- 172.21.15.167:0/3929872986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62441439a0 msgr2=0x7f62440c5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:03.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.375+0000 7f624a400640 1 --2- 172.21.15.167:0/3929872986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62441439a0 0x7f62440c5c50 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f623c00c770 tx=0x7f623c00cc30 comp rx=0 tx=0).stop 2024-05-11T12:14:03.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.376+0000 7f624a400640 1 -- 172.21.15.167:0/3929872986 shutdown_connections 2024-05-11T12:14:03.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.376+0000 7f624a400640 1 --2- 172.21.15.167:0/3929872986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62441439a0 0x7f62440c5c50 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:03.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.376+0000 7f624a400640 1 --2- 172.21.15.167:0/3929872986 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62440cf820 0x7f6244158ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:03.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.376+0000 7f624a400640 1 --2- 172.21.15.167:0/3929872986 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f62200762d0 0x7f6220078790 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:03.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.376+0000 7f624a400640 1 --2- 172.21.15.167:0/3929872986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62440b59f0 0x7f6244159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:03.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.376+0000 7f624a400640 1 -- 172.21.15.167:0/3929872986 >> 172.21.15.167:0/3929872986 conn(0x7f62440d0c70 msgr2=0x7f62440b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:03.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.376+0000 7f624a400640 1 -- 172.21.15.167:0/3929872986 shutdown_connections 2024-05-11T12:14:03.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.376+0000 7f624a400640 1 -- 172.21.15.167:0/3929872986 wait complete. 2024-05-11T12:14:03.404 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-11T12:14:03.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.602+0000 7f5fa1000640 1 Processor -- start 2024-05-11T12:14:03.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.602+0000 7f5fa1000640 1 -- start start 2024-05-11T12:14:03.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.604+0000 7f5fa1000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5f9c072ff0 0x7f5f9c0b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:03.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.604+0000 7f5fa1000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5f9c06d460 0x7f5f9c06d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:03.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.604+0000 7f5fa1000640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f9c0733d0 con 0x7f5f9c15f850 2024-05-11T12:14:03.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.604+0000 7f5fa1000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f5f9c06e5b0 con 0x7f5f9c06d460 2024-05-11T12:14:03.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.604+0000 7f5fa1000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5f9c160110 con 0x7f5f9c072ff0 2024-05-11T12:14:03.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.604+0000 7f5f93e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f5f9c15f850 0x7f5f9c06ea40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:40382/0 (socket says 172.21.15.167:40382) 2024-05-11T12:14:03.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.604+0000 7f5f93e00640 1 -- 172.21.15.167:0/3599192468 learned_addr learned my addr 172.21.15.167:0/3599192468 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:14:03.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.605+0000 7f5f9b400640 1 --2- 172.21.15.167:0/3599192468 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5f9c072ff0 0x7f5f9c0b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:03.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.605+0000 7f5f9aa00640 1 -- 172.21.15.167:0/3599192468 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2930060946 0 0) 0x7f5f9c0733d0 con 0x7f5f9c15f850 2024-05-11T12:14:03.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.605+0000 7f5f9aa00640 1 -- 172.21.15.167:0/3599192468 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f80003520 con 0x7f5f9c15f850 2024-05-11T12:14:03.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.605+0000 7f5f9be00640 1 --2- 172.21.15.167:0/3599192468 >> v2:172.21.15.204:3300/0 conn(0x7f5f9c06d460 0x7f5f9c06d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:03.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.605+0000 7f5f9aa00640 1 -- 172.21.15.167:0/3599192468 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4173955664 0 0) 0x7f5f80003520 con 0x7f5f9c15f850 2024-05-11T12:14:03.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.605+0000 7f5f9aa00640 1 -- 172.21.15.167:0/3599192468 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5f9c072ff0 msgr2=0x7f5f9c0b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:03.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.605+0000 7f5f9aa00640 1 --2- 172.21.15.167:0/3599192468 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5f9c072ff0 0x7f5f9c0b5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:03.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.605+0000 7f5f9aa00640 1 -- 172.21.15.167:0/3599192468 >> v2:172.21.15.204:3300/0 conn(0x7f5f9c06d460 msgr2=0x7f5f9c06d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:03.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.605+0000 7f5f9aa00640 1 --2- 172.21.15.167:0/3599192468 >> v2:172.21.15.204:3300/0 conn(0x7f5f9c06d460 0x7f5f9c06d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:03.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.605+0000 7f5f9aa00640 1 -- 172.21.15.167:0/3599192468 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f9c1603f0 con 0x7f5f9c15f850 2024-05-11T12:14:03.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.605+0000 7f5f9aa00640 1 -- 172.21.15.167:0/3599192468 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f5f840016f0 con 0x7f5f9c15f850 2024-05-11T12:14:03.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.605+0000 7f5f9aa00640 1 -- 172.21.15.167:0/3599192468 >> v1:172.21.15.167:6789/0 conn(0x7f5f9c15f850 legacy=0x7f5f9c06ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:03.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.605+0000 7f5f9aa00640 1 --2- 172.21.15.167:0/3599192468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f80003a00 0x7f5f80003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:03.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.605+0000 7f5f9aa00640 1 --2- 172.21.15.167:0/3599192468 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5f9c072ff0 0x7f5f80004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:03.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.605+0000 7f5f9aa00640 1 --2- 172.21.15.167:0/3599192468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f80004d40 0x7f5f800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:03.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.605+0000 7f5f9aa00640 1 -- 172.21.15.167:0/3599192468 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9c06e5b0 con 0x7f5f80004d40 2024-05-11T12:14:03.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.605+0000 7f5f9aa00640 1 -- 172.21.15.167:0/3599192468 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9c160110 con 0x7f5f80003a00 2024-05-11T12:14:03.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.605+0000 7f5f9aa00640 1 -- 172.21.15.167:0/3599192468 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5f8000ba70 con 0x7f5f9c072ff0 2024-05-11T12:14:03.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.606+0000 7f5f9be00640 1 --2- 172.21.15.167:0/3599192468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f80004d40 0x7f5f800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:03.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.606+0000 7f5f93e00640 1 --2- 172.21.15.167:0/3599192468 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5f9c072ff0 0x7f5f80004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:03.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.606+0000 7f5f9b400640 1 --2- 172.21.15.167:0/3599192468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f80003a00 0x7f5f80003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:03.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.606+0000 7f5f9be00640 1 -- 172.21.15.167:0/3599192468 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5f9c072ff0 msgr2=0x7f5f80004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:03.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.606+0000 7f5f9be00640 1 --2- 172.21.15.167:0/3599192468 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5f9c072ff0 0x7f5f80004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:03.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.606+0000 7f5f9be00640 1 -- 172.21.15.167:0/3599192468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f80003a00 msgr2=0x7f5f80003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:03.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.606+0000 7f5f9be00640 1 --2- 172.21.15.167:0/3599192468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f80003a00 0x7f5f80003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:03.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.606+0000 7f5f9be00640 1 -- 172.21.15.167:0/3599192468 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f80003520 con 0x7f5f80004d40 2024-05-11T12:14:03.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.606+0000 7f5f9b400640 1 --2- 172.21.15.167:0/3599192468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f80003a00 0x7f5f80003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:03.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.606+0000 7f5f93e00640 1 --2- 172.21.15.167:0/3599192468 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5f9c072ff0 0x7f5f80004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:03.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.607+0000 7f5f9be00640 1 --2- 172.21.15.167:0/3599192468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f80004d40 0x7f5f800051c0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f5f94007360 tx=0x7f5f80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f80019b8bd34eb00 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:03.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.607+0000 7f5f9aa00640 1 -- 172.21.15.167:0/3599192468 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f94009be0 con 0x7f5f80004d40 2024-05-11T12:14:03.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.607+0000 7f5f9aa00640 1 -- 172.21.15.167:0/3599192468 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f9402e620 con 0x7f5f80004d40 2024-05-11T12:14:03.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.607+0000 7f5f9aa00640 1 -- 172.21.15.167:0/3599192468 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9402e9f0 con 0x7f5f80004d40 2024-05-11T12:14:03.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.608+0000 7f5fa1000640 1 -- 172.21.15.167:0/3599192468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f80004d40 msgr2=0x7f5f800051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:03.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.608+0000 7f5fa1000640 1 --2- 172.21.15.167:0/3599192468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f80004d40 0x7f5f800051c0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f5f94007360 tx=0x7f5f80005830 comp rx=0 tx=0).stop 2024-05-11T12:14:03.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.608+0000 7f5f9be00640 1 -- 172.21.15.167:0/3599192468 reap_dead start 2024-05-11T12:14:03.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.608+0000 7f5fa1000640 1 -- 172.21.15.167:0/3599192468 shutdown_connections 2024-05-11T12:14:03.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.608+0000 7f5fa1000640 1 -- 172.21.15.167:0/3599192468 >> 172.21.15.167:0/3599192468 conn(0x7f5f9c0b32a0 msgr2=0x7f5f9c0d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:03.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.608+0000 7f5fa1000640 1 -- 172.21.15.167:0/3599192468 shutdown_connections 2024-05-11T12:14:03.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.608+0000 7f5fa1000640 1 -- 172.21.15.167:0/3599192468 wait complete. 2024-05-11T12:14:03.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.610+0000 7f5fa1000640 1 Processor -- start 2024-05-11T12:14:03.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.610+0000 7f5fa1000640 1 -- start start 2024-05-11T12:14:03.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.611+0000 7f5fa1000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f80004d40 0x7f5f9c149c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:03.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.611+0000 7f5fa1000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f9c1467c0 0x7f5f9c1f4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:03.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.611+0000 7f5fa1000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5f9c0b06b0 0x7f5f9c0b0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:03.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.611+0000 7f5fa1000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9c15fcb0 con 0x7f5f80004d40 2024-05-11T12:14:03.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.611+0000 7f5fa1000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9c1606d0 con 0x7f5f9c1467c0 2024-05-11T12:14:03.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.611+0000 7f5fa1000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5f9c15fe30 con 0x7f5f9c0b06b0 2024-05-11T12:14:03.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.611+0000 7f5f9b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f80004d40 0x7f5f9c149c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:03.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.611+0000 7f5f9b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f80004d40 0x7f5f9c149c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45310/0 (socket says 172.21.15.167:45310) 2024-05-11T12:14:03.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.611+0000 7f5f9b400640 1 -- 172.21.15.167:0/2487346248 learned_addr learned my addr 172.21.15.167:0/2487346248 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:03.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.612+0000 7f5f9be00640 1 --2- 172.21.15.167:0/2487346248 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5f9c0b06b0 0x7f5f9c0b0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:03.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.612+0000 7f5f93e00640 1 --2- 172.21.15.167:0/2487346248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f9c1467c0 0x7f5f9c1f4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:03.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.612+0000 7f5f9b400640 1 -- 172.21.15.167:0/2487346248 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5f9c0b06b0 msgr2=0x7f5f9c0b0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:03.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.612+0000 7f5f9b400640 1 --2- 172.21.15.167:0/2487346248 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5f9c0b06b0 0x7f5f9c0b0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:03.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.612+0000 7f5f9b400640 1 -- 172.21.15.167:0/2487346248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f9c1467c0 msgr2=0x7f5f9c1f4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:03.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.612+0000 7f5f9b400640 1 --2- 172.21.15.167:0/2487346248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f9c1467c0 0x7f5f9c1f4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:03.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.612+0000 7f5f9b400640 1 -- 172.21.15.167:0/2487346248 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f9c1f5500 con 0x7f5f80004d40 2024-05-11T12:14:03.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.612+0000 7f5f9b400640 1 --2- 172.21.15.167:0/2487346248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f80004d40 0x7f5f9c149c50 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c0108b0 tx=0x7f5f8c010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:03.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.612+0000 7f5f93400640 1 -- 172.21.15.167:0/2487346248 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c00f040 con 0x7f5f80004d40 2024-05-11T12:14:03.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.612+0000 7f5f93400640 1 -- 172.21.15.167:0/2487346248 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f8c00eb60 con 0x7f5f80004d40 2024-05-11T12:14:03.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.612+0000 7f5f93400640 1 -- 172.21.15.167:0/2487346248 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c01c400 con 0x7f5f80004d40 2024-05-11T12:14:03.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.612+0000 7f5fa1000640 1 -- 172.21.15.167:0/2487346248 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f9c1f57e0 con 0x7f5f80004d40 2024-05-11T12:14:03.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.613+0000 7f5fa1000640 1 -- 172.21.15.167:0/2487346248 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f9c147240 con 0x7f5f80004d40 2024-05-11T12:14:03.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.614+0000 7f5fa1000640 1 -- 172.21.15.167:0/2487346248 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f9c0b1070 con 0x7f5f80004d40 2024-05-11T12:14:03.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.615+0000 7f5f93400640 1 -- 172.21.15.167:0/2487346248 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5f8c01c5a0 con 0x7f5f80004d40 2024-05-11T12:14:03.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.617+0000 7f5f93400640 1 --2- 172.21.15.167:0/2487346248 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5f78076300 0x7f5f780787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:03.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.617+0000 7f5f93400640 1 -- 172.21.15.167:0/2487346248 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5619+0+0 (secure 0 0 0) 0x7f5f8c097cd0 con 0x7f5f80004d40 2024-05-11T12:14:03.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.617+0000 7f5f93e00640 1 --2- 172.21.15.167:0/2487346248 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5f78076300 0x7f5f780787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:03.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.618+0000 7f5f93e00640 1 --2- 172.21.15.167:0/2487346248 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5f78076300 0x7f5f780787c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5f84000f60 tx=0x7f5f84001c30 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:03.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.622+0000 7f5f93400640 1 -- 172.21.15.167:0/2487346248 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5f8c018030 con 0x7f5f80004d40 2024-05-11T12:14:03.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:03.829+0000 7f5fa1000640 1 -- 172.21.15.167:0/2487346248 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f5f9c159130 con 0x7f5f80004d40 2024-05-11T12:14:04.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.388+0000 7f5f93400640 1 -- 172.21.15.167:0/2487346248 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v84) ==== 170+0+0 (secure 0 0 0) 0x7f5f8c061670 con 0x7f5f80004d40 2024-05-11T12:14:04.389 INFO:teuthology.orchestra.run.smithi167.stderr:pool 'cephfs_data' created 2024-05-11T12:14:04.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.392+0000 7f5fa1000640 1 -- 172.21.15.167:0/2487346248 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5f78076300 msgr2=0x7f5f780787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:04.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.392+0000 7f5fa1000640 1 --2- 172.21.15.167:0/2487346248 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5f78076300 0x7f5f780787c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5f84000f60 tx=0x7f5f84001c30 comp rx=0 tx=0).stop 2024-05-11T12:14:04.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.392+0000 7f5fa1000640 1 -- 172.21.15.167:0/2487346248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f80004d40 msgr2=0x7f5f9c149c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:04.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.392+0000 7f5fa1000640 1 --2- 172.21.15.167:0/2487346248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f80004d40 0x7f5f9c149c50 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c0108b0 tx=0x7f5f8c010d70 comp rx=0 tx=0).stop 2024-05-11T12:14:04.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.392+0000 7f5fa1000640 1 -- 172.21.15.167:0/2487346248 shutdown_connections 2024-05-11T12:14:04.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.392+0000 7f5fa1000640 1 --2- 172.21.15.167:0/2487346248 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5f9c0b06b0 0x7f5f9c0b0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:04.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.392+0000 7f5fa1000640 1 --2- 172.21.15.167:0/2487346248 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5f78076300 0x7f5f780787c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:04.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.392+0000 7f5fa1000640 1 --2- 172.21.15.167:0/2487346248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f9c1467c0 0x7f5f9c1f4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:04.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.392+0000 7f5fa1000640 1 --2- 172.21.15.167:0/2487346248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f80004d40 0x7f5f9c149c50 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:04.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.392+0000 7f5fa1000640 1 -- 172.21.15.167:0/2487346248 >> 172.21.15.167:0/2487346248 conn(0x7f5f9c0b32a0 msgr2=0x7f5f9c0681c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:04.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.392+0000 7f5fa1000640 1 -- 172.21.15.167:0/2487346248 shutdown_connections 2024-05-11T12:14:04.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.392+0000 7f5fa1000640 1 -- 172.21.15.167:0/2487346248 wait complete. 2024-05-11T12:14:04.426 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-11T12:14:04.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.601+0000 7fabb0000640 1 Processor -- start 2024-05-11T12:14:04.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.601+0000 7fabb0000640 1 -- start start 2024-05-11T12:14:04.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.602+0000 7fabb0000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7faba806f420 0x7faba80b6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:04.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.602+0000 7fabb0000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faba8070050 0x7faba8072a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:04.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.602+0000 7fabb0000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faba8156140 con 0x7faba8072fa0 2024-05-11T12:14:04.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.603+0000 7fabb0000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7faba80b6e80 con 0x7faba806f420 2024-05-11T12:14:04.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.603+0000 7fabb0000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7faba815bc60 con 0x7faba8070050 2024-05-11T12:14:04.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.603+0000 7fabad600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7faba806f420 0x7faba80b6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:04.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.603+0000 7fabae000640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7faba8072fa0 0x7faba815f250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.167:46284/0 (socket says 172.21.15.167:46284) 2024-05-11T12:14:04.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.603+0000 7fabae000640 1 -- 172.21.15.167:0/1705555544 learned_addr learned my addr 172.21.15.167:0/1705555544 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:14:04.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.603+0000 7fabacc00640 1 --2- 172.21.15.167:0/1705555544 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faba8070050 0x7faba8072a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:04.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.604+0000 7fabad600640 1 -- 172.21.15.167:0/1705555544 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faba8070050 msgr2=0x7faba8072a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:04.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.604+0000 7fabad600640 1 --2- 172.21.15.167:0/1705555544 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faba8070050 0x7faba8072a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:04.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.604+0000 7fabad600640 1 -- 172.21.15.167:0/1705555544 >> v1:172.21.15.204:6789/0 conn(0x7faba8072fa0 legacy=0x7faba815f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:04.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.605+0000 7fabad600640 1 -- 172.21.15.167:0/1705555544 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faba815bf20 con 0x7faba806f420 2024-05-11T12:14:04.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.604+0000 7faba7e00640 1 -- 172.21.15.167:0/1705555544 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3591498537 0 0) 0x7faba0001490 con 0x7faba8072fa0 2024-05-11T12:14:04.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.606+0000 7fabad600640 1 --2- 172.21.15.167:0/1705555544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faba806f420 0x7faba80b6940 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fab9c0097a0 tx=0x7fab9c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1ca2fc8bae81c9a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:04.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.606+0000 7faba7e00640 1 -- 172.21.15.167:0/1705555544 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9c037070 con 0x7faba806f420 2024-05-11T12:14:04.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.606+0000 7faba7e00640 1 -- 172.21.15.167:0/1705555544 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab9c032440 con 0x7faba806f420 2024-05-11T12:14:04.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.606+0000 7faba7e00640 1 -- 172.21.15.167:0/1705555544 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9c032810 con 0x7faba806f420 2024-05-11T12:14:04.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.607+0000 7fabb0000640 1 -- 172.21.15.167:0/1705555544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faba806f420 msgr2=0x7faba80b6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:04.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.607+0000 7fabb0000640 1 --2- 172.21.15.167:0/1705555544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faba806f420 0x7faba80b6940 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fab9c0097a0 tx=0x7fab9c02a180 comp rx=0 tx=0).stop 2024-05-11T12:14:04.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.607+0000 7fabb0000640 1 -- 172.21.15.167:0/1705555544 shutdown_connections 2024-05-11T12:14:04.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.607+0000 7fabb0000640 1 --2- 172.21.15.167:0/1705555544 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faba8070050 0x7faba8072a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:04.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.607+0000 7fabb0000640 1 --2- 172.21.15.167:0/1705555544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faba806f420 0x7faba80b6940 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:04.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.607+0000 7fabb0000640 1 -- 172.21.15.167:0/1705555544 >> 172.21.15.167:0/1705555544 conn(0x7faba806c450 msgr2=0x7faba806c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:04.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.607+0000 7fabb0000640 1 -- 172.21.15.167:0/1705555544 shutdown_connections 2024-05-11T12:14:04.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.607+0000 7fabb0000640 1 -- 172.21.15.167:0/1705555544 wait complete. 2024-05-11T12:14:04.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.609+0000 7fabb0000640 1 Processor -- start 2024-05-11T12:14:04.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.609+0000 7fabb0000640 1 -- start start 2024-05-11T12:14:04.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.610+0000 7fabb0000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faba806f420 0x7faba81f4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:04.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.610+0000 7fabb0000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faba8070050 0x7faba806e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:04.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.610+0000 7fabb0000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faba8072fa0 0x7faba806e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:04.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.610+0000 7fabb0000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faba815c7c0 con 0x7faba8072fa0 2024-05-11T12:14:04.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.610+0000 7fabb0000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7faba815c4e0 con 0x7faba8070050 2024-05-11T12:14:04.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.610+0000 7fabb0000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7faba80cfab0 con 0x7faba806f420 2024-05-11T12:14:04.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.610+0000 7fabae000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faba8072fa0 0x7faba806e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:04.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.610+0000 7fabae000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faba8072fa0 0x7faba806e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45324/0 (socket says 172.21.15.167:45324) 2024-05-11T12:14:04.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.610+0000 7fabae000640 1 -- 172.21.15.167:0/1222884169 learned_addr learned my addr 172.21.15.167:0/1222884169 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:04.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.610+0000 7fabad600640 1 --2- 172.21.15.167:0/1222884169 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faba806f420 0x7faba81f4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:04.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.611+0000 7fabacc00640 1 --2- 172.21.15.167:0/1222884169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faba8070050 0x7faba806e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:04.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.611+0000 7fabae000640 1 -- 172.21.15.167:0/1222884169 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faba806f420 msgr2=0x7faba81f4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:04.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.611+0000 7fabae000640 1 --2- 172.21.15.167:0/1222884169 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faba806f420 0x7faba81f4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:04.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.611+0000 7fabae000640 1 -- 172.21.15.167:0/1222884169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faba8070050 msgr2=0x7faba806e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:04.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.611+0000 7fabae000640 1 --2- 172.21.15.167:0/1222884169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faba8070050 0x7faba806e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:04.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.611+0000 7fabae000640 1 -- 172.21.15.167:0/1222884169 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faba81f1500 con 0x7faba8072fa0 2024-05-11T12:14:04.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.611+0000 7fabae000640 1 --2- 172.21.15.167:0/1222884169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faba8072fa0 0x7faba806e800 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7faba000c6b0 tx=0x7faba000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:04.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.611+0000 7faba5600640 1 -- 172.21.15.167:0/1222884169 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba0018070 con 0x7faba8072fa0 2024-05-11T12:14:04.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.611+0000 7faba5600640 1 -- 172.21.15.167:0/1222884169 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faba0004da0 con 0x7faba8072fa0 2024-05-11T12:14:04.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.611+0000 7faba5600640 1 -- 172.21.15.167:0/1222884169 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba000a880 con 0x7faba8072fa0 2024-05-11T12:14:04.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.611+0000 7fabb0000640 1 -- 172.21.15.167:0/1222884169 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faba81f17e0 con 0x7faba8072fa0 2024-05-11T12:14:04.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.612+0000 7fabb0000640 1 -- 172.21.15.167:0/1222884169 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faba81f20f0 con 0x7faba8072fa0 2024-05-11T12:14:04.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.613+0000 7fabb0000640 1 -- 172.21.15.167:0/1222884169 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faba81f1d00 con 0x7faba8072fa0 2024-05-11T12:14:04.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.615+0000 7faba5600640 1 -- 172.21.15.167:0/1222884169 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7faba001b070 con 0x7faba8072fa0 2024-05-11T12:14:04.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.616+0000 7faba5600640 1 --2- 172.21.15.167:0/1222884169 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fab880762d0 0x7fab88078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:04.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.616+0000 7faba5600640 1 -- 172.21.15.167:0/1222884169 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6001+0+0 (secure 0 0 0) 0x7faba0097110 con 0x7faba8072fa0 2024-05-11T12:14:04.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.617+0000 7fabad600640 1 --2- 172.21.15.167:0/1222884169 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fab880762d0 0x7fab88078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:04.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.617+0000 7fabad600640 1 --2- 172.21.15.167:0/1222884169 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fab880762d0 0x7fab88078790 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fab9c034000 tx=0x7fab9c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:04.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.620+0000 7faba5600640 1 -- 172.21.15.167:0/1222884169 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7faba81f1d00 con 0x7faba8072fa0 2024-05-11T12:14:04.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:04.860+0000 7fabb0000640 1 -- 172.21.15.167:0/1222884169 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7faba8069520 con 0x7faba8072fa0 2024-05-11T12:14:05.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.392+0000 7faba5600640 1 -- 172.21.15.167:0/1222884169 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v49) ==== 174+0+0 (secure 0 0 0) 0x7faba0022020 con 0x7faba8072fa0 2024-05-11T12:14:05.393 INFO:teuthology.orchestra.run.smithi167.stderr:new fs with metadata pool 13 and data pool 14 2024-05-11T12:14:05.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.399+0000 7fabb0000640 1 -- 172.21.15.167:0/1222884169 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fab880762d0 msgr2=0x7fab88078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:05.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.399+0000 7fabb0000640 1 --2- 172.21.15.167:0/1222884169 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fab880762d0 0x7fab88078790 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fab9c034000 tx=0x7fab9c02e000 comp rx=0 tx=0).stop 2024-05-11T12:14:05.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.399+0000 7fabb0000640 1 -- 172.21.15.167:0/1222884169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faba8072fa0 msgr2=0x7faba806e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:05.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.399+0000 7fabb0000640 1 --2- 172.21.15.167:0/1222884169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faba8072fa0 0x7faba806e800 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7faba000c6b0 tx=0x7faba000cb70 comp rx=0 tx=0).stop 2024-05-11T12:14:05.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.399+0000 7fabb0000640 1 -- 172.21.15.167:0/1222884169 shutdown_connections 2024-05-11T12:14:05.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.399+0000 7fabb0000640 1 --2- 172.21.15.167:0/1222884169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faba8072fa0 0x7faba806e800 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:05.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.399+0000 7fabb0000640 1 --2- 172.21.15.167:0/1222884169 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fab880762d0 0x7fab88078790 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:05.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.399+0000 7fabb0000640 1 --2- 172.21.15.167:0/1222884169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faba8070050 0x7faba806e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:05.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.399+0000 7fabb0000640 1 --2- 172.21.15.167:0/1222884169 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faba806f420 0x7faba81f4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:05.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.399+0000 7fabb0000640 1 -- 172.21.15.167:0/1222884169 >> 172.21.15.167:0/1222884169 conn(0x7faba806c450 msgr2=0x7faba81557a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:05.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.403+0000 7fabb0000640 1 -- 172.21.15.167:0/1222884169 shutdown_connections 2024-05-11T12:14:05.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.403+0000 7fabb0000640 1 -- 172.21.15.167:0/1222884169 wait complete. 2024-05-11T12:14:05.440 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:14:05.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.612+0000 7fd503400640 1 Processor -- start 2024-05-11T12:14:05.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.613+0000 7fd503400640 1 -- start start 2024-05-11T12:14:05.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.614+0000 7fd503400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd4fc157750 0x7fd4fc15d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:05.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.614+0000 7fd503400640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4fc15dc80 con 0x7fd4fc0b3ea0 2024-05-11T12:14:05.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.614+0000 7fd503400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4fc15df30 con 0x7fd4fc06d460 2024-05-11T12:14:05.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.614+0000 7fd503400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fd4fc161000 con 0x7fd4fc157750 2024-05-11T12:14:05.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.614+0000 7fd4f3e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fd4fc0b3ea0 0x7fd4fc161490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:40396/0 (socket says 172.21.15.167:40396) 2024-05-11T12:14:05.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.615+0000 7fd4f3e00640 1 -- 172.21.15.167:0/924990608 learned_addr learned my addr 172.21.15.167:0/924990608 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:14:05.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.615+0000 7fd4fbe00640 1 -- 172.21.15.167:0/924990608 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1902606331 0 0) 0x7fd4fc15dc80 con 0x7fd4fc0b3ea0 2024-05-11T12:14:05.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.615+0000 7fd4fbe00640 1 -- 172.21.15.167:0/924990608 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd4dc003520 con 0x7fd4fc0b3ea0 2024-05-11T12:14:05.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.615+0000 7fd500a00640 1 --2- 172.21.15.167:0/924990608 >> v2:172.21.15.204:3300/0 conn(0x7fd4fc157750 0x7fd4fc15d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:05.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.615+0000 7fd4fbe00640 1 -- 172.21.15.167:0/924990608 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2456235323 0 0) 0x7fd4dc003520 con 0x7fd4fc0b3ea0 2024-05-11T12:14:05.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.615+0000 7fd4fbe00640 1 -- 172.21.15.167:0/924990608 >> v2:172.21.15.204:3300/0 conn(0x7fd4fc157750 msgr2=0x7fd4fc15d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:05.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.615+0000 7fd4fbe00640 1 --2- 172.21.15.167:0/924990608 >> v2:172.21.15.204:3300/0 conn(0x7fd4fc157750 0x7fd4fc15d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:05.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.615+0000 7fd4fbe00640 1 -- 172.21.15.167:0/924990608 >> v1:172.21.15.204:6789/0 conn(0x7fd4fc06d460 legacy=0x7fd4fc0cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:05.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.615+0000 7fd4fbe00640 1 -- 172.21.15.167:0/924990608 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4fc15e210 con 0x7fd4fc0b3ea0 2024-05-11T12:14:05.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.615+0000 7fd4fbe00640 1 -- 172.21.15.167:0/924990608 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7fd4ec0016f0 con 0x7fd4fc0b3ea0 2024-05-11T12:14:05.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.615+0000 7fd4fbe00640 1 -- 172.21.15.167:0/924990608 >> v1:172.21.15.167:6789/0 conn(0x7fd4fc0b3ea0 legacy=0x7fd4fc161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:05.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.615+0000 7fd4fbe00640 1 --2- 172.21.15.167:0/924990608 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4dc003a50 0x7fd4dc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:05.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.616+0000 7fd4fbe00640 1 --2- 172.21.15.167:0/924990608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4dc004800 0x7fd4dc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:05.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.616+0000 7fd4fbe00640 1 --2- 172.21.15.167:0/924990608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4dc0052e0 0x7fd4dc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:05.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.616+0000 7fd4fbe00640 1 -- 172.21.15.167:0/924990608 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd4fc161000 con 0x7fd4dc0052e0 2024-05-11T12:14:05.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.616+0000 7fd4fbe00640 1 -- 172.21.15.167:0/924990608 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd4dc00c1b0 con 0x7fd4dc004800 2024-05-11T12:14:05.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.616+0000 7fd4fbe00640 1 -- 172.21.15.167:0/924990608 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd4dc00c3b0 con 0x7fd4dc003a50 2024-05-11T12:14:05.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.616+0000 7fd501400640 1 --2- 172.21.15.167:0/924990608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4dc0052e0 0x7fd4dc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:05.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.616+0000 7fd501400640 1 -- 172.21.15.167:0/924990608 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4dc003a50 msgr2=0x7fd4dc003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:05.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.616+0000 7fd501400640 1 --2- 172.21.15.167:0/924990608 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4dc003a50 0x7fd4dc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:05.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.616+0000 7fd501400640 1 -- 172.21.15.167:0/924990608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4dc004800 msgr2=0x7fd4dc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:05.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.616+0000 7fd501400640 1 --2- 172.21.15.167:0/924990608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4dc004800 0x7fd4dc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:05.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.616+0000 7fd501400640 1 -- 172.21.15.167:0/924990608 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4dc003520 con 0x7fd4dc0052e0 2024-05-11T12:14:05.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.617+0000 7fd501400640 1 -- 172.21.15.167:0/924990608 reap_dead start 2024-05-11T12:14:05.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.617+0000 7fd501400640 1 --2- 172.21.15.167:0/924990608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4dc0052e0 0x7fd4dc00b9c0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4001e60 tx=0x7fd4dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2217e5b59574ed3c server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:05.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.617+0000 7fd4fbe00640 1 -- 172.21.15.167:0/924990608 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f4001ab0 con 0x7fd4dc0052e0 2024-05-11T12:14:05.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.618+0000 7fd4fbe00640 1 -- 172.21.15.167:0/924990608 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4f402ae00 con 0x7fd4dc0052e0 2024-05-11T12:14:05.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.618+0000 7fd4fbe00640 1 -- 172.21.15.167:0/924990608 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f4003bb0 con 0x7fd4dc0052e0 2024-05-11T12:14:05.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.618+0000 7fd503400640 1 -- 172.21.15.167:0/924990608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4dc0052e0 msgr2=0x7fd4dc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:05.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.618+0000 7fd503400640 1 --2- 172.21.15.167:0/924990608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4dc0052e0 0x7fd4dc00b9c0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4001e60 tx=0x7fd4dc005830 comp rx=0 tx=0).stop 2024-05-11T12:14:05.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.618+0000 7fd503400640 1 -- 172.21.15.167:0/924990608 shutdown_connections 2024-05-11T12:14:05.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.618+0000 7fd503400640 1 --2- 172.21.15.167:0/924990608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4dc0052e0 0x7fd4dc00b9c0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:05.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.618+0000 7fd503400640 1 -- 172.21.15.167:0/924990608 >> 172.21.15.167:0/924990608 conn(0x7fd4fc0cd8f0 msgr2=0x7fd4fc0cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:05.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.618+0000 7fd503400640 1 -- 172.21.15.167:0/924990608 shutdown_connections 2024-05-11T12:14:05.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.618+0000 7fd503400640 1 -- 172.21.15.167:0/924990608 wait complete. 2024-05-11T12:14:05.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.620+0000 7fd503400640 1 Processor -- start 2024-05-11T12:14:05.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.620+0000 7fd503400640 1 -- start start 2024-05-11T12:14:05.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.621+0000 7fd503400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4dc0052e0 0x7fd4fc157cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:05.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.621+0000 7fd503400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4fc143630 0x7fd4fc13f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:05.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.621+0000 7fd503400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4fc071f70 0x7fd4fc072400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:05.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.621+0000 7fd503400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd4fc15d740 con 0x7fd4dc0052e0 2024-05-11T12:14:05.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.621+0000 7fd503400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd4fc157b30 con 0x7fd4fc143630 2024-05-11T12:14:05.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.621+0000 7fd503400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd4fc15d8c0 con 0x7fd4fc071f70 2024-05-11T12:14:05.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.621+0000 7fd500a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4dc0052e0 0x7fd4fc157cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:05.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.621+0000 7fd500a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4dc0052e0 0x7fd4fc157cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45346/0 (socket says 172.21.15.167:45346) 2024-05-11T12:14:05.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.621+0000 7fd500a00640 1 -- 172.21.15.167:0/3928105058 learned_addr learned my addr 172.21.15.167:0/3928105058 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:05.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.622+0000 7fd4f3e00640 1 --2- 172.21.15.167:0/3928105058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4fc143630 0x7fd4fc13f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:05.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.622+0000 7fd501400640 1 --2- 172.21.15.167:0/3928105058 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4fc071f70 0x7fd4fc072400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:05.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.622+0000 7fd500a00640 1 -- 172.21.15.167:0/3928105058 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4fc071f70 msgr2=0x7fd4fc072400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:05.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.622+0000 7fd500a00640 1 --2- 172.21.15.167:0/3928105058 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4fc071f70 0x7fd4fc072400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:05.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.622+0000 7fd500a00640 1 -- 172.21.15.167:0/3928105058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4fc143630 msgr2=0x7fd4fc13f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:05.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.622+0000 7fd500a00640 1 --2- 172.21.15.167:0/3928105058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4fc143630 0x7fd4fc13f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:05.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.622+0000 7fd500a00640 1 -- 172.21.15.167:0/3928105058 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4fc158370 con 0x7fd4dc0052e0 2024-05-11T12:14:05.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.622+0000 7fd500a00640 1 --2- 172.21.15.167:0/3928105058 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4dc0052e0 0x7fd4fc157cb0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400d800 tx=0x7fd4e400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:05.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.622+0000 7fd4f9600640 1 -- 172.21.15.167:0/3928105058 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e40096a0 con 0x7fd4dc0052e0 2024-05-11T12:14:05.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.622+0000 7fd4f9600640 1 -- 172.21.15.167:0/3928105058 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4e4019070 con 0x7fd4dc0052e0 2024-05-11T12:14:05.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.623+0000 7fd4f9600640 1 -- 172.21.15.167:0/3928105058 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4009a80 con 0x7fd4dc0052e0 2024-05-11T12:14:05.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.623+0000 7fd503400640 1 -- 172.21.15.167:0/3928105058 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4fc12d450 con 0x7fd4dc0052e0 2024-05-11T12:14:05.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.623+0000 7fd503400640 1 -- 172.21.15.167:0/3928105058 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4fc12da10 con 0x7fd4dc0052e0 2024-05-11T12:14:05.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.624+0000 7fd503400640 1 -- 172.21.15.167:0/3928105058 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4fc069590 con 0x7fd4dc0052e0 2024-05-11T12:14:05.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.625+0000 7fd4f9600640 1 -- 172.21.15.167:0/3928105058 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd4e4009c20 con 0x7fd4dc0052e0 2024-05-11T12:14:05.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.626+0000 7fd4f9600640 1 --2- 172.21.15.167:0/3928105058 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd4d8076300 0x7fd4d80787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:05.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.626+0000 7fd4f9600640 1 -- 172.21.15.167:0/3928105058 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 6101+0+0 (secure 0 0 0) 0x7fd4e4097bb0 con 0x7fd4dc0052e0 2024-05-11T12:14:05.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.626+0000 7fd4f3e00640 1 --2- 172.21.15.167:0/3928105058 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd4d8076300 0x7fd4d80787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:05.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.627+0000 7fd4f3e00640 1 --2- 172.21.15.167:0/3928105058 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd4d8076300 0x7fd4d80787c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc143a10 tx=0x7fd4ec001780 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:05.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.631+0000 7fd4f9600640 1 -- 172.21.15.167:0/3928105058 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd4e40612f0 con 0x7fd4dc0052e0 2024-05-11T12:14:05.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.825+0000 7fd503400640 1 -- 172.21.15.167:0/3928105058 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd4fc13f690 con 0x7fd4dc0052e0 2024-05-11T12:14:05.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.827+0000 7fd4f9600640 1 -- 172.21.15.167:0/3928105058 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+16946 (secure 0 0 0) 0x7fd4e4060bf0 con 0x7fd4dc0052e0 2024-05-11T12:14:05.828 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:05.828 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":85,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:14:04.862029+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:14:02.772030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-11T12:14:03.847508+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:14:05.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.830+0000 7fd503400640 1 -- 172.21.15.167:0/3928105058 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd4d8076300 msgr2=0x7fd4d80787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:05.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.830+0000 7fd503400640 1 --2- 172.21.15.167:0/3928105058 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd4d8076300 0x7fd4d80787c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc143a10 tx=0x7fd4ec001780 comp rx=0 tx=0).stop 2024-05-11T12:14:05.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.830+0000 7fd503400640 1 -- 172.21.15.167:0/3928105058 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4dc0052e0 msgr2=0x7fd4fc157cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:05.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.830+0000 7fd503400640 1 --2- 172.21.15.167:0/3928105058 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4dc0052e0 0x7fd4fc157cb0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400d800 tx=0x7fd4e400dcc0 comp rx=0 tx=0).stop 2024-05-11T12:14:05.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.830+0000 7fd503400640 1 -- 172.21.15.167:0/3928105058 shutdown_connections 2024-05-11T12:14:05.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.830+0000 7fd503400640 1 --2- 172.21.15.167:0/3928105058 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4fc071f70 0x7fd4fc072400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:05.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.830+0000 7fd503400640 1 --2- 172.21.15.167:0/3928105058 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd4d8076300 0x7fd4d80787c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:05.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.830+0000 7fd503400640 1 --2- 172.21.15.167:0/3928105058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4fc143630 0x7fd4fc13f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:05.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.830+0000 7fd503400640 1 --2- 172.21.15.167:0/3928105058 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4dc0052e0 0x7fd4fc157cb0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:05.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.830+0000 7fd503400640 1 -- 172.21.15.167:0/3928105058 >> 172.21.15.167:0/3928105058 conn(0x7fd4fc0cd8f0 msgr2=0x7fd4fc142780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:05.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.830+0000 7fd503400640 1 -- 172.21.15.167:0/3928105058 shutdown_connections 2024-05-11T12:14:05.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:05.830+0000 7fd503400640 1 -- 172.21.15.167:0/3928105058 wait complete. 2024-05-11T12:14:05.848 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:14:06.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.028+0000 7fc038400640 1 Processor -- start 2024-05-11T12:14:06.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.028+0000 7fc038400640 1 -- start start 2024-05-11T12:14:06.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.029+0000 7fc038400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc030157750 0x7fc03015d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:06.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.030+0000 7fc038400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc0300b3ea0 0x7fc030161490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:06.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.030+0000 7fc038400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc0301619d0 0x7fc03006d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:06.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.030+0000 7fc038400640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fc030161000 con 0x7fc0300b3ea0 2024-05-11T12:14:06.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.030+0000 7fc038400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fc03015dc80 con 0x7fc0301619d0 2024-05-11T12:14:06.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.030+0000 7fc038400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc03015df30 con 0x7fc030157750 2024-05-11T12:14:06.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.030+0000 7fc035000640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc0300b3ea0 0x7fc030161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:06.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.030+0000 7fc035000640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc0300b3ea0 0x7fc030161490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45362/0 (socket says 172.21.15.167:45362) 2024-05-11T12:14:06.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.030+0000 7fc035000640 1 -- 172.21.15.167:0/239619802 learned_addr learned my addr 172.21.15.167:0/239619802 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:06.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.030+0000 7fc035a00640 1 --2- 172.21.15.167:0/239619802 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc030157750 0x7fc03015d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:06.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.030+0000 7fc036400640 1 --2- 172.21.15.167:0/239619802 >> v2:172.21.15.204:3300/0 conn(0x7fc0301619d0 0x7fc03006d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:06.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.031+0000 7fc035000640 1 -- 172.21.15.167:0/239619802 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc030157750 msgr2=0x7fc03015d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:06.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.031+0000 7fc035000640 1 --2- 172.21.15.167:0/239619802 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc030157750 0x7fc03015d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:06.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.031+0000 7fc035000640 1 -- 172.21.15.167:0/239619802 >> v2:172.21.15.204:3300/0 conn(0x7fc0301619d0 msgr2=0x7fc03006d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:06.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.031+0000 7fc035000640 1 --2- 172.21.15.167:0/239619802 >> v2:172.21.15.204:3300/0 conn(0x7fc0301619d0 0x7fc03006d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:06.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.031+0000 7fc035000640 1 -- 172.21.15.167:0/239619802 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc03015e210 con 0x7fc0300b3ea0 2024-05-11T12:14:06.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.031+0000 7fc035a00640 1 --2- 172.21.15.167:0/239619802 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc030157750 0x7fc03015d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:06.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.032+0000 7fc035000640 1 --2- 172.21.15.167:0/239619802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0300b3ea0 0x7fc030161490 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fc030160f70 tx=0x7fc01c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5d7747f30e76103 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:06.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.032+0000 7fc02fe00640 1 -- 172.21.15.167:0/239619802 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01c037070 con 0x7fc0300b3ea0 2024-05-11T12:14:06.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.032+0000 7fc02fe00640 1 -- 172.21.15.167:0/239619802 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc01c032440 con 0x7fc0300b3ea0 2024-05-11T12:14:06.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.032+0000 7fc02fe00640 1 -- 172.21.15.167:0/239619802 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01c032810 con 0x7fc0300b3ea0 2024-05-11T12:14:06.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.032+0000 7fc038400640 1 -- 172.21.15.167:0/239619802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0300b3ea0 msgr2=0x7fc030161490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:06.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.032+0000 7fc038400640 1 --2- 172.21.15.167:0/239619802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0300b3ea0 0x7fc030161490 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fc030160f70 tx=0x7fc01c02a1e0 comp rx=0 tx=0).stop 2024-05-11T12:14:06.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.033+0000 7fc038400640 1 -- 172.21.15.167:0/239619802 shutdown_connections 2024-05-11T12:14:06.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.033+0000 7fc038400640 1 --2- 172.21.15.167:0/239619802 >> v2:172.21.15.204:3300/0 conn(0x7fc0301619d0 0x7fc03006d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:06.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.033+0000 7fc038400640 1 --2- 172.21.15.167:0/239619802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0300b3ea0 0x7fc030161490 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:06.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.033+0000 7fc038400640 1 --2- 172.21.15.167:0/239619802 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc030157750 0x7fc03015d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:06.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.033+0000 7fc038400640 1 -- 172.21.15.167:0/239619802 >> 172.21.15.167:0/239619802 conn(0x7fc0300cd8f0 msgr2=0x7fc0300cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:06.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.033+0000 7fc038400640 1 -- 172.21.15.167:0/239619802 shutdown_connections 2024-05-11T12:14:06.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.033+0000 7fc038400640 1 -- 172.21.15.167:0/239619802 wait complete. 2024-05-11T12:14:06.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.034+0000 7fc038400640 1 Processor -- start 2024-05-11T12:14:06.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.034+0000 7fc038400640 1 -- start start 2024-05-11T12:14:06.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.035+0000 7fc038400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0300b3ea0 0x7fc030140740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:06.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.035+0000 7fc038400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc030157750 0x7fc030141fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:06.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.035+0000 7fc038400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0301619d0 0x7fc03006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:06.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.035+0000 7fc038400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc03006d9a0 con 0x7fc0301619d0 2024-05-11T12:14:06.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.035+0000 7fc038400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc030157b30 con 0x7fc0300b3ea0 2024-05-11T12:14:06.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.035+0000 7fc038400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc03006db20 con 0x7fc030157750 2024-05-11T12:14:06.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.036+0000 7fc036400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0301619d0 0x7fc03006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:06.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.036+0000 7fc036400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0301619d0 0x7fc03006e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45366/0 (socket says 172.21.15.167:45366) 2024-05-11T12:14:06.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.036+0000 7fc036400640 1 -- 172.21.15.167:0/1920005374 learned_addr learned my addr 172.21.15.167:0/1920005374 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:06.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.036+0000 7fc035a00640 1 --2- 172.21.15.167:0/1920005374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0300b3ea0 0x7fc030140740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:06.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.036+0000 7fc035000640 1 --2- 172.21.15.167:0/1920005374 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc030157750 0x7fc030141fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:06.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.036+0000 7fc036400640 1 -- 172.21.15.167:0/1920005374 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc030157750 msgr2=0x7fc030141fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:06.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.036+0000 7fc036400640 1 --2- 172.21.15.167:0/1920005374 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc030157750 0x7fc030141fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:06.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.036+0000 7fc036400640 1 -- 172.21.15.167:0/1920005374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0300b3ea0 msgr2=0x7fc030140740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:06.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.036+0000 7fc036400640 1 --2- 172.21.15.167:0/1920005374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0300b3ea0 0x7fc030140740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:06.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.036+0000 7fc036400640 1 -- 172.21.15.167:0/1920005374 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc030142850 con 0x7fc0301619d0 2024-05-11T12:14:06.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.036+0000 7fc035a00640 1 --2- 172.21.15.167:0/1920005374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0300b3ea0 0x7fc030140740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:06.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.037+0000 7fc036400640 1 --2- 172.21.15.167:0/1920005374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0301619d0 0x7fc03006e370 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fc028007e30 tx=0x7fc02800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:06.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.037+0000 7fc02d600640 1 -- 172.21.15.167:0/1920005374 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc028014620 con 0x7fc0301619d0 2024-05-11T12:14:06.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.037+0000 7fc02d600640 1 -- 172.21.15.167:0/1920005374 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc028019070 con 0x7fc0301619d0 2024-05-11T12:14:06.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.037+0000 7fc02d600640 1 -- 172.21.15.167:0/1920005374 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc028014a00 con 0x7fc0301619d0 2024-05-11T12:14:06.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.037+0000 7fc038400640 1 -- 172.21.15.167:0/1920005374 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc030142b30 con 0x7fc0301619d0 2024-05-11T12:14:06.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.037+0000 7fc038400640 1 -- 172.21.15.167:0/1920005374 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0301f4fe0 con 0x7fc0301619d0 2024-05-11T12:14:06.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.038+0000 7fc038400640 1 -- 172.21.15.167:0/1920005374 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc030143050 con 0x7fc0301619d0 2024-05-11T12:14:06.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.039+0000 7fc02d600640 1 -- 172.21.15.167:0/1920005374 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc028014ba0 con 0x7fc0301619d0 2024-05-11T12:14:06.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.040+0000 7fc02d600640 1 --2- 172.21.15.167:0/1920005374 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc0100762d0 0x7fc010078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:06.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.040+0000 7fc02d600640 1 -- 172.21.15.167:0/1920005374 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 6101+0+0 (secure 0 0 0) 0x7fc028098100 con 0x7fc0301619d0 2024-05-11T12:14:06.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.040+0000 7fc035a00640 1 --2- 172.21.15.167:0/1920005374 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc0100762d0 0x7fc010078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:06.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.041+0000 7fc035a00640 1 --2- 172.21.15.167:0/1920005374 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc0100762d0 0x7fc010078790 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc02000b840 tx=0x7fc020008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:06.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.045+0000 7fc02d600640 1 -- 172.21.15.167:0/1920005374 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc030143050 con 0x7fc0301619d0 2024-05-11T12:14:06.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.250+0000 7fc038400640 1 -- 172.21.15.167:0/1920005374 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc0301f4dd0 con 0x7fc0301619d0 2024-05-11T12:14:06.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.253+0000 7fc02d600640 1 -- 172.21.15.167:0/1920005374 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+16946 (secure 0 0 0) 0x7fc028061830 con 0x7fc0301619d0 2024-05-11T12:14:06.254 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:06.254 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":85,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:14:04.862029+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:14:02.772030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-11T12:14:03.847508+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:14:06.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.256+0000 7fc038400640 1 -- 172.21.15.167:0/1920005374 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc0100762d0 msgr2=0x7fc010078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:06.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.256+0000 7fc038400640 1 --2- 172.21.15.167:0/1920005374 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc0100762d0 0x7fc010078790 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc02000b840 tx=0x7fc020008040 comp rx=0 tx=0).stop 2024-05-11T12:14:06.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.256+0000 7fc038400640 1 -- 172.21.15.167:0/1920005374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0301619d0 msgr2=0x7fc03006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:06.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.256+0000 7fc038400640 1 --2- 172.21.15.167:0/1920005374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0301619d0 0x7fc03006e370 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fc028007e30 tx=0x7fc02800e4b0 comp rx=0 tx=0).stop 2024-05-11T12:14:06.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.256+0000 7fc038400640 1 -- 172.21.15.167:0/1920005374 shutdown_connections 2024-05-11T12:14:06.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.256+0000 7fc038400640 1 --2- 172.21.15.167:0/1920005374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0301619d0 0x7fc03006e370 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:06.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.256+0000 7fc038400640 1 --2- 172.21.15.167:0/1920005374 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc030157750 0x7fc030141fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:06.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.256+0000 7fc038400640 1 --2- 172.21.15.167:0/1920005374 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc0100762d0 0x7fc010078790 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:06.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.256+0000 7fc038400640 1 --2- 172.21.15.167:0/1920005374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0300b3ea0 0x7fc030140740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:06.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.256+0000 7fc038400640 1 -- 172.21.15.167:0/1920005374 >> 172.21.15.167:0/1920005374 conn(0x7fc0300cd8f0 msgr2=0x7fc0300cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:06.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.256+0000 7fc038400640 1 -- 172.21.15.167:0/1920005374 shutdown_connections 2024-05-11T12:14:06.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.256+0000 7fc038400640 1 -- 172.21.15.167:0/1920005374 wait complete. 2024-05-11T12:14:06.275 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-11T12:14:06.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.481+0000 7fdd7e600640 1 Processor -- start 2024-05-11T12:14:06.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.482+0000 7fdd7e600640 1 -- start start 2024-05-11T12:14:06.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.483+0000 7fdd7e600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd78159970 0x7fdd780d1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:06.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.483+0000 7fdd7e600640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fdd780d2db0 0x7fdd780d3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:06.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.483+0000 7fdd7e600640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd78159d50 con 0x7fdd780d17b0 2024-05-11T12:14:06.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.483+0000 7fdd7e600640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fdd780d04b0 con 0x7fdd780d2db0 2024-05-11T12:14:06.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.483+0000 7fdd7e600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fdd780d0790 con 0x7fdd78159970 2024-05-11T12:14:06.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.483+0000 7fdd76a00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fdd780d17b0 0x7fdd7806d200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:40410/0 (socket says 172.21.15.167:40410) 2024-05-11T12:14:06.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.483+0000 7fdd76a00640 1 -- 172.21.15.167:0/2267929398 learned_addr learned my addr 172.21.15.167:0/2267929398 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:14:06.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.483+0000 7fdd77e00640 1 --2- 172.21.15.167:0/2267929398 >> v2:172.21.15.167:3300/0 conn(0x7fdd780d2db0 0x7fdd780d3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:06.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.483+0000 7fdd77400640 1 --2- 172.21.15.167:0/2267929398 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd78159970 0x7fdd780d1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:06.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.484+0000 7fdd76000640 1 -- 172.21.15.167:0/2267929398 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3644923000 0 0) 0x7fdd78159d50 con 0x7fdd780d17b0 2024-05-11T12:14:06.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.484+0000 7fdd77e00640 1 -- 172.21.15.167:0/2267929398 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd78159970 msgr2=0x7fdd780d1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:06.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.484+0000 7fdd77e00640 1 --2- 172.21.15.167:0/2267929398 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd78159970 0x7fdd780d1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:06.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.484+0000 7fdd77e00640 1 -- 172.21.15.167:0/2267929398 >> v1:172.21.15.167:6789/0 conn(0x7fdd780d17b0 legacy=0x7fdd7806d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:06.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.484+0000 7fdd77e00640 1 -- 172.21.15.167:0/2267929398 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd780d0a70 con 0x7fdd780d2db0 2024-05-11T12:14:06.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.485+0000 7fdd77e00640 1 --2- 172.21.15.167:0/2267929398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd780d2db0 0x7fdd780d3240 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fdd700097d0 tx=0x7fdd7002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=379d7d79aabc6f9d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:06.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.485+0000 7fdd76000640 1 -- 172.21.15.167:0/2267929398 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd70037070 con 0x7fdd780d2db0 2024-05-11T12:14:06.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.485+0000 7fdd76000640 1 -- 172.21.15.167:0/2267929398 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd70032440 con 0x7fdd780d2db0 2024-05-11T12:14:06.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.485+0000 7fdd76000640 1 -- 172.21.15.167:0/2267929398 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd70032810 con 0x7fdd780d2db0 2024-05-11T12:14:06.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.486+0000 7fdd7e600640 1 -- 172.21.15.167:0/2267929398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd780d2db0 msgr2=0x7fdd780d3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:06.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.486+0000 7fdd7e600640 1 --2- 172.21.15.167:0/2267929398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd780d2db0 0x7fdd780d3240 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fdd700097d0 tx=0x7fdd7002a1c0 comp rx=0 tx=0).stop 2024-05-11T12:14:06.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.486+0000 7fdd7e600640 1 -- 172.21.15.167:0/2267929398 shutdown_connections 2024-05-11T12:14:06.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.486+0000 7fdd7e600640 1 --2- 172.21.15.167:0/2267929398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd780d2db0 0x7fdd780d3240 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:06.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.486+0000 7fdd7e600640 1 --2- 172.21.15.167:0/2267929398 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd78159970 0x7fdd780d1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:06.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.486+0000 7fdd7e600640 1 -- 172.21.15.167:0/2267929398 >> 172.21.15.167:0/2267929398 conn(0x7fdd780d51e0 msgr2=0x7fdd780d55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:06.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.486+0000 7fdd7e600640 1 -- 172.21.15.167:0/2267929398 shutdown_connections 2024-05-11T12:14:06.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.486+0000 7fdd7e600640 1 -- 172.21.15.167:0/2267929398 wait complete. 2024-05-11T12:14:06.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.488+0000 7fdd7e600640 1 Processor -- start 2024-05-11T12:14:06.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.488+0000 7fdd7e600640 1 -- start start 2024-05-11T12:14:06.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.489+0000 7fdd7e600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd780d17b0 0x7fdd78153c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:06.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.489+0000 7fdd7e600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd780d2db0 0x7fdd7814fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:06.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.489+0000 7fdd7e600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd78159970 0x7fdd781f51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:06.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.489+0000 7fdd7e600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdd7806d910 con 0x7fdd78159970 2024-05-11T12:14:06.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.489+0000 7fdd7e600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdd78163d00 con 0x7fdd780d17b0 2024-05-11T12:14:06.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.489+0000 7fdd7e600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fdd7806da90 con 0x7fdd780d2db0 2024-05-11T12:14:06.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.489+0000 7fdd77e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd78159970 0x7fdd781f51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:06.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.489+0000 7fdd77e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd78159970 0x7fdd781f51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45394/0 (socket says 172.21.15.167:45394) 2024-05-11T12:14:06.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.489+0000 7fdd77e00640 1 -- 172.21.15.167:0/3160743605 learned_addr learned my addr 172.21.15.167:0/3160743605 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:06.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.489+0000 7fdd77e00640 1 -- 172.21.15.167:0/3160743605 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd780d2db0 msgr2=0x7fdd7814fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:06.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.489+0000 7fdd77400640 1 --2- 172.21.15.167:0/3160743605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd780d17b0 0x7fdd78153c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:06.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.489+0000 7fdd77e00640 1 --2- 172.21.15.167:0/3160743605 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd780d2db0 0x7fdd7814fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:06.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.489+0000 7fdd77e00640 1 -- 172.21.15.167:0/3160743605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd780d17b0 msgr2=0x7fdd78153c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:06.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.489+0000 7fdd77e00640 1 --2- 172.21.15.167:0/3160743605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd780d17b0 0x7fdd78153c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:06.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.489+0000 7fdd77e00640 1 -- 172.21.15.167:0/3160743605 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd780cd900 con 0x7fdd78159970 2024-05-11T12:14:06.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.490+0000 7fdd77e00640 1 --2- 172.21.15.167:0/3160743605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd78159970 0x7fdd781f51b0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fdd700098f0 tx=0x7fdd700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:06.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.490+0000 7fdd6f400640 1 -- 172.21.15.167:0/3160743605 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd70037050 con 0x7fdd78159970 2024-05-11T12:14:06.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.490+0000 7fdd6f400640 1 -- 172.21.15.167:0/3160743605 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd70038040 con 0x7fdd78159970 2024-05-11T12:14:06.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.490+0000 7fdd6f400640 1 -- 172.21.15.167:0/3160743605 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd70006e10 con 0x7fdd78159970 2024-05-11T12:14:06.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.490+0000 7fdd7e600640 1 -- 172.21.15.167:0/3160743605 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd780cdbe0 con 0x7fdd78159970 2024-05-11T12:14:06.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.490+0000 7fdd7e600640 1 -- 172.21.15.167:0/3160743605 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd780ce1a0 con 0x7fdd78159970 2024-05-11T12:14:06.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.491+0000 7fdd7e600640 1 -- 172.21.15.167:0/3160743605 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd78069bc0 con 0x7fdd78159970 2024-05-11T12:14:06.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.492+0000 7fdd6f400640 1 -- 172.21.15.167:0/3160743605 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fdd70030980 con 0x7fdd78159970 2024-05-11T12:14:06.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.493+0000 7fdd6f400640 1 --2- 172.21.15.167:0/3160743605 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fdd54076300 0x7fdd540787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:06.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.493+0000 7fdd6f400640 1 -- 172.21.15.167:0/3160743605 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7fdd700b8230 con 0x7fdd78159970 2024-05-11T12:14:06.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.494+0000 7fdd77400640 1 --2- 172.21.15.167:0/3160743605 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fdd54076300 0x7fdd540787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:06.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.494+0000 7fdd77400640 1 --2- 172.21.15.167:0/3160743605 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fdd54076300 0x7fdd540787c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fdd68009f30 tx=0x7fdd6800b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:06.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.498+0000 7fdd6f400640 1 -- 172.21.15.167:0/3160743605 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fdd70081960 con 0x7fdd78159970 2024-05-11T12:14:06.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:06.753+0000 7fdd7e600640 1 -- 172.21.15.167:0/3160743605 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fdd780d00f0 con 0x7fdd78159970 2024-05-11T12:14:07.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.422+0000 7fdd6f400640 1 -- 172.21.15.167:0/3160743605 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v52) ==== 118+0+0 (secure 0 0 0) 0x7fdd70032e20 con 0x7fdd78159970 2024-05-11T12:14:07.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.426+0000 7fdd7e600640 1 -- 172.21.15.167:0/3160743605 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fdd54076300 msgr2=0x7fdd540787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:07.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.426+0000 7fdd7e600640 1 --2- 172.21.15.167:0/3160743605 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fdd54076300 0x7fdd540787c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fdd68009f30 tx=0x7fdd6800b040 comp rx=0 tx=0).stop 2024-05-11T12:14:07.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.426+0000 7fdd7e600640 1 -- 172.21.15.167:0/3160743605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd78159970 msgr2=0x7fdd781f51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:07.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.426+0000 7fdd7e600640 1 --2- 172.21.15.167:0/3160743605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd78159970 0x7fdd781f51b0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fdd700098f0 tx=0x7fdd700046c0 comp rx=0 tx=0).stop 2024-05-11T12:14:07.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.426+0000 7fdd7e600640 1 -- 172.21.15.167:0/3160743605 shutdown_connections 2024-05-11T12:14:07.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.426+0000 7fdd7e600640 1 --2- 172.21.15.167:0/3160743605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd78159970 0x7fdd781f51b0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:07.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.426+0000 7fdd7e600640 1 --2- 172.21.15.167:0/3160743605 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd780d2db0 0x7fdd7814fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:07.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.426+0000 7fdd7e600640 1 --2- 172.21.15.167:0/3160743605 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fdd54076300 0x7fdd540787c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:07.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.426+0000 7fdd7e600640 1 --2- 172.21.15.167:0/3160743605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd780d17b0 0x7fdd78153c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:07.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.426+0000 7fdd7e600640 1 -- 172.21.15.167:0/3160743605 >> 172.21.15.167:0/3160743605 conn(0x7fdd780d51e0 msgr2=0x7fdd781621e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:07.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.426+0000 7fdd7e600640 1 -- 172.21.15.167:0/3160743605 shutdown_connections 2024-05-11T12:14:07.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.426+0000 7fdd7e600640 1 -- 172.21.15.167:0/3160743605 wait complete. 2024-05-11T12:14:07.450 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-11T12:14:07.450 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-11T12:14:07.511 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T12:14:07.516 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T12:14:07.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.653+0000 7f01ee600640 1 Processor -- start 2024-05-11T12:14:07.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.653+0000 7f01ee600640 1 -- start start 2024-05-11T12:14:07.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.654+0000 7f01ee600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f01e8161380 0x7f01e8074260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:07.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.655+0000 7f01ee600640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01e80b65e0 con 0x7f01e815ac80 2024-05-11T12:14:07.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.655+0000 7f01ee600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01e80b68c0 con 0x7f01e80b3d50 2024-05-11T12:14:07.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.655+0000 7f01ee600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f01e806e3c0 con 0x7f01e8161380 2024-05-11T12:14:07.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.655+0000 7f01e7400640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f01e815ac80 0x7f01e8160c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:40422/0 (socket says 172.21.15.167:40422) 2024-05-11T12:14:07.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.655+0000 7f01e7400640 1 -- 172.21.15.167:0/1556474942 learned_addr learned my addr 172.21.15.167:0/1556474942 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:14:07.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.655+0000 7f01e6000640 1 -- 172.21.15.167:0/1556474942 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1635725187 0 0) 0x7f01e80b65e0 con 0x7f01e815ac80 2024-05-11T12:14:07.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.655+0000 7f01e6a00640 1 --2- 172.21.15.167:0/1556474942 >> v2:172.21.15.204:3300/0 conn(0x7f01e8161380 0x7f01e8074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:07.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.655+0000 7f01e6000640 1 -- 172.21.15.167:0/1556474942 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01cc003520 con 0x7f01e815ac80 2024-05-11T12:14:07.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.656+0000 7f01e6000640 1 -- 172.21.15.167:0/1556474942 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2476634517 0 0) 0x7f01cc003520 con 0x7f01e815ac80 2024-05-11T12:14:07.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.656+0000 7f01e6000640 1 -- 172.21.15.167:0/1556474942 >> v2:172.21.15.204:3300/0 conn(0x7f01e8161380 msgr2=0x7f01e8074260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:07.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.656+0000 7f01e6000640 1 --2- 172.21.15.167:0/1556474942 >> v2:172.21.15.204:3300/0 conn(0x7f01e8161380 0x7f01e8074260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:07.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.656+0000 7f01e6000640 1 -- 172.21.15.167:0/1556474942 >> v1:172.21.15.204:6789/0 conn(0x7f01e80b3d50 legacy=0x7f01e8072950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:07.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.656+0000 7f01e6000640 1 -- 172.21.15.167:0/1556474942 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01e80b6ba0 con 0x7f01e815ac80 2024-05-11T12:14:07.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.656+0000 7f01e6000640 1 -- 172.21.15.167:0/1556474942 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f01d80016f0 con 0x7f01e815ac80 2024-05-11T12:14:07.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.656+0000 7f01e6000640 1 -- 172.21.15.167:0/1556474942 >> v1:172.21.15.167:6789/0 conn(0x7f01e815ac80 legacy=0x7f01e8160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:07.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.656+0000 7f01e6000640 1 --2- 172.21.15.167:0/1556474942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01cc003a50 0x7f01cc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:07.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.656+0000 7f01e6000640 1 --2- 172.21.15.167:0/1556474942 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01cc004800 0x7f01cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:07.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.656+0000 7f01e6000640 1 --2- 172.21.15.167:0/1556474942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01cc0052e0 0x7f01cc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:07.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.656+0000 7f01e6000640 1 -- 172.21.15.167:0/1556474942 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f01cc00c180 con 0x7f01cc0052e0 2024-05-11T12:14:07.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.656+0000 7f01e6000640 1 -- 172.21.15.167:0/1556474942 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f01cc00c330 con 0x7f01cc003a50 2024-05-11T12:14:07.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.656+0000 7f01e6000640 1 -- 172.21.15.167:0/1556474942 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f01cc00c530 con 0x7f01cc004800 2024-05-11T12:14:07.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.656+0000 7f01e7e00640 1 --2- 172.21.15.167:0/1556474942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01cc0052e0 0x7f01cc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:07.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.656+0000 7f01e7400640 1 --2- 172.21.15.167:0/1556474942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01cc003a50 0x7f01cc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:07.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.657+0000 7f01e6a00640 1 --2- 172.21.15.167:0/1556474942 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01cc004800 0x7f01cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:07.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.657+0000 7f01e7e00640 1 -- 172.21.15.167:0/1556474942 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01cc004800 msgr2=0x7f01cc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:07.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.657+0000 7f01e7e00640 1 --2- 172.21.15.167:0/1556474942 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01cc004800 0x7f01cc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:07.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.657+0000 7f01e7e00640 1 -- 172.21.15.167:0/1556474942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01cc003a50 msgr2=0x7f01cc003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:07.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.657+0000 7f01e7e00640 1 --2- 172.21.15.167:0/1556474942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01cc003a50 0x7f01cc003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:07.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.657+0000 7f01e7e00640 1 -- 172.21.15.167:0/1556474942 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01cc003520 con 0x7f01cc0052e0 2024-05-11T12:14:07.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.658+0000 7f01e7e00640 1 -- 172.21.15.167:0/1556474942 reap_dead start 2024-05-11T12:14:07.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.658+0000 7f01e7e00640 1 --2- 172.21.15.167:0/1556474942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01cc0052e0 0x7f01cc00b9c0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f01e0001e60 tx=0x7f01cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51c962fdb1dfb044 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:07.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.658+0000 7f01e6000640 1 -- 172.21.15.167:0/1556474942 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e0001ab0 con 0x7f01cc0052e0 2024-05-11T12:14:07.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.658+0000 7f01e6000640 1 -- 172.21.15.167:0/1556474942 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01e002ae40 con 0x7f01cc0052e0 2024-05-11T12:14:07.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.658+0000 7f01e6000640 1 -- 172.21.15.167:0/1556474942 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e0003bb0 con 0x7f01cc0052e0 2024-05-11T12:14:07.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.659+0000 7f01ee600640 1 -- 172.21.15.167:0/1556474942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01cc0052e0 msgr2=0x7f01cc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:07.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.659+0000 7f01ee600640 1 --2- 172.21.15.167:0/1556474942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01cc0052e0 0x7f01cc00b9c0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f01e0001e60 tx=0x7f01cc005830 comp rx=0 tx=0).stop 2024-05-11T12:14:07.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.659+0000 7f01ee600640 1 -- 172.21.15.167:0/1556474942 shutdown_connections 2024-05-11T12:14:07.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.659+0000 7f01ee600640 1 --2- 172.21.15.167:0/1556474942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01cc0052e0 0x7f01cc00b9c0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:07.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.659+0000 7f01ee600640 1 -- 172.21.15.167:0/1556474942 >> 172.21.15.167:0/1556474942 conn(0x7f01e806c450 msgr2=0x7f01e806c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:07.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.659+0000 7f01ee600640 1 -- 172.21.15.167:0/1556474942 shutdown_connections 2024-05-11T12:14:07.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.659+0000 7f01ee600640 1 -- 172.21.15.167:0/1556474942 wait complete. 2024-05-11T12:14:07.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.661+0000 7f01ee600640 1 Processor -- start 2024-05-11T12:14:07.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.661+0000 7f01ee600640 1 -- start start 2024-05-11T12:14:07.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.662+0000 7f01ee600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01cc0052e0 0x7f01e813f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:07.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.662+0000 7f01ee600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01e81f6210 0x7f01e81f6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:07.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.662+0000 7f01ee600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01e815c360 0x7f01e815c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:07.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.662+0000 7f01ee600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f01e80b4160 con 0x7f01cc0052e0 2024-05-11T12:14:07.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.662+0000 7f01ee600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f01e815b060 con 0x7f01e81f6210 2024-05-11T12:14:07.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.662+0000 7f01ee600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f01e8074260 con 0x7f01e815c360 2024-05-11T12:14:07.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.662+0000 7f01e7400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01cc0052e0 0x7f01e813f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:07.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.662+0000 7f01e7400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01cc0052e0 0x7f01e813f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45418/0 (socket says 172.21.15.167:45418) 2024-05-11T12:14:07.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.662+0000 7f01e7400640 1 -- 172.21.15.167:0/3279626321 learned_addr learned my addr 172.21.15.167:0/3279626321 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:07.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.662+0000 7f01e6a00640 1 --2- 172.21.15.167:0/3279626321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01e81f6210 0x7f01e81f6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:07.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.663+0000 7f01e7400640 1 -- 172.21.15.167:0/3279626321 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01e815c360 msgr2=0x7f01e815c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:07.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.663+0000 7f01e7400640 1 --2- 172.21.15.167:0/3279626321 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01e815c360 0x7f01e815c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:07.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.663+0000 7f01e7400640 1 -- 172.21.15.167:0/3279626321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01e81f6210 msgr2=0x7f01e81f6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:07.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.663+0000 7f01e7400640 1 --2- 172.21.15.167:0/3279626321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01e81f6210 0x7f01e81f6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:07.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.663+0000 7f01e7400640 1 -- 172.21.15.167:0/3279626321 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01e81f6cd0 con 0x7f01cc0052e0 2024-05-11T12:14:07.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.663+0000 7f01e7400640 1 --2- 172.21.15.167:0/3279626321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01cc0052e0 0x7f01e813f730 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f01d8007fd0 tx=0x7f01d80077a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:07.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.663+0000 7f01df400640 1 -- 172.21.15.167:0/3279626321 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d801a070 con 0x7f01cc0052e0 2024-05-11T12:14:07.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.663+0000 7f01df400640 1 -- 172.21.15.167:0/3279626321 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01d800f960 con 0x7f01cc0052e0 2024-05-11T12:14:07.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.663+0000 7f01df400640 1 -- 172.21.15.167:0/3279626321 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d800fd30 con 0x7f01cc0052e0 2024-05-11T12:14:07.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.663+0000 7f01ee600640 1 -- 172.21.15.167:0/3279626321 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01e81f6fb0 con 0x7f01cc0052e0 2024-05-11T12:14:07.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.663+0000 7f01ee600640 1 -- 172.21.15.167:0/3279626321 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01e813fc70 con 0x7f01cc0052e0 2024-05-11T12:14:07.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.665+0000 7f01ee600640 1 -- 172.21.15.167:0/3279626321 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01e8069520 con 0x7f01cc0052e0 2024-05-11T12:14:07.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.666+0000 7f01df400640 1 -- 172.21.15.167:0/3279626321 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f01d801d360 con 0x7f01cc0052e0 2024-05-11T12:14:07.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.667+0000 7f01df400640 1 --2- 172.21.15.167:0/3279626321 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f01c80762d0 0x7f01c8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:07.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.667+0000 7f01df400640 1 -- 172.21.15.167:0/3279626321 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7f01d8097200 con 0x7f01cc0052e0 2024-05-11T12:14:07.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.667+0000 7f01e6a00640 1 --2- 172.21.15.167:0/3279626321 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f01c80762d0 0x7f01c8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:07.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.668+0000 7f01e6a00640 1 --2- 172.21.15.167:0/3279626321 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f01c80762d0 0x7f01c8078790 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f01e8145700 tx=0x7f01d4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:07.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.672+0000 7f01df400640 1 -- 172.21.15.167:0/3279626321 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f01d8060a40 con 0x7f01cc0052e0 2024-05-11T12:14:07.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:07.924+0000 7f01ee600640 1 -- 172.21.15.167:0/3279626321 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f01e8070120 con 0x7f01cc0052e0 2024-05-11T12:14:08.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.442+0000 7f01df400640 1 -- 172.21.15.167:0/3279626321 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v53) ==== 122+0+0 (secure 0 0 0) 0x7f01d8060340 con 0x7f01cc0052e0 2024-05-11T12:14:08.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.446+0000 7f01ee600640 1 -- 172.21.15.167:0/3279626321 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f01c80762d0 msgr2=0x7f01c8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:08.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.446+0000 7f01ee600640 1 --2- 172.21.15.167:0/3279626321 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f01c80762d0 0x7f01c8078790 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f01e8145700 tx=0x7f01d4008040 comp rx=0 tx=0).stop 2024-05-11T12:14:08.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.446+0000 7f01ee600640 1 -- 172.21.15.167:0/3279626321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01cc0052e0 msgr2=0x7f01e813f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:08.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.446+0000 7f01ee600640 1 --2- 172.21.15.167:0/3279626321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01cc0052e0 0x7f01e813f730 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f01d8007fd0 tx=0x7f01d80077a0 comp rx=0 tx=0).stop 2024-05-11T12:14:08.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.446+0000 7f01ee600640 1 -- 172.21.15.167:0/3279626321 shutdown_connections 2024-05-11T12:14:08.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.446+0000 7f01ee600640 1 --2- 172.21.15.167:0/3279626321 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01e815c360 0x7f01e815c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:08.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.446+0000 7f01ee600640 1 --2- 172.21.15.167:0/3279626321 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f01c80762d0 0x7f01c8078790 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:08.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.446+0000 7f01ee600640 1 --2- 172.21.15.167:0/3279626321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01e81f6210 0x7f01e81f6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:08.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.446+0000 7f01ee600640 1 --2- 172.21.15.167:0/3279626321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01cc0052e0 0x7f01e813f730 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:08.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.446+0000 7f01ee600640 1 -- 172.21.15.167:0/3279626321 >> 172.21.15.167:0/3279626321 conn(0x7f01e806c450 msgr2=0x7f01e806eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:08.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.447+0000 7f01ee600640 1 -- 172.21.15.167:0/3279626321 shutdown_connections 2024-05-11T12:14:08.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.447+0000 7f01ee600640 1 -- 172.21.15.167:0/3279626321 wait complete. 2024-05-11T12:14:08.470 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-11T12:14:08.470 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:14:08.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.717+0000 7f9d2ea00640 1 Processor -- start 2024-05-11T12:14:08.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.717+0000 7f9d2ea00640 1 -- start start 2024-05-11T12:14:08.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.719+0000 7f9d2ea00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9d28157750 0x7f9d2815d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:08.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.719+0000 7f9d2ea00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d280b3ea0 0x7f9d28161490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:08.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.719+0000 7f9d2ea00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d2815dc80 con 0x7f9d281619d0 2024-05-11T12:14:08.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.719+0000 7f9d2ea00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f9d28161000 con 0x7f9d28157750 2024-05-11T12:14:08.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.719+0000 7f9d2ea00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f9d2815df30 con 0x7f9d280b3ea0 2024-05-11T12:14:08.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.720+0000 7f9d1fe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d280b3ea0 0x7f9d28161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:08.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.720+0000 7f9d1fe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d280b3ea0 0x7f9d28161490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:50634/0 (socket says 172.21.15.167:50634) 2024-05-11T12:14:08.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.720+0000 7f9d1fe00640 1 -- 172.21.15.167:0/97009152 learned_addr learned my addr 172.21.15.167:0/97009152 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:08.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.720+0000 7f9d27e00640 1 --2- 172.21.15.167:0/97009152 >> v2:172.21.15.204:3300/0 conn(0x7f9d28157750 0x7f9d2815d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:08.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.721+0000 7f9d27400640 1 -- 172.21.15.167:0/97009152 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3721840753 0 0) 0x7f9d2815dc80 con 0x7f9d281619d0 2024-05-11T12:14:08.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.721+0000 7f9d27400640 1 -- 172.21.15.167:0/97009152 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d0c003520 con 0x7f9d281619d0 2024-05-11T12:14:08.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.721+0000 7f9d1fe00640 1 -- 172.21.15.167:0/97009152 >> v2:172.21.15.204:3300/0 conn(0x7f9d28157750 msgr2=0x7f9d2815d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:08.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.721+0000 7f9d1fe00640 1 --2- 172.21.15.167:0/97009152 >> v2:172.21.15.204:3300/0 conn(0x7f9d28157750 0x7f9d2815d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:08.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.721+0000 7f9d1fe00640 1 -- 172.21.15.167:0/97009152 >> v1:172.21.15.204:6789/0 conn(0x7f9d281619d0 legacy=0x7f9d2806d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:08.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.721+0000 7f9d1fe00640 1 -- 172.21.15.167:0/97009152 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d2815e210 con 0x7f9d280b3ea0 2024-05-11T12:14:08.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.722+0000 7f9d1fe00640 1 --2- 172.21.15.167:0/97009152 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d280b3ea0 0x7f9d28161490 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9d10009870 tx=0x7f9d1002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64613e3746f3d01b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:08.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.723+0000 7f9d27400640 1 -- 172.21.15.167:0/97009152 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d10037070 con 0x7f9d280b3ea0 2024-05-11T12:14:08.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.723+0000 7f9d27400640 1 -- 172.21.15.167:0/97009152 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d10032440 con 0x7f9d280b3ea0 2024-05-11T12:14:08.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.723+0000 7f9d27400640 1 -- 172.21.15.167:0/97009152 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d100327f0 con 0x7f9d280b3ea0 2024-05-11T12:14:08.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.723+0000 7f9d2ea00640 1 -- 172.21.15.167:0/97009152 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d280b3ea0 msgr2=0x7f9d28161490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:08.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.723+0000 7f9d2ea00640 1 --2- 172.21.15.167:0/97009152 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d280b3ea0 0x7f9d28161490 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9d10009870 tx=0x7f9d1002a250 comp rx=0 tx=0).stop 2024-05-11T12:14:08.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.724+0000 7f9d2ea00640 1 -- 172.21.15.167:0/97009152 shutdown_connections 2024-05-11T12:14:08.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.724+0000 7f9d2ea00640 1 --2- 172.21.15.167:0/97009152 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d280b3ea0 0x7f9d28161490 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:08.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.724+0000 7f9d2ea00640 1 --2- 172.21.15.167:0/97009152 >> v2:172.21.15.204:3300/0 conn(0x7f9d28157750 0x7f9d2815d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:08.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.724+0000 7f9d2ea00640 1 -- 172.21.15.167:0/97009152 >> 172.21.15.167:0/97009152 conn(0x7f9d280cd8f0 msgr2=0x7f9d280cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:08.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.724+0000 7f9d2ea00640 1 -- 172.21.15.167:0/97009152 shutdown_connections 2024-05-11T12:14:08.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.724+0000 7f9d2ea00640 1 -- 172.21.15.167:0/97009152 wait complete. 2024-05-11T12:14:08.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.725+0000 7f9d2ea00640 1 Processor -- start 2024-05-11T12:14:08.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.726+0000 7f9d2ea00640 1 -- start start 2024-05-11T12:14:08.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.726+0000 7f9d2ea00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d280b3ea0 0x7f9d28158750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:08.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.726+0000 7f9d2ea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d28157750 0x7f9d28073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:08.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.726+0000 7f9d2ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d281619d0 0x7f9d2813d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:08.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.726+0000 7f9d2ea00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9d2806db70 con 0x7f9d28157750 2024-05-11T12:14:08.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.726+0000 7f9d2ea00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9d28157b30 con 0x7f9d281619d0 2024-05-11T12:14:08.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.727+0000 7f9d2ea00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f9d2806dcf0 con 0x7f9d280b3ea0 2024-05-11T12:14:08.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.727+0000 7f9d1fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d28157750 0x7f9d28073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:08.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.727+0000 7f9d1fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d28157750 0x7f9d28073330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45424/0 (socket says 172.21.15.167:45424) 2024-05-11T12:14:08.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.727+0000 7f9d1fe00640 1 -- 172.21.15.167:0/1663871072 learned_addr learned my addr 172.21.15.167:0/1663871072 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:08.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.727+0000 7f9d27e00640 1 --2- 172.21.15.167:0/1663871072 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d280b3ea0 0x7f9d28158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:08.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.727+0000 7f9d2ca00640 1 --2- 172.21.15.167:0/1663871072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d281619d0 0x7f9d2813d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:08.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.727+0000 7f9d1fe00640 1 -- 172.21.15.167:0/1663871072 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d280b3ea0 msgr2=0x7f9d28158750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:08.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.727+0000 7f9d1fe00640 1 --2- 172.21.15.167:0/1663871072 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d280b3ea0 0x7f9d28158750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:08.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.727+0000 7f9d1fe00640 1 -- 172.21.15.167:0/1663871072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d281619d0 msgr2=0x7f9d2813d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:08.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.727+0000 7f9d1fe00640 1 --2- 172.21.15.167:0/1663871072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d281619d0 0x7f9d2813d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:08.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.727+0000 7f9d1fe00640 1 -- 172.21.15.167:0/1663871072 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d2815a140 con 0x7f9d28157750 2024-05-11T12:14:08.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.728+0000 7f9d1fe00640 1 --2- 172.21.15.167:0/1663871072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d28157750 0x7f9d28073330 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f9d10035040 tx=0x7f9d100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:08.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.728+0000 7f9d24c00640 1 -- 172.21.15.167:0/1663871072 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d10037040 con 0x7f9d28157750 2024-05-11T12:14:08.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.728+0000 7f9d24c00640 1 -- 172.21.15.167:0/1663871072 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d10003a40 con 0x7f9d28157750 2024-05-11T12:14:08.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.728+0000 7f9d2ea00640 1 -- 172.21.15.167:0/1663871072 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d28159380 con 0x7f9d28157750 2024-05-11T12:14:08.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.728+0000 7f9d24c00640 1 -- 172.21.15.167:0/1663871072 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d10003e10 con 0x7f9d28157750 2024-05-11T12:14:08.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.728+0000 7f9d2ea00640 1 -- 172.21.15.167:0/1663871072 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d28159940 con 0x7f9d28157750 2024-05-11T12:14:08.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.729+0000 7f9d2ea00640 1 -- 172.21.15.167:0/1663871072 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d280b1b00 con 0x7f9d28157750 2024-05-11T12:14:08.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.731+0000 7f9d24c00640 1 -- 172.21.15.167:0/1663871072 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9d10044020 con 0x7f9d28157750 2024-05-11T12:14:08.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.732+0000 7f9d24c00640 1 --2- 172.21.15.167:0/1663871072 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f9d040762d0 0x7f9d04078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:08.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.732+0000 7f9d24c00640 1 -- 172.21.15.167:0/1663871072 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7f9d100b7710 con 0x7f9d28157750 2024-05-11T12:14:08.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.732+0000 7f9d27e00640 1 --2- 172.21.15.167:0/1663871072 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f9d040762d0 0x7f9d04078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:08.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.733+0000 7f9d27e00640 1 --2- 172.21.15.167:0/1663871072 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f9d040762d0 0x7f9d04078790 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9d18009ad0 tx=0x7f9d18008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:08.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.737+0000 7f9d24c00640 1 -- 172.21.15.167:0/1663871072 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9d280b1b00 con 0x7f9d28157750 2024-05-11T12:14:08.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.984+0000 7f9d2ea00640 1 -- 172.21.15.167:0/1663871072 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9d2815e820 con 0x7f9d28157750 2024-05-11T12:14:08.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.985+0000 7f9d24c00640 1 -- 172.21.15.167:0/1663871072 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5195 (secure 0 0 0) 0x7f9d1004b020 con 0x7f9d28157750 2024-05-11T12:14:08.986 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:08.986 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4922,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1154357925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1154357925},{"type":"v1","addr":"172.21.15.167:6837","nonce":1154357925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":5155,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1256314893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1256314893},{"type":"v1","addr":"172.21.15.204:6833","nonce":1256314893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":5158,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1578748665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1578748665},{"type":"v1","addr":"172.21.15.204:6835","nonce":1578748665}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:14:04.861998+0000","modified":"2024-05-11T12:14:07.926157+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6300":{"gid":6300,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6836/3484070468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3484070468},{"type":"v1","addr":"172.21.15.167:6836","nonce":3484070468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6300,"qdb_cluster":[6300]},"id":6}]} 2024-05-11T12:14:08.986 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 53 2024-05-11T12:14:08.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.988+0000 7f9d2ea00640 1 -- 172.21.15.167:0/1663871072 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f9d040762d0 msgr2=0x7f9d04078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:08.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.988+0000 7f9d2ea00640 1 --2- 172.21.15.167:0/1663871072 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f9d040762d0 0x7f9d04078790 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9d18009ad0 tx=0x7f9d18008040 comp rx=0 tx=0).stop 2024-05-11T12:14:08.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.988+0000 7f9d2ea00640 1 -- 172.21.15.167:0/1663871072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d28157750 msgr2=0x7f9d28073330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:08.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.988+0000 7f9d2ea00640 1 --2- 172.21.15.167:0/1663871072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d28157750 0x7f9d28073330 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f9d10035040 tx=0x7f9d100046c0 comp rx=0 tx=0).stop 2024-05-11T12:14:08.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.988+0000 7f9d2ea00640 1 -- 172.21.15.167:0/1663871072 shutdown_connections 2024-05-11T12:14:08.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.988+0000 7f9d2ea00640 1 --2- 172.21.15.167:0/1663871072 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f9d040762d0 0x7f9d04078790 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:08.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.988+0000 7f9d2ea00640 1 --2- 172.21.15.167:0/1663871072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d281619d0 0x7f9d2813d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:08.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.988+0000 7f9d2ea00640 1 --2- 172.21.15.167:0/1663871072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d28157750 0x7f9d28073330 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:08.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.988+0000 7f9d2ea00640 1 --2- 172.21.15.167:0/1663871072 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d280b3ea0 0x7f9d28158750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:08.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.988+0000 7f9d2ea00640 1 -- 172.21.15.167:0/1663871072 >> 172.21.15.167:0/1663871072 conn(0x7f9d280cd8f0 msgr2=0x7f9d280cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:08.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.988+0000 7f9d2ea00640 1 -- 172.21.15.167:0/1663871072 shutdown_connections 2024-05-11T12:14:08.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:08.989+0000 7f9d2ea00640 1 -- 172.21.15.167:0/1663871072 wait complete. 2024-05-11T12:14:09.008 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:14:09.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.211+0000 7eff5e200640 1 Processor -- start 2024-05-11T12:14:09.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.211+0000 7eff5e200640 1 -- start start 2024-05-11T12:14:09.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.212+0000 7eff5e200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7eff58145b00 0x7eff5806f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:09.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.212+0000 7eff5e200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff5806c660 0x7eff5806ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:09.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.213+0000 7eff5e200640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff5806cfb0 con 0x7eff58146770 2024-05-11T12:14:09.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.213+0000 7eff5e200640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7eff5806fbc0 con 0x7eff58145b00 2024-05-11T12:14:09.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.213+0000 7eff5e200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7eff580b4520 con 0x7eff5806c660 2024-05-11T12:14:09.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.213+0000 7eff57e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7eff58146770 0x7eff580ce9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:46456/0 (socket says 172.21.15.167:46456) 2024-05-11T12:14:09.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.213+0000 7eff57e00640 1 -- 172.21.15.167:0/2507185026 learned_addr learned my addr 172.21.15.167:0/2507185026 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:14:09.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.213+0000 7eff5ce00640 1 --2- 172.21.15.167:0/2507185026 >> v2:172.21.15.167:3300/0 conn(0x7eff58145b00 0x7eff5806f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:09.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.213+0000 7eff5d800640 1 --2- 172.21.15.167:0/2507185026 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff5806c660 0x7eff5806ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:09.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.213+0000 7eff5ce00640 1 -- 172.21.15.167:0/2507185026 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff5806c660 msgr2=0x7eff5806ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:09.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.213+0000 7eff5ce00640 1 --2- 172.21.15.167:0/2507185026 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff5806c660 0x7eff5806ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:09.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.214+0000 7eff5ce00640 1 -- 172.21.15.167:0/2507185026 >> v1:172.21.15.167:6789/0 conn(0x7eff58146770 legacy=0x7eff580ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:09.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.214+0000 7eff5ce00640 1 -- 172.21.15.167:0/2507185026 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff580b4800 con 0x7eff58145b00 2024-05-11T12:14:09.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.214+0000 7eff57400640 1 -- 172.21.15.167:0/2507185026 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1299575326 0 0) 0x7eff48001110 con 0x7eff58146770 2024-05-11T12:14:09.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.215+0000 7eff5ce00640 1 --2- 172.21.15.167:0/2507185026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff58145b00 0x7eff5806f2c0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7eff4c0097a0 tx=0x7eff4c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9127194f77435fea server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:09.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.215+0000 7eff57400640 1 -- 172.21.15.167:0/2507185026 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4c037070 con 0x7eff58145b00 2024-05-11T12:14:09.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.215+0000 7eff57400640 1 -- 172.21.15.167:0/2507185026 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff4c02ae50 con 0x7eff58145b00 2024-05-11T12:14:09.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.215+0000 7eff57400640 1 -- 172.21.15.167:0/2507185026 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4c032660 con 0x7eff58145b00 2024-05-11T12:14:09.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.216+0000 7eff5e200640 1 -- 172.21.15.167:0/2507185026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff58145b00 msgr2=0x7eff5806f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:09.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.216+0000 7eff5e200640 1 --2- 172.21.15.167:0/2507185026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff58145b00 0x7eff5806f2c0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7eff4c0097a0 tx=0x7eff4c02a100 comp rx=0 tx=0).stop 2024-05-11T12:14:09.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.216+0000 7eff5e200640 1 -- 172.21.15.167:0/2507185026 shutdown_connections 2024-05-11T12:14:09.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.216+0000 7eff5e200640 1 --2- 172.21.15.167:0/2507185026 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff5806c660 0x7eff5806ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:09.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.216+0000 7eff5e200640 1 --2- 172.21.15.167:0/2507185026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff58145b00 0x7eff5806f2c0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:09.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.216+0000 7eff5e200640 1 -- 172.21.15.167:0/2507185026 >> 172.21.15.167:0/2507185026 conn(0x7eff580b3460 msgr2=0x7eff580d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:09.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.216+0000 7eff5e200640 1 -- 172.21.15.167:0/2507185026 shutdown_connections 2024-05-11T12:14:09.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.216+0000 7eff5e200640 1 -- 172.21.15.167:0/2507185026 wait complete. 2024-05-11T12:14:09.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.218+0000 7eff5e200640 1 Processor -- start 2024-05-11T12:14:09.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.218+0000 7eff5e200640 1 -- start start 2024-05-11T12:14:09.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.219+0000 7eff5e200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff5806c660 0x7eff581e7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:09.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.219+0000 7eff5e200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff58145b00 0x7eff581e7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:09.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.219+0000 7eff5e200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff58146770 0x7eff5815c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:09.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.219+0000 7eff5e200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7eff58145ee0 con 0x7eff58146770 2024-05-11T12:14:09.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.219+0000 7eff5e200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7eff580cf0e0 con 0x7eff5806c660 2024-05-11T12:14:09.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.219+0000 7eff5e200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7eff5806f800 con 0x7eff58145b00 2024-05-11T12:14:09.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.219+0000 7eff5d800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff58146770 0x7eff5815c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:09.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.219+0000 7eff5d800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff58146770 0x7eff5815c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35532/0 (socket says 172.21.15.167:35532) 2024-05-11T12:14:09.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.219+0000 7eff5d800640 1 -- 172.21.15.167:0/2756526345 learned_addr learned my addr 172.21.15.167:0/2756526345 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:09.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.219+0000 7eff57e00640 1 --2- 172.21.15.167:0/2756526345 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff58145b00 0x7eff581e7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:09.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.219+0000 7eff5ce00640 1 --2- 172.21.15.167:0/2756526345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff5806c660 0x7eff581e7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:09.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.219+0000 7eff5d800640 1 -- 172.21.15.167:0/2756526345 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff58145b00 msgr2=0x7eff581e7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:09.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.219+0000 7eff5d800640 1 --2- 172.21.15.167:0/2756526345 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff58145b00 0x7eff581e7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:09.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.219+0000 7eff5d800640 1 -- 172.21.15.167:0/2756526345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff5806c660 msgr2=0x7eff581e7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:09.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.219+0000 7eff5d800640 1 --2- 172.21.15.167:0/2756526345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff5806c660 0x7eff581e7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:09.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.220+0000 7eff5d800640 1 -- 172.21.15.167:0/2756526345 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff5813de00 con 0x7eff58146770 2024-05-11T12:14:09.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.220+0000 7eff5d800640 1 --2- 172.21.15.167:0/2756526345 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff58146770 0x7eff5815c140 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7eff50007e40 tx=0x7eff5000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:09.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.220+0000 7eff54c00640 1 -- 172.21.15.167:0/2756526345 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff50014620 con 0x7eff58146770 2024-05-11T12:14:09.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.220+0000 7eff54c00640 1 -- 172.21.15.167:0/2756526345 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff50019070 con 0x7eff58146770 2024-05-11T12:14:09.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.220+0000 7eff54c00640 1 -- 172.21.15.167:0/2756526345 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff50014a20 con 0x7eff58146770 2024-05-11T12:14:09.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.220+0000 7eff5e200640 1 -- 172.21.15.167:0/2756526345 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff581e85e0 con 0x7eff58146770 2024-05-11T12:14:09.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.220+0000 7eff5e200640 1 -- 172.21.15.167:0/2756526345 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff581e8ba0 con 0x7eff58146770 2024-05-11T12:14:09.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.221+0000 7eff5e200640 1 -- 172.21.15.167:0/2756526345 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff581e9370 con 0x7eff58146770 2024-05-11T12:14:09.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.223+0000 7eff54c00640 1 -- 172.21.15.167:0/2756526345 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7eff50014bc0 con 0x7eff58146770 2024-05-11T12:14:09.224 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.224+0000 7eff54c00640 1 --2- 172.21.15.167:0/2756526345 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7eff3c0762d0 0x7eff3c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:09.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.224+0000 7eff54c00640 1 -- 172.21.15.167:0/2756526345 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7eff50098120 con 0x7eff58146770 2024-05-11T12:14:09.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.224+0000 7eff5ce00640 1 --2- 172.21.15.167:0/2756526345 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7eff3c0762d0 0x7eff3c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:09.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.225+0000 7eff5ce00640 1 --2- 172.21.15.167:0/2756526345 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7eff3c0762d0 0x7eff3c078790 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7eff4c034000 tx=0x7eff4c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:09.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.229+0000 7eff54c00640 1 -- 172.21.15.167:0/2756526345 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7eff50061850 con 0x7eff58146770 2024-05-11T12:14:09.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.439+0000 7eff5e200640 1 -- 172.21.15.167:0/2756526345 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7eff580694b0 con 0x7eff58146770 2024-05-11T12:14:09.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.440+0000 7eff54c00640 1 -- 172.21.15.167:0/2756526345 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+16933 (secure 0 0 0) 0x7eff50061150 con 0x7eff58146770 2024-05-11T12:14:09.441 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:09.441 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":86,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:14:06.410610+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:14:02.772030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-11T12:14:03.847508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:14:09.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.443+0000 7eff5e200640 1 -- 172.21.15.167:0/2756526345 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7eff3c0762d0 msgr2=0x7eff3c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:09.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.443+0000 7eff5e200640 1 --2- 172.21.15.167:0/2756526345 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7eff3c0762d0 0x7eff3c078790 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7eff4c034000 tx=0x7eff4c02e000 comp rx=0 tx=0).stop 2024-05-11T12:14:09.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.443+0000 7eff5e200640 1 -- 172.21.15.167:0/2756526345 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff58146770 msgr2=0x7eff5815c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:09.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.443+0000 7eff5e200640 1 --2- 172.21.15.167:0/2756526345 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff58146770 0x7eff5815c140 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7eff50007e40 tx=0x7eff5000e4b0 comp rx=0 tx=0).stop 2024-05-11T12:14:09.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.443+0000 7eff5e200640 1 -- 172.21.15.167:0/2756526345 shutdown_connections 2024-05-11T12:14:09.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.443+0000 7eff5e200640 1 --2- 172.21.15.167:0/2756526345 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff58146770 0x7eff5815c140 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:09.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.443+0000 7eff5e200640 1 --2- 172.21.15.167:0/2756526345 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff58145b00 0x7eff581e7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:09.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.443+0000 7eff5e200640 1 --2- 172.21.15.167:0/2756526345 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7eff3c0762d0 0x7eff3c078790 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:09.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.443+0000 7eff5e200640 1 --2- 172.21.15.167:0/2756526345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff5806c660 0x7eff581e7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:09.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.443+0000 7eff5e200640 1 -- 172.21.15.167:0/2756526345 >> 172.21.15.167:0/2756526345 conn(0x7eff580b3460 msgr2=0x7eff580ce9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:09.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.444+0000 7eff5e200640 1 -- 172.21.15.167:0/2756526345 shutdown_connections 2024-05-11T12:14:09.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.444+0000 7eff5e200640 1 -- 172.21.15.167:0/2756526345 wait complete. 2024-05-11T12:14:09.462 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-11T12:14:09.462 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T12:14:09.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.656+0000 7f8ec9e00640 1 Processor -- start 2024-05-11T12:14:09.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.657+0000 7f8ec9e00640 1 -- start start 2024-05-11T12:14:09.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.658+0000 7f8ec9e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8ec41598d0 0x7f8ec40d2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:09.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.658+0000 7f8ec9e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f8ec40d0640 0x7f8ec40d0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:09.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.658+0000 7f8ec9e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ec40d1190 con 0x7f8ec40cafb0 2024-05-11T12:14:09.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.658+0000 7f8ec9e00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f8ec406d500 con 0x7f8ec40d0640 2024-05-11T12:14:09.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.658+0000 7f8ec9e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f8ec406d7e0 con 0x7f8ec41598d0 2024-05-11T12:14:09.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.659+0000 7f8ec2000640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f8ec40d0640 0x7f8ec40d0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:09.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.659+0000 7f8ec2000640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f8ec40d0640 0x7f8ec40d0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35536/0 (socket says 172.21.15.167:35536) 2024-05-11T12:14:09.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.659+0000 7f8ec2000640 1 -- 172.21.15.167:0/4281507737 learned_addr learned my addr 172.21.15.167:0/4281507737 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:09.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.659+0000 7f8ec2a00640 1 --2- 172.21.15.167:0/4281507737 >> v2:172.21.15.204:3300/0 conn(0x7f8ec41598d0 0x7f8ec40d2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:09.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.659+0000 7f8ec2000640 1 -- 172.21.15.167:0/4281507737 >> v2:172.21.15.204:3300/0 conn(0x7f8ec41598d0 msgr2=0x7f8ec40d2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:09.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.659+0000 7f8ec2000640 1 --2- 172.21.15.167:0/4281507737 >> v2:172.21.15.204:3300/0 conn(0x7f8ec41598d0 0x7f8ec40d2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:09.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.659+0000 7f8ec2000640 1 -- 172.21.15.167:0/4281507737 >> v1:172.21.15.204:6789/0 conn(0x7f8ec40cafb0 legacy=0x7f8ec406dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:09.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.659+0000 7f8ec2000640 1 -- 172.21.15.167:0/4281507737 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ec406dac0 con 0x7f8ec40d0640 2024-05-11T12:14:09.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.659+0000 7f8ec2a00640 1 --2- 172.21.15.167:0/4281507737 >> v2:172.21.15.204:3300/0 conn(0x7f8ec41598d0 0x7f8ec40d2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:09.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.660+0000 7f8ec2000640 1 --2- 172.21.15.167:0/4281507737 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ec40d0640 0x7f8ec40d0ad0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f8ec408d610 tx=0x7f8eac02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16ca87f2ba1cfe46 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:09.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.661+0000 7f8ec1600640 1 -- 172.21.15.167:0/4281507737 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac037070 con 0x7f8ec40d0640 2024-05-11T12:14:09.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.661+0000 7f8ec1600640 1 -- 172.21.15.167:0/4281507737 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eac032440 con 0x7f8ec40d0640 2024-05-11T12:14:09.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.661+0000 7f8ec1600640 1 -- 172.21.15.167:0/4281507737 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac032810 con 0x7f8ec40d0640 2024-05-11T12:14:09.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.661+0000 7f8ec9e00640 1 -- 172.21.15.167:0/4281507737 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ec40d0640 msgr2=0x7f8ec40d0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:09.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.661+0000 7f8ec9e00640 1 --2- 172.21.15.167:0/4281507737 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ec40d0640 0x7f8ec40d0ad0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f8ec408d610 tx=0x7f8eac02a1e0 comp rx=0 tx=0).stop 2024-05-11T12:14:09.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.661+0000 7f8ec9e00640 1 -- 172.21.15.167:0/4281507737 shutdown_connections 2024-05-11T12:14:09.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.661+0000 7f8ec9e00640 1 --2- 172.21.15.167:0/4281507737 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ec40d0640 0x7f8ec40d0ad0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:09.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.661+0000 7f8ec9e00640 1 --2- 172.21.15.167:0/4281507737 >> v2:172.21.15.204:3300/0 conn(0x7f8ec41598d0 0x7f8ec40d2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:09.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.661+0000 7f8ec9e00640 1 -- 172.21.15.167:0/4281507737 >> 172.21.15.167:0/4281507737 conn(0x7f8ec40d1ba0 msgr2=0x7f8ec40d1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:09.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.661+0000 7f8ec9e00640 1 -- 172.21.15.167:0/4281507737 shutdown_connections 2024-05-11T12:14:09.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.661+0000 7f8ec9e00640 1 -- 172.21.15.167:0/4281507737 wait complete. 2024-05-11T12:14:09.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.663+0000 7f8ec9e00640 1 Processor -- start 2024-05-11T12:14:09.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.663+0000 7f8ec9e00640 1 -- start start 2024-05-11T12:14:09.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.664+0000 7f8ec9e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ec40cafb0 0x7f8ec4149a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:09.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.664+0000 7f8ec9e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8ec40d0640 0x7f8ec40ca200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:09.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.664+0000 7f8ec9e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ec41598d0 0x7f8ec40ca740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:09.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.664+0000 7f8ec9e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec406d1b0 con 0x7f8ec41598d0 2024-05-11T12:14:09.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.664+0000 7f8ec9e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec4159cb0 con 0x7f8ec40cafb0 2024-05-11T12:14:09.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.664+0000 7f8ec9e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f8ec40cb3e0 con 0x7f8ec40d0640 2024-05-11T12:14:09.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.664+0000 7f8ec3400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ec41598d0 0x7f8ec40ca740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:09.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.664+0000 7f8ec3400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ec41598d0 0x7f8ec40ca740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35542/0 (socket says 172.21.15.167:35542) 2024-05-11T12:14:09.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.665+0000 7f8ec3400640 1 -- 172.21.15.167:0/3880879475 learned_addr learned my addr 172.21.15.167:0/3880879475 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:09.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.665+0000 7f8ec2000640 1 --2- 172.21.15.167:0/3880879475 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8ec40d0640 0x7f8ec40ca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:09.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.665+0000 7f8ec2a00640 1 --2- 172.21.15.167:0/3880879475 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ec40cafb0 0x7f8ec4149a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:09.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.665+0000 7f8ec3400640 1 -- 172.21.15.167:0/3880879475 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8ec40d0640 msgr2=0x7f8ec40ca200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:09.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.665+0000 7f8ec3400640 1 --2- 172.21.15.167:0/3880879475 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8ec40d0640 0x7f8ec40ca200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:09.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.665+0000 7f8ec3400640 1 -- 172.21.15.167:0/3880879475 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ec40cafb0 msgr2=0x7f8ec4149a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:09.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.665+0000 7f8ec3400640 1 --2- 172.21.15.167:0/3880879475 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ec40cafb0 0x7f8ec4149a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:09.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.665+0000 7f8ec3400640 1 -- 172.21.15.167:0/3880879475 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ec4149fd0 con 0x7f8ec41598d0 2024-05-11T12:14:09.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.665+0000 7f8ec2000640 1 --2- 172.21.15.167:0/3880879475 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8ec40d0640 0x7f8ec40ca200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:09.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.665+0000 7f8ec3400640 1 --2- 172.21.15.167:0/3880879475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ec41598d0 0x7f8ec40ca740 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc00c440 tx=0x7f8ebc00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:09.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.666+0000 7f8ebb400640 1 -- 172.21.15.167:0/3880879475 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc011790 con 0x7f8ec41598d0 2024-05-11T12:14:09.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.666+0000 7f8ebb400640 1 -- 172.21.15.167:0/3880879475 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ebc017070 con 0x7f8ec41598d0 2024-05-11T12:14:09.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.666+0000 7f8ebb400640 1 -- 172.21.15.167:0/3880879475 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc011b70 con 0x7f8ec41598d0 2024-05-11T12:14:09.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.666+0000 7f8ec9e00640 1 -- 172.21.15.167:0/3880879475 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ec40bb200 con 0x7f8ec41598d0 2024-05-11T12:14:09.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.666+0000 7f8ec9e00640 1 -- 172.21.15.167:0/3880879475 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ec41e9040 con 0x7f8ec41598d0 2024-05-11T12:14:09.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.667+0000 7f8ec9e00640 1 -- 172.21.15.167:0/3880879475 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ec4069590 con 0x7f8ec41598d0 2024-05-11T12:14:09.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.668+0000 7f8ebb400640 1 -- 172.21.15.167:0/3880879475 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8ebc011d10 con 0x7f8ec41598d0 2024-05-11T12:14:09.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.669+0000 7f8ebb400640 1 --2- 172.21.15.167:0/3880879475 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f8ea00762d0 0x7f8ea0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:09.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.669+0000 7f8ebb400640 1 -- 172.21.15.167:0/3880879475 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7f8ebc097600 con 0x7f8ec41598d0 2024-05-11T12:14:09.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.670+0000 7f8ec2a00640 1 --2- 172.21.15.167:0/3880879475 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f8ea00762d0 0x7f8ea0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:09.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.670+0000 7f8ec2a00640 1 --2- 172.21.15.167:0/3880879475 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f8ea00762d0 0x7f8ea0078790 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4007510 tx=0x7f8eb40074a0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:09.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.674+0000 7f8ebb400640 1 -- 172.21.15.167:0/3880879475 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8ebc060d30 con 0x7f8ec41598d0 2024-05-11T12:14:09.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.879+0000 7f8ec9e00640 1 -- 172.21.15.167:0/3880879475 --> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8ec4157740 con 0x7f8ea00762d0 2024-05-11T12:14:09.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.888+0000 7f8ebb400640 1 -- 172.21.15.167:0/3880879475 <== mgr.4106 v2:172.21.15.167:6832/578352559 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252057 (secure 0 0 0) 0x7f8ec4157740 con 0x7f8ea00762d0 2024-05-11T12:14:09.892 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:09.893 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2024-05-11T12:14:09.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.894+0000 7f8ec9e00640 1 -- 172.21.15.167:0/3880879475 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f8ea00762d0 msgr2=0x7f8ea0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:09.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.894+0000 7f8ec9e00640 1 --2- 172.21.15.167:0/3880879475 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f8ea00762d0 0x7f8ea0078790 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4007510 tx=0x7f8eb40074a0 comp rx=0 tx=0).stop 2024-05-11T12:14:09.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.894+0000 7f8ec9e00640 1 -- 172.21.15.167:0/3880879475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ec41598d0 msgr2=0x7f8ec40ca740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:09.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.894+0000 7f8ec9e00640 1 --2- 172.21.15.167:0/3880879475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ec41598d0 0x7f8ec40ca740 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc00c440 tx=0x7f8ebc00c900 comp rx=0 tx=0).stop 2024-05-11T12:14:09.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.894+0000 7f8ec9e00640 1 -- 172.21.15.167:0/3880879475 shutdown_connections 2024-05-11T12:14:09.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.894+0000 7f8ec9e00640 1 --2- 172.21.15.167:0/3880879475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ec41598d0 0x7f8ec40ca740 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:09.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.894+0000 7f8ec9e00640 1 --2- 172.21.15.167:0/3880879475 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8ec40d0640 0x7f8ec40ca200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:09.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.894+0000 7f8ec9e00640 1 --2- 172.21.15.167:0/3880879475 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f8ea00762d0 0x7f8ea0078790 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:09.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.894+0000 7f8ec9e00640 1 --2- 172.21.15.167:0/3880879475 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ec40cafb0 0x7f8ec4149a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:09.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.894+0000 7f8ec9e00640 1 -- 172.21.15.167:0/3880879475 >> 172.21.15.167:0/3880879475 conn(0x7f8ec40d1ba0 msgr2=0x7f8ec4160ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:09.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.894+0000 7f8ec9e00640 1 -- 172.21.15.167:0/3880879475 shutdown_connections 2024-05-11T12:14:09.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:09.894+0000 7f8ec9e00640 1 -- 172.21.15.167:0/3880879475 wait complete. 2024-05-11T12:14:09.913 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":188,"stamp":"2024-05-11T12:14:09.299464+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":174,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":246392,"kb_used_data":9392,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754728328,"statfs":{"total":773094113280,"available":772841807872,"internally_reserved":0,"allocated":9617408,"data_stored":4969644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.925448"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.426058+0000","last_change":"2024-05-11T12:14:05.421973+0000","last_active":"2024-05-11T12:14:06.426058+0000","last_peered":"2024-05-11T12:14:06.426058+0000","last_clean":"2024-05-11T12:14:06.426058+0000","last_became_active":"2024-05-11T12:14:05.421634+0000","last_became_peered":"2024-05-11T12:14:05.421634+0000","last_unstale":"2024-05-11T12:14:06.426058+0000","last_undegraded":"2024-05-11T12:14:06.426058+0000","last_fullsized":"2024-05-11T12:14:06.426058+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:46:38.094747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.427751+0000","last_change":"2024-05-11T12:14:05.427759+0000","last_active":"2024-05-11T12:14:06.427751+0000","last_peered":"2024-05-11T12:14:06.427751+0000","last_clean":"2024-05-11T12:14:06.427751+0000","last_became_active":"2024-05-11T12:14:05.427273+0000","last_became_peered":"2024-05-11T12:14:05.427273+0000","last_unstale":"2024-05-11T12:14:06.427751+0000","last_undegraded":"2024-05-11T12:14:06.427751+0000","last_fullsized":"2024-05-11T12:14:06.427751+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:38:21.667711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.425641+0000","last_change":"2024-05-11T12:14:05.450113+0000","last_active":"2024-05-11T12:14:06.425641+0000","last_peered":"2024-05-11T12:14:06.425641+0000","last_clean":"2024-05-11T12:14:06.425641+0000","last_became_active":"2024-05-11T12:14:05.449774+0000","last_became_peered":"2024-05-11T12:14:05.449774+0000","last_unstale":"2024-05-11T12:14:06.425641+0000","last_undegraded":"2024-05-11T12:14:06.425641+0000","last_fullsized":"2024-05-11T12:14:06.425641+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:45:31.493940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.608617+0000","last_change":"2024-05-11T12:14:05.439504+0000","last_active":"2024-05-11T12:14:06.608617+0000","last_peered":"2024-05-11T12:14:06.608617+0000","last_clean":"2024-05-11T12:14:06.608617+0000","last_became_active":"2024-05-11T12:14:05.438152+0000","last_became_peered":"2024-05-11T12:14:05.438152+0000","last_unstale":"2024-05-11T12:14:06.608617+0000","last_undegraded":"2024-05-11T12:14:06.608617+0000","last_fullsized":"2024-05-11T12:14:06.608617+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:12:22.260345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.430672+0000","last_change":"2024-05-11T12:14:05.444070+0000","last_active":"2024-05-11T12:14:06.430672+0000","last_peered":"2024-05-11T12:14:06.430672+0000","last_clean":"2024-05-11T12:14:06.430672+0000","last_became_active":"2024-05-11T12:14:05.439711+0000","last_became_peered":"2024-05-11T12:14:05.439711+0000","last_unstale":"2024-05-11T12:14:06.430672+0000","last_undegraded":"2024-05-11T12:14:06.430672+0000","last_fullsized":"2024-05-11T12:14:06.430672+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:40:08.268417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.607026+0000","last_change":"2024-05-11T12:14:05.431059+0000","last_active":"2024-05-11T12:14:06.607026+0000","last_peered":"2024-05-11T12:14:06.607026+0000","last_clean":"2024-05-11T12:14:06.607026+0000","last_became_active":"2024-05-11T12:14:05.429549+0000","last_became_peered":"2024-05-11T12:14:05.429549+0000","last_unstale":"2024-05-11T12:14:06.607026+0000","last_undegraded":"2024-05-11T12:14:06.607026+0000","last_fullsized":"2024-05-11T12:14:06.607026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:44:29.512178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.607355+0000","last_change":"2024-05-11T12:14:05.410648+0000","last_active":"2024-05-11T12:14:06.607355+0000","last_peered":"2024-05-11T12:14:06.607355+0000","last_clean":"2024-05-11T12:14:06.607355+0000","last_became_active":"2024-05-11T12:14:05.409914+0000","last_became_peered":"2024-05-11T12:14:05.409914+0000","last_unstale":"2024-05-11T12:14:06.607355+0000","last_undegraded":"2024-05-11T12:14:06.607355+0000","last_fullsized":"2024-05-11T12:14:06.607355+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:45:31.493940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.607245+0000","last_change":"2024-05-11T12:14:05.431246+0000","last_active":"2024-05-11T12:14:06.607245+0000","last_peered":"2024-05-11T12:14:06.607245+0000","last_clean":"2024-05-11T12:14:06.607245+0000","last_became_active":"2024-05-11T12:14:05.430140+0000","last_became_peered":"2024-05-11T12:14:05.430140+0000","last_unstale":"2024-05-11T12:14:06.607245+0000","last_undegraded":"2024-05-11T12:14:06.607245+0000","last_fullsized":"2024-05-11T12:14:06.607245+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:12:51.106509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.431582+0000","last_change":"2024-05-11T12:14:05.438362+0000","last_active":"2024-05-11T12:14:06.431582+0000","last_peered":"2024-05-11T12:14:06.431582+0000","last_clean":"2024-05-11T12:14:06.431582+0000","last_became_active":"2024-05-11T12:14:05.438083+0000","last_became_peered":"2024-05-11T12:14:05.438083+0000","last_unstale":"2024-05-11T12:14:06.431582+0000","last_undegraded":"2024-05-11T12:14:06.431582+0000","last_fullsized":"2024-05-11T12:14:06.431582+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:33:56.761089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-11T12:14:05.444695+0000","last_change":"2024-05-11T12:14:05.437957+0000","last_active":"2024-05-11T12:14:05.444695+0000","last_peered":"2024-05-11T12:14:05.444695+0000","last_clean":"2024-05-11T12:14:05.444695+0000","last_became_active":"2024-05-11T12:14:05.437680+0000","last_became_peered":"2024-05-11T12:14:05.437680+0000","last_unstale":"2024-05-11T12:14:05.444695+0000","last_undegraded":"2024-05-11T12:14:05.444695+0000","last_fullsized":"2024-05-11T12:14:05.444695+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:15:34.687976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.430739+0000","last_change":"2024-05-11T12:14:05.441958+0000","last_active":"2024-05-11T12:14:06.430739+0000","last_peered":"2024-05-11T12:14:06.430739+0000","last_clean":"2024-05-11T12:14:06.430739+0000","last_became_active":"2024-05-11T12:14:05.441608+0000","last_became_peered":"2024-05-11T12:14:05.441608+0000","last_unstale":"2024-05-11T12:14:06.430739+0000","last_undegraded":"2024-05-11T12:14:06.430739+0000","last_fullsized":"2024-05-11T12:14:06.430739+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:11:03.580774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-11T12:14:05.417890+0000","last_change":"2024-05-11T12:14:05.408860+0000","last_active":"2024-05-11T12:14:05.417890+0000","last_peered":"2024-05-11T12:14:05.417890+0000","last_clean":"2024-05-11T12:14:05.417890+0000","last_became_active":"2024-05-11T12:14:05.408576+0000","last_became_peered":"2024-05-11T12:14:05.408576+0000","last_unstale":"2024-05-11T12:14:05.417890+0000","last_undegraded":"2024-05-11T12:14:05.417890+0000","last_fullsized":"2024-05-11T12:14:05.417890+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:18:27.729212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.607893+0000","last_change":"2024-05-11T12:14:05.435827+0000","last_active":"2024-05-11T12:14:06.607893+0000","last_peered":"2024-05-11T12:14:06.607893+0000","last_clean":"2024-05-11T12:14:06.607893+0000","last_became_active":"2024-05-11T12:14:05.431822+0000","last_became_peered":"2024-05-11T12:14:05.431822+0000","last_unstale":"2024-05-11T12:14:06.607893+0000","last_undegraded":"2024-05-11T12:14:06.607893+0000","last_fullsized":"2024-05-11T12:14:06.607893+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:33:56.761089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.429961+0000","last_change":"2024-05-11T12:14:05.442631+0000","last_active":"2024-05-11T12:14:06.429961+0000","last_peered":"2024-05-11T12:14:06.429961+0000","last_clean":"2024-05-11T12:14:06.429961+0000","last_became_active":"2024-05-11T12:14:05.442301+0000","last_became_peered":"2024-05-11T12:14:05.442301+0000","last_unstale":"2024-05-11T12:14:06.429961+0000","last_undegraded":"2024-05-11T12:14:06.429961+0000","last_fullsized":"2024-05-11T12:14:06.429961+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:53:03.501470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.607543+0000","last_change":"2024-05-11T12:14:05.436161+0000","last_active":"2024-05-11T12:14:06.607543+0000","last_peered":"2024-05-11T12:14:06.607543+0000","last_clean":"2024-05-11T12:14:06.607543+0000","last_became_active":"2024-05-11T12:14:05.435898+0000","last_became_peered":"2024-05-11T12:14:05.435898+0000","last_unstale":"2024-05-11T12:14:06.607543+0000","last_undegraded":"2024-05-11T12:14:06.607543+0000","last_fullsized":"2024-05-11T12:14:06.607543+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:12:19.761617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.606204+0000","last_change":"2024-05-11T12:14:05.426591+0000","last_active":"2024-05-11T12:14:06.606204+0000","last_peered":"2024-05-11T12:14:06.606204+0000","last_clean":"2024-05-11T12:14:06.606204+0000","last_became_active":"2024-05-11T12:14:05.426213+0000","last_became_peered":"2024-05-11T12:14:05.426213+0000","last_unstale":"2024-05-11T12:14:06.606204+0000","last_undegraded":"2024-05-11T12:14:06.606204+0000","last_fullsized":"2024-05-11T12:14:06.606204+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:47:30.304698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.609592+0000","last_change":"2024-05-11T12:14:05.425958+0000","last_active":"2024-05-11T12:14:06.609592+0000","last_peered":"2024-05-11T12:14:06.609592+0000","last_clean":"2024-05-11T12:14:06.609592+0000","last_became_active":"2024-05-11T12:14:05.425089+0000","last_became_peered":"2024-05-11T12:14:05.425089+0000","last_unstale":"2024-05-11T12:14:06.609592+0000","last_undegraded":"2024-05-11T12:14:06.609592+0000","last_fullsized":"2024-05-11T12:14:06.609592+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:21:47.157444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.425942+0000","last_change":"2024-05-11T12:14:05.423083+0000","last_active":"2024-05-11T12:14:06.425942+0000","last_peered":"2024-05-11T12:14:06.425942+0000","last_clean":"2024-05-11T12:14:06.425942+0000","last_became_active":"2024-05-11T12:14:05.422817+0000","last_became_peered":"2024-05-11T12:14:05.422817+0000","last_unstale":"2024-05-11T12:14:06.425942+0000","last_undegraded":"2024-05-11T12:14:06.425942+0000","last_fullsized":"2024-05-11T12:14:06.425942+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:15:28.003250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.431299+0000","last_change":"2024-05-11T12:14:05.442561+0000","last_active":"2024-05-11T12:14:06.431299+0000","last_peered":"2024-05-11T12:14:06.431299+0000","last_clean":"2024-05-11T12:14:06.431299+0000","last_became_active":"2024-05-11T12:14:05.442267+0000","last_became_peered":"2024-05-11T12:14:05.442267+0000","last_unstale":"2024-05-11T12:14:06.431299+0000","last_undegraded":"2024-05-11T12:14:06.431299+0000","last_fullsized":"2024-05-11T12:14:06.431299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:46:10.965993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.609001+0000","last_change":"2024-05-11T12:14:05.423746+0000","last_active":"2024-05-11T12:14:06.609001+0000","last_peered":"2024-05-11T12:14:06.609001+0000","last_clean":"2024-05-11T12:14:06.609001+0000","last_became_active":"2024-05-11T12:14:05.422919+0000","last_became_peered":"2024-05-11T12:14:05.422919+0000","last_unstale":"2024-05-11T12:14:06.609001+0000","last_undegraded":"2024-05-11T12:14:06.609001+0000","last_fullsized":"2024-05-11T12:14:06.609001+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:09:14.521912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.430232+0000","last_change":"2024-05-11T12:14:05.440553+0000","last_active":"2024-05-11T12:14:06.430232+0000","last_peered":"2024-05-11T12:14:06.430232+0000","last_clean":"2024-05-11T12:14:06.430232+0000","last_became_active":"2024-05-11T12:14:05.439842+0000","last_became_peered":"2024-05-11T12:14:05.439842+0000","last_unstale":"2024-05-11T12:14:06.430232+0000","last_undegraded":"2024-05-11T12:14:06.430232+0000","last_fullsized":"2024-05-11T12:14:06.430232+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:31:10.245818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.431197+0000","last_change":"2024-05-11T12:14:05.442418+0000","last_active":"2024-05-11T12:14:06.431197+0000","last_peered":"2024-05-11T12:14:06.431197+0000","last_clean":"2024-05-11T12:14:06.431197+0000","last_became_active":"2024-05-11T12:14:05.442104+0000","last_became_peered":"2024-05-11T12:14:05.442104+0000","last_unstale":"2024-05-11T12:14:06.431197+0000","last_undegraded":"2024-05-11T12:14:06.431197+0000","last_fullsized":"2024-05-11T12:14:06.431197+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:12:19.761617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.607279+0000","last_change":"2024-05-11T12:14:05.441909+0000","last_active":"2024-05-11T12:14:06.607279+0000","last_peered":"2024-05-11T12:14:06.607279+0000","last_clean":"2024-05-11T12:14:06.607279+0000","last_became_active":"2024-05-11T12:14:05.440813+0000","last_became_peered":"2024-05-11T12:14:05.440813+0000","last_unstale":"2024-05-11T12:14:06.607279+0000","last_undegraded":"2024-05-11T12:14:06.607279+0000","last_fullsized":"2024-05-11T12:14:06.607279+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:12:51.106509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.425420+0000","last_change":"2024-05-11T12:14:05.448498+0000","last_active":"2024-05-11T12:14:06.425420+0000","last_peered":"2024-05-11T12:14:06.425420+0000","last_clean":"2024-05-11T12:14:06.425420+0000","last_became_active":"2024-05-11T12:14:05.448160+0000","last_became_peered":"2024-05-11T12:14:05.448160+0000","last_unstale":"2024-05-11T12:14:06.425420+0000","last_undegraded":"2024-05-11T12:14:06.425420+0000","last_fullsized":"2024-05-11T12:14:06.425420+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:04:48.659957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.427908+0000","last_change":"2024-05-11T12:14:05.439126+0000","last_active":"2024-05-11T12:14:06.427908+0000","last_peered":"2024-05-11T12:14:06.427908+0000","last_clean":"2024-05-11T12:14:06.427908+0000","last_became_active":"2024-05-11T12:14:05.438829+0000","last_became_peered":"2024-05-11T12:14:05.438829+0000","last_unstale":"2024-05-11T12:14:06.427908+0000","last_undegraded":"2024-05-11T12:14:06.427908+0000","last_fullsized":"2024-05-11T12:14:06.427908+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:10:13.238904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.431996+0000","last_change":"2024-05-11T12:14:05.443650+0000","last_active":"2024-05-11T12:14:06.431996+0000","last_peered":"2024-05-11T12:14:06.431996+0000","last_clean":"2024-05-11T12:14:06.431996+0000","last_became_active":"2024-05-11T12:14:05.443218+0000","last_became_peered":"2024-05-11T12:14:05.443218+0000","last_unstale":"2024-05-11T12:14:06.431996+0000","last_undegraded":"2024-05-11T12:14:06.431996+0000","last_fullsized":"2024-05-11T12:14:06.431996+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:28:06.950871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-11T12:14:05.441473+0000","last_change":"2024-05-11T12:14:05.437816+0000","last_active":"2024-05-11T12:14:05.441473+0000","last_peered":"2024-05-11T12:14:05.441473+0000","last_clean":"2024-05-11T12:14:05.441473+0000","last_became_active":"2024-05-11T12:14:05.437542+0000","last_became_peered":"2024-05-11T12:14:05.437542+0000","last_unstale":"2024-05-11T12:14:05.441473+0000","last_undegraded":"2024-05-11T12:14:05.441473+0000","last_fullsized":"2024-05-11T12:14:05.441473+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:42:41.681582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.609217+0000","last_change":"2024-05-11T12:14:05.444359+0000","last_active":"2024-05-11T12:14:06.609217+0000","last_peered":"2024-05-11T12:14:06.609217+0000","last_clean":"2024-05-11T12:14:06.609217+0000","last_became_active":"2024-05-11T12:14:05.444070+0000","last_became_peered":"2024-05-11T12:14:05.444070+0000","last_unstale":"2024-05-11T12:14:06.609217+0000","last_undegraded":"2024-05-11T12:14:06.609217+0000","last_fullsized":"2024-05-11T12:14:06.609217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:00:50.141176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.608939+0000","last_change":"2024-05-11T12:14:05.424047+0000","last_active":"2024-05-11T12:14:06.608939+0000","last_peered":"2024-05-11T12:14:06.608939+0000","last_clean":"2024-05-11T12:14:06.608939+0000","last_became_active":"2024-05-11T12:14:05.423742+0000","last_became_peered":"2024-05-11T12:14:05.423742+0000","last_unstale":"2024-05-11T12:14:06.608939+0000","last_undegraded":"2024-05-11T12:14:06.608939+0000","last_fullsized":"2024-05-11T12:14:06.608939+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:46:38.094747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.430678+0000","last_change":"2024-05-11T12:14:05.438867+0000","last_active":"2024-05-11T12:14:06.430678+0000","last_peered":"2024-05-11T12:14:06.430678+0000","last_clean":"2024-05-11T12:14:06.430678+0000","last_became_active":"2024-05-11T12:14:05.438031+0000","last_became_peered":"2024-05-11T12:14:05.438031+0000","last_unstale":"2024-05-11T12:14:06.430678+0000","last_undegraded":"2024-05-11T12:14:06.430678+0000","last_fullsized":"2024-05-11T12:14:06.430678+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:03:23.795707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.426352+0000","last_change":"2024-05-11T12:14:05.441006+0000","last_active":"2024-05-11T12:14:06.426352+0000","last_peered":"2024-05-11T12:14:06.426352+0000","last_clean":"2024-05-11T12:14:06.426352+0000","last_became_active":"2024-05-11T12:14:05.440145+0000","last_became_peered":"2024-05-11T12:14:05.440145+0000","last_unstale":"2024-05-11T12:14:06.426352+0000","last_undegraded":"2024-05-11T12:14:06.426352+0000","last_fullsized":"2024-05-11T12:14:06.426352+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:09:14.521912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.606549+0000","last_change":"2024-05-11T12:14:05.425221+0000","last_active":"2024-05-11T12:14:06.606549+0000","last_peered":"2024-05-11T12:14:06.606549+0000","last_clean":"2024-05-11T12:14:06.606549+0000","last_became_active":"2024-05-11T12:14:05.424733+0000","last_became_peered":"2024-05-11T12:14:05.424733+0000","last_unstale":"2024-05-11T12:14:06.606549+0000","last_undegraded":"2024-05-11T12:14:06.606549+0000","last_fullsized":"2024-05-11T12:14:06.606549+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:00:28.938873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.608349+0000","last_change":"2024-05-11T12:14:05.448987+0000","last_active":"2024-05-11T12:14:06.608349+0000","last_peered":"2024-05-11T12:14:06.608349+0000","last_clean":"2024-05-11T12:14:06.608349+0000","last_became_active":"2024-05-11T12:14:05.448334+0000","last_became_peered":"2024-05-11T12:14:05.448334+0000","last_unstale":"2024-05-11T12:14:06.608349+0000","last_undegraded":"2024-05-11T12:14:06.608349+0000","last_fullsized":"2024-05-11T12:14:06.608349+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:14:53.991948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"85'4","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.609606+0000","last_change":"2024-05-11T12:14:04.425347+0000","last_active":"2024-05-11T12:14:06.609606+0000","last_peered":"2024-05-11T12:14:06.609606+0000","last_clean":"2024-05-11T12:14:06.609606+0000","last_became_active":"2024-05-11T12:14:04.425025+0000","last_became_peered":"2024-05-11T12:14:04.425025+0000","last_unstale":"2024-05-11T12:14:06.609606+0000","last_undegraded":"2024-05-11T12:14:06.609606+0000","last_fullsized":"2024-05-11T12:14:06.609606+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:23:15.339626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.608343+0000","last_change":"2024-05-11T12:14:05.427020+0000","last_active":"2024-05-11T12:14:06.608343+0000","last_peered":"2024-05-11T12:14:06.608343+0000","last_clean":"2024-05-11T12:14:06.608343+0000","last_became_active":"2024-05-11T12:14:05.425119+0000","last_became_peered":"2024-05-11T12:14:05.425119+0000","last_unstale":"2024-05-11T12:14:06.608343+0000","last_undegraded":"2024-05-11T12:14:06.608343+0000","last_fullsized":"2024-05-11T12:14:06.608343+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:12:46.904813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.432142+0000","last_change":"2024-05-11T12:14:04.432813+0000","last_active":"2024-05-11T12:14:06.432142+0000","last_peered":"2024-05-11T12:14:06.432142+0000","last_clean":"2024-05-11T12:14:06.432142+0000","last_became_active":"2024-05-11T12:14:04.432455+0000","last_became_peered":"2024-05-11T12:14:04.432455+0000","last_unstale":"2024-05-11T12:14:06.432142+0000","last_undegraded":"2024-05-11T12:14:06.432142+0000","last_fullsized":"2024-05-11T12:14:06.432142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:30:38.517473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.608212+0000","last_change":"2024-05-11T12:14:05.424740+0000","last_active":"2024-05-11T12:14:06.608212+0000","last_peered":"2024-05-11T12:14:06.608212+0000","last_clean":"2024-05-11T12:14:06.608212+0000","last_became_active":"2024-05-11T12:14:05.424106+0000","last_became_peered":"2024-05-11T12:14:05.424106+0000","last_unstale":"2024-05-11T12:14:06.608212+0000","last_undegraded":"2024-05-11T12:14:06.608212+0000","last_fullsized":"2024-05-11T12:14:06.608212+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:07:02.964303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.607806+0000","last_change":"2024-05-11T12:14:04.420632+0000","last_active":"2024-05-11T12:14:06.607806+0000","last_peered":"2024-05-11T12:14:06.607806+0000","last_clean":"2024-05-11T12:14:06.607806+0000","last_became_active":"2024-05-11T12:14:04.416930+0000","last_became_peered":"2024-05-11T12:14:04.416930+0000","last_unstale":"2024-05-11T12:14:06.607806+0000","last_undegraded":"2024-05-11T12:14:06.607806+0000","last_fullsized":"2024-05-11T12:14:06.607806+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:00:49.124605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.609910+0000","last_change":"2024-05-11T12:14:05.438796+0000","last_active":"2024-05-11T12:14:06.609910+0000","last_peered":"2024-05-11T12:14:06.609910+0000","last_clean":"2024-05-11T12:14:06.609910+0000","last_became_active":"2024-05-11T12:14:05.438526+0000","last_became_peered":"2024-05-11T12:14:05.438526+0000","last_unstale":"2024-05-11T12:14:06.609910+0000","last_undegraded":"2024-05-11T12:14:06.609910+0000","last_fullsized":"2024-05-11T12:14:06.609910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:34:51.856988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.608740+0000","last_change":"2024-05-11T12:14:04.426774+0000","last_active":"2024-05-11T12:14:06.608740+0000","last_peered":"2024-05-11T12:14:06.608740+0000","last_clean":"2024-05-11T12:14:06.608740+0000","last_became_active":"2024-05-11T12:14:04.424806+0000","last_became_peered":"2024-05-11T12:14:04.424806+0000","last_unstale":"2024-05-11T12:14:06.608740+0000","last_undegraded":"2024-05-11T12:14:06.608740+0000","last_fullsized":"2024-05-11T12:14:06.608740+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:00:44.293728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.607087+0000","last_change":"2024-05-11T12:14:05.423593+0000","last_active":"2024-05-11T12:14:06.607087+0000","last_peered":"2024-05-11T12:14:06.607087+0000","last_clean":"2024-05-11T12:14:06.607087+0000","last_became_active":"2024-05-11T12:14:05.423304+0000","last_became_peered":"2024-05-11T12:14:05.423304+0000","last_unstale":"2024-05-11T12:14:06.607087+0000","last_undegraded":"2024-05-11T12:14:06.607087+0000","last_fullsized":"2024-05-11T12:14:06.607087+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:11:03.580774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.431065+0000","last_change":"2024-05-11T12:14:04.420760+0000","last_active":"2024-05-11T12:14:06.431065+0000","last_peered":"2024-05-11T12:14:06.431065+0000","last_clean":"2024-05-11T12:14:06.431065+0000","last_became_active":"2024-05-11T12:14:04.419701+0000","last_became_peered":"2024-05-11T12:14:04.419701+0000","last_unstale":"2024-05-11T12:14:06.431065+0000","last_undegraded":"2024-05-11T12:14:06.431065+0000","last_fullsized":"2024-05-11T12:14:06.431065+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:46:46.547028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.607706+0000","last_change":"2024-05-11T12:14:05.435844+0000","last_active":"2024-05-11T12:14:06.607706+0000","last_peered":"2024-05-11T12:14:06.607706+0000","last_clean":"2024-05-11T12:14:06.607706+0000","last_became_active":"2024-05-11T12:14:05.435126+0000","last_became_peered":"2024-05-11T12:14:05.435126+0000","last_unstale":"2024-05-11T12:14:06.607706+0000","last_undegraded":"2024-05-11T12:14:06.607706+0000","last_fullsized":"2024-05-11T12:14:06.607706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:12:19.761617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.610177+0000","last_change":"2024-05-11T12:14:04.417449+0000","last_active":"2024-05-11T12:14:06.610177+0000","last_peered":"2024-05-11T12:14:06.610177+0000","last_clean":"2024-05-11T12:14:06.610177+0000","last_became_active":"2024-05-11T12:14:04.417195+0000","last_became_peered":"2024-05-11T12:14:04.417195+0000","last_unstale":"2024-05-11T12:14:06.610177+0000","last_undegraded":"2024-05-11T12:14:06.610177+0000","last_fullsized":"2024-05-11T12:14:06.610177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:16:17.661069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.607072+0000","last_change":"2024-05-11T12:14:05.438916+0000","last_active":"2024-05-11T12:14:06.607072+0000","last_peered":"2024-05-11T12:14:06.607072+0000","last_clean":"2024-05-11T12:14:06.607072+0000","last_became_active":"2024-05-11T12:14:05.438608+0000","last_became_peered":"2024-05-11T12:14:05.438608+0000","last_unstale":"2024-05-11T12:14:06.607072+0000","last_undegraded":"2024-05-11T12:14:06.607072+0000","last_fullsized":"2024-05-11T12:14:06.607072+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:55:32.026118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.425470+0000","last_change":"2024-05-11T12:14:04.407920+0000","last_active":"2024-05-11T12:14:06.425470+0000","last_peered":"2024-05-11T12:14:06.425470+0000","last_clean":"2024-05-11T12:14:06.425470+0000","last_became_active":"2024-05-11T12:14:04.407605+0000","last_became_peered":"2024-05-11T12:14:04.407605+0000","last_unstale":"2024-05-11T12:14:06.425470+0000","last_undegraded":"2024-05-11T12:14:06.425470+0000","last_fullsized":"2024-05-11T12:14:06.425470+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:09:13.505342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.608016+0000","last_change":"2024-05-11T12:14:05.435212+0000","last_active":"2024-05-11T12:14:06.608016+0000","last_peered":"2024-05-11T12:14:06.608016+0000","last_clean":"2024-05-11T12:14:06.608016+0000","last_became_active":"2024-05-11T12:14:05.434838+0000","last_became_peered":"2024-05-11T12:14:05.434838+0000","last_unstale":"2024-05-11T12:14:06.608016+0000","last_undegraded":"2024-05-11T12:14:06.608016+0000","last_fullsized":"2024-05-11T12:14:06.608016+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:28:06.950871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.607924+0000","last_change":"2024-05-11T12:14:04.422280+0000","last_active":"2024-05-11T12:14:06.607924+0000","last_peered":"2024-05-11T12:14:06.607924+0000","last_clean":"2024-05-11T12:14:06.607924+0000","last_became_active":"2024-05-11T12:14:04.420929+0000","last_became_peered":"2024-05-11T12:14:04.420929+0000","last_unstale":"2024-05-11T12:14:06.607924+0000","last_undegraded":"2024-05-11T12:14:06.607924+0000","last_fullsized":"2024-05-11T12:14:06.607924+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:12:21.243774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.431063+0000","last_change":"2024-05-11T12:14:05.438352+0000","last_active":"2024-05-11T12:14:06.431063+0000","last_peered":"2024-05-11T12:14:06.431063+0000","last_clean":"2024-05-11T12:14:06.431063+0000","last_became_active":"2024-05-11T12:14:05.438027+0000","last_became_peered":"2024-05-11T12:14:05.438027+0000","last_unstale":"2024-05-11T12:14:06.431063+0000","last_undegraded":"2024-05-11T12:14:06.431063+0000","last_fullsized":"2024-05-11T12:14:06.431063+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:12:51.106509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.428110+0000","last_change":"2024-05-11T12:14:04.423576+0000","last_active":"2024-05-11T12:14:06.428110+0000","last_peered":"2024-05-11T12:14:06.428110+0000","last_clean":"2024-05-11T12:14:06.428110+0000","last_became_active":"2024-05-11T12:14:04.423046+0000","last_became_peered":"2024-05-11T12:14:04.423046+0000","last_unstale":"2024-05-11T12:14:06.428110+0000","last_undegraded":"2024-05-11T12:14:06.428110+0000","last_fullsized":"2024-05-11T12:14:06.428110+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:11:02.564203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.607534+0000","last_change":"2024-05-11T12:14:05.445807+0000","last_active":"2024-05-11T12:14:06.607534+0000","last_peered":"2024-05-11T12:14:06.607534+0000","last_clean":"2024-05-11T12:14:06.607534+0000","last_became_active":"2024-05-11T12:14:05.445496+0000","last_became_peered":"2024-05-11T12:14:05.445496+0000","last_unstale":"2024-05-11T12:14:06.607534+0000","last_undegraded":"2024-05-11T12:14:06.607534+0000","last_fullsized":"2024-05-11T12:14:06.607534+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:15:28.003250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.426081+0000","last_change":"2024-05-11T12:14:04.417845+0000","last_active":"2024-05-11T12:14:06.426081+0000","last_peered":"2024-05-11T12:14:06.426081+0000","last_clean":"2024-05-11T12:14:06.426081+0000","last_became_active":"2024-05-11T12:14:04.417547+0000","last_became_peered":"2024-05-11T12:14:04.417547+0000","last_unstale":"2024-05-11T12:14:06.426081+0000","last_undegraded":"2024-05-11T12:14:06.426081+0000","last_fullsized":"2024-05-11T12:14:06.426081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:00:49.124605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.425966+0000","last_change":"2024-05-11T12:14:05.420993+0000","last_active":"2024-05-11T12:14:06.425966+0000","last_peered":"2024-05-11T12:14:06.425966+0000","last_clean":"2024-05-11T12:14:06.425966+0000","last_became_active":"2024-05-11T12:14:05.420702+0000","last_became_peered":"2024-05-11T12:14:05.420702+0000","last_unstale":"2024-05-11T12:14:06.425966+0000","last_undegraded":"2024-05-11T12:14:06.425966+0000","last_fullsized":"2024-05-11T12:14:06.425966+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:07:02.964303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"85'3","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.431214+0000","last_change":"2024-05-11T12:14:04.431738+0000","last_active":"2024-05-11T12:14:06.431214+0000","last_peered":"2024-05-11T12:14:06.431214+0000","last_clean":"2024-05-11T12:14:06.431214+0000","last_became_active":"2024-05-11T12:14:04.430502+0000","last_became_peered":"2024-05-11T12:14:04.430502+0000","last_unstale":"2024-05-11T12:14:06.431214+0000","last_undegraded":"2024-05-11T12:14:06.431214+0000","last_fullsized":"2024-05-11T12:14:06.431214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:22:38.635128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.610402+0000","last_change":"2024-05-11T12:14:05.433943+0000","last_active":"2024-05-11T12:14:06.610402+0000","last_peered":"2024-05-11T12:14:06.610402+0000","last_clean":"2024-05-11T12:14:06.610402+0000","last_became_active":"2024-05-11T12:14:05.433604+0000","last_became_peered":"2024-05-11T12:14:05.433604+0000","last_unstale":"2024-05-11T12:14:06.610402+0000","last_undegraded":"2024-05-11T12:14:06.610402+0000","last_fullsized":"2024-05-11T12:14:06.610402+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:20:57.538769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"85'6","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.607836+0000","last_change":"2024-05-11T12:14:04.427471+0000","last_active":"2024-05-11T12:14:06.607836+0000","last_peered":"2024-05-11T12:14:06.607836+0000","last_clean":"2024-05-11T12:14:06.607836+0000","last_became_active":"2024-05-11T12:14:04.427148+0000","last_became_peered":"2024-05-11T12:14:04.427148+0000","last_unstale":"2024-05-11T12:14:06.607836+0000","last_undegraded":"2024-05-11T12:14:06.607836+0000","last_fullsized":"2024-05-11T12:14:06.607836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:19:35.359726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.607694+0000","last_change":"2024-05-11T12:14:05.431244+0000","last_active":"2024-05-11T12:14:06.607694+0000","last_peered":"2024-05-11T12:14:06.607694+0000","last_clean":"2024-05-11T12:14:06.607694+0000","last_became_active":"2024-05-11T12:14:05.430962+0000","last_became_peered":"2024-05-11T12:14:05.430962+0000","last_unstale":"2024-05-11T12:14:06.607694+0000","last_undegraded":"2024-05-11T12:14:06.607694+0000","last_fullsized":"2024-05-11T12:14:06.607694+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:46:10.965993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.608529+0000","last_change":"2024-05-11T12:14:04.419824+0000","last_active":"2024-05-11T12:14:06.608529+0000","last_peered":"2024-05-11T12:14:06.608529+0000","last_clean":"2024-05-11T12:14:06.608529+0000","last_became_active":"2024-05-11T12:14:04.419546+0000","last_became_peered":"2024-05-11T12:14:04.419546+0000","last_unstale":"2024-05-11T12:14:06.608529+0000","last_undegraded":"2024-05-11T12:14:06.608529+0000","last_fullsized":"2024-05-11T12:14:06.608529+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:38:20.651140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.429894+0000","last_change":"2024-05-11T12:14:05.439531+0000","last_active":"2024-05-11T12:14:06.429894+0000","last_peered":"2024-05-11T12:14:06.429894+0000","last_clean":"2024-05-11T12:14:06.429894+0000","last_became_active":"2024-05-11T12:14:05.437967+0000","last_became_peered":"2024-05-11T12:14:05.437967+0000","last_unstale":"2024-05-11T12:14:06.429894+0000","last_undegraded":"2024-05-11T12:14:06.429894+0000","last_fullsized":"2024-05-11T12:14:06.429894+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:23:16.356197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.607226+0000","last_change":"2024-05-11T12:14:04.429861+0000","last_active":"2024-05-11T12:14:06.607226+0000","last_peered":"2024-05-11T12:14:06.607226+0000","last_clean":"2024-05-11T12:14:06.607226+0000","last_became_active":"2024-05-11T12:14:04.429546+0000","last_became_peered":"2024-05-11T12:14:04.429546+0000","last_unstale":"2024-05-11T12:14:06.607226+0000","last_undegraded":"2024-05-11T12:14:06.607226+0000","last_fullsized":"2024-05-11T12:14:06.607226+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:38:20.651140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-11T12:14:05.433205+0000","last_change":"2024-05-11T12:14:05.425575+0000","last_active":"2024-05-11T12:14:05.433205+0000","last_peered":"2024-05-11T12:14:05.433205+0000","last_clean":"2024-05-11T12:14:05.433205+0000","last_became_active":"2024-05-11T12:14:05.425139+0000","last_became_peered":"2024-05-11T12:14:05.425139+0000","last_unstale":"2024-05-11T12:14:05.433205+0000","last_undegraded":"2024-05-11T12:14:05.433205+0000","last_fullsized":"2024-05-11T12:14:05.433205+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:12:21.767350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.607723+0000","last_change":"2024-05-11T12:14:04.427924+0000","last_active":"2024-05-11T12:14:06.607723+0000","last_peered":"2024-05-11T12:14:06.607723+0000","last_clean":"2024-05-11T12:14:06.607723+0000","last_became_active":"2024-05-11T12:14:04.427196+0000","last_became_peered":"2024-05-11T12:14:04.427196+0000","last_unstale":"2024-05-11T12:14:06.607723+0000","last_undegraded":"2024-05-11T12:14:06.607723+0000","last_fullsized":"2024-05-11T12:14:06.607723+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:16:55.623835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-11T12:14:05.450072+0000","last_change":"2024-05-11T12:14:05.445549+0000","last_active":"2024-05-11T12:14:05.450072+0000","last_peered":"2024-05-11T12:14:05.450072+0000","last_clean":"2024-05-11T12:14:05.450072+0000","last_became_active":"2024-05-11T12:14:05.445136+0000","last_became_peered":"2024-05-11T12:14:05.445136+0000","last_unstale":"2024-05-11T12:14:05.450072+0000","last_undegraded":"2024-05-11T12:14:05.450072+0000","last_fullsized":"2024-05-11T12:14:05.450072+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:19:09.883922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"85'8","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.607998+0000","last_change":"2024-05-11T12:14:04.417414+0000","last_active":"2024-05-11T12:14:06.607998+0000","last_peered":"2024-05-11T12:14:06.607998+0000","last_clean":"2024-05-11T12:14:06.607998+0000","last_became_active":"2024-05-11T12:14:04.417109+0000","last_became_peered":"2024-05-11T12:14:04.417109+0000","last_unstale":"2024-05-11T12:14:06.607998+0000","last_undegraded":"2024-05-11T12:14:06.607998+0000","last_fullsized":"2024-05-11T12:14:06.607998+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:48:53.320829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.607615+0000","last_change":"2024-05-11T12:14:05.449206+0000","last_active":"2024-05-11T12:14:06.607615+0000","last_peered":"2024-05-11T12:14:06.607615+0000","last_clean":"2024-05-11T12:14:06.607615+0000","last_became_active":"2024-05-11T12:14:05.448720+0000","last_became_peered":"2024-05-11T12:14:05.448720+0000","last_unstale":"2024-05-11T12:14:06.607615+0000","last_undegraded":"2024-05-11T12:14:06.607615+0000","last_fullsized":"2024-05-11T12:14:06.607615+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:16:36.104898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.609743+0000","last_change":"2024-05-11T12:14:04.429540+0000","last_active":"2024-05-11T12:14:06.609743+0000","last_peered":"2024-05-11T12:14:06.609743+0000","last_clean":"2024-05-11T12:14:06.609743+0000","last_became_active":"2024-05-11T12:14:04.429111+0000","last_became_peered":"2024-05-11T12:14:04.429111+0000","last_unstale":"2024-05-11T12:14:06.609743+0000","last_undegraded":"2024-05-11T12:14:06.609743+0000","last_fullsized":"2024-05-11T12:14:06.609743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:50:43.887550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.608587+0000","last_change":"2024-05-11T12:14:05.445374+0000","last_active":"2024-05-11T12:14:06.608587+0000","last_peered":"2024-05-11T12:14:06.608587+0000","last_clean":"2024-05-11T12:14:06.608587+0000","last_became_active":"2024-05-11T12:14:05.444923+0000","last_became_peered":"2024-05-11T12:14:05.444923+0000","last_unstale":"2024-05-11T12:14:06.608587+0000","last_undegraded":"2024-05-11T12:14:06.608587+0000","last_fullsized":"2024-05-11T12:14:06.608587+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:28:06.950871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.608535+0000","last_change":"2024-05-11T12:14:04.429525+0000","last_active":"2024-05-11T12:14:06.608535+0000","last_peered":"2024-05-11T12:14:06.608535+0000","last_clean":"2024-05-11T12:14:06.608535+0000","last_became_active":"2024-05-11T12:14:04.429223+0000","last_became_peered":"2024-05-11T12:14:04.429223+0000","last_unstale":"2024-05-11T12:14:06.608535+0000","last_undegraded":"2024-05-11T12:14:06.608535+0000","last_fullsized":"2024-05-11T12:14:06.608535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:20:56.522199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.432050+0000","last_change":"2024-05-11T12:14:05.439212+0000","last_active":"2024-05-11T12:14:06.432050+0000","last_peered":"2024-05-11T12:14:06.432050+0000","last_clean":"2024-05-11T12:14:06.432050+0000","last_became_active":"2024-05-11T12:14:05.438869+0000","last_became_peered":"2024-05-11T12:14:05.438869+0000","last_unstale":"2024-05-11T12:14:06.432050+0000","last_undegraded":"2024-05-11T12:14:06.432050+0000","last_fullsized":"2024-05-11T12:14:06.432050+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:12:22.260345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.431508+0000","last_change":"2024-05-11T12:14:04.419221+0000","last_active":"2024-05-11T12:14:06.431508+0000","last_peered":"2024-05-11T12:14:06.431508+0000","last_clean":"2024-05-11T12:14:06.431508+0000","last_became_active":"2024-05-11T12:14:04.418879+0000","last_became_peered":"2024-05-11T12:14:04.418879+0000","last_unstale":"2024-05-11T12:14:06.431508+0000","last_undegraded":"2024-05-11T12:14:06.431508+0000","last_fullsized":"2024-05-11T12:14:06.431508+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:43:10.502481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.606368+0000","last_change":"2024-05-11T12:14:05.420121+0000","last_active":"2024-05-11T12:14:06.606368+0000","last_peered":"2024-05-11T12:14:06.606368+0000","last_clean":"2024-05-11T12:14:06.606368+0000","last_became_active":"2024-05-11T12:14:05.419581+0000","last_became_peered":"2024-05-11T12:14:05.419581+0000","last_unstale":"2024-05-11T12:14:06.606368+0000","last_undegraded":"2024-05-11T12:14:06.606368+0000","last_fullsized":"2024-05-11T12:14:06.606368+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:56:45.292884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.427462+0000","last_change":"2024-05-11T12:14:04.426347+0000","last_active":"2024-05-11T12:14:06.427462+0000","last_peered":"2024-05-11T12:14:06.427462+0000","last_clean":"2024-05-11T12:14:06.427462+0000","last_became_active":"2024-05-11T12:14:04.426020+0000","last_became_peered":"2024-05-11T12:14:04.426020+0000","last_unstale":"2024-05-11T12:14:06.427462+0000","last_undegraded":"2024-05-11T12:14:06.427462+0000","last_fullsized":"2024-05-11T12:14:06.427462+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:31:09.229247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.426380+0000","last_change":"2024-05-11T12:14:05.423884+0000","last_active":"2024-05-11T12:14:06.426380+0000","last_peered":"2024-05-11T12:14:06.426380+0000","last_clean":"2024-05-11T12:14:06.426380+0000","last_became_active":"2024-05-11T12:14:05.423588+0000","last_became_peered":"2024-05-11T12:14:05.423588+0000","last_unstale":"2024-05-11T12:14:06.426380+0000","last_undegraded":"2024-05-11T12:14:06.426380+0000","last_fullsized":"2024-05-11T12:14:06.426380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:00:50.141176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-11T12:14:05.400337+0000","last_change":"2024-05-11T12:14:04.418068+0000","last_active":"2024-05-11T12:14:05.400337+0000","last_peered":"2024-05-11T12:14:05.400337+0000","last_clean":"2024-05-11T12:14:05.400337+0000","last_became_active":"2024-05-11T12:14:04.417799+0000","last_became_peered":"2024-05-11T12:14:04.417799+0000","last_unstale":"2024-05-11T12:14:05.400337+0000","last_undegraded":"2024-05-11T12:14:05.400337+0000","last_fullsized":"2024-05-11T12:14:05.400337+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:26:06.820640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.427057+0000","last_change":"2024-05-11T12:14:05.435022+0000","last_active":"2024-05-11T12:14:06.427057+0000","last_peered":"2024-05-11T12:14:06.427057+0000","last_clean":"2024-05-11T12:14:06.427057+0000","last_became_active":"2024-05-11T12:14:05.434731+0000","last_became_peered":"2024-05-11T12:14:05.434731+0000","last_unstale":"2024-05-11T12:14:06.427057+0000","last_undegraded":"2024-05-11T12:14:06.427057+0000","last_fullsized":"2024-05-11T12:14:06.427057+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:20:57.538769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.426758+0000","last_change":"2024-05-11T12:14:04.425226+0000","last_active":"2024-05-11T12:14:06.426758+0000","last_peered":"2024-05-11T12:14:06.426758+0000","last_clean":"2024-05-11T12:14:06.426758+0000","last_became_active":"2024-05-11T12:14:04.424079+0000","last_became_peered":"2024-05-11T12:14:04.424079+0000","last_unstale":"2024-05-11T12:14:06.426758+0000","last_undegraded":"2024-05-11T12:14:06.426758+0000","last_fullsized":"2024-05-11T12:14:06.426758+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:03:22.779136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.607617+0000","last_change":"2024-05-11T12:14:05.430748+0000","last_active":"2024-05-11T12:14:06.607617+0000","last_peered":"2024-05-11T12:14:06.607617+0000","last_clean":"2024-05-11T12:14:06.607617+0000","last_became_active":"2024-05-11T12:14:05.426138+0000","last_became_peered":"2024-05-11T12:14:05.426138+0000","last_unstale":"2024-05-11T12:14:06.607617+0000","last_undegraded":"2024-05-11T12:14:06.607617+0000","last_fullsized":"2024-05-11T12:14:06.607617+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:58:44.327216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-11T12:14:05.400232+0000","last_change":"2024-05-11T12:14:04.421430+0000","last_active":"2024-05-11T12:14:05.400232+0000","last_peered":"2024-05-11T12:14:05.400232+0000","last_clean":"2024-05-11T12:14:05.400232+0000","last_became_active":"2024-05-11T12:14:04.420955+0000","last_became_peered":"2024-05-11T12:14:04.420955+0000","last_unstale":"2024-05-11T12:14:05.400232+0000","last_undegraded":"2024-05-11T12:14:05.400232+0000","last_fullsized":"2024-05-11T12:14:05.400232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:09:13.866774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.608543+0000","last_change":"2024-05-11T12:14:05.443776+0000","last_active":"2024-05-11T12:14:06.608543+0000","last_peered":"2024-05-11T12:14:06.608543+0000","last_clean":"2024-05-11T12:14:06.608543+0000","last_became_active":"2024-05-11T12:14:05.443504+0000","last_became_peered":"2024-05-11T12:14:05.443504+0000","last_unstale":"2024-05-11T12:14:06.608543+0000","last_undegraded":"2024-05-11T12:14:06.608543+0000","last_fullsized":"2024-05-11T12:14:06.608543+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:33:56.761089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.609166+0000","last_change":"2024-05-11T12:14:04.428661+0000","last_active":"2024-05-11T12:14:06.609166+0000","last_peered":"2024-05-11T12:14:06.609166+0000","last_clean":"2024-05-11T12:14:06.609166+0000","last_became_active":"2024-05-11T12:14:04.427340+0000","last_became_peered":"2024-05-11T12:14:04.427340+0000","last_unstale":"2024-05-11T12:14:06.609166+0000","last_undegraded":"2024-05-11T12:14:06.609166+0000","last_fullsized":"2024-05-11T12:14:06.609166+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:10:12.222334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.608765+0000","last_change":"2024-05-11T12:14:05.448920+0000","last_active":"2024-05-11T12:14:06.608765+0000","last_peered":"2024-05-11T12:14:06.608765+0000","last_clean":"2024-05-11T12:14:06.608765+0000","last_became_active":"2024-05-11T12:14:05.448475+0000","last_became_peered":"2024-05-11T12:14:05.448475+0000","last_unstale":"2024-05-11T12:14:06.608765+0000","last_undegraded":"2024-05-11T12:14:06.608765+0000","last_fullsized":"2024-05-11T12:14:06.608765+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:10:34.562552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"86'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:07.409528+0000","last_change":"2024-05-11T12:14:04.431814+0000","last_active":"2024-05-11T12:14:07.409528+0000","last_peered":"2024-05-11T12:14:07.409528+0000","last_clean":"2024-05-11T12:14:07.409528+0000","last_became_active":"2024-05-11T12:14:04.431516+0000","last_became_peered":"2024-05-11T12:14:04.431516+0000","last_unstale":"2024-05-11T12:14:07.409528+0000","last_undegraded":"2024-05-11T12:14:07.409528+0000","last_fullsized":"2024-05-11T12:14:07.409528+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:12:42.194230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.608899+0000","last_change":"2024-05-11T12:14:05.425737+0000","last_active":"2024-05-11T12:14:06.608899+0000","last_peered":"2024-05-11T12:14:06.608899+0000","last_clean":"2024-05-11T12:14:06.608899+0000","last_became_active":"2024-05-11T12:14:05.425475+0000","last_became_peered":"2024-05-11T12:14:05.425475+0000","last_unstale":"2024-05-11T12:14:06.608899+0000","last_undegraded":"2024-05-11T12:14:06.608899+0000","last_fullsized":"2024-05-11T12:14:06.608899+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:59:32.672735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.431564+0000","last_change":"2024-05-11T12:14:04.431591+0000","last_active":"2024-05-11T12:14:06.431564+0000","last_peered":"2024-05-11T12:14:06.431564+0000","last_clean":"2024-05-11T12:14:06.431564+0000","last_became_active":"2024-05-11T12:14:04.429447+0000","last_became_peered":"2024-05-11T12:14:04.429447+0000","last_unstale":"2024-05-11T12:14:06.431564+0000","last_undegraded":"2024-05-11T12:14:06.431564+0000","last_fullsized":"2024-05-11T12:14:06.431564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:23:26.454412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.428113+0000","last_change":"2024-05-11T12:14:05.441203+0000","last_active":"2024-05-11T12:14:06.428113+0000","last_peered":"2024-05-11T12:14:06.428113+0000","last_clean":"2024-05-11T12:14:06.428113+0000","last_became_active":"2024-05-11T12:14:05.440747+0000","last_became_peered":"2024-05-11T12:14:05.440747+0000","last_unstale":"2024-05-11T12:14:06.428113+0000","last_undegraded":"2024-05-11T12:14:06.428113+0000","last_fullsized":"2024-05-11T12:14:06.428113+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:19:36.376296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.608804+0000","last_change":"2024-05-11T12:14:04.424655+0000","last_active":"2024-05-11T12:14:06.608804+0000","last_peered":"2024-05-11T12:14:06.608804+0000","last_clean":"2024-05-11T12:14:06.608804+0000","last_became_active":"2024-05-11T12:14:04.423752+0000","last_became_peered":"2024-05-11T12:14:04.423752+0000","last_unstale":"2024-05-11T12:14:06.608804+0000","last_undegraded":"2024-05-11T12:14:06.608804+0000","last_fullsized":"2024-05-11T12:14:06.608804+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:25:39.149525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-11T12:14:05.423260+0000","last_change":"2024-05-11T12:14:05.418926+0000","last_active":"2024-05-11T12:14:05.423260+0000","last_peered":"2024-05-11T12:14:05.423260+0000","last_clean":"2024-05-11T12:14:05.423260+0000","last_became_active":"2024-05-11T12:14:05.418662+0000","last_became_peered":"2024-05-11T12:14:05.418662+0000","last_unstale":"2024-05-11T12:14:05.423260+0000","last_undegraded":"2024-05-11T12:14:05.423260+0000","last_fullsized":"2024-05-11T12:14:05.423260+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:38:12.001547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.608104+0000","last_change":"2024-05-11T12:14:04.429826+0000","last_active":"2024-05-11T12:14:06.608104+0000","last_peered":"2024-05-11T12:14:06.608104+0000","last_clean":"2024-05-11T12:14:06.608104+0000","last_became_active":"2024-05-11T12:14:04.429540+0000","last_became_peered":"2024-05-11T12:14:04.429540+0000","last_unstale":"2024-05-11T12:14:06.608104+0000","last_undegraded":"2024-05-11T12:14:06.608104+0000","last_fullsized":"2024-05-11T12:14:06.608104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:10:12.222334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.425320+0000","last_change":"2024-05-11T12:14:05.422361+0000","last_active":"2024-05-11T12:14:06.425320+0000","last_peered":"2024-05-11T12:14:06.425320+0000","last_clean":"2024-05-11T12:14:06.425320+0000","last_became_active":"2024-05-11T12:14:05.422077+0000","last_became_peered":"2024-05-11T12:14:05.422077+0000","last_unstale":"2024-05-11T12:14:06.425320+0000","last_undegraded":"2024-05-11T12:14:06.425320+0000","last_fullsized":"2024-05-11T12:14:06.425320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:12:35.736138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.427232+0000","last_change":"2024-05-11T12:14:04.426263+0000","last_active":"2024-05-11T12:14:06.427232+0000","last_peered":"2024-05-11T12:14:06.427232+0000","last_clean":"2024-05-11T12:14:06.427232+0000","last_became_active":"2024-05-11T12:14:04.425967+0000","last_became_peered":"2024-05-11T12:14:04.425967+0000","last_unstale":"2024-05-11T12:14:06.427232+0000","last_undegraded":"2024-05-11T12:14:06.427232+0000","last_fullsized":"2024-05-11T12:14:06.427232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:40:07.251846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.608057+0000","last_change":"2024-05-11T12:14:05.444013+0000","last_active":"2024-05-11T12:14:06.608057+0000","last_peered":"2024-05-11T12:14:06.608057+0000","last_clean":"2024-05-11T12:14:06.608057+0000","last_became_active":"2024-05-11T12:14:05.442985+0000","last_became_peered":"2024-05-11T12:14:05.442985+0000","last_unstale":"2024-05-11T12:14:06.608057+0000","last_undegraded":"2024-05-11T12:14:06.608057+0000","last_fullsized":"2024-05-11T12:14:06.608057+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:46:10.965993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.607839+0000","last_change":"2024-05-11T12:14:04.425086+0000","last_active":"2024-05-11T12:14:06.607839+0000","last_peered":"2024-05-11T12:14:06.607839+0000","last_clean":"2024-05-11T12:14:06.607839+0000","last_became_active":"2024-05-11T12:14:04.424035+0000","last_became_peered":"2024-05-11T12:14:04.424035+0000","last_unstale":"2024-05-11T12:14:06.607839+0000","last_undegraded":"2024-05-11T12:14:06.607839+0000","last_fullsized":"2024-05-11T12:14:06.607839+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:08:25.339038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.607919+0000","last_change":"2024-05-11T12:14:05.426326+0000","last_active":"2024-05-11T12:14:06.607919+0000","last_peered":"2024-05-11T12:14:06.607919+0000","last_clean":"2024-05-11T12:14:06.607919+0000","last_became_active":"2024-05-11T12:14:05.424685+0000","last_became_peered":"2024-05-11T12:14:05.424685+0000","last_unstale":"2024-05-11T12:14:06.607919+0000","last_undegraded":"2024-05-11T12:14:06.607919+0000","last_fullsized":"2024-05-11T12:14:06.607919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:15:24.413933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.607423+0000","last_change":"2024-05-11T12:14:04.423204+0000","last_active":"2024-05-11T12:14:06.607423+0000","last_peered":"2024-05-11T12:14:06.607423+0000","last_clean":"2024-05-11T12:14:06.607423+0000","last_became_active":"2024-05-11T12:14:04.422071+0000","last_became_peered":"2024-05-11T12:14:04.422071+0000","last_unstale":"2024-05-11T12:14:06.607423+0000","last_undegraded":"2024-05-11T12:14:06.607423+0000","last_fullsized":"2024-05-11T12:14:06.607423+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:09:13.505342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.609299+0000","last_change":"2024-05-11T12:14:05.419683+0000","last_active":"2024-05-11T12:14:06.609299+0000","last_peered":"2024-05-11T12:14:06.609299+0000","last_clean":"2024-05-11T12:14:06.609299+0000","last_became_active":"2024-05-11T12:14:05.419245+0000","last_became_peered":"2024-05-11T12:14:05.419245+0000","last_unstale":"2024-05-11T12:14:06.609299+0000","last_undegraded":"2024-05-11T12:14:06.609299+0000","last_fullsized":"2024-05-11T12:14:06.609299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:01:03.933240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.609254+0000","last_change":"2024-05-11T12:14:04.424087+0000","last_active":"2024-05-11T12:14:06.609254+0000","last_peered":"2024-05-11T12:14:06.609254+0000","last_clean":"2024-05-11T12:14:06.609254+0000","last_became_active":"2024-05-11T12:14:04.423810+0000","last_became_peered":"2024-05-11T12:14:04.423810+0000","last_unstale":"2024-05-11T12:14:06.609254+0000","last_undegraded":"2024-05-11T12:14:06.609254+0000","last_fullsized":"2024-05-11T12:14:06.609254+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:15:09.957490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":227,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.607598+0000","last_change":"2024-05-11T12:10:35.066673+0000","last_active":"2024-05-11T12:14:06.607598+0000","last_peered":"2024-05-11T12:14:06.607598+0000","last_clean":"2024-05-11T12:14:06.607598+0000","last_became_active":"2024-05-11T12:10:35.065942+0000","last_became_peered":"2024-05-11T12:10:35.065942+0000","last_unstale":"2024-05-11T12:14:06.607598+0000","last_undegraded":"2024-05-11T12:14:06.607598+0000","last_fullsized":"2024-05-11T12:14:06.607598+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_clean_scrub_stamp":"2024-05-11T12:10:33.601136+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:12:02.133182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476782,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32796,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339044,"statfs":{"total":96636764160,"available":96603181056,"internally_reserved":0,"allocated":1650688,"data_stored":1065108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476783,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343772,"statfs":{"total":96636764160,"available":96608022528,"internally_reserved":0,"allocated":1069056,"data_stored":476900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476782,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":1048576,"data_stored":474684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476784,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1056768,"data_stored":474844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509487,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1048576,"data_stored":474684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509488,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343800,"statfs":{"total":96636764160,"available":96608051200,"internally_reserved":0,"allocated":1040384,"data_stored":468062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509487,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32812,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1667072,"data_stored":1067370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542191,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1036288,"data_stored":467992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T12:14:09.917 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T12:14:10.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.155+0000 7fae4b000640 1 Processor -- start 2024-05-11T12:14:10.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.155+0000 7fae4b000640 1 -- start start 2024-05-11T12:14:10.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.157+0000 7fae4b000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fae44157750 0x7fae4415d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:10.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.157+0000 7fae4b000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fae440b3ea0 0x7fae44161490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:10.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.157+0000 7fae4b000640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae4415dc80 con 0x7fae441619d0 2024-05-11T12:14:10.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.157+0000 7fae4b000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fae44161000 con 0x7fae440b3ea0 2024-05-11T12:14:10.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.157+0000 7fae4b000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fae4415df30 con 0x7fae44157750 2024-05-11T12:14:10.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.157+0000 7fae49000640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fae441619d0 0x7fae4406d460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:46464/0 (socket says 172.21.15.167:46464) 2024-05-11T12:14:10.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.157+0000 7fae49000640 1 -- 172.21.15.167:0/1659647206 learned_addr learned my addr 172.21.15.167:0/1659647206 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:14:10.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.158+0000 7fae3be00640 1 --2- 172.21.15.167:0/1659647206 >> v2:172.21.15.204:3300/0 conn(0x7fae440b3ea0 0x7fae44161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:10.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.158+0000 7fae43e00640 1 --2- 172.21.15.167:0/1659647206 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fae44157750 0x7fae4415d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:10.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.158+0000 7fae43400640 1 -- 172.21.15.167:0/1659647206 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 740587648 0 0) 0x7fae4415dc80 con 0x7fae441619d0 2024-05-11T12:14:10.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.158+0000 7fae43400640 1 -- 172.21.15.167:0/1659647206 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fae28003520 con 0x7fae441619d0 2024-05-11T12:14:10.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.158+0000 7fae43400640 1 -- 172.21.15.167:0/1659647206 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3731815458 0 0) 0x7fae28003520 con 0x7fae441619d0 2024-05-11T12:14:10.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.158+0000 7fae43400640 1 -- 172.21.15.167:0/1659647206 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fae44157750 msgr2=0x7fae4415d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:10.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.158+0000 7fae43400640 1 --2- 172.21.15.167:0/1659647206 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fae44157750 0x7fae4415d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:10.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.158+0000 7fae43400640 1 -- 172.21.15.167:0/1659647206 >> v2:172.21.15.204:3300/0 conn(0x7fae440b3ea0 msgr2=0x7fae44161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:10.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.158+0000 7fae43400640 1 --2- 172.21.15.167:0/1659647206 >> v2:172.21.15.204:3300/0 conn(0x7fae440b3ea0 0x7fae44161490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:10.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.158+0000 7fae43400640 1 -- 172.21.15.167:0/1659647206 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae4415e210 con 0x7fae441619d0 2024-05-11T12:14:10.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.158+0000 7fae43400640 1 -- 172.21.15.167:0/1659647206 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7fae3c001a70 con 0x7fae441619d0 2024-05-11T12:14:10.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.158+0000 7fae43400640 1 -- 172.21.15.167:0/1659647206 >> v1:172.21.15.167:6789/0 conn(0x7fae441619d0 legacy=0x7fae4406d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:10.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.158+0000 7fae43400640 1 --2- 172.21.15.167:0/1659647206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae28003a50 0x7fae28003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:10.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.158+0000 7fae43400640 1 --2- 172.21.15.167:0/1659647206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae28004800 0x7fae28004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:10.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.158+0000 7fae43400640 1 --2- 172.21.15.167:0/1659647206 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fae44157750 0x7fae280051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:10.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.158+0000 7fae43400640 1 -- 172.21.15.167:0/1659647206 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fae44161000 con 0x7fae28004800 2024-05-11T12:14:10.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.158+0000 7fae43400640 1 -- 172.21.15.167:0/1659647206 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fae4415df30 con 0x7fae28003a50 2024-05-11T12:14:10.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.158+0000 7fae43400640 1 -- 172.21.15.167:0/1659647206 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fae2800bb50 con 0x7fae44157750 2024-05-11T12:14:10.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.158+0000 7fae3be00640 1 --2- 172.21.15.167:0/1659647206 >> v2:172.21.15.204:3300/0 conn(0x7fae440b3ea0 0x7fae44161490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:14:10.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.159+0000 7fae3be00640 1 --2- 172.21.15.167:0/1659647206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae28004800 0x7fae28004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:10.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.159+0000 7fae43e00640 1 --2- 172.21.15.167:0/1659647206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae28003a50 0x7fae28003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:10.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.159+0000 7fae3be00640 1 -- 172.21.15.167:0/1659647206 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fae44157750 msgr2=0x7fae280051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:10.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.159+0000 7fae49000640 1 --2- 172.21.15.167:0/1659647206 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fae44157750 0x7fae280051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:10.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.159+0000 7fae3be00640 1 --2- 172.21.15.167:0/1659647206 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fae44157750 0x7fae280051e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:10.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.159+0000 7fae3be00640 1 -- 172.21.15.167:0/1659647206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae28003a50 msgr2=0x7fae28003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:10.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.159+0000 7fae3be00640 1 --2- 172.21.15.167:0/1659647206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae28003a50 0x7fae28003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:10.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.159+0000 7fae3be00640 1 -- 172.21.15.167:0/1659647206 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae28003520 con 0x7fae28004800 2024-05-11T12:14:10.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.159+0000 7fae43e00640 1 --2- 172.21.15.167:0/1659647206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae28003a50 0x7fae28003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:14:10.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.160+0000 7fae3be00640 1 --2- 172.21.15.167:0/1659647206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae28004800 0x7fae28004c80 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fae2c0098a0 tx=0x7fae28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b248de80d302126 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:10.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.161+0000 7fae43400640 1 -- 172.21.15.167:0/1659647206 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2c037070 con 0x7fae28004800 2024-05-11T12:14:10.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.161+0000 7fae43400640 1 -- 172.21.15.167:0/1659647206 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae2c02ad20 con 0x7fae28004800 2024-05-11T12:14:10.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.161+0000 7fae43400640 1 -- 172.21.15.167:0/1659647206 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2c00f040 con 0x7fae28004800 2024-05-11T12:14:10.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.161+0000 7fae4b000640 1 -- 172.21.15.167:0/1659647206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae28004800 msgr2=0x7fae28004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:10.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.161+0000 7fae4b000640 1 --2- 172.21.15.167:0/1659647206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae28004800 0x7fae28004c80 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fae2c0098a0 tx=0x7fae28005830 comp rx=0 tx=0).stop 2024-05-11T12:14:10.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.161+0000 7fae49000640 1 -- 172.21.15.167:0/1659647206 reap_dead start 2024-05-11T12:14:10.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.161+0000 7fae4b000640 1 -- 172.21.15.167:0/1659647206 shutdown_connections 2024-05-11T12:14:10.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.161+0000 7fae4b000640 1 -- 172.21.15.167:0/1659647206 >> 172.21.15.167:0/1659647206 conn(0x7fae440cd8f0 msgr2=0x7fae440cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:10.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.161+0000 7fae4b000640 1 -- 172.21.15.167:0/1659647206 shutdown_connections 2024-05-11T12:14:10.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.161+0000 7fae4b000640 1 -- 172.21.15.167:0/1659647206 wait complete. 2024-05-11T12:14:10.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.163+0000 7fae4b000640 1 Processor -- start 2024-05-11T12:14:10.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.163+0000 7fae4b000640 1 -- start start 2024-05-11T12:14:10.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.164+0000 7fae4b000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae28004800 0x7fae441f5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:10.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.164+0000 7fae4b000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae441f5fa0 0x7fae440ab270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:10.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.164+0000 7fae4b000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fae440ab7b0 0x7fae440730c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:10.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.164+0000 7fae4b000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fae4406db70 con 0x7fae28004800 2024-05-11T12:14:10.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.164+0000 7fae4b000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fae44157b30 con 0x7fae441f5fa0 2024-05-11T12:14:10.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.164+0000 7fae4b000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fae4406dcf0 con 0x7fae440ab7b0 2024-05-11T12:14:10.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.164+0000 7fae43e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae28004800 0x7fae441f5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:10.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.164+0000 7fae43e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae28004800 0x7fae441f5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35564/0 (socket says 172.21.15.167:35564) 2024-05-11T12:14:10.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.164+0000 7fae43e00640 1 -- 172.21.15.167:0/662511678 learned_addr learned my addr 172.21.15.167:0/662511678 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:10.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.165+0000 7fae43e00640 1 -- 172.21.15.167:0/662511678 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fae440ab7b0 msgr2=0x7fae440730c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:10.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.165+0000 7fae43e00640 1 --2- 172.21.15.167:0/662511678 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fae440ab7b0 0x7fae440730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:10.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.165+0000 7fae43e00640 1 -- 172.21.15.167:0/662511678 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae441f5fa0 msgr2=0x7fae440ab270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:10.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.165+0000 7fae43e00640 1 --2- 172.21.15.167:0/662511678 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae441f5fa0 0x7fae440ab270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:10.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.165+0000 7fae43e00640 1 -- 172.21.15.167:0/662511678 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae441f5d80 con 0x7fae28004800 2024-05-11T12:14:10.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.165+0000 7fae43e00640 1 --2- 172.21.15.167:0/662511678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae28004800 0x7fae441f5840 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fae3400f510 tx=0x7fae34004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:10.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.165+0000 7fae40c00640 1 -- 172.21.15.167:0/662511678 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae34018020 con 0x7fae28004800 2024-05-11T12:14:10.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.165+0000 7fae40c00640 1 -- 172.21.15.167:0/662511678 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae34014870 con 0x7fae28004800 2024-05-11T12:14:10.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.165+0000 7fae40c00640 1 -- 172.21.15.167:0/662511678 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae34014c40 con 0x7fae28004800 2024-05-11T12:14:10.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.165+0000 7fae4b000640 1 -- 172.21.15.167:0/662511678 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae44143cd0 con 0x7fae28004800 2024-05-11T12:14:10.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.165+0000 7fae4b000640 1 -- 172.21.15.167:0/662511678 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae44144270 con 0x7fae28004800 2024-05-11T12:14:10.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.166+0000 7fae4b000640 1 -- 172.21.15.167:0/662511678 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae44069590 con 0x7fae28004800 2024-05-11T12:14:10.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.167+0000 7fae40c00640 1 -- 172.21.15.167:0/662511678 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fae3401b020 con 0x7fae28004800 2024-05-11T12:14:10.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.168+0000 7fae40c00640 1 --2- 172.21.15.167:0/662511678 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fae20076300 0x7fae200787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:10.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.169+0000 7fae40c00640 1 -- 172.21.15.167:0/662511678 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7fae34097b40 con 0x7fae28004800 2024-05-11T12:14:10.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.169+0000 7fae3be00640 1 --2- 172.21.15.167:0/662511678 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fae20076300 0x7fae200787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:10.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.169+0000 7fae3be00640 1 --2- 172.21.15.167:0/662511678 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fae20076300 0x7fae200787c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fae2c034000 tx=0x7fae2c02c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:10.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.173+0000 7fae40c00640 1 -- 172.21.15.167:0/662511678 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fae34022020 con 0x7fae28004800 2024-05-11T12:14:10.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.362+0000 7fae4b000640 1 -- 172.21.15.167:0/662511678 --> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fae440ce0c0 con 0x7fae20076300 2024-05-11T12:14:10.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.381+0000 7fae40c00640 1 -- 172.21.15.167:0/662511678 <== mgr.4106 v2:172.21.15.167:6832/578352559 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252057 (secure 0 0 0) 0x7fae440ce0c0 con 0x7fae20076300 2024-05-11T12:14:10.385 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:10.387 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2024-05-11T12:14:10.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.387+0000 7fae4b000640 1 -- 172.21.15.167:0/662511678 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fae20076300 msgr2=0x7fae200787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:10.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.387+0000 7fae4b000640 1 --2- 172.21.15.167:0/662511678 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fae20076300 0x7fae200787c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fae2c034000 tx=0x7fae2c02c000 comp rx=0 tx=0).stop 2024-05-11T12:14:10.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.387+0000 7fae4b000640 1 -- 172.21.15.167:0/662511678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae28004800 msgr2=0x7fae441f5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:10.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.387+0000 7fae4b000640 1 --2- 172.21.15.167:0/662511678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae28004800 0x7fae441f5840 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fae3400f510 tx=0x7fae34004610 comp rx=0 tx=0).stop 2024-05-11T12:14:10.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.387+0000 7fae4b000640 1 -- 172.21.15.167:0/662511678 shutdown_connections 2024-05-11T12:14:10.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.387+0000 7fae4b000640 1 --2- 172.21.15.167:0/662511678 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fae440ab7b0 0x7fae440730c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:10.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.387+0000 7fae4b000640 1 --2- 172.21.15.167:0/662511678 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fae20076300 0x7fae200787c0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:10.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.387+0000 7fae4b000640 1 --2- 172.21.15.167:0/662511678 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae441f5fa0 0x7fae440ab270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:10.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.387+0000 7fae4b000640 1 --2- 172.21.15.167:0/662511678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae28004800 0x7fae441f5840 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:10.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.387+0000 7fae4b000640 1 -- 172.21.15.167:0/662511678 >> 172.21.15.167:0/662511678 conn(0x7fae440cd8f0 msgr2=0x7fae4406d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:10.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.388+0000 7fae4b000640 1 -- 172.21.15.167:0/662511678 shutdown_connections 2024-05-11T12:14:10.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.388+0000 7fae4b000640 1 -- 172.21.15.167:0/662511678 wait complete. 2024-05-11T12:14:10.407 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":188,"stamp":"2024-05-11T12:14:09.299464+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":174,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":246392,"kb_used_data":9392,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754728328,"statfs":{"total":773094113280,"available":772841807872,"internally_reserved":0,"allocated":9617408,"data_stored":4969644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.925448"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.426058+0000","last_change":"2024-05-11T12:14:05.421973+0000","last_active":"2024-05-11T12:14:06.426058+0000","last_peered":"2024-05-11T12:14:06.426058+0000","last_clean":"2024-05-11T12:14:06.426058+0000","last_became_active":"2024-05-11T12:14:05.421634+0000","last_became_peered":"2024-05-11T12:14:05.421634+0000","last_unstale":"2024-05-11T12:14:06.426058+0000","last_undegraded":"2024-05-11T12:14:06.426058+0000","last_fullsized":"2024-05-11T12:14:06.426058+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:46:38.094747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.427751+0000","last_change":"2024-05-11T12:14:05.427759+0000","last_active":"2024-05-11T12:14:06.427751+0000","last_peered":"2024-05-11T12:14:06.427751+0000","last_clean":"2024-05-11T12:14:06.427751+0000","last_became_active":"2024-05-11T12:14:05.427273+0000","last_became_peered":"2024-05-11T12:14:05.427273+0000","last_unstale":"2024-05-11T12:14:06.427751+0000","last_undegraded":"2024-05-11T12:14:06.427751+0000","last_fullsized":"2024-05-11T12:14:06.427751+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:38:21.667711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.425641+0000","last_change":"2024-05-11T12:14:05.450113+0000","last_active":"2024-05-11T12:14:06.425641+0000","last_peered":"2024-05-11T12:14:06.425641+0000","last_clean":"2024-05-11T12:14:06.425641+0000","last_became_active":"2024-05-11T12:14:05.449774+0000","last_became_peered":"2024-05-11T12:14:05.449774+0000","last_unstale":"2024-05-11T12:14:06.425641+0000","last_undegraded":"2024-05-11T12:14:06.425641+0000","last_fullsized":"2024-05-11T12:14:06.425641+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:45:31.493940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.608617+0000","last_change":"2024-05-11T12:14:05.439504+0000","last_active":"2024-05-11T12:14:06.608617+0000","last_peered":"2024-05-11T12:14:06.608617+0000","last_clean":"2024-05-11T12:14:06.608617+0000","last_became_active":"2024-05-11T12:14:05.438152+0000","last_became_peered":"2024-05-11T12:14:05.438152+0000","last_unstale":"2024-05-11T12:14:06.608617+0000","last_undegraded":"2024-05-11T12:14:06.608617+0000","last_fullsized":"2024-05-11T12:14:06.608617+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:12:22.260345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.430672+0000","last_change":"2024-05-11T12:14:05.444070+0000","last_active":"2024-05-11T12:14:06.430672+0000","last_peered":"2024-05-11T12:14:06.430672+0000","last_clean":"2024-05-11T12:14:06.430672+0000","last_became_active":"2024-05-11T12:14:05.439711+0000","last_became_peered":"2024-05-11T12:14:05.439711+0000","last_unstale":"2024-05-11T12:14:06.430672+0000","last_undegraded":"2024-05-11T12:14:06.430672+0000","last_fullsized":"2024-05-11T12:14:06.430672+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:40:08.268417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.607026+0000","last_change":"2024-05-11T12:14:05.431059+0000","last_active":"2024-05-11T12:14:06.607026+0000","last_peered":"2024-05-11T12:14:06.607026+0000","last_clean":"2024-05-11T12:14:06.607026+0000","last_became_active":"2024-05-11T12:14:05.429549+0000","last_became_peered":"2024-05-11T12:14:05.429549+0000","last_unstale":"2024-05-11T12:14:06.607026+0000","last_undegraded":"2024-05-11T12:14:06.607026+0000","last_fullsized":"2024-05-11T12:14:06.607026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:44:29.512178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.607355+0000","last_change":"2024-05-11T12:14:05.410648+0000","last_active":"2024-05-11T12:14:06.607355+0000","last_peered":"2024-05-11T12:14:06.607355+0000","last_clean":"2024-05-11T12:14:06.607355+0000","last_became_active":"2024-05-11T12:14:05.409914+0000","last_became_peered":"2024-05-11T12:14:05.409914+0000","last_unstale":"2024-05-11T12:14:06.607355+0000","last_undegraded":"2024-05-11T12:14:06.607355+0000","last_fullsized":"2024-05-11T12:14:06.607355+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:45:31.493940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.607245+0000","last_change":"2024-05-11T12:14:05.431246+0000","last_active":"2024-05-11T12:14:06.607245+0000","last_peered":"2024-05-11T12:14:06.607245+0000","last_clean":"2024-05-11T12:14:06.607245+0000","last_became_active":"2024-05-11T12:14:05.430140+0000","last_became_peered":"2024-05-11T12:14:05.430140+0000","last_unstale":"2024-05-11T12:14:06.607245+0000","last_undegraded":"2024-05-11T12:14:06.607245+0000","last_fullsized":"2024-05-11T12:14:06.607245+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:12:51.106509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.431582+0000","last_change":"2024-05-11T12:14:05.438362+0000","last_active":"2024-05-11T12:14:06.431582+0000","last_peered":"2024-05-11T12:14:06.431582+0000","last_clean":"2024-05-11T12:14:06.431582+0000","last_became_active":"2024-05-11T12:14:05.438083+0000","last_became_peered":"2024-05-11T12:14:05.438083+0000","last_unstale":"2024-05-11T12:14:06.431582+0000","last_undegraded":"2024-05-11T12:14:06.431582+0000","last_fullsized":"2024-05-11T12:14:06.431582+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:33:56.761089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-11T12:14:05.444695+0000","last_change":"2024-05-11T12:14:05.437957+0000","last_active":"2024-05-11T12:14:05.444695+0000","last_peered":"2024-05-11T12:14:05.444695+0000","last_clean":"2024-05-11T12:14:05.444695+0000","last_became_active":"2024-05-11T12:14:05.437680+0000","last_became_peered":"2024-05-11T12:14:05.437680+0000","last_unstale":"2024-05-11T12:14:05.444695+0000","last_undegraded":"2024-05-11T12:14:05.444695+0000","last_fullsized":"2024-05-11T12:14:05.444695+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:15:34.687976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.430739+0000","last_change":"2024-05-11T12:14:05.441958+0000","last_active":"2024-05-11T12:14:06.430739+0000","last_peered":"2024-05-11T12:14:06.430739+0000","last_clean":"2024-05-11T12:14:06.430739+0000","last_became_active":"2024-05-11T12:14:05.441608+0000","last_became_peered":"2024-05-11T12:14:05.441608+0000","last_unstale":"2024-05-11T12:14:06.430739+0000","last_undegraded":"2024-05-11T12:14:06.430739+0000","last_fullsized":"2024-05-11T12:14:06.430739+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:11:03.580774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-11T12:14:05.417890+0000","last_change":"2024-05-11T12:14:05.408860+0000","last_active":"2024-05-11T12:14:05.417890+0000","last_peered":"2024-05-11T12:14:05.417890+0000","last_clean":"2024-05-11T12:14:05.417890+0000","last_became_active":"2024-05-11T12:14:05.408576+0000","last_became_peered":"2024-05-11T12:14:05.408576+0000","last_unstale":"2024-05-11T12:14:05.417890+0000","last_undegraded":"2024-05-11T12:14:05.417890+0000","last_fullsized":"2024-05-11T12:14:05.417890+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:18:27.729212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.607893+0000","last_change":"2024-05-11T12:14:05.435827+0000","last_active":"2024-05-11T12:14:06.607893+0000","last_peered":"2024-05-11T12:14:06.607893+0000","last_clean":"2024-05-11T12:14:06.607893+0000","last_became_active":"2024-05-11T12:14:05.431822+0000","last_became_peered":"2024-05-11T12:14:05.431822+0000","last_unstale":"2024-05-11T12:14:06.607893+0000","last_undegraded":"2024-05-11T12:14:06.607893+0000","last_fullsized":"2024-05-11T12:14:06.607893+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:33:56.761089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.429961+0000","last_change":"2024-05-11T12:14:05.442631+0000","last_active":"2024-05-11T12:14:06.429961+0000","last_peered":"2024-05-11T12:14:06.429961+0000","last_clean":"2024-05-11T12:14:06.429961+0000","last_became_active":"2024-05-11T12:14:05.442301+0000","last_became_peered":"2024-05-11T12:14:05.442301+0000","last_unstale":"2024-05-11T12:14:06.429961+0000","last_undegraded":"2024-05-11T12:14:06.429961+0000","last_fullsized":"2024-05-11T12:14:06.429961+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:53:03.501470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.607543+0000","last_change":"2024-05-11T12:14:05.436161+0000","last_active":"2024-05-11T12:14:06.607543+0000","last_peered":"2024-05-11T12:14:06.607543+0000","last_clean":"2024-05-11T12:14:06.607543+0000","last_became_active":"2024-05-11T12:14:05.435898+0000","last_became_peered":"2024-05-11T12:14:05.435898+0000","last_unstale":"2024-05-11T12:14:06.607543+0000","last_undegraded":"2024-05-11T12:14:06.607543+0000","last_fullsized":"2024-05-11T12:14:06.607543+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:12:19.761617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.606204+0000","last_change":"2024-05-11T12:14:05.426591+0000","last_active":"2024-05-11T12:14:06.606204+0000","last_peered":"2024-05-11T12:14:06.606204+0000","last_clean":"2024-05-11T12:14:06.606204+0000","last_became_active":"2024-05-11T12:14:05.426213+0000","last_became_peered":"2024-05-11T12:14:05.426213+0000","last_unstale":"2024-05-11T12:14:06.606204+0000","last_undegraded":"2024-05-11T12:14:06.606204+0000","last_fullsized":"2024-05-11T12:14:06.606204+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:47:30.304698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.609592+0000","last_change":"2024-05-11T12:14:05.425958+0000","last_active":"2024-05-11T12:14:06.609592+0000","last_peered":"2024-05-11T12:14:06.609592+0000","last_clean":"2024-05-11T12:14:06.609592+0000","last_became_active":"2024-05-11T12:14:05.425089+0000","last_became_peered":"2024-05-11T12:14:05.425089+0000","last_unstale":"2024-05-11T12:14:06.609592+0000","last_undegraded":"2024-05-11T12:14:06.609592+0000","last_fullsized":"2024-05-11T12:14:06.609592+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:21:47.157444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.425942+0000","last_change":"2024-05-11T12:14:05.423083+0000","last_active":"2024-05-11T12:14:06.425942+0000","last_peered":"2024-05-11T12:14:06.425942+0000","last_clean":"2024-05-11T12:14:06.425942+0000","last_became_active":"2024-05-11T12:14:05.422817+0000","last_became_peered":"2024-05-11T12:14:05.422817+0000","last_unstale":"2024-05-11T12:14:06.425942+0000","last_undegraded":"2024-05-11T12:14:06.425942+0000","last_fullsized":"2024-05-11T12:14:06.425942+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:15:28.003250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.431299+0000","last_change":"2024-05-11T12:14:05.442561+0000","last_active":"2024-05-11T12:14:06.431299+0000","last_peered":"2024-05-11T12:14:06.431299+0000","last_clean":"2024-05-11T12:14:06.431299+0000","last_became_active":"2024-05-11T12:14:05.442267+0000","last_became_peered":"2024-05-11T12:14:05.442267+0000","last_unstale":"2024-05-11T12:14:06.431299+0000","last_undegraded":"2024-05-11T12:14:06.431299+0000","last_fullsized":"2024-05-11T12:14:06.431299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:46:10.965993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.609001+0000","last_change":"2024-05-11T12:14:05.423746+0000","last_active":"2024-05-11T12:14:06.609001+0000","last_peered":"2024-05-11T12:14:06.609001+0000","last_clean":"2024-05-11T12:14:06.609001+0000","last_became_active":"2024-05-11T12:14:05.422919+0000","last_became_peered":"2024-05-11T12:14:05.422919+0000","last_unstale":"2024-05-11T12:14:06.609001+0000","last_undegraded":"2024-05-11T12:14:06.609001+0000","last_fullsized":"2024-05-11T12:14:06.609001+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:09:14.521912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.430232+0000","last_change":"2024-05-11T12:14:05.440553+0000","last_active":"2024-05-11T12:14:06.430232+0000","last_peered":"2024-05-11T12:14:06.430232+0000","last_clean":"2024-05-11T12:14:06.430232+0000","last_became_active":"2024-05-11T12:14:05.439842+0000","last_became_peered":"2024-05-11T12:14:05.439842+0000","last_unstale":"2024-05-11T12:14:06.430232+0000","last_undegraded":"2024-05-11T12:14:06.430232+0000","last_fullsized":"2024-05-11T12:14:06.430232+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:31:10.245818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.431197+0000","last_change":"2024-05-11T12:14:05.442418+0000","last_active":"2024-05-11T12:14:06.431197+0000","last_peered":"2024-05-11T12:14:06.431197+0000","last_clean":"2024-05-11T12:14:06.431197+0000","last_became_active":"2024-05-11T12:14:05.442104+0000","last_became_peered":"2024-05-11T12:14:05.442104+0000","last_unstale":"2024-05-11T12:14:06.431197+0000","last_undegraded":"2024-05-11T12:14:06.431197+0000","last_fullsized":"2024-05-11T12:14:06.431197+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:12:19.761617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.607279+0000","last_change":"2024-05-11T12:14:05.441909+0000","last_active":"2024-05-11T12:14:06.607279+0000","last_peered":"2024-05-11T12:14:06.607279+0000","last_clean":"2024-05-11T12:14:06.607279+0000","last_became_active":"2024-05-11T12:14:05.440813+0000","last_became_peered":"2024-05-11T12:14:05.440813+0000","last_unstale":"2024-05-11T12:14:06.607279+0000","last_undegraded":"2024-05-11T12:14:06.607279+0000","last_fullsized":"2024-05-11T12:14:06.607279+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:12:51.106509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.425420+0000","last_change":"2024-05-11T12:14:05.448498+0000","last_active":"2024-05-11T12:14:06.425420+0000","last_peered":"2024-05-11T12:14:06.425420+0000","last_clean":"2024-05-11T12:14:06.425420+0000","last_became_active":"2024-05-11T12:14:05.448160+0000","last_became_peered":"2024-05-11T12:14:05.448160+0000","last_unstale":"2024-05-11T12:14:06.425420+0000","last_undegraded":"2024-05-11T12:14:06.425420+0000","last_fullsized":"2024-05-11T12:14:06.425420+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:04:48.659957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.427908+0000","last_change":"2024-05-11T12:14:05.439126+0000","last_active":"2024-05-11T12:14:06.427908+0000","last_peered":"2024-05-11T12:14:06.427908+0000","last_clean":"2024-05-11T12:14:06.427908+0000","last_became_active":"2024-05-11T12:14:05.438829+0000","last_became_peered":"2024-05-11T12:14:05.438829+0000","last_unstale":"2024-05-11T12:14:06.427908+0000","last_undegraded":"2024-05-11T12:14:06.427908+0000","last_fullsized":"2024-05-11T12:14:06.427908+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:10:13.238904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.431996+0000","last_change":"2024-05-11T12:14:05.443650+0000","last_active":"2024-05-11T12:14:06.431996+0000","last_peered":"2024-05-11T12:14:06.431996+0000","last_clean":"2024-05-11T12:14:06.431996+0000","last_became_active":"2024-05-11T12:14:05.443218+0000","last_became_peered":"2024-05-11T12:14:05.443218+0000","last_unstale":"2024-05-11T12:14:06.431996+0000","last_undegraded":"2024-05-11T12:14:06.431996+0000","last_fullsized":"2024-05-11T12:14:06.431996+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:28:06.950871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-11T12:14:05.441473+0000","last_change":"2024-05-11T12:14:05.437816+0000","last_active":"2024-05-11T12:14:05.441473+0000","last_peered":"2024-05-11T12:14:05.441473+0000","last_clean":"2024-05-11T12:14:05.441473+0000","last_became_active":"2024-05-11T12:14:05.437542+0000","last_became_peered":"2024-05-11T12:14:05.437542+0000","last_unstale":"2024-05-11T12:14:05.441473+0000","last_undegraded":"2024-05-11T12:14:05.441473+0000","last_fullsized":"2024-05-11T12:14:05.441473+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:42:41.681582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.609217+0000","last_change":"2024-05-11T12:14:05.444359+0000","last_active":"2024-05-11T12:14:06.609217+0000","last_peered":"2024-05-11T12:14:06.609217+0000","last_clean":"2024-05-11T12:14:06.609217+0000","last_became_active":"2024-05-11T12:14:05.444070+0000","last_became_peered":"2024-05-11T12:14:05.444070+0000","last_unstale":"2024-05-11T12:14:06.609217+0000","last_undegraded":"2024-05-11T12:14:06.609217+0000","last_fullsized":"2024-05-11T12:14:06.609217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:00:50.141176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.608939+0000","last_change":"2024-05-11T12:14:05.424047+0000","last_active":"2024-05-11T12:14:06.608939+0000","last_peered":"2024-05-11T12:14:06.608939+0000","last_clean":"2024-05-11T12:14:06.608939+0000","last_became_active":"2024-05-11T12:14:05.423742+0000","last_became_peered":"2024-05-11T12:14:05.423742+0000","last_unstale":"2024-05-11T12:14:06.608939+0000","last_undegraded":"2024-05-11T12:14:06.608939+0000","last_fullsized":"2024-05-11T12:14:06.608939+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:46:38.094747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.430678+0000","last_change":"2024-05-11T12:14:05.438867+0000","last_active":"2024-05-11T12:14:06.430678+0000","last_peered":"2024-05-11T12:14:06.430678+0000","last_clean":"2024-05-11T12:14:06.430678+0000","last_became_active":"2024-05-11T12:14:05.438031+0000","last_became_peered":"2024-05-11T12:14:05.438031+0000","last_unstale":"2024-05-11T12:14:06.430678+0000","last_undegraded":"2024-05-11T12:14:06.430678+0000","last_fullsized":"2024-05-11T12:14:06.430678+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:03:23.795707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.426352+0000","last_change":"2024-05-11T12:14:05.441006+0000","last_active":"2024-05-11T12:14:06.426352+0000","last_peered":"2024-05-11T12:14:06.426352+0000","last_clean":"2024-05-11T12:14:06.426352+0000","last_became_active":"2024-05-11T12:14:05.440145+0000","last_became_peered":"2024-05-11T12:14:05.440145+0000","last_unstale":"2024-05-11T12:14:06.426352+0000","last_undegraded":"2024-05-11T12:14:06.426352+0000","last_fullsized":"2024-05-11T12:14:06.426352+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:09:14.521912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.606549+0000","last_change":"2024-05-11T12:14:05.425221+0000","last_active":"2024-05-11T12:14:06.606549+0000","last_peered":"2024-05-11T12:14:06.606549+0000","last_clean":"2024-05-11T12:14:06.606549+0000","last_became_active":"2024-05-11T12:14:05.424733+0000","last_became_peered":"2024-05-11T12:14:05.424733+0000","last_unstale":"2024-05-11T12:14:06.606549+0000","last_undegraded":"2024-05-11T12:14:06.606549+0000","last_fullsized":"2024-05-11T12:14:06.606549+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:00:28.938873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.608349+0000","last_change":"2024-05-11T12:14:05.448987+0000","last_active":"2024-05-11T12:14:06.608349+0000","last_peered":"2024-05-11T12:14:06.608349+0000","last_clean":"2024-05-11T12:14:06.608349+0000","last_became_active":"2024-05-11T12:14:05.448334+0000","last_became_peered":"2024-05-11T12:14:05.448334+0000","last_unstale":"2024-05-11T12:14:06.608349+0000","last_undegraded":"2024-05-11T12:14:06.608349+0000","last_fullsized":"2024-05-11T12:14:06.608349+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:14:53.991948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"85'4","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.609606+0000","last_change":"2024-05-11T12:14:04.425347+0000","last_active":"2024-05-11T12:14:06.609606+0000","last_peered":"2024-05-11T12:14:06.609606+0000","last_clean":"2024-05-11T12:14:06.609606+0000","last_became_active":"2024-05-11T12:14:04.425025+0000","last_became_peered":"2024-05-11T12:14:04.425025+0000","last_unstale":"2024-05-11T12:14:06.609606+0000","last_undegraded":"2024-05-11T12:14:06.609606+0000","last_fullsized":"2024-05-11T12:14:06.609606+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:23:15.339626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.608343+0000","last_change":"2024-05-11T12:14:05.427020+0000","last_active":"2024-05-11T12:14:06.608343+0000","last_peered":"2024-05-11T12:14:06.608343+0000","last_clean":"2024-05-11T12:14:06.608343+0000","last_became_active":"2024-05-11T12:14:05.425119+0000","last_became_peered":"2024-05-11T12:14:05.425119+0000","last_unstale":"2024-05-11T12:14:06.608343+0000","last_undegraded":"2024-05-11T12:14:06.608343+0000","last_fullsized":"2024-05-11T12:14:06.608343+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:12:46.904813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.432142+0000","last_change":"2024-05-11T12:14:04.432813+0000","last_active":"2024-05-11T12:14:06.432142+0000","last_peered":"2024-05-11T12:14:06.432142+0000","last_clean":"2024-05-11T12:14:06.432142+0000","last_became_active":"2024-05-11T12:14:04.432455+0000","last_became_peered":"2024-05-11T12:14:04.432455+0000","last_unstale":"2024-05-11T12:14:06.432142+0000","last_undegraded":"2024-05-11T12:14:06.432142+0000","last_fullsized":"2024-05-11T12:14:06.432142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:30:38.517473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.608212+0000","last_change":"2024-05-11T12:14:05.424740+0000","last_active":"2024-05-11T12:14:06.608212+0000","last_peered":"2024-05-11T12:14:06.608212+0000","last_clean":"2024-05-11T12:14:06.608212+0000","last_became_active":"2024-05-11T12:14:05.424106+0000","last_became_peered":"2024-05-11T12:14:05.424106+0000","last_unstale":"2024-05-11T12:14:06.608212+0000","last_undegraded":"2024-05-11T12:14:06.608212+0000","last_fullsized":"2024-05-11T12:14:06.608212+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:07:02.964303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.607806+0000","last_change":"2024-05-11T12:14:04.420632+0000","last_active":"2024-05-11T12:14:06.607806+0000","last_peered":"2024-05-11T12:14:06.607806+0000","last_clean":"2024-05-11T12:14:06.607806+0000","last_became_active":"2024-05-11T12:14:04.416930+0000","last_became_peered":"2024-05-11T12:14:04.416930+0000","last_unstale":"2024-05-11T12:14:06.607806+0000","last_undegraded":"2024-05-11T12:14:06.607806+0000","last_fullsized":"2024-05-11T12:14:06.607806+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:00:49.124605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.609910+0000","last_change":"2024-05-11T12:14:05.438796+0000","last_active":"2024-05-11T12:14:06.609910+0000","last_peered":"2024-05-11T12:14:06.609910+0000","last_clean":"2024-05-11T12:14:06.609910+0000","last_became_active":"2024-05-11T12:14:05.438526+0000","last_became_peered":"2024-05-11T12:14:05.438526+0000","last_unstale":"2024-05-11T12:14:06.609910+0000","last_undegraded":"2024-05-11T12:14:06.609910+0000","last_fullsized":"2024-05-11T12:14:06.609910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:34:51.856988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.608740+0000","last_change":"2024-05-11T12:14:04.426774+0000","last_active":"2024-05-11T12:14:06.608740+0000","last_peered":"2024-05-11T12:14:06.608740+0000","last_clean":"2024-05-11T12:14:06.608740+0000","last_became_active":"2024-05-11T12:14:04.424806+0000","last_became_peered":"2024-05-11T12:14:04.424806+0000","last_unstale":"2024-05-11T12:14:06.608740+0000","last_undegraded":"2024-05-11T12:14:06.608740+0000","last_fullsized":"2024-05-11T12:14:06.608740+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:00:44.293728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.607087+0000","last_change":"2024-05-11T12:14:05.423593+0000","last_active":"2024-05-11T12:14:06.607087+0000","last_peered":"2024-05-11T12:14:06.607087+0000","last_clean":"2024-05-11T12:14:06.607087+0000","last_became_active":"2024-05-11T12:14:05.423304+0000","last_became_peered":"2024-05-11T12:14:05.423304+0000","last_unstale":"2024-05-11T12:14:06.607087+0000","last_undegraded":"2024-05-11T12:14:06.607087+0000","last_fullsized":"2024-05-11T12:14:06.607087+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:11:03.580774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.431065+0000","last_change":"2024-05-11T12:14:04.420760+0000","last_active":"2024-05-11T12:14:06.431065+0000","last_peered":"2024-05-11T12:14:06.431065+0000","last_clean":"2024-05-11T12:14:06.431065+0000","last_became_active":"2024-05-11T12:14:04.419701+0000","last_became_peered":"2024-05-11T12:14:04.419701+0000","last_unstale":"2024-05-11T12:14:06.431065+0000","last_undegraded":"2024-05-11T12:14:06.431065+0000","last_fullsized":"2024-05-11T12:14:06.431065+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:46:46.547028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.607706+0000","last_change":"2024-05-11T12:14:05.435844+0000","last_active":"2024-05-11T12:14:06.607706+0000","last_peered":"2024-05-11T12:14:06.607706+0000","last_clean":"2024-05-11T12:14:06.607706+0000","last_became_active":"2024-05-11T12:14:05.435126+0000","last_became_peered":"2024-05-11T12:14:05.435126+0000","last_unstale":"2024-05-11T12:14:06.607706+0000","last_undegraded":"2024-05-11T12:14:06.607706+0000","last_fullsized":"2024-05-11T12:14:06.607706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:12:19.761617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.610177+0000","last_change":"2024-05-11T12:14:04.417449+0000","last_active":"2024-05-11T12:14:06.610177+0000","last_peered":"2024-05-11T12:14:06.610177+0000","last_clean":"2024-05-11T12:14:06.610177+0000","last_became_active":"2024-05-11T12:14:04.417195+0000","last_became_peered":"2024-05-11T12:14:04.417195+0000","last_unstale":"2024-05-11T12:14:06.610177+0000","last_undegraded":"2024-05-11T12:14:06.610177+0000","last_fullsized":"2024-05-11T12:14:06.610177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:16:17.661069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.607072+0000","last_change":"2024-05-11T12:14:05.438916+0000","last_active":"2024-05-11T12:14:06.607072+0000","last_peered":"2024-05-11T12:14:06.607072+0000","last_clean":"2024-05-11T12:14:06.607072+0000","last_became_active":"2024-05-11T12:14:05.438608+0000","last_became_peered":"2024-05-11T12:14:05.438608+0000","last_unstale":"2024-05-11T12:14:06.607072+0000","last_undegraded":"2024-05-11T12:14:06.607072+0000","last_fullsized":"2024-05-11T12:14:06.607072+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:55:32.026118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.425470+0000","last_change":"2024-05-11T12:14:04.407920+0000","last_active":"2024-05-11T12:14:06.425470+0000","last_peered":"2024-05-11T12:14:06.425470+0000","last_clean":"2024-05-11T12:14:06.425470+0000","last_became_active":"2024-05-11T12:14:04.407605+0000","last_became_peered":"2024-05-11T12:14:04.407605+0000","last_unstale":"2024-05-11T12:14:06.425470+0000","last_undegraded":"2024-05-11T12:14:06.425470+0000","last_fullsized":"2024-05-11T12:14:06.425470+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:09:13.505342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.608016+0000","last_change":"2024-05-11T12:14:05.435212+0000","last_active":"2024-05-11T12:14:06.608016+0000","last_peered":"2024-05-11T12:14:06.608016+0000","last_clean":"2024-05-11T12:14:06.608016+0000","last_became_active":"2024-05-11T12:14:05.434838+0000","last_became_peered":"2024-05-11T12:14:05.434838+0000","last_unstale":"2024-05-11T12:14:06.608016+0000","last_undegraded":"2024-05-11T12:14:06.608016+0000","last_fullsized":"2024-05-11T12:14:06.608016+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:28:06.950871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.607924+0000","last_change":"2024-05-11T12:14:04.422280+0000","last_active":"2024-05-11T12:14:06.607924+0000","last_peered":"2024-05-11T12:14:06.607924+0000","last_clean":"2024-05-11T12:14:06.607924+0000","last_became_active":"2024-05-11T12:14:04.420929+0000","last_became_peered":"2024-05-11T12:14:04.420929+0000","last_unstale":"2024-05-11T12:14:06.607924+0000","last_undegraded":"2024-05-11T12:14:06.607924+0000","last_fullsized":"2024-05-11T12:14:06.607924+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:12:21.243774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.431063+0000","last_change":"2024-05-11T12:14:05.438352+0000","last_active":"2024-05-11T12:14:06.431063+0000","last_peered":"2024-05-11T12:14:06.431063+0000","last_clean":"2024-05-11T12:14:06.431063+0000","last_became_active":"2024-05-11T12:14:05.438027+0000","last_became_peered":"2024-05-11T12:14:05.438027+0000","last_unstale":"2024-05-11T12:14:06.431063+0000","last_undegraded":"2024-05-11T12:14:06.431063+0000","last_fullsized":"2024-05-11T12:14:06.431063+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:12:51.106509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.428110+0000","last_change":"2024-05-11T12:14:04.423576+0000","last_active":"2024-05-11T12:14:06.428110+0000","last_peered":"2024-05-11T12:14:06.428110+0000","last_clean":"2024-05-11T12:14:06.428110+0000","last_became_active":"2024-05-11T12:14:04.423046+0000","last_became_peered":"2024-05-11T12:14:04.423046+0000","last_unstale":"2024-05-11T12:14:06.428110+0000","last_undegraded":"2024-05-11T12:14:06.428110+0000","last_fullsized":"2024-05-11T12:14:06.428110+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:11:02.564203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.607534+0000","last_change":"2024-05-11T12:14:05.445807+0000","last_active":"2024-05-11T12:14:06.607534+0000","last_peered":"2024-05-11T12:14:06.607534+0000","last_clean":"2024-05-11T12:14:06.607534+0000","last_became_active":"2024-05-11T12:14:05.445496+0000","last_became_peered":"2024-05-11T12:14:05.445496+0000","last_unstale":"2024-05-11T12:14:06.607534+0000","last_undegraded":"2024-05-11T12:14:06.607534+0000","last_fullsized":"2024-05-11T12:14:06.607534+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:15:28.003250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.426081+0000","last_change":"2024-05-11T12:14:04.417845+0000","last_active":"2024-05-11T12:14:06.426081+0000","last_peered":"2024-05-11T12:14:06.426081+0000","last_clean":"2024-05-11T12:14:06.426081+0000","last_became_active":"2024-05-11T12:14:04.417547+0000","last_became_peered":"2024-05-11T12:14:04.417547+0000","last_unstale":"2024-05-11T12:14:06.426081+0000","last_undegraded":"2024-05-11T12:14:06.426081+0000","last_fullsized":"2024-05-11T12:14:06.426081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:00:49.124605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.425966+0000","last_change":"2024-05-11T12:14:05.420993+0000","last_active":"2024-05-11T12:14:06.425966+0000","last_peered":"2024-05-11T12:14:06.425966+0000","last_clean":"2024-05-11T12:14:06.425966+0000","last_became_active":"2024-05-11T12:14:05.420702+0000","last_became_peered":"2024-05-11T12:14:05.420702+0000","last_unstale":"2024-05-11T12:14:06.425966+0000","last_undegraded":"2024-05-11T12:14:06.425966+0000","last_fullsized":"2024-05-11T12:14:06.425966+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:07:02.964303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"85'3","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.431214+0000","last_change":"2024-05-11T12:14:04.431738+0000","last_active":"2024-05-11T12:14:06.431214+0000","last_peered":"2024-05-11T12:14:06.431214+0000","last_clean":"2024-05-11T12:14:06.431214+0000","last_became_active":"2024-05-11T12:14:04.430502+0000","last_became_peered":"2024-05-11T12:14:04.430502+0000","last_unstale":"2024-05-11T12:14:06.431214+0000","last_undegraded":"2024-05-11T12:14:06.431214+0000","last_fullsized":"2024-05-11T12:14:06.431214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:22:38.635128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.610402+0000","last_change":"2024-05-11T12:14:05.433943+0000","last_active":"2024-05-11T12:14:06.610402+0000","last_peered":"2024-05-11T12:14:06.610402+0000","last_clean":"2024-05-11T12:14:06.610402+0000","last_became_active":"2024-05-11T12:14:05.433604+0000","last_became_peered":"2024-05-11T12:14:05.433604+0000","last_unstale":"2024-05-11T12:14:06.610402+0000","last_undegraded":"2024-05-11T12:14:06.610402+0000","last_fullsized":"2024-05-11T12:14:06.610402+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:20:57.538769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"85'6","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.607836+0000","last_change":"2024-05-11T12:14:04.427471+0000","last_active":"2024-05-11T12:14:06.607836+0000","last_peered":"2024-05-11T12:14:06.607836+0000","last_clean":"2024-05-11T12:14:06.607836+0000","last_became_active":"2024-05-11T12:14:04.427148+0000","last_became_peered":"2024-05-11T12:14:04.427148+0000","last_unstale":"2024-05-11T12:14:06.607836+0000","last_undegraded":"2024-05-11T12:14:06.607836+0000","last_fullsized":"2024-05-11T12:14:06.607836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:19:35.359726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.607694+0000","last_change":"2024-05-11T12:14:05.431244+0000","last_active":"2024-05-11T12:14:06.607694+0000","last_peered":"2024-05-11T12:14:06.607694+0000","last_clean":"2024-05-11T12:14:06.607694+0000","last_became_active":"2024-05-11T12:14:05.430962+0000","last_became_peered":"2024-05-11T12:14:05.430962+0000","last_unstale":"2024-05-11T12:14:06.607694+0000","last_undegraded":"2024-05-11T12:14:06.607694+0000","last_fullsized":"2024-05-11T12:14:06.607694+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:46:10.965993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.608529+0000","last_change":"2024-05-11T12:14:04.419824+0000","last_active":"2024-05-11T12:14:06.608529+0000","last_peered":"2024-05-11T12:14:06.608529+0000","last_clean":"2024-05-11T12:14:06.608529+0000","last_became_active":"2024-05-11T12:14:04.419546+0000","last_became_peered":"2024-05-11T12:14:04.419546+0000","last_unstale":"2024-05-11T12:14:06.608529+0000","last_undegraded":"2024-05-11T12:14:06.608529+0000","last_fullsized":"2024-05-11T12:14:06.608529+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:38:20.651140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.429894+0000","last_change":"2024-05-11T12:14:05.439531+0000","last_active":"2024-05-11T12:14:06.429894+0000","last_peered":"2024-05-11T12:14:06.429894+0000","last_clean":"2024-05-11T12:14:06.429894+0000","last_became_active":"2024-05-11T12:14:05.437967+0000","last_became_peered":"2024-05-11T12:14:05.437967+0000","last_unstale":"2024-05-11T12:14:06.429894+0000","last_undegraded":"2024-05-11T12:14:06.429894+0000","last_fullsized":"2024-05-11T12:14:06.429894+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:23:16.356197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.607226+0000","last_change":"2024-05-11T12:14:04.429861+0000","last_active":"2024-05-11T12:14:06.607226+0000","last_peered":"2024-05-11T12:14:06.607226+0000","last_clean":"2024-05-11T12:14:06.607226+0000","last_became_active":"2024-05-11T12:14:04.429546+0000","last_became_peered":"2024-05-11T12:14:04.429546+0000","last_unstale":"2024-05-11T12:14:06.607226+0000","last_undegraded":"2024-05-11T12:14:06.607226+0000","last_fullsized":"2024-05-11T12:14:06.607226+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:38:20.651140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-11T12:14:05.433205+0000","last_change":"2024-05-11T12:14:05.425575+0000","last_active":"2024-05-11T12:14:05.433205+0000","last_peered":"2024-05-11T12:14:05.433205+0000","last_clean":"2024-05-11T12:14:05.433205+0000","last_became_active":"2024-05-11T12:14:05.425139+0000","last_became_peered":"2024-05-11T12:14:05.425139+0000","last_unstale":"2024-05-11T12:14:05.433205+0000","last_undegraded":"2024-05-11T12:14:05.433205+0000","last_fullsized":"2024-05-11T12:14:05.433205+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:12:21.767350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.607723+0000","last_change":"2024-05-11T12:14:04.427924+0000","last_active":"2024-05-11T12:14:06.607723+0000","last_peered":"2024-05-11T12:14:06.607723+0000","last_clean":"2024-05-11T12:14:06.607723+0000","last_became_active":"2024-05-11T12:14:04.427196+0000","last_became_peered":"2024-05-11T12:14:04.427196+0000","last_unstale":"2024-05-11T12:14:06.607723+0000","last_undegraded":"2024-05-11T12:14:06.607723+0000","last_fullsized":"2024-05-11T12:14:06.607723+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:16:55.623835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-11T12:14:05.450072+0000","last_change":"2024-05-11T12:14:05.445549+0000","last_active":"2024-05-11T12:14:05.450072+0000","last_peered":"2024-05-11T12:14:05.450072+0000","last_clean":"2024-05-11T12:14:05.450072+0000","last_became_active":"2024-05-11T12:14:05.445136+0000","last_became_peered":"2024-05-11T12:14:05.445136+0000","last_unstale":"2024-05-11T12:14:05.450072+0000","last_undegraded":"2024-05-11T12:14:05.450072+0000","last_fullsized":"2024-05-11T12:14:05.450072+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:19:09.883922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"85'8","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.607998+0000","last_change":"2024-05-11T12:14:04.417414+0000","last_active":"2024-05-11T12:14:06.607998+0000","last_peered":"2024-05-11T12:14:06.607998+0000","last_clean":"2024-05-11T12:14:06.607998+0000","last_became_active":"2024-05-11T12:14:04.417109+0000","last_became_peered":"2024-05-11T12:14:04.417109+0000","last_unstale":"2024-05-11T12:14:06.607998+0000","last_undegraded":"2024-05-11T12:14:06.607998+0000","last_fullsized":"2024-05-11T12:14:06.607998+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:48:53.320829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.607615+0000","last_change":"2024-05-11T12:14:05.449206+0000","last_active":"2024-05-11T12:14:06.607615+0000","last_peered":"2024-05-11T12:14:06.607615+0000","last_clean":"2024-05-11T12:14:06.607615+0000","last_became_active":"2024-05-11T12:14:05.448720+0000","last_became_peered":"2024-05-11T12:14:05.448720+0000","last_unstale":"2024-05-11T12:14:06.607615+0000","last_undegraded":"2024-05-11T12:14:06.607615+0000","last_fullsized":"2024-05-11T12:14:06.607615+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:16:36.104898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.609743+0000","last_change":"2024-05-11T12:14:04.429540+0000","last_active":"2024-05-11T12:14:06.609743+0000","last_peered":"2024-05-11T12:14:06.609743+0000","last_clean":"2024-05-11T12:14:06.609743+0000","last_became_active":"2024-05-11T12:14:04.429111+0000","last_became_peered":"2024-05-11T12:14:04.429111+0000","last_unstale":"2024-05-11T12:14:06.609743+0000","last_undegraded":"2024-05-11T12:14:06.609743+0000","last_fullsized":"2024-05-11T12:14:06.609743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:50:43.887550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.608587+0000","last_change":"2024-05-11T12:14:05.445374+0000","last_active":"2024-05-11T12:14:06.608587+0000","last_peered":"2024-05-11T12:14:06.608587+0000","last_clean":"2024-05-11T12:14:06.608587+0000","last_became_active":"2024-05-11T12:14:05.444923+0000","last_became_peered":"2024-05-11T12:14:05.444923+0000","last_unstale":"2024-05-11T12:14:06.608587+0000","last_undegraded":"2024-05-11T12:14:06.608587+0000","last_fullsized":"2024-05-11T12:14:06.608587+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:28:06.950871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.608535+0000","last_change":"2024-05-11T12:14:04.429525+0000","last_active":"2024-05-11T12:14:06.608535+0000","last_peered":"2024-05-11T12:14:06.608535+0000","last_clean":"2024-05-11T12:14:06.608535+0000","last_became_active":"2024-05-11T12:14:04.429223+0000","last_became_peered":"2024-05-11T12:14:04.429223+0000","last_unstale":"2024-05-11T12:14:06.608535+0000","last_undegraded":"2024-05-11T12:14:06.608535+0000","last_fullsized":"2024-05-11T12:14:06.608535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:20:56.522199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.432050+0000","last_change":"2024-05-11T12:14:05.439212+0000","last_active":"2024-05-11T12:14:06.432050+0000","last_peered":"2024-05-11T12:14:06.432050+0000","last_clean":"2024-05-11T12:14:06.432050+0000","last_became_active":"2024-05-11T12:14:05.438869+0000","last_became_peered":"2024-05-11T12:14:05.438869+0000","last_unstale":"2024-05-11T12:14:06.432050+0000","last_undegraded":"2024-05-11T12:14:06.432050+0000","last_fullsized":"2024-05-11T12:14:06.432050+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:12:22.260345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.431508+0000","last_change":"2024-05-11T12:14:04.419221+0000","last_active":"2024-05-11T12:14:06.431508+0000","last_peered":"2024-05-11T12:14:06.431508+0000","last_clean":"2024-05-11T12:14:06.431508+0000","last_became_active":"2024-05-11T12:14:04.418879+0000","last_became_peered":"2024-05-11T12:14:04.418879+0000","last_unstale":"2024-05-11T12:14:06.431508+0000","last_undegraded":"2024-05-11T12:14:06.431508+0000","last_fullsized":"2024-05-11T12:14:06.431508+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:43:10.502481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.606368+0000","last_change":"2024-05-11T12:14:05.420121+0000","last_active":"2024-05-11T12:14:06.606368+0000","last_peered":"2024-05-11T12:14:06.606368+0000","last_clean":"2024-05-11T12:14:06.606368+0000","last_became_active":"2024-05-11T12:14:05.419581+0000","last_became_peered":"2024-05-11T12:14:05.419581+0000","last_unstale":"2024-05-11T12:14:06.606368+0000","last_undegraded":"2024-05-11T12:14:06.606368+0000","last_fullsized":"2024-05-11T12:14:06.606368+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:56:45.292884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.427462+0000","last_change":"2024-05-11T12:14:04.426347+0000","last_active":"2024-05-11T12:14:06.427462+0000","last_peered":"2024-05-11T12:14:06.427462+0000","last_clean":"2024-05-11T12:14:06.427462+0000","last_became_active":"2024-05-11T12:14:04.426020+0000","last_became_peered":"2024-05-11T12:14:04.426020+0000","last_unstale":"2024-05-11T12:14:06.427462+0000","last_undegraded":"2024-05-11T12:14:06.427462+0000","last_fullsized":"2024-05-11T12:14:06.427462+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:31:09.229247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.426380+0000","last_change":"2024-05-11T12:14:05.423884+0000","last_active":"2024-05-11T12:14:06.426380+0000","last_peered":"2024-05-11T12:14:06.426380+0000","last_clean":"2024-05-11T12:14:06.426380+0000","last_became_active":"2024-05-11T12:14:05.423588+0000","last_became_peered":"2024-05-11T12:14:05.423588+0000","last_unstale":"2024-05-11T12:14:06.426380+0000","last_undegraded":"2024-05-11T12:14:06.426380+0000","last_fullsized":"2024-05-11T12:14:06.426380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:00:50.141176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-11T12:14:05.400337+0000","last_change":"2024-05-11T12:14:04.418068+0000","last_active":"2024-05-11T12:14:05.400337+0000","last_peered":"2024-05-11T12:14:05.400337+0000","last_clean":"2024-05-11T12:14:05.400337+0000","last_became_active":"2024-05-11T12:14:04.417799+0000","last_became_peered":"2024-05-11T12:14:04.417799+0000","last_unstale":"2024-05-11T12:14:05.400337+0000","last_undegraded":"2024-05-11T12:14:05.400337+0000","last_fullsized":"2024-05-11T12:14:05.400337+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:26:06.820640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.427057+0000","last_change":"2024-05-11T12:14:05.435022+0000","last_active":"2024-05-11T12:14:06.427057+0000","last_peered":"2024-05-11T12:14:06.427057+0000","last_clean":"2024-05-11T12:14:06.427057+0000","last_became_active":"2024-05-11T12:14:05.434731+0000","last_became_peered":"2024-05-11T12:14:05.434731+0000","last_unstale":"2024-05-11T12:14:06.427057+0000","last_undegraded":"2024-05-11T12:14:06.427057+0000","last_fullsized":"2024-05-11T12:14:06.427057+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:20:57.538769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.426758+0000","last_change":"2024-05-11T12:14:04.425226+0000","last_active":"2024-05-11T12:14:06.426758+0000","last_peered":"2024-05-11T12:14:06.426758+0000","last_clean":"2024-05-11T12:14:06.426758+0000","last_became_active":"2024-05-11T12:14:04.424079+0000","last_became_peered":"2024-05-11T12:14:04.424079+0000","last_unstale":"2024-05-11T12:14:06.426758+0000","last_undegraded":"2024-05-11T12:14:06.426758+0000","last_fullsized":"2024-05-11T12:14:06.426758+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:03:22.779136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.607617+0000","last_change":"2024-05-11T12:14:05.430748+0000","last_active":"2024-05-11T12:14:06.607617+0000","last_peered":"2024-05-11T12:14:06.607617+0000","last_clean":"2024-05-11T12:14:06.607617+0000","last_became_active":"2024-05-11T12:14:05.426138+0000","last_became_peered":"2024-05-11T12:14:05.426138+0000","last_unstale":"2024-05-11T12:14:06.607617+0000","last_undegraded":"2024-05-11T12:14:06.607617+0000","last_fullsized":"2024-05-11T12:14:06.607617+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:58:44.327216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-11T12:14:05.400232+0000","last_change":"2024-05-11T12:14:04.421430+0000","last_active":"2024-05-11T12:14:05.400232+0000","last_peered":"2024-05-11T12:14:05.400232+0000","last_clean":"2024-05-11T12:14:05.400232+0000","last_became_active":"2024-05-11T12:14:04.420955+0000","last_became_peered":"2024-05-11T12:14:04.420955+0000","last_unstale":"2024-05-11T12:14:05.400232+0000","last_undegraded":"2024-05-11T12:14:05.400232+0000","last_fullsized":"2024-05-11T12:14:05.400232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:09:13.866774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.608543+0000","last_change":"2024-05-11T12:14:05.443776+0000","last_active":"2024-05-11T12:14:06.608543+0000","last_peered":"2024-05-11T12:14:06.608543+0000","last_clean":"2024-05-11T12:14:06.608543+0000","last_became_active":"2024-05-11T12:14:05.443504+0000","last_became_peered":"2024-05-11T12:14:05.443504+0000","last_unstale":"2024-05-11T12:14:06.608543+0000","last_undegraded":"2024-05-11T12:14:06.608543+0000","last_fullsized":"2024-05-11T12:14:06.608543+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:33:56.761089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.609166+0000","last_change":"2024-05-11T12:14:04.428661+0000","last_active":"2024-05-11T12:14:06.609166+0000","last_peered":"2024-05-11T12:14:06.609166+0000","last_clean":"2024-05-11T12:14:06.609166+0000","last_became_active":"2024-05-11T12:14:04.427340+0000","last_became_peered":"2024-05-11T12:14:04.427340+0000","last_unstale":"2024-05-11T12:14:06.609166+0000","last_undegraded":"2024-05-11T12:14:06.609166+0000","last_fullsized":"2024-05-11T12:14:06.609166+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:10:12.222334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.608765+0000","last_change":"2024-05-11T12:14:05.448920+0000","last_active":"2024-05-11T12:14:06.608765+0000","last_peered":"2024-05-11T12:14:06.608765+0000","last_clean":"2024-05-11T12:14:06.608765+0000","last_became_active":"2024-05-11T12:14:05.448475+0000","last_became_peered":"2024-05-11T12:14:05.448475+0000","last_unstale":"2024-05-11T12:14:06.608765+0000","last_undegraded":"2024-05-11T12:14:06.608765+0000","last_fullsized":"2024-05-11T12:14:06.608765+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:10:34.562552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"86'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:07.409528+0000","last_change":"2024-05-11T12:14:04.431814+0000","last_active":"2024-05-11T12:14:07.409528+0000","last_peered":"2024-05-11T12:14:07.409528+0000","last_clean":"2024-05-11T12:14:07.409528+0000","last_became_active":"2024-05-11T12:14:04.431516+0000","last_became_peered":"2024-05-11T12:14:04.431516+0000","last_unstale":"2024-05-11T12:14:07.409528+0000","last_undegraded":"2024-05-11T12:14:07.409528+0000","last_fullsized":"2024-05-11T12:14:07.409528+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:12:42.194230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.608899+0000","last_change":"2024-05-11T12:14:05.425737+0000","last_active":"2024-05-11T12:14:06.608899+0000","last_peered":"2024-05-11T12:14:06.608899+0000","last_clean":"2024-05-11T12:14:06.608899+0000","last_became_active":"2024-05-11T12:14:05.425475+0000","last_became_peered":"2024-05-11T12:14:05.425475+0000","last_unstale":"2024-05-11T12:14:06.608899+0000","last_undegraded":"2024-05-11T12:14:06.608899+0000","last_fullsized":"2024-05-11T12:14:06.608899+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:59:32.672735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.431564+0000","last_change":"2024-05-11T12:14:04.431591+0000","last_active":"2024-05-11T12:14:06.431564+0000","last_peered":"2024-05-11T12:14:06.431564+0000","last_clean":"2024-05-11T12:14:06.431564+0000","last_became_active":"2024-05-11T12:14:04.429447+0000","last_became_peered":"2024-05-11T12:14:04.429447+0000","last_unstale":"2024-05-11T12:14:06.431564+0000","last_undegraded":"2024-05-11T12:14:06.431564+0000","last_fullsized":"2024-05-11T12:14:06.431564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:23:26.454412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.428113+0000","last_change":"2024-05-11T12:14:05.441203+0000","last_active":"2024-05-11T12:14:06.428113+0000","last_peered":"2024-05-11T12:14:06.428113+0000","last_clean":"2024-05-11T12:14:06.428113+0000","last_became_active":"2024-05-11T12:14:05.440747+0000","last_became_peered":"2024-05-11T12:14:05.440747+0000","last_unstale":"2024-05-11T12:14:06.428113+0000","last_undegraded":"2024-05-11T12:14:06.428113+0000","last_fullsized":"2024-05-11T12:14:06.428113+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:19:36.376296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.608804+0000","last_change":"2024-05-11T12:14:04.424655+0000","last_active":"2024-05-11T12:14:06.608804+0000","last_peered":"2024-05-11T12:14:06.608804+0000","last_clean":"2024-05-11T12:14:06.608804+0000","last_became_active":"2024-05-11T12:14:04.423752+0000","last_became_peered":"2024-05-11T12:14:04.423752+0000","last_unstale":"2024-05-11T12:14:06.608804+0000","last_undegraded":"2024-05-11T12:14:06.608804+0000","last_fullsized":"2024-05-11T12:14:06.608804+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:25:39.149525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-11T12:14:05.423260+0000","last_change":"2024-05-11T12:14:05.418926+0000","last_active":"2024-05-11T12:14:05.423260+0000","last_peered":"2024-05-11T12:14:05.423260+0000","last_clean":"2024-05-11T12:14:05.423260+0000","last_became_active":"2024-05-11T12:14:05.418662+0000","last_became_peered":"2024-05-11T12:14:05.418662+0000","last_unstale":"2024-05-11T12:14:05.423260+0000","last_undegraded":"2024-05-11T12:14:05.423260+0000","last_fullsized":"2024-05-11T12:14:05.423260+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:38:12.001547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.608104+0000","last_change":"2024-05-11T12:14:04.429826+0000","last_active":"2024-05-11T12:14:06.608104+0000","last_peered":"2024-05-11T12:14:06.608104+0000","last_clean":"2024-05-11T12:14:06.608104+0000","last_became_active":"2024-05-11T12:14:04.429540+0000","last_became_peered":"2024-05-11T12:14:04.429540+0000","last_unstale":"2024-05-11T12:14:06.608104+0000","last_undegraded":"2024-05-11T12:14:06.608104+0000","last_fullsized":"2024-05-11T12:14:06.608104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:10:12.222334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.425320+0000","last_change":"2024-05-11T12:14:05.422361+0000","last_active":"2024-05-11T12:14:06.425320+0000","last_peered":"2024-05-11T12:14:06.425320+0000","last_clean":"2024-05-11T12:14:06.425320+0000","last_became_active":"2024-05-11T12:14:05.422077+0000","last_became_peered":"2024-05-11T12:14:05.422077+0000","last_unstale":"2024-05-11T12:14:06.425320+0000","last_undegraded":"2024-05-11T12:14:06.425320+0000","last_fullsized":"2024-05-11T12:14:06.425320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:12:35.736138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.427232+0000","last_change":"2024-05-11T12:14:04.426263+0000","last_active":"2024-05-11T12:14:06.427232+0000","last_peered":"2024-05-11T12:14:06.427232+0000","last_clean":"2024-05-11T12:14:06.427232+0000","last_became_active":"2024-05-11T12:14:04.425967+0000","last_became_peered":"2024-05-11T12:14:04.425967+0000","last_unstale":"2024-05-11T12:14:06.427232+0000","last_undegraded":"2024-05-11T12:14:06.427232+0000","last_fullsized":"2024-05-11T12:14:06.427232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:40:07.251846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.608057+0000","last_change":"2024-05-11T12:14:05.444013+0000","last_active":"2024-05-11T12:14:06.608057+0000","last_peered":"2024-05-11T12:14:06.608057+0000","last_clean":"2024-05-11T12:14:06.608057+0000","last_became_active":"2024-05-11T12:14:05.442985+0000","last_became_peered":"2024-05-11T12:14:05.442985+0000","last_unstale":"2024-05-11T12:14:06.608057+0000","last_undegraded":"2024-05-11T12:14:06.608057+0000","last_fullsized":"2024-05-11T12:14:06.608057+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:46:10.965993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.607839+0000","last_change":"2024-05-11T12:14:04.425086+0000","last_active":"2024-05-11T12:14:06.607839+0000","last_peered":"2024-05-11T12:14:06.607839+0000","last_clean":"2024-05-11T12:14:06.607839+0000","last_became_active":"2024-05-11T12:14:04.424035+0000","last_became_peered":"2024-05-11T12:14:04.424035+0000","last_unstale":"2024-05-11T12:14:06.607839+0000","last_undegraded":"2024-05-11T12:14:06.607839+0000","last_fullsized":"2024-05-11T12:14:06.607839+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:08:25.339038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.607919+0000","last_change":"2024-05-11T12:14:05.426326+0000","last_active":"2024-05-11T12:14:06.607919+0000","last_peered":"2024-05-11T12:14:06.607919+0000","last_clean":"2024-05-11T12:14:06.607919+0000","last_became_active":"2024-05-11T12:14:05.424685+0000","last_became_peered":"2024-05-11T12:14:05.424685+0000","last_unstale":"2024-05-11T12:14:06.607919+0000","last_undegraded":"2024-05-11T12:14:06.607919+0000","last_fullsized":"2024-05-11T12:14:06.607919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:15:24.413933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.607423+0000","last_change":"2024-05-11T12:14:04.423204+0000","last_active":"2024-05-11T12:14:06.607423+0000","last_peered":"2024-05-11T12:14:06.607423+0000","last_clean":"2024-05-11T12:14:06.607423+0000","last_became_active":"2024-05-11T12:14:04.422071+0000","last_became_peered":"2024-05-11T12:14:04.422071+0000","last_unstale":"2024-05-11T12:14:06.607423+0000","last_undegraded":"2024-05-11T12:14:06.607423+0000","last_fullsized":"2024-05-11T12:14:06.607423+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:09:13.505342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.609299+0000","last_change":"2024-05-11T12:14:05.419683+0000","last_active":"2024-05-11T12:14:06.609299+0000","last_peered":"2024-05-11T12:14:06.609299+0000","last_clean":"2024-05-11T12:14:06.609299+0000","last_became_active":"2024-05-11T12:14:05.419245+0000","last_became_peered":"2024-05-11T12:14:05.419245+0000","last_unstale":"2024-05-11T12:14:06.609299+0000","last_undegraded":"2024-05-11T12:14:06.609299+0000","last_fullsized":"2024-05-11T12:14:06.609299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:04.373274+0000","last_clean_scrub_stamp":"2024-05-11T12:14:04.373274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:01:03.933240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.609254+0000","last_change":"2024-05-11T12:14:04.424087+0000","last_active":"2024-05-11T12:14:06.609254+0000","last_peered":"2024-05-11T12:14:06.609254+0000","last_clean":"2024-05-11T12:14:06.609254+0000","last_became_active":"2024-05-11T12:14:04.423810+0000","last_became_peered":"2024-05-11T12:14:04.423810+0000","last_unstale":"2024-05-11T12:14:06.609254+0000","last_undegraded":"2024-05-11T12:14:06.609254+0000","last_fullsized":"2024-05-11T12:14:06.609254+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:03.356703+0000","last_clean_scrub_stamp":"2024-05-11T12:14:03.356703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:15:09.957490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":227,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T12:14:06.607598+0000","last_change":"2024-05-11T12:10:35.066673+0000","last_active":"2024-05-11T12:14:06.607598+0000","last_peered":"2024-05-11T12:14:06.607598+0000","last_clean":"2024-05-11T12:14:06.607598+0000","last_became_active":"2024-05-11T12:10:35.065942+0000","last_became_peered":"2024-05-11T12:10:35.065942+0000","last_unstale":"2024-05-11T12:14:06.607598+0000","last_undegraded":"2024-05-11T12:14:06.607598+0000","last_fullsized":"2024-05-11T12:14:06.607598+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_clean_scrub_stamp":"2024-05-11T12:10:33.601136+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:12:02.133182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476782,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32796,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339044,"statfs":{"total":96636764160,"available":96603181056,"internally_reserved":0,"allocated":1650688,"data_stored":1065108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476783,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343772,"statfs":{"total":96636764160,"available":96608022528,"internally_reserved":0,"allocated":1069056,"data_stored":476900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476782,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":1048576,"data_stored":474684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476784,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1056768,"data_stored":474844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509487,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1048576,"data_stored":474684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509488,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343800,"statfs":{"total":96636764160,"available":96608051200,"internally_reserved":0,"allocated":1040384,"data_stored":468062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509487,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32812,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1667072,"data_stored":1067370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542191,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1036288,"data_stored":467992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T12:14:10.411 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-11T12:14:10.411 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-11T12:14:10.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.649+0000 7f4ab6800640 1 Processor -- start 2024-05-11T12:14:10.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.650+0000 7f4ab6800640 1 -- start start 2024-05-11T12:14:10.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.651+0000 7f4ab6800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ab0162a40 0x7f4ab00b84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:10.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.651+0000 7f4ab6800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4ab00b3fb0 0x7f4ab00d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:10.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.651+0000 7f4ab6800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ab006dec0 con 0x7f4ab00d3de0 2024-05-11T12:14:10.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.651+0000 7f4ab6800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f4ab00b89f0 con 0x7f4ab00b3fb0 2024-05-11T12:14:10.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.651+0000 7f4ab6800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4ab00d2b70 con 0x7f4ab0162a40 2024-05-11T12:14:10.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.652+0000 7f4ab5400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ab0162a40 0x7f4ab00b84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:10.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.652+0000 7f4ab4a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4ab00b3fb0 0x7f4ab00d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:10.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.652+0000 7f4ab5e00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f4ab00d3de0 0x7f4ab00cff70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.167:50394/0 (socket says 172.21.15.167:50394) 2024-05-11T12:14:10.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.652+0000 7f4ab5e00640 1 -- 172.21.15.167:0/343370085 learned_addr learned my addr 172.21.15.167:0/343370085 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:14:10.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.653+0000 7f4ab5400640 1 -- 172.21.15.167:0/343370085 >> v2:172.21.15.204:3300/0 conn(0x7f4ab00b3fb0 msgr2=0x7f4ab00d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:10.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.653+0000 7f4ab5400640 1 --2- 172.21.15.167:0/343370085 >> v2:172.21.15.204:3300/0 conn(0x7f4ab00b3fb0 0x7f4ab00d38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:10.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.653+0000 7f4ab5400640 1 -- 172.21.15.167:0/343370085 >> v1:172.21.15.204:6789/0 conn(0x7f4ab00d3de0 legacy=0x7f4ab00cff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:10.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.653+0000 7f4ab5400640 1 -- 172.21.15.167:0/343370085 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ab00d2dc0 con 0x7f4ab0162a40 2024-05-11T12:14:10.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.653+0000 7f4ab4a00640 1 --2- 172.21.15.167:0/343370085 >> v2:172.21.15.204:3300/0 conn(0x7f4ab00b3fb0 0x7f4ab00d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:14:10.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.654+0000 7f4ab5400640 1 --2- 172.21.15.167:0/343370085 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ab0162a40 0x7f4ab00b84b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4aa00097a0 tx=0x7f4aa002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a021e536a219d6b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:10.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.655+0000 7f4aafe00640 1 -- 172.21.15.167:0/343370085 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0037070 con 0x7f4ab0162a40 2024-05-11T12:14:10.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.655+0000 7f4aafe00640 1 -- 172.21.15.167:0/343370085 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4aa0032440 con 0x7f4ab0162a40 2024-05-11T12:14:10.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.655+0000 7f4aafe00640 1 -- 172.21.15.167:0/343370085 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0032810 con 0x7f4ab0162a40 2024-05-11T12:14:10.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.656+0000 7f4ab6800640 1 -- 172.21.15.167:0/343370085 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ab0162a40 msgr2=0x7f4ab00b84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:10.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.656+0000 7f4ab6800640 1 --2- 172.21.15.167:0/343370085 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ab0162a40 0x7f4ab00b84b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4aa00097a0 tx=0x7f4aa002a180 comp rx=0 tx=0).stop 2024-05-11T12:14:10.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.656+0000 7f4ab6800640 1 -- 172.21.15.167:0/343370085 shutdown_connections 2024-05-11T12:14:10.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.656+0000 7f4ab6800640 1 --2- 172.21.15.167:0/343370085 >> v2:172.21.15.204:3300/0 conn(0x7f4ab00b3fb0 0x7f4ab00d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:10.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.656+0000 7f4ab6800640 1 --2- 172.21.15.167:0/343370085 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ab0162a40 0x7f4ab00b84b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:10.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.656+0000 7f4ab6800640 1 -- 172.21.15.167:0/343370085 >> 172.21.15.167:0/343370085 conn(0x7f4ab00d1c00 msgr2=0x7f4ab00d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:10.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.656+0000 7f4ab6800640 1 -- 172.21.15.167:0/343370085 shutdown_connections 2024-05-11T12:14:10.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.656+0000 7f4ab6800640 1 -- 172.21.15.167:0/343370085 wait complete. 2024-05-11T12:14:10.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.657+0000 7f4ab6800640 1 Processor -- start 2024-05-11T12:14:10.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.658+0000 7f4ab6800640 1 -- start start 2024-05-11T12:14:10.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.658+0000 7f4ab6800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ab00b3fb0 0x7f4ab0149f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:10.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.659+0000 7f4ab6800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ab00d3de0 0x7f4ab015f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:10.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.659+0000 7f4ab6800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ab0162a40 0x7f4ab015fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:10.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.659+0000 7f4ab6800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab00d4270 con 0x7f4ab0162a40 2024-05-11T12:14:10.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.659+0000 7f4ab6800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab00b3c60 con 0x7f4ab00b3fb0 2024-05-11T12:14:10.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.659+0000 7f4ab6800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4ab00d43f0 con 0x7f4ab00d3de0 2024-05-11T12:14:10.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.659+0000 7f4ab5e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ab0162a40 0x7f4ab015fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:10.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.659+0000 7f4ab5e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ab0162a40 0x7f4ab015fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35578/0 (socket says 172.21.15.167:35578) 2024-05-11T12:14:10.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.659+0000 7f4ab5e00640 1 -- 172.21.15.167:0/2046456625 learned_addr learned my addr 172.21.15.167:0/2046456625 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:10.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.659+0000 7f4ab4a00640 1 --2- 172.21.15.167:0/2046456625 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ab00d3de0 0x7f4ab015f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:10.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.659+0000 7f4ab5400640 1 --2- 172.21.15.167:0/2046456625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ab00b3fb0 0x7f4ab0149f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:10.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.659+0000 7f4ab5e00640 1 -- 172.21.15.167:0/2046456625 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ab00d3de0 msgr2=0x7f4ab015f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:10.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.659+0000 7f4ab5e00640 1 --2- 172.21.15.167:0/2046456625 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ab00d3de0 0x7f4ab015f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:10.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.659+0000 7f4ab5e00640 1 -- 172.21.15.167:0/2046456625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ab00b3fb0 msgr2=0x7f4ab0149f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:10.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.660+0000 7f4ab5e00640 1 --2- 172.21.15.167:0/2046456625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ab00b3fb0 0x7f4ab0149f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:10.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.660+0000 7f4ab5e00640 1 -- 172.21.15.167:0/2046456625 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ab00ca0f0 con 0x7f4ab0162a40 2024-05-11T12:14:10.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.660+0000 7f4ab5e00640 1 --2- 172.21.15.167:0/2046456625 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ab0162a40 0x7f4ab015fa10 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f4aa800cb50 tx=0x7f4aa800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:10.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.660+0000 7f4aad600640 1 -- 172.21.15.167:0/2046456625 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa8017070 con 0x7f4ab0162a40 2024-05-11T12:14:10.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.660+0000 7f4aad600640 1 -- 172.21.15.167:0/2046456625 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4aa800ad40 con 0x7f4ab0162a40 2024-05-11T12:14:10.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.660+0000 7f4ab6800640 1 -- 172.21.15.167:0/2046456625 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ab00ca3d0 con 0x7f4ab0162a40 2024-05-11T12:14:10.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.660+0000 7f4aad600640 1 -- 172.21.15.167:0/2046456625 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa8012600 con 0x7f4ab0162a40 2024-05-11T12:14:10.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.660+0000 7f4ab6800640 1 -- 172.21.15.167:0/2046456625 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ab015d910 con 0x7f4ab0162a40 2024-05-11T12:14:10.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.661+0000 7f4ab6800640 1 -- 172.21.15.167:0/2046456625 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ab015db10 con 0x7f4ab0162a40 2024-05-11T12:14:10.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.663+0000 7f4aad600640 1 -- 172.21.15.167:0/2046456625 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4aa801a070 con 0x7f4ab0162a40 2024-05-11T12:14:10.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.664+0000 7f4aad600640 1 --2- 172.21.15.167:0/2046456625 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4a90076300 0x7f4a900787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:10.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.664+0000 7f4aad600640 1 -- 172.21.15.167:0/2046456625 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7f4aa80974a0 con 0x7f4ab0162a40 2024-05-11T12:14:10.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.664+0000 7f4ab5400640 1 --2- 172.21.15.167:0/2046456625 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4a90076300 0x7f4a900787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:10.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.665+0000 7f4ab5400640 1 --2- 172.21.15.167:0/2046456625 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4a90076300 0x7f4a900787c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0034000 tx=0x7f4aa002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:10.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.669+0000 7f4aad600640 1 -- 172.21.15.167:0/2046456625 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4ab015db10 con 0x7f4ab0162a40 2024-05-11T12:14:10.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.952+0000 7f4ab6800640 1 -- 172.21.15.167:0/2046456625 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f4ab00694b0 con 0x7f4ab0162a40 2024-05-11T12:14:10.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.956+0000 7f4aad600640 1 -- 172.21.15.167:0/2046456625 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) ==== 190+0+147 (secure 0 0 0) 0x7f4aa8060610 con 0x7f4ab0162a40 2024-05-11T12:14:10.957 INFO:teuthology.orchestra.run.smithi167.stdout:[client.0] 2024-05-11T12:14:10.957 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQAFYD9mLK/lEhAAyERTYMl250KkBvIrAXWYSw== 2024-05-11T12:14:10.957 INFO:teuthology.orchestra.run.smithi167.stdout: caps mds = "allow" 2024-05-11T12:14:10.957 INFO:teuthology.orchestra.run.smithi167.stdout: caps mon = "allow r" 2024-05-11T12:14:10.957 INFO:teuthology.orchestra.run.smithi167.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-11T12:14:10.957 INFO:teuthology.orchestra.run.smithi167.stderr:updated caps for client.0 2024-05-11T12:14:10.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.959+0000 7f4ab6800640 1 -- 172.21.15.167:0/2046456625 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4a90076300 msgr2=0x7f4a900787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:10.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.959+0000 7f4ab6800640 1 --2- 172.21.15.167:0/2046456625 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4a90076300 0x7f4a900787c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0034000 tx=0x7f4aa002e000 comp rx=0 tx=0).stop 2024-05-11T12:14:10.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.959+0000 7f4ab6800640 1 -- 172.21.15.167:0/2046456625 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ab0162a40 msgr2=0x7f4ab015fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:10.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.959+0000 7f4ab6800640 1 --2- 172.21.15.167:0/2046456625 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ab0162a40 0x7f4ab015fa10 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f4aa800cb50 tx=0x7f4aa800a430 comp rx=0 tx=0).stop 2024-05-11T12:14:10.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.960+0000 7f4ab6800640 1 -- 172.21.15.167:0/2046456625 shutdown_connections 2024-05-11T12:14:10.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.960+0000 7f4ab6800640 1 --2- 172.21.15.167:0/2046456625 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ab0162a40 0x7f4ab015fa10 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:10.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.960+0000 7f4ab6800640 1 --2- 172.21.15.167:0/2046456625 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ab00d3de0 0x7f4ab015f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:10.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.960+0000 7f4ab6800640 1 --2- 172.21.15.167:0/2046456625 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4a90076300 0x7f4a900787c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:10.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.960+0000 7f4ab6800640 1 --2- 172.21.15.167:0/2046456625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ab00b3fb0 0x7f4ab0149f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:10.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.960+0000 7f4ab6800640 1 -- 172.21.15.167:0/2046456625 >> 172.21.15.167:0/2046456625 conn(0x7f4ab00d1c00 msgr2=0x7f4ab00b8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:10.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.960+0000 7f4ab6800640 1 -- 172.21.15.167:0/2046456625 shutdown_connections 2024-05-11T12:14:10.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:10.960+0000 7f4ab6800640 1 -- 172.21.15.167:0/2046456625 wait complete. 2024-05-11T12:14:10.979 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:14:11.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.174+0000 7f202c400640 1 Processor -- start 2024-05-11T12:14:11.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.175+0000 7f202c400640 1 -- start start 2024-05-11T12:14:11.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.176+0000 7f202c400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20240b3ea0 0x7f2024161490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:11.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.176+0000 7f202c400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f20241619d0 0x7f202406d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:11.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.176+0000 7f202c400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f202415df30 con 0x7f2024157750 2024-05-11T12:14:11.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.176+0000 7f202c400640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f2024161000 con 0x7f20241619d0 2024-05-11T12:14:11.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.176+0000 7f202c400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f202415e210 con 0x7f20240b3ea0 2024-05-11T12:14:11.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.176+0000 7f202a400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f20241619d0 0x7f202406d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:11.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.177+0000 7f202a400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f20241619d0 0x7f202406d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35580/0 (socket says 172.21.15.167:35580) 2024-05-11T12:14:11.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.177+0000 7f202a400640 1 -- 172.21.15.167:0/836021547 learned_addr learned my addr 172.21.15.167:0/836021547 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:11.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.177+0000 7f2029000640 1 --2- 172.21.15.167:0/836021547 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20240b3ea0 0x7f2024161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:11.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.177+0000 7f202a400640 1 -- 172.21.15.167:0/836021547 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20240b3ea0 msgr2=0x7f2024161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:11.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.177+0000 7f202a400640 1 --2- 172.21.15.167:0/836021547 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20240b3ea0 0x7f2024161490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:11.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.177+0000 7f202a400640 1 -- 172.21.15.167:0/836021547 >> v1:172.21.15.204:6789/0 conn(0x7f2024157750 legacy=0x7f202415d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:11.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.177+0000 7f202a400640 1 -- 172.21.15.167:0/836021547 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f202415e4f0 con 0x7f20241619d0 2024-05-11T12:14:11.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.177+0000 7f2029000640 1 --2- 172.21.15.167:0/836021547 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20240b3ea0 0x7f2024161490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:11.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.178+0000 7f202a400640 1 --2- 172.21.15.167:0/836021547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20241619d0 0x7f202406d460 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f201c009e50 tx=0x7f201c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7658549d68fdca4 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:11.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.179+0000 7f2023e00640 1 -- 172.21.15.167:0/836021547 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201c036070 con 0x7f20241619d0 2024-05-11T12:14:11.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.179+0000 7f2023e00640 1 -- 172.21.15.167:0/836021547 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f201c031600 con 0x7f20241619d0 2024-05-11T12:14:11.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.179+0000 7f2023e00640 1 -- 172.21.15.167:0/836021547 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201c0319d0 con 0x7f20241619d0 2024-05-11T12:14:11.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.179+0000 7f202c400640 1 -- 172.21.15.167:0/836021547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20241619d0 msgr2=0x7f202406d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:11.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.179+0000 7f202c400640 1 --2- 172.21.15.167:0/836021547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20241619d0 0x7f202406d460 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f201c009e50 tx=0x7f201c02a680 comp rx=0 tx=0).stop 2024-05-11T12:14:11.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.179+0000 7f202c400640 1 -- 172.21.15.167:0/836021547 shutdown_connections 2024-05-11T12:14:11.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.179+0000 7f202c400640 1 --2- 172.21.15.167:0/836021547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20241619d0 0x7f202406d460 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:11.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.179+0000 7f202c400640 1 --2- 172.21.15.167:0/836021547 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20240b3ea0 0x7f2024161490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:11.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.179+0000 7f202c400640 1 -- 172.21.15.167:0/836021547 >> 172.21.15.167:0/836021547 conn(0x7f20240cd8f0 msgr2=0x7f20240cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:11.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.180+0000 7f202c400640 1 -- 172.21.15.167:0/836021547 shutdown_connections 2024-05-11T12:14:11.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.180+0000 7f202c400640 1 -- 172.21.15.167:0/836021547 wait complete. 2024-05-11T12:14:11.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.181+0000 7f202c400640 1 Processor -- start 2024-05-11T12:14:11.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.181+0000 7f202c400640 1 -- start start 2024-05-11T12:14:11.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.182+0000 7f202c400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20240b3ea0 0x7f2024141830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:11.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.182+0000 7f202c400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2024157750 0x7f2024140660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:11.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.182+0000 7f202c400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20241619d0 0x7f20241f3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:11.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.182+0000 7f202c400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f202406db20 con 0x7f2024157750 2024-05-11T12:14:11.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.182+0000 7f202c400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f202406d9a0 con 0x7f20241619d0 2024-05-11T12:14:11.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.182+0000 7f202c400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f202406dca0 con 0x7f20240b3ea0 2024-05-11T12:14:11.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.183+0000 7f2029000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2024157750 0x7f2024140660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:11.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.183+0000 7f2029000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2024157750 0x7f2024140660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35586/0 (socket says 172.21.15.167:35586) 2024-05-11T12:14:11.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.183+0000 7f2029000640 1 -- 172.21.15.167:0/1775693881 learned_addr learned my addr 172.21.15.167:0/1775693881 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:11.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.183+0000 7f2029a00640 1 --2- 172.21.15.167:0/1775693881 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20240b3ea0 0x7f2024141830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:11.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.183+0000 7f2029000640 1 -- 172.21.15.167:0/1775693881 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20240b3ea0 msgr2=0x7f2024141830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:11.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.183+0000 7f202a400640 1 --2- 172.21.15.167:0/1775693881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20241619d0 0x7f20241f3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:11.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.183+0000 7f2029000640 1 --2- 172.21.15.167:0/1775693881 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20240b3ea0 0x7f2024141830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:11.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.183+0000 7f2029000640 1 -- 172.21.15.167:0/1775693881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20241619d0 msgr2=0x7f20241f3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:11.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.183+0000 7f2029000640 1 --2- 172.21.15.167:0/1775693881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20241619d0 0x7f20241f3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:11.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.183+0000 7f2029000640 1 -- 172.21.15.167:0/1775693881 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2024140ba0 con 0x7f2024157750 2024-05-11T12:14:11.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.183+0000 7f202a400640 1 --2- 172.21.15.167:0/1775693881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20241619d0 0x7f20241f3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:11.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.183+0000 7f2029a00640 1 --2- 172.21.15.167:0/1775693881 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20240b3ea0 0x7f2024141830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:11.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.184+0000 7f2029000640 1 --2- 172.21.15.167:0/1775693881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2024157750 0x7f2024140660 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f201000c820 tx=0x7f201000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:11.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.184+0000 7f2021600640 1 -- 172.21.15.167:0/1775693881 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2010015070 con 0x7f2024157750 2024-05-11T12:14:11.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.184+0000 7f2021600640 1 -- 172.21.15.167:0/1775693881 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2010007a30 con 0x7f2024157750 2024-05-11T12:14:11.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.184+0000 7f2021600640 1 -- 172.21.15.167:0/1775693881 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2010007e00 con 0x7f2024157750 2024-05-11T12:14:11.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.184+0000 7f202c400640 1 -- 172.21.15.167:0/1775693881 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2024141e70 con 0x7f2024157750 2024-05-11T12:14:11.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.184+0000 7f202c400640 1 -- 172.21.15.167:0/1775693881 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2024159320 con 0x7f2024157750 2024-05-11T12:14:11.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.185+0000 7f202c400640 1 -- 172.21.15.167:0/1775693881 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2024076420 con 0x7f2024157750 2024-05-11T12:14:11.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.186+0000 7f2021600640 1 -- 172.21.15.167:0/1775693881 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2010018070 con 0x7f2024157750 2024-05-11T12:14:11.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.187+0000 7f2021600640 1 --2- 172.21.15.167:0/1775693881 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f20040762d0 0x7f2004078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:11.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.187+0000 7f2021600640 1 -- 172.21.15.167:0/1775693881 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7f2010097700 con 0x7f2024157750 2024-05-11T12:14:11.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.188+0000 7f2029a00640 1 --2- 172.21.15.167:0/1775693881 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f20040762d0 0x7f2004078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:11.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.188+0000 7f2029a00640 1 --2- 172.21.15.167:0/1775693881 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f20040762d0 0x7f2004078790 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f20241605e0 tx=0x7f2018009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:11.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.192+0000 7f2021600640 1 -- 172.21.15.167:0/1775693881 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2024076420 con 0x7f2024157750 2024-05-11T12:14:11.351 INFO:tasks.ceph.mds.a.smithi167.stderr:2024-05-11T12:14:11.351+0000 7fb27e000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-11T12:14:11.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.434+0000 7f202c400640 1 -- 172.21.15.167:0/1775693881 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2024158e50 con 0x7f2024157750 2024-05-11T12:14:11.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.435+0000 7f2021600640 1 -- 172.21.15.167:0/1775693881 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5195 (secure 0 0 0) 0x7f2010060ed0 con 0x7f2024157750 2024-05-11T12:14:11.435 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:11.435 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4922,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1154357925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1154357925},{"type":"v1","addr":"172.21.15.167:6837","nonce":1154357925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":5155,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1256314893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1256314893},{"type":"v1","addr":"172.21.15.204:6833","nonce":1256314893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":5158,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1578748665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1578748665},{"type":"v1","addr":"172.21.15.204:6835","nonce":1578748665}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:14:04.861998+0000","modified":"2024-05-11T12:14:07.926157+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6300":{"gid":6300,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6836/3484070468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3484070468},{"type":"v1","addr":"172.21.15.167:6836","nonce":3484070468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6300,"qdb_cluster":[6300]},"id":6}]} 2024-05-11T12:14:11.435 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 53 2024-05-11T12:14:11.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.437+0000 7f202c400640 1 -- 172.21.15.167:0/1775693881 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f20040762d0 msgr2=0x7f2004078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:11.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.437+0000 7f202c400640 1 --2- 172.21.15.167:0/1775693881 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f20040762d0 0x7f2004078790 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f20241605e0 tx=0x7f2018009040 comp rx=0 tx=0).stop 2024-05-11T12:14:11.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.437+0000 7f202c400640 1 -- 172.21.15.167:0/1775693881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2024157750 msgr2=0x7f2024140660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:11.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.437+0000 7f202c400640 1 --2- 172.21.15.167:0/1775693881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2024157750 0x7f2024140660 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f201000c820 tx=0x7f201000cc20 comp rx=0 tx=0).stop 2024-05-11T12:14:11.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.438+0000 7f202c400640 1 -- 172.21.15.167:0/1775693881 shutdown_connections 2024-05-11T12:14:11.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.438+0000 7f202c400640 1 --2- 172.21.15.167:0/1775693881 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f20040762d0 0x7f2004078790 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:11.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.438+0000 7f202c400640 1 --2- 172.21.15.167:0/1775693881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20241619d0 0x7f20241f3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:11.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.438+0000 7f202c400640 1 --2- 172.21.15.167:0/1775693881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2024157750 0x7f2024140660 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:11.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.438+0000 7f202c400640 1 --2- 172.21.15.167:0/1775693881 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20240b3ea0 0x7f2024141830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:11.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.438+0000 7f202c400640 1 -- 172.21.15.167:0/1775693881 >> 172.21.15.167:0/1775693881 conn(0x7f20240cd8f0 msgr2=0x7f202406e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:11.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.438+0000 7f202c400640 1 -- 172.21.15.167:0/1775693881 shutdown_connections 2024-05-11T12:14:11.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:11.438+0000 7f202c400640 1 -- 172.21.15.167:0/1775693881 wait complete. 2024-05-11T12:14:11.859 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 53, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-11T12:14:04.861998+0000', 'modified': '2024-05-11T12:14:07.926157+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6300}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6300': {'gid': 6300, 'name': 'a', 'rank': 0, 'incarnation': 49, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.167:6836/3484070468', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6835', 'nonce': 3484070468}, {'type': 'v1', 'addr': '172.21.15.167:6836', 'nonce': 3484070468}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6300, 'qdb_cluster': [6300]} 2024-05-11T12:14:11.859 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-11T12:14:11.860 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-11T12:14:12.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.070+0000 7f602ec00640 1 Processor -- start 2024-05-11T12:14:12.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.071+0000 7f602ec00640 1 -- start start 2024-05-11T12:14:12.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.072+0000 7f602ec00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f60281625d0 0x7f60280d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:12.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.072+0000 7f602ec00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f60280b8390 0x7f60280b8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:12.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.072+0000 7f602ec00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60280d2e40 con 0x7f60280d3c00 2024-05-11T12:14:12.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.072+0000 7f602ec00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f602806df00 con 0x7f60280b8390 2024-05-11T12:14:12.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.072+0000 7f602ec00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f60280d3120 con 0x7f60281625d0 2024-05-11T12:14:12.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.072+0000 7f602ce00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f60280b8390 0x7f60280b8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:12.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.073+0000 7f602ce00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f60280b8390 0x7f60280b8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35592/0 (socket says 172.21.15.167:35592) 2024-05-11T12:14:12.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.073+0000 7f602ce00640 1 -- 172.21.15.167:0/713402862 learned_addr learned my addr 172.21.15.167:0/713402862 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:12.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.073+0000 7f602d800640 1 --2- 172.21.15.167:0/713402862 >> v2:172.21.15.204:3300/0 conn(0x7f60281625d0 0x7f60280d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:12.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.073+0000 7f602ce00640 1 -- 172.21.15.167:0/713402862 >> v2:172.21.15.204:3300/0 conn(0x7f60281625d0 msgr2=0x7f60280d0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:12.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.073+0000 7f602ce00640 1 --2- 172.21.15.167:0/713402862 >> v2:172.21.15.204:3300/0 conn(0x7f60281625d0 0x7f60280d0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:12.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.073+0000 7f602ce00640 1 -- 172.21.15.167:0/713402862 >> v1:172.21.15.204:6789/0 conn(0x7f60280d3c00 legacy=0x7f60280d4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:12.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.073+0000 7f602ce00640 1 -- 172.21.15.167:0/713402862 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60280d3400 con 0x7f60280b8390 2024-05-11T12:14:12.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.074+0000 7f602ce00640 1 --2- 172.21.15.167:0/713402862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60280b8390 0x7f60280b8820 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f60280b7c10 tx=0x7f601802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f805e558b620b4c server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:12.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.075+0000 7f6027e00640 1 -- 172.21.15.167:0/713402862 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6018036070 con 0x7f60280b8390 2024-05-11T12:14:12.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.075+0000 7f6027e00640 1 -- 172.21.15.167:0/713402862 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6018031600 con 0x7f60280b8390 2024-05-11T12:14:12.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.075+0000 7f6027e00640 1 -- 172.21.15.167:0/713402862 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60180319d0 con 0x7f60280b8390 2024-05-11T12:14:12.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.075+0000 7f602ec00640 1 -- 172.21.15.167:0/713402862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60280b8390 msgr2=0x7f60280b8820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:12.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.075+0000 7f602ec00640 1 --2- 172.21.15.167:0/713402862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60280b8390 0x7f60280b8820 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f60280b7c10 tx=0x7f601802a730 comp rx=0 tx=0).stop 2024-05-11T12:14:12.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.075+0000 7f602ec00640 1 -- 172.21.15.167:0/713402862 shutdown_connections 2024-05-11T12:14:12.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.075+0000 7f602ec00640 1 --2- 172.21.15.167:0/713402862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60280b8390 0x7f60280b8820 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:12.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.075+0000 7f602ec00640 1 --2- 172.21.15.167:0/713402862 >> v2:172.21.15.204:3300/0 conn(0x7f60281625d0 0x7f60280d0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:12.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.076+0000 7f602ec00640 1 -- 172.21.15.167:0/713402862 >> 172.21.15.167:0/713402862 conn(0x7f60280d50f0 msgr2=0x7f60280d5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:12.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.076+0000 7f602ec00640 1 -- 172.21.15.167:0/713402862 shutdown_connections 2024-05-11T12:14:12.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.076+0000 7f602ec00640 1 -- 172.21.15.167:0/713402862 wait complete. 2024-05-11T12:14:12.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.077+0000 7f602ec00640 1 Processor -- start 2024-05-11T12:14:12.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.077+0000 7f602ec00640 1 -- start start 2024-05-11T12:14:12.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.078+0000 7f602ec00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60280b8390 0x7f602815db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:12.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.078+0000 7f602ec00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f60280d3c00 0x7f60281f8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:12.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.078+0000 7f602ec00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60281625d0 0x7f60280cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:12.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.078+0000 7f602ec00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6028162b30 con 0x7f60281625d0 2024-05-11T12:14:12.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.078+0000 7f602ec00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f60281629b0 con 0x7f60280b8390 2024-05-11T12:14:12.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.078+0000 7f602ec00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f602806cef0 con 0x7f60280d3c00 2024-05-11T12:14:12.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.079+0000 7f602e200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60281625d0 0x7f60280cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:12.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.079+0000 7f602e200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60281625d0 0x7f60280cd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35596/0 (socket says 172.21.15.167:35596) 2024-05-11T12:14:12.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.079+0000 7f602e200640 1 -- 172.21.15.167:0/1756414679 learned_addr learned my addr 172.21.15.167:0/1756414679 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:12.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.079+0000 7f602ce00640 1 --2- 172.21.15.167:0/1756414679 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f60280d3c00 0x7f60281f8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:12.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.079+0000 7f602d800640 1 --2- 172.21.15.167:0/1756414679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60280b8390 0x7f602815db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:12.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.079+0000 7f602e200640 1 -- 172.21.15.167:0/1756414679 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f60280d3c00 msgr2=0x7f60281f8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:12.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.079+0000 7f602e200640 1 --2- 172.21.15.167:0/1756414679 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f60280d3c00 0x7f60281f8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:12.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.079+0000 7f602e200640 1 -- 172.21.15.167:0/1756414679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60280b8390 msgr2=0x7f602815db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:12.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.079+0000 7f602e200640 1 --2- 172.21.15.167:0/1756414679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60280b8390 0x7f602815db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:12.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.079+0000 7f602e200640 1 -- 172.21.15.167:0/1756414679 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6028160430 con 0x7f60281625d0 2024-05-11T12:14:12.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.079+0000 7f602ce00640 1 --2- 172.21.15.167:0/1756414679 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f60280d3c00 0x7f60281f8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:12.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.079+0000 7f602d800640 1 --2- 172.21.15.167:0/1756414679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60280b8390 0x7f602815db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:12.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.079+0000 7f602e200640 1 --2- 172.21.15.167:0/1756414679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60281625d0 0x7f60280cd5f0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f602000c430 tx=0x7f602000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:12.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.080+0000 7f6025600640 1 -- 172.21.15.167:0/1756414679 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6020017070 con 0x7f60281625d0 2024-05-11T12:14:12.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.080+0000 7f6025600640 1 -- 172.21.15.167:0/1756414679 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60200052d0 con 0x7f60281625d0 2024-05-11T12:14:12.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.080+0000 7f6025600640 1 -- 172.21.15.167:0/1756414679 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60200037f0 con 0x7f60281625d0 2024-05-11T12:14:12.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.080+0000 7f602ec00640 1 -- 172.21.15.167:0/1756414679 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6028160710 con 0x7f60281625d0 2024-05-11T12:14:12.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.080+0000 7f602ec00640 1 -- 172.21.15.167:0/1756414679 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6028160cd0 con 0x7f60281625d0 2024-05-11T12:14:12.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.082+0000 7f6025600640 1 -- 172.21.15.167:0/1756414679 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f602001a070 con 0x7f60281625d0 2024-05-11T12:14:12.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.083+0000 7f6025600640 1 --2- 172.21.15.167:0/1756414679 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f600c0762d0 0x7f600c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:12.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.084+0000 7f6025600640 1 -- 172.21.15.167:0/1756414679 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7f6020096a60 con 0x7f60281625d0 2024-05-11T12:14:12.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.084+0000 7f602d800640 1 --2- 172.21.15.167:0/1756414679 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f600c0762d0 0x7f600c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:12.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.084+0000 7f6033021740 1 Processor -- start 2024-05-11T12:14:12.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.084+0000 7f6033021740 1 -- start start 2024-05-11T12:14:12.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.084+0000 7f602d800640 1 --2- 172.21.15.167:0/1756414679 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f600c0762d0 0x7f600c078790 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f60280cef00 tx=0x7f601c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:12.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.085+0000 7f6033021740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa9e90a10 0x55faa9e90df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:12.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.085+0000 7f6033021740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55faa9e8b8a0 0x55faa9e8f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:12.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.085+0000 7f6033021740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55faa9e8fb50 0x55faa9e8ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:12.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.085+0000 7f6033021740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55faa9df4790 con 0x55faa9e90a10 2024-05-11T12:14:12.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.085+0000 7f6033021740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55faa9d99f20 con 0x55faa9e8b8a0 2024-05-11T12:14:12.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.085+0000 7f6033021740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55faa9df0750 con 0x55faa9e8fb50 2024-05-11T12:14:12.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.085+0000 7f602d800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa9e90a10 0x55faa9e90df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:12.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.085+0000 7f602d800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa9e90a10 0x55faa9e90df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35598/0 (socket says 172.21.15.167:35598) 2024-05-11T12:14:12.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.085+0000 7f602d800640 1 -- 172.21.15.167:0/580330861 learned_addr learned my addr 172.21.15.167:0/580330861 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:12.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.086+0000 7f602d800640 1 -- 172.21.15.167:0/580330861 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55faa9e8fb50 msgr2=0x55faa9e8ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:12.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.086+0000 7f602d800640 1 --2- 172.21.15.167:0/580330861 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55faa9e8fb50 0x55faa9e8ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:12.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.086+0000 7f602d800640 1 -- 172.21.15.167:0/580330861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55faa9e8b8a0 msgr2=0x55faa9e8f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:12.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.086+0000 7f602d800640 1 --2- 172.21.15.167:0/580330861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55faa9e8b8a0 0x55faa9e8f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:12.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.086+0000 7f602d800640 1 -- 172.21.15.167:0/580330861 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faa9b062d0 con 0x55faa9e90a10 2024-05-11T12:14:12.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.086+0000 7f602d800640 1 --2- 172.21.15.167:0/580330861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa9e90a10 0x55faa9e90df0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f601c0042f0 tx=0x7f601c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=427262a33d8b36df server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:12.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.086+0000 7f6016000640 1 -- 172.21.15.167:0/580330861 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c027070 con 0x55faa9e90a10 2024-05-11T12:14:12.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.086+0000 7f6016000640 1 -- 172.21.15.167:0/580330861 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f601c003b90 con 0x55faa9e90a10 2024-05-11T12:14:12.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.086+0000 7f6016000640 1 -- 172.21.15.167:0/580330861 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c00f040 con 0x55faa9e90a10 2024-05-11T12:14:12.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.087+0000 7f6033021740 1 -- 172.21.15.167:0/580330861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa9e90a10 msgr2=0x55faa9e90df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:12.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.087+0000 7f6033021740 1 --2- 172.21.15.167:0/580330861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa9e90a10 0x55faa9e90df0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f601c0042f0 tx=0x7f601c013000 comp rx=0 tx=0).stop 2024-05-11T12:14:12.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.087+0000 7f6033021740 1 -- 172.21.15.167:0/580330861 shutdown_connections 2024-05-11T12:14:12.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.087+0000 7f6033021740 1 --2- 172.21.15.167:0/580330861 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55faa9e8fb50 0x55faa9e8ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:12.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.087+0000 7f6033021740 1 --2- 172.21.15.167:0/580330861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55faa9e8b8a0 0x55faa9e8f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:12.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.087+0000 7f6033021740 1 --2- 172.21.15.167:0/580330861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa9e90a10 0x55faa9e90df0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:12.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.087+0000 7f6033021740 1 -- 172.21.15.167:0/580330861 >> 172.21.15.167:0/580330861 conn(0x55faa9e8db10 msgr2=0x55faa9ed65d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:12.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.087+0000 7f6033021740 1 -- 172.21.15.167:0/580330861 shutdown_connections 2024-05-11T12:14:12.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.087+0000 7f6033021740 1 -- 172.21.15.167:0/580330861 wait complete. 2024-05-11T12:14:12.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.088+0000 7f6033021740 1 -- start start 2024-05-11T12:14:12.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.088+0000 7f6033021740 1 Processor -- start 2024-05-11T12:14:12.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.089+0000 7f6033021740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa9e8fb50 0x55faa9edbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:12.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.089+0000 7f6033021740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55faa9e90a10 0x55faa9eb0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:12.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.089+0000 7f6033021740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55faa9eae4e0 0x55faa9eb0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:12.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.089+0000 7f6033021740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55faa9ebc5a0 con 0x55faa9e8fb50 2024-05-11T12:14:12.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.089+0000 7f6033021740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55faa9dad9c0 con 0x55faa9e90a10 2024-05-11T12:14:12.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.089+0000 7f6033021740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55faa9ebba20 con 0x55faa9eae4e0 2024-05-11T12:14:12.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.089+0000 7f6033021740 10 client.0 _ll_register_callbacks cb 0x55faa9d7a120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:14:12.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.089+0000 7f602d800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa9e8fb50 0x55faa9edbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:12.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.089+0000 7f602d800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa9e8fb50 0x55faa9edbee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35612/0 (socket says 172.21.15.167:35612) 2024-05-11T12:14:12.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.089+0000 7f602d800640 1 -- 172.21.15.167:0/1346797673 learned_addr learned my addr 172.21.15.167:0/1346797673 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:12.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.089+0000 7f602d800640 1 -- 172.21.15.167:0/1346797673 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55faa9eae4e0 msgr2=0x55faa9eb0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:12.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.089+0000 7f602d800640 1 --2- 172.21.15.167:0/1346797673 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55faa9eae4e0 0x55faa9eb0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:12.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.089+0000 7f602d800640 1 -- 172.21.15.167:0/1346797673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55faa9e90a10 msgr2=0x55faa9eb0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:12.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.089+0000 7f602d800640 1 --2- 172.21.15.167:0/1346797673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55faa9e90a10 0x55faa9eb0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:12.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.089+0000 7f602d800640 1 -- 172.21.15.167:0/1346797673 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55faa9bd06b0 con 0x55faa9e8fb50 2024-05-11T12:14:12.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.090+0000 7f602d800640 1 --2- 172.21.15.167:0/1346797673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa9e8fb50 0x55faa9edbee0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f601c025040 tx=0x7f601c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc82765b407634b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:12.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.090+0000 7f600aa00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-11T12:14:12.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.090+0000 7f600aa00640 1 -- 172.21.15.167:0/1346797673 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c031020 con 0x55faa9e8fb50 2024-05-11T12:14:12.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.090+0000 7f600aa00640 1 -- 172.21.15.167:0/1346797673 <== mon.0 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7f601c016950 con 0x55faa9e8fb50 2024-05-11T12:14:12.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.090+0000 7f6033021740 1 -- 172.21.15.167:0/1346797673 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f601c003b90 con 0x55faa9e8fb50 2024-05-11T12:14:12.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.090+0000 7f600aa00640 1 -- 172.21.15.167:0/1346797673 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f601c03b050 con 0x55faa9e8fb50 2024-05-11T12:14:12.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.090+0000 7f6033021740 10 client.6429 fetch_fsmap learned FSMap version 53 2024-05-11T12:14:12.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.090+0000 7f6033021740 1 -- 172.21.15.167:0/1346797673 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x55faa9edb400 con 0x55faa9e8fb50 2024-05-11T12:14:12.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.091+0000 7f600aa00640 1 -- 172.21.15.167:0/1346797673 <== mon.0 v2:172.21.15.167:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f601c003b90 con 0x55faa9e8fb50 2024-05-11T12:14:12.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.091+0000 7f600aa00640 1 -- 172.21.15.167:0/1346797673 <== mon.0 v2:172.21.15.167:3300/0 5 ==== fsmap(e 53) ==== 3494+0+0 (secure 0 0 0) 0x7f601c03e020 con 0x55faa9e8fb50 2024-05-11T12:14:12.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.091+0000 7f6033021740 10 client.6429 fetch_fsmap finished waiting for FSMap version 53 2024-05-11T12:14:12.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.091+0000 7f6033021740 10 client.6429 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-11T12:14:12.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.091+0000 7f6033021740 20 client.6429 populate_metadata read hostname 'smithi167' 2024-05-11T12:14:12.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.091+0000 7f6033021740 1 --2- 172.21.15.167:0/1346797673 >> [v2:172.21.15.167:6835/3484070468,v1:172.21.15.167:6836/3484070468] conn(0x55faa9edcc70 0x55faa9edd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:12.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.091+0000 7f6033021740 4 client.6429 mds_command: new command op to 6300 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:14:12.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.091+0000 7f6033021740 1 -- 172.21.15.167:0/1346797673 --> [v2:172.21.15.167:6835/3484070468,v1:172.21.15.167:6836/3484070468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55faa9e023a0 con 0x55faa9edcc70 2024-05-11T12:14:12.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.091+0000 7f602ce00640 1 --2- 172.21.15.167:0/1346797673 >> [v2:172.21.15.167:6835/3484070468,v1:172.21.15.167:6836/3484070468] conn(0x55faa9edcc70 0x55faa9edd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:12.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.092+0000 7f602ce00640 1 --2- 172.21.15.167:0/1346797673 >> [v2:172.21.15.167:6835/3484070468,v1:172.21.15.167:6836/3484070468] conn(0x55faa9edcc70 0x55faa9edd050 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d35ed1c11ec1ce17 server_cookie=28115754f4b469f8 in_seq=0 out_seq=0 2024-05-11T12:14:12.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.092+0000 7f600aa00640 10 client.6429 ms_handle_connect on v2:172.21.15.167:6835/3484070468 2024-05-11T12:14:12.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.092+0000 7f600aa00640 1 -- 172.21.15.167:0/1346797673 <== mds.0 v2:172.21.15.167:6835/3484070468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55faa9e023a0 con 0x55faa9edcc70 2024-05-11T12:14:12.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.092+0000 7f600aa00640 10 client.6429 handle_command_reply: tid=0 2024-05-11T12:14:12.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.093+0000 7f6033021740 1 client.6429 shutdown 2024-05-11T12:14:12.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.093+0000 7f6033021740 1 -- 172.21.15.167:0/1346797673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa9e8fb50 msgr2=0x55faa9edbee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:12.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.093+0000 7f6033021740 1 --2- 172.21.15.167:0/1346797673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa9e8fb50 0x55faa9edbee0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f601c025040 tx=0x7f601c0046c0 comp rx=0 tx=0).stop 2024-05-11T12:14:12.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.093+0000 7f6033021740 1 -- 172.21.15.167:0/1346797673 shutdown_connections 2024-05-11T12:14:12.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.093+0000 7f6033021740 1 --2- 172.21.15.167:0/1346797673 >> [v2:172.21.15.167:6835/3484070468,v1:172.21.15.167:6836/3484070468] conn(0x55faa9edcc70 0x55faa9edd050 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:12.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.093+0000 7f6033021740 1 --2- 172.21.15.167:0/1346797673 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55faa9eae4e0 0x55faa9eb0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:12.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.093+0000 7f6033021740 1 --2- 172.21.15.167:0/1346797673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55faa9e90a10 0x55faa9eb0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:12.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.093+0000 7f6033021740 1 --2- 172.21.15.167:0/1346797673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa9e8fb50 0x55faa9edbee0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:12.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.093+0000 7f600aa00640 0 client.6429 ms_handle_reset on v2:172.21.15.167:6835/3484070468 2024-05-11T12:14:12.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.093+0000 7f6033021740 1 -- 172.21.15.167:0/1346797673 >> 172.21.15.167:0/1346797673 conn(0x55faa9e8b8a0 msgr2=0x55faa9e8ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:12.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.093+0000 7f6033021740 1 -- 172.21.15.167:0/1346797673 shutdown_connections 2024-05-11T12:14:12.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.093+0000 7f6033021740 1 -- 172.21.15.167:0/1346797673 wait complete. 2024-05-11T12:14:12.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.093+0000 7f6033021740 20 client.6429 trim_cache size 0 max 16384 2024-05-11T12:14:12.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.113+0000 7f6033021740 1 Processor -- start 2024-05-11T12:14:12.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.113+0000 7f6033021740 1 -- start start 2024-05-11T12:14:12.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.114+0000 7f6033021740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa9e8fb50 0x55faa9efa6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:12.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.114+0000 7f6033021740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55faa9e90a10 0x55faa9eafd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:12.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.114+0000 7f6033021740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55faa9eae4e0 0x55faa9eb0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:12.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.114+0000 7f6033021740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6028200d20 con 0x55faa9e8fb50 2024-05-11T12:14:12.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.114+0000 7f6033021740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f60281f5760 con 0x55faa9eae4e0 2024-05-11T12:14:12.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.114+0000 7f6033021740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f60282006d0 con 0x55faa9e90a10 2024-05-11T12:14:12.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.114+0000 7f602d800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa9e8fb50 0x55faa9efa6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:12.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.114+0000 7f602d800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa9e8fb50 0x55faa9efa6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35618/0 (socket says 172.21.15.167:35618) 2024-05-11T12:14:12.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.114+0000 7f602d800640 1 -- 172.21.15.167:0/129303200 learned_addr learned my addr 172.21.15.167:0/129303200 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:12.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.115+0000 7f602d800640 1 -- 172.21.15.167:0/129303200 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55faa9e90a10 msgr2=0x55faa9eafd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:12.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.115+0000 7f602d800640 1 --2- 172.21.15.167:0/129303200 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55faa9e90a10 0x55faa9eafd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:12.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.115+0000 7f602d800640 1 -- 172.21.15.167:0/129303200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55faa9eae4e0 msgr2=0x55faa9eb0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:12.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.115+0000 7f602d800640 1 --2- 172.21.15.167:0/129303200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55faa9eae4e0 0x55faa9eb0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:12.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.115+0000 7f602d800640 1 -- 172.21.15.167:0/129303200 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faa9e8efe0 con 0x55faa9e8fb50 2024-05-11T12:14:12.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.115+0000 7f602d800640 1 --2- 172.21.15.167:0/129303200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa9e8fb50 0x55faa9efa6a0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f601c008010 tx=0x7f601c029000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1388e5c6b3279637 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:12.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.115+0000 7f6015600640 1 -- 172.21.15.167:0/129303200 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c01b070 con 0x55faa9e8fb50 2024-05-11T12:14:12.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.115+0000 7f6015600640 1 -- 172.21.15.167:0/129303200 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f601c016630 con 0x55faa9e8fb50 2024-05-11T12:14:12.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.115+0000 7f6015600640 1 -- 172.21.15.167:0/129303200 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c031070 con 0x55faa9e8fb50 2024-05-11T12:14:12.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.116+0000 7f6033021740 1 -- 172.21.15.167:0/129303200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa9e8fb50 msgr2=0x55faa9efa6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:12.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.116+0000 7f6033021740 1 --2- 172.21.15.167:0/129303200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa9e8fb50 0x55faa9efa6a0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f601c008010 tx=0x7f601c029000 comp rx=0 tx=0).stop 2024-05-11T12:14:12.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.116+0000 7f6033021740 1 -- 172.21.15.167:0/129303200 shutdown_connections 2024-05-11T12:14:12.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.116+0000 7f6033021740 1 --2- 172.21.15.167:0/129303200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55faa9eae4e0 0x55faa9eb0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:12.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.116+0000 7f6033021740 1 --2- 172.21.15.167:0/129303200 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55faa9e90a10 0x55faa9eafd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:12.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.116+0000 7f6033021740 1 --2- 172.21.15.167:0/129303200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa9e8fb50 0x55faa9efa6a0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:12.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.116+0000 7f6033021740 1 -- 172.21.15.167:0/129303200 >> 172.21.15.167:0/129303200 conn(0x55faa9e8b8a0 msgr2=0x55faa9e8e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:12.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.116+0000 7f6033021740 1 -- 172.21.15.167:0/129303200 shutdown_connections 2024-05-11T12:14:12.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.116+0000 7f6033021740 1 -- 172.21.15.167:0/129303200 wait complete. 2024-05-11T12:14:12.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.117+0000 7f6033021740 1 -- start start 2024-05-11T12:14:12.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.117+0000 7f6033021740 1 Processor -- start 2024-05-11T12:14:12.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.118+0000 7f6033021740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55faa9e8fb50 0x55faa9f3d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:12.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.118+0000 7f6033021740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55faa9e90a10 0x55faa9eba570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:12.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.118+0000 7f6033021740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa9eae4e0 0x55faa9eb11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:12.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.118+0000 7f6033021740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55faa9ead5a0 con 0x55faa9eae4e0 2024-05-11T12:14:12.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.118+0000 7f6033021740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55faa9edb730 con 0x55faa9e90a10 2024-05-11T12:14:12.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.118+0000 7f6033021740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55faa9eea450 con 0x55faa9e8fb50 2024-05-11T12:14:12.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.118+0000 7f6033021740 10 client.0 _ll_register_callbacks cb 0x7f601c025070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:14:12.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.118+0000 7f602e200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa9eae4e0 0x55faa9eb11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:12.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.118+0000 7f602e200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa9eae4e0 0x55faa9eb11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35626/0 (socket says 172.21.15.167:35626) 2024-05-11T12:14:12.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.118+0000 7f602e200640 1 -- 172.21.15.167:0/1209499057 learned_addr learned my addr 172.21.15.167:0/1209499057 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:12.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.118+0000 7f602e200640 1 -- 172.21.15.167:0/1209499057 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55faa9e8fb50 msgr2=0x55faa9f3d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:12.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.118+0000 7f602e200640 1 --2- 172.21.15.167:0/1209499057 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55faa9e8fb50 0x55faa9f3d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:12.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.118+0000 7f602e200640 1 -- 172.21.15.167:0/1209499057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55faa9e90a10 msgr2=0x55faa9eba570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:12.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.118+0000 7f602e200640 1 --2- 172.21.15.167:0/1209499057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55faa9e90a10 0x55faa9eba570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:12.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.118+0000 7f602e200640 1 -- 172.21.15.167:0/1209499057 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55faa9f544a0 con 0x55faa9eae4e0 2024-05-11T12:14:12.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.119+0000 7f602e200640 1 --2- 172.21.15.167:0/1209499057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa9eae4e0 0x55faa9eb11a0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f602005fad0 tx=0x7f602005fa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d859b38270e90d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:12.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.119+0000 7f600aa00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-11T12:14:12.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.119+0000 7f600aa00640 1 -- 172.21.15.167:0/1209499057 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602005fb40 con 0x55faa9eae4e0 2024-05-11T12:14:12.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.119+0000 7f600aa00640 1 -- 172.21.15.167:0/1209499057 <== mon.0 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7f602005eda0 con 0x55faa9eae4e0 2024-05-11T12:14:12.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.119+0000 7f6033021740 1 -- 172.21.15.167:0/1209499057 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f601c016630 con 0x55faa9eae4e0 2024-05-11T12:14:12.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.119+0000 7f600aa00640 1 -- 172.21.15.167:0/1209499057 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f6020063e40 con 0x55faa9eae4e0 2024-05-11T12:14:12.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.119+0000 7f6033021740 10 client.6435 fetch_fsmap learned FSMap version 53 2024-05-11T12:14:12.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.119+0000 7f6033021740 1 -- 172.21.15.167:0/1209499057 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x55faa9eaec60 con 0x55faa9eae4e0 2024-05-11T12:14:12.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.119+0000 7f600aa00640 1 -- 172.21.15.167:0/1209499057 <== mon.0 v2:172.21.15.167:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f601c016630 con 0x55faa9eae4e0 2024-05-11T12:14:12.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.119+0000 7f600aa00640 1 -- 172.21.15.167:0/1209499057 <== mon.0 v2:172.21.15.167:3300/0 5 ==== fsmap(e 53) ==== 3494+0+0 (secure 0 0 0) 0x7f602005b4b0 con 0x55faa9eae4e0 2024-05-11T12:14:12.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.119+0000 7f6033021740 10 client.6435 fetch_fsmap finished waiting for FSMap version 53 2024-05-11T12:14:12.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.120+0000 7f6033021740 10 client.6435 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-11T12:14:12.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.120+0000 7f6033021740 20 client.6435 populate_metadata read hostname 'smithi167' 2024-05-11T12:14:12.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.120+0000 7f6033021740 1 --2- 172.21.15.167:0/1209499057 >> [v2:172.21.15.167:6835/3484070468,v1:172.21.15.167:6836/3484070468] conn(0x55faa9f58570 0x55faa9e8a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:12.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.120+0000 7f6033021740 4 client.6435 mds_command: new command op to 6300 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-11T12:14:12.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.120+0000 7f6033021740 1 -- 172.21.15.167:0/1209499057 --> [v2:172.21.15.167:6835/3484070468,v1:172.21.15.167:6836/3484070468] -- command(tid 0: {"prefix": "status"}) -- 0x55faa9dfc210 con 0x55faa9f58570 2024-05-11T12:14:12.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.120+0000 7f602d800640 1 --2- 172.21.15.167:0/1209499057 >> [v2:172.21.15.167:6835/3484070468,v1:172.21.15.167:6836/3484070468] conn(0x55faa9f58570 0x55faa9e8a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:12.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.120+0000 7f602d800640 1 --2- 172.21.15.167:0/1209499057 >> [v2:172.21.15.167:6835/3484070468,v1:172.21.15.167:6836/3484070468] conn(0x55faa9f58570 0x55faa9e8a600 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b5e1c47f1ae58b8 server_cookie=6a215049b9b886da in_seq=0 out_seq=0 2024-05-11T12:14:12.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.120+0000 7f600aa00640 10 client.6435 ms_handle_connect on v2:172.21.15.167:6835/3484070468 2024-05-11T12:14:12.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.121+0000 7f600aa00640 1 -- 172.21.15.167:0/1209499057 <== mds.0 v2:172.21.15.167:6835/3484070468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55faa9dfc210 con 0x55faa9f58570 2024-05-11T12:14:12.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.121+0000 7f600aa00640 10 client.6435 handle_command_reply: tid=0 2024-05-11T12:14:12.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.121+0000 7f6033021740 1 client.6435 shutdown 2024-05-11T12:14:12.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.121+0000 7f6033021740 1 -- 172.21.15.167:0/1209499057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa9eae4e0 msgr2=0x55faa9eb11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:12.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.121+0000 7f6033021740 1 --2- 172.21.15.167:0/1209499057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa9eae4e0 0x55faa9eb11a0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f602005fad0 tx=0x7f602005fa20 comp rx=0 tx=0).stop 2024-05-11T12:14:12.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.121+0000 7f6033021740 1 -- 172.21.15.167:0/1209499057 shutdown_connections 2024-05-11T12:14:12.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.121+0000 7f6033021740 1 --2- 172.21.15.167:0/1209499057 >> [v2:172.21.15.167:6835/3484070468,v1:172.21.15.167:6836/3484070468] conn(0x55faa9f58570 0x55faa9e8a600 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:12.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.121+0000 7f6033021740 1 --2- 172.21.15.167:0/1209499057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55faa9eae4e0 0x55faa9eb11a0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:12.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.121+0000 7f6033021740 1 --2- 172.21.15.167:0/1209499057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55faa9e90a10 0x55faa9eba570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:12.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.121+0000 7f6033021740 1 --2- 172.21.15.167:0/1209499057 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55faa9e8fb50 0x55faa9f3d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:12.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.121+0000 7f600aa00640 0 client.6435 ms_handle_reset on v2:172.21.15.167:6835/3484070468 2024-05-11T12:14:12.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.121+0000 7f6033021740 1 -- 172.21.15.167:0/1209499057 >> 172.21.15.167:0/1209499057 conn(0x55faa9e8b8a0 msgr2=0x55faa9eafd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:12.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.121+0000 7f6033021740 1 -- 172.21.15.167:0/1209499057 shutdown_connections 2024-05-11T12:14:12.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.121+0000 7f6033021740 1 -- 172.21.15.167:0/1209499057 wait complete. 2024-05-11T12:14:12.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.121+0000 7f6033021740 20 client.6435 trim_cache size 0 max 16384 2024-05-11T12:14:12.126 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T12:14:12.126 INFO:teuthology.orchestra.run.smithi167.stdout: "cluster_fsid": "5f32d344-7289-4c27-881b-1533419ab678", 2024-05-11T12:14:12.126 INFO:teuthology.orchestra.run.smithi167.stdout: "whoami": 0, 2024-05-11T12:14:12.126 INFO:teuthology.orchestra.run.smithi167.stdout: "id": 6300, 2024-05-11T12:14:12.126 INFO:teuthology.orchestra.run.smithi167.stdout: "want_state": "up:active", 2024-05-11T12:14:12.126 INFO:teuthology.orchestra.run.smithi167.stdout: "state": "up:active", 2024-05-11T12:14:12.127 INFO:teuthology.orchestra.run.smithi167.stdout: "fs_name": "cephfs", 2024-05-11T12:14:12.127 INFO:teuthology.orchestra.run.smithi167.stdout: "rank_uptime": 6.7265443779999998, 2024-05-11T12:14:12.127 INFO:teuthology.orchestra.run.smithi167.stdout: "mdsmap_epoch": 53, 2024-05-11T12:14:12.127 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_epoch": 86, 2024-05-11T12:14:12.127 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_epoch_barrier": 86, 2024-05-11T12:14:12.127 INFO:teuthology.orchestra.run.smithi167.stdout: "uptime": 11.163322161 2024-05-11T12:14:12.127 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T12:14:12.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.122+0000 7f602ec00640 1 -- 172.21.15.167:0/1756414679 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f600c0762d0 msgr2=0x7f600c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:12.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.122+0000 7f602ec00640 1 --2- 172.21.15.167:0/1756414679 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f600c0762d0 0x7f600c078790 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f60280cef00 tx=0x7f601c008040 comp rx=0 tx=0).stop 2024-05-11T12:14:12.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.122+0000 7f602ec00640 1 -- 172.21.15.167:0/1756414679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60281625d0 msgr2=0x7f60280cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:12.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.122+0000 7f602ec00640 1 --2- 172.21.15.167:0/1756414679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60281625d0 0x7f60280cd5f0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f602000c430 tx=0x7f602000c8f0 comp rx=0 tx=0).stop 2024-05-11T12:14:12.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.122+0000 7f602ec00640 1 -- 172.21.15.167:0/1756414679 shutdown_connections 2024-05-11T12:14:12.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.122+0000 7f602ec00640 1 --2- 172.21.15.167:0/1756414679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60281625d0 0x7f60280cd5f0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:12.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.122+0000 7f602ec00640 1 --2- 172.21.15.167:0/1756414679 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f60280d3c00 0x7f60281f8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:12.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.122+0000 7f602ec00640 1 --2- 172.21.15.167:0/1756414679 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f600c0762d0 0x7f600c078790 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:12.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.122+0000 7f602ec00640 1 --2- 172.21.15.167:0/1756414679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60280b8390 0x7f602815db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:12.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.122+0000 7f602ec00640 1 -- 172.21.15.167:0/1756414679 >> 172.21.15.167:0/1756414679 conn(0x7f60280d50f0 msgr2=0x7f60280b4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:12.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.122+0000 7f602ec00640 1 -- 172.21.15.167:0/1756414679 shutdown_connections 2024-05-11T12:14:12.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.122+0000 7f602ec00640 1 -- 172.21.15.167:0/1756414679 wait complete. 2024-05-11T12:14:12.140 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-11T12:14:12.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.328+0000 7f67e2800640 1 Processor -- start 2024-05-11T12:14:12.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.328+0000 7f67e2800640 1 -- start start 2024-05-11T12:14:12.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.329+0000 7f67e2800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67dc14c8a0 0x7f67dc0cf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:12.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.329+0000 7f67e2800640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67dc0cfc10 con 0x7f67dc06db70 2024-05-11T12:14:12.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.329+0000 7f67e2800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67dc0cfef0 con 0x7f67dc0cf700 2024-05-11T12:14:12.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.329+0000 7f67e2800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f67dc0d01d0 con 0x7f67dc14c8a0 2024-05-11T12:14:12.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.330+0000 7f67dbe00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f67dc06db70 0x7f67dc0b47d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:46468/0 (socket says 172.21.15.167:46468) 2024-05-11T12:14:12.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.330+0000 7f67dbe00640 1 -- 172.21.15.167:0/1038566367 learned_addr learned my addr 172.21.15.167:0/1038566367 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:14:12.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.330+0000 7f67db400640 1 --2- 172.21.15.167:0/1038566367 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67dc14c8a0 0x7f67dc0cf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:12.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.331+0000 7f67daa00640 1 -- 172.21.15.167:0/1038566367 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3705257608 0 0) 0x7f67dc0cfc10 con 0x7f67dc06db70 2024-05-11T12:14:12.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.331+0000 7f67daa00640 1 -- 172.21.15.167:0/1038566367 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f67c0003520 con 0x7f67dc06db70 2024-05-11T12:14:12.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.331+0000 7f67daa00640 1 -- 172.21.15.167:0/1038566367 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 187440202 0 0) 0x7f67dc0cfef0 con 0x7f67dc0cf700 2024-05-11T12:14:12.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.331+0000 7f67daa00640 1 -- 172.21.15.167:0/1038566367 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f67dc0cfc10 con 0x7f67dc0cf700 2024-05-11T12:14:12.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.331+0000 7f67daa00640 1 -- 172.21.15.167:0/1038566367 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2358084108 0 0) 0x7f67c0003520 con 0x7f67dc06db70 2024-05-11T12:14:12.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.331+0000 7f67daa00640 1 -- 172.21.15.167:0/1038566367 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67dc14c8a0 msgr2=0x7f67dc0cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:12.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.331+0000 7f67daa00640 1 --2- 172.21.15.167:0/1038566367 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67dc14c8a0 0x7f67dc0cf1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:12.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.331+0000 7f67daa00640 1 -- 172.21.15.167:0/1038566367 >> v1:172.21.15.204:6789/0 conn(0x7f67dc0cf700 legacy=0x7f67dc06d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:12.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.331+0000 7f67daa00640 1 -- 172.21.15.167:0/1038566367 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67dc0b42b0 con 0x7f67dc06db70 2024-05-11T12:14:12.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.331+0000 7f67daa00640 1 -- 172.21.15.167:0/1038566367 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f67d4001a70 con 0x7f67dc06db70 2024-05-11T12:14:12.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.331+0000 7f67daa00640 1 -- 172.21.15.167:0/1038566367 >> v1:172.21.15.167:6789/0 conn(0x7f67dc06db70 legacy=0x7f67dc0b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:12.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.331+0000 7f67daa00640 1 --2- 172.21.15.167:0/1038566367 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67dc14c8a0 0x7f67c0003c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:12.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.331+0000 7f67daa00640 1 --2- 172.21.15.167:0/1038566367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67c0004210 0x7f67c0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:12.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.331+0000 7f67daa00640 1 --2- 172.21.15.167:0/1038566367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67c0004d90 0x7f67c0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:12.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.331+0000 7f67daa00640 1 -- 172.21.15.167:0/1038566367 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc0d01d0 con 0x7f67c0004d90 2024-05-11T12:14:12.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.331+0000 7f67daa00640 1 -- 172.21.15.167:0/1038566367 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f67c000bbd0 con 0x7f67c0004210 2024-05-11T12:14:12.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.331+0000 7f67daa00640 1 -- 172.21.15.167:0/1038566367 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f67c000bdd0 con 0x7f67dc14c8a0 2024-05-11T12:14:12.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.332+0000 7f67dbe00640 1 --2- 172.21.15.167:0/1038566367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67c0004d90 0x7f67c0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:12.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.332+0000 7f67db400640 1 --2- 172.21.15.167:0/1038566367 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67dc14c8a0 0x7f67c0003c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:12.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.332+0000 7f67dbe00640 1 -- 172.21.15.167:0/1038566367 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67dc14c8a0 msgr2=0x7f67c0003c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:12.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.332+0000 7f67dbe00640 1 --2- 172.21.15.167:0/1038566367 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67dc14c8a0 0x7f67c0003c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:12.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.332+0000 7f67dbe00640 1 -- 172.21.15.167:0/1038566367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67c0004210 msgr2=0x7f67c0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:12.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.332+0000 7f67d3e00640 1 --2- 172.21.15.167:0/1038566367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67c0004210 0x7f67c0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:12.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.332+0000 7f67dbe00640 1 --2- 172.21.15.167:0/1038566367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67c0004210 0x7f67c0004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:12.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.332+0000 7f67dbe00640 1 -- 172.21.15.167:0/1038566367 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67c0003520 con 0x7f67c0004d90 2024-05-11T12:14:12.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.333+0000 7f67dbe00640 1 --2- 172.21.15.167:0/1038566367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67c0004d90 0x7f67c0005210 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f67dc0b0ae0 tx=0x7f67c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7313a1764f83e84 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:12.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.333+0000 7f67daa00640 1 -- 172.21.15.167:0/1038566367 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d400a3b0 con 0x7f67c0004d90 2024-05-11T12:14:12.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.333+0000 7f67daa00640 1 -- 172.21.15.167:0/1038566367 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67d402f070 con 0x7f67c0004d90 2024-05-11T12:14:12.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.333+0000 7f67daa00640 1 -- 172.21.15.167:0/1038566367 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d402bb00 con 0x7f67c0004d90 2024-05-11T12:14:12.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.334+0000 7f67e2800640 1 -- 172.21.15.167:0/1038566367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67c0004d90 msgr2=0x7f67c0005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:12.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.334+0000 7f67e2800640 1 --2- 172.21.15.167:0/1038566367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67c0004d90 0x7f67c0005210 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f67dc0b0ae0 tx=0x7f67c0005830 comp rx=0 tx=0).stop 2024-05-11T12:14:12.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.334+0000 7f67dbe00640 1 -- 172.21.15.167:0/1038566367 reap_dead start 2024-05-11T12:14:12.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.334+0000 7f67e2800640 1 -- 172.21.15.167:0/1038566367 shutdown_connections 2024-05-11T12:14:12.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.334+0000 7f67e2800640 1 -- 172.21.15.167:0/1038566367 >> 172.21.15.167:0/1038566367 conn(0x7f67dc0cd8f0 msgr2=0x7f67dc0cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:12.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.334+0000 7f67e2800640 1 -- 172.21.15.167:0/1038566367 shutdown_connections 2024-05-11T12:14:12.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.334+0000 7f67e2800640 1 -- 172.21.15.167:0/1038566367 wait complete. 2024-05-11T12:14:12.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.336+0000 7f67e2800640 1 Processor -- start 2024-05-11T12:14:12.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.336+0000 7f67e2800640 1 -- start start 2024-05-11T12:14:12.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.337+0000 7f67e2800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67c0004d90 0x7f67dc0ae560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:12.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.337+0000 7f67e2800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67dc1ef040 0x7f67dc1ef4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:12.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.337+0000 7f67e2800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67dc13ae20 0x7f67dc13b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:12.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.337+0000 7f67e2800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc06feb0 con 0x7f67dc13ae20 2024-05-11T12:14:12.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.337+0000 7f67e2800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc06fd30 con 0x7f67dc1ef040 2024-05-11T12:14:12.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.337+0000 7f67e2800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f67dc14cc80 con 0x7f67c0004d90 2024-05-11T12:14:12.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.337+0000 7f67dbe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67dc13ae20 0x7f67dc13b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:12.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.337+0000 7f67dbe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67dc13ae20 0x7f67dc13b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35634/0 (socket says 172.21.15.167:35634) 2024-05-11T12:14:12.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.337+0000 7f67dbe00640 1 -- 172.21.15.167:0/2198766193 learned_addr learned my addr 172.21.15.167:0/2198766193 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:12.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.338+0000 7f67db400640 1 --2- 172.21.15.167:0/2198766193 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67c0004d90 0x7f67dc0ae560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:12.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.338+0000 7f67dbe00640 1 -- 172.21.15.167:0/2198766193 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67c0004d90 msgr2=0x7f67dc0ae560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:12.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.338+0000 7f67dbe00640 1 --2- 172.21.15.167:0/2198766193 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67c0004d90 0x7f67dc0ae560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:12.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.338+0000 7f67dbe00640 1 -- 172.21.15.167:0/2198766193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67dc1ef040 msgr2=0x7f67dc1ef4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:12.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.338+0000 7f67dbe00640 1 --2- 172.21.15.167:0/2198766193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67dc1ef040 0x7f67dc1ef4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:12.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.338+0000 7f67dbe00640 1 -- 172.21.15.167:0/2198766193 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67dc13bb30 con 0x7f67dc13ae20 2024-05-11T12:14:12.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.338+0000 7f67dbe00640 1 --2- 172.21.15.167:0/2198766193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67dc13ae20 0x7f67dc13b290 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f67d400a850 tx=0x7f67dc0d03a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:12.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.338+0000 7f67d3400640 1 -- 172.21.15.167:0/2198766193 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d4044070 con 0x7f67dc13ae20 2024-05-11T12:14:12.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.338+0000 7f67d3400640 1 -- 172.21.15.167:0/2198766193 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67d402cd50 con 0x7f67dc13ae20 2024-05-11T12:14:12.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.338+0000 7f67e2800640 1 -- 172.21.15.167:0/2198766193 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67dc138f30 con 0x7f67dc13ae20 2024-05-11T12:14:12.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.338+0000 7f67d3400640 1 -- 172.21.15.167:0/2198766193 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d400a3b0 con 0x7f67dc13ae20 2024-05-11T12:14:12.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.339+0000 7f67e2800640 1 -- 172.21.15.167:0/2198766193 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67dc13b7d0 con 0x7f67dc13ae20 2024-05-11T12:14:12.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.340+0000 7f67e2800640 1 -- 172.21.15.167:0/2198766193 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67dc0cf4f0 con 0x7f67dc13ae20 2024-05-11T12:14:12.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.341+0000 7f67d3400640 1 -- 172.21.15.167:0/2198766193 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f67d4047070 con 0x7f67dc13ae20 2024-05-11T12:14:12.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.342+0000 7f67d3400640 1 --2- 172.21.15.167:0/2198766193 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f67b80762d0 0x7f67b8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:12.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.342+0000 7f67d3400640 1 -- 172.21.15.167:0/2198766193 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7f67d40b7620 con 0x7f67dc13ae20 2024-05-11T12:14:12.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.342+0000 7f67db400640 1 --2- 172.21.15.167:0/2198766193 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f67b80762d0 0x7f67b8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:12.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.342+0000 7f67db400640 1 --2- 172.21.15.167:0/2198766193 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f67b80762d0 0x7f67b8078790 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f67cc009950 tx=0x7f67cc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:12.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.346+0000 7f67d3400640 1 -- 172.21.15.167:0/2198766193 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f67dc0cf4f0 con 0x7f67dc13ae20 2024-05-11T12:14:12.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.603+0000 7f67e2800640 1 -- 172.21.15.167:0/2198766193 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f67dc069590 con 0x7f67dc13ae20 2024-05-11T12:14:12.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.603+0000 7f67d3400640 1 -- 172.21.15.167:0/2198766193 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v53) ==== 53+0+73 (secure 0 0 0) 0x7f67d4080d50 con 0x7f67dc13ae20 2024-05-11T12:14:12.604 INFO:teuthology.orchestra.run.smithi167.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-11T12:14:12.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.606+0000 7f67e2800640 1 -- 172.21.15.167:0/2198766193 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f67b80762d0 msgr2=0x7f67b8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:12.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.606+0000 7f67e2800640 1 --2- 172.21.15.167:0/2198766193 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f67b80762d0 0x7f67b8078790 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f67cc009950 tx=0x7f67cc008040 comp rx=0 tx=0).stop 2024-05-11T12:14:12.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.606+0000 7f67e2800640 1 -- 172.21.15.167:0/2198766193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67dc13ae20 msgr2=0x7f67dc13b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:12.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.606+0000 7f67e2800640 1 --2- 172.21.15.167:0/2198766193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67dc13ae20 0x7f67dc13b290 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f67d400a850 tx=0x7f67dc0d03a0 comp rx=0 tx=0).stop 2024-05-11T12:14:12.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.606+0000 7f67e2800640 1 -- 172.21.15.167:0/2198766193 shutdown_connections 2024-05-11T12:14:12.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.606+0000 7f67e2800640 1 --2- 172.21.15.167:0/2198766193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67dc13ae20 0x7f67dc13b290 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:12.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.606+0000 7f67e2800640 1 --2- 172.21.15.167:0/2198766193 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f67b80762d0 0x7f67b8078790 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:12.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.606+0000 7f67e2800640 1 --2- 172.21.15.167:0/2198766193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67dc1ef040 0x7f67dc1ef4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:12.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.606+0000 7f67e2800640 1 --2- 172.21.15.167:0/2198766193 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67c0004d90 0x7f67dc0ae560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:12.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.606+0000 7f67e2800640 1 -- 172.21.15.167:0/2198766193 >> 172.21.15.167:0/2198766193 conn(0x7f67dc0cd8f0 msgr2=0x7f67dc0b1840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:12.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.606+0000 7f67e2800640 1 -- 172.21.15.167:0/2198766193 shutdown_connections 2024-05-11T12:14:12.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12.607+0000 7f67e2800640 1 -- 172.21.15.167:0/2198766193 wait complete. 2024-05-11T12:14:12.625 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-11T12:14:12.625 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-11T12:14:12.625 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi167.front.sepia.ceph.com 2024-05-11T12:14:12.625 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-11T12:14:12.625 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-11T12:14:12.625 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-11T12:14:12.625 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-11T12:14:12.625 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-11T12:14:12.626 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-11T12:14:12.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:12.626 DEBUG:teuthology.orchestra.run.smithi167:> ip addr 2024-05-11T12:14:12.673 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-11T12:14:12.673 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-11T12:14:12.673 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2024-05-11T12:14:12.673 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:14:12.674 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2024-05-11T12:14:12.674 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:14:12.674 INFO:teuthology.orchestra.run.smithi167.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:14:12.674 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e6 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:14:12.674 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f0 2024-05-11T12:14:12.674 INFO:teuthology.orchestra.run.smithi167.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:14:12.674 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e7 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:14:12.674 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f1 2024-05-11T12:14:12.674 INFO:teuthology.orchestra.run.smithi167.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:14:12.674 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:08 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:14:12.674 INFO:teuthology.orchestra.run.smithi167.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-11T12:14:12.674 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:09 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:14:12.675 INFO:teuthology.orchestra.run.smithi167.stdout: inet 172.21.15.167/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-11T12:14:12.675 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft 42416sec preferred_lft 42416sec 2024-05-11T12:14:12.675 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::ec4:7aff:fe88:7309/64 scope link 2024-05-11T12:14:12.675 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:14:12.675 INFO:teuthology.orchestra.run.smithi167.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-11T12:14:12.675 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 1a:a8:8d:b3:f4:32 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:14:12.675 INFO:teuthology.orchestra.run.smithi167.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-11T12:14:12.675 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:14:12.675 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::b017:48ff:fea0:7183/64 scope link 2024-05-11T12:14:12.675 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:14:12.675 INFO:teuthology.orchestra.run.smithi167.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-11T12:14:12.675 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 1a:a8:8d:b3:f4:32 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-11T12:14:12.675 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::18a8:8dff:feb3:f432/64 scope link 2024-05-11T12:14:12.676 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:14:12.676 INFO:teuthology.orchestra.run.smithi167.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-11T12:14:12.676 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 7e:79:1f:cb:d2:83 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-11T12:14:12.676 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::7c79:1fff:fecb:d283/64 scope link 2024-05-11T12:14:12.676 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:14:12.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:12.677 DEBUG:teuthology.orchestra.run.smithi167:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-11T12:14:12.793 INFO:teuthology.orchestra.run.smithi167.stdout:1 2024-05-11T12:14:12.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:12.797 DEBUG:teuthology.orchestra.run.smithi167:> route 2024-05-11T12:14:12.848 INFO:teuthology.orchestra.run.smithi167.stdout:Kernel IP routing table 2024-05-11T12:14:12.848 INFO:teuthology.orchestra.run.smithi167.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-11T12:14:12.848 INFO:teuthology.orchestra.run.smithi167.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-11T12:14:12.848 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-11T12:14:12.849 INFO:teuthology.orchestra.run.smithi167.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-11T12:14:12.849 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T12:14:12.849 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-05-11T12:14:12.849 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-11T12:14:12.849 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-11T12:14:12.849 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-11T12:14:12.849 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-05-11T12:14:12.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T12:14:13.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:14:13.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:13.056 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2024-05-11T12:14:13.103 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-11T12:14:13.103 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-11T12:14:13.104 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-11T12:14:13.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:13.104 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set brx.0 up 2024-05-11T12:14:13.185 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T12:14:13.186 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-11T12:14:13.239 INFO:teuthology.orchestra.run.smithi167.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-11T12:14:13.240 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T12:14:13.240 DEBUG:teuthology.orchestra.run.smithi167:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-11T12:14:13.290 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-05-11T12:14:13.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:13.291 DEBUG:teuthology.orchestra.run.smithi167:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:14:13.424 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:14:13.424 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-11T12:14:13.458 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:14:13.458 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-11T12:14:13.458 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.167:3300,172.21.15.204:3300,172.21.15.204:3301" 2024-05-11T12:14:13.458 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: trying mount with new device syntax: 0@5f32d344-7289-4c27-881b-1533419ab678.cephfs=/ 2024-05-11T12:14:13.458 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.167:3300/172.21.15.204:3300/172.21.15.204:3301" will pass to kernel 2024-05-11T12:14:13.460 INFO:teuthology.orchestra.run.smithi167.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-11T12:14:13.460 INFO:teuthology.orchestra.run.smithi167.stdout: You just mounted a file system that supports labels which does not 2024-05-11T12:14:13.460 INFO:teuthology.orchestra.run.smithi167.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-11T12:14:13.460 INFO:teuthology.orchestra.run.smithi167.stdout: applications will generate AVC messages and not be allowed access to 2024-05-11T12:14:13.460 INFO:teuthology.orchestra.run.smithi167.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-11T12:14:13.464 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-11T12:14:13.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:13.464 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-11T12:14:13.532 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-11T12:14:13.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-11T12:14:13.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:14:13.658 INFO:teuthology.orchestra.run.smithi167.stdout:client5230 2024-05-11T12:14:13.659 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:14:13.659 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:14:13.706 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-11T12:14:13.950 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.950+0000 7f706c200640 1 Processor -- start 2024-05-11T12:14:13.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.950+0000 7f706c200640 1 -- start start 2024-05-11T12:14:13.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.951+0000 7f706c200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f70640cd610 0x7f7064163be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:13.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.951+0000 7f706c200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f70640b7b90 0x7f70640b8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:13.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.951+0000 7f706c200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70640b6300 con 0x7f70640cd9f0 2024-05-11T12:14:13.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.951+0000 7f706c200640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f70640b65e0 con 0x7f70640b7b90 2024-05-11T12:14:13.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.951+0000 7f706c200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f7064163100 con 0x7f70640cd610 2024-05-11T12:14:13.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.952+0000 7f706a200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f70640b7b90 0x7f70640b8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:13.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.952+0000 7f706a200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f70640b7b90 0x7f70640b8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35648/0 (socket says 172.21.15.167:35648) 2024-05-11T12:14:13.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.952+0000 7f706a200640 1 -- 172.21.15.167:0/3746313047 learned_addr learned my addr 172.21.15.167:0/3746313047 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:13.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.952+0000 7f706a200640 1 -- 172.21.15.167:0/3746313047 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f70640cd610 msgr2=0x7f7064163be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:13.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.952+0000 7f7069800640 1 --2- 172.21.15.167:0/3746313047 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f70640cd610 0x7f7064163be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:13.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.952+0000 7f706a200640 1 --2- 172.21.15.167:0/3746313047 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f70640cd610 0x7f7064163be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:13.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.952+0000 7f706a200640 1 -- 172.21.15.167:0/3746313047 >> v1:172.21.15.204:6789/0 conn(0x7f70640cd9f0 legacy=0x7f7064160230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:13.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.952+0000 7f706a200640 1 -- 172.21.15.167:0/3746313047 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70641633e0 con 0x7f70640b7b90 2024-05-11T12:14:13.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.953+0000 7f706a200640 1 --2- 172.21.15.167:0/3746313047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70640b7b90 0x7f70640b8020 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f705c009e50 tx=0x7f705c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38915326b78fbb99 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:13.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.954+0000 7f7063e00640 1 -- 172.21.15.167:0/3746313047 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f705c036070 con 0x7f70640b7b90 2024-05-11T12:14:13.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.954+0000 7f7063e00640 1 -- 172.21.15.167:0/3746313047 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f705c031600 con 0x7f70640b7b90 2024-05-11T12:14:13.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.954+0000 7f7063e00640 1 -- 172.21.15.167:0/3746313047 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f705c0319d0 con 0x7f70640b7b90 2024-05-11T12:14:13.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.954+0000 7f706c200640 1 -- 172.21.15.167:0/3746313047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70640b7b90 msgr2=0x7f70640b8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:13.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.954+0000 7f706c200640 1 --2- 172.21.15.167:0/3746313047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70640b7b90 0x7f70640b8020 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f705c009e50 tx=0x7f705c02a680 comp rx=0 tx=0).stop 2024-05-11T12:14:13.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.954+0000 7f706c200640 1 -- 172.21.15.167:0/3746313047 shutdown_connections 2024-05-11T12:14:13.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.954+0000 7f706c200640 1 --2- 172.21.15.167:0/3746313047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70640b7b90 0x7f70640b8020 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:13.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.954+0000 7f706c200640 1 --2- 172.21.15.167:0/3746313047 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f70640cd610 0x7f7064163be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:13.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.954+0000 7f706c200640 1 -- 172.21.15.167:0/3746313047 >> 172.21.15.167:0/3746313047 conn(0x7f7064072730 msgr2=0x7f70640d2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:13.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.954+0000 7f706c200640 1 -- 172.21.15.167:0/3746313047 shutdown_connections 2024-05-11T12:14:13.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.954+0000 7f706c200640 1 -- 172.21.15.167:0/3746313047 wait complete. 2024-05-11T12:14:13.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.956+0000 7f706c200640 1 Processor -- start 2024-05-11T12:14:13.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.956+0000 7f706c200640 1 -- start start 2024-05-11T12:14:13.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.957+0000 7f706c200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f70640b7b90 0x7f7064157020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:13.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.957+0000 7f706c200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70640cd610 0x7f7064149e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:13.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.957+0000 7f706c200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70640cd9f0 0x7f706414a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:13.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.957+0000 7f706c200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f70640d2a80 con 0x7f70640cd610 2024-05-11T12:14:13.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.957+0000 7f706c200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7064160940 con 0x7f70640cd9f0 2024-05-11T12:14:13.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.957+0000 7f706c200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f70640cde50 con 0x7f70640b7b90 2024-05-11T12:14:13.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.958+0000 7f7068e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70640cd610 0x7f7064149e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:13.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.958+0000 7f7068e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70640cd610 0x7f7064149e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35652/0 (socket says 172.21.15.167:35652) 2024-05-11T12:14:13.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.958+0000 7f7068e00640 1 -- 172.21.15.167:0/3934633505 learned_addr learned my addr 172.21.15.167:0/3934633505 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:13.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.958+0000 7f7069800640 1 --2- 172.21.15.167:0/3934633505 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f70640b7b90 0x7f7064157020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:13.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.958+0000 7f706a200640 1 --2- 172.21.15.167:0/3934633505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70640cd9f0 0x7f706414a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:13.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.958+0000 7f7068e00640 1 -- 172.21.15.167:0/3934633505 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f70640b7b90 msgr2=0x7f7064157020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:13.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.958+0000 7f7068e00640 1 --2- 172.21.15.167:0/3934633505 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f70640b7b90 0x7f7064157020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:13.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.958+0000 7f7068e00640 1 -- 172.21.15.167:0/3934633505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70640cd9f0 msgr2=0x7f706414a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:13.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.958+0000 7f7068e00640 1 --2- 172.21.15.167:0/3934633505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70640cd9f0 0x7f706414a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:13.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.958+0000 7f7068e00640 1 -- 172.21.15.167:0/3934633505 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70640cb060 con 0x7f70640cd610 2024-05-11T12:14:13.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.958+0000 7f706a200640 1 --2- 172.21.15.167:0/3934633505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70640cd9f0 0x7f706414a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:13.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.959+0000 7f7068e00640 1 --2- 172.21.15.167:0/3934633505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70640cd610 0x7f7064149e30 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f705000b3f0 tx=0x7f705000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:13.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.959+0000 7f7061600640 1 -- 172.21.15.167:0/3934633505 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7050015020 con 0x7f70640cd610 2024-05-11T12:14:13.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.959+0000 7f7061600640 1 -- 172.21.15.167:0/3934633505 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7050011830 con 0x7f70640cd610 2024-05-11T12:14:13.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.959+0000 7f7061600640 1 -- 172.21.15.167:0/3934633505 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7050011c20 con 0x7f70640cd610 2024-05-11T12:14:13.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.959+0000 7f706c200640 1 -- 172.21.15.167:0/3934633505 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70640ca450 con 0x7f70640cd610 2024-05-11T12:14:13.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.959+0000 7f706c200640 1 -- 172.21.15.167:0/3934633505 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70640caa10 con 0x7f70640cd610 2024-05-11T12:14:13.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.960+0000 7f706c200640 1 -- 172.21.15.167:0/3934633505 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7064069590 con 0x7f70640cd610 2024-05-11T12:14:13.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.962+0000 7f7061600640 1 -- 172.21.15.167:0/3934633505 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7050018020 con 0x7f70640cd610 2024-05-11T12:14:13.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.962+0000 7f7061600640 1 --2- 172.21.15.167:0/3934633505 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f70440762d0 0x7f7044078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:13.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.963+0000 7f7061600640 1 -- 172.21.15.167:0/3934633505 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7f7050097410 con 0x7f70640cd610 2024-05-11T12:14:13.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.963+0000 7f7069800640 1 --2- 172.21.15.167:0/3934633505 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f70440762d0 0x7f7044078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:13.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.963+0000 7f7069800640 1 --2- 172.21.15.167:0/3934633505 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f70440762d0 0x7f7044078790 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f705800a970 tx=0x7f7058008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:13.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:13.967+0000 7f7061600640 1 -- 172.21.15.167:0/3934633505 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f705001f030 con 0x7f70640cd610 2024-05-11T12:14:14.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:14.214+0000 7f706c200640 1 -- 172.21.15.167:0/3934633505 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f70640cb290 con 0x7f70640cd610 2024-05-11T12:14:14.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:14.214+0000 7f7061600640 1 -- 172.21.15.167:0/3934633505 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v53) ==== 53+0+73 (secure 0 0 0) 0x7f7050060be0 con 0x7f70640cd610 2024-05-11T12:14:14.215 INFO:teuthology.orchestra.run.smithi167.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-11T12:14:14.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:14.217+0000 7f706c200640 1 -- 172.21.15.167:0/3934633505 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f70440762d0 msgr2=0x7f7044078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:14.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:14.217+0000 7f706c200640 1 --2- 172.21.15.167:0/3934633505 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f70440762d0 0x7f7044078790 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f705800a970 tx=0x7f7058008040 comp rx=0 tx=0).stop 2024-05-11T12:14:14.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:14.217+0000 7f706c200640 1 -- 172.21.15.167:0/3934633505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70640cd610 msgr2=0x7f7064149e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:14.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:14.217+0000 7f706c200640 1 --2- 172.21.15.167:0/3934633505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70640cd610 0x7f7064149e30 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f705000b3f0 tx=0x7f705000b8b0 comp rx=0 tx=0).stop 2024-05-11T12:14:14.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:14.217+0000 7f706c200640 1 -- 172.21.15.167:0/3934633505 shutdown_connections 2024-05-11T12:14:14.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:14.217+0000 7f706c200640 1 --2- 172.21.15.167:0/3934633505 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f70440762d0 0x7f7044078790 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:14.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:14.217+0000 7f706c200640 1 --2- 172.21.15.167:0/3934633505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70640cd9f0 0x7f706414a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:14.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:14.217+0000 7f706c200640 1 --2- 172.21.15.167:0/3934633505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70640cd610 0x7f7064149e30 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:14.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:14.217+0000 7f706c200640 1 --2- 172.21.15.167:0/3934633505 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f70640b7b90 0x7f7064157020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:14.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:14.217+0000 7f706c200640 1 -- 172.21.15.167:0/3934633505 >> 172.21.15.167:0/3934633505 conn(0x7f7064072730 msgr2=0x7f706415dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:14.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:14.217+0000 7f706c200640 1 -- 172.21.15.167:0/3934633505 shutdown_connections 2024-05-11T12:14:14.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:14.218+0000 7f706c200640 1 -- 172.21.15.167:0/3934633505 wait complete. 2024-05-11T12:14:14.236 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-11T12:14:14.236 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-11T12:14:14.236 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi204.front.sepia.ceph.com 2024-05-11T12:14:14.236 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-11T12:14:14.236 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-11T12:14:14.236 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-11T12:14:14.236 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-11T12:14:14.236 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-11T12:14:14.236 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-11T12:14:14.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:14.237 DEBUG:teuthology.orchestra.run.smithi204:> ip addr 2024-05-11T12:14:14.295 INFO:teuthology.orchestra.run.smithi204.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-11T12:14:14.295 INFO:teuthology.orchestra.run.smithi204.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-11T12:14:14.295 INFO:teuthology.orchestra.run.smithi204.stdout: inet 127.0.0.1/8 scope host lo 2024-05-11T12:14:14.295 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:14:14.295 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 ::1/128 scope host 2024-05-11T12:14:14.295 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:14:14.296 INFO:teuthology.orchestra.run.smithi204.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:14:14.296 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2a brd ff:ff:ff:ff:ff:ff 2024-05-11T12:14:14.296 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f0 2024-05-11T12:14:14.296 INFO:teuthology.orchestra.run.smithi204.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:14:14.296 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2b brd ff:ff:ff:ff:ff:ff 2024-05-11T12:14:14.296 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f1 2024-05-11T12:14:14.296 INFO:teuthology.orchestra.run.smithi204.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:14:14.296 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:aa brd ff:ff:ff:ff:ff:ff 2024-05-11T12:14:14.296 INFO:teuthology.orchestra.run.smithi204.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-11T12:14:14.296 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:ab brd ff:ff:ff:ff:ff:ff 2024-05-11T12:14:14.296 INFO:teuthology.orchestra.run.smithi204.stdout: inet 172.21.15.204/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-11T12:14:14.296 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft 42410sec preferred_lft 42410sec 2024-05-11T12:14:14.297 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::ec4:7aff:fe88:77ab/64 scope link 2024-05-11T12:14:14.297 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:14:14.297 INFO:teuthology.orchestra.run.smithi204.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-11T12:14:14.297 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether e2:9e:5b:47:45:5d brd ff:ff:ff:ff:ff:ff 2024-05-11T12:14:14.297 INFO:teuthology.orchestra.run.smithi204.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-11T12:14:14.297 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:14:14.297 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::a498:7aff:fe50:48c3/64 scope link 2024-05-11T12:14:14.297 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:14:14.297 INFO:teuthology.orchestra.run.smithi204.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-11T12:14:14.298 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether fa:0a:7f:e4:30:35 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-11T12:14:14.298 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::f80a:7fff:fee4:3035/64 scope link 2024-05-11T12:14:14.298 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:14:14.298 INFO:teuthology.orchestra.run.smithi204.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-11T12:14:14.298 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether e2:9e:5b:47:45:5d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-11T12:14:14.298 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::e09e:5bff:fe47:455d/64 scope link 2024-05-11T12:14:14.298 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:14:14.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:14.299 DEBUG:teuthology.orchestra.run.smithi204:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-11T12:14:14.378 INFO:teuthology.orchestra.run.smithi204.stdout:1 2024-05-11T12:14:14.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:14.382 DEBUG:teuthology.orchestra.run.smithi204:> route 2024-05-11T12:14:14.434 INFO:teuthology.orchestra.run.smithi204.stdout:Kernel IP routing table 2024-05-11T12:14:14.435 INFO:teuthology.orchestra.run.smithi204.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-11T12:14:14.435 INFO:teuthology.orchestra.run.smithi204.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-11T12:14:14.435 INFO:teuthology.orchestra.run.smithi204.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-11T12:14:14.435 INFO:teuthology.orchestra.run.smithi204.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-11T12:14:14.435 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T12:14:14.435 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-05-11T12:14:14.435 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-11T12:14:14.435 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-11T12:14:14.435 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-11T12:14:14.435 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-05-11T12:14:14.542 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:14:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T12:14:14.640 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:14:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:14:14.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:14.651 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list 2024-05-11T12:14:14.703 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-11T12:14:14.703 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-11T12:14:14.704 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-11T12:14:14.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:14.704 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link set brx.0 up 2024-05-11T12:14:14.790 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T12:14:14.790 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-11T12:14:14.845 INFO:teuthology.orchestra.run.smithi204.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-11T12:14:14.846 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T12:14:14.846 DEBUG:teuthology.orchestra.run.smithi204:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-11T12:14:14.896 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-05-11T12:14:14.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:14.896 DEBUG:teuthology.orchestra.run.smithi204:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:14:15.019 INFO:teuthology.orchestra.run.smithi204.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:14:15.020 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-11T12:14:15.056 INFO:teuthology.orchestra.run.smithi204.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:14:15.056 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-11T12:14:15.056 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: resolved to: "172.21.15.167:3300,172.21.15.204:3300,172.21.15.204:3301" 2024-05-11T12:14:15.056 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: trying mount with new device syntax: 1@5f32d344-7289-4c27-881b-1533419ab678.cephfs=/ 2024-05-11T12:14:15.056 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.167:3300/172.21.15.204:3300/172.21.15.204:3301" will pass to kernel 2024-05-11T12:14:15.058 INFO:teuthology.orchestra.run.smithi204.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-05-11T12:14:15.058 INFO:teuthology.orchestra.run.smithi204.stdout: You just mounted a file system that supports labels which does not 2024-05-11T12:14:15.058 INFO:teuthology.orchestra.run.smithi204.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-11T12:14:15.058 INFO:teuthology.orchestra.run.smithi204.stdout: applications will generate AVC messages and not be allowed access to 2024-05-11T12:14:15.058 INFO:teuthology.orchestra.run.smithi204.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-11T12:14:15.061 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-11T12:14:15.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:15.061 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-11T12:14:15.136 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-11T12:14:15.243 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:14:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-11T12:14:15.248 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:14:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:14:15.257 INFO:teuthology.orchestra.run.smithi204.stdout:client6450 2024-05-11T12:14:15.257 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:14:15.257 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:14:15.305 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:14:15.305 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:14:15.364 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:14:15.364 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:14:15.418 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-11T12:14:15.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.623+0000 7f3b0da00640 1 Processor -- start 2024-05-11T12:14:15.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.624+0000 7f3b0da00640 1 -- start start 2024-05-11T12:14:15.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.625+0000 7f3b0da00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3b08157940 0x7f3b0815e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:15.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.625+0000 7f3b0da00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b080ceab0 0x7f3b080702a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:15.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.625+0000 7f3b0da00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b0815e890 con 0x7f3b080d0da0 2024-05-11T12:14:15.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.625+0000 7f3b0da00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f3b080727e0 con 0x7f3b08157940 2024-05-11T12:14:15.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.625+0000 7f3b0da00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3b08072ac0 con 0x7f3b080ceab0 2024-05-11T12:14:15.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.625+0000 7f3b07e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3b08157940 0x7f3b0815e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:15.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.626+0000 7f3b07e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3b08157940 0x7f3b0815e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35660/0 (socket says 172.21.15.167:35660) 2024-05-11T12:14:15.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.626+0000 7f3b07e00640 1 -- 172.21.15.167:0/1936827671 learned_addr learned my addr 172.21.15.167:0/1936827671 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:15.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.626+0000 7f3b07400640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f3b080d0da0 0x7f3b080ce3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:46472/0 (socket says 172.21.15.167:46472) 2024-05-11T12:14:15.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.626+0000 7f3b0d000640 1 --2- 172.21.15.167:0/1936827671 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b080ceab0 0x7f3b080702a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:15.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.626+0000 7f3b07e00640 1 -- 172.21.15.167:0/1936827671 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b080ceab0 msgr2=0x7f3b080702a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:15.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.626+0000 7f3b07e00640 1 --2- 172.21.15.167:0/1936827671 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b080ceab0 0x7f3b080702a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:15.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.626+0000 7f3b07e00640 1 -- 172.21.15.167:0/1936827671 >> v1:172.21.15.167:6789/0 conn(0x7f3b080d0da0 legacy=0x7f3b080ce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:15.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.626+0000 7f3b06a00640 1 -- 172.21.15.167:0/1936827671 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3505659551 0 0) 0x7f3af0001110 con 0x7f3b080d0da0 2024-05-11T12:14:15.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.626+0000 7f3b07e00640 1 -- 172.21.15.167:0/1936827671 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b080cd8c0 con 0x7f3b08157940 2024-05-11T12:14:15.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.626+0000 7f3b0d000640 1 --2- 172.21.15.167:0/1936827671 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b080ceab0 0x7f3b080702a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:15.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.627+0000 7f3b07e00640 1 --2- 172.21.15.167:0/1936827671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b08157940 0x7f3b0815e350 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f3af8009e50 tx=0x7f3af802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cf6335e9b763ee3 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:15.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.628+0000 7f3b06a00640 1 -- 172.21.15.167:0/1936827671 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af8036070 con 0x7f3b08157940 2024-05-11T12:14:15.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.628+0000 7f3b06a00640 1 -- 172.21.15.167:0/1936827671 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3af8031600 con 0x7f3b08157940 2024-05-11T12:14:15.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.628+0000 7f3b06a00640 1 -- 172.21.15.167:0/1936827671 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af80319d0 con 0x7f3b08157940 2024-05-11T12:14:15.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.628+0000 7f3b0da00640 1 -- 172.21.15.167:0/1936827671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b08157940 msgr2=0x7f3b0815e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:15.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.628+0000 7f3b0da00640 1 --2- 172.21.15.167:0/1936827671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b08157940 0x7f3b0815e350 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f3af8009e50 tx=0x7f3af802a710 comp rx=0 tx=0).stop 2024-05-11T12:14:15.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.629+0000 7f3b0da00640 1 -- 172.21.15.167:0/1936827671 shutdown_connections 2024-05-11T12:14:15.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.629+0000 7f3b0da00640 1 --2- 172.21.15.167:0/1936827671 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b080ceab0 0x7f3b080702a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:15.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.629+0000 7f3b0da00640 1 --2- 172.21.15.167:0/1936827671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b08157940 0x7f3b0815e350 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:15.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.629+0000 7f3b0da00640 1 -- 172.21.15.167:0/1936827671 >> 172.21.15.167:0/1936827671 conn(0x7f3b080d2ea0 msgr2=0x7f3b080d32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:15.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.629+0000 7f3b0da00640 1 -- 172.21.15.167:0/1936827671 shutdown_connections 2024-05-11T12:14:15.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.629+0000 7f3b0da00640 1 -- 172.21.15.167:0/1936827671 wait complete. 2024-05-11T12:14:15.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.630+0000 7f3b0da00640 1 Processor -- start 2024-05-11T12:14:15.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.630+0000 7f3b0da00640 1 -- start start 2024-05-11T12:14:15.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.631+0000 7f3b0da00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b080ceab0 0x7f3b080ca980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:15.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.631+0000 7f3b0da00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b080d0da0 0x7f3b081556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:15.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.631+0000 7f3b0da00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b08157940 0x7f3b08150c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:15.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.631+0000 7f3b0da00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3b08157d20 con 0x7f3b08157940 2024-05-11T12:14:15.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.631+0000 7f3b0da00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3b080cde80 con 0x7f3b080d0da0 2024-05-11T12:14:15.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.631+0000 7f3b0da00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3b080cf2a0 con 0x7f3b080ceab0 2024-05-11T12:14:15.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.632+0000 7f3b0d000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b08157940 0x7f3b08150c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:15.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.632+0000 7f3b0d000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b08157940 0x7f3b08150c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35676/0 (socket says 172.21.15.167:35676) 2024-05-11T12:14:15.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.632+0000 7f3b0d000640 1 -- 172.21.15.167:0/247808560 learned_addr learned my addr 172.21.15.167:0/247808560 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:15.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.632+0000 7f3b07e00640 1 --2- 172.21.15.167:0/247808560 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b080ceab0 0x7f3b080ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:15.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.632+0000 7f3b0d000640 1 -- 172.21.15.167:0/247808560 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b080ceab0 msgr2=0x7f3b080ca980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:15.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.632+0000 7f3b0d000640 1 --2- 172.21.15.167:0/247808560 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b080ceab0 0x7f3b080ca980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:15.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.632+0000 7f3b0d000640 1 -- 172.21.15.167:0/247808560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b080d0da0 msgr2=0x7f3b081556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:15.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.632+0000 7f3b0d000640 1 --2- 172.21.15.167:0/247808560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b080d0da0 0x7f3b081556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:15.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.632+0000 7f3b0d000640 1 -- 172.21.15.167:0/247808560 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b080b61b0 con 0x7f3b08157940 2024-05-11T12:14:15.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.633+0000 7f3b0d000640 1 --2- 172.21.15.167:0/247808560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b08157940 0x7f3b08150c00 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f3b0000c7f0 tx=0x7f3b0000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:15.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.633+0000 7f3affe00640 1 -- 172.21.15.167:0/247808560 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b00015070 con 0x7f3b08157940 2024-05-11T12:14:15.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.633+0000 7f3affe00640 1 -- 172.21.15.167:0/247808560 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b00007820 con 0x7f3b08157940 2024-05-11T12:14:15.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.633+0000 7f3affe00640 1 -- 172.21.15.167:0/247808560 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b00007c10 con 0x7f3b08157940 2024-05-11T12:14:15.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.633+0000 7f3b0da00640 1 -- 172.21.15.167:0/247808560 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b080b6490 con 0x7f3b08157940 2024-05-11T12:14:15.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.633+0000 7f3b0da00640 1 -- 172.21.15.167:0/247808560 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b080b6a50 con 0x7f3b08157940 2024-05-11T12:14:15.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.634+0000 7f3b0da00640 1 -- 172.21.15.167:0/247808560 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b08069ea0 con 0x7f3b08157940 2024-05-11T12:14:15.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.635+0000 7f3affe00640 1 -- 172.21.15.167:0/247808560 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3b00018070 con 0x7f3b08157940 2024-05-11T12:14:15.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.636+0000 7f3affe00640 1 --2- 172.21.15.167:0/247808560 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3ae40762d0 0x7f3ae4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:15.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.636+0000 7f3affe00640 1 -- 172.21.15.167:0/247808560 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7f3b00097940 con 0x7f3b08157940 2024-05-11T12:14:15.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.637+0000 7f3b07e00640 1 --2- 172.21.15.167:0/247808560 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3ae40762d0 0x7f3ae4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:15.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.637+0000 7f3b07e00640 1 --2- 172.21.15.167:0/247808560 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3ae40762d0 0x7f3ae4078790 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3af8033000 tx=0x7f3af8009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:15.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.641+0000 7f3affe00640 1 -- 172.21.15.167:0/247808560 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3b08069ea0 con 0x7f3b08157940 2024-05-11T12:14:15.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.862+0000 7f3b0da00640 1 -- 172.21.15.167:0/247808560 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3b080694b0 con 0x7f3b08157940 2024-05-11T12:14:15.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.862+0000 7f3affe00640 1 -- 172.21.15.167:0/247808560 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v86) ==== 80+0+0 (secure 0 0 0) 0x7f3b00061100 con 0x7f3b08157940 2024-05-11T12:14:15.863 INFO:teuthology.orchestra.run.smithi167.stderr:listed 0 entries 2024-05-11T12:14:15.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.865+0000 7f3b0da00640 1 -- 172.21.15.167:0/247808560 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3ae40762d0 msgr2=0x7f3ae4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:15.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.865+0000 7f3b0da00640 1 --2- 172.21.15.167:0/247808560 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3ae40762d0 0x7f3ae4078790 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3af8033000 tx=0x7f3af8009000 comp rx=0 tx=0).stop 2024-05-11T12:14:15.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.865+0000 7f3b0da00640 1 -- 172.21.15.167:0/247808560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b08157940 msgr2=0x7f3b08150c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:15.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.865+0000 7f3b0da00640 1 --2- 172.21.15.167:0/247808560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b08157940 0x7f3b08150c00 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f3b0000c7f0 tx=0x7f3b0000ccb0 comp rx=0 tx=0).stop 2024-05-11T12:14:15.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.865+0000 7f3b0da00640 1 -- 172.21.15.167:0/247808560 shutdown_connections 2024-05-11T12:14:15.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.865+0000 7f3b0da00640 1 --2- 172.21.15.167:0/247808560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b08157940 0x7f3b08150c00 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:15.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.865+0000 7f3b0da00640 1 --2- 172.21.15.167:0/247808560 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3ae40762d0 0x7f3ae4078790 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:15.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.865+0000 7f3b0da00640 1 --2- 172.21.15.167:0/247808560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b080d0da0 0x7f3b081556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:15.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.865+0000 7f3b0da00640 1 --2- 172.21.15.167:0/247808560 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b080ceab0 0x7f3b080ca980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:15.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.865+0000 7f3b0da00640 1 -- 172.21.15.167:0/247808560 >> 172.21.15.167:0/247808560 conn(0x7f3b080d2ea0 msgr2=0x7f3b080d1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:15.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.865+0000 7f3b0da00640 1 -- 172.21.15.167:0/247808560 shutdown_connections 2024-05-11T12:14:15.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:15.865+0000 7f3b0da00640 1 -- 172.21.15.167:0/247808560 wait complete. 2024-05-11T12:14:15.883 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-11T12:14:15.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:15.883 DEBUG:teuthology.orchestra.run.smithi167:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-11T12:14:15.976 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2024-05-11T12:14:15.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:15.976 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-11T12:14:16.024 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:14:16.025 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:14:16.083 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:14:16.083 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:14:16.136 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-11T12:14:16.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.349+0000 7f3363200640 1 Processor -- start 2024-05-11T12:14:16.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.349+0000 7f3363200640 1 -- start start 2024-05-11T12:14:16.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.351+0000 7f3363200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f336413fc30 0x7f3364160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:16.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.351+0000 7f3363200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f33640cfc90 0x7f3364161020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:16.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.351+0000 7f3363200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33640d0220 con 0x7f3364140010 2024-05-11T12:14:16.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.351+0000 7f3363200640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f33641619d0 con 0x7f33640cfc90 2024-05-11T12:14:16.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.351+0000 7f3363200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f33640d0ac0 con 0x7f336413fc30 2024-05-11T12:14:16.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.351+0000 7f3362800640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f33640cfc90 0x7f3364161020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:16.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.351+0000 7f3362800640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f33640cfc90 0x7f3364161020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35682/0 (socket says 172.21.15.167:35682) 2024-05-11T12:14:16.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.351+0000 7f3362800640 1 -- 172.21.15.167:0/529920863 learned_addr learned my addr 172.21.15.167:0/529920863 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:16.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.351+0000 7f3361e00640 1 --2- 172.21.15.167:0/529920863 >> v2:172.21.15.204:3300/0 conn(0x7f336413fc30 0x7f3364160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:16.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.351+0000 7f3362800640 1 -- 172.21.15.167:0/529920863 >> v2:172.21.15.204:3300/0 conn(0x7f336413fc30 msgr2=0x7f3364160ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:16.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.352+0000 7f3362800640 1 --2- 172.21.15.167:0/529920863 >> v2:172.21.15.204:3300/0 conn(0x7f336413fc30 0x7f3364160ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:16.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.352+0000 7f3362800640 1 -- 172.21.15.167:0/529920863 >> v1:172.21.15.204:6789/0 conn(0x7f3364140010 legacy=0x7f33640cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:16.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.352+0000 7f3362800640 1 -- 172.21.15.167:0/529920863 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33640d0500 con 0x7f33640cfc90 2024-05-11T12:14:16.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.352+0000 7f3361e00640 1 --2- 172.21.15.167:0/529920863 >> v2:172.21.15.204:3300/0 conn(0x7f336413fc30 0x7f3364160ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:16.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.353+0000 7f3362800640 1 --2- 172.21.15.167:0/529920863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33640cfc90 0x7f3364161020 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f3364070100 tx=0x7f335c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9042408f9a32777f server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:16.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.353+0000 7f3361400640 1 -- 172.21.15.167:0/529920863 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335c036070 con 0x7f33640cfc90 2024-05-11T12:14:16.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.353+0000 7f3361400640 1 -- 172.21.15.167:0/529920863 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f335c031600 con 0x7f33640cfc90 2024-05-11T12:14:16.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.353+0000 7f3361400640 1 -- 172.21.15.167:0/529920863 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335c0319d0 con 0x7f33640cfc90 2024-05-11T12:14:16.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.353+0000 7f3363200640 1 -- 172.21.15.167:0/529920863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33640cfc90 msgr2=0x7f3364161020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:16.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.353+0000 7f3363200640 1 --2- 172.21.15.167:0/529920863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33640cfc90 0x7f3364161020 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f3364070100 tx=0x7f335c02a610 comp rx=0 tx=0).stop 2024-05-11T12:14:16.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.354+0000 7f3363200640 1 -- 172.21.15.167:0/529920863 shutdown_connections 2024-05-11T12:14:16.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.354+0000 7f3363200640 1 --2- 172.21.15.167:0/529920863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33640cfc90 0x7f3364161020 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:16.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.354+0000 7f3363200640 1 --2- 172.21.15.167:0/529920863 >> v2:172.21.15.204:3300/0 conn(0x7f336413fc30 0x7f3364160ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:16.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.354+0000 7f3363200640 1 -- 172.21.15.167:0/529920863 >> 172.21.15.167:0/529920863 conn(0x7f336406c450 msgr2=0x7f336406c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:16.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.354+0000 7f3363200640 1 -- 172.21.15.167:0/529920863 shutdown_connections 2024-05-11T12:14:16.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.354+0000 7f3363200640 1 -- 172.21.15.167:0/529920863 wait complete. 2024-05-11T12:14:16.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.355+0000 7f3363200640 1 Processor -- start 2024-05-11T12:14:16.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.356+0000 7f3363200640 1 -- start start 2024-05-11T12:14:16.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.356+0000 7f3363200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33640cfc90 0x7f33641f99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:16.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.356+0000 7f3363200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f336413fc30 0x7f3364154880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:16.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.356+0000 7f3363200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3364140010 0x7f336413b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:16.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.357+0000 7f3363200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f336406eab0 con 0x7f3364140010 2024-05-11T12:14:16.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.357+0000 7f3363200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f336406e590 con 0x7f336413fc30 2024-05-11T12:14:16.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.357+0000 7f3363200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f336406ec30 con 0x7f33640cfc90 2024-05-11T12:14:16.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.357+0000 7f3362800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3364140010 0x7f336413b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:16.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.357+0000 7f3362800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3364140010 0x7f336413b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35698/0 (socket says 172.21.15.167:35698) 2024-05-11T12:14:16.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.357+0000 7f3362800640 1 -- 172.21.15.167:0/2342701155 learned_addr learned my addr 172.21.15.167:0/2342701155 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:16.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.357+0000 7f3361e00640 1 --2- 172.21.15.167:0/2342701155 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33640cfc90 0x7f33641f99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:16.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.357+0000 7f3362800640 1 -- 172.21.15.167:0/2342701155 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33640cfc90 msgr2=0x7f33641f99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:16.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.357+0000 7f3362800640 1 --2- 172.21.15.167:0/2342701155 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33640cfc90 0x7f33641f99a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:16.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.357+0000 7f3362800640 1 -- 172.21.15.167:0/2342701155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f336413fc30 msgr2=0x7f3364154880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:16.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.357+0000 7f335be00640 1 --2- 172.21.15.167:0/2342701155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f336413fc30 0x7f3364154880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:16.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.357+0000 7f3362800640 1 --2- 172.21.15.167:0/2342701155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f336413fc30 0x7f3364154880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:16.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.357+0000 7f3362800640 1 -- 172.21.15.167:0/2342701155 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f336415f7a0 con 0x7f3364140010 2024-05-11T12:14:16.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.357+0000 7f335be00640 1 --2- 172.21.15.167:0/2342701155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f336413fc30 0x7f3364154880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:16.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.358+0000 7f3362800640 1 --2- 172.21.15.167:0/2342701155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3364140010 0x7f336413b200 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f335c009f40 tx=0x7f335c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:16.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.358+0000 7f335a000640 1 -- 172.21.15.167:0/2342701155 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335c036050 con 0x7f3364140010 2024-05-11T12:14:16.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.358+0000 7f335a000640 1 -- 172.21.15.167:0/2342701155 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f335c042070 con 0x7f3364140010 2024-05-11T12:14:16.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.358+0000 7f335a000640 1 -- 172.21.15.167:0/2342701155 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335c009510 con 0x7f3364140010 2024-05-11T12:14:16.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.358+0000 7f3363200640 1 -- 172.21.15.167:0/2342701155 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f336415fa80 con 0x7f3364140010 2024-05-11T12:14:16.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.358+0000 7f3363200640 1 -- 172.21.15.167:0/2342701155 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3364160040 con 0x7f3364140010 2024-05-11T12:14:16.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.359+0000 7f3363200640 1 -- 172.21.15.167:0/2342701155 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3364154370 con 0x7f3364140010 2024-05-11T12:14:16.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.361+0000 7f335a000640 1 -- 172.21.15.167:0/2342701155 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f335c007bd0 con 0x7f3364140010 2024-05-11T12:14:16.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.362+0000 7f335a000640 1 --2- 172.21.15.167:0/2342701155 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f33400762d0 0x7f3340078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:16.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.362+0000 7f335a000640 1 -- 172.21.15.167:0/2342701155 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7f335c0b86f0 con 0x7f3364140010 2024-05-11T12:14:16.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.362+0000 7f3361e00640 1 --2- 172.21.15.167:0/2342701155 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f33400762d0 0x7f3340078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:16.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.363+0000 7f3361e00640 1 --2- 172.21.15.167:0/2342701155 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f33400762d0 0x7f3340078790 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3354008850 tx=0x7f3354008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:16.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.366+0000 7f335a000640 1 -- 172.21.15.167:0/2342701155 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3364154370 con 0x7f3364140010 2024-05-11T12:14:16.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.588+0000 7f3363200640 1 -- 172.21.15.167:0/2342701155 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f33640694b0 con 0x7f3364140010 2024-05-11T12:14:16.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.589+0000 7f335a000640 1 -- 172.21.15.167:0/2342701155 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v86) ==== 80+0+0 (secure 0 0 0) 0x7f335c081eb0 con 0x7f3364140010 2024-05-11T12:14:16.589 INFO:teuthology.orchestra.run.smithi167.stderr:listed 0 entries 2024-05-11T12:14:16.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.591+0000 7f3363200640 1 -- 172.21.15.167:0/2342701155 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f33400762d0 msgr2=0x7f3340078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:16.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.591+0000 7f3363200640 1 --2- 172.21.15.167:0/2342701155 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f33400762d0 0x7f3340078790 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3354008850 tx=0x7f3354008040 comp rx=0 tx=0).stop 2024-05-11T12:14:16.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.591+0000 7f3363200640 1 -- 172.21.15.167:0/2342701155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3364140010 msgr2=0x7f336413b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:16.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.591+0000 7f3363200640 1 --2- 172.21.15.167:0/2342701155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3364140010 0x7f336413b200 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f335c009f40 tx=0x7f335c004a40 comp rx=0 tx=0).stop 2024-05-11T12:14:16.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.591+0000 7f3363200640 1 -- 172.21.15.167:0/2342701155 shutdown_connections 2024-05-11T12:14:16.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.591+0000 7f3363200640 1 --2- 172.21.15.167:0/2342701155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3364140010 0x7f336413b200 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:16.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.591+0000 7f3363200640 1 --2- 172.21.15.167:0/2342701155 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f33400762d0 0x7f3340078790 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:16.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.591+0000 7f3363200640 1 --2- 172.21.15.167:0/2342701155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f336413fc30 0x7f3364154880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:16.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.591+0000 7f3363200640 1 --2- 172.21.15.167:0/2342701155 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33640cfc90 0x7f33641f99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:16.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.591+0000 7f3363200640 1 -- 172.21.15.167:0/2342701155 >> 172.21.15.167:0/2342701155 conn(0x7f336406c450 msgr2=0x7f336406d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:16.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.591+0000 7f3363200640 1 -- 172.21.15.167:0/2342701155 shutdown_connections 2024-05-11T12:14:16.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.591+0000 7f3363200640 1 -- 172.21.15.167:0/2342701155 wait complete. 2024-05-11T12:14:16.611 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-11T12:14:16.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:16.612 DEBUG:teuthology.orchestra.run.smithi204:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-05-11T12:14:16.711 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi204.front.sepia.ceph.com 2024-05-11T12:14:16.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:16.711 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-11T12:14:16.762 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:14:16.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.971+0000 7f99bda00640 1 Processor -- start 2024-05-11T12:14:16.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.971+0000 7f99bda00640 1 -- start start 2024-05-11T12:14:16.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.972+0000 7f99bda00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99b8157750 0x7f99b815d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:16.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.973+0000 7f99bda00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f99b806d460 0x7f99b806d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:16.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.973+0000 7f99bda00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99b815dc80 con 0x7f99b80b3ea0 2024-05-11T12:14:16.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.973+0000 7f99bda00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f99b8161000 con 0x7f99b806d460 2024-05-11T12:14:16.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.973+0000 7f99bda00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f99b815df30 con 0x7f99b8157750 2024-05-11T12:14:16.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.973+0000 7f99b6000640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f99b80b3ea0 0x7f99b8161490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:46476/0 (socket says 172.21.15.167:46476) 2024-05-11T12:14:16.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.973+0000 7f99b6000640 1 -- 172.21.15.167:0/2398983347 learned_addr learned my addr 172.21.15.167:0/2398983347 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:14:16.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.973+0000 7f99b7400640 1 --2- 172.21.15.167:0/2398983347 >> v2:172.21.15.167:3300/0 conn(0x7f99b806d460 0x7f99b806d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:16.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.973+0000 7f99b6a00640 1 --2- 172.21.15.167:0/2398983347 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99b8157750 0x7f99b815d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:16.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.973+0000 7f99b5600640 1 -- 172.21.15.167:0/2398983347 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4006755033 0 0) 0x7f99b815dc80 con 0x7f99b80b3ea0 2024-05-11T12:14:16.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.973+0000 7f99b5600640 1 -- 172.21.15.167:0/2398983347 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f999c003520 con 0x7f99b80b3ea0 2024-05-11T12:14:16.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.974+0000 7f99b7400640 1 -- 172.21.15.167:0/2398983347 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99b8157750 msgr2=0x7f99b815d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:16.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.974+0000 7f99b7400640 1 --2- 172.21.15.167:0/2398983347 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99b8157750 0x7f99b815d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:16.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.974+0000 7f99b7400640 1 -- 172.21.15.167:0/2398983347 >> v1:172.21.15.167:6789/0 conn(0x7f99b80b3ea0 legacy=0x7f99b8161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:16.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.974+0000 7f99b7400640 1 -- 172.21.15.167:0/2398983347 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99b815e210 con 0x7f99b806d460 2024-05-11T12:14:16.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.975+0000 7f99b7400640 1 --2- 172.21.15.167:0/2398983347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99b806d460 0x7f99b806d8f0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f99b0009d90 tx=0x7f99b002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b2f793846c161c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:16.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.975+0000 7f99b5600640 1 -- 172.21.15.167:0/2398983347 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b0007740 con 0x7f99b806d460 2024-05-11T12:14:16.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.975+0000 7f99b5600640 1 -- 172.21.15.167:0/2398983347 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99b00078e0 con 0x7f99b806d460 2024-05-11T12:14:16.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.975+0000 7f99b5600640 1 -- 172.21.15.167:0/2398983347 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b0007cb0 con 0x7f99b806d460 2024-05-11T12:14:16.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.975+0000 7f99bda00640 1 -- 172.21.15.167:0/2398983347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99b806d460 msgr2=0x7f99b806d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:16.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.975+0000 7f99bda00640 1 --2- 172.21.15.167:0/2398983347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99b806d460 0x7f99b806d8f0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f99b0009d90 tx=0x7f99b002a660 comp rx=0 tx=0).stop 2024-05-11T12:14:16.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.976+0000 7f99bda00640 1 -- 172.21.15.167:0/2398983347 shutdown_connections 2024-05-11T12:14:16.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.976+0000 7f99bda00640 1 --2- 172.21.15.167:0/2398983347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99b806d460 0x7f99b806d8f0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:16.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.976+0000 7f99bda00640 1 --2- 172.21.15.167:0/2398983347 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99b8157750 0x7f99b815d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:16.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.976+0000 7f99bda00640 1 -- 172.21.15.167:0/2398983347 >> 172.21.15.167:0/2398983347 conn(0x7f99b80cd8f0 msgr2=0x7f99b80cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:16.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.976+0000 7f99bda00640 1 -- 172.21.15.167:0/2398983347 shutdown_connections 2024-05-11T12:14:16.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.976+0000 7f99bda00640 1 -- 172.21.15.167:0/2398983347 wait complete. 2024-05-11T12:14:16.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.977+0000 7f99bda00640 1 Processor -- start 2024-05-11T12:14:16.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.977+0000 7f99bda00640 1 -- start start 2024-05-11T12:14:16.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.978+0000 7f99bda00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99b806d460 0x7f99b8132150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:16.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.978+0000 7f99bda00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99b80b3ea0 0x7f99b8134bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:16.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.978+0000 7f99bda00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99b8157750 0x7f99b81f20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:16.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.978+0000 7f99bda00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f99b8161ba0 con 0x7f99b8157750 2024-05-11T12:14:16.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.978+0000 7f99bda00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f99b8157b30 con 0x7f99b80b3ea0 2024-05-11T12:14:16.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.978+0000 7f99bda00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f99b80cf1c0 con 0x7f99b806d460 2024-05-11T12:14:16.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.979+0000 7f99b7400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99b8157750 0x7f99b81f20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:16.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.979+0000 7f99b7400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99b8157750 0x7f99b81f20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35714/0 (socket says 172.21.15.167:35714) 2024-05-11T12:14:16.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.979+0000 7f99b7400640 1 -- 172.21.15.167:0/2876003646 learned_addr learned my addr 172.21.15.167:0/2876003646 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:16.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.979+0000 7f99b6a00640 1 --2- 172.21.15.167:0/2876003646 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99b806d460 0x7f99b8132150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:16.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.979+0000 7f99b6000640 1 --2- 172.21.15.167:0/2876003646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99b80b3ea0 0x7f99b8134bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:16.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.979+0000 7f99b7400640 1 -- 172.21.15.167:0/2876003646 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99b806d460 msgr2=0x7f99b8132150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:16.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.979+0000 7f99b7400640 1 --2- 172.21.15.167:0/2876003646 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99b806d460 0x7f99b8132150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:16.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.979+0000 7f99b7400640 1 -- 172.21.15.167:0/2876003646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99b80b3ea0 msgr2=0x7f99b8134bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:16.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.979+0000 7f99b7400640 1 --2- 172.21.15.167:0/2876003646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99b80b3ea0 0x7f99b8134bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:16.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.979+0000 7f99b7400640 1 -- 172.21.15.167:0/2876003646 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99b8135130 con 0x7f99b8157750 2024-05-11T12:14:16.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.980+0000 7f99b7400640 1 --2- 172.21.15.167:0/2876003646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99b8157750 0x7f99b81f20d0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f99b0009eb0 tx=0x7f99b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:16.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.980+0000 7f99aea00640 1 -- 172.21.15.167:0/2876003646 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b0040070 con 0x7f99b8157750 2024-05-11T12:14:16.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.980+0000 7f99aea00640 1 -- 172.21.15.167:0/2876003646 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99b0031bc0 con 0x7f99b8157750 2024-05-11T12:14:16.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.980+0000 7f99aea00640 1 -- 172.21.15.167:0/2876003646 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b0009450 con 0x7f99b8157750 2024-05-11T12:14:16.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.980+0000 7f99bda00640 1 -- 172.21.15.167:0/2876003646 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99b8132730 con 0x7f99b8157750 2024-05-11T12:14:16.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.980+0000 7f99bda00640 1 -- 172.21.15.167:0/2876003646 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99b8132cf0 con 0x7f99b8157750 2024-05-11T12:14:16.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.982+0000 7f99bda00640 1 -- 172.21.15.167:0/2876003646 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99b8069590 con 0x7f99b8157750 2024-05-11T12:14:16.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.983+0000 7f99aea00640 1 -- 172.21.15.167:0/2876003646 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f99b0039040 con 0x7f99b8157750 2024-05-11T12:14:16.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.984+0000 7f99aea00640 1 --2- 172.21.15.167:0/2876003646 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f99980762d0 0x7f9998078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:16.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.984+0000 7f99aea00640 1 -- 172.21.15.167:0/2876003646 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7f99b00b9080 con 0x7f99b8157750 2024-05-11T12:14:16.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.984+0000 7f99b6a00640 1 --2- 172.21.15.167:0/2876003646 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f99980762d0 0x7f9998078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:16.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.985+0000 7f99b6a00640 1 --2- 172.21.15.167:0/2876003646 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f99980762d0 0x7f9998078790 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f99a8006fd0 tx=0x7f99a8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:16.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:16.990+0000 7f99aea00640 1 -- 172.21.15.167:0/2876003646 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f99b00827b0 con 0x7f99b8157750 2024-05-11T12:14:17.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.234+0000 7f99bda00640 1 -- 172.21.15.167:0/2876003646 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99b8075b40 con 0x7f99b8157750 2024-05-11T12:14:17.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.235+0000 7f99aea00640 1 -- 172.21.15.167:0/2876003646 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5195 (secure 0 0 0) 0x7f99b00820b0 con 0x7f99b8157750 2024-05-11T12:14:17.235 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:17.236 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4922,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1154357925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1154357925},{"type":"v1","addr":"172.21.15.167:6837","nonce":1154357925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":5155,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1256314893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1256314893},{"type":"v1","addr":"172.21.15.204:6833","nonce":1256314893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":5158,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1578748665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1578748665},{"type":"v1","addr":"172.21.15.204:6835","nonce":1578748665}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:14:04.861998+0000","modified":"2024-05-11T12:14:07.926157+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6300":{"gid":6300,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6836/3484070468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3484070468},{"type":"v1","addr":"172.21.15.167:6836","nonce":3484070468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6300,"qdb_cluster":[6300]},"id":6}]} 2024-05-11T12:14:17.236 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 53 2024-05-11T12:14:17.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.237+0000 7f99bda00640 1 -- 172.21.15.167:0/2876003646 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f99980762d0 msgr2=0x7f9998078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:17.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.237+0000 7f99bda00640 1 --2- 172.21.15.167:0/2876003646 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f99980762d0 0x7f9998078790 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f99a8006fd0 tx=0x7f99a8008040 comp rx=0 tx=0).stop 2024-05-11T12:14:17.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.238+0000 7f99bda00640 1 -- 172.21.15.167:0/2876003646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99b8157750 msgr2=0x7f99b81f20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:17.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.238+0000 7f99bda00640 1 --2- 172.21.15.167:0/2876003646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99b8157750 0x7f99b81f20d0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f99b0009eb0 tx=0x7f99b0004a40 comp rx=0 tx=0).stop 2024-05-11T12:14:17.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.238+0000 7f99bda00640 1 -- 172.21.15.167:0/2876003646 shutdown_connections 2024-05-11T12:14:17.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.238+0000 7f99bda00640 1 --2- 172.21.15.167:0/2876003646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99b8157750 0x7f99b81f20d0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:17.239 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.238+0000 7f99bda00640 1 --2- 172.21.15.167:0/2876003646 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f99980762d0 0x7f9998078790 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:17.239 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.238+0000 7f99bda00640 1 --2- 172.21.15.167:0/2876003646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99b80b3ea0 0x7f99b8134bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:17.239 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.238+0000 7f99bda00640 1 --2- 172.21.15.167:0/2876003646 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99b806d460 0x7f99b8132150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:17.239 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.238+0000 7f99bda00640 1 -- 172.21.15.167:0/2876003646 >> 172.21.15.167:0/2876003646 conn(0x7f99b80cd8f0 msgr2=0x7f99b80cf5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:17.239 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.238+0000 7f99bda00640 1 -- 172.21.15.167:0/2876003646 shutdown_connections 2024-05-11T12:14:17.239 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.238+0000 7f99bda00640 1 -- 172.21.15.167:0/2876003646 wait complete. 2024-05-11T12:14:17.257 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 session ls 2024-05-11T12:14:17.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.452+0000 7f21a3000640 1 Processor -- start 2024-05-11T12:14:17.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.452+0000 7f21a3000640 1 -- start start 2024-05-11T12:14:17.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.454+0000 7f21a3000640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f219c074260 0x7f219c0b3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:17.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.454+0000 7f21a3000640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f219c0b65e0 con 0x7f219c161380 2024-05-11T12:14:17.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.454+0000 7f21a3000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f219c0b68c0 con 0x7f219c15ac80 2024-05-11T12:14:17.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.454+0000 7f21a3000640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f219c074670 con 0x7f219c074260 2024-05-11T12:14:17.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.454+0000 7f21a1000640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f219c074260 0x7f219c0b3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:17.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.454+0000 7f219b400640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f219c161380 0x7f219c072950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:46490/0 (socket says 172.21.15.167:46490) 2024-05-11T12:14:17.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.454+0000 7f219b400640 1 -- 172.21.15.167:0/373227364 learned_addr learned my addr 172.21.15.167:0/373227364 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:14:17.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.455+0000 7f21a1000640 1 -- 172.21.15.167:0/373227364 >> v1:172.21.15.204:6789/0 conn(0x7f219c15ac80 legacy=0x7f219c160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:17.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.455+0000 7f21a1000640 1 -- 172.21.15.167:0/373227364 >> v1:172.21.15.167:6789/0 conn(0x7f219c161380 legacy=0x7f219c072950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:17.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.455+0000 7f21a1000640 1 -- 172.21.15.167:0/373227364 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f219c0b6ba0 con 0x7f219c074260 2024-05-11T12:14:17.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.455+0000 7f219aa00640 1 -- 172.21.15.167:0/373227364 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3038692032 0 0) 0x7f219c0b65e0 con 0x7f219c161380 2024-05-11T12:14:17.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.456+0000 7f21a1000640 1 --2- 172.21.15.167:0/373227364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f219c074260 0x7f219c0b3d50 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f2194009d90 tx=0x7f219402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87928a8bad43c56e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:17.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.456+0000 7f219aa00640 1 -- 172.21.15.167:0/373227364 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2194007740 con 0x7f219c074260 2024-05-11T12:14:17.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.456+0000 7f219aa00640 1 -- 172.21.15.167:0/373227364 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21940078e0 con 0x7f219c074260 2024-05-11T12:14:17.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.456+0000 7f219aa00640 1 -- 172.21.15.167:0/373227364 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2194007cb0 con 0x7f219c074260 2024-05-11T12:14:17.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.457+0000 7f21a3000640 1 -- 172.21.15.167:0/373227364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f219c074260 msgr2=0x7f219c0b3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:17.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.457+0000 7f21a3000640 1 --2- 172.21.15.167:0/373227364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f219c074260 0x7f219c0b3d50 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f2194009d90 tx=0x7f219402a660 comp rx=0 tx=0).stop 2024-05-11T12:14:17.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.457+0000 7f21a3000640 1 -- 172.21.15.167:0/373227364 shutdown_connections 2024-05-11T12:14:17.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.457+0000 7f21a3000640 1 --2- 172.21.15.167:0/373227364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f219c074260 0x7f219c0b3d50 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:17.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.457+0000 7f21a3000640 1 -- 172.21.15.167:0/373227364 >> 172.21.15.167:0/373227364 conn(0x7f219c06c450 msgr2=0x7f219c06c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:17.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.457+0000 7f21a3000640 1 -- 172.21.15.167:0/373227364 shutdown_connections 2024-05-11T12:14:17.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.457+0000 7f21a3000640 1 -- 172.21.15.167:0/373227364 wait complete. 2024-05-11T12:14:17.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.458+0000 7f21a3000640 1 Processor -- start 2024-05-11T12:14:17.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.459+0000 7f21a3000640 1 -- start start 2024-05-11T12:14:17.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.459+0000 7f21a3000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f219c074260 0x7f219c0cc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:17.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.459+0000 7f21a3000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f219c15ac80 0x7f219c0aac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:17.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.459+0000 7f21a3000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f219c161380 0x7f219c0ab600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:17.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.460+0000 7f21a3000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f219c15b060 con 0x7f219c161380 2024-05-11T12:14:17.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.460+0000 7f21a3000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f219c0b4290 con 0x7f219c074260 2024-05-11T12:14:17.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.460+0000 7f21a3000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f219c073060 con 0x7f219c15ac80 2024-05-11T12:14:17.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.460+0000 7f21a1000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f219c161380 0x7f219c0ab600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:17.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.460+0000 7f21a1000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f219c161380 0x7f219c0ab600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35732/0 (socket says 172.21.15.167:35732) 2024-05-11T12:14:17.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.460+0000 7f21a1000640 1 -- 172.21.15.167:0/1727336549 learned_addr learned my addr 172.21.15.167:0/1727336549 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:17.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.460+0000 7f219be00640 1 --2- 172.21.15.167:0/1727336549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f219c074260 0x7f219c0cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:17.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.460+0000 7f21a1000640 1 -- 172.21.15.167:0/1727336549 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f219c15ac80 msgr2=0x7f219c0aac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:17.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.460+0000 7f21a1000640 1 --2- 172.21.15.167:0/1727336549 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f219c15ac80 0x7f219c0aac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:17.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.460+0000 7f21a1000640 1 -- 172.21.15.167:0/1727336549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f219c074260 msgr2=0x7f219c0cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:17.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.460+0000 7f21a1000640 1 --2- 172.21.15.167:0/1727336549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f219c074260 0x7f219c0cc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:17.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.460+0000 7f21a1000640 1 -- 172.21.15.167:0/1727336549 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f219c144810 con 0x7f219c161380 2024-05-11T12:14:17.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.460+0000 7f21a1000640 1 --2- 172.21.15.167:0/1727336549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f219c161380 0x7f219c0ab600 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f219402ac50 tx=0x7f2194004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:17.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.461+0000 7f2193e00640 1 -- 172.21.15.167:0/1727336549 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2194040070 con 0x7f219c161380 2024-05-11T12:14:17.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.461+0000 7f2193e00640 1 -- 172.21.15.167:0/1727336549 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2194031c90 con 0x7f219c161380 2024-05-11T12:14:17.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.461+0000 7f21a3000640 1 -- 172.21.15.167:0/1727336549 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f219c0ab1b0 con 0x7f219c161380 2024-05-11T12:14:17.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.461+0000 7f2193e00640 1 -- 172.21.15.167:0/1727336549 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2194045660 con 0x7f219c161380 2024-05-11T12:14:17.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.461+0000 7f21a3000640 1 -- 172.21.15.167:0/1727336549 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f219c0afd90 con 0x7f219c161380 2024-05-11T12:14:17.463 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.463+0000 7f2193e00640 1 -- 172.21.15.167:0/1727336549 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2194039040 con 0x7f219c161380 2024-05-11T12:14:17.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.465+0000 7f2193e00640 1 --2- 172.21.15.167:0/1727336549 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f217c0762d0 0x7f217c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:17.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.465+0000 7f2193e00640 1 -- 172.21.15.167:0/1727336549 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7f21940b9280 con 0x7f219c161380 2024-05-11T12:14:17.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.465+0000 7f219be00640 1 --2- 172.21.15.167:0/1727336549 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f217c0762d0 0x7f217c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:17.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.466+0000 7f21a5df5740 1 Processor -- start 2024-05-11T12:14:17.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.466+0000 7f21a5df5740 1 -- start start 2024-05-11T12:14:17.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.466+0000 7f219be00640 1 --2- 172.21.15.167:0/1727336549 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f217c0762d0 0x7f217c078790 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f218c003d90 tx=0x7f218c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:17.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.467+0000 7f21a5df5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9ad80b770 0x55f9ad80bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:17.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.467+0000 7f21a5df5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9ad829010 0x55f9ad829560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:17.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.467+0000 7f21a5df5740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55f9ad7e1450 0x55f9ad8276d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:17.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.467+0000 7f21a5df5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f9ad745d50 con 0x55f9ad829010 2024-05-11T12:14:17.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.467+0000 7f21a5df5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f9ad6eb0c0 con 0x55f9ad80b770 2024-05-11T12:14:17.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.467+0000 7f21a5df5740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55f9ad741d10 con 0x55f9ad7e1450 2024-05-11T12:14:17.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.467+0000 7f219b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9ad829010 0x55f9ad829560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:17.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.467+0000 7f219b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9ad829010 0x55f9ad829560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35748/0 (socket says 172.21.15.167:35748) 2024-05-11T12:14:17.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.467+0000 7f219b400640 1 -- 172.21.15.167:0/2445201765 learned_addr learned my addr 172.21.15.167:0/2445201765 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:17.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.467+0000 7f219be00640 1 --2- 172.21.15.167:0/2445201765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9ad80b770 0x55f9ad80bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:17.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.467+0000 7f21a1000640 1 --2- 172.21.15.167:0/2445201765 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55f9ad7e1450 0x55f9ad8276d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:17.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.467+0000 7f219b400640 1 -- 172.21.15.167:0/2445201765 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55f9ad7e1450 msgr2=0x55f9ad8276d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:17.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.467+0000 7f219b400640 1 --2- 172.21.15.167:0/2445201765 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55f9ad7e1450 0x55f9ad8276d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:17.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.467+0000 7f219b400640 1 -- 172.21.15.167:0/2445201765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9ad80b770 msgr2=0x55f9ad80bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:17.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.467+0000 7f219b400640 1 --2- 172.21.15.167:0/2445201765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9ad80b770 0x55f9ad80bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:17.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.467+0000 7f219b400640 1 -- 172.21.15.167:0/2445201765 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9ad4573f0 con 0x55f9ad829010 2024-05-11T12:14:17.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.467+0000 7f21a1000640 1 --2- 172.21.15.167:0/2445201765 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55f9ad7e1450 0x55f9ad8276d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:17.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.468+0000 7f219b400640 1 --2- 172.21.15.167:0/2445201765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9ad829010 0x55f9ad829560 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f2188001580 tx=0x7f2188001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c9cacb991ad8b2e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:17.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.468+0000 7f2190c00640 1 -- 172.21.15.167:0/2445201765 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21880072d0 con 0x55f9ad829010 2024-05-11T12:14:17.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.468+0000 7f2190c00640 1 -- 172.21.15.167:0/2445201765 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2188007470 con 0x55f9ad829010 2024-05-11T12:14:17.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.468+0000 7f2190c00640 1 -- 172.21.15.167:0/2445201765 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2188004a20 con 0x55f9ad829010 2024-05-11T12:14:17.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.469+0000 7f21a5df5740 1 -- 172.21.15.167:0/2445201765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9ad829010 msgr2=0x55f9ad829560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:17.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.469+0000 7f21a5df5740 1 --2- 172.21.15.167:0/2445201765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9ad829010 0x55f9ad829560 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f2188001580 tx=0x7f2188001a40 comp rx=0 tx=0).stop 2024-05-11T12:14:17.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.469+0000 7f21a5df5740 1 -- 172.21.15.167:0/2445201765 shutdown_connections 2024-05-11T12:14:17.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.469+0000 7f21a5df5740 1 --2- 172.21.15.167:0/2445201765 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55f9ad7e1450 0x55f9ad8276d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:17.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.469+0000 7f21a5df5740 1 --2- 172.21.15.167:0/2445201765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9ad829010 0x55f9ad829560 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:17.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.469+0000 7f21a5df5740 1 --2- 172.21.15.167:0/2445201765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9ad80b770 0x55f9ad80bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:17.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.469+0000 7f21a5df5740 1 -- 172.21.15.167:0/2445201765 >> 172.21.15.167:0/2445201765 conn(0x55f9ad7ddd80 msgr2=0x55f9ad826580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:17.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.469+0000 7f21a5df5740 1 -- 172.21.15.167:0/2445201765 shutdown_connections 2024-05-11T12:14:17.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.469+0000 7f21a5df5740 1 -- 172.21.15.167:0/2445201765 wait complete. 2024-05-11T12:14:17.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.469+0000 7f21a5df5740 1 -- start start 2024-05-11T12:14:17.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.470+0000 7f21a5df5740 1 Processor -- start 2024-05-11T12:14:17.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.471+0000 7f21a5df5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9ad80b770 0x55f9ad7fad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:17.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.471+0000 7f21a5df5740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55f9ad829010 0x55f9ad80d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:17.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.471+0000 7f21a5df5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9ad81d630 0x55f9ad81dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:17.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.471+0000 7f21a5df5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f9ad7e21f0 con 0x55f9ad81d630 2024-05-11T12:14:17.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.471+0000 7f21a5df5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f9ad6feb50 con 0x55f9ad80b770 2024-05-11T12:14:17.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.471+0000 7f21a5df5740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55f9ad7dbcb0 con 0x55f9ad829010 2024-05-11T12:14:17.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.471+0000 7f21a5df5740 10 client.0 _ll_register_callbacks cb 0x55f9ad6c8420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:14:17.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.471+0000 7f21a1000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9ad81d630 0x55f9ad81dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:17.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.471+0000 7f21a1000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9ad81d630 0x55f9ad81dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35762/0 (socket says 172.21.15.167:35762) 2024-05-11T12:14:17.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.471+0000 7f21a1000640 1 -- 172.21.15.167:0/2497588731 learned_addr learned my addr 172.21.15.167:0/2497588731 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:17.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.471+0000 7f219b400640 1 --2- 172.21.15.167:0/2497588731 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55f9ad829010 0x55f9ad80d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:17.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.471+0000 7f219be00640 1 --2- 172.21.15.167:0/2497588731 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9ad80b770 0x55f9ad7fad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:17.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.471+0000 7f21a1000640 1 -- 172.21.15.167:0/2497588731 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55f9ad829010 msgr2=0x55f9ad80d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:17.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.471+0000 7f21a1000640 1 --2- 172.21.15.167:0/2497588731 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55f9ad829010 0x55f9ad80d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:17.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.471+0000 7f21a1000640 1 -- 172.21.15.167:0/2497588731 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9ad80b770 msgr2=0x55f9ad7fad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:17.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.471+0000 7f21a1000640 1 --2- 172.21.15.167:0/2497588731 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9ad80b770 0x55f9ad7fad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:17.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.471+0000 7f21a1000640 1 -- 172.21.15.167:0/2497588731 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9ad5217d0 con 0x55f9ad81d630 2024-05-11T12:14:17.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.472+0000 7f21a1000640 1 --2- 172.21.15.167:0/2497588731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9ad81d630 0x55f9ad81dab0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f219407e3f0 tx=0x7f2194031f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9f8bc6dea58d2d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:17.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.472+0000 7f2185600640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-11T12:14:17.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.472+0000 7f2185600640 1 -- 172.21.15.167:0/2497588731 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2194082370 con 0x55f9ad81d630 2024-05-11T12:14:17.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.472+0000 7f2185600640 1 -- 172.21.15.167:0/2497588731 <== mon.0 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7f219408acb0 con 0x55f9ad81d630 2024-05-11T12:14:17.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.472+0000 7f21a5df5740 1 -- 172.21.15.167:0/2497588731 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2188007470 con 0x55f9ad81d630 2024-05-11T12:14:17.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.472+0000 7f2185600640 1 -- 172.21.15.167:0/2497588731 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f2194086720 con 0x55f9ad81d630 2024-05-11T12:14:17.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.472+0000 7f21a5df5740 10 client.6489 fetch_fsmap learned FSMap version 53 2024-05-11T12:14:17.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.472+0000 7f21a5df5740 1 -- 172.21.15.167:0/2497588731 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x55f9ad80a160 con 0x55f9ad81d630 2024-05-11T12:14:17.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.472+0000 7f2185600640 1 -- 172.21.15.167:0/2497588731 <== mon.0 v2:172.21.15.167:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2188007470 con 0x55f9ad81d630 2024-05-11T12:14:17.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.472+0000 7f2185600640 1 -- 172.21.15.167:0/2497588731 <== mon.0 v2:172.21.15.167:3300/0 5 ==== fsmap(e 53) ==== 3494+0+0 (secure 0 0 0) 0x7f21940815d0 con 0x55f9ad81d630 2024-05-11T12:14:17.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.472+0000 7f21a5df5740 10 client.6489 fetch_fsmap finished waiting for FSMap version 53 2024-05-11T12:14:17.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.472+0000 7f21a5df5740 10 client.6489 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-11T12:14:17.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.472+0000 7f21a5df5740 20 client.6489 populate_metadata read hostname 'smithi167' 2024-05-11T12:14:17.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.472+0000 7f21a5df5740 1 --2- 172.21.15.167:0/2497588731 >> [v2:172.21.15.167:6835/3484070468,v1:172.21.15.167:6836/3484070468] conn(0x55f9ad83a1e0 0x55f9ad83a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:17.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.473+0000 7f21a5df5740 4 client.6489 mds_command: new command op to 6300 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:14:17.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.473+0000 7f21a5df5740 1 -- 172.21.15.167:0/2497588731 --> [v2:172.21.15.167:6835/3484070468,v1:172.21.15.167:6836/3484070468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9ad753960 con 0x55f9ad83a1e0 2024-05-11T12:14:17.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.473+0000 7f219be00640 1 --2- 172.21.15.167:0/2497588731 >> [v2:172.21.15.167:6835/3484070468,v1:172.21.15.167:6836/3484070468] conn(0x55f9ad83a1e0 0x55f9ad83a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:17.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.473+0000 7f219be00640 1 --2- 172.21.15.167:0/2497588731 >> [v2:172.21.15.167:6835/3484070468,v1:172.21.15.167:6836/3484070468] conn(0x55f9ad83a1e0 0x55f9ad83a610 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5805fb1a013a4467 server_cookie=9ce8a13965c1739d in_seq=0 out_seq=0 2024-05-11T12:14:17.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.473+0000 7f2185600640 10 client.6489 ms_handle_connect on v2:172.21.15.167:6835/3484070468 2024-05-11T12:14:17.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.474+0000 7f2185600640 1 -- 172.21.15.167:0/2497588731 <== mds.0 v2:172.21.15.167:6835/3484070468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f9ad753960 con 0x55f9ad83a1e0 2024-05-11T12:14:17.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.474+0000 7f2185600640 10 client.6489 handle_command_reply: tid=0 2024-05-11T12:14:17.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.474+0000 7f21a5df5740 1 client.6489 shutdown 2024-05-11T12:14:17.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.474+0000 7f21a5df5740 1 -- 172.21.15.167:0/2497588731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9ad81d630 msgr2=0x55f9ad81dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:17.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.474+0000 7f21a5df5740 1 --2- 172.21.15.167:0/2497588731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9ad81d630 0x55f9ad81dab0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f219407e3f0 tx=0x7f2194031f80 comp rx=0 tx=0).stop 2024-05-11T12:14:17.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.474+0000 7f21a5df5740 1 -- 172.21.15.167:0/2497588731 shutdown_connections 2024-05-11T12:14:17.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.474+0000 7f21a5df5740 1 --2- 172.21.15.167:0/2497588731 >> [v2:172.21.15.167:6835/3484070468,v1:172.21.15.167:6836/3484070468] conn(0x55f9ad83a1e0 0x55f9ad83a610 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:17.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.474+0000 7f21a5df5740 1 --2- 172.21.15.167:0/2497588731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9ad81d630 0x55f9ad81dab0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:17.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.474+0000 7f21a5df5740 1 --2- 172.21.15.167:0/2497588731 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55f9ad829010 0x55f9ad80d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:17.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.474+0000 7f2185600640 0 client.6489 ms_handle_reset on v2:172.21.15.167:6835/3484070468 2024-05-11T12:14:17.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.474+0000 7f21a5df5740 1 --2- 172.21.15.167:0/2497588731 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9ad80b770 0x55f9ad7fad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:17.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.474+0000 7f21a5df5740 1 -- 172.21.15.167:0/2497588731 >> 172.21.15.167:0/2497588731 conn(0x55f9ad7e1450 msgr2=0x55f9ad829b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:17.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.475+0000 7f21a5df5740 1 -- 172.21.15.167:0/2497588731 shutdown_connections 2024-05-11T12:14:17.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.475+0000 7f21a5df5740 1 -- 172.21.15.167:0/2497588731 wait complete. 2024-05-11T12:14:17.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.475+0000 7f21a5df5740 20 client.6489 trim_cache size 0 max 16384 2024-05-11T12:14:17.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.495+0000 7f21a5df5740 1 Processor -- start 2024-05-11T12:14:17.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.495+0000 7f21a5df5740 1 -- start start 2024-05-11T12:14:17.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.496+0000 7f21a5df5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9ad80b770 0x55f9ad80f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:17.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.496+0000 7f21a5df5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9ad81d630 0x55f9ad8492b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:17.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.496+0000 7f21a5df5740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55f9ad829010 0x55f9ad89eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:17.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.496+0000 7f21a5df5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f219c1ffcd0 con 0x55f9ad81d630 2024-05-11T12:14:17.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.496+0000 7f21a5df5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f219c15c8b0 con 0x55f9ad80b770 2024-05-11T12:14:17.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.496+0000 7f21a5df5740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f219c1f6f50 con 0x55f9ad829010 2024-05-11T12:14:17.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.496+0000 7f219b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9ad81d630 0x55f9ad8492b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:17.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.496+0000 7f219b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9ad81d630 0x55f9ad8492b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35764/0 (socket says 172.21.15.167:35764) 2024-05-11T12:14:17.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.496+0000 7f219b400640 1 -- 172.21.15.167:0/3143614003 learned_addr learned my addr 172.21.15.167:0/3143614003 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:17.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.496+0000 7f219be00640 1 --2- 172.21.15.167:0/3143614003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9ad80b770 0x55f9ad80f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:17.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.497+0000 7f219b400640 1 -- 172.21.15.167:0/3143614003 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55f9ad829010 msgr2=0x55f9ad89eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:17.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.497+0000 7f21a1000640 1 --2- 172.21.15.167:0/3143614003 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55f9ad829010 0x55f9ad89eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:17.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.497+0000 7f219b400640 1 --2- 172.21.15.167:0/3143614003 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55f9ad829010 0x55f9ad89eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:17.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.497+0000 7f219b400640 1 -- 172.21.15.167:0/3143614003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9ad80b770 msgr2=0x55f9ad80f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:17.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.497+0000 7f219b400640 1 --2- 172.21.15.167:0/3143614003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9ad80b770 0x55f9ad80f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:17.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.497+0000 7f219b400640 1 -- 172.21.15.167:0/3143614003 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9ad81f270 con 0x55f9ad81d630 2024-05-11T12:14:17.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.497+0000 7f219be00640 1 --2- 172.21.15.167:0/3143614003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9ad80b770 0x55f9ad80f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:17.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.497+0000 7f219b400640 1 --2- 172.21.15.167:0/3143614003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9ad81d630 0x55f9ad8492b0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f2188007fd0 tx=0x7f2188006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41a16c1312f84069 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:17.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.497+0000 7f2187e00640 1 -- 172.21.15.167:0/3143614003 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2188026070 con 0x55f9ad81d630 2024-05-11T12:14:17.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.497+0000 7f2187e00640 1 -- 172.21.15.167:0/3143614003 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f218800b030 con 0x55f9ad81d630 2024-05-11T12:14:17.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.497+0000 7f2187e00640 1 -- 172.21.15.167:0/3143614003 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2188007630 con 0x55f9ad81d630 2024-05-11T12:14:17.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.498+0000 7f21a5df5740 1 -- 172.21.15.167:0/3143614003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9ad81d630 msgr2=0x55f9ad8492b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:17.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.498+0000 7f21a5df5740 1 --2- 172.21.15.167:0/3143614003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9ad81d630 0x55f9ad8492b0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f2188007fd0 tx=0x7f2188006000 comp rx=0 tx=0).stop 2024-05-11T12:14:17.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.498+0000 7f21a5df5740 1 -- 172.21.15.167:0/3143614003 shutdown_connections 2024-05-11T12:14:17.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.498+0000 7f21a5df5740 1 --2- 172.21.15.167:0/3143614003 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55f9ad829010 0x55f9ad89eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:17.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.498+0000 7f21a5df5740 1 --2- 172.21.15.167:0/3143614003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9ad81d630 0x55f9ad8492b0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:17.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.498+0000 7f21a5df5740 1 --2- 172.21.15.167:0/3143614003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9ad80b770 0x55f9ad80f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:17.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.498+0000 7f21a5df5740 1 -- 172.21.15.167:0/3143614003 >> 172.21.15.167:0/3143614003 conn(0x55f9ad7e1450 msgr2=0x55f9ad7d7180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:17.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.498+0000 7f21a5df5740 1 -- 172.21.15.167:0/3143614003 shutdown_connections 2024-05-11T12:14:17.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.498+0000 7f21a5df5740 1 -- 172.21.15.167:0/3143614003 wait complete. 2024-05-11T12:14:17.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.499+0000 7f21a5df5740 1 -- start start 2024-05-11T12:14:17.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.499+0000 7f21a5df5740 1 Processor -- start 2024-05-11T12:14:17.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.500+0000 7f21a5df5740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55f9ad80b770 0x55f9ad81c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:17.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.500+0000 7f21a5df5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9ad81d630 0x55f9ad89db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:17.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.500+0000 7f21a5df5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9ad829010 0x55f9ad838580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:17.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.500+0000 7f21a5df5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f219c15c260 con 0x55f9ad829010 2024-05-11T12:14:17.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.500+0000 7f21a5df5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f9ad82a520 con 0x55f9ad81d630 2024-05-11T12:14:17.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.500+0000 7f21a5df5740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55f9ad7db190 con 0x55f9ad80b770 2024-05-11T12:14:17.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.500+0000 7f21a5df5740 10 client.0 _ll_register_callbacks cb 0x7f219407ea40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:14:17.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.500+0000 7f21a1000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9ad829010 0x55f9ad838580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:17.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.500+0000 7f21a1000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9ad829010 0x55f9ad838580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35766/0 (socket says 172.21.15.167:35766) 2024-05-11T12:14:17.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.500+0000 7f21a1000640 1 -- 172.21.15.167:0/1161462827 learned_addr learned my addr 172.21.15.167:0/1161462827 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:17.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.501+0000 7f219be00640 1 --2- 172.21.15.167:0/1161462827 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55f9ad80b770 0x55f9ad81c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:17.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.501+0000 7f219b400640 1 --2- 172.21.15.167:0/1161462827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9ad81d630 0x55f9ad89db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:17.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.501+0000 7f21a1000640 1 -- 172.21.15.167:0/1161462827 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55f9ad80b770 msgr2=0x55f9ad81c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:17.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.501+0000 7f21a1000640 1 --2- 172.21.15.167:0/1161462827 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55f9ad80b770 0x55f9ad81c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:17.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.501+0000 7f21a1000640 1 -- 172.21.15.167:0/1161462827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9ad81d630 msgr2=0x55f9ad89db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:17.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.501+0000 7f21a1000640 1 --2- 172.21.15.167:0/1161462827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9ad81d630 0x55f9ad89db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:17.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.501+0000 7f21a1000640 1 -- 172.21.15.167:0/1161462827 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9ad83a940 con 0x55f9ad829010 2024-05-11T12:14:17.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.501+0000 7f219be00640 1 --2- 172.21.15.167:0/1161462827 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55f9ad80b770 0x55f9ad81c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:17.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.501+0000 7f21a1000640 1 --2- 172.21.15.167:0/1161462827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9ad829010 0x55f9ad838580 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f219403f040 tx=0x7f2194009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba2539d85b5f4e36 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:17.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.501+0000 7f2185600640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-11T12:14:17.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.501+0000 7f2185600640 1 -- 172.21.15.167:0/1161462827 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2194081ae0 con 0x55f9ad829010 2024-05-11T12:14:17.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.501+0000 7f2185600640 1 -- 172.21.15.167:0/1161462827 <== mon.0 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7f219407dce0 con 0x55f9ad829010 2024-05-11T12:14:17.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.501+0000 7f21a5df5740 1 -- 172.21.15.167:0/1161462827 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f218800b030 con 0x55f9ad829010 2024-05-11T12:14:17.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.501+0000 7f2185600640 1 -- 172.21.15.167:0/1161462827 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f21940b0050 con 0x55f9ad829010 2024-05-11T12:14:17.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.501+0000 7f21a5df5740 10 client.6495 fetch_fsmap learned FSMap version 53 2024-05-11T12:14:17.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.501+0000 7f21a5df5740 1 -- 172.21.15.167:0/1161462827 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x55f9ad83a940 con 0x55f9ad829010 2024-05-11T12:14:17.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.502+0000 7f2185600640 1 -- 172.21.15.167:0/1161462827 <== mon.0 v2:172.21.15.167:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f218800b030 con 0x55f9ad829010 2024-05-11T12:14:17.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.502+0000 7f2185600640 1 -- 172.21.15.167:0/1161462827 <== mon.0 v2:172.21.15.167:3300/0 5 ==== fsmap(e 53) ==== 3494+0+0 (secure 0 0 0) 0x7f219407dee0 con 0x55f9ad829010 2024-05-11T12:14:17.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.502+0000 7f21a5df5740 10 client.6495 fetch_fsmap finished waiting for FSMap version 53 2024-05-11T12:14:17.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.502+0000 7f21a5df5740 10 client.6495 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-11T12:14:17.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.502+0000 7f21a5df5740 20 client.6495 populate_metadata read hostname 'smithi167' 2024-05-11T12:14:17.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.502+0000 7f21a5df5740 1 --2- 172.21.15.167:0/1161462827 >> [v2:172.21.15.167:6835/3484070468,v1:172.21.15.167:6836/3484070468] conn(0x55f9ad7daa60 0x55f9ad838ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:17.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.502+0000 7f21a5df5740 4 client.6495 mds_command: new command op to 6300 tid=0 multi_id=0 [{"prefix": "session ls"}] 2024-05-11T12:14:17.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.502+0000 7f21a5df5740 1 -- 172.21.15.167:0/1161462827 --> [v2:172.21.15.167:6835/3484070468,v1:172.21.15.167:6836/3484070468] -- command(tid 0: {"prefix": "session ls"}) -- 0x55f9ad74d7d0 con 0x55f9ad7daa60 2024-05-11T12:14:17.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.502+0000 7f219be00640 1 --2- 172.21.15.167:0/1161462827 >> [v2:172.21.15.167:6835/3484070468,v1:172.21.15.167:6836/3484070468] conn(0x55f9ad7daa60 0x55f9ad838ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:17.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.503+0000 7f219be00640 1 --2- 172.21.15.167:0/1161462827 >> [v2:172.21.15.167:6835/3484070468,v1:172.21.15.167:6836/3484070468] conn(0x55f9ad7daa60 0x55f9ad838ac0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e54f767472999887 server_cookie=66b04a19e4e22972 in_seq=0 out_seq=0 2024-05-11T12:14:17.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.503+0000 7f2185600640 10 client.6495 ms_handle_connect on v2:172.21.15.167:6835/3484070468 2024-05-11T12:14:17.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.503+0000 7f2185600640 1 -- 172.21.15.167:0/1161462827 <== mds.0 v2:172.21.15.167:6835/3484070468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f9ad74d7d0 con 0x55f9ad7daa60 2024-05-11T12:14:17.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.503+0000 7f2185600640 10 client.6495 handle_command_reply: tid=0 2024-05-11T12:14:17.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.503+0000 7f21a5df5740 1 client.6495 shutdown 2024-05-11T12:14:17.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.503+0000 7f21a5df5740 1 -- 172.21.15.167:0/1161462827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9ad829010 msgr2=0x55f9ad838580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:17.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.503+0000 7f21a5df5740 1 --2- 172.21.15.167:0/1161462827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9ad829010 0x55f9ad838580 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f219403f040 tx=0x7f2194009750 comp rx=0 tx=0).stop 2024-05-11T12:14:17.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.503+0000 7f21a5df5740 1 -- 172.21.15.167:0/1161462827 shutdown_connections 2024-05-11T12:14:17.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.503+0000 7f21a5df5740 1 --2- 172.21.15.167:0/1161462827 >> [v2:172.21.15.167:6835/3484070468,v1:172.21.15.167:6836/3484070468] conn(0x55f9ad7daa60 0x55f9ad838ac0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:17.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.503+0000 7f21a5df5740 1 --2- 172.21.15.167:0/1161462827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9ad829010 0x55f9ad838580 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:17.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.503+0000 7f21a5df5740 1 --2- 172.21.15.167:0/1161462827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9ad81d630 0x55f9ad89db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:17.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.503+0000 7f21a5df5740 1 --2- 172.21.15.167:0/1161462827 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55f9ad80b770 0x55f9ad81c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:17.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.503+0000 7f21a5df5740 1 -- 172.21.15.167:0/1161462827 >> 172.21.15.167:0/1161462827 conn(0x55f9ad7e1450 msgr2=0x55f9ad809750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:17.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.503+0000 7f2185600640 0 client.6495 ms_handle_reset on v2:172.21.15.167:6835/3484070468 2024-05-11T12:14:17.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.503+0000 7f21a5df5740 1 -- 172.21.15.167:0/1161462827 shutdown_connections 2024-05-11T12:14:17.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.503+0000 7f21a5df5740 1 -- 172.21.15.167:0/1161462827 wait complete. 2024-05-11T12:14:17.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.503+0000 7f21a5df5740 20 client.6495 trim_cache size 0 max 16384 2024-05-11T12:14:17.508 INFO:teuthology.orchestra.run.smithi167.stdout:[] 2024-05-11T12:14:17.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.504+0000 7f21a3000640 1 -- 172.21.15.167:0/1727336549 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f217c0762d0 msgr2=0x7f217c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:17.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.504+0000 7f21a3000640 1 --2- 172.21.15.167:0/1727336549 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f217c0762d0 0x7f217c078790 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f218c003d90 tx=0x7f218c009040 comp rx=0 tx=0).stop 2024-05-11T12:14:17.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.504+0000 7f21a3000640 1 -- 172.21.15.167:0/1727336549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f219c161380 msgr2=0x7f219c0ab600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:17.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.504+0000 7f21a3000640 1 --2- 172.21.15.167:0/1727336549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f219c161380 0x7f219c0ab600 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f219402ac50 tx=0x7f2194004a40 comp rx=0 tx=0).stop 2024-05-11T12:14:17.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.504+0000 7f21a3000640 1 -- 172.21.15.167:0/1727336549 shutdown_connections 2024-05-11T12:14:17.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.504+0000 7f21a3000640 1 --2- 172.21.15.167:0/1727336549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f219c161380 0x7f219c0ab600 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:17.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.504+0000 7f21a3000640 1 --2- 172.21.15.167:0/1727336549 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f219c15ac80 0x7f219c0aac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:17.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.504+0000 7f21a3000640 1 --2- 172.21.15.167:0/1727336549 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f217c0762d0 0x7f217c078790 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:17.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.504+0000 7f21a3000640 1 --2- 172.21.15.167:0/1727336549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f219c074260 0x7f219c0cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:17.510 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.504+0000 7f21a3000640 1 -- 172.21.15.167:0/1727336549 >> 172.21.15.167:0/1727336549 conn(0x7f219c06c450 msgr2=0x7f219c06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:17.510 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.504+0000 7f21a3000640 1 -- 172.21.15.167:0/1727336549 shutdown_connections 2024-05-11T12:14:17.510 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:17.504+0000 7f21a3000640 1 -- 172.21.15.167:0/1727336549 wait complete. 2024-05-11T12:14:17.522 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-05-11T12:14:17.730 INFO:teuthology.orchestra.run.smithi167.stdout:[] 2024-05-11T12:14:17.730 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2024-05-11T12:14:17.730 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-11T12:14:17.730 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-11T12:14:17.827 DEBUG:teuthology.orchestra.run.smithi204:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-11T12:14:17.935 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-11T12:14:17.935 INFO:tasks.cephfs.mount:Terminating background process 2024-05-11T12:14:17.936 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:14:17.936 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:14:17.994 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-11T12:14:17.994 INFO:tasks.cephfs.mount:Terminating background process 2024-05-11T12:14:17.995 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:14:17.995 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:14:18.055 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052713,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-11T12:14:18.055 INFO:tasks.cephfs.mount:Terminating background process 2024-05-11T12:14:18.055 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:14:18.055 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:14:18.113 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-11T12:14:18.114 INFO:tasks.cephfs.mount:Terminating background process 2024-05-11T12:14:18.114 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:14:18.114 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:14:18.169 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052713,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-11T12:14:18.169 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:14:18.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.378+0000 7f285fe00640 1 Processor -- start 2024-05-11T12:14:18.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.378+0000 7f285fe00640 1 -- start start 2024-05-11T12:14:18.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.379+0000 7f285fe00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f28601458f0 0x7f286006d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:18.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.379+0000 7f285fe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28600b5220 0x7f28600b56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:18.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.380+0000 7f285fe00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2860071f30 con 0x7f28600b3a80 2024-05-11T12:14:18.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.380+0000 7f285fe00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f28600b5bf0 con 0x7f28601458f0 2024-05-11T12:14:18.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.380+0000 7f285fe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f286006db80 con 0x7f28600b5220 2024-05-11T12:14:18.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.380+0000 7f285e000640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f28600b3a80 0x7f28600b4970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:46500/0 (socket says 172.21.15.167:46500) 2024-05-11T12:14:18.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.380+0000 7f285e000640 1 -- 172.21.15.167:0/1416064478 learned_addr learned my addr 172.21.15.167:0/1416064478 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:14:18.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.380+0000 7f285ea00640 1 --2- 172.21.15.167:0/1416064478 >> v2:172.21.15.204:3300/0 conn(0x7f28601458f0 0x7f286006d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:18.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.380+0000 7f285f400640 1 --2- 172.21.15.167:0/1416064478 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28600b5220 0x7f28600b56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:18.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.380+0000 7f285d600640 1 -- 172.21.15.167:0/1416064478 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 700330696 0 0) 0x7f2860071f30 con 0x7f28600b3a80 2024-05-11T12:14:18.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.380+0000 7f285d600640 1 -- 172.21.15.167:0/1416064478 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2844005730 con 0x7f28600b3a80 2024-05-11T12:14:18.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.381+0000 7f285d600640 1 -- 172.21.15.167:0/1416064478 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1498183697 0 0) 0x7f2844005730 con 0x7f28600b3a80 2024-05-11T12:14:18.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.381+0000 7f285d600640 1 -- 172.21.15.167:0/1416064478 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28600b5220 msgr2=0x7f28600b56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:18.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.381+0000 7f285d600640 1 --2- 172.21.15.167:0/1416064478 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28600b5220 0x7f28600b56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:18.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.381+0000 7f285d600640 1 -- 172.21.15.167:0/1416064478 >> v2:172.21.15.204:3300/0 conn(0x7f28601458f0 msgr2=0x7f286006d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:18.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.381+0000 7f285d600640 1 --2- 172.21.15.167:0/1416064478 >> v2:172.21.15.204:3300/0 conn(0x7f28601458f0 0x7f286006d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:18.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.381+0000 7f285d600640 1 -- 172.21.15.167:0/1416064478 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2860072210 con 0x7f28600b3a80 2024-05-11T12:14:18.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.381+0000 7f285d600640 1 -- 172.21.15.167:0/1416064478 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f284c0016f0 con 0x7f28600b3a80 2024-05-11T12:14:18.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.381+0000 7f285d600640 1 -- 172.21.15.167:0/1416064478 >> v1:172.21.15.167:6789/0 conn(0x7f28600b3a80 legacy=0x7f28600b4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:18.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.381+0000 7f285d600640 1 --2- 172.21.15.167:0/1416064478 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28600b5220 0x7f2844005c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:18.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.381+0000 7f285d600640 1 --2- 172.21.15.167:0/1416064478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2844006230 0x7f2844006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:18.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.381+0000 7f285d600640 1 --2- 172.21.15.167:0/1416064478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2844006f50 0x7f28440073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:18.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.381+0000 7f285d600640 1 -- 172.21.15.167:0/1416064478 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f28600b5bf0 con 0x7f2844006230 2024-05-11T12:14:18.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.381+0000 7f285d600640 1 -- 172.21.15.167:0/1416064478 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f284400dcb0 con 0x7f2844006f50 2024-05-11T12:14:18.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.381+0000 7f285d600640 1 -- 172.21.15.167:0/1416064478 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f284400deb0 con 0x7f28600b5220 2024-05-11T12:14:18.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.381+0000 7f285e000640 1 --2- 172.21.15.167:0/1416064478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2844006230 0x7f2844006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:18.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.382+0000 7f285ea00640 1 --2- 172.21.15.167:0/1416064478 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28600b5220 0x7f2844005c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:18.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.382+0000 7f285e000640 1 -- 172.21.15.167:0/1416064478 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28600b5220 msgr2=0x7f2844005c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:18.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.382+0000 7f285e000640 1 --2- 172.21.15.167:0/1416064478 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28600b5220 0x7f2844005c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:18.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.382+0000 7f285e000640 1 -- 172.21.15.167:0/1416064478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2844006f50 msgr2=0x7f28440073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:18.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.382+0000 7f285e000640 1 --2- 172.21.15.167:0/1416064478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2844006f50 0x7f28440073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:18.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.382+0000 7f285e000640 1 -- 172.21.15.167:0/1416064478 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2844005730 con 0x7f2844006230 2024-05-11T12:14:18.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.383+0000 7f285e000640 1 --2- 172.21.15.167:0/1416064478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2844006230 0x7f2844006a10 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f284c00c210 tx=0x7f2844007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e448bda33614f66a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:18.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.383+0000 7f285d600640 1 -- 172.21.15.167:0/1416064478 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c007ce0 con 0x7f2844006230 2024-05-11T12:14:18.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.383+0000 7f285d600640 1 -- 172.21.15.167:0/1416064478 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f284c005af0 con 0x7f2844006230 2024-05-11T12:14:18.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.383+0000 7f285d600640 1 -- 172.21.15.167:0/1416064478 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c007070 con 0x7f2844006230 2024-05-11T12:14:18.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.383+0000 7f285fe00640 1 -- 172.21.15.167:0/1416064478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2844006230 msgr2=0x7f2844006a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:18.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.383+0000 7f285fe00640 1 --2- 172.21.15.167:0/1416064478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2844006230 0x7f2844006a10 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f284c00c210 tx=0x7f2844007a40 comp rx=0 tx=0).stop 2024-05-11T12:14:18.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.383+0000 7f285f400640 1 -- 172.21.15.167:0/1416064478 reap_dead start 2024-05-11T12:14:18.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.384+0000 7f285fe00640 1 -- 172.21.15.167:0/1416064478 shutdown_connections 2024-05-11T12:14:18.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.384+0000 7f285fe00640 1 -- 172.21.15.167:0/1416064478 >> 172.21.15.167:0/1416064478 conn(0x7f28600b3460 msgr2=0x7f28600d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:18.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.384+0000 7f285fe00640 1 -- 172.21.15.167:0/1416064478 shutdown_connections 2024-05-11T12:14:18.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.384+0000 7f285fe00640 1 -- 172.21.15.167:0/1416064478 wait complete. 2024-05-11T12:14:18.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.385+0000 7f285fe00640 1 Processor -- start 2024-05-11T12:14:18.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.386+0000 7f285fe00640 1 -- start start 2024-05-11T12:14:18.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.387+0000 7f285fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2844006230 0x7f2860143ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:18.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.387+0000 7f285fe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f286015fd10 0x7f2860160190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:18.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.387+0000 7f285fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28601425e0 0x7f2860142a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:18.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.387+0000 7f285fe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2860072ab0 con 0x7f28601425e0 2024-05-11T12:14:18.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.387+0000 7f285fe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f28600727d0 con 0x7f2844006230 2024-05-11T12:14:18.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.387+0000 7f285fe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f28600b56b0 con 0x7f286015fd10 2024-05-11T12:14:18.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.387+0000 7f285f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28601425e0 0x7f2860142a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:18.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.387+0000 7f285f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28601425e0 0x7f2860142a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35778/0 (socket says 172.21.15.167:35778) 2024-05-11T12:14:18.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.387+0000 7f285f400640 1 -- 172.21.15.167:0/1271478510 learned_addr learned my addr 172.21.15.167:0/1271478510 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:18.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.387+0000 7f285ea00640 1 --2- 172.21.15.167:0/1271478510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2844006230 0x7f2860143ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:18.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.387+0000 7f285e000640 1 --2- 172.21.15.167:0/1271478510 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f286015fd10 0x7f2860160190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:18.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.387+0000 7f285f400640 1 -- 172.21.15.167:0/1271478510 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f286015fd10 msgr2=0x7f2860160190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:18.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.387+0000 7f285f400640 1 --2- 172.21.15.167:0/1271478510 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f286015fd10 0x7f2860160190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:18.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.387+0000 7f285f400640 1 -- 172.21.15.167:0/1271478510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2844006230 msgr2=0x7f2860143ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:18.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.387+0000 7f285f400640 1 --2- 172.21.15.167:0/1271478510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2844006230 0x7f2860143ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:18.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.388+0000 7f285f400640 1 -- 172.21.15.167:0/1271478510 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f286015c6f0 con 0x7f28601425e0 2024-05-11T12:14:18.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.388+0000 7f285f400640 1 --2- 172.21.15.167:0/1271478510 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28601425e0 0x7f2860142a60 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f2860074360 tx=0x7f285800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:18.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.388+0000 7f2856a00640 1 -- 172.21.15.167:0/1271478510 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285800ebc0 con 0x7f28601425e0 2024-05-11T12:14:18.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.388+0000 7f2856a00640 1 -- 172.21.15.167:0/1271478510 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f285800ed80 con 0x7f28601425e0 2024-05-11T12:14:18.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.388+0000 7f2856a00640 1 -- 172.21.15.167:0/1271478510 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2858003bd0 con 0x7f28601425e0 2024-05-11T12:14:18.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.388+0000 7f285fe00640 1 -- 172.21.15.167:0/1271478510 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f286015c9d0 con 0x7f28601425e0 2024-05-11T12:14:18.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.388+0000 7f285fe00640 1 -- 172.21.15.167:0/1271478510 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f286015cf90 con 0x7f28601425e0 2024-05-11T12:14:18.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.389+0000 7f285fe00640 1 -- 172.21.15.167:0/1271478510 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28601606d0 con 0x7f28601425e0 2024-05-11T12:14:18.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.391+0000 7f2856a00640 1 -- 172.21.15.167:0/1271478510 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f285801b020 con 0x7f28601425e0 2024-05-11T12:14:18.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.391+0000 7f2856a00640 1 --2- 172.21.15.167:0/1271478510 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f28400762d0 0x7f2840078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:18.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.392+0000 7f2856a00640 1 -- 172.21.15.167:0/1271478510 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7f28580974a0 con 0x7f28601425e0 2024-05-11T12:14:18.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.392+0000 7f285ea00640 1 --2- 172.21.15.167:0/1271478510 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f28400762d0 0x7f2840078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:18.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.392+0000 7f285ea00640 1 --2- 172.21.15.167:0/1271478510 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f28400762d0 0x7f2840078790 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2850009a90 tx=0x7f2850008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:18.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.396+0000 7f2856a00640 1 -- 172.21.15.167:0/1271478510 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2858060c60 con 0x7f28601425e0 2024-05-11T12:14:18.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.643+0000 7f285fe00640 1 -- 172.21.15.167:0/1271478510 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28600694b0 con 0x7f28601425e0 2024-05-11T12:14:18.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.644+0000 7f2856a00640 1 -- 172.21.15.167:0/1271478510 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5195 (secure 0 0 0) 0x7f2858060560 con 0x7f28601425e0 2024-05-11T12:14:18.644 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:18.644 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4922,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1154357925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1154357925},{"type":"v1","addr":"172.21.15.167:6837","nonce":1154357925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":5155,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1256314893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1256314893},{"type":"v1","addr":"172.21.15.204:6833","nonce":1256314893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":5158,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1578748665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1578748665},{"type":"v1","addr":"172.21.15.204:6835","nonce":1578748665}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:14:04.861998+0000","modified":"2024-05-11T12:14:07.926157+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6300":{"gid":6300,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6836/3484070468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3484070468},{"type":"v1","addr":"172.21.15.167:6836","nonce":3484070468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6300,"qdb_cluster":[6300]},"id":6}]} 2024-05-11T12:14:18.644 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 53 2024-05-11T12:14:18.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.646+0000 7f285fe00640 1 -- 172.21.15.167:0/1271478510 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f28400762d0 msgr2=0x7f2840078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:18.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.646+0000 7f285fe00640 1 --2- 172.21.15.167:0/1271478510 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f28400762d0 0x7f2840078790 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2850009a90 tx=0x7f2850008040 comp rx=0 tx=0).stop 2024-05-11T12:14:18.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.646+0000 7f285fe00640 1 -- 172.21.15.167:0/1271478510 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28601425e0 msgr2=0x7f2860142a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:18.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.646+0000 7f285fe00640 1 --2- 172.21.15.167:0/1271478510 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28601425e0 0x7f2860142a60 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f2860074360 tx=0x7f285800e4b0 comp rx=0 tx=0).stop 2024-05-11T12:14:18.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.647+0000 7f285fe00640 1 -- 172.21.15.167:0/1271478510 shutdown_connections 2024-05-11T12:14:18.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.647+0000 7f285fe00640 1 --2- 172.21.15.167:0/1271478510 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28601425e0 0x7f2860142a60 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:18.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.647+0000 7f285fe00640 1 --2- 172.21.15.167:0/1271478510 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f286015fd10 0x7f2860160190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:18.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.647+0000 7f285fe00640 1 --2- 172.21.15.167:0/1271478510 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f28400762d0 0x7f2840078790 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:18.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.647+0000 7f285fe00640 1 --2- 172.21.15.167:0/1271478510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2844006230 0x7f2860143ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:18.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.647+0000 7f285fe00640 1 -- 172.21.15.167:0/1271478510 >> 172.21.15.167:0/1271478510 conn(0x7f28600b3460 msgr2=0x7f286006d640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:18.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.647+0000 7f285fe00640 1 -- 172.21.15.167:0/1271478510 shutdown_connections 2024-05-11T12:14:18.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.647+0000 7f285fe00640 1 -- 172.21.15.167:0/1271478510 wait complete. 2024-05-11T12:14:18.667 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:14:18.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.870+0000 7fa89c200640 1 Processor -- start 2024-05-11T12:14:18.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.871+0000 7fa89c200640 1 -- start start 2024-05-11T12:14:18.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.872+0000 7fa89c200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa89413f290 0x7fa8940735c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:18.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.872+0000 7fa89c200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa8940b50a0 0x7fa894073b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:18.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.872+0000 7fa89c200640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa89415fb90 con 0x7fa89413eeb0 2024-05-11T12:14:18.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.872+0000 7fa89c200640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fa89415d5b0 con 0x7fa8940b50a0 2024-05-11T12:14:18.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.872+0000 7fa89c200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fa89415fe70 con 0x7fa89413f290 2024-05-11T12:14:18.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.873+0000 7fa899800640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fa89413eeb0 0x7fa89406f480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:39256/0 (socket says 172.21.15.167:39256) 2024-05-11T12:14:18.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.873+0000 7fa899800640 1 -- 172.21.15.167:0/2212174237 learned_addr learned my addr 172.21.15.167:0/2212174237 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:14:18.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.873+0000 7fa89a200640 1 --2- 172.21.15.167:0/2212174237 >> v2:172.21.15.167:3300/0 conn(0x7fa8940b50a0 0x7fa894073b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:18.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.873+0000 7fa898e00640 1 --2- 172.21.15.167:0/2212174237 >> v2:172.21.15.204:3300/0 conn(0x7fa89413f290 0x7fa8940735c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:18.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.873+0000 7fa893e00640 1 -- 172.21.15.167:0/2212174237 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2587442671 0 0) 0x7fa89415fb90 con 0x7fa89413eeb0 2024-05-11T12:14:18.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.873+0000 7fa893e00640 1 -- 172.21.15.167:0/2212174237 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa87c005730 con 0x7fa89413eeb0 2024-05-11T12:14:18.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.873+0000 7fa89a200640 1 -- 172.21.15.167:0/2212174237 >> v2:172.21.15.204:3300/0 conn(0x7fa89413f290 msgr2=0x7fa8940735c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:18.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.873+0000 7fa89a200640 1 --2- 172.21.15.167:0/2212174237 >> v2:172.21.15.204:3300/0 conn(0x7fa89413f290 0x7fa8940735c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:18.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.873+0000 7fa89a200640 1 -- 172.21.15.167:0/2212174237 >> v1:172.21.15.167:6789/0 conn(0x7fa89413eeb0 legacy=0x7fa89406f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:18.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.873+0000 7fa89a200640 1 -- 172.21.15.167:0/2212174237 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa894160150 con 0x7fa8940b50a0 2024-05-11T12:14:18.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.874+0000 7fa89a200640 1 --2- 172.21.15.167:0/2212174237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8940b50a0 0x7fa894073b00 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fa894074520 tx=0x7fa88c02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6149b58cd768cc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:18.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.875+0000 7fa893e00640 1 -- 172.21.15.167:0/2212174237 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa88c007740 con 0x7fa8940b50a0 2024-05-11T12:14:18.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.875+0000 7fa893e00640 1 -- 172.21.15.167:0/2212174237 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa88c0078e0 con 0x7fa8940b50a0 2024-05-11T12:14:18.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.875+0000 7fa893e00640 1 -- 172.21.15.167:0/2212174237 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa88c007cb0 con 0x7fa8940b50a0 2024-05-11T12:14:18.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.875+0000 7fa89c200640 1 -- 172.21.15.167:0/2212174237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8940b50a0 msgr2=0x7fa894073b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:18.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.875+0000 7fa89c200640 1 --2- 172.21.15.167:0/2212174237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8940b50a0 0x7fa894073b00 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fa894074520 tx=0x7fa88c02a5f0 comp rx=0 tx=0).stop 2024-05-11T12:14:18.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.875+0000 7fa89c200640 1 -- 172.21.15.167:0/2212174237 shutdown_connections 2024-05-11T12:14:18.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.875+0000 7fa89c200640 1 --2- 172.21.15.167:0/2212174237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8940b50a0 0x7fa894073b00 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:18.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.875+0000 7fa89c200640 1 --2- 172.21.15.167:0/2212174237 >> v2:172.21.15.204:3300/0 conn(0x7fa89413f290 0x7fa8940735c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:18.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.875+0000 7fa89c200640 1 -- 172.21.15.167:0/2212174237 >> 172.21.15.167:0/2212174237 conn(0x7fa8940cd8f0 msgr2=0x7fa8940cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:18.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.875+0000 7fa89c200640 1 -- 172.21.15.167:0/2212174237 shutdown_connections 2024-05-11T12:14:18.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.876+0000 7fa89c200640 1 -- 172.21.15.167:0/2212174237 wait complete. 2024-05-11T12:14:18.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.877+0000 7fa89c200640 1 Processor -- start 2024-05-11T12:14:18.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.877+0000 7fa89c200640 1 -- start start 2024-05-11T12:14:18.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.878+0000 7fa89c200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8940b50a0 0x7fa894138730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:18.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.878+0000 7fa89c200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa89413eeb0 0x7fa8940cfab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:18.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.878+0000 7fa89c200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa89413f290 0x7fa8940b1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:18.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.878+0000 7fa89c200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa894074040 con 0x7fa89413f290 2024-05-11T12:14:18.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.878+0000 7fa89c200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa894160710 con 0x7fa89413eeb0 2024-05-11T12:14:18.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.878+0000 7fa89c200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa89406fb90 con 0x7fa8940b50a0 2024-05-11T12:14:18.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.879+0000 7fa89a200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa89413f290 0x7fa8940b1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:18.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.879+0000 7fa89a200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa89413f290 0x7fa8940b1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:51420/0 (socket says 172.21.15.167:51420) 2024-05-11T12:14:18.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.879+0000 7fa89a200640 1 -- 172.21.15.167:0/3315464727 learned_addr learned my addr 172.21.15.167:0/3315464727 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:18.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.879+0000 7fa89a200640 1 -- 172.21.15.167:0/3315464727 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8940b50a0 msgr2=0x7fa894138730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:18.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.879+0000 7fa899800640 1 --2- 172.21.15.167:0/3315464727 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8940b50a0 0x7fa894138730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:18.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.879+0000 7fa898e00640 1 --2- 172.21.15.167:0/3315464727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa89413eeb0 0x7fa8940cfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:18.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.879+0000 7fa89a200640 1 --2- 172.21.15.167:0/3315464727 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8940b50a0 0x7fa894138730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:18.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.879+0000 7fa89a200640 1 -- 172.21.15.167:0/3315464727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa89413eeb0 msgr2=0x7fa8940cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:18.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.879+0000 7fa89a200640 1 --2- 172.21.15.167:0/3315464727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa89413eeb0 0x7fa8940cfab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:18.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.879+0000 7fa89a200640 1 -- 172.21.15.167:0/3315464727 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa894152fb0 con 0x7fa89413f290 2024-05-11T12:14:18.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.879+0000 7fa89a200640 1 --2- 172.21.15.167:0/3315464727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa89413f290 0x7fa8940b1e70 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fa88c009e50 tx=0x7fa88c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:18.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.880+0000 7fa891600640 1 -- 172.21.15.167:0/3315464727 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa88c040070 con 0x7fa89413f290 2024-05-11T12:14:18.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.880+0000 7fa891600640 1 -- 172.21.15.167:0/3315464727 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa88c031bf0 con 0x7fa89413f290 2024-05-11T12:14:18.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.880+0000 7fa891600640 1 -- 172.21.15.167:0/3315464727 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa88c009450 con 0x7fa89413f290 2024-05-11T12:14:18.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.880+0000 7fa89c200640 1 -- 172.21.15.167:0/3315464727 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8940cecf0 con 0x7fa89413f290 2024-05-11T12:14:18.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.880+0000 7fa89c200640 1 -- 172.21.15.167:0/3315464727 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8940cf2b0 con 0x7fa89413f290 2024-05-11T12:14:18.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.881+0000 7fa89c200640 1 -- 172.21.15.167:0/3315464727 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa894069590 con 0x7fa89413f290 2024-05-11T12:14:18.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.882+0000 7fa891600640 1 -- 172.21.15.167:0/3315464727 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa88c039040 con 0x7fa89413f290 2024-05-11T12:14:18.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.883+0000 7fa891600640 1 --2- 172.21.15.167:0/3315464727 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa8740762d0 0x7fa874078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:18.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.883+0000 7fa891600640 1 -- 172.21.15.167:0/3315464727 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7fa88c0b9020 con 0x7fa89413f290 2024-05-11T12:14:18.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.883+0000 7fa899800640 1 --2- 172.21.15.167:0/3315464727 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa8740762d0 0x7fa874078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:18.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.884+0000 7fa899800640 1 --2- 172.21.15.167:0/3315464727 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa8740762d0 0x7fa874078790 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa888001220 tx=0x7fa888009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:18.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:18.888+0000 7fa891600640 1 -- 172.21.15.167:0/3315464727 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa88c0827f0 con 0x7fa89413f290 2024-05-11T12:14:19.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.121+0000 7fa89c200640 1 -- 172.21.15.167:0/3315464727 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa894155f60 con 0x7fa89413f290 2024-05-11T12:14:19.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.123+0000 7fa891600640 1 -- 172.21.15.167:0/3315464727 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5195 (secure 0 0 0) 0x7fa88c0820f0 con 0x7fa89413f290 2024-05-11T12:14:19.123 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:19.123 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4922,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1154357925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1154357925},{"type":"v1","addr":"172.21.15.167:6837","nonce":1154357925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":5155,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1256314893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1256314893},{"type":"v1","addr":"172.21.15.204:6833","nonce":1256314893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":5158,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1578748665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1578748665},{"type":"v1","addr":"172.21.15.204:6835","nonce":1578748665}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:14:04.861998+0000","modified":"2024-05-11T12:14:07.926157+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6300":{"gid":6300,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6836/3484070468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3484070468},{"type":"v1","addr":"172.21.15.167:6836","nonce":3484070468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6300,"qdb_cluster":[6300]},"id":6}]} 2024-05-11T12:14:19.123 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 53 2024-05-11T12:14:19.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.125+0000 7fa89c200640 1 -- 172.21.15.167:0/3315464727 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa8740762d0 msgr2=0x7fa874078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:19.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.125+0000 7fa89c200640 1 --2- 172.21.15.167:0/3315464727 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa8740762d0 0x7fa874078790 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa888001220 tx=0x7fa888009040 comp rx=0 tx=0).stop 2024-05-11T12:14:19.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.125+0000 7fa89c200640 1 -- 172.21.15.167:0/3315464727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa89413f290 msgr2=0x7fa8940b1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:19.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.125+0000 7fa89c200640 1 --2- 172.21.15.167:0/3315464727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa89413f290 0x7fa8940b1e70 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fa88c009e50 tx=0x7fa88c004a40 comp rx=0 tx=0).stop 2024-05-11T12:14:19.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.125+0000 7fa89c200640 1 -- 172.21.15.167:0/3315464727 shutdown_connections 2024-05-11T12:14:19.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.125+0000 7fa89c200640 1 --2- 172.21.15.167:0/3315464727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa89413f290 0x7fa8940b1e70 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:19.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.125+0000 7fa89c200640 1 --2- 172.21.15.167:0/3315464727 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa8740762d0 0x7fa874078790 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:19.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.125+0000 7fa89c200640 1 --2- 172.21.15.167:0/3315464727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa89413eeb0 0x7fa8940cfab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:19.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.125+0000 7fa89c200640 1 --2- 172.21.15.167:0/3315464727 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8940b50a0 0x7fa894138730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:19.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.126+0000 7fa89c200640 1 -- 172.21.15.167:0/3315464727 >> 172.21.15.167:0/3315464727 conn(0x7fa8940cd8f0 msgr2=0x7fa89406eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:19.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.126+0000 7fa89c200640 1 -- 172.21.15.167:0/3315464727 shutdown_connections 2024-05-11T12:14:19.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.126+0000 7fa89c200640 1 -- 172.21.15.167:0/3315464727 wait complete. 2024-05-11T12:14:19.143 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:14:19.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.335+0000 7fb369e00640 1 Processor -- start 2024-05-11T12:14:19.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.335+0000 7fb369e00640 1 -- start start 2024-05-11T12:14:19.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.337+0000 7fb369e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb364145b00 0x7fb36406f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:19.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.337+0000 7fb369e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb364146770 0x7fb3640ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:19.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.337+0000 7fb369e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3640cef10 0x7fb36406c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:19.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.337+0000 7fb369e00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fb36406fbc0 con 0x7fb364146770 2024-05-11T12:14:19.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.337+0000 7fb369e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fb3640b4520 con 0x7fb364145b00 2024-05-11T12:14:19.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.337+0000 7fb369e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb3640b4800 con 0x7fb3640cef10 2024-05-11T12:14:19.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.337+0000 7fb35be00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb364146770 0x7fb3640ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:19.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.337+0000 7fb35be00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb364146770 0x7fb3640ce9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:51436/0 (socket says 172.21.15.167:51436) 2024-05-11T12:14:19.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.337+0000 7fb35be00640 1 -- 172.21.15.167:0/3276056198 learned_addr learned my addr 172.21.15.167:0/3276056198 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:19.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.338+0000 7fb369400640 1 --2- 172.21.15.167:0/3276056198 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3640cef10 0x7fb36406c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:19.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.338+0000 7fb368a00640 1 --2- 172.21.15.167:0/3276056198 >> v2:172.21.15.204:3300/0 conn(0x7fb364145b00 0x7fb36406f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:19.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.338+0000 7fb35be00640 1 -- 172.21.15.167:0/3276056198 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3640cef10 msgr2=0x7fb36406c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:19.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.338+0000 7fb35be00640 1 --2- 172.21.15.167:0/3276056198 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3640cef10 0x7fb36406c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:19.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.338+0000 7fb35be00640 1 -- 172.21.15.167:0/3276056198 >> v2:172.21.15.204:3300/0 conn(0x7fb364145b00 msgr2=0x7fb36406f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:19.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.338+0000 7fb35be00640 1 --2- 172.21.15.167:0/3276056198 >> v2:172.21.15.204:3300/0 conn(0x7fb364145b00 0x7fb36406f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:19.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.338+0000 7fb35be00640 1 -- 172.21.15.167:0/3276056198 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3640b4ae0 con 0x7fb364146770 2024-05-11T12:14:19.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.339+0000 7fb35be00640 1 --2- 172.21.15.167:0/3276056198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb364146770 0x7fb3640ce9d0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fb3640b4730 tx=0x7fb35402a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d8ac256cb907b2b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:19.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.339+0000 7fb363e00640 1 -- 172.21.15.167:0/3276056198 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb354037070 con 0x7fb364146770 2024-05-11T12:14:19.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.339+0000 7fb363e00640 1 -- 172.21.15.167:0/3276056198 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb354032440 con 0x7fb364146770 2024-05-11T12:14:19.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.339+0000 7fb363e00640 1 -- 172.21.15.167:0/3276056198 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb354032810 con 0x7fb364146770 2024-05-11T12:14:19.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.340+0000 7fb369e00640 1 -- 172.21.15.167:0/3276056198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb364146770 msgr2=0x7fb3640ce9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:19.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.340+0000 7fb369e00640 1 --2- 172.21.15.167:0/3276056198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb364146770 0x7fb3640ce9d0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fb3640b4730 tx=0x7fb35402a130 comp rx=0 tx=0).stop 2024-05-11T12:14:19.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.340+0000 7fb369e00640 1 -- 172.21.15.167:0/3276056198 shutdown_connections 2024-05-11T12:14:19.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.340+0000 7fb369e00640 1 --2- 172.21.15.167:0/3276056198 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3640cef10 0x7fb36406c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:19.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.340+0000 7fb369e00640 1 --2- 172.21.15.167:0/3276056198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb364146770 0x7fb3640ce9d0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:19.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.340+0000 7fb369e00640 1 --2- 172.21.15.167:0/3276056198 >> v2:172.21.15.204:3300/0 conn(0x7fb364145b00 0x7fb36406f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:19.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.340+0000 7fb369e00640 1 -- 172.21.15.167:0/3276056198 >> 172.21.15.167:0/3276056198 conn(0x7fb3640b3460 msgr2=0x7fb3640d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:19.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.340+0000 7fb369e00640 1 -- 172.21.15.167:0/3276056198 shutdown_connections 2024-05-11T12:14:19.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.340+0000 7fb369e00640 1 -- 172.21.15.167:0/3276056198 wait complete. 2024-05-11T12:14:19.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.342+0000 7fb369e00640 1 Processor -- start 2024-05-11T12:14:19.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.342+0000 7fb369e00640 1 -- start start 2024-05-11T12:14:19.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.343+0000 7fb369e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3640cef10 0x7fb3641600c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:19.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.343+0000 7fb369e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb364145b00 0x7fb36413c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:19.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.343+0000 7fb369e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb364146770 0x7fb364072940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:19.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.343+0000 7fb369e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb36406f980 con 0x7fb364145b00 2024-05-11T12:14:19.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.343+0000 7fb369e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb36406f800 con 0x7fb364146770 2024-05-11T12:14:19.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.343+0000 7fb369e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb36406cba0 con 0x7fb3640cef10 2024-05-11T12:14:19.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.343+0000 7fb35be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb364145b00 0x7fb36413c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:19.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.343+0000 7fb35be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb364145b00 0x7fb36413c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:51450/0 (socket says 172.21.15.167:51450) 2024-05-11T12:14:19.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.343+0000 7fb35be00640 1 -- 172.21.15.167:0/2139761903 learned_addr learned my addr 172.21.15.167:0/2139761903 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:19.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.343+0000 7fb368a00640 1 --2- 172.21.15.167:0/2139761903 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3640cef10 0x7fb3641600c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:19.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.343+0000 7fb35be00640 1 -- 172.21.15.167:0/2139761903 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3640cef10 msgr2=0x7fb3641600c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:19.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.343+0000 7fb35be00640 1 --2- 172.21.15.167:0/2139761903 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3640cef10 0x7fb3641600c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:19.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.343+0000 7fb35be00640 1 -- 172.21.15.167:0/2139761903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb364146770 msgr2=0x7fb364072940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:19.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.343+0000 7fb35be00640 1 --2- 172.21.15.167:0/2139761903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb364146770 0x7fb364072940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:19.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.343+0000 7fb35be00640 1 -- 172.21.15.167:0/2139761903 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3641f4f90 con 0x7fb364145b00 2024-05-11T12:14:19.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.344+0000 7fb35be00640 1 --2- 172.21.15.167:0/2139761903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb364145b00 0x7fb36413c540 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fb354000c00 tx=0x7fb3540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:19.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.344+0000 7fb361600640 1 -- 172.21.15.167:0/2139761903 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb354041020 con 0x7fb364145b00 2024-05-11T12:14:19.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.344+0000 7fb361600640 1 -- 172.21.15.167:0/2139761903 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb354030800 con 0x7fb364145b00 2024-05-11T12:14:19.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.344+0000 7fb361600640 1 -- 172.21.15.167:0/2139761903 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb354037040 con 0x7fb364145b00 2024-05-11T12:14:19.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.344+0000 7fb369e00640 1 -- 172.21.15.167:0/2139761903 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb364143950 con 0x7fb364145b00 2024-05-11T12:14:19.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.344+0000 7fb369e00640 1 -- 172.21.15.167:0/2139761903 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb364073120 con 0x7fb364145b00 2024-05-11T12:14:19.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.345+0000 7fb369e00640 1 -- 172.21.15.167:0/2139761903 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3640694b0 con 0x7fb364145b00 2024-05-11T12:14:19.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.346+0000 7fb361600640 1 -- 172.21.15.167:0/2139761903 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb354044020 con 0x7fb364145b00 2024-05-11T12:14:19.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.347+0000 7fb361600640 1 --2- 172.21.15.167:0/2139761903 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb3400762d0 0x7fb340078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:19.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.347+0000 7fb361600640 1 -- 172.21.15.167:0/2139761903 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7fb3540b7ff0 con 0x7fb364145b00 2024-05-11T12:14:19.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.347+0000 7fb368a00640 1 --2- 172.21.15.167:0/2139761903 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb3400762d0 0x7fb340078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:19.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.348+0000 7fb368a00640 1 --2- 172.21.15.167:0/2139761903 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb3400762d0 0x7fb340078790 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb34c006fd0 tx=0x7fb34c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:19.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.351+0000 7fb361600640 1 -- 172.21.15.167:0/2139761903 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb35404b030 con 0x7fb364145b00 2024-05-11T12:14:19.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.546+0000 7fb369e00640 1 -- 172.21.15.167:0/2139761903 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb3640b5320 con 0x7fb364145b00 2024-05-11T12:14:19.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.547+0000 7fb361600640 1 -- 172.21.15.167:0/2139761903 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+16933 (secure 0 0 0) 0x7fb364143950 con 0x7fb364145b00 2024-05-11T12:14:19.548 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:19.548 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":86,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:14:06.410610+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:14:02.772030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-11T12:14:03.847508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:14:19.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.550+0000 7fb369e00640 1 -- 172.21.15.167:0/2139761903 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb3400762d0 msgr2=0x7fb340078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:19.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.550+0000 7fb369e00640 1 --2- 172.21.15.167:0/2139761903 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb3400762d0 0x7fb340078790 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb34c006fd0 tx=0x7fb34c008040 comp rx=0 tx=0).stop 2024-05-11T12:14:19.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.550+0000 7fb369e00640 1 -- 172.21.15.167:0/2139761903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb364145b00 msgr2=0x7fb36413c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:19.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.550+0000 7fb369e00640 1 --2- 172.21.15.167:0/2139761903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb364145b00 0x7fb36413c540 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fb354000c00 tx=0x7fb3540046c0 comp rx=0 tx=0).stop 2024-05-11T12:14:19.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.550+0000 7fb369e00640 1 -- 172.21.15.167:0/2139761903 shutdown_connections 2024-05-11T12:14:19.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.550+0000 7fb369e00640 1 --2- 172.21.15.167:0/2139761903 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb3400762d0 0x7fb340078790 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:19.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.550+0000 7fb369e00640 1 --2- 172.21.15.167:0/2139761903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb364146770 0x7fb364072940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:19.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.550+0000 7fb369e00640 1 --2- 172.21.15.167:0/2139761903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb364145b00 0x7fb36413c540 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:19.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.550+0000 7fb369e00640 1 --2- 172.21.15.167:0/2139761903 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3640cef10 0x7fb3641600c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:19.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.550+0000 7fb369e00640 1 -- 172.21.15.167:0/2139761903 >> 172.21.15.167:0/2139761903 conn(0x7fb3640b3460 msgr2=0x7fb3640d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:19.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.550+0000 7fb369e00640 1 -- 172.21.15.167:0/2139761903 shutdown_connections 2024-05-11T12:14:19.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.550+0000 7fb369e00640 1 -- 172.21.15.167:0/2139761903 wait complete. 2024-05-11T12:14:19.568 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-11T12:14:19.568 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:14:19.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.764+0000 7f7323200640 1 Processor -- start 2024-05-11T12:14:19.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.765+0000 7f7323200640 1 -- start start 2024-05-11T12:14:19.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.766+0000 7f7323200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f73240d42c0 0x7f73240d0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:19.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.766+0000 7f7323200640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73240cf990 con 0x7f732415f460 2024-05-11T12:14:19.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.766+0000 7f7323200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73240cfc70 con 0x7f732406cc40 2024-05-11T12:14:19.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.766+0000 7f7323200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f73240cff50 con 0x7f73240d42c0 2024-05-11T12:14:19.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.766+0000 7f7321e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f732415f460 0x7f73240b83a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:39268/0 (socket says 172.21.15.167:39268) 2024-05-11T12:14:19.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.766+0000 7f7321e00640 1 -- 172.21.15.167:0/3559145560 learned_addr learned my addr 172.21.15.167:0/3559145560 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:14:19.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.767+0000 7f7320a00640 1 -- 172.21.15.167:0/3559145560 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2183478022 0 0) 0x7f73240cf990 con 0x7f732415f460 2024-05-11T12:14:19.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.767+0000 7f7320a00640 1 -- 172.21.15.167:0/3559145560 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7308003520 con 0x7f732415f460 2024-05-11T12:14:19.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.767+0000 7f7322800640 1 --2- 172.21.15.167:0/3559145560 >> v2:172.21.15.204:3300/0 conn(0x7f73240d42c0 0x7f73240d0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:19.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.767+0000 7f7320a00640 1 -- 172.21.15.167:0/3559145560 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1608972748 0 0) 0x7f7308003520 con 0x7f732415f460 2024-05-11T12:14:19.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.767+0000 7f7320a00640 1 -- 172.21.15.167:0/3559145560 >> v2:172.21.15.204:3300/0 conn(0x7f73240d42c0 msgr2=0x7f73240d0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:19.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.767+0000 7f7320a00640 1 --2- 172.21.15.167:0/3559145560 >> v2:172.21.15.204:3300/0 conn(0x7f73240d42c0 0x7f73240d0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:19.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.767+0000 7f7320a00640 1 -- 172.21.15.167:0/3559145560 >> v1:172.21.15.204:6789/0 conn(0x7f732406cc40 legacy=0x7f73240d3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:19.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.767+0000 7f7320a00640 1 -- 172.21.15.167:0/3559145560 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73240d0230 con 0x7f732415f460 2024-05-11T12:14:19.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.767+0000 7f7320a00640 1 -- 172.21.15.167:0/3559145560 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f7314001710 con 0x7f732415f460 2024-05-11T12:14:19.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.767+0000 7f7320a00640 1 -- 172.21.15.167:0/3559145560 >> v1:172.21.15.167:6789/0 conn(0x7f732415f460 legacy=0x7f73240b83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:19.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.767+0000 7f7320a00640 1 --2- 172.21.15.167:0/3559145560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7308003a50 0x7f7308003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:19.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.767+0000 7f7320a00640 1 --2- 172.21.15.167:0/3559145560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7308004800 0x7f7308004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:19.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.767+0000 7f7320a00640 1 --2- 172.21.15.167:0/3559145560 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f73080052e0 0x7f730800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:19.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.767+0000 7f7320a00640 1 -- 172.21.15.167:0/3559145560 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f73240cff50 con 0x7f7308004800 2024-05-11T12:14:19.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.767+0000 7f7320a00640 1 -- 172.21.15.167:0/3559145560 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f730800c1b0 con 0x7f7308003a50 2024-05-11T12:14:19.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.767+0000 7f7320a00640 1 -- 172.21.15.167:0/3559145560 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f730800c3b0 con 0x7f73080052e0 2024-05-11T12:14:19.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.768+0000 7f7321400640 1 --2- 172.21.15.167:0/3559145560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7308004800 0x7f7308004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:19.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.768+0000 7f7321e00640 1 --2- 172.21.15.167:0/3559145560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7308003a50 0x7f7308003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:19.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.768+0000 7f7321400640 1 -- 172.21.15.167:0/3559145560 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f73080052e0 msgr2=0x7f730800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:19.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.768+0000 7f7321400640 1 --2- 172.21.15.167:0/3559145560 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f73080052e0 0x7f730800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:19.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.768+0000 7f7321400640 1 -- 172.21.15.167:0/3559145560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7308003a50 msgr2=0x7f7308003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:19.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.768+0000 7f7321400640 1 --2- 172.21.15.167:0/3559145560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7308003a50 0x7f7308003e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:19.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.768+0000 7f7321400640 1 -- 172.21.15.167:0/3559145560 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7308003520 con 0x7f7308004800 2024-05-11T12:14:19.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.768+0000 7f7322800640 1 -- 172.21.15.167:0/3559145560 reap_dead start 2024-05-11T12:14:19.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.769+0000 7f7321400640 1 --2- 172.21.15.167:0/3559145560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7308004800 0x7f7308004c80 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f7310001760 tx=0x7f7308005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c92d77f66b8b2f9a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:19.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.769+0000 7f7320a00640 1 -- 172.21.15.167:0/3559145560 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73100013b0 con 0x7f7308004800 2024-05-11T12:14:19.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.769+0000 7f7320a00640 1 -- 172.21.15.167:0/3559145560 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73100052d0 con 0x7f7308004800 2024-05-11T12:14:19.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.769+0000 7f7320a00640 1 -- 172.21.15.167:0/3559145560 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731002aa30 con 0x7f7308004800 2024-05-11T12:14:19.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.770+0000 7f7323200640 1 -- 172.21.15.167:0/3559145560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7308004800 msgr2=0x7f7308004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:19.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.770+0000 7f7323200640 1 --2- 172.21.15.167:0/3559145560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7308004800 0x7f7308004c80 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f7310001760 tx=0x7f7308005830 comp rx=0 tx=0).stop 2024-05-11T12:14:19.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.770+0000 7f7323200640 1 -- 172.21.15.167:0/3559145560 shutdown_connections 2024-05-11T12:14:19.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.770+0000 7f7323200640 1 --2- 172.21.15.167:0/3559145560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7308004800 0x7f7308004c80 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:19.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.770+0000 7f7323200640 1 -- 172.21.15.167:0/3559145560 >> 172.21.15.167:0/3559145560 conn(0x7f73240d1a50 msgr2=0x7f73240d1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:19.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.770+0000 7f7323200640 1 -- 172.21.15.167:0/3559145560 shutdown_connections 2024-05-11T12:14:19.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.770+0000 7f7323200640 1 -- 172.21.15.167:0/3559145560 wait complete. 2024-05-11T12:14:19.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.772+0000 7f7323200640 1 Processor -- start 2024-05-11T12:14:19.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.772+0000 7f7323200640 1 -- start start 2024-05-11T12:14:19.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.773+0000 7f7323200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7308004800 0x7f73240cb270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:19.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.773+0000 7f7323200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7324161e80 0x7f7324162300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:19.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.773+0000 7f7323200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7324154630 0x7f7324154a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:19.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.773+0000 7f7323200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f732406db20 con 0x7f7324154630 2024-05-11T12:14:19.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.773+0000 7f7323200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f73240cf620 con 0x7f7308004800 2024-05-11T12:14:19.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.773+0000 7f7323200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f732406dca0 con 0x7f7324161e80 2024-05-11T12:14:19.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.773+0000 7f7322800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7324154630 0x7f7324154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:19.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.773+0000 7f7322800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7324154630 0x7f7324154a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:51466/0 (socket says 172.21.15.167:51466) 2024-05-11T12:14:19.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.773+0000 7f7322800640 1 -- 172.21.15.167:0/1101952912 learned_addr learned my addr 172.21.15.167:0/1101952912 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:19.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.773+0000 7f7321400640 1 --2- 172.21.15.167:0/1101952912 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7324161e80 0x7f7324162300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:19.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.773+0000 7f7322800640 1 -- 172.21.15.167:0/1101952912 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7324161e80 msgr2=0x7f7324162300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:19.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.774+0000 7f7322800640 1 --2- 172.21.15.167:0/1101952912 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7324161e80 0x7f7324162300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:19.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.774+0000 7f7321e00640 1 --2- 172.21.15.167:0/1101952912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7308004800 0x7f73240cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:19.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.774+0000 7f7322800640 1 -- 172.21.15.167:0/1101952912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7308004800 msgr2=0x7f73240cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:19.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.774+0000 7f7322800640 1 --2- 172.21.15.167:0/1101952912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7308004800 0x7f73240cb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:19.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.774+0000 7f7322800640 1 -- 172.21.15.167:0/1101952912 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7324160980 con 0x7f7324154630 2024-05-11T12:14:19.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.774+0000 7f7322800640 1 --2- 172.21.15.167:0/1101952912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7324154630 0x7f7324154a90 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f73240bb110 tx=0x7f731c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:19.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.774+0000 7f731a000640 1 -- 172.21.15.167:0/1101952912 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c00ed90 con 0x7f7324154630 2024-05-11T12:14:19.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.774+0000 7f731a000640 1 -- 172.21.15.167:0/1101952912 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f731c019070 con 0x7f7324154630 2024-05-11T12:14:19.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.774+0000 7f731a000640 1 -- 172.21.15.167:0/1101952912 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c014660 con 0x7f7324154630 2024-05-11T12:14:19.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.774+0000 7f7323200640 1 -- 172.21.15.167:0/1101952912 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73241613a0 con 0x7f7324154630 2024-05-11T12:14:19.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.774+0000 7f7323200640 1 -- 172.21.15.167:0/1101952912 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7324161960 con 0x7f7324154630 2024-05-11T12:14:19.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.776+0000 7f7323200640 1 -- 172.21.15.167:0/1101952912 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73240694b0 con 0x7f7324154630 2024-05-11T12:14:19.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.777+0000 7f731a000640 1 -- 172.21.15.167:0/1101952912 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f731c014800 con 0x7f7324154630 2024-05-11T12:14:19.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.778+0000 7f731a000640 1 --2- 172.21.15.167:0/1101952912 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f73040762d0 0x7f7304078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:19.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.778+0000 7f731a000640 1 -- 172.21.15.167:0/1101952912 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7f731c097df0 con 0x7f7324154630 2024-05-11T12:14:19.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.778+0000 7f7321e00640 1 --2- 172.21.15.167:0/1101952912 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f73040762d0 0x7f7304078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:19.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.779+0000 7f7321e00640 1 --2- 172.21.15.167:0/1101952912 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f73040762d0 0x7f7304078790 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f73240520f0 tx=0x7f7314001510 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:19.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:19.782+0000 7f731a000640 1 -- 172.21.15.167:0/1101952912 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f731c061490 con 0x7f7324154630 2024-05-11T12:14:20.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.030+0000 7f7323200640 1 -- 172.21.15.167:0/1101952912 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73240d4900 con 0x7f7324154630 2024-05-11T12:14:20.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.030+0000 7f731a000640 1 -- 172.21.15.167:0/1101952912 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5195 (secure 0 0 0) 0x7f731c060d90 con 0x7f7324154630 2024-05-11T12:14:20.031 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:20.031 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4922,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1154357925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1154357925},{"type":"v1","addr":"172.21.15.167:6837","nonce":1154357925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":5155,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1256314893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1256314893},{"type":"v1","addr":"172.21.15.204:6833","nonce":1256314893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":5158,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1578748665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1578748665},{"type":"v1","addr":"172.21.15.204:6835","nonce":1578748665}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:14:04.861998+0000","modified":"2024-05-11T12:14:07.926157+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6300":{"gid":6300,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6836/3484070468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3484070468},{"type":"v1","addr":"172.21.15.167:6836","nonce":3484070468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6300,"qdb_cluster":[6300]},"id":6}]} 2024-05-11T12:14:20.031 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 53 2024-05-11T12:14:20.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.033+0000 7f7323200640 1 -- 172.21.15.167:0/1101952912 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f73040762d0 msgr2=0x7f7304078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:20.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.033+0000 7f7323200640 1 --2- 172.21.15.167:0/1101952912 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f73040762d0 0x7f7304078790 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f73240520f0 tx=0x7f7314001510 comp rx=0 tx=0).stop 2024-05-11T12:14:20.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.033+0000 7f7323200640 1 -- 172.21.15.167:0/1101952912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7324154630 msgr2=0x7f7324154a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:20.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.033+0000 7f7323200640 1 --2- 172.21.15.167:0/1101952912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7324154630 0x7f7324154a90 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f73240bb110 tx=0x7f731c00e4b0 comp rx=0 tx=0).stop 2024-05-11T12:14:20.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.033+0000 7f7323200640 1 -- 172.21.15.167:0/1101952912 shutdown_connections 2024-05-11T12:14:20.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.033+0000 7f7323200640 1 --2- 172.21.15.167:0/1101952912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7324154630 0x7f7324154a90 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:20.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.033+0000 7f7323200640 1 --2- 172.21.15.167:0/1101952912 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7324161e80 0x7f7324162300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:20.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.033+0000 7f7323200640 1 --2- 172.21.15.167:0/1101952912 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f73040762d0 0x7f7304078790 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:20.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.033+0000 7f7323200640 1 --2- 172.21.15.167:0/1101952912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7308004800 0x7f73240cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:20.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.033+0000 7f7323200640 1 -- 172.21.15.167:0/1101952912 >> 172.21.15.167:0/1101952912 conn(0x7f73240d1a50 msgr2=0x7f73240b5bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:20.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.034+0000 7f7323200640 1 -- 172.21.15.167:0/1101952912 shutdown_connections 2024-05-11T12:14:20.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.034+0000 7f7323200640 1 -- 172.21.15.167:0/1101952912 wait complete. 2024-05-11T12:14:20.053 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:14:20.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.256+0000 7efe16e00640 1 Processor -- start 2024-05-11T12:14:20.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.256+0000 7efe16e00640 1 -- start start 2024-05-11T12:14:20.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.258+0000 7efe16e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7efe10157750 0x7efe1015d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:20.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.258+0000 7efe16e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efe100b3ea0 0x7efe10161490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:20.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.258+0000 7efe16e00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe1015dc80 con 0x7efe101619d0 2024-05-11T12:14:20.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.258+0000 7efe16e00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7efe10161000 con 0x7efe10157750 2024-05-11T12:14:20.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.258+0000 7efe16e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7efe1015df30 con 0x7efe100b3ea0 2024-05-11T12:14:20.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.258+0000 7efe14e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7efe101619d0 0x7efe1006d460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:39274/0 (socket says 172.21.15.167:39274) 2024-05-11T12:14:20.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.258+0000 7efe0fe00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7efe10157750 0x7efe1015d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:20.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.258+0000 7efe14e00640 1 -- 172.21.15.167:0/1714088148 learned_addr learned my addr 172.21.15.167:0/1714088148 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:14:20.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.259+0000 7efe0f400640 1 --2- 172.21.15.167:0/1714088148 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efe100b3ea0 0x7efe10161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:20.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.259+0000 7efe0fe00640 1 -- 172.21.15.167:0/1714088148 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efe100b3ea0 msgr2=0x7efe10161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:20.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.259+0000 7efe0fe00640 1 --2- 172.21.15.167:0/1714088148 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efe100b3ea0 0x7efe10161490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:20.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.259+0000 7efe0fe00640 1 -- 172.21.15.167:0/1714088148 >> v1:172.21.15.167:6789/0 conn(0x7efe101619d0 legacy=0x7efe1006d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:20.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.259+0000 7efe0fe00640 1 -- 172.21.15.167:0/1714088148 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe1015e210 con 0x7efe10157750 2024-05-11T12:14:20.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.260+0000 7efe0fe00640 1 --2- 172.21.15.167:0/1714088148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe10157750 0x7efe1015d740 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7efe000097a0 tx=0x7efe0002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5867ce1fac7d5688 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:20.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.260+0000 7efe0ea00640 1 -- 172.21.15.167:0/1714088148 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe00037070 con 0x7efe10157750 2024-05-11T12:14:20.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.260+0000 7efe0ea00640 1 -- 172.21.15.167:0/1714088148 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe00032440 con 0x7efe10157750 2024-05-11T12:14:20.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.260+0000 7efe0ea00640 1 -- 172.21.15.167:0/1714088148 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe00032810 con 0x7efe10157750 2024-05-11T12:14:20.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.261+0000 7efe16e00640 1 -- 172.21.15.167:0/1714088148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe10157750 msgr2=0x7efe1015d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:20.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.261+0000 7efe16e00640 1 --2- 172.21.15.167:0/1714088148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe10157750 0x7efe1015d740 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7efe000097a0 tx=0x7efe0002a180 comp rx=0 tx=0).stop 2024-05-11T12:14:20.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.261+0000 7efe16e00640 1 -- 172.21.15.167:0/1714088148 shutdown_connections 2024-05-11T12:14:20.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.261+0000 7efe16e00640 1 --2- 172.21.15.167:0/1714088148 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efe100b3ea0 0x7efe10161490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:20.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.261+0000 7efe16e00640 1 --2- 172.21.15.167:0/1714088148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe10157750 0x7efe1015d740 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:20.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.261+0000 7efe16e00640 1 -- 172.21.15.167:0/1714088148 >> 172.21.15.167:0/1714088148 conn(0x7efe100cd8f0 msgr2=0x7efe100cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:20.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.261+0000 7efe16e00640 1 -- 172.21.15.167:0/1714088148 shutdown_connections 2024-05-11T12:14:20.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.261+0000 7efe16e00640 1 -- 172.21.15.167:0/1714088148 wait complete. 2024-05-11T12:14:20.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.262+0000 7efe16e00640 1 Processor -- start 2024-05-11T12:14:20.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.263+0000 7efe16e00640 1 -- start start 2024-05-11T12:14:20.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.263+0000 7efe16e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efe100b3ea0 0x7efe1013f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:20.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.263+0000 7efe16e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe10157750 0x7efe1013da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:20.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.263+0000 7efe16e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe101619d0 0x7efe101407f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:20.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.263+0000 7efe16e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efe1006db70 con 0x7efe10157750 2024-05-11T12:14:20.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.263+0000 7efe16e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7efe10157b30 con 0x7efe101619d0 2024-05-11T12:14:20.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.263+0000 7efe16e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7efe1006dcf0 con 0x7efe100b3ea0 2024-05-11T12:14:20.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.264+0000 7efe0f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe10157750 0x7efe1013da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:20.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.264+0000 7efe0f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe10157750 0x7efe1013da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:51474/0 (socket says 172.21.15.167:51474) 2024-05-11T12:14:20.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.264+0000 7efe0f400640 1 -- 172.21.15.167:0/3831485739 learned_addr learned my addr 172.21.15.167:0/3831485739 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:20.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.264+0000 7efe0fe00640 1 --2- 172.21.15.167:0/3831485739 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efe100b3ea0 0x7efe1013f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:20.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.264+0000 7efe14e00640 1 --2- 172.21.15.167:0/3831485739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe101619d0 0x7efe101407f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:20.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.264+0000 7efe0f400640 1 -- 172.21.15.167:0/3831485739 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efe100b3ea0 msgr2=0x7efe1013f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:20.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.264+0000 7efe0f400640 1 --2- 172.21.15.167:0/3831485739 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efe100b3ea0 0x7efe1013f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:20.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.264+0000 7efe0f400640 1 -- 172.21.15.167:0/3831485739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe101619d0 msgr2=0x7efe101407f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:20.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.264+0000 7efe0f400640 1 --2- 172.21.15.167:0/3831485739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe101619d0 0x7efe101407f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:20.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.264+0000 7efe0f400640 1 -- 172.21.15.167:0/3831485739 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe1013df70 con 0x7efe10157750 2024-05-11T12:14:20.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.264+0000 7efe14e00640 1 --2- 172.21.15.167:0/3831485739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe101619d0 0x7efe101407f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:20.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.265+0000 7efe0f400640 1 --2- 172.21.15.167:0/3831485739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe10157750 0x7efe1013da30 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7efdfc00d800 tx=0x7efdfc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:20.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.265+0000 7efe07e00640 1 -- 172.21.15.167:0/3831485739 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc00b6c0 con 0x7efe10157750 2024-05-11T12:14:20.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.265+0000 7efe07e00640 1 -- 172.21.15.167:0/3831485739 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdfc019070 con 0x7efe10157750 2024-05-11T12:14:20.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.265+0000 7efe07e00640 1 -- 172.21.15.167:0/3831485739 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc00ba80 con 0x7efe10157750 2024-05-11T12:14:20.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.265+0000 7efe16e00640 1 -- 172.21.15.167:0/3831485739 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe1013f7d0 con 0x7efe10157750 2024-05-11T12:14:20.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.265+0000 7efe16e00640 1 -- 172.21.15.167:0/3831485739 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe1013fd90 con 0x7efe10157750 2024-05-11T12:14:20.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.266+0000 7efe16e00640 1 -- 172.21.15.167:0/3831485739 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe10069590 con 0x7efe10157750 2024-05-11T12:14:20.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.267+0000 7efe07e00640 1 -- 172.21.15.167:0/3831485739 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7efdfc00bc20 con 0x7efe10157750 2024-05-11T12:14:20.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.268+0000 7efe07e00640 1 --2- 172.21.15.167:0/3831485739 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7efdf00762d0 0x7efdf0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:20.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.268+0000 7efe07e00640 1 -- 172.21.15.167:0/3831485739 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7efdfc097c50 con 0x7efe10157750 2024-05-11T12:14:20.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.268+0000 7efe0fe00640 1 --2- 172.21.15.167:0/3831485739 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7efdf00762d0 0x7efdf0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:20.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.269+0000 7efe0fe00640 1 --2- 172.21.15.167:0/3831485739 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7efdf00762d0 0x7efdf0078790 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7efe00034000 tx=0x7efe0002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:20.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.273+0000 7efe07e00640 1 -- 172.21.15.167:0/3831485739 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7efdfc061410 con 0x7efe10157750 2024-05-11T12:14:20.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.524+0000 7efe16e00640 1 -- 172.21.15.167:0/3831485739 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe10140d30 con 0x7efe10157750 2024-05-11T12:14:20.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.525+0000 7efe07e00640 1 -- 172.21.15.167:0/3831485739 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5195 (secure 0 0 0) 0x7efdfc060d10 con 0x7efe10157750 2024-05-11T12:14:20.526 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:20.526 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4922,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1154357925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1154357925},{"type":"v1","addr":"172.21.15.167:6837","nonce":1154357925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":5155,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1256314893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1256314893},{"type":"v1","addr":"172.21.15.204:6833","nonce":1256314893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":5158,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1578748665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1578748665},{"type":"v1","addr":"172.21.15.204:6835","nonce":1578748665}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:14:04.861998+0000","modified":"2024-05-11T12:14:07.926157+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6300":{"gid":6300,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6836/3484070468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3484070468},{"type":"v1","addr":"172.21.15.167:6836","nonce":3484070468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6300,"qdb_cluster":[6300]},"id":6}]} 2024-05-11T12:14:20.526 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 53 2024-05-11T12:14:20.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.528+0000 7efe16e00640 1 -- 172.21.15.167:0/3831485739 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7efdf00762d0 msgr2=0x7efdf0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:20.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.528+0000 7efe16e00640 1 --2- 172.21.15.167:0/3831485739 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7efdf00762d0 0x7efdf0078790 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7efe00034000 tx=0x7efe0002e000 comp rx=0 tx=0).stop 2024-05-11T12:14:20.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.528+0000 7efe16e00640 1 -- 172.21.15.167:0/3831485739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe10157750 msgr2=0x7efe1013da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:20.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.528+0000 7efe16e00640 1 --2- 172.21.15.167:0/3831485739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe10157750 0x7efe1013da30 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7efdfc00d800 tx=0x7efdfc00dcc0 comp rx=0 tx=0).stop 2024-05-11T12:14:20.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.528+0000 7efe16e00640 1 -- 172.21.15.167:0/3831485739 shutdown_connections 2024-05-11T12:14:20.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.528+0000 7efe16e00640 1 --2- 172.21.15.167:0/3831485739 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7efdf00762d0 0x7efdf0078790 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:20.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.528+0000 7efe16e00640 1 --2- 172.21.15.167:0/3831485739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe101619d0 0x7efe101407f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:20.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.528+0000 7efe16e00640 1 --2- 172.21.15.167:0/3831485739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe10157750 0x7efe1013da30 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:20.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.528+0000 7efe16e00640 1 --2- 172.21.15.167:0/3831485739 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efe100b3ea0 0x7efe1013f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:20.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.528+0000 7efe16e00640 1 -- 172.21.15.167:0/3831485739 >> 172.21.15.167:0/3831485739 conn(0x7efe100cd8f0 msgr2=0x7efe1015e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:20.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.528+0000 7efe16e00640 1 -- 172.21.15.167:0/3831485739 shutdown_connections 2024-05-11T12:14:20.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.528+0000 7efe16e00640 1 -- 172.21.15.167:0/3831485739 wait complete. 2024-05-11T12:14:20.547 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:14:20.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.743+0000 7ff39fa00640 1 Processor -- start 2024-05-11T12:14:20.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.743+0000 7ff39fa00640 1 -- start start 2024-05-11T12:14:20.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.745+0000 7ff39fa00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff398157750 0x7ff39815d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:20.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.745+0000 7ff39fa00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff3980b3ea0 0x7ff398161490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:20.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.745+0000 7ff39fa00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff39815dc80 con 0x7ff3981619d0 2024-05-11T12:14:20.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.745+0000 7ff39fa00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7ff398161000 con 0x7ff3980b3ea0 2024-05-11T12:14:20.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.745+0000 7ff39fa00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff39815df30 con 0x7ff398157750 2024-05-11T12:14:20.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.745+0000 7ff39da00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7ff3981619d0 0x7ff39806d460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:39282/0 (socket says 172.21.15.167:39282) 2024-05-11T12:14:20.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.745+0000 7ff39da00640 1 -- 172.21.15.167:0/1544628768 learned_addr learned my addr 172.21.15.167:0/1544628768 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:14:20.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.745+0000 7ff397e00640 1 --2- 172.21.15.167:0/1544628768 >> v2:172.21.15.167:3300/0 conn(0x7ff3980b3ea0 0x7ff398161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:20.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.746+0000 7ff39d000640 1 --2- 172.21.15.167:0/1544628768 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff398157750 0x7ff39815d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:20.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.746+0000 7ff397400640 1 -- 172.21.15.167:0/1544628768 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2997216291 0 0) 0x7ff39815dc80 con 0x7ff3981619d0 2024-05-11T12:14:20.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.746+0000 7ff397400640 1 -- 172.21.15.167:0/1544628768 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff37c003520 con 0x7ff3981619d0 2024-05-11T12:14:20.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.746+0000 7ff397e00640 1 -- 172.21.15.167:0/1544628768 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff398157750 msgr2=0x7ff39815d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:20.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.746+0000 7ff397e00640 1 --2- 172.21.15.167:0/1544628768 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff398157750 0x7ff39815d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:20.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.746+0000 7ff397e00640 1 -- 172.21.15.167:0/1544628768 >> v1:172.21.15.167:6789/0 conn(0x7ff3981619d0 legacy=0x7ff39806d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:20.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.746+0000 7ff397e00640 1 -- 172.21.15.167:0/1544628768 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff39815e210 con 0x7ff3980b3ea0 2024-05-11T12:14:20.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.747+0000 7ff397e00640 1 --2- 172.21.15.167:0/1544628768 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3980b3ea0 0x7ff398161490 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7ff3800097a0 tx=0x7ff38002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47875ba6f108433e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:20.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.747+0000 7ff397400640 1 -- 172.21.15.167:0/1544628768 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff380037070 con 0x7ff3980b3ea0 2024-05-11T12:14:20.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.747+0000 7ff397400640 1 -- 172.21.15.167:0/1544628768 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff380032440 con 0x7ff3980b3ea0 2024-05-11T12:14:20.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.747+0000 7ff397400640 1 -- 172.21.15.167:0/1544628768 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff380032810 con 0x7ff3980b3ea0 2024-05-11T12:14:20.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.748+0000 7ff39fa00640 1 -- 172.21.15.167:0/1544628768 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3980b3ea0 msgr2=0x7ff398161490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:20.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.748+0000 7ff39fa00640 1 --2- 172.21.15.167:0/1544628768 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3980b3ea0 0x7ff398161490 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7ff3800097a0 tx=0x7ff38002a180 comp rx=0 tx=0).stop 2024-05-11T12:14:20.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.748+0000 7ff39fa00640 1 -- 172.21.15.167:0/1544628768 shutdown_connections 2024-05-11T12:14:20.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.748+0000 7ff39fa00640 1 --2- 172.21.15.167:0/1544628768 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3980b3ea0 0x7ff398161490 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:20.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.748+0000 7ff39fa00640 1 --2- 172.21.15.167:0/1544628768 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff398157750 0x7ff39815d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:20.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.748+0000 7ff39fa00640 1 -- 172.21.15.167:0/1544628768 >> 172.21.15.167:0/1544628768 conn(0x7ff3980cd8f0 msgr2=0x7ff3980cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:20.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.748+0000 7ff39fa00640 1 -- 172.21.15.167:0/1544628768 shutdown_connections 2024-05-11T12:14:20.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.748+0000 7ff39fa00640 1 -- 172.21.15.167:0/1544628768 wait complete. 2024-05-11T12:14:20.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.750+0000 7ff39fa00640 1 Processor -- start 2024-05-11T12:14:20.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.750+0000 7ff39fa00640 1 -- start start 2024-05-11T12:14:20.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.751+0000 7ff39fa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3980b3ea0 0x7ff398156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:20.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.751+0000 7ff39fa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff398157750 0x7ff3980b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:20.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.751+0000 7ff39fa00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff3981619d0 0x7ff3980b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:20.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.751+0000 7ff39fa00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff39806db70 con 0x7ff3980b3ea0 2024-05-11T12:14:20.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.751+0000 7ff39fa00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff398157b30 con 0x7ff398157750 2024-05-11T12:14:20.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.751+0000 7ff39fa00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff39806dcf0 con 0x7ff3981619d0 2024-05-11T12:14:20.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.751+0000 7ff39d000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3980b3ea0 0x7ff398156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:20.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.751+0000 7ff39d000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3980b3ea0 0x7ff398156b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:51484/0 (socket says 172.21.15.167:51484) 2024-05-11T12:14:20.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.751+0000 7ff39d000640 1 -- 172.21.15.167:0/3397914838 learned_addr learned my addr 172.21.15.167:0/3397914838 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:20.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.751+0000 7ff39d000640 1 -- 172.21.15.167:0/3397914838 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff3981619d0 msgr2=0x7ff3980b2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:20.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.751+0000 7ff39d000640 1 --2- 172.21.15.167:0/3397914838 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff3981619d0 0x7ff3980b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:20.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.751+0000 7ff39d000640 1 -- 172.21.15.167:0/3397914838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff398157750 msgr2=0x7ff3980b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:20.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.751+0000 7ff39d000640 1 --2- 172.21.15.167:0/3397914838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff398157750 0x7ff3980b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:20.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.751+0000 7ff39d000640 1 -- 172.21.15.167:0/3397914838 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3980aea70 con 0x7ff3980b3ea0 2024-05-11T12:14:20.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.752+0000 7ff39d000640 1 --2- 172.21.15.167:0/3397914838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3980b3ea0 0x7ff398156b50 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7ff38800d830 tx=0x7ff38800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:20.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.752+0000 7ff394c00640 1 -- 172.21.15.167:0/3397914838 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38800b6c0 con 0x7ff3980b3ea0 2024-05-11T12:14:20.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.752+0000 7ff394c00640 1 -- 172.21.15.167:0/3397914838 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff388019070 con 0x7ff3980b3ea0 2024-05-11T12:14:20.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.752+0000 7ff39fa00640 1 -- 172.21.15.167:0/3397914838 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3980aed50 con 0x7ff3980b3ea0 2024-05-11T12:14:20.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.752+0000 7ff394c00640 1 -- 172.21.15.167:0/3397914838 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38800ba80 con 0x7ff3980b3ea0 2024-05-11T12:14:20.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.752+0000 7ff39fa00640 1 -- 172.21.15.167:0/3397914838 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3980af310 con 0x7ff3980b3ea0 2024-05-11T12:14:20.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.753+0000 7ff39fa00640 1 -- 172.21.15.167:0/3397914838 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff398069590 con 0x7ff3980b3ea0 2024-05-11T12:14:20.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.754+0000 7ff394c00640 1 -- 172.21.15.167:0/3397914838 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff38801e460 con 0x7ff3980b3ea0 2024-05-11T12:14:20.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.755+0000 7ff394c00640 1 --2- 172.21.15.167:0/3397914838 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff3740762d0 0x7ff374078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:20.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.755+0000 7ff394c00640 1 -- 172.21.15.167:0/3397914838 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7ff388097f30 con 0x7ff3980b3ea0 2024-05-11T12:14:20.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.755+0000 7ff397e00640 1 --2- 172.21.15.167:0/3397914838 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff3740762d0 0x7ff374078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:20.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.756+0000 7ff397e00640 1 --2- 172.21.15.167:0/3397914838 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff3740762d0 0x7ff374078790 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff380034000 tx=0x7ff38002c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:20.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.760+0000 7ff394c00640 1 -- 172.21.15.167:0/3397914838 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff388061670 con 0x7ff3980b3ea0 2024-05-11T12:14:20.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.971+0000 7ff39fa00640 1 -- 172.21.15.167:0/3397914838 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff39815eb40 con 0x7ff3980b3ea0 2024-05-11T12:14:20.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.974+0000 7ff394c00640 1 -- 172.21.15.167:0/3397914838 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+16933 (secure 0 0 0) 0x7ff388060f70 con 0x7ff3980b3ea0 2024-05-11T12:14:20.975 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:20.976 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":86,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:14:06.410610+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:14:02.772030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-11T12:14:03.847508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:14:20.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.977+0000 7ff39fa00640 1 -- 172.21.15.167:0/3397914838 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff3740762d0 msgr2=0x7ff374078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:20.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.977+0000 7ff39fa00640 1 --2- 172.21.15.167:0/3397914838 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff3740762d0 0x7ff374078790 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff380034000 tx=0x7ff38002c000 comp rx=0 tx=0).stop 2024-05-11T12:14:20.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.977+0000 7ff39fa00640 1 -- 172.21.15.167:0/3397914838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3980b3ea0 msgr2=0x7ff398156b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:20.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.977+0000 7ff39fa00640 1 --2- 172.21.15.167:0/3397914838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3980b3ea0 0x7ff398156b50 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7ff38800d830 tx=0x7ff38800dcf0 comp rx=0 tx=0).stop 2024-05-11T12:14:20.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.977+0000 7ff39fa00640 1 -- 172.21.15.167:0/3397914838 shutdown_connections 2024-05-11T12:14:20.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.977+0000 7ff39fa00640 1 --2- 172.21.15.167:0/3397914838 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff3981619d0 0x7ff3980b2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:20.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.977+0000 7ff39fa00640 1 --2- 172.21.15.167:0/3397914838 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff3740762d0 0x7ff374078790 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:20.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.977+0000 7ff39fa00640 1 --2- 172.21.15.167:0/3397914838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff398157750 0x7ff3980b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:20.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.977+0000 7ff39fa00640 1 --2- 172.21.15.167:0/3397914838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3980b3ea0 0x7ff398156b50 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:20.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.977+0000 7ff39fa00640 1 -- 172.21.15.167:0/3397914838 >> 172.21.15.167:0/3397914838 conn(0x7ff3980cd8f0 msgr2=0x7ff3980b4360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:20.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.978+0000 7ff39fa00640 1 -- 172.21.15.167:0/3397914838 shutdown_connections 2024-05-11T12:14:20.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:20.978+0000 7ff39fa00640 1 -- 172.21.15.167:0/3397914838 wait complete. 2024-05-11T12:14:20.997 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-11T12:14:21.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.200+0000 7f601ca00640 1 Processor -- start 2024-05-11T12:14:21.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.200+0000 7f601ca00640 1 -- start start 2024-05-11T12:14:21.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.202+0000 7f601ca00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f601813fc30 0x7f6018160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:21.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.202+0000 7f601ca00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f60180cfc90 0x7f6018161020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:21.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.202+0000 7f601ca00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60180d0220 con 0x7f6018140010 2024-05-11T12:14:21.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.202+0000 7f601ca00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f60181619d0 con 0x7f60180cfc90 2024-05-11T12:14:21.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.202+0000 7f601ca00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f60180d0ac0 con 0x7f601813fc30 2024-05-11T12:14:21.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.202+0000 7f6015e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f6018140010 0x7f60180cf580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:39288/0 (socket says 172.21.15.167:39288) 2024-05-11T12:14:21.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.202+0000 7f6015e00640 1 -- 172.21.15.167:0/3313736363 learned_addr learned my addr 172.21.15.167:0/3313736363 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:14:21.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.202+0000 7f6017200640 1 --2- 172.21.15.167:0/3313736363 >> v2:172.21.15.204:3300/0 conn(0x7f60180cfc90 0x7f6018161020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:21.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.202+0000 7f6016800640 1 --2- 172.21.15.167:0/3313736363 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f601813fc30 0x7f6018160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:21.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.202+0000 7f6015400640 1 -- 172.21.15.167:0/3313736363 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3445632445 0 0) 0x7f60180d0220 con 0x7f6018140010 2024-05-11T12:14:21.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.203+0000 7f6015400640 1 -- 172.21.15.167:0/3313736363 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ffc003520 con 0x7f6018140010 2024-05-11T12:14:21.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.203+0000 7f6015400640 1 -- 172.21.15.167:0/3313736363 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1877004097 0 0) 0x7f5ffc003520 con 0x7f6018140010 2024-05-11T12:14:21.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.203+0000 7f6015400640 1 -- 172.21.15.167:0/3313736363 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f601813fc30 msgr2=0x7f6018160ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:21.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.203+0000 7f6015400640 1 --2- 172.21.15.167:0/3313736363 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f601813fc30 0x7f6018160ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:21.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.203+0000 7f6015400640 1 -- 172.21.15.167:0/3313736363 >> v2:172.21.15.204:3300/0 conn(0x7f60180cfc90 msgr2=0x7f6018161020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:21.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.203+0000 7f6015400640 1 --2- 172.21.15.167:0/3313736363 >> v2:172.21.15.204:3300/0 conn(0x7f60180cfc90 0x7f6018161020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:21.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.203+0000 7f6015400640 1 -- 172.21.15.167:0/3313736363 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60180d0500 con 0x7f6018140010 2024-05-11T12:14:21.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.203+0000 7f6015400640 1 -- 172.21.15.167:0/3313736363 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f60040016f0 con 0x7f6018140010 2024-05-11T12:14:21.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.203+0000 7f6015400640 1 -- 172.21.15.167:0/3313736363 >> v1:172.21.15.167:6789/0 conn(0x7f6018140010 legacy=0x7f60180cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:21.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.203+0000 7f6015400640 1 --2- 172.21.15.167:0/3313736363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ffc003ac0 0x7f5ffc003ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:21.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.203+0000 7f6015400640 1 --2- 172.21.15.167:0/3313736363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ffc004800 0x7f5ffc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:21.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.203+0000 7f6015400640 1 --2- 172.21.15.167:0/3313736363 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f601813fc30 0x7f5ffc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:21.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.203+0000 7f6015400640 1 -- 172.21.15.167:0/3313736363 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f60181619d0 con 0x7f5ffc003ac0 2024-05-11T12:14:21.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.203+0000 7f6015400640 1 -- 172.21.15.167:0/3313736363 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f60180d0ac0 con 0x7f5ffc004800 2024-05-11T12:14:21.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.203+0000 7f6015400640 1 -- 172.21.15.167:0/3313736363 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5ffc0055a0 con 0x7f601813fc30 2024-05-11T12:14:21.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.203+0000 7f6016800640 1 --2- 172.21.15.167:0/3313736363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ffc003ac0 0x7f5ffc003ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:21.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.204+0000 7f6016800640 1 -- 172.21.15.167:0/3313736363 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f601813fc30 msgr2=0x7f5ffc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:21.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.204+0000 7f6016800640 1 --2- 172.21.15.167:0/3313736363 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f601813fc30 0x7f5ffc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:21.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.204+0000 7f6016800640 1 -- 172.21.15.167:0/3313736363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ffc004800 msgr2=0x7f5ffc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:21.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.204+0000 7f6016800640 1 --2- 172.21.15.167:0/3313736363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ffc004800 0x7f5ffc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:21.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.204+0000 7f6016800640 1 -- 172.21.15.167:0/3313736363 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ffc003520 con 0x7f5ffc003ac0 2024-05-11T12:14:21.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.205+0000 7f6016800640 1 --2- 172.21.15.167:0/3313736363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ffc003ac0 0x7f5ffc003ee0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f600800ba30 tx=0x7f5ffc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=457bf3ff83b1943a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:21.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.205+0000 7f6015400640 1 -- 172.21.15.167:0/3313736363 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6008030070 con 0x7f5ffc003ac0 2024-05-11T12:14:21.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.205+0000 7f6015400640 1 -- 172.21.15.167:0/3313736363 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f600802ae10 con 0x7f5ffc003ac0 2024-05-11T12:14:21.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.205+0000 7f6015400640 1 -- 172.21.15.167:0/3313736363 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600800b600 con 0x7f5ffc003ac0 2024-05-11T12:14:21.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.205+0000 7f601ca00640 1 -- 172.21.15.167:0/3313736363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ffc003ac0 msgr2=0x7f5ffc003ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:21.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.205+0000 7f601ca00640 1 --2- 172.21.15.167:0/3313736363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ffc003ac0 0x7f5ffc003ee0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f600800ba30 tx=0x7f5ffc005830 comp rx=0 tx=0).stop 2024-05-11T12:14:21.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.205+0000 7f6017200640 1 -- 172.21.15.167:0/3313736363 reap_dead start 2024-05-11T12:14:21.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.206+0000 7f601ca00640 1 -- 172.21.15.167:0/3313736363 shutdown_connections 2024-05-11T12:14:21.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.206+0000 7f601ca00640 1 -- 172.21.15.167:0/3313736363 >> 172.21.15.167:0/3313736363 conn(0x7f601806c450 msgr2=0x7f601806c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:21.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.206+0000 7f601ca00640 1 -- 172.21.15.167:0/3313736363 shutdown_connections 2024-05-11T12:14:21.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.206+0000 7f601ca00640 1 -- 172.21.15.167:0/3313736363 wait complete. 2024-05-11T12:14:21.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.207+0000 7f601ca00640 1 Processor -- start 2024-05-11T12:14:21.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.208+0000 7f601ca00640 1 -- start start 2024-05-11T12:14:21.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.208+0000 7f601ca00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ffc003ac0 0x7f601815e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:21.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.208+0000 7f601ca00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f601813b6a0 0x7f6018160230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:21.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.208+0000 7f601ca00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6018072a90 0x7f6018072f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:21.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.208+0000 7f601ca00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6018160ae0 con 0x7f601813b6a0 2024-05-11T12:14:21.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.208+0000 7f601ca00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f601806e590 con 0x7f5ffc003ac0 2024-05-11T12:14:21.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.208+0000 7f601ca00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6018160c60 con 0x7f6018072a90 2024-05-11T12:14:21.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.209+0000 7f6015e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f601813b6a0 0x7f6018160230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:21.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.209+0000 7f6015e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f601813b6a0 0x7f6018160230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:51502/0 (socket says 172.21.15.167:51502) 2024-05-11T12:14:21.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.209+0000 7f6015e00640 1 -- 172.21.15.167:0/4224522580 learned_addr learned my addr 172.21.15.167:0/4224522580 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:21.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.209+0000 7f6016800640 1 --2- 172.21.15.167:0/4224522580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ffc003ac0 0x7f601815e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:21.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.209+0000 7f6015e00640 1 -- 172.21.15.167:0/4224522580 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6018072a90 msgr2=0x7f6018072f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:21.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.209+0000 7f6017200640 1 --2- 172.21.15.167:0/4224522580 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6018072a90 0x7f6018072f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:21.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.209+0000 7f6015e00640 1 --2- 172.21.15.167:0/4224522580 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6018072a90 0x7f6018072f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:21.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.209+0000 7f6015e00640 1 -- 172.21.15.167:0/4224522580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ffc003ac0 msgr2=0x7f601815e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:21.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.209+0000 7f6015e00640 1 --2- 172.21.15.167:0/4224522580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ffc003ac0 0x7f601815e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:21.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.209+0000 7f6015e00640 1 -- 172.21.15.167:0/4224522580 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60180b47c0 con 0x7f601813b6a0 2024-05-11T12:14:21.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.210+0000 7f6015e00640 1 --2- 172.21.15.167:0/4224522580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f601813b6a0 0x7f6018160230 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f6004001160 tx=0x7f600400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:21.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.210+0000 7f600ea00640 1 -- 172.21.15.167:0/4224522580 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60040072d0 con 0x7f601813b6a0 2024-05-11T12:14:21.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.210+0000 7f600ea00640 1 -- 172.21.15.167:0/4224522580 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6004007470 con 0x7f601813b6a0 2024-05-11T12:14:21.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.210+0000 7f600ea00640 1 -- 172.21.15.167:0/4224522580 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6004004a20 con 0x7f601813b6a0 2024-05-11T12:14:21.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.210+0000 7f601ca00640 1 -- 172.21.15.167:0/4224522580 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f601815eb20 con 0x7f601813b6a0 2024-05-11T12:14:21.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.210+0000 7f601ca00640 1 -- 172.21.15.167:0/4224522580 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f601815f0e0 con 0x7f601813b6a0 2024-05-11T12:14:21.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.211+0000 7f601ca00640 1 -- 172.21.15.167:0/4224522580 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60180694b0 con 0x7f601813b6a0 2024-05-11T12:14:21.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.212+0000 7f600ea00640 1 -- 172.21.15.167:0/4224522580 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6004011070 con 0x7f601813b6a0 2024-05-11T12:14:21.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.213+0000 7f600ea00640 1 --2- 172.21.15.167:0/4224522580 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5ff80762d0 0x7f5ff8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:21.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.213+0000 7f600ea00640 1 -- 172.21.15.167:0/4224522580 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7f6004097490 con 0x7f601813b6a0 2024-05-11T12:14:21.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.214+0000 7f6016800640 1 --2- 172.21.15.167:0/4224522580 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5ff80762d0 0x7f5ff8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:21.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.214+0000 7f6016800640 1 --2- 172.21.15.167:0/4224522580 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5ff80762d0 0x7f5ff8078790 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f600800ba00 tx=0x7f6008030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:21.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.218+0000 7f600ea00640 1 -- 172.21.15.167:0/4224522580 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6004060bc0 con 0x7f601813b6a0 2024-05-11T12:14:21.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.474+0000 7f601ca00640 1 -- 172.21.15.167:0/4224522580 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f6018161080 con 0x7f601813b6a0 2024-05-11T12:14:21.475 INFO:tasks.ceph.mon.a.smithi167.stderr:2024-05-11T12:14:21.475+0000 7f3aac000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-11T12:14:21.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.479+0000 7f600ea00640 1 -- 172.21.15.167:0/4224522580 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v54) ==== 161+0+0 (secure 0 0 0) 0x7f60040604c0 con 0x7f601813b6a0 2024-05-11T12:14:21.480 INFO:teuthology.orchestra.run.smithi167.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-11T12:14:21.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.482+0000 7f601ca00640 1 -- 172.21.15.167:0/4224522580 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5ff80762d0 msgr2=0x7f5ff8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:21.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.482+0000 7f601ca00640 1 --2- 172.21.15.167:0/4224522580 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5ff80762d0 0x7f5ff8078790 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f600800ba00 tx=0x7f6008030000 comp rx=0 tx=0).stop 2024-05-11T12:14:21.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.482+0000 7f601ca00640 1 -- 172.21.15.167:0/4224522580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f601813b6a0 msgr2=0x7f6018160230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:21.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.482+0000 7f601ca00640 1 --2- 172.21.15.167:0/4224522580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f601813b6a0 0x7f6018160230 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f6004001160 tx=0x7f600400cb90 comp rx=0 tx=0).stop 2024-05-11T12:14:21.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.483+0000 7f601ca00640 1 -- 172.21.15.167:0/4224522580 shutdown_connections 2024-05-11T12:14:21.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.483+0000 7f601ca00640 1 --2- 172.21.15.167:0/4224522580 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6018072a90 0x7f6018072f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:21.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.483+0000 7f601ca00640 1 --2- 172.21.15.167:0/4224522580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f601813b6a0 0x7f6018160230 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:21.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.483+0000 7f601ca00640 1 --2- 172.21.15.167:0/4224522580 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f5ff80762d0 0x7f5ff8078790 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:21.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.483+0000 7f601ca00640 1 --2- 172.21.15.167:0/4224522580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ffc003ac0 0x7f601815e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:21.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.483+0000 7f601ca00640 1 -- 172.21.15.167:0/4224522580 >> 172.21.15.167:0/4224522580 conn(0x7f601806c450 msgr2=0x7f60180af130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:21.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.483+0000 7f601ca00640 1 -- 172.21.15.167:0/4224522580 shutdown_connections 2024-05-11T12:14:21.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.483+0000 7f601ca00640 1 -- 172.21.15.167:0/4224522580 wait complete. 2024-05-11T12:14:21.500 INFO:tasks.ceph.mds.a.smithi167.stderr: -2108> 2024-05-11T12:14:11.351+0000 7fb27e000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-11T12:14:21.505 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-11T12:14:21.564 INFO:tasks.ceph.mds.a.smithi167.stdout:starting mds.a at 2024-05-11T12:14:21.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.685+0000 7f0459600640 1 Processor -- start 2024-05-11T12:14:21.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.685+0000 7f0459600640 1 -- start start 2024-05-11T12:14:21.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.687+0000 7f0459600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0454073270 0x7f045415d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:21.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.687+0000 7f0459600640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f04540ce8f0 0x7f04540ced80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:21.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.687+0000 7f0459600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f045415c6d0 con 0x7f0454073650 2024-05-11T12:14:21.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.687+0000 7f0459600640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f045406ead0 con 0x7f04540ce8f0 2024-05-11T12:14:21.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.687+0000 7f0459600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0454160b00 con 0x7f0454073270 2024-05-11T12:14:21.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.687+0000 7f0453400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f04540ce8f0 0x7f04540ced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:21.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.687+0000 7f0453400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f04540ce8f0 0x7f04540ced80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:51534/0 (socket says 172.21.15.167:51534) 2024-05-11T12:14:21.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.688+0000 7f0453400640 1 -- 172.21.15.167:0/214408384 learned_addr learned my addr 172.21.15.167:0/214408384 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:21.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.688+0000 7f0452a00640 1 --2- 172.21.15.167:0/214408384 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0454073270 0x7f045415d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:21.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.688+0000 7f0453400640 1 -- 172.21.15.167:0/214408384 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0454073270 msgr2=0x7f045415d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:21.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.688+0000 7f0453400640 1 --2- 172.21.15.167:0/214408384 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0454073270 0x7f045415d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:21.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.688+0000 7f0453400640 1 -- 172.21.15.167:0/214408384 >> v1:172.21.15.204:6789/0 conn(0x7f0454073650 legacy=0x7f045406e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:21.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.688+0000 7f0453400640 1 -- 172.21.15.167:0/214408384 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04540700b0 con 0x7f04540ce8f0 2024-05-11T12:14:21.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.689+0000 7f0453400640 1 --2- 172.21.15.167:0/214408384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04540ce8f0 0x7f04540ced80 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f044c009e50 tx=0x7f044c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bad3e0c96d48e393 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:21.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.689+0000 7f0452000640 1 -- 172.21.15.167:0/214408384 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044c036070 con 0x7f04540ce8f0 2024-05-11T12:14:21.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.689+0000 7f0452000640 1 -- 172.21.15.167:0/214408384 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f044c031600 con 0x7f04540ce8f0 2024-05-11T12:14:21.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.689+0000 7f0452000640 1 -- 172.21.15.167:0/214408384 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044c0319d0 con 0x7f04540ce8f0 2024-05-11T12:14:21.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.690+0000 7f0459600640 1 -- 172.21.15.167:0/214408384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04540ce8f0 msgr2=0x7f04540ced80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:21.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.690+0000 7f0459600640 1 --2- 172.21.15.167:0/214408384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04540ce8f0 0x7f04540ced80 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f044c009e50 tx=0x7f044c02a680 comp rx=0 tx=0).stop 2024-05-11T12:14:21.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.690+0000 7f0459600640 1 -- 172.21.15.167:0/214408384 shutdown_connections 2024-05-11T12:14:21.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.690+0000 7f0459600640 1 --2- 172.21.15.167:0/214408384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04540ce8f0 0x7f04540ced80 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:21.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.690+0000 7f0459600640 1 --2- 172.21.15.167:0/214408384 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0454073270 0x7f045415d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:21.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.690+0000 7f0459600640 1 -- 172.21.15.167:0/214408384 >> 172.21.15.167:0/214408384 conn(0x7f045406c450 msgr2=0x7f045406c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:21.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.690+0000 7f0459600640 1 -- 172.21.15.167:0/214408384 shutdown_connections 2024-05-11T12:14:21.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.690+0000 7f0459600640 1 -- 172.21.15.167:0/214408384 wait complete. 2024-05-11T12:14:21.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.692+0000 7f0459600640 1 Processor -- start 2024-05-11T12:14:21.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.692+0000 7f0459600640 1 -- start start 2024-05-11T12:14:21.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.693+0000 7f0459600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0454073270 0x7f0454160320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:21.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.693+0000 7f0459600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0454073650 0x7f045415ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:21.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.693+0000 7f0459600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04540ce8f0 0x7f0454144520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:21.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.693+0000 7f0459600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0454070390 con 0x7f0454073270 2024-05-11T12:14:21.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.693+0000 7f0459600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f045415cce0 con 0x7f04540ce8f0 2024-05-11T12:14:21.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.693+0000 7f0459600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0454073ad0 con 0x7f0454073650 2024-05-11T12:14:21.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.693+0000 7f0452a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0454073270 0x7f0454160320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:21.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.693+0000 7f0452a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0454073270 0x7f0454160320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:51548/0 (socket says 172.21.15.167:51548) 2024-05-11T12:14:21.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.693+0000 7f0452a00640 1 -- 172.21.15.167:0/1064536558 learned_addr learned my addr 172.21.15.167:0/1064536558 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:21.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.694+0000 7f0452a00640 1 -- 172.21.15.167:0/1064536558 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0454073650 msgr2=0x7f045415ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:21.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.694+0000 7f044be00640 1 --2- 172.21.15.167:0/1064536558 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0454073650 0x7f045415ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:21.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.694+0000 7f0453400640 1 --2- 172.21.15.167:0/1064536558 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04540ce8f0 0x7f0454144520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:21.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.694+0000 7f0452a00640 1 --2- 172.21.15.167:0/1064536558 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0454073650 0x7f045415ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:21.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.694+0000 7f0452a00640 1 -- 172.21.15.167:0/1064536558 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04540ce8f0 msgr2=0x7f0454144520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:21.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.694+0000 7f0452a00640 1 --2- 172.21.15.167:0/1064536558 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04540ce8f0 0x7f0454144520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:21.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.694+0000 7f0452a00640 1 -- 172.21.15.167:0/1064536558 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04541f6b90 con 0x7f0454073270 2024-05-11T12:14:21.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.694+0000 7f0453400640 1 --2- 172.21.15.167:0/1064536558 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04540ce8f0 0x7f0454144520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:21.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.694+0000 7f0452a00640 1 --2- 172.21.15.167:0/1064536558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0454073270 0x7f0454160320 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f044400c7f0 tx=0x7f044400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:21.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.694+0000 7f044aa00640 1 -- 172.21.15.167:0/1064536558 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0444015070 con 0x7f0454073270 2024-05-11T12:14:21.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.694+0000 7f044aa00640 1 -- 172.21.15.167:0/1064536558 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0444004240 con 0x7f0454073270 2024-05-11T12:14:21.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.694+0000 7f044aa00640 1 -- 172.21.15.167:0/1064536558 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0444007620 con 0x7f0454073270 2024-05-11T12:14:21.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.694+0000 7f0459600640 1 -- 172.21.15.167:0/1064536558 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04540cf960 con 0x7f0454073270 2024-05-11T12:14:21.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.694+0000 7f0459600640 1 -- 172.21.15.167:0/1064536558 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f045415dc90 con 0x7f0454073270 2024-05-11T12:14:21.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.695+0000 7f0459600640 1 -- 172.21.15.167:0/1064536558 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0454069520 con 0x7f0454073270 2024-05-11T12:14:21.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.697+0000 7f044aa00640 1 -- 172.21.15.167:0/1064536558 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f04440079d0 con 0x7f0454073270 2024-05-11T12:14:21.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.698+0000 7f044aa00640 1 --2- 172.21.15.167:0/1064536558 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f04300762d0 0x7f0430078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:21.698 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.698+0000 7f044aa00640 1 -- 172.21.15.167:0/1064536558 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6187+0+0 (secure 0 0 0) 0x7f0444097b90 con 0x7f0454073270 2024-05-11T12:14:21.698 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.698+0000 7f044be00640 1 --2- 172.21.15.167:0/1064536558 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f04300762d0 0x7f0430078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:21.698 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.699+0000 7f044be00640 1 --2- 172.21.15.167:0/1064536558 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f04300762d0 0x7f0430078790 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f04541455c0 tx=0x7f043c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:21.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.702+0000 7f044aa00640 1 -- 172.21.15.167:0/1064536558 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f04440613a0 con 0x7f0454073270 2024-05-11T12:14:21.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:21.951+0000 7f0459600640 1 -- 172.21.15.167:0/1064536558 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f04540d0a60 con 0x7f0454073270 2024-05-11T12:14:22.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.483+0000 7f044aa00640 1 -- 172.21.15.167:0/1064536558 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v55) ==== 104+0+0 (secure 0 0 0) 0x7f0444060ca0 con 0x7f0454073270 2024-05-11T12:14:22.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.486+0000 7f0459600640 1 -- 172.21.15.167:0/1064536558 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f04300762d0 msgr2=0x7f0430078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:22.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.486+0000 7f0459600640 1 --2- 172.21.15.167:0/1064536558 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f04300762d0 0x7f0430078790 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f04541455c0 tx=0x7f043c009040 comp rx=0 tx=0).stop 2024-05-11T12:14:22.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.486+0000 7f0459600640 1 -- 172.21.15.167:0/1064536558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0454073270 msgr2=0x7f0454160320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:22.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.486+0000 7f0459600640 1 --2- 172.21.15.167:0/1064536558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0454073270 0x7f0454160320 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f044400c7f0 tx=0x7f044400ccb0 comp rx=0 tx=0).stop 2024-05-11T12:14:22.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.487+0000 7f0459600640 1 -- 172.21.15.167:0/1064536558 shutdown_connections 2024-05-11T12:14:22.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.487+0000 7f0459600640 1 --2- 172.21.15.167:0/1064536558 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f04300762d0 0x7f0430078790 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:22.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.487+0000 7f0459600640 1 --2- 172.21.15.167:0/1064536558 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04540ce8f0 0x7f0454144520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:22.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.487+0000 7f0459600640 1 --2- 172.21.15.167:0/1064536558 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0454073650 0x7f045415ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:22.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.487+0000 7f0459600640 1 --2- 172.21.15.167:0/1064536558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0454073270 0x7f0454160320 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:22.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.487+0000 7f0459600640 1 -- 172.21.15.167:0/1064536558 >> 172.21.15.167:0/1064536558 conn(0x7f045406c450 msgr2=0x7f04540af2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:22.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.487+0000 7f0459600640 1 -- 172.21.15.167:0/1064536558 shutdown_connections 2024-05-11T12:14:22.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.487+0000 7f0459600640 1 -- 172.21.15.167:0/1064536558 wait complete. 2024-05-11T12:14:22.508 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-11T12:14:22.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.711+0000 7f3997e00640 1 Processor -- start 2024-05-11T12:14:22.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.712+0000 7f3997e00640 1 -- start start 2024-05-11T12:14:22.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.713+0000 7f3997e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f39901628c0 0x7f39900d3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:22.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.713+0000 7f3997e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f39900b3f10 0x7f399006db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:22.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.713+0000 7f3997e00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3990163420 con 0x7f39900b4460 2024-05-11T12:14:22.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.713+0000 7f3997e00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f3990164030 con 0x7f39900b3f10 2024-05-11T12:14:22.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.713+0000 7f3997e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f3990163700 con 0x7f39901628c0 2024-05-11T12:14:22.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.714+0000 7f3995e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f39900b3f10 0x7f399006db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:22.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.714+0000 7f3994a00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f39900b4460 0x7f39900cf810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:39304/0 (socket says 172.21.15.167:39304) 2024-05-11T12:14:22.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.714+0000 7f3994a00640 1 -- 172.21.15.167:0/1032725436 learned_addr learned my addr 172.21.15.167:0/1032725436 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:14:22.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.714+0000 7f3995400640 1 --2- 172.21.15.167:0/1032725436 >> v2:172.21.15.204:3300/0 conn(0x7f39901628c0 0x7f39900d3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:22.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.714+0000 7f3995e00640 1 -- 172.21.15.167:0/1032725436 >> v2:172.21.15.204:3300/0 conn(0x7f39901628c0 msgr2=0x7f39900d3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:22.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.714+0000 7f3995e00640 1 --2- 172.21.15.167:0/1032725436 >> v2:172.21.15.204:3300/0 conn(0x7f39901628c0 0x7f39900d3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:22.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.714+0000 7f3995e00640 1 -- 172.21.15.167:0/1032725436 >> v1:172.21.15.167:6789/0 conn(0x7f39900b4460 legacy=0x7f39900cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:22.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.714+0000 7f3995e00640 1 -- 172.21.15.167:0/1032725436 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39901639e0 con 0x7f39900b3f10 2024-05-11T12:14:22.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.716+0000 7f3995e00640 1 --2- 172.21.15.167:0/1032725436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39900b3f10 0x7f399006db70 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f399008dd20 tx=0x7f398802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c820d4cf2666ce9b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:22.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.716+0000 7f398fe00640 1 -- 172.21.15.167:0/1032725436 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3988007740 con 0x7f39900b3f10 2024-05-11T12:14:22.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.716+0000 7f398fe00640 1 -- 172.21.15.167:0/1032725436 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39880078e0 con 0x7f39900b3f10 2024-05-11T12:14:22.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.716+0000 7f398fe00640 1 -- 172.21.15.167:0/1032725436 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3988007cb0 con 0x7f39900b3f10 2024-05-11T12:14:22.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.717+0000 7f3997e00640 1 -- 172.21.15.167:0/1032725436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39900b3f10 msgr2=0x7f399006db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:22.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.717+0000 7f3997e00640 1 --2- 172.21.15.167:0/1032725436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39900b3f10 0x7f399006db70 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f399008dd20 tx=0x7f398802a680 comp rx=0 tx=0).stop 2024-05-11T12:14:22.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.717+0000 7f3997e00640 1 -- 172.21.15.167:0/1032725436 shutdown_connections 2024-05-11T12:14:22.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.717+0000 7f3997e00640 1 --2- 172.21.15.167:0/1032725436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39900b3f10 0x7f399006db70 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:22.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.717+0000 7f3997e00640 1 --2- 172.21.15.167:0/1032725436 >> v2:172.21.15.204:3300/0 conn(0x7f39901628c0 0x7f39900d3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:22.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.717+0000 7f3997e00640 1 -- 172.21.15.167:0/1032725436 >> 172.21.15.167:0/1032725436 conn(0x7f39900b6cb0 msgr2=0x7f39900b7090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:22.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.717+0000 7f3997e00640 1 -- 172.21.15.167:0/1032725436 shutdown_connections 2024-05-11T12:14:22.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.717+0000 7f3997e00640 1 -- 172.21.15.167:0/1032725436 wait complete. 2024-05-11T12:14:22.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.718+0000 7f3997e00640 1 Processor -- start 2024-05-11T12:14:22.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.719+0000 7f3997e00640 1 -- start start 2024-05-11T12:14:22.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.719+0000 7f3997e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39900b3f10 0x7f39900cb290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:22.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.719+0000 7f3997e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39900b4460 0x7f399015d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:22.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.719+0000 7f3997e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39901628c0 0x7f39901f57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:22.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.719+0000 7f3997e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3990162ca0 con 0x7f39900b4460 2024-05-11T12:14:22.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.719+0000 7f3997e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f39901630d0 con 0x7f39901628c0 2024-05-11T12:14:22.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.719+0000 7f3997e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3990162e20 con 0x7f39900b3f10 2024-05-11T12:14:22.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.720+0000 7f3994a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39900b4460 0x7f399015d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:22.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.720+0000 7f3994a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39900b4460 0x7f399015d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:51564/0 (socket says 172.21.15.167:51564) 2024-05-11T12:14:22.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.720+0000 7f3994a00640 1 -- 172.21.15.167:0/2163242462 learned_addr learned my addr 172.21.15.167:0/2163242462 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:22.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.720+0000 7f3995400640 1 --2- 172.21.15.167:0/2163242462 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39900b3f10 0x7f39900cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:22.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.720+0000 7f3995e00640 1 --2- 172.21.15.167:0/2163242462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39901628c0 0x7f39901f57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:22.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.720+0000 7f3994a00640 1 -- 172.21.15.167:0/2163242462 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39900b3f10 msgr2=0x7f39900cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:22.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.720+0000 7f3994a00640 1 --2- 172.21.15.167:0/2163242462 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39900b3f10 0x7f39900cb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:22.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.720+0000 7f3994a00640 1 -- 172.21.15.167:0/2163242462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39901628c0 msgr2=0x7f39901f57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:22.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.720+0000 7f3994a00640 1 --2- 172.21.15.167:0/2163242462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39901628c0 0x7f39901f57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:22.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.720+0000 7f3994a00640 1 -- 172.21.15.167:0/2163242462 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3990159040 con 0x7f39900b4460 2024-05-11T12:14:22.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.720+0000 7f3995400640 1 --2- 172.21.15.167:0/2163242462 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39900b3f10 0x7f39900cb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:22.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.721+0000 7f3994a00640 1 --2- 172.21.15.167:0/2163242462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39900b4460 0x7f399015d4d0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f397c00c450 tx=0x7f397c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:22.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.721+0000 7f398d600640 1 -- 172.21.15.167:0/2163242462 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c017070 con 0x7f39900b4460 2024-05-11T12:14:22.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.721+0000 7f398d600640 1 -- 172.21.15.167:0/2163242462 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f397c005ce0 con 0x7f39900b4460 2024-05-11T12:14:22.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.721+0000 7f398d600640 1 -- 172.21.15.167:0/2163242462 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c0072b0 con 0x7f39900b4460 2024-05-11T12:14:22.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.721+0000 7f3997e00640 1 -- 172.21.15.167:0/2163242462 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3990159320 con 0x7f39900b4460 2024-05-11T12:14:22.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.722+0000 7f3997e00640 1 -- 172.21.15.167:0/2163242462 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39901598e0 con 0x7f39900b4460 2024-05-11T12:14:22.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.723+0000 7f3997e00640 1 -- 172.21.15.167:0/2163242462 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3990159ae0 con 0x7f39900b4460 2024-05-11T12:14:22.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.723+0000 7f398d600640 1 -- 172.21.15.167:0/2163242462 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f397c01a070 con 0x7f39900b4460 2024-05-11T12:14:22.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.724+0000 7f398d600640 1 --2- 172.21.15.167:0/2163242462 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f39700762d0 0x7f3970078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:22.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.724+0000 7f398d600640 1 -- 172.21.15.167:0/2163242462 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6187+0+0 (secure 0 0 0) 0x7f397c096ae0 con 0x7f39900b4460 2024-05-11T12:14:22.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.725+0000 7f3995400640 1 --2- 172.21.15.167:0/2163242462 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f39700762d0 0x7f3970078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:22.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.725+0000 7f3995400640 1 --2- 172.21.15.167:0/2163242462 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f39700762d0 0x7f3970078790 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3980006fd0 tx=0x7f3980008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:22.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.729+0000 7f398d600640 1 -- 172.21.15.167:0/2163242462 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3990159ae0 con 0x7f39900b4460 2024-05-11T12:14:22.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:22.937+0000 7f3997e00640 1 -- 172.21.15.167:0/2163242462 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3990069590 con 0x7f39900b4460 2024-05-11T12:14:23.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.488+0000 7f398d600640 1 -- 172.21.15.167:0/2163242462 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v88) ==== 181+0+0 (secure 0 0 0) 0x7f397c060260 con 0x7f39900b4460 2024-05-11T12:14:23.487 INFO:teuthology.orchestra.run.smithi167.stderr:pool 'cephfs_metadata' removed 2024-05-11T12:14:23.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.493+0000 7f3997e00640 1 -- 172.21.15.167:0/2163242462 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f39700762d0 msgr2=0x7f3970078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:23.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.493+0000 7f3997e00640 1 --2- 172.21.15.167:0/2163242462 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f39700762d0 0x7f3970078790 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3980006fd0 tx=0x7f3980008040 comp rx=0 tx=0).stop 2024-05-11T12:14:23.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.493+0000 7f3997e00640 1 -- 172.21.15.167:0/2163242462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39900b4460 msgr2=0x7f399015d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:23.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.493+0000 7f3997e00640 1 --2- 172.21.15.167:0/2163242462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39900b4460 0x7f399015d4d0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f397c00c450 tx=0x7f397c00c910 comp rx=0 tx=0).stop 2024-05-11T12:14:23.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.494+0000 7f3997e00640 1 -- 172.21.15.167:0/2163242462 shutdown_connections 2024-05-11T12:14:23.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.494+0000 7f3997e00640 1 --2- 172.21.15.167:0/2163242462 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f39700762d0 0x7f3970078790 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:23.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.494+0000 7f3997e00640 1 --2- 172.21.15.167:0/2163242462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39901628c0 0x7f39901f57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:23.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.494+0000 7f3997e00640 1 --2- 172.21.15.167:0/2163242462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39900b4460 0x7f399015d4d0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:23.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.494+0000 7f3997e00640 1 --2- 172.21.15.167:0/2163242462 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39900b3f10 0x7f39900cb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:23.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.494+0000 7f3997e00640 1 -- 172.21.15.167:0/2163242462 >> 172.21.15.167:0/2163242462 conn(0x7f39900b6cb0 msgr2=0x7f399006ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:23.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.494+0000 7f3997e00640 1 -- 172.21.15.167:0/2163242462 shutdown_connections 2024-05-11T12:14:23.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.495+0000 7f3997e00640 1 -- 172.21.15.167:0/2163242462 wait complete. 2024-05-11T12:14:23.520 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-11T12:14:23.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.721+0000 7f3e16600640 1 Processor -- start 2024-05-11T12:14:23.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.722+0000 7f3e16600640 1 -- start start 2024-05-11T12:14:23.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.723+0000 7f3e16600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e101439a0 0x7f3e100cfee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:23.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.723+0000 7f3e16600640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e100d0420 con 0x7f3e100b59f0 2024-05-11T12:14:23.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.723+0000 7f3e16600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e100b3f30 con 0x7f3e100cf820 2024-05-11T12:14:23.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.723+0000 7f3e16600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3e100b3ce0 con 0x7f3e101439a0 2024-05-11T12:14:23.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.724+0000 7f3e0fe00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f3e100b59f0 0x7f3e10073220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:39316/0 (socket says 172.21.15.167:39316) 2024-05-11T12:14:23.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.724+0000 7f3e0fe00640 1 -- 172.21.15.167:0/539159859 learned_addr learned my addr 172.21.15.167:0/539159859 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:14:23.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.724+0000 7f3e0f400640 1 --2- 172.21.15.167:0/539159859 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e101439a0 0x7f3e100cfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:23.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.724+0000 7f3e0ea00640 1 -- 172.21.15.167:0/539159859 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2506121203 0 0) 0x7f3e100d0420 con 0x7f3e100b59f0 2024-05-11T12:14:23.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.724+0000 7f3e0ea00640 1 -- 172.21.15.167:0/539159859 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3df4003520 con 0x7f3e100b59f0 2024-05-11T12:14:23.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.724+0000 7f3e0ea00640 1 -- 172.21.15.167:0/539159859 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 318206058 0 0) 0x7f3df4003520 con 0x7f3e100b59f0 2024-05-11T12:14:23.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.724+0000 7f3e0ea00640 1 -- 172.21.15.167:0/539159859 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e101439a0 msgr2=0x7f3e100cfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:23.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.724+0000 7f3e0ea00640 1 --2- 172.21.15.167:0/539159859 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e101439a0 0x7f3e100cfee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:23.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.724+0000 7f3e0ea00640 1 -- 172.21.15.167:0/539159859 >> v1:172.21.15.204:6789/0 conn(0x7f3e100cf820 legacy=0x7f3e100b52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:23.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.725+0000 7f3e0ea00640 1 -- 172.21.15.167:0/539159859 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e100b4210 con 0x7f3e100b59f0 2024-05-11T12:14:23.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.725+0000 7f3e0ea00640 1 -- 172.21.15.167:0/539159859 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f3e08001a70 con 0x7f3e100b59f0 2024-05-11T12:14:23.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.725+0000 7f3e0ea00640 1 -- 172.21.15.167:0/539159859 >> v1:172.21.15.167:6789/0 conn(0x7f3e100b59f0 legacy=0x7f3e10073220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:23.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.725+0000 7f3e0ea00640 1 --2- 172.21.15.167:0/539159859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3df40039e0 0x7f3df4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:23.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.725+0000 7f3e0ea00640 1 --2- 172.21.15.167:0/539159859 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e101439a0 0x7f3df4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:23.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.725+0000 7f3e0ea00640 1 --2- 172.21.15.167:0/539159859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3df4004d40 0x7f3df40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:23.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.725+0000 7f3e0ea00640 1 -- 172.21.15.167:0/539159859 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3e100b3ce0 con 0x7f3df4004d40 2024-05-11T12:14:23.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.725+0000 7f3e0ea00640 1 -- 172.21.15.167:0/539159859 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3df400bb50 con 0x7f3df40039e0 2024-05-11T12:14:23.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.725+0000 7f3e0ea00640 1 -- 172.21.15.167:0/539159859 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3df400bd20 con 0x7f3e101439a0 2024-05-11T12:14:23.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.725+0000 7f3e0fe00640 1 --2- 172.21.15.167:0/539159859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3df4004d40 0x7f3df40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:23.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.725+0000 7f3e0f400640 1 --2- 172.21.15.167:0/539159859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3df40039e0 0x7f3df4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:23.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.725+0000 7f3e07e00640 1 --2- 172.21.15.167:0/539159859 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e101439a0 0x7f3df4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:23.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.725+0000 7f3e0fe00640 1 -- 172.21.15.167:0/539159859 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e101439a0 msgr2=0x7f3df4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:23.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.725+0000 7f3e0fe00640 1 --2- 172.21.15.167:0/539159859 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e101439a0 0x7f3df4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:23.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.725+0000 7f3e0fe00640 1 -- 172.21.15.167:0/539159859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3df40039e0 msgr2=0x7f3df4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:23.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.725+0000 7f3e0fe00640 1 --2- 172.21.15.167:0/539159859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3df40039e0 0x7f3df4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:23.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.725+0000 7f3e0fe00640 1 -- 172.21.15.167:0/539159859 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e100b4210 con 0x7f3df4004d40 2024-05-11T12:14:23.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.727+0000 7f3e0fe00640 1 --2- 172.21.15.167:0/539159859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3df4004d40 0x7f3df40051c0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f3e0800a970 tx=0x7f3df4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57c5ece2a7700340 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:23.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.727+0000 7f3e0ea00640 1 -- 172.21.15.167:0/539159859 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0800a5c0 con 0x7f3df4004d40 2024-05-11T12:14:23.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.727+0000 7f3e0ea00640 1 -- 172.21.15.167:0/539159859 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e0802f070 con 0x7f3df4004d40 2024-05-11T12:14:23.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.727+0000 7f3e0ea00640 1 -- 172.21.15.167:0/539159859 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0802bd80 con 0x7f3df4004d40 2024-05-11T12:14:23.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.727+0000 7f3e16600640 1 -- 172.21.15.167:0/539159859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3df4004d40 msgr2=0x7f3df40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:23.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.727+0000 7f3e16600640 1 --2- 172.21.15.167:0/539159859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3df4004d40 0x7f3df40051c0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f3e0800a970 tx=0x7f3df4005830 comp rx=0 tx=0).stop 2024-05-11T12:14:23.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.727+0000 7f3e0fe00640 1 -- 172.21.15.167:0/539159859 reap_dead start 2024-05-11T12:14:23.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.727+0000 7f3e16600640 1 -- 172.21.15.167:0/539159859 shutdown_connections 2024-05-11T12:14:23.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.727+0000 7f3e16600640 1 -- 172.21.15.167:0/539159859 >> 172.21.15.167:0/539159859 conn(0x7f3e100d0c70 msgr2=0x7f3e100d1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:23.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.728+0000 7f3e16600640 1 -- 172.21.15.167:0/539159859 shutdown_connections 2024-05-11T12:14:23.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.728+0000 7f3e16600640 1 -- 172.21.15.167:0/539159859 wait complete. 2024-05-11T12:14:23.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.729+0000 7f3e16600640 1 Processor -- start 2024-05-11T12:14:23.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.729+0000 7f3e16600640 1 -- start start 2024-05-11T12:14:23.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.730+0000 7f3e16600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3df4004d40 0x7f3e1015a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:23.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.730+0000 7f3e16600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e10158c00 0x7f3e100afac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:23.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.730+0000 7f3e16600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e101f5ee0 0x7f3e101f6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:23.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.730+0000 7f3e16600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3e100b4ab0 con 0x7f3e101f5ee0 2024-05-11T12:14:23.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.730+0000 7f3e16600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3e100b47d0 con 0x7f3e10158c00 2024-05-11T12:14:23.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.730+0000 7f3e16600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3e100cfee0 con 0x7f3df4004d40 2024-05-11T12:14:23.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.731+0000 7f3e0fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e101f5ee0 0x7f3e101f6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:23.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.731+0000 7f3e0fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e101f5ee0 0x7f3e101f6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:51572/0 (socket says 172.21.15.167:51572) 2024-05-11T12:14:23.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.731+0000 7f3e0fe00640 1 -- 172.21.15.167:0/850079644 learned_addr learned my addr 172.21.15.167:0/850079644 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:23.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.731+0000 7f3e0f400640 1 --2- 172.21.15.167:0/850079644 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3df4004d40 0x7f3e1015a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:23.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.731+0000 7f3e0fe00640 1 -- 172.21.15.167:0/850079644 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3df4004d40 msgr2=0x7f3e1015a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:23.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.731+0000 7f3e0fe00640 1 --2- 172.21.15.167:0/850079644 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3df4004d40 0x7f3e1015a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:23.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.731+0000 7f3e0fe00640 1 -- 172.21.15.167:0/850079644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e10158c00 msgr2=0x7f3e100afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:23.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.731+0000 7f3e0fe00640 1 --2- 172.21.15.167:0/850079644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e10158c00 0x7f3e100afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:23.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.731+0000 7f3e0fe00640 1 -- 172.21.15.167:0/850079644 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e10159080 con 0x7f3e101f5ee0 2024-05-11T12:14:23.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.731+0000 7f3e0fe00640 1 --2- 172.21.15.167:0/850079644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e101f5ee0 0x7f3e101f6390 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f3e0802b630 tx=0x7f3e08001db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:23.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.732+0000 7f3e07400640 1 -- 172.21.15.167:0/850079644 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0800a5c0 con 0x7f3e101f5ee0 2024-05-11T12:14:23.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.732+0000 7f3e07400640 1 -- 172.21.15.167:0/850079644 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e08034470 con 0x7f3e101f5ee0 2024-05-11T12:14:23.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.732+0000 7f3e07400640 1 -- 172.21.15.167:0/850079644 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e08034820 con 0x7f3e101f5ee0 2024-05-11T12:14:23.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.732+0000 7f3e16600640 1 -- 172.21.15.167:0/850079644 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e1015b1f0 con 0x7f3e101f5ee0 2024-05-11T12:14:23.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.732+0000 7f3e16600640 1 -- 172.21.15.167:0/850079644 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e1015b7b0 con 0x7f3e101f5ee0 2024-05-11T12:14:23.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.733+0000 7f3e16600640 1 -- 172.21.15.167:0/850079644 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e10069590 con 0x7f3e101f5ee0 2024-05-11T12:14:23.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.734+0000 7f3e07400640 1 -- 172.21.15.167:0/850079644 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3e0803e070 con 0x7f3e101f5ee0 2024-05-11T12:14:23.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.735+0000 7f3e07400640 1 --2- 172.21.15.167:0/850079644 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3dec0762d0 0x7f3dec078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:23.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.735+0000 7f3e07400640 1 -- 172.21.15.167:0/850079644 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5749+0+0 (secure 0 0 0) 0x7f3e080b6e50 con 0x7f3e101f5ee0 2024-05-11T12:14:23.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.736+0000 7f3e0f400640 1 --2- 172.21.15.167:0/850079644 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3dec0762d0 0x7f3dec078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:23.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.736+0000 7f3e0f400640 1 --2- 172.21.15.167:0/850079644 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3dec0762d0 0x7f3dec078790 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3e00006fd0 tx=0x7f3e00008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:23.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.740+0000 7f3e07400640 1 -- 172.21.15.167:0/850079644 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3e08080770 con 0x7f3e101f5ee0 2024-05-11T12:14:23.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:23.949+0000 7f3e16600640 1 -- 172.21.15.167:0/850079644 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3e10075bb0 con 0x7f3e101f5ee0 2024-05-11T12:14:24.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.491+0000 7f3e07400640 1 -- 172.21.15.167:0/850079644 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v89) ==== 169+0+0 (secure 0 0 0) 0x7f3e08080070 con 0x7f3e101f5ee0 2024-05-11T12:14:24.491 INFO:teuthology.orchestra.run.smithi167.stderr:pool 'cephfs_data' removed 2024-05-11T12:14:24.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.495+0000 7f3e16600640 1 -- 172.21.15.167:0/850079644 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3dec0762d0 msgr2=0x7f3dec078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:24.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.495+0000 7f3e16600640 1 --2- 172.21.15.167:0/850079644 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3dec0762d0 0x7f3dec078790 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3e00006fd0 tx=0x7f3e00008040 comp rx=0 tx=0).stop 2024-05-11T12:14:24.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.495+0000 7f3e16600640 1 -- 172.21.15.167:0/850079644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e101f5ee0 msgr2=0x7f3e101f6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:24.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.495+0000 7f3e16600640 1 --2- 172.21.15.167:0/850079644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e101f5ee0 0x7f3e101f6390 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f3e0802b630 tx=0x7f3e08001db0 comp rx=0 tx=0).stop 2024-05-11T12:14:24.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.495+0000 7f3e16600640 1 -- 172.21.15.167:0/850079644 shutdown_connections 2024-05-11T12:14:24.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.495+0000 7f3e16600640 1 --2- 172.21.15.167:0/850079644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e101f5ee0 0x7f3e101f6390 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:24.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.495+0000 7f3e16600640 1 --2- 172.21.15.167:0/850079644 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3dec0762d0 0x7f3dec078790 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:24.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.495+0000 7f3e16600640 1 --2- 172.21.15.167:0/850079644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e10158c00 0x7f3e100afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:24.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.495+0000 7f3e16600640 1 --2- 172.21.15.167:0/850079644 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3df4004d40 0x7f3e1015a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:24.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.495+0000 7f3e16600640 1 -- 172.21.15.167:0/850079644 >> 172.21.15.167:0/850079644 conn(0x7f3e100d0c70 msgr2=0x7f3e10161640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:24.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.496+0000 7f3e16600640 1 -- 172.21.15.167:0/850079644 shutdown_connections 2024-05-11T12:14:24.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.496+0000 7f3e16600640 1 -- 172.21.15.167:0/850079644 wait complete. 2024-05-11T12:14:24.518 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-11T12:14:24.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.711+0000 7fd809800640 1 Processor -- start 2024-05-11T12:14:24.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.711+0000 7fd809800640 1 -- start start 2024-05-11T12:14:24.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.714+0000 7fd809800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd80415ac80 0x7fd804074260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:24.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.714+0000 7fd809800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd8040b3d50 0x7fd804160c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:24.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.714+0000 7fd809800640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd8041611b0 0x7fd80415d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:24.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.714+0000 7fd809800640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fd80406e3c0 con 0x7fd8041611b0 2024-05-11T12:14:24.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.714+0000 7fd809800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fd80415b060 con 0x7fd80415ac80 2024-05-11T12:14:24.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.714+0000 7fd809800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd8040b65e0 con 0x7fd8040b3d50 2024-05-11T12:14:24.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.714+0000 7fd803400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd8041611b0 0x7fd80415d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:24.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.714+0000 7fd803400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd8041611b0 0x7fd80415d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:51582/0 (socket says 172.21.15.167:51582) 2024-05-11T12:14:24.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.714+0000 7fd803400640 1 -- 172.21.15.167:0/1141503613 learned_addr learned my addr 172.21.15.167:0/1141503613 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:24.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.715+0000 7fd803400640 1 -- 172.21.15.167:0/1141503613 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd8040b3d50 msgr2=0x7fd804160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:24.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.715+0000 7fd803400640 1 --2- 172.21.15.167:0/1141503613 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd8040b3d50 0x7fd804160c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:24.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.715+0000 7fd803400640 1 -- 172.21.15.167:0/1141503613 >> v2:172.21.15.204:3300/0 conn(0x7fd80415ac80 msgr2=0x7fd804074260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:24.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.715+0000 7fd803400640 1 --2- 172.21.15.167:0/1141503613 >> v2:172.21.15.204:3300/0 conn(0x7fd80415ac80 0x7fd804074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:24.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.715+0000 7fd803400640 1 -- 172.21.15.167:0/1141503613 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8040b68c0 con 0x7fd8041611b0 2024-05-11T12:14:24.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.716+0000 7fd803400640 1 --2- 172.21.15.167:0/1141503613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8041611b0 0x7fd80415d680 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fd80406a770 tx=0x7fd7fc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85261b537d01f4c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:24.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.717+0000 7fd802000640 1 -- 172.21.15.167:0/1141503613 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc037070 con 0x7fd8041611b0 2024-05-11T12:14:24.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.717+0000 7fd802000640 1 -- 172.21.15.167:0/1141503613 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7fc032440 con 0x7fd8041611b0 2024-05-11T12:14:24.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.717+0000 7fd802000640 1 -- 172.21.15.167:0/1141503613 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc032810 con 0x7fd8041611b0 2024-05-11T12:14:24.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.717+0000 7fd809800640 1 -- 172.21.15.167:0/1141503613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8041611b0 msgr2=0x7fd80415d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:24.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.717+0000 7fd809800640 1 --2- 172.21.15.167:0/1141503613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8041611b0 0x7fd80415d680 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fd80406a770 tx=0x7fd7fc02a1e0 comp rx=0 tx=0).stop 2024-05-11T12:14:24.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.718+0000 7fd809800640 1 -- 172.21.15.167:0/1141503613 shutdown_connections 2024-05-11T12:14:24.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.718+0000 7fd809800640 1 --2- 172.21.15.167:0/1141503613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8041611b0 0x7fd80415d680 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:24.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.718+0000 7fd809800640 1 --2- 172.21.15.167:0/1141503613 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd8040b3d50 0x7fd804160c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:24.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.718+0000 7fd809800640 1 --2- 172.21.15.167:0/1141503613 >> v2:172.21.15.204:3300/0 conn(0x7fd80415ac80 0x7fd804074260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:24.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.718+0000 7fd809800640 1 -- 172.21.15.167:0/1141503613 >> 172.21.15.167:0/1141503613 conn(0x7fd80406c450 msgr2=0x7fd80406c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:24.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.718+0000 7fd809800640 1 -- 172.21.15.167:0/1141503613 shutdown_connections 2024-05-11T12:14:24.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.718+0000 7fd809800640 1 -- 172.21.15.167:0/1141503613 wait complete. 2024-05-11T12:14:24.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.719+0000 7fd809800640 1 Processor -- start 2024-05-11T12:14:24.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.720+0000 7fd809800640 1 -- start start 2024-05-11T12:14:24.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.720+0000 7fd809800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8040b3d50 0x7fd8040b1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:24.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.720+0000 7fd809800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd80415ac80 0x7fd804159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:24.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.720+0000 7fd809800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8041611b0 0x7fd804159b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:24.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.721+0000 7fd809800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd8040b44f0 con 0x7fd8041611b0 2024-05-11T12:14:24.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.721+0000 7fd809800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd8040b41b0 con 0x7fd8040b3d50 2024-05-11T12:14:24.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.721+0000 7fd809800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd80415dbc0 con 0x7fd80415ac80 2024-05-11T12:14:24.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.721+0000 7fd803400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8041611b0 0x7fd804159b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:24.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.721+0000 7fd803400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8041611b0 0x7fd804159b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:51598/0 (socket says 172.21.15.167:51598) 2024-05-11T12:14:24.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.721+0000 7fd803400640 1 -- 172.21.15.167:0/3814894912 learned_addr learned my addr 172.21.15.167:0/3814894912 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:24.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.721+0000 7fd7fa000640 1 --2- 172.21.15.167:0/3814894912 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd80415ac80 0x7fd804159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:24.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.721+0000 7fd802a00640 1 --2- 172.21.15.167:0/3814894912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8040b3d50 0x7fd8040b1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:24.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.721+0000 7fd803400640 1 -- 172.21.15.167:0/3814894912 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd80415ac80 msgr2=0x7fd804159640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:24.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.721+0000 7fd803400640 1 --2- 172.21.15.167:0/3814894912 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd80415ac80 0x7fd804159640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:24.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.721+0000 7fd803400640 1 -- 172.21.15.167:0/3814894912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8040b3d50 msgr2=0x7fd8040b1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:24.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.721+0000 7fd803400640 1 --2- 172.21.15.167:0/3814894912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8040b3d50 0x7fd8040b1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:24.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.721+0000 7fd803400640 1 -- 172.21.15.167:0/3814894912 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8040aa410 con 0x7fd8041611b0 2024-05-11T12:14:24.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.721+0000 7fd802a00640 1 --2- 172.21.15.167:0/3814894912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8040b3d50 0x7fd8040b1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:24.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.722+0000 7fd803400640 1 --2- 172.21.15.167:0/3814894912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8041611b0 0x7fd804159b80 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc0090b0 tx=0x7fd7fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:24.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.722+0000 7fd7fb400640 1 -- 172.21.15.167:0/3814894912 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc041020 con 0x7fd8041611b0 2024-05-11T12:14:24.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.722+0000 7fd7fb400640 1 -- 172.21.15.167:0/3814894912 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7fc030990 con 0x7fd8041611b0 2024-05-11T12:14:24.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.722+0000 7fd7fb400640 1 -- 172.21.15.167:0/3814894912 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc037040 con 0x7fd8041611b0 2024-05-11T12:14:24.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.722+0000 7fd809800640 1 -- 172.21.15.167:0/3814894912 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8040afcb0 con 0x7fd8041611b0 2024-05-11T12:14:24.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.722+0000 7fd809800640 1 -- 172.21.15.167:0/3814894912 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8040b0270 con 0x7fd8041611b0 2024-05-11T12:14:24.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.724+0000 7fd809800640 1 -- 172.21.15.167:0/3814894912 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd804069520 con 0x7fd8041611b0 2024-05-11T12:14:24.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.725+0000 7fd7fb400640 1 -- 172.21.15.167:0/3814894912 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd7fc044020 con 0x7fd8041611b0 2024-05-11T12:14:24.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.725+0000 7fd7fb400640 1 --2- 172.21.15.167:0/3814894912 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd7e00762d0 0x7fd7e0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:24.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.726+0000 7fd7fb400640 1 -- 172.21.15.167:0/3814894912 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5335+0+0 (secure 0 0 0) 0x7fd7fc0b7ef0 con 0x7fd8041611b0 2024-05-11T12:14:24.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.726+0000 7fd802a00640 1 --2- 172.21.15.167:0/3814894912 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd7e00762d0 0x7fd7e0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:24.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.727+0000 7fd802a00640 1 --2- 172.21.15.167:0/3814894912 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd7e00762d0 0x7fd7e0078790 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4007eb0 tx=0x7fd7f40073c0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:24.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.730+0000 7fd7fb400640 1 -- 172.21.15.167:0/3814894912 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd7fc04b030 con 0x7fd8041611b0 2024-05-11T12:14:24.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.949+0000 7fd809800640 1 -- 172.21.15.167:0/3814894912 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fd804075bb0 con 0x7fd8041611b0 2024-05-11T12:14:24.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.957+0000 7fd7fb400640 1 -- 172.21.15.167:0/3814894912 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) ==== 104+0+0 (secure 0 0 0) 0x7fd7fc081930 con 0x7fd8041611b0 2024-05-11T12:14:24.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.960+0000 7fd809800640 1 -- 172.21.15.167:0/3814894912 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd7e00762d0 msgr2=0x7fd7e0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:24.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.960+0000 7fd809800640 1 --2- 172.21.15.167:0/3814894912 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd7e00762d0 0x7fd7e0078790 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4007eb0 tx=0x7fd7f40073c0 comp rx=0 tx=0).stop 2024-05-11T12:14:24.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.960+0000 7fd809800640 1 -- 172.21.15.167:0/3814894912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8041611b0 msgr2=0x7fd804159b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:24.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.960+0000 7fd809800640 1 --2- 172.21.15.167:0/3814894912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8041611b0 0x7fd804159b80 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc0090b0 tx=0x7fd7fc0046c0 comp rx=0 tx=0).stop 2024-05-11T12:14:24.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.960+0000 7fd809800640 1 -- 172.21.15.167:0/3814894912 shutdown_connections 2024-05-11T12:14:24.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.960+0000 7fd809800640 1 --2- 172.21.15.167:0/3814894912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8041611b0 0x7fd804159b80 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:24.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.960+0000 7fd809800640 1 --2- 172.21.15.167:0/3814894912 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd80415ac80 0x7fd804159640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:24.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.960+0000 7fd809800640 1 --2- 172.21.15.167:0/3814894912 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd7e00762d0 0x7fd7e0078790 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:24.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.960+0000 7fd809800640 1 --2- 172.21.15.167:0/3814894912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8040b3d50 0x7fd8040b1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:24.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.960+0000 7fd809800640 1 -- 172.21.15.167:0/3814894912 >> 172.21.15.167:0/3814894912 conn(0x7fd80406c450 msgr2=0x7fd8040b5a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:24.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.960+0000 7fd809800640 1 -- 172.21.15.167:0/3814894912 shutdown_connections 2024-05-11T12:14:24.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:24.960+0000 7fd809800640 1 -- 172.21.15.167:0/3814894912 wait complete. 2024-05-11T12:14:24.979 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_session_drop' 2024-05-11T12:14:25.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.178+0000 7fac25400640 1 Processor -- start 2024-05-11T12:14:25.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.178+0000 7fac25400640 1 -- start start 2024-05-11T12:14:25.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.180+0000 7fac25400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fac200af5d0 0x7fac20073ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:25.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.180+0000 7fac25400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fac2006d460 0x7fac2006d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:25.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.180+0000 7fac25400640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac2006fe10 con 0x7fac200af1f0 2024-05-11T12:14:25.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.180+0000 7fac25400640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fac201608d0 con 0x7fac200af5d0 2024-05-11T12:14:25.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.180+0000 7fac25400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fac2006de10 con 0x7fac2006d460 2024-05-11T12:14:25.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.180+0000 7fac17e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fac200af5d0 0x7fac20073ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:25.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.180+0000 7fac1e800640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fac200af1f0 0x7fac200cf1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:39324/0 (socket says 172.21.15.167:39324) 2024-05-11T12:14:25.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.180+0000 7fac1e800640 1 -- 172.21.15.167:0/20741956 learned_addr learned my addr 172.21.15.167:0/20741956 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:14:25.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.181+0000 7fac1f200640 1 --2- 172.21.15.167:0/20741956 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fac2006d460 0x7fac2006d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:25.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.181+0000 7fac17e00640 1 -- 172.21.15.167:0/20741956 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fac2006d460 msgr2=0x7fac2006d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:25.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.181+0000 7fac17e00640 1 --2- 172.21.15.167:0/20741956 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fac2006d460 0x7fac2006d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:25.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.181+0000 7fac17e00640 1 -- 172.21.15.167:0/20741956 >> v1:172.21.15.167:6789/0 conn(0x7fac200af1f0 legacy=0x7fac200cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:25.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.181+0000 7fac17e00640 1 -- 172.21.15.167:0/20741956 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac2015fcc0 con 0x7fac200af5d0 2024-05-11T12:14:25.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.181+0000 7fac1de00640 1 -- 172.21.15.167:0/20741956 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3242397292 0 0) 0x7fac2006fe10 con 0x7fac200af1f0 2024-05-11T12:14:25.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.181+0000 7fac1f200640 1 --2- 172.21.15.167:0/20741956 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fac2006d460 0x7fac2006d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:25.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.182+0000 7fac17e00640 1 --2- 172.21.15.167:0/20741956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac200af5d0 0x7fac20073ce0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fac080097a0 tx=0x7fac0802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5daf59dccfad7794 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:25.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.182+0000 7fac1de00640 1 -- 172.21.15.167:0/20741956 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac08037070 con 0x7fac200af5d0 2024-05-11T12:14:25.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.182+0000 7fac1de00640 1 -- 172.21.15.167:0/20741956 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac0802ae50 con 0x7fac200af5d0 2024-05-11T12:14:25.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.182+0000 7fac1de00640 1 -- 172.21.15.167:0/20741956 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac08032660 con 0x7fac200af5d0 2024-05-11T12:14:25.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.183+0000 7fac25400640 1 -- 172.21.15.167:0/20741956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac200af5d0 msgr2=0x7fac20073ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:25.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.183+0000 7fac25400640 1 --2- 172.21.15.167:0/20741956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac200af5d0 0x7fac20073ce0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fac080097a0 tx=0x7fac0802a100 comp rx=0 tx=0).stop 2024-05-11T12:14:25.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.183+0000 7fac25400640 1 -- 172.21.15.167:0/20741956 shutdown_connections 2024-05-11T12:14:25.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.183+0000 7fac25400640 1 --2- 172.21.15.167:0/20741956 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fac2006d460 0x7fac2006d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:25.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.183+0000 7fac25400640 1 --2- 172.21.15.167:0/20741956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac200af5d0 0x7fac20073ce0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:25.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.183+0000 7fac25400640 1 -- 172.21.15.167:0/20741956 >> 172.21.15.167:0/20741956 conn(0x7fac200cd8f0 msgr2=0x7fac200cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:25.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.183+0000 7fac25400640 1 -- 172.21.15.167:0/20741956 shutdown_connections 2024-05-11T12:14:25.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.183+0000 7fac25400640 1 -- 172.21.15.167:0/20741956 wait complete. 2024-05-11T12:14:25.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.185+0000 7fac25400640 1 Processor -- start 2024-05-11T12:14:25.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.186+0000 7fac25400640 1 -- start start 2024-05-11T12:14:25.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.187+0000 7fac25400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fac2006d460 0x7fac2015b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:25.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.187+0000 7fac25400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac200af1f0 0x7fac20156f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:25.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.187+0000 7fac25400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fac200af5d0 0x7fac20162e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:25.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.187+0000 7fac25400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fac20160560 con 0x7fac200af1f0 2024-05-11T12:14:25.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.187+0000 7fac25400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fac20160280 con 0x7fac2006d460 2024-05-11T12:14:25.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.187+0000 7fac25400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fac2015f920 con 0x7fac200af5d0 2024-05-11T12:14:25.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.187+0000 7fac17e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac200af1f0 0x7fac20156f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:25.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.187+0000 7fac17e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac200af1f0 0x7fac20156f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:51614/0 (socket says 172.21.15.167:51614) 2024-05-11T12:14:25.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.187+0000 7fac17e00640 1 -- 172.21.15.167:0/1648968286 learned_addr learned my addr 172.21.15.167:0/1648968286 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:25.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.187+0000 7fac1e800640 1 --2- 172.21.15.167:0/1648968286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fac2006d460 0x7fac2015b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:25.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.187+0000 7fac1f200640 1 --2- 172.21.15.167:0/1648968286 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fac200af5d0 0x7fac20162e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:25.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.187+0000 7fac17e00640 1 -- 172.21.15.167:0/1648968286 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fac200af5d0 msgr2=0x7fac20162e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:25.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.188+0000 7fac17e00640 1 --2- 172.21.15.167:0/1648968286 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fac200af5d0 0x7fac20162e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:25.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.188+0000 7fac17e00640 1 -- 172.21.15.167:0/1648968286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fac2006d460 msgr2=0x7fac2015b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:25.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.188+0000 7fac17e00640 1 --2- 172.21.15.167:0/1648968286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fac2006d460 0x7fac2015b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:25.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.188+0000 7fac17e00640 1 -- 172.21.15.167:0/1648968286 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac20157490 con 0x7fac200af1f0 2024-05-11T12:14:25.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.188+0000 7fac1f200640 1 --2- 172.21.15.167:0/1648968286 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fac200af5d0 0x7fac20162e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:25.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.188+0000 7fac17e00640 1 --2- 172.21.15.167:0/1648968286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac200af1f0 0x7fac20156f50 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fac080097a0 tx=0x7fac080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:25.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.188+0000 7fac16a00640 1 -- 172.21.15.167:0/1648968286 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac08037040 con 0x7fac200af1f0 2024-05-11T12:14:25.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.188+0000 7fac16a00640 1 -- 172.21.15.167:0/1648968286 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac08004250 con 0x7fac200af1f0 2024-05-11T12:14:25.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.188+0000 7fac16a00640 1 -- 172.21.15.167:0/1648968286 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac08003610 con 0x7fac200af1f0 2024-05-11T12:14:25.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.188+0000 7fac25400640 1 -- 172.21.15.167:0/1648968286 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac201f4c30 con 0x7fac200af1f0 2024-05-11T12:14:25.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.188+0000 7fac25400640 1 -- 172.21.15.167:0/1648968286 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac201f51f0 con 0x7fac200af1f0 2024-05-11T12:14:25.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.190+0000 7fac25400640 1 -- 172.21.15.167:0/1648968286 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac20069590 con 0x7fac200af1f0 2024-05-11T12:14:25.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.191+0000 7fac16a00640 1 -- 172.21.15.167:0/1648968286 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fac080037b0 con 0x7fac200af1f0 2024-05-11T12:14:25.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.192+0000 7fac16a00640 1 --2- 172.21.15.167:0/1648968286 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fabfc0762d0 0x7fabfc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:25.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.193+0000 7fac16a00640 1 -- 172.21.15.167:0/1648968286 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5335+0+0 (secure 0 0 0) 0x7fac080b8400 con 0x7fac200af1f0 2024-05-11T12:14:25.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.193+0000 7fac1e800640 1 --2- 172.21.15.167:0/1648968286 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fabfc0762d0 0x7fabfc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:25.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.194+0000 7fac1e800640 1 --2- 172.21.15.167:0/1648968286 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fabfc0762d0 0x7fabfc078790 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fac10000f60 tx=0x7fac10007b10 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:25.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.197+0000 7fac16a00640 1 -- 172.21.15.167:0/1648968286 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fac08081e30 con 0x7fac200af1f0 2024-05-11T12:14:25.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.478+0000 7fac25400640 1 -- 172.21.15.167:0/1648968286 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_session_drop"]} v 0) -- 0x7fac2015e330 con 0x7fac200af1f0 2024-05-11T12:14:25.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.509+0000 7fac16a00640 1 -- 172.21.15.167:0/1648968286 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_session_drop"]}]=0 v223) ==== 145+0+0 (secure 0 0 0) 0x7fac08081730 con 0x7fac200af1f0 2024-05-11T12:14:25.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.512+0000 7fac25400640 1 -- 172.21.15.167:0/1648968286 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fabfc0762d0 msgr2=0x7fabfc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:25.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.512+0000 7fac25400640 1 --2- 172.21.15.167:0/1648968286 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fabfc0762d0 0x7fabfc078790 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fac10000f60 tx=0x7fac10007b10 comp rx=0 tx=0).stop 2024-05-11T12:14:25.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.512+0000 7fac25400640 1 -- 172.21.15.167:0/1648968286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac200af1f0 msgr2=0x7fac20156f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:25.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.512+0000 7fac25400640 1 --2- 172.21.15.167:0/1648968286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac200af1f0 0x7fac20156f50 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fac080097a0 tx=0x7fac080046c0 comp rx=0 tx=0).stop 2024-05-11T12:14:25.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.512+0000 7fac25400640 1 -- 172.21.15.167:0/1648968286 shutdown_connections 2024-05-11T12:14:25.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.512+0000 7fac25400640 1 --2- 172.21.15.167:0/1648968286 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fac200af5d0 0x7fac20162e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:25.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.512+0000 7fac25400640 1 --2- 172.21.15.167:0/1648968286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac200af1f0 0x7fac20156f50 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:25.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.512+0000 7fac25400640 1 --2- 172.21.15.167:0/1648968286 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fabfc0762d0 0x7fabfc078790 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:25.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.512+0000 7fac25400640 1 --2- 172.21.15.167:0/1648968286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fac2006d460 0x7fac2015b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:25.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.512+0000 7fac25400640 1 -- 172.21.15.167:0/1648968286 >> 172.21.15.167:0/1648968286 conn(0x7fac200cd8f0 msgr2=0x7fac2006e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:25.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.513+0000 7fac25400640 1 -- 172.21.15.167:0/1648968286 shutdown_connections 2024-05-11T12:14:25.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.513+0000 7fac25400640 1 -- 172.21.15.167:0/1648968286 wait complete. 2024-05-11T12:14:25.530 INFO:tasks.cephfs_test_runner:That when a `tell` command is sent using the python CLI, ... ok 2024-05-11T12:14:25.530 INFO:tasks.cephfs_test_runner:Starting test: test_version_splitting (tasks.cephfs.test_sessionmap.TestSessionMap) That when many sessions are updated, they are correctly 2024-05-11T12:14:25.530 INFO:tasks.cephfs_test_runner:test_version_splitting (tasks.cephfs.test_sessionmap.TestSessionMap) 2024-05-11T12:14:25.531 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:14:25.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.723+0000 7f0361000640 1 Processor -- start 2024-05-11T12:14:25.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.724+0000 7f0361000640 1 -- start start 2024-05-11T12:14:25.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.725+0000 7f0361000640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f035c0b3ea0 0x7f035c161490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:25.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.725+0000 7f0361000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f035c1619d0 0x7f035c06d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:25.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.725+0000 7f0361000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f035c15df30 con 0x7f035c157750 2024-05-11T12:14:25.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.725+0000 7f0361000640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f035c161000 con 0x7f035c0b3ea0 2024-05-11T12:14:25.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.725+0000 7f0361000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f035c15e210 con 0x7f035c1619d0 2024-05-11T12:14:25.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.726+0000 7f0359400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f035c0b3ea0 0x7f035c161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:25.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.726+0000 7f0359400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f035c0b3ea0 0x7f035c161490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:51616/0 (socket says 172.21.15.167:51616) 2024-05-11T12:14:25.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.726+0000 7f0359400640 1 -- 172.21.15.167:0/2097335094 learned_addr learned my addr 172.21.15.167:0/2097335094 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:25.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.726+0000 7f0359400640 1 -- 172.21.15.167:0/2097335094 >> v2:172.21.15.204:3300/0 conn(0x7f035c1619d0 msgr2=0x7f035c06d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:25.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.726+0000 7f035a800640 1 --2- 172.21.15.167:0/2097335094 >> v2:172.21.15.204:3300/0 conn(0x7f035c1619d0 0x7f035c06d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:25.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.726+0000 7f0359400640 1 --2- 172.21.15.167:0/2097335094 >> v2:172.21.15.204:3300/0 conn(0x7f035c1619d0 0x7f035c06d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:25.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.726+0000 7f0359400640 1 -- 172.21.15.167:0/2097335094 >> v1:172.21.15.204:6789/0 conn(0x7f035c157750 legacy=0x7f035c15d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:25.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.726+0000 7f0359400640 1 -- 172.21.15.167:0/2097335094 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f035c15e4f0 con 0x7f035c0b3ea0 2024-05-11T12:14:25.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.726+0000 7f035a800640 1 --2- 172.21.15.167:0/2097335094 >> v2:172.21.15.204:3300/0 conn(0x7f035c1619d0 0x7f035c06d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:25.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.727+0000 7f0359400640 1 --2- 172.21.15.167:0/2097335094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f035c0b3ea0 0x7f035c161490 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f035c06f6c0 tx=0x7f034402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b27bad6ce181acc7 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:25.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.728+0000 7f0358a00640 1 -- 172.21.15.167:0/2097335094 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0344036070 con 0x7f035c0b3ea0 2024-05-11T12:14:25.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.728+0000 7f0358a00640 1 -- 172.21.15.167:0/2097335094 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0344031600 con 0x7f035c0b3ea0 2024-05-11T12:14:25.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.728+0000 7f0358a00640 1 -- 172.21.15.167:0/2097335094 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03440319d0 con 0x7f035c0b3ea0 2024-05-11T12:14:25.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.728+0000 7f0361000640 1 -- 172.21.15.167:0/2097335094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f035c0b3ea0 msgr2=0x7f035c161490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:25.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.728+0000 7f0361000640 1 --2- 172.21.15.167:0/2097335094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f035c0b3ea0 0x7f035c161490 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f035c06f6c0 tx=0x7f034402a6a0 comp rx=0 tx=0).stop 2024-05-11T12:14:25.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.728+0000 7f0361000640 1 -- 172.21.15.167:0/2097335094 shutdown_connections 2024-05-11T12:14:25.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.728+0000 7f0361000640 1 --2- 172.21.15.167:0/2097335094 >> v2:172.21.15.204:3300/0 conn(0x7f035c1619d0 0x7f035c06d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:25.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.728+0000 7f0361000640 1 --2- 172.21.15.167:0/2097335094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f035c0b3ea0 0x7f035c161490 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:25.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.728+0000 7f0361000640 1 -- 172.21.15.167:0/2097335094 >> 172.21.15.167:0/2097335094 conn(0x7f035c0cd8f0 msgr2=0x7f035c0cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:25.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.728+0000 7f0361000640 1 -- 172.21.15.167:0/2097335094 shutdown_connections 2024-05-11T12:14:25.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.728+0000 7f0361000640 1 -- 172.21.15.167:0/2097335094 wait complete. 2024-05-11T12:14:25.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.730+0000 7f0361000640 1 Processor -- start 2024-05-11T12:14:25.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.730+0000 7f0361000640 1 -- start start 2024-05-11T12:14:25.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.731+0000 7f0361000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f035c0b3ea0 0x7f035c0aa410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:25.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.731+0000 7f0361000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f035c157750 0x7f035c0aa950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:25.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.731+0000 7f0361000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f035c1619d0 0x7f035c134430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:25.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.731+0000 7f0361000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f035c06db20 con 0x7f035c0b3ea0 2024-05-11T12:14:25.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.731+0000 7f0361000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f035c06d9a0 con 0x7f035c157750 2024-05-11T12:14:25.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.731+0000 7f0361000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f035c06dca0 con 0x7f035c1619d0 2024-05-11T12:14:25.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.731+0000 7f0359e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f035c0b3ea0 0x7f035c0aa410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:25.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.732+0000 7f0359e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f035c0b3ea0 0x7f035c0aa410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:51626/0 (socket says 172.21.15.167:51626) 2024-05-11T12:14:25.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.732+0000 7f0359e00640 1 -- 172.21.15.167:0/2022091351 learned_addr learned my addr 172.21.15.167:0/2022091351 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:25.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.732+0000 7f0359400640 1 --2- 172.21.15.167:0/2022091351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f035c157750 0x7f035c0aa950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:25.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.732+0000 7f0359e00640 1 -- 172.21.15.167:0/2022091351 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f035c1619d0 msgr2=0x7f035c134430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:25.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.732+0000 7f0359e00640 1 --2- 172.21.15.167:0/2022091351 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f035c1619d0 0x7f035c134430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:25.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.732+0000 7f0359e00640 1 -- 172.21.15.167:0/2022091351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f035c157750 msgr2=0x7f035c0aa950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:25.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.732+0000 7f0359e00640 1 --2- 172.21.15.167:0/2022091351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f035c157750 0x7f035c0aa950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:25.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.732+0000 7f0359e00640 1 -- 172.21.15.167:0/2022091351 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f035c15cb40 con 0x7f035c0b3ea0 2024-05-11T12:14:25.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.732+0000 7f0359400640 1 --2- 172.21.15.167:0/2022091351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f035c157750 0x7f035c0aa950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:25.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.732+0000 7f0359e00640 1 --2- 172.21.15.167:0/2022091351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f035c0b3ea0 0x7f035c0aa410 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f034c00c460 tx=0x7f034c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:25.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.733+0000 7f0352000640 1 -- 172.21.15.167:0/2022091351 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034c016020 con 0x7f035c0b3ea0 2024-05-11T12:14:25.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.733+0000 7f0352000640 1 -- 172.21.15.167:0/2022091351 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f034c012870 con 0x7f035c0b3ea0 2024-05-11T12:14:25.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.733+0000 7f0352000640 1 -- 172.21.15.167:0/2022091351 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034c012c60 con 0x7f035c0b3ea0 2024-05-11T12:14:25.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.733+0000 7f0361000640 1 -- 172.21.15.167:0/2022091351 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f035c132510 con 0x7f035c0b3ea0 2024-05-11T12:14:25.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.733+0000 7f0361000640 1 -- 172.21.15.167:0/2022091351 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f035c132ad0 con 0x7f035c0b3ea0 2024-05-11T12:14:25.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.734+0000 7f0361000640 1 -- 172.21.15.167:0/2022091351 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f035c06f7f0 con 0x7f035c0b3ea0 2024-05-11T12:14:25.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.735+0000 7f0352000640 1 -- 172.21.15.167:0/2022091351 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f034c019020 con 0x7f035c0b3ea0 2024-05-11T12:14:25.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.736+0000 7f0352000640 1 --2- 172.21.15.167:0/2022091351 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f03380762d0 0x7f0338078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:25.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.736+0000 7f0352000640 1 -- 172.21.15.167:0/2022091351 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5335+0+0 (secure 0 0 0) 0x7f034c0970e0 con 0x7f035c0b3ea0 2024-05-11T12:14:25.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.736+0000 7f0359400640 1 --2- 172.21.15.167:0/2022091351 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f03380762d0 0x7f0338078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:25.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.737+0000 7f0359400640 1 --2- 172.21.15.167:0/2022091351 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f03380762d0 0x7f0338078790 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0344033000 tx=0x7f0344009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:25.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.740+0000 7f0352000640 1 -- 172.21.15.167:0/2022091351 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f035c06f7f0 con 0x7f035c0b3ea0 2024-05-11T12:14:25.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.950+0000 7f0361000640 1 -- 172.21.15.167:0/2022091351 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f035c15f120 con 0x7f035c0b3ea0 2024-05-11T12:14:25.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.952+0000 7f0352000640 1 -- 172.21.15.167:0/2022091351 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+13344 (secure 0 0 0) 0x7f035c15f120 con 0x7f035c0b3ea0 2024-05-11T12:14:25.951 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:25.952 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":89,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:14:24.487967+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.167:6836/3484070468":"2024-05-12T12:14:21.476934+0000","172.21.15.167:6835/3484070468":"2024-05-12T12:14:21.476934+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:14:25.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.955+0000 7f0361000640 1 -- 172.21.15.167:0/2022091351 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f03380762d0 msgr2=0x7f0338078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:25.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.955+0000 7f0361000640 1 --2- 172.21.15.167:0/2022091351 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f03380762d0 0x7f0338078790 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0344033000 tx=0x7f0344009000 comp rx=0 tx=0).stop 2024-05-11T12:14:25.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.955+0000 7f0361000640 1 -- 172.21.15.167:0/2022091351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f035c0b3ea0 msgr2=0x7f035c0aa410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:25.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.955+0000 7f0361000640 1 --2- 172.21.15.167:0/2022091351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f035c0b3ea0 0x7f035c0aa410 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f034c00c460 tx=0x7f034c00c920 comp rx=0 tx=0).stop 2024-05-11T12:14:25.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.955+0000 7f0361000640 1 -- 172.21.15.167:0/2022091351 shutdown_connections 2024-05-11T12:14:25.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.955+0000 7f0361000640 1 --2- 172.21.15.167:0/2022091351 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f035c1619d0 0x7f035c134430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:25.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.955+0000 7f0361000640 1 --2- 172.21.15.167:0/2022091351 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f03380762d0 0x7f0338078790 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:25.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.955+0000 7f0361000640 1 --2- 172.21.15.167:0/2022091351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f035c157750 0x7f035c0aa950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:25.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.955+0000 7f0361000640 1 --2- 172.21.15.167:0/2022091351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f035c0b3ea0 0x7f035c0aa410 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:25.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.955+0000 7f0361000640 1 -- 172.21.15.167:0/2022091351 >> 172.21.15.167:0/2022091351 conn(0x7f035c0cd8f0 msgr2=0x7f035c15f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:25.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.955+0000 7f0361000640 1 -- 172.21.15.167:0/2022091351 shutdown_connections 2024-05-11T12:14:25.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:25.955+0000 7f0361000640 1 -- 172.21.15.167:0/2022091351 wait complete. 2024-05-11T12:14:25.974 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-11T12:10:33.260270+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-11T12:14:25.975 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-11T12:14:26.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.169+0000 7ff03b400640 1 Processor -- start 2024-05-11T12:14:26.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.170+0000 7ff03b400640 1 -- start start 2024-05-11T12:14:26.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.171+0000 7ff03b400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff034159970 0x7ff0340d1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:26.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.171+0000 7ff03b400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff0340d2db0 0x7ff0340d3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:26.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.171+0000 7ff03b400640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff034159d50 con 0x7ff0340d17b0 2024-05-11T12:14:26.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.171+0000 7ff03b400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7ff0340d04b0 con 0x7ff0340d2db0 2024-05-11T12:14:26.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.171+0000 7ff03b400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff0340d0790 con 0x7ff034159970 2024-05-11T12:14:26.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.171+0000 7ff033e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7ff0340d17b0 0x7ff03406d200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:39332/0 (socket says 172.21.15.167:39332) 2024-05-11T12:14:26.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.172+0000 7ff033e00640 1 -- 172.21.15.167:0/3955549871 learned_addr learned my addr 172.21.15.167:0/3955549871 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:14:26.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.172+0000 7ff038a00640 1 --2- 172.21.15.167:0/3955549871 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff034159970 0x7ff0340d1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:26.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.172+0000 7ff039400640 1 --2- 172.21.15.167:0/3955549871 >> v2:172.21.15.204:3300/0 conn(0x7ff0340d2db0 0x7ff0340d3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:26.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.172+0000 7ff033400640 1 -- 172.21.15.167:0/3955549871 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3050550583 0 0) 0x7ff034159d50 con 0x7ff0340d17b0 2024-05-11T12:14:26.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.172+0000 7ff033400640 1 -- 172.21.15.167:0/3955549871 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff018003520 con 0x7ff0340d17b0 2024-05-11T12:14:26.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.172+0000 7ff033400640 1 -- 172.21.15.167:0/3955549871 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1823981267 0 0) 0x7ff018003520 con 0x7ff0340d17b0 2024-05-11T12:14:26.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.172+0000 7ff033400640 1 -- 172.21.15.167:0/3955549871 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff034159970 msgr2=0x7ff0340d1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:26.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.172+0000 7ff033400640 1 --2- 172.21.15.167:0/3955549871 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff034159970 0x7ff0340d1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:26.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.172+0000 7ff033400640 1 -- 172.21.15.167:0/3955549871 >> v2:172.21.15.204:3300/0 conn(0x7ff0340d2db0 msgr2=0x7ff0340d3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:26.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.172+0000 7ff033400640 1 --2- 172.21.15.167:0/3955549871 >> v2:172.21.15.204:3300/0 conn(0x7ff0340d2db0 0x7ff0340d3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:26.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.172+0000 7ff033400640 1 -- 172.21.15.167:0/3955549871 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0340d0a70 con 0x7ff0340d17b0 2024-05-11T12:14:26.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.172+0000 7ff033400640 1 -- 172.21.15.167:0/3955549871 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7ff01c0016f0 con 0x7ff0340d17b0 2024-05-11T12:14:26.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.173+0000 7ff033400640 1 -- 172.21.15.167:0/3955549871 >> v1:172.21.15.167:6789/0 conn(0x7ff0340d17b0 legacy=0x7ff03406d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:26.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.173+0000 7ff033400640 1 --2- 172.21.15.167:0/3955549871 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff034159970 0x7ff018003a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:26.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.173+0000 7ff033400640 1 --2- 172.21.15.167:0/3955549871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff018004020 0x7ff018004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:26.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.173+0000 7ff033400640 1 --2- 172.21.15.167:0/3955549871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff018004d40 0x7ff0180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:26.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.173+0000 7ff033400640 1 -- 172.21.15.167:0/3955549871 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff0340d04b0 con 0x7ff018004020 2024-05-11T12:14:26.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.173+0000 7ff033400640 1 -- 172.21.15.167:0/3955549871 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff0340d0790 con 0x7ff018004d40 2024-05-11T12:14:26.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.173+0000 7ff033400640 1 -- 172.21.15.167:0/3955549871 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff01800ba70 con 0x7ff034159970 2024-05-11T12:14:26.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.173+0000 7ff033e00640 1 --2- 172.21.15.167:0/3955549871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff018004020 0x7ff018004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:26.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.173+0000 7ff038a00640 1 --2- 172.21.15.167:0/3955549871 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff034159970 0x7ff018003a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:26.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.173+0000 7ff039400640 1 --2- 172.21.15.167:0/3955549871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff018004d40 0x7ff0180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:26.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.173+0000 7ff033e00640 1 -- 172.21.15.167:0/3955549871 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff034159970 msgr2=0x7ff018003a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:26.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.173+0000 7ff033e00640 1 --2- 172.21.15.167:0/3955549871 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff034159970 0x7ff018003a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:26.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.173+0000 7ff033e00640 1 -- 172.21.15.167:0/3955549871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff018004d40 msgr2=0x7ff0180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:26.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.174+0000 7ff033e00640 1 --2- 172.21.15.167:0/3955549871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff018004d40 0x7ff0180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:26.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.174+0000 7ff033e00640 1 -- 172.21.15.167:0/3955549871 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff018003520 con 0x7ff018004020 2024-05-11T12:14:26.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.174+0000 7ff039400640 1 --2- 172.21.15.167:0/3955549871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff018004d40 0x7ff0180051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:26.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.175+0000 7ff033e00640 1 --2- 172.21.15.167:0/3955549871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff018004020 0x7ff018004800 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7ff01c00c190 tx=0x7ff018005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d758a5f39c8f0044 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:26.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.175+0000 7ff033400640 1 -- 172.21.15.167:0/3955549871 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01c007cc0 con 0x7ff018004020 2024-05-11T12:14:26.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.175+0000 7ff033400640 1 -- 172.21.15.167:0/3955549871 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff01c005ad0 con 0x7ff018004020 2024-05-11T12:14:26.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.175+0000 7ff033400640 1 -- 172.21.15.167:0/3955549871 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01c007070 con 0x7ff018004020 2024-05-11T12:14:26.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.175+0000 7ff03b400640 1 -- 172.21.15.167:0/3955549871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff018004020 msgr2=0x7ff018004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:26.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.175+0000 7ff03b400640 1 --2- 172.21.15.167:0/3955549871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff018004020 0x7ff018004800 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7ff01c00c190 tx=0x7ff018005830 comp rx=0 tx=0).stop 2024-05-11T12:14:26.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.175+0000 7ff039400640 1 -- 172.21.15.167:0/3955549871 reap_dead start 2024-05-11T12:14:26.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.175+0000 7ff03b400640 1 -- 172.21.15.167:0/3955549871 shutdown_connections 2024-05-11T12:14:26.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.175+0000 7ff03b400640 1 -- 172.21.15.167:0/3955549871 >> 172.21.15.167:0/3955549871 conn(0x7ff0340d51e0 msgr2=0x7ff0340d55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:26.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.176+0000 7ff03b400640 1 -- 172.21.15.167:0/3955549871 shutdown_connections 2024-05-11T12:14:26.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.176+0000 7ff03b400640 1 -- 172.21.15.167:0/3955549871 wait complete. 2024-05-11T12:14:26.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.177+0000 7ff03b400640 1 Processor -- start 2024-05-11T12:14:26.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.177+0000 7ff03b400640 1 -- start start 2024-05-11T12:14:26.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.178+0000 7ff03b400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff018004020 0x7ff03415ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:26.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.178+0000 7ff03b400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0340b67e0 0x7ff0341f8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:26.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.178+0000 7ff03b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0341f87d0 0x7ff0340b9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:26.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.178+0000 7ff03b400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff03406d910 con 0x7ff0341f87d0 2024-05-11T12:14:26.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.178+0000 7ff03b400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff034163d00 con 0x7ff0340b67e0 2024-05-11T12:14:26.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.178+0000 7ff03b400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff03406da90 con 0x7ff018004020 2024-05-11T12:14:26.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.179+0000 7ff039400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0341f87d0 0x7ff0340b9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:26.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.179+0000 7ff039400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0341f87d0 0x7ff0340b9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:51642/0 (socket says 172.21.15.167:51642) 2024-05-11T12:14:26.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.179+0000 7ff039400640 1 -- 172.21.15.167:0/2629092239 learned_addr learned my addr 172.21.15.167:0/2629092239 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:26.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.179+0000 7ff033e00640 1 --2- 172.21.15.167:0/2629092239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0340b67e0 0x7ff0341f8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:26.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.179+0000 7ff038a00640 1 --2- 172.21.15.167:0/2629092239 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff018004020 0x7ff03415ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:26.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.179+0000 7ff039400640 1 -- 172.21.15.167:0/2629092239 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff018004020 msgr2=0x7ff03415ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:26.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.179+0000 7ff039400640 1 --2- 172.21.15.167:0/2629092239 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff018004020 0x7ff03415ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:26.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.179+0000 7ff039400640 1 -- 172.21.15.167:0/2629092239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0340b67e0 msgr2=0x7ff0341f8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:26.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.179+0000 7ff039400640 1 --2- 172.21.15.167:0/2629092239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0340b67e0 0x7ff0341f8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:26.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.179+0000 7ff039400640 1 -- 172.21.15.167:0/2629092239 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff03415afa0 con 0x7ff0341f87d0 2024-05-11T12:14:26.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.179+0000 7ff038a00640 1 --2- 172.21.15.167:0/2629092239 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff018004020 0x7ff03415ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:26.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.180+0000 7ff039400640 1 --2- 172.21.15.167:0/2629092239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0341f87d0 0x7ff0340b9a60 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7ff0340d4b20 tx=0x7ff02c00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:26.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.180+0000 7ff030c00640 1 -- 172.21.15.167:0/2629092239 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02c01c070 con 0x7ff0341f87d0 2024-05-11T12:14:26.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.180+0000 7ff030c00640 1 -- 172.21.15.167:0/2629092239 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff02c017400 con 0x7ff0341f87d0 2024-05-11T12:14:26.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.180+0000 7ff030c00640 1 -- 172.21.15.167:0/2629092239 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02c0177d0 con 0x7ff0341f87d0 2024-05-11T12:14:26.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.180+0000 7ff03b400640 1 -- 172.21.15.167:0/2629092239 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0341f8d50 con 0x7ff0341f87d0 2024-05-11T12:14:26.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.180+0000 7ff03b400640 1 -- 172.21.15.167:0/2629092239 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff03415b240 con 0x7ff0341f87d0 2024-05-11T12:14:26.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.181+0000 7ff03b400640 1 -- 172.21.15.167:0/2629092239 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff034069520 con 0x7ff0341f87d0 2024-05-11T12:14:26.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.182+0000 7ff030c00640 1 -- 172.21.15.167:0/2629092239 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff02c005d10 con 0x7ff0341f87d0 2024-05-11T12:14:26.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.183+0000 7ff030c00640 1 --2- 172.21.15.167:0/2629092239 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff010076300 0x7ff0100787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:26.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.183+0000 7ff030c00640 1 -- 172.21.15.167:0/2629092239 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5335+0+0 (secure 0 0 0) 0x7ff02c097660 con 0x7ff0341f87d0 2024-05-11T12:14:26.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.184+0000 7ff038a00640 1 --2- 172.21.15.167:0/2629092239 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff010076300 0x7ff0100787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:26.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.184+0000 7ff038a00640 1 --2- 172.21.15.167:0/2629092239 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff010076300 0x7ff0100787c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff024009910 tx=0x7ff024008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:26.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.188+0000 7ff030c00640 1 -- 172.21.15.167:0/2629092239 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff02c012070 con 0x7ff0341f87d0 2024-05-11T12:14:26.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.393+0000 7ff03b400640 1 -- 172.21.15.167:0/2629092239 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff034161f30 con 0x7ff0341f87d0 2024-05-11T12:14:26.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.393+0000 7ff030c00640 1 -- 172.21.15.167:0/2629092239 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7ff02c061090 con 0x7ff0341f87d0 2024-05-11T12:14:26.393 INFO:teuthology.orchestra.run.smithi167.stdout:pg_num: 1 2024-05-11T12:14:26.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.396+0000 7ff03b400640 1 -- 172.21.15.167:0/2629092239 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff010076300 msgr2=0x7ff0100787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:26.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.396+0000 7ff03b400640 1 --2- 172.21.15.167:0/2629092239 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff010076300 0x7ff0100787c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff024009910 tx=0x7ff024008040 comp rx=0 tx=0).stop 2024-05-11T12:14:26.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.396+0000 7ff03b400640 1 -- 172.21.15.167:0/2629092239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0341f87d0 msgr2=0x7ff0340b9a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:26.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.396+0000 7ff03b400640 1 --2- 172.21.15.167:0/2629092239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0341f87d0 0x7ff0340b9a60 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7ff0340d4b20 tx=0x7ff02c00f4a0 comp rx=0 tx=0).stop 2024-05-11T12:14:26.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.397+0000 7ff03b400640 1 -- 172.21.15.167:0/2629092239 shutdown_connections 2024-05-11T12:14:26.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.397+0000 7ff03b400640 1 --2- 172.21.15.167:0/2629092239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0341f87d0 0x7ff0340b9a60 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:26.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.397+0000 7ff03b400640 1 --2- 172.21.15.167:0/2629092239 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff010076300 0x7ff0100787c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:26.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.397+0000 7ff03b400640 1 --2- 172.21.15.167:0/2629092239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0340b67e0 0x7ff0341f8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:26.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.397+0000 7ff03b400640 1 --2- 172.21.15.167:0/2629092239 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff018004020 0x7ff03415ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:26.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.397+0000 7ff03b400640 1 -- 172.21.15.167:0/2629092239 >> 172.21.15.167:0/2629092239 conn(0x7ff0340d51e0 msgr2=0x7ff0340b9070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:26.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.397+0000 7ff03b400640 1 -- 172.21.15.167:0/2629092239 shutdown_connections 2024-05-11T12:14:26.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.397+0000 7ff03b400640 1 -- 172.21.15.167:0/2629092239 wait complete. 2024-05-11T12:14:26.414 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_version_splitting' 2024-05-11T12:14:26.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.602+0000 7f7972600640 1 Processor -- start 2024-05-11T12:14:26.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.602+0000 7f7972600640 1 -- start start 2024-05-11T12:14:26.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.604+0000 7f7972600640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f796c13eeb0 0x7f796c06f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:26.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.604+0000 7f7972600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f796c0b50a0 0x7f796c073cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:26.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.604+0000 7f7972600640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f796c15fb90 con 0x7f796c13f290 2024-05-11T12:14:26.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.604+0000 7f7972600640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f796c15d5b0 con 0x7f796c13eeb0 2024-05-11T12:14:26.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.604+0000 7f7972600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f796c15fe70 con 0x7f796c0b50a0 2024-05-11T12:14:26.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.604+0000 7f796aa00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f796c13f290 0x7f796c0735c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:39334/0 (socket says 172.21.15.167:39334) 2024-05-11T12:14:26.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.604+0000 7f796aa00640 1 -- 172.21.15.167:0/3114664944 learned_addr learned my addr 172.21.15.167:0/3114664944 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:14:26.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.604+0000 7f796b400640 1 --2- 172.21.15.167:0/3114664944 >> v2:172.21.15.167:3300/0 conn(0x7f796c13eeb0 0x7f796c06f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:26.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.605+0000 7f796be00640 1 --2- 172.21.15.167:0/3114664944 >> v2:172.21.15.204:3300/0 conn(0x7f796c0b50a0 0x7f796c073cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:26.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.605+0000 7f796a000640 1 -- 172.21.15.167:0/3114664944 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3348670189 0 0) 0x7f796c15fb90 con 0x7f796c13f290 2024-05-11T12:14:26.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.605+0000 7f796b400640 1 -- 172.21.15.167:0/3114664944 >> v2:172.21.15.204:3300/0 conn(0x7f796c0b50a0 msgr2=0x7f796c073cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:26.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.605+0000 7f796b400640 1 --2- 172.21.15.167:0/3114664944 >> v2:172.21.15.204:3300/0 conn(0x7f796c0b50a0 0x7f796c073cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:26.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.605+0000 7f796b400640 1 -- 172.21.15.167:0/3114664944 >> v1:172.21.15.167:6789/0 conn(0x7f796c13f290 legacy=0x7f796c0735c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:26.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.605+0000 7f796b400640 1 -- 172.21.15.167:0/3114664944 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f796c160150 con 0x7f796c13eeb0 2024-05-11T12:14:26.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.606+0000 7f796b400640 1 --2- 172.21.15.167:0/3114664944 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f796c13eeb0 0x7f796c06f480 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f796c15faf0 tx=0x7f795c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89b4119bf31243e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:26.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.606+0000 7f796a000640 1 -- 172.21.15.167:0/3114664944 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795c037070 con 0x7f796c13eeb0 2024-05-11T12:14:26.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.606+0000 7f796a000640 1 -- 172.21.15.167:0/3114664944 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f795c032440 con 0x7f796c13eeb0 2024-05-11T12:14:26.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.606+0000 7f796a000640 1 -- 172.21.15.167:0/3114664944 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795c02ae60 con 0x7f796c13eeb0 2024-05-11T12:14:26.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.607+0000 7f7972600640 1 -- 172.21.15.167:0/3114664944 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f796c13eeb0 msgr2=0x7f796c06f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:26.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.607+0000 7f7972600640 1 --2- 172.21.15.167:0/3114664944 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f796c13eeb0 0x7f796c06f480 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f796c15faf0 tx=0x7f795c02a110 comp rx=0 tx=0).stop 2024-05-11T12:14:26.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.607+0000 7f7972600640 1 -- 172.21.15.167:0/3114664944 shutdown_connections 2024-05-11T12:14:26.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.607+0000 7f7972600640 1 --2- 172.21.15.167:0/3114664944 >> v2:172.21.15.204:3300/0 conn(0x7f796c0b50a0 0x7f796c073cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:26.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.607+0000 7f7972600640 1 --2- 172.21.15.167:0/3114664944 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f796c13eeb0 0x7f796c06f480 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:26.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.607+0000 7f7972600640 1 -- 172.21.15.167:0/3114664944 >> 172.21.15.167:0/3114664944 conn(0x7f796c0cd8f0 msgr2=0x7f796c0cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:26.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.607+0000 7f7972600640 1 -- 172.21.15.167:0/3114664944 shutdown_connections 2024-05-11T12:14:26.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.607+0000 7f7972600640 1 -- 172.21.15.167:0/3114664944 wait complete. 2024-05-11T12:14:26.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.609+0000 7f7972600640 1 Processor -- start 2024-05-11T12:14:26.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.609+0000 7f7972600640 1 -- start start 2024-05-11T12:14:26.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.610+0000 7f7972600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f796c0b50a0 0x7f796c14c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:26.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.610+0000 7f7972600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f796c13eeb0 0x7f796c14f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:26.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.610+0000 7f7972600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f796c13f290 0x7f796c13a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:26.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.610+0000 7f7972600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f796c06f9c0 con 0x7f796c13f290 2024-05-11T12:14:26.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.610+0000 7f7972600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f796c160710 con 0x7f796c0b50a0 2024-05-11T12:14:26.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.610+0000 7f7972600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f796c06fb40 con 0x7f796c13eeb0 2024-05-11T12:14:26.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.610+0000 7f796be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f796c13f290 0x7f796c13a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:26.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.610+0000 7f796be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f796c13f290 0x7f796c13a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:51666/0 (socket says 172.21.15.167:51666) 2024-05-11T12:14:26.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.610+0000 7f796be00640 1 -- 172.21.15.167:0/968536539 learned_addr learned my addr 172.21.15.167:0/968536539 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:26.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.610+0000 7f796b400640 1 --2- 172.21.15.167:0/968536539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f796c0b50a0 0x7f796c14c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:26.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.610+0000 7f796aa00640 1 --2- 172.21.15.167:0/968536539 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f796c13eeb0 0x7f796c14f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:26.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.610+0000 7f796be00640 1 -- 172.21.15.167:0/968536539 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f796c13eeb0 msgr2=0x7f796c14f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:26.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.610+0000 7f796be00640 1 --2- 172.21.15.167:0/968536539 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f796c13eeb0 0x7f796c14f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:26.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.610+0000 7f796be00640 1 -- 172.21.15.167:0/968536539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f796c0b50a0 msgr2=0x7f796c14c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:26.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.610+0000 7f796be00640 1 --2- 172.21.15.167:0/968536539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f796c0b50a0 0x7f796c14c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:26.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.610+0000 7f796be00640 1 -- 172.21.15.167:0/968536539 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f796c14fbf0 con 0x7f796c13f290 2024-05-11T12:14:26.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.611+0000 7f796be00640 1 --2- 172.21.15.167:0/968536539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f796c13f290 0x7f796c13a5e0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f796400d800 tx=0x7f796400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:26.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.611+0000 7f7963400640 1 -- 172.21.15.167:0/968536539 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796400b6c0 con 0x7f796c13f290 2024-05-11T12:14:26.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.611+0000 7f7963400640 1 -- 172.21.15.167:0/968536539 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7964019070 con 0x7f796c13f290 2024-05-11T12:14:26.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.611+0000 7f7963400640 1 -- 172.21.15.167:0/968536539 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796400bac0 con 0x7f796c13f290 2024-05-11T12:14:26.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.611+0000 7f7972600640 1 -- 172.21.15.167:0/968536539 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f796c0cf4a0 con 0x7f796c13f290 2024-05-11T12:14:26.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.611+0000 7f7972600640 1 -- 172.21.15.167:0/968536539 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f796c0cfa60 con 0x7f796c13f290 2024-05-11T12:14:26.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.612+0000 7f7972600640 1 -- 172.21.15.167:0/968536539 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f796c069590 con 0x7f796c13f290 2024-05-11T12:14:26.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.614+0000 7f7963400640 1 -- 172.21.15.167:0/968536539 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f796400bc60 con 0x7f796c13f290 2024-05-11T12:14:26.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.614+0000 7f7963400640 1 --2- 172.21.15.167:0/968536539 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f7948076320 0x7f79480787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:26.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.615+0000 7f7963400640 1 -- 172.21.15.167:0/968536539 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5335+0+0 (secure 0 0 0) 0x7f7964097810 con 0x7f796c13f290 2024-05-11T12:14:26.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.615+0000 7f796b400640 1 --2- 172.21.15.167:0/968536539 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f7948076320 0x7f79480787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:26.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.615+0000 7f796b400640 1 --2- 172.21.15.167:0/968536539 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f7948076320 0x7f79480787e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f795c034000 tx=0x7f795c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:26.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.619+0000 7f7963400640 1 -- 172.21.15.167:0/968536539 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7964060d70 con 0x7f796c13f290 2024-05-11T12:14:26.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:26.888+0000 7f7972600640 1 -- 172.21.15.167:0/968536539 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_version_splitting"]} v 0) -- 0x7f796c0cfc30 con 0x7f796c13f290 2024-05-11T12:14:27.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.515+0000 7f7963400640 1 -- 172.21.15.167:0/968536539 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_version_splitting"]}]=0 v225) ==== 148+0+0 (secure 0 0 0) 0x7f7964065070 con 0x7f796c13f290 2024-05-11T12:14:27.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.519+0000 7f7972600640 1 -- 172.21.15.167:0/968536539 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f7948076320 msgr2=0x7f79480787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:27.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.519+0000 7f7972600640 1 --2- 172.21.15.167:0/968536539 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f7948076320 0x7f79480787e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f795c034000 tx=0x7f795c02e000 comp rx=0 tx=0).stop 2024-05-11T12:14:27.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.519+0000 7f7972600640 1 -- 172.21.15.167:0/968536539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f796c13f290 msgr2=0x7f796c13a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:27.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.519+0000 7f7972600640 1 --2- 172.21.15.167:0/968536539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f796c13f290 0x7f796c13a5e0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f796400d800 tx=0x7f796400dcc0 comp rx=0 tx=0).stop 2024-05-11T12:14:27.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.519+0000 7f7972600640 1 -- 172.21.15.167:0/968536539 shutdown_connections 2024-05-11T12:14:27.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.519+0000 7f7972600640 1 --2- 172.21.15.167:0/968536539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f796c13f290 0x7f796c13a5e0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:27.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.519+0000 7f7972600640 1 --2- 172.21.15.167:0/968536539 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f796c13eeb0 0x7f796c14f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:27.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.519+0000 7f7972600640 1 --2- 172.21.15.167:0/968536539 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f7948076320 0x7f79480787e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:27.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.519+0000 7f7972600640 1 --2- 172.21.15.167:0/968536539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f796c0b50a0 0x7f796c14c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:27.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.519+0000 7f7972600640 1 -- 172.21.15.167:0/968536539 >> 172.21.15.167:0/968536539 conn(0x7f796c0cd8f0 msgr2=0x7f796c0b5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:27.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.519+0000 7f7972600640 1 -- 172.21.15.167:0/968536539 shutdown_connections 2024-05-11T12:14:27.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.519+0000 7f7972600640 1 -- 172.21.15.167:0/968536539 wait complete. 2024-05-11T12:14:27.542 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-11T12:14:27.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.745+0000 7f3797200640 1 Processor -- start 2024-05-11T12:14:27.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.745+0000 7f3797200640 1 -- start start 2024-05-11T12:14:27.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.747+0000 7f3797200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3798074370 0x7f37980b0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:27.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.747+0000 7f3797200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37980b0dd0 0x7f37980aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:27.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.747+0000 7f3797200640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37980b1550 con 0x7f37981596e0 2024-05-11T12:14:27.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.747+0000 7f3797200640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f37980b6dd0 con 0x7f3798074370 2024-05-11T12:14:27.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.747+0000 7f3797200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f37980b6010 con 0x7f37980b0dd0 2024-05-11T12:14:27.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.747+0000 7f378d400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3798074370 0x7f37980b0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:27.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.747+0000 7f3795e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f37981596e0 0x7f3798073c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:39344/0 (socket says 172.21.15.167:39344) 2024-05-11T12:14:27.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.747+0000 7f3795e00640 1 -- 172.21.15.167:0/977119050 learned_addr learned my addr 172.21.15.167:0/977119050 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:14:27.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.748+0000 7f3796800640 1 --2- 172.21.15.167:0/977119050 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37980b0dd0 0x7f37980aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:27.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.748+0000 7f378d400640 1 -- 172.21.15.167:0/977119050 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37980b0dd0 msgr2=0x7f37980aecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:27.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.748+0000 7f378d400640 1 --2- 172.21.15.167:0/977119050 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37980b0dd0 0x7f37980aecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:27.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.748+0000 7f378d400640 1 -- 172.21.15.167:0/977119050 >> v1:172.21.15.167:6789/0 conn(0x7f37981596e0 legacy=0x7f3798073c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:27.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.748+0000 7f378d400640 1 -- 172.21.15.167:0/977119050 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37980b62f0 con 0x7f3798074370 2024-05-11T12:14:27.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.748+0000 7f3795400640 1 -- 172.21.15.167:0/977119050 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1063810458 0 0) 0x7f37980b1550 con 0x7f37981596e0 2024-05-11T12:14:27.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.749+0000 7f378d400640 1 --2- 172.21.15.167:0/977119050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3798074370 0x7f37980b0890 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f37800097a0 tx=0x7f378002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90e0237b92f4d459 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:27.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.749+0000 7f3795400640 1 -- 172.21.15.167:0/977119050 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3780037070 con 0x7f3798074370 2024-05-11T12:14:27.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.749+0000 7f3795400640 1 -- 172.21.15.167:0/977119050 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3780032440 con 0x7f3798074370 2024-05-11T12:14:27.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.749+0000 7f3795400640 1 -- 172.21.15.167:0/977119050 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3780032810 con 0x7f3798074370 2024-05-11T12:14:27.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.750+0000 7f3797200640 1 -- 172.21.15.167:0/977119050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3798074370 msgr2=0x7f37980b0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:27.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.750+0000 7f3797200640 1 --2- 172.21.15.167:0/977119050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3798074370 0x7f37980b0890 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f37800097a0 tx=0x7f378002a180 comp rx=0 tx=0).stop 2024-05-11T12:14:27.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.750+0000 7f3797200640 1 -- 172.21.15.167:0/977119050 shutdown_connections 2024-05-11T12:14:27.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.750+0000 7f3797200640 1 --2- 172.21.15.167:0/977119050 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37980b0dd0 0x7f37980aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:27.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.750+0000 7f3797200640 1 --2- 172.21.15.167:0/977119050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3798074370 0x7f37980b0890 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:27.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.750+0000 7f3797200640 1 -- 172.21.15.167:0/977119050 >> 172.21.15.167:0/977119050 conn(0x7f3798073140 msgr2=0x7f3798073550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:27.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.750+0000 7f3797200640 1 -- 172.21.15.167:0/977119050 shutdown_connections 2024-05-11T12:14:27.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.750+0000 7f3797200640 1 -- 172.21.15.167:0/977119050 wait complete. 2024-05-11T12:14:27.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.751+0000 7f3797200640 1 Processor -- start 2024-05-11T12:14:27.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.752+0000 7f3797200640 1 -- start start 2024-05-11T12:14:27.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.753+0000 7f3797200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3798074370 0x7f379814b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:27.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.753+0000 7f3797200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37980b0dd0 0x7f3798158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:27.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.753+0000 7f3797200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37981596e0 0x7f37980af240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:27.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.753+0000 7f3797200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f37980b1230 con 0x7f37981596e0 2024-05-11T12:14:27.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.753+0000 7f3797200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f37980b68b0 con 0x7f37980b0dd0 2024-05-11T12:14:27.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.753+0000 7f3797200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3798159ac0 con 0x7f3798074370 2024-05-11T12:14:27.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.753+0000 7f3796800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37981596e0 0x7f37980af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:27.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.753+0000 7f3796800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37981596e0 0x7f37980af240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:51688/0 (socket says 172.21.15.167:51688) 2024-05-11T12:14:27.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.753+0000 7f3796800640 1 -- 172.21.15.167:0/2776699658 learned_addr learned my addr 172.21.15.167:0/2776699658 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:27.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.753+0000 7f3795e00640 1 --2- 172.21.15.167:0/2776699658 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3798074370 0x7f379814b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:27.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.753+0000 7f3796800640 1 -- 172.21.15.167:0/2776699658 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3798074370 msgr2=0x7f379814b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:27.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.753+0000 7f3796800640 1 --2- 172.21.15.167:0/2776699658 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3798074370 0x7f379814b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:27.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.753+0000 7f378d400640 1 --2- 172.21.15.167:0/2776699658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37980b0dd0 0x7f3798158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:27.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.753+0000 7f3796800640 1 -- 172.21.15.167:0/2776699658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37980b0dd0 msgr2=0x7f3798158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:27.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.753+0000 7f3796800640 1 --2- 172.21.15.167:0/2776699658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37980b0dd0 0x7f3798158870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:27.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.753+0000 7f3796800640 1 -- 172.21.15.167:0/2776699658 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3798154b60 con 0x7f37981596e0 2024-05-11T12:14:27.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.754+0000 7f3796800640 1 --2- 172.21.15.167:0/2776699658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37981596e0 0x7f37980af240 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f379000cd70 tx=0x7f379000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:27.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.754+0000 7f378ea00640 1 -- 172.21.15.167:0/2776699658 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3790003b90 con 0x7f37981596e0 2024-05-11T12:14:27.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.754+0000 7f378ea00640 1 -- 172.21.15.167:0/2776699658 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3790003d30 con 0x7f37981596e0 2024-05-11T12:14:27.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.754+0000 7f378ea00640 1 -- 172.21.15.167:0/2776699658 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3790004240 con 0x7f37981596e0 2024-05-11T12:14:27.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.754+0000 7f3797200640 1 -- 172.21.15.167:0/2776699658 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3798154e40 con 0x7f37981596e0 2024-05-11T12:14:27.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.754+0000 7f3797200640 1 -- 172.21.15.167:0/2776699658 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3798155400 con 0x7f37981596e0 2024-05-11T12:14:27.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.755+0000 7f3797200640 1 -- 172.21.15.167:0/2776699658 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37980694b0 con 0x7f37981596e0 2024-05-11T12:14:27.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.756+0000 7f378ea00640 1 -- 172.21.15.167:0/2776699658 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f37900043e0 con 0x7f37981596e0 2024-05-11T12:14:27.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.757+0000 7f378ea00640 1 --2- 172.21.15.167:0/2776699658 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f37740762d0 0x7f3774078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:27.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.758+0000 7f378ea00640 1 -- 172.21.15.167:0/2776699658 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5335+0+0 (secure 0 0 0) 0x7f3790097fe0 con 0x7f37981596e0 2024-05-11T12:14:27.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.758+0000 7f3795e00640 1 --2- 172.21.15.167:0/2776699658 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f37740762d0 0x7f3774078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:27.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.759+0000 7f3795e00640 1 --2- 172.21.15.167:0/2776699658 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f37740762d0 0x7f3774078790 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3788000f60 tx=0x7f3788009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:27.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.762+0000 7f378ea00640 1 -- 172.21.15.167:0/2776699658 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3790061400 con 0x7f37981596e0 2024-05-11T12:14:27.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.954+0000 7f3797200640 1 -- 172.21.15.167:0/2776699658 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f37980d1820 con 0x7f37981596e0 2024-05-11T12:14:27.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.959+0000 7f378ea00640 1 -- 172.21.15.167:0/2776699658 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) ==== 122+0+0 (secure 0 0 0) 0x7f3790065700 con 0x7f37981596e0 2024-05-11T12:14:27.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.961+0000 7f3797200640 1 -- 172.21.15.167:0/2776699658 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f37740762d0 msgr2=0x7f3774078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:27.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.961+0000 7f3797200640 1 --2- 172.21.15.167:0/2776699658 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f37740762d0 0x7f3774078790 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3788000f60 tx=0x7f3788009040 comp rx=0 tx=0).stop 2024-05-11T12:14:27.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.962+0000 7f3797200640 1 -- 172.21.15.167:0/2776699658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37981596e0 msgr2=0x7f37980af240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:27.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.962+0000 7f3797200640 1 --2- 172.21.15.167:0/2776699658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37981596e0 0x7f37980af240 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f379000cd70 tx=0x7f379000a660 comp rx=0 tx=0).stop 2024-05-11T12:14:27.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.962+0000 7f3797200640 1 -- 172.21.15.167:0/2776699658 shutdown_connections 2024-05-11T12:14:27.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.962+0000 7f3797200640 1 --2- 172.21.15.167:0/2776699658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37981596e0 0x7f37980af240 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:27.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.962+0000 7f3797200640 1 --2- 172.21.15.167:0/2776699658 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f37740762d0 0x7f3774078790 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:27.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.962+0000 7f3797200640 1 --2- 172.21.15.167:0/2776699658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37980b0dd0 0x7f3798158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:27.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.962+0000 7f3797200640 1 --2- 172.21.15.167:0/2776699658 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3798074370 0x7f379814b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:27.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.962+0000 7f3797200640 1 -- 172.21.15.167:0/2776699658 >> 172.21.15.167:0/2776699658 conn(0x7f3798073140 msgr2=0x7f37980b4b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:27.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.962+0000 7f3797200640 1 -- 172.21.15.167:0/2776699658 shutdown_connections 2024-05-11T12:14:27.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:27.962+0000 7f3797200640 1 -- 172.21.15.167:0/2776699658 wait complete. 2024-05-11T12:14:27.979 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-11T12:14:27.979 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-11T12:14:28.076 DEBUG:teuthology.orchestra.run.smithi204:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-11T12:14:28.181 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-11T12:14:28.182 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:14:28.182 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:14:28.240 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-11T12:14:28.240 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:14:28.240 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:14:28.296 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052713,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-11T12:14:28.296 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:14:28.296 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:14:28.352 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-11T12:14:28.352 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:14:28.352 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:14:28.408 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052713,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-11T12:14:28.409 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-11T12:14:28.409 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:14:28.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.615+0000 7f2976200640 1 Processor -- start 2024-05-11T12:14:28.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.615+0000 7f2976200640 1 -- start start 2024-05-11T12:14:28.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.616+0000 7f2976200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f29700cf970 0x7f297015bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:28.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.616+0000 7f2976200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f297015c280 0x7f29700b4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:28.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.616+0000 7f2976200640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f297015cab0 con 0x7f2970159350 2024-05-11T12:14:28.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.616+0000 7f2976200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f297015fff0 con 0x7f29700cf970 2024-05-11T12:14:28.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.616+0000 7f2976200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f297015f230 con 0x7f297015c280 2024-05-11T12:14:28.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.616+0000 7f2974e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f2970159350 0x7f29700cf260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:39350/0 (socket says 172.21.15.167:39350) 2024-05-11T12:14:28.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.616+0000 7f2974e00640 1 -- 172.21.15.167:0/971501905 learned_addr learned my addr 172.21.15.167:0/971501905 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:14:28.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.617+0000 7f296f400640 1 -- 172.21.15.167:0/971501905 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3278433427 0 0) 0x7f297015cab0 con 0x7f2970159350 2024-05-11T12:14:28.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.617+0000 7f296f400640 1 -- 172.21.15.167:0/971501905 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2958003520 con 0x7f2970159350 2024-05-11T12:14:28.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.617+0000 7f2975800640 1 --2- 172.21.15.167:0/971501905 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f297015c280 0x7f29700b4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:28.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.617+0000 7f296fe00640 1 --2- 172.21.15.167:0/971501905 >> v2:172.21.15.204:3300/0 conn(0x7f29700cf970 0x7f297015bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:28.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.617+0000 7f296f400640 1 -- 172.21.15.167:0/971501905 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2168651258 0 0) 0x7f2958003520 con 0x7f2970159350 2024-05-11T12:14:28.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.617+0000 7f296f400640 1 -- 172.21.15.167:0/971501905 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f297015c280 msgr2=0x7f29700b4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:28.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.617+0000 7f296f400640 1 --2- 172.21.15.167:0/971501905 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f297015c280 0x7f29700b4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:28.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.617+0000 7f296f400640 1 -- 172.21.15.167:0/971501905 >> v2:172.21.15.204:3300/0 conn(0x7f29700cf970 msgr2=0x7f297015bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:28.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.617+0000 7f296f400640 1 --2- 172.21.15.167:0/971501905 >> v2:172.21.15.204:3300/0 conn(0x7f29700cf970 0x7f297015bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:28.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.617+0000 7f296f400640 1 -- 172.21.15.167:0/971501905 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f297015f510 con 0x7f2970159350 2024-05-11T12:14:28.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.617+0000 7f296f400640 1 -- 172.21.15.167:0/971501905 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f29640016f0 con 0x7f2970159350 2024-05-11T12:14:28.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.617+0000 7f296f400640 1 -- 172.21.15.167:0/971501905 >> v1:172.21.15.167:6789/0 conn(0x7f2970159350 legacy=0x7f29700cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:28.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.617+0000 7f296f400640 1 --2- 172.21.15.167:0/971501905 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2958003a00 0x7f2958003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:28.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.617+0000 7f296f400640 1 --2- 172.21.15.167:0/971501905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2958004800 0x7f2958004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:28.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.617+0000 7f296f400640 1 --2- 172.21.15.167:0/971501905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29580051c0 0x7f295800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:28.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.617+0000 7f296f400640 1 -- 172.21.15.167:0/971501905 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f297015fff0 con 0x7f2958004800 2024-05-11T12:14:28.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.617+0000 7f296f400640 1 -- 172.21.15.167:0/971501905 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f297015f230 con 0x7f29580051c0 2024-05-11T12:14:28.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.617+0000 7f296f400640 1 -- 172.21.15.167:0/971501905 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f295800c140 con 0x7f2958003a00 2024-05-11T12:14:28.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.618+0000 7f2975800640 1 --2- 172.21.15.167:0/971501905 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f297015c280 0x7f29700b4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:14:28.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.618+0000 7f296fe00640 1 --2- 172.21.15.167:0/971501905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2958004800 0x7f2958004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:28.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.618+0000 7f296fe00640 1 -- 172.21.15.167:0/971501905 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2958003a00 msgr2=0x7f2958003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:28.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.618+0000 7f2974e00640 1 --2- 172.21.15.167:0/971501905 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2958003a00 0x7f2958003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:28.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.618+0000 7f296fe00640 1 --2- 172.21.15.167:0/971501905 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2958003a00 0x7f2958003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:28.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.618+0000 7f296fe00640 1 -- 172.21.15.167:0/971501905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29580051c0 msgr2=0x7f295800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:28.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.618+0000 7f296fe00640 1 --2- 172.21.15.167:0/971501905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29580051c0 0x7f295800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:28.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.618+0000 7f296fe00640 1 -- 172.21.15.167:0/971501905 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2958003520 con 0x7f2958004800 2024-05-11T12:14:28.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.619+0000 7f296fe00640 1 --2- 172.21.15.167:0/971501905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2958004800 0x7f2958004c80 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f296000ba30 tx=0x7f2958005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa57aabd39bdbd5 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:28.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.619+0000 7f296f400640 1 -- 172.21.15.167:0/971501905 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2960030070 con 0x7f2958004800 2024-05-11T12:14:28.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.619+0000 7f296f400640 1 -- 172.21.15.167:0/971501905 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f296002ae10 con 0x7f2958004800 2024-05-11T12:14:28.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.619+0000 7f296f400640 1 -- 172.21.15.167:0/971501905 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296000b600 con 0x7f2958004800 2024-05-11T12:14:28.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.620+0000 7f2976200640 1 -- 172.21.15.167:0/971501905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2958004800 msgr2=0x7f2958004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:28.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.620+0000 7f2976200640 1 --2- 172.21.15.167:0/971501905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2958004800 0x7f2958004c80 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f296000ba30 tx=0x7f2958005830 comp rx=0 tx=0).stop 2024-05-11T12:14:28.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.620+0000 7f2975800640 1 -- 172.21.15.167:0/971501905 reap_dead start 2024-05-11T12:14:28.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.620+0000 7f2976200640 1 -- 172.21.15.167:0/971501905 shutdown_connections 2024-05-11T12:14:28.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.620+0000 7f2976200640 1 -- 172.21.15.167:0/971501905 >> 172.21.15.167:0/971501905 conn(0x7f29700b3460 msgr2=0x7f29700d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:28.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.620+0000 7f2976200640 1 -- 172.21.15.167:0/971501905 shutdown_connections 2024-05-11T12:14:28.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.620+0000 7f2976200640 1 -- 172.21.15.167:0/971501905 wait complete. 2024-05-11T12:14:28.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.622+0000 7f2976200640 1 Processor -- start 2024-05-11T12:14:28.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.622+0000 7f2976200640 1 -- start start 2024-05-11T12:14:28.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.623+0000 7f2976200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2958004800 0x7f2970147d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:28.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.623+0000 7f2976200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29701696d0 0x7f297014cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:28.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.623+0000 7f2976200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f297014d0f0 0x7f29700b4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:28.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.623+0000 7f2976200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f297015c6f0 con 0x7f2958004800 2024-05-11T12:14:28.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.623+0000 7f2976200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f297015fad0 con 0x7f29701696d0 2024-05-11T12:14:28.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.623+0000 7f2976200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f297015c870 con 0x7f297014d0f0 2024-05-11T12:14:28.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.623+0000 7f2974e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2958004800 0x7f2970147d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:28.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.623+0000 7f296fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29701696d0 0x7f297014cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:28.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.623+0000 7f2974e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2958004800 0x7f2970147d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:51704/0 (socket says 172.21.15.167:51704) 2024-05-11T12:14:28.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.623+0000 7f2974e00640 1 -- 172.21.15.167:0/1446969327 learned_addr learned my addr 172.21.15.167:0/1446969327 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:28.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.624+0000 7f2975800640 1 --2- 172.21.15.167:0/1446969327 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f297014d0f0 0x7f29700b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:28.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.624+0000 7f2974e00640 1 -- 172.21.15.167:0/1446969327 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f297014d0f0 msgr2=0x7f29700b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:28.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.624+0000 7f2974e00640 1 --2- 172.21.15.167:0/1446969327 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f297014d0f0 0x7f29700b4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:28.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.624+0000 7f2974e00640 1 -- 172.21.15.167:0/1446969327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29701696d0 msgr2=0x7f297014cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:28.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.624+0000 7f2974e00640 1 --2- 172.21.15.167:0/1446969327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29701696d0 0x7f297014cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:28.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.624+0000 7f2974e00640 1 -- 172.21.15.167:0/1446969327 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f297014d5b0 con 0x7f2958004800 2024-05-11T12:14:28.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.624+0000 7f296fe00640 1 --2- 172.21.15.167:0/1446969327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29701696d0 0x7f297014cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:14:28.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.624+0000 7f2974e00640 1 --2- 172.21.15.167:0/1446969327 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2958004800 0x7f2970147d40 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f296400d7f0 tx=0x7f296400dcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:28.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.624+0000 7f296cc00640 1 -- 172.21.15.167:0/1446969327 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2964019070 con 0x7f2958004800 2024-05-11T12:14:28.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.624+0000 7f296cc00640 1 -- 172.21.15.167:0/1446969327 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29640074e0 con 0x7f2958004800 2024-05-11T12:14:28.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.624+0000 7f296cc00640 1 -- 172.21.15.167:0/1446969327 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2964004a00 con 0x7f2958004800 2024-05-11T12:14:28.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.624+0000 7f2976200640 1 -- 172.21.15.167:0/1446969327 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f297014d890 con 0x7f2958004800 2024-05-11T12:14:28.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.625+0000 7f2976200640 1 -- 172.21.15.167:0/1446969327 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f297014de50 con 0x7f2958004800 2024-05-11T12:14:28.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.626+0000 7f2976200640 1 -- 172.21.15.167:0/1446969327 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29700694b0 con 0x7f2958004800 2024-05-11T12:14:28.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.627+0000 7f296cc00640 1 -- 172.21.15.167:0/1446969327 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f296401c070 con 0x7f2958004800 2024-05-11T12:14:28.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.628+0000 7f296cc00640 1 --2- 172.21.15.167:0/1446969327 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f29540762d0 0x7f2954078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:28.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.628+0000 7f296cc00640 1 -- 172.21.15.167:0/1446969327 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5335+0+0 (secure 0 0 0) 0x7f2964096db0 con 0x7f2958004800 2024-05-11T12:14:28.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.628+0000 7f296fe00640 1 --2- 172.21.15.167:0/1446969327 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f29540762d0 0x7f2954078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:28.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.629+0000 7f296fe00640 1 --2- 172.21.15.167:0/1446969327 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f29540762d0 0x7f2954078790 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f296000ba00 tx=0x7f2960030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:28.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.632+0000 7f296cc00640 1 -- 172.21.15.167:0/1446969327 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2964060230 con 0x7f2958004800 2024-05-11T12:14:28.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.860+0000 7f2976200640 1 -- 172.21.15.167:0/1446969327 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f297014c1d0 con 0x7f2958004800 2024-05-11T12:14:28.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.860+0000 7f296cc00640 1 -- 172.21.15.167:0/1446969327 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+3881 (secure 0 0 0) 0x7f2964064530 con 0x7f2958004800 2024-05-11T12:14:28.860 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:28.860 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":55,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4922,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1154357925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1154357925},{"type":"v1","addr":"172.21.15.167:6837","nonce":1154357925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":5155,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1256314893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1256314893},{"type":"v1","addr":"172.21.15.204:6833","nonce":1256314893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":5158,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1578748665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1578748665},{"type":"v1","addr":"172.21.15.204:6835","nonce":1578748665}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6552,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/2461489513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2461489513},{"type":"v1","addr":"172.21.15.167:6836","nonce":2461489513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[]} 2024-05-11T12:14:28.860 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 55 2024-05-11T12:14:28.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.863+0000 7f2976200640 1 -- 172.21.15.167:0/1446969327 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f29540762d0 msgr2=0x7f2954078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:28.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.863+0000 7f2976200640 1 --2- 172.21.15.167:0/1446969327 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f29540762d0 0x7f2954078790 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f296000ba00 tx=0x7f2960030000 comp rx=0 tx=0).stop 2024-05-11T12:14:28.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.863+0000 7f2976200640 1 -- 172.21.15.167:0/1446969327 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2958004800 msgr2=0x7f2970147d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:28.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.863+0000 7f2976200640 1 --2- 172.21.15.167:0/1446969327 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2958004800 0x7f2970147d40 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f296400d7f0 tx=0x7f296400dcb0 comp rx=0 tx=0).stop 2024-05-11T12:14:28.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.863+0000 7f2976200640 1 -- 172.21.15.167:0/1446969327 shutdown_connections 2024-05-11T12:14:28.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.863+0000 7f2976200640 1 --2- 172.21.15.167:0/1446969327 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f297014d0f0 0x7f29700b4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:28.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.863+0000 7f2976200640 1 --2- 172.21.15.167:0/1446969327 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f29540762d0 0x7f2954078790 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:28.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.863+0000 7f2976200640 1 --2- 172.21.15.167:0/1446969327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29701696d0 0x7f297014cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:28.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.863+0000 7f2976200640 1 --2- 172.21.15.167:0/1446969327 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2958004800 0x7f2970147d40 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:28.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.863+0000 7f2976200640 1 -- 172.21.15.167:0/1446969327 >> 172.21.15.167:0/1446969327 conn(0x7f29700b3460 msgr2=0x7f297015de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:28.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.863+0000 7f2976200640 1 -- 172.21.15.167:0/1446969327 shutdown_connections 2024-05-11T12:14:28.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:28.864+0000 7f2976200640 1 -- 172.21.15.167:0/1446969327 wait complete. 2024-05-11T12:14:28.881 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-11T12:14:28.881 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-11T12:14:28.881 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-11T12:14:28.881 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T12:14:28.882 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-11T12:14:28.882 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-11T12:14:28.882 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-11T12:14:28.882 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T12:14:28.882 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-11T12:14:28.882 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-11T12:14:28.882 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-11T12:14:28.883 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T12:14:28.883 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-11T12:14:28.883 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-11T12:14:28.883 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-11T12:14:28.883 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T12:14:28.898 INFO:tasks.ceph.mds.a:Stopped 2024-05-11T12:14:28.898 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-11T12:14:28.902 INFO:tasks.ceph.mds.b:Stopped 2024-05-11T12:14:28.902 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-11T12:14:28.903 INFO:tasks.ceph.mds.c:Stopped 2024-05-11T12:14:28.904 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-11T12:14:28.912 INFO:tasks.ceph.mds.d:Stopped 2024-05-11T12:14:28.912 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-11T12:14:28.949 INFO:tasks.ceph.mds.a:Started 2024-05-11T12:14:28.949 DEBUG:teuthology.parallel:result is None 2024-05-11T12:14:28.955 INFO:tasks.ceph.mds.c:Started 2024-05-11T12:14:28.955 DEBUG:teuthology.parallel:result is None 2024-05-11T12:14:28.960 INFO:tasks.ceph.mds.b:Started 2024-05-11T12:14:28.960 DEBUG:teuthology.parallel:result is None 2024-05-11T12:14:28.966 INFO:tasks.ceph.mds.d:Started 2024-05-11T12:14:28.966 DEBUG:teuthology.parallel:result is None 2024-05-11T12:14:28.967 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-11T12:14:29.098 INFO:tasks.ceph.mds.a.smithi167.stdout:starting mds.a at 2024-05-11T12:14:29.101 INFO:tasks.ceph.mds.c.smithi167.stdout:starting mds.c at 2024-05-11T12:14:29.117 INFO:tasks.ceph.mds.b.smithi204.stdout:starting mds.b at 2024-05-11T12:14:29.117 INFO:tasks.ceph.mds.d.smithi204.stdout:starting mds.d at 2024-05-11T12:14:29.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.144+0000 7ff1c8000640 1 Processor -- start 2024-05-11T12:14:29.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.144+0000 7ff1c8000640 1 -- start start 2024-05-11T12:14:29.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.146+0000 7ff1c8000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1c00d2e80 0x7ff1c00d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:29.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.146+0000 7ff1c8000640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1c00d37c0 con 0x7ff1c0147ff0 2024-05-11T12:14:29.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.146+0000 7ff1c8000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1c00d1430 con 0x7ff1c0148cf0 2024-05-11T12:14:29.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.146+0000 7ff1c8000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff1c00b64f0 con 0x7ff1c00d2e80 2024-05-11T12:14:29.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.146+0000 7ff1c5600640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7ff1c0147ff0 0x7ff1c00d24d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:55570/0 (socket says 172.21.15.167:55570) 2024-05-11T12:14:29.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.146+0000 7ff1c5600640 1 -- 172.21.15.167:0/685548048 learned_addr learned my addr 172.21.15.167:0/685548048 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:14:29.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.146+0000 7ff1c6000640 1 --2- 172.21.15.167:0/685548048 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1c00d2e80 0x7ff1c00d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:29.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.146+0000 7ff1bfe00640 1 -- 172.21.15.167:0/685548048 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 12547889 0 0) 0x7ff1c00d37c0 con 0x7ff1c0147ff0 2024-05-11T12:14:29.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.146+0000 7ff1bfe00640 1 -- 172.21.15.167:0/685548048 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff1a8003520 con 0x7ff1c0147ff0 2024-05-11T12:14:29.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.146+0000 7ff1bfe00640 1 -- 172.21.15.167:0/685548048 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2508960501 0 0) 0x7ff1c00d1430 con 0x7ff1c0148cf0 2024-05-11T12:14:29.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.147+0000 7ff1bfe00640 1 -- 172.21.15.167:0/685548048 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff1c00d37c0 con 0x7ff1c0148cf0 2024-05-11T12:14:29.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.147+0000 7ff1c6000640 1 -- 172.21.15.167:0/685548048 >> v1:172.21.15.204:6789/0 conn(0x7ff1c0148cf0 legacy=0x7ff1c00b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:29.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.147+0000 7ff1c6000640 1 -- 172.21.15.167:0/685548048 >> v1:172.21.15.167:6789/0 conn(0x7ff1c0147ff0 legacy=0x7ff1c00d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:29.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.147+0000 7ff1c6000640 1 -- 172.21.15.167:0/685548048 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1c00d1710 con 0x7ff1c00d2e80 2024-05-11T12:14:29.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.148+0000 7ff1c6000640 1 --2- 172.21.15.167:0/685548048 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1c00d2e80 0x7ff1c00d3280 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8009dc0 tx=0x7ff1b802a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=885ab167d30195d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:29.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.148+0000 7ff1bfe00640 1 -- 172.21.15.167:0/685548048 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b802ae60 con 0x7ff1c00d2e80 2024-05-11T12:14:29.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.148+0000 7ff1bfe00640 1 -- 172.21.15.167:0/685548048 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1b8007740 con 0x7ff1c00d2e80 2024-05-11T12:14:29.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.148+0000 7ff1bfe00640 1 -- 172.21.15.167:0/685548048 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8007b10 con 0x7ff1c00d2e80 2024-05-11T12:14:29.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.148+0000 7ff1c8000640 1 -- 172.21.15.167:0/685548048 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1c00d2e80 msgr2=0x7ff1c00d3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:29.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.148+0000 7ff1c8000640 1 --2- 172.21.15.167:0/685548048 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1c00d2e80 0x7ff1c00d3280 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8009dc0 tx=0x7ff1b802a640 comp rx=0 tx=0).stop 2024-05-11T12:14:29.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.149+0000 7ff1c8000640 1 -- 172.21.15.167:0/685548048 shutdown_connections 2024-05-11T12:14:29.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.149+0000 7ff1c8000640 1 --2- 172.21.15.167:0/685548048 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1c00d2e80 0x7ff1c00d3280 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:29.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.149+0000 7ff1c8000640 1 -- 172.21.15.167:0/685548048 >> 172.21.15.167:0/685548048 conn(0x7ff1c00b8300 msgr2=0x7ff1c00b86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:29.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.149+0000 7ff1c8000640 1 -- 172.21.15.167:0/685548048 shutdown_connections 2024-05-11T12:14:29.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.149+0000 7ff1c8000640 1 -- 172.21.15.167:0/685548048 wait complete. 2024-05-11T12:14:29.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.150+0000 7ff1c8000640 1 Processor -- start 2024-05-11T12:14:29.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.150+0000 7ff1c8000640 1 -- start start 2024-05-11T12:14:29.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.151+0000 7ff1c8000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1c00d2e80 0x7ff1c0154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:29.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.151+0000 7ff1c8000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1c0147ff0 0x7ff1c01447b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:29.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.151+0000 7ff1c8000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1c0148cf0 0x7ff1c0161e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:29.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.151+0000 7ff1c8000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c00d1cd0 con 0x7ff1c0148cf0 2024-05-11T12:14:29.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.151+0000 7ff1c8000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c00d2040 con 0x7ff1c0147ff0 2024-05-11T12:14:29.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.151+0000 7ff1c8000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff1c01483d0 con 0x7ff1c00d2e80 2024-05-11T12:14:29.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.152+0000 7ff1c6000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1c0148cf0 0x7ff1c0161e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:29.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.152+0000 7ff1c6000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1c0148cf0 0x7ff1c0161e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:40244/0 (socket says 172.21.15.167:40244) 2024-05-11T12:14:29.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.152+0000 7ff1c6000640 1 -- 172.21.15.167:0/1406079146 learned_addr learned my addr 172.21.15.167:0/1406079146 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:29.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.152+0000 7ff1c5600640 1 --2- 172.21.15.167:0/1406079146 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1c00d2e80 0x7ff1c0154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:29.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.152+0000 7ff1c4c00640 1 --2- 172.21.15.167:0/1406079146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1c0147ff0 0x7ff1c01447b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:29.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.152+0000 7ff1c6000640 1 -- 172.21.15.167:0/1406079146 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1c00d2e80 msgr2=0x7ff1c0154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:29.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.152+0000 7ff1c6000640 1 --2- 172.21.15.167:0/1406079146 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1c00d2e80 0x7ff1c0154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:29.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.152+0000 7ff1c6000640 1 -- 172.21.15.167:0/1406079146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1c0147ff0 msgr2=0x7ff1c01447b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:29.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.152+0000 7ff1c6000640 1 --2- 172.21.15.167:0/1406079146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1c0147ff0 0x7ff1c01447b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:29.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.152+0000 7ff1c6000640 1 -- 172.21.15.167:0/1406079146 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1c0145090 con 0x7ff1c0148cf0 2024-05-11T12:14:29.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.152+0000 7ff1c6000640 1 --2- 172.21.15.167:0/1406079146 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1c0148cf0 0x7ff1c0161e30 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8009ee0 tx=0x7ff1b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:29.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.153+0000 7ff1bd600640 1 -- 172.21.15.167:0/1406079146 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8040070 con 0x7ff1c0148cf0 2024-05-11T12:14:29.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.153+0000 7ff1bd600640 1 -- 172.21.15.167:0/1406079146 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1b802ae60 con 0x7ff1c0148cf0 2024-05-11T12:14:29.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.153+0000 7ff1bd600640 1 -- 172.21.15.167:0/1406079146 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8031be0 con 0x7ff1c0148cf0 2024-05-11T12:14:29.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.153+0000 7ff1c8000640 1 -- 172.21.15.167:0/1406079146 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1c0145370 con 0x7ff1c0148cf0 2024-05-11T12:14:29.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.153+0000 7ff1c8000640 1 -- 172.21.15.167:0/1406079146 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1c0145930 con 0x7ff1c0148cf0 2024-05-11T12:14:29.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.154+0000 7ff1c8000640 1 -- 172.21.15.167:0/1406079146 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1c00d3db0 con 0x7ff1c0148cf0 2024-05-11T12:14:29.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.155+0000 7ff1bd600640 1 -- 172.21.15.167:0/1406079146 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff1b8039040 con 0x7ff1c0148cf0 2024-05-11T12:14:29.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.156+0000 7ff1bd600640 1 --2- 172.21.15.167:0/1406079146 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff1a0076300 0x7ff1a00787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:29.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.156+0000 7ff1bd600640 1 -- 172.21.15.167:0/1406079146 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5335+0+0 (secure 0 0 0) 0x7ff1b80b8600 con 0x7ff1c0148cf0 2024-05-11T12:14:29.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.157+0000 7ff1c5600640 1 --2- 172.21.15.167:0/1406079146 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff1a0076300 0x7ff1a00787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:29.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.157+0000 7ff1c5600640 1 --2- 172.21.15.167:0/1406079146 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff1a0076300 0x7ff1a00787c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4007a80 tx=0x7ff1b4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:29.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.161+0000 7ff1bd600640 1 -- 172.21.15.167:0/1406079146 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff1c00d3db0 con 0x7ff1c0148cf0 2024-05-11T12:14:29.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.378+0000 7ff1c8000640 1 -- 172.21.15.167:0/1406079146 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7ff1c00694b0 con 0x7ff1c0148cf0 2024-05-11T12:14:29.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.543+0000 7ff1bd600640 1 -- 172.21.15.167:0/1406079146 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v91) ==== 97+0+0 (secure 0 0 0) 0x7ff1b8082180 con 0x7ff1c0148cf0 2024-05-11T12:14:29.543 INFO:teuthology.orchestra.run.smithi167.stderr: removed all blocklist entries 2024-05-11T12:14:29.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.547+0000 7ff1c8000640 1 -- 172.21.15.167:0/1406079146 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff1a0076300 msgr2=0x7ff1a00787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:29.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.547+0000 7ff1c8000640 1 --2- 172.21.15.167:0/1406079146 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff1a0076300 0x7ff1a00787c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4007a80 tx=0x7ff1b4009040 comp rx=0 tx=0).stop 2024-05-11T12:14:29.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.547+0000 7ff1c8000640 1 -- 172.21.15.167:0/1406079146 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1c0148cf0 msgr2=0x7ff1c0161e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:29.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.547+0000 7ff1c8000640 1 --2- 172.21.15.167:0/1406079146 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1c0148cf0 0x7ff1c0161e30 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8009ee0 tx=0x7ff1b8004a40 comp rx=0 tx=0).stop 2024-05-11T12:14:29.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.548+0000 7ff1c8000640 1 -- 172.21.15.167:0/1406079146 shutdown_connections 2024-05-11T12:14:29.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.548+0000 7ff1c8000640 1 --2- 172.21.15.167:0/1406079146 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1c0148cf0 0x7ff1c0161e30 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:29.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.548+0000 7ff1c8000640 1 --2- 172.21.15.167:0/1406079146 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff1a0076300 0x7ff1a00787c0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:29.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.548+0000 7ff1c8000640 1 --2- 172.21.15.167:0/1406079146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1c0147ff0 0x7ff1c01447b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:29.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.548+0000 7ff1c8000640 1 --2- 172.21.15.167:0/1406079146 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1c00d2e80 0x7ff1c0154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:29.548 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.548+0000 7ff1c8000640 1 -- 172.21.15.167:0/1406079146 >> 172.21.15.167:0/1406079146 conn(0x7ff1c00b8300 msgr2=0x7ff1c0143040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:29.548 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.548+0000 7ff1c8000640 1 -- 172.21.15.167:0/1406079146 shutdown_connections 2024-05-11T12:14:29.548 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.548+0000 7ff1c8000640 1 -- 172.21.15.167:0/1406079146 wait complete. 2024-05-11T12:14:29.568 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-11T12:14:29.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.754+0000 7fbdc6a00640 1 Processor -- start 2024-05-11T12:14:29.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.755+0000 7fbdc6a00640 1 -- start start 2024-05-11T12:14:29.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.756+0000 7fbdc6a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbdc01458f0 0x7fbdc006d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:29.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.756+0000 7fbdc6a00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdc0071f30 con 0x7fbdc00b3a80 2024-05-11T12:14:29.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.756+0000 7fbdc6a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdc0072210 con 0x7fbdc00b5220 2024-05-11T12:14:29.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.756+0000 7fbdc6a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbdc006db80 con 0x7fbdc01458f0 2024-05-11T12:14:29.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.757+0000 7fbdc4c00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fbdc00b3a80 0x7fbdc00b4970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:55574/0 (socket says 172.21.15.167:55574) 2024-05-11T12:14:29.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.757+0000 7fbdc4c00640 1 -- 172.21.15.167:0/3716907933 learned_addr learned my addr 172.21.15.167:0/3716907933 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:14:29.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.757+0000 7fbdbfe00640 1 -- 172.21.15.167:0/3716907933 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 107052658 0 0) 0x7fbdc0071f30 con 0x7fbdc00b3a80 2024-05-11T12:14:29.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.757+0000 7fbdbfe00640 1 -- 172.21.15.167:0/3716907933 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbda8005730 con 0x7fbdc00b3a80 2024-05-11T12:14:29.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.757+0000 7fbdc5600640 1 --2- 172.21.15.167:0/3716907933 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbdc01458f0 0x7fbdc006d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:29.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.757+0000 7fbdbfe00640 1 -- 172.21.15.167:0/3716907933 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 435850703 0 0) 0x7fbda8005730 con 0x7fbdc00b3a80 2024-05-11T12:14:29.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.758+0000 7fbdbfe00640 1 -- 172.21.15.167:0/3716907933 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbdc01458f0 msgr2=0x7fbdc006d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:29.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.758+0000 7fbdbfe00640 1 --2- 172.21.15.167:0/3716907933 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbdc01458f0 0x7fbdc006d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:29.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.758+0000 7fbdbfe00640 1 -- 172.21.15.167:0/3716907933 >> v1:172.21.15.204:6789/0 conn(0x7fbdc00b5220 legacy=0x7fbdc0072fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:29.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.758+0000 7fbdbfe00640 1 -- 172.21.15.167:0/3716907933 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdc00724f0 con 0x7fbdc00b3a80 2024-05-11T12:14:29.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.758+0000 7fbdbfe00640 1 -- 172.21.15.167:0/3716907933 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7fbdac0016f0 con 0x7fbdc00b3a80 2024-05-11T12:14:29.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.758+0000 7fbdbfe00640 1 -- 172.21.15.167:0/3716907933 >> v1:172.21.15.167:6789/0 conn(0x7fbdc00b3a80 legacy=0x7fbdc00b4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:29.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.758+0000 7fbdbfe00640 1 --2- 172.21.15.167:0/3716907933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbda8005c40 0x7fbda8006060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:29.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.758+0000 7fbdbfe00640 1 --2- 172.21.15.167:0/3716907933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbda8006a10 0x7fbda8006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:29.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.758+0000 7fbdbfe00640 1 --2- 172.21.15.167:0/3716907933 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbdc01458f0 0x7fbda8007410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:29.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.758+0000 7fbdbfe00640 1 -- 172.21.15.167:0/3716907933 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbda800dcd0 con 0x7fbda8005c40 2024-05-11T12:14:29.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.758+0000 7fbdbfe00640 1 -- 172.21.15.167:0/3716907933 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbda800de80 con 0x7fbda8006a10 2024-05-11T12:14:29.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.758+0000 7fbdbfe00640 1 -- 172.21.15.167:0/3716907933 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbda800e080 con 0x7fbdc01458f0 2024-05-11T12:14:29.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.758+0000 7fbdc5600640 1 --2- 172.21.15.167:0/3716907933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbda8005c40 0x7fbda8006060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:29.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.758+0000 7fbdc5600640 1 -- 172.21.15.167:0/3716907933 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbdc01458f0 msgr2=0x7fbda8007410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:29.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.758+0000 7fbdc5600640 1 --2- 172.21.15.167:0/3716907933 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbdc01458f0 0x7fbda8007410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:29.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.758+0000 7fbdc5600640 1 -- 172.21.15.167:0/3716907933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbda8006a10 msgr2=0x7fbda8006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:29.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.758+0000 7fbdc5600640 1 --2- 172.21.15.167:0/3716907933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbda8006a10 0x7fbda8006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:29.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.758+0000 7fbdc5600640 1 -- 172.21.15.167:0/3716907933 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbda8005730 con 0x7fbda8005c40 2024-05-11T12:14:29.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.759+0000 7fbdc5600640 1 --2- 172.21.15.167:0/3716907933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbda8005c40 0x7fbda8006060 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4009ac0 tx=0x7fbda8007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eea240c782109715 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:29.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.760+0000 7fbdbfe00640 1 -- 172.21.15.167:0/3716907933 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4036070 con 0x7fbda8005c40 2024-05-11T12:14:29.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.760+0000 7fbdbfe00640 1 -- 172.21.15.167:0/3716907933 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdb4031400 con 0x7fbda8005c40 2024-05-11T12:14:29.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.760+0000 7fbdbfe00640 1 -- 172.21.15.167:0/3716907933 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4009690 con 0x7fbda8005c40 2024-05-11T12:14:29.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.760+0000 7fbdc6a00640 1 -- 172.21.15.167:0/3716907933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbda8005c40 msgr2=0x7fbda8006060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:29.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.760+0000 7fbdc6a00640 1 --2- 172.21.15.167:0/3716907933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbda8005c40 0x7fbda8006060 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4009ac0 tx=0x7fbda8007a40 comp rx=0 tx=0).stop 2024-05-11T12:14:29.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.760+0000 7fbdc6000640 1 -- 172.21.15.167:0/3716907933 reap_dead start 2024-05-11T12:14:29.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.760+0000 7fbdc6a00640 1 -- 172.21.15.167:0/3716907933 shutdown_connections 2024-05-11T12:14:29.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.760+0000 7fbdc6a00640 1 -- 172.21.15.167:0/3716907933 >> 172.21.15.167:0/3716907933 conn(0x7fbdc00b3460 msgr2=0x7fbdc00d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:29.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.761+0000 7fbdc6a00640 1 -- 172.21.15.167:0/3716907933 shutdown_connections 2024-05-11T12:14:29.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.761+0000 7fbdc6a00640 1 -- 172.21.15.167:0/3716907933 wait complete. 2024-05-11T12:14:29.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.762+0000 7fbdc6a00640 1 Processor -- start 2024-05-11T12:14:29.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.762+0000 7fbdc6a00640 1 -- start start 2024-05-11T12:14:29.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.763+0000 7fbdc6a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbda8005c40 0x7fbdc014d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:29.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.763+0000 7fbdc6a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbdc01f5620 0x7fbdc015efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:29.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.763+0000 7fbdc6a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbdc01f5aa0 0x7fbdc00caf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:29.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.763+0000 7fbdc6a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc006d640 con 0x7fbda8005c40 2024-05-11T12:14:29.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.763+0000 7fbdc6a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc0072ab0 con 0x7fbdc01f5620 2024-05-11T12:14:29.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.763+0000 7fbdc6a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbdc006d7c0 con 0x7fbdc01f5aa0 2024-05-11T12:14:29.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.764+0000 7fbdc5600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbda8005c40 0x7fbdc014d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:29.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.764+0000 7fbdc5600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbda8005c40 0x7fbdc014d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:40270/0 (socket says 172.21.15.167:40270) 2024-05-11T12:14:29.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.764+0000 7fbdc5600640 1 -- 172.21.15.167:0/2660511917 learned_addr learned my addr 172.21.15.167:0/2660511917 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:29.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.764+0000 7fbdc5600640 1 -- 172.21.15.167:0/2660511917 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbdc01f5aa0 msgr2=0x7fbdc00caf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:29.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.764+0000 7fbdc5600640 1 --2- 172.21.15.167:0/2660511917 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbdc01f5aa0 0x7fbdc00caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:29.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.764+0000 7fbdc5600640 1 -- 172.21.15.167:0/2660511917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbdc01f5620 msgr2=0x7fbdc015efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:29.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.764+0000 7fbdc5600640 1 --2- 172.21.15.167:0/2660511917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbdc01f5620 0x7fbdc015efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:29.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.764+0000 7fbdc5600640 1 -- 172.21.15.167:0/2660511917 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdc0142a00 con 0x7fbda8005c40 2024-05-11T12:14:29.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.764+0000 7fbdc5600640 1 --2- 172.21.15.167:0/2660511917 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbda8005c40 0x7fbdc014d7c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4009ac0 tx=0x7fbdb4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:29.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.765+0000 7fbdbd600640 1 -- 172.21.15.167:0/2660511917 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4040020 con 0x7fbda8005c40 2024-05-11T12:14:29.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.765+0000 7fbdbd600640 1 -- 172.21.15.167:0/2660511917 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdb4003bc0 con 0x7fbda8005c40 2024-05-11T12:14:29.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.765+0000 7fbdbd600640 1 -- 172.21.15.167:0/2660511917 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4036040 con 0x7fbda8005c40 2024-05-11T12:14:29.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.765+0000 7fbdc6a00640 1 -- 172.21.15.167:0/2660511917 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdc0142ce0 con 0x7fbda8005c40 2024-05-11T12:14:29.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.765+0000 7fbdc6a00640 1 -- 172.21.15.167:0/2660511917 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdc01432a0 con 0x7fbda8005c40 2024-05-11T12:14:29.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.766+0000 7fbdc6a00640 1 -- 172.21.15.167:0/2660511917 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdc00694b0 con 0x7fbda8005c40 2024-05-11T12:14:29.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.767+0000 7fbdbd600640 1 -- 172.21.15.167:0/2660511917 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbdb4043020 con 0x7fbda8005c40 2024-05-11T12:14:29.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.768+0000 7fbdbd600640 1 --2- 172.21.15.167:0/2660511917 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbda0076300 0x7fbda00787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:29.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.768+0000 7fbdbd600640 1 -- 172.21.15.167:0/2660511917 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5249+0+0 (secure 0 0 0) 0x7fbdb40b7790 con 0x7fbda8005c40 2024-05-11T12:14:29.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.768+0000 7fbdc4c00640 1 --2- 172.21.15.167:0/2660511917 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbda0076300 0x7fbda00787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:29.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.769+0000 7fbdc4c00640 1 --2- 172.21.15.167:0/2660511917 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbda0076300 0x7fbda00787c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0160580 tx=0x7fbdac0015e0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:29.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:29.773+0000 7fbdbd600640 1 -- 172.21.15.167:0/2660511917 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fbdb404a030 con 0x7fbda8005c40 2024-05-11T12:14:30.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.060+0000 7fbdc6a00640 1 -- 172.21.15.167:0/2660511917 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fbdc00ceb80 con 0x7fbda8005c40 2024-05-11T12:14:30.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.062+0000 7fbdbd600640 1 -- 172.21.15.167:0/2660511917 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) ==== 80+0+5304 (secure 0 0 0) 0x7fbdb40044e0 con 0x7fbda8005c40 2024-05-11T12:14:30.061 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:30.061 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T12:14:30.061 INFO:teuthology.orchestra.run.smithi167.stdout: "auth_dump": [ 2024-05-11T12:14:30.061 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:14:30.061 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "mds.a", 2024-05-11T12:14:30.061 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAEYD9msMGnFBAAamqokXQ+znxPHtHhhs611g==", 2024-05-11T12:14:30.061 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:14:30.061 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T12:14:30.062 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow *", 2024-05-11T12:14:30.062 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow *", 2024-05-11T12:14:30.062 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:14:30.062 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:14:30.062 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:14:30.062 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:14:30.062 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "mds.b", 2024-05-11T12:14:30.062 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAEYD9mqbCgMhAA5CNUf1cz77nUqLli2bTFcQ==", 2024-05-11T12:14:30.062 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:14:30.062 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T12:14:30.062 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow *", 2024-05-11T12:14:30.062 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow *", 2024-05-11T12:14:30.063 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:14:30.063 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:14:30.063 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:14:30.063 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:14:30.063 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "mds.c", 2024-05-11T12:14:30.063 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAEYD9mhylUIxAAqrIRYuNIHKfoI00zx+Gh1Q==", 2024-05-11T12:14:30.063 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:14:30.063 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T12:14:30.063 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow *", 2024-05-11T12:14:30.063 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow *", 2024-05-11T12:14:30.063 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:14:30.063 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:14:30.063 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:14:30.063 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:14:30.063 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "mds.d", 2024-05-11T12:14:30.064 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAFYD9mcUjpBRAAbx54iDHcqexfNXGKk/avlw==", 2024-05-11T12:14:30.064 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:14:30.064 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T12:14:30.064 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow *", 2024-05-11T12:14:30.064 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow *", 2024-05-11T12:14:30.064 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:14:30.064 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:14:30.064 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:14:30.064 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:14:30.064 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.0", 2024-05-11T12:14:30.064 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAJYD9mb71mBxAAKMSzkfppZVbEevwELKrqvw==", 2024-05-11T12:14:30.064 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:14:30.064 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T12:14:30.064 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T12:14:30.064 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:14:30.065 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:14:30.065 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:14:30.065 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:14:30.065 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.1", 2024-05-11T12:14:30.065 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAMYD9m/qx7EhAAUcZntottFNBwqjdc/CqFbw==", 2024-05-11T12:14:30.065 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:14:30.065 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T12:14:30.065 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T12:14:30.065 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:14:30.065 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:14:30.065 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:14:30.065 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:14:30.065 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.2", 2024-05-11T12:14:30.065 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAPYD9m5PKRKxAAFQuzh7dAN8PVEHBq+UdQvg==", 2024-05-11T12:14:30.066 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:14:30.066 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T12:14:30.066 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T12:14:30.066 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:14:30.066 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:14:30.066 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:14:30.066 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:14:30.066 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.3", 2024-05-11T12:14:30.066 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQASYD9mZAy+NhAA49k6QbSvtSxA/5t8bWbPuQ==", 2024-05-11T12:14:30.066 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:14:30.066 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T12:14:30.066 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T12:14:30.066 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:14:30.066 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:14:30.067 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:14:30.067 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:14:30.067 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.4", 2024-05-11T12:14:30.067 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAZYD9mEbsvIBAA3nkQXQTbIf7ubq1SJtUeIA==", 2024-05-11T12:14:30.067 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:14:30.067 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T12:14:30.067 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T12:14:30.067 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:14:30.067 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:14:30.067 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:14:30.067 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:14:30.067 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.5", 2024-05-11T12:14:30.067 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAcYD9mtdsHOxAAvNUSkMgQlDc2n1ak/lfRgg==", 2024-05-11T12:14:30.067 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:14:30.067 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T12:14:30.068 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T12:14:30.068 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:14:30.068 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:14:30.068 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:14:30.068 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:14:30.068 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.6", 2024-05-11T12:14:30.068 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAgYD9mr3u8CRAAm2ShwV0lJpIVudodmviusg==", 2024-05-11T12:14:30.068 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:14:30.068 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T12:14:30.068 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T12:14:30.068 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:14:30.068 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:14:30.068 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:14:30.068 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:14:30.068 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.7", 2024-05-11T12:14:30.069 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAjYD9m+0oyFRAAUVQW0kQmvtU8IE599nrWJQ==", 2024-05-11T12:14:30.069 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:14:30.069 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T12:14:30.069 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T12:14:30.069 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:14:30.069 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:14:30.069 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:14:30.069 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:14:30.069 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "client.0", 2024-05-11T12:14:30.069 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAFYD9mLK/lEhAAyERTYMl250KkBvIrAXWYSw==", 2024-05-11T12:14:30.069 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:14:30.069 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T12:14:30.069 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow r", 2024-05-11T12:14:30.069 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-11T12:14:30.069 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:14:30.070 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:14:30.070 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:14:30.070 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "client.1", 2024-05-11T12:14:30.070 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAFYD9m2oSjJhAAz53H1SvPStkxGq5V+2VFeA==", 2024-05-11T12:14:30.070 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:14:30.070 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T12:14:30.070 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow r", 2024-05-11T12:14:30.070 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow rw", 2024-05-11T12:14:30.070 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow rwx" 2024-05-11T12:14:30.070 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:14:30.070 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:14:30.070 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:14:30.070 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "client.2", 2024-05-11T12:14:30.070 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAFYD9mMzXnHBAAmJ//hwSofaZpk+6ykxVM/A==", 2024-05-11T12:14:30.071 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:14:30.071 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T12:14:30.071 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow r", 2024-05-11T12:14:30.071 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow rw", 2024-05-11T12:14:30.071 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow rwx" 2024-05-11T12:14:30.071 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:14:30.071 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:14:30.071 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:14:30.071 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "client.3", 2024-05-11T12:14:30.071 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAFYD9muOmsLxAA/tYFT4YNy3adENgVNLljdw==", 2024-05-11T12:14:30.071 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:14:30.071 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T12:14:30.072 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow r", 2024-05-11T12:14:30.072 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow rw", 2024-05-11T12:14:30.072 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow rwx" 2024-05-11T12:14:30.072 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:14:30.072 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:14:30.072 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:14:30.072 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "client.admin", 2024-05-11T12:14:30.072 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQADYD9mItegDBAALtuDP6qzAvbrwiROf94biw==", 2024-05-11T12:14:30.072 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:14:30.072 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow *", 2024-05-11T12:14:30.072 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow *", 2024-05-11T12:14:30.072 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow *", 2024-05-11T12:14:30.072 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:14:30.073 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:14:30.073 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:14:30.073 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:14:30.073 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "mgr.x", 2024-05-11T12:14:30.073 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQAEYD9mAZTKCRAAJbY821O+4pUMC0DgxRa3og==", 2024-05-11T12:14:30.073 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:14:30.073 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow *", 2024-05-11T12:14:30.073 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile mgr", 2024-05-11T12:14:30.073 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:14:30.073 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:14:30.073 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:14:30.073 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:14:30.073 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "mgr.y", 2024-05-11T12:14:30.073 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQADYD9m2cZRORAAX9kYcT5/t4f5jQRZwXHHzw==", 2024-05-11T12:14:30.074 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T12:14:30.074 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow *", 2024-05-11T12:14:30.074 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile mgr", 2024-05-11T12:14:30.074 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T12:14:30.074 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:14:30.074 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:14:30.074 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-05-11T12:14:30.074 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T12:14:30.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.064+0000 7fbdc6a00640 1 -- 172.21.15.167:0/2660511917 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbda0076300 msgr2=0x7fbda00787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:30.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.064+0000 7fbdc6a00640 1 --2- 172.21.15.167:0/2660511917 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbda0076300 0x7fbda00787c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0160580 tx=0x7fbdac0015e0 comp rx=0 tx=0).stop 2024-05-11T12:14:30.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.064+0000 7fbdc6a00640 1 -- 172.21.15.167:0/2660511917 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbda8005c40 msgr2=0x7fbdc014d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:30.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.064+0000 7fbdc6a00640 1 --2- 172.21.15.167:0/2660511917 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbda8005c40 0x7fbdc014d7c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4009ac0 tx=0x7fbdb4004730 comp rx=0 tx=0).stop 2024-05-11T12:14:30.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.064+0000 7fbdc6a00640 1 -- 172.21.15.167:0/2660511917 shutdown_connections 2024-05-11T12:14:30.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.064+0000 7fbdc6a00640 1 --2- 172.21.15.167:0/2660511917 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbdc01f5aa0 0x7fbdc00caf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:30.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.064+0000 7fbdc6a00640 1 --2- 172.21.15.167:0/2660511917 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbda0076300 0x7fbda00787c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:30.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.064+0000 7fbdc6a00640 1 --2- 172.21.15.167:0/2660511917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbdc01f5620 0x7fbdc015efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:30.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.064+0000 7fbdc6a00640 1 --2- 172.21.15.167:0/2660511917 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbda8005c40 0x7fbdc014d7c0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:30.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.064+0000 7fbdc6a00640 1 -- 172.21.15.167:0/2660511917 >> 172.21.15.167:0/2660511917 conn(0x7fbdc00b3460 msgr2=0x7fbdc006def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:30.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.065+0000 7fbdc6a00640 1 -- 172.21.15.167:0/2660511917 shutdown_connections 2024-05-11T12:14:30.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.065+0000 7fbdc6a00640 1 -- 172.21.15.167:0/2660511917 wait complete. 2024-05-11T12:14:30.083 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-11T12:14:30.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.280+0000 7f67f1c00640 1 Processor -- start 2024-05-11T12:14:30.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.280+0000 7f67f1c00640 1 -- start start 2024-05-11T12:14:30.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.281+0000 7f67f1c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67ec0af1f0 0x7f67ec0cf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:30.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.282+0000 7f67f1c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f67ec0af5d0 0x7f67ec073ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:30.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.282+0000 7f67f1c00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f67ec06d460 0x7f67ec06d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:30.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.282+0000 7f67f1c00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f67ec1608d0 con 0x7f67ec06d460 2024-05-11T12:14:30.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.282+0000 7f67f1c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f67ec0cf700 con 0x7f67ec0af5d0 2024-05-11T12:14:30.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.282+0000 7f67f1c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f67ec06fe10 con 0x7f67ec0af1f0 2024-05-11T12:14:30.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.282+0000 7f67eb400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f67ec06d460 0x7f67ec06d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:30.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.282+0000 7f67eb400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f67ec06d460 0x7f67ec06d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:40280/0 (socket says 172.21.15.167:40280) 2024-05-11T12:14:30.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.282+0000 7f67eb400640 1 -- 172.21.15.167:0/1732357817 learned_addr learned my addr 172.21.15.167:0/1732357817 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:30.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.282+0000 7f67eaa00640 1 --2- 172.21.15.167:0/1732357817 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67ec0af1f0 0x7f67ec0cf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:30.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.282+0000 7f67ea000640 1 --2- 172.21.15.167:0/1732357817 >> v2:172.21.15.204:3300/0 conn(0x7f67ec0af5d0 0x7f67ec073ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:30.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.282+0000 7f67eb400640 1 -- 172.21.15.167:0/1732357817 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67ec0af1f0 msgr2=0x7f67ec0cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:30.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.282+0000 7f67eb400640 1 --2- 172.21.15.167:0/1732357817 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67ec0af1f0 0x7f67ec0cf1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:30.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.282+0000 7f67eb400640 1 -- 172.21.15.167:0/1732357817 >> v2:172.21.15.204:3300/0 conn(0x7f67ec0af5d0 msgr2=0x7f67ec073ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:30.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.282+0000 7f67eb400640 1 --2- 172.21.15.167:0/1732357817 >> v2:172.21.15.204:3300/0 conn(0x7f67ec0af5d0 0x7f67ec073ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:30.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.283+0000 7f67eb400640 1 -- 172.21.15.167:0/1732357817 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67ec15fcc0 con 0x7f67ec06d460 2024-05-11T12:14:30.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.283+0000 7f67ea000640 1 --2- 172.21.15.167:0/1732357817 >> v2:172.21.15.204:3300/0 conn(0x7f67ec0af5d0 0x7f67ec073ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:30.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.283+0000 7f67eaa00640 1 --2- 172.21.15.167:0/1732357817 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67ec0af1f0 0x7f67ec0cf1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:30.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.284+0000 7f67eb400640 1 --2- 172.21.15.167:0/1732357817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67ec06d460 0x7f67ec06d8f0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f67ec06e0a0 tx=0x7f67e402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eee5b93b1c34833d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:30.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.284+0000 7f67e9600640 1 -- 172.21.15.167:0/1732357817 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e4036070 con 0x7f67ec06d460 2024-05-11T12:14:30.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.284+0000 7f67e9600640 1 -- 172.21.15.167:0/1732357817 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67e4031600 con 0x7f67ec06d460 2024-05-11T12:14:30.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.284+0000 7f67e9600640 1 -- 172.21.15.167:0/1732357817 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e40319d0 con 0x7f67ec06d460 2024-05-11T12:14:30.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.285+0000 7f67f1c00640 1 -- 172.21.15.167:0/1732357817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67ec06d460 msgr2=0x7f67ec06d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:30.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.285+0000 7f67f1c00640 1 --2- 172.21.15.167:0/1732357817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67ec06d460 0x7f67ec06d8f0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f67ec06e0a0 tx=0x7f67e402a610 comp rx=0 tx=0).stop 2024-05-11T12:14:30.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.285+0000 7f67f1c00640 1 -- 172.21.15.167:0/1732357817 shutdown_connections 2024-05-11T12:14:30.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.285+0000 7f67f1c00640 1 --2- 172.21.15.167:0/1732357817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67ec06d460 0x7f67ec06d8f0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:30.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.285+0000 7f67f1c00640 1 --2- 172.21.15.167:0/1732357817 >> v2:172.21.15.204:3300/0 conn(0x7f67ec0af5d0 0x7f67ec073ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:30.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.285+0000 7f67f1c00640 1 --2- 172.21.15.167:0/1732357817 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67ec0af1f0 0x7f67ec0cf1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:30.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.285+0000 7f67f1c00640 1 -- 172.21.15.167:0/1732357817 >> 172.21.15.167:0/1732357817 conn(0x7f67ec0cd8f0 msgr2=0x7f67ec0cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:30.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.285+0000 7f67f1c00640 1 -- 172.21.15.167:0/1732357817 shutdown_connections 2024-05-11T12:14:30.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.285+0000 7f67f1c00640 1 -- 172.21.15.167:0/1732357817 wait complete. 2024-05-11T12:14:30.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.286+0000 7f67f1c00640 1 Processor -- start 2024-05-11T12:14:30.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.287+0000 7f67f1c00640 1 -- start start 2024-05-11T12:14:30.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.287+0000 7f67f1c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67ec06d460 0x7f67ec157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:30.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.288+0000 7f67f1c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67ec0af1f0 0x7f67ec153c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:30.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.288+0000 7f67f1c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67ec0af5d0 0x7f67ec163220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:30.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.288+0000 7f67f1c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec160560 con 0x7f67ec06d460 2024-05-11T12:14:30.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.288+0000 7f67f1c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec160280 con 0x7f67ec0af1f0 2024-05-11T12:14:30.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.288+0000 7f67f1c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f67ec15f920 con 0x7f67ec0af5d0 2024-05-11T12:14:30.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.288+0000 7f67eaa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67ec06d460 0x7f67ec157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:30.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.288+0000 7f67eaa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67ec06d460 0x7f67ec157710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:40292/0 (socket says 172.21.15.167:40292) 2024-05-11T12:14:30.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.288+0000 7f67eaa00640 1 -- 172.21.15.167:0/294827135 learned_addr learned my addr 172.21.15.167:0/294827135 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:30.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.288+0000 7f67eaa00640 1 -- 172.21.15.167:0/294827135 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67ec0af5d0 msgr2=0x7f67ec163220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:30.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.288+0000 7f67eaa00640 1 --2- 172.21.15.167:0/294827135 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67ec0af5d0 0x7f67ec163220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:30.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.288+0000 7f67eaa00640 1 -- 172.21.15.167:0/294827135 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67ec0af1f0 msgr2=0x7f67ec153c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:30.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.288+0000 7f67eaa00640 1 --2- 172.21.15.167:0/294827135 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67ec0af1f0 0x7f67ec153c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:30.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.288+0000 7f67eaa00640 1 -- 172.21.15.167:0/294827135 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67ec1f5070 con 0x7f67ec06d460 2024-05-11T12:14:30.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.289+0000 7f67eaa00640 1 --2- 172.21.15.167:0/294827135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67ec06d460 0x7f67ec157710 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f67d400c7f0 tx=0x7f67d400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:30.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.289+0000 7f67e2a00640 1 -- 172.21.15.167:0/294827135 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d4015070 con 0x7f67ec06d460 2024-05-11T12:14:30.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.289+0000 7f67e2a00640 1 -- 172.21.15.167:0/294827135 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67d4007a30 con 0x7f67ec06d460 2024-05-11T12:14:30.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.289+0000 7f67e2a00640 1 -- 172.21.15.167:0/294827135 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d4007e00 con 0x7f67ec06d460 2024-05-11T12:14:30.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.289+0000 7f67f1c00640 1 -- 172.21.15.167:0/294827135 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67ec1f5350 con 0x7f67ec06d460 2024-05-11T12:14:30.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.289+0000 7f67f1c00640 1 -- 172.21.15.167:0/294827135 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67ec1f5910 con 0x7f67ec06d460 2024-05-11T12:14:30.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.290+0000 7f67f1c00640 1 -- 172.21.15.167:0/294827135 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67ec1f5b10 con 0x7f67ec06d460 2024-05-11T12:14:30.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.291+0000 7f67e2a00640 1 -- 172.21.15.167:0/294827135 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f67d4018070 con 0x7f67ec06d460 2024-05-11T12:14:30.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.292+0000 7f67e2a00640 1 --2- 172.21.15.167:0/294827135 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f67c8076300 0x7f67c80787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:30.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.293+0000 7f67e2a00640 1 -- 172.21.15.167:0/294827135 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5249+0+0 (secure 0 0 0) 0x7f67d4097430 con 0x7f67ec06d460 2024-05-11T12:14:30.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.293+0000 7f67ea000640 1 --2- 172.21.15.167:0/294827135 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f67c8076300 0x7f67c80787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:30.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.293+0000 7f67ea000640 1 --2- 172.21.15.167:0/294827135 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f67c8076300 0x7f67c80787c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f67dc006b10 tx=0x7f67dc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:30.296 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.297+0000 7f67e2a00640 1 -- 172.21.15.167:0/294827135 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f67ec1f5b10 con 0x7f67ec06d460 2024-05-11T12:14:30.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.505+0000 7f67f1c00640 1 -- 172.21.15.167:0/294827135 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f67ec069570 con 0x7f67ec06d460 2024-05-11T12:14:30.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.506+0000 7f67e2a00640 1 -- 172.21.15.167:0/294827135 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v93) ==== 84+0+65 (secure 0 0 0) 0x7f67d4060eb0 con 0x7f67ec06d460 2024-05-11T12:14:30.505 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:30.505 INFO:teuthology.orchestra.run.smithi167.stdout:[ 2024-05-11T12:14:30.506 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:14:30.506 INFO:teuthology.orchestra.run.smithi167.stdout: "poolnum": 1, 2024-05-11T12:14:30.506 INFO:teuthology.orchestra.run.smithi167.stdout: "poolname": ".mgr" 2024-05-11T12:14:30.506 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:14:30.506 INFO:teuthology.orchestra.run.smithi167.stdout:] 2024-05-11T12:14:30.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.509+0000 7f67f1c00640 1 -- 172.21.15.167:0/294827135 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f67c8076300 msgr2=0x7f67c80787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:30.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.509+0000 7f67f1c00640 1 --2- 172.21.15.167:0/294827135 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f67c8076300 0x7f67c80787c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f67dc006b10 tx=0x7f67dc008040 comp rx=0 tx=0).stop 2024-05-11T12:14:30.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.509+0000 7f67f1c00640 1 -- 172.21.15.167:0/294827135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67ec06d460 msgr2=0x7f67ec157710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:30.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.509+0000 7f67f1c00640 1 --2- 172.21.15.167:0/294827135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67ec06d460 0x7f67ec157710 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f67d400c7f0 tx=0x7f67d400cbf0 comp rx=0 tx=0).stop 2024-05-11T12:14:30.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.509+0000 7f67f1c00640 1 -- 172.21.15.167:0/294827135 shutdown_connections 2024-05-11T12:14:30.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.509+0000 7f67f1c00640 1 --2- 172.21.15.167:0/294827135 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67ec0af5d0 0x7f67ec163220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:30.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.509+0000 7f67f1c00640 1 --2- 172.21.15.167:0/294827135 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f67c8076300 0x7f67c80787c0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:30.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.509+0000 7f67f1c00640 1 --2- 172.21.15.167:0/294827135 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67ec0af1f0 0x7f67ec153c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:30.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.509+0000 7f67f1c00640 1 --2- 172.21.15.167:0/294827135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67ec06d460 0x7f67ec157710 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:30.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.509+0000 7f67f1c00640 1 -- 172.21.15.167:0/294827135 >> 172.21.15.167:0/294827135 conn(0x7f67ec0cd8f0 msgr2=0x7f67ec072c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:30.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.509+0000 7f67f1c00640 1 -- 172.21.15.167:0/294827135 shutdown_connections 2024-05-11T12:14:30.510 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.509+0000 7f67f1c00640 1 -- 172.21.15.167:0/294827135 wait complete. 2024-05-11T12:14:30.527 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-11T12:14:30.527 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-11T12:14:30.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.730+0000 7f35c6000640 1 Processor -- start 2024-05-11T12:14:30.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.730+0000 7f35c6000640 1 -- start start 2024-05-11T12:14:30.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.732+0000 7f35c6000640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f35c015fba0 0x7f35c006d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:30.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.732+0000 7f35c6000640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35c0160340 con 0x7f35c006d9a0 2024-05-11T12:14:30.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.732+0000 7f35c6000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35c0160620 con 0x7f35c015cb90 2024-05-11T12:14:30.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.732+0000 7f35c6000640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f35c006ddd0 con 0x7f35c015fba0 2024-05-11T12:14:30.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.732+0000 7f35bfe00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f35c015fba0 0x7f35c006d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:30.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.732+0000 7f35bfe00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f35c015fba0 0x7f35c006d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:40308/0 (socket says 172.21.15.167:40308) 2024-05-11T12:14:30.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.732+0000 7f35bfe00640 1 -- 172.21.15.167:0/1705798878 learned_addr learned my addr 172.21.15.167:0/1705798878 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:30.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.732+0000 7f35c5600640 1 --1- 172.21.15.167:0/1705798878 >> v1:172.21.15.167:6789/0 conn(0x7f35c006d9a0 0x7f35c0072680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:55586/0 (socket says 172.21.15.167:55586) 2024-05-11T12:14:30.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.732+0000 7f35bfe00640 1 -- 172.21.15.167:0/1705798878 >> v1:172.21.15.204:6789/0 conn(0x7f35c015cb90 legacy=0x7f35c015cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:30.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.733+0000 7f35bfe00640 1 -- 172.21.15.167:0/1705798878 >> v1:172.21.15.167:6789/0 conn(0x7f35c006d9a0 legacy=0x7f35c0072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:30.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.733+0000 7f35bfe00640 1 -- 172.21.15.167:0/1705798878 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35c0160900 con 0x7f35c015fba0 2024-05-11T12:14:30.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.733+0000 7f35bf400640 1 -- 172.21.15.167:0/1705798878 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1230957723 0 0) 0x7f35c0160340 con 0x7f35c006d9a0 2024-05-11T12:14:30.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.734+0000 7f35bfe00640 1 --2- 172.21.15.167:0/1705798878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35c015fba0 0x7f35c006d460 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f35ac009870 tx=0x7f35ac02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbd0ab3211ef21f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:30.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.734+0000 7f35bf400640 1 -- 172.21.15.167:0/1705798878 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ac037070 con 0x7f35c015fba0 2024-05-11T12:14:30.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.734+0000 7f35bf400640 1 -- 172.21.15.167:0/1705798878 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35ac032440 con 0x7f35c015fba0 2024-05-11T12:14:30.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.734+0000 7f35bf400640 1 -- 172.21.15.167:0/1705798878 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ac032810 con 0x7f35c015fba0 2024-05-11T12:14:30.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.735+0000 7f35c6000640 1 -- 172.21.15.167:0/1705798878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35c015fba0 msgr2=0x7f35c006d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:30.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.735+0000 7f35c6000640 1 --2- 172.21.15.167:0/1705798878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35c015fba0 0x7f35c006d460 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f35ac009870 tx=0x7f35ac02a250 comp rx=0 tx=0).stop 2024-05-11T12:14:30.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.735+0000 7f35c6000640 1 -- 172.21.15.167:0/1705798878 shutdown_connections 2024-05-11T12:14:30.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.735+0000 7f35c6000640 1 --2- 172.21.15.167:0/1705798878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35c015fba0 0x7f35c006d460 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:30.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.735+0000 7f35c6000640 1 -- 172.21.15.167:0/1705798878 >> 172.21.15.167:0/1705798878 conn(0x7f35c00b32a0 msgr2=0x7f35c00d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:30.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.735+0000 7f35c6000640 1 -- 172.21.15.167:0/1705798878 shutdown_connections 2024-05-11T12:14:30.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.735+0000 7f35c6000640 1 -- 172.21.15.167:0/1705798878 wait complete. 2024-05-11T12:14:30.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.737+0000 7f35c6000640 1 Processor -- start 2024-05-11T12:14:30.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.737+0000 7f35c6000640 1 -- start start 2024-05-11T12:14:30.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.738+0000 7f35c6000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35c006d9a0 0x7f35c0155b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:30.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.738+0000 7f35c6000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35c015cb90 0x7f35c01f7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:30.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.738+0000 7f35c6000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35c015fba0 0x7f35c0158e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:30.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.738+0000 7f35c6000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f35c0072d90 con 0x7f35c015fba0 2024-05-11T12:14:30.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.738+0000 7f35c6000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f35c0160050 con 0x7f35c006d9a0 2024-05-11T12:14:30.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.738+0000 7f35c6000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f35c0072f10 con 0x7f35c015cb90 2024-05-11T12:14:30.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.738+0000 7f35c5600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35c015fba0 0x7f35c0158e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:30.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.738+0000 7f35c5600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35c015fba0 0x7f35c0158e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:40312/0 (socket says 172.21.15.167:40312) 2024-05-11T12:14:30.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.738+0000 7f35c5600640 1 -- 172.21.15.167:0/2743376532 learned_addr learned my addr 172.21.15.167:0/2743376532 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:30.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.738+0000 7f35bfe00640 1 --2- 172.21.15.167:0/2743376532 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35c015cb90 0x7f35c01f7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:30.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.738+0000 7f35c4c00640 1 --2- 172.21.15.167:0/2743376532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35c006d9a0 0x7f35c0155b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:30.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.738+0000 7f35c5600640 1 -- 172.21.15.167:0/2743376532 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35c015cb90 msgr2=0x7f35c01f7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:30.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.738+0000 7f35c5600640 1 --2- 172.21.15.167:0/2743376532 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35c015cb90 0x7f35c01f7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:30.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.738+0000 7f35c5600640 1 -- 172.21.15.167:0/2743376532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35c006d9a0 msgr2=0x7f35c0155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:30.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.738+0000 7f35c5600640 1 --2- 172.21.15.167:0/2743376532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35c006d9a0 0x7f35c0155b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:30.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.738+0000 7f35c5600640 1 -- 172.21.15.167:0/2743376532 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35c01f7680 con 0x7f35c015fba0 2024-05-11T12:14:30.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.738+0000 7f35c4c00640 1 --2- 172.21.15.167:0/2743376532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35c006d9a0 0x7f35c0155b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:30.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.739+0000 7f35c5600640 1 --2- 172.21.15.167:0/2743376532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35c015fba0 0x7f35c0158e20 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f35b800c7a0 tx=0x7f35b800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:30.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.739+0000 7f35bcc00640 1 -- 172.21.15.167:0/2743376532 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b8018070 con 0x7f35c015fba0 2024-05-11T12:14:30.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.739+0000 7f35bcc00640 1 -- 172.21.15.167:0/2743376532 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35b800a5e0 con 0x7f35c015fba0 2024-05-11T12:14:30.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.739+0000 7f35bcc00640 1 -- 172.21.15.167:0/2743376532 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b800aa50 con 0x7f35c015fba0 2024-05-11T12:14:30.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.739+0000 7f35c6000640 1 -- 172.21.15.167:0/2743376532 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35c01f3560 con 0x7f35c015fba0 2024-05-11T12:14:30.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.739+0000 7f35c6000640 1 -- 172.21.15.167:0/2743376532 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35c01f3b20 con 0x7f35c015fba0 2024-05-11T12:14:30.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.740+0000 7f35c6000640 1 -- 172.21.15.167:0/2743376532 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35c00694b0 con 0x7f35c015fba0 2024-05-11T12:14:30.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.741+0000 7f35bcc00640 1 -- 172.21.15.167:0/2743376532 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f35b801b070 con 0x7f35c015fba0 2024-05-11T12:14:30.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.742+0000 7f35bcc00640 1 --2- 172.21.15.167:0/2743376532 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f35a00762d0 0x7f35a0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:30.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.743+0000 7f35bcc00640 1 -- 172.21.15.167:0/2743376532 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5249+0+0 (secure 0 0 0) 0x7f35b80969d0 con 0x7f35c015fba0 2024-05-11T12:14:30.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.743+0000 7f35c4c00640 1 --2- 172.21.15.167:0/2743376532 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f35a00762d0 0x7f35a0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:30.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.743+0000 7f35c4c00640 1 --2- 172.21.15.167:0/2743376532 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f35a00762d0 0x7f35a0078790 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f35b4003eb0 tx=0x7f35b4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:30.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.747+0000 7f35bcc00640 1 -- 172.21.15.167:0/2743376532 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f35c01f7680 con 0x7f35c015fba0 2024-05-11T12:14:30.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:30.958+0000 7f35c6000640 1 -- 172.21.15.167:0/2743376532 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f35c006fae0 con 0x7f35c015fba0 2024-05-11T12:14:31.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.561+0000 7f35bcc00640 1 -- 172.21.15.167:0/2743376532 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v94) ==== 138+0+0 (secure 0 0 0) 0x7f35b80604e0 con 0x7f35c015fba0 2024-05-11T12:14:31.560 INFO:teuthology.orchestra.run.smithi167.stderr:pool 'cephfs_metadata' created 2024-05-11T12:14:31.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.565+0000 7f35c6000640 1 -- 172.21.15.167:0/2743376532 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f35a00762d0 msgr2=0x7f35a0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:31.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.565+0000 7f35c6000640 1 --2- 172.21.15.167:0/2743376532 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f35a00762d0 0x7f35a0078790 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f35b4003eb0 tx=0x7f35b4009040 comp rx=0 tx=0).stop 2024-05-11T12:14:31.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.565+0000 7f35c6000640 1 -- 172.21.15.167:0/2743376532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35c015fba0 msgr2=0x7f35c0158e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:31.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.565+0000 7f35c6000640 1 --2- 172.21.15.167:0/2743376532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35c015fba0 0x7f35c0158e20 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f35b800c7a0 tx=0x7f35b800cc60 comp rx=0 tx=0).stop 2024-05-11T12:14:31.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.565+0000 7f35c6000640 1 -- 172.21.15.167:0/2743376532 shutdown_connections 2024-05-11T12:14:31.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.565+0000 7f35c6000640 1 --2- 172.21.15.167:0/2743376532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35c015fba0 0x7f35c0158e20 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:31.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.565+0000 7f35c6000640 1 --2- 172.21.15.167:0/2743376532 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35c015cb90 0x7f35c01f7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:31.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.565+0000 7f35c6000640 1 --2- 172.21.15.167:0/2743376532 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f35a00762d0 0x7f35a0078790 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:31.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.565+0000 7f35c6000640 1 --2- 172.21.15.167:0/2743376532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35c006d9a0 0x7f35c0155b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:31.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.565+0000 7f35c6000640 1 -- 172.21.15.167:0/2743376532 >> 172.21.15.167:0/2743376532 conn(0x7f35c00b32a0 msgr2=0x7f35c0072680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:31.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.565+0000 7f35c6000640 1 -- 172.21.15.167:0/2743376532 shutdown_connections 2024-05-11T12:14:31.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.565+0000 7f35c6000640 1 -- 172.21.15.167:0/2743376532 wait complete. 2024-05-11T12:14:31.594 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-11T12:14:31.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.792+0000 7fbf35e00640 1 Processor -- start 2024-05-11T12:14:31.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.792+0000 7fbf35e00640 1 -- start start 2024-05-11T12:14:31.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.793+0000 7fbf35e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf30072ff0 0x7fbf300b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:31.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.793+0000 7fbf35e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbf3006d460 0x7fbf3006d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:31.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.793+0000 7fbf35e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf300733d0 con 0x7fbf3015f850 2024-05-11T12:14:31.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.793+0000 7fbf35e00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fbf3006e5b0 con 0x7fbf3006d460 2024-05-11T12:14:31.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.793+0000 7fbf35e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbf30160110 con 0x7fbf30072ff0 2024-05-11T12:14:31.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.794+0000 7fbf35400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbf3006d460 0x7fbf3006d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:31.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.794+0000 7fbf35400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbf3006d460 0x7fbf3006d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:40322/0 (socket says 172.21.15.167:40322) 2024-05-11T12:14:31.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.794+0000 7fbf35400640 1 -- 172.21.15.167:0/3696872262 learned_addr learned my addr 172.21.15.167:0/3696872262 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:31.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.794+0000 7fbf34a00640 1 --2- 172.21.15.167:0/3696872262 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf30072ff0 0x7fbf300b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:31.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.795+0000 7fbf35400640 1 -- 172.21.15.167:0/3696872262 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf30072ff0 msgr2=0x7fbf300b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:31.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.795+0000 7fbf35400640 1 --2- 172.21.15.167:0/3696872262 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf30072ff0 0x7fbf300b5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:31.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.795+0000 7fbf35400640 1 -- 172.21.15.167:0/3696872262 >> v1:172.21.15.204:6789/0 conn(0x7fbf3015f850 legacy=0x7fbf3006ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:31.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.795+0000 7fbf35400640 1 -- 172.21.15.167:0/3696872262 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf301603f0 con 0x7fbf3006d460 2024-05-11T12:14:31.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.796+0000 7fbf35400640 1 --2- 172.21.15.167:0/3696872262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf3006d460 0x7fbf3006d8f0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fbf28009e50 tx=0x7fbf2802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebb6854488f85738 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:31.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.796+0000 7fbf2f400640 1 -- 172.21.15.167:0/3696872262 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf28036070 con 0x7fbf3006d460 2024-05-11T12:14:31.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.796+0000 7fbf2f400640 1 -- 172.21.15.167:0/3696872262 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf28031600 con 0x7fbf3006d460 2024-05-11T12:14:31.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.796+0000 7fbf2f400640 1 -- 172.21.15.167:0/3696872262 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf280319d0 con 0x7fbf3006d460 2024-05-11T12:14:31.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.797+0000 7fbf35e00640 1 -- 172.21.15.167:0/3696872262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf3006d460 msgr2=0x7fbf3006d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:31.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.797+0000 7fbf35e00640 1 --2- 172.21.15.167:0/3696872262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf3006d460 0x7fbf3006d8f0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fbf28009e50 tx=0x7fbf2802a680 comp rx=0 tx=0).stop 2024-05-11T12:14:31.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.797+0000 7fbf35e00640 1 -- 172.21.15.167:0/3696872262 shutdown_connections 2024-05-11T12:14:31.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.797+0000 7fbf35e00640 1 --2- 172.21.15.167:0/3696872262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf3006d460 0x7fbf3006d8f0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:31.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.797+0000 7fbf35e00640 1 --2- 172.21.15.167:0/3696872262 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf30072ff0 0x7fbf300b5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:31.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.797+0000 7fbf35e00640 1 -- 172.21.15.167:0/3696872262 >> 172.21.15.167:0/3696872262 conn(0x7fbf300b32a0 msgr2=0x7fbf300d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:31.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.797+0000 7fbf35e00640 1 -- 172.21.15.167:0/3696872262 shutdown_connections 2024-05-11T12:14:31.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.797+0000 7fbf35e00640 1 -- 172.21.15.167:0/3696872262 wait complete. 2024-05-11T12:14:31.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.799+0000 7fbf35e00640 1 Processor -- start 2024-05-11T12:14:31.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.799+0000 7fbf35e00640 1 -- start start 2024-05-11T12:14:31.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.800+0000 7fbf35e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf3006d460 0x7fbf300ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:31.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.800+0000 7fbf35e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf30072ff0 0x7fbf30158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:31.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.800+0000 7fbf35e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf3015f850 0x7fbf3015dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:31.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.800+0000 7fbf35e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3015fcb0 con 0x7fbf30072ff0 2024-05-11T12:14:31.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.800+0000 7fbf35e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbf301606d0 con 0x7fbf3015f850 2024-05-11T12:14:31.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.800+0000 7fbf35e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbf3015fe30 con 0x7fbf3006d460 2024-05-11T12:14:31.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.800+0000 7fbf2fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf30072ff0 0x7fbf30158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:31.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.800+0000 7fbf2fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf30072ff0 0x7fbf30158ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:40332/0 (socket says 172.21.15.167:40332) 2024-05-11T12:14:31.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.800+0000 7fbf2fe00640 1 -- 172.21.15.167:0/1331375449 learned_addr learned my addr 172.21.15.167:0/1331375449 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:31.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.800+0000 7fbf34a00640 1 --2- 172.21.15.167:0/1331375449 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf3006d460 0x7fbf300ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:31.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.800+0000 7fbf35400640 1 --2- 172.21.15.167:0/1331375449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf3015f850 0x7fbf3015dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:31.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.800+0000 7fbf2fe00640 1 -- 172.21.15.167:0/1331375449 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf3006d460 msgr2=0x7fbf300ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:31.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.800+0000 7fbf2fe00640 1 --2- 172.21.15.167:0/1331375449 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf3006d460 0x7fbf300ceeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:31.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.800+0000 7fbf2fe00640 1 -- 172.21.15.167:0/1331375449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf3015f850 msgr2=0x7fbf3015dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:31.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.800+0000 7fbf2fe00640 1 --2- 172.21.15.167:0/1331375449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf3015f850 0x7fbf3015dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:31.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.800+0000 7fbf2fe00640 1 -- 172.21.15.167:0/1331375449 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf3015e770 con 0x7fbf30072ff0 2024-05-11T12:14:31.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.800+0000 7fbf35400640 1 --2- 172.21.15.167:0/1331375449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf3015f850 0x7fbf3015dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:31.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.801+0000 7fbf2fe00640 1 --2- 172.21.15.167:0/1331375449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf30072ff0 0x7fbf30158ac0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c00b3f0 tx=0x7fbf1c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:31.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.801+0000 7fbf2cc00640 1 -- 172.21.15.167:0/1331375449 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c015020 con 0x7fbf30072ff0 2024-05-11T12:14:31.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.801+0000 7fbf2cc00640 1 -- 172.21.15.167:0/1331375449 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf1c011830 con 0x7fbf30072ff0 2024-05-11T12:14:31.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.801+0000 7fbf2cc00640 1 -- 172.21.15.167:0/1331375449 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c011c20 con 0x7fbf30072ff0 2024-05-11T12:14:31.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.801+0000 7fbf35e00640 1 -- 172.21.15.167:0/1331375449 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf3015ea50 con 0x7fbf30072ff0 2024-05-11T12:14:31.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.801+0000 7fbf35e00640 1 -- 172.21.15.167:0/1331375449 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf300ce990 con 0x7fbf30072ff0 2024-05-11T12:14:31.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.803+0000 7fbf35e00640 1 -- 172.21.15.167:0/1331375449 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf300694b0 con 0x7fbf30072ff0 2024-05-11T12:14:31.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.804+0000 7fbf2cc00640 1 -- 172.21.15.167:0/1331375449 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbf1c018020 con 0x7fbf30072ff0 2024-05-11T12:14:31.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.805+0000 7fbf2cc00640 1 --2- 172.21.15.167:0/1331375449 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbf100762d0 0x7fbf10078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:31.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.805+0000 7fbf2cc00640 1 -- 172.21.15.167:0/1331375449 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5619+0+0 (secure 0 0 0) 0x7fbf1c0975e0 con 0x7fbf30072ff0 2024-05-11T12:14:31.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.805+0000 7fbf34a00640 1 --2- 172.21.15.167:0/1331375449 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbf100762d0 0x7fbf10078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:31.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.806+0000 7fbf34a00640 1 --2- 172.21.15.167:0/1331375449 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbf100762d0 0x7fbf10078790 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbf24006fd0 tx=0x7fbf24008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:31.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:31.810+0000 7fbf2cc00640 1 -- 172.21.15.167:0/1331375449 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fbf1c01f020 con 0x7fbf30072ff0 2024-05-11T12:14:32.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.020+0000 7fbf35e00640 1 -- 172.21.15.167:0/1331375449 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fbf300ceb60 con 0x7fbf30072ff0 2024-05-11T12:14:32.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.577+0000 7fbf2cc00640 1 -- 172.21.15.167:0/1331375449 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v95) ==== 170+0+0 (secure 0 0 0) 0x7fbf1c060f00 con 0x7fbf30072ff0 2024-05-11T12:14:32.577 INFO:teuthology.orchestra.run.smithi167.stderr:pool 'cephfs_data' created 2024-05-11T12:14:32.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.582+0000 7fbf35e00640 1 -- 172.21.15.167:0/1331375449 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbf100762d0 msgr2=0x7fbf10078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:32.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.582+0000 7fbf35e00640 1 --2- 172.21.15.167:0/1331375449 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbf100762d0 0x7fbf10078790 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbf24006fd0 tx=0x7fbf24008040 comp rx=0 tx=0).stop 2024-05-11T12:14:32.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.582+0000 7fbf35e00640 1 -- 172.21.15.167:0/1331375449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf30072ff0 msgr2=0x7fbf30158ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:32.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.582+0000 7fbf35e00640 1 --2- 172.21.15.167:0/1331375449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf30072ff0 0x7fbf30158ac0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c00b3f0 tx=0x7fbf1c00b8b0 comp rx=0 tx=0).stop 2024-05-11T12:14:32.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.582+0000 7fbf35e00640 1 -- 172.21.15.167:0/1331375449 shutdown_connections 2024-05-11T12:14:32.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.582+0000 7fbf35e00640 1 --2- 172.21.15.167:0/1331375449 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbf100762d0 0x7fbf10078790 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:32.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.582+0000 7fbf35e00640 1 --2- 172.21.15.167:0/1331375449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf3015f850 0x7fbf3015dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:32.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.582+0000 7fbf35e00640 1 --2- 172.21.15.167:0/1331375449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf30072ff0 0x7fbf30158ac0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:32.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.582+0000 7fbf35e00640 1 --2- 172.21.15.167:0/1331375449 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf3006d460 0x7fbf300ceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:32.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.582+0000 7fbf35e00640 1 -- 172.21.15.167:0/1331375449 >> 172.21.15.167:0/1331375449 conn(0x7fbf300b32a0 msgr2=0x7fbf300af0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:32.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.583+0000 7fbf35e00640 1 -- 172.21.15.167:0/1331375449 shutdown_connections 2024-05-11T12:14:32.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.583+0000 7fbf35e00640 1 -- 172.21.15.167:0/1331375449 wait complete. 2024-05-11T12:14:32.617 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-11T12:14:32.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.793+0000 7f49b5800640 1 Processor -- start 2024-05-11T12:14:32.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.793+0000 7f49b5800640 1 -- start start 2024-05-11T12:14:32.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.795+0000 7f49b5800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f49b0147ff0 0x7f49b00d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:32.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.795+0000 7f49b5800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49b00d2e80 0x7f49b00d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:32.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.795+0000 7f49b5800640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49b00d37c0 con 0x7f49b0148cf0 2024-05-11T12:14:32.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.795+0000 7f49b5800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f49b00b64f0 con 0x7f49b0147ff0 2024-05-11T12:14:32.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.795+0000 7f49b5800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f49b00d2040 con 0x7f49b00d2e80 2024-05-11T12:14:32.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.795+0000 7f49ae000640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f49b0148cf0 0x7f49b00b5de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:55594/0 (socket says 172.21.15.167:55594) 2024-05-11T12:14:32.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.795+0000 7f49ae000640 1 -- 172.21.15.167:0/1132515676 learned_addr learned my addr 172.21.15.167:0/1132515676 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:14:32.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.795+0000 7f49af400640 1 --2- 172.21.15.167:0/1132515676 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49b00d2e80 0x7f49b00d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:32.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.795+0000 7f49aea00640 1 --2- 172.21.15.167:0/1132515676 >> v2:172.21.15.204:3300/0 conn(0x7f49b0147ff0 0x7f49b00d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:32.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.796+0000 7f49ad600640 1 -- 172.21.15.167:0/1132515676 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2760591015 0 0) 0x7f49b00d37c0 con 0x7f49b0148cf0 2024-05-11T12:14:32.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.796+0000 7f49ad600640 1 -- 172.21.15.167:0/1132515676 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4994003520 con 0x7f49b0148cf0 2024-05-11T12:14:32.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.796+0000 7f49ad600640 1 -- 172.21.15.167:0/1132515676 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3233764800 0 0) 0x7f4994003520 con 0x7f49b0148cf0 2024-05-11T12:14:32.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.796+0000 7f49ad600640 1 -- 172.21.15.167:0/1132515676 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49b00d2e80 msgr2=0x7f49b00d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:32.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.796+0000 7f49ad600640 1 --2- 172.21.15.167:0/1132515676 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49b00d2e80 0x7f49b00d3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:32.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.796+0000 7f49ad600640 1 -- 172.21.15.167:0/1132515676 >> v2:172.21.15.204:3300/0 conn(0x7f49b0147ff0 msgr2=0x7f49b00d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:32.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.796+0000 7f49ad600640 1 --2- 172.21.15.167:0/1132515676 >> v2:172.21.15.204:3300/0 conn(0x7f49b0147ff0 0x7f49b00d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:32.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.796+0000 7f49ad600640 1 -- 172.21.15.167:0/1132515676 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49b00d1430 con 0x7f49b0148cf0 2024-05-11T12:14:32.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.796+0000 7f49ad600640 1 -- 172.21.15.167:0/1132515676 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f499c0016f0 con 0x7f49b0148cf0 2024-05-11T12:14:32.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.796+0000 7f49ad600640 1 -- 172.21.15.167:0/1132515676 >> v1:172.21.15.167:6789/0 conn(0x7f49b0148cf0 legacy=0x7f49b00b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:32.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.796+0000 7f49ad600640 1 --2- 172.21.15.167:0/1132515676 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49b00d2e80 0x7f4994003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:32.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.796+0000 7f49ad600640 1 --2- 172.21.15.167:0/1132515676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4994003fd0 0x7f4994004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:32.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.796+0000 7f49ad600640 1 --2- 172.21.15.167:0/1132515676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4994004d40 0x7f49940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:32.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.796+0000 7f49ad600640 1 -- 172.21.15.167:0/1132515676 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f49b00d2040 con 0x7f4994004d40 2024-05-11T12:14:32.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.796+0000 7f49ad600640 1 -- 172.21.15.167:0/1132515676 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f499400b9c0 con 0x7f4994003fd0 2024-05-11T12:14:32.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.796+0000 7f49ad600640 1 -- 172.21.15.167:0/1132515676 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f499400bb90 con 0x7f49b00d2e80 2024-05-11T12:14:32.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.797+0000 7f49af400640 1 --2- 172.21.15.167:0/1132515676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4994004d40 0x7f49940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:32.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.797+0000 7f49aea00640 1 --2- 172.21.15.167:0/1132515676 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49b00d2e80 0x7f4994003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:32.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.797+0000 7f49ae000640 1 --2- 172.21.15.167:0/1132515676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4994003fd0 0x7f4994004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:32.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.797+0000 7f49af400640 1 -- 172.21.15.167:0/1132515676 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49b00d2e80 msgr2=0x7f4994003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:32.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.797+0000 7f49af400640 1 --2- 172.21.15.167:0/1132515676 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49b00d2e80 0x7f4994003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:32.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.797+0000 7f49af400640 1 -- 172.21.15.167:0/1132515676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4994003fd0 msgr2=0x7f4994004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:32.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.797+0000 7f49af400640 1 --2- 172.21.15.167:0/1132515676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4994003fd0 0x7f4994004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:32.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.797+0000 7f49af400640 1 -- 172.21.15.167:0/1132515676 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4994003520 con 0x7f4994004d40 2024-05-11T12:14:32.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.797+0000 7f49ae000640 1 --2- 172.21.15.167:0/1132515676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4994003fd0 0x7f4994004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:32.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.798+0000 7f49af400640 1 --2- 172.21.15.167:0/1132515676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4994004d40 0x7f49940051c0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f49a8009af0 tx=0x7f4994005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5e4935f6b7f2f76 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:32.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.798+0000 7f49ad600640 1 -- 172.21.15.167:0/1132515676 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a8036070 con 0x7f4994004d40 2024-05-11T12:14:32.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.798+0000 7f49ad600640 1 -- 172.21.15.167:0/1132515676 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49a8031400 con 0x7f4994004d40 2024-05-11T12:14:32.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.798+0000 7f49ad600640 1 -- 172.21.15.167:0/1132515676 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a8009690 con 0x7f4994004d40 2024-05-11T12:14:32.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.799+0000 7f49b5800640 1 -- 172.21.15.167:0/1132515676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4994004d40 msgr2=0x7f49940051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:32.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.799+0000 7f49b5800640 1 --2- 172.21.15.167:0/1132515676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4994004d40 0x7f49940051c0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f49a8009af0 tx=0x7f4994005830 comp rx=0 tx=0).stop 2024-05-11T12:14:32.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.799+0000 7f49af400640 1 -- 172.21.15.167:0/1132515676 reap_dead start 2024-05-11T12:14:32.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.799+0000 7f49b5800640 1 -- 172.21.15.167:0/1132515676 shutdown_connections 2024-05-11T12:14:32.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.799+0000 7f49b5800640 1 -- 172.21.15.167:0/1132515676 >> 172.21.15.167:0/1132515676 conn(0x7f49b00b8300 msgr2=0x7f49b00b86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:32.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.799+0000 7f49b5800640 1 -- 172.21.15.167:0/1132515676 shutdown_connections 2024-05-11T12:14:32.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.799+0000 7f49b5800640 1 -- 172.21.15.167:0/1132515676 wait complete. 2024-05-11T12:14:32.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.801+0000 7f49b5800640 1 Processor -- start 2024-05-11T12:14:32.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.801+0000 7f49b5800640 1 -- start start 2024-05-11T12:14:32.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.802+0000 7f49b5800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4994004d40 0x7f49b0139d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:32.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.802+0000 7f49b5800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49b013bae0 0x7f49b01f4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:32.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.802+0000 7f49b5800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49b00ccce0 0x7f49b00cd160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:32.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.802+0000 7f49b5800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f49b00d1cd0 con 0x7f49b00ccce0 2024-05-11T12:14:32.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.802+0000 7f49b5800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f49b00d19f0 con 0x7f49b013bae0 2024-05-11T12:14:32.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.802+0000 7f49b5800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f49b01483d0 con 0x7f4994004d40 2024-05-11T12:14:32.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.802+0000 7f49af400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49b00ccce0 0x7f49b00cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:32.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.802+0000 7f49af400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49b00ccce0 0x7f49b00cd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:40348/0 (socket says 172.21.15.167:40348) 2024-05-11T12:14:32.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.802+0000 7f49af400640 1 -- 172.21.15.167:0/1977458835 learned_addr learned my addr 172.21.15.167:0/1977458835 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:32.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.803+0000 7f49af400640 1 -- 172.21.15.167:0/1977458835 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4994004d40 msgr2=0x7f49b0139d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:32.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.803+0000 7f49af400640 1 --2- 172.21.15.167:0/1977458835 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4994004d40 0x7f49b0139d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:32.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.803+0000 7f49af400640 1 -- 172.21.15.167:0/1977458835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49b013bae0 msgr2=0x7f49b01f4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:32.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.803+0000 7f49af400640 1 --2- 172.21.15.167:0/1977458835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49b013bae0 0x7f49b01f4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:32.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.803+0000 7f49af400640 1 -- 172.21.15.167:0/1977458835 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49b00cdb20 con 0x7f49b00ccce0 2024-05-11T12:14:32.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.803+0000 7f49af400640 1 --2- 172.21.15.167:0/1977458835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49b00ccce0 0x7f49b00cd160 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f49a8009c10 tx=0x7f49a8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:32.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.803+0000 7f49a6a00640 1 -- 172.21.15.167:0/1977458835 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a8036040 con 0x7f49b00ccce0 2024-05-11T12:14:32.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.803+0000 7f49a6a00640 1 -- 172.21.15.167:0/1977458835 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49a800f000 con 0x7f49b00ccce0 2024-05-11T12:14:32.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.803+0000 7f49a6a00640 1 -- 172.21.15.167:0/1977458835 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a8009690 con 0x7f49b00ccce0 2024-05-11T12:14:32.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.803+0000 7f49b5800640 1 -- 172.21.15.167:0/1977458835 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49b013a400 con 0x7f49b00ccce0 2024-05-11T12:14:32.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.803+0000 7f49b5800640 1 -- 172.21.15.167:0/1977458835 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49b013a9c0 con 0x7f49b00ccce0 2024-05-11T12:14:32.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.805+0000 7f49b5800640 1 -- 172.21.15.167:0/1977458835 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49b0069520 con 0x7f49b00ccce0 2024-05-11T12:14:32.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.806+0000 7f49a6a00640 1 -- 172.21.15.167:0/1977458835 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f49a8043020 con 0x7f49b00ccce0 2024-05-11T12:14:32.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.807+0000 7f49a6a00640 1 --2- 172.21.15.167:0/1977458835 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4990076300 0x7f49900787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:32.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.807+0000 7f49a6a00640 1 -- 172.21.15.167:0/1977458835 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6001+0+0 (secure 0 0 0) 0x7f49a80b6f80 con 0x7f49b00ccce0 2024-05-11T12:14:32.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.807+0000 7f49aea00640 1 --2- 172.21.15.167:0/1977458835 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4990076300 0x7f49900787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:32.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.808+0000 7f49aea00640 1 --2- 172.21.15.167:0/1977458835 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4990076300 0x7f49900787c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f49a0009fd0 tx=0x7f49a0009750 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:32.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:32.813+0000 7f49a6a00640 1 -- 172.21.15.167:0/1977458835 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f49a8080710 con 0x7f49b00ccce0 2024-05-11T12:14:33.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.066+0000 7f49b5800640 1 -- 172.21.15.167:0/1977458835 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f49b00b63c0 con 0x7f49b00ccce0 2024-05-11T12:14:33.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.604+0000 7f49a6a00640 1 -- 172.21.15.167:0/1977458835 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v58) ==== 174+0+0 (secure 0 0 0) 0x7f49a8080010 con 0x7f49b00ccce0 2024-05-11T12:14:33.610 INFO:teuthology.orchestra.run.smithi167.stderr:new fs with metadata pool 15 and data pool 16 2024-05-11T12:14:33.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.611+0000 7f49b5800640 1 -- 172.21.15.167:0/1977458835 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4990076300 msgr2=0x7f49900787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:33.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.611+0000 7f49b5800640 1 --2- 172.21.15.167:0/1977458835 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4990076300 0x7f49900787c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f49a0009fd0 tx=0x7f49a0009750 comp rx=0 tx=0).stop 2024-05-11T12:14:33.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.611+0000 7f49b5800640 1 -- 172.21.15.167:0/1977458835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49b00ccce0 msgr2=0x7f49b00cd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:33.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.611+0000 7f49b5800640 1 --2- 172.21.15.167:0/1977458835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49b00ccce0 0x7f49b00cd160 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f49a8009c10 tx=0x7f49a8004730 comp rx=0 tx=0).stop 2024-05-11T12:14:33.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.611+0000 7f49b5800640 1 -- 172.21.15.167:0/1977458835 shutdown_connections 2024-05-11T12:14:33.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.611+0000 7f49b5800640 1 --2- 172.21.15.167:0/1977458835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49b00ccce0 0x7f49b00cd160 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:33.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.611+0000 7f49b5800640 1 --2- 172.21.15.167:0/1977458835 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4990076300 0x7f49900787c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:33.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.611+0000 7f49b5800640 1 --2- 172.21.15.167:0/1977458835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49b013bae0 0x7f49b01f4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:33.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.611+0000 7f49b5800640 1 --2- 172.21.15.167:0/1977458835 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4994004d40 0x7f49b0139d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:33.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.611+0000 7f49b5800640 1 -- 172.21.15.167:0/1977458835 >> 172.21.15.167:0/1977458835 conn(0x7f49b00b8300 msgr2=0x7f49b00b5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:33.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.614+0000 7f49b5800640 1 -- 172.21.15.167:0/1977458835 shutdown_connections 2024-05-11T12:14:33.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.616+0000 7f49b5800640 1 -- 172.21.15.167:0/1977458835 wait complete. 2024-05-11T12:14:33.654 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:14:33.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.826+0000 7f311e000640 1 Processor -- start 2024-05-11T12:14:33.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.826+0000 7f311e000640 1 -- start start 2024-05-11T12:14:33.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.827+0000 7f311e000640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f31180b3ea0 0x7f3118161490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:33.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.827+0000 7f311e000640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f311815df30 con 0x7f31181619d0 2024-05-11T12:14:33.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.828+0000 7f311e000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f311815e210 con 0x7f3118157750 2024-05-11T12:14:33.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.828+0000 7f311e000640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f3118161000 con 0x7f31180b3ea0 2024-05-11T12:14:33.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.828+0000 7f310ea00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f31180b3ea0 0x7f3118161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:33.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.828+0000 7f310ea00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f31180b3ea0 0x7f3118161490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:40362/0 (socket says 172.21.15.167:40362) 2024-05-11T12:14:33.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.828+0000 7f310ea00640 1 -- 172.21.15.167:0/1623383276 learned_addr learned my addr 172.21.15.167:0/1623383276 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:33.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.828+0000 7f3117e00640 1 --1- 172.21.15.167:0/1623383276 >> v1:172.21.15.167:6789/0 conn(0x7f31181619d0 0x7f311806d460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:55608/0 (socket says 172.21.15.167:55608) 2024-05-11T12:14:33.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.828+0000 7f310ea00640 1 -- 172.21.15.167:0/1623383276 >> v1:172.21.15.204:6789/0 conn(0x7f3118157750 legacy=0x7f311815d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:33.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.828+0000 7f310ea00640 1 -- 172.21.15.167:0/1623383276 >> v1:172.21.15.167:6789/0 conn(0x7f31181619d0 legacy=0x7f311806d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:33.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.828+0000 7f310ea00640 1 -- 172.21.15.167:0/1623383276 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f311815e4f0 con 0x7f31180b3ea0 2024-05-11T12:14:33.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.829+0000 7f310ea00640 1 --2- 172.21.15.167:0/1623383276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31180b3ea0 0x7f3118161490 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f3100009870 tx=0x7f310002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e8863dd3e9919 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:33.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.830+0000 7f3116a00640 1 -- 172.21.15.167:0/1623383276 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3100037070 con 0x7f31180b3ea0 2024-05-11T12:14:33.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.830+0000 7f3116a00640 1 -- 172.21.15.167:0/1623383276 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3100032440 con 0x7f31180b3ea0 2024-05-11T12:14:33.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.830+0000 7f3116a00640 1 -- 172.21.15.167:0/1623383276 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3100032810 con 0x7f31180b3ea0 2024-05-11T12:14:33.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.830+0000 7f311e000640 1 -- 172.21.15.167:0/1623383276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31180b3ea0 msgr2=0x7f3118161490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:33.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.830+0000 7f311e000640 1 --2- 172.21.15.167:0/1623383276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31180b3ea0 0x7f3118161490 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f3100009870 tx=0x7f310002a250 comp rx=0 tx=0).stop 2024-05-11T12:14:33.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.830+0000 7f311e000640 1 -- 172.21.15.167:0/1623383276 shutdown_connections 2024-05-11T12:14:33.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.830+0000 7f311e000640 1 --2- 172.21.15.167:0/1623383276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31180b3ea0 0x7f3118161490 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:33.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.830+0000 7f311e000640 1 -- 172.21.15.167:0/1623383276 >> 172.21.15.167:0/1623383276 conn(0x7f31180cd8f0 msgr2=0x7f31180cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:33.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.830+0000 7f311e000640 1 -- 172.21.15.167:0/1623383276 shutdown_connections 2024-05-11T12:14:33.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.830+0000 7f311e000640 1 -- 172.21.15.167:0/1623383276 wait complete. 2024-05-11T12:14:33.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.832+0000 7f311e000640 1 Processor -- start 2024-05-11T12:14:33.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.832+0000 7f311e000640 1 -- start start 2024-05-11T12:14:33.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.833+0000 7f311e000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31180b3ea0 0x7f3118143ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:33.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.833+0000 7f311e000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3118157750 0x7f3118141790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:33.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.833+0000 7f311e000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31181619d0 0x7f3118141cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:33.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.833+0000 7f311e000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f311806dcf0 con 0x7f3118157750 2024-05-11T12:14:33.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.833+0000 7f311e000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f311806db70 con 0x7f31180b3ea0 2024-05-11T12:14:33.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.833+0000 7f311e000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f31180cf1c0 con 0x7f31181619d0 2024-05-11T12:14:33.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.833+0000 7f310ea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3118157750 0x7f3118141790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:33.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.833+0000 7f310ea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3118157750 0x7f3118141790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:40364/0 (socket says 172.21.15.167:40364) 2024-05-11T12:14:33.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.834+0000 7f310ea00640 1 -- 172.21.15.167:0/2139645358 learned_addr learned my addr 172.21.15.167:0/2139645358 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:33.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.834+0000 7f3117400640 1 --2- 172.21.15.167:0/2139645358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31180b3ea0 0x7f3118143ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:33.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.834+0000 7f310ea00640 1 -- 172.21.15.167:0/2139645358 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31181619d0 msgr2=0x7f3118141cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:33.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.834+0000 7f310ea00640 1 --2- 172.21.15.167:0/2139645358 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31181619d0 0x7f3118141cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:33.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.834+0000 7f310ea00640 1 -- 172.21.15.167:0/2139645358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31180b3ea0 msgr2=0x7f3118143ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:33.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.834+0000 7f310ea00640 1 --2- 172.21.15.167:0/2139645358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31180b3ea0 0x7f3118143ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:33.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.834+0000 7f310ea00640 1 -- 172.21.15.167:0/2139645358 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f311806d3e0 con 0x7f3118157750 2024-05-11T12:14:33.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.834+0000 7f310ea00640 1 --2- 172.21.15.167:0/2139645358 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3118157750 0x7f3118141790 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f3100009870 tx=0x7f31000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:33.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.835+0000 7f310fe00640 1 -- 172.21.15.167:0/2139645358 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3100041070 con 0x7f3118157750 2024-05-11T12:14:33.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.835+0000 7f310fe00640 1 -- 172.21.15.167:0/2139645358 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f310003c870 con 0x7f3118157750 2024-05-11T12:14:33.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.835+0000 7f310fe00640 1 -- 172.21.15.167:0/2139645358 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3100037040 con 0x7f3118157750 2024-05-11T12:14:33.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.835+0000 7f311e000640 1 -- 172.21.15.167:0/2139645358 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f311806d6c0 con 0x7f3118157750 2024-05-11T12:14:33.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.835+0000 7f311e000640 1 -- 172.21.15.167:0/2139645358 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31181f2c50 con 0x7f3118157750 2024-05-11T12:14:33.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.836+0000 7f311e000640 1 -- 172.21.15.167:0/2139645358 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3118069590 con 0x7f3118157750 2024-05-11T12:14:33.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.837+0000 7f310fe00640 1 -- 172.21.15.167:0/2139645358 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3100044070 con 0x7f3118157750 2024-05-11T12:14:33.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.838+0000 7f310fe00640 1 --2- 172.21.15.167:0/2139645358 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f30f4076320 0x7f30f40787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:33.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.838+0000 7f310fe00640 1 -- 172.21.15.167:0/2139645358 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6101+0+0 (secure 0 0 0) 0x7f31000b82e0 con 0x7f3118157750 2024-05-11T12:14:33.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.838+0000 7f3117400640 1 --2- 172.21.15.167:0/2139645358 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f30f4076320 0x7f30f40787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:33.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.839+0000 7f3117400640 1 --2- 172.21.15.167:0/2139645358 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f30f4076320 0x7f30f40787e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3108007fd0 tx=0x7f3108009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:33.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:33.842+0000 7f310fe00640 1 -- 172.21.15.167:0/2139645358 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3100081b20 con 0x7f3118157750 2024-05-11T12:14:34.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.036+0000 7f311e000640 1 -- 172.21.15.167:0/2139645358 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f31180b48c0 con 0x7f3118157750 2024-05-11T12:14:34.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.039+0000 7f310fe00640 1 -- 172.21.15.167:0/2139645358 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+16934 (secure 0 0 0) 0x7f3100081420 con 0x7f3118157750 2024-05-11T12:14:34.039 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:34.039 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":96,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:14:33.067275+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:14:30.972447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-11T12:14:32.037637+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:14:34.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.041+0000 7f311e000640 1 -- 172.21.15.167:0/2139645358 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f30f4076320 msgr2=0x7f30f40787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:34.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.041+0000 7f311e000640 1 --2- 172.21.15.167:0/2139645358 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f30f4076320 0x7f30f40787e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3108007fd0 tx=0x7f3108009040 comp rx=0 tx=0).stop 2024-05-11T12:14:34.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.042+0000 7f311e000640 1 -- 172.21.15.167:0/2139645358 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3118157750 msgr2=0x7f3118141790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:34.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.042+0000 7f311e000640 1 --2- 172.21.15.167:0/2139645358 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3118157750 0x7f3118141790 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f3100009870 tx=0x7f31000046c0 comp rx=0 tx=0).stop 2024-05-11T12:14:34.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.042+0000 7f311e000640 1 -- 172.21.15.167:0/2139645358 shutdown_connections 2024-05-11T12:14:34.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.042+0000 7f311e000640 1 --2- 172.21.15.167:0/2139645358 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31181619d0 0x7f3118141cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:34.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.042+0000 7f311e000640 1 --2- 172.21.15.167:0/2139645358 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3118157750 0x7f3118141790 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:34.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.042+0000 7f311e000640 1 --2- 172.21.15.167:0/2139645358 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f30f4076320 0x7f30f40787e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:34.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.042+0000 7f311e000640 1 --2- 172.21.15.167:0/2139645358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31180b3ea0 0x7f3118143ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:34.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.042+0000 7f311e000640 1 -- 172.21.15.167:0/2139645358 >> 172.21.15.167:0/2139645358 conn(0x7f31180cd8f0 msgr2=0x7f31180cf5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:34.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.042+0000 7f311e000640 1 -- 172.21.15.167:0/2139645358 shutdown_connections 2024-05-11T12:14:34.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.042+0000 7f311e000640 1 -- 172.21.15.167:0/2139645358 wait complete. 2024-05-11T12:14:34.058 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:14:34.239 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.240+0000 7f3db0c00640 1 Processor -- start 2024-05-11T12:14:34.239 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.240+0000 7f3db0c00640 1 -- start start 2024-05-11T12:14:34.241 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.241+0000 7f3db0c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3dac1458f0 0x7f3dac06d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:34.241 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.241+0000 7f3db0c00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3dac0b5220 0x7f3dac0b5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:34.242 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.241+0000 7f3db0c00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dac071f30 con 0x7f3dac0b3a80 2024-05-11T12:14:34.242 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.242+0000 7f3db0c00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f3dac06db80 con 0x7f3dac0b5220 2024-05-11T12:14:34.242 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.242+0000 7f3db0c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f3dac0b5bd0 con 0x7f3dac1458f0 2024-05-11T12:14:34.242 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.242+0000 7f3dabe00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3dac0b5220 0x7f3dac0b5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:34.242 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.242+0000 7f3daaa00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f3dac0b3a80 0x7f3dac0b4970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:55612/0 (socket says 172.21.15.167:55612) 2024-05-11T12:14:34.242 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.242+0000 7f3daaa00640 1 -- 172.21.15.167:0/3488711333 learned_addr learned my addr 172.21.15.167:0/3488711333 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:14:34.242 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.242+0000 7f3dab400640 1 --2- 172.21.15.167:0/3488711333 >> v2:172.21.15.204:3300/0 conn(0x7f3dac1458f0 0x7f3dac06d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:34.242 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.242+0000 7f3dabe00640 1 -- 172.21.15.167:0/3488711333 >> v2:172.21.15.204:3300/0 conn(0x7f3dac1458f0 msgr2=0x7f3dac06d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:34.242 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.242+0000 7f3dabe00640 1 --2- 172.21.15.167:0/3488711333 >> v2:172.21.15.204:3300/0 conn(0x7f3dac1458f0 0x7f3dac06d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:34.242 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.242+0000 7f3dabe00640 1 -- 172.21.15.167:0/3488711333 >> v1:172.21.15.167:6789/0 conn(0x7f3dac0b3a80 legacy=0x7f3dac0b4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:34.242 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.242+0000 7f3dabe00640 1 -- 172.21.15.167:0/3488711333 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dac072210 con 0x7f3dac0b5220 2024-05-11T12:14:34.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.243+0000 7f3dabe00640 1 --2- 172.21.15.167:0/3488711333 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3dac0b5220 0x7f3dac0b5690 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f3dac0d0980 tx=0x7f3da402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f418860b9bb26e57 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:34.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.244+0000 7f3daa000640 1 -- 172.21.15.167:0/3488711333 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da4037070 con 0x7f3dac0b5220 2024-05-11T12:14:34.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.244+0000 7f3daa000640 1 -- 172.21.15.167:0/3488711333 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3da4032440 con 0x7f3dac0b5220 2024-05-11T12:14:34.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.244+0000 7f3daa000640 1 -- 172.21.15.167:0/3488711333 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da4032810 con 0x7f3dac0b5220 2024-05-11T12:14:34.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.244+0000 7f3db0c00640 1 -- 172.21.15.167:0/3488711333 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3dac0b5220 msgr2=0x7f3dac0b5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:34.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.244+0000 7f3db0c00640 1 --2- 172.21.15.167:0/3488711333 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3dac0b5220 0x7f3dac0b5690 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f3dac0d0980 tx=0x7f3da402a1a0 comp rx=0 tx=0).stop 2024-05-11T12:14:34.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.244+0000 7f3db0c00640 1 -- 172.21.15.167:0/3488711333 shutdown_connections 2024-05-11T12:14:34.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.244+0000 7f3db0c00640 1 --2- 172.21.15.167:0/3488711333 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3dac0b5220 0x7f3dac0b5690 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:34.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.244+0000 7f3db0c00640 1 --2- 172.21.15.167:0/3488711333 >> v2:172.21.15.204:3300/0 conn(0x7f3dac1458f0 0x7f3dac06d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:34.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.244+0000 7f3db0c00640 1 -- 172.21.15.167:0/3488711333 >> 172.21.15.167:0/3488711333 conn(0x7f3dac0b3460 msgr2=0x7f3dac0d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:34.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.245+0000 7f3db0c00640 1 -- 172.21.15.167:0/3488711333 shutdown_connections 2024-05-11T12:14:34.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.245+0000 7f3db0c00640 1 -- 172.21.15.167:0/3488711333 wait complete. 2024-05-11T12:14:34.245 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.246+0000 7f3db0c00640 1 Processor -- start 2024-05-11T12:14:34.245 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.246+0000 7f3db0c00640 1 -- start start 2024-05-11T12:14:34.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.247+0000 7f3db0c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3dac0b3a80 0x7f3dac13db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:34.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.247+0000 7f3db0c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3dac0b5220 0x7f3dac15e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:34.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.247+0000 7f3db0c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3dac1458f0 0x7f3dac15faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:34.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.247+0000 7f3db0c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3dac072ab0 con 0x7f3dac0b3a80 2024-05-11T12:14:34.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.247+0000 7f3db0c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3dac0727d0 con 0x7f3dac0b5220 2024-05-11T12:14:34.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.247+0000 7f3db0c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3dac072fd0 con 0x7f3dac1458f0 2024-05-11T12:14:34.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.247+0000 7f3dab400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3dac0b3a80 0x7f3dac13db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:34.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.247+0000 7f3dab400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3dac0b3a80 0x7f3dac13db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:40380/0 (socket says 172.21.15.167:40380) 2024-05-11T12:14:34.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.247+0000 7f3dab400640 1 -- 172.21.15.167:0/3184394043 learned_addr learned my addr 172.21.15.167:0/3184394043 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:34.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.248+0000 7f3dab400640 1 -- 172.21.15.167:0/3184394043 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3dac1458f0 msgr2=0x7f3dac15faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:34.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.248+0000 7f3dab400640 1 --2- 172.21.15.167:0/3184394043 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3dac1458f0 0x7f3dac15faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:34.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.248+0000 7f3dab400640 1 -- 172.21.15.167:0/3184394043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3dac0b5220 msgr2=0x7f3dac15e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:34.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.248+0000 7f3dab400640 1 --2- 172.21.15.167:0/3184394043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3dac0b5220 0x7f3dac15e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:34.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.248+0000 7f3dab400640 1 -- 172.21.15.167:0/3184394043 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dac15e830 con 0x7f3dac0b3a80 2024-05-11T12:14:34.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.248+0000 7f3dab400640 1 --2- 172.21.15.167:0/3184394043 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3dac0b3a80 0x7f3dac13db10 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f3d98007e70 tx=0x7f3d9800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:34.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.248+0000 7f3da3400640 1 -- 172.21.15.167:0/3184394043 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d98014620 con 0x7f3dac0b3a80 2024-05-11T12:14:34.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.248+0000 7f3da3400640 1 -- 172.21.15.167:0/3184394043 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d98019070 con 0x7f3dac0b3a80 2024-05-11T12:14:34.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.248+0000 7f3da3400640 1 -- 172.21.15.167:0/3184394043 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d98014a00 con 0x7f3dac0b3a80 2024-05-11T12:14:34.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.248+0000 7f3db0c00640 1 -- 172.21.15.167:0/3184394043 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dac15eb10 con 0x7f3dac0b3a80 2024-05-11T12:14:34.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.249+0000 7f3db0c00640 1 -- 172.21.15.167:0/3184394043 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dac15f0d0 con 0x7f3dac0b3a80 2024-05-11T12:14:34.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.250+0000 7f3db0c00640 1 -- 172.21.15.167:0/3184394043 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dac0694b0 con 0x7f3dac0b3a80 2024-05-11T12:14:34.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.251+0000 7f3da3400640 1 -- 172.21.15.167:0/3184394043 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3d98014ba0 con 0x7f3dac0b3a80 2024-05-11T12:14:34.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.252+0000 7f3da3400640 1 --2- 172.21.15.167:0/3184394043 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3d8c0762d0 0x7f3d8c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:34.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.252+0000 7f3da3400640 1 -- 172.21.15.167:0/3184394043 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6101+0+0 (secure 0 0 0) 0x7f3d98098070 con 0x7f3dac0b3a80 2024-05-11T12:14:34.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.252+0000 7f3daaa00640 1 --2- 172.21.15.167:0/3184394043 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3d8c0762d0 0x7f3d8c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:34.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.253+0000 7f3daaa00640 1 --2- 172.21.15.167:0/3184394043 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3d8c0762d0 0x7f3d8c078790 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3dac0b0d40 tx=0x7f3d9c007a40 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:34.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.256+0000 7f3da3400640 1 -- 172.21.15.167:0/3184394043 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3d98061840 con 0x7f3dac0b3a80 2024-05-11T12:14:34.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.446+0000 7f3db0c00640 1 -- 172.21.15.167:0/3184394043 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3dac15bda0 con 0x7f3dac0b3a80 2024-05-11T12:14:34.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.449+0000 7f3da3400640 1 -- 172.21.15.167:0/3184394043 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+16934 (secure 0 0 0) 0x7f3d98061140 con 0x7f3dac0b3a80 2024-05-11T12:14:34.449 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:34.450 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":96,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:14:33.067275+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:14:30.972447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-11T12:14:32.037637+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:14:34.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.452+0000 7f3db0c00640 1 -- 172.21.15.167:0/3184394043 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3d8c0762d0 msgr2=0x7f3d8c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:34.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.452+0000 7f3db0c00640 1 --2- 172.21.15.167:0/3184394043 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3d8c0762d0 0x7f3d8c078790 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3dac0b0d40 tx=0x7f3d9c007a40 comp rx=0 tx=0).stop 2024-05-11T12:14:34.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.452+0000 7f3db0c00640 1 -- 172.21.15.167:0/3184394043 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3dac0b3a80 msgr2=0x7f3dac13db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:34.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.452+0000 7f3db0c00640 1 --2- 172.21.15.167:0/3184394043 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3dac0b3a80 0x7f3dac13db10 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f3d98007e70 tx=0x7f3d9800e4b0 comp rx=0 tx=0).stop 2024-05-11T12:14:34.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.452+0000 7f3db0c00640 1 -- 172.21.15.167:0/3184394043 shutdown_connections 2024-05-11T12:14:34.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.452+0000 7f3db0c00640 1 --2- 172.21.15.167:0/3184394043 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3dac1458f0 0x7f3dac15faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:34.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.452+0000 7f3db0c00640 1 --2- 172.21.15.167:0/3184394043 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3d8c0762d0 0x7f3d8c078790 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:34.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.452+0000 7f3db0c00640 1 --2- 172.21.15.167:0/3184394043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3dac0b5220 0x7f3dac15e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:34.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.452+0000 7f3db0c00640 1 --2- 172.21.15.167:0/3184394043 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3dac0b3a80 0x7f3dac13db10 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:34.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.452+0000 7f3db0c00640 1 -- 172.21.15.167:0/3184394043 >> 172.21.15.167:0/3184394043 conn(0x7f3dac0b3460 msgr2=0x7f3dac0b47d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:34.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.452+0000 7f3db0c00640 1 -- 172.21.15.167:0/3184394043 shutdown_connections 2024-05-11T12:14:34.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.452+0000 7f3db0c00640 1 -- 172.21.15.167:0/3184394043 wait complete. 2024-05-11T12:14:34.469 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-11T12:14:34.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.671+0000 7f29dde00640 1 Processor -- start 2024-05-11T12:14:34.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.671+0000 7f29dde00640 1 -- start start 2024-05-11T12:14:34.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.673+0000 7f29dde00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f29d815f310 0x7f29d806e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:34.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.673+0000 7f29dde00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29d806e9a0 0x7f29d8073130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:34.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.673+0000 7f29dde00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29d80af9f0 con 0x7f29d815ef30 2024-05-11T12:14:34.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.673+0000 7f29dde00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f29d80b5100 con 0x7f29d815f310 2024-05-11T12:14:34.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.673+0000 7f29dde00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f29d80afcd0 con 0x7f29d806e9a0 2024-05-11T12:14:34.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.673+0000 7f29d7e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f29d815f310 0x7f29d806e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:34.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.673+0000 7f29dca00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f29d815ef30 0x7f29d80728d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:55616/0 (socket says 172.21.15.167:55616) 2024-05-11T12:14:34.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.673+0000 7f29dca00640 1 -- 172.21.15.167:0/3951133831 learned_addr learned my addr 172.21.15.167:0/3951133831 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:14:34.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.674+0000 7f29d7e00640 1 -- 172.21.15.167:0/3951133831 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29d806e9a0 msgr2=0x7f29d8073130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:34.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.674+0000 7f29dd400640 1 --2- 172.21.15.167:0/3951133831 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29d806e9a0 0x7f29d8073130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:34.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.674+0000 7f29d7e00640 1 --2- 172.21.15.167:0/3951133831 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29d806e9a0 0x7f29d8073130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:34.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.674+0000 7f29d7e00640 1 -- 172.21.15.167:0/3951133831 >> v1:172.21.15.167:6789/0 conn(0x7f29d815ef30 legacy=0x7f29d80728d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:34.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.674+0000 7f29d7e00640 1 -- 172.21.15.167:0/3951133831 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d80b4d90 con 0x7f29d815f310 2024-05-11T12:14:34.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.674+0000 7f29d7400640 1 -- 172.21.15.167:0/3951133831 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3986149741 0 0) 0x7f29d80af9f0 con 0x7f29d815ef30 2024-05-11T12:14:34.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.674+0000 7f29dd400640 1 --2- 172.21.15.167:0/3951133831 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29d806e9a0 0x7f29d8073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:34.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.675+0000 7f29d7e00640 1 --2- 172.21.15.167:0/3951133831 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29d815f310 0x7f29d806e460 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f29c40097a0 tx=0x7f29c402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa6c6f0f6030dbfd server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:34.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.675+0000 7f29d7400640 1 -- 172.21.15.167:0/3951133831 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c4037070 con 0x7f29d815f310 2024-05-11T12:14:34.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.675+0000 7f29d7400640 1 -- 172.21.15.167:0/3951133831 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29c402ae50 con 0x7f29d815f310 2024-05-11T12:14:34.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.675+0000 7f29d7400640 1 -- 172.21.15.167:0/3951133831 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c4032660 con 0x7f29d815f310 2024-05-11T12:14:34.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.676+0000 7f29dde00640 1 -- 172.21.15.167:0/3951133831 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29d815f310 msgr2=0x7f29d806e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:34.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.676+0000 7f29dde00640 1 --2- 172.21.15.167:0/3951133831 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29d815f310 0x7f29d806e460 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f29c40097a0 tx=0x7f29c402a100 comp rx=0 tx=0).stop 2024-05-11T12:14:34.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.676+0000 7f29dde00640 1 -- 172.21.15.167:0/3951133831 shutdown_connections 2024-05-11T12:14:34.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.676+0000 7f29dde00640 1 --2- 172.21.15.167:0/3951133831 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29d806e9a0 0x7f29d8073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:34.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.676+0000 7f29dde00640 1 --2- 172.21.15.167:0/3951133831 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29d815f310 0x7f29d806e460 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:34.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.676+0000 7f29dde00640 1 -- 172.21.15.167:0/3951133831 >> 172.21.15.167:0/3951133831 conn(0x7f29d80b2f10 msgr2=0x7f29d806c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:34.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.676+0000 7f29dde00640 1 -- 172.21.15.167:0/3951133831 shutdown_connections 2024-05-11T12:14:34.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.676+0000 7f29dde00640 1 -- 172.21.15.167:0/3951133831 wait complete. 2024-05-11T12:14:34.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.678+0000 7f29dde00640 1 Processor -- start 2024-05-11T12:14:34.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.678+0000 7f29dde00640 1 -- start start 2024-05-11T12:14:34.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.679+0000 7f29dde00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29d815ef30 0x7f29d8151760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:34.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.679+0000 7f29dde00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29d814ee50 0x7f29d80c6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:34.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.679+0000 7f29dde00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29d80c6aa0 0x7f29d815ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:34.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.679+0000 7f29dde00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f29d80af700 con 0x7f29d815ef30 2024-05-11T12:14:34.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.679+0000 7f29dde00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f29d80af580 con 0x7f29d80c6aa0 2024-05-11T12:14:34.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.679+0000 7f29dde00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f29d8073670 con 0x7f29d814ee50 2024-05-11T12:14:34.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.679+0000 7f29dca00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29d815ef30 0x7f29d8151760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:34.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.679+0000 7f29dca00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29d815ef30 0x7f29d8151760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:40404/0 (socket says 172.21.15.167:40404) 2024-05-11T12:14:34.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.679+0000 7f29dca00640 1 -- 172.21.15.167:0/732693722 learned_addr learned my addr 172.21.15.167:0/732693722 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:34.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.680+0000 7f29d7e00640 1 --2- 172.21.15.167:0/732693722 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29d814ee50 0x7f29d80c6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:34.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.680+0000 7f29dca00640 1 -- 172.21.15.167:0/732693722 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29d814ee50 msgr2=0x7f29d80c6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:34.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.680+0000 7f29dca00640 1 --2- 172.21.15.167:0/732693722 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29d814ee50 0x7f29d80c6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:34.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.680+0000 7f29dca00640 1 -- 172.21.15.167:0/732693722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29d80c6aa0 msgr2=0x7f29d815ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:34.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.680+0000 7f29dca00640 1 --2- 172.21.15.167:0/732693722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29d80c6aa0 0x7f29d815ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:34.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.680+0000 7f29dca00640 1 -- 172.21.15.167:0/732693722 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d80c7270 con 0x7f29d815ef30 2024-05-11T12:14:34.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.680+0000 7f29dca00640 1 --2- 172.21.15.167:0/732693722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29d815ef30 0x7f29d8151760 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f29cc00c490 tx=0x7f29cc00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:34.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.680+0000 7f29d4c00640 1 -- 172.21.15.167:0/732693722 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29cc018070 con 0x7f29d815ef30 2024-05-11T12:14:34.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.680+0000 7f29d4c00640 1 -- 172.21.15.167:0/732693722 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29cc007500 con 0x7f29d815ef30 2024-05-11T12:14:34.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.680+0000 7f29d4c00640 1 -- 172.21.15.167:0/732693722 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29cc004a20 con 0x7f29d815ef30 2024-05-11T12:14:34.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.680+0000 7f29dde00640 1 -- 172.21.15.167:0/732693722 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29d80c6f90 con 0x7f29d815ef30 2024-05-11T12:14:34.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.680+0000 7f29dde00640 1 -- 172.21.15.167:0/732693722 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29d815b340 con 0x7f29d815ef30 2024-05-11T12:14:34.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.681+0000 7f29dde00640 1 -- 172.21.15.167:0/732693722 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29d80694b0 con 0x7f29d815ef30 2024-05-11T12:14:34.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.683+0000 7f29d4c00640 1 -- 172.21.15.167:0/732693722 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f29cc01b070 con 0x7f29d815ef30 2024-05-11T12:14:34.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.683+0000 7f29d4c00640 1 --2- 172.21.15.167:0/732693722 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f29b80762d0 0x7f29b8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:34.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.684+0000 7f29d4c00640 1 -- 172.21.15.167:0/732693722 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6101+0+0 (secure 0 0 0) 0x7f29cc096ae0 con 0x7f29d815ef30 2024-05-11T12:14:34.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.684+0000 7f29d7e00640 1 --2- 172.21.15.167:0/732693722 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f29b80762d0 0x7f29b8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:34.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.685+0000 7f29d7e00640 1 --2- 172.21.15.167:0/732693722 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f29b80762d0 0x7f29b8078790 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f29c4034000 tx=0x7f29c402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:34.687 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.688+0000 7f29d4c00640 1 -- 172.21.15.167:0/732693722 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f29cc060210 con 0x7f29d815ef30 2024-05-11T12:14:34.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:34.932+0000 7f29dde00640 1 -- 172.21.15.167:0/732693722 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f29d80c7720 con 0x7f29d815ef30 2024-05-11T12:14:35.555 INFO:tasks.ceph.mds.c.smithi167.stderr:2024-05-11T12:14:35.556+0000 7fca14800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-11T12:14:35.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.632+0000 7f29d4c00640 1 -- 172.21.15.167:0/732693722 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v61) ==== 118+0+0 (secure 0 0 0) 0x7f29cc05fb10 con 0x7f29d815ef30 2024-05-11T12:14:35.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.635+0000 7f29dde00640 1 -- 172.21.15.167:0/732693722 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f29b80762d0 msgr2=0x7f29b8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:35.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.635+0000 7f29dde00640 1 --2- 172.21.15.167:0/732693722 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f29b80762d0 0x7f29b8078790 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f29c4034000 tx=0x7f29c402e000 comp rx=0 tx=0).stop 2024-05-11T12:14:35.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.635+0000 7f29dde00640 1 -- 172.21.15.167:0/732693722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29d815ef30 msgr2=0x7f29d8151760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:35.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.635+0000 7f29dde00640 1 --2- 172.21.15.167:0/732693722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29d815ef30 0x7f29d8151760 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f29cc00c490 tx=0x7f29cc00c950 comp rx=0 tx=0).stop 2024-05-11T12:14:35.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.635+0000 7f29dde00640 1 -- 172.21.15.167:0/732693722 shutdown_connections 2024-05-11T12:14:35.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.635+0000 7f29dde00640 1 --2- 172.21.15.167:0/732693722 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f29b80762d0 0x7f29b8078790 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:35.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.635+0000 7f29dde00640 1 --2- 172.21.15.167:0/732693722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29d80c6aa0 0x7f29d815ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:35.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.635+0000 7f29dde00640 1 --2- 172.21.15.167:0/732693722 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29d814ee50 0x7f29d80c6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:35.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.635+0000 7f29dde00640 1 --2- 172.21.15.167:0/732693722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29d815ef30 0x7f29d8151760 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:35.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.635+0000 7f29dde00640 1 -- 172.21.15.167:0/732693722 >> 172.21.15.167:0/732693722 conn(0x7f29d80b2f10 msgr2=0x7f29d8161810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:35.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.636+0000 7f29dde00640 1 -- 172.21.15.167:0/732693722 shutdown_connections 2024-05-11T12:14:35.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.636+0000 7f29dde00640 1 -- 172.21.15.167:0/732693722 wait complete. 2024-05-11T12:14:35.658 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-11T12:14:35.659 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-11T12:14:35.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.860+0000 7fa9e1a00640 1 Processor -- start 2024-05-11T12:14:35.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.860+0000 7fa9e1a00640 1 -- start start 2024-05-11T12:14:35.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.862+0000 7fa9e1a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa9dc15ef30 0x7fa9dc0728d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:35.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.862+0000 7fa9e1a00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9dc06eb70 con 0x7fa9dc15f310 2024-05-11T12:14:35.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.862+0000 7fa9e1a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9dc0af9f0 con 0x7fa9dc073130 2024-05-11T12:14:35.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.862+0000 7fa9e1a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fa9dc0b5100 con 0x7fa9dc15ef30 2024-05-11T12:14:35.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.862+0000 7fa9d3400640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fa9dc15f310 0x7fa9dc06e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:55624/0 (socket says 172.21.15.167:55624) 2024-05-11T12:14:35.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.863+0000 7fa9d3400640 1 -- 172.21.15.167:0/2166529058 learned_addr learned my addr 172.21.15.167:0/2166529058 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:14:35.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.863+0000 7fa9db400640 1 -- 172.21.15.167:0/2166529058 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2851169330 0 0) 0x7fa9dc06eb70 con 0x7fa9dc15f310 2024-05-11T12:14:35.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.863+0000 7fa9dbe00640 1 --2- 172.21.15.167:0/2166529058 >> v2:172.21.15.204:3300/0 conn(0x7fa9dc15ef30 0x7fa9dc0728d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:35.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.863+0000 7fa9db400640 1 -- 172.21.15.167:0/2166529058 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa9c0003520 con 0x7fa9dc15f310 2024-05-11T12:14:35.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.863+0000 7fa9db400640 1 -- 172.21.15.167:0/2166529058 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1191543509 0 0) 0x7fa9c0003520 con 0x7fa9dc15f310 2024-05-11T12:14:35.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.863+0000 7fa9db400640 1 -- 172.21.15.167:0/2166529058 >> v2:172.21.15.204:3300/0 conn(0x7fa9dc15ef30 msgr2=0x7fa9dc0728d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:35.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.863+0000 7fa9db400640 1 --2- 172.21.15.167:0/2166529058 >> v2:172.21.15.204:3300/0 conn(0x7fa9dc15ef30 0x7fa9dc0728d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:35.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.863+0000 7fa9db400640 1 -- 172.21.15.167:0/2166529058 >> v1:172.21.15.204:6789/0 conn(0x7fa9dc073130 legacy=0x7fa9dc0b5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:35.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.863+0000 7fa9db400640 1 -- 172.21.15.167:0/2166529058 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9dc0afcd0 con 0x7fa9dc15f310 2024-05-11T12:14:35.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.863+0000 7fa9db400640 1 -- 172.21.15.167:0/2166529058 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7fa9c40016f0 con 0x7fa9dc15f310 2024-05-11T12:14:35.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.863+0000 7fa9db400640 1 -- 172.21.15.167:0/2166529058 >> v1:172.21.15.167:6789/0 conn(0x7fa9dc15f310 legacy=0x7fa9dc06e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:35.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.863+0000 7fa9db400640 1 --2- 172.21.15.167:0/2166529058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9c0003a50 0x7fa9c0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:35.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.864+0000 7fa9db400640 1 --2- 172.21.15.167:0/2166529058 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9c0004800 0x7fa9c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:35.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.864+0000 7fa9db400640 1 --2- 172.21.15.167:0/2166529058 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9c00052e0 0x7fa9c000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:35.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.864+0000 7fa9db400640 1 -- 172.21.15.167:0/2166529058 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa9dc0b5100 con 0x7fa9c00052e0 2024-05-11T12:14:35.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.864+0000 7fa9db400640 1 -- 172.21.15.167:0/2166529058 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c000c230 con 0x7fa9c0003a50 2024-05-11T12:14:35.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.864+0000 7fa9db400640 1 -- 172.21.15.167:0/2166529058 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa9c000c430 con 0x7fa9c0004800 2024-05-11T12:14:35.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.864+0000 7fa9e1000640 1 --2- 172.21.15.167:0/2166529058 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9c00052e0 0x7fa9c000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:35.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.864+0000 7fa9d3400640 1 --2- 172.21.15.167:0/2166529058 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9c0004800 0x7fa9c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:35.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.864+0000 7fa9dbe00640 1 --2- 172.21.15.167:0/2166529058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9c0003a50 0x7fa9c0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:35.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.864+0000 7fa9e1000640 1 -- 172.21.15.167:0/2166529058 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9c0004800 msgr2=0x7fa9c0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:35.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.864+0000 7fa9e1000640 1 --2- 172.21.15.167:0/2166529058 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9c0004800 0x7fa9c0004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:35.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.864+0000 7fa9e1000640 1 -- 172.21.15.167:0/2166529058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9c0003a50 msgr2=0x7fa9c0003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:35.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.864+0000 7fa9e1000640 1 --2- 172.21.15.167:0/2166529058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9c0003a50 0x7fa9c0003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:35.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.864+0000 7fa9e1000640 1 -- 172.21.15.167:0/2166529058 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9c0003520 con 0x7fa9c00052e0 2024-05-11T12:14:35.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.864+0000 7fa9dbe00640 1 --2- 172.21.15.167:0/2166529058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9c0003a50 0x7fa9c0003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:35.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.865+0000 7fa9e1000640 1 -- 172.21.15.167:0/2166529058 reap_dead start 2024-05-11T12:14:35.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.865+0000 7fa9e1000640 1 --2- 172.21.15.167:0/2166529058 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9c00052e0 0x7fa9c000b9c0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4004480 tx=0x7fa9c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=344c047f044702a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:35.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.866+0000 7fa9db400640 1 -- 172.21.15.167:0/2166529058 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4004080 con 0x7fa9c00052e0 2024-05-11T12:14:35.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.866+0000 7fa9db400640 1 -- 172.21.15.167:0/2166529058 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9d402ad80 con 0x7fa9c00052e0 2024-05-11T12:14:35.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.866+0000 7fa9db400640 1 -- 172.21.15.167:0/2166529058 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4006d90 con 0x7fa9c00052e0 2024-05-11T12:14:35.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.866+0000 7fa9e1a00640 1 -- 172.21.15.167:0/2166529058 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9c00052e0 msgr2=0x7fa9c000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:35.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.866+0000 7fa9e1a00640 1 --2- 172.21.15.167:0/2166529058 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9c00052e0 0x7fa9c000b9c0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4004480 tx=0x7fa9c0005830 comp rx=0 tx=0).stop 2024-05-11T12:14:35.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.866+0000 7fa9e1a00640 1 -- 172.21.15.167:0/2166529058 shutdown_connections 2024-05-11T12:14:35.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.866+0000 7fa9e1a00640 1 --2- 172.21.15.167:0/2166529058 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9c00052e0 0x7fa9c000b9c0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:35.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.866+0000 7fa9e1a00640 1 -- 172.21.15.167:0/2166529058 >> 172.21.15.167:0/2166529058 conn(0x7fa9dc0b2f10 msgr2=0x7fa9dc06c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:35.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.866+0000 7fa9e1a00640 1 -- 172.21.15.167:0/2166529058 shutdown_connections 2024-05-11T12:14:35.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.866+0000 7fa9e1a00640 1 -- 172.21.15.167:0/2166529058 wait complete. 2024-05-11T12:14:35.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.868+0000 7fa9e1a00640 1 Processor -- start 2024-05-11T12:14:35.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.868+0000 7fa9e1a00640 1 -- start start 2024-05-11T12:14:35.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.869+0000 7fa9e1a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9c00052e0 0x7fa9dc1f51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:35.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.869+0000 7fa9e1a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9dc15d560 0x7fa9dc15b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:35.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.869+0000 7fa9e1a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9dc1522d0 0x7fa9dc152740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:35.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.869+0000 7fa9e1a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa9dc0af580 con 0x7fa9dc15d560 2024-05-11T12:14:35.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.869+0000 7fa9e1a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa9dc0b4a40 con 0x7fa9dc1522d0 2024-05-11T12:14:35.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.869+0000 7fa9e1a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa9dc0af700 con 0x7fa9c00052e0 2024-05-11T12:14:35.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.869+0000 7fa9d3400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9dc15d560 0x7fa9dc15b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:35.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.869+0000 7fa9d3400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9dc15d560 0x7fa9dc15b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:40420/0 (socket says 172.21.15.167:40420) 2024-05-11T12:14:35.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.869+0000 7fa9d3400640 1 -- 172.21.15.167:0/722665445 learned_addr learned my addr 172.21.15.167:0/722665445 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:35.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.870+0000 7fa9dbe00640 1 --2- 172.21.15.167:0/722665445 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9c00052e0 0x7fa9dc1f51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:35.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.870+0000 7fa9e1000640 1 --2- 172.21.15.167:0/722665445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9dc1522d0 0x7fa9dc152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:35.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.870+0000 7fa9d3400640 1 -- 172.21.15.167:0/722665445 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9c00052e0 msgr2=0x7fa9dc1f51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:35.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.870+0000 7fa9d3400640 1 --2- 172.21.15.167:0/722665445 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9c00052e0 0x7fa9dc1f51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:35.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.870+0000 7fa9d3400640 1 -- 172.21.15.167:0/722665445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9dc1522d0 msgr2=0x7fa9dc152740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:35.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.870+0000 7fa9d3400640 1 --2- 172.21.15.167:0/722665445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9dc1522d0 0x7fa9dc152740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:35.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.870+0000 7fa9d3400640 1 -- 172.21.15.167:0/722665445 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9dc15dd20 con 0x7fa9dc15d560 2024-05-11T12:14:35.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.870+0000 7fa9d3400640 1 --2- 172.21.15.167:0/722665445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9dc15d560 0x7fa9dc15b5b0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fa9c400d770 tx=0x7fa9c400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:35.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.870+0000 7fa9d8c00640 1 -- 172.21.15.167:0/722665445 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c40074e0 con 0x7fa9dc15d560 2024-05-11T12:14:35.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.870+0000 7fa9d8c00640 1 -- 172.21.15.167:0/722665445 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9c400de20 con 0x7fa9dc15d560 2024-05-11T12:14:35.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.870+0000 7fa9e1a00640 1 -- 172.21.15.167:0/722665445 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9dc15e000 con 0x7fa9dc15d560 2024-05-11T12:14:35.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.871+0000 7fa9d8c00640 1 -- 172.21.15.167:0/722665445 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c4004a20 con 0x7fa9dc15d560 2024-05-11T12:14:35.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.871+0000 7fa9e1a00640 1 -- 172.21.15.167:0/722665445 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9dc15e5c0 con 0x7fa9dc15d560 2024-05-11T12:14:35.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.872+0000 7fa9e1a00640 1 -- 172.21.15.167:0/722665445 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9dc0694b0 con 0x7fa9dc15d560 2024-05-11T12:14:35.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.873+0000 7fa9d8c00640 1 -- 172.21.15.167:0/722665445 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa9c401b020 con 0x7fa9dc15d560 2024-05-11T12:14:35.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.874+0000 7fa9d8c00640 1 --2- 172.21.15.167:0/722665445 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa9b8076300 0x7fa9b80787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:35.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.874+0000 7fa9d8c00640 1 -- 172.21.15.167:0/722665445 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6101+0+0 (secure 0 0 0) 0x7fa9c40974f0 con 0x7fa9dc15d560 2024-05-11T12:14:35.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.874+0000 7fa9dbe00640 1 --2- 172.21.15.167:0/722665445 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa9b8076300 0x7fa9b80787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:35.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.875+0000 7fa9dbe00640 1 --2- 172.21.15.167:0/722665445 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa9b8076300 0x7fa9b80787c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc006fd0 tx=0x7fa9cc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:35.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:35.878+0000 7fa9d8c00640 1 -- 172.21.15.167:0/722665445 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa9c4060670 con 0x7fa9dc15d560 2024-05-11T12:14:36.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.129+0000 7fa9e1a00640 1 -- 172.21.15.167:0/722665445 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fa9dc0b3a60 con 0x7fa9dc15d560 2024-05-11T12:14:36.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.635+0000 7fa9d8c00640 1 -- 172.21.15.167:0/722665445 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v62) ==== 122+0+0 (secure 0 0 0) 0x7fa9c4064970 con 0x7fa9dc15d560 2024-05-11T12:14:36.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.639+0000 7fa9e1a00640 1 -- 172.21.15.167:0/722665445 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa9b8076300 msgr2=0x7fa9b80787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:36.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.639+0000 7fa9e1a00640 1 --2- 172.21.15.167:0/722665445 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa9b8076300 0x7fa9b80787c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc006fd0 tx=0x7fa9cc008040 comp rx=0 tx=0).stop 2024-05-11T12:14:36.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.639+0000 7fa9e1a00640 1 -- 172.21.15.167:0/722665445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9dc15d560 msgr2=0x7fa9dc15b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:36.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.639+0000 7fa9e1a00640 1 --2- 172.21.15.167:0/722665445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9dc15d560 0x7fa9dc15b5b0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fa9c400d770 tx=0x7fa9c400dc30 comp rx=0 tx=0).stop 2024-05-11T12:14:36.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.639+0000 7fa9e1a00640 1 -- 172.21.15.167:0/722665445 shutdown_connections 2024-05-11T12:14:36.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.639+0000 7fa9e1a00640 1 --2- 172.21.15.167:0/722665445 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa9b8076300 0x7fa9b80787c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:36.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.639+0000 7fa9e1a00640 1 --2- 172.21.15.167:0/722665445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9dc1522d0 0x7fa9dc152740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:36.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.639+0000 7fa9e1a00640 1 --2- 172.21.15.167:0/722665445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9dc15d560 0x7fa9dc15b5b0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:36.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.639+0000 7fa9e1a00640 1 --2- 172.21.15.167:0/722665445 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9c00052e0 0x7fa9dc1f51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:36.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.639+0000 7fa9e1a00640 1 -- 172.21.15.167:0/722665445 >> 172.21.15.167:0/722665445 conn(0x7fa9dc0b2f10 msgr2=0x7fa9dc1547d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:36.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.640+0000 7fa9e1a00640 1 -- 172.21.15.167:0/722665445 shutdown_connections 2024-05-11T12:14:36.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.640+0000 7fa9e1a00640 1 -- 172.21.15.167:0/722665445 wait complete. 2024-05-11T12:14:36.661 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-11T12:14:36.661 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:14:36.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.865+0000 7f066d000640 1 Processor -- start 2024-05-11T12:14:36.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.865+0000 7f066d000640 1 -- start start 2024-05-11T12:14:36.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.866+0000 7f066d000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f066806d460 0x7f066806d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:36.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.866+0000 7f066d000640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f066815df30 con 0x7f06680b3ea0 2024-05-11T12:14:36.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.866+0000 7f066d000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f066815e210 con 0x7f0668157750 2024-05-11T12:14:36.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.866+0000 7f066d000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f066806ddf0 con 0x7f066806d460 2024-05-11T12:14:36.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.866+0000 7f0665400640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f06680b3ea0 0x7f0668161490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:55634/0 (socket says 172.21.15.167:55634) 2024-05-11T12:14:36.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.866+0000 7f0665400640 1 -- 172.21.15.167:0/1594684065 learned_addr learned my addr 172.21.15.167:0/1594684065 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:14:36.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.867+0000 7f0666800640 1 --2- 172.21.15.167:0/1594684065 >> v2:172.21.15.204:3300/0 conn(0x7f066806d460 0x7f066806d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:36.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.867+0000 7f0664a00640 1 -- 172.21.15.167:0/1594684065 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1531386345 0 0) 0x7f066815df30 con 0x7f06680b3ea0 2024-05-11T12:14:36.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.867+0000 7f0664a00640 1 -- 172.21.15.167:0/1594684065 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f064c003520 con 0x7f06680b3ea0 2024-05-11T12:14:36.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.867+0000 7f0664a00640 1 -- 172.21.15.167:0/1594684065 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1016415651 0 0) 0x7f064c003520 con 0x7f06680b3ea0 2024-05-11T12:14:36.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.867+0000 7f0664a00640 1 -- 172.21.15.167:0/1594684065 >> v2:172.21.15.204:3300/0 conn(0x7f066806d460 msgr2=0x7f066806d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:36.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.867+0000 7f0664a00640 1 --2- 172.21.15.167:0/1594684065 >> v2:172.21.15.204:3300/0 conn(0x7f066806d460 0x7f066806d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:36.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.867+0000 7f0664a00640 1 -- 172.21.15.167:0/1594684065 >> v1:172.21.15.204:6789/0 conn(0x7f0668157750 legacy=0x7f066815d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:36.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.867+0000 7f0664a00640 1 -- 172.21.15.167:0/1594684065 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f066815e4f0 con 0x7f06680b3ea0 2024-05-11T12:14:36.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.867+0000 7f0664a00640 1 -- 172.21.15.167:0/1594684065 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f06500016f0 con 0x7f06680b3ea0 2024-05-11T12:14:36.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.867+0000 7f0666800640 1 --2- 172.21.15.167:0/1594684065 >> v2:172.21.15.204:3300/0 conn(0x7f066806d460 0x7f066806d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:14:36.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.867+0000 7f0664a00640 1 -- 172.21.15.167:0/1594684065 >> v1:172.21.15.167:6789/0 conn(0x7f06680b3ea0 legacy=0x7f0668161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:36.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.867+0000 7f0664a00640 1 --2- 172.21.15.167:0/1594684065 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f064c003a50 0x7f064c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:36.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.867+0000 7f0664a00640 1 --2- 172.21.15.167:0/1594684065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f064c004800 0x7f064c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:36.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.867+0000 7f0664a00640 1 --2- 172.21.15.167:0/1594684065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f064c0052e0 0x7f064c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:36.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.868+0000 7f0664a00640 1 -- 172.21.15.167:0/1594684065 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f066806ddf0 con 0x7f064c004800 2024-05-11T12:14:36.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.868+0000 7f0664a00640 1 -- 172.21.15.167:0/1594684065 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f064c00c190 con 0x7f064c0052e0 2024-05-11T12:14:36.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.868+0000 7f0664a00640 1 -- 172.21.15.167:0/1594684065 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f064c00c390 con 0x7f064c003a50 2024-05-11T12:14:36.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.868+0000 7f0665400640 1 --2- 172.21.15.167:0/1594684065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f064c004800 0x7f064c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:36.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.868+0000 7f0666800640 1 --2- 172.21.15.167:0/1594684065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f064c0052e0 0x7f064c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:36.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.868+0000 7f0665e00640 1 --2- 172.21.15.167:0/1594684065 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f064c003a50 0x7f064c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:36.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.868+0000 7f0665400640 1 -- 172.21.15.167:0/1594684065 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f064c003a50 msgr2=0x7f064c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:36.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.868+0000 7f0665400640 1 --2- 172.21.15.167:0/1594684065 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f064c003a50 0x7f064c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:36.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.868+0000 7f0665400640 1 -- 172.21.15.167:0/1594684065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f064c0052e0 msgr2=0x7f064c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:36.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.868+0000 7f0665400640 1 --2- 172.21.15.167:0/1594684065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f064c0052e0 0x7f064c00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:36.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.868+0000 7f0665400640 1 -- 172.21.15.167:0/1594684065 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f064c003520 con 0x7f064c004800 2024-05-11T12:14:36.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.868+0000 7f0666800640 1 -- 172.21.15.167:0/1594684065 reap_dead start 2024-05-11T12:14:36.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.868+0000 7f0665e00640 1 --2- 172.21.15.167:0/1594684065 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f064c003a50 0x7f064c003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:36.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.869+0000 7f0665400640 1 --2- 172.21.15.167:0/1594684065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f064c004800 0x7f064c004c80 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f0650007fd0 tx=0x7f064c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0de57a3e7220a9d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:36.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.870+0000 7f0664a00640 1 -- 172.21.15.167:0/1594684065 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0650007ce0 con 0x7f064c004800 2024-05-11T12:14:36.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.870+0000 7f0664a00640 1 -- 172.21.15.167:0/1594684065 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0650005af0 con 0x7f064c004800 2024-05-11T12:14:36.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.870+0000 7f0664a00640 1 -- 172.21.15.167:0/1594684065 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0650007070 con 0x7f064c004800 2024-05-11T12:14:36.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.870+0000 7f066d000640 1 -- 172.21.15.167:0/1594684065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f064c004800 msgr2=0x7f064c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:36.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.870+0000 7f066d000640 1 --2- 172.21.15.167:0/1594684065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f064c004800 0x7f064c004c80 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f0650007fd0 tx=0x7f064c005830 comp rx=0 tx=0).stop 2024-05-11T12:14:36.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.870+0000 7f066d000640 1 -- 172.21.15.167:0/1594684065 shutdown_connections 2024-05-11T12:14:36.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.870+0000 7f066d000640 1 --2- 172.21.15.167:0/1594684065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f064c004800 0x7f064c004c80 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:36.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.870+0000 7f066d000640 1 -- 172.21.15.167:0/1594684065 >> 172.21.15.167:0/1594684065 conn(0x7f06680cd8f0 msgr2=0x7f06680cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:36.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.870+0000 7f066d000640 1 -- 172.21.15.167:0/1594684065 shutdown_connections 2024-05-11T12:14:36.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.870+0000 7f066d000640 1 -- 172.21.15.167:0/1594684065 wait complete. 2024-05-11T12:14:36.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.872+0000 7f066d000640 1 Processor -- start 2024-05-11T12:14:36.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.872+0000 7f066d000640 1 -- start start 2024-05-11T12:14:36.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.873+0000 7f066d000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f064c004800 0x7f066813c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:36.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.873+0000 7f066d000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0668144720 0x7f0668157da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:36.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.873+0000 7f066d000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0668158a70 0x7f0668158ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:36.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.873+0000 7f066d000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f066806d8b0 con 0x7f0668144720 2024-05-11T12:14:36.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.873+0000 7f066d000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0668161000 con 0x7f0668158a70 2024-05-11T12:14:36.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.873+0000 7f066d000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f066806da30 con 0x7f064c004800 2024-05-11T12:14:36.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.873+0000 7f0665400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0668144720 0x7f0668157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:36.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.873+0000 7f0665400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0668144720 0x7f0668157da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:40446/0 (socket says 172.21.15.167:40446) 2024-05-11T12:14:36.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.873+0000 7f0665400640 1 -- 172.21.15.167:0/381300947 learned_addr learned my addr 172.21.15.167:0/381300947 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:36.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.874+0000 7f0666800640 1 --2- 172.21.15.167:0/381300947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0668158a70 0x7f0668158ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:36.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.874+0000 7f0665400640 1 -- 172.21.15.167:0/381300947 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f064c004800 msgr2=0x7f066813c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:36.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.874+0000 7f0665400640 1 --2- 172.21.15.167:0/381300947 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f064c004800 0x7f066813c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:36.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.874+0000 7f0665400640 1 -- 172.21.15.167:0/381300947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0668158a70 msgr2=0x7f0668158ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:36.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.874+0000 7f0665400640 1 --2- 172.21.15.167:0/381300947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0668158a70 0x7f0668158ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:36.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.874+0000 7f0665400640 1 -- 172.21.15.167:0/381300947 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06681598a0 con 0x7f0668144720 2024-05-11T12:14:36.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.874+0000 7f0666800640 1 --2- 172.21.15.167:0/381300947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0668158a70 0x7f0668158ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:36.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.874+0000 7f0665400640 1 --2- 172.21.15.167:0/381300947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0668144720 0x7f0668157da0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f065002acf0 tx=0x7f066815e100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:36.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.874+0000 7f065e000640 1 -- 172.21.15.167:0/381300947 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0650037040 con 0x7f0668144720 2024-05-11T12:14:36.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.874+0000 7f065e000640 1 -- 172.21.15.167:0/381300947 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06500014b0 con 0x7f0668144720 2024-05-11T12:14:36.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.874+0000 7f065e000640 1 -- 172.21.15.167:0/381300947 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06500047e0 con 0x7f0668144720 2024-05-11T12:14:36.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.874+0000 7f066d000640 1 -- 172.21.15.167:0/381300947 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06681582e0 con 0x7f0668144720 2024-05-11T12:14:36.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.875+0000 7f066d000640 1 -- 172.21.15.167:0/381300947 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f066815c9b0 con 0x7f0668144720 2024-05-11T12:14:36.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.876+0000 7f066d000640 1 -- 172.21.15.167:0/381300947 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f066806f7c0 con 0x7f0668144720 2024-05-11T12:14:36.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.877+0000 7f065e000640 1 -- 172.21.15.167:0/381300947 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0650043020 con 0x7f0668144720 2024-05-11T12:14:36.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.878+0000 7f065e000640 1 --2- 172.21.15.167:0/381300947 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f06440762d0 0x7f0644078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:36.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.878+0000 7f065e000640 1 -- 172.21.15.167:0/381300947 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6101+0+0 (secure 0 0 0) 0x7f06500b6e30 con 0x7f0668144720 2024-05-11T12:14:36.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.879+0000 7f0665e00640 1 --2- 172.21.15.167:0/381300947 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f06440762d0 0x7f0644078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:36.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.880+0000 7f0665e00640 1 --2- 172.21.15.167:0/381300947 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f06440762d0 0x7f0644078790 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f06680520f0 tx=0x7f0658009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:36.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:36.883+0000 7f065e000640 1 -- 172.21.15.167:0/381300947 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f066806f7c0 con 0x7f0668144720 2024-05-11T12:14:37.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.135+0000 7f066d000640 1 -- 172.21.15.167:0/381300947 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0668069590 con 0x7f0668144720 2024-05-11T12:14:37.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.137+0000 7f065e000640 1 -- 172.21.15.167:0/381300947 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5195 (secure 0 0 0) 0x7f06500805f0 con 0x7f0668144720 2024-05-11T12:14:37.136 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:37.136 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":62,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5033,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/4026834899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4026834899},{"type":"v1","addr":"172.21.15.204:6834","nonce":4026834899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5036,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1440200187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":1440200187},{"type":"v1","addr":"172.21.15.204:6835","nonce":1440200187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":6636,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1570676224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1570676224},{"type":"v1","addr":"172.21.15.167:6835","nonce":1570676224}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:14:33.067226+0000","modified":"2024-05-11T12:14:36.130783+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6639},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6639":{"gid":6639,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/3493868276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3493868276},{"type":"v1","addr":"172.21.15.167:6837","nonce":3493868276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6639,"qdb_cluster":[6639]},"id":7}]} 2024-05-11T12:14:37.137 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 62 2024-05-11T12:14:37.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.140+0000 7f066d000640 1 -- 172.21.15.167:0/381300947 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f06440762d0 msgr2=0x7f0644078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:37.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.140+0000 7f066d000640 1 --2- 172.21.15.167:0/381300947 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f06440762d0 0x7f0644078790 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f06680520f0 tx=0x7f0658009040 comp rx=0 tx=0).stop 2024-05-11T12:14:37.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.140+0000 7f066d000640 1 -- 172.21.15.167:0/381300947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0668144720 msgr2=0x7f0668157da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:37.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.140+0000 7f066d000640 1 --2- 172.21.15.167:0/381300947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0668144720 0x7f0668157da0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f065002acf0 tx=0x7f066815e100 comp rx=0 tx=0).stop 2024-05-11T12:14:37.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.140+0000 7f066d000640 1 -- 172.21.15.167:0/381300947 shutdown_connections 2024-05-11T12:14:37.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.140+0000 7f066d000640 1 --2- 172.21.15.167:0/381300947 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f06440762d0 0x7f0644078790 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:37.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.140+0000 7f066d000640 1 --2- 172.21.15.167:0/381300947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0668158a70 0x7f0668158ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:37.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.140+0000 7f066d000640 1 --2- 172.21.15.167:0/381300947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0668144720 0x7f0668157da0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:37.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.140+0000 7f066d000640 1 --2- 172.21.15.167:0/381300947 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f064c004800 0x7f066813c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:37.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.140+0000 7f066d000640 1 -- 172.21.15.167:0/381300947 >> 172.21.15.167:0/381300947 conn(0x7f06680cd8f0 msgr2=0x7f06680cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:37.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.140+0000 7f066d000640 1 -- 172.21.15.167:0/381300947 shutdown_connections 2024-05-11T12:14:37.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.140+0000 7f066d000640 1 -- 172.21.15.167:0/381300947 wait complete. 2024-05-11T12:14:37.159 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:14:37.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.368+0000 7f5089c00640 1 Processor -- start 2024-05-11T12:14:37.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.369+0000 7f5089c00640 1 -- start start 2024-05-11T12:14:37.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.370+0000 7f5089c00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f50840d0640 0x7f50840d3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:37.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.370+0000 7f5089c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f508406c7f0 0x7f50840d4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:37.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.370+0000 7f5089c00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50840d2db0 con 0x7f508415f3f0 2024-05-11T12:14:37.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.370+0000 7f5089c00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f508415f7d0 con 0x7f50840d0640 2024-05-11T12:14:37.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.370+0000 7f5089c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f50840d3090 con 0x7f508406c7f0 2024-05-11T12:14:37.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.370+0000 7f5082000640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f50840d0640 0x7f50840d3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:37.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.370+0000 7f5082a00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f508415f3f0 0x7f50840b83a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:55644/0 (socket says 172.21.15.167:55644) 2024-05-11T12:14:37.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.370+0000 7f5082a00640 1 -- 172.21.15.167:0/3096280038 learned_addr learned my addr 172.21.15.167:0/3096280038 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:14:37.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.371+0000 7f5083400640 1 --2- 172.21.15.167:0/3096280038 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f508406c7f0 0x7f50840d4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:37.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.371+0000 7f5082000640 1 -- 172.21.15.167:0/3096280038 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f508406c7f0 msgr2=0x7f50840d4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:37.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.371+0000 7f5082000640 1 --2- 172.21.15.167:0/3096280038 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f508406c7f0 0x7f50840d4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:37.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.371+0000 7f5082000640 1 -- 172.21.15.167:0/3096280038 >> v1:172.21.15.167:6789/0 conn(0x7f508415f3f0 legacy=0x7f50840b83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:37.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.371+0000 7f5082000640 1 -- 172.21.15.167:0/3096280038 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50840d3370 con 0x7f50840d0640 2024-05-11T12:14:37.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.372+0000 7f5082000640 1 --2- 172.21.15.167:0/3096280038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50840d0640 0x7f50840d3e50 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f506c009d90 tx=0x7f506c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d116f5ff3090347 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:37.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.372+0000 7f5081600640 1 -- 172.21.15.167:0/3096280038 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506c007740 con 0x7f50840d0640 2024-05-11T12:14:37.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.372+0000 7f5081600640 1 -- 172.21.15.167:0/3096280038 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f506c0078e0 con 0x7f50840d0640 2024-05-11T12:14:37.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.372+0000 7f5081600640 1 -- 172.21.15.167:0/3096280038 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506c007cb0 con 0x7f50840d0640 2024-05-11T12:14:37.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.373+0000 7f5089c00640 1 -- 172.21.15.167:0/3096280038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50840d0640 msgr2=0x7f50840d3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:37.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.373+0000 7f5089c00640 1 --2- 172.21.15.167:0/3096280038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50840d0640 0x7f50840d3e50 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f506c009d90 tx=0x7f506c02a660 comp rx=0 tx=0).stop 2024-05-11T12:14:37.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.373+0000 7f5089c00640 1 -- 172.21.15.167:0/3096280038 shutdown_connections 2024-05-11T12:14:37.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.373+0000 7f5089c00640 1 --2- 172.21.15.167:0/3096280038 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f508406c7f0 0x7f50840d4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:37.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.373+0000 7f5089c00640 1 --2- 172.21.15.167:0/3096280038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50840d0640 0x7f50840d3e50 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:37.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.373+0000 7f5089c00640 1 -- 172.21.15.167:0/3096280038 >> 172.21.15.167:0/3096280038 conn(0x7f50840d51e0 msgr2=0x7f50840d55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:37.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.373+0000 7f5089c00640 1 -- 172.21.15.167:0/3096280038 shutdown_connections 2024-05-11T12:14:37.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.373+0000 7f5089c00640 1 -- 172.21.15.167:0/3096280038 wait complete. 2024-05-11T12:14:37.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.375+0000 7f5089c00640 1 Processor -- start 2024-05-11T12:14:37.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.375+0000 7f5089c00640 1 -- start start 2024-05-11T12:14:37.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.376+0000 7f5089c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f508406c7f0 0x7f50840ce430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:37.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.376+0000 7f5089c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50840d0640 0x7f50840caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:37.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.376+0000 7f5089c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f508415f3f0 0x7f50840cef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:37.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.376+0000 7f5089c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f50840b8ab0 con 0x7f508415f3f0 2024-05-11T12:14:37.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.376+0000 7f5089c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f50840d3930 con 0x7f508406c7f0 2024-05-11T12:14:37.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.376+0000 7f5089c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f50840b8c30 con 0x7f50840d0640 2024-05-11T12:14:37.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.377+0000 7f5083400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f508415f3f0 0x7f50840cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:37.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.377+0000 7f5083400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f508415f3f0 0x7f50840cef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:40468/0 (socket says 172.21.15.167:40468) 2024-05-11T12:14:37.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.377+0000 7f5083400640 1 -- 172.21.15.167:0/398160204 learned_addr learned my addr 172.21.15.167:0/398160204 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:37.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.377+0000 7f5082000640 1 --2- 172.21.15.167:0/398160204 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50840d0640 0x7f50840caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:37.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.377+0000 7f5082a00640 1 --2- 172.21.15.167:0/398160204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f508406c7f0 0x7f50840ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:37.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.377+0000 7f5083400640 1 -- 172.21.15.167:0/398160204 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50840d0640 msgr2=0x7f50840caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:37.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.377+0000 7f5083400640 1 --2- 172.21.15.167:0/398160204 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50840d0640 0x7f50840caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:37.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.377+0000 7f5083400640 1 -- 172.21.15.167:0/398160204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f508406c7f0 msgr2=0x7f50840ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:37.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.377+0000 7f5083400640 1 --2- 172.21.15.167:0/398160204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f508406c7f0 0x7f50840ce430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:37.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.377+0000 7f5083400640 1 -- 172.21.15.167:0/398160204 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50840cd0b0 con 0x7f508415f3f0 2024-05-11T12:14:37.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.377+0000 7f5082000640 1 --2- 172.21.15.167:0/398160204 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50840d0640 0x7f50840caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:37.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.377+0000 7f5083400640 1 --2- 172.21.15.167:0/398160204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f508415f3f0 0x7f50840cef70 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f507c00c7f0 tx=0x7f507c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:37.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.378+0000 7f507aa00640 1 -- 172.21.15.167:0/398160204 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c015070 con 0x7f508415f3f0 2024-05-11T12:14:37.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.378+0000 7f507aa00640 1 -- 172.21.15.167:0/398160204 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f507c004220 con 0x7f508415f3f0 2024-05-11T12:14:37.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.378+0000 7f507aa00640 1 -- 172.21.15.167:0/398160204 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c007620 con 0x7f508415f3f0 2024-05-11T12:14:37.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.378+0000 7f5089c00640 1 -- 172.21.15.167:0/398160204 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50840cd390 con 0x7f508415f3f0 2024-05-11T12:14:37.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.378+0000 7f5089c00640 1 -- 172.21.15.167:0/398160204 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50840cd950 con 0x7f508415f3f0 2024-05-11T12:14:37.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.379+0000 7f5089c00640 1 -- 172.21.15.167:0/398160204 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5084069bc0 con 0x7f508415f3f0 2024-05-11T12:14:37.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.381+0000 7f507aa00640 1 -- 172.21.15.167:0/398160204 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f507c0079d0 con 0x7f508415f3f0 2024-05-11T12:14:37.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.381+0000 7f507aa00640 1 --2- 172.21.15.167:0/398160204 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f50600762d0 0x7f5060078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:37.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.382+0000 7f507aa00640 1 -- 172.21.15.167:0/398160204 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6101+0+0 (secure 0 0 0) 0x7f507c097890 con 0x7f508415f3f0 2024-05-11T12:14:37.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.382+0000 7f5082a00640 1 --2- 172.21.15.167:0/398160204 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f50600762d0 0x7f5060078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:37.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.383+0000 7f5082a00640 1 --2- 172.21.15.167:0/398160204 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f50600762d0 0x7f5060078790 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5074000f60 tx=0x7f5074009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:37.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.386+0000 7f507aa00640 1 -- 172.21.15.167:0/398160204 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f507c060fc0 con 0x7f508415f3f0 2024-05-11T12:14:37.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.593+0000 7f5089c00640 1 -- 172.21.15.167:0/398160204 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5084160b30 con 0x7f508415f3f0 2024-05-11T12:14:37.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.595+0000 7f507aa00640 1 -- 172.21.15.167:0/398160204 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+16921 (secure 0 0 0) 0x7f507c0608c0 con 0x7f508415f3f0 2024-05-11T12:14:37.595 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:37.595 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":97,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:14:34.600263+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:14:30.972447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-11T12:14:32.037637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:14:37.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.598+0000 7f5089c00640 1 -- 172.21.15.167:0/398160204 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f50600762d0 msgr2=0x7f5060078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:37.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.598+0000 7f5089c00640 1 --2- 172.21.15.167:0/398160204 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f50600762d0 0x7f5060078790 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5074000f60 tx=0x7f5074009040 comp rx=0 tx=0).stop 2024-05-11T12:14:37.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.598+0000 7f5089c00640 1 -- 172.21.15.167:0/398160204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f508415f3f0 msgr2=0x7f50840cef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:37.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.598+0000 7f5089c00640 1 --2- 172.21.15.167:0/398160204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f508415f3f0 0x7f50840cef70 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f507c00c7f0 tx=0x7f507c00ccb0 comp rx=0 tx=0).stop 2024-05-11T12:14:37.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.598+0000 7f5089c00640 1 -- 172.21.15.167:0/398160204 shutdown_connections 2024-05-11T12:14:37.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.599+0000 7f5089c00640 1 --2- 172.21.15.167:0/398160204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f508415f3f0 0x7f50840cef70 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:37.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.599+0000 7f5089c00640 1 --2- 172.21.15.167:0/398160204 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50840d0640 0x7f50840caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:37.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.599+0000 7f5089c00640 1 --2- 172.21.15.167:0/398160204 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f50600762d0 0x7f5060078790 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:37.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.599+0000 7f5089c00640 1 --2- 172.21.15.167:0/398160204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f508406c7f0 0x7f50840ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:37.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.599+0000 7f5089c00640 1 -- 172.21.15.167:0/398160204 >> 172.21.15.167:0/398160204 conn(0x7f50840d51e0 msgr2=0x7f50840d55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:37.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.599+0000 7f5089c00640 1 -- 172.21.15.167:0/398160204 shutdown_connections 2024-05-11T12:14:37.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.599+0000 7f5089c00640 1 -- 172.21.15.167:0/398160204 wait complete. 2024-05-11T12:14:37.616 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-11T12:14:37.617 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T12:14:37.635 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T12:14:37.668 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T12:14:37.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.800+0000 7f2c4be00640 1 Processor -- start 2024-05-11T12:14:37.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.801+0000 7f2c4be00640 1 -- start start 2024-05-11T12:14:37.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.802+0000 7f2c4be00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2c4c146770 0x7f2c4c0ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:37.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.802+0000 7f2c4be00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c4c06fbc0 con 0x7f2c4c145b00 2024-05-11T12:14:37.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.802+0000 7f2c4be00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c4c0b4520 con 0x7f2c4c0cef10 2024-05-11T12:14:37.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.802+0000 7f2c4be00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f2c4c0b4800 con 0x7f2c4c146770 2024-05-11T12:14:37.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.802+0000 7f2c4aa00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f2c4c145b00 0x7f2c4c06f2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:55652/0 (socket says 172.21.15.167:55652) 2024-05-11T12:14:37.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.802+0000 7f2c4aa00640 1 -- 172.21.15.167:0/611974174 learned_addr learned my addr 172.21.15.167:0/611974174 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:14:37.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.803+0000 7f2c43e00640 1 --2- 172.21.15.167:0/611974174 >> v2:172.21.15.204:3300/0 conn(0x7f2c4c146770 0x7f2c4c0ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:37.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.803+0000 7f2c4a000640 1 -- 172.21.15.167:0/611974174 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3386483541 0 0) 0x7f2c4c06fbc0 con 0x7f2c4c145b00 2024-05-11T12:14:37.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.803+0000 7f2c4a000640 1 -- 172.21.15.167:0/611974174 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2c30003520 con 0x7f2c4c145b00 2024-05-11T12:14:37.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.803+0000 7f2c4a000640 1 -- 172.21.15.167:0/611974174 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4284294016 0 0) 0x7f2c30003520 con 0x7f2c4c145b00 2024-05-11T12:14:37.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.803+0000 7f2c4a000640 1 -- 172.21.15.167:0/611974174 >> v2:172.21.15.204:3300/0 conn(0x7f2c4c146770 msgr2=0x7f2c4c0ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:37.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.803+0000 7f2c4a000640 1 --2- 172.21.15.167:0/611974174 >> v2:172.21.15.204:3300/0 conn(0x7f2c4c146770 0x7f2c4c0ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:37.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.803+0000 7f2c4a000640 1 -- 172.21.15.167:0/611974174 >> v1:172.21.15.204:6789/0 conn(0x7f2c4c0cef10 legacy=0x7f2c4c06c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:37.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.803+0000 7f2c4a000640 1 -- 172.21.15.167:0/611974174 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c4c0b4ae0 con 0x7f2c4c145b00 2024-05-11T12:14:37.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.803+0000 7f2c4a000640 1 -- 172.21.15.167:0/611974174 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f2c3c0016f0 con 0x7f2c4c145b00 2024-05-11T12:14:37.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.803+0000 7f2c4a000640 1 -- 172.21.15.167:0/611974174 >> v1:172.21.15.167:6789/0 conn(0x7f2c4c145b00 legacy=0x7f2c4c06f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:37.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.803+0000 7f2c4a000640 1 --2- 172.21.15.167:0/611974174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c30003a50 0x7f2c30003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:37.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.803+0000 7f2c4a000640 1 --2- 172.21.15.167:0/611974174 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c30004800 0x7f2c30004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:37.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.803+0000 7f2c4a000640 1 --2- 172.21.15.167:0/611974174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c300053c0 0x7f2c3000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:37.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.804+0000 7f2c4a000640 1 -- 172.21.15.167:0/611974174 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4c0b4800 con 0x7f2c300053c0 2024-05-11T12:14:37.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.804+0000 7f2c4a000640 1 -- 172.21.15.167:0/611974174 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3000c250 con 0x7f2c30003a50 2024-05-11T12:14:37.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.804+0000 7f2c4a000640 1 -- 172.21.15.167:0/611974174 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2c3000c450 con 0x7f2c30004800 2024-05-11T12:14:37.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.804+0000 7f2c4b400640 1 --2- 172.21.15.167:0/611974174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c300053c0 0x7f2c3000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:37.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.804+0000 7f2c4aa00640 1 --2- 172.21.15.167:0/611974174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c30003a50 0x7f2c30003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:37.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.804+0000 7f2c43e00640 1 --2- 172.21.15.167:0/611974174 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c30004800 0x7f2c30004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:37.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.804+0000 7f2c4b400640 1 -- 172.21.15.167:0/611974174 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c30004800 msgr2=0x7f2c30004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:37.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.804+0000 7f2c4b400640 1 --2- 172.21.15.167:0/611974174 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c30004800 0x7f2c30004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:37.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.804+0000 7f2c4b400640 1 -- 172.21.15.167:0/611974174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c30003a50 msgr2=0x7f2c30003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:37.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.804+0000 7f2c4b400640 1 --2- 172.21.15.167:0/611974174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c30003a50 0x7f2c30003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:37.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.804+0000 7f2c4b400640 1 -- 172.21.15.167:0/611974174 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c30003520 con 0x7f2c300053c0 2024-05-11T12:14:37.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.804+0000 7f2c4aa00640 1 --2- 172.21.15.167:0/611974174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c30003a50 0x7f2c30003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:14:37.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.805+0000 7f2c4b400640 1 -- 172.21.15.167:0/611974174 reap_dead start 2024-05-11T12:14:37.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.805+0000 7f2c4b400640 1 --2- 172.21.15.167:0/611974174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c300053c0 0x7f2c3000b9c0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f2c44001e90 tx=0x7f2c30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4828a723a948cb60 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:37.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.805+0000 7f2c4a000640 1 -- 172.21.15.167:0/611974174 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c44001ab0 con 0x7f2c300053c0 2024-05-11T12:14:37.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.805+0000 7f2c4a000640 1 -- 172.21.15.167:0/611974174 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c4402ae10 con 0x7f2c300053c0 2024-05-11T12:14:37.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.806+0000 7f2c4a000640 1 -- 172.21.15.167:0/611974174 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c44003bb0 con 0x7f2c300053c0 2024-05-11T12:14:37.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.806+0000 7f2c4be00640 1 -- 172.21.15.167:0/611974174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c300053c0 msgr2=0x7f2c3000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:37.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.806+0000 7f2c4be00640 1 --2- 172.21.15.167:0/611974174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c300053c0 0x7f2c3000b9c0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f2c44001e90 tx=0x7f2c30005830 comp rx=0 tx=0).stop 2024-05-11T12:14:37.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.806+0000 7f2c4be00640 1 -- 172.21.15.167:0/611974174 shutdown_connections 2024-05-11T12:14:37.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.806+0000 7f2c4be00640 1 --2- 172.21.15.167:0/611974174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c300053c0 0x7f2c3000b9c0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:37.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.806+0000 7f2c4be00640 1 -- 172.21.15.167:0/611974174 >> 172.21.15.167:0/611974174 conn(0x7f2c4c0b3460 msgr2=0x7f2c4c0d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:37.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.806+0000 7f2c4be00640 1 -- 172.21.15.167:0/611974174 shutdown_connections 2024-05-11T12:14:37.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.806+0000 7f2c4be00640 1 -- 172.21.15.167:0/611974174 wait complete. 2024-05-11T12:14:37.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.808+0000 7f2c4be00640 1 Processor -- start 2024-05-11T12:14:37.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.808+0000 7f2c4be00640 1 -- start start 2024-05-11T12:14:37.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.809+0000 7f2c4be00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c300053c0 0x7f2c4c15fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:37.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.809+0000 7f2c4be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c4c1f5c10 0x7f2c4c071f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:37.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.809+0000 7f2c4be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c4c15c6b0 0x7f2c4c160900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:37.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.809+0000 7f2c4be00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4c0ceb50 con 0x7f2c4c15c6b0 2024-05-11T12:14:37.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.809+0000 7f2c4be00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4c0ce9d0 con 0x7f2c4c1f5c10 2024-05-11T12:14:37.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.809+0000 7f2c4be00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2c4c0cecd0 con 0x7f2c300053c0 2024-05-11T12:14:37.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.809+0000 7f2c4b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c4c15c6b0 0x7f2c4c160900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:37.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.809+0000 7f2c4b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c4c15c6b0 0x7f2c4c160900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:40486/0 (socket says 172.21.15.167:40486) 2024-05-11T12:14:37.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.809+0000 7f2c4b400640 1 -- 172.21.15.167:0/1861308815 learned_addr learned my addr 172.21.15.167:0/1861308815 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:37.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.810+0000 7f2c4aa00640 1 --2- 172.21.15.167:0/1861308815 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c300053c0 0x7f2c4c15fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:37.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.810+0000 7f2c43e00640 1 --2- 172.21.15.167:0/1861308815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c4c1f5c10 0x7f2c4c071f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:37.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.810+0000 7f2c4b400640 1 -- 172.21.15.167:0/1861308815 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c300053c0 msgr2=0x7f2c4c15fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:37.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.810+0000 7f2c4b400640 1 --2- 172.21.15.167:0/1861308815 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c300053c0 0x7f2c4c15fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:37.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.810+0000 7f2c4b400640 1 -- 172.21.15.167:0/1861308815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c4c1f5c10 msgr2=0x7f2c4c071f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:37.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.810+0000 7f2c4b400640 1 --2- 172.21.15.167:0/1861308815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c4c1f5c10 0x7f2c4c071f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:37.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.810+0000 7f2c4b400640 1 -- 172.21.15.167:0/1861308815 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c4c1f6090 con 0x7f2c4c15c6b0 2024-05-11T12:14:37.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.810+0000 7f2c4aa00640 1 --2- 172.21.15.167:0/1861308815 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c300053c0 0x7f2c4c15fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:37.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.810+0000 7f2c4b400640 1 --2- 172.21.15.167:0/1861308815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c4c15c6b0 0x7f2c4c160900 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f2c44001fb0 tx=0x7f2c440014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:37.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.810+0000 7f2c42a00640 1 -- 172.21.15.167:0/1861308815 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c44007df0 con 0x7f2c4c15c6b0 2024-05-11T12:14:37.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.810+0000 7f2c42a00640 1 -- 172.21.15.167:0/1861308815 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c4402ac10 con 0x7f2c4c15c6b0 2024-05-11T12:14:37.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.810+0000 7f2c4be00640 1 -- 172.21.15.167:0/1861308815 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c4c160310 con 0x7f2c4c15c6b0 2024-05-11T12:14:37.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.810+0000 7f2c42a00640 1 -- 172.21.15.167:0/1861308815 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c44001ab0 con 0x7f2c4c15c6b0 2024-05-11T12:14:37.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.811+0000 7f2c4be00640 1 -- 172.21.15.167:0/1861308815 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c4c15e4a0 con 0x7f2c4c15c6b0 2024-05-11T12:14:37.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.812+0000 7f2c4be00640 1 -- 172.21.15.167:0/1861308815 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c4c0694b0 con 0x7f2c4c15c6b0 2024-05-11T12:14:37.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.813+0000 7f2c42a00640 1 -- 172.21.15.167:0/1861308815 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2c44043020 con 0x7f2c4c15c6b0 2024-05-11T12:14:37.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.814+0000 7f2c42a00640 1 --2- 172.21.15.167:0/1861308815 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2c28076300 0x7f2c280787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:37.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.814+0000 7f2c42a00640 1 -- 172.21.15.167:0/1861308815 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6101+0+0 (secure 0 0 0) 0x7f2c440b6ba0 con 0x7f2c4c15c6b0 2024-05-11T12:14:37.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.814+0000 7f2c4aa00640 1 --2- 172.21.15.167:0/1861308815 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2c28076300 0x7f2c280787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:37.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.815+0000 7f2c4aa00640 1 --2- 172.21.15.167:0/1861308815 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2c28076300 0x7f2c280787c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c001eb0 tx=0x7f2c3c0014b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:37.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:37.818+0000 7f2c42a00640 1 -- 172.21.15.167:0/1861308815 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2c440804e0 con 0x7f2c4c15c6b0 2024-05-11T12:14:38.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.023+0000 7f2c4be00640 1 -- 172.21.15.167:0/1861308815 --> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2c4c072470 con 0x7f2c28076300 2024-05-11T12:14:38.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.030+0000 7f2c42a00640 1 -- 172.21.15.167:0/1861308815 <== mgr.4106 v2:172.21.15.167:6832/578352559 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248374 (secure 0 0 0) 0x7f2c4c072470 con 0x7f2c28076300 2024-05-11T12:14:38.031 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:38.034 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2024-05-11T12:14:38.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.036+0000 7f2c4be00640 1 -- 172.21.15.167:0/1861308815 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2c28076300 msgr2=0x7f2c280787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:38.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.037+0000 7f2c4be00640 1 --2- 172.21.15.167:0/1861308815 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2c28076300 0x7f2c280787c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c001eb0 tx=0x7f2c3c0014b0 comp rx=0 tx=0).stop 2024-05-11T12:14:38.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.037+0000 7f2c4be00640 1 -- 172.21.15.167:0/1861308815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c4c15c6b0 msgr2=0x7f2c4c160900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:38.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.037+0000 7f2c4be00640 1 --2- 172.21.15.167:0/1861308815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c4c15c6b0 0x7f2c4c160900 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f2c44001fb0 tx=0x7f2c440014a0 comp rx=0 tx=0).stop 2024-05-11T12:14:38.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.037+0000 7f2c4be00640 1 -- 172.21.15.167:0/1861308815 shutdown_connections 2024-05-11T12:14:38.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.037+0000 7f2c4be00640 1 --2- 172.21.15.167:0/1861308815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c4c15c6b0 0x7f2c4c160900 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:38.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.037+0000 7f2c4be00640 1 --2- 172.21.15.167:0/1861308815 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2c28076300 0x7f2c280787c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:38.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.037+0000 7f2c4be00640 1 --2- 172.21.15.167:0/1861308815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c4c1f5c10 0x7f2c4c071f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:38.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.037+0000 7f2c4be00640 1 --2- 172.21.15.167:0/1861308815 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c300053c0 0x7f2c4c15fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:38.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.037+0000 7f2c4be00640 1 -- 172.21.15.167:0/1861308815 >> 172.21.15.167:0/1861308815 conn(0x7f2c4c0b3460 msgr2=0x7f2c4c0d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:38.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.037+0000 7f2c4be00640 1 -- 172.21.15.167:0/1861308815 shutdown_connections 2024-05-11T12:14:38.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.037+0000 7f2c4be00640 1 -- 172.21.15.167:0/1861308815 wait complete. 2024-05-11T12:14:38.056 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":213,"stamp":"2024-05-11T12:14:37.307961+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590950,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":110,"num_write_kb":1629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82,"ondisk_log_size":82,"up":180,"acting":180,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":98,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259828,"kb_used_data":10352,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714892,"statfs":{"total":773094113280,"available":772828049408,"internally_reserved":0,"allocated":10600448,"data_stored":5438451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.745236"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615834+0000","last_change":"2024-05-11T12:14:33.622872+0000","last_active":"2024-05-11T12:14:34.615834+0000","last_peered":"2024-05-11T12:14:34.615834+0000","last_clean":"2024-05-11T12:14:34.615834+0000","last_became_active":"2024-05-11T12:14:33.619136+0000","last_became_peered":"2024-05-11T12:14:33.619136+0000","last_unstale":"2024-05-11T12:14:34.615834+0000","last_undegraded":"2024-05-11T12:14:34.615834+0000","last_fullsized":"2024-05-11T12:14:34.615834+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:18:44.962195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.601568+0000","last_change":"2024-05-11T12:14:32.601568+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.601568+0000","last_undegraded":"2024-05-11T12:14:32.601568+0000","last_fullsized":"2024-05-11T12:14:32.601568+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615616+0000","last_change":"2024-05-11T12:14:33.626436+0000","last_active":"2024-05-11T12:14:34.615616+0000","last_peered":"2024-05-11T12:14:34.615616+0000","last_clean":"2024-05-11T12:14:34.615616+0000","last_became_active":"2024-05-11T12:14:33.626091+0000","last_became_peered":"2024-05-11T12:14:33.626091+0000","last_unstale":"2024-05-11T12:14:34.615616+0000","last_undegraded":"2024-05-11T12:14:34.615616+0000","last_fullsized":"2024-05-11T12:14:34.615616+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:17:05.023315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.284831+0000","last_change":"2024-05-11T12:14:33.624886+0000","last_active":"2024-05-11T12:14:35.284831+0000","last_peered":"2024-05-11T12:14:35.284831+0000","last_clean":"2024-05-11T12:14:35.284831+0000","last_became_active":"2024-05-11T12:14:33.618826+0000","last_became_peered":"2024-05-11T12:14:33.618826+0000","last_unstale":"2024-05-11T12:14:35.284831+0000","last_undegraded":"2024-05-11T12:14:35.284831+0000","last_fullsized":"2024-05-11T12:14:35.284831+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:56:09.727119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T12:14:32.562048+0000","last_change":"2024-05-11T12:14:32.562048+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.562048+0000","last_undegraded":"2024-05-11T12:14:32.562048+0000","last_fullsized":"2024-05-11T12:14:32.562048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.598150+0000","last_change":"2024-05-11T12:14:32.598150+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.598150+0000","last_undegraded":"2024-05-11T12:14:32.598150+0000","last_fullsized":"2024-05-11T12:14:32.598150+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.602749+0000","last_change":"2024-05-11T12:14:32.602749+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.602749+0000","last_undegraded":"2024-05-11T12:14:32.602749+0000","last_fullsized":"2024-05-11T12:14:32.602749+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.208341+0000","last_change":"2024-05-11T12:14:33.621312+0000","last_active":"2024-05-11T12:14:35.208341+0000","last_peered":"2024-05-11T12:14:35.208341+0000","last_clean":"2024-05-11T12:14:35.208341+0000","last_became_active":"2024-05-11T12:14:33.621021+0000","last_became_peered":"2024-05-11T12:14:33.621021+0000","last_unstale":"2024-05-11T12:14:35.208341+0000","last_undegraded":"2024-05-11T12:14:35.208341+0000","last_fullsized":"2024-05-11T12:14:35.208341+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:54:45.084719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.595249+0000","last_change":"2024-05-11T12:14:32.595249+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.595249+0000","last_undegraded":"2024-05-11T12:14:32.595249+0000","last_fullsized":"2024-05-11T12:14:32.595249+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T12:14:32.562048+0000","last_change":"2024-05-11T12:14:32.562048+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.562048+0000","last_undegraded":"2024-05-11T12:14:32.562048+0000","last_fullsized":"2024-05-11T12:14:32.562048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.592511+0000","last_change":"2024-05-11T12:14:32.592511+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.592511+0000","last_undegraded":"2024-05-11T12:14:32.592511+0000","last_fullsized":"2024-05-11T12:14:32.592511+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.606336+0000","last_change":"2024-05-11T12:14:32.606336+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.606336+0000","last_undegraded":"2024-05-11T12:14:32.606336+0000","last_fullsized":"2024-05-11T12:14:32.606336+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.283623+0000","last_change":"2024-05-11T12:14:33.607961+0000","last_active":"2024-05-11T12:14:35.283623+0000","last_peered":"2024-05-11T12:14:35.283623+0000","last_clean":"2024-05-11T12:14:35.283623+0000","last_became_active":"2024-05-11T12:14:33.607599+0000","last_became_peered":"2024-05-11T12:14:33.607599+0000","last_unstale":"2024-05-11T12:14:35.283623+0000","last_undegraded":"2024-05-11T12:14:35.283623+0000","last_fullsized":"2024-05-11T12:14:35.283623+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:22:15.346218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.600693+0000","last_change":"2024-05-11T12:14:32.600693+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.600693+0000","last_undegraded":"2024-05-11T12:14:32.600693+0000","last_fullsized":"2024-05-11T12:14:32.600693+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.599082+0000","last_change":"2024-05-11T12:14:32.599082+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.599082+0000","last_undegraded":"2024-05-11T12:14:32.599082+0000","last_fullsized":"2024-05-11T12:14:32.599082+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.603185+0000","last_change":"2024-05-11T12:14:32.603185+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.603185+0000","last_undegraded":"2024-05-11T12:14:32.603185+0000","last_fullsized":"2024-05-11T12:14:32.603185+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615878+0000","last_change":"2024-05-11T12:14:33.622549+0000","last_active":"2024-05-11T12:14:34.615878+0000","last_peered":"2024-05-11T12:14:34.615878+0000","last_clean":"2024-05-11T12:14:34.615878+0000","last_became_active":"2024-05-11T12:14:33.619603+0000","last_became_peered":"2024-05-11T12:14:33.619603+0000","last_unstale":"2024-05-11T12:14:34.615878+0000","last_undegraded":"2024-05-11T12:14:34.615878+0000","last_fullsized":"2024-05-11T12:14:34.615878+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:51:30.951966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.600768+0000","last_change":"2024-05-11T12:14:32.600768+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.600768+0000","last_undegraded":"2024-05-11T12:14:32.600768+0000","last_fullsized":"2024-05-11T12:14:32.600768+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615768+0000","last_change":"2024-05-11T12:14:33.635269+0000","last_active":"2024-05-11T12:14:34.615768+0000","last_peered":"2024-05-11T12:14:34.615768+0000","last_clean":"2024-05-11T12:14:34.615768+0000","last_became_active":"2024-05-11T12:14:33.634795+0000","last_became_peered":"2024-05-11T12:14:33.634795+0000","last_unstale":"2024-05-11T12:14:34.615768+0000","last_undegraded":"2024-05-11T12:14:34.615768+0000","last_fullsized":"2024-05-11T12:14:34.615768+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:13:16.630356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.605840+0000","last_change":"2024-05-11T12:14:32.605840+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.605840+0000","last_undegraded":"2024-05-11T12:14:32.605840+0000","last_fullsized":"2024-05-11T12:14:32.605840+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.208740+0000","last_change":"2024-05-11T12:14:33.621162+0000","last_active":"2024-05-11T12:14:35.208740+0000","last_peered":"2024-05-11T12:14:35.208740+0000","last_clean":"2024-05-11T12:14:35.208740+0000","last_became_active":"2024-05-11T12:14:33.620843+0000","last_became_peered":"2024-05-11T12:14:33.620843+0000","last_unstale":"2024-05-11T12:14:35.208740+0000","last_undegraded":"2024-05-11T12:14:35.208740+0000","last_fullsized":"2024-05-11T12:14:35.208740+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:29:39.246958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T12:14:32.562048+0000","last_change":"2024-05-11T12:14:32.562048+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.562048+0000","last_undegraded":"2024-05-11T12:14:32.562048+0000","last_fullsized":"2024-05-11T12:14:32.562048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615236+0000","last_change":"2024-05-11T12:14:33.619917+0000","last_active":"2024-05-11T12:14:34.615236+0000","last_peered":"2024-05-11T12:14:34.615236+0000","last_clean":"2024-05-11T12:14:34.615236+0000","last_became_active":"2024-05-11T12:14:33.619614+0000","last_became_peered":"2024-05-11T12:14:33.619614+0000","last_unstale":"2024-05-11T12:14:34.615236+0000","last_undegraded":"2024-05-11T12:14:34.615236+0000","last_fullsized":"2024-05-11T12:14:34.615236+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:56:17.539601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T12:14:32.562048+0000","last_change":"2024-05-11T12:14:32.562048+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.562048+0000","last_undegraded":"2024-05-11T12:14:32.562048+0000","last_fullsized":"2024-05-11T12:14:32.562048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.607235+0000","last_change":"2024-05-11T12:14:32.607235+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.607235+0000","last_undegraded":"2024-05-11T12:14:32.607235+0000","last_fullsized":"2024-05-11T12:14:32.607235+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.601792+0000","last_change":"2024-05-11T12:14:32.601792+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.601792+0000","last_undegraded":"2024-05-11T12:14:32.601792+0000","last_fullsized":"2024-05-11T12:14:32.601792+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.591419+0000","last_change":"2024-05-11T12:14:32.591419+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.591419+0000","last_undegraded":"2024-05-11T12:14:32.591419+0000","last_fullsized":"2024-05-11T12:14:32.591419+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.595401+0000","last_change":"2024-05-11T12:14:32.595401+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.595401+0000","last_undegraded":"2024-05-11T12:14:32.595401+0000","last_fullsized":"2024-05-11T12:14:32.595401+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209481+0000","last_change":"2024-05-11T12:14:33.623708+0000","last_active":"2024-05-11T12:14:35.209481+0000","last_peered":"2024-05-11T12:14:35.209481+0000","last_clean":"2024-05-11T12:14:35.209481+0000","last_became_active":"2024-05-11T12:14:33.623429+0000","last_became_peered":"2024-05-11T12:14:33.623429+0000","last_unstale":"2024-05-11T12:14:35.209481+0000","last_undegraded":"2024-05-11T12:14:35.209481+0000","last_fullsized":"2024-05-11T12:14:35.209481+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:30:11.411742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.603561+0000","last_change":"2024-05-11T12:14:32.603561+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.603561+0000","last_undegraded":"2024-05-11T12:14:32.603561+0000","last_fullsized":"2024-05-11T12:14:32.603561+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.596127+0000","last_change":"2024-05-11T12:14:32.596127+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.596127+0000","last_undegraded":"2024-05-11T12:14:32.596127+0000","last_fullsized":"2024-05-11T12:14:32.596127+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.208839+0000","last_change":"2024-05-11T12:14:33.621606+0000","last_active":"2024-05-11T12:14:35.208839+0000","last_peered":"2024-05-11T12:14:35.208839+0000","last_clean":"2024-05-11T12:14:35.208839+0000","last_became_active":"2024-05-11T12:14:33.621288+0000","last_became_peered":"2024-05-11T12:14:33.621288+0000","last_unstale":"2024-05-11T12:14:35.208839+0000","last_undegraded":"2024-05-11T12:14:35.208839+0000","last_fullsized":"2024-05-11T12:14:35.208839+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:37:19.519120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615469+0000","last_change":"2024-05-11T12:14:33.633794+0000","last_active":"2024-05-11T12:14:34.615469+0000","last_peered":"2024-05-11T12:14:34.615469+0000","last_clean":"2024-05-11T12:14:34.615469+0000","last_became_active":"2024-05-11T12:14:33.633451+0000","last_became_peered":"2024-05-11T12:14:33.633451+0000","last_unstale":"2024-05-11T12:14:34.615469+0000","last_undegraded":"2024-05-11T12:14:34.615469+0000","last_fullsized":"2024-05-11T12:14:34.615469+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:31:39.465305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"96'4","reported_seq":18,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.285496+0000","last_change":"2024-05-11T12:14:32.610045+0000","last_active":"2024-05-11T12:14:35.285496+0000","last_peered":"2024-05-11T12:14:35.285496+0000","last_clean":"2024-05-11T12:14:35.285496+0000","last_became_active":"2024-05-11T12:14:32.609746+0000","last_became_peered":"2024-05-11T12:14:32.609746+0000","last_unstale":"2024-05-11T12:14:35.285496+0000","last_undegraded":"2024-05-11T12:14:35.285496+0000","last_fullsized":"2024-05-11T12:14:35.285496+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:44:56.684605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.208579+0000","last_change":"2024-05-11T12:14:33.630664+0000","last_active":"2024-05-11T12:14:35.208579+0000","last_peered":"2024-05-11T12:14:35.208579+0000","last_clean":"2024-05-11T12:14:35.208579+0000","last_became_active":"2024-05-11T12:14:33.630171+0000","last_became_peered":"2024-05-11T12:14:33.630171+0000","last_unstale":"2024-05-11T12:14:35.208579+0000","last_undegraded":"2024-05-11T12:14:35.208579+0000","last_fullsized":"2024-05-11T12:14:35.208579+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:00:10.906551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209543+0000","last_change":"2024-05-11T12:14:32.619446+0000","last_active":"2024-05-11T12:14:35.209543+0000","last_peered":"2024-05-11T12:14:35.209543+0000","last_clean":"2024-05-11T12:14:35.209543+0000","last_became_active":"2024-05-11T12:14:32.619219+0000","last_became_peered":"2024-05-11T12:14:32.619219+0000","last_unstale":"2024-05-11T12:14:35.209543+0000","last_undegraded":"2024-05-11T12:14:35.209543+0000","last_fullsized":"2024-05-11T12:14:35.209543+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:34:15.881914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.603030+0000","last_change":"2024-05-11T12:14:32.603030+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.603030+0000","last_undegraded":"2024-05-11T12:14:32.603030+0000","last_fullsized":"2024-05-11T12:14:32.603030+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-11T12:14:32.621991+0000","last_change":"2024-05-11T12:14:32.618332+0000","last_active":"2024-05-11T12:14:32.621991+0000","last_peered":"2024-05-11T12:14:32.621991+0000","last_clean":"2024-05-11T12:14:32.621991+0000","last_became_active":"2024-05-11T12:14:32.614777+0000","last_became_peered":"2024-05-11T12:14:32.614777+0000","last_unstale":"2024-05-11T12:14:32.621991+0000","last_undegraded":"2024-05-11T12:14:32.621991+0000","last_fullsized":"2024-05-11T12:14:32.621991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:01:31.105667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.591054+0000","last_change":"2024-05-11T12:14:32.591054+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.591054+0000","last_undegraded":"2024-05-11T12:14:32.591054+0000","last_fullsized":"2024-05-11T12:14:32.591054+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-05-11T12:14:31.585654+0000","last_change":"2024-05-11T12:14:31.585654+0000","last_active":"2024-05-11T12:14:31.545700+0000","last_peered":"2024-05-11T12:14:31.545700+0000","last_clean":"2024-05-11T12:14:31.545700+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:31.585654+0000","last_undegraded":"2024-05-11T12:14:31.585654+0000","last_fullsized":"2024-05-11T12:14:31.585654+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.284849+0000","last_change":"2024-05-11T12:14:33.616596+0000","last_active":"2024-05-11T12:14:35.284849+0000","last_peered":"2024-05-11T12:14:35.284849+0000","last_clean":"2024-05-11T12:14:35.284849+0000","last_became_active":"2024-05-11T12:14:33.616294+0000","last_became_peered":"2024-05-11T12:14:33.616294+0000","last_unstale":"2024-05-11T12:14:35.284849+0000","last_undegraded":"2024-05-11T12:14:35.284849+0000","last_fullsized":"2024-05-11T12:14:35.284849+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:27:47.349240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-11T12:14:32.621828+0000","last_change":"2024-05-11T12:14:32.619606+0000","last_active":"2024-05-11T12:14:32.621828+0000","last_peered":"2024-05-11T12:14:32.621828+0000","last_clean":"2024-05-11T12:14:32.621828+0000","last_became_active":"2024-05-11T12:14:32.619133+0000","last_became_peered":"2024-05-11T12:14:32.619133+0000","last_unstale":"2024-05-11T12:14:32.621828+0000","last_undegraded":"2024-05-11T12:14:32.621828+0000","last_fullsized":"2024-05-11T12:14:32.621828+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:00:56.111300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.616178+0000","last_change":"2024-05-11T12:14:33.634226+0000","last_active":"2024-05-11T12:14:34.616178+0000","last_peered":"2024-05-11T12:14:34.616178+0000","last_clean":"2024-05-11T12:14:34.616178+0000","last_became_active":"2024-05-11T12:14:33.633932+0000","last_became_peered":"2024-05-11T12:14:33.633932+0000","last_unstale":"2024-05-11T12:14:34.616178+0000","last_undegraded":"2024-05-11T12:14:34.616178+0000","last_fullsized":"2024-05-11T12:14:34.616178+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:01:12.394223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-11T12:14:32.616593+0000","last_change":"2024-05-11T12:14:32.614159+0000","last_active":"2024-05-11T12:14:32.616593+0000","last_peered":"2024-05-11T12:14:32.616593+0000","last_clean":"2024-05-11T12:14:32.616593+0000","last_became_active":"2024-05-11T12:14:32.613421+0000","last_became_peered":"2024-05-11T12:14:32.613421+0000","last_unstale":"2024-05-11T12:14:32.616593+0000","last_undegraded":"2024-05-11T12:14:32.616593+0000","last_fullsized":"2024-05-11T12:14:32.616593+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:58:41.059975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.599834+0000","last_change":"2024-05-11T12:14:32.599834+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.599834+0000","last_undegraded":"2024-05-11T12:14:32.599834+0000","last_fullsized":"2024-05-11T12:14:32.599834+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-11T12:14:32.620324+0000","last_change":"2024-05-11T12:14:32.614638+0000","last_active":"2024-05-11T12:14:32.620324+0000","last_peered":"2024-05-11T12:14:32.620324+0000","last_clean":"2024-05-11T12:14:32.620324+0000","last_became_active":"2024-05-11T12:14:32.614242+0000","last_became_peered":"2024-05-11T12:14:32.614242+0000","last_unstale":"2024-05-11T12:14:32.620324+0000","last_undegraded":"2024-05-11T12:14:32.620324+0000","last_fullsized":"2024-05-11T12:14:32.620324+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:45:58.666366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.285649+0000","last_change":"2024-05-11T12:14:33.615167+0000","last_active":"2024-05-11T12:14:35.285649+0000","last_peered":"2024-05-11T12:14:35.285649+0000","last_clean":"2024-05-11T12:14:35.285649+0000","last_became_active":"2024-05-11T12:14:33.614898+0000","last_became_peered":"2024-05-11T12:14:33.614898+0000","last_unstale":"2024-05-11T12:14:35.285649+0000","last_undegraded":"2024-05-11T12:14:35.285649+0000","last_fullsized":"2024-05-11T12:14:35.285649+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:52:46.883659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.283863+0000","last_change":"2024-05-11T12:14:32.605627+0000","last_active":"2024-05-11T12:14:35.283863+0000","last_peered":"2024-05-11T12:14:35.283863+0000","last_clean":"2024-05-11T12:14:35.283863+0000","last_became_active":"2024-05-11T12:14:32.604729+0000","last_became_peered":"2024-05-11T12:14:32.604729+0000","last_unstale":"2024-05-11T12:14:35.283863+0000","last_undegraded":"2024-05-11T12:14:35.283863+0000","last_fullsized":"2024-05-11T12:14:35.283863+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:00:56.111300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615229+0000","last_change":"2024-05-11T12:14:33.635074+0000","last_active":"2024-05-11T12:14:34.615229+0000","last_peered":"2024-05-11T12:14:34.615229+0000","last_clean":"2024-05-11T12:14:34.615229+0000","last_became_active":"2024-05-11T12:14:33.634606+0000","last_became_peered":"2024-05-11T12:14:33.634606+0000","last_unstale":"2024-05-11T12:14:34.615229+0000","last_undegraded":"2024-05-11T12:14:34.615229+0000","last_fullsized":"2024-05-11T12:14:34.615229+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:03:07.386179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-11T12:14:32.620414+0000","last_change":"2024-05-11T12:14:32.615191+0000","last_active":"2024-05-11T12:14:32.620414+0000","last_peered":"2024-05-11T12:14:32.620414+0000","last_clean":"2024-05-11T12:14:32.620414+0000","last_became_active":"2024-05-11T12:14:32.614809+0000","last_became_peered":"2024-05-11T12:14:32.614809+0000","last_unstale":"2024-05-11T12:14:32.620414+0000","last_undegraded":"2024-05-11T12:14:32.620414+0000","last_fullsized":"2024-05-11T12:14:32.620414+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:15:55.175677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T12:14:32.562048+0000","last_change":"2024-05-11T12:14:32.562048+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.562048+0000","last_undegraded":"2024-05-11T12:14:32.562048+0000","last_fullsized":"2024-05-11T12:14:32.562048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615602+0000","last_change":"2024-05-11T12:14:32.609650+0000","last_active":"2024-05-11T12:14:34.615602+0000","last_peered":"2024-05-11T12:14:34.615602+0000","last_clean":"2024-05-11T12:14:34.615602+0000","last_became_active":"2024-05-11T12:14:32.609340+0000","last_became_peered":"2024-05-11T12:14:32.609340+0000","last_unstale":"2024-05-11T12:14:34.615602+0000","last_undegraded":"2024-05-11T12:14:34.615602+0000","last_fullsized":"2024-05-11T12:14:34.615602+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:40:56.223685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209330+0000","last_change":"2024-05-11T12:14:33.631013+0000","last_active":"2024-05-11T12:14:35.209330+0000","last_peered":"2024-05-11T12:14:35.209330+0000","last_clean":"2024-05-11T12:14:35.209330+0000","last_became_active":"2024-05-11T12:14:33.630698+0000","last_became_peered":"2024-05-11T12:14:33.630698+0000","last_unstale":"2024-05-11T12:14:35.209330+0000","last_undegraded":"2024-05-11T12:14:35.209330+0000","last_fullsized":"2024-05-11T12:14:35.209330+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:32:48.749691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-11T12:14:32.621778+0000","last_change":"2024-05-11T12:14:32.618526+0000","last_active":"2024-05-11T12:14:32.621778+0000","last_peered":"2024-05-11T12:14:32.621778+0000","last_clean":"2024-05-11T12:14:32.621778+0000","last_became_active":"2024-05-11T12:14:32.618198+0000","last_became_peered":"2024-05-11T12:14:32.618198+0000","last_unstale":"2024-05-11T12:14:32.621778+0000","last_undegraded":"2024-05-11T12:14:32.621778+0000","last_fullsized":"2024-05-11T12:14:32.621778+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:57:12.465310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T12:14:32.562048+0000","last_change":"2024-05-11T12:14:32.562048+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.562048+0000","last_undegraded":"2024-05-11T12:14:32.562048+0000","last_fullsized":"2024-05-11T12:14:32.562048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-11T12:14:32.613159+0000","last_change":"2024-05-11T12:14:32.607489+0000","last_active":"2024-05-11T12:14:32.613159+0000","last_peered":"2024-05-11T12:14:32.613159+0000","last_clean":"2024-05-11T12:14:32.613159+0000","last_became_active":"2024-05-11T12:14:32.606078+0000","last_became_peered":"2024-05-11T12:14:32.606078+0000","last_unstale":"2024-05-11T12:14:32.613159+0000","last_undegraded":"2024-05-11T12:14:32.613159+0000","last_fullsized":"2024-05-11T12:14:32.613159+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:07:30.136729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.591204+0000","last_change":"2024-05-11T12:14:32.591204+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.591204+0000","last_undegraded":"2024-05-11T12:14:32.591204+0000","last_fullsized":"2024-05-11T12:14:32.591204+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-05-11T12:14:31.584097+0000","last_change":"2024-05-11T12:14:31.584097+0000","last_active":"2024-05-11T12:14:31.545700+0000","last_peered":"2024-05-11T12:14:31.545700+0000","last_clean":"2024-05-11T12:14:31.545700+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:31.584097+0000","last_undegraded":"2024-05-11T12:14:31.584097+0000","last_fullsized":"2024-05-11T12:14:31.584097+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.594883+0000","last_change":"2024-05-11T12:14:32.594883+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.594883+0000","last_undegraded":"2024-05-11T12:14:32.594883+0000","last_fullsized":"2024-05-11T12:14:32.594883+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.284356+0000","last_change":"2024-05-11T12:14:32.616922+0000","last_active":"2024-05-11T12:14:35.284356+0000","last_peered":"2024-05-11T12:14:35.284356+0000","last_clean":"2024-05-11T12:14:35.284356+0000","last_became_active":"2024-05-11T12:14:32.616599+0000","last_became_peered":"2024-05-11T12:14:32.616599+0000","last_unstale":"2024-05-11T12:14:35.284356+0000","last_undegraded":"2024-05-11T12:14:35.284356+0000","last_fullsized":"2024-05-11T12:14:35.284356+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:55:59.198545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209297+0000","last_change":"2024-05-11T12:14:33.624455+0000","last_active":"2024-05-11T12:14:35.209297+0000","last_peered":"2024-05-11T12:14:35.209297+0000","last_clean":"2024-05-11T12:14:35.209297+0000","last_became_active":"2024-05-11T12:14:33.624158+0000","last_became_peered":"2024-05-11T12:14:33.624158+0000","last_unstale":"2024-05-11T12:14:35.209297+0000","last_undegraded":"2024-05-11T12:14:35.209297+0000","last_fullsized":"2024-05-11T12:14:35.209297+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:44:32.910714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.208576+0000","last_change":"2024-05-11T12:14:32.615486+0000","last_active":"2024-05-11T12:14:35.208576+0000","last_peered":"2024-05-11T12:14:35.208576+0000","last_clean":"2024-05-11T12:14:35.208576+0000","last_became_active":"2024-05-11T12:14:32.615204+0000","last_became_peered":"2024-05-11T12:14:32.615204+0000","last_unstale":"2024-05-11T12:14:35.208576+0000","last_undegraded":"2024-05-11T12:14:35.208576+0000","last_fullsized":"2024-05-11T12:14:35.208576+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:27:00.766407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615938+0000","last_change":"2024-05-11T12:14:33.623140+0000","last_active":"2024-05-11T12:14:34.615938+0000","last_peered":"2024-05-11T12:14:34.615938+0000","last_clean":"2024-05-11T12:14:34.615938+0000","last_became_active":"2024-05-11T12:14:33.618444+0000","last_became_peered":"2024-05-11T12:14:33.618444+0000","last_unstale":"2024-05-11T12:14:34.615938+0000","last_undegraded":"2024-05-11T12:14:34.615938+0000","last_fullsized":"2024-05-11T12:14:34.615938+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:35:51.107577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-11T12:14:32.620528+0000","last_change":"2024-05-11T12:14:32.616263+0000","last_active":"2024-05-11T12:14:32.620528+0000","last_peered":"2024-05-11T12:14:32.620528+0000","last_clean":"2024-05-11T12:14:32.620528+0000","last_became_active":"2024-05-11T12:14:32.615004+0000","last_became_peered":"2024-05-11T12:14:32.615004+0000","last_unstale":"2024-05-11T12:14:32.620528+0000","last_undegraded":"2024-05-11T12:14:32.620528+0000","last_fullsized":"2024-05-11T12:14:32.620528+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:07:30.136729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.208800+0000","last_change":"2024-05-11T12:14:33.616435+0000","last_active":"2024-05-11T12:14:35.208800+0000","last_peered":"2024-05-11T12:14:35.208800+0000","last_clean":"2024-05-11T12:14:35.208800+0000","last_became_active":"2024-05-11T12:14:33.616110+0000","last_became_peered":"2024-05-11T12:14:33.616110+0000","last_unstale":"2024-05-11T12:14:35.208800+0000","last_undegraded":"2024-05-11T12:14:35.208800+0000","last_fullsized":"2024-05-11T12:14:35.208800+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:51:10.706767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-11T12:14:32.616852+0000","last_change":"2024-05-11T12:14:32.613751+0000","last_active":"2024-05-11T12:14:32.616852+0000","last_peered":"2024-05-11T12:14:32.616852+0000","last_clean":"2024-05-11T12:14:32.616852+0000","last_became_active":"2024-05-11T12:14:32.613431+0000","last_became_peered":"2024-05-11T12:14:32.613431+0000","last_unstale":"2024-05-11T12:14:32.616852+0000","last_undegraded":"2024-05-11T12:14:32.616852+0000","last_fullsized":"2024-05-11T12:14:32.616852+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:47:05.267174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.597272+0000","last_change":"2024-05-11T12:14:32.597272+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.597272+0000","last_undegraded":"2024-05-11T12:14:32.597272+0000","last_fullsized":"2024-05-11T12:14:32.597272+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-11T12:14:32.626697+0000","last_change":"2024-05-11T12:14:32.621615+0000","last_active":"2024-05-11T12:14:32.626697+0000","last_peered":"2024-05-11T12:14:32.626697+0000","last_clean":"2024-05-11T12:14:32.626697+0000","last_became_active":"2024-05-11T12:14:32.620973+0000","last_became_peered":"2024-05-11T12:14:32.620973+0000","last_unstale":"2024-05-11T12:14:32.626697+0000","last_undegraded":"2024-05-11T12:14:32.626697+0000","last_fullsized":"2024-05-11T12:14:32.626697+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:44:56.684605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T12:14:32.562048+0000","last_change":"2024-05-11T12:14:32.562048+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.562048+0000","last_undegraded":"2024-05-11T12:14:32.562048+0000","last_fullsized":"2024-05-11T12:14:32.562048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-11T12:14:32.612302+0000","last_change":"2024-05-11T12:14:32.609813+0000","last_active":"2024-05-11T12:14:32.612302+0000","last_peered":"2024-05-11T12:14:32.612302+0000","last_clean":"2024-05-11T12:14:32.612302+0000","last_became_active":"2024-05-11T12:14:32.609531+0000","last_became_peered":"2024-05-11T12:14:32.609531+0000","last_unstale":"2024-05-11T12:14:32.612302+0000","last_undegraded":"2024-05-11T12:14:32.612302+0000","last_fullsized":"2024-05-11T12:14:32.612302+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:54:11.073754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615796+0000","last_change":"2024-05-11T12:14:33.629837+0000","last_active":"2024-05-11T12:14:34.615796+0000","last_peered":"2024-05-11T12:14:34.615796+0000","last_clean":"2024-05-11T12:14:34.615796+0000","last_became_active":"2024-05-11T12:14:33.629561+0000","last_became_peered":"2024-05-11T12:14:33.629561+0000","last_unstale":"2024-05-11T12:14:34.615796+0000","last_undegraded":"2024-05-11T12:14:34.615796+0000","last_fullsized":"2024-05-11T12:14:34.615796+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:24:04.957521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209466+0000","last_change":"2024-05-11T12:14:32.605666+0000","last_active":"2024-05-11T12:14:35.209466+0000","last_peered":"2024-05-11T12:14:35.209466+0000","last_clean":"2024-05-11T12:14:35.209466+0000","last_became_active":"2024-05-11T12:14:32.605407+0000","last_became_peered":"2024-05-11T12:14:32.605407+0000","last_unstale":"2024-05-11T12:14:35.209466+0000","last_undegraded":"2024-05-11T12:14:35.209466+0000","last_fullsized":"2024-05-11T12:14:35.209466+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:31:24.122325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615946+0000","last_change":"2024-05-11T12:14:33.623726+0000","last_active":"2024-05-11T12:14:34.615946+0000","last_peered":"2024-05-11T12:14:34.615946+0000","last_clean":"2024-05-11T12:14:34.615946+0000","last_became_active":"2024-05-11T12:14:33.616682+0000","last_became_peered":"2024-05-11T12:14:33.616682+0000","last_unstale":"2024-05-11T12:14:34.615946+0000","last_undegraded":"2024-05-11T12:14:34.615946+0000","last_fullsized":"2024-05-11T12:14:34.615946+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:11:09.187757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.208810+0000","last_change":"2024-05-11T12:14:32.612947+0000","last_active":"2024-05-11T12:14:35.208810+0000","last_peered":"2024-05-11T12:14:35.208810+0000","last_clean":"2024-05-11T12:14:35.208810+0000","last_became_active":"2024-05-11T12:14:32.612545+0000","last_became_peered":"2024-05-11T12:14:32.612545+0000","last_unstale":"2024-05-11T12:14:35.208810+0000","last_undegraded":"2024-05-11T12:14:35.208810+0000","last_fullsized":"2024-05-11T12:14:35.208810+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:10:08.253641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.285245+0000","last_change":"2024-05-11T12:14:33.628473+0000","last_active":"2024-05-11T12:14:35.285245+0000","last_peered":"2024-05-11T12:14:35.285245+0000","last_clean":"2024-05-11T12:14:35.285245+0000","last_became_active":"2024-05-11T12:14:33.627301+0000","last_became_peered":"2024-05-11T12:14:33.627301+0000","last_unstale":"2024-05-11T12:14:35.285245+0000","last_undegraded":"2024-05-11T12:14:35.285245+0000","last_fullsized":"2024-05-11T12:14:35.285245+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:16:12.500928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-11T12:14:32.613383+0000","last_change":"2024-05-11T12:14:32.607325+0000","last_active":"2024-05-11T12:14:32.613383+0000","last_peered":"2024-05-11T12:14:32.613383+0000","last_clean":"2024-05-11T12:14:32.613383+0000","last_became_active":"2024-05-11T12:14:32.606982+0000","last_became_peered":"2024-05-11T12:14:32.606982+0000","last_unstale":"2024-05-11T12:14:32.613383+0000","last_undegraded":"2024-05-11T12:14:32.613383+0000","last_fullsized":"2024-05-11T12:14:32.613383+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:15:55.175677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209260+0000","last_change":"2024-05-11T12:14:33.618158+0000","last_active":"2024-05-11T12:14:35.209260+0000","last_peered":"2024-05-11T12:14:35.209260+0000","last_clean":"2024-05-11T12:14:35.209260+0000","last_became_active":"2024-05-11T12:14:33.617850+0000","last_became_peered":"2024-05-11T12:14:33.617850+0000","last_unstale":"2024-05-11T12:14:35.209260+0000","last_undegraded":"2024-05-11T12:14:35.209260+0000","last_fullsized":"2024-05-11T12:14:35.209260+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:38:25.159336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209121+0000","last_change":"2024-05-11T12:14:32.607439+0000","last_active":"2024-05-11T12:14:35.209121+0000","last_peered":"2024-05-11T12:14:35.209121+0000","last_clean":"2024-05-11T12:14:35.209121+0000","last_became_active":"2024-05-11T12:14:32.606952+0000","last_became_peered":"2024-05-11T12:14:32.606952+0000","last_unstale":"2024-05-11T12:14:35.209121+0000","last_undegraded":"2024-05-11T12:14:35.209121+0000","last_fullsized":"2024-05-11T12:14:35.209121+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:14:57.743590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615535+0000","last_change":"2024-05-11T12:14:33.619178+0000","last_active":"2024-05-11T12:14:34.615535+0000","last_peered":"2024-05-11T12:14:34.615535+0000","last_clean":"2024-05-11T12:14:34.615535+0000","last_became_active":"2024-05-11T12:14:33.618894+0000","last_became_peered":"2024-05-11T12:14:33.618894+0000","last_unstale":"2024-05-11T12:14:34.615535+0000","last_undegraded":"2024-05-11T12:14:34.615535+0000","last_fullsized":"2024-05-11T12:14:34.615535+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:10:50.170349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-05-11T12:14:31.584956+0000","last_change":"2024-05-11T12:14:31.584956+0000","last_active":"2024-05-11T12:14:31.545700+0000","last_peered":"2024-05-11T12:14:31.545700+0000","last_clean":"2024-05-11T12:14:31.545700+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:31.584956+0000","last_undegraded":"2024-05-11T12:14:31.584956+0000","last_fullsized":"2024-05-11T12:14:31.584956+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.602318+0000","last_change":"2024-05-11T12:14:32.602318+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.602318+0000","last_undegraded":"2024-05-11T12:14:32.602318+0000","last_fullsized":"2024-05-11T12:14:32.602318+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"97'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615678+0000","last_change":"2024-05-11T12:14:32.605888+0000","last_active":"2024-05-11T12:14:34.615678+0000","last_peered":"2024-05-11T12:14:34.615678+0000","last_clean":"2024-05-11T12:14:34.615678+0000","last_became_active":"2024-05-11T12:14:32.605588+0000","last_became_peered":"2024-05-11T12:14:32.605588+0000","last_unstale":"2024-05-11T12:14:34.615678+0000","last_undegraded":"2024-05-11T12:14:34.615678+0000","last_fullsized":"2024-05-11T12:14:34.615678+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:56:02.427960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.284207+0000","last_change":"2024-05-11T12:14:33.627479+0000","last_active":"2024-05-11T12:14:35.284207+0000","last_peered":"2024-05-11T12:14:35.284207+0000","last_clean":"2024-05-11T12:14:35.284207+0000","last_became_active":"2024-05-11T12:14:33.627179+0000","last_became_peered":"2024-05-11T12:14:33.627179+0000","last_unstale":"2024-05-11T12:14:35.284207+0000","last_undegraded":"2024-05-11T12:14:35.284207+0000","last_fullsized":"2024-05-11T12:14:35.284207+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:35:20.045762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-11T12:14:32.612410+0000","last_change":"2024-05-11T12:14:32.605594+0000","last_active":"2024-05-11T12:14:32.612410+0000","last_peered":"2024-05-11T12:14:32.612410+0000","last_clean":"2024-05-11T12:14:32.612410+0000","last_became_active":"2024-05-11T12:14:32.605097+0000","last_became_peered":"2024-05-11T12:14:32.605097+0000","last_unstale":"2024-05-11T12:14:32.612410+0000","last_undegraded":"2024-05-11T12:14:32.612410+0000","last_fullsized":"2024-05-11T12:14:32.612410+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:05:15.832383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.592181+0000","last_change":"2024-05-11T12:14:32.592181+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.592181+0000","last_undegraded":"2024-05-11T12:14:32.592181+0000","last_fullsized":"2024-05-11T12:14:32.592181+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-05-11T12:14:31.584482+0000","last_change":"2024-05-11T12:14:31.584482+0000","last_active":"2024-05-11T12:14:31.545700+0000","last_peered":"2024-05-11T12:14:31.545700+0000","last_clean":"2024-05-11T12:14:31.545700+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:31.584482+0000","last_undegraded":"2024-05-11T12:14:31.584482+0000","last_fullsized":"2024-05-11T12:14:31.584482+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.208801+0000","last_change":"2024-05-11T12:14:33.630741+0000","last_active":"2024-05-11T12:14:35.208801+0000","last_peered":"2024-05-11T12:14:35.208801+0000","last_clean":"2024-05-11T12:14:35.208801+0000","last_became_active":"2024-05-11T12:14:33.630423+0000","last_became_peered":"2024-05-11T12:14:33.630423+0000","last_unstale":"2024-05-11T12:14:35.208801+0000","last_undegraded":"2024-05-11T12:14:35.208801+0000","last_fullsized":"2024-05-11T12:14:35.208801+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:47:57.386158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-11T12:14:32.621836+0000","last_change":"2024-05-11T12:14:32.616942+0000","last_active":"2024-05-11T12:14:32.621836+0000","last_peered":"2024-05-11T12:14:32.621836+0000","last_clean":"2024-05-11T12:14:32.621836+0000","last_became_active":"2024-05-11T12:14:32.616577+0000","last_became_peered":"2024-05-11T12:14:32.616577+0000","last_unstale":"2024-05-11T12:14:32.621836+0000","last_undegraded":"2024-05-11T12:14:32.621836+0000","last_fullsized":"2024-05-11T12:14:32.621836+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:47:05.267174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.208768+0000","last_change":"2024-05-11T12:14:33.620388+0000","last_active":"2024-05-11T12:14:35.208768+0000","last_peered":"2024-05-11T12:14:35.208768+0000","last_clean":"2024-05-11T12:14:35.208768+0000","last_became_active":"2024-05-11T12:14:33.618621+0000","last_became_peered":"2024-05-11T12:14:33.618621+0000","last_unstale":"2024-05-11T12:14:35.208768+0000","last_undegraded":"2024-05-11T12:14:35.208768+0000","last_fullsized":"2024-05-11T12:14:35.208768+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:57:26.186975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209282+0000","last_change":"2024-05-11T12:14:32.615893+0000","last_active":"2024-05-11T12:14:35.209282+0000","last_peered":"2024-05-11T12:14:35.209282+0000","last_clean":"2024-05-11T12:14:35.209282+0000","last_became_active":"2024-05-11T12:14:32.615639+0000","last_became_peered":"2024-05-11T12:14:32.615639+0000","last_unstale":"2024-05-11T12:14:35.209282+0000","last_undegraded":"2024-05-11T12:14:35.209282+0000","last_fullsized":"2024-05-11T12:14:35.209282+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:51:28.062961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.601116+0000","last_change":"2024-05-11T12:14:32.601116+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.601116+0000","last_undegraded":"2024-05-11T12:14:32.601116+0000","last_fullsized":"2024-05-11T12:14:32.601116+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-11T12:14:32.613358+0000","last_change":"2024-05-11T12:14:32.607229+0000","last_active":"2024-05-11T12:14:32.613358+0000","last_peered":"2024-05-11T12:14:32.613358+0000","last_clean":"2024-05-11T12:14:32.613358+0000","last_became_active":"2024-05-11T12:14:32.606938+0000","last_became_peered":"2024-05-11T12:14:32.606938+0000","last_unstale":"2024-05-11T12:14:32.613358+0000","last_undegraded":"2024-05-11T12:14:32.613358+0000","last_fullsized":"2024-05-11T12:14:32.613358+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:45:58.666366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.593567+0000","last_change":"2024-05-11T12:14:32.593567+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.593567+0000","last_undegraded":"2024-05-11T12:14:32.593567+0000","last_fullsized":"2024-05-11T12:14:32.593567+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209088+0000","last_change":"2024-05-11T12:14:32.610511+0000","last_active":"2024-05-11T12:14:35.209088+0000","last_peered":"2024-05-11T12:14:35.209088+0000","last_clean":"2024-05-11T12:14:35.209088+0000","last_became_active":"2024-05-11T12:14:32.610243+0000","last_became_peered":"2024-05-11T12:14:32.610243+0000","last_unstale":"2024-05-11T12:14:35.209088+0000","last_undegraded":"2024-05-11T12:14:35.209088+0000","last_fullsized":"2024-05-11T12:14:35.209088+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:39:24.599004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.589733+0000","last_change":"2024-05-11T12:14:32.589733+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.589733+0000","last_undegraded":"2024-05-11T12:14:32.589733+0000","last_fullsized":"2024-05-11T12:14:32.589733+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209535+0000","last_change":"2024-05-11T12:14:32.612907+0000","last_active":"2024-05-11T12:14:35.209535+0000","last_peered":"2024-05-11T12:14:35.209535+0000","last_clean":"2024-05-11T12:14:35.209535+0000","last_became_active":"2024-05-11T12:14:32.612548+0000","last_became_peered":"2024-05-11T12:14:32.612548+0000","last_unstale":"2024-05-11T12:14:35.209535+0000","last_undegraded":"2024-05-11T12:14:35.209535+0000","last_fullsized":"2024-05-11T12:14:35.209535+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:49:45.457016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":243,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-11T12:14:32.605441+0000","last_change":"2024-05-11T12:10:35.066673+0000","last_active":"2024-05-11T12:14:32.605441+0000","last_peered":"2024-05-11T12:14:32.605441+0000","last_clean":"2024-05-11T12:14:32.605441+0000","last_became_active":"2024-05-11T12:10:35.065942+0000","last_became_peered":"2024-05-11T12:10:35.065942+0000","last_unstale":"2024-05-11T12:14:32.605441+0000","last_undegraded":"2024-05-11T12:14:32.605441+0000","last_fullsized":"2024-05-11T12:14:32.605441+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_clean_scrub_stamp":"2024-05-11T12:10:33.601136+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:12:02.133182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":114,"acting":114,"num_store_stats":7},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476787,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32900,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338940,"statfs":{"total":96636764160,"available":96603074560,"internally_reserved":0,"allocated":1761280,"data_stored":1118705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476788,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32324,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1167360,"data_stored":528337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476787,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32324,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1167360,"data_stored":528337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476789,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32324,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1167360,"data_stored":528337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509493,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1196032,"data_stored":542778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509493,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32304,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339536,"statfs":{"total":96636764160,"available":96603684864,"internally_reserved":0,"allocated":1142784,"data_stored":515461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509493,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32948,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338892,"statfs":{"total":96636764160,"available":96603025408,"internally_reserved":0,"allocated":1802240,"data_stored":1134266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542197,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1196032,"data_stored":542230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T12:14:38.060 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T12:14:38.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.302+0000 7f953a400640 1 Processor -- start 2024-05-11T12:14:38.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.302+0000 7f953a400640 1 -- start start 2024-05-11T12:14:38.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.304+0000 7f953a400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9534146770 0x7f95340ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:38.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.304+0000 7f953a400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f95340cef10 0x7f953406c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:38.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.304+0000 7f953a400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f953406fbc0 con 0x7f9534145b00 2024-05-11T12:14:38.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.304+0000 7f953a400640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f95340b4520 con 0x7f9534146770 2024-05-11T12:14:38.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.304+0000 7f953a400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f95340b4800 con 0x7f95340cef10 2024-05-11T12:14:38.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.304+0000 7f9533e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9534146770 0x7f95340ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:38.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.304+0000 7f9533e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9534146770 0x7f95340ce9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:40498/0 (socket says 172.21.15.167:40498) 2024-05-11T12:14:38.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.304+0000 7f9533e00640 1 -- 172.21.15.167:0/2861815395 learned_addr learned my addr 172.21.15.167:0/2861815395 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:38.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.304+0000 7f9539a00640 1 --2- 172.21.15.167:0/2861815395 >> v2:172.21.15.204:3300/0 conn(0x7f95340cef10 0x7f953406c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:38.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.305+0000 7f9533e00640 1 -- 172.21.15.167:0/2861815395 >> v2:172.21.15.204:3300/0 conn(0x7f95340cef10 msgr2=0x7f953406c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:38.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.305+0000 7f9533e00640 1 --2- 172.21.15.167:0/2861815395 >> v2:172.21.15.204:3300/0 conn(0x7f95340cef10 0x7f953406c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:38.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.305+0000 7f9533e00640 1 -- 172.21.15.167:0/2861815395 >> v1:172.21.15.204:6789/0 conn(0x7f9534145b00 legacy=0x7f953406f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:38.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.305+0000 7f9533e00640 1 -- 172.21.15.167:0/2861815395 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95340b4ae0 con 0x7f9534146770 2024-05-11T12:14:38.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.306+0000 7f9533e00640 1 --2- 172.21.15.167:0/2861815395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9534146770 0x7f95340ce9d0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f953406e010 tx=0x7f952002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c140903a667ab57b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:38.306 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.306+0000 7f9533400640 1 -- 172.21.15.167:0/2861815395 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9520037070 con 0x7f9534146770 2024-05-11T12:14:38.306 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.307+0000 7f9533400640 1 -- 172.21.15.167:0/2861815395 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9520032440 con 0x7f9534146770 2024-05-11T12:14:38.306 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.307+0000 7f9533400640 1 -- 172.21.15.167:0/2861815395 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9520032810 con 0x7f9534146770 2024-05-11T12:14:38.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.307+0000 7f953a400640 1 -- 172.21.15.167:0/2861815395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9534146770 msgr2=0x7f95340ce9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:38.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.307+0000 7f953a400640 1 --2- 172.21.15.167:0/2861815395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9534146770 0x7f95340ce9d0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f953406e010 tx=0x7f952002a140 comp rx=0 tx=0).stop 2024-05-11T12:14:38.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.307+0000 7f953a400640 1 -- 172.21.15.167:0/2861815395 shutdown_connections 2024-05-11T12:14:38.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.307+0000 7f953a400640 1 --2- 172.21.15.167:0/2861815395 >> v2:172.21.15.204:3300/0 conn(0x7f95340cef10 0x7f953406c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:38.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.307+0000 7f953a400640 1 --2- 172.21.15.167:0/2861815395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9534146770 0x7f95340ce9d0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:38.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.307+0000 7f953a400640 1 -- 172.21.15.167:0/2861815395 >> 172.21.15.167:0/2861815395 conn(0x7f95340b3460 msgr2=0x7f95340d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:38.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.307+0000 7f953a400640 1 -- 172.21.15.167:0/2861815395 shutdown_connections 2024-05-11T12:14:38.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.307+0000 7f953a400640 1 -- 172.21.15.167:0/2861815395 wait complete. 2024-05-11T12:14:38.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.309+0000 7f953a400640 1 Processor -- start 2024-05-11T12:14:38.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.309+0000 7f953a400640 1 -- start start 2024-05-11T12:14:38.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.310+0000 7f953a400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95340cef10 0x7f953413d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:38.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.310+0000 7f953a400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9534145b00 0x7f953415df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:38.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.310+0000 7f953a400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9534146770 0x7f953415f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:38.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.310+0000 7f953a400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f953406cd20 con 0x7f9534146770 2024-05-11T12:14:38.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.310+0000 7f953a400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f953406cba0 con 0x7f9534145b00 2024-05-11T12:14:38.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.310+0000 7f953a400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f953406cea0 con 0x7f95340cef10 2024-05-11T12:14:38.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.310+0000 7f9539a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9534146770 0x7f953415f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:38.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.311+0000 7f9539a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9534146770 0x7f953415f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:40514/0 (socket says 172.21.15.167:40514) 2024-05-11T12:14:38.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.311+0000 7f9539a00640 1 -- 172.21.15.167:0/97985514 learned_addr learned my addr 172.21.15.167:0/97985514 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:38.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.311+0000 7f9533e00640 1 --2- 172.21.15.167:0/97985514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9534145b00 0x7f953415df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:38.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.311+0000 7f9539000640 1 --2- 172.21.15.167:0/97985514 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95340cef10 0x7f953413d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:38.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.311+0000 7f9539a00640 1 -- 172.21.15.167:0/97985514 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95340cef10 msgr2=0x7f953413d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:38.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.311+0000 7f9539a00640 1 --2- 172.21.15.167:0/97985514 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95340cef10 0x7f953413d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:38.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.311+0000 7f9539a00640 1 -- 172.21.15.167:0/97985514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9534145b00 msgr2=0x7f953415df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:38.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.311+0000 7f9539a00640 1 --2- 172.21.15.167:0/97985514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9534145b00 0x7f953415df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:38.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.311+0000 7f9539a00640 1 -- 172.21.15.167:0/97985514 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f953415e600 con 0x7f9534146770 2024-05-11T12:14:38.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.311+0000 7f9533e00640 1 --2- 172.21.15.167:0/97985514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9534145b00 0x7f953415df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:38.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.311+0000 7f9539a00640 1 --2- 172.21.15.167:0/97985514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9534146770 0x7f953415f6a0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f952c007e70 tx=0x7f952c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:38.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.311+0000 7f9530c00640 1 -- 172.21.15.167:0/97985514 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c014620 con 0x7f9534146770 2024-05-11T12:14:38.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.312+0000 7f9530c00640 1 -- 172.21.15.167:0/97985514 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f952c019070 con 0x7f9534146770 2024-05-11T12:14:38.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.312+0000 7f9530c00640 1 -- 172.21.15.167:0/97985514 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c014a20 con 0x7f9534146770 2024-05-11T12:14:38.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.312+0000 7f953a400640 1 -- 172.21.15.167:0/97985514 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f953415e8e0 con 0x7f9534146770 2024-05-11T12:14:38.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.312+0000 7f953a400640 1 -- 172.21.15.167:0/97985514 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f953415eea0 con 0x7f9534146770 2024-05-11T12:14:38.313 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.313+0000 7f953a400640 1 -- 172.21.15.167:0/97985514 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95340694b0 con 0x7f9534146770 2024-05-11T12:14:38.313 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.314+0000 7f9530c00640 1 -- 172.21.15.167:0/97985514 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f952c014bc0 con 0x7f9534146770 2024-05-11T12:14:38.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.315+0000 7f9530c00640 1 --2- 172.21.15.167:0/97985514 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f9514076300 0x7f95140787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:38.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.315+0000 7f9530c00640 1 -- 172.21.15.167:0/97985514 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6101+0+0 (secure 0 0 0) 0x7f952c098100 con 0x7f9534146770 2024-05-11T12:14:38.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.315+0000 7f9539000640 1 --2- 172.21.15.167:0/97985514 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f9514076300 0x7f95140787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:38.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.316+0000 7f9539000640 1 --2- 172.21.15.167:0/97985514 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f9514076300 0x7f95140787c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f95280015d0 tx=0x7f9528009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:38.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.319+0000 7f9530c00640 1 -- 172.21.15.167:0/97985514 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f952c061830 con 0x7f9534146770 2024-05-11T12:14:38.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.505+0000 7f953a400640 1 -- 172.21.15.167:0/97985514 --> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9534145100 con 0x7f9514076300 2024-05-11T12:14:38.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.520+0000 7f9530c00640 1 -- 172.21.15.167:0/97985514 <== mgr.4106 v2:172.21.15.167:6832/578352559 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248374 (secure 0 0 0) 0x7f9534145100 con 0x7f9514076300 2024-05-11T12:14:38.523 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:38.524 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2024-05-11T12:14:38.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.526+0000 7f953a400640 1 -- 172.21.15.167:0/97985514 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f9514076300 msgr2=0x7f95140787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:38.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.526+0000 7f953a400640 1 --2- 172.21.15.167:0/97985514 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f9514076300 0x7f95140787c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f95280015d0 tx=0x7f9528009040 comp rx=0 tx=0).stop 2024-05-11T12:14:38.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.526+0000 7f953a400640 1 -- 172.21.15.167:0/97985514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9534146770 msgr2=0x7f953415f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:38.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.526+0000 7f953a400640 1 --2- 172.21.15.167:0/97985514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9534146770 0x7f953415f6a0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f952c007e70 tx=0x7f952c00e4b0 comp rx=0 tx=0).stop 2024-05-11T12:14:38.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.526+0000 7f953a400640 1 -- 172.21.15.167:0/97985514 shutdown_connections 2024-05-11T12:14:38.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.526+0000 7f953a400640 1 --2- 172.21.15.167:0/97985514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9534146770 0x7f953415f6a0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:38.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.526+0000 7f953a400640 1 --2- 172.21.15.167:0/97985514 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f9514076300 0x7f95140787c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:38.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.526+0000 7f953a400640 1 --2- 172.21.15.167:0/97985514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9534145b00 0x7f953415df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:38.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.526+0000 7f953a400640 1 --2- 172.21.15.167:0/97985514 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95340cef10 0x7f953413d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:38.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.526+0000 7f953a400640 1 -- 172.21.15.167:0/97985514 >> 172.21.15.167:0/97985514 conn(0x7f95340b3460 msgr2=0x7f95340cf440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:38.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.526+0000 7f953a400640 1 -- 172.21.15.167:0/97985514 shutdown_connections 2024-05-11T12:14:38.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.526+0000 7f953a400640 1 -- 172.21.15.167:0/97985514 wait complete. 2024-05-11T12:14:38.544 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":213,"stamp":"2024-05-11T12:14:37.307961+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590950,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":110,"num_write_kb":1629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82,"ondisk_log_size":82,"up":180,"acting":180,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":98,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259828,"kb_used_data":10352,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714892,"statfs":{"total":773094113280,"available":772828049408,"internally_reserved":0,"allocated":10600448,"data_stored":5438451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.745236"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615834+0000","last_change":"2024-05-11T12:14:33.622872+0000","last_active":"2024-05-11T12:14:34.615834+0000","last_peered":"2024-05-11T12:14:34.615834+0000","last_clean":"2024-05-11T12:14:34.615834+0000","last_became_active":"2024-05-11T12:14:33.619136+0000","last_became_peered":"2024-05-11T12:14:33.619136+0000","last_unstale":"2024-05-11T12:14:34.615834+0000","last_undegraded":"2024-05-11T12:14:34.615834+0000","last_fullsized":"2024-05-11T12:14:34.615834+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:18:44.962195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.601568+0000","last_change":"2024-05-11T12:14:32.601568+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.601568+0000","last_undegraded":"2024-05-11T12:14:32.601568+0000","last_fullsized":"2024-05-11T12:14:32.601568+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615616+0000","last_change":"2024-05-11T12:14:33.626436+0000","last_active":"2024-05-11T12:14:34.615616+0000","last_peered":"2024-05-11T12:14:34.615616+0000","last_clean":"2024-05-11T12:14:34.615616+0000","last_became_active":"2024-05-11T12:14:33.626091+0000","last_became_peered":"2024-05-11T12:14:33.626091+0000","last_unstale":"2024-05-11T12:14:34.615616+0000","last_undegraded":"2024-05-11T12:14:34.615616+0000","last_fullsized":"2024-05-11T12:14:34.615616+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:17:05.023315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.284831+0000","last_change":"2024-05-11T12:14:33.624886+0000","last_active":"2024-05-11T12:14:35.284831+0000","last_peered":"2024-05-11T12:14:35.284831+0000","last_clean":"2024-05-11T12:14:35.284831+0000","last_became_active":"2024-05-11T12:14:33.618826+0000","last_became_peered":"2024-05-11T12:14:33.618826+0000","last_unstale":"2024-05-11T12:14:35.284831+0000","last_undegraded":"2024-05-11T12:14:35.284831+0000","last_fullsized":"2024-05-11T12:14:35.284831+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:56:09.727119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T12:14:32.562048+0000","last_change":"2024-05-11T12:14:32.562048+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.562048+0000","last_undegraded":"2024-05-11T12:14:32.562048+0000","last_fullsized":"2024-05-11T12:14:32.562048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.598150+0000","last_change":"2024-05-11T12:14:32.598150+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.598150+0000","last_undegraded":"2024-05-11T12:14:32.598150+0000","last_fullsized":"2024-05-11T12:14:32.598150+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.602749+0000","last_change":"2024-05-11T12:14:32.602749+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.602749+0000","last_undegraded":"2024-05-11T12:14:32.602749+0000","last_fullsized":"2024-05-11T12:14:32.602749+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.208341+0000","last_change":"2024-05-11T12:14:33.621312+0000","last_active":"2024-05-11T12:14:35.208341+0000","last_peered":"2024-05-11T12:14:35.208341+0000","last_clean":"2024-05-11T12:14:35.208341+0000","last_became_active":"2024-05-11T12:14:33.621021+0000","last_became_peered":"2024-05-11T12:14:33.621021+0000","last_unstale":"2024-05-11T12:14:35.208341+0000","last_undegraded":"2024-05-11T12:14:35.208341+0000","last_fullsized":"2024-05-11T12:14:35.208341+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:54:45.084719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.595249+0000","last_change":"2024-05-11T12:14:32.595249+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.595249+0000","last_undegraded":"2024-05-11T12:14:32.595249+0000","last_fullsized":"2024-05-11T12:14:32.595249+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T12:14:32.562048+0000","last_change":"2024-05-11T12:14:32.562048+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.562048+0000","last_undegraded":"2024-05-11T12:14:32.562048+0000","last_fullsized":"2024-05-11T12:14:32.562048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.592511+0000","last_change":"2024-05-11T12:14:32.592511+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.592511+0000","last_undegraded":"2024-05-11T12:14:32.592511+0000","last_fullsized":"2024-05-11T12:14:32.592511+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.606336+0000","last_change":"2024-05-11T12:14:32.606336+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.606336+0000","last_undegraded":"2024-05-11T12:14:32.606336+0000","last_fullsized":"2024-05-11T12:14:32.606336+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.283623+0000","last_change":"2024-05-11T12:14:33.607961+0000","last_active":"2024-05-11T12:14:35.283623+0000","last_peered":"2024-05-11T12:14:35.283623+0000","last_clean":"2024-05-11T12:14:35.283623+0000","last_became_active":"2024-05-11T12:14:33.607599+0000","last_became_peered":"2024-05-11T12:14:33.607599+0000","last_unstale":"2024-05-11T12:14:35.283623+0000","last_undegraded":"2024-05-11T12:14:35.283623+0000","last_fullsized":"2024-05-11T12:14:35.283623+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:22:15.346218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.600693+0000","last_change":"2024-05-11T12:14:32.600693+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.600693+0000","last_undegraded":"2024-05-11T12:14:32.600693+0000","last_fullsized":"2024-05-11T12:14:32.600693+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.599082+0000","last_change":"2024-05-11T12:14:32.599082+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.599082+0000","last_undegraded":"2024-05-11T12:14:32.599082+0000","last_fullsized":"2024-05-11T12:14:32.599082+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.603185+0000","last_change":"2024-05-11T12:14:32.603185+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.603185+0000","last_undegraded":"2024-05-11T12:14:32.603185+0000","last_fullsized":"2024-05-11T12:14:32.603185+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615878+0000","last_change":"2024-05-11T12:14:33.622549+0000","last_active":"2024-05-11T12:14:34.615878+0000","last_peered":"2024-05-11T12:14:34.615878+0000","last_clean":"2024-05-11T12:14:34.615878+0000","last_became_active":"2024-05-11T12:14:33.619603+0000","last_became_peered":"2024-05-11T12:14:33.619603+0000","last_unstale":"2024-05-11T12:14:34.615878+0000","last_undegraded":"2024-05-11T12:14:34.615878+0000","last_fullsized":"2024-05-11T12:14:34.615878+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:51:30.951966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.600768+0000","last_change":"2024-05-11T12:14:32.600768+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.600768+0000","last_undegraded":"2024-05-11T12:14:32.600768+0000","last_fullsized":"2024-05-11T12:14:32.600768+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615768+0000","last_change":"2024-05-11T12:14:33.635269+0000","last_active":"2024-05-11T12:14:34.615768+0000","last_peered":"2024-05-11T12:14:34.615768+0000","last_clean":"2024-05-11T12:14:34.615768+0000","last_became_active":"2024-05-11T12:14:33.634795+0000","last_became_peered":"2024-05-11T12:14:33.634795+0000","last_unstale":"2024-05-11T12:14:34.615768+0000","last_undegraded":"2024-05-11T12:14:34.615768+0000","last_fullsized":"2024-05-11T12:14:34.615768+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:13:16.630356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.605840+0000","last_change":"2024-05-11T12:14:32.605840+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.605840+0000","last_undegraded":"2024-05-11T12:14:32.605840+0000","last_fullsized":"2024-05-11T12:14:32.605840+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.208740+0000","last_change":"2024-05-11T12:14:33.621162+0000","last_active":"2024-05-11T12:14:35.208740+0000","last_peered":"2024-05-11T12:14:35.208740+0000","last_clean":"2024-05-11T12:14:35.208740+0000","last_became_active":"2024-05-11T12:14:33.620843+0000","last_became_peered":"2024-05-11T12:14:33.620843+0000","last_unstale":"2024-05-11T12:14:35.208740+0000","last_undegraded":"2024-05-11T12:14:35.208740+0000","last_fullsized":"2024-05-11T12:14:35.208740+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:29:39.246958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T12:14:32.562048+0000","last_change":"2024-05-11T12:14:32.562048+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.562048+0000","last_undegraded":"2024-05-11T12:14:32.562048+0000","last_fullsized":"2024-05-11T12:14:32.562048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615236+0000","last_change":"2024-05-11T12:14:33.619917+0000","last_active":"2024-05-11T12:14:34.615236+0000","last_peered":"2024-05-11T12:14:34.615236+0000","last_clean":"2024-05-11T12:14:34.615236+0000","last_became_active":"2024-05-11T12:14:33.619614+0000","last_became_peered":"2024-05-11T12:14:33.619614+0000","last_unstale":"2024-05-11T12:14:34.615236+0000","last_undegraded":"2024-05-11T12:14:34.615236+0000","last_fullsized":"2024-05-11T12:14:34.615236+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:56:17.539601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T12:14:32.562048+0000","last_change":"2024-05-11T12:14:32.562048+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.562048+0000","last_undegraded":"2024-05-11T12:14:32.562048+0000","last_fullsized":"2024-05-11T12:14:32.562048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.607235+0000","last_change":"2024-05-11T12:14:32.607235+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.607235+0000","last_undegraded":"2024-05-11T12:14:32.607235+0000","last_fullsized":"2024-05-11T12:14:32.607235+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.601792+0000","last_change":"2024-05-11T12:14:32.601792+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.601792+0000","last_undegraded":"2024-05-11T12:14:32.601792+0000","last_fullsized":"2024-05-11T12:14:32.601792+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.591419+0000","last_change":"2024-05-11T12:14:32.591419+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.591419+0000","last_undegraded":"2024-05-11T12:14:32.591419+0000","last_fullsized":"2024-05-11T12:14:32.591419+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.595401+0000","last_change":"2024-05-11T12:14:32.595401+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.595401+0000","last_undegraded":"2024-05-11T12:14:32.595401+0000","last_fullsized":"2024-05-11T12:14:32.595401+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209481+0000","last_change":"2024-05-11T12:14:33.623708+0000","last_active":"2024-05-11T12:14:35.209481+0000","last_peered":"2024-05-11T12:14:35.209481+0000","last_clean":"2024-05-11T12:14:35.209481+0000","last_became_active":"2024-05-11T12:14:33.623429+0000","last_became_peered":"2024-05-11T12:14:33.623429+0000","last_unstale":"2024-05-11T12:14:35.209481+0000","last_undegraded":"2024-05-11T12:14:35.209481+0000","last_fullsized":"2024-05-11T12:14:35.209481+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:30:11.411742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.603561+0000","last_change":"2024-05-11T12:14:32.603561+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.603561+0000","last_undegraded":"2024-05-11T12:14:32.603561+0000","last_fullsized":"2024-05-11T12:14:32.603561+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.596127+0000","last_change":"2024-05-11T12:14:32.596127+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.596127+0000","last_undegraded":"2024-05-11T12:14:32.596127+0000","last_fullsized":"2024-05-11T12:14:32.596127+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.208839+0000","last_change":"2024-05-11T12:14:33.621606+0000","last_active":"2024-05-11T12:14:35.208839+0000","last_peered":"2024-05-11T12:14:35.208839+0000","last_clean":"2024-05-11T12:14:35.208839+0000","last_became_active":"2024-05-11T12:14:33.621288+0000","last_became_peered":"2024-05-11T12:14:33.621288+0000","last_unstale":"2024-05-11T12:14:35.208839+0000","last_undegraded":"2024-05-11T12:14:35.208839+0000","last_fullsized":"2024-05-11T12:14:35.208839+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:37:19.519120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615469+0000","last_change":"2024-05-11T12:14:33.633794+0000","last_active":"2024-05-11T12:14:34.615469+0000","last_peered":"2024-05-11T12:14:34.615469+0000","last_clean":"2024-05-11T12:14:34.615469+0000","last_became_active":"2024-05-11T12:14:33.633451+0000","last_became_peered":"2024-05-11T12:14:33.633451+0000","last_unstale":"2024-05-11T12:14:34.615469+0000","last_undegraded":"2024-05-11T12:14:34.615469+0000","last_fullsized":"2024-05-11T12:14:34.615469+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:31:39.465305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"96'4","reported_seq":18,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.285496+0000","last_change":"2024-05-11T12:14:32.610045+0000","last_active":"2024-05-11T12:14:35.285496+0000","last_peered":"2024-05-11T12:14:35.285496+0000","last_clean":"2024-05-11T12:14:35.285496+0000","last_became_active":"2024-05-11T12:14:32.609746+0000","last_became_peered":"2024-05-11T12:14:32.609746+0000","last_unstale":"2024-05-11T12:14:35.285496+0000","last_undegraded":"2024-05-11T12:14:35.285496+0000","last_fullsized":"2024-05-11T12:14:35.285496+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:44:56.684605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.208579+0000","last_change":"2024-05-11T12:14:33.630664+0000","last_active":"2024-05-11T12:14:35.208579+0000","last_peered":"2024-05-11T12:14:35.208579+0000","last_clean":"2024-05-11T12:14:35.208579+0000","last_became_active":"2024-05-11T12:14:33.630171+0000","last_became_peered":"2024-05-11T12:14:33.630171+0000","last_unstale":"2024-05-11T12:14:35.208579+0000","last_undegraded":"2024-05-11T12:14:35.208579+0000","last_fullsized":"2024-05-11T12:14:35.208579+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:00:10.906551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209543+0000","last_change":"2024-05-11T12:14:32.619446+0000","last_active":"2024-05-11T12:14:35.209543+0000","last_peered":"2024-05-11T12:14:35.209543+0000","last_clean":"2024-05-11T12:14:35.209543+0000","last_became_active":"2024-05-11T12:14:32.619219+0000","last_became_peered":"2024-05-11T12:14:32.619219+0000","last_unstale":"2024-05-11T12:14:35.209543+0000","last_undegraded":"2024-05-11T12:14:35.209543+0000","last_fullsized":"2024-05-11T12:14:35.209543+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:34:15.881914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.603030+0000","last_change":"2024-05-11T12:14:32.603030+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.603030+0000","last_undegraded":"2024-05-11T12:14:32.603030+0000","last_fullsized":"2024-05-11T12:14:32.603030+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-11T12:14:32.621991+0000","last_change":"2024-05-11T12:14:32.618332+0000","last_active":"2024-05-11T12:14:32.621991+0000","last_peered":"2024-05-11T12:14:32.621991+0000","last_clean":"2024-05-11T12:14:32.621991+0000","last_became_active":"2024-05-11T12:14:32.614777+0000","last_became_peered":"2024-05-11T12:14:32.614777+0000","last_unstale":"2024-05-11T12:14:32.621991+0000","last_undegraded":"2024-05-11T12:14:32.621991+0000","last_fullsized":"2024-05-11T12:14:32.621991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:01:31.105667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.591054+0000","last_change":"2024-05-11T12:14:32.591054+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.591054+0000","last_undegraded":"2024-05-11T12:14:32.591054+0000","last_fullsized":"2024-05-11T12:14:32.591054+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-05-11T12:14:31.585654+0000","last_change":"2024-05-11T12:14:31.585654+0000","last_active":"2024-05-11T12:14:31.545700+0000","last_peered":"2024-05-11T12:14:31.545700+0000","last_clean":"2024-05-11T12:14:31.545700+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:31.585654+0000","last_undegraded":"2024-05-11T12:14:31.585654+0000","last_fullsized":"2024-05-11T12:14:31.585654+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.284849+0000","last_change":"2024-05-11T12:14:33.616596+0000","last_active":"2024-05-11T12:14:35.284849+0000","last_peered":"2024-05-11T12:14:35.284849+0000","last_clean":"2024-05-11T12:14:35.284849+0000","last_became_active":"2024-05-11T12:14:33.616294+0000","last_became_peered":"2024-05-11T12:14:33.616294+0000","last_unstale":"2024-05-11T12:14:35.284849+0000","last_undegraded":"2024-05-11T12:14:35.284849+0000","last_fullsized":"2024-05-11T12:14:35.284849+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:27:47.349240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-11T12:14:32.621828+0000","last_change":"2024-05-11T12:14:32.619606+0000","last_active":"2024-05-11T12:14:32.621828+0000","last_peered":"2024-05-11T12:14:32.621828+0000","last_clean":"2024-05-11T12:14:32.621828+0000","last_became_active":"2024-05-11T12:14:32.619133+0000","last_became_peered":"2024-05-11T12:14:32.619133+0000","last_unstale":"2024-05-11T12:14:32.621828+0000","last_undegraded":"2024-05-11T12:14:32.621828+0000","last_fullsized":"2024-05-11T12:14:32.621828+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:00:56.111300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.616178+0000","last_change":"2024-05-11T12:14:33.634226+0000","last_active":"2024-05-11T12:14:34.616178+0000","last_peered":"2024-05-11T12:14:34.616178+0000","last_clean":"2024-05-11T12:14:34.616178+0000","last_became_active":"2024-05-11T12:14:33.633932+0000","last_became_peered":"2024-05-11T12:14:33.633932+0000","last_unstale":"2024-05-11T12:14:34.616178+0000","last_undegraded":"2024-05-11T12:14:34.616178+0000","last_fullsized":"2024-05-11T12:14:34.616178+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:01:12.394223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-11T12:14:32.616593+0000","last_change":"2024-05-11T12:14:32.614159+0000","last_active":"2024-05-11T12:14:32.616593+0000","last_peered":"2024-05-11T12:14:32.616593+0000","last_clean":"2024-05-11T12:14:32.616593+0000","last_became_active":"2024-05-11T12:14:32.613421+0000","last_became_peered":"2024-05-11T12:14:32.613421+0000","last_unstale":"2024-05-11T12:14:32.616593+0000","last_undegraded":"2024-05-11T12:14:32.616593+0000","last_fullsized":"2024-05-11T12:14:32.616593+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:58:41.059975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.599834+0000","last_change":"2024-05-11T12:14:32.599834+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.599834+0000","last_undegraded":"2024-05-11T12:14:32.599834+0000","last_fullsized":"2024-05-11T12:14:32.599834+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-11T12:14:32.620324+0000","last_change":"2024-05-11T12:14:32.614638+0000","last_active":"2024-05-11T12:14:32.620324+0000","last_peered":"2024-05-11T12:14:32.620324+0000","last_clean":"2024-05-11T12:14:32.620324+0000","last_became_active":"2024-05-11T12:14:32.614242+0000","last_became_peered":"2024-05-11T12:14:32.614242+0000","last_unstale":"2024-05-11T12:14:32.620324+0000","last_undegraded":"2024-05-11T12:14:32.620324+0000","last_fullsized":"2024-05-11T12:14:32.620324+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:45:58.666366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.285649+0000","last_change":"2024-05-11T12:14:33.615167+0000","last_active":"2024-05-11T12:14:35.285649+0000","last_peered":"2024-05-11T12:14:35.285649+0000","last_clean":"2024-05-11T12:14:35.285649+0000","last_became_active":"2024-05-11T12:14:33.614898+0000","last_became_peered":"2024-05-11T12:14:33.614898+0000","last_unstale":"2024-05-11T12:14:35.285649+0000","last_undegraded":"2024-05-11T12:14:35.285649+0000","last_fullsized":"2024-05-11T12:14:35.285649+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:52:46.883659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.283863+0000","last_change":"2024-05-11T12:14:32.605627+0000","last_active":"2024-05-11T12:14:35.283863+0000","last_peered":"2024-05-11T12:14:35.283863+0000","last_clean":"2024-05-11T12:14:35.283863+0000","last_became_active":"2024-05-11T12:14:32.604729+0000","last_became_peered":"2024-05-11T12:14:32.604729+0000","last_unstale":"2024-05-11T12:14:35.283863+0000","last_undegraded":"2024-05-11T12:14:35.283863+0000","last_fullsized":"2024-05-11T12:14:35.283863+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:00:56.111300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615229+0000","last_change":"2024-05-11T12:14:33.635074+0000","last_active":"2024-05-11T12:14:34.615229+0000","last_peered":"2024-05-11T12:14:34.615229+0000","last_clean":"2024-05-11T12:14:34.615229+0000","last_became_active":"2024-05-11T12:14:33.634606+0000","last_became_peered":"2024-05-11T12:14:33.634606+0000","last_unstale":"2024-05-11T12:14:34.615229+0000","last_undegraded":"2024-05-11T12:14:34.615229+0000","last_fullsized":"2024-05-11T12:14:34.615229+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:03:07.386179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-11T12:14:32.620414+0000","last_change":"2024-05-11T12:14:32.615191+0000","last_active":"2024-05-11T12:14:32.620414+0000","last_peered":"2024-05-11T12:14:32.620414+0000","last_clean":"2024-05-11T12:14:32.620414+0000","last_became_active":"2024-05-11T12:14:32.614809+0000","last_became_peered":"2024-05-11T12:14:32.614809+0000","last_unstale":"2024-05-11T12:14:32.620414+0000","last_undegraded":"2024-05-11T12:14:32.620414+0000","last_fullsized":"2024-05-11T12:14:32.620414+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:15:55.175677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T12:14:32.562048+0000","last_change":"2024-05-11T12:14:32.562048+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.562048+0000","last_undegraded":"2024-05-11T12:14:32.562048+0000","last_fullsized":"2024-05-11T12:14:32.562048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615602+0000","last_change":"2024-05-11T12:14:32.609650+0000","last_active":"2024-05-11T12:14:34.615602+0000","last_peered":"2024-05-11T12:14:34.615602+0000","last_clean":"2024-05-11T12:14:34.615602+0000","last_became_active":"2024-05-11T12:14:32.609340+0000","last_became_peered":"2024-05-11T12:14:32.609340+0000","last_unstale":"2024-05-11T12:14:34.615602+0000","last_undegraded":"2024-05-11T12:14:34.615602+0000","last_fullsized":"2024-05-11T12:14:34.615602+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:40:56.223685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209330+0000","last_change":"2024-05-11T12:14:33.631013+0000","last_active":"2024-05-11T12:14:35.209330+0000","last_peered":"2024-05-11T12:14:35.209330+0000","last_clean":"2024-05-11T12:14:35.209330+0000","last_became_active":"2024-05-11T12:14:33.630698+0000","last_became_peered":"2024-05-11T12:14:33.630698+0000","last_unstale":"2024-05-11T12:14:35.209330+0000","last_undegraded":"2024-05-11T12:14:35.209330+0000","last_fullsized":"2024-05-11T12:14:35.209330+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:32:48.749691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-11T12:14:32.621778+0000","last_change":"2024-05-11T12:14:32.618526+0000","last_active":"2024-05-11T12:14:32.621778+0000","last_peered":"2024-05-11T12:14:32.621778+0000","last_clean":"2024-05-11T12:14:32.621778+0000","last_became_active":"2024-05-11T12:14:32.618198+0000","last_became_peered":"2024-05-11T12:14:32.618198+0000","last_unstale":"2024-05-11T12:14:32.621778+0000","last_undegraded":"2024-05-11T12:14:32.621778+0000","last_fullsized":"2024-05-11T12:14:32.621778+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:57:12.465310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T12:14:32.562048+0000","last_change":"2024-05-11T12:14:32.562048+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.562048+0000","last_undegraded":"2024-05-11T12:14:32.562048+0000","last_fullsized":"2024-05-11T12:14:32.562048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-11T12:14:32.613159+0000","last_change":"2024-05-11T12:14:32.607489+0000","last_active":"2024-05-11T12:14:32.613159+0000","last_peered":"2024-05-11T12:14:32.613159+0000","last_clean":"2024-05-11T12:14:32.613159+0000","last_became_active":"2024-05-11T12:14:32.606078+0000","last_became_peered":"2024-05-11T12:14:32.606078+0000","last_unstale":"2024-05-11T12:14:32.613159+0000","last_undegraded":"2024-05-11T12:14:32.613159+0000","last_fullsized":"2024-05-11T12:14:32.613159+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:07:30.136729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.591204+0000","last_change":"2024-05-11T12:14:32.591204+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.591204+0000","last_undegraded":"2024-05-11T12:14:32.591204+0000","last_fullsized":"2024-05-11T12:14:32.591204+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-05-11T12:14:31.584097+0000","last_change":"2024-05-11T12:14:31.584097+0000","last_active":"2024-05-11T12:14:31.545700+0000","last_peered":"2024-05-11T12:14:31.545700+0000","last_clean":"2024-05-11T12:14:31.545700+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:31.584097+0000","last_undegraded":"2024-05-11T12:14:31.584097+0000","last_fullsized":"2024-05-11T12:14:31.584097+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.594883+0000","last_change":"2024-05-11T12:14:32.594883+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.594883+0000","last_undegraded":"2024-05-11T12:14:32.594883+0000","last_fullsized":"2024-05-11T12:14:32.594883+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.284356+0000","last_change":"2024-05-11T12:14:32.616922+0000","last_active":"2024-05-11T12:14:35.284356+0000","last_peered":"2024-05-11T12:14:35.284356+0000","last_clean":"2024-05-11T12:14:35.284356+0000","last_became_active":"2024-05-11T12:14:32.616599+0000","last_became_peered":"2024-05-11T12:14:32.616599+0000","last_unstale":"2024-05-11T12:14:35.284356+0000","last_undegraded":"2024-05-11T12:14:35.284356+0000","last_fullsized":"2024-05-11T12:14:35.284356+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:55:59.198545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209297+0000","last_change":"2024-05-11T12:14:33.624455+0000","last_active":"2024-05-11T12:14:35.209297+0000","last_peered":"2024-05-11T12:14:35.209297+0000","last_clean":"2024-05-11T12:14:35.209297+0000","last_became_active":"2024-05-11T12:14:33.624158+0000","last_became_peered":"2024-05-11T12:14:33.624158+0000","last_unstale":"2024-05-11T12:14:35.209297+0000","last_undegraded":"2024-05-11T12:14:35.209297+0000","last_fullsized":"2024-05-11T12:14:35.209297+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:44:32.910714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.208576+0000","last_change":"2024-05-11T12:14:32.615486+0000","last_active":"2024-05-11T12:14:35.208576+0000","last_peered":"2024-05-11T12:14:35.208576+0000","last_clean":"2024-05-11T12:14:35.208576+0000","last_became_active":"2024-05-11T12:14:32.615204+0000","last_became_peered":"2024-05-11T12:14:32.615204+0000","last_unstale":"2024-05-11T12:14:35.208576+0000","last_undegraded":"2024-05-11T12:14:35.208576+0000","last_fullsized":"2024-05-11T12:14:35.208576+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:27:00.766407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615938+0000","last_change":"2024-05-11T12:14:33.623140+0000","last_active":"2024-05-11T12:14:34.615938+0000","last_peered":"2024-05-11T12:14:34.615938+0000","last_clean":"2024-05-11T12:14:34.615938+0000","last_became_active":"2024-05-11T12:14:33.618444+0000","last_became_peered":"2024-05-11T12:14:33.618444+0000","last_unstale":"2024-05-11T12:14:34.615938+0000","last_undegraded":"2024-05-11T12:14:34.615938+0000","last_fullsized":"2024-05-11T12:14:34.615938+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:35:51.107577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-11T12:14:32.620528+0000","last_change":"2024-05-11T12:14:32.616263+0000","last_active":"2024-05-11T12:14:32.620528+0000","last_peered":"2024-05-11T12:14:32.620528+0000","last_clean":"2024-05-11T12:14:32.620528+0000","last_became_active":"2024-05-11T12:14:32.615004+0000","last_became_peered":"2024-05-11T12:14:32.615004+0000","last_unstale":"2024-05-11T12:14:32.620528+0000","last_undegraded":"2024-05-11T12:14:32.620528+0000","last_fullsized":"2024-05-11T12:14:32.620528+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:07:30.136729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.208800+0000","last_change":"2024-05-11T12:14:33.616435+0000","last_active":"2024-05-11T12:14:35.208800+0000","last_peered":"2024-05-11T12:14:35.208800+0000","last_clean":"2024-05-11T12:14:35.208800+0000","last_became_active":"2024-05-11T12:14:33.616110+0000","last_became_peered":"2024-05-11T12:14:33.616110+0000","last_unstale":"2024-05-11T12:14:35.208800+0000","last_undegraded":"2024-05-11T12:14:35.208800+0000","last_fullsized":"2024-05-11T12:14:35.208800+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:51:10.706767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-11T12:14:32.616852+0000","last_change":"2024-05-11T12:14:32.613751+0000","last_active":"2024-05-11T12:14:32.616852+0000","last_peered":"2024-05-11T12:14:32.616852+0000","last_clean":"2024-05-11T12:14:32.616852+0000","last_became_active":"2024-05-11T12:14:32.613431+0000","last_became_peered":"2024-05-11T12:14:32.613431+0000","last_unstale":"2024-05-11T12:14:32.616852+0000","last_undegraded":"2024-05-11T12:14:32.616852+0000","last_fullsized":"2024-05-11T12:14:32.616852+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:47:05.267174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.597272+0000","last_change":"2024-05-11T12:14:32.597272+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.597272+0000","last_undegraded":"2024-05-11T12:14:32.597272+0000","last_fullsized":"2024-05-11T12:14:32.597272+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-11T12:14:32.626697+0000","last_change":"2024-05-11T12:14:32.621615+0000","last_active":"2024-05-11T12:14:32.626697+0000","last_peered":"2024-05-11T12:14:32.626697+0000","last_clean":"2024-05-11T12:14:32.626697+0000","last_became_active":"2024-05-11T12:14:32.620973+0000","last_became_peered":"2024-05-11T12:14:32.620973+0000","last_unstale":"2024-05-11T12:14:32.626697+0000","last_undegraded":"2024-05-11T12:14:32.626697+0000","last_fullsized":"2024-05-11T12:14:32.626697+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:44:56.684605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T12:14:32.562048+0000","last_change":"2024-05-11T12:14:32.562048+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.562048+0000","last_undegraded":"2024-05-11T12:14:32.562048+0000","last_fullsized":"2024-05-11T12:14:32.562048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-11T12:14:32.612302+0000","last_change":"2024-05-11T12:14:32.609813+0000","last_active":"2024-05-11T12:14:32.612302+0000","last_peered":"2024-05-11T12:14:32.612302+0000","last_clean":"2024-05-11T12:14:32.612302+0000","last_became_active":"2024-05-11T12:14:32.609531+0000","last_became_peered":"2024-05-11T12:14:32.609531+0000","last_unstale":"2024-05-11T12:14:32.612302+0000","last_undegraded":"2024-05-11T12:14:32.612302+0000","last_fullsized":"2024-05-11T12:14:32.612302+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:54:11.073754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615796+0000","last_change":"2024-05-11T12:14:33.629837+0000","last_active":"2024-05-11T12:14:34.615796+0000","last_peered":"2024-05-11T12:14:34.615796+0000","last_clean":"2024-05-11T12:14:34.615796+0000","last_became_active":"2024-05-11T12:14:33.629561+0000","last_became_peered":"2024-05-11T12:14:33.629561+0000","last_unstale":"2024-05-11T12:14:34.615796+0000","last_undegraded":"2024-05-11T12:14:34.615796+0000","last_fullsized":"2024-05-11T12:14:34.615796+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:24:04.957521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209466+0000","last_change":"2024-05-11T12:14:32.605666+0000","last_active":"2024-05-11T12:14:35.209466+0000","last_peered":"2024-05-11T12:14:35.209466+0000","last_clean":"2024-05-11T12:14:35.209466+0000","last_became_active":"2024-05-11T12:14:32.605407+0000","last_became_peered":"2024-05-11T12:14:32.605407+0000","last_unstale":"2024-05-11T12:14:35.209466+0000","last_undegraded":"2024-05-11T12:14:35.209466+0000","last_fullsized":"2024-05-11T12:14:35.209466+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:31:24.122325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615946+0000","last_change":"2024-05-11T12:14:33.623726+0000","last_active":"2024-05-11T12:14:34.615946+0000","last_peered":"2024-05-11T12:14:34.615946+0000","last_clean":"2024-05-11T12:14:34.615946+0000","last_became_active":"2024-05-11T12:14:33.616682+0000","last_became_peered":"2024-05-11T12:14:33.616682+0000","last_unstale":"2024-05-11T12:14:34.615946+0000","last_undegraded":"2024-05-11T12:14:34.615946+0000","last_fullsized":"2024-05-11T12:14:34.615946+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:11:09.187757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.208810+0000","last_change":"2024-05-11T12:14:32.612947+0000","last_active":"2024-05-11T12:14:35.208810+0000","last_peered":"2024-05-11T12:14:35.208810+0000","last_clean":"2024-05-11T12:14:35.208810+0000","last_became_active":"2024-05-11T12:14:32.612545+0000","last_became_peered":"2024-05-11T12:14:32.612545+0000","last_unstale":"2024-05-11T12:14:35.208810+0000","last_undegraded":"2024-05-11T12:14:35.208810+0000","last_fullsized":"2024-05-11T12:14:35.208810+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:10:08.253641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.285245+0000","last_change":"2024-05-11T12:14:33.628473+0000","last_active":"2024-05-11T12:14:35.285245+0000","last_peered":"2024-05-11T12:14:35.285245+0000","last_clean":"2024-05-11T12:14:35.285245+0000","last_became_active":"2024-05-11T12:14:33.627301+0000","last_became_peered":"2024-05-11T12:14:33.627301+0000","last_unstale":"2024-05-11T12:14:35.285245+0000","last_undegraded":"2024-05-11T12:14:35.285245+0000","last_fullsized":"2024-05-11T12:14:35.285245+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:16:12.500928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-11T12:14:32.613383+0000","last_change":"2024-05-11T12:14:32.607325+0000","last_active":"2024-05-11T12:14:32.613383+0000","last_peered":"2024-05-11T12:14:32.613383+0000","last_clean":"2024-05-11T12:14:32.613383+0000","last_became_active":"2024-05-11T12:14:32.606982+0000","last_became_peered":"2024-05-11T12:14:32.606982+0000","last_unstale":"2024-05-11T12:14:32.613383+0000","last_undegraded":"2024-05-11T12:14:32.613383+0000","last_fullsized":"2024-05-11T12:14:32.613383+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:15:55.175677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209260+0000","last_change":"2024-05-11T12:14:33.618158+0000","last_active":"2024-05-11T12:14:35.209260+0000","last_peered":"2024-05-11T12:14:35.209260+0000","last_clean":"2024-05-11T12:14:35.209260+0000","last_became_active":"2024-05-11T12:14:33.617850+0000","last_became_peered":"2024-05-11T12:14:33.617850+0000","last_unstale":"2024-05-11T12:14:35.209260+0000","last_undegraded":"2024-05-11T12:14:35.209260+0000","last_fullsized":"2024-05-11T12:14:35.209260+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:38:25.159336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209121+0000","last_change":"2024-05-11T12:14:32.607439+0000","last_active":"2024-05-11T12:14:35.209121+0000","last_peered":"2024-05-11T12:14:35.209121+0000","last_clean":"2024-05-11T12:14:35.209121+0000","last_became_active":"2024-05-11T12:14:32.606952+0000","last_became_peered":"2024-05-11T12:14:32.606952+0000","last_unstale":"2024-05-11T12:14:35.209121+0000","last_undegraded":"2024-05-11T12:14:35.209121+0000","last_fullsized":"2024-05-11T12:14:35.209121+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:14:57.743590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615535+0000","last_change":"2024-05-11T12:14:33.619178+0000","last_active":"2024-05-11T12:14:34.615535+0000","last_peered":"2024-05-11T12:14:34.615535+0000","last_clean":"2024-05-11T12:14:34.615535+0000","last_became_active":"2024-05-11T12:14:33.618894+0000","last_became_peered":"2024-05-11T12:14:33.618894+0000","last_unstale":"2024-05-11T12:14:34.615535+0000","last_undegraded":"2024-05-11T12:14:34.615535+0000","last_fullsized":"2024-05-11T12:14:34.615535+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:10:50.170349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-05-11T12:14:31.584956+0000","last_change":"2024-05-11T12:14:31.584956+0000","last_active":"2024-05-11T12:14:31.545700+0000","last_peered":"2024-05-11T12:14:31.545700+0000","last_clean":"2024-05-11T12:14:31.545700+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:31.584956+0000","last_undegraded":"2024-05-11T12:14:31.584956+0000","last_fullsized":"2024-05-11T12:14:31.584956+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.602318+0000","last_change":"2024-05-11T12:14:32.602318+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.602318+0000","last_undegraded":"2024-05-11T12:14:32.602318+0000","last_fullsized":"2024-05-11T12:14:32.602318+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"97'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615678+0000","last_change":"2024-05-11T12:14:32.605888+0000","last_active":"2024-05-11T12:14:34.615678+0000","last_peered":"2024-05-11T12:14:34.615678+0000","last_clean":"2024-05-11T12:14:34.615678+0000","last_became_active":"2024-05-11T12:14:32.605588+0000","last_became_peered":"2024-05-11T12:14:32.605588+0000","last_unstale":"2024-05-11T12:14:34.615678+0000","last_undegraded":"2024-05-11T12:14:34.615678+0000","last_fullsized":"2024-05-11T12:14:34.615678+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:56:02.427960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.284207+0000","last_change":"2024-05-11T12:14:33.627479+0000","last_active":"2024-05-11T12:14:35.284207+0000","last_peered":"2024-05-11T12:14:35.284207+0000","last_clean":"2024-05-11T12:14:35.284207+0000","last_became_active":"2024-05-11T12:14:33.627179+0000","last_became_peered":"2024-05-11T12:14:33.627179+0000","last_unstale":"2024-05-11T12:14:35.284207+0000","last_undegraded":"2024-05-11T12:14:35.284207+0000","last_fullsized":"2024-05-11T12:14:35.284207+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:35:20.045762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-11T12:14:32.612410+0000","last_change":"2024-05-11T12:14:32.605594+0000","last_active":"2024-05-11T12:14:32.612410+0000","last_peered":"2024-05-11T12:14:32.612410+0000","last_clean":"2024-05-11T12:14:32.612410+0000","last_became_active":"2024-05-11T12:14:32.605097+0000","last_became_peered":"2024-05-11T12:14:32.605097+0000","last_unstale":"2024-05-11T12:14:32.612410+0000","last_undegraded":"2024-05-11T12:14:32.612410+0000","last_fullsized":"2024-05-11T12:14:32.612410+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:05:15.832383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.592181+0000","last_change":"2024-05-11T12:14:32.592181+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.592181+0000","last_undegraded":"2024-05-11T12:14:32.592181+0000","last_fullsized":"2024-05-11T12:14:32.592181+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-05-11T12:14:31.584482+0000","last_change":"2024-05-11T12:14:31.584482+0000","last_active":"2024-05-11T12:14:31.545700+0000","last_peered":"2024-05-11T12:14:31.545700+0000","last_clean":"2024-05-11T12:14:31.545700+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:31.584482+0000","last_undegraded":"2024-05-11T12:14:31.584482+0000","last_fullsized":"2024-05-11T12:14:31.584482+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.208801+0000","last_change":"2024-05-11T12:14:33.630741+0000","last_active":"2024-05-11T12:14:35.208801+0000","last_peered":"2024-05-11T12:14:35.208801+0000","last_clean":"2024-05-11T12:14:35.208801+0000","last_became_active":"2024-05-11T12:14:33.630423+0000","last_became_peered":"2024-05-11T12:14:33.630423+0000","last_unstale":"2024-05-11T12:14:35.208801+0000","last_undegraded":"2024-05-11T12:14:35.208801+0000","last_fullsized":"2024-05-11T12:14:35.208801+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:47:57.386158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-11T12:14:32.621836+0000","last_change":"2024-05-11T12:14:32.616942+0000","last_active":"2024-05-11T12:14:32.621836+0000","last_peered":"2024-05-11T12:14:32.621836+0000","last_clean":"2024-05-11T12:14:32.621836+0000","last_became_active":"2024-05-11T12:14:32.616577+0000","last_became_peered":"2024-05-11T12:14:32.616577+0000","last_unstale":"2024-05-11T12:14:32.621836+0000","last_undegraded":"2024-05-11T12:14:32.621836+0000","last_fullsized":"2024-05-11T12:14:32.621836+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:47:05.267174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.208768+0000","last_change":"2024-05-11T12:14:33.620388+0000","last_active":"2024-05-11T12:14:35.208768+0000","last_peered":"2024-05-11T12:14:35.208768+0000","last_clean":"2024-05-11T12:14:35.208768+0000","last_became_active":"2024-05-11T12:14:33.618621+0000","last_became_peered":"2024-05-11T12:14:33.618621+0000","last_unstale":"2024-05-11T12:14:35.208768+0000","last_undegraded":"2024-05-11T12:14:35.208768+0000","last_fullsized":"2024-05-11T12:14:35.208768+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:57:26.186975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209282+0000","last_change":"2024-05-11T12:14:32.615893+0000","last_active":"2024-05-11T12:14:35.209282+0000","last_peered":"2024-05-11T12:14:35.209282+0000","last_clean":"2024-05-11T12:14:35.209282+0000","last_became_active":"2024-05-11T12:14:32.615639+0000","last_became_peered":"2024-05-11T12:14:32.615639+0000","last_unstale":"2024-05-11T12:14:35.209282+0000","last_undegraded":"2024-05-11T12:14:35.209282+0000","last_fullsized":"2024-05-11T12:14:35.209282+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:51:28.062961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.601116+0000","last_change":"2024-05-11T12:14:32.601116+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.601116+0000","last_undegraded":"2024-05-11T12:14:32.601116+0000","last_fullsized":"2024-05-11T12:14:32.601116+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-11T12:14:32.613358+0000","last_change":"2024-05-11T12:14:32.607229+0000","last_active":"2024-05-11T12:14:32.613358+0000","last_peered":"2024-05-11T12:14:32.613358+0000","last_clean":"2024-05-11T12:14:32.613358+0000","last_became_active":"2024-05-11T12:14:32.606938+0000","last_became_peered":"2024-05-11T12:14:32.606938+0000","last_unstale":"2024-05-11T12:14:32.613358+0000","last_undegraded":"2024-05-11T12:14:32.613358+0000","last_fullsized":"2024-05-11T12:14:32.613358+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:45:58.666366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.593567+0000","last_change":"2024-05-11T12:14:32.593567+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.593567+0000","last_undegraded":"2024-05-11T12:14:32.593567+0000","last_fullsized":"2024-05-11T12:14:32.593567+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209088+0000","last_change":"2024-05-11T12:14:32.610511+0000","last_active":"2024-05-11T12:14:35.209088+0000","last_peered":"2024-05-11T12:14:35.209088+0000","last_clean":"2024-05-11T12:14:35.209088+0000","last_became_active":"2024-05-11T12:14:32.610243+0000","last_became_peered":"2024-05-11T12:14:32.610243+0000","last_unstale":"2024-05-11T12:14:35.209088+0000","last_undegraded":"2024-05-11T12:14:35.209088+0000","last_fullsized":"2024-05-11T12:14:35.209088+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:39:24.599004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.589733+0000","last_change":"2024-05-11T12:14:32.589733+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.589733+0000","last_undegraded":"2024-05-11T12:14:32.589733+0000","last_fullsized":"2024-05-11T12:14:32.589733+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209535+0000","last_change":"2024-05-11T12:14:32.612907+0000","last_active":"2024-05-11T12:14:35.209535+0000","last_peered":"2024-05-11T12:14:35.209535+0000","last_clean":"2024-05-11T12:14:35.209535+0000","last_became_active":"2024-05-11T12:14:32.612548+0000","last_became_peered":"2024-05-11T12:14:32.612548+0000","last_unstale":"2024-05-11T12:14:35.209535+0000","last_undegraded":"2024-05-11T12:14:35.209535+0000","last_fullsized":"2024-05-11T12:14:35.209535+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:49:45.457016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":243,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-11T12:14:32.605441+0000","last_change":"2024-05-11T12:10:35.066673+0000","last_active":"2024-05-11T12:14:32.605441+0000","last_peered":"2024-05-11T12:14:32.605441+0000","last_clean":"2024-05-11T12:14:32.605441+0000","last_became_active":"2024-05-11T12:10:35.065942+0000","last_became_peered":"2024-05-11T12:10:35.065942+0000","last_unstale":"2024-05-11T12:14:32.605441+0000","last_undegraded":"2024-05-11T12:14:32.605441+0000","last_fullsized":"2024-05-11T12:14:32.605441+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_clean_scrub_stamp":"2024-05-11T12:10:33.601136+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:12:02.133182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":114,"acting":114,"num_store_stats":7},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476787,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32900,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338940,"statfs":{"total":96636764160,"available":96603074560,"internally_reserved":0,"allocated":1761280,"data_stored":1118705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476788,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32324,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1167360,"data_stored":528337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476787,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32324,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1167360,"data_stored":528337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476789,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32324,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1167360,"data_stored":528337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509493,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1196032,"data_stored":542778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509493,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32304,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339536,"statfs":{"total":96636764160,"available":96603684864,"internally_reserved":0,"allocated":1142784,"data_stored":515461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509493,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32948,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338892,"statfs":{"total":96636764160,"available":96603025408,"internally_reserved":0,"allocated":1802240,"data_stored":1134266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542197,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1196032,"data_stored":542230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T12:14:38.548 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T12:14:38.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.786+0000 7f4cad600640 1 Processor -- start 2024-05-11T12:14:38.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.786+0000 7f4cad600640 1 -- start start 2024-05-11T12:14:38.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.788+0000 7f4cad600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ca80af5d0 0x7f4ca8073ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:38.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.788+0000 7f4cad600640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4ca806d460 0x7f4ca806d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:38.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.788+0000 7f4cad600640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ca806fe10 con 0x7f4ca80af1f0 2024-05-11T12:14:38.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.788+0000 7f4cad600640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f4ca81608d0 con 0x7f4ca806d460 2024-05-11T12:14:38.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.788+0000 7f4cad600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4ca815fcc0 con 0x7f4ca80af5d0 2024-05-11T12:14:38.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.788+0000 7f4ca7400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4ca806d460 0x7f4ca806d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:38.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.788+0000 7f4ca6a00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f4ca80af1f0 0x7f4ca80cf1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:55654/0 (socket says 172.21.15.167:55654) 2024-05-11T12:14:38.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.788+0000 7f4ca6a00640 1 -- 172.21.15.167:0/4191690330 learned_addr learned my addr 172.21.15.167:0/4191690330 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:14:38.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.788+0000 7f4c9fe00640 1 --2- 172.21.15.167:0/4191690330 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ca80af5d0 0x7f4ca8073ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:38.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.789+0000 7f4ca6000640 1 -- 172.21.15.167:0/4191690330 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 623231788 0 0) 0x7f4ca806fe10 con 0x7f4ca80af1f0 2024-05-11T12:14:38.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.789+0000 7f4ca6000640 1 -- 172.21.15.167:0/4191690330 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c8c003520 con 0x7f4ca80af1f0 2024-05-11T12:14:38.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.789+0000 7f4ca7400640 1 -- 172.21.15.167:0/4191690330 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ca80af5d0 msgr2=0x7f4ca8073ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:38.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.789+0000 7f4ca7400640 1 --2- 172.21.15.167:0/4191690330 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ca80af5d0 0x7f4ca8073ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:38.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.789+0000 7f4ca7400640 1 -- 172.21.15.167:0/4191690330 >> v1:172.21.15.167:6789/0 conn(0x7f4ca80af1f0 legacy=0x7f4ca80cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:38.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.789+0000 7f4ca7400640 1 -- 172.21.15.167:0/4191690330 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ca815ffa0 con 0x7f4ca806d460 2024-05-11T12:14:38.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.790+0000 7f4ca7400640 1 --2- 172.21.15.167:0/4191690330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ca806d460 0x7f4ca806d8f0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0009d90 tx=0x7f4ca002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b029599514cde5f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:38.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.790+0000 7f4ca6000640 1 -- 172.21.15.167:0/4191690330 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca0031600 con 0x7f4ca806d460 2024-05-11T12:14:38.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.790+0000 7f4ca6000640 1 -- 172.21.15.167:0/4191690330 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ca00317a0 con 0x7f4ca806d460 2024-05-11T12:14:38.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.790+0000 7f4ca6000640 1 -- 172.21.15.167:0/4191690330 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca0031b50 con 0x7f4ca806d460 2024-05-11T12:14:38.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.791+0000 7f4cad600640 1 -- 172.21.15.167:0/4191690330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ca806d460 msgr2=0x7f4ca806d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:38.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.791+0000 7f4cad600640 1 --2- 172.21.15.167:0/4191690330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ca806d460 0x7f4ca806d8f0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0009d90 tx=0x7f4ca002a5d0 comp rx=0 tx=0).stop 2024-05-11T12:14:38.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.791+0000 7f4cad600640 1 -- 172.21.15.167:0/4191690330 shutdown_connections 2024-05-11T12:14:38.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.791+0000 7f4cad600640 1 --2- 172.21.15.167:0/4191690330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ca806d460 0x7f4ca806d8f0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:38.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.791+0000 7f4cad600640 1 --2- 172.21.15.167:0/4191690330 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ca80af5d0 0x7f4ca8073ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:38.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.791+0000 7f4cad600640 1 -- 172.21.15.167:0/4191690330 >> 172.21.15.167:0/4191690330 conn(0x7f4ca80cd8f0 msgr2=0x7f4ca80cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:38.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.791+0000 7f4cad600640 1 -- 172.21.15.167:0/4191690330 shutdown_connections 2024-05-11T12:14:38.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.791+0000 7f4cad600640 1 -- 172.21.15.167:0/4191690330 wait complete. 2024-05-11T12:14:38.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.792+0000 7f4cad600640 1 Processor -- start 2024-05-11T12:14:38.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.793+0000 7f4cad600640 1 -- start start 2024-05-11T12:14:38.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.793+0000 7f4cad600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ca806d460 0x7f4ca8154670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:38.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.793+0000 7f4cad600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ca80af1f0 0x7f4ca80b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:38.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.793+0000 7f4cad600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ca80af5d0 0x7f4ca80b51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:38.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.793+0000 7f4cad600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca815f920 con 0x7f4ca806d460 2024-05-11T12:14:38.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.794+0000 7f4cad600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca8160560 con 0x7f4ca80af5d0 2024-05-11T12:14:38.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.794+0000 7f4cad600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4ca815faa0 con 0x7f4ca80af1f0 2024-05-11T12:14:38.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.794+0000 7f4ca6a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ca806d460 0x7f4ca8154670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:38.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.794+0000 7f4ca6a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ca806d460 0x7f4ca8154670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:40540/0 (socket says 172.21.15.167:40540) 2024-05-11T12:14:38.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.794+0000 7f4ca6a00640 1 -- 172.21.15.167:0/112137082 learned_addr learned my addr 172.21.15.167:0/112137082 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:38.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.794+0000 7f4ca6a00640 1 -- 172.21.15.167:0/112137082 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ca80af1f0 msgr2=0x7f4ca80b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:38.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.794+0000 7f4c9fe00640 1 --2- 172.21.15.167:0/112137082 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ca80af1f0 0x7f4ca80b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:38.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.794+0000 7f4ca6a00640 1 --2- 172.21.15.167:0/112137082 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ca80af1f0 0x7f4ca80b4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:38.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.794+0000 7f4ca7400640 1 --2- 172.21.15.167:0/112137082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ca80af5d0 0x7f4ca80b51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:38.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.794+0000 7f4ca6a00640 1 -- 172.21.15.167:0/112137082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ca80af5d0 msgr2=0x7f4ca80b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:38.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.794+0000 7f4ca6a00640 1 --2- 172.21.15.167:0/112137082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ca80af5d0 0x7f4ca80b51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:38.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.794+0000 7f4ca6a00640 1 -- 172.21.15.167:0/112137082 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ca80b59e0 con 0x7f4ca806d460 2024-05-11T12:14:38.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.795+0000 7f4ca6a00640 1 --2- 172.21.15.167:0/112137082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ca806d460 0x7f4ca8154670 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f4c9800c500 tx=0x7f4c9800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:38.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.795+0000 7f4c9ea00640 1 -- 172.21.15.167:0/112137082 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c98017070 con 0x7f4ca806d460 2024-05-11T12:14:38.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.795+0000 7f4c9ea00640 1 -- 172.21.15.167:0/112137082 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c98005ce0 con 0x7f4ca806d460 2024-05-11T12:14:38.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.795+0000 7f4cad600640 1 -- 172.21.15.167:0/112137082 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ca80b0c70 con 0x7f4ca806d460 2024-05-11T12:14:38.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.795+0000 7f4c9ea00640 1 -- 172.21.15.167:0/112137082 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c98007270 con 0x7f4ca806d460 2024-05-11T12:14:38.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.795+0000 7f4cad600640 1 -- 172.21.15.167:0/112137082 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ca80b1230 con 0x7f4ca806d460 2024-05-11T12:14:38.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.796+0000 7f4cad600640 1 -- 172.21.15.167:0/112137082 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ca80b1430 con 0x7f4ca806d460 2024-05-11T12:14:38.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.797+0000 7f4c9ea00640 1 -- 172.21.15.167:0/112137082 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4c9801a070 con 0x7f4ca806d460 2024-05-11T12:14:38.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.798+0000 7f4c9ea00640 1 --2- 172.21.15.167:0/112137082 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4c840762d0 0x7f4c84078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:38.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.798+0000 7f4c9ea00640 1 -- 172.21.15.167:0/112137082 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6101+0+0 (secure 0 0 0) 0x7f4c98096a30 con 0x7f4ca806d460 2024-05-11T12:14:38.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.799+0000 7f4c9fe00640 1 --2- 172.21.15.167:0/112137082 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4c840762d0 0x7f4c84078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:38.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.799+0000 7f4c9fe00640 1 --2- 172.21.15.167:0/112137082 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4c840762d0 0x7f4c84078790 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4c90006fd0 tx=0x7f4c90008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:38.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:38.803+0000 7f4c9ea00640 1 -- 172.21.15.167:0/112137082 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4c9805fc40 con 0x7f4ca806d460 2024-05-11T12:14:39.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.007+0000 7f4cad600640 1 -- 172.21.15.167:0/112137082 --> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4ca8069590 con 0x7f4c840762d0 2024-05-11T12:14:39.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.014+0000 7f4c9ea00640 1 -- 172.21.15.167:0/112137082 <== mgr.4106 v2:172.21.15.167:6832/578352559 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248374 (secure 0 0 0) 0x7f4ca8069590 con 0x7f4c840762d0 2024-05-11T12:14:39.017 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:39.018 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2024-05-11T12:14:39.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.020+0000 7f4cad600640 1 -- 172.21.15.167:0/112137082 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4c840762d0 msgr2=0x7f4c84078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:39.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.020+0000 7f4cad600640 1 --2- 172.21.15.167:0/112137082 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4c840762d0 0x7f4c84078790 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4c90006fd0 tx=0x7f4c90008040 comp rx=0 tx=0).stop 2024-05-11T12:14:39.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.020+0000 7f4cad600640 1 -- 172.21.15.167:0/112137082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ca806d460 msgr2=0x7f4ca8154670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:39.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.020+0000 7f4cad600640 1 --2- 172.21.15.167:0/112137082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ca806d460 0x7f4ca8154670 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f4c9800c500 tx=0x7f4c9800c9c0 comp rx=0 tx=0).stop 2024-05-11T12:14:39.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.020+0000 7f4cad600640 1 -- 172.21.15.167:0/112137082 shutdown_connections 2024-05-11T12:14:39.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.020+0000 7f4cad600640 1 --2- 172.21.15.167:0/112137082 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4c840762d0 0x7f4c84078790 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:39.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.020+0000 7f4cad600640 1 --2- 172.21.15.167:0/112137082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ca80af5d0 0x7f4ca80b51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:39.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.020+0000 7f4cad600640 1 --2- 172.21.15.167:0/112137082 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ca80af1f0 0x7f4ca80b4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:39.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.020+0000 7f4cad600640 1 --2- 172.21.15.167:0/112137082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ca806d460 0x7f4ca8154670 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:39.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.020+0000 7f4cad600640 1 -- 172.21.15.167:0/112137082 >> 172.21.15.167:0/112137082 conn(0x7f4ca80cd8f0 msgr2=0x7f4ca815d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:39.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.020+0000 7f4cad600640 1 -- 172.21.15.167:0/112137082 shutdown_connections 2024-05-11T12:14:39.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.021+0000 7f4cad600640 1 -- 172.21.15.167:0/112137082 wait complete. 2024-05-11T12:14:39.037 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":213,"stamp":"2024-05-11T12:14:37.307961+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590950,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":110,"num_write_kb":1629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82,"ondisk_log_size":82,"up":180,"acting":180,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":98,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259828,"kb_used_data":10352,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714892,"statfs":{"total":773094113280,"available":772828049408,"internally_reserved":0,"allocated":10600448,"data_stored":5438451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.745236"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615834+0000","last_change":"2024-05-11T12:14:33.622872+0000","last_active":"2024-05-11T12:14:34.615834+0000","last_peered":"2024-05-11T12:14:34.615834+0000","last_clean":"2024-05-11T12:14:34.615834+0000","last_became_active":"2024-05-11T12:14:33.619136+0000","last_became_peered":"2024-05-11T12:14:33.619136+0000","last_unstale":"2024-05-11T12:14:34.615834+0000","last_undegraded":"2024-05-11T12:14:34.615834+0000","last_fullsized":"2024-05-11T12:14:34.615834+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:18:44.962195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.601568+0000","last_change":"2024-05-11T12:14:32.601568+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.601568+0000","last_undegraded":"2024-05-11T12:14:32.601568+0000","last_fullsized":"2024-05-11T12:14:32.601568+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615616+0000","last_change":"2024-05-11T12:14:33.626436+0000","last_active":"2024-05-11T12:14:34.615616+0000","last_peered":"2024-05-11T12:14:34.615616+0000","last_clean":"2024-05-11T12:14:34.615616+0000","last_became_active":"2024-05-11T12:14:33.626091+0000","last_became_peered":"2024-05-11T12:14:33.626091+0000","last_unstale":"2024-05-11T12:14:34.615616+0000","last_undegraded":"2024-05-11T12:14:34.615616+0000","last_fullsized":"2024-05-11T12:14:34.615616+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:17:05.023315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.284831+0000","last_change":"2024-05-11T12:14:33.624886+0000","last_active":"2024-05-11T12:14:35.284831+0000","last_peered":"2024-05-11T12:14:35.284831+0000","last_clean":"2024-05-11T12:14:35.284831+0000","last_became_active":"2024-05-11T12:14:33.618826+0000","last_became_peered":"2024-05-11T12:14:33.618826+0000","last_unstale":"2024-05-11T12:14:35.284831+0000","last_undegraded":"2024-05-11T12:14:35.284831+0000","last_fullsized":"2024-05-11T12:14:35.284831+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:56:09.727119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T12:14:32.562048+0000","last_change":"2024-05-11T12:14:32.562048+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.562048+0000","last_undegraded":"2024-05-11T12:14:32.562048+0000","last_fullsized":"2024-05-11T12:14:32.562048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.598150+0000","last_change":"2024-05-11T12:14:32.598150+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.598150+0000","last_undegraded":"2024-05-11T12:14:32.598150+0000","last_fullsized":"2024-05-11T12:14:32.598150+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.602749+0000","last_change":"2024-05-11T12:14:32.602749+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.602749+0000","last_undegraded":"2024-05-11T12:14:32.602749+0000","last_fullsized":"2024-05-11T12:14:32.602749+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.208341+0000","last_change":"2024-05-11T12:14:33.621312+0000","last_active":"2024-05-11T12:14:35.208341+0000","last_peered":"2024-05-11T12:14:35.208341+0000","last_clean":"2024-05-11T12:14:35.208341+0000","last_became_active":"2024-05-11T12:14:33.621021+0000","last_became_peered":"2024-05-11T12:14:33.621021+0000","last_unstale":"2024-05-11T12:14:35.208341+0000","last_undegraded":"2024-05-11T12:14:35.208341+0000","last_fullsized":"2024-05-11T12:14:35.208341+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:54:45.084719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.595249+0000","last_change":"2024-05-11T12:14:32.595249+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.595249+0000","last_undegraded":"2024-05-11T12:14:32.595249+0000","last_fullsized":"2024-05-11T12:14:32.595249+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T12:14:32.562048+0000","last_change":"2024-05-11T12:14:32.562048+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.562048+0000","last_undegraded":"2024-05-11T12:14:32.562048+0000","last_fullsized":"2024-05-11T12:14:32.562048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.592511+0000","last_change":"2024-05-11T12:14:32.592511+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.592511+0000","last_undegraded":"2024-05-11T12:14:32.592511+0000","last_fullsized":"2024-05-11T12:14:32.592511+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.606336+0000","last_change":"2024-05-11T12:14:32.606336+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.606336+0000","last_undegraded":"2024-05-11T12:14:32.606336+0000","last_fullsized":"2024-05-11T12:14:32.606336+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.283623+0000","last_change":"2024-05-11T12:14:33.607961+0000","last_active":"2024-05-11T12:14:35.283623+0000","last_peered":"2024-05-11T12:14:35.283623+0000","last_clean":"2024-05-11T12:14:35.283623+0000","last_became_active":"2024-05-11T12:14:33.607599+0000","last_became_peered":"2024-05-11T12:14:33.607599+0000","last_unstale":"2024-05-11T12:14:35.283623+0000","last_undegraded":"2024-05-11T12:14:35.283623+0000","last_fullsized":"2024-05-11T12:14:35.283623+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:22:15.346218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.600693+0000","last_change":"2024-05-11T12:14:32.600693+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.600693+0000","last_undegraded":"2024-05-11T12:14:32.600693+0000","last_fullsized":"2024-05-11T12:14:32.600693+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.599082+0000","last_change":"2024-05-11T12:14:32.599082+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.599082+0000","last_undegraded":"2024-05-11T12:14:32.599082+0000","last_fullsized":"2024-05-11T12:14:32.599082+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.603185+0000","last_change":"2024-05-11T12:14:32.603185+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.603185+0000","last_undegraded":"2024-05-11T12:14:32.603185+0000","last_fullsized":"2024-05-11T12:14:32.603185+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615878+0000","last_change":"2024-05-11T12:14:33.622549+0000","last_active":"2024-05-11T12:14:34.615878+0000","last_peered":"2024-05-11T12:14:34.615878+0000","last_clean":"2024-05-11T12:14:34.615878+0000","last_became_active":"2024-05-11T12:14:33.619603+0000","last_became_peered":"2024-05-11T12:14:33.619603+0000","last_unstale":"2024-05-11T12:14:34.615878+0000","last_undegraded":"2024-05-11T12:14:34.615878+0000","last_fullsized":"2024-05-11T12:14:34.615878+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:51:30.951966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.600768+0000","last_change":"2024-05-11T12:14:32.600768+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.600768+0000","last_undegraded":"2024-05-11T12:14:32.600768+0000","last_fullsized":"2024-05-11T12:14:32.600768+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615768+0000","last_change":"2024-05-11T12:14:33.635269+0000","last_active":"2024-05-11T12:14:34.615768+0000","last_peered":"2024-05-11T12:14:34.615768+0000","last_clean":"2024-05-11T12:14:34.615768+0000","last_became_active":"2024-05-11T12:14:33.634795+0000","last_became_peered":"2024-05-11T12:14:33.634795+0000","last_unstale":"2024-05-11T12:14:34.615768+0000","last_undegraded":"2024-05-11T12:14:34.615768+0000","last_fullsized":"2024-05-11T12:14:34.615768+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:13:16.630356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.605840+0000","last_change":"2024-05-11T12:14:32.605840+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.605840+0000","last_undegraded":"2024-05-11T12:14:32.605840+0000","last_fullsized":"2024-05-11T12:14:32.605840+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.208740+0000","last_change":"2024-05-11T12:14:33.621162+0000","last_active":"2024-05-11T12:14:35.208740+0000","last_peered":"2024-05-11T12:14:35.208740+0000","last_clean":"2024-05-11T12:14:35.208740+0000","last_became_active":"2024-05-11T12:14:33.620843+0000","last_became_peered":"2024-05-11T12:14:33.620843+0000","last_unstale":"2024-05-11T12:14:35.208740+0000","last_undegraded":"2024-05-11T12:14:35.208740+0000","last_fullsized":"2024-05-11T12:14:35.208740+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:29:39.246958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T12:14:32.562048+0000","last_change":"2024-05-11T12:14:32.562048+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.562048+0000","last_undegraded":"2024-05-11T12:14:32.562048+0000","last_fullsized":"2024-05-11T12:14:32.562048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615236+0000","last_change":"2024-05-11T12:14:33.619917+0000","last_active":"2024-05-11T12:14:34.615236+0000","last_peered":"2024-05-11T12:14:34.615236+0000","last_clean":"2024-05-11T12:14:34.615236+0000","last_became_active":"2024-05-11T12:14:33.619614+0000","last_became_peered":"2024-05-11T12:14:33.619614+0000","last_unstale":"2024-05-11T12:14:34.615236+0000","last_undegraded":"2024-05-11T12:14:34.615236+0000","last_fullsized":"2024-05-11T12:14:34.615236+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:56:17.539601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T12:14:32.562048+0000","last_change":"2024-05-11T12:14:32.562048+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.562048+0000","last_undegraded":"2024-05-11T12:14:32.562048+0000","last_fullsized":"2024-05-11T12:14:32.562048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.607235+0000","last_change":"2024-05-11T12:14:32.607235+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.607235+0000","last_undegraded":"2024-05-11T12:14:32.607235+0000","last_fullsized":"2024-05-11T12:14:32.607235+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.601792+0000","last_change":"2024-05-11T12:14:32.601792+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.601792+0000","last_undegraded":"2024-05-11T12:14:32.601792+0000","last_fullsized":"2024-05-11T12:14:32.601792+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.591419+0000","last_change":"2024-05-11T12:14:32.591419+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.591419+0000","last_undegraded":"2024-05-11T12:14:32.591419+0000","last_fullsized":"2024-05-11T12:14:32.591419+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.595401+0000","last_change":"2024-05-11T12:14:32.595401+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.595401+0000","last_undegraded":"2024-05-11T12:14:32.595401+0000","last_fullsized":"2024-05-11T12:14:32.595401+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209481+0000","last_change":"2024-05-11T12:14:33.623708+0000","last_active":"2024-05-11T12:14:35.209481+0000","last_peered":"2024-05-11T12:14:35.209481+0000","last_clean":"2024-05-11T12:14:35.209481+0000","last_became_active":"2024-05-11T12:14:33.623429+0000","last_became_peered":"2024-05-11T12:14:33.623429+0000","last_unstale":"2024-05-11T12:14:35.209481+0000","last_undegraded":"2024-05-11T12:14:35.209481+0000","last_fullsized":"2024-05-11T12:14:35.209481+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:30:11.411742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.603561+0000","last_change":"2024-05-11T12:14:32.603561+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.603561+0000","last_undegraded":"2024-05-11T12:14:32.603561+0000","last_fullsized":"2024-05-11T12:14:32.603561+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.596127+0000","last_change":"2024-05-11T12:14:32.596127+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.596127+0000","last_undegraded":"2024-05-11T12:14:32.596127+0000","last_fullsized":"2024-05-11T12:14:32.596127+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.208839+0000","last_change":"2024-05-11T12:14:33.621606+0000","last_active":"2024-05-11T12:14:35.208839+0000","last_peered":"2024-05-11T12:14:35.208839+0000","last_clean":"2024-05-11T12:14:35.208839+0000","last_became_active":"2024-05-11T12:14:33.621288+0000","last_became_peered":"2024-05-11T12:14:33.621288+0000","last_unstale":"2024-05-11T12:14:35.208839+0000","last_undegraded":"2024-05-11T12:14:35.208839+0000","last_fullsized":"2024-05-11T12:14:35.208839+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:37:19.519120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615469+0000","last_change":"2024-05-11T12:14:33.633794+0000","last_active":"2024-05-11T12:14:34.615469+0000","last_peered":"2024-05-11T12:14:34.615469+0000","last_clean":"2024-05-11T12:14:34.615469+0000","last_became_active":"2024-05-11T12:14:33.633451+0000","last_became_peered":"2024-05-11T12:14:33.633451+0000","last_unstale":"2024-05-11T12:14:34.615469+0000","last_undegraded":"2024-05-11T12:14:34.615469+0000","last_fullsized":"2024-05-11T12:14:34.615469+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:31:39.465305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"96'4","reported_seq":18,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.285496+0000","last_change":"2024-05-11T12:14:32.610045+0000","last_active":"2024-05-11T12:14:35.285496+0000","last_peered":"2024-05-11T12:14:35.285496+0000","last_clean":"2024-05-11T12:14:35.285496+0000","last_became_active":"2024-05-11T12:14:32.609746+0000","last_became_peered":"2024-05-11T12:14:32.609746+0000","last_unstale":"2024-05-11T12:14:35.285496+0000","last_undegraded":"2024-05-11T12:14:35.285496+0000","last_fullsized":"2024-05-11T12:14:35.285496+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:44:56.684605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.208579+0000","last_change":"2024-05-11T12:14:33.630664+0000","last_active":"2024-05-11T12:14:35.208579+0000","last_peered":"2024-05-11T12:14:35.208579+0000","last_clean":"2024-05-11T12:14:35.208579+0000","last_became_active":"2024-05-11T12:14:33.630171+0000","last_became_peered":"2024-05-11T12:14:33.630171+0000","last_unstale":"2024-05-11T12:14:35.208579+0000","last_undegraded":"2024-05-11T12:14:35.208579+0000","last_fullsized":"2024-05-11T12:14:35.208579+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:00:10.906551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209543+0000","last_change":"2024-05-11T12:14:32.619446+0000","last_active":"2024-05-11T12:14:35.209543+0000","last_peered":"2024-05-11T12:14:35.209543+0000","last_clean":"2024-05-11T12:14:35.209543+0000","last_became_active":"2024-05-11T12:14:32.619219+0000","last_became_peered":"2024-05-11T12:14:32.619219+0000","last_unstale":"2024-05-11T12:14:35.209543+0000","last_undegraded":"2024-05-11T12:14:35.209543+0000","last_fullsized":"2024-05-11T12:14:35.209543+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:34:15.881914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.603030+0000","last_change":"2024-05-11T12:14:32.603030+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.603030+0000","last_undegraded":"2024-05-11T12:14:32.603030+0000","last_fullsized":"2024-05-11T12:14:32.603030+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-11T12:14:32.621991+0000","last_change":"2024-05-11T12:14:32.618332+0000","last_active":"2024-05-11T12:14:32.621991+0000","last_peered":"2024-05-11T12:14:32.621991+0000","last_clean":"2024-05-11T12:14:32.621991+0000","last_became_active":"2024-05-11T12:14:32.614777+0000","last_became_peered":"2024-05-11T12:14:32.614777+0000","last_unstale":"2024-05-11T12:14:32.621991+0000","last_undegraded":"2024-05-11T12:14:32.621991+0000","last_fullsized":"2024-05-11T12:14:32.621991+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:01:31.105667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.591054+0000","last_change":"2024-05-11T12:14:32.591054+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.591054+0000","last_undegraded":"2024-05-11T12:14:32.591054+0000","last_fullsized":"2024-05-11T12:14:32.591054+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-05-11T12:14:31.585654+0000","last_change":"2024-05-11T12:14:31.585654+0000","last_active":"2024-05-11T12:14:31.545700+0000","last_peered":"2024-05-11T12:14:31.545700+0000","last_clean":"2024-05-11T12:14:31.545700+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:31.585654+0000","last_undegraded":"2024-05-11T12:14:31.585654+0000","last_fullsized":"2024-05-11T12:14:31.585654+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.284849+0000","last_change":"2024-05-11T12:14:33.616596+0000","last_active":"2024-05-11T12:14:35.284849+0000","last_peered":"2024-05-11T12:14:35.284849+0000","last_clean":"2024-05-11T12:14:35.284849+0000","last_became_active":"2024-05-11T12:14:33.616294+0000","last_became_peered":"2024-05-11T12:14:33.616294+0000","last_unstale":"2024-05-11T12:14:35.284849+0000","last_undegraded":"2024-05-11T12:14:35.284849+0000","last_fullsized":"2024-05-11T12:14:35.284849+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:27:47.349240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-11T12:14:32.621828+0000","last_change":"2024-05-11T12:14:32.619606+0000","last_active":"2024-05-11T12:14:32.621828+0000","last_peered":"2024-05-11T12:14:32.621828+0000","last_clean":"2024-05-11T12:14:32.621828+0000","last_became_active":"2024-05-11T12:14:32.619133+0000","last_became_peered":"2024-05-11T12:14:32.619133+0000","last_unstale":"2024-05-11T12:14:32.621828+0000","last_undegraded":"2024-05-11T12:14:32.621828+0000","last_fullsized":"2024-05-11T12:14:32.621828+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:00:56.111300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.616178+0000","last_change":"2024-05-11T12:14:33.634226+0000","last_active":"2024-05-11T12:14:34.616178+0000","last_peered":"2024-05-11T12:14:34.616178+0000","last_clean":"2024-05-11T12:14:34.616178+0000","last_became_active":"2024-05-11T12:14:33.633932+0000","last_became_peered":"2024-05-11T12:14:33.633932+0000","last_unstale":"2024-05-11T12:14:34.616178+0000","last_undegraded":"2024-05-11T12:14:34.616178+0000","last_fullsized":"2024-05-11T12:14:34.616178+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:01:12.394223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-11T12:14:32.616593+0000","last_change":"2024-05-11T12:14:32.614159+0000","last_active":"2024-05-11T12:14:32.616593+0000","last_peered":"2024-05-11T12:14:32.616593+0000","last_clean":"2024-05-11T12:14:32.616593+0000","last_became_active":"2024-05-11T12:14:32.613421+0000","last_became_peered":"2024-05-11T12:14:32.613421+0000","last_unstale":"2024-05-11T12:14:32.616593+0000","last_undegraded":"2024-05-11T12:14:32.616593+0000","last_fullsized":"2024-05-11T12:14:32.616593+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:58:41.059975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.599834+0000","last_change":"2024-05-11T12:14:32.599834+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.599834+0000","last_undegraded":"2024-05-11T12:14:32.599834+0000","last_fullsized":"2024-05-11T12:14:32.599834+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-11T12:14:32.620324+0000","last_change":"2024-05-11T12:14:32.614638+0000","last_active":"2024-05-11T12:14:32.620324+0000","last_peered":"2024-05-11T12:14:32.620324+0000","last_clean":"2024-05-11T12:14:32.620324+0000","last_became_active":"2024-05-11T12:14:32.614242+0000","last_became_peered":"2024-05-11T12:14:32.614242+0000","last_unstale":"2024-05-11T12:14:32.620324+0000","last_undegraded":"2024-05-11T12:14:32.620324+0000","last_fullsized":"2024-05-11T12:14:32.620324+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:45:58.666366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.285649+0000","last_change":"2024-05-11T12:14:33.615167+0000","last_active":"2024-05-11T12:14:35.285649+0000","last_peered":"2024-05-11T12:14:35.285649+0000","last_clean":"2024-05-11T12:14:35.285649+0000","last_became_active":"2024-05-11T12:14:33.614898+0000","last_became_peered":"2024-05-11T12:14:33.614898+0000","last_unstale":"2024-05-11T12:14:35.285649+0000","last_undegraded":"2024-05-11T12:14:35.285649+0000","last_fullsized":"2024-05-11T12:14:35.285649+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:52:46.883659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.283863+0000","last_change":"2024-05-11T12:14:32.605627+0000","last_active":"2024-05-11T12:14:35.283863+0000","last_peered":"2024-05-11T12:14:35.283863+0000","last_clean":"2024-05-11T12:14:35.283863+0000","last_became_active":"2024-05-11T12:14:32.604729+0000","last_became_peered":"2024-05-11T12:14:32.604729+0000","last_unstale":"2024-05-11T12:14:35.283863+0000","last_undegraded":"2024-05-11T12:14:35.283863+0000","last_fullsized":"2024-05-11T12:14:35.283863+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:00:56.111300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615229+0000","last_change":"2024-05-11T12:14:33.635074+0000","last_active":"2024-05-11T12:14:34.615229+0000","last_peered":"2024-05-11T12:14:34.615229+0000","last_clean":"2024-05-11T12:14:34.615229+0000","last_became_active":"2024-05-11T12:14:33.634606+0000","last_became_peered":"2024-05-11T12:14:33.634606+0000","last_unstale":"2024-05-11T12:14:34.615229+0000","last_undegraded":"2024-05-11T12:14:34.615229+0000","last_fullsized":"2024-05-11T12:14:34.615229+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:03:07.386179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-11T12:14:32.620414+0000","last_change":"2024-05-11T12:14:32.615191+0000","last_active":"2024-05-11T12:14:32.620414+0000","last_peered":"2024-05-11T12:14:32.620414+0000","last_clean":"2024-05-11T12:14:32.620414+0000","last_became_active":"2024-05-11T12:14:32.614809+0000","last_became_peered":"2024-05-11T12:14:32.614809+0000","last_unstale":"2024-05-11T12:14:32.620414+0000","last_undegraded":"2024-05-11T12:14:32.620414+0000","last_fullsized":"2024-05-11T12:14:32.620414+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:15:55.175677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T12:14:32.562048+0000","last_change":"2024-05-11T12:14:32.562048+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.562048+0000","last_undegraded":"2024-05-11T12:14:32.562048+0000","last_fullsized":"2024-05-11T12:14:32.562048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615602+0000","last_change":"2024-05-11T12:14:32.609650+0000","last_active":"2024-05-11T12:14:34.615602+0000","last_peered":"2024-05-11T12:14:34.615602+0000","last_clean":"2024-05-11T12:14:34.615602+0000","last_became_active":"2024-05-11T12:14:32.609340+0000","last_became_peered":"2024-05-11T12:14:32.609340+0000","last_unstale":"2024-05-11T12:14:34.615602+0000","last_undegraded":"2024-05-11T12:14:34.615602+0000","last_fullsized":"2024-05-11T12:14:34.615602+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:40:56.223685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209330+0000","last_change":"2024-05-11T12:14:33.631013+0000","last_active":"2024-05-11T12:14:35.209330+0000","last_peered":"2024-05-11T12:14:35.209330+0000","last_clean":"2024-05-11T12:14:35.209330+0000","last_became_active":"2024-05-11T12:14:33.630698+0000","last_became_peered":"2024-05-11T12:14:33.630698+0000","last_unstale":"2024-05-11T12:14:35.209330+0000","last_undegraded":"2024-05-11T12:14:35.209330+0000","last_fullsized":"2024-05-11T12:14:35.209330+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:32:48.749691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-11T12:14:32.621778+0000","last_change":"2024-05-11T12:14:32.618526+0000","last_active":"2024-05-11T12:14:32.621778+0000","last_peered":"2024-05-11T12:14:32.621778+0000","last_clean":"2024-05-11T12:14:32.621778+0000","last_became_active":"2024-05-11T12:14:32.618198+0000","last_became_peered":"2024-05-11T12:14:32.618198+0000","last_unstale":"2024-05-11T12:14:32.621778+0000","last_undegraded":"2024-05-11T12:14:32.621778+0000","last_fullsized":"2024-05-11T12:14:32.621778+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:57:12.465310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T12:14:32.562048+0000","last_change":"2024-05-11T12:14:32.562048+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.562048+0000","last_undegraded":"2024-05-11T12:14:32.562048+0000","last_fullsized":"2024-05-11T12:14:32.562048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-11T12:14:32.613159+0000","last_change":"2024-05-11T12:14:32.607489+0000","last_active":"2024-05-11T12:14:32.613159+0000","last_peered":"2024-05-11T12:14:32.613159+0000","last_clean":"2024-05-11T12:14:32.613159+0000","last_became_active":"2024-05-11T12:14:32.606078+0000","last_became_peered":"2024-05-11T12:14:32.606078+0000","last_unstale":"2024-05-11T12:14:32.613159+0000","last_undegraded":"2024-05-11T12:14:32.613159+0000","last_fullsized":"2024-05-11T12:14:32.613159+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:07:30.136729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.591204+0000","last_change":"2024-05-11T12:14:32.591204+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.591204+0000","last_undegraded":"2024-05-11T12:14:32.591204+0000","last_fullsized":"2024-05-11T12:14:32.591204+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-05-11T12:14:31.584097+0000","last_change":"2024-05-11T12:14:31.584097+0000","last_active":"2024-05-11T12:14:31.545700+0000","last_peered":"2024-05-11T12:14:31.545700+0000","last_clean":"2024-05-11T12:14:31.545700+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:31.584097+0000","last_undegraded":"2024-05-11T12:14:31.584097+0000","last_fullsized":"2024-05-11T12:14:31.584097+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.594883+0000","last_change":"2024-05-11T12:14:32.594883+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.594883+0000","last_undegraded":"2024-05-11T12:14:32.594883+0000","last_fullsized":"2024-05-11T12:14:32.594883+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.284356+0000","last_change":"2024-05-11T12:14:32.616922+0000","last_active":"2024-05-11T12:14:35.284356+0000","last_peered":"2024-05-11T12:14:35.284356+0000","last_clean":"2024-05-11T12:14:35.284356+0000","last_became_active":"2024-05-11T12:14:32.616599+0000","last_became_peered":"2024-05-11T12:14:32.616599+0000","last_unstale":"2024-05-11T12:14:35.284356+0000","last_undegraded":"2024-05-11T12:14:35.284356+0000","last_fullsized":"2024-05-11T12:14:35.284356+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:55:59.198545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209297+0000","last_change":"2024-05-11T12:14:33.624455+0000","last_active":"2024-05-11T12:14:35.209297+0000","last_peered":"2024-05-11T12:14:35.209297+0000","last_clean":"2024-05-11T12:14:35.209297+0000","last_became_active":"2024-05-11T12:14:33.624158+0000","last_became_peered":"2024-05-11T12:14:33.624158+0000","last_unstale":"2024-05-11T12:14:35.209297+0000","last_undegraded":"2024-05-11T12:14:35.209297+0000","last_fullsized":"2024-05-11T12:14:35.209297+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:44:32.910714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.208576+0000","last_change":"2024-05-11T12:14:32.615486+0000","last_active":"2024-05-11T12:14:35.208576+0000","last_peered":"2024-05-11T12:14:35.208576+0000","last_clean":"2024-05-11T12:14:35.208576+0000","last_became_active":"2024-05-11T12:14:32.615204+0000","last_became_peered":"2024-05-11T12:14:32.615204+0000","last_unstale":"2024-05-11T12:14:35.208576+0000","last_undegraded":"2024-05-11T12:14:35.208576+0000","last_fullsized":"2024-05-11T12:14:35.208576+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:27:00.766407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615938+0000","last_change":"2024-05-11T12:14:33.623140+0000","last_active":"2024-05-11T12:14:34.615938+0000","last_peered":"2024-05-11T12:14:34.615938+0000","last_clean":"2024-05-11T12:14:34.615938+0000","last_became_active":"2024-05-11T12:14:33.618444+0000","last_became_peered":"2024-05-11T12:14:33.618444+0000","last_unstale":"2024-05-11T12:14:34.615938+0000","last_undegraded":"2024-05-11T12:14:34.615938+0000","last_fullsized":"2024-05-11T12:14:34.615938+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:35:51.107577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-11T12:14:32.620528+0000","last_change":"2024-05-11T12:14:32.616263+0000","last_active":"2024-05-11T12:14:32.620528+0000","last_peered":"2024-05-11T12:14:32.620528+0000","last_clean":"2024-05-11T12:14:32.620528+0000","last_became_active":"2024-05-11T12:14:32.615004+0000","last_became_peered":"2024-05-11T12:14:32.615004+0000","last_unstale":"2024-05-11T12:14:32.620528+0000","last_undegraded":"2024-05-11T12:14:32.620528+0000","last_fullsized":"2024-05-11T12:14:32.620528+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:07:30.136729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.208800+0000","last_change":"2024-05-11T12:14:33.616435+0000","last_active":"2024-05-11T12:14:35.208800+0000","last_peered":"2024-05-11T12:14:35.208800+0000","last_clean":"2024-05-11T12:14:35.208800+0000","last_became_active":"2024-05-11T12:14:33.616110+0000","last_became_peered":"2024-05-11T12:14:33.616110+0000","last_unstale":"2024-05-11T12:14:35.208800+0000","last_undegraded":"2024-05-11T12:14:35.208800+0000","last_fullsized":"2024-05-11T12:14:35.208800+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:51:10.706767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-11T12:14:32.616852+0000","last_change":"2024-05-11T12:14:32.613751+0000","last_active":"2024-05-11T12:14:32.616852+0000","last_peered":"2024-05-11T12:14:32.616852+0000","last_clean":"2024-05-11T12:14:32.616852+0000","last_became_active":"2024-05-11T12:14:32.613431+0000","last_became_peered":"2024-05-11T12:14:32.613431+0000","last_unstale":"2024-05-11T12:14:32.616852+0000","last_undegraded":"2024-05-11T12:14:32.616852+0000","last_fullsized":"2024-05-11T12:14:32.616852+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:47:05.267174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.597272+0000","last_change":"2024-05-11T12:14:32.597272+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.597272+0000","last_undegraded":"2024-05-11T12:14:32.597272+0000","last_fullsized":"2024-05-11T12:14:32.597272+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-11T12:14:32.626697+0000","last_change":"2024-05-11T12:14:32.621615+0000","last_active":"2024-05-11T12:14:32.626697+0000","last_peered":"2024-05-11T12:14:32.626697+0000","last_clean":"2024-05-11T12:14:32.626697+0000","last_became_active":"2024-05-11T12:14:32.620973+0000","last_became_peered":"2024-05-11T12:14:32.620973+0000","last_unstale":"2024-05-11T12:14:32.626697+0000","last_undegraded":"2024-05-11T12:14:32.626697+0000","last_fullsized":"2024-05-11T12:14:32.626697+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:44:56.684605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T12:14:32.562048+0000","last_change":"2024-05-11T12:14:32.562048+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.562048+0000","last_undegraded":"2024-05-11T12:14:32.562048+0000","last_fullsized":"2024-05-11T12:14:32.562048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-11T12:14:32.612302+0000","last_change":"2024-05-11T12:14:32.609813+0000","last_active":"2024-05-11T12:14:32.612302+0000","last_peered":"2024-05-11T12:14:32.612302+0000","last_clean":"2024-05-11T12:14:32.612302+0000","last_became_active":"2024-05-11T12:14:32.609531+0000","last_became_peered":"2024-05-11T12:14:32.609531+0000","last_unstale":"2024-05-11T12:14:32.612302+0000","last_undegraded":"2024-05-11T12:14:32.612302+0000","last_fullsized":"2024-05-11T12:14:32.612302+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:54:11.073754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615796+0000","last_change":"2024-05-11T12:14:33.629837+0000","last_active":"2024-05-11T12:14:34.615796+0000","last_peered":"2024-05-11T12:14:34.615796+0000","last_clean":"2024-05-11T12:14:34.615796+0000","last_became_active":"2024-05-11T12:14:33.629561+0000","last_became_peered":"2024-05-11T12:14:33.629561+0000","last_unstale":"2024-05-11T12:14:34.615796+0000","last_undegraded":"2024-05-11T12:14:34.615796+0000","last_fullsized":"2024-05-11T12:14:34.615796+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:24:04.957521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209466+0000","last_change":"2024-05-11T12:14:32.605666+0000","last_active":"2024-05-11T12:14:35.209466+0000","last_peered":"2024-05-11T12:14:35.209466+0000","last_clean":"2024-05-11T12:14:35.209466+0000","last_became_active":"2024-05-11T12:14:32.605407+0000","last_became_peered":"2024-05-11T12:14:32.605407+0000","last_unstale":"2024-05-11T12:14:35.209466+0000","last_undegraded":"2024-05-11T12:14:35.209466+0000","last_fullsized":"2024-05-11T12:14:35.209466+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:31:24.122325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615946+0000","last_change":"2024-05-11T12:14:33.623726+0000","last_active":"2024-05-11T12:14:34.615946+0000","last_peered":"2024-05-11T12:14:34.615946+0000","last_clean":"2024-05-11T12:14:34.615946+0000","last_became_active":"2024-05-11T12:14:33.616682+0000","last_became_peered":"2024-05-11T12:14:33.616682+0000","last_unstale":"2024-05-11T12:14:34.615946+0000","last_undegraded":"2024-05-11T12:14:34.615946+0000","last_fullsized":"2024-05-11T12:14:34.615946+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:11:09.187757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.208810+0000","last_change":"2024-05-11T12:14:32.612947+0000","last_active":"2024-05-11T12:14:35.208810+0000","last_peered":"2024-05-11T12:14:35.208810+0000","last_clean":"2024-05-11T12:14:35.208810+0000","last_became_active":"2024-05-11T12:14:32.612545+0000","last_became_peered":"2024-05-11T12:14:32.612545+0000","last_unstale":"2024-05-11T12:14:35.208810+0000","last_undegraded":"2024-05-11T12:14:35.208810+0000","last_fullsized":"2024-05-11T12:14:35.208810+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:10:08.253641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.285245+0000","last_change":"2024-05-11T12:14:33.628473+0000","last_active":"2024-05-11T12:14:35.285245+0000","last_peered":"2024-05-11T12:14:35.285245+0000","last_clean":"2024-05-11T12:14:35.285245+0000","last_became_active":"2024-05-11T12:14:33.627301+0000","last_became_peered":"2024-05-11T12:14:33.627301+0000","last_unstale":"2024-05-11T12:14:35.285245+0000","last_undegraded":"2024-05-11T12:14:35.285245+0000","last_fullsized":"2024-05-11T12:14:35.285245+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:16:12.500928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-11T12:14:32.613383+0000","last_change":"2024-05-11T12:14:32.607325+0000","last_active":"2024-05-11T12:14:32.613383+0000","last_peered":"2024-05-11T12:14:32.613383+0000","last_clean":"2024-05-11T12:14:32.613383+0000","last_became_active":"2024-05-11T12:14:32.606982+0000","last_became_peered":"2024-05-11T12:14:32.606982+0000","last_unstale":"2024-05-11T12:14:32.613383+0000","last_undegraded":"2024-05-11T12:14:32.613383+0000","last_fullsized":"2024-05-11T12:14:32.613383+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:15:55.175677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209260+0000","last_change":"2024-05-11T12:14:33.618158+0000","last_active":"2024-05-11T12:14:35.209260+0000","last_peered":"2024-05-11T12:14:35.209260+0000","last_clean":"2024-05-11T12:14:35.209260+0000","last_became_active":"2024-05-11T12:14:33.617850+0000","last_became_peered":"2024-05-11T12:14:33.617850+0000","last_unstale":"2024-05-11T12:14:35.209260+0000","last_undegraded":"2024-05-11T12:14:35.209260+0000","last_fullsized":"2024-05-11T12:14:35.209260+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:38:25.159336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209121+0000","last_change":"2024-05-11T12:14:32.607439+0000","last_active":"2024-05-11T12:14:35.209121+0000","last_peered":"2024-05-11T12:14:35.209121+0000","last_clean":"2024-05-11T12:14:35.209121+0000","last_became_active":"2024-05-11T12:14:32.606952+0000","last_became_peered":"2024-05-11T12:14:32.606952+0000","last_unstale":"2024-05-11T12:14:35.209121+0000","last_undegraded":"2024-05-11T12:14:35.209121+0000","last_fullsized":"2024-05-11T12:14:35.209121+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:14:57.743590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615535+0000","last_change":"2024-05-11T12:14:33.619178+0000","last_active":"2024-05-11T12:14:34.615535+0000","last_peered":"2024-05-11T12:14:34.615535+0000","last_clean":"2024-05-11T12:14:34.615535+0000","last_became_active":"2024-05-11T12:14:33.618894+0000","last_became_peered":"2024-05-11T12:14:33.618894+0000","last_unstale":"2024-05-11T12:14:34.615535+0000","last_undegraded":"2024-05-11T12:14:34.615535+0000","last_fullsized":"2024-05-11T12:14:34.615535+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:10:50.170349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-05-11T12:14:31.584956+0000","last_change":"2024-05-11T12:14:31.584956+0000","last_active":"2024-05-11T12:14:31.545700+0000","last_peered":"2024-05-11T12:14:31.545700+0000","last_clean":"2024-05-11T12:14:31.545700+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:31.584956+0000","last_undegraded":"2024-05-11T12:14:31.584956+0000","last_fullsized":"2024-05-11T12:14:31.584956+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.602318+0000","last_change":"2024-05-11T12:14:32.602318+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.602318+0000","last_undegraded":"2024-05-11T12:14:32.602318+0000","last_fullsized":"2024-05-11T12:14:32.602318+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"97'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615678+0000","last_change":"2024-05-11T12:14:32.605888+0000","last_active":"2024-05-11T12:14:34.615678+0000","last_peered":"2024-05-11T12:14:34.615678+0000","last_clean":"2024-05-11T12:14:34.615678+0000","last_became_active":"2024-05-11T12:14:32.605588+0000","last_became_peered":"2024-05-11T12:14:32.605588+0000","last_unstale":"2024-05-11T12:14:34.615678+0000","last_undegraded":"2024-05-11T12:14:34.615678+0000","last_fullsized":"2024-05-11T12:14:34.615678+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:56:02.427960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.284207+0000","last_change":"2024-05-11T12:14:33.627479+0000","last_active":"2024-05-11T12:14:35.284207+0000","last_peered":"2024-05-11T12:14:35.284207+0000","last_clean":"2024-05-11T12:14:35.284207+0000","last_became_active":"2024-05-11T12:14:33.627179+0000","last_became_peered":"2024-05-11T12:14:33.627179+0000","last_unstale":"2024-05-11T12:14:35.284207+0000","last_undegraded":"2024-05-11T12:14:35.284207+0000","last_fullsized":"2024-05-11T12:14:35.284207+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:35:20.045762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-11T12:14:32.612410+0000","last_change":"2024-05-11T12:14:32.605594+0000","last_active":"2024-05-11T12:14:32.612410+0000","last_peered":"2024-05-11T12:14:32.612410+0000","last_clean":"2024-05-11T12:14:32.612410+0000","last_became_active":"2024-05-11T12:14:32.605097+0000","last_became_peered":"2024-05-11T12:14:32.605097+0000","last_unstale":"2024-05-11T12:14:32.612410+0000","last_undegraded":"2024-05-11T12:14:32.612410+0000","last_fullsized":"2024-05-11T12:14:32.612410+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:05:15.832383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.592181+0000","last_change":"2024-05-11T12:14:32.592181+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.592181+0000","last_undegraded":"2024-05-11T12:14:32.592181+0000","last_fullsized":"2024-05-11T12:14:32.592181+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-05-11T12:14:31.584482+0000","last_change":"2024-05-11T12:14:31.584482+0000","last_active":"2024-05-11T12:14:31.545700+0000","last_peered":"2024-05-11T12:14:31.545700+0000","last_clean":"2024-05-11T12:14:31.545700+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:31.584482+0000","last_undegraded":"2024-05-11T12:14:31.584482+0000","last_fullsized":"2024-05-11T12:14:31.584482+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.208801+0000","last_change":"2024-05-11T12:14:33.630741+0000","last_active":"2024-05-11T12:14:35.208801+0000","last_peered":"2024-05-11T12:14:35.208801+0000","last_clean":"2024-05-11T12:14:35.208801+0000","last_became_active":"2024-05-11T12:14:33.630423+0000","last_became_peered":"2024-05-11T12:14:33.630423+0000","last_unstale":"2024-05-11T12:14:35.208801+0000","last_undegraded":"2024-05-11T12:14:35.208801+0000","last_fullsized":"2024-05-11T12:14:35.208801+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:47:57.386158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-11T12:14:32.621836+0000","last_change":"2024-05-11T12:14:32.616942+0000","last_active":"2024-05-11T12:14:32.621836+0000","last_peered":"2024-05-11T12:14:32.621836+0000","last_clean":"2024-05-11T12:14:32.621836+0000","last_became_active":"2024-05-11T12:14:32.616577+0000","last_became_peered":"2024-05-11T12:14:32.616577+0000","last_unstale":"2024-05-11T12:14:32.621836+0000","last_undegraded":"2024-05-11T12:14:32.621836+0000","last_fullsized":"2024-05-11T12:14:32.621836+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:47:05.267174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.208768+0000","last_change":"2024-05-11T12:14:33.620388+0000","last_active":"2024-05-11T12:14:35.208768+0000","last_peered":"2024-05-11T12:14:35.208768+0000","last_clean":"2024-05-11T12:14:35.208768+0000","last_became_active":"2024-05-11T12:14:33.618621+0000","last_became_peered":"2024-05-11T12:14:33.618621+0000","last_unstale":"2024-05-11T12:14:35.208768+0000","last_undegraded":"2024-05-11T12:14:35.208768+0000","last_fullsized":"2024-05-11T12:14:35.208768+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:57:26.186975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209282+0000","last_change":"2024-05-11T12:14:32.615893+0000","last_active":"2024-05-11T12:14:35.209282+0000","last_peered":"2024-05-11T12:14:35.209282+0000","last_clean":"2024-05-11T12:14:35.209282+0000","last_became_active":"2024-05-11T12:14:32.615639+0000","last_became_peered":"2024-05-11T12:14:32.615639+0000","last_unstale":"2024-05-11T12:14:35.209282+0000","last_undegraded":"2024-05-11T12:14:35.209282+0000","last_fullsized":"2024-05-11T12:14:35.209282+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:51:28.062961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.601116+0000","last_change":"2024-05-11T12:14:32.601116+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.601116+0000","last_undegraded":"2024-05-11T12:14:32.601116+0000","last_fullsized":"2024-05-11T12:14:32.601116+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-11T12:14:32.613358+0000","last_change":"2024-05-11T12:14:32.607229+0000","last_active":"2024-05-11T12:14:32.613358+0000","last_peered":"2024-05-11T12:14:32.613358+0000","last_clean":"2024-05-11T12:14:32.613358+0000","last_became_active":"2024-05-11T12:14:32.606938+0000","last_became_peered":"2024-05-11T12:14:32.606938+0000","last_unstale":"2024-05-11T12:14:32.613358+0000","last_undegraded":"2024-05-11T12:14:32.613358+0000","last_fullsized":"2024-05-11T12:14:32.613358+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:45:58.666366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.593567+0000","last_change":"2024-05-11T12:14:32.593567+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.593567+0000","last_undegraded":"2024-05-11T12:14:32.593567+0000","last_fullsized":"2024-05-11T12:14:32.593567+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209088+0000","last_change":"2024-05-11T12:14:32.610511+0000","last_active":"2024-05-11T12:14:35.209088+0000","last_peered":"2024-05-11T12:14:35.209088+0000","last_clean":"2024-05-11T12:14:35.209088+0000","last_became_active":"2024-05-11T12:14:32.610243+0000","last_became_peered":"2024-05-11T12:14:32.610243+0000","last_unstale":"2024-05-11T12:14:35.209088+0000","last_undegraded":"2024-05-11T12:14:35.209088+0000","last_fullsized":"2024-05-11T12:14:35.209088+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:39:24.599004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-05-11T12:14:32.589733+0000","last_change":"2024-05-11T12:14:32.589733+0000","last_active":"2024-05-11T12:14:32.562048+0000","last_peered":"2024-05-11T12:14:32.562048+0000","last_clean":"2024-05-11T12:14:32.562048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T12:14:32.589733+0000","last_undegraded":"2024-05-11T12:14:32.589733+0000","last_fullsized":"2024-05-11T12:14:32.589733+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209535+0000","last_change":"2024-05-11T12:14:32.612907+0000","last_active":"2024-05-11T12:14:35.209535+0000","last_peered":"2024-05-11T12:14:35.209535+0000","last_clean":"2024-05-11T12:14:35.209535+0000","last_became_active":"2024-05-11T12:14:32.612548+0000","last_became_peered":"2024-05-11T12:14:32.612548+0000","last_unstale":"2024-05-11T12:14:35.209535+0000","last_undegraded":"2024-05-11T12:14:35.209535+0000","last_fullsized":"2024-05-11T12:14:35.209535+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:49:45.457016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":243,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-11T12:14:32.605441+0000","last_change":"2024-05-11T12:10:35.066673+0000","last_active":"2024-05-11T12:14:32.605441+0000","last_peered":"2024-05-11T12:14:32.605441+0000","last_clean":"2024-05-11T12:14:32.605441+0000","last_became_active":"2024-05-11T12:10:35.065942+0000","last_became_peered":"2024-05-11T12:10:35.065942+0000","last_unstale":"2024-05-11T12:14:32.605441+0000","last_undegraded":"2024-05-11T12:14:32.605441+0000","last_fullsized":"2024-05-11T12:14:32.605441+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_clean_scrub_stamp":"2024-05-11T12:10:33.601136+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:12:02.133182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":114,"acting":114,"num_store_stats":7},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476787,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32900,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338940,"statfs":{"total":96636764160,"available":96603074560,"internally_reserved":0,"allocated":1761280,"data_stored":1118705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476788,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32324,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1167360,"data_stored":528337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476787,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32324,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1167360,"data_stored":528337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476789,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32324,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1167360,"data_stored":528337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509493,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1196032,"data_stored":542778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509493,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32304,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339536,"statfs":{"total":96636764160,"available":96603684864,"internally_reserved":0,"allocated":1142784,"data_stored":515461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509493,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32948,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338892,"statfs":{"total":96636764160,"available":96603025408,"internally_reserved":0,"allocated":1802240,"data_stored":1134266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542197,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1196032,"data_stored":542230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T12:14:39.041 INFO:tasks.ceph.ceph_manager.ceph:PG 16.3e is not active+clean 2024-05-11T12:14:39.041 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:14:32.601568+0000', 'last_change': '2024-05-11T12:14:32.601568+0000', 'last_active': '2024-05-11T12:14:32.562048+0000', 'last_peered': '2024-05-11T12:14:32.562048+0000', 'last_clean': '2024-05-11T12:14:32.562048+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:14:32.601568+0000', 'last_undegraded': '2024-05-11T12:14:32.601568+0000', 'last_fullsized': '2024-05-11T12:14:32.601568+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_clean_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-11T12:14:39.041 INFO:tasks.ceph.ceph_manager.ceph:PG 16.3b is not active+clean 2024-05-11T12:14:39.042 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T12:14:32.562048+0000', 'last_change': '2024-05-11T12:14:32.562048+0000', 'last_active': '2024-05-11T12:14:32.562048+0000', 'last_peered': '2024-05-11T12:14:32.562048+0000', 'last_clean': '2024-05-11T12:14:32.562048+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:14:32.562048+0000', 'last_undegraded': '2024-05-11T12:14:32.562048+0000', 'last_fullsized': '2024-05-11T12:14:32.562048+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_clean_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T12:14:39.042 INFO:tasks.ceph.ceph_manager.ceph:PG 16.3a is not active+clean 2024-05-11T12:14:39.042 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:14:32.598150+0000', 'last_change': '2024-05-11T12:14:32.598150+0000', 'last_active': '2024-05-11T12:14:32.562048+0000', 'last_peered': '2024-05-11T12:14:32.562048+0000', 'last_clean': '2024-05-11T12:14:32.562048+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:14:32.598150+0000', 'last_undegraded': '2024-05-11T12:14:32.598150+0000', 'last_fullsized': '2024-05-11T12:14:32.598150+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_clean_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-11T12:14:39.042 INFO:tasks.ceph.ceph_manager.ceph:PG 16.39 is not active+clean 2024-05-11T12:14:39.042 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:14:32.602749+0000', 'last_change': '2024-05-11T12:14:32.602749+0000', 'last_active': '2024-05-11T12:14:32.562048+0000', 'last_peered': '2024-05-11T12:14:32.562048+0000', 'last_clean': '2024-05-11T12:14:32.562048+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:14:32.602749+0000', 'last_undegraded': '2024-05-11T12:14:32.602749+0000', 'last_fullsized': '2024-05-11T12:14:32.602749+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_clean_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-11T12:14:39.042 INFO:tasks.ceph.ceph_manager.ceph:PG 16.37 is not active+clean 2024-05-11T12:14:39.042 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:14:32.595249+0000', 'last_change': '2024-05-11T12:14:32.595249+0000', 'last_active': '2024-05-11T12:14:32.562048+0000', 'last_peered': '2024-05-11T12:14:32.562048+0000', 'last_clean': '2024-05-11T12:14:32.562048+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:14:32.595249+0000', 'last_undegraded': '2024-05-11T12:14:32.595249+0000', 'last_fullsized': '2024-05-11T12:14:32.595249+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_clean_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-11T12:14:39.042 INFO:tasks.ceph.ceph_manager.ceph:PG 16.36 is not active+clean 2024-05-11T12:14:39.042 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T12:14:32.562048+0000', 'last_change': '2024-05-11T12:14:32.562048+0000', 'last_active': '2024-05-11T12:14:32.562048+0000', 'last_peered': '2024-05-11T12:14:32.562048+0000', 'last_clean': '2024-05-11T12:14:32.562048+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:14:32.562048+0000', 'last_undegraded': '2024-05-11T12:14:32.562048+0000', 'last_fullsized': '2024-05-11T12:14:32.562048+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_clean_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T12:14:39.042 INFO:tasks.ceph.ceph_manager.ceph:PG 16.35 is not active+clean 2024-05-11T12:14:39.043 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:14:32.592511+0000', 'last_change': '2024-05-11T12:14:32.592511+0000', 'last_active': '2024-05-11T12:14:32.562048+0000', 'last_peered': '2024-05-11T12:14:32.562048+0000', 'last_clean': '2024-05-11T12:14:32.562048+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:14:32.592511+0000', 'last_undegraded': '2024-05-11T12:14:32.592511+0000', 'last_fullsized': '2024-05-11T12:14:32.592511+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_clean_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-11T12:14:39.043 INFO:tasks.ceph.ceph_manager.ceph:PG 16.34 is not active+clean 2024-05-11T12:14:39.043 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:14:32.606336+0000', 'last_change': '2024-05-11T12:14:32.606336+0000', 'last_active': '2024-05-11T12:14:32.562048+0000', 'last_peered': '2024-05-11T12:14:32.562048+0000', 'last_clean': '2024-05-11T12:14:32.562048+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:14:32.606336+0000', 'last_undegraded': '2024-05-11T12:14:32.606336+0000', 'last_fullsized': '2024-05-11T12:14:32.606336+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_clean_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-11T12:14:39.043 INFO:tasks.ceph.ceph_manager.ceph:PG 16.32 is not active+clean 2024-05-11T12:14:39.043 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:14:32.600693+0000', 'last_change': '2024-05-11T12:14:32.600693+0000', 'last_active': '2024-05-11T12:14:32.562048+0000', 'last_peered': '2024-05-11T12:14:32.562048+0000', 'last_clean': '2024-05-11T12:14:32.562048+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:14:32.600693+0000', 'last_undegraded': '2024-05-11T12:14:32.600693+0000', 'last_fullsized': '2024-05-11T12:14:32.600693+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_clean_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-11T12:14:39.043 INFO:tasks.ceph.ceph_manager.ceph:PG 16.31 is not active+clean 2024-05-11T12:14:39.043 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:14:32.599082+0000', 'last_change': '2024-05-11T12:14:32.599082+0000', 'last_active': '2024-05-11T12:14:32.562048+0000', 'last_peered': '2024-05-11T12:14:32.562048+0000', 'last_clean': '2024-05-11T12:14:32.562048+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:14:32.599082+0000', 'last_undegraded': '2024-05-11T12:14:32.599082+0000', 'last_fullsized': '2024-05-11T12:14:32.599082+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_clean_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-11T12:14:39.043 INFO:tasks.ceph.ceph_manager.ceph:PG 16.30 is not active+clean 2024-05-11T12:14:39.043 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:14:32.603185+0000', 'last_change': '2024-05-11T12:14:32.603185+0000', 'last_active': '2024-05-11T12:14:32.562048+0000', 'last_peered': '2024-05-11T12:14:32.562048+0000', 'last_clean': '2024-05-11T12:14:32.562048+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:14:32.603185+0000', 'last_undegraded': '2024-05-11T12:14:32.603185+0000', 'last_fullsized': '2024-05-11T12:14:32.603185+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_clean_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-11T12:14:39.043 INFO:tasks.ceph.ceph_manager.ceph:PG 16.2e is not active+clean 2024-05-11T12:14:39.043 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:14:32.600768+0000', 'last_change': '2024-05-11T12:14:32.600768+0000', 'last_active': '2024-05-11T12:14:32.562048+0000', 'last_peered': '2024-05-11T12:14:32.562048+0000', 'last_clean': '2024-05-11T12:14:32.562048+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:14:32.600768+0000', 'last_undegraded': '2024-05-11T12:14:32.600768+0000', 'last_fullsized': '2024-05-11T12:14:32.600768+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_clean_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-11T12:14:39.043 INFO:tasks.ceph.ceph_manager.ceph:PG 16.2c is not active+clean 2024-05-11T12:14:39.043 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:14:32.605840+0000', 'last_change': '2024-05-11T12:14:32.605840+0000', 'last_active': '2024-05-11T12:14:32.562048+0000', 'last_peered': '2024-05-11T12:14:32.562048+0000', 'last_clean': '2024-05-11T12:14:32.562048+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:14:32.605840+0000', 'last_undegraded': '2024-05-11T12:14:32.605840+0000', 'last_fullsized': '2024-05-11T12:14:32.605840+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_clean_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-11T12:14:39.044 INFO:tasks.ceph.ceph_manager.ceph:PG 16.2a is not active+clean 2024-05-11T12:14:39.044 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T12:14:32.562048+0000', 'last_change': '2024-05-11T12:14:32.562048+0000', 'last_active': '2024-05-11T12:14:32.562048+0000', 'last_peered': '2024-05-11T12:14:32.562048+0000', 'last_clean': '2024-05-11T12:14:32.562048+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:14:32.562048+0000', 'last_undegraded': '2024-05-11T12:14:32.562048+0000', 'last_fullsized': '2024-05-11T12:14:32.562048+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_clean_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T12:14:39.044 INFO:tasks.ceph.ceph_manager.ceph:PG 16.28 is not active+clean 2024-05-11T12:14:39.044 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T12:14:32.562048+0000', 'last_change': '2024-05-11T12:14:32.562048+0000', 'last_active': '2024-05-11T12:14:32.562048+0000', 'last_peered': '2024-05-11T12:14:32.562048+0000', 'last_clean': '2024-05-11T12:14:32.562048+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:14:32.562048+0000', 'last_undegraded': '2024-05-11T12:14:32.562048+0000', 'last_fullsized': '2024-05-11T12:14:32.562048+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_clean_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T12:14:39.044 INFO:tasks.ceph.ceph_manager.ceph:PG 16.27 is not active+clean 2024-05-11T12:14:39.044 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:14:32.607235+0000', 'last_change': '2024-05-11T12:14:32.607235+0000', 'last_active': '2024-05-11T12:14:32.562048+0000', 'last_peered': '2024-05-11T12:14:32.562048+0000', 'last_clean': '2024-05-11T12:14:32.562048+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:14:32.607235+0000', 'last_undegraded': '2024-05-11T12:14:32.607235+0000', 'last_fullsized': '2024-05-11T12:14:32.607235+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_clean_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-11T12:14:39.044 INFO:tasks.ceph.ceph_manager.ceph:PG 16.26 is not active+clean 2024-05-11T12:14:39.044 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:14:32.601792+0000', 'last_change': '2024-05-11T12:14:32.601792+0000', 'last_active': '2024-05-11T12:14:32.562048+0000', 'last_peered': '2024-05-11T12:14:32.562048+0000', 'last_clean': '2024-05-11T12:14:32.562048+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:14:32.601792+0000', 'last_undegraded': '2024-05-11T12:14:32.601792+0000', 'last_fullsized': '2024-05-11T12:14:32.601792+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_clean_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-11T12:14:39.044 INFO:tasks.ceph.ceph_manager.ceph:PG 16.25 is not active+clean 2024-05-11T12:14:39.044 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:14:32.591419+0000', 'last_change': '2024-05-11T12:14:32.591419+0000', 'last_active': '2024-05-11T12:14:32.562048+0000', 'last_peered': '2024-05-11T12:14:32.562048+0000', 'last_clean': '2024-05-11T12:14:32.562048+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:14:32.591419+0000', 'last_undegraded': '2024-05-11T12:14:32.591419+0000', 'last_fullsized': '2024-05-11T12:14:32.591419+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_clean_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-11T12:14:39.044 INFO:tasks.ceph.ceph_manager.ceph:PG 16.24 is not active+clean 2024-05-11T12:14:39.044 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:14:32.595401+0000', 'last_change': '2024-05-11T12:14:32.595401+0000', 'last_active': '2024-05-11T12:14:32.562048+0000', 'last_peered': '2024-05-11T12:14:32.562048+0000', 'last_clean': '2024-05-11T12:14:32.562048+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:14:32.595401+0000', 'last_undegraded': '2024-05-11T12:14:32.595401+0000', 'last_fullsized': '2024-05-11T12:14:32.595401+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_clean_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-11T12:14:39.044 INFO:tasks.ceph.ceph_manager.ceph:PG 16.22 is not active+clean 2024-05-11T12:14:39.045 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:14:32.603561+0000', 'last_change': '2024-05-11T12:14:32.603561+0000', 'last_active': '2024-05-11T12:14:32.562048+0000', 'last_peered': '2024-05-11T12:14:32.562048+0000', 'last_clean': '2024-05-11T12:14:32.562048+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:14:32.603561+0000', 'last_undegraded': '2024-05-11T12:14:32.603561+0000', 'last_fullsized': '2024-05-11T12:14:32.603561+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_clean_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-11T12:14:39.045 INFO:tasks.ceph.ceph_manager.ceph:PG 16.21 is not active+clean 2024-05-11T12:14:39.045 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:14:32.596127+0000', 'last_change': '2024-05-11T12:14:32.596127+0000', 'last_active': '2024-05-11T12:14:32.562048+0000', 'last_peered': '2024-05-11T12:14:32.562048+0000', 'last_clean': '2024-05-11T12:14:32.562048+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:14:32.596127+0000', 'last_undegraded': '2024-05-11T12:14:32.596127+0000', 'last_fullsized': '2024-05-11T12:14:32.596127+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_clean_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-11T12:14:39.045 INFO:tasks.ceph.ceph_manager.ceph:PG 16.2 is not active+clean 2024-05-11T12:14:39.045 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:14:32.603030+0000', 'last_change': '2024-05-11T12:14:32.603030+0000', 'last_active': '2024-05-11T12:14:32.562048+0000', 'last_peered': '2024-05-11T12:14:32.562048+0000', 'last_clean': '2024-05-11T12:14:32.562048+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:14:32.603030+0000', 'last_undegraded': '2024-05-11T12:14:32.603030+0000', 'last_fullsized': '2024-05-11T12:14:32.603030+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_clean_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-11T12:14:39.045 INFO:tasks.ceph.ceph_manager.ceph:PG 16.3 is not active+clean 2024-05-11T12:14:39.045 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:14:32.591054+0000', 'last_change': '2024-05-11T12:14:32.591054+0000', 'last_active': '2024-05-11T12:14:32.562048+0000', 'last_peered': '2024-05-11T12:14:32.562048+0000', 'last_clean': '2024-05-11T12:14:32.562048+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:14:32.591054+0000', 'last_undegraded': '2024-05-11T12:14:32.591054+0000', 'last_fullsized': '2024-05-11T12:14:32.591054+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_clean_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-11T12:14:39.045 INFO:tasks.ceph.ceph_manager.ceph:PG 15.1c is not active+clean 2024-05-11T12:14:39.045 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 94, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:14:31.585654+0000', 'last_change': '2024-05-11T12:14:31.585654+0000', 'last_active': '2024-05-11T12:14:31.545700+0000', 'last_peered': '2024-05-11T12:14:31.545700+0000', 'last_clean': '2024-05-11T12:14:31.545700+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:14:31.585654+0000', 'last_undegraded': '2024-05-11T12:14:31.585654+0000', 'last_fullsized': '2024-05-11T12:14:31.585654+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:14:31.545700+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:14:31.545700+0000', 'last_clean_scrub_stamp': '2024-05-11T12:14:31.545700+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-11T12:14:39.045 INFO:tasks.ceph.ceph_manager.ceph:PG 16.6 is not active+clean 2024-05-11T12:14:39.045 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:14:32.599834+0000', 'last_change': '2024-05-11T12:14:32.599834+0000', 'last_active': '2024-05-11T12:14:32.562048+0000', 'last_peered': '2024-05-11T12:14:32.562048+0000', 'last_clean': '2024-05-11T12:14:32.562048+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:14:32.599834+0000', 'last_undegraded': '2024-05-11T12:14:32.599834+0000', 'last_fullsized': '2024-05-11T12:14:32.599834+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_clean_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-11T12:14:39.045 INFO:tasks.ceph.ceph_manager.ceph:PG 16.9 is not active+clean 2024-05-11T12:14:39.046 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T12:14:32.562048+0000', 'last_change': '2024-05-11T12:14:32.562048+0000', 'last_active': '2024-05-11T12:14:32.562048+0000', 'last_peered': '2024-05-11T12:14:32.562048+0000', 'last_clean': '2024-05-11T12:14:32.562048+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:14:32.562048+0000', 'last_undegraded': '2024-05-11T12:14:32.562048+0000', 'last_fullsized': '2024-05-11T12:14:32.562048+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_clean_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T12:14:39.046 INFO:tasks.ceph.ceph_manager.ceph:PG 16.b is not active+clean 2024-05-11T12:14:39.046 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T12:14:32.562048+0000', 'last_change': '2024-05-11T12:14:32.562048+0000', 'last_active': '2024-05-11T12:14:32.562048+0000', 'last_peered': '2024-05-11T12:14:32.562048+0000', 'last_clean': '2024-05-11T12:14:32.562048+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:14:32.562048+0000', 'last_undegraded': '2024-05-11T12:14:32.562048+0000', 'last_fullsized': '2024-05-11T12:14:32.562048+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_clean_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T12:14:39.046 INFO:tasks.ceph.ceph_manager.ceph:PG 16.c is not active+clean 2024-05-11T12:14:39.046 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:14:32.591204+0000', 'last_change': '2024-05-11T12:14:32.591204+0000', 'last_active': '2024-05-11T12:14:32.562048+0000', 'last_peered': '2024-05-11T12:14:32.562048+0000', 'last_clean': '2024-05-11T12:14:32.562048+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:14:32.591204+0000', 'last_undegraded': '2024-05-11T12:14:32.591204+0000', 'last_fullsized': '2024-05-11T12:14:32.591204+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_clean_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-11T12:14:39.046 INFO:tasks.ceph.ceph_manager.ceph:PG 15.13 is not active+clean 2024-05-11T12:14:39.046 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 94, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:14:31.584097+0000', 'last_change': '2024-05-11T12:14:31.584097+0000', 'last_active': '2024-05-11T12:14:31.545700+0000', 'last_peered': '2024-05-11T12:14:31.545700+0000', 'last_clean': '2024-05-11T12:14:31.545700+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:14:31.584097+0000', 'last_undegraded': '2024-05-11T12:14:31.584097+0000', 'last_fullsized': '2024-05-11T12:14:31.584097+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:14:31.545700+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:14:31.545700+0000', 'last_clean_scrub_stamp': '2024-05-11T12:14:31.545700+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-11T12:14:39.046 INFO:tasks.ceph.ceph_manager.ceph:PG 16.d is not active+clean 2024-05-11T12:14:39.046 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:14:32.594883+0000', 'last_change': '2024-05-11T12:14:32.594883+0000', 'last_active': '2024-05-11T12:14:32.562048+0000', 'last_peered': '2024-05-11T12:14:32.562048+0000', 'last_clean': '2024-05-11T12:14:32.562048+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:14:32.594883+0000', 'last_undegraded': '2024-05-11T12:14:32.594883+0000', 'last_fullsized': '2024-05-11T12:14:32.594883+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_clean_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-11T12:14:39.046 INFO:tasks.ceph.ceph_manager.ceph:PG 16.12 is not active+clean 2024-05-11T12:14:39.046 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:14:32.597272+0000', 'last_change': '2024-05-11T12:14:32.597272+0000', 'last_active': '2024-05-11T12:14:32.562048+0000', 'last_peered': '2024-05-11T12:14:32.562048+0000', 'last_clean': '2024-05-11T12:14:32.562048+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:14:32.597272+0000', 'last_undegraded': '2024-05-11T12:14:32.597272+0000', 'last_fullsized': '2024-05-11T12:14:32.597272+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_clean_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-11T12:14:39.047 INFO:tasks.ceph.ceph_manager.ceph:PG 16.13 is not active+clean 2024-05-11T12:14:39.047 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T12:14:32.562048+0000', 'last_change': '2024-05-11T12:14:32.562048+0000', 'last_active': '2024-05-11T12:14:32.562048+0000', 'last_peered': '2024-05-11T12:14:32.562048+0000', 'last_clean': '2024-05-11T12:14:32.562048+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:14:32.562048+0000', 'last_undegraded': '2024-05-11T12:14:32.562048+0000', 'last_fullsized': '2024-05-11T12:14:32.562048+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_clean_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T12:14:39.047 INFO:tasks.ceph.ceph_manager.ceph:PG 15.7 is not active+clean 2024-05-11T12:14:39.047 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 94, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:14:31.584956+0000', 'last_change': '2024-05-11T12:14:31.584956+0000', 'last_active': '2024-05-11T12:14:31.545700+0000', 'last_peered': '2024-05-11T12:14:31.545700+0000', 'last_clean': '2024-05-11T12:14:31.545700+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:14:31.584956+0000', 'last_undegraded': '2024-05-11T12:14:31.584956+0000', 'last_fullsized': '2024-05-11T12:14:31.584956+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:14:31.545700+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:14:31.545700+0000', 'last_clean_scrub_stamp': '2024-05-11T12:14:31.545700+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-11T12:14:39.047 INFO:tasks.ceph.ceph_manager.ceph:PG 16.19 is not active+clean 2024-05-11T12:14:39.047 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:14:32.602318+0000', 'last_change': '2024-05-11T12:14:32.602318+0000', 'last_active': '2024-05-11T12:14:32.562048+0000', 'last_peered': '2024-05-11T12:14:32.562048+0000', 'last_clean': '2024-05-11T12:14:32.562048+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:14:32.602318+0000', 'last_undegraded': '2024-05-11T12:14:32.602318+0000', 'last_fullsized': '2024-05-11T12:14:32.602318+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_clean_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-11T12:14:39.047 INFO:tasks.ceph.ceph_manager.ceph:PG 16.1b is not active+clean 2024-05-11T12:14:39.047 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:14:32.592181+0000', 'last_change': '2024-05-11T12:14:32.592181+0000', 'last_active': '2024-05-11T12:14:32.562048+0000', 'last_peered': '2024-05-11T12:14:32.562048+0000', 'last_clean': '2024-05-11T12:14:32.562048+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:14:32.592181+0000', 'last_undegraded': '2024-05-11T12:14:32.592181+0000', 'last_fullsized': '2024-05-11T12:14:32.592181+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_clean_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-11T12:14:39.047 INFO:tasks.ceph.ceph_manager.ceph:PG 15.4 is not active+clean 2024-05-11T12:14:39.047 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 94, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:14:31.584482+0000', 'last_change': '2024-05-11T12:14:31.584482+0000', 'last_active': '2024-05-11T12:14:31.545700+0000', 'last_peered': '2024-05-11T12:14:31.545700+0000', 'last_clean': '2024-05-11T12:14:31.545700+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:14:31.584482+0000', 'last_undegraded': '2024-05-11T12:14:31.584482+0000', 'last_fullsized': '2024-05-11T12:14:31.584482+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:14:31.545700+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:14:31.545700+0000', 'last_clean_scrub_stamp': '2024-05-11T12:14:31.545700+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-11T12:14:39.047 INFO:tasks.ceph.ceph_manager.ceph:PG 16.1e is not active+clean 2024-05-11T12:14:39.047 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:14:32.601116+0000', 'last_change': '2024-05-11T12:14:32.601116+0000', 'last_active': '2024-05-11T12:14:32.562048+0000', 'last_peered': '2024-05-11T12:14:32.562048+0000', 'last_clean': '2024-05-11T12:14:32.562048+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:14:32.601116+0000', 'last_undegraded': '2024-05-11T12:14:32.601116+0000', 'last_fullsized': '2024-05-11T12:14:32.601116+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_clean_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-11T12:14:39.048 INFO:tasks.ceph.ceph_manager.ceph:PG 16.1f is not active+clean 2024-05-11T12:14:39.048 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:14:32.593567+0000', 'last_change': '2024-05-11T12:14:32.593567+0000', 'last_active': '2024-05-11T12:14:32.562048+0000', 'last_peered': '2024-05-11T12:14:32.562048+0000', 'last_clean': '2024-05-11T12:14:32.562048+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:14:32.593567+0000', 'last_undegraded': '2024-05-11T12:14:32.593567+0000', 'last_fullsized': '2024-05-11T12:14:32.593567+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_clean_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-11T12:14:39.048 INFO:tasks.ceph.ceph_manager.ceph:PG 16.11 is not active+clean 2024-05-11T12:14:39.048 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-05-11T12:14:32.589733+0000', 'last_change': '2024-05-11T12:14:32.589733+0000', 'last_active': '2024-05-11T12:14:32.562048+0000', 'last_peered': '2024-05-11T12:14:32.562048+0000', 'last_clean': '2024-05-11T12:14:32.562048+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T12:14:32.589733+0000', 'last_undegraded': '2024-05-11T12:14:32.589733+0000', 'last_fullsized': '2024-05-11T12:14:32.589733+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'last_clean_scrub_stamp': '2024-05-11T12:14:32.562048+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-11T12:14:39.048 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-11T12:14:39.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.279+0000 7f7113200640 1 Processor -- start 2024-05-11T12:14:39.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.280+0000 7f7113200640 1 -- start start 2024-05-11T12:14:39.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.281+0000 7f7113200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7114156120 0x7f7114162bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:39.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.281+0000 7f7113200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f711406c660 0x7f7114163110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:39.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.281+0000 7f7113200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7114156500 con 0x7f711415f620 2024-05-11T12:14:39.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.281+0000 7f7113200640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f711406d700 con 0x7f711406c660 2024-05-11T12:14:39.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.281+0000 7f7113200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f71140d3b80 con 0x7f7114156120 2024-05-11T12:14:39.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.281+0000 7f7111400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f711406c660 0x7f7114163110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:39.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.281+0000 7f7111400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f711406c660 0x7f7114163110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:47580/0 (socket says 172.21.15.167:47580) 2024-05-11T12:14:39.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.281+0000 7f7111400640 1 -- 172.21.15.167:0/2381714581 learned_addr learned my addr 172.21.15.167:0/2381714581 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:39.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.282+0000 7f7111e00640 1 --2- 172.21.15.167:0/2381714581 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7114156120 0x7f7114162bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:39.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.282+0000 7f7111400640 1 -- 172.21.15.167:0/2381714581 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7114156120 msgr2=0x7f7114162bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:39.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.282+0000 7f7111400640 1 --2- 172.21.15.167:0/2381714581 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7114156120 0x7f7114162bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:39.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.282+0000 7f7111400640 1 -- 172.21.15.167:0/2381714581 >> v1:172.21.15.204:6789/0 conn(0x7f711415f620 legacy=0x7f711415fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:39.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.282+0000 7f7111400640 1 -- 172.21.15.167:0/2381714581 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f711406caf0 con 0x7f711406c660 2024-05-11T12:14:39.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.282+0000 7f7111e00640 1 --2- 172.21.15.167:0/2381714581 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7114156120 0x7f7114162bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:39.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.283+0000 7f7111400640 1 --2- 172.21.15.167:0/2381714581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f711406c660 0x7f7114163110 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f71141566d0 tx=0x7f70fc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=673dee9088fc996a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:39.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.284+0000 7f7110a00640 1 -- 172.21.15.167:0/2381714581 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc036070 con 0x7f711406c660 2024-05-11T12:14:39.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.284+0000 7f7110a00640 1 -- 172.21.15.167:0/2381714581 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70fc031600 con 0x7f711406c660 2024-05-11T12:14:39.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.284+0000 7f7110a00640 1 -- 172.21.15.167:0/2381714581 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc0319d0 con 0x7f711406c660 2024-05-11T12:14:39.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.284+0000 7f7113200640 1 -- 172.21.15.167:0/2381714581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f711406c660 msgr2=0x7f7114163110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:39.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.284+0000 7f7113200640 1 --2- 172.21.15.167:0/2381714581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f711406c660 0x7f7114163110 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f71141566d0 tx=0x7f70fc02a660 comp rx=0 tx=0).stop 2024-05-11T12:14:39.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.284+0000 7f7113200640 1 -- 172.21.15.167:0/2381714581 shutdown_connections 2024-05-11T12:14:39.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.284+0000 7f7113200640 1 --2- 172.21.15.167:0/2381714581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f711406c660 0x7f7114163110 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:39.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.284+0000 7f7113200640 1 --2- 172.21.15.167:0/2381714581 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7114156120 0x7f7114162bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:39.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.284+0000 7f7113200640 1 -- 172.21.15.167:0/2381714581 >> 172.21.15.167:0/2381714581 conn(0x7f71140b6af0 msgr2=0x7f71140b6f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:39.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.285+0000 7f7113200640 1 -- 172.21.15.167:0/2381714581 shutdown_connections 2024-05-11T12:14:39.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.285+0000 7f7113200640 1 -- 172.21.15.167:0/2381714581 wait complete. 2024-05-11T12:14:39.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.286+0000 7f7113200640 1 Processor -- start 2024-05-11T12:14:39.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.286+0000 7f7113200640 1 -- start start 2024-05-11T12:14:39.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.287+0000 7f7113200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7114156120 0x7f71140cb110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:39.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.287+0000 7f7113200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f711415f620 0x7f7114151690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:39.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.287+0000 7f7113200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f711415e690 0x7f711415eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:39.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.287+0000 7f7113200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f711406d390 con 0x7f711415e690 2024-05-11T12:14:39.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.287+0000 7f7113200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f711406d0b0 con 0x7f711415f620 2024-05-11T12:14:39.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.287+0000 7f7113200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f711406db90 con 0x7f7114156120 2024-05-11T12:14:39.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.288+0000 7f7112800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f711415e690 0x7f711415eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:39.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.288+0000 7f7112800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f711415e690 0x7f711415eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:47582/0 (socket says 172.21.15.167:47582) 2024-05-11T12:14:39.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.288+0000 7f7112800640 1 -- 172.21.15.167:0/2284656980 learned_addr learned my addr 172.21.15.167:0/2284656980 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:39.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.288+0000 7f7111e00640 1 --2- 172.21.15.167:0/2284656980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7114156120 0x7f71140cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:39.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.288+0000 7f7111400640 1 --2- 172.21.15.167:0/2284656980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f711415f620 0x7f7114151690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:39.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.288+0000 7f7112800640 1 -- 172.21.15.167:0/2284656980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7114156120 msgr2=0x7f71140cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:39.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.288+0000 7f7112800640 1 --2- 172.21.15.167:0/2284656980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7114156120 0x7f71140cb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:39.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.288+0000 7f7112800640 1 -- 172.21.15.167:0/2284656980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f711415f620 msgr2=0x7f7114151690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:39.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.288+0000 7f7112800640 1 --2- 172.21.15.167:0/2284656980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f711415f620 0x7f7114151690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:39.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.288+0000 7f7112800640 1 -- 172.21.15.167:0/2284656980 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7114151bd0 con 0x7f711415e690 2024-05-11T12:14:39.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.288+0000 7f7111e00640 1 --2- 172.21.15.167:0/2284656980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7114156120 0x7f71140cb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:39.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.288+0000 7f7111400640 1 --2- 172.21.15.167:0/2284656980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f711415f620 0x7f7114151690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:39.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.288+0000 7f7112800640 1 --2- 172.21.15.167:0/2284656980 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f711415e690 0x7f711415eb20 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f710c00c410 tx=0x7f710c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:39.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.289+0000 7f710a000640 1 -- 172.21.15.167:0/2284656980 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710c011790 con 0x7f711415e690 2024-05-11T12:14:39.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.289+0000 7f710a000640 1 -- 172.21.15.167:0/2284656980 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f710c017070 con 0x7f711415e690 2024-05-11T12:14:39.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.289+0000 7f710a000640 1 -- 172.21.15.167:0/2284656980 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710c011b70 con 0x7f711415e690 2024-05-11T12:14:39.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.289+0000 7f7113200640 1 -- 172.21.15.167:0/2284656980 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71141f5080 con 0x7f711415e690 2024-05-11T12:14:39.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.289+0000 7f7113200640 1 -- 172.21.15.167:0/2284656980 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71141f5640 con 0x7f711415e690 2024-05-11T12:14:39.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.290+0000 7f7113200640 1 -- 172.21.15.167:0/2284656980 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7114069bc0 con 0x7f711415e690 2024-05-11T12:14:39.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.291+0000 7f710a000640 1 -- 172.21.15.167:0/2284656980 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f710c011d10 con 0x7f711415e690 2024-05-11T12:14:39.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.293+0000 7f710a000640 1 --2- 172.21.15.167:0/2284656980 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f70f0076300 0x7f70f00787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:39.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.293+0000 7f710a000640 1 -- 172.21.15.167:0/2284656980 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6101+0+0 (secure 0 0 0) 0x7f710c0973b0 con 0x7f711415e690 2024-05-11T12:14:39.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.293+0000 7f7111e00640 1 --2- 172.21.15.167:0/2284656980 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f70f0076300 0x7f70f00787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:39.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.294+0000 7f7111e00640 1 --2- 172.21.15.167:0/2284656980 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f70f0076300 0x7f70f00787c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7104006fd0 tx=0x7f7104008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:39.296 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.297+0000 7f710a000640 1 -- 172.21.15.167:0/2284656980 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f710c021020 con 0x7f711415e690 2024-05-11T12:14:39.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.586+0000 7f7113200640 1 -- 172.21.15.167:0/2284656980 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f71141604d0 con 0x7f711415e690 2024-05-11T12:14:39.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.587+0000 7f710a000640 1 -- 172.21.15.167:0/2284656980 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1592 (secure 0 0 0) 0x7f7114151bd0 con 0x7f711415e690 2024-05-11T12:14:39.586 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:39.587 INFO:teuthology.orchestra.run.smithi167.stdout:{"fsid":"5f32d344-7289-4c27-881b-1533419ab678","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":259,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":97,"num_osds":8,"num_up_osds":8,"osd_up_since":1715429432,"num_in_osds":8,"osd_in_since":1715429423,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":58},{"state_name":"creating+peering","count":32},{"state_name":"unknown","count":7}],"num_pgs":97,"num_pools":3,"num_objects":7,"data_bytes":590950,"bytes_used":266063872,"bytes_avail":772828049408,"bytes_total":773094113280,"unknown_pgs_ratio":0.072164945304393768,"inactive_pgs_ratio":0.32989689707756042,"write_bytes_sec":178,"read_op_per_sec":0,"write_op_per_sec":1},"fsmap":{"epoch":62,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":6639}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-05-11T12:13:23.284443+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-11T12:14:39.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.589+0000 7f7113200640 1 -- 172.21.15.167:0/2284656980 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f70f0076300 msgr2=0x7f70f00787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:39.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.589+0000 7f7113200640 1 --2- 172.21.15.167:0/2284656980 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f70f0076300 0x7f70f00787c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7104006fd0 tx=0x7f7104008040 comp rx=0 tx=0).stop 2024-05-11T12:14:39.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.590+0000 7f7113200640 1 -- 172.21.15.167:0/2284656980 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f711415e690 msgr2=0x7f711415eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:39.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.590+0000 7f7113200640 1 --2- 172.21.15.167:0/2284656980 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f711415e690 0x7f711415eb20 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f710c00c410 tx=0x7f710c00c8d0 comp rx=0 tx=0).stop 2024-05-11T12:14:39.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.590+0000 7f7113200640 1 -- 172.21.15.167:0/2284656980 shutdown_connections 2024-05-11T12:14:39.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.590+0000 7f7113200640 1 --2- 172.21.15.167:0/2284656980 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f711415e690 0x7f711415eb20 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:39.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.590+0000 7f7113200640 1 --2- 172.21.15.167:0/2284656980 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f70f0076300 0x7f70f00787c0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:39.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.590+0000 7f7113200640 1 --2- 172.21.15.167:0/2284656980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f711415f620 0x7f7114151690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:39.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.590+0000 7f7113200640 1 --2- 172.21.15.167:0/2284656980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7114156120 0x7f71140cb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:39.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.590+0000 7f7113200640 1 -- 172.21.15.167:0/2284656980 >> 172.21.15.167:0/2284656980 conn(0x7f71140b6af0 msgr2=0x7f7114161920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:39.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.590+0000 7f7113200640 1 -- 172.21.15.167:0/2284656980 shutdown_connections 2024-05-11T12:14:39.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.590+0000 7f7113200640 1 -- 172.21.15.167:0/2284656980 wait complete. 2024-05-11T12:14:39.608 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-05-11T12:14:39.608 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T12:14:39.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.798+0000 7fa886c00640 1 Processor -- start 2024-05-11T12:14:39.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.798+0000 7fa886c00640 1 -- start start 2024-05-11T12:14:39.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.800+0000 7fa886c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa8800cf970 0x7fa8801601f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:39.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.800+0000 7fa886c00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa8800d04a0 0x7fa8800b55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:39.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.800+0000 7fa886c00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8800b4560 con 0x7fa8800afcd0 2024-05-11T12:14:39.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.800+0000 7fa886c00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fa8800b5130 con 0x7fa8800d04a0 2024-05-11T12:14:39.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.800+0000 7fa886c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fa8800b4800 con 0x7fa8800cf970 2024-05-11T12:14:39.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.800+0000 7fa886200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa8800d04a0 0x7fa8800b55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:39.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.800+0000 7fa885800640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fa8800afcd0 0x7fa8800cf260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:44010/0 (socket says 172.21.15.167:44010) 2024-05-11T12:14:39.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.800+0000 7fa885800640 1 -- 172.21.15.167:0/1080677533 learned_addr learned my addr 172.21.15.167:0/1080677533 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:14:39.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.801+0000 7fa884e00640 1 --2- 172.21.15.167:0/1080677533 >> v2:172.21.15.204:3300/0 conn(0x7fa8800cf970 0x7fa8801601f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:39.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.801+0000 7fa886200640 1 -- 172.21.15.167:0/1080677533 >> v2:172.21.15.204:3300/0 conn(0x7fa8800cf970 msgr2=0x7fa8801601f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:39.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.801+0000 7fa886200640 1 --2- 172.21.15.167:0/1080677533 >> v2:172.21.15.204:3300/0 conn(0x7fa8800cf970 0x7fa8801601f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:39.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.801+0000 7fa87fe00640 1 -- 172.21.15.167:0/1080677533 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 196250913 0 0) 0x7fa8800b4560 con 0x7fa8800afcd0 2024-05-11T12:14:39.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.801+0000 7fa886200640 1 -- 172.21.15.167:0/1080677533 >> v1:172.21.15.167:6789/0 conn(0x7fa8800afcd0 legacy=0x7fa8800cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:39.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.801+0000 7fa886200640 1 -- 172.21.15.167:0/1080677533 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8800b4ae0 con 0x7fa8800d04a0 2024-05-11T12:14:39.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.801+0000 7fa884e00640 1 --2- 172.21.15.167:0/1080677533 >> v2:172.21.15.204:3300/0 conn(0x7fa8800cf970 0x7fa8801601f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:39.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.802+0000 7fa886200640 1 --2- 172.21.15.167:0/1080677533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8800d04a0 0x7fa8800b55c0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fa8800d0260 tx=0x7fa87802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5c02aeb7f8645e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:39.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.802+0000 7fa87fe00640 1 -- 172.21.15.167:0/1080677533 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa878007740 con 0x7fa8800d04a0 2024-05-11T12:14:39.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.802+0000 7fa87fe00640 1 -- 172.21.15.167:0/1080677533 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8780078e0 con 0x7fa8800d04a0 2024-05-11T12:14:39.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.802+0000 7fa87fe00640 1 -- 172.21.15.167:0/1080677533 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa878007cb0 con 0x7fa8800d04a0 2024-05-11T12:14:39.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.803+0000 7fa886c00640 1 -- 172.21.15.167:0/1080677533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8800d04a0 msgr2=0x7fa8800b55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:39.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.803+0000 7fa886c00640 1 --2- 172.21.15.167:0/1080677533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8800d04a0 0x7fa8800b55c0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fa8800d0260 tx=0x7fa87802a680 comp rx=0 tx=0).stop 2024-05-11T12:14:39.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.803+0000 7fa886c00640 1 -- 172.21.15.167:0/1080677533 shutdown_connections 2024-05-11T12:14:39.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.803+0000 7fa886c00640 1 --2- 172.21.15.167:0/1080677533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8800d04a0 0x7fa8800b55c0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:39.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.803+0000 7fa886c00640 1 --2- 172.21.15.167:0/1080677533 >> v2:172.21.15.204:3300/0 conn(0x7fa8800cf970 0x7fa8801601f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:39.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.803+0000 7fa886c00640 1 -- 172.21.15.167:0/1080677533 >> 172.21.15.167:0/1080677533 conn(0x7fa8800b3460 msgr2=0x7fa8800d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:39.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.803+0000 7fa886c00640 1 -- 172.21.15.167:0/1080677533 shutdown_connections 2024-05-11T12:14:39.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.803+0000 7fa886c00640 1 -- 172.21.15.167:0/1080677533 wait complete. 2024-05-11T12:14:39.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.805+0000 7fa886c00640 1 Processor -- start 2024-05-11T12:14:39.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.805+0000 7fa886c00640 1 -- start start 2024-05-11T12:14:39.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.806+0000 7fa886c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8800afcd0 0x7fa88006f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:39.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.806+0000 7fa886c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8800cf970 0x7fa880154aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:39.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.806+0000 7fa886c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8800d04a0 0x7fa8801f8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:39.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.806+0000 7fa886c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa8800cff40 con 0x7fa8800afcd0 2024-05-11T12:14:39.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.806+0000 7fa886c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa8800b00b0 con 0x7fa8800cf970 2024-05-11T12:14:39.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.806+0000 7fa886c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa8800d00c0 con 0x7fa8800d04a0 2024-05-11T12:14:39.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.806+0000 7fa885800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8800afcd0 0x7fa88006f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:39.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.806+0000 7fa885800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8800afcd0 0x7fa88006f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:47594/0 (socket says 172.21.15.167:47594) 2024-05-11T12:14:39.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.806+0000 7fa885800640 1 -- 172.21.15.167:0/380789275 learned_addr learned my addr 172.21.15.167:0/380789275 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:39.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.807+0000 7fa885800640 1 -- 172.21.15.167:0/380789275 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8800d04a0 msgr2=0x7fa8801f8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:39.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.807+0000 7fa886200640 1 --2- 172.21.15.167:0/380789275 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8800d04a0 0x7fa8801f8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:39.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.807+0000 7fa884e00640 1 --2- 172.21.15.167:0/380789275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8800cf970 0x7fa880154aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:39.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.807+0000 7fa885800640 1 --2- 172.21.15.167:0/380789275 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8800d04a0 0x7fa8801f8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:39.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.807+0000 7fa885800640 1 -- 172.21.15.167:0/380789275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8800cf970 msgr2=0x7fa880154aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:39.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.807+0000 7fa885800640 1 --2- 172.21.15.167:0/380789275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8800cf970 0x7fa880154aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:39.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.807+0000 7fa885800640 1 -- 172.21.15.167:0/380789275 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa88006faf0 con 0x7fa8800afcd0 2024-05-11T12:14:39.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.807+0000 7fa885800640 1 --2- 172.21.15.167:0/380789275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8800afcd0 0x7fa88006f5b0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fa87400c4d0 tx=0x7fa87400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:39.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.807+0000 7fa87d600640 1 -- 172.21.15.167:0/380789275 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa874017070 con 0x7fa8800afcd0 2024-05-11T12:14:39.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.807+0000 7fa87d600640 1 -- 172.21.15.167:0/380789275 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa874005d00 con 0x7fa8800afcd0 2024-05-11T12:14:39.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.807+0000 7fa87d600640 1 -- 172.21.15.167:0/380789275 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8740072b0 con 0x7fa8800afcd0 2024-05-11T12:14:39.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.807+0000 7fa886c00640 1 -- 172.21.15.167:0/380789275 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa880162560 con 0x7fa8800afcd0 2024-05-11T12:14:39.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.807+0000 7fa886c00640 1 -- 172.21.15.167:0/380789275 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa880162b20 con 0x7fa8800afcd0 2024-05-11T12:14:39.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.809+0000 7fa886c00640 1 -- 172.21.15.167:0/380789275 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8800694b0 con 0x7fa8800afcd0 2024-05-11T12:14:39.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.810+0000 7fa87d600640 1 -- 172.21.15.167:0/380789275 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa87401a070 con 0x7fa8800afcd0 2024-05-11T12:14:39.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.811+0000 7fa87d600640 1 --2- 172.21.15.167:0/380789275 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa8600762d0 0x7fa860078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:39.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.811+0000 7fa87d600640 1 -- 172.21.15.167:0/380789275 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6101+0+0 (secure 0 0 0) 0x7fa874096b50 con 0x7fa8800afcd0 2024-05-11T12:14:39.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.812+0000 7fa884e00640 1 --2- 172.21.15.167:0/380789275 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa8600762d0 0x7fa860078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:39.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.812+0000 7fa884e00640 1 --2- 172.21.15.167:0/380789275 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa8600762d0 0x7fa860078790 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa880154080 tx=0x7fa86c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:39.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:39.816+0000 7fa87d600640 1 -- 172.21.15.167:0/380789275 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa88006faf0 con 0x7fa8800afcd0 2024-05-11T12:14:40.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:40.021+0000 7fa886c00640 1 -- 172.21.15.167:0/380789275 --> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa88006c890 con 0x7fa8600762d0 2024-05-11T12:14:40.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:40.030+0000 7fa87d600640 1 -- 172.21.15.167:0/380789275 <== mgr.4106 v2:172.21.15.167:6832/578352559 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252074 (secure 0 0 0) 0x7fa86c008540 con 0x7fa8600762d0 2024-05-11T12:14:40.033 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:40.034 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2024-05-11T12:14:40.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:40.035+0000 7fa886c00640 1 -- 172.21.15.167:0/380789275 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa8600762d0 msgr2=0x7fa860078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:40.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:40.035+0000 7fa886c00640 1 --2- 172.21.15.167:0/380789275 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa8600762d0 0x7fa860078790 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa880154080 tx=0x7fa86c008040 comp rx=0 tx=0).stop 2024-05-11T12:14:40.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:40.035+0000 7fa886c00640 1 -- 172.21.15.167:0/380789275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8800afcd0 msgr2=0x7fa88006f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:40.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:40.035+0000 7fa886c00640 1 --2- 172.21.15.167:0/380789275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8800afcd0 0x7fa88006f5b0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fa87400c4d0 tx=0x7fa87400c990 comp rx=0 tx=0).stop 2024-05-11T12:14:40.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:40.035+0000 7fa886c00640 1 -- 172.21.15.167:0/380789275 shutdown_connections 2024-05-11T12:14:40.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:40.035+0000 7fa886c00640 1 --2- 172.21.15.167:0/380789275 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8800d04a0 0x7fa8801f8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:40.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:40.035+0000 7fa886c00640 1 --2- 172.21.15.167:0/380789275 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa8600762d0 0x7fa860078790 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:40.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:40.035+0000 7fa886c00640 1 --2- 172.21.15.167:0/380789275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8800cf970 0x7fa880154aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:40.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:40.035+0000 7fa886c00640 1 --2- 172.21.15.167:0/380789275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8800afcd0 0x7fa88006f5b0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:40.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:40.035+0000 7fa886c00640 1 -- 172.21.15.167:0/380789275 >> 172.21.15.167:0/380789275 conn(0x7fa8800b3460 msgr2=0x7fa8801596a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:40.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:40.036+0000 7fa886c00640 1 -- 172.21.15.167:0/380789275 shutdown_connections 2024-05-11T12:14:40.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:40.036+0000 7fa886c00640 1 -- 172.21.15.167:0/380789275 wait complete. 2024-05-11T12:14:40.051 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":214,"stamp":"2024-05-11T12:14:39.308657+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259996,"kb_used_data":10536,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714724,"statfs":{"total":773094113280,"available":772827877376,"internally_reserved":0,"allocated":10788864,"data_stored":5523124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.729196"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615834+0000","last_change":"2024-05-11T12:14:33.622872+0000","last_active":"2024-05-11T12:14:34.615834+0000","last_peered":"2024-05-11T12:14:34.615834+0000","last_clean":"2024-05-11T12:14:34.615834+0000","last_became_active":"2024-05-11T12:14:33.619136+0000","last_became_peered":"2024-05-11T12:14:33.619136+0000","last_unstale":"2024-05-11T12:14:34.615834+0000","last_undegraded":"2024-05-11T12:14:34.615834+0000","last_fullsized":"2024-05-11T12:14:34.615834+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:18:44.962195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.284196+0000","last_change":"2024-05-11T12:14:33.624137+0000","last_active":"2024-05-11T12:14:35.284196+0000","last_peered":"2024-05-11T12:14:35.284196+0000","last_clean":"2024-05-11T12:14:35.284196+0000","last_became_active":"2024-05-11T12:14:33.623671+0000","last_became_peered":"2024-05-11T12:14:33.623671+0000","last_unstale":"2024-05-11T12:14:35.284196+0000","last_undegraded":"2024-05-11T12:14:35.284196+0000","last_fullsized":"2024-05-11T12:14:35.284196+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:27:47.349240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615616+0000","last_change":"2024-05-11T12:14:33.626436+0000","last_active":"2024-05-11T12:14:34.615616+0000","last_peered":"2024-05-11T12:14:34.615616+0000","last_clean":"2024-05-11T12:14:34.615616+0000","last_became_active":"2024-05-11T12:14:33.626091+0000","last_became_peered":"2024-05-11T12:14:33.626091+0000","last_unstale":"2024-05-11T12:14:34.615616+0000","last_undegraded":"2024-05-11T12:14:34.615616+0000","last_fullsized":"2024-05-11T12:14:34.615616+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:17:05.023315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.284831+0000","last_change":"2024-05-11T12:14:33.624886+0000","last_active":"2024-05-11T12:14:35.284831+0000","last_peered":"2024-05-11T12:14:35.284831+0000","last_clean":"2024-05-11T12:14:35.284831+0000","last_became_active":"2024-05-11T12:14:33.618826+0000","last_became_peered":"2024-05-11T12:14:33.618826+0000","last_unstale":"2024-05-11T12:14:35.284831+0000","last_undegraded":"2024-05-11T12:14:35.284831+0000","last_fullsized":"2024-05-11T12:14:35.284831+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:56:09.727119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.283813+0000","last_change":"2024-05-11T12:14:33.618388+0000","last_active":"2024-05-11T12:14:35.283813+0000","last_peered":"2024-05-11T12:14:35.283813+0000","last_clean":"2024-05-11T12:14:35.283813+0000","last_became_active":"2024-05-11T12:14:33.618090+0000","last_became_peered":"2024-05-11T12:14:33.618090+0000","last_unstale":"2024-05-11T12:14:35.283813+0000","last_undegraded":"2024-05-11T12:14:35.283813+0000","last_fullsized":"2024-05-11T12:14:35.283813+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:00:00.861509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.283162+0000","last_change":"2024-05-11T12:14:33.634835+0000","last_active":"2024-05-11T12:14:35.283162+0000","last_peered":"2024-05-11T12:14:35.283162+0000","last_clean":"2024-05-11T12:14:35.283162+0000","last_became_active":"2024-05-11T12:14:33.634564+0000","last_became_peered":"2024-05-11T12:14:33.634564+0000","last_unstale":"2024-05-11T12:14:35.283162+0000","last_undegraded":"2024-05-11T12:14:35.283162+0000","last_fullsized":"2024-05-11T12:14:35.283162+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:11:02.751326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.282828+0000","last_change":"2024-05-11T12:14:33.634371+0000","last_active":"2024-05-11T12:14:35.282828+0000","last_peered":"2024-05-11T12:14:35.282828+0000","last_clean":"2024-05-11T12:14:35.282828+0000","last_became_active":"2024-05-11T12:14:33.634080+0000","last_became_peered":"2024-05-11T12:14:33.634080+0000","last_unstale":"2024-05-11T12:14:35.282828+0000","last_undegraded":"2024-05-11T12:14:35.282828+0000","last_fullsized":"2024-05-11T12:14:35.282828+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:15:22.180722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.208341+0000","last_change":"2024-05-11T12:14:33.621312+0000","last_active":"2024-05-11T12:14:35.208341+0000","last_peered":"2024-05-11T12:14:35.208341+0000","last_clean":"2024-05-11T12:14:35.208341+0000","last_became_active":"2024-05-11T12:14:33.621021+0000","last_became_peered":"2024-05-11T12:14:33.621021+0000","last_unstale":"2024-05-11T12:14:35.208341+0000","last_undegraded":"2024-05-11T12:14:35.208341+0000","last_fullsized":"2024-05-11T12:14:35.208341+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:54:45.084719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.284171+0000","last_change":"2024-05-11T12:14:33.622698+0000","last_active":"2024-05-11T12:14:35.284171+0000","last_peered":"2024-05-11T12:14:35.284171+0000","last_clean":"2024-05-11T12:14:35.284171+0000","last_became_active":"2024-05-11T12:14:33.622347+0000","last_became_peered":"2024-05-11T12:14:33.622347+0000","last_unstale":"2024-05-11T12:14:35.284171+0000","last_undegraded":"2024-05-11T12:14:35.284171+0000","last_fullsized":"2024-05-11T12:14:35.284171+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:16:12.500928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.283614+0000","last_change":"2024-05-11T12:14:33.623008+0000","last_active":"2024-05-11T12:14:35.283614+0000","last_peered":"2024-05-11T12:14:35.283614+0000","last_clean":"2024-05-11T12:14:35.283614+0000","last_became_active":"2024-05-11T12:14:33.622017+0000","last_became_peered":"2024-05-11T12:14:33.622017+0000","last_unstale":"2024-05-11T12:14:35.283614+0000","last_undegraded":"2024-05-11T12:14:35.283614+0000","last_fullsized":"2024-05-11T12:14:35.283614+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:11:02.751326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.619851+0000","last_change":"2024-05-11T12:14:33.626097+0000","last_active":"2024-05-11T12:14:34.619851+0000","last_peered":"2024-05-11T12:14:34.619851+0000","last_clean":"2024-05-11T12:14:34.619851+0000","last_became_active":"2024-05-11T12:14:33.624668+0000","last_became_peered":"2024-05-11T12:14:33.624668+0000","last_unstale":"2024-05-11T12:14:34.619851+0000","last_undegraded":"2024-05-11T12:14:34.619851+0000","last_fullsized":"2024-05-11T12:14:34.619851+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:13:15.093587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.621111+0000","last_change":"2024-05-11T12:14:33.629494+0000","last_active":"2024-05-11T12:14:34.621111+0000","last_peered":"2024-05-11T12:14:34.621111+0000","last_clean":"2024-05-11T12:14:34.621111+0000","last_became_active":"2024-05-11T12:14:33.629203+0000","last_became_peered":"2024-05-11T12:14:33.629203+0000","last_unstale":"2024-05-11T12:14:34.621111+0000","last_undegraded":"2024-05-11T12:14:34.621111+0000","last_fullsized":"2024-05-11T12:14:34.621111+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:15:22.180722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.283623+0000","last_change":"2024-05-11T12:14:33.607961+0000","last_active":"2024-05-11T12:14:35.283623+0000","last_peered":"2024-05-11T12:14:35.283623+0000","last_clean":"2024-05-11T12:14:35.283623+0000","last_became_active":"2024-05-11T12:14:33.607599+0000","last_became_peered":"2024-05-11T12:14:33.607599+0000","last_unstale":"2024-05-11T12:14:35.283623+0000","last_undegraded":"2024-05-11T12:14:35.283623+0000","last_fullsized":"2024-05-11T12:14:35.283623+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:22:15.346218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.184802+0000","last_change":"2024-05-11T12:14:33.626456+0000","last_active":"2024-05-11T12:14:35.184802+0000","last_peered":"2024-05-11T12:14:35.184802+0000","last_clean":"2024-05-11T12:14:35.184802+0000","last_became_active":"2024-05-11T12:14:33.625698+0000","last_became_peered":"2024-05-11T12:14:33.625698+0000","last_unstale":"2024-05-11T12:14:35.184802+0000","last_undegraded":"2024-05-11T12:14:35.184802+0000","last_fullsized":"2024-05-11T12:14:35.184802+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:35:20.045762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.184995+0000","last_change":"2024-05-11T12:14:33.632118+0000","last_active":"2024-05-11T12:14:35.184995+0000","last_peered":"2024-05-11T12:14:35.184995+0000","last_clean":"2024-05-11T12:14:35.184995+0000","last_became_active":"2024-05-11T12:14:33.631202+0000","last_became_peered":"2024-05-11T12:14:33.631202+0000","last_unstale":"2024-05-11T12:14:35.184995+0000","last_undegraded":"2024-05-11T12:14:35.184995+0000","last_fullsized":"2024-05-11T12:14:35.184995+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:56:09.727119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.184745+0000","last_change":"2024-05-11T12:14:33.626275+0000","last_active":"2024-05-11T12:14:35.184745+0000","last_peered":"2024-05-11T12:14:35.184745+0000","last_clean":"2024-05-11T12:14:35.184745+0000","last_became_active":"2024-05-11T12:14:33.625954+0000","last_became_peered":"2024-05-11T12:14:33.625954+0000","last_unstale":"2024-05-11T12:14:35.184745+0000","last_undegraded":"2024-05-11T12:14:35.184745+0000","last_fullsized":"2024-05-11T12:14:35.184745+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:22:15.346218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615878+0000","last_change":"2024-05-11T12:14:33.622549+0000","last_active":"2024-05-11T12:14:34.615878+0000","last_peered":"2024-05-11T12:14:34.615878+0000","last_clean":"2024-05-11T12:14:34.615878+0000","last_became_active":"2024-05-11T12:14:33.619603+0000","last_became_peered":"2024-05-11T12:14:33.619603+0000","last_unstale":"2024-05-11T12:14:34.615878+0000","last_undegraded":"2024-05-11T12:14:34.615878+0000","last_fullsized":"2024-05-11T12:14:34.615878+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:51:30.951966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.284305+0000","last_change":"2024-05-11T12:14:33.636255+0000","last_active":"2024-05-11T12:14:35.284305+0000","last_peered":"2024-05-11T12:14:35.284305+0000","last_clean":"2024-05-11T12:14:35.284305+0000","last_became_active":"2024-05-11T12:14:33.635916+0000","last_became_peered":"2024-05-11T12:14:33.635916+0000","last_unstale":"2024-05-11T12:14:35.284305+0000","last_undegraded":"2024-05-11T12:14:35.284305+0000","last_fullsized":"2024-05-11T12:14:35.284305+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:52:46.883659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615768+0000","last_change":"2024-05-11T12:14:33.635269+0000","last_active":"2024-05-11T12:14:34.615768+0000","last_peered":"2024-05-11T12:14:34.615768+0000","last_clean":"2024-05-11T12:14:34.615768+0000","last_became_active":"2024-05-11T12:14:33.634795+0000","last_became_peered":"2024-05-11T12:14:33.634795+0000","last_unstale":"2024-05-11T12:14:34.615768+0000","last_undegraded":"2024-05-11T12:14:34.615768+0000","last_fullsized":"2024-05-11T12:14:34.615768+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:13:16.630356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.621591+0000","last_change":"2024-05-11T12:14:33.615290+0000","last_active":"2024-05-11T12:14:34.621591+0000","last_peered":"2024-05-11T12:14:34.621591+0000","last_clean":"2024-05-11T12:14:34.621591+0000","last_became_active":"2024-05-11T12:14:33.614910+0000","last_became_peered":"2024-05-11T12:14:33.614910+0000","last_unstale":"2024-05-11T12:14:34.621591+0000","last_undegraded":"2024-05-11T12:14:34.621591+0000","last_fullsized":"2024-05-11T12:14:34.621591+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:22:15.346218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.208740+0000","last_change":"2024-05-11T12:14:33.621162+0000","last_active":"2024-05-11T12:14:35.208740+0000","last_peered":"2024-05-11T12:14:35.208740+0000","last_clean":"2024-05-11T12:14:35.208740+0000","last_became_active":"2024-05-11T12:14:33.620843+0000","last_became_peered":"2024-05-11T12:14:33.620843+0000","last_unstale":"2024-05-11T12:14:35.208740+0000","last_undegraded":"2024-05-11T12:14:35.208740+0000","last_fullsized":"2024-05-11T12:14:35.208740+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:29:39.246958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.284670+0000","last_change":"2024-05-11T12:14:33.623513+0000","last_active":"2024-05-11T12:14:35.284670+0000","last_peered":"2024-05-11T12:14:35.284670+0000","last_clean":"2024-05-11T12:14:35.284670+0000","last_became_active":"2024-05-11T12:14:33.619839+0000","last_became_peered":"2024-05-11T12:14:33.619839+0000","last_unstale":"2024-05-11T12:14:35.284670+0000","last_undegraded":"2024-05-11T12:14:35.284670+0000","last_fullsized":"2024-05-11T12:14:35.284670+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:35:20.045762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615236+0000","last_change":"2024-05-11T12:14:33.619917+0000","last_active":"2024-05-11T12:14:34.615236+0000","last_peered":"2024-05-11T12:14:34.615236+0000","last_clean":"2024-05-11T12:14:34.615236+0000","last_became_active":"2024-05-11T12:14:33.619614+0000","last_became_peered":"2024-05-11T12:14:33.619614+0000","last_unstale":"2024-05-11T12:14:34.615236+0000","last_undegraded":"2024-05-11T12:14:34.615236+0000","last_fullsized":"2024-05-11T12:14:34.615236+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:56:17.539601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.283165+0000","last_change":"2024-05-11T12:14:33.614192+0000","last_active":"2024-05-11T12:14:35.283165+0000","last_peered":"2024-05-11T12:14:35.283165+0000","last_clean":"2024-05-11T12:14:35.283165+0000","last_became_active":"2024-05-11T12:14:33.613880+0000","last_became_peered":"2024-05-11T12:14:33.613880+0000","last_unstale":"2024-05-11T12:14:35.283165+0000","last_undegraded":"2024-05-11T12:14:35.283165+0000","last_fullsized":"2024-05-11T12:14:35.283165+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:13:15.093587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.620105+0000","last_change":"2024-05-11T12:14:33.620900+0000","last_active":"2024-05-11T12:14:34.620105+0000","last_peered":"2024-05-11T12:14:34.620105+0000","last_clean":"2024-05-11T12:14:34.620105+0000","last_became_active":"2024-05-11T12:14:33.620614+0000","last_became_peered":"2024-05-11T12:14:33.620614+0000","last_unstale":"2024-05-11T12:14:34.620105+0000","last_undegraded":"2024-05-11T12:14:34.620105+0000","last_fullsized":"2024-05-11T12:14:34.620105+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:17:04.293672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.620227+0000","last_change":"2024-05-11T12:14:33.628258+0000","last_active":"2024-05-11T12:14:34.620227+0000","last_peered":"2024-05-11T12:14:34.620227+0000","last_clean":"2024-05-11T12:14:34.620227+0000","last_became_active":"2024-05-11T12:14:33.627691+0000","last_became_peered":"2024-05-11T12:14:33.627691+0000","last_unstale":"2024-05-11T12:14:34.620227+0000","last_undegraded":"2024-05-11T12:14:34.620227+0000","last_fullsized":"2024-05-11T12:14:34.620227+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:15:52.602707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.621886+0000","last_change":"2024-05-11T12:14:33.614817+0000","last_active":"2024-05-11T12:14:34.621886+0000","last_peered":"2024-05-11T12:14:34.621886+0000","last_clean":"2024-05-11T12:14:34.621886+0000","last_became_active":"2024-05-11T12:14:33.614446+0000","last_became_peered":"2024-05-11T12:14:33.614446+0000","last_unstale":"2024-05-11T12:14:34.621886+0000","last_undegraded":"2024-05-11T12:14:34.621886+0000","last_fullsized":"2024-05-11T12:14:34.621886+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:00:00.861509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.283265+0000","last_change":"2024-05-11T12:14:33.630308+0000","last_active":"2024-05-11T12:14:35.283265+0000","last_peered":"2024-05-11T12:14:35.283265+0000","last_clean":"2024-05-11T12:14:35.283265+0000","last_became_active":"2024-05-11T12:14:33.629962+0000","last_became_peered":"2024-05-11T12:14:33.629962+0000","last_unstale":"2024-05-11T12:14:35.283265+0000","last_undegraded":"2024-05-11T12:14:35.283265+0000","last_fullsized":"2024-05-11T12:14:35.283265+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:35:20.045762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209481+0000","last_change":"2024-05-11T12:14:33.623708+0000","last_active":"2024-05-11T12:14:35.209481+0000","last_peered":"2024-05-11T12:14:35.209481+0000","last_clean":"2024-05-11T12:14:35.209481+0000","last_became_active":"2024-05-11T12:14:33.623429+0000","last_became_peered":"2024-05-11T12:14:33.623429+0000","last_unstale":"2024-05-11T12:14:35.209481+0000","last_undegraded":"2024-05-11T12:14:35.209481+0000","last_fullsized":"2024-05-11T12:14:35.209481+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:30:11.411742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.621113+0000","last_change":"2024-05-11T12:14:33.625235+0000","last_active":"2024-05-11T12:14:34.621113+0000","last_peered":"2024-05-11T12:14:34.621113+0000","last_clean":"2024-05-11T12:14:34.621113+0000","last_became_active":"2024-05-11T12:14:33.624914+0000","last_became_peered":"2024-05-11T12:14:33.624914+0000","last_unstale":"2024-05-11T12:14:34.621113+0000","last_undegraded":"2024-05-11T12:14:34.621113+0000","last_fullsized":"2024-05-11T12:14:34.621113+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:13:15.093587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.282836+0000","last_change":"2024-05-11T12:14:33.625241+0000","last_active":"2024-05-11T12:14:35.282836+0000","last_peered":"2024-05-11T12:14:35.282836+0000","last_clean":"2024-05-11T12:14:35.282836+0000","last_became_active":"2024-05-11T12:14:33.623619+0000","last_became_peered":"2024-05-11T12:14:33.623619+0000","last_unstale":"2024-05-11T12:14:35.282836+0000","last_undegraded":"2024-05-11T12:14:35.282836+0000","last_fullsized":"2024-05-11T12:14:35.282836+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:00:00.861509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.208839+0000","last_change":"2024-05-11T12:14:33.621606+0000","last_active":"2024-05-11T12:14:35.208839+0000","last_peered":"2024-05-11T12:14:35.208839+0000","last_clean":"2024-05-11T12:14:35.208839+0000","last_became_active":"2024-05-11T12:14:33.621288+0000","last_became_peered":"2024-05-11T12:14:33.621288+0000","last_unstale":"2024-05-11T12:14:35.208839+0000","last_undegraded":"2024-05-11T12:14:35.208839+0000","last_fullsized":"2024-05-11T12:14:35.208839+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:37:19.519120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615469+0000","last_change":"2024-05-11T12:14:33.633794+0000","last_active":"2024-05-11T12:14:34.615469+0000","last_peered":"2024-05-11T12:14:34.615469+0000","last_clean":"2024-05-11T12:14:34.615469+0000","last_became_active":"2024-05-11T12:14:33.633451+0000","last_became_peered":"2024-05-11T12:14:33.633451+0000","last_unstale":"2024-05-11T12:14:34.615469+0000","last_undegraded":"2024-05-11T12:14:34.615469+0000","last_fullsized":"2024-05-11T12:14:34.615469+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:31:39.465305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"96'4","reported_seq":18,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.285496+0000","last_change":"2024-05-11T12:14:32.610045+0000","last_active":"2024-05-11T12:14:35.285496+0000","last_peered":"2024-05-11T12:14:35.285496+0000","last_clean":"2024-05-11T12:14:35.285496+0000","last_became_active":"2024-05-11T12:14:32.609746+0000","last_became_peered":"2024-05-11T12:14:32.609746+0000","last_unstale":"2024-05-11T12:14:35.285496+0000","last_undegraded":"2024-05-11T12:14:35.285496+0000","last_fullsized":"2024-05-11T12:14:35.285496+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:44:56.684605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.208579+0000","last_change":"2024-05-11T12:14:33.630664+0000","last_active":"2024-05-11T12:14:35.208579+0000","last_peered":"2024-05-11T12:14:35.208579+0000","last_clean":"2024-05-11T12:14:35.208579+0000","last_became_active":"2024-05-11T12:14:33.630171+0000","last_became_peered":"2024-05-11T12:14:33.630171+0000","last_unstale":"2024-05-11T12:14:35.208579+0000","last_undegraded":"2024-05-11T12:14:35.208579+0000","last_fullsized":"2024-05-11T12:14:35.208579+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:00:10.906551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209543+0000","last_change":"2024-05-11T12:14:32.619446+0000","last_active":"2024-05-11T12:14:35.209543+0000","last_peered":"2024-05-11T12:14:35.209543+0000","last_clean":"2024-05-11T12:14:35.209543+0000","last_became_active":"2024-05-11T12:14:32.619219+0000","last_became_peered":"2024-05-11T12:14:32.619219+0000","last_unstale":"2024-05-11T12:14:35.209543+0000","last_undegraded":"2024-05-11T12:14:35.209543+0000","last_fullsized":"2024-05-11T12:14:35.209543+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:34:15.881914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.621298+0000","last_change":"2024-05-11T12:14:33.623282+0000","last_active":"2024-05-11T12:14:34.621298+0000","last_peered":"2024-05-11T12:14:34.621298+0000","last_clean":"2024-05-11T12:14:34.621298+0000","last_became_active":"2024-05-11T12:14:33.622997+0000","last_became_peered":"2024-05-11T12:14:33.622997+0000","last_unstale":"2024-05-11T12:14:34.621298+0000","last_undegraded":"2024-05-11T12:14:34.621298+0000","last_fullsized":"2024-05-11T12:14:34.621298+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:00:00.861509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.619909+0000","last_change":"2024-05-11T12:14:32.618332+0000","last_active":"2024-05-11T12:14:34.619909+0000","last_peered":"2024-05-11T12:14:34.619909+0000","last_clean":"2024-05-11T12:14:34.619909+0000","last_became_active":"2024-05-11T12:14:32.614777+0000","last_became_peered":"2024-05-11T12:14:32.614777+0000","last_unstale":"2024-05-11T12:14:34.619909+0000","last_undegraded":"2024-05-11T12:14:34.619909+0000","last_fullsized":"2024-05-11T12:14:34.619909+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:55:53.087982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.620038+0000","last_change":"2024-05-11T12:14:33.617665+0000","last_active":"2024-05-11T12:14:34.620038+0000","last_peered":"2024-05-11T12:14:34.620038+0000","last_clean":"2024-05-11T12:14:34.620038+0000","last_became_active":"2024-05-11T12:14:33.617251+0000","last_became_peered":"2024-05-11T12:14:33.617251+0000","last_unstale":"2024-05-11T12:14:34.620038+0000","last_undegraded":"2024-05-11T12:14:34.620038+0000","last_fullsized":"2024-05-11T12:14:34.620038+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:15:22.180722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.283263+0000","last_change":"2024-05-11T12:14:32.610836+0000","last_active":"2024-05-11T12:14:35.283263+0000","last_peered":"2024-05-11T12:14:35.283263+0000","last_clean":"2024-05-11T12:14:35.283263+0000","last_became_active":"2024-05-11T12:14:32.610542+0000","last_became_peered":"2024-05-11T12:14:32.610542+0000","last_unstale":"2024-05-11T12:14:35.283263+0000","last_undegraded":"2024-05-11T12:14:35.283263+0000","last_fullsized":"2024-05-11T12:14:35.283263+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:55:53.087982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.284849+0000","last_change":"2024-05-11T12:14:33.616596+0000","last_active":"2024-05-11T12:14:35.284849+0000","last_peered":"2024-05-11T12:14:35.284849+0000","last_clean":"2024-05-11T12:14:35.284849+0000","last_became_active":"2024-05-11T12:14:33.616294+0000","last_became_peered":"2024-05-11T12:14:33.616294+0000","last_unstale":"2024-05-11T12:14:35.284849+0000","last_undegraded":"2024-05-11T12:14:35.284849+0000","last_fullsized":"2024-05-11T12:14:35.284849+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:27:47.349240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.184868+0000","last_change":"2024-05-11T12:14:32.619606+0000","last_active":"2024-05-11T12:14:35.184868+0000","last_peered":"2024-05-11T12:14:35.184868+0000","last_clean":"2024-05-11T12:14:35.184868+0000","last_became_active":"2024-05-11T12:14:32.619133+0000","last_became_peered":"2024-05-11T12:14:32.619133+0000","last_unstale":"2024-05-11T12:14:35.184868+0000","last_undegraded":"2024-05-11T12:14:35.184868+0000","last_fullsized":"2024-05-11T12:14:35.184868+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:17:03.277324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.616178+0000","last_change":"2024-05-11T12:14:33.634226+0000","last_active":"2024-05-11T12:14:34.616178+0000","last_peered":"2024-05-11T12:14:34.616178+0000","last_clean":"2024-05-11T12:14:34.616178+0000","last_became_active":"2024-05-11T12:14:33.633932+0000","last_became_peered":"2024-05-11T12:14:33.633932+0000","last_unstale":"2024-05-11T12:14:34.616178+0000","last_undegraded":"2024-05-11T12:14:34.616178+0000","last_fullsized":"2024-05-11T12:14:34.616178+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:01:12.394223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.622171+0000","last_change":"2024-05-11T12:14:32.614159+0000","last_active":"2024-05-11T12:14:34.622171+0000","last_peered":"2024-05-11T12:14:34.622171+0000","last_clean":"2024-05-11T12:14:34.622171+0000","last_became_active":"2024-05-11T12:14:32.613421+0000","last_became_peered":"2024-05-11T12:14:32.613421+0000","last_unstale":"2024-05-11T12:14:34.622171+0000","last_undegraded":"2024-05-11T12:14:34.622171+0000","last_fullsized":"2024-05-11T12:14:34.622171+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:55:53.087982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.284582+0000","last_change":"2024-05-11T12:14:33.632315+0000","last_active":"2024-05-11T12:14:35.284582+0000","last_peered":"2024-05-11T12:14:35.284582+0000","last_clean":"2024-05-11T12:14:35.284582+0000","last_became_active":"2024-05-11T12:14:33.626868+0000","last_became_peered":"2024-05-11T12:14:33.626868+0000","last_unstale":"2024-05-11T12:14:35.284582+0000","last_undegraded":"2024-05-11T12:14:35.284582+0000","last_fullsized":"2024-05-11T12:14:35.284582+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:26:18.712084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.621042+0000","last_change":"2024-05-11T12:14:32.614638+0000","last_active":"2024-05-11T12:14:34.621042+0000","last_peered":"2024-05-11T12:14:34.621042+0000","last_clean":"2024-05-11T12:14:34.621042+0000","last_became_active":"2024-05-11T12:14:32.614242+0000","last_became_peered":"2024-05-11T12:14:32.614242+0000","last_unstale":"2024-05-11T12:14:34.621042+0000","last_undegraded":"2024-05-11T12:14:34.621042+0000","last_fullsized":"2024-05-11T12:14:34.621042+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:58:41.059975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.285649+0000","last_change":"2024-05-11T12:14:33.615167+0000","last_active":"2024-05-11T12:14:35.285649+0000","last_peered":"2024-05-11T12:14:35.285649+0000","last_clean":"2024-05-11T12:14:35.285649+0000","last_became_active":"2024-05-11T12:14:33.614898+0000","last_became_peered":"2024-05-11T12:14:33.614898+0000","last_unstale":"2024-05-11T12:14:35.285649+0000","last_undegraded":"2024-05-11T12:14:35.285649+0000","last_fullsized":"2024-05-11T12:14:35.285649+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:52:46.883659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.283863+0000","last_change":"2024-05-11T12:14:32.605627+0000","last_active":"2024-05-11T12:14:35.283863+0000","last_peered":"2024-05-11T12:14:35.283863+0000","last_clean":"2024-05-11T12:14:35.283863+0000","last_became_active":"2024-05-11T12:14:32.604729+0000","last_became_peered":"2024-05-11T12:14:32.604729+0000","last_unstale":"2024-05-11T12:14:35.283863+0000","last_undegraded":"2024-05-11T12:14:35.283863+0000","last_fullsized":"2024-05-11T12:14:35.283863+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:00:56.111300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615229+0000","last_change":"2024-05-11T12:14:33.635074+0000","last_active":"2024-05-11T12:14:34.615229+0000","last_peered":"2024-05-11T12:14:34.615229+0000","last_clean":"2024-05-11T12:14:34.615229+0000","last_became_active":"2024-05-11T12:14:33.634606+0000","last_became_peered":"2024-05-11T12:14:33.634606+0000","last_unstale":"2024-05-11T12:14:34.615229+0000","last_undegraded":"2024-05-11T12:14:34.615229+0000","last_fullsized":"2024-05-11T12:14:34.615229+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:03:07.386179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.620538+0000","last_change":"2024-05-11T12:14:32.615191+0000","last_active":"2024-05-11T12:14:34.620538+0000","last_peered":"2024-05-11T12:14:34.620538+0000","last_clean":"2024-05-11T12:14:34.620538+0000","last_became_active":"2024-05-11T12:14:32.614809+0000","last_became_peered":"2024-05-11T12:14:32.614809+0000","last_unstale":"2024-05-11T12:14:34.620538+0000","last_undegraded":"2024-05-11T12:14:34.620538+0000","last_fullsized":"2024-05-11T12:14:34.620538+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:30:51.604601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.282918+0000","last_change":"2024-05-11T12:14:33.624689+0000","last_active":"2024-05-11T12:14:35.282918+0000","last_peered":"2024-05-11T12:14:35.282918+0000","last_clean":"2024-05-11T12:14:35.282918+0000","last_became_active":"2024-05-11T12:14:33.624275+0000","last_became_peered":"2024-05-11T12:14:33.624275+0000","last_unstale":"2024-05-11T12:14:35.282918+0000","last_undegraded":"2024-05-11T12:14:35.282918+0000","last_fullsized":"2024-05-11T12:14:35.282918+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:15:52.602707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615602+0000","last_change":"2024-05-11T12:14:32.609650+0000","last_active":"2024-05-11T12:14:34.615602+0000","last_peered":"2024-05-11T12:14:34.615602+0000","last_clean":"2024-05-11T12:14:34.615602+0000","last_became_active":"2024-05-11T12:14:32.609340+0000","last_became_peered":"2024-05-11T12:14:32.609340+0000","last_unstale":"2024-05-11T12:14:34.615602+0000","last_undegraded":"2024-05-11T12:14:34.615602+0000","last_fullsized":"2024-05-11T12:14:34.615602+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:40:56.223685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209330+0000","last_change":"2024-05-11T12:14:33.631013+0000","last_active":"2024-05-11T12:14:35.209330+0000","last_peered":"2024-05-11T12:14:35.209330+0000","last_clean":"2024-05-11T12:14:35.209330+0000","last_became_active":"2024-05-11T12:14:33.630698+0000","last_became_peered":"2024-05-11T12:14:33.630698+0000","last_unstale":"2024-05-11T12:14:35.209330+0000","last_undegraded":"2024-05-11T12:14:35.209330+0000","last_fullsized":"2024-05-11T12:14:35.209330+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:32:48.749691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"96'3","reported_seq":17,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.185206+0000","last_change":"2024-05-11T12:14:32.618526+0000","last_active":"2024-05-11T12:14:35.185206+0000","last_peered":"2024-05-11T12:14:35.185206+0000","last_clean":"2024-05-11T12:14:35.185206+0000","last_became_active":"2024-05-11T12:14:32.618198+0000","last_became_peered":"2024-05-11T12:14:32.618198+0000","last_unstale":"2024-05-11T12:14:35.185206+0000","last_undegraded":"2024-05-11T12:14:35.185206+0000","last_fullsized":"2024-05-11T12:14:35.185206+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:55:59.198545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.283938+0000","last_change":"2024-05-11T12:14:33.623135+0000","last_active":"2024-05-11T12:14:35.283938+0000","last_peered":"2024-05-11T12:14:35.283938+0000","last_clean":"2024-05-11T12:14:35.283938+0000","last_became_active":"2024-05-11T12:14:33.622788+0000","last_became_peered":"2024-05-11T12:14:33.622788+0000","last_unstale":"2024-05-11T12:14:35.283938+0000","last_undegraded":"2024-05-11T12:14:35.283938+0000","last_fullsized":"2024-05-11T12:14:35.283938+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:22:15.346218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"96'6","reported_seq":20,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.283185+0000","last_change":"2024-05-11T12:14:32.607489+0000","last_active":"2024-05-11T12:14:35.283185+0000","last_peered":"2024-05-11T12:14:35.283185+0000","last_clean":"2024-05-11T12:14:35.283185+0000","last_became_active":"2024-05-11T12:14:32.606078+0000","last_became_peered":"2024-05-11T12:14:32.606078+0000","last_unstale":"2024-05-11T12:14:35.283185+0000","last_undegraded":"2024-05-11T12:14:35.283185+0000","last_fullsized":"2024-05-11T12:14:35.283185+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:55:53.087982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.619221+0000","last_change":"2024-05-11T12:14:33.616862+0000","last_active":"2024-05-11T12:14:34.619221+0000","last_peered":"2024-05-11T12:14:34.619221+0000","last_clean":"2024-05-11T12:14:34.619221+0000","last_became_active":"2024-05-11T12:14:33.616400+0000","last_became_peered":"2024-05-11T12:14:33.616400+0000","last_unstale":"2024-05-11T12:14:34.619221+0000","last_undegraded":"2024-05-11T12:14:34.619221+0000","last_fullsized":"2024-05-11T12:14:34.619221+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:59:12.515990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.283960+0000","last_change":"2024-05-11T12:14:32.607882+0000","last_active":"2024-05-11T12:14:35.283960+0000","last_peered":"2024-05-11T12:14:35.283960+0000","last_clean":"2024-05-11T12:14:35.283960+0000","last_became_active":"2024-05-11T12:14:32.607528+0000","last_became_peered":"2024-05-11T12:14:32.607528+0000","last_unstale":"2024-05-11T12:14:35.283960+0000","last_undegraded":"2024-05-11T12:14:35.283960+0000","last_fullsized":"2024-05-11T12:14:35.283960+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:34:52.216177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.284047+0000","last_change":"2024-05-11T12:14:33.628739+0000","last_active":"2024-05-11T12:14:35.284047+0000","last_peered":"2024-05-11T12:14:35.284047+0000","last_clean":"2024-05-11T12:14:35.284047+0000","last_became_active":"2024-05-11T12:14:33.628423+0000","last_became_peered":"2024-05-11T12:14:33.628423+0000","last_unstale":"2024-05-11T12:14:35.284047+0000","last_undegraded":"2024-05-11T12:14:35.284047+0000","last_fullsized":"2024-05-11T12:14:35.284047+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:56:09.727119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.284356+0000","last_change":"2024-05-11T12:14:32.616922+0000","last_active":"2024-05-11T12:14:35.284356+0000","last_peered":"2024-05-11T12:14:35.284356+0000","last_clean":"2024-05-11T12:14:35.284356+0000","last_became_active":"2024-05-11T12:14:32.616599+0000","last_became_peered":"2024-05-11T12:14:32.616599+0000","last_unstale":"2024-05-11T12:14:35.284356+0000","last_undegraded":"2024-05-11T12:14:35.284356+0000","last_fullsized":"2024-05-11T12:14:35.284356+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:55:59.198545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209297+0000","last_change":"2024-05-11T12:14:33.624455+0000","last_active":"2024-05-11T12:14:35.209297+0000","last_peered":"2024-05-11T12:14:35.209297+0000","last_clean":"2024-05-11T12:14:35.209297+0000","last_became_active":"2024-05-11T12:14:33.624158+0000","last_became_peered":"2024-05-11T12:14:33.624158+0000","last_unstale":"2024-05-11T12:14:35.209297+0000","last_undegraded":"2024-05-11T12:14:35.209297+0000","last_fullsized":"2024-05-11T12:14:35.209297+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:44:32.910714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.208576+0000","last_change":"2024-05-11T12:14:32.615486+0000","last_active":"2024-05-11T12:14:35.208576+0000","last_peered":"2024-05-11T12:14:35.208576+0000","last_clean":"2024-05-11T12:14:35.208576+0000","last_became_active":"2024-05-11T12:14:32.615204+0000","last_became_peered":"2024-05-11T12:14:32.615204+0000","last_unstale":"2024-05-11T12:14:35.208576+0000","last_undegraded":"2024-05-11T12:14:35.208576+0000","last_fullsized":"2024-05-11T12:14:35.208576+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:27:00.766407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615938+0000","last_change":"2024-05-11T12:14:33.623140+0000","last_active":"2024-05-11T12:14:34.615938+0000","last_peered":"2024-05-11T12:14:34.615938+0000","last_clean":"2024-05-11T12:14:34.615938+0000","last_became_active":"2024-05-11T12:14:33.618444+0000","last_became_peered":"2024-05-11T12:14:33.618444+0000","last_unstale":"2024-05-11T12:14:34.615938+0000","last_undegraded":"2024-05-11T12:14:34.615938+0000","last_fullsized":"2024-05-11T12:14:34.615938+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:35:51.107577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"96'8","reported_seq":22,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.619539+0000","last_change":"2024-05-11T12:14:32.616263+0000","last_active":"2024-05-11T12:14:34.619539+0000","last_peered":"2024-05-11T12:14:34.619539+0000","last_clean":"2024-05-11T12:14:34.619539+0000","last_became_active":"2024-05-11T12:14:32.615004+0000","last_became_peered":"2024-05-11T12:14:32.615004+0000","last_unstale":"2024-05-11T12:14:34.619539+0000","last_undegraded":"2024-05-11T12:14:34.619539+0000","last_fullsized":"2024-05-11T12:14:34.619539+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:54:11.073754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.208800+0000","last_change":"2024-05-11T12:14:33.616435+0000","last_active":"2024-05-11T12:14:35.208800+0000","last_peered":"2024-05-11T12:14:35.208800+0000","last_clean":"2024-05-11T12:14:35.208800+0000","last_became_active":"2024-05-11T12:14:33.616110+0000","last_became_peered":"2024-05-11T12:14:33.616110+0000","last_unstale":"2024-05-11T12:14:35.208800+0000","last_undegraded":"2024-05-11T12:14:35.208800+0000","last_fullsized":"2024-05-11T12:14:35.208800+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:51:10.706767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.284168+0000","last_change":"2024-05-11T12:14:32.613751+0000","last_active":"2024-05-11T12:14:35.284168+0000","last_peered":"2024-05-11T12:14:35.284168+0000","last_clean":"2024-05-11T12:14:35.284168+0000","last_became_active":"2024-05-11T12:14:32.613431+0000","last_became_peered":"2024-05-11T12:14:32.613431+0000","last_unstale":"2024-05-11T12:14:35.284168+0000","last_undegraded":"2024-05-11T12:14:35.284168+0000","last_fullsized":"2024-05-11T12:14:35.284168+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:07:53.042878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.283203+0000","last_change":"2024-05-11T12:14:33.637186+0000","last_active":"2024-05-11T12:14:35.283203+0000","last_peered":"2024-05-11T12:14:35.283203+0000","last_clean":"2024-05-11T12:14:35.283203+0000","last_became_active":"2024-05-11T12:14:33.636807+0000","last_became_peered":"2024-05-11T12:14:33.636807+0000","last_unstale":"2024-05-11T12:14:35.283203+0000","last_undegraded":"2024-05-11T12:14:35.283203+0000","last_fullsized":"2024-05-11T12:14:35.283203+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:22:15.346218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.185187+0000","last_change":"2024-05-11T12:14:32.621615+0000","last_active":"2024-05-11T12:14:35.185187+0000","last_peered":"2024-05-11T12:14:35.185187+0000","last_clean":"2024-05-11T12:14:35.185187+0000","last_became_active":"2024-05-11T12:14:32.620973+0000","last_became_peered":"2024-05-11T12:14:32.620973+0000","last_unstale":"2024-05-11T12:14:35.185187+0000","last_undegraded":"2024-05-11T12:14:35.185187+0000","last_fullsized":"2024-05-11T12:14:35.185187+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:59:11.499643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.283520+0000","last_change":"2024-05-11T12:14:33.615559+0000","last_active":"2024-05-11T12:14:35.283520+0000","last_peered":"2024-05-11T12:14:35.283520+0000","last_clean":"2024-05-11T12:14:35.283520+0000","last_became_active":"2024-05-11T12:14:33.615276+0000","last_became_peered":"2024-05-11T12:14:33.615276+0000","last_unstale":"2024-05-11T12:14:35.283520+0000","last_undegraded":"2024-05-11T12:14:35.283520+0000","last_fullsized":"2024-05-11T12:14:35.283520+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:15:22.180722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.619522+0000","last_change":"2024-05-11T12:14:32.609813+0000","last_active":"2024-05-11T12:14:34.619522+0000","last_peered":"2024-05-11T12:14:34.619522+0000","last_clean":"2024-05-11T12:14:34.619522+0000","last_became_active":"2024-05-11T12:14:32.609531+0000","last_became_peered":"2024-05-11T12:14:32.609531+0000","last_unstale":"2024-05-11T12:14:34.619522+0000","last_undegraded":"2024-05-11T12:14:34.619522+0000","last_fullsized":"2024-05-11T12:14:34.619522+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:30:51.604601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615796+0000","last_change":"2024-05-11T12:14:33.629837+0000","last_active":"2024-05-11T12:14:34.615796+0000","last_peered":"2024-05-11T12:14:34.615796+0000","last_clean":"2024-05-11T12:14:34.615796+0000","last_became_active":"2024-05-11T12:14:33.629561+0000","last_became_peered":"2024-05-11T12:14:33.629561+0000","last_unstale":"2024-05-11T12:14:34.615796+0000","last_undegraded":"2024-05-11T12:14:34.615796+0000","last_fullsized":"2024-05-11T12:14:34.615796+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:24:04.957521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209466+0000","last_change":"2024-05-11T12:14:32.605666+0000","last_active":"2024-05-11T12:14:35.209466+0000","last_peered":"2024-05-11T12:14:35.209466+0000","last_clean":"2024-05-11T12:14:35.209466+0000","last_became_active":"2024-05-11T12:14:32.605407+0000","last_became_peered":"2024-05-11T12:14:32.605407+0000","last_unstale":"2024-05-11T12:14:35.209466+0000","last_undegraded":"2024-05-11T12:14:35.209466+0000","last_fullsized":"2024-05-11T12:14:35.209466+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:31:24.122325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615946+0000","last_change":"2024-05-11T12:14:33.623726+0000","last_active":"2024-05-11T12:14:34.615946+0000","last_peered":"2024-05-11T12:14:34.615946+0000","last_clean":"2024-05-11T12:14:34.615946+0000","last_became_active":"2024-05-11T12:14:33.616682+0000","last_became_peered":"2024-05-11T12:14:33.616682+0000","last_unstale":"2024-05-11T12:14:34.615946+0000","last_undegraded":"2024-05-11T12:14:34.615946+0000","last_fullsized":"2024-05-11T12:14:34.615946+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:11:09.187757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.208810+0000","last_change":"2024-05-11T12:14:32.612947+0000","last_active":"2024-05-11T12:14:35.208810+0000","last_peered":"2024-05-11T12:14:35.208810+0000","last_clean":"2024-05-11T12:14:35.208810+0000","last_became_active":"2024-05-11T12:14:32.612545+0000","last_became_peered":"2024-05-11T12:14:32.612545+0000","last_unstale":"2024-05-11T12:14:35.208810+0000","last_undegraded":"2024-05-11T12:14:35.208810+0000","last_fullsized":"2024-05-11T12:14:35.208810+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:10:08.253641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.285245+0000","last_change":"2024-05-11T12:14:33.628473+0000","last_active":"2024-05-11T12:14:35.285245+0000","last_peered":"2024-05-11T12:14:35.285245+0000","last_clean":"2024-05-11T12:14:35.285245+0000","last_became_active":"2024-05-11T12:14:33.627301+0000","last_became_peered":"2024-05-11T12:14:33.627301+0000","last_unstale":"2024-05-11T12:14:35.285245+0000","last_undegraded":"2024-05-11T12:14:35.285245+0000","last_fullsized":"2024-05-11T12:14:35.285245+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:16:12.500928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.283089+0000","last_change":"2024-05-11T12:14:32.607325+0000","last_active":"2024-05-11T12:14:35.283089+0000","last_peered":"2024-05-11T12:14:35.283089+0000","last_clean":"2024-05-11T12:14:35.283089+0000","last_became_active":"2024-05-11T12:14:32.606982+0000","last_became_peered":"2024-05-11T12:14:32.606982+0000","last_unstale":"2024-05-11T12:14:35.283089+0000","last_undegraded":"2024-05-11T12:14:35.283089+0000","last_fullsized":"2024-05-11T12:14:35.283089+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:54:11.073754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209260+0000","last_change":"2024-05-11T12:14:33.618158+0000","last_active":"2024-05-11T12:14:35.209260+0000","last_peered":"2024-05-11T12:14:35.209260+0000","last_clean":"2024-05-11T12:14:35.209260+0000","last_became_active":"2024-05-11T12:14:33.617850+0000","last_became_peered":"2024-05-11T12:14:33.617850+0000","last_unstale":"2024-05-11T12:14:35.209260+0000","last_undegraded":"2024-05-11T12:14:35.209260+0000","last_fullsized":"2024-05-11T12:14:35.209260+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:38:25.159336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209121+0000","last_change":"2024-05-11T12:14:32.607439+0000","last_active":"2024-05-11T12:14:35.209121+0000","last_peered":"2024-05-11T12:14:35.209121+0000","last_clean":"2024-05-11T12:14:35.209121+0000","last_became_active":"2024-05-11T12:14:32.606952+0000","last_became_peered":"2024-05-11T12:14:32.606952+0000","last_unstale":"2024-05-11T12:14:35.209121+0000","last_undegraded":"2024-05-11T12:14:35.209121+0000","last_fullsized":"2024-05-11T12:14:35.209121+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:14:57.743590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615535+0000","last_change":"2024-05-11T12:14:33.619178+0000","last_active":"2024-05-11T12:14:34.615535+0000","last_peered":"2024-05-11T12:14:34.615535+0000","last_clean":"2024-05-11T12:14:34.615535+0000","last_became_active":"2024-05-11T12:14:33.618894+0000","last_became_peered":"2024-05-11T12:14:33.618894+0000","last_unstale":"2024-05-11T12:14:34.615535+0000","last_undegraded":"2024-05-11T12:14:34.615535+0000","last_fullsized":"2024-05-11T12:14:34.615535+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:10:50.170349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.283804+0000","last_change":"2024-05-11T12:14:32.608235+0000","last_active":"2024-05-11T12:14:35.283804+0000","last_peered":"2024-05-11T12:14:35.283804+0000","last_clean":"2024-05-11T12:14:35.283804+0000","last_became_active":"2024-05-11T12:14:32.607966+0000","last_became_peered":"2024-05-11T12:14:32.607966+0000","last_unstale":"2024-05-11T12:14:35.283804+0000","last_undegraded":"2024-05-11T12:14:35.283804+0000","last_fullsized":"2024-05-11T12:14:35.283804+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:30:51.604601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.621157+0000","last_change":"2024-05-11T12:14:33.626562+0000","last_active":"2024-05-11T12:14:34.621157+0000","last_peered":"2024-05-11T12:14:34.621157+0000","last_clean":"2024-05-11T12:14:34.621157+0000","last_became_active":"2024-05-11T12:14:33.625547+0000","last_became_peered":"2024-05-11T12:14:33.625547+0000","last_unstale":"2024-05-11T12:14:34.621157+0000","last_undegraded":"2024-05-11T12:14:34.621157+0000","last_fullsized":"2024-05-11T12:14:34.621157+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:11:02.751326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"97'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615678+0000","last_change":"2024-05-11T12:14:32.605888+0000","last_active":"2024-05-11T12:14:34.615678+0000","last_peered":"2024-05-11T12:14:34.615678+0000","last_clean":"2024-05-11T12:14:34.615678+0000","last_became_active":"2024-05-11T12:14:32.605588+0000","last_became_peered":"2024-05-11T12:14:32.605588+0000","last_unstale":"2024-05-11T12:14:34.615678+0000","last_undegraded":"2024-05-11T12:14:34.615678+0000","last_fullsized":"2024-05-11T12:14:34.615678+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:56:02.427960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.284207+0000","last_change":"2024-05-11T12:14:33.627479+0000","last_active":"2024-05-11T12:14:35.284207+0000","last_peered":"2024-05-11T12:14:35.284207+0000","last_clean":"2024-05-11T12:14:35.284207+0000","last_became_active":"2024-05-11T12:14:33.627179+0000","last_became_peered":"2024-05-11T12:14:33.627179+0000","last_unstale":"2024-05-11T12:14:35.284207+0000","last_undegraded":"2024-05-11T12:14:35.284207+0000","last_fullsized":"2024-05-11T12:14:35.284207+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:35:20.045762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.283940+0000","last_change":"2024-05-11T12:14:32.605594+0000","last_active":"2024-05-11T12:14:35.283940+0000","last_peered":"2024-05-11T12:14:35.283940+0000","last_clean":"2024-05-11T12:14:35.283940+0000","last_became_active":"2024-05-11T12:14:32.605097+0000","last_became_peered":"2024-05-11T12:14:32.605097+0000","last_unstale":"2024-05-11T12:14:35.283940+0000","last_undegraded":"2024-05-11T12:14:35.283940+0000","last_fullsized":"2024-05-11T12:14:35.283940+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:30:51.604601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.619647+0000","last_change":"2024-05-11T12:14:33.622085+0000","last_active":"2024-05-11T12:14:34.619647+0000","last_peered":"2024-05-11T12:14:34.619647+0000","last_clean":"2024-05-11T12:14:34.619647+0000","last_became_active":"2024-05-11T12:14:33.621754+0000","last_became_peered":"2024-05-11T12:14:33.621754+0000","last_unstale":"2024-05-11T12:14:34.619647+0000","last_undegraded":"2024-05-11T12:14:34.619647+0000","last_fullsized":"2024-05-11T12:14:34.619647+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:15:52.602707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.283625+0000","last_change":"2024-05-11T12:14:32.613449+0000","last_active":"2024-05-11T12:14:35.283625+0000","last_peered":"2024-05-11T12:14:35.283625+0000","last_clean":"2024-05-11T12:14:35.283625+0000","last_became_active":"2024-05-11T12:14:32.613108+0000","last_became_peered":"2024-05-11T12:14:32.613108+0000","last_unstale":"2024-05-11T12:14:35.283625+0000","last_undegraded":"2024-05-11T12:14:35.283625+0000","last_fullsized":"2024-05-11T12:14:35.283625+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:07:53.042878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.208801+0000","last_change":"2024-05-11T12:14:33.630741+0000","last_active":"2024-05-11T12:14:35.208801+0000","last_peered":"2024-05-11T12:14:35.208801+0000","last_clean":"2024-05-11T12:14:35.208801+0000","last_became_active":"2024-05-11T12:14:33.630423+0000","last_became_peered":"2024-05-11T12:14:33.630423+0000","last_unstale":"2024-05-11T12:14:35.208801+0000","last_undegraded":"2024-05-11T12:14:35.208801+0000","last_fullsized":"2024-05-11T12:14:35.208801+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:47:57.386158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.621831+0000","last_change":"2024-05-11T12:14:32.616942+0000","last_active":"2024-05-11T12:14:34.621831+0000","last_peered":"2024-05-11T12:14:34.621831+0000","last_clean":"2024-05-11T12:14:34.621831+0000","last_became_active":"2024-05-11T12:14:32.616577+0000","last_became_peered":"2024-05-11T12:14:32.616577+0000","last_unstale":"2024-05-11T12:14:34.621831+0000","last_undegraded":"2024-05-11T12:14:34.621831+0000","last_fullsized":"2024-05-11T12:14:34.621831+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:07:53.042878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.208768+0000","last_change":"2024-05-11T12:14:33.620388+0000","last_active":"2024-05-11T12:14:35.208768+0000","last_peered":"2024-05-11T12:14:35.208768+0000","last_clean":"2024-05-11T12:14:35.208768+0000","last_became_active":"2024-05-11T12:14:33.618621+0000","last_became_peered":"2024-05-11T12:14:33.618621+0000","last_unstale":"2024-05-11T12:14:35.208768+0000","last_undegraded":"2024-05-11T12:14:35.208768+0000","last_fullsized":"2024-05-11T12:14:35.208768+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:57:26.186975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209282+0000","last_change":"2024-05-11T12:14:32.615893+0000","last_active":"2024-05-11T12:14:35.209282+0000","last_peered":"2024-05-11T12:14:35.209282+0000","last_clean":"2024-05-11T12:14:35.209282+0000","last_became_active":"2024-05-11T12:14:32.615639+0000","last_became_peered":"2024-05-11T12:14:32.615639+0000","last_unstale":"2024-05-11T12:14:35.209282+0000","last_undegraded":"2024-05-11T12:14:35.209282+0000","last_fullsized":"2024-05-11T12:14:35.209282+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:51:28.062961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.185419+0000","last_change":"2024-05-11T12:14:33.628888+0000","last_active":"2024-05-11T12:14:35.185419+0000","last_peered":"2024-05-11T12:14:35.185419+0000","last_clean":"2024-05-11T12:14:35.185419+0000","last_became_active":"2024-05-11T12:14:33.625527+0000","last_became_peered":"2024-05-11T12:14:33.625527+0000","last_unstale":"2024-05-11T12:14:35.185419+0000","last_undegraded":"2024-05-11T12:14:35.185419+0000","last_fullsized":"2024-05-11T12:14:35.185419+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:27:47.349240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.283457+0000","last_change":"2024-05-11T12:14:32.607229+0000","last_active":"2024-05-11T12:14:35.283457+0000","last_peered":"2024-05-11T12:14:35.283457+0000","last_clean":"2024-05-11T12:14:35.283457+0000","last_became_active":"2024-05-11T12:14:32.606938+0000","last_became_peered":"2024-05-11T12:14:32.606938+0000","last_unstale":"2024-05-11T12:14:35.283457+0000","last_undegraded":"2024-05-11T12:14:35.283457+0000","last_fullsized":"2024-05-11T12:14:35.283457+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:58:41.059975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.620675+0000","last_change":"2024-05-11T12:14:33.614925+0000","last_active":"2024-05-11T12:14:34.620675+0000","last_peered":"2024-05-11T12:14:34.620675+0000","last_clean":"2024-05-11T12:14:34.620675+0000","last_became_active":"2024-05-11T12:14:33.614629+0000","last_became_peered":"2024-05-11T12:14:33.614629+0000","last_unstale":"2024-05-11T12:14:34.620675+0000","last_undegraded":"2024-05-11T12:14:34.620675+0000","last_fullsized":"2024-05-11T12:14:34.620675+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:11:02.751326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209088+0000","last_change":"2024-05-11T12:14:32.610511+0000","last_active":"2024-05-11T12:14:35.209088+0000","last_peered":"2024-05-11T12:14:35.209088+0000","last_clean":"2024-05-11T12:14:35.209088+0000","last_became_active":"2024-05-11T12:14:32.610243+0000","last_became_peered":"2024-05-11T12:14:32.610243+0000","last_unstale":"2024-05-11T12:14:35.209088+0000","last_undegraded":"2024-05-11T12:14:35.209088+0000","last_fullsized":"2024-05-11T12:14:35.209088+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:39:24.599004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.619309+0000","last_change":"2024-05-11T12:14:33.617894+0000","last_active":"2024-05-11T12:14:34.619309+0000","last_peered":"2024-05-11T12:14:34.619309+0000","last_clean":"2024-05-11T12:14:34.619309+0000","last_became_active":"2024-05-11T12:14:33.617537+0000","last_became_peered":"2024-05-11T12:14:33.617537+0000","last_unstale":"2024-05-11T12:14:34.619309+0000","last_undegraded":"2024-05-11T12:14:34.619309+0000","last_fullsized":"2024-05-11T12:14:34.619309+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:17:04.293672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209535+0000","last_change":"2024-05-11T12:14:32.612907+0000","last_active":"2024-05-11T12:14:35.209535+0000","last_peered":"2024-05-11T12:14:35.209535+0000","last_clean":"2024-05-11T12:14:35.209535+0000","last_became_active":"2024-05-11T12:14:32.612548+0000","last_became_peered":"2024-05-11T12:14:32.612548+0000","last_unstale":"2024-05-11T12:14:35.209535+0000","last_undegraded":"2024-05-11T12:14:35.209535+0000","last_fullsized":"2024-05-11T12:14:35.209535+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:49:45.457016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":248,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.184579+0000","last_change":"2024-05-11T12:10:35.066673+0000","last_active":"2024-05-11T12:14:35.184579+0000","last_peered":"2024-05-11T12:14:35.184579+0000","last_clean":"2024-05-11T12:14:35.184579+0000","last_became_active":"2024-05-11T12:10:35.065942+0000","last_became_peered":"2024-05-11T12:10:35.065942+0000","last_unstale":"2024-05-11T12:14:35.184579+0000","last_undegraded":"2024-05-11T12:14:35.184579+0000","last_fullsized":"2024-05-11T12:14:35.184579+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_clean_scrub_stamp":"2024-05-11T12:10:33.601136+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:12:02.133182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476788,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32932,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338908,"statfs":{"total":96636764160,"available":96603041792,"internally_reserved":0,"allocated":1794048,"data_stored":1132724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476789,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32364,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339476,"statfs":{"total":96636764160,"available":96603623424,"internally_reserved":0,"allocated":1212416,"data_stored":544412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476788,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32348,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339492,"statfs":{"total":96636764160,"available":96603639808,"internally_reserved":0,"allocated":1191936,"data_stored":542196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476790,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1200128,"data_stored":542288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509493,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1196032,"data_stored":542778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509494,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32348,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339492,"statfs":{"total":96636764160,"available":96603639808,"internally_reserved":0,"allocated":1196032,"data_stored":542230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509493,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32948,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338892,"statfs":{"total":96636764160,"available":96603025408,"internally_reserved":0,"allocated":1802240,"data_stored":1134266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542197,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1196032,"data_stored":542230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T12:14:43.089 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T12:14:43.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.299+0000 7fe37aa00640 1 Processor -- start 2024-05-11T12:14:43.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.299+0000 7fe37aa00640 1 -- start start 2024-05-11T12:14:43.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.300+0000 7fe37aa00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3740b3ea0 0x7fe374161490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:43.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.300+0000 7fe37aa00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe37415df30 con 0x7fe3741619d0 2024-05-11T12:14:43.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.300+0000 7fe37aa00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe37415e210 con 0x7fe374157750 2024-05-11T12:14:43.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.301+0000 7fe37aa00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe374161000 con 0x7fe3740b3ea0 2024-05-11T12:14:43.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.301+0000 7fe378a00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fe3741619d0 0x7fe37406d460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:44014/0 (socket says 172.21.15.167:44014) 2024-05-11T12:14:43.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.301+0000 7fe378a00640 1 -- 172.21.15.167:0/495465097 learned_addr learned my addr 172.21.15.167:0/495465097 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:14:43.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.301+0000 7fe36b400640 1 --2- 172.21.15.167:0/495465097 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3740b3ea0 0x7fe374161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:43.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.301+0000 7fe373400640 1 -- 172.21.15.167:0/495465097 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 327035273 0 0) 0x7fe37415df30 con 0x7fe3741619d0 2024-05-11T12:14:43.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.302+0000 7fe373400640 1 -- 172.21.15.167:0/495465097 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe358003520 con 0x7fe3741619d0 2024-05-11T12:14:43.302 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.302+0000 7fe373400640 1 -- 172.21.15.167:0/495465097 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1911736033 0 0) 0x7fe358003520 con 0x7fe3741619d0 2024-05-11T12:14:43.302 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.302+0000 7fe373400640 1 -- 172.21.15.167:0/495465097 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3740b3ea0 msgr2=0x7fe374161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:43.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.302+0000 7fe373400640 1 --2- 172.21.15.167:0/495465097 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3740b3ea0 0x7fe374161490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:43.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.302+0000 7fe373400640 1 -- 172.21.15.167:0/495465097 >> v1:172.21.15.204:6789/0 conn(0x7fe374157750 legacy=0x7fe37415d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:43.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.302+0000 7fe373400640 1 -- 172.21.15.167:0/495465097 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe37415e4f0 con 0x7fe3741619d0 2024-05-11T12:14:43.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.302+0000 7fe373400640 1 -- 172.21.15.167:0/495465097 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7fe36c0016f0 con 0x7fe3741619d0 2024-05-11T12:14:43.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.302+0000 7fe373400640 1 -- 172.21.15.167:0/495465097 >> v1:172.21.15.167:6789/0 conn(0x7fe3741619d0 legacy=0x7fe37406d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:43.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.302+0000 7fe373400640 1 --2- 172.21.15.167:0/495465097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe358003a30 0x7fe358003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:43.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.302+0000 7fe373400640 1 --2- 172.21.15.167:0/495465097 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3740b3ea0 0x7fe358004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:43.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.302+0000 7fe373400640 1 --2- 172.21.15.167:0/495465097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe358004d40 0x7fe3580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:43.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.302+0000 7fe373400640 1 -- 172.21.15.167:0/495465097 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe374161000 con 0x7fe358003a30 2024-05-11T12:14:43.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.302+0000 7fe373400640 1 -- 172.21.15.167:0/495465097 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe35800baf0 con 0x7fe358004d40 2024-05-11T12:14:43.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.302+0000 7fe373400640 1 -- 172.21.15.167:0/495465097 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe35800bcf0 con 0x7fe3740b3ea0 2024-05-11T12:14:43.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.302+0000 7fe373e00640 1 --2- 172.21.15.167:0/495465097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe358003a30 0x7fe358003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:43.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.303+0000 7fe378a00640 1 --2- 172.21.15.167:0/495465097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe358004d40 0x7fe3580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:43.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.303+0000 7fe36b400640 1 --2- 172.21.15.167:0/495465097 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3740b3ea0 0x7fe358004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:43.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.303+0000 7fe373e00640 1 -- 172.21.15.167:0/495465097 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3740b3ea0 msgr2=0x7fe358004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:43.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.303+0000 7fe373e00640 1 --2- 172.21.15.167:0/495465097 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3740b3ea0 0x7fe358004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:43.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.303+0000 7fe373e00640 1 -- 172.21.15.167:0/495465097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe358004d40 msgr2=0x7fe3580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:43.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.303+0000 7fe373e00640 1 --2- 172.21.15.167:0/495465097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe358004d40 0x7fe3580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:43.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.303+0000 7fe373e00640 1 -- 172.21.15.167:0/495465097 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe358003520 con 0x7fe358003a30 2024-05-11T12:14:43.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.303+0000 7fe378a00640 1 --2- 172.21.15.167:0/495465097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe358004d40 0x7fe3580051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:43.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.304+0000 7fe373e00640 1 --2- 172.21.15.167:0/495465097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe358003a30 0x7fe358003e50 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fe364001e90 tx=0x7fe358005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b7e84abf0771e0b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:43.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.304+0000 7fe373400640 1 -- 172.21.15.167:0/495465097 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe364001ab0 con 0x7fe358003a30 2024-05-11T12:14:43.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.304+0000 7fe373400640 1 -- 172.21.15.167:0/495465097 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe36402ada0 con 0x7fe358003a30 2024-05-11T12:14:43.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.304+0000 7fe373400640 1 -- 172.21.15.167:0/495465097 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe364003b70 con 0x7fe358003a30 2024-05-11T12:14:43.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.305+0000 7fe37aa00640 1 -- 172.21.15.167:0/495465097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe358003a30 msgr2=0x7fe358003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:43.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.305+0000 7fe37aa00640 1 --2- 172.21.15.167:0/495465097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe358003a30 0x7fe358003e50 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fe364001e90 tx=0x7fe358005830 comp rx=0 tx=0).stop 2024-05-11T12:14:43.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.305+0000 7fe378a00640 1 -- 172.21.15.167:0/495465097 reap_dead start 2024-05-11T12:14:43.306 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.305+0000 7fe37aa00640 1 -- 172.21.15.167:0/495465097 shutdown_connections 2024-05-11T12:14:43.306 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.305+0000 7fe37aa00640 1 -- 172.21.15.167:0/495465097 >> 172.21.15.167:0/495465097 conn(0x7fe3740cd8f0 msgr2=0x7fe3740cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:43.306 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.305+0000 7fe37aa00640 1 -- 172.21.15.167:0/495465097 shutdown_connections 2024-05-11T12:14:43.306 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.305+0000 7fe37aa00640 1 -- 172.21.15.167:0/495465097 wait complete. 2024-05-11T12:14:43.306 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.307+0000 7fe37aa00640 1 Processor -- start 2024-05-11T12:14:43.306 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.307+0000 7fe37aa00640 1 -- start start 2024-05-11T12:14:43.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.309+0000 7fe37aa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe358003a30 0x7fe374159fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:43.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.309+0000 7fe37aa00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3741f4020 0x7fe37413bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:43.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.309+0000 7fe37aa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe37413c2e0 0x7fe374145e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:43.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.309+0000 7fe37aa00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe37406dcf0 con 0x7fe358003a30 2024-05-11T12:14:43.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.309+0000 7fe37aa00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe37406db70 con 0x7fe37413c2e0 2024-05-11T12:14:43.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.309+0000 7fe37aa00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe374161490 con 0x7fe3741f4020 2024-05-11T12:14:43.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.309+0000 7fe373e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe358003a30 0x7fe374159fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:43.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.309+0000 7fe373e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe358003a30 0x7fe374159fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:47624/0 (socket says 172.21.15.167:47624) 2024-05-11T12:14:43.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.309+0000 7fe373e00640 1 -- 172.21.15.167:0/3648628770 learned_addr learned my addr 172.21.15.167:0/3648628770 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:43.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.309+0000 7fe378a00640 1 --2- 172.21.15.167:0/3648628770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe37413c2e0 0x7fe374145e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:43.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.309+0000 7fe36b400640 1 --2- 172.21.15.167:0/3648628770 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3741f4020 0x7fe37413bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:43.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.309+0000 7fe373e00640 1 -- 172.21.15.167:0/3648628770 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3741f4020 msgr2=0x7fe37413bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:43.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.309+0000 7fe373e00640 1 --2- 172.21.15.167:0/3648628770 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3741f4020 0x7fe37413bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:43.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.309+0000 7fe373e00640 1 -- 172.21.15.167:0/3648628770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe37413c2e0 msgr2=0x7fe374145e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:43.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.309+0000 7fe373e00640 1 --2- 172.21.15.167:0/3648628770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe37413c2e0 0x7fe374145e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:43.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.309+0000 7fe373e00640 1 -- 172.21.15.167:0/3648628770 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3741f45a0 con 0x7fe358003a30 2024-05-11T12:14:43.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.310+0000 7fe373e00640 1 --2- 172.21.15.167:0/3648628770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe358003a30 0x7fe374159fd0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fe364001e90 tx=0x7fe3640014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:43.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.310+0000 7fe370c00640 1 -- 172.21.15.167:0/3648628770 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe364042070 con 0x7fe358003a30 2024-05-11T12:14:43.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.310+0000 7fe370c00640 1 -- 172.21.15.167:0/3648628770 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3640050d0 con 0x7fe358003a30 2024-05-11T12:14:43.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.310+0000 7fe370c00640 1 -- 172.21.15.167:0/3648628770 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe364037040 con 0x7fe358003a30 2024-05-11T12:14:43.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.310+0000 7fe37aa00640 1 -- 172.21.15.167:0/3648628770 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3741f4880 con 0x7fe358003a30 2024-05-11T12:14:43.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.311+0000 7fe37aa00640 1 -- 172.21.15.167:0/3648628770 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3741f4e40 con 0x7fe358003a30 2024-05-11T12:14:43.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.312+0000 7fe37aa00640 1 -- 172.21.15.167:0/3648628770 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe374069590 con 0x7fe358003a30 2024-05-11T12:14:43.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.313+0000 7fe370c00640 1 -- 172.21.15.167:0/3648628770 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe3640054c0 con 0x7fe358003a30 2024-05-11T12:14:43.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.314+0000 7fe370c00640 1 --2- 172.21.15.167:0/3648628770 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe350076300 0x7fe3500787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:43.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.314+0000 7fe370c00640 1 -- 172.21.15.167:0/3648628770 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6101+0+0 (secure 0 0 0) 0x7fe3640b73a0 con 0x7fe358003a30 2024-05-11T12:14:43.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.314+0000 7fe36b400640 1 --2- 172.21.15.167:0/3648628770 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe350076300 0x7fe3500787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:43.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.315+0000 7fe36b400640 1 --2- 172.21.15.167:0/3648628770 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe350076300 0x7fe3500787c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe3740cf110 tx=0x7fe35c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:43.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.319+0000 7fe370c00640 1 -- 172.21.15.167:0/3648628770 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe364080ad0 con 0x7fe358003a30 2024-05-11T12:14:43.525 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.526+0000 7fe37aa00640 1 -- 172.21.15.167:0/3648628770 --> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe37406d670 con 0x7fe350076300 2024-05-11T12:14:43.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.533+0000 7fe370c00640 1 -- 172.21.15.167:0/3648628770 <== mgr.4106 v2:172.21.15.167:6832/578352559 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252014 (secure 0 0 0) 0x7fe37406d670 con 0x7fe350076300 2024-05-11T12:14:43.536 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:43.537 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2024-05-11T12:14:43.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.539+0000 7fe37aa00640 1 -- 172.21.15.167:0/3648628770 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe350076300 msgr2=0x7fe3500787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:43.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.539+0000 7fe37aa00640 1 --2- 172.21.15.167:0/3648628770 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe350076300 0x7fe3500787c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe3740cf110 tx=0x7fe35c008040 comp rx=0 tx=0).stop 2024-05-11T12:14:43.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.539+0000 7fe37aa00640 1 -- 172.21.15.167:0/3648628770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe358003a30 msgr2=0x7fe374159fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:43.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.539+0000 7fe37aa00640 1 --2- 172.21.15.167:0/3648628770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe358003a30 0x7fe374159fd0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fe364001e90 tx=0x7fe3640014a0 comp rx=0 tx=0).stop 2024-05-11T12:14:43.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.539+0000 7fe37aa00640 1 -- 172.21.15.167:0/3648628770 shutdown_connections 2024-05-11T12:14:43.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.539+0000 7fe37aa00640 1 --2- 172.21.15.167:0/3648628770 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe350076300 0x7fe3500787c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:43.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.539+0000 7fe37aa00640 1 --2- 172.21.15.167:0/3648628770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe37413c2e0 0x7fe374145e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:43.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.539+0000 7fe37aa00640 1 --2- 172.21.15.167:0/3648628770 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3741f4020 0x7fe37413bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:43.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.539+0000 7fe37aa00640 1 --2- 172.21.15.167:0/3648628770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe358003a30 0x7fe374159fd0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:43.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.539+0000 7fe37aa00640 1 -- 172.21.15.167:0/3648628770 >> 172.21.15.167:0/3648628770 conn(0x7fe3740cd8f0 msgr2=0x7fe3741600c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:43.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.540+0000 7fe37aa00640 1 -- 172.21.15.167:0/3648628770 shutdown_connections 2024-05-11T12:14:43.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.540+0000 7fe37aa00640 1 -- 172.21.15.167:0/3648628770 wait complete. 2024-05-11T12:14:43.557 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":216,"stamp":"2024-05-11T12:14:43.310073+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259996,"kb_used_data":10536,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714724,"statfs":{"total":773094113280,"available":772827877376,"internally_reserved":0,"allocated":10788864,"data_stored":5523124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.724901"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615834+0000","last_change":"2024-05-11T12:14:33.622872+0000","last_active":"2024-05-11T12:14:34.615834+0000","last_peered":"2024-05-11T12:14:34.615834+0000","last_clean":"2024-05-11T12:14:34.615834+0000","last_became_active":"2024-05-11T12:14:33.619136+0000","last_became_peered":"2024-05-11T12:14:33.619136+0000","last_unstale":"2024-05-11T12:14:34.615834+0000","last_undegraded":"2024-05-11T12:14:34.615834+0000","last_fullsized":"2024-05-11T12:14:34.615834+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:18:44.962195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.284196+0000","last_change":"2024-05-11T12:14:33.624137+0000","last_active":"2024-05-11T12:14:35.284196+0000","last_peered":"2024-05-11T12:14:35.284196+0000","last_clean":"2024-05-11T12:14:35.284196+0000","last_became_active":"2024-05-11T12:14:33.623671+0000","last_became_peered":"2024-05-11T12:14:33.623671+0000","last_unstale":"2024-05-11T12:14:35.284196+0000","last_undegraded":"2024-05-11T12:14:35.284196+0000","last_fullsized":"2024-05-11T12:14:35.284196+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:27:47.349240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615616+0000","last_change":"2024-05-11T12:14:33.626436+0000","last_active":"2024-05-11T12:14:34.615616+0000","last_peered":"2024-05-11T12:14:34.615616+0000","last_clean":"2024-05-11T12:14:34.615616+0000","last_became_active":"2024-05-11T12:14:33.626091+0000","last_became_peered":"2024-05-11T12:14:33.626091+0000","last_unstale":"2024-05-11T12:14:34.615616+0000","last_undegraded":"2024-05-11T12:14:34.615616+0000","last_fullsized":"2024-05-11T12:14:34.615616+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:17:05.023315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.284831+0000","last_change":"2024-05-11T12:14:33.624886+0000","last_active":"2024-05-11T12:14:35.284831+0000","last_peered":"2024-05-11T12:14:35.284831+0000","last_clean":"2024-05-11T12:14:35.284831+0000","last_became_active":"2024-05-11T12:14:33.618826+0000","last_became_peered":"2024-05-11T12:14:33.618826+0000","last_unstale":"2024-05-11T12:14:35.284831+0000","last_undegraded":"2024-05-11T12:14:35.284831+0000","last_fullsized":"2024-05-11T12:14:35.284831+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:56:09.727119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.283813+0000","last_change":"2024-05-11T12:14:33.618388+0000","last_active":"2024-05-11T12:14:35.283813+0000","last_peered":"2024-05-11T12:14:35.283813+0000","last_clean":"2024-05-11T12:14:35.283813+0000","last_became_active":"2024-05-11T12:14:33.618090+0000","last_became_peered":"2024-05-11T12:14:33.618090+0000","last_unstale":"2024-05-11T12:14:35.283813+0000","last_undegraded":"2024-05-11T12:14:35.283813+0000","last_fullsized":"2024-05-11T12:14:35.283813+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:00:00.861509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.283162+0000","last_change":"2024-05-11T12:14:33.634835+0000","last_active":"2024-05-11T12:14:35.283162+0000","last_peered":"2024-05-11T12:14:35.283162+0000","last_clean":"2024-05-11T12:14:35.283162+0000","last_became_active":"2024-05-11T12:14:33.634564+0000","last_became_peered":"2024-05-11T12:14:33.634564+0000","last_unstale":"2024-05-11T12:14:35.283162+0000","last_undegraded":"2024-05-11T12:14:35.283162+0000","last_fullsized":"2024-05-11T12:14:35.283162+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:11:02.751326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.282828+0000","last_change":"2024-05-11T12:14:33.634371+0000","last_active":"2024-05-11T12:14:35.282828+0000","last_peered":"2024-05-11T12:14:35.282828+0000","last_clean":"2024-05-11T12:14:35.282828+0000","last_became_active":"2024-05-11T12:14:33.634080+0000","last_became_peered":"2024-05-11T12:14:33.634080+0000","last_unstale":"2024-05-11T12:14:35.282828+0000","last_undegraded":"2024-05-11T12:14:35.282828+0000","last_fullsized":"2024-05-11T12:14:35.282828+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:15:22.180722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.208341+0000","last_change":"2024-05-11T12:14:33.621312+0000","last_active":"2024-05-11T12:14:35.208341+0000","last_peered":"2024-05-11T12:14:35.208341+0000","last_clean":"2024-05-11T12:14:35.208341+0000","last_became_active":"2024-05-11T12:14:33.621021+0000","last_became_peered":"2024-05-11T12:14:33.621021+0000","last_unstale":"2024-05-11T12:14:35.208341+0000","last_undegraded":"2024-05-11T12:14:35.208341+0000","last_fullsized":"2024-05-11T12:14:35.208341+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:54:45.084719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.284171+0000","last_change":"2024-05-11T12:14:33.622698+0000","last_active":"2024-05-11T12:14:35.284171+0000","last_peered":"2024-05-11T12:14:35.284171+0000","last_clean":"2024-05-11T12:14:35.284171+0000","last_became_active":"2024-05-11T12:14:33.622347+0000","last_became_peered":"2024-05-11T12:14:33.622347+0000","last_unstale":"2024-05-11T12:14:35.284171+0000","last_undegraded":"2024-05-11T12:14:35.284171+0000","last_fullsized":"2024-05-11T12:14:35.284171+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:16:12.500928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.283614+0000","last_change":"2024-05-11T12:14:33.623008+0000","last_active":"2024-05-11T12:14:35.283614+0000","last_peered":"2024-05-11T12:14:35.283614+0000","last_clean":"2024-05-11T12:14:35.283614+0000","last_became_active":"2024-05-11T12:14:33.622017+0000","last_became_peered":"2024-05-11T12:14:33.622017+0000","last_unstale":"2024-05-11T12:14:35.283614+0000","last_undegraded":"2024-05-11T12:14:35.283614+0000","last_fullsized":"2024-05-11T12:14:35.283614+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:11:02.751326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.619851+0000","last_change":"2024-05-11T12:14:33.626097+0000","last_active":"2024-05-11T12:14:34.619851+0000","last_peered":"2024-05-11T12:14:34.619851+0000","last_clean":"2024-05-11T12:14:34.619851+0000","last_became_active":"2024-05-11T12:14:33.624668+0000","last_became_peered":"2024-05-11T12:14:33.624668+0000","last_unstale":"2024-05-11T12:14:34.619851+0000","last_undegraded":"2024-05-11T12:14:34.619851+0000","last_fullsized":"2024-05-11T12:14:34.619851+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:13:15.093587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.621111+0000","last_change":"2024-05-11T12:14:33.629494+0000","last_active":"2024-05-11T12:14:34.621111+0000","last_peered":"2024-05-11T12:14:34.621111+0000","last_clean":"2024-05-11T12:14:34.621111+0000","last_became_active":"2024-05-11T12:14:33.629203+0000","last_became_peered":"2024-05-11T12:14:33.629203+0000","last_unstale":"2024-05-11T12:14:34.621111+0000","last_undegraded":"2024-05-11T12:14:34.621111+0000","last_fullsized":"2024-05-11T12:14:34.621111+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:15:22.180722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.283623+0000","last_change":"2024-05-11T12:14:33.607961+0000","last_active":"2024-05-11T12:14:35.283623+0000","last_peered":"2024-05-11T12:14:35.283623+0000","last_clean":"2024-05-11T12:14:35.283623+0000","last_became_active":"2024-05-11T12:14:33.607599+0000","last_became_peered":"2024-05-11T12:14:33.607599+0000","last_unstale":"2024-05-11T12:14:35.283623+0000","last_undegraded":"2024-05-11T12:14:35.283623+0000","last_fullsized":"2024-05-11T12:14:35.283623+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:22:15.346218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.184802+0000","last_change":"2024-05-11T12:14:33.626456+0000","last_active":"2024-05-11T12:14:35.184802+0000","last_peered":"2024-05-11T12:14:35.184802+0000","last_clean":"2024-05-11T12:14:35.184802+0000","last_became_active":"2024-05-11T12:14:33.625698+0000","last_became_peered":"2024-05-11T12:14:33.625698+0000","last_unstale":"2024-05-11T12:14:35.184802+0000","last_undegraded":"2024-05-11T12:14:35.184802+0000","last_fullsized":"2024-05-11T12:14:35.184802+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:35:20.045762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.184995+0000","last_change":"2024-05-11T12:14:33.632118+0000","last_active":"2024-05-11T12:14:35.184995+0000","last_peered":"2024-05-11T12:14:35.184995+0000","last_clean":"2024-05-11T12:14:35.184995+0000","last_became_active":"2024-05-11T12:14:33.631202+0000","last_became_peered":"2024-05-11T12:14:33.631202+0000","last_unstale":"2024-05-11T12:14:35.184995+0000","last_undegraded":"2024-05-11T12:14:35.184995+0000","last_fullsized":"2024-05-11T12:14:35.184995+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:56:09.727119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.184745+0000","last_change":"2024-05-11T12:14:33.626275+0000","last_active":"2024-05-11T12:14:35.184745+0000","last_peered":"2024-05-11T12:14:35.184745+0000","last_clean":"2024-05-11T12:14:35.184745+0000","last_became_active":"2024-05-11T12:14:33.625954+0000","last_became_peered":"2024-05-11T12:14:33.625954+0000","last_unstale":"2024-05-11T12:14:35.184745+0000","last_undegraded":"2024-05-11T12:14:35.184745+0000","last_fullsized":"2024-05-11T12:14:35.184745+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:22:15.346218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615878+0000","last_change":"2024-05-11T12:14:33.622549+0000","last_active":"2024-05-11T12:14:34.615878+0000","last_peered":"2024-05-11T12:14:34.615878+0000","last_clean":"2024-05-11T12:14:34.615878+0000","last_became_active":"2024-05-11T12:14:33.619603+0000","last_became_peered":"2024-05-11T12:14:33.619603+0000","last_unstale":"2024-05-11T12:14:34.615878+0000","last_undegraded":"2024-05-11T12:14:34.615878+0000","last_fullsized":"2024-05-11T12:14:34.615878+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:51:30.951966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.284305+0000","last_change":"2024-05-11T12:14:33.636255+0000","last_active":"2024-05-11T12:14:35.284305+0000","last_peered":"2024-05-11T12:14:35.284305+0000","last_clean":"2024-05-11T12:14:35.284305+0000","last_became_active":"2024-05-11T12:14:33.635916+0000","last_became_peered":"2024-05-11T12:14:33.635916+0000","last_unstale":"2024-05-11T12:14:35.284305+0000","last_undegraded":"2024-05-11T12:14:35.284305+0000","last_fullsized":"2024-05-11T12:14:35.284305+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:52:46.883659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615768+0000","last_change":"2024-05-11T12:14:33.635269+0000","last_active":"2024-05-11T12:14:34.615768+0000","last_peered":"2024-05-11T12:14:34.615768+0000","last_clean":"2024-05-11T12:14:34.615768+0000","last_became_active":"2024-05-11T12:14:33.634795+0000","last_became_peered":"2024-05-11T12:14:33.634795+0000","last_unstale":"2024-05-11T12:14:34.615768+0000","last_undegraded":"2024-05-11T12:14:34.615768+0000","last_fullsized":"2024-05-11T12:14:34.615768+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:13:16.630356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.621591+0000","last_change":"2024-05-11T12:14:33.615290+0000","last_active":"2024-05-11T12:14:34.621591+0000","last_peered":"2024-05-11T12:14:34.621591+0000","last_clean":"2024-05-11T12:14:34.621591+0000","last_became_active":"2024-05-11T12:14:33.614910+0000","last_became_peered":"2024-05-11T12:14:33.614910+0000","last_unstale":"2024-05-11T12:14:34.621591+0000","last_undegraded":"2024-05-11T12:14:34.621591+0000","last_fullsized":"2024-05-11T12:14:34.621591+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:22:15.346218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.208740+0000","last_change":"2024-05-11T12:14:33.621162+0000","last_active":"2024-05-11T12:14:35.208740+0000","last_peered":"2024-05-11T12:14:35.208740+0000","last_clean":"2024-05-11T12:14:35.208740+0000","last_became_active":"2024-05-11T12:14:33.620843+0000","last_became_peered":"2024-05-11T12:14:33.620843+0000","last_unstale":"2024-05-11T12:14:35.208740+0000","last_undegraded":"2024-05-11T12:14:35.208740+0000","last_fullsized":"2024-05-11T12:14:35.208740+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:29:39.246958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.284670+0000","last_change":"2024-05-11T12:14:33.623513+0000","last_active":"2024-05-11T12:14:35.284670+0000","last_peered":"2024-05-11T12:14:35.284670+0000","last_clean":"2024-05-11T12:14:35.284670+0000","last_became_active":"2024-05-11T12:14:33.619839+0000","last_became_peered":"2024-05-11T12:14:33.619839+0000","last_unstale":"2024-05-11T12:14:35.284670+0000","last_undegraded":"2024-05-11T12:14:35.284670+0000","last_fullsized":"2024-05-11T12:14:35.284670+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:35:20.045762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615236+0000","last_change":"2024-05-11T12:14:33.619917+0000","last_active":"2024-05-11T12:14:34.615236+0000","last_peered":"2024-05-11T12:14:34.615236+0000","last_clean":"2024-05-11T12:14:34.615236+0000","last_became_active":"2024-05-11T12:14:33.619614+0000","last_became_peered":"2024-05-11T12:14:33.619614+0000","last_unstale":"2024-05-11T12:14:34.615236+0000","last_undegraded":"2024-05-11T12:14:34.615236+0000","last_fullsized":"2024-05-11T12:14:34.615236+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:56:17.539601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.283165+0000","last_change":"2024-05-11T12:14:33.614192+0000","last_active":"2024-05-11T12:14:35.283165+0000","last_peered":"2024-05-11T12:14:35.283165+0000","last_clean":"2024-05-11T12:14:35.283165+0000","last_became_active":"2024-05-11T12:14:33.613880+0000","last_became_peered":"2024-05-11T12:14:33.613880+0000","last_unstale":"2024-05-11T12:14:35.283165+0000","last_undegraded":"2024-05-11T12:14:35.283165+0000","last_fullsized":"2024-05-11T12:14:35.283165+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:13:15.093587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.620105+0000","last_change":"2024-05-11T12:14:33.620900+0000","last_active":"2024-05-11T12:14:34.620105+0000","last_peered":"2024-05-11T12:14:34.620105+0000","last_clean":"2024-05-11T12:14:34.620105+0000","last_became_active":"2024-05-11T12:14:33.620614+0000","last_became_peered":"2024-05-11T12:14:33.620614+0000","last_unstale":"2024-05-11T12:14:34.620105+0000","last_undegraded":"2024-05-11T12:14:34.620105+0000","last_fullsized":"2024-05-11T12:14:34.620105+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:17:04.293672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.620227+0000","last_change":"2024-05-11T12:14:33.628258+0000","last_active":"2024-05-11T12:14:34.620227+0000","last_peered":"2024-05-11T12:14:34.620227+0000","last_clean":"2024-05-11T12:14:34.620227+0000","last_became_active":"2024-05-11T12:14:33.627691+0000","last_became_peered":"2024-05-11T12:14:33.627691+0000","last_unstale":"2024-05-11T12:14:34.620227+0000","last_undegraded":"2024-05-11T12:14:34.620227+0000","last_fullsized":"2024-05-11T12:14:34.620227+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:15:52.602707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.621886+0000","last_change":"2024-05-11T12:14:33.614817+0000","last_active":"2024-05-11T12:14:34.621886+0000","last_peered":"2024-05-11T12:14:34.621886+0000","last_clean":"2024-05-11T12:14:34.621886+0000","last_became_active":"2024-05-11T12:14:33.614446+0000","last_became_peered":"2024-05-11T12:14:33.614446+0000","last_unstale":"2024-05-11T12:14:34.621886+0000","last_undegraded":"2024-05-11T12:14:34.621886+0000","last_fullsized":"2024-05-11T12:14:34.621886+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:00:00.861509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.283265+0000","last_change":"2024-05-11T12:14:33.630308+0000","last_active":"2024-05-11T12:14:35.283265+0000","last_peered":"2024-05-11T12:14:35.283265+0000","last_clean":"2024-05-11T12:14:35.283265+0000","last_became_active":"2024-05-11T12:14:33.629962+0000","last_became_peered":"2024-05-11T12:14:33.629962+0000","last_unstale":"2024-05-11T12:14:35.283265+0000","last_undegraded":"2024-05-11T12:14:35.283265+0000","last_fullsized":"2024-05-11T12:14:35.283265+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:35:20.045762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209481+0000","last_change":"2024-05-11T12:14:33.623708+0000","last_active":"2024-05-11T12:14:35.209481+0000","last_peered":"2024-05-11T12:14:35.209481+0000","last_clean":"2024-05-11T12:14:35.209481+0000","last_became_active":"2024-05-11T12:14:33.623429+0000","last_became_peered":"2024-05-11T12:14:33.623429+0000","last_unstale":"2024-05-11T12:14:35.209481+0000","last_undegraded":"2024-05-11T12:14:35.209481+0000","last_fullsized":"2024-05-11T12:14:35.209481+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:30:11.411742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.621113+0000","last_change":"2024-05-11T12:14:33.625235+0000","last_active":"2024-05-11T12:14:34.621113+0000","last_peered":"2024-05-11T12:14:34.621113+0000","last_clean":"2024-05-11T12:14:34.621113+0000","last_became_active":"2024-05-11T12:14:33.624914+0000","last_became_peered":"2024-05-11T12:14:33.624914+0000","last_unstale":"2024-05-11T12:14:34.621113+0000","last_undegraded":"2024-05-11T12:14:34.621113+0000","last_fullsized":"2024-05-11T12:14:34.621113+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:13:15.093587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.282836+0000","last_change":"2024-05-11T12:14:33.625241+0000","last_active":"2024-05-11T12:14:35.282836+0000","last_peered":"2024-05-11T12:14:35.282836+0000","last_clean":"2024-05-11T12:14:35.282836+0000","last_became_active":"2024-05-11T12:14:33.623619+0000","last_became_peered":"2024-05-11T12:14:33.623619+0000","last_unstale":"2024-05-11T12:14:35.282836+0000","last_undegraded":"2024-05-11T12:14:35.282836+0000","last_fullsized":"2024-05-11T12:14:35.282836+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:00:00.861509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.208839+0000","last_change":"2024-05-11T12:14:33.621606+0000","last_active":"2024-05-11T12:14:35.208839+0000","last_peered":"2024-05-11T12:14:35.208839+0000","last_clean":"2024-05-11T12:14:35.208839+0000","last_became_active":"2024-05-11T12:14:33.621288+0000","last_became_peered":"2024-05-11T12:14:33.621288+0000","last_unstale":"2024-05-11T12:14:35.208839+0000","last_undegraded":"2024-05-11T12:14:35.208839+0000","last_fullsized":"2024-05-11T12:14:35.208839+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:37:19.519120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615469+0000","last_change":"2024-05-11T12:14:33.633794+0000","last_active":"2024-05-11T12:14:34.615469+0000","last_peered":"2024-05-11T12:14:34.615469+0000","last_clean":"2024-05-11T12:14:34.615469+0000","last_became_active":"2024-05-11T12:14:33.633451+0000","last_became_peered":"2024-05-11T12:14:33.633451+0000","last_unstale":"2024-05-11T12:14:34.615469+0000","last_undegraded":"2024-05-11T12:14:34.615469+0000","last_fullsized":"2024-05-11T12:14:34.615469+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:31:39.465305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"96'4","reported_seq":18,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.285496+0000","last_change":"2024-05-11T12:14:32.610045+0000","last_active":"2024-05-11T12:14:35.285496+0000","last_peered":"2024-05-11T12:14:35.285496+0000","last_clean":"2024-05-11T12:14:35.285496+0000","last_became_active":"2024-05-11T12:14:32.609746+0000","last_became_peered":"2024-05-11T12:14:32.609746+0000","last_unstale":"2024-05-11T12:14:35.285496+0000","last_undegraded":"2024-05-11T12:14:35.285496+0000","last_fullsized":"2024-05-11T12:14:35.285496+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:44:56.684605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.208579+0000","last_change":"2024-05-11T12:14:33.630664+0000","last_active":"2024-05-11T12:14:35.208579+0000","last_peered":"2024-05-11T12:14:35.208579+0000","last_clean":"2024-05-11T12:14:35.208579+0000","last_became_active":"2024-05-11T12:14:33.630171+0000","last_became_peered":"2024-05-11T12:14:33.630171+0000","last_unstale":"2024-05-11T12:14:35.208579+0000","last_undegraded":"2024-05-11T12:14:35.208579+0000","last_fullsized":"2024-05-11T12:14:35.208579+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:00:10.906551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209543+0000","last_change":"2024-05-11T12:14:32.619446+0000","last_active":"2024-05-11T12:14:35.209543+0000","last_peered":"2024-05-11T12:14:35.209543+0000","last_clean":"2024-05-11T12:14:35.209543+0000","last_became_active":"2024-05-11T12:14:32.619219+0000","last_became_peered":"2024-05-11T12:14:32.619219+0000","last_unstale":"2024-05-11T12:14:35.209543+0000","last_undegraded":"2024-05-11T12:14:35.209543+0000","last_fullsized":"2024-05-11T12:14:35.209543+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:34:15.881914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.621298+0000","last_change":"2024-05-11T12:14:33.623282+0000","last_active":"2024-05-11T12:14:34.621298+0000","last_peered":"2024-05-11T12:14:34.621298+0000","last_clean":"2024-05-11T12:14:34.621298+0000","last_became_active":"2024-05-11T12:14:33.622997+0000","last_became_peered":"2024-05-11T12:14:33.622997+0000","last_unstale":"2024-05-11T12:14:34.621298+0000","last_undegraded":"2024-05-11T12:14:34.621298+0000","last_fullsized":"2024-05-11T12:14:34.621298+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:00:00.861509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.619909+0000","last_change":"2024-05-11T12:14:32.618332+0000","last_active":"2024-05-11T12:14:34.619909+0000","last_peered":"2024-05-11T12:14:34.619909+0000","last_clean":"2024-05-11T12:14:34.619909+0000","last_became_active":"2024-05-11T12:14:32.614777+0000","last_became_peered":"2024-05-11T12:14:32.614777+0000","last_unstale":"2024-05-11T12:14:34.619909+0000","last_undegraded":"2024-05-11T12:14:34.619909+0000","last_fullsized":"2024-05-11T12:14:34.619909+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:55:53.087982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.620038+0000","last_change":"2024-05-11T12:14:33.617665+0000","last_active":"2024-05-11T12:14:34.620038+0000","last_peered":"2024-05-11T12:14:34.620038+0000","last_clean":"2024-05-11T12:14:34.620038+0000","last_became_active":"2024-05-11T12:14:33.617251+0000","last_became_peered":"2024-05-11T12:14:33.617251+0000","last_unstale":"2024-05-11T12:14:34.620038+0000","last_undegraded":"2024-05-11T12:14:34.620038+0000","last_fullsized":"2024-05-11T12:14:34.620038+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:15:22.180722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.283263+0000","last_change":"2024-05-11T12:14:32.610836+0000","last_active":"2024-05-11T12:14:35.283263+0000","last_peered":"2024-05-11T12:14:35.283263+0000","last_clean":"2024-05-11T12:14:35.283263+0000","last_became_active":"2024-05-11T12:14:32.610542+0000","last_became_peered":"2024-05-11T12:14:32.610542+0000","last_unstale":"2024-05-11T12:14:35.283263+0000","last_undegraded":"2024-05-11T12:14:35.283263+0000","last_fullsized":"2024-05-11T12:14:35.283263+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:55:53.087982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.284849+0000","last_change":"2024-05-11T12:14:33.616596+0000","last_active":"2024-05-11T12:14:35.284849+0000","last_peered":"2024-05-11T12:14:35.284849+0000","last_clean":"2024-05-11T12:14:35.284849+0000","last_became_active":"2024-05-11T12:14:33.616294+0000","last_became_peered":"2024-05-11T12:14:33.616294+0000","last_unstale":"2024-05-11T12:14:35.284849+0000","last_undegraded":"2024-05-11T12:14:35.284849+0000","last_fullsized":"2024-05-11T12:14:35.284849+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:27:47.349240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.184868+0000","last_change":"2024-05-11T12:14:32.619606+0000","last_active":"2024-05-11T12:14:35.184868+0000","last_peered":"2024-05-11T12:14:35.184868+0000","last_clean":"2024-05-11T12:14:35.184868+0000","last_became_active":"2024-05-11T12:14:32.619133+0000","last_became_peered":"2024-05-11T12:14:32.619133+0000","last_unstale":"2024-05-11T12:14:35.184868+0000","last_undegraded":"2024-05-11T12:14:35.184868+0000","last_fullsized":"2024-05-11T12:14:35.184868+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:17:03.277324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.616178+0000","last_change":"2024-05-11T12:14:33.634226+0000","last_active":"2024-05-11T12:14:34.616178+0000","last_peered":"2024-05-11T12:14:34.616178+0000","last_clean":"2024-05-11T12:14:34.616178+0000","last_became_active":"2024-05-11T12:14:33.633932+0000","last_became_peered":"2024-05-11T12:14:33.633932+0000","last_unstale":"2024-05-11T12:14:34.616178+0000","last_undegraded":"2024-05-11T12:14:34.616178+0000","last_fullsized":"2024-05-11T12:14:34.616178+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:01:12.394223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.622171+0000","last_change":"2024-05-11T12:14:32.614159+0000","last_active":"2024-05-11T12:14:34.622171+0000","last_peered":"2024-05-11T12:14:34.622171+0000","last_clean":"2024-05-11T12:14:34.622171+0000","last_became_active":"2024-05-11T12:14:32.613421+0000","last_became_peered":"2024-05-11T12:14:32.613421+0000","last_unstale":"2024-05-11T12:14:34.622171+0000","last_undegraded":"2024-05-11T12:14:34.622171+0000","last_fullsized":"2024-05-11T12:14:34.622171+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:55:53.087982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.284582+0000","last_change":"2024-05-11T12:14:33.632315+0000","last_active":"2024-05-11T12:14:35.284582+0000","last_peered":"2024-05-11T12:14:35.284582+0000","last_clean":"2024-05-11T12:14:35.284582+0000","last_became_active":"2024-05-11T12:14:33.626868+0000","last_became_peered":"2024-05-11T12:14:33.626868+0000","last_unstale":"2024-05-11T12:14:35.284582+0000","last_undegraded":"2024-05-11T12:14:35.284582+0000","last_fullsized":"2024-05-11T12:14:35.284582+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:26:18.712084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.621042+0000","last_change":"2024-05-11T12:14:32.614638+0000","last_active":"2024-05-11T12:14:34.621042+0000","last_peered":"2024-05-11T12:14:34.621042+0000","last_clean":"2024-05-11T12:14:34.621042+0000","last_became_active":"2024-05-11T12:14:32.614242+0000","last_became_peered":"2024-05-11T12:14:32.614242+0000","last_unstale":"2024-05-11T12:14:34.621042+0000","last_undegraded":"2024-05-11T12:14:34.621042+0000","last_fullsized":"2024-05-11T12:14:34.621042+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:58:41.059975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.285649+0000","last_change":"2024-05-11T12:14:33.615167+0000","last_active":"2024-05-11T12:14:35.285649+0000","last_peered":"2024-05-11T12:14:35.285649+0000","last_clean":"2024-05-11T12:14:35.285649+0000","last_became_active":"2024-05-11T12:14:33.614898+0000","last_became_peered":"2024-05-11T12:14:33.614898+0000","last_unstale":"2024-05-11T12:14:35.285649+0000","last_undegraded":"2024-05-11T12:14:35.285649+0000","last_fullsized":"2024-05-11T12:14:35.285649+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:52:46.883659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.283863+0000","last_change":"2024-05-11T12:14:32.605627+0000","last_active":"2024-05-11T12:14:35.283863+0000","last_peered":"2024-05-11T12:14:35.283863+0000","last_clean":"2024-05-11T12:14:35.283863+0000","last_became_active":"2024-05-11T12:14:32.604729+0000","last_became_peered":"2024-05-11T12:14:32.604729+0000","last_unstale":"2024-05-11T12:14:35.283863+0000","last_undegraded":"2024-05-11T12:14:35.283863+0000","last_fullsized":"2024-05-11T12:14:35.283863+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:00:56.111300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615229+0000","last_change":"2024-05-11T12:14:33.635074+0000","last_active":"2024-05-11T12:14:34.615229+0000","last_peered":"2024-05-11T12:14:34.615229+0000","last_clean":"2024-05-11T12:14:34.615229+0000","last_became_active":"2024-05-11T12:14:33.634606+0000","last_became_peered":"2024-05-11T12:14:33.634606+0000","last_unstale":"2024-05-11T12:14:34.615229+0000","last_undegraded":"2024-05-11T12:14:34.615229+0000","last_fullsized":"2024-05-11T12:14:34.615229+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:03:07.386179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.620538+0000","last_change":"2024-05-11T12:14:32.615191+0000","last_active":"2024-05-11T12:14:34.620538+0000","last_peered":"2024-05-11T12:14:34.620538+0000","last_clean":"2024-05-11T12:14:34.620538+0000","last_became_active":"2024-05-11T12:14:32.614809+0000","last_became_peered":"2024-05-11T12:14:32.614809+0000","last_unstale":"2024-05-11T12:14:34.620538+0000","last_undegraded":"2024-05-11T12:14:34.620538+0000","last_fullsized":"2024-05-11T12:14:34.620538+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:30:51.604601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.282918+0000","last_change":"2024-05-11T12:14:33.624689+0000","last_active":"2024-05-11T12:14:35.282918+0000","last_peered":"2024-05-11T12:14:35.282918+0000","last_clean":"2024-05-11T12:14:35.282918+0000","last_became_active":"2024-05-11T12:14:33.624275+0000","last_became_peered":"2024-05-11T12:14:33.624275+0000","last_unstale":"2024-05-11T12:14:35.282918+0000","last_undegraded":"2024-05-11T12:14:35.282918+0000","last_fullsized":"2024-05-11T12:14:35.282918+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:15:52.602707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615602+0000","last_change":"2024-05-11T12:14:32.609650+0000","last_active":"2024-05-11T12:14:34.615602+0000","last_peered":"2024-05-11T12:14:34.615602+0000","last_clean":"2024-05-11T12:14:34.615602+0000","last_became_active":"2024-05-11T12:14:32.609340+0000","last_became_peered":"2024-05-11T12:14:32.609340+0000","last_unstale":"2024-05-11T12:14:34.615602+0000","last_undegraded":"2024-05-11T12:14:34.615602+0000","last_fullsized":"2024-05-11T12:14:34.615602+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:40:56.223685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209330+0000","last_change":"2024-05-11T12:14:33.631013+0000","last_active":"2024-05-11T12:14:35.209330+0000","last_peered":"2024-05-11T12:14:35.209330+0000","last_clean":"2024-05-11T12:14:35.209330+0000","last_became_active":"2024-05-11T12:14:33.630698+0000","last_became_peered":"2024-05-11T12:14:33.630698+0000","last_unstale":"2024-05-11T12:14:35.209330+0000","last_undegraded":"2024-05-11T12:14:35.209330+0000","last_fullsized":"2024-05-11T12:14:35.209330+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:32:48.749691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"96'3","reported_seq":17,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.185206+0000","last_change":"2024-05-11T12:14:32.618526+0000","last_active":"2024-05-11T12:14:35.185206+0000","last_peered":"2024-05-11T12:14:35.185206+0000","last_clean":"2024-05-11T12:14:35.185206+0000","last_became_active":"2024-05-11T12:14:32.618198+0000","last_became_peered":"2024-05-11T12:14:32.618198+0000","last_unstale":"2024-05-11T12:14:35.185206+0000","last_undegraded":"2024-05-11T12:14:35.185206+0000","last_fullsized":"2024-05-11T12:14:35.185206+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:55:59.198545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.283938+0000","last_change":"2024-05-11T12:14:33.623135+0000","last_active":"2024-05-11T12:14:35.283938+0000","last_peered":"2024-05-11T12:14:35.283938+0000","last_clean":"2024-05-11T12:14:35.283938+0000","last_became_active":"2024-05-11T12:14:33.622788+0000","last_became_peered":"2024-05-11T12:14:33.622788+0000","last_unstale":"2024-05-11T12:14:35.283938+0000","last_undegraded":"2024-05-11T12:14:35.283938+0000","last_fullsized":"2024-05-11T12:14:35.283938+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:22:15.346218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"96'6","reported_seq":20,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.283185+0000","last_change":"2024-05-11T12:14:32.607489+0000","last_active":"2024-05-11T12:14:35.283185+0000","last_peered":"2024-05-11T12:14:35.283185+0000","last_clean":"2024-05-11T12:14:35.283185+0000","last_became_active":"2024-05-11T12:14:32.606078+0000","last_became_peered":"2024-05-11T12:14:32.606078+0000","last_unstale":"2024-05-11T12:14:35.283185+0000","last_undegraded":"2024-05-11T12:14:35.283185+0000","last_fullsized":"2024-05-11T12:14:35.283185+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:55:53.087982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.619221+0000","last_change":"2024-05-11T12:14:33.616862+0000","last_active":"2024-05-11T12:14:34.619221+0000","last_peered":"2024-05-11T12:14:34.619221+0000","last_clean":"2024-05-11T12:14:34.619221+0000","last_became_active":"2024-05-11T12:14:33.616400+0000","last_became_peered":"2024-05-11T12:14:33.616400+0000","last_unstale":"2024-05-11T12:14:34.619221+0000","last_undegraded":"2024-05-11T12:14:34.619221+0000","last_fullsized":"2024-05-11T12:14:34.619221+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:59:12.515990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.283960+0000","last_change":"2024-05-11T12:14:32.607882+0000","last_active":"2024-05-11T12:14:35.283960+0000","last_peered":"2024-05-11T12:14:35.283960+0000","last_clean":"2024-05-11T12:14:35.283960+0000","last_became_active":"2024-05-11T12:14:32.607528+0000","last_became_peered":"2024-05-11T12:14:32.607528+0000","last_unstale":"2024-05-11T12:14:35.283960+0000","last_undegraded":"2024-05-11T12:14:35.283960+0000","last_fullsized":"2024-05-11T12:14:35.283960+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:34:52.216177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.284047+0000","last_change":"2024-05-11T12:14:33.628739+0000","last_active":"2024-05-11T12:14:35.284047+0000","last_peered":"2024-05-11T12:14:35.284047+0000","last_clean":"2024-05-11T12:14:35.284047+0000","last_became_active":"2024-05-11T12:14:33.628423+0000","last_became_peered":"2024-05-11T12:14:33.628423+0000","last_unstale":"2024-05-11T12:14:35.284047+0000","last_undegraded":"2024-05-11T12:14:35.284047+0000","last_fullsized":"2024-05-11T12:14:35.284047+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:56:09.727119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.284356+0000","last_change":"2024-05-11T12:14:32.616922+0000","last_active":"2024-05-11T12:14:35.284356+0000","last_peered":"2024-05-11T12:14:35.284356+0000","last_clean":"2024-05-11T12:14:35.284356+0000","last_became_active":"2024-05-11T12:14:32.616599+0000","last_became_peered":"2024-05-11T12:14:32.616599+0000","last_unstale":"2024-05-11T12:14:35.284356+0000","last_undegraded":"2024-05-11T12:14:35.284356+0000","last_fullsized":"2024-05-11T12:14:35.284356+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:55:59.198545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209297+0000","last_change":"2024-05-11T12:14:33.624455+0000","last_active":"2024-05-11T12:14:35.209297+0000","last_peered":"2024-05-11T12:14:35.209297+0000","last_clean":"2024-05-11T12:14:35.209297+0000","last_became_active":"2024-05-11T12:14:33.624158+0000","last_became_peered":"2024-05-11T12:14:33.624158+0000","last_unstale":"2024-05-11T12:14:35.209297+0000","last_undegraded":"2024-05-11T12:14:35.209297+0000","last_fullsized":"2024-05-11T12:14:35.209297+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:44:32.910714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.208576+0000","last_change":"2024-05-11T12:14:32.615486+0000","last_active":"2024-05-11T12:14:35.208576+0000","last_peered":"2024-05-11T12:14:35.208576+0000","last_clean":"2024-05-11T12:14:35.208576+0000","last_became_active":"2024-05-11T12:14:32.615204+0000","last_became_peered":"2024-05-11T12:14:32.615204+0000","last_unstale":"2024-05-11T12:14:35.208576+0000","last_undegraded":"2024-05-11T12:14:35.208576+0000","last_fullsized":"2024-05-11T12:14:35.208576+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:27:00.766407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615938+0000","last_change":"2024-05-11T12:14:33.623140+0000","last_active":"2024-05-11T12:14:34.615938+0000","last_peered":"2024-05-11T12:14:34.615938+0000","last_clean":"2024-05-11T12:14:34.615938+0000","last_became_active":"2024-05-11T12:14:33.618444+0000","last_became_peered":"2024-05-11T12:14:33.618444+0000","last_unstale":"2024-05-11T12:14:34.615938+0000","last_undegraded":"2024-05-11T12:14:34.615938+0000","last_fullsized":"2024-05-11T12:14:34.615938+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:35:51.107577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"96'8","reported_seq":22,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.619539+0000","last_change":"2024-05-11T12:14:32.616263+0000","last_active":"2024-05-11T12:14:34.619539+0000","last_peered":"2024-05-11T12:14:34.619539+0000","last_clean":"2024-05-11T12:14:34.619539+0000","last_became_active":"2024-05-11T12:14:32.615004+0000","last_became_peered":"2024-05-11T12:14:32.615004+0000","last_unstale":"2024-05-11T12:14:34.619539+0000","last_undegraded":"2024-05-11T12:14:34.619539+0000","last_fullsized":"2024-05-11T12:14:34.619539+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:54:11.073754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.208800+0000","last_change":"2024-05-11T12:14:33.616435+0000","last_active":"2024-05-11T12:14:35.208800+0000","last_peered":"2024-05-11T12:14:35.208800+0000","last_clean":"2024-05-11T12:14:35.208800+0000","last_became_active":"2024-05-11T12:14:33.616110+0000","last_became_peered":"2024-05-11T12:14:33.616110+0000","last_unstale":"2024-05-11T12:14:35.208800+0000","last_undegraded":"2024-05-11T12:14:35.208800+0000","last_fullsized":"2024-05-11T12:14:35.208800+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:51:10.706767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.284168+0000","last_change":"2024-05-11T12:14:32.613751+0000","last_active":"2024-05-11T12:14:35.284168+0000","last_peered":"2024-05-11T12:14:35.284168+0000","last_clean":"2024-05-11T12:14:35.284168+0000","last_became_active":"2024-05-11T12:14:32.613431+0000","last_became_peered":"2024-05-11T12:14:32.613431+0000","last_unstale":"2024-05-11T12:14:35.284168+0000","last_undegraded":"2024-05-11T12:14:35.284168+0000","last_fullsized":"2024-05-11T12:14:35.284168+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:07:53.042878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.283203+0000","last_change":"2024-05-11T12:14:33.637186+0000","last_active":"2024-05-11T12:14:35.283203+0000","last_peered":"2024-05-11T12:14:35.283203+0000","last_clean":"2024-05-11T12:14:35.283203+0000","last_became_active":"2024-05-11T12:14:33.636807+0000","last_became_peered":"2024-05-11T12:14:33.636807+0000","last_unstale":"2024-05-11T12:14:35.283203+0000","last_undegraded":"2024-05-11T12:14:35.283203+0000","last_fullsized":"2024-05-11T12:14:35.283203+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:22:15.346218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.185187+0000","last_change":"2024-05-11T12:14:32.621615+0000","last_active":"2024-05-11T12:14:35.185187+0000","last_peered":"2024-05-11T12:14:35.185187+0000","last_clean":"2024-05-11T12:14:35.185187+0000","last_became_active":"2024-05-11T12:14:32.620973+0000","last_became_peered":"2024-05-11T12:14:32.620973+0000","last_unstale":"2024-05-11T12:14:35.185187+0000","last_undegraded":"2024-05-11T12:14:35.185187+0000","last_fullsized":"2024-05-11T12:14:35.185187+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:59:11.499643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.283520+0000","last_change":"2024-05-11T12:14:33.615559+0000","last_active":"2024-05-11T12:14:35.283520+0000","last_peered":"2024-05-11T12:14:35.283520+0000","last_clean":"2024-05-11T12:14:35.283520+0000","last_became_active":"2024-05-11T12:14:33.615276+0000","last_became_peered":"2024-05-11T12:14:33.615276+0000","last_unstale":"2024-05-11T12:14:35.283520+0000","last_undegraded":"2024-05-11T12:14:35.283520+0000","last_fullsized":"2024-05-11T12:14:35.283520+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:15:22.180722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.619522+0000","last_change":"2024-05-11T12:14:32.609813+0000","last_active":"2024-05-11T12:14:34.619522+0000","last_peered":"2024-05-11T12:14:34.619522+0000","last_clean":"2024-05-11T12:14:34.619522+0000","last_became_active":"2024-05-11T12:14:32.609531+0000","last_became_peered":"2024-05-11T12:14:32.609531+0000","last_unstale":"2024-05-11T12:14:34.619522+0000","last_undegraded":"2024-05-11T12:14:34.619522+0000","last_fullsized":"2024-05-11T12:14:34.619522+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:30:51.604601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615796+0000","last_change":"2024-05-11T12:14:33.629837+0000","last_active":"2024-05-11T12:14:34.615796+0000","last_peered":"2024-05-11T12:14:34.615796+0000","last_clean":"2024-05-11T12:14:34.615796+0000","last_became_active":"2024-05-11T12:14:33.629561+0000","last_became_peered":"2024-05-11T12:14:33.629561+0000","last_unstale":"2024-05-11T12:14:34.615796+0000","last_undegraded":"2024-05-11T12:14:34.615796+0000","last_fullsized":"2024-05-11T12:14:34.615796+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:24:04.957521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209466+0000","last_change":"2024-05-11T12:14:32.605666+0000","last_active":"2024-05-11T12:14:35.209466+0000","last_peered":"2024-05-11T12:14:35.209466+0000","last_clean":"2024-05-11T12:14:35.209466+0000","last_became_active":"2024-05-11T12:14:32.605407+0000","last_became_peered":"2024-05-11T12:14:32.605407+0000","last_unstale":"2024-05-11T12:14:35.209466+0000","last_undegraded":"2024-05-11T12:14:35.209466+0000","last_fullsized":"2024-05-11T12:14:35.209466+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:31:24.122325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615946+0000","last_change":"2024-05-11T12:14:33.623726+0000","last_active":"2024-05-11T12:14:34.615946+0000","last_peered":"2024-05-11T12:14:34.615946+0000","last_clean":"2024-05-11T12:14:34.615946+0000","last_became_active":"2024-05-11T12:14:33.616682+0000","last_became_peered":"2024-05-11T12:14:33.616682+0000","last_unstale":"2024-05-11T12:14:34.615946+0000","last_undegraded":"2024-05-11T12:14:34.615946+0000","last_fullsized":"2024-05-11T12:14:34.615946+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:11:09.187757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.208810+0000","last_change":"2024-05-11T12:14:32.612947+0000","last_active":"2024-05-11T12:14:35.208810+0000","last_peered":"2024-05-11T12:14:35.208810+0000","last_clean":"2024-05-11T12:14:35.208810+0000","last_became_active":"2024-05-11T12:14:32.612545+0000","last_became_peered":"2024-05-11T12:14:32.612545+0000","last_unstale":"2024-05-11T12:14:35.208810+0000","last_undegraded":"2024-05-11T12:14:35.208810+0000","last_fullsized":"2024-05-11T12:14:35.208810+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:10:08.253641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.285245+0000","last_change":"2024-05-11T12:14:33.628473+0000","last_active":"2024-05-11T12:14:35.285245+0000","last_peered":"2024-05-11T12:14:35.285245+0000","last_clean":"2024-05-11T12:14:35.285245+0000","last_became_active":"2024-05-11T12:14:33.627301+0000","last_became_peered":"2024-05-11T12:14:33.627301+0000","last_unstale":"2024-05-11T12:14:35.285245+0000","last_undegraded":"2024-05-11T12:14:35.285245+0000","last_fullsized":"2024-05-11T12:14:35.285245+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:16:12.500928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.283089+0000","last_change":"2024-05-11T12:14:32.607325+0000","last_active":"2024-05-11T12:14:35.283089+0000","last_peered":"2024-05-11T12:14:35.283089+0000","last_clean":"2024-05-11T12:14:35.283089+0000","last_became_active":"2024-05-11T12:14:32.606982+0000","last_became_peered":"2024-05-11T12:14:32.606982+0000","last_unstale":"2024-05-11T12:14:35.283089+0000","last_undegraded":"2024-05-11T12:14:35.283089+0000","last_fullsized":"2024-05-11T12:14:35.283089+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:54:11.073754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209260+0000","last_change":"2024-05-11T12:14:33.618158+0000","last_active":"2024-05-11T12:14:35.209260+0000","last_peered":"2024-05-11T12:14:35.209260+0000","last_clean":"2024-05-11T12:14:35.209260+0000","last_became_active":"2024-05-11T12:14:33.617850+0000","last_became_peered":"2024-05-11T12:14:33.617850+0000","last_unstale":"2024-05-11T12:14:35.209260+0000","last_undegraded":"2024-05-11T12:14:35.209260+0000","last_fullsized":"2024-05-11T12:14:35.209260+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:38:25.159336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209121+0000","last_change":"2024-05-11T12:14:32.607439+0000","last_active":"2024-05-11T12:14:35.209121+0000","last_peered":"2024-05-11T12:14:35.209121+0000","last_clean":"2024-05-11T12:14:35.209121+0000","last_became_active":"2024-05-11T12:14:32.606952+0000","last_became_peered":"2024-05-11T12:14:32.606952+0000","last_unstale":"2024-05-11T12:14:35.209121+0000","last_undegraded":"2024-05-11T12:14:35.209121+0000","last_fullsized":"2024-05-11T12:14:35.209121+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:14:57.743590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615535+0000","last_change":"2024-05-11T12:14:33.619178+0000","last_active":"2024-05-11T12:14:34.615535+0000","last_peered":"2024-05-11T12:14:34.615535+0000","last_clean":"2024-05-11T12:14:34.615535+0000","last_became_active":"2024-05-11T12:14:33.618894+0000","last_became_peered":"2024-05-11T12:14:33.618894+0000","last_unstale":"2024-05-11T12:14:34.615535+0000","last_undegraded":"2024-05-11T12:14:34.615535+0000","last_fullsized":"2024-05-11T12:14:34.615535+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:10:50.170349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.283804+0000","last_change":"2024-05-11T12:14:32.608235+0000","last_active":"2024-05-11T12:14:35.283804+0000","last_peered":"2024-05-11T12:14:35.283804+0000","last_clean":"2024-05-11T12:14:35.283804+0000","last_became_active":"2024-05-11T12:14:32.607966+0000","last_became_peered":"2024-05-11T12:14:32.607966+0000","last_unstale":"2024-05-11T12:14:35.283804+0000","last_undegraded":"2024-05-11T12:14:35.283804+0000","last_fullsized":"2024-05-11T12:14:35.283804+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:30:51.604601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.621157+0000","last_change":"2024-05-11T12:14:33.626562+0000","last_active":"2024-05-11T12:14:34.621157+0000","last_peered":"2024-05-11T12:14:34.621157+0000","last_clean":"2024-05-11T12:14:34.621157+0000","last_became_active":"2024-05-11T12:14:33.625547+0000","last_became_peered":"2024-05-11T12:14:33.625547+0000","last_unstale":"2024-05-11T12:14:34.621157+0000","last_undegraded":"2024-05-11T12:14:34.621157+0000","last_fullsized":"2024-05-11T12:14:34.621157+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:11:02.751326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"97'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.615678+0000","last_change":"2024-05-11T12:14:32.605888+0000","last_active":"2024-05-11T12:14:34.615678+0000","last_peered":"2024-05-11T12:14:34.615678+0000","last_clean":"2024-05-11T12:14:34.615678+0000","last_became_active":"2024-05-11T12:14:32.605588+0000","last_became_peered":"2024-05-11T12:14:32.605588+0000","last_unstale":"2024-05-11T12:14:34.615678+0000","last_undegraded":"2024-05-11T12:14:34.615678+0000","last_fullsized":"2024-05-11T12:14:34.615678+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:56:02.427960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.284207+0000","last_change":"2024-05-11T12:14:33.627479+0000","last_active":"2024-05-11T12:14:35.284207+0000","last_peered":"2024-05-11T12:14:35.284207+0000","last_clean":"2024-05-11T12:14:35.284207+0000","last_became_active":"2024-05-11T12:14:33.627179+0000","last_became_peered":"2024-05-11T12:14:33.627179+0000","last_unstale":"2024-05-11T12:14:35.284207+0000","last_undegraded":"2024-05-11T12:14:35.284207+0000","last_fullsized":"2024-05-11T12:14:35.284207+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:35:20.045762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.283940+0000","last_change":"2024-05-11T12:14:32.605594+0000","last_active":"2024-05-11T12:14:35.283940+0000","last_peered":"2024-05-11T12:14:35.283940+0000","last_clean":"2024-05-11T12:14:35.283940+0000","last_became_active":"2024-05-11T12:14:32.605097+0000","last_became_peered":"2024-05-11T12:14:32.605097+0000","last_unstale":"2024-05-11T12:14:35.283940+0000","last_undegraded":"2024-05-11T12:14:35.283940+0000","last_fullsized":"2024-05-11T12:14:35.283940+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:30:51.604601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.619647+0000","last_change":"2024-05-11T12:14:33.622085+0000","last_active":"2024-05-11T12:14:34.619647+0000","last_peered":"2024-05-11T12:14:34.619647+0000","last_clean":"2024-05-11T12:14:34.619647+0000","last_became_active":"2024-05-11T12:14:33.621754+0000","last_became_peered":"2024-05-11T12:14:33.621754+0000","last_unstale":"2024-05-11T12:14:34.619647+0000","last_undegraded":"2024-05-11T12:14:34.619647+0000","last_fullsized":"2024-05-11T12:14:34.619647+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:15:52.602707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.283625+0000","last_change":"2024-05-11T12:14:32.613449+0000","last_active":"2024-05-11T12:14:35.283625+0000","last_peered":"2024-05-11T12:14:35.283625+0000","last_clean":"2024-05-11T12:14:35.283625+0000","last_became_active":"2024-05-11T12:14:32.613108+0000","last_became_peered":"2024-05-11T12:14:32.613108+0000","last_unstale":"2024-05-11T12:14:35.283625+0000","last_undegraded":"2024-05-11T12:14:35.283625+0000","last_fullsized":"2024-05-11T12:14:35.283625+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:07:53.042878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.208801+0000","last_change":"2024-05-11T12:14:33.630741+0000","last_active":"2024-05-11T12:14:35.208801+0000","last_peered":"2024-05-11T12:14:35.208801+0000","last_clean":"2024-05-11T12:14:35.208801+0000","last_became_active":"2024-05-11T12:14:33.630423+0000","last_became_peered":"2024-05-11T12:14:33.630423+0000","last_unstale":"2024-05-11T12:14:35.208801+0000","last_undegraded":"2024-05-11T12:14:35.208801+0000","last_fullsized":"2024-05-11T12:14:35.208801+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:47:57.386158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.621831+0000","last_change":"2024-05-11T12:14:32.616942+0000","last_active":"2024-05-11T12:14:34.621831+0000","last_peered":"2024-05-11T12:14:34.621831+0000","last_clean":"2024-05-11T12:14:34.621831+0000","last_became_active":"2024-05-11T12:14:32.616577+0000","last_became_peered":"2024-05-11T12:14:32.616577+0000","last_unstale":"2024-05-11T12:14:34.621831+0000","last_undegraded":"2024-05-11T12:14:34.621831+0000","last_fullsized":"2024-05-11T12:14:34.621831+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:07:53.042878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.208768+0000","last_change":"2024-05-11T12:14:33.620388+0000","last_active":"2024-05-11T12:14:35.208768+0000","last_peered":"2024-05-11T12:14:35.208768+0000","last_clean":"2024-05-11T12:14:35.208768+0000","last_became_active":"2024-05-11T12:14:33.618621+0000","last_became_peered":"2024-05-11T12:14:33.618621+0000","last_unstale":"2024-05-11T12:14:35.208768+0000","last_undegraded":"2024-05-11T12:14:35.208768+0000","last_fullsized":"2024-05-11T12:14:35.208768+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:57:26.186975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209282+0000","last_change":"2024-05-11T12:14:32.615893+0000","last_active":"2024-05-11T12:14:35.209282+0000","last_peered":"2024-05-11T12:14:35.209282+0000","last_clean":"2024-05-11T12:14:35.209282+0000","last_became_active":"2024-05-11T12:14:32.615639+0000","last_became_peered":"2024-05-11T12:14:32.615639+0000","last_unstale":"2024-05-11T12:14:35.209282+0000","last_undegraded":"2024-05-11T12:14:35.209282+0000","last_fullsized":"2024-05-11T12:14:35.209282+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:51:28.062961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.185419+0000","last_change":"2024-05-11T12:14:33.628888+0000","last_active":"2024-05-11T12:14:35.185419+0000","last_peered":"2024-05-11T12:14:35.185419+0000","last_clean":"2024-05-11T12:14:35.185419+0000","last_became_active":"2024-05-11T12:14:33.625527+0000","last_became_peered":"2024-05-11T12:14:33.625527+0000","last_unstale":"2024-05-11T12:14:35.185419+0000","last_undegraded":"2024-05-11T12:14:35.185419+0000","last_fullsized":"2024-05-11T12:14:35.185419+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:27:47.349240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.283457+0000","last_change":"2024-05-11T12:14:32.607229+0000","last_active":"2024-05-11T12:14:35.283457+0000","last_peered":"2024-05-11T12:14:35.283457+0000","last_clean":"2024-05-11T12:14:35.283457+0000","last_became_active":"2024-05-11T12:14:32.606938+0000","last_became_peered":"2024-05-11T12:14:32.606938+0000","last_unstale":"2024-05-11T12:14:35.283457+0000","last_undegraded":"2024-05-11T12:14:35.283457+0000","last_fullsized":"2024-05-11T12:14:35.283457+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:58:41.059975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.620675+0000","last_change":"2024-05-11T12:14:33.614925+0000","last_active":"2024-05-11T12:14:34.620675+0000","last_peered":"2024-05-11T12:14:34.620675+0000","last_clean":"2024-05-11T12:14:34.620675+0000","last_became_active":"2024-05-11T12:14:33.614629+0000","last_became_peered":"2024-05-11T12:14:33.614629+0000","last_unstale":"2024-05-11T12:14:34.620675+0000","last_undegraded":"2024-05-11T12:14:34.620675+0000","last_fullsized":"2024-05-11T12:14:34.620675+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:11:02.751326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209088+0000","last_change":"2024-05-11T12:14:32.610511+0000","last_active":"2024-05-11T12:14:35.209088+0000","last_peered":"2024-05-11T12:14:35.209088+0000","last_clean":"2024-05-11T12:14:35.209088+0000","last_became_active":"2024-05-11T12:14:32.610243+0000","last_became_peered":"2024-05-11T12:14:32.610243+0000","last_unstale":"2024-05-11T12:14:35.209088+0000","last_undegraded":"2024-05-11T12:14:35.209088+0000","last_fullsized":"2024-05-11T12:14:35.209088+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:39:24.599004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:34.619309+0000","last_change":"2024-05-11T12:14:33.617894+0000","last_active":"2024-05-11T12:14:34.619309+0000","last_peered":"2024-05-11T12:14:34.619309+0000","last_clean":"2024-05-11T12:14:34.619309+0000","last_became_active":"2024-05-11T12:14:33.617537+0000","last_became_peered":"2024-05-11T12:14:33.617537+0000","last_unstale":"2024-05-11T12:14:34.619309+0000","last_undegraded":"2024-05-11T12:14:34.619309+0000","last_fullsized":"2024-05-11T12:14:34.619309+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:32.562048+0000","last_clean_scrub_stamp":"2024-05-11T12:14:32.562048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:17:04.293672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.209535+0000","last_change":"2024-05-11T12:14:32.612907+0000","last_active":"2024-05-11T12:14:35.209535+0000","last_peered":"2024-05-11T12:14:35.209535+0000","last_clean":"2024-05-11T12:14:35.209535+0000","last_became_active":"2024-05-11T12:14:32.612548+0000","last_became_peered":"2024-05-11T12:14:32.612548+0000","last_unstale":"2024-05-11T12:14:35.209535+0000","last_undegraded":"2024-05-11T12:14:35.209535+0000","last_fullsized":"2024-05-11T12:14:35.209535+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:14:31.545700+0000","last_clean_scrub_stamp":"2024-05-11T12:14:31.545700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:49:45.457016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":248,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-05-11T12:14:35.184579+0000","last_change":"2024-05-11T12:10:35.066673+0000","last_active":"2024-05-11T12:14:35.184579+0000","last_peered":"2024-05-11T12:14:35.184579+0000","last_clean":"2024-05-11T12:14:35.184579+0000","last_became_active":"2024-05-11T12:10:35.065942+0000","last_became_peered":"2024-05-11T12:10:35.065942+0000","last_unstale":"2024-05-11T12:14:35.184579+0000","last_undegraded":"2024-05-11T12:14:35.184579+0000","last_fullsized":"2024-05-11T12:14:35.184579+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_clean_scrub_stamp":"2024-05-11T12:10:33.601136+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:12:02.133182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476788,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32932,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338908,"statfs":{"total":96636764160,"available":96603041792,"internally_reserved":0,"allocated":1794048,"data_stored":1132724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476789,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32364,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339476,"statfs":{"total":96636764160,"available":96603623424,"internally_reserved":0,"allocated":1212416,"data_stored":544412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476788,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32348,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339492,"statfs":{"total":96636764160,"available":96603639808,"internally_reserved":0,"allocated":1191936,"data_stored":542196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476791,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1200128,"data_stored":542288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509494,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1196032,"data_stored":542778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509495,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32348,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339492,"statfs":{"total":96636764160,"available":96603639808,"internally_reserved":0,"allocated":1196032,"data_stored":542230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509494,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32948,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338892,"statfs":{"total":96636764160,"available":96603025408,"internally_reserved":0,"allocated":1802240,"data_stored":1134266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542198,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1196032,"data_stored":542230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T12:14:43.561 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-11T12:14:43.561 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-11T12:14:43.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.810+0000 7f7c17c00640 1 Processor -- start 2024-05-11T12:14:43.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.810+0000 7f7c17c00640 1 -- start start 2024-05-11T12:14:43.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.812+0000 7f7c17c00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7c10072aa0 0x7f7c1006e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:43.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.812+0000 7f7c17c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f7c1015e360 0x7f7c1006e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:43.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.812+0000 7f7c17c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c10142750 0x7f7c10142c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:43.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.812+0000 7f7c17c00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f7c1015d620 con 0x7f7c10072aa0 2024-05-11T12:14:43.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.812+0000 7f7c17c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f7c10161a60 con 0x7f7c1015e360 2024-05-11T12:14:43.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.812+0000 7f7c17c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f7c10143440 con 0x7f7c10142750 2024-05-11T12:14:43.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.812+0000 7f7c15200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7c10072aa0 0x7f7c1006e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:43.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.812+0000 7f7c15200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7c10072aa0 0x7f7c1006e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:47632/0 (socket says 172.21.15.167:47632) 2024-05-11T12:14:43.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.812+0000 7f7c15200640 1 -- 172.21.15.167:0/2879789902 learned_addr learned my addr 172.21.15.167:0/2879789902 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:43.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.813+0000 7f7c15200640 1 -- 172.21.15.167:0/2879789902 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c10142750 msgr2=0x7f7c10142c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:43.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.813+0000 7f7c15c00640 1 --2- 172.21.15.167:0/2879789902 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c10142750 0x7f7c10142c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:43.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.813+0000 7f7c0fe00640 1 --2- 172.21.15.167:0/2879789902 >> v2:172.21.15.204:3300/0 conn(0x7f7c1015e360 0x7f7c1006e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:43.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.813+0000 7f7c15200640 1 --2- 172.21.15.167:0/2879789902 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c10142750 0x7f7c10142c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:43.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.813+0000 7f7c15200640 1 -- 172.21.15.167:0/2879789902 >> v2:172.21.15.204:3300/0 conn(0x7f7c1015e360 msgr2=0x7f7c1006e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:43.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.813+0000 7f7c15200640 1 --2- 172.21.15.167:0/2879789902 >> v2:172.21.15.204:3300/0 conn(0x7f7c1015e360 0x7f7c1006e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:43.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.813+0000 7f7c15200640 1 -- 172.21.15.167:0/2879789902 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c1015d880 con 0x7f7c10072aa0 2024-05-11T12:14:43.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.813+0000 7f7c0fe00640 1 --2- 172.21.15.167:0/2879789902 >> v2:172.21.15.204:3300/0 conn(0x7f7c1015e360 0x7f7c1006e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:43.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.814+0000 7f7c15200640 1 --2- 172.21.15.167:0/2879789902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c10072aa0 0x7f7c1006e350 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f7c100b3e40 tx=0x7f7c0402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e10d8d12fd3fcae server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:43.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.814+0000 7f7c0f400640 1 -- 172.21.15.167:0/2879789902 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c04037070 con 0x7f7c10072aa0 2024-05-11T12:14:43.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.814+0000 7f7c0f400640 1 -- 172.21.15.167:0/2879789902 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c04032440 con 0x7f7c10072aa0 2024-05-11T12:14:43.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.814+0000 7f7c0f400640 1 -- 172.21.15.167:0/2879789902 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c04032810 con 0x7f7c10072aa0 2024-05-11T12:14:43.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.815+0000 7f7c17c00640 1 -- 172.21.15.167:0/2879789902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c10072aa0 msgr2=0x7f7c1006e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:43.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.815+0000 7f7c17c00640 1 --2- 172.21.15.167:0/2879789902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c10072aa0 0x7f7c1006e350 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f7c100b3e40 tx=0x7f7c0402a1e0 comp rx=0 tx=0).stop 2024-05-11T12:14:43.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.815+0000 7f7c17c00640 1 -- 172.21.15.167:0/2879789902 shutdown_connections 2024-05-11T12:14:43.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.815+0000 7f7c17c00640 1 --2- 172.21.15.167:0/2879789902 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c10142750 0x7f7c10142c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:43.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.815+0000 7f7c17c00640 1 --2- 172.21.15.167:0/2879789902 >> v2:172.21.15.204:3300/0 conn(0x7f7c1015e360 0x7f7c1006e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:43.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.815+0000 7f7c17c00640 1 --2- 172.21.15.167:0/2879789902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c10072aa0 0x7f7c1006e350 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:43.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.815+0000 7f7c17c00640 1 -- 172.21.15.167:0/2879789902 >> 172.21.15.167:0/2879789902 conn(0x7f7c100b2f60 msgr2=0x7f7c100b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:43.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.815+0000 7f7c17c00640 1 -- 172.21.15.167:0/2879789902 shutdown_connections 2024-05-11T12:14:43.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.815+0000 7f7c17c00640 1 -- 172.21.15.167:0/2879789902 wait complete. 2024-05-11T12:14:43.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.817+0000 7f7c17c00640 1 Processor -- start 2024-05-11T12:14:43.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.817+0000 7f7c17c00640 1 -- start start 2024-05-11T12:14:43.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.818+0000 7f7c17c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c10072aa0 0x7f7c100aa410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:43.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.818+0000 7f7c17c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c10142750 0x7f7c101f48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:43.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.818+0000 7f7c17c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c1015e360 0x7f7c100b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:43.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.818+0000 7f7c17c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7c10143140 con 0x7f7c10072aa0 2024-05-11T12:14:43.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.818+0000 7f7c17c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1015de40 con 0x7f7c10142750 2024-05-11T12:14:43.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.818+0000 7f7c17c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f7c10072e80 con 0x7f7c1015e360 2024-05-11T12:14:43.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.818+0000 7f7c15200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c10072aa0 0x7f7c100aa410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:43.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.818+0000 7f7c15200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c10072aa0 0x7f7c100aa410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:47648/0 (socket says 172.21.15.167:47648) 2024-05-11T12:14:43.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.818+0000 7f7c15200640 1 -- 172.21.15.167:0/3720068486 learned_addr learned my addr 172.21.15.167:0/3720068486 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:43.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.819+0000 7f7c15c00640 1 --2- 172.21.15.167:0/3720068486 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c1015e360 0x7f7c100b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:43.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.819+0000 7f7c0fe00640 1 --2- 172.21.15.167:0/3720068486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c10142750 0x7f7c101f48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:43.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.819+0000 7f7c15200640 1 -- 172.21.15.167:0/3720068486 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c1015e360 msgr2=0x7f7c100b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:43.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.819+0000 7f7c15200640 1 --2- 172.21.15.167:0/3720068486 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c1015e360 0x7f7c100b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:43.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.819+0000 7f7c15200640 1 -- 172.21.15.167:0/3720068486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c10142750 msgr2=0x7f7c101f48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:43.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.819+0000 7f7c15200640 1 --2- 172.21.15.167:0/3720068486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c10142750 0x7f7c101f48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:43.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.819+0000 7f7c15200640 1 -- 172.21.15.167:0/3720068486 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c100aac60 con 0x7f7c10072aa0 2024-05-11T12:14:43.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.819+0000 7f7c15200640 1 --2- 172.21.15.167:0/3720068486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c10072aa0 0x7f7c100aa410 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f7c0402a220 tx=0x7f7c040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:43.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.819+0000 7f7c0cc00640 1 -- 172.21.15.167:0/3720068486 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c04037040 con 0x7f7c10072aa0 2024-05-11T12:14:43.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.820+0000 7f7c0cc00640 1 -- 172.21.15.167:0/3720068486 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c04042070 con 0x7f7c10072aa0 2024-05-11T12:14:43.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.820+0000 7f7c0cc00640 1 -- 172.21.15.167:0/3720068486 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c04030bb0 con 0x7f7c10072aa0 2024-05-11T12:14:43.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.820+0000 7f7c17c00640 1 -- 172.21.15.167:0/3720068486 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c100b4320 con 0x7f7c10072aa0 2024-05-11T12:14:43.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.820+0000 7f7c17c00640 1 -- 172.21.15.167:0/3720068486 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c100b48e0 con 0x7f7c10072aa0 2024-05-11T12:14:43.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.821+0000 7f7c17c00640 1 -- 172.21.15.167:0/3720068486 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c10069630 con 0x7f7c10072aa0 2024-05-11T12:14:43.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.822+0000 7f7c0cc00640 1 -- 172.21.15.167:0/3720068486 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7c04047460 con 0x7f7c10072aa0 2024-05-11T12:14:43.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.823+0000 7f7c0cc00640 1 --2- 172.21.15.167:0/3720068486 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f7bf00762d0 0x7f7bf0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:43.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.823+0000 7f7c0cc00640 1 -- 172.21.15.167:0/3720068486 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6101+0+0 (secure 0 0 0) 0x7f7c040b83a0 con 0x7f7c10072aa0 2024-05-11T12:14:43.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.823+0000 7f7c0fe00640 1 --2- 172.21.15.167:0/3720068486 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f7bf00762d0 0x7f7bf0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:43.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.824+0000 7f7c0fe00640 1 --2- 172.21.15.167:0/3720068486 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f7bf00762d0 0x7f7bf0078790 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7c1015f310 tx=0x7f7bfc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:43.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:43.828+0000 7f7c0cc00640 1 -- 172.21.15.167:0/3720068486 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7c04081b70 con 0x7f7c10072aa0 2024-05-11T12:14:44.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.102+0000 7f7c17c00640 1 -- 172.21.15.167:0/3720068486 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f7c10073250 con 0x7f7c10072aa0 2024-05-11T12:14:44.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.108+0000 7f7c0cc00640 1 -- 172.21.15.167:0/3720068486 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) ==== 190+0+147 (secure 0 0 0) 0x7f7c04081470 con 0x7f7c10072aa0 2024-05-11T12:14:44.107 INFO:teuthology.orchestra.run.smithi167.stdout:[client.0] 2024-05-11T12:14:44.107 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQAFYD9mLK/lEhAAyERTYMl250KkBvIrAXWYSw== 2024-05-11T12:14:44.108 INFO:teuthology.orchestra.run.smithi167.stdout: caps mds = "allow" 2024-05-11T12:14:44.108 INFO:teuthology.orchestra.run.smithi167.stdout: caps mon = "allow r" 2024-05-11T12:14:44.108 INFO:teuthology.orchestra.run.smithi167.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-11T12:14:44.108 INFO:teuthology.orchestra.run.smithi167.stderr:updated caps for client.0 2024-05-11T12:14:44.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.110+0000 7f7c17c00640 1 -- 172.21.15.167:0/3720068486 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f7bf00762d0 msgr2=0x7f7bf0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:44.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.110+0000 7f7c17c00640 1 --2- 172.21.15.167:0/3720068486 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f7bf00762d0 0x7f7bf0078790 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7c1015f310 tx=0x7f7bfc008040 comp rx=0 tx=0).stop 2024-05-11T12:14:44.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.110+0000 7f7c17c00640 1 -- 172.21.15.167:0/3720068486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c10072aa0 msgr2=0x7f7c100aa410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:44.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.110+0000 7f7c17c00640 1 --2- 172.21.15.167:0/3720068486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c10072aa0 0x7f7c100aa410 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f7c0402a220 tx=0x7f7c040046c0 comp rx=0 tx=0).stop 2024-05-11T12:14:44.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.110+0000 7f7c17c00640 1 -- 172.21.15.167:0/3720068486 shutdown_connections 2024-05-11T12:14:44.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.110+0000 7f7c17c00640 1 --2- 172.21.15.167:0/3720068486 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c1015e360 0x7f7c100b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:44.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.110+0000 7f7c17c00640 1 --2- 172.21.15.167:0/3720068486 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f7bf00762d0 0x7f7bf0078790 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:44.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.110+0000 7f7c17c00640 1 --2- 172.21.15.167:0/3720068486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c10142750 0x7f7c101f48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:44.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.110+0000 7f7c17c00640 1 --2- 172.21.15.167:0/3720068486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c10072aa0 0x7f7c100aa410 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:44.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.110+0000 7f7c17c00640 1 -- 172.21.15.167:0/3720068486 >> 172.21.15.167:0/3720068486 conn(0x7f7c100b2f60 msgr2=0x7f7c100b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:44.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.111+0000 7f7c17c00640 1 -- 172.21.15.167:0/3720068486 shutdown_connections 2024-05-11T12:14:44.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.111+0000 7f7c17c00640 1 -- 172.21.15.167:0/3720068486 wait complete. 2024-05-11T12:14:44.127 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:14:44.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.320+0000 7f2705800640 1 Processor -- start 2024-05-11T12:14:44.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.320+0000 7f2705800640 1 -- start start 2024-05-11T12:14:44.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.321+0000 7f2705800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2700145b00 0x7f270006f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:44.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.322+0000 7f2705800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2700146770 0x7f27000ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:44.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.322+0000 7f2705800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f270006fbc0 con 0x7f27000cef10 2024-05-11T12:14:44.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.322+0000 7f2705800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f27000b4520 con 0x7f2700146770 2024-05-11T12:14:44.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.322+0000 7f2705800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f27000b4800 con 0x7f2700145b00 2024-05-11T12:14:44.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.322+0000 7f26ffe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2700145b00 0x7f270006f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:44.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.322+0000 7f26ff400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2700146770 0x7f27000ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:44.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.322+0000 7f26ffe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2700145b00 0x7f270006f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:54550/0 (socket says 172.21.15.167:54550) 2024-05-11T12:14:44.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.322+0000 7f26ffe00640 1 -- 172.21.15.167:0/4203565933 learned_addr learned my addr 172.21.15.167:0/4203565933 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:44.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.323+0000 7f26ffe00640 1 -- 172.21.15.167:0/4203565933 >> v2:172.21.15.204:3300/0 conn(0x7f2700146770 msgr2=0x7f27000ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:44.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.323+0000 7f26ffe00640 1 --2- 172.21.15.167:0/4203565933 >> v2:172.21.15.204:3300/0 conn(0x7f2700146770 0x7f27000ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:44.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.323+0000 7f26ffe00640 1 -- 172.21.15.167:0/4203565933 >> v1:172.21.15.204:6789/0 conn(0x7f27000cef10 legacy=0x7f270006c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:44.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.323+0000 7f26ffe00640 1 -- 172.21.15.167:0/4203565933 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27000b4ae0 con 0x7f2700145b00 2024-05-11T12:14:44.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.325+0000 7f26ffe00640 1 --2- 172.21.15.167:0/4203565933 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2700145b00 0x7f270006f2c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f26f0009870 tx=0x7f26f002a1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dfbedc9d582e3dd server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:44.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.325+0000 7f26fea00640 1 -- 172.21.15.167:0/4203565933 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f0037070 con 0x7f2700145b00 2024-05-11T12:14:44.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.325+0000 7f26fea00640 1 -- 172.21.15.167:0/4203565933 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26f0032440 con 0x7f2700145b00 2024-05-11T12:14:44.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.325+0000 7f26fea00640 1 -- 172.21.15.167:0/4203565933 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f0032810 con 0x7f2700145b00 2024-05-11T12:14:44.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.326+0000 7f2705800640 1 -- 172.21.15.167:0/4203565933 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2700145b00 msgr2=0x7f270006f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:44.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.326+0000 7f2705800640 1 --2- 172.21.15.167:0/4203565933 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2700145b00 0x7f270006f2c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f26f0009870 tx=0x7f26f002a1e0 comp rx=0 tx=0).stop 2024-05-11T12:14:44.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.326+0000 7f2705800640 1 -- 172.21.15.167:0/4203565933 shutdown_connections 2024-05-11T12:14:44.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.326+0000 7f2705800640 1 --2- 172.21.15.167:0/4203565933 >> v2:172.21.15.204:3300/0 conn(0x7f2700146770 0x7f27000ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:44.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.326+0000 7f2705800640 1 --2- 172.21.15.167:0/4203565933 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2700145b00 0x7f270006f2c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:44.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.326+0000 7f2705800640 1 -- 172.21.15.167:0/4203565933 >> 172.21.15.167:0/4203565933 conn(0x7f27000b3460 msgr2=0x7f27000d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:44.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.326+0000 7f2705800640 1 -- 172.21.15.167:0/4203565933 shutdown_connections 2024-05-11T12:14:44.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.326+0000 7f2705800640 1 -- 172.21.15.167:0/4203565933 wait complete. 2024-05-11T12:14:44.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.328+0000 7f2705800640 1 Processor -- start 2024-05-11T12:14:44.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.328+0000 7f2705800640 1 -- start start 2024-05-11T12:14:44.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.329+0000 7f2705800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27000cef10 0x7f270015c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:44.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.329+0000 7f2705800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2700145b00 0x7f2700072940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:44.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.329+0000 7f2705800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2700146770 0x7f27000b0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:44.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.329+0000 7f2705800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f270006f980 con 0x7f2700146770 2024-05-11T12:14:44.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.329+0000 7f2705800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f270006f800 con 0x7f2700145b00 2024-05-11T12:14:44.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.329+0000 7f2705800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f270006cd70 con 0x7f27000cef10 2024-05-11T12:14:44.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.329+0000 7f2704e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2700146770 0x7f27000b0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:44.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.329+0000 7f2704e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2700146770 0x7f27000b0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:47662/0 (socket says 172.21.15.167:47662) 2024-05-11T12:14:44.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.329+0000 7f2704e00640 1 -- 172.21.15.167:0/322356994 learned_addr learned my addr 172.21.15.167:0/322356994 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:44.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.330+0000 7f26ffe00640 1 --2- 172.21.15.167:0/322356994 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27000cef10 0x7f270015c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:44.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.330+0000 7f26ff400640 1 --2- 172.21.15.167:0/322356994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2700145b00 0x7f2700072940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:44.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.330+0000 7f2704e00640 1 -- 172.21.15.167:0/322356994 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27000cef10 msgr2=0x7f270015c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:44.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.330+0000 7f2704e00640 1 --2- 172.21.15.167:0/322356994 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27000cef10 0x7f270015c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:44.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.330+0000 7f2704e00640 1 -- 172.21.15.167:0/322356994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2700145b00 msgr2=0x7f2700072940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:44.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.330+0000 7f2704e00640 1 --2- 172.21.15.167:0/322356994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2700145b00 0x7f2700072940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:44.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.330+0000 7f2704e00640 1 -- 172.21.15.167:0/322356994 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2700073270 con 0x7f2700146770 2024-05-11T12:14:44.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.330+0000 7f2704e00640 1 --2- 172.21.15.167:0/322356994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2700146770 0x7f27000b0e60 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f26f800c7a0 tx=0x7f26f800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:44.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.330+0000 7f26f7e00640 1 -- 172.21.15.167:0/322356994 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f8003b70 con 0x7f2700146770 2024-05-11T12:14:44.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.331+0000 7f26f7e00640 1 -- 172.21.15.167:0/322356994 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26f8003d10 con 0x7f2700146770 2024-05-11T12:14:44.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.331+0000 7f26f7e00640 1 -- 172.21.15.167:0/322356994 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f800aa70 con 0x7f2700146770 2024-05-11T12:14:44.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.331+0000 7f2705800640 1 -- 172.21.15.167:0/322356994 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27000735b0 con 0x7f2700146770 2024-05-11T12:14:44.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.331+0000 7f2705800640 1 -- 172.21.15.167:0/322356994 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27001f4e30 con 0x7f2700146770 2024-05-11T12:14:44.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.332+0000 7f2705800640 1 -- 172.21.15.167:0/322356994 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27000694b0 con 0x7f2700146770 2024-05-11T12:14:44.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.333+0000 7f26f7e00640 1 -- 172.21.15.167:0/322356994 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f26f800ac10 con 0x7f2700146770 2024-05-11T12:14:44.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.334+0000 7f26f7e00640 1 --2- 172.21.15.167:0/322356994 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f26dc0762d0 0x7f26dc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:44.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.334+0000 7f26f7e00640 1 -- 172.21.15.167:0/322356994 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6101+0+0 (secure 0 0 0) 0x7f26f8097790 con 0x7f2700146770 2024-05-11T12:14:44.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.334+0000 7f26ffe00640 1 --2- 172.21.15.167:0/322356994 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f26dc0762d0 0x7f26dc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:44.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.335+0000 7f26ffe00640 1 --2- 172.21.15.167:0/322356994 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f26dc0762d0 0x7f26dc078790 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f26f0034000 tx=0x7f26f002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:44.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.338+0000 7f26f7e00640 1 -- 172.21.15.167:0/322356994 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f27000735b0 con 0x7f2700146770 2024-05-11T12:14:44.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.567+0000 7f2705800640 1 -- 172.21.15.167:0/322356994 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27000d0c70 con 0x7f2700146770 2024-05-11T12:14:44.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.568+0000 7f26f7e00640 1 -- 172.21.15.167:0/322356994 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5195 (secure 0 0 0) 0x7f26f8060ec0 con 0x7f2700146770 2024-05-11T12:14:44.568 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:44.568 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":62,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5033,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/4026834899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4026834899},{"type":"v1","addr":"172.21.15.204:6834","nonce":4026834899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5036,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1440200187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":1440200187},{"type":"v1","addr":"172.21.15.204:6835","nonce":1440200187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":6636,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1570676224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1570676224},{"type":"v1","addr":"172.21.15.167:6835","nonce":1570676224}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:14:33.067226+0000","modified":"2024-05-11T12:14:36.130783+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6639},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6639":{"gid":6639,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/3493868276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3493868276},{"type":"v1","addr":"172.21.15.167:6837","nonce":3493868276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6639,"qdb_cluster":[6639]},"id":7}]} 2024-05-11T12:14:44.568 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 62 2024-05-11T12:14:44.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.571+0000 7f2705800640 1 -- 172.21.15.167:0/322356994 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f26dc0762d0 msgr2=0x7f26dc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:44.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.571+0000 7f2705800640 1 --2- 172.21.15.167:0/322356994 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f26dc0762d0 0x7f26dc078790 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f26f0034000 tx=0x7f26f002e000 comp rx=0 tx=0).stop 2024-05-11T12:14:44.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.571+0000 7f2705800640 1 -- 172.21.15.167:0/322356994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2700146770 msgr2=0x7f27000b0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:44.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.571+0000 7f2705800640 1 --2- 172.21.15.167:0/322356994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2700146770 0x7f27000b0e60 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f26f800c7a0 tx=0x7f26f800cc60 comp rx=0 tx=0).stop 2024-05-11T12:14:44.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.571+0000 7f2705800640 1 -- 172.21.15.167:0/322356994 shutdown_connections 2024-05-11T12:14:44.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.571+0000 7f2705800640 1 --2- 172.21.15.167:0/322356994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2700146770 0x7f27000b0e60 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:44.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.571+0000 7f2705800640 1 --2- 172.21.15.167:0/322356994 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f26dc0762d0 0x7f26dc078790 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:44.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.571+0000 7f2705800640 1 --2- 172.21.15.167:0/322356994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2700145b00 0x7f2700072940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:44.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.571+0000 7f2705800640 1 --2- 172.21.15.167:0/322356994 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27000cef10 0x7f270015c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:44.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.571+0000 7f2705800640 1 -- 172.21.15.167:0/322356994 >> 172.21.15.167:0/322356994 conn(0x7f27000b3460 msgr2=0x7f27000b4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:44.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.571+0000 7f2705800640 1 -- 172.21.15.167:0/322356994 shutdown_connections 2024-05-11T12:14:44.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:44.571+0000 7f2705800640 1 -- 172.21.15.167:0/322356994 wait complete. 2024-05-11T12:14:44.990 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 62, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-11T12:14:33.067226+0000', 'modified': '2024-05-11T12:14:36.130783+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6639}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6639': {'gid': 6639, 'name': 'c', 'rank': 0, 'incarnation': 58, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.167:6837/3493868276', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6836', 'nonce': 3493868276}, {'type': 'v1', 'addr': '172.21.15.167:6837', 'nonce': 3493868276}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6639, 'qdb_cluster': [6639]} 2024-05-11T12:14:44.990 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-11T12:14:44.990 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-11T12:14:45.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.202+0000 7fbb0ca00640 1 Processor -- start 2024-05-11T12:14:45.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.203+0000 7fbb0ca00640 1 -- start start 2024-05-11T12:14:45.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.204+0000 7fbb0ca00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fbb081625d0 0x7fbb080d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:45.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.204+0000 7fbb0ca00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbb080b8390 0x7fbb080b8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:45.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.204+0000 7fbb0ca00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb080d3c00 0x7fbb080d4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:45.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.204+0000 7fbb0ca00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fbb080d2e40 con 0x7fbb080b8390 2024-05-11T12:14:45.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.204+0000 7fbb0ca00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fbb080d3120 con 0x7fbb081625d0 2024-05-11T12:14:45.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.204+0000 7fbb0ca00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbb080d3400 con 0x7fbb080d3c00 2024-05-11T12:14:45.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.205+0000 7fbb06a00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbb080b8390 0x7fbb080b8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:45.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.205+0000 7fbb06a00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbb080b8390 0x7fbb080b8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:47666/0 (socket says 172.21.15.167:47666) 2024-05-11T12:14:45.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.205+0000 7fbb06a00640 1 -- 172.21.15.167:0/449036560 learned_addr learned my addr 172.21.15.167:0/449036560 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:45.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.205+0000 7fbb07e00640 1 --2- 172.21.15.167:0/449036560 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb080d3c00 0x7fbb080d4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:45.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.205+0000 7fbb07400640 1 --2- 172.21.15.167:0/449036560 >> v2:172.21.15.204:3300/0 conn(0x7fbb081625d0 0x7fbb080d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:45.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.205+0000 7fbb06a00640 1 -- 172.21.15.167:0/449036560 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb080d3c00 msgr2=0x7fbb080d4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:45.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.205+0000 7fbb06a00640 1 --2- 172.21.15.167:0/449036560 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb080d3c00 0x7fbb080d4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:45.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.205+0000 7fbb06a00640 1 -- 172.21.15.167:0/449036560 >> v2:172.21.15.204:3300/0 conn(0x7fbb081625d0 msgr2=0x7fbb080d0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:45.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.205+0000 7fbb06a00640 1 --2- 172.21.15.167:0/449036560 >> v2:172.21.15.204:3300/0 conn(0x7fbb081625d0 0x7fbb080d0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:45.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.205+0000 7fbb06a00640 1 -- 172.21.15.167:0/449036560 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb080d36e0 con 0x7fbb080b8390 2024-05-11T12:14:45.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.206+0000 7fbb06a00640 1 --2- 172.21.15.167:0/449036560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb080b8390 0x7fbb080b8820 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fbb080b7c10 tx=0x7fbaf802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c9ea0927207ce0b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:45.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.206+0000 7fbb06000640 1 -- 172.21.15.167:0/449036560 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf8036070 con 0x7fbb080b8390 2024-05-11T12:14:45.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.207+0000 7fbb06000640 1 -- 172.21.15.167:0/449036560 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaf8031600 con 0x7fbb080b8390 2024-05-11T12:14:45.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.207+0000 7fbb06000640 1 -- 172.21.15.167:0/449036560 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf80319d0 con 0x7fbb080b8390 2024-05-11T12:14:45.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.207+0000 7fbb0ca00640 1 -- 172.21.15.167:0/449036560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb080b8390 msgr2=0x7fbb080b8820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:45.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.207+0000 7fbb0ca00640 1 --2- 172.21.15.167:0/449036560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb080b8390 0x7fbb080b8820 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fbb080b7c10 tx=0x7fbaf802a6a0 comp rx=0 tx=0).stop 2024-05-11T12:14:45.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.207+0000 7fbb0ca00640 1 -- 172.21.15.167:0/449036560 shutdown_connections 2024-05-11T12:14:45.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.207+0000 7fbb0ca00640 1 --2- 172.21.15.167:0/449036560 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb080d3c00 0x7fbb080d4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:45.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.207+0000 7fbb0ca00640 1 --2- 172.21.15.167:0/449036560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb080b8390 0x7fbb080b8820 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:45.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.207+0000 7fbb0ca00640 1 --2- 172.21.15.167:0/449036560 >> v2:172.21.15.204:3300/0 conn(0x7fbb081625d0 0x7fbb080d0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:45.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.207+0000 7fbb0ca00640 1 -- 172.21.15.167:0/449036560 >> 172.21.15.167:0/449036560 conn(0x7fbb080d50f0 msgr2=0x7fbb080d5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:45.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.207+0000 7fbb0ca00640 1 -- 172.21.15.167:0/449036560 shutdown_connections 2024-05-11T12:14:45.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.207+0000 7fbb0ca00640 1 -- 172.21.15.167:0/449036560 wait complete. 2024-05-11T12:14:45.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.209+0000 7fbb0ca00640 1 Processor -- start 2024-05-11T12:14:45.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.209+0000 7fbb0ca00640 1 -- start start 2024-05-11T12:14:45.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.210+0000 7fbb0ca00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb080b8390 0x7fbb08159900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:45.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.210+0000 7fbb0ca00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb080d3c00 0x7fbb0814f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:45.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.210+0000 7fbb0ca00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb081625d0 0x7fbb080b8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:45.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.210+0000 7fbb0ca00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbb08162b30 con 0x7fbb081625d0 2024-05-11T12:14:45.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.210+0000 7fbb0ca00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbb081629b0 con 0x7fbb080b8390 2024-05-11T12:14:45.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.210+0000 7fbb0ca00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbb08162d50 con 0x7fbb080d3c00 2024-05-11T12:14:45.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.211+0000 7fbb07e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb081625d0 0x7fbb080b8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:45.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.211+0000 7fbb07e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb081625d0 0x7fbb080b8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:47674/0 (socket says 172.21.15.167:47674) 2024-05-11T12:14:45.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.211+0000 7fbb07e00640 1 -- 172.21.15.167:0/2233589479 learned_addr learned my addr 172.21.15.167:0/2233589479 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:45.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.211+0000 7fbb06a00640 1 --2- 172.21.15.167:0/2233589479 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb080d3c00 0x7fbb0814f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:45.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.211+0000 7fbb07e00640 1 -- 172.21.15.167:0/2233589479 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb080d3c00 msgr2=0x7fbb0814f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:45.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.211+0000 7fbb07400640 1 --2- 172.21.15.167:0/2233589479 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb080b8390 0x7fbb08159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:45.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.211+0000 7fbb07e00640 1 --2- 172.21.15.167:0/2233589479 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb080d3c00 0x7fbb0814f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:45.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.211+0000 7fbb07e00640 1 -- 172.21.15.167:0/2233589479 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb080b8390 msgr2=0x7fbb08159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:45.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.211+0000 7fbb07e00640 1 --2- 172.21.15.167:0/2233589479 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb080b8390 0x7fbb08159900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:45.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.211+0000 7fbb07e00640 1 -- 172.21.15.167:0/2233589479 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb0815a3f0 con 0x7fbb081625d0 2024-05-11T12:14:45.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.211+0000 7fbb07400640 1 --2- 172.21.15.167:0/2233589479 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb080b8390 0x7fbb08159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:45.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.211+0000 7fbb07e00640 1 --2- 172.21.15.167:0/2233589479 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb081625d0 0x7fbb080b8dc0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fbb0000d800 tx=0x7fbb0000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:45.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.212+0000 7fbaff400640 1 -- 172.21.15.167:0/2233589479 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb00018020 con 0x7fbb081625d0 2024-05-11T12:14:45.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.212+0000 7fbaff400640 1 -- 172.21.15.167:0/2233589479 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb000096c0 con 0x7fbb081625d0 2024-05-11T12:14:45.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.212+0000 7fbb0ca00640 1 -- 172.21.15.167:0/2233589479 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb0815a6d0 con 0x7fbb081625d0 2024-05-11T12:14:45.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.212+0000 7fbaff400640 1 -- 172.21.15.167:0/2233589479 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb00009a90 con 0x7fbb081625d0 2024-05-11T12:14:45.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.212+0000 7fbb0ca00640 1 -- 172.21.15.167:0/2233589479 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb0815ac90 con 0x7fbb081625d0 2024-05-11T12:14:45.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.215+0000 7fbaff400640 1 -- 172.21.15.167:0/2233589479 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbb0001b020 con 0x7fbb081625d0 2024-05-11T12:14:45.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.216+0000 7fbaff400640 1 --2- 172.21.15.167:0/2233589479 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbae40762d0 0x7fbae4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:45.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.216+0000 7fbaff400640 1 -- 172.21.15.167:0/2233589479 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6101+0+0 (secure 0 0 0) 0x7fbb000979f0 con 0x7fbb081625d0 2024-05-11T12:14:45.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.216+0000 7fbb07400640 1 --2- 172.21.15.167:0/2233589479 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbae40762d0 0x7fbae4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:45.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.216+0000 7fbb10fcd740 1 Processor -- start 2024-05-11T12:14:45.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.217+0000 7fbb10fcd740 1 -- start start 2024-05-11T12:14:45.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.217+0000 7fbb07400640 1 --2- 172.21.15.167:0/2233589479 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbae40762d0 0x7fbae4078790 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbaf000b820 tx=0x7fbaf0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:45.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.217+0000 7fbb10fcd740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c2f94e6a10 0x55c2f94e6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:45.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.217+0000 7fbb10fcd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2f94e18a0 0x55c2f94e5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:45.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.217+0000 7fbb10fcd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2f94e5b50 0x55c2f94e5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:45.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.217+0000 7fbb10fcd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c2f944a790 con 0x55c2f94e18a0 2024-05-11T12:14:45.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.217+0000 7fbb10fcd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c2f93eff20 con 0x55c2f94e5b50 2024-05-11T12:14:45.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.217+0000 7fbb10fcd740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55c2f9446750 con 0x55c2f94e6a10 2024-05-11T12:14:45.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.218+0000 7fbb06a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2f94e18a0 0x55c2f94e5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:45.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.218+0000 7fbb06a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2f94e18a0 0x55c2f94e5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:47688/0 (socket says 172.21.15.167:47688) 2024-05-11T12:14:45.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.218+0000 7fbb06a00640 1 -- 172.21.15.167:0/3406723440 learned_addr learned my addr 172.21.15.167:0/3406723440 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:45.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.218+0000 7fbb07400640 1 --2- 172.21.15.167:0/3406723440 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c2f94e6a10 0x55c2f94e6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:45.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.218+0000 7fbb06a00640 1 -- 172.21.15.167:0/3406723440 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c2f94e6a10 msgr2=0x55c2f94e6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:45.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.218+0000 7fbb06a00640 1 --2- 172.21.15.167:0/3406723440 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c2f94e6a10 0x55c2f94e6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:45.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.218+0000 7fbb06a00640 1 -- 172.21.15.167:0/3406723440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2f94e5b50 msgr2=0x55c2f94e5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:45.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.218+0000 7fbb06a00640 1 --2- 172.21.15.167:0/3406723440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2f94e5b50 0x55c2f94e5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:45.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.218+0000 7fbb06a00640 1 -- 172.21.15.167:0/3406723440 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2f915c2d0 con 0x55c2f94e18a0 2024-05-11T12:14:45.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.218+0000 7fbb06a00640 1 --2- 172.21.15.167:0/3406723440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2f94e18a0 0x55c2f94e5610 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8034040 tx=0x7fbaf802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e7c476fa87cc05d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:45.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.219+0000 7fbaf7400640 1 -- 172.21.15.167:0/3406723440 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf8036040 con 0x55c2f94e18a0 2024-05-11T12:14:45.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.219+0000 7fbaf7400640 1 -- 172.21.15.167:0/3406723440 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaf803c470 con 0x55c2f94e18a0 2024-05-11T12:14:45.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.219+0000 7fbaf7400640 1 -- 172.21.15.167:0/3406723440 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf803c820 con 0x55c2f94e18a0 2024-05-11T12:14:45.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.219+0000 7fbb10fcd740 1 -- 172.21.15.167:0/3406723440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2f94e18a0 msgr2=0x55c2f94e5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:45.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.219+0000 7fbb10fcd740 1 --2- 172.21.15.167:0/3406723440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2f94e18a0 0x55c2f94e5610 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8034040 tx=0x7fbaf802e000 comp rx=0 tx=0).stop 2024-05-11T12:14:45.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.219+0000 7fbb10fcd740 1 -- 172.21.15.167:0/3406723440 shutdown_connections 2024-05-11T12:14:45.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.219+0000 7fbb10fcd740 1 --2- 172.21.15.167:0/3406723440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2f94e5b50 0x55c2f94e5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:45.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.219+0000 7fbb10fcd740 1 --2- 172.21.15.167:0/3406723440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2f94e18a0 0x55c2f94e5610 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:45.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.219+0000 7fbb10fcd740 1 --2- 172.21.15.167:0/3406723440 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c2f94e6a10 0x55c2f94e6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:45.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.219+0000 7fbb10fcd740 1 -- 172.21.15.167:0/3406723440 >> 172.21.15.167:0/3406723440 conn(0x55c2f94e3b10 msgr2=0x55c2f952c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:45.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.219+0000 7fbb10fcd740 1 -- 172.21.15.167:0/3406723440 shutdown_connections 2024-05-11T12:14:45.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.220+0000 7fbb10fcd740 1 -- 172.21.15.167:0/3406723440 wait complete. 2024-05-11T12:14:45.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.220+0000 7fbb10fcd740 1 -- start start 2024-05-11T12:14:45.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.220+0000 7fbb10fcd740 1 Processor -- start 2024-05-11T12:14:45.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.221+0000 7fbb10fcd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2f94e5b50 0x55c2f950a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:45.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.221+0000 7fbb10fcd740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c2f94e6a10 0x55c2f9531ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:45.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.221+0000 7fbb10fcd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2f94f9e70 0x55c2f9533800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:45.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.221+0000 7fbb10fcd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c2f95125a0 con 0x55c2f94f9e70 2024-05-11T12:14:45.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.221+0000 7fbb10fcd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c2f94039c0 con 0x55c2f94e5b50 2024-05-11T12:14:45.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.221+0000 7fbb10fcd740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55c2f9511a20 con 0x55c2f94e6a10 2024-05-11T12:14:45.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.221+0000 7fbb10fcd740 10 client.0 _ll_register_callbacks cb 0x55c2f93d0120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:14:45.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.222+0000 7fbb07e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2f94f9e70 0x55c2f9533800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:45.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.222+0000 7fbb07e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2f94f9e70 0x55c2f9533800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:47694/0 (socket says 172.21.15.167:47694) 2024-05-11T12:14:45.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.222+0000 7fbb07e00640 1 -- 172.21.15.167:0/427163292 learned_addr learned my addr 172.21.15.167:0/427163292 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:45.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.222+0000 7fbb07e00640 1 -- 172.21.15.167:0/427163292 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c2f94e6a10 msgr2=0x55c2f9531ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:45.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.222+0000 7fbb07e00640 1 --2- 172.21.15.167:0/427163292 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c2f94e6a10 0x55c2f9531ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:45.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.222+0000 7fbb07400640 1 --2- 172.21.15.167:0/427163292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2f94e5b50 0x55c2f950a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:45.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.222+0000 7fbb07e00640 1 -- 172.21.15.167:0/427163292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2f94e5b50 msgr2=0x55c2f950a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:45.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.222+0000 7fbb07e00640 1 --2- 172.21.15.167:0/427163292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2f94e5b50 0x55c2f950a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:45.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.222+0000 7fbb07e00640 1 -- 172.21.15.167:0/427163292 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2f92266b0 con 0x55c2f94f9e70 2024-05-11T12:14:45.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.222+0000 7fbb07e00640 1 --2- 172.21.15.167:0/427163292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2f94f9e70 0x55c2f9533800 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fbb0005caa0 tx=0x7fbb00069260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cbeafd9631c871a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:45.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.222+0000 7fbaebe00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-11T12:14:45.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.222+0000 7fbaebe00640 1 -- 172.21.15.167:0/427163292 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb00069560 con 0x55c2f94f9e70 2024-05-11T12:14:45.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.222+0000 7fbaebe00640 1 -- 172.21.15.167:0/427163292 <== mon.0 v2:172.21.15.167:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 6101+0+0 (secure 0 0 0) 0x7fbb0005fc80 con 0x55c2f94f9e70 2024-05-11T12:14:45.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.222+0000 7fbb10fcd740 1 -- 172.21.15.167:0/427163292 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbaf803c470 con 0x55c2f94f9e70 2024-05-11T12:14:45.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.223+0000 7fbaebe00640 1 -- 172.21.15.167:0/427163292 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fbb00095050 con 0x55c2f94f9e70 2024-05-11T12:14:45.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.223+0000 7fbb10fcd740 10 client.6804 fetch_fsmap learned FSMap version 62 2024-05-11T12:14:45.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.223+0000 7fbb10fcd740 1 -- 172.21.15.167:0/427163292 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) -- 0x55c2f95316e0 con 0x55c2f94f9e70 2024-05-11T12:14:45.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.223+0000 7fbaebe00640 1 -- 172.21.15.167:0/427163292 <== mon.0 v2:172.21.15.167:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb0007b030 con 0x55c2f94f9e70 2024-05-11T12:14:45.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.223+0000 7fbaebe00640 1 -- 172.21.15.167:0/427163292 <== mon.0 v2:172.21.15.167:3300/0 5 ==== fsmap(e 62) ==== 3494+0+0 (secure 0 0 0) 0x7fbb00013050 con 0x55c2f94f9e70 2024-05-11T12:14:45.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.223+0000 7fbb10fcd740 10 client.6804 fetch_fsmap finished waiting for FSMap version 62 2024-05-11T12:14:45.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.223+0000 7fbb10fcd740 10 client.6804 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-11T12:14:45.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.223+0000 7fbb10fcd740 20 client.6804 populate_metadata read hostname 'smithi167' 2024-05-11T12:14:45.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.223+0000 7fbb10fcd740 1 --2- 172.21.15.167:0/427163292 >> [v2:172.21.15.167:6836/3493868276,v1:172.21.15.167:6837/3493868276] conn(0x55c2f950dfa0 0x55c2f950e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:45.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.223+0000 7fbb10fcd740 4 client.6804 mds_command: new command op to 6639 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:14:45.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.223+0000 7fbb10fcd740 1 -- 172.21.15.167:0/427163292 --> [v2:172.21.15.167:6836/3493868276,v1:172.21.15.167:6837/3493868276] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2f94583a0 con 0x55c2f950dfa0 2024-05-11T12:14:45.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.224+0000 7fbb07400640 1 --2- 172.21.15.167:0/427163292 >> [v2:172.21.15.167:6836/3493868276,v1:172.21.15.167:6837/3493868276] conn(0x55c2f950dfa0 0x55c2f950e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:45.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.224+0000 7fbaebe00640 10 client.6804 ms_handle_connect on v2:172.21.15.167:6836/3493868276 2024-05-11T12:14:45.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.224+0000 7fbb07400640 1 --2- 172.21.15.167:0/427163292 >> [v2:172.21.15.167:6836/3493868276,v1:172.21.15.167:6837/3493868276] conn(0x55c2f950dfa0 0x55c2f950e380 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2f95a55f55407e server_cookie=c52e526d3394f541 in_seq=0 out_seq=0 2024-05-11T12:14:45.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.225+0000 7fbaebe00640 1 -- 172.21.15.167:0/427163292 <== mds.0 v2:172.21.15.167:6836/3493868276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c2f94583a0 con 0x55c2f950dfa0 2024-05-11T12:14:45.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.225+0000 7fbaebe00640 10 client.6804 handle_command_reply: tid=0 2024-05-11T12:14:45.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.225+0000 7fbb10fcd740 1 client.6804 shutdown 2024-05-11T12:14:45.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.226+0000 7fbb10fcd740 1 -- 172.21.15.167:0/427163292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2f94f9e70 msgr2=0x55c2f9533800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:45.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.226+0000 7fbb10fcd740 1 --2- 172.21.15.167:0/427163292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2f94f9e70 0x55c2f9533800 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fbb0005caa0 tx=0x7fbb00069260 comp rx=0 tx=0).stop 2024-05-11T12:14:45.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.226+0000 7fbb10fcd740 1 -- 172.21.15.167:0/427163292 shutdown_connections 2024-05-11T12:14:45.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.226+0000 7fbb10fcd740 1 --2- 172.21.15.167:0/427163292 >> [v2:172.21.15.167:6836/3493868276,v1:172.21.15.167:6837/3493868276] conn(0x55c2f950dfa0 0x55c2f950e380 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:45.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.226+0000 7fbb10fcd740 1 --2- 172.21.15.167:0/427163292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2f94f9e70 0x55c2f9533800 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:45.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.226+0000 7fbb10fcd740 1 --2- 172.21.15.167:0/427163292 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c2f94e6a10 0x55c2f9531ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:45.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.226+0000 7fbb10fcd740 1 --2- 172.21.15.167:0/427163292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2f94e5b50 0x55c2f950a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:45.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.226+0000 7fbaebe00640 0 client.6804 ms_handle_reset on v2:172.21.15.167:6836/3493868276 2024-05-11T12:14:45.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.226+0000 7fbb10fcd740 1 -- 172.21.15.167:0/427163292 >> 172.21.15.167:0/427163292 conn(0x55c2f94e18a0 msgr2=0x55c2f94e0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:45.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.226+0000 7fbb10fcd740 1 -- 172.21.15.167:0/427163292 shutdown_connections 2024-05-11T12:14:45.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.226+0000 7fbb10fcd740 1 -- 172.21.15.167:0/427163292 wait complete. 2024-05-11T12:14:45.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.226+0000 7fbb10fcd740 20 client.6804 trim_cache size 0 max 16384 2024-05-11T12:14:45.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.246+0000 7fbb10fcd740 1 Processor -- start 2024-05-11T12:14:45.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.246+0000 7fbb10fcd740 1 -- start start 2024-05-11T12:14:45.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.247+0000 7fbb10fcd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2f94e5b50 0x55c2f94e87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:45.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.247+0000 7fbb10fcd740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c2f94e6a10 0x55c2f94e61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:45.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.247+0000 7fbb10fcd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2f94f9e70 0x55c2f94e0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:45.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.247+0000 7fbb10fcd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbb08201d80 con 0x55c2f94e5b50 2024-05-11T12:14:45.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.247+0000 7fbb10fcd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbb08200e30 con 0x55c2f94f9e70 2024-05-11T12:14:45.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.247+0000 7fbb10fcd740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbb081f5a60 con 0x55c2f94e6a10 2024-05-11T12:14:45.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.247+0000 7fbb07400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2f94e5b50 0x55c2f94e87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:45.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.247+0000 7fbb07400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2f94e5b50 0x55c2f94e87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:47698/0 (socket says 172.21.15.167:47698) 2024-05-11T12:14:45.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.247+0000 7fbb07400640 1 -- 172.21.15.167:0/2273502007 learned_addr learned my addr 172.21.15.167:0/2273502007 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:45.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.247+0000 7fbb07e00640 1 --2- 172.21.15.167:0/2273502007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2f94f9e70 0x55c2f94e0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:45.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.247+0000 7fbb06a00640 1 --2- 172.21.15.167:0/2273502007 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c2f94e6a10 0x55c2f94e61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:45.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.247+0000 7fbb07400640 1 -- 172.21.15.167:0/2273502007 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c2f94e6a10 msgr2=0x55c2f94e61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:45.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.247+0000 7fbb07400640 1 --2- 172.21.15.167:0/2273502007 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c2f94e6a10 0x55c2f94e61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:45.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.248+0000 7fbb07400640 1 -- 172.21.15.167:0/2273502007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2f94f9e70 msgr2=0x55c2f94e0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:45.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.248+0000 7fbb07400640 1 --2- 172.21.15.167:0/2273502007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2f94f9e70 0x55c2f94e0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:45.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.248+0000 7fbb07400640 1 -- 172.21.15.167:0/2273502007 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2f950ce70 con 0x55c2f94e5b50 2024-05-11T12:14:45.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.248+0000 7fbb07400640 1 --2- 172.21.15.167:0/2273502007 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2f94e5b50 0x55c2f94e87b0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fbaf000d590 tx=0x7fbaf0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c62756d568dbb64 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:45.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.248+0000 7fbaf6a00640 1 -- 172.21.15.167:0/2273502007 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf002a020 con 0x55c2f94e5b50 2024-05-11T12:14:45.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.248+0000 7fbaf6a00640 1 -- 172.21.15.167:0/2273502007 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaf0012b40 con 0x55c2f94e5b50 2024-05-11T12:14:45.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.248+0000 7fbaf6a00640 1 -- 172.21.15.167:0/2273502007 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf000dea0 con 0x55c2f94e5b50 2024-05-11T12:14:45.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.249+0000 7fbb10fcd740 1 -- 172.21.15.167:0/2273502007 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2f94e5b50 msgr2=0x55c2f94e87b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:45.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.249+0000 7fbb10fcd740 1 --2- 172.21.15.167:0/2273502007 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2f94e5b50 0x55c2f94e87b0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fbaf000d590 tx=0x7fbaf0013000 comp rx=0 tx=0).stop 2024-05-11T12:14:45.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.249+0000 7fbb10fcd740 1 -- 172.21.15.167:0/2273502007 shutdown_connections 2024-05-11T12:14:45.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.249+0000 7fbb10fcd740 1 --2- 172.21.15.167:0/2273502007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2f94f9e70 0x55c2f94e0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:45.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.249+0000 7fbb10fcd740 1 --2- 172.21.15.167:0/2273502007 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c2f94e6a10 0x55c2f94e61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:45.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.249+0000 7fbb10fcd740 1 --2- 172.21.15.167:0/2273502007 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2f94e5b50 0x55c2f94e87b0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:45.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.249+0000 7fbb10fcd740 1 -- 172.21.15.167:0/2273502007 >> 172.21.15.167:0/2273502007 conn(0x55c2f94e18a0 msgr2=0x55c2f94dcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:45.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.249+0000 7fbb10fcd740 1 -- 172.21.15.167:0/2273502007 shutdown_connections 2024-05-11T12:14:45.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.249+0000 7fbb10fcd740 1 -- 172.21.15.167:0/2273502007 wait complete. 2024-05-11T12:14:45.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.250+0000 7fbb10fcd740 1 -- start start 2024-05-11T12:14:45.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.250+0000 7fbb10fcd740 1 Processor -- start 2024-05-11T12:14:45.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.251+0000 7fbb10fcd740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c2f94e5b50 0x55c2f959bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:45.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.251+0000 7fbb10fcd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2f94e6a10 0x55c2f9509e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:45.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.251+0000 7fbb10fcd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2f94f9e70 0x55c2f95b5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:45.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.251+0000 7fbb10fcd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c2f9508ba0 con 0x55c2f94e6a10 2024-05-11T12:14:45.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.251+0000 7fbb10fcd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbb082007e0 con 0x55c2f94f9e70 2024-05-11T12:14:45.253 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.251+0000 7fbb10fcd740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55c2f9532610 con 0x55c2f94e5b50 2024-05-11T12:14:45.253 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.251+0000 7fbb10fcd740 10 client.0 _ll_register_callbacks cb 0x7fbb0005d070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:14:45.253 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.251+0000 7fbb06a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2f94e6a10 0x55c2f9509e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:45.253 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.251+0000 7fbb06a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2f94e6a10 0x55c2f9509e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:47700/0 (socket says 172.21.15.167:47700) 2024-05-11T12:14:45.253 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.251+0000 7fbb06a00640 1 -- 172.21.15.167:0/4292989565 learned_addr learned my addr 172.21.15.167:0/4292989565 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:45.253 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.251+0000 7fbb07400640 1 --2- 172.21.15.167:0/4292989565 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c2f94e5b50 0x55c2f959bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:45.253 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.251+0000 7fbb06a00640 1 -- 172.21.15.167:0/4292989565 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c2f94e5b50 msgr2=0x55c2f959bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:45.253 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.251+0000 7fbb06a00640 1 --2- 172.21.15.167:0/4292989565 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c2f94e5b50 0x55c2f959bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:45.253 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.251+0000 7fbb07e00640 1 --2- 172.21.15.167:0/4292989565 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2f94f9e70 0x55c2f95b5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:45.253 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.251+0000 7fbb06a00640 1 -- 172.21.15.167:0/4292989565 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2f94f9e70 msgr2=0x55c2f95b5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:45.253 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.251+0000 7fbb06a00640 1 --2- 172.21.15.167:0/4292989565 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2f94f9e70 0x55c2f95b5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:45.253 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.251+0000 7fbb06a00640 1 -- 172.21.15.167:0/4292989565 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2f9512380 con 0x55c2f94e6a10 2024-05-11T12:14:45.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.252+0000 7fbb06a00640 1 --2- 172.21.15.167:0/4292989565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2f94e6a10 0x55c2f9509e90 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fbaf802ac90 tx=0x7fbaf8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4504812341a1997 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:45.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.252+0000 7fbaebe00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-11T12:14:45.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.252+0000 7fbaebe00640 1 -- 172.21.15.167:0/4292989565 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf804a070 con 0x55c2f94e6a10 2024-05-11T12:14:45.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.252+0000 7fbaebe00640 1 -- 172.21.15.167:0/4292989565 <== mon.0 v2:172.21.15.167:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 6101+0+0 (secure 0 0 0) 0x7fbaf8004250 con 0x55c2f94e6a10 2024-05-11T12:14:45.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.252+0000 7fbb10fcd740 1 -- 172.21.15.167:0/4292989565 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbaf0012b40 con 0x55c2f94e6a10 2024-05-11T12:14:45.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.252+0000 7fbaebe00640 1 -- 172.21.15.167:0/4292989565 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fbaf8054050 con 0x55c2f94e6a10 2024-05-11T12:14:45.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.252+0000 7fbb10fcd740 10 client.6810 fetch_fsmap learned FSMap version 62 2024-05-11T12:14:45.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.252+0000 7fbb10fcd740 1 -- 172.21.15.167:0/4292989565 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) -- 0x55c2f9509970 con 0x55c2f94e6a10 2024-05-11T12:14:45.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.252+0000 7fbaebe00640 1 -- 172.21.15.167:0/4292989565 <== mon.0 v2:172.21.15.167:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaf0012b40 con 0x55c2f94e6a10 2024-05-11T12:14:45.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.253+0000 7fbaebe00640 1 -- 172.21.15.167:0/4292989565 <== mon.0 v2:172.21.15.167:3300/0 5 ==== fsmap(e 62) ==== 3494+0+0 (secure 0 0 0) 0x7fbaf8057020 con 0x55c2f94e6a10 2024-05-11T12:14:45.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.253+0000 7fbb10fcd740 10 client.6810 fetch_fsmap finished waiting for FSMap version 62 2024-05-11T12:14:45.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.253+0000 7fbb10fcd740 10 client.6810 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-11T12:14:45.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.253+0000 7fbb10fcd740 20 client.6810 populate_metadata read hostname 'smithi167' 2024-05-11T12:14:45.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.253+0000 7fbb10fcd740 1 --2- 172.21.15.167:0/4292989565 >> [v2:172.21.15.167:6836/3493868276,v1:172.21.15.167:6837/3493868276] conn(0x55c2f94e09f0 0x55c2f95062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:45.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.253+0000 7fbb10fcd740 4 client.6810 mds_command: new command op to 6639 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-11T12:14:45.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.253+0000 7fbb10fcd740 1 -- 172.21.15.167:0/4292989565 --> [v2:172.21.15.167:6836/3493868276,v1:172.21.15.167:6837/3493868276] -- command(tid 0: {"prefix": "status"}) -- 0x55c2f9452210 con 0x55c2f94e09f0 2024-05-11T12:14:45.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.253+0000 7fbb07400640 1 --2- 172.21.15.167:0/4292989565 >> [v2:172.21.15.167:6836/3493868276,v1:172.21.15.167:6837/3493868276] conn(0x55c2f94e09f0 0x55c2f95062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:45.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.253+0000 7fbb07400640 1 --2- 172.21.15.167:0/4292989565 >> [v2:172.21.15.167:6836/3493868276,v1:172.21.15.167:6837/3493868276] conn(0x55c2f94e09f0 0x55c2f95062d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25f9c70b4b0ed46e server_cookie=8242af6dc70d8c1 in_seq=0 out_seq=0 2024-05-11T12:14:45.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.253+0000 7fbaebe00640 10 client.6810 ms_handle_connect on v2:172.21.15.167:6836/3493868276 2024-05-11T12:14:45.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.254+0000 7fbaebe00640 1 -- 172.21.15.167:0/4292989565 <== mds.0 v2:172.21.15.167:6836/3493868276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+319 (crc 0 0 0) 0x55c2f9452210 con 0x55c2f94e09f0 2024-05-11T12:14:45.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.254+0000 7fbaebe00640 10 client.6810 handle_command_reply: tid=0 2024-05-11T12:14:45.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.254+0000 7fbb10fcd740 1 client.6810 shutdown 2024-05-11T12:14:45.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.254+0000 7fbb10fcd740 1 -- 172.21.15.167:0/4292989565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2f94e6a10 msgr2=0x55c2f9509e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:45.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.254+0000 7fbb10fcd740 1 --2- 172.21.15.167:0/4292989565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2f94e6a10 0x55c2f9509e90 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fbaf802ac90 tx=0x7fbaf8004a40 comp rx=0 tx=0).stop 2024-05-11T12:14:45.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.254+0000 7fbb10fcd740 1 -- 172.21.15.167:0/4292989565 shutdown_connections 2024-05-11T12:14:45.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.254+0000 7fbb10fcd740 1 --2- 172.21.15.167:0/4292989565 >> [v2:172.21.15.167:6836/3493868276,v1:172.21.15.167:6837/3493868276] conn(0x55c2f94e09f0 0x55c2f95062d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:45.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.254+0000 7fbb10fcd740 1 --2- 172.21.15.167:0/4292989565 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2f94f9e70 0x55c2f95b5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:45.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.254+0000 7fbb10fcd740 1 --2- 172.21.15.167:0/4292989565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2f94e6a10 0x55c2f9509e90 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:45.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.254+0000 7fbb10fcd740 1 --2- 172.21.15.167:0/4292989565 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c2f94e5b50 0x55c2f959bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:45.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.254+0000 7fbaebe00640 0 client.6810 ms_handle_reset on v2:172.21.15.167:6836/3493868276 2024-05-11T12:14:45.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.254+0000 7fbb10fcd740 1 -- 172.21.15.167:0/4292989565 >> 172.21.15.167:0/4292989565 conn(0x55c2f94e18a0 msgr2=0x55c2f95113f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:45.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.254+0000 7fbb10fcd740 1 -- 172.21.15.167:0/4292989565 shutdown_connections 2024-05-11T12:14:45.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.254+0000 7fbb10fcd740 1 -- 172.21.15.167:0/4292989565 wait complete. 2024-05-11T12:14:45.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.254+0000 7fbb10fcd740 20 client.6810 trim_cache size 0 max 16384 2024-05-11T12:14:45.259 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T12:14:45.259 INFO:teuthology.orchestra.run.smithi167.stdout: "cluster_fsid": "5f32d344-7289-4c27-881b-1533419ab678", 2024-05-11T12:14:45.259 INFO:teuthology.orchestra.run.smithi167.stdout: "whoami": 0, 2024-05-11T12:14:45.259 INFO:teuthology.orchestra.run.smithi167.stdout: "id": 6639, 2024-05-11T12:14:45.259 INFO:teuthology.orchestra.run.smithi167.stdout: "want_state": "up:active", 2024-05-11T12:14:45.259 INFO:teuthology.orchestra.run.smithi167.stdout: "state": "up:active", 2024-05-11T12:14:45.259 INFO:teuthology.orchestra.run.smithi167.stdout: "fs_name": "cephfs", 2024-05-11T12:14:45.259 INFO:teuthology.orchestra.run.smithi167.stdout: "rank_uptime": 11.652147368, 2024-05-11T12:14:45.259 INFO:teuthology.orchestra.run.smithi167.stdout: "mdsmap_epoch": 62, 2024-05-11T12:14:45.259 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_epoch": 97, 2024-05-11T12:14:45.260 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_epoch_barrier": 97, 2024-05-11T12:14:45.260 INFO:teuthology.orchestra.run.smithi167.stdout: "uptime": 16.148217309 2024-05-11T12:14:45.260 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T12:14:45.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.255+0000 7fbb0ca00640 1 -- 172.21.15.167:0/2233589479 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbae40762d0 msgr2=0x7fbae4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:45.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.255+0000 7fbb0ca00640 1 --2- 172.21.15.167:0/2233589479 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbae40762d0 0x7fbae4078790 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbaf000b820 tx=0x7fbaf0008040 comp rx=0 tx=0).stop 2024-05-11T12:14:45.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.255+0000 7fbb0ca00640 1 -- 172.21.15.167:0/2233589479 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb081625d0 msgr2=0x7fbb080b8dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:45.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.255+0000 7fbb0ca00640 1 --2- 172.21.15.167:0/2233589479 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb081625d0 0x7fbb080b8dc0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fbb0000d800 tx=0x7fbb0000dcc0 comp rx=0 tx=0).stop 2024-05-11T12:14:45.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.255+0000 7fbb0ca00640 1 -- 172.21.15.167:0/2233589479 shutdown_connections 2024-05-11T12:14:45.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.255+0000 7fbb0ca00640 1 --2- 172.21.15.167:0/2233589479 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb081625d0 0x7fbb080b8dc0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:45.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.255+0000 7fbb0ca00640 1 --2- 172.21.15.167:0/2233589479 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb080d3c00 0x7fbb0814f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:45.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.255+0000 7fbb0ca00640 1 --2- 172.21.15.167:0/2233589479 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbae40762d0 0x7fbae4078790 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:45.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.255+0000 7fbb0ca00640 1 --2- 172.21.15.167:0/2233589479 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb080b8390 0x7fbb08159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:45.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.255+0000 7fbb0ca00640 1 -- 172.21.15.167:0/2233589479 >> 172.21.15.167:0/2233589479 conn(0x7fbb080d50f0 msgr2=0x7fbb080b5c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:45.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.255+0000 7fbb0ca00640 1 -- 172.21.15.167:0/2233589479 shutdown_connections 2024-05-11T12:14:45.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.255+0000 7fbb0ca00640 1 -- 172.21.15.167:0/2233589479 wait complete. 2024-05-11T12:14:45.274 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-11T12:14:45.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.475+0000 7f3dd4400640 1 Processor -- start 2024-05-11T12:14:45.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.475+0000 7f3dd4400640 1 -- start start 2024-05-11T12:14:45.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.476+0000 7f3dd4400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3dcc06d460 0x7f3dcc06d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:45.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.476+0000 7f3dd4400640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dcc15df30 con 0x7f3dcc0b3ea0 2024-05-11T12:14:45.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.477+0000 7f3dd4400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dcc15e210 con 0x7f3dcc157750 2024-05-11T12:14:45.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.477+0000 7f3dd4400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f3dcc06ddf0 con 0x7f3dcc06d460 2024-05-11T12:14:45.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.477+0000 7f3dd1000640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f3dcc0b3ea0 0x7f3dcc161490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:44022/0 (socket says 172.21.15.167:44022) 2024-05-11T12:14:45.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.477+0000 7f3dd1000640 1 -- 172.21.15.167:0/4124969627 learned_addr learned my addr 172.21.15.167:0/4124969627 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:14:45.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.477+0000 7f3dd2400640 1 --2- 172.21.15.167:0/4124969627 >> v2:172.21.15.204:3300/0 conn(0x7f3dcc06d460 0x7f3dcc06d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:45.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.478+0000 7f3dcbe00640 1 -- 172.21.15.167:0/4124969627 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3629587934 0 0) 0x7f3dcc15df30 con 0x7f3dcc0b3ea0 2024-05-11T12:14:45.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.478+0000 7f3dcbe00640 1 -- 172.21.15.167:0/4124969627 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3db4003520 con 0x7f3dcc0b3ea0 2024-05-11T12:14:45.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.478+0000 7f3dcbe00640 1 -- 172.21.15.167:0/4124969627 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 495895233 0 0) 0x7f3db4003520 con 0x7f3dcc0b3ea0 2024-05-11T12:14:45.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.478+0000 7f3dcbe00640 1 -- 172.21.15.167:0/4124969627 >> v2:172.21.15.204:3300/0 conn(0x7f3dcc06d460 msgr2=0x7f3dcc06d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:45.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.478+0000 7f3dcbe00640 1 --2- 172.21.15.167:0/4124969627 >> v2:172.21.15.204:3300/0 conn(0x7f3dcc06d460 0x7f3dcc06d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:45.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.478+0000 7f3dcbe00640 1 -- 172.21.15.167:0/4124969627 >> v1:172.21.15.204:6789/0 conn(0x7f3dcc157750 legacy=0x7f3dcc15d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:45.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.478+0000 7f3dcbe00640 1 -- 172.21.15.167:0/4124969627 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dcc15e4f0 con 0x7f3dcc0b3ea0 2024-05-11T12:14:45.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.478+0000 7f3dcbe00640 1 -- 172.21.15.167:0/4124969627 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f3db80016f0 con 0x7f3dcc0b3ea0 2024-05-11T12:14:45.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.478+0000 7f3dcbe00640 1 -- 172.21.15.167:0/4124969627 >> v1:172.21.15.167:6789/0 conn(0x7f3dcc0b3ea0 legacy=0x7f3dcc161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:45.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.478+0000 7f3dcbe00640 1 --2- 172.21.15.167:0/4124969627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3db4003aa0 0x7f3db4003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:45.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.478+0000 7f3dcbe00640 1 --2- 172.21.15.167:0/4124969627 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3db4004800 0x7f3db4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:45.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.478+0000 7f3dcbe00640 1 --2- 172.21.15.167:0/4124969627 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3db4005210 0x7f3db400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:45.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.478+0000 7f3dcbe00640 1 -- 172.21.15.167:0/4124969627 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3dcc06ddf0 con 0x7f3db4003aa0 2024-05-11T12:14:45.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.478+0000 7f3dcbe00640 1 -- 172.21.15.167:0/4124969627 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3db400c160 con 0x7f3db4004800 2024-05-11T12:14:45.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.478+0000 7f3dcbe00640 1 -- 172.21.15.167:0/4124969627 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3db400c360 con 0x7f3db4005210 2024-05-11T12:14:45.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.478+0000 7f3dd1a00640 1 --2- 172.21.15.167:0/4124969627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3db4003aa0 0x7f3db4003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:45.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.479+0000 7f3dd1000640 1 --2- 172.21.15.167:0/4124969627 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3db4004800 0x7f3db4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:45.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.479+0000 7f3dd1a00640 1 -- 172.21.15.167:0/4124969627 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3db4005210 msgr2=0x7f3db400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:45.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.479+0000 7f3dd1a00640 1 --2- 172.21.15.167:0/4124969627 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3db4005210 0x7f3db400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:45.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.479+0000 7f3dd1a00640 1 -- 172.21.15.167:0/4124969627 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3db4004800 msgr2=0x7f3db4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:45.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.479+0000 7f3dd1a00640 1 --2- 172.21.15.167:0/4124969627 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3db4004800 0x7f3db4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:45.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.479+0000 7f3dd1a00640 1 -- 172.21.15.167:0/4124969627 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3db4003520 con 0x7f3db4003aa0 2024-05-11T12:14:45.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.479+0000 7f3dd2400640 1 -- 172.21.15.167:0/4124969627 reap_dead start 2024-05-11T12:14:45.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.479+0000 7f3dd1000640 1 --2- 172.21.15.167:0/4124969627 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3db4004800 0x7f3db4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:45.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.480+0000 7f3dd1a00640 1 --2- 172.21.15.167:0/4124969627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3db4003aa0 0x7f3db4003ec0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0001760 tx=0x7f3db4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67d4992fcdd948fa server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:45.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.480+0000 7f3dcbe00640 1 -- 172.21.15.167:0/4124969627 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc00013b0 con 0x7f3db4003aa0 2024-05-11T12:14:45.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.480+0000 7f3dcbe00640 1 -- 172.21.15.167:0/4124969627 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dc00052d0 con 0x7f3db4003aa0 2024-05-11T12:14:45.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.480+0000 7f3dcbe00640 1 -- 172.21.15.167:0/4124969627 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc002aa70 con 0x7f3db4003aa0 2024-05-11T12:14:45.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.481+0000 7f3dd4400640 1 -- 172.21.15.167:0/4124969627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3db4003aa0 msgr2=0x7f3db4003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:45.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.481+0000 7f3dd4400640 1 --2- 172.21.15.167:0/4124969627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3db4003aa0 0x7f3db4003ec0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0001760 tx=0x7f3db4005830 comp rx=0 tx=0).stop 2024-05-11T12:14:45.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.481+0000 7f3dd4400640 1 -- 172.21.15.167:0/4124969627 shutdown_connections 2024-05-11T12:14:45.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.481+0000 7f3dd4400640 1 --2- 172.21.15.167:0/4124969627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3db4003aa0 0x7f3db4003ec0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:45.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.481+0000 7f3dd4400640 1 -- 172.21.15.167:0/4124969627 >> 172.21.15.167:0/4124969627 conn(0x7f3dcc0cd8f0 msgr2=0x7f3dcc0cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:45.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.481+0000 7f3dd4400640 1 -- 172.21.15.167:0/4124969627 shutdown_connections 2024-05-11T12:14:45.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.481+0000 7f3dd4400640 1 -- 172.21.15.167:0/4124969627 wait complete. 2024-05-11T12:14:45.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.483+0000 7f3dd4400640 1 Processor -- start 2024-05-11T12:14:45.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.483+0000 7f3dd4400640 1 -- start start 2024-05-11T12:14:45.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.484+0000 7f3dd4400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3db4003aa0 0x7f3dcc0730c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:45.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.484+0000 7f3dd4400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3dcc0af6a0 0x7f3dcc0ad710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:45.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.484+0000 7f3dd4400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3dcc0ac410 0x7f3dcc0ac870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:45.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.484+0000 7f3dd4400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3dcc06d8b0 con 0x7f3db4003aa0 2024-05-11T12:14:45.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.484+0000 7f3dd4400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3dcc161000 con 0x7f3dcc0af6a0 2024-05-11T12:14:45.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.484+0000 7f3dd4400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3dcc06da30 con 0x7f3dcc0ac410 2024-05-11T12:14:45.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.484+0000 7f3dd1a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3db4003aa0 0x7f3dcc0730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:45.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.484+0000 7f3dd1a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3db4003aa0 0x7f3dcc0730c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:47722/0 (socket says 172.21.15.167:47722) 2024-05-11T12:14:45.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.484+0000 7f3dd1a00640 1 -- 172.21.15.167:0/3838079559 learned_addr learned my addr 172.21.15.167:0/3838079559 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:45.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.485+0000 7f3dd2400640 1 --2- 172.21.15.167:0/3838079559 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3dcc0ac410 0x7f3dcc0ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:45.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.485+0000 7f3dd1a00640 1 -- 172.21.15.167:0/3838079559 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3dcc0ac410 msgr2=0x7f3dcc0ac870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:45.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.485+0000 7f3dd1a00640 1 --2- 172.21.15.167:0/3838079559 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3dcc0ac410 0x7f3dcc0ac870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:45.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.485+0000 7f3dd1a00640 1 -- 172.21.15.167:0/3838079559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3dcc0af6a0 msgr2=0x7f3dcc0ad710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:45.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.485+0000 7f3dd1000640 1 --2- 172.21.15.167:0/3838079559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3dcc0af6a0 0x7f3dcc0ad710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:45.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.485+0000 7f3dd1a00640 1 --2- 172.21.15.167:0/3838079559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3dcc0af6a0 0x7f3dcc0ad710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:45.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.485+0000 7f3dd1a00640 1 -- 172.21.15.167:0/3838079559 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dcc0acdb0 con 0x7f3db4003aa0 2024-05-11T12:14:45.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.485+0000 7f3dd1a00640 1 --2- 172.21.15.167:0/3838079559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3db4003aa0 0x7f3dcc0730c0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0001760 tx=0x7f3dc0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:45.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.485+0000 7f3dc9600640 1 -- 172.21.15.167:0/3838079559 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0037040 con 0x7f3db4003aa0 2024-05-11T12:14:45.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.485+0000 7f3dc9600640 1 -- 172.21.15.167:0/3838079559 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dc0005480 con 0x7f3db4003aa0 2024-05-11T12:14:45.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.485+0000 7f3dd4400640 1 -- 172.21.15.167:0/3838079559 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dcc073950 con 0x7f3db4003aa0 2024-05-11T12:14:45.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.485+0000 7f3dc9600640 1 -- 172.21.15.167:0/3838079559 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc00013b0 con 0x7f3db4003aa0 2024-05-11T12:14:45.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.486+0000 7f3dd4400640 1 -- 172.21.15.167:0/3838079559 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dcc073f10 con 0x7f3db4003aa0 2024-05-11T12:14:45.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.487+0000 7f3dd4400640 1 -- 172.21.15.167:0/3838079559 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dcc069590 con 0x7f3db4003aa0 2024-05-11T12:14:45.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.488+0000 7f3dc9600640 1 -- 172.21.15.167:0/3838079559 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3dc0044020 con 0x7f3db4003aa0 2024-05-11T12:14:45.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.489+0000 7f3dc9600640 1 --2- 172.21.15.167:0/3838079559 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3dac076300 0x7f3dac0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:45.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.489+0000 7f3dc9600640 1 -- 172.21.15.167:0/3838079559 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6101+0+0 (secure 0 0 0) 0x7f3dc00b72a0 con 0x7f3db4003aa0 2024-05-11T12:14:45.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.489+0000 7f3dd1000640 1 --2- 172.21.15.167:0/3838079559 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3dac076300 0x7f3dac0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:45.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.490+0000 7f3dd1000640 1 --2- 172.21.15.167:0/3838079559 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3dac076300 0x7f3dac0787c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3db8000fb0 tx=0x7f3db8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:45.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.494+0000 7f3dc9600640 1 -- 172.21.15.167:0/3838079559 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3dc0080a70 con 0x7f3db4003aa0 2024-05-11T12:14:45.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.743+0000 7f3dd4400640 1 -- 172.21.15.167:0/3838079559 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3dcc075b40 con 0x7f3db4003aa0 2024-05-11T12:14:45.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.744+0000 7f3dc9600640 1 -- 172.21.15.167:0/3838079559 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v62) ==== 53+0+73 (secure 0 0 0) 0x7f3dc00b70c0 con 0x7f3db4003aa0 2024-05-11T12:14:45.744 INFO:teuthology.orchestra.run.smithi167.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-11T12:14:45.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.746+0000 7f3dd4400640 1 -- 172.21.15.167:0/3838079559 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3dac076300 msgr2=0x7f3dac0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:45.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.746+0000 7f3dd4400640 1 --2- 172.21.15.167:0/3838079559 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3dac076300 0x7f3dac0787c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3db8000fb0 tx=0x7f3db8009040 comp rx=0 tx=0).stop 2024-05-11T12:14:45.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.746+0000 7f3dd4400640 1 -- 172.21.15.167:0/3838079559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3db4003aa0 msgr2=0x7f3dcc0730c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:45.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.746+0000 7f3dd4400640 1 --2- 172.21.15.167:0/3838079559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3db4003aa0 0x7f3dcc0730c0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0001760 tx=0x7f3dc0001120 comp rx=0 tx=0).stop 2024-05-11T12:14:45.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.747+0000 7f3dd4400640 1 -- 172.21.15.167:0/3838079559 shutdown_connections 2024-05-11T12:14:45.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.747+0000 7f3dd4400640 1 --2- 172.21.15.167:0/3838079559 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3dcc0ac410 0x7f3dcc0ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:45.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.747+0000 7f3dd4400640 1 --2- 172.21.15.167:0/3838079559 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3dac076300 0x7f3dac0787c0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:45.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.747+0000 7f3dd4400640 1 --2- 172.21.15.167:0/3838079559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3dcc0af6a0 0x7f3dcc0ad710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:45.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.747+0000 7f3dd4400640 1 --2- 172.21.15.167:0/3838079559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3db4003aa0 0x7f3dcc0730c0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:45.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.747+0000 7f3dd4400640 1 -- 172.21.15.167:0/3838079559 >> 172.21.15.167:0/3838079559 conn(0x7f3dcc0cd8f0 msgr2=0x7f3dcc0cfbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:45.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.747+0000 7f3dd4400640 1 -- 172.21.15.167:0/3838079559 shutdown_connections 2024-05-11T12:14:45.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:45.747+0000 7f3dd4400640 1 -- 172.21.15.167:0/3838079559 wait complete. 2024-05-11T12:14:45.765 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-11T12:14:45.765 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-11T12:14:45.765 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi167.front.sepia.ceph.com 2024-05-11T12:14:45.765 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-11T12:14:45.765 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-11T12:14:45.765 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-11T12:14:45.765 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-11T12:14:45.765 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-11T12:14:45.765 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-11T12:14:45.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:45.766 DEBUG:teuthology.orchestra.run.smithi167:> ip addr 2024-05-11T12:14:45.817 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-11T12:14:45.817 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-11T12:14:45.817 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2024-05-11T12:14:45.818 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:14:45.818 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2024-05-11T12:14:45.818 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:14:45.818 INFO:teuthology.orchestra.run.smithi167.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:14:45.818 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e6 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:14:45.818 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f0 2024-05-11T12:14:45.818 INFO:teuthology.orchestra.run.smithi167.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:14:45.818 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e7 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:14:45.818 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f1 2024-05-11T12:14:45.818 INFO:teuthology.orchestra.run.smithi167.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:14:45.818 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:08 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:14:45.818 INFO:teuthology.orchestra.run.smithi167.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-11T12:14:45.818 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:09 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:14:45.819 INFO:teuthology.orchestra.run.smithi167.stdout: inet 172.21.15.167/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-11T12:14:45.819 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft 42383sec preferred_lft 42383sec 2024-05-11T12:14:45.819 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::ec4:7aff:fe88:7309/64 scope link 2024-05-11T12:14:45.819 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:14:45.819 INFO:teuthology.orchestra.run.smithi167.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-11T12:14:45.819 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 1a:a8:8d:b3:f4:32 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:14:45.819 INFO:teuthology.orchestra.run.smithi167.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-11T12:14:45.819 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:14:45.819 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::b017:48ff:fea0:7183/64 scope link 2024-05-11T12:14:45.819 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:14:45.819 INFO:teuthology.orchestra.run.smithi167.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-11T12:14:45.819 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 1a:a8:8d:b3:f4:32 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-11T12:14:45.819 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::18a8:8dff:feb3:f432/64 scope link 2024-05-11T12:14:45.820 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:14:45.820 INFO:teuthology.orchestra.run.smithi167.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-11T12:14:45.820 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 7e:79:1f:cb:d2:83 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-11T12:14:45.820 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::7c79:1fff:fecb:d283/64 scope link 2024-05-11T12:14:45.820 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:14:45.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:45.821 DEBUG:teuthology.orchestra.run.smithi167:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-11T12:14:45.892 INFO:teuthology.orchestra.run.smithi167.stdout:1 2024-05-11T12:14:45.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:45.896 DEBUG:teuthology.orchestra.run.smithi167:> route 2024-05-11T12:14:45.945 INFO:teuthology.orchestra.run.smithi167.stdout:Kernel IP routing table 2024-05-11T12:14:45.945 INFO:teuthology.orchestra.run.smithi167.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-11T12:14:45.945 INFO:teuthology.orchestra.run.smithi167.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-11T12:14:45.945 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-11T12:14:45.945 INFO:teuthology.orchestra.run.smithi167.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-11T12:14:45.945 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T12:14:45.945 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-05-11T12:14:45.945 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-11T12:14:45.945 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-11T12:14:45.945 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-11T12:14:45.946 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-05-11T12:14:46.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T12:14:46.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:14:46.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:46.152 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2024-05-11T12:14:46.200 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-11T12:14:46.200 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-11T12:14:46.201 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-11T12:14:46.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:46.201 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set brx.0 up 2024-05-11T12:14:46.277 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T12:14:46.277 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-11T12:14:46.329 INFO:teuthology.orchestra.run.smithi167.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-11T12:14:46.330 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T12:14:46.330 DEBUG:teuthology.orchestra.run.smithi167:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-11T12:14:46.382 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-05-11T12:14:46.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:46.382 DEBUG:teuthology.orchestra.run.smithi167:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:14:46.508 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:14:46.508 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-11T12:14:46.543 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:14:46.543 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-11T12:14:46.543 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.167:3300,172.21.15.204:3300,172.21.15.204:3301" 2024-05-11T12:14:46.543 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: trying mount with new device syntax: 0@5f32d344-7289-4c27-881b-1533419ab678.cephfs=/ 2024-05-11T12:14:46.543 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.167:3300/172.21.15.204:3300/172.21.15.204:3301" will pass to kernel 2024-05-11T12:14:46.546 INFO:teuthology.orchestra.run.smithi167.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-11T12:14:46.546 INFO:teuthology.orchestra.run.smithi167.stdout: You just mounted a file system that supports labels which does not 2024-05-11T12:14:46.546 INFO:teuthology.orchestra.run.smithi167.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-11T12:14:46.546 INFO:teuthology.orchestra.run.smithi167.stdout: applications will generate AVC messages and not be allowed access to 2024-05-11T12:14:46.546 INFO:teuthology.orchestra.run.smithi167.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-11T12:14:46.550 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-11T12:14:46.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:46.550 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-11T12:14:46.623 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-11T12:14:46.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-11T12:14:46.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:14:46.751 INFO:teuthology.orchestra.run.smithi167.stdout:client6819 2024-05-11T12:14:46.751 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:14:46.751 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:14:46.798 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-11T12:14:47.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.044+0000 7f1295c00640 1 Processor -- start 2024-05-11T12:14:47.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.045+0000 7f1295c00640 1 -- start start 2024-05-11T12:14:47.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.046+0000 7f1295c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1290148a50 0x7f12900d4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:47.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.046+0000 7f1295c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f12900b95a0 0x7f129015f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:47.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.046+0000 7f1295c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f129006d600 con 0x7f129006ce60 2024-05-11T12:14:47.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.046+0000 7f1295c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f129006e210 con 0x7f1290148a50 2024-05-11T12:14:47.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.046+0000 7f1295c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f129006d8e0 con 0x7f12900b95a0 2024-05-11T12:14:47.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.047+0000 7f128fe00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1290148a50 0x7f12900d4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:47.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.047+0000 7f1295200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f12900b95a0 0x7f129015f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:47.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.047+0000 7f128fe00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1290148a50 0x7f12900d4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.167:34518/0 (socket says 172.21.15.167:34518) 2024-05-11T12:14:47.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.047+0000 7f128fe00640 1 -- 172.21.15.167:0/704385940 learned_addr learned my addr 172.21.15.167:0/704385940 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:47.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.048+0000 7f1295200640 1 -- 172.21.15.167:0/704385940 >> v2:172.21.15.204:3300/0 conn(0x7f1290148a50 msgr2=0x7f12900d4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:47.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.048+0000 7f1295200640 1 --2- 172.21.15.167:0/704385940 >> v2:172.21.15.204:3300/0 conn(0x7f1290148a50 0x7f12900d4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:47.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.048+0000 7f1295200640 1 -- 172.21.15.167:0/704385940 >> v1:172.21.15.204:6789/0 conn(0x7f129006ce60 legacy=0x7f12900b8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:47.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.048+0000 7f1295200640 1 -- 172.21.15.167:0/704385940 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f129006dbc0 con 0x7f12900b95a0 2024-05-11T12:14:47.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.049+0000 7f1295200640 1 --2- 172.21.15.167:0/704385940 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f12900b95a0 0x7f129015f800 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f12880097a0 tx=0x7f128802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69fbdfab0644dd63 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:47.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.050+0000 7f128f400640 1 -- 172.21.15.167:0/704385940 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1288037070 con 0x7f12900b95a0 2024-05-11T12:14:47.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.050+0000 7f128f400640 1 -- 172.21.15.167:0/704385940 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f128802ae50 con 0x7f12900b95a0 2024-05-11T12:14:47.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.050+0000 7f128f400640 1 -- 172.21.15.167:0/704385940 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1288032660 con 0x7f12900b95a0 2024-05-11T12:14:47.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.051+0000 7f1295c00640 1 -- 172.21.15.167:0/704385940 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f12900b95a0 msgr2=0x7f129015f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:47.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.051+0000 7f1295c00640 1 --2- 172.21.15.167:0/704385940 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f12900b95a0 0x7f129015f800 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f12880097a0 tx=0x7f128802a100 comp rx=0 tx=0).stop 2024-05-11T12:14:47.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.051+0000 7f1295c00640 1 -- 172.21.15.167:0/704385940 shutdown_connections 2024-05-11T12:14:47.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.051+0000 7f1295c00640 1 --2- 172.21.15.167:0/704385940 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f12900b95a0 0x7f129015f800 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:47.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.051+0000 7f1295c00640 1 --2- 172.21.15.167:0/704385940 >> v2:172.21.15.204:3300/0 conn(0x7f1290148a50 0x7f12900d4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:47.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.051+0000 7f1295c00640 1 -- 172.21.15.167:0/704385940 >> 172.21.15.167:0/704385940 conn(0x7f12900d4f10 msgr2=0x7f12900d5320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:47.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.051+0000 7f1295c00640 1 -- 172.21.15.167:0/704385940 shutdown_connections 2024-05-11T12:14:47.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.051+0000 7f1295c00640 1 -- 172.21.15.167:0/704385940 wait complete. 2024-05-11T12:14:47.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.053+0000 7f1295c00640 1 Processor -- start 2024-05-11T12:14:47.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.053+0000 7f1295c00640 1 -- start start 2024-05-11T12:14:47.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.054+0000 7f1295c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f129006ce60 0x7f12901f57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:47.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.054+0000 7f1295c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12900b95a0 0x7f12900adc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:47.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.054+0000 7f1295c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1290148a50 0x7f12900ae190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:47.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.054+0000 7f1295c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1290148e30 con 0x7f1290148a50 2024-05-11T12:14:47.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.054+0000 7f1295c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f129006d2c0 con 0x7f12900b95a0 2024-05-11T12:14:47.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.054+0000 7f1295c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f129006e6a0 con 0x7f129006ce60 2024-05-11T12:14:47.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.054+0000 7f1295200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1290148a50 0x7f12900ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:47.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.054+0000 7f1295200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1290148a50 0x7f12900ae190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:47724/0 (socket says 172.21.15.167:47724) 2024-05-11T12:14:47.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.054+0000 7f1295200640 1 -- 172.21.15.167:0/832678253 learned_addr learned my addr 172.21.15.167:0/832678253 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:47.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.055+0000 7f128fe00640 1 --2- 172.21.15.167:0/832678253 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f129006ce60 0x7f12901f57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:47.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.055+0000 7f1287400640 1 --2- 172.21.15.167:0/832678253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12900b95a0 0x7f12900adc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:47.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.055+0000 7f1295200640 1 -- 172.21.15.167:0/832678253 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f129006ce60 msgr2=0x7f12901f57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:47.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.055+0000 7f1295200640 1 --2- 172.21.15.167:0/832678253 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f129006ce60 0x7f12901f57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:47.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.055+0000 7f1295200640 1 -- 172.21.15.167:0/832678253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12900b95a0 msgr2=0x7f12900adc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:47.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.055+0000 7f1295200640 1 --2- 172.21.15.167:0/832678253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12900b95a0 0x7f12900adc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:47.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.055+0000 7f1295200640 1 -- 172.21.15.167:0/832678253 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12900abdd0 con 0x7f1290148a50 2024-05-11T12:14:47.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.055+0000 7f1295200640 1 --2- 172.21.15.167:0/832678253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1290148a50 0x7f12900ae190 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f12880098c0 tx=0x7f12880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:47.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.056+0000 7f128cc00640 1 -- 172.21.15.167:0/832678253 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1288042020 con 0x7f1290148a50 2024-05-11T12:14:47.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.056+0000 7f128cc00640 1 -- 172.21.15.167:0/832678253 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f128803d470 con 0x7f1290148a50 2024-05-11T12:14:47.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.056+0000 7f128cc00640 1 -- 172.21.15.167:0/832678253 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1288037040 con 0x7f1290148a50 2024-05-11T12:14:47.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.056+0000 7f1295c00640 1 -- 172.21.15.167:0/832678253 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12900ac0b0 con 0x7f1290148a50 2024-05-11T12:14:47.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.056+0000 7f1295c00640 1 -- 172.21.15.167:0/832678253 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12900ac670 con 0x7f1290148a50 2024-05-11T12:14:47.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.057+0000 7f1295c00640 1 -- 172.21.15.167:0/832678253 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12900694b0 con 0x7f1290148a50 2024-05-11T12:14:47.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.058+0000 7f128cc00640 1 -- 172.21.15.167:0/832678253 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1288045020 con 0x7f1290148a50 2024-05-11T12:14:47.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.059+0000 7f128cc00640 1 --2- 172.21.15.167:0/832678253 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f126c0762d0 0x7f126c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:47.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.059+0000 7f128cc00640 1 -- 172.21.15.167:0/832678253 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6101+0+0 (secure 0 0 0) 0x7f12880b7d80 con 0x7f1290148a50 2024-05-11T12:14:47.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.059+0000 7f128fe00640 1 --2- 172.21.15.167:0/832678253 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f126c0762d0 0x7f126c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:47.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.060+0000 7f128fe00640 1 --2- 172.21.15.167:0/832678253 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f126c0762d0 0x7f126c078790 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1280009ce0 tx=0x7f128000b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:47.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.064+0000 7f128cc00640 1 -- 172.21.15.167:0/832678253 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f128804c070 con 0x7f1290148a50 2024-05-11T12:14:47.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.314+0000 7f1295c00640 1 -- 172.21.15.167:0/832678253 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1290148200 con 0x7f1290148a50 2024-05-11T12:14:47.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.315+0000 7f128cc00640 1 -- 172.21.15.167:0/832678253 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v62) ==== 53+0+73 (secure 0 0 0) 0x7f12880814b0 con 0x7f1290148a50 2024-05-11T12:14:47.315 INFO:teuthology.orchestra.run.smithi167.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-11T12:14:47.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.317+0000 7f1295c00640 1 -- 172.21.15.167:0/832678253 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f126c0762d0 msgr2=0x7f126c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:47.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.317+0000 7f1295c00640 1 --2- 172.21.15.167:0/832678253 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f126c0762d0 0x7f126c078790 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1280009ce0 tx=0x7f128000b040 comp rx=0 tx=0).stop 2024-05-11T12:14:47.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.318+0000 7f1295c00640 1 -- 172.21.15.167:0/832678253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1290148a50 msgr2=0x7f12900ae190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:47.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.318+0000 7f1295c00640 1 --2- 172.21.15.167:0/832678253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1290148a50 0x7f12900ae190 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f12880098c0 tx=0x7f12880046c0 comp rx=0 tx=0).stop 2024-05-11T12:14:47.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.318+0000 7f1295c00640 1 -- 172.21.15.167:0/832678253 shutdown_connections 2024-05-11T12:14:47.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.318+0000 7f1295c00640 1 --2- 172.21.15.167:0/832678253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1290148a50 0x7f12900ae190 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:47.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.318+0000 7f1295c00640 1 --2- 172.21.15.167:0/832678253 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f126c0762d0 0x7f126c078790 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:47.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.318+0000 7f1295c00640 1 --2- 172.21.15.167:0/832678253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12900b95a0 0x7f12900adc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:47.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.318+0000 7f1295c00640 1 --2- 172.21.15.167:0/832678253 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f129006ce60 0x7f12901f57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:47.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.318+0000 7f1295c00640 1 -- 172.21.15.167:0/832678253 >> 172.21.15.167:0/832678253 conn(0x7f12900d4f10 msgr2=0x7f1290144ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:47.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.318+0000 7f1295c00640 1 -- 172.21.15.167:0/832678253 shutdown_connections 2024-05-11T12:14:47.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:47.318+0000 7f1295c00640 1 -- 172.21.15.167:0/832678253 wait complete. 2024-05-11T12:14:47.336 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-11T12:14:47.336 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-11T12:14:47.336 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi204.front.sepia.ceph.com 2024-05-11T12:14:47.336 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-11T12:14:47.336 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-11T12:14:47.336 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-11T12:14:47.336 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-11T12:14:47.336 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-11T12:14:47.336 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-11T12:14:47.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:47.336 DEBUG:teuthology.orchestra.run.smithi204:> ip addr 2024-05-11T12:14:47.396 INFO:teuthology.orchestra.run.smithi204.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-11T12:14:47.396 INFO:teuthology.orchestra.run.smithi204.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-11T12:14:47.396 INFO:teuthology.orchestra.run.smithi204.stdout: inet 127.0.0.1/8 scope host lo 2024-05-11T12:14:47.396 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:14:47.396 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 ::1/128 scope host 2024-05-11T12:14:47.396 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:14:47.396 INFO:teuthology.orchestra.run.smithi204.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:14:47.396 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2a brd ff:ff:ff:ff:ff:ff 2024-05-11T12:14:47.396 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f0 2024-05-11T12:14:47.397 INFO:teuthology.orchestra.run.smithi204.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:14:47.397 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2b brd ff:ff:ff:ff:ff:ff 2024-05-11T12:14:47.397 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f1 2024-05-11T12:14:47.397 INFO:teuthology.orchestra.run.smithi204.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:14:47.397 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:aa brd ff:ff:ff:ff:ff:ff 2024-05-11T12:14:47.397 INFO:teuthology.orchestra.run.smithi204.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-11T12:14:47.397 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:ab brd ff:ff:ff:ff:ff:ff 2024-05-11T12:14:47.397 INFO:teuthology.orchestra.run.smithi204.stdout: inet 172.21.15.204/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-11T12:14:47.397 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft 42377sec preferred_lft 42377sec 2024-05-11T12:14:47.397 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::ec4:7aff:fe88:77ab/64 scope link 2024-05-11T12:14:47.397 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:14:47.397 INFO:teuthology.orchestra.run.smithi204.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-11T12:14:47.397 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether e2:9e:5b:47:45:5d brd ff:ff:ff:ff:ff:ff 2024-05-11T12:14:47.397 INFO:teuthology.orchestra.run.smithi204.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-11T12:14:47.397 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:14:47.397 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::a498:7aff:fe50:48c3/64 scope link 2024-05-11T12:14:47.398 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:14:47.398 INFO:teuthology.orchestra.run.smithi204.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-11T12:14:47.398 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether fa:0a:7f:e4:30:35 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-11T12:14:47.398 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::f80a:7fff:fee4:3035/64 scope link 2024-05-11T12:14:47.398 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:14:47.398 INFO:teuthology.orchestra.run.smithi204.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-11T12:14:47.398 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether e2:9e:5b:47:45:5d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-11T12:14:47.398 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::e09e:5bff:fe47:455d/64 scope link 2024-05-11T12:14:47.398 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:14:47.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:47.399 DEBUG:teuthology.orchestra.run.smithi204:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-11T12:14:47.479 INFO:teuthology.orchestra.run.smithi204.stdout:1 2024-05-11T12:14:47.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:47.483 DEBUG:teuthology.orchestra.run.smithi204:> route 2024-05-11T12:14:47.533 INFO:teuthology.orchestra.run.smithi204.stdout:Kernel IP routing table 2024-05-11T12:14:47.533 INFO:teuthology.orchestra.run.smithi204.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-11T12:14:47.533 INFO:teuthology.orchestra.run.smithi204.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-11T12:14:47.533 INFO:teuthology.orchestra.run.smithi204.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-11T12:14:47.533 INFO:teuthology.orchestra.run.smithi204.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-11T12:14:47.534 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T12:14:47.534 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-05-11T12:14:47.534 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-11T12:14:47.534 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-11T12:14:47.535 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-11T12:14:47.535 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-05-11T12:14:47.639 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:14:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T12:14:47.735 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:14:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:14:47.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:47.745 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list 2024-05-11T12:14:47.790 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-11T12:14:47.790 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-11T12:14:47.791 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-11T12:14:47.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:47.791 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link set brx.0 up 2024-05-11T12:14:47.918 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T12:14:47.918 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-11T12:14:47.970 INFO:teuthology.orchestra.run.smithi204.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-11T12:14:47.970 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T12:14:47.970 DEBUG:teuthology.orchestra.run.smithi204:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-11T12:14:48.019 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-05-11T12:14:48.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:48.020 DEBUG:teuthology.orchestra.run.smithi204:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:14:48.147 INFO:teuthology.orchestra.run.smithi204.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:14:48.147 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-11T12:14:48.187 INFO:teuthology.orchestra.run.smithi204.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:14:48.187 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-11T12:14:48.187 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: resolved to: "172.21.15.167:3300,172.21.15.204:3300,172.21.15.204:3301" 2024-05-11T12:14:48.187 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: trying mount with new device syntax: 1@5f32d344-7289-4c27-881b-1533419ab678.cephfs=/ 2024-05-11T12:14:48.187 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.167:3300/172.21.15.204:3300/172.21.15.204:3301" will pass to kernel 2024-05-11T12:14:48.191 INFO:teuthology.orchestra.run.smithi204.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-05-11T12:14:48.191 INFO:teuthology.orchestra.run.smithi204.stdout: You just mounted a file system that supports labels which does not 2024-05-11T12:14:48.191 INFO:teuthology.orchestra.run.smithi204.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-11T12:14:48.191 INFO:teuthology.orchestra.run.smithi204.stdout: applications will generate AVC messages and not be allowed access to 2024-05-11T12:14:48.191 INFO:teuthology.orchestra.run.smithi204.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-11T12:14:48.196 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-11T12:14:48.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:48.196 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-11T12:14:48.266 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-11T12:14:48.365 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:14:48 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-11T12:14:48.370 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:14:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:14:48.379 INFO:teuthology.orchestra.run.smithi204.stdout:client6825 2024-05-11T12:14:48.380 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:14:48.380 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:14:48.430 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:14:48.431 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:14:48.488 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:14:48.488 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:14:48.541 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-11T12:14:48.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.748+0000 7fafedc00640 1 Processor -- start 2024-05-11T12:14:48.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.749+0000 7fafedc00640 1 -- start start 2024-05-11T12:14:48.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.750+0000 7fafedc00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fafe81625d0 0x7fafe80d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:48.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.750+0000 7fafedc00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fafe80b8390 0x7fafe80b8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:48.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.750+0000 7fafedc00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafe80d2e40 con 0x7fafe80d3c00 2024-05-11T12:14:48.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.750+0000 7fafedc00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fafe806df00 con 0x7fafe81625d0 2024-05-11T12:14:48.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.750+0000 7fafedc00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fafe80d3120 con 0x7fafe80b8390 2024-05-11T12:14:48.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.750+0000 7fafe7e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fafe81625d0 0x7fafe80d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:48.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.750+0000 7fafe7e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fafe81625d0 0x7fafe80d0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:47730/0 (socket says 172.21.15.167:47730) 2024-05-11T12:14:48.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.750+0000 7fafe7e00640 1 -- 172.21.15.167:0/2675639400 learned_addr learned my addr 172.21.15.167:0/2675639400 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:48.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.750+0000 7fafe7400640 1 --2- 172.21.15.167:0/2675639400 >> v2:172.21.15.204:3300/0 conn(0x7fafe80b8390 0x7fafe80b8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:48.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.750+0000 7fafe7e00640 1 -- 172.21.15.167:0/2675639400 >> v2:172.21.15.204:3300/0 conn(0x7fafe80b8390 msgr2=0x7fafe80b8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:48.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.750+0000 7fafe7e00640 1 --2- 172.21.15.167:0/2675639400 >> v2:172.21.15.204:3300/0 conn(0x7fafe80b8390 0x7fafe80b8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:48.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.750+0000 7fafe7e00640 1 -- 172.21.15.167:0/2675639400 >> v1:172.21.15.204:6789/0 conn(0x7fafe80d3c00 legacy=0x7fafe80d4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:48.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.750+0000 7fafe7e00640 1 -- 172.21.15.167:0/2675639400 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafe80d3400 con 0x7fafe81625d0 2024-05-11T12:14:48.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.751+0000 7fafe7400640 1 --2- 172.21.15.167:0/2675639400 >> v2:172.21.15.204:3300/0 conn(0x7fafe80b8390 0x7fafe80b8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:48.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.752+0000 7fafe7e00640 1 --2- 172.21.15.167:0/2675639400 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafe81625d0 0x7fafe80d0f60 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fafe80d48e0 tx=0x7fafd802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d8d15347ca2c11c server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:48.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.752+0000 7fafe6a00640 1 -- 172.21.15.167:0/2675639400 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd8037070 con 0x7fafe81625d0 2024-05-11T12:14:48.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.752+0000 7fafe6a00640 1 -- 172.21.15.167:0/2675639400 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafd8032440 con 0x7fafe81625d0 2024-05-11T12:14:48.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.752+0000 7fafe6a00640 1 -- 172.21.15.167:0/2675639400 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd8032810 con 0x7fafe81625d0 2024-05-11T12:14:48.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.753+0000 7fafedc00640 1 -- 172.21.15.167:0/2675639400 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafe81625d0 msgr2=0x7fafe80d0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:48.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.753+0000 7fafedc00640 1 --2- 172.21.15.167:0/2675639400 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafe81625d0 0x7fafe80d0f60 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fafe80d48e0 tx=0x7fafd802a1e0 comp rx=0 tx=0).stop 2024-05-11T12:14:48.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.753+0000 7fafedc00640 1 -- 172.21.15.167:0/2675639400 shutdown_connections 2024-05-11T12:14:48.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.753+0000 7fafedc00640 1 --2- 172.21.15.167:0/2675639400 >> v2:172.21.15.204:3300/0 conn(0x7fafe80b8390 0x7fafe80b8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:48.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.753+0000 7fafedc00640 1 --2- 172.21.15.167:0/2675639400 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafe81625d0 0x7fafe80d0f60 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:48.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.753+0000 7fafedc00640 1 -- 172.21.15.167:0/2675639400 >> 172.21.15.167:0/2675639400 conn(0x7fafe80d50f0 msgr2=0x7fafe80d5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:48.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.753+0000 7fafedc00640 1 -- 172.21.15.167:0/2675639400 shutdown_connections 2024-05-11T12:14:48.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.753+0000 7fafedc00640 1 -- 172.21.15.167:0/2675639400 wait complete. 2024-05-11T12:14:48.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.754+0000 7fafedc00640 1 Processor -- start 2024-05-11T12:14:48.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.755+0000 7fafedc00640 1 -- start start 2024-05-11T12:14:48.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.755+0000 7fafedc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafe80b8390 0x7fafe815f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:48.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.755+0000 7fafedc00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafe80d3c00 0x7fafe80ab940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:48.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.755+0000 7fafedc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fafe81625d0 0x7fafe80b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:48.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.755+0000 7fafedc00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fafe8162b30 con 0x7fafe80d3c00 2024-05-11T12:14:48.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.755+0000 7fafedc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fafe81629b0 con 0x7fafe80b8390 2024-05-11T12:14:48.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.755+0000 7fafedc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fafe806cef0 con 0x7fafe81625d0 2024-05-11T12:14:48.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.756+0000 7fafe7400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafe80d3c00 0x7fafe80ab940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:48.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.756+0000 7fafe7400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafe80d3c00 0x7fafe80ab940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:47746/0 (socket says 172.21.15.167:47746) 2024-05-11T12:14:48.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.756+0000 7fafe7400640 1 -- 172.21.15.167:0/683082463 learned_addr learned my addr 172.21.15.167:0/683082463 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:48.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.756+0000 7fafed200640 1 --2- 172.21.15.167:0/683082463 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fafe81625d0 0x7fafe80b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:48.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.756+0000 7fafe7e00640 1 --2- 172.21.15.167:0/683082463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafe80b8390 0x7fafe815f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:48.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.756+0000 7fafe7400640 1 -- 172.21.15.167:0/683082463 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fafe81625d0 msgr2=0x7fafe80b4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:48.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.756+0000 7fafe7400640 1 --2- 172.21.15.167:0/683082463 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fafe81625d0 0x7fafe80b4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:48.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.756+0000 7fafe7400640 1 -- 172.21.15.167:0/683082463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafe80b8390 msgr2=0x7fafe815f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:48.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.756+0000 7fafe7400640 1 --2- 172.21.15.167:0/683082463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafe80b8390 0x7fafe815f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:48.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.756+0000 7fafe7400640 1 -- 172.21.15.167:0/683082463 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafe815fbe0 con 0x7fafe80d3c00 2024-05-11T12:14:48.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.756+0000 7fafed200640 1 --2- 172.21.15.167:0/683082463 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fafe81625d0 0x7fafe80b4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:48.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.756+0000 7fafe7e00640 1 --2- 172.21.15.167:0/683082463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafe80b8390 0x7fafe815f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:48.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.757+0000 7fafe7400640 1 --2- 172.21.15.167:0/683082463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafe80d3c00 0x7fafe80ab940 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fafd000cf80 tx=0x7fafd000a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:48.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.757+0000 7fafdfe00640 1 -- 172.21.15.167:0/683082463 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd001a070 con 0x7fafe80d3c00 2024-05-11T12:14:48.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.757+0000 7fafdfe00640 1 -- 172.21.15.167:0/683082463 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafd000d840 con 0x7fafe80d3c00 2024-05-11T12:14:48.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.757+0000 7fafdfe00640 1 -- 172.21.15.167:0/683082463 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd000dc30 con 0x7fafe80d3c00 2024-05-11T12:14:48.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.757+0000 7fafedc00640 1 -- 172.21.15.167:0/683082463 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafe80ac400 con 0x7fafe80d3c00 2024-05-11T12:14:48.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.757+0000 7fafedc00640 1 -- 172.21.15.167:0/683082463 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafe80ac9c0 con 0x7fafe80d3c00 2024-05-11T12:14:48.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.758+0000 7fafedc00640 1 -- 172.21.15.167:0/683082463 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafe81f50b0 con 0x7fafe80d3c00 2024-05-11T12:14:48.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.760+0000 7fafdfe00640 1 -- 172.21.15.167:0/683082463 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fafd001d070 con 0x7fafe80d3c00 2024-05-11T12:14:48.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.760+0000 7fafdfe00640 1 --2- 172.21.15.167:0/683082463 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fafc40762d0 0x7fafc4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:48.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.761+0000 7fafdfe00640 1 -- 172.21.15.167:0/683082463 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6101+0+0 (secure 0 0 0) 0x7fafd0097ff0 con 0x7fafe80d3c00 2024-05-11T12:14:48.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.761+0000 7fafe7e00640 1 --2- 172.21.15.167:0/683082463 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fafc40762d0 0x7fafc4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:48.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.762+0000 7fafe7e00640 1 --2- 172.21.15.167:0/683082463 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fafc40762d0 0x7fafc4078790 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fafd8034000 tx=0x7fafd802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:48.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.765+0000 7fafdfe00640 1 -- 172.21.15.167:0/683082463 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fafe815fbe0 con 0x7fafe80d3c00 2024-05-11T12:14:48.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.988+0000 7fafedc00640 1 -- 172.21.15.167:0/683082463 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fafe80694b0 con 0x7fafe80d3c00 2024-05-11T12:14:48.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.989+0000 7fafdfe00640 1 -- 172.21.15.167:0/683082463 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v97) ==== 80+0+0 (secure 0 0 0) 0x7fafd0061780 con 0x7fafe80d3c00 2024-05-11T12:14:48.989 INFO:teuthology.orchestra.run.smithi167.stderr:listed 0 entries 2024-05-11T12:14:48.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.992+0000 7fafedc00640 1 -- 172.21.15.167:0/683082463 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fafc40762d0 msgr2=0x7fafc4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:48.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.992+0000 7fafedc00640 1 --2- 172.21.15.167:0/683082463 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fafc40762d0 0x7fafc4078790 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fafd8034000 tx=0x7fafd802e000 comp rx=0 tx=0).stop 2024-05-11T12:14:48.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.992+0000 7fafedc00640 1 -- 172.21.15.167:0/683082463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafe80d3c00 msgr2=0x7fafe80ab940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:48.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.992+0000 7fafedc00640 1 --2- 172.21.15.167:0/683082463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafe80d3c00 0x7fafe80ab940 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fafd000cf80 tx=0x7fafd000a7d0 comp rx=0 tx=0).stop 2024-05-11T12:14:48.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.992+0000 7fafedc00640 1 -- 172.21.15.167:0/683082463 shutdown_connections 2024-05-11T12:14:48.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.992+0000 7fafedc00640 1 --2- 172.21.15.167:0/683082463 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fafe81625d0 0x7fafe80b4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:48.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.992+0000 7fafedc00640 1 --2- 172.21.15.167:0/683082463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafe80d3c00 0x7fafe80ab940 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:48.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.992+0000 7fafedc00640 1 --2- 172.21.15.167:0/683082463 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fafc40762d0 0x7fafc4078790 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:48.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.992+0000 7fafedc00640 1 --2- 172.21.15.167:0/683082463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafe80b8390 0x7fafe815f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:48.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.992+0000 7fafedc00640 1 -- 172.21.15.167:0/683082463 >> 172.21.15.167:0/683082463 conn(0x7fafe80d50f0 msgr2=0x7fafe80d54d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:48.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.992+0000 7fafedc00640 1 -- 172.21.15.167:0/683082463 shutdown_connections 2024-05-11T12:14:48.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:48.992+0000 7fafedc00640 1 -- 172.21.15.167:0/683082463 wait complete. 2024-05-11T12:14:49.013 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-11T12:14:49.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:49.013 DEBUG:teuthology.orchestra.run.smithi167:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-11T12:14:49.105 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2024-05-11T12:14:49.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:49.105 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-11T12:14:49.159 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:14:49.159 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:14:49.217 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:14:49.217 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:14:49.269 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-11T12:14:49.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.483+0000 7f0a3b600640 1 Processor -- start 2024-05-11T12:14:49.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.483+0000 7f0a3b600640 1 -- start start 2024-05-11T12:14:49.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.484+0000 7f0a3b600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0a3415ac80 0x7f0a34074260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:49.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.485+0000 7f0a3b600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a340b3d50 0x7f0a34160c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:49.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.485+0000 7f0a3b600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a340b65e0 con 0x7f0a341611b0 2024-05-11T12:14:49.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.485+0000 7f0a3b600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f0a3406e3c0 con 0x7f0a3415ac80 2024-05-11T12:14:49.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.485+0000 7f0a3b600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0a3415b060 con 0x7f0a340b3d50 2024-05-11T12:14:49.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.485+0000 7f0a2be00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a340b3d50 0x7f0a34160c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:49.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.485+0000 7f0a38c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0a3415ac80 0x7f0a34074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:49.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.486+0000 7f0a38c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0a3415ac80 0x7f0a34074260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.167:42530/0 (socket says 172.21.15.167:42530) 2024-05-11T12:14:49.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.486+0000 7f0a38c00640 1 -- 172.21.15.167:0/1304906675 learned_addr learned my addr 172.21.15.167:0/1304906675 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:49.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.486+0000 7f0a2be00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a340b3d50 0x7f0a34160c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:39204/0 (socket says 172.21.15.167:39204) 2024-05-11T12:14:49.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.486+0000 7f0a38c00640 1 -- 172.21.15.167:0/1304906675 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a340b3d50 msgr2=0x7f0a34160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:49.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.486+0000 7f0a38c00640 1 --2- 172.21.15.167:0/1304906675 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a340b3d50 0x7f0a34160c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:49.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.486+0000 7f0a38c00640 1 -- 172.21.15.167:0/1304906675 >> v1:172.21.15.204:6789/0 conn(0x7f0a341611b0 legacy=0x7f0a3415d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:49.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.486+0000 7f0a38c00640 1 -- 172.21.15.167:0/1304906675 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a340b68c0 con 0x7f0a3415ac80 2024-05-11T12:14:49.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.486+0000 7f0a33e00640 1 -- 172.21.15.167:0/1304906675 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1518072055 0 0) 0x7f0a340b65e0 con 0x7f0a341611b0 2024-05-11T12:14:49.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.487+0000 7f0a2be00640 1 --2- 172.21.15.167:0/1304906675 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a340b3d50 0x7f0a34160c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:14:49.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.488+0000 7f0a38c00640 1 --2- 172.21.15.167:0/1304906675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a3415ac80 0x7f0a34074260 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0a24009870 tx=0x7f0a2402a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb4c4ef1b0777544 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:49.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.488+0000 7f0a33e00640 1 -- 172.21.15.167:0/1304906675 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a24037070 con 0x7f0a3415ac80 2024-05-11T12:14:49.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.488+0000 7f0a33e00640 1 -- 172.21.15.167:0/1304906675 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a24032440 con 0x7f0a3415ac80 2024-05-11T12:14:49.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.488+0000 7f0a33e00640 1 -- 172.21.15.167:0/1304906675 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a24032810 con 0x7f0a3415ac80 2024-05-11T12:14:49.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.489+0000 7f0a3b600640 1 -- 172.21.15.167:0/1304906675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a3415ac80 msgr2=0x7f0a34074260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:49.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.489+0000 7f0a3b600640 1 --2- 172.21.15.167:0/1304906675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a3415ac80 0x7f0a34074260 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0a24009870 tx=0x7f0a2402a250 comp rx=0 tx=0).stop 2024-05-11T12:14:49.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.489+0000 7f0a3b600640 1 -- 172.21.15.167:0/1304906675 shutdown_connections 2024-05-11T12:14:49.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.489+0000 7f0a3b600640 1 --2- 172.21.15.167:0/1304906675 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a340b3d50 0x7f0a34160c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:49.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.489+0000 7f0a3b600640 1 --2- 172.21.15.167:0/1304906675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a3415ac80 0x7f0a34074260 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:49.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.489+0000 7f0a3b600640 1 -- 172.21.15.167:0/1304906675 >> 172.21.15.167:0/1304906675 conn(0x7f0a3406c450 msgr2=0x7f0a3406c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:49.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.489+0000 7f0a3b600640 1 -- 172.21.15.167:0/1304906675 shutdown_connections 2024-05-11T12:14:49.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.489+0000 7f0a3b600640 1 -- 172.21.15.167:0/1304906675 wait complete. 2024-05-11T12:14:49.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.491+0000 7f0a3b600640 1 Processor -- start 2024-05-11T12:14:49.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.491+0000 7f0a3b600640 1 -- start start 2024-05-11T12:14:49.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.492+0000 7f0a3b600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a340b3d50 0x7f0a341598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:49.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.492+0000 7f0a3b600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a3415ac80 0x7f0a3415f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:49.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.492+0000 7f0a3b600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a341611b0 0x7f0a3415fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:49.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.492+0000 7f0a3b600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0a340b44f0 con 0x7f0a340b3d50 2024-05-11T12:14:49.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.492+0000 7f0a3b600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0a340b41b0 con 0x7f0a341611b0 2024-05-11T12:14:49.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.492+0000 7f0a3b600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0a34073960 con 0x7f0a3415ac80 2024-05-11T12:14:49.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.492+0000 7f0a38c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a340b3d50 0x7f0a341598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:49.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.492+0000 7f0a38c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a340b3d50 0x7f0a341598b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:52282/0 (socket says 172.21.15.167:52282) 2024-05-11T12:14:49.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.492+0000 7f0a38c00640 1 -- 172.21.15.167:0/3811687542 learned_addr learned my addr 172.21.15.167:0/3811687542 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:49.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.493+0000 7f0a2be00640 1 --2- 172.21.15.167:0/3811687542 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a3415ac80 0x7f0a3415f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:49.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.493+0000 7f0a39600640 1 --2- 172.21.15.167:0/3811687542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a341611b0 0x7f0a3415fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:49.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.493+0000 7f0a38c00640 1 -- 172.21.15.167:0/3811687542 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a3415ac80 msgr2=0x7f0a3415f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:49.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.493+0000 7f0a38c00640 1 --2- 172.21.15.167:0/3811687542 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a3415ac80 0x7f0a3415f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:49.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.493+0000 7f0a38c00640 1 -- 172.21.15.167:0/3811687542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a341611b0 msgr2=0x7f0a3415fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:49.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.493+0000 7f0a38c00640 1 --2- 172.21.15.167:0/3811687542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a341611b0 0x7f0a3415fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:49.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.493+0000 7f0a38c00640 1 -- 172.21.15.167:0/3811687542 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a3415e0d0 con 0x7f0a340b3d50 2024-05-11T12:14:49.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.493+0000 7f0a2be00640 1 --2- 172.21.15.167:0/3811687542 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a3415ac80 0x7f0a3415f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:49.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.493+0000 7f0a38c00640 1 --2- 172.21.15.167:0/3811687542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a340b3d50 0x7f0a341598b0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f0a240090b0 tx=0x7f0a240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:49.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.494+0000 7f0a31600640 1 -- 172.21.15.167:0/3811687542 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a24037040 con 0x7f0a340b3d50 2024-05-11T12:14:49.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.494+0000 7f0a31600640 1 -- 172.21.15.167:0/3811687542 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a24004250 con 0x7f0a340b3d50 2024-05-11T12:14:49.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.494+0000 7f0a31600640 1 -- 172.21.15.167:0/3811687542 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2403c670 con 0x7f0a340b3d50 2024-05-11T12:14:49.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.494+0000 7f0a3b600640 1 -- 172.21.15.167:0/3811687542 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a3415e3b0 con 0x7f0a340b3d50 2024-05-11T12:14:49.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.494+0000 7f0a3b600640 1 -- 172.21.15.167:0/3811687542 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a3415e970 con 0x7f0a340b3d50 2024-05-11T12:14:49.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.495+0000 7f0a3b600640 1 -- 172.21.15.167:0/3811687542 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a340694b0 con 0x7f0a340b3d50 2024-05-11T12:14:49.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.496+0000 7f0a31600640 1 -- 172.21.15.167:0/3811687542 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0a2403a070 con 0x7f0a340b3d50 2024-05-11T12:14:49.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.497+0000 7f0a31600640 1 --2- 172.21.15.167:0/3811687542 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0a10076300 0x7f0a100787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:49.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.497+0000 7f0a31600640 1 -- 172.21.15.167:0/3811687542 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6101+0+0 (secure 0 0 0) 0x7f0a240b7fd0 con 0x7f0a340b3d50 2024-05-11T12:14:49.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.497+0000 7f0a2be00640 1 --2- 172.21.15.167:0/3811687542 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0a10076300 0x7f0a100787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:49.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.498+0000 7f0a2be00640 1 --2- 172.21.15.167:0/3811687542 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0a10076300 0x7f0a100787c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0a3406f3a0 tx=0x7f0a1c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:49.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.502+0000 7f0a31600640 1 -- 172.21.15.167:0/3811687542 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0a24041020 con 0x7f0a340b3d50 2024-05-11T12:14:49.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.730+0000 7f0a3b600640 1 -- 172.21.15.167:0/3811687542 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0a34070120 con 0x7f0a340b3d50 2024-05-11T12:14:49.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.731+0000 7f0a31600640 1 -- 172.21.15.167:0/3811687542 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v97) ==== 80+0+0 (secure 0 0 0) 0x7f0a24081700 con 0x7f0a340b3d50 2024-05-11T12:14:49.732 INFO:teuthology.orchestra.run.smithi167.stderr:listed 0 entries 2024-05-11T12:14:49.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.734+0000 7f0a3b600640 1 -- 172.21.15.167:0/3811687542 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0a10076300 msgr2=0x7f0a100787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:49.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.734+0000 7f0a3b600640 1 --2- 172.21.15.167:0/3811687542 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0a10076300 0x7f0a100787c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0a3406f3a0 tx=0x7f0a1c008040 comp rx=0 tx=0).stop 2024-05-11T12:14:49.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.734+0000 7f0a3b600640 1 -- 172.21.15.167:0/3811687542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a340b3d50 msgr2=0x7f0a341598b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:49.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.734+0000 7f0a3b600640 1 --2- 172.21.15.167:0/3811687542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a340b3d50 0x7f0a341598b0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f0a240090b0 tx=0x7f0a240046c0 comp rx=0 tx=0).stop 2024-05-11T12:14:49.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.734+0000 7f0a3b600640 1 -- 172.21.15.167:0/3811687542 shutdown_connections 2024-05-11T12:14:49.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.734+0000 7f0a3b600640 1 --2- 172.21.15.167:0/3811687542 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0a10076300 0x7f0a100787c0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:49.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.734+0000 7f0a3b600640 1 --2- 172.21.15.167:0/3811687542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a341611b0 0x7f0a3415fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:49.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.734+0000 7f0a3b600640 1 --2- 172.21.15.167:0/3811687542 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a3415ac80 0x7f0a3415f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:49.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.734+0000 7f0a3b600640 1 --2- 172.21.15.167:0/3811687542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a340b3d50 0x7f0a341598b0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:49.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.734+0000 7f0a3b600640 1 -- 172.21.15.167:0/3811687542 >> 172.21.15.167:0/3811687542 conn(0x7f0a3406c450 msgr2=0x7f0a3406f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:49.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.735+0000 7f0a3b600640 1 -- 172.21.15.167:0/3811687542 shutdown_connections 2024-05-11T12:14:49.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:49.735+0000 7f0a3b600640 1 -- 172.21.15.167:0/3811687542 wait complete. 2024-05-11T12:14:49.754 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-11T12:14:49.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:49.755 DEBUG:teuthology.orchestra.run.smithi204:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-05-11T12:14:49.856 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi204.front.sepia.ceph.com 2024-05-11T12:14:49.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:49.857 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-11T12:14:49.905 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-11T12:14:49.912 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-11T12:14:50.045 INFO:teuthology.orchestra.run.smithi167.stdout:[global] 2024-05-11T12:14:50.045 INFO:teuthology.orchestra.run.smithi167.stdout: chdir = "" 2024-05-11T12:14:50.045 INFO:teuthology.orchestra.run.smithi167.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-11T12:14:50.046 INFO:teuthology.orchestra.run.smithi167.stdout: auth supported = cephx 2024-05-11T12:14:50.046 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:50.046 INFO:teuthology.orchestra.run.smithi167.stdout: filestore xattr use omap = true 2024-05-11T12:14:50.046 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:50.046 INFO:teuthology.orchestra.run.smithi167.stdout: mon clock drift allowed = 1.000 2024-05-11T12:14:50.046 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:50.046 INFO:teuthology.orchestra.run.smithi167.stdout: osd crush chooseleaf type = 0 2024-05-11T12:14:50.046 INFO:teuthology.orchestra.run.smithi167.stdout: auth debug = true 2024-05-11T12:14:50.046 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:50.046 INFO:teuthology.orchestra.run.smithi167.stdout: ms die on old message = true 2024-05-11T12:14:50.046 INFO:teuthology.orchestra.run.smithi167.stdout: ms die on bug = true 2024-05-11T12:14:50.046 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:50.046 INFO:teuthology.orchestra.run.smithi167.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-11T12:14:50.047 INFO:teuthology.orchestra.run.smithi167.stdout: mon pg warn max object skew = 0 2024-05-11T12:14:50.047 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:50.047 INFO:teuthology.orchestra.run.smithi167.stdout: # disable pg_autoscaler by default for new pools 2024-05-11T12:14:50.047 INFO:teuthology.orchestra.run.smithi167.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-11T12:14:50.047 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:50.047 INFO:teuthology.orchestra.run.smithi167.stdout: osd pool default size = 2 2024-05-11T12:14:50.047 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:50.047 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd allow primary affinity = true 2024-05-11T12:14:50.047 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd allow pg remap = true 2024-05-11T12:14:50.047 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on legacy crush tunables = false 2024-05-11T12:14:50.047 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on crush straw calc version zero = false 2024-05-11T12:14:50.047 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on no sortbitwise = false 2024-05-11T12:14:50.047 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on osd down out interval zero = false 2024-05-11T12:14:50.047 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on too few osds = false 2024-05-11T12:14:50.047 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-11T12:14:50.047 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-11T12:14:50.048 INFO:teuthology.orchestra.run.smithi167.stdout: mon_allow_pool_size_one = true 2024-05-11T12:14:50.048 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:50.048 INFO:teuthology.orchestra.run.smithi167.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-11T12:14:50.048 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:50.048 INFO:teuthology.orchestra.run.smithi167.stdout: osd default data pool replay window = 5 2024-05-11T12:14:50.048 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:50.048 INFO:teuthology.orchestra.run.smithi167.stdout: mon allow pool delete = true 2024-05-11T12:14:50.048 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:50.048 INFO:teuthology.orchestra.run.smithi167.stdout: mon cluster log file level = debug 2024-05-11T12:14:50.048 INFO:teuthology.orchestra.run.smithi167.stdout: debug asserts on shutdown = true 2024-05-11T12:14:50.048 INFO:teuthology.orchestra.run.smithi167.stdout: mon health detail to clog = false 2024-05-11T12:14:50.048 INFO:teuthology.orchestra.run.smithi167.stdout: mon host = "172.21.15.167,172.21.15.204,[v2:172.21.15.204:3301,v1:172.21.15.204:6790]" 2024-05-11T12:14:50.048 INFO:teuthology.orchestra.run.smithi167.stdout: ms die on skipped message = False 2024-05-11T12:14:50.048 INFO:teuthology.orchestra.run.smithi167.stdout: fsid = 5f32d344-7289-4c27-881b-1533419ab678 2024-05-11T12:14:50.049 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:50.049 INFO:teuthology.orchestra.run.smithi167.stdout:[osd] 2024-05-11T12:14:50.049 INFO:teuthology.orchestra.run.smithi167.stdout: osd journal size = 100 2024-05-11T12:14:50.049 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:50.049 INFO:teuthology.orchestra.run.smithi167.stdout: osd scrub load threshold = 5.0 2024-05-11T12:14:50.049 INFO:teuthology.orchestra.run.smithi167.stdout: osd scrub max interval = 600 2024-05-11T12:14:50.049 INFO:teuthology.orchestra.run.smithi167.stdout: osd mclock profile = high_recovery_ops 2024-05-11T12:14:50.049 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:50.049 INFO:teuthology.orchestra.run.smithi167.stdout: osd recover clone overlap = true 2024-05-11T12:14:50.049 INFO:teuthology.orchestra.run.smithi167.stdout: osd recovery max chunk = 1048576 2024-05-11T12:14:50.049 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:50.049 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug shutdown = true 2024-05-11T12:14:50.049 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug op order = true 2024-05-11T12:14:50.050 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug verify stray on activate = true 2024-05-11T12:14:50.050 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug trim objects = true 2024-05-11T12:14:50.050 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:50.050 INFO:teuthology.orchestra.run.smithi167.stdout: osd open classes on start = true 2024-05-11T12:14:50.050 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug pg log writeout = true 2024-05-11T12:14:50.050 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:50.050 INFO:teuthology.orchestra.run.smithi167.stdout: osd deep scrub update digest min age = 30 2024-05-11T12:14:50.050 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:50.050 INFO:teuthology.orchestra.run.smithi167.stdout: osd map max advance = 10 2024-05-11T12:14:50.050 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:50.050 INFO:teuthology.orchestra.run.smithi167.stdout: journal zero on create = true 2024-05-11T12:14:50.050 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:50.050 INFO:teuthology.orchestra.run.smithi167.stdout: filestore ondisk finisher threads = 3 2024-05-11T12:14:50.050 INFO:teuthology.orchestra.run.smithi167.stdout: filestore apply finisher threads = 3 2024-05-11T12:14:50.051 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:50.051 INFO:teuthology.orchestra.run.smithi167.stdout: bdev debug aio = true 2024-05-11T12:14:50.051 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug misdirected ops = true 2024-05-11T12:14:50.051 INFO:teuthology.orchestra.run.smithi167.stdout: bluestore block size = 96636764160 2024-05-11T12:14:50.051 INFO:teuthology.orchestra.run.smithi167.stdout: bluestore fsck on mount = True 2024-05-11T12:14:50.051 INFO:teuthology.orchestra.run.smithi167.stdout: debug bluefs = 20 2024-05-11T12:14:50.051 INFO:teuthology.orchestra.run.smithi167.stdout: debug bluestore = 20 2024-05-11T12:14:50.051 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2024-05-11T12:14:50.051 INFO:teuthology.orchestra.run.smithi167.stdout: debug osd = 20 2024-05-11T12:14:50.051 INFO:teuthology.orchestra.run.smithi167.stdout: debug rocksdb = 10 2024-05-11T12:14:50.051 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd backfillfull_ratio = 0.85 2024-05-11T12:14:50.051 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd full ratio = 0.9 2024-05-11T12:14:50.051 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd nearfull ratio = 0.8 2024-05-11T12:14:50.051 INFO:teuthology.orchestra.run.smithi167.stdout: osd failsafe full ratio = 0.95 2024-05-11T12:14:50.051 INFO:teuthology.orchestra.run.smithi167.stdout: osd objectstore = bluestore 2024-05-11T12:14:50.052 INFO:teuthology.orchestra.run.smithi167.stdout: osd op complaint time = 180 2024-05-11T12:14:50.052 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:50.052 INFO:teuthology.orchestra.run.smithi167.stdout:[mgr] 2024-05-11T12:14:50.052 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2024-05-11T12:14:50.052 INFO:teuthology.orchestra.run.smithi167.stdout: debug mgr = 20 2024-05-11T12:14:50.052 INFO:teuthology.orchestra.run.smithi167.stdout: debug mon = 20 2024-05-11T12:14:50.052 INFO:teuthology.orchestra.run.smithi167.stdout: debug auth = 20 2024-05-11T12:14:50.052 INFO:teuthology.orchestra.run.smithi167.stdout: mon reweight min pgs per osd = 4 2024-05-11T12:14:50.052 INFO:teuthology.orchestra.run.smithi167.stdout: mon reweight min bytes per osd = 10 2024-05-11T12:14:50.052 INFO:teuthology.orchestra.run.smithi167.stdout: mgr/telemetry/nag = false 2024-05-11T12:14:50.052 INFO:teuthology.orchestra.run.smithi167.stdout: debug client = 20 2024-05-11T12:14:50.052 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:50.052 INFO:teuthology.orchestra.run.smithi167.stdout:[mon] 2024-05-11T12:14:50.052 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2024-05-11T12:14:50.052 INFO:teuthology.orchestra.run.smithi167.stdout: debug mon = 20 2024-05-11T12:14:50.052 INFO:teuthology.orchestra.run.smithi167.stdout: debug paxos = 20 2024-05-11T12:14:50.052 INFO:teuthology.orchestra.run.smithi167.stdout: debug auth = 20 2024-05-11T12:14:50.053 INFO:teuthology.orchestra.run.smithi167.stdout: mon data avail warn = 5 2024-05-11T12:14:50.053 INFO:teuthology.orchestra.run.smithi167.stdout: mon mgr mkfs grace = 240 2024-05-11T12:14:50.053 INFO:teuthology.orchestra.run.smithi167.stdout: mon reweight min pgs per osd = 4 2024-05-11T12:14:50.053 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd reporter subtree level = osd 2024-05-11T12:14:50.053 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd prime pg temp = true 2024-05-11T12:14:50.053 INFO:teuthology.orchestra.run.smithi167.stdout: mon reweight min bytes per osd = 10 2024-05-11T12:14:50.053 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:50.053 INFO:teuthology.orchestra.run.smithi167.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-11T12:14:50.053 INFO:teuthology.orchestra.run.smithi167.stdout: auth mon ticket ttl = 660 # 11m 2024-05-11T12:14:50.053 INFO:teuthology.orchestra.run.smithi167.stdout: auth service ticket ttl = 240 # 4m 2024-05-11T12:14:50.053 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:50.053 INFO:teuthology.orchestra.run.smithi167.stdout: # don't complain about insecure global_id in the test suite 2024-05-11T12:14:50.053 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-11T12:14:50.053 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-11T12:14:50.053 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:50.054 INFO:teuthology.orchestra.run.smithi167.stdout: # 1m isn't quite enough 2024-05-11T12:14:50.054 INFO:teuthology.orchestra.run.smithi167.stdout: mon_down_mkfs_grace = 2m 2024-05-11T12:14:50.054 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:50.054 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_filestore_osds = false 2024-05-11T12:14:50.054 INFO:teuthology.orchestra.run.smithi167.stdout: mon down mkfs grace = 300 2024-05-11T12:14:50.054 INFO:teuthology.orchestra.run.smithi167.stdout: mon op complaint time = 120 2024-05-11T12:14:50.054 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:50.054 INFO:teuthology.orchestra.run.smithi167.stdout:[client] 2024-05-11T12:14:50.054 INFO:teuthology.orchestra.run.smithi167.stdout: rgw cache enabled = true 2024-05-11T12:14:50.054 INFO:teuthology.orchestra.run.smithi167.stdout: rgw enable ops log = true 2024-05-11T12:14:50.054 INFO:teuthology.orchestra.run.smithi167.stdout: rgw enable usage log = true 2024-05-11T12:14:50.054 INFO:teuthology.orchestra.run.smithi167.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-11T12:14:50.055 INFO:teuthology.orchestra.run.smithi167.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-11T12:14:50.055 INFO:teuthology.orchestra.run.smithi167.stdout: client mount timeout = 600 2024-05-11T12:14:50.055 INFO:teuthology.orchestra.run.smithi167.stdout: debug client = 20 2024-05-11T12:14:50.055 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2024-05-11T12:14:50.055 INFO:teuthology.orchestra.run.smithi167.stdout: pid file = "" 2024-05-11T12:14:50.055 INFO:teuthology.orchestra.run.smithi167.stdout: rados mon op timeout = 900 2024-05-11T12:14:50.055 INFO:teuthology.orchestra.run.smithi167.stdout: rados osd op timeout = 900 2024-05-11T12:14:50.055 INFO:teuthology.orchestra.run.smithi167.stdout:[mon.a] 2024-05-11T12:14:50.055 INFO:teuthology.orchestra.run.smithi167.stdout:[mon.b] 2024-05-11T12:14:50.055 INFO:teuthology.orchestra.run.smithi167.stdout:[mon.c] 2024-05-11T12:14:50.055 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.a] 2024-05-11T12:14:50.055 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.c] 2024-05-11T12:14:50.055 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.b] 2024-05-11T12:14:50.056 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.d] 2024-05-11T12:14:50.056 INFO:teuthology.orchestra.run.smithi167.stdout:[mds] 2024-05-11T12:14:50.056 INFO:teuthology.orchestra.run.smithi167.stdout: debug mds = 20 2024-05-11T12:14:50.056 INFO:teuthology.orchestra.run.smithi167.stdout: debug mds balancer = 20 2024-05-11T12:14:50.056 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2024-05-11T12:14:50.056 INFO:teuthology.orchestra.run.smithi167.stdout: mds debug frag = True 2024-05-11T12:14:50.056 INFO:teuthology.orchestra.run.smithi167.stdout: mds debug scatterstat = True 2024-05-11T12:14:50.056 INFO:teuthology.orchestra.run.smithi167.stdout: mds op complaint time = 180 2024-05-11T12:14:50.056 INFO:teuthology.orchestra.run.smithi167.stdout: mds verify scatter = True 2024-05-11T12:14:50.056 INFO:teuthology.orchestra.run.smithi167.stdout: osd op complaint time = 180 2024-05-11T12:14:50.056 INFO:teuthology.orchestra.run.smithi167.stdout: rados mon op timeout = 900 2024-05-11T12:14:50.056 INFO:teuthology.orchestra.run.smithi167.stdout: rados osd op timeout = 900 2024-05-11T12:14:50.056 INFO:teuthology.orchestra.run.smithi167.stdout: mds_sessionmap_keys_per_op = 1 2024-05-11T12:14:50.057 INFO:teuthology.orchestra.run.smithi204.stdout:[global] 2024-05-11T12:14:50.057 INFO:teuthology.orchestra.run.smithi204.stdout: chdir = "" 2024-05-11T12:14:50.057 INFO:teuthology.orchestra.run.smithi204.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-11T12:14:50.057 INFO:teuthology.orchestra.run.smithi204.stdout: auth supported = cephx 2024-05-11T12:14:50.057 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:14:50.057 INFO:teuthology.orchestra.run.smithi204.stdout: filestore xattr use omap = true 2024-05-11T12:14:50.057 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:14:50.057 INFO:teuthology.orchestra.run.smithi204.stdout: mon clock drift allowed = 1.000 2024-05-11T12:14:50.057 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:14:50.058 INFO:teuthology.orchestra.run.smithi204.stdout: osd crush chooseleaf type = 0 2024-05-11T12:14:50.058 INFO:teuthology.orchestra.run.smithi204.stdout: auth debug = true 2024-05-11T12:14:50.058 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:14:50.058 INFO:teuthology.orchestra.run.smithi204.stdout: ms die on old message = true 2024-05-11T12:14:50.058 INFO:teuthology.orchestra.run.smithi204.stdout: ms die on bug = true 2024-05-11T12:14:50.058 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:14:50.058 INFO:teuthology.orchestra.run.smithi204.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-11T12:14:50.058 INFO:teuthology.orchestra.run.smithi204.stdout: mon pg warn max object skew = 0 2024-05-11T12:14:50.058 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:14:50.058 INFO:teuthology.orchestra.run.smithi204.stdout: # disable pg_autoscaler by default for new pools 2024-05-11T12:14:50.058 INFO:teuthology.orchestra.run.smithi204.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-11T12:14:50.058 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:14:50.058 INFO:teuthology.orchestra.run.smithi204.stdout: osd pool default size = 2 2024-05-11T12:14:50.059 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:14:50.059 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd allow primary affinity = true 2024-05-11T12:14:50.059 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd allow pg remap = true 2024-05-11T12:14:50.059 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on legacy crush tunables = false 2024-05-11T12:14:50.059 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on crush straw calc version zero = false 2024-05-11T12:14:50.059 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on no sortbitwise = false 2024-05-11T12:14:50.059 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on osd down out interval zero = false 2024-05-11T12:14:50.059 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on too few osds = false 2024-05-11T12:14:50.059 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-11T12:14:50.059 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-11T12:14:50.059 INFO:teuthology.orchestra.run.smithi204.stdout: mon_allow_pool_size_one = true 2024-05-11T12:14:50.059 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:14:50.059 INFO:teuthology.orchestra.run.smithi204.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-11T12:14:50.060 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:14:50.060 INFO:teuthology.orchestra.run.smithi204.stdout: osd default data pool replay window = 5 2024-05-11T12:14:50.060 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:14:50.060 INFO:teuthology.orchestra.run.smithi204.stdout: mon allow pool delete = true 2024-05-11T12:14:50.060 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:14:50.060 INFO:teuthology.orchestra.run.smithi204.stdout: mon cluster log file level = debug 2024-05-11T12:14:50.060 INFO:teuthology.orchestra.run.smithi204.stdout: debug asserts on shutdown = true 2024-05-11T12:14:50.060 INFO:teuthology.orchestra.run.smithi204.stdout: mon health detail to clog = false 2024-05-11T12:14:50.060 INFO:teuthology.orchestra.run.smithi204.stdout: mon host = "172.21.15.167,172.21.15.204,[v2:172.21.15.204:3301,v1:172.21.15.204:6790]" 2024-05-11T12:14:50.060 INFO:teuthology.orchestra.run.smithi204.stdout: ms die on skipped message = False 2024-05-11T12:14:50.060 INFO:teuthology.orchestra.run.smithi204.stdout: fsid = 5f32d344-7289-4c27-881b-1533419ab678 2024-05-11T12:14:50.060 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:14:50.060 INFO:teuthology.orchestra.run.smithi204.stdout:[osd] 2024-05-11T12:14:50.060 INFO:teuthology.orchestra.run.smithi204.stdout: osd journal size = 100 2024-05-11T12:14:50.060 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:14:50.061 INFO:teuthology.orchestra.run.smithi204.stdout: osd scrub load threshold = 5.0 2024-05-11T12:14:50.061 INFO:teuthology.orchestra.run.smithi204.stdout: osd scrub max interval = 600 2024-05-11T12:14:50.061 INFO:teuthology.orchestra.run.smithi204.stdout: osd mclock profile = high_recovery_ops 2024-05-11T12:14:50.061 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:14:50.061 INFO:teuthology.orchestra.run.smithi204.stdout: osd recover clone overlap = true 2024-05-11T12:14:50.061 INFO:teuthology.orchestra.run.smithi204.stdout: osd recovery max chunk = 1048576 2024-05-11T12:14:50.061 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:14:50.061 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug shutdown = true 2024-05-11T12:14:50.061 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug op order = true 2024-05-11T12:14:50.061 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug verify stray on activate = true 2024-05-11T12:14:50.061 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug trim objects = true 2024-05-11T12:14:50.061 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:14:50.062 INFO:teuthology.orchestra.run.smithi204.stdout: osd open classes on start = true 2024-05-11T12:14:50.062 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug pg log writeout = true 2024-05-11T12:14:50.062 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:14:50.062 INFO:teuthology.orchestra.run.smithi204.stdout: osd deep scrub update digest min age = 30 2024-05-11T12:14:50.062 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:14:50.062 INFO:teuthology.orchestra.run.smithi204.stdout: osd map max advance = 10 2024-05-11T12:14:50.062 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:14:50.062 INFO:teuthology.orchestra.run.smithi204.stdout: journal zero on create = true 2024-05-11T12:14:50.062 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:14:50.062 INFO:teuthology.orchestra.run.smithi204.stdout: filestore ondisk finisher threads = 3 2024-05-11T12:14:50.062 INFO:teuthology.orchestra.run.smithi204.stdout: filestore apply finisher threads = 3 2024-05-11T12:14:50.062 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:14:50.063 INFO:teuthology.orchestra.run.smithi204.stdout: bdev debug aio = true 2024-05-11T12:14:50.063 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug misdirected ops = true 2024-05-11T12:14:50.063 INFO:teuthology.orchestra.run.smithi204.stdout: bluestore block size = 96636764160 2024-05-11T12:14:50.063 INFO:teuthology.orchestra.run.smithi204.stdout: bluestore fsck on mount = True 2024-05-11T12:14:50.063 INFO:teuthology.orchestra.run.smithi204.stdout: debug bluefs = 20 2024-05-11T12:14:50.063 INFO:teuthology.orchestra.run.smithi204.stdout: debug bluestore = 20 2024-05-11T12:14:50.063 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-05-11T12:14:50.063 INFO:teuthology.orchestra.run.smithi204.stdout: debug osd = 20 2024-05-11T12:14:50.063 INFO:teuthology.orchestra.run.smithi204.stdout: debug rocksdb = 10 2024-05-11T12:14:50.063 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd backfillfull_ratio = 0.85 2024-05-11T12:14:50.063 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd full ratio = 0.9 2024-05-11T12:14:50.063 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd nearfull ratio = 0.8 2024-05-11T12:14:50.063 INFO:teuthology.orchestra.run.smithi204.stdout: osd failsafe full ratio = 0.95 2024-05-11T12:14:50.063 INFO:teuthology.orchestra.run.smithi204.stdout: osd objectstore = bluestore 2024-05-11T12:14:50.063 INFO:teuthology.orchestra.run.smithi204.stdout: osd op complaint time = 180 2024-05-11T12:14:50.064 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:14:50.064 INFO:teuthology.orchestra.run.smithi204.stdout:[mgr] 2024-05-11T12:14:50.064 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-05-11T12:14:50.064 INFO:teuthology.orchestra.run.smithi204.stdout: debug mgr = 20 2024-05-11T12:14:50.064 INFO:teuthology.orchestra.run.smithi204.stdout: debug mon = 20 2024-05-11T12:14:50.064 INFO:teuthology.orchestra.run.smithi204.stdout: debug auth = 20 2024-05-11T12:14:50.064 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min pgs per osd = 4 2024-05-11T12:14:50.064 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min bytes per osd = 10 2024-05-11T12:14:50.064 INFO:teuthology.orchestra.run.smithi204.stdout: mgr/telemetry/nag = false 2024-05-11T12:14:50.064 INFO:teuthology.orchestra.run.smithi204.stdout: debug client = 20 2024-05-11T12:14:50.064 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:14:50.064 INFO:teuthology.orchestra.run.smithi204.stdout:[mon] 2024-05-11T12:14:50.064 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-05-11T12:14:50.065 INFO:teuthology.orchestra.run.smithi204.stdout: debug mon = 20 2024-05-11T12:14:50.065 INFO:teuthology.orchestra.run.smithi204.stdout: debug paxos = 20 2024-05-11T12:14:50.065 INFO:teuthology.orchestra.run.smithi204.stdout: debug auth = 20 2024-05-11T12:14:50.065 INFO:teuthology.orchestra.run.smithi204.stdout: mon data avail warn = 5 2024-05-11T12:14:50.065 INFO:teuthology.orchestra.run.smithi204.stdout: mon mgr mkfs grace = 240 2024-05-11T12:14:50.065 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min pgs per osd = 4 2024-05-11T12:14:50.065 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd reporter subtree level = osd 2024-05-11T12:14:50.065 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd prime pg temp = true 2024-05-11T12:14:50.065 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min bytes per osd = 10 2024-05-11T12:14:50.065 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:14:50.065 INFO:teuthology.orchestra.run.smithi204.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-11T12:14:50.065 INFO:teuthology.orchestra.run.smithi204.stdout: auth mon ticket ttl = 660 # 11m 2024-05-11T12:14:50.065 INFO:teuthology.orchestra.run.smithi204.stdout: auth service ticket ttl = 240 # 4m 2024-05-11T12:14:50.065 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:14:50.065 INFO:teuthology.orchestra.run.smithi204.stdout: # don't complain about insecure global_id in the test suite 2024-05-11T12:14:50.066 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-11T12:14:50.066 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-11T12:14:50.066 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:14:50.066 INFO:teuthology.orchestra.run.smithi204.stdout: # 1m isn't quite enough 2024-05-11T12:14:50.066 INFO:teuthology.orchestra.run.smithi204.stdout: mon_down_mkfs_grace = 2m 2024-05-11T12:14:50.066 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:14:50.066 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_filestore_osds = false 2024-05-11T12:14:50.066 INFO:teuthology.orchestra.run.smithi204.stdout: mon down mkfs grace = 300 2024-05-11T12:14:50.066 INFO:teuthology.orchestra.run.smithi204.stdout: mon op complaint time = 120 2024-05-11T12:14:50.066 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:14:50.066 INFO:teuthology.orchestra.run.smithi204.stdout:[client] 2024-05-11T12:14:50.066 INFO:teuthology.orchestra.run.smithi204.stdout: rgw cache enabled = true 2024-05-11T12:14:50.066 INFO:teuthology.orchestra.run.smithi204.stdout: rgw enable ops log = true 2024-05-11T12:14:50.066 INFO:teuthology.orchestra.run.smithi204.stdout: rgw enable usage log = true 2024-05-11T12:14:50.067 INFO:teuthology.orchestra.run.smithi204.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-11T12:14:50.067 INFO:teuthology.orchestra.run.smithi204.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-11T12:14:50.067 INFO:teuthology.orchestra.run.smithi204.stdout: client mount timeout = 600 2024-05-11T12:14:50.067 INFO:teuthology.orchestra.run.smithi204.stdout: debug client = 20 2024-05-11T12:14:50.067 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-05-11T12:14:50.067 INFO:teuthology.orchestra.run.smithi204.stdout: pid file = "" 2024-05-11T12:14:50.067 INFO:teuthology.orchestra.run.smithi204.stdout: rados mon op timeout = 900 2024-05-11T12:14:50.067 INFO:teuthology.orchestra.run.smithi204.stdout: rados osd op timeout = 900 2024-05-11T12:14:50.067 INFO:teuthology.orchestra.run.smithi204.stdout:[mon.a] 2024-05-11T12:14:50.067 INFO:teuthology.orchestra.run.smithi204.stdout:[mon.b] 2024-05-11T12:14:50.067 INFO:teuthology.orchestra.run.smithi204.stdout:[mon.c] 2024-05-11T12:14:50.067 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.a] 2024-05-11T12:14:50.067 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.c] 2024-05-11T12:14:50.067 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.b] 2024-05-11T12:14:50.068 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.d] 2024-05-11T12:14:50.068 INFO:teuthology.orchestra.run.smithi204.stdout:[mds] 2024-05-11T12:14:50.068 INFO:teuthology.orchestra.run.smithi204.stdout: debug mds = 20 2024-05-11T12:14:50.068 INFO:teuthology.orchestra.run.smithi204.stdout: debug mds balancer = 20 2024-05-11T12:14:50.068 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-05-11T12:14:50.068 INFO:teuthology.orchestra.run.smithi204.stdout: mds debug frag = True 2024-05-11T12:14:50.068 INFO:teuthology.orchestra.run.smithi204.stdout: mds debug scatterstat = True 2024-05-11T12:14:50.068 INFO:teuthology.orchestra.run.smithi204.stdout: mds op complaint time = 180 2024-05-11T12:14:50.068 INFO:teuthology.orchestra.run.smithi204.stdout: mds verify scatter = True 2024-05-11T12:14:50.068 INFO:teuthology.orchestra.run.smithi204.stdout: osd op complaint time = 180 2024-05-11T12:14:50.068 INFO:teuthology.orchestra.run.smithi204.stdout: rados mon op timeout = 900 2024-05-11T12:14:50.068 INFO:teuthology.orchestra.run.smithi204.stdout: rados osd op timeout = 900 2024-05-11T12:14:50.068 INFO:teuthology.orchestra.run.smithi204.stdout: mds_sessionmap_keys_per_op = 1 2024-05-11T12:14:50.085 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-11T12:14:50.085 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T12:14:50.085 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-11T12:14:50.085 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T12:14:50.086 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-11T12:14:50.086 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T12:14:50.086 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-11T12:14:50.086 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T12:14:50.104 INFO:tasks.ceph.mds.b:Stopped 2024-05-11T12:14:50.104 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail b 2024-05-11T12:14:50.105 INFO:tasks.ceph.mds.c:Stopped 2024-05-11T12:14:50.105 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail c 2024-05-11T12:14:50.106 INFO:tasks.ceph.mds.a:Stopped 2024-05-11T12:14:50.107 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail a 2024-05-11T12:14:50.107 INFO:tasks.ceph.mds.d:Stopped 2024-05-11T12:14:50.108 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail d 2024-05-11T12:14:50.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.330+0000 7f65b3200640 1 Processor -- start 2024-05-11T12:14:50.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.330+0000 7f65b3200640 1 -- start start 2024-05-11T12:14:50.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.331+0000 7f65b3200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f65b40d3bf0 0x7f65b40cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:50.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.331+0000 7f65b3200640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65b40c9600 con 0x7f65b415f4d0 2024-05-11T12:14:50.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.331+0000 7f65b3200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65b40c98e0 con 0x7f65b40d15a0 2024-05-11T12:14:50.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.331+0000 7f65b3200640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f65b40d5e90 con 0x7f65b40d3bf0 2024-05-11T12:14:50.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.332+0000 7f65b1400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f65b40d3bf0 0x7f65b40cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:50.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.332+0000 7f65b1400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f65b40d3bf0 0x7f65b40cfb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:52294/0 (socket says 172.21.15.167:52294) 2024-05-11T12:14:50.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.332+0000 7f65b1400640 1 -- 172.21.15.167:0/437014772 learned_addr learned my addr 172.21.15.167:0/437014772 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:50.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.332+0000 7f65b1400640 1 -- 172.21.15.167:0/437014772 >> v1:172.21.15.204:6789/0 conn(0x7f65b40d15a0 legacy=0x7f65b40d0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:50.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.332+0000 7f65b1400640 1 -- 172.21.15.167:0/437014772 >> v1:172.21.15.167:6789/0 conn(0x7f65b415f4d0 legacy=0x7f65b40d34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:50.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.332+0000 7f65b1400640 1 -- 172.21.15.167:0/437014772 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b40c9bc0 con 0x7f65b40d3bf0 2024-05-11T12:14:50.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.334+0000 7f65b1400640 1 --2- 172.21.15.167:0/437014772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65b40d3bf0 0x7f65b40cfb10 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f65a0009870 tx=0x7f65a002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3513176f1c834cd3 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:50.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.334+0000 7f65b0a00640 1 -- 172.21.15.167:0/437014772 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a0037070 con 0x7f65b40d3bf0 2024-05-11T12:14:50.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.334+0000 7f65b0a00640 1 -- 172.21.15.167:0/437014772 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65a0032440 con 0x7f65b40d3bf0 2024-05-11T12:14:50.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.334+0000 7f65b0a00640 1 -- 172.21.15.167:0/437014772 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a0032810 con 0x7f65b40d3bf0 2024-05-11T12:14:50.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.335+0000 7f65b3200640 1 -- 172.21.15.167:0/437014772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65b40d3bf0 msgr2=0x7f65b40cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:50.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.335+0000 7f65b3200640 1 --2- 172.21.15.167:0/437014772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65b40d3bf0 0x7f65b40cfb10 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f65a0009870 tx=0x7f65a002a250 comp rx=0 tx=0).stop 2024-05-11T12:14:50.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.335+0000 7f65b3200640 1 -- 172.21.15.167:0/437014772 shutdown_connections 2024-05-11T12:14:50.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.335+0000 7f65b3200640 1 --2- 172.21.15.167:0/437014772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65b40d3bf0 0x7f65b40cfb10 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:50.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.335+0000 7f65b3200640 1 -- 172.21.15.167:0/437014772 >> 172.21.15.167:0/437014772 conn(0x7f65b40d4f90 msgr2=0x7f65b40d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:50.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.335+0000 7f65b3200640 1 -- 172.21.15.167:0/437014772 shutdown_connections 2024-05-11T12:14:50.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.335+0000 7f65b3200640 1 -- 172.21.15.167:0/437014772 wait complete. 2024-05-11T12:14:50.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.337+0000 7f65b3200640 1 Processor -- start 2024-05-11T12:14:50.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.337+0000 7f65b3200640 1 -- start start 2024-05-11T12:14:50.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.338+0000 7f65b3200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65b40d15a0 0x7f65b414f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:50.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.338+0000 7f65b3200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65b40d3bf0 0x7f65b41605b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:50.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.338+0000 7f65b3200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65b415f4d0 0x7f65b40a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:50.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.338+0000 7f65b3200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f65b40cb3d0 con 0x7f65b40d3bf0 2024-05-11T12:14:50.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.338+0000 7f65b3200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f65b40c9f30 con 0x7f65b415f4d0 2024-05-11T12:14:50.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.338+0000 7f65b3200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f65b40cb550 con 0x7f65b40d15a0 2024-05-11T12:14:50.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.338+0000 7f65b1400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65b40d3bf0 0x7f65b41605b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:50.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.338+0000 7f65b1400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65b40d3bf0 0x7f65b41605b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:52298/0 (socket says 172.21.15.167:52298) 2024-05-11T12:14:50.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.338+0000 7f65b1400640 1 -- 172.21.15.167:0/3914910747 learned_addr learned my addr 172.21.15.167:0/3914910747 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:50.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.339+0000 7f65b2800640 1 --2- 172.21.15.167:0/3914910747 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65b415f4d0 0x7f65b40a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:50.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.339+0000 7f65b1e00640 1 --2- 172.21.15.167:0/3914910747 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65b40d15a0 0x7f65b414f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:50.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.339+0000 7fb0c6800640 1 Processor -- start 2024-05-11T12:14:50.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.339+0000 7f65b1400640 1 -- 172.21.15.167:0/3914910747 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65b40d15a0 msgr2=0x7f65b414f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:50.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.339+0000 7f65b1400640 1 --2- 172.21.15.167:0/3914910747 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65b40d15a0 0x7f65b414f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:50.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.339+0000 7f65b1400640 1 -- 172.21.15.167:0/3914910747 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65b415f4d0 msgr2=0x7f65b40a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:50.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.339+0000 7f65b1400640 1 --2- 172.21.15.167:0/3914910747 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65b415f4d0 0x7f65b40a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:50.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.339+0000 7fac70a00640 1 Processor -- start 2024-05-11T12:14:50.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.339+0000 7f65b1400640 1 -- 172.21.15.167:0/3914910747 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b40a11c0 con 0x7f65b40d3bf0 2024-05-11T12:14:50.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.339+0000 7fac70a00640 1 -- start start 2024-05-11T12:14:50.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.339+0000 7fb0c6800640 1 -- start start 2024-05-11T12:14:50.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.339+0000 7eff0a400640 1 Processor -- start 2024-05-11T12:14:50.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.340+0000 7f65b1400640 1 --2- 172.21.15.167:0/3914910747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65b40d3bf0 0x7f65b41605b0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f65a0009870 tx=0x7f65a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:50.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.340+0000 7eff0a400640 1 -- start start 2024-05-11T12:14:50.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.340+0000 7f65aa000640 1 -- 172.21.15.167:0/3914910747 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a0041070 con 0x7f65b40d3bf0 2024-05-11T12:14:50.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.340+0000 7f65aa000640 1 -- 172.21.15.167:0/3914910747 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65a003c870 con 0x7f65b40d3bf0 2024-05-11T12:14:50.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.340+0000 7f65b3200640 1 -- 172.21.15.167:0/3914910747 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65b40a1480 con 0x7f65b40d3bf0 2024-05-11T12:14:50.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.340+0000 7f65aa000640 1 -- 172.21.15.167:0/3914910747 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a0037040 con 0x7f65b40d3bf0 2024-05-11T12:14:50.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.340+0000 7f65b3200640 1 -- 172.21.15.167:0/3914910747 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65b40a1a40 con 0x7f65b40d3bf0 2024-05-11T12:14:50.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.341+0000 7fb0c6800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb0c0163730 0x7fb0c006e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:50.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.341+0000 7fb0c6800640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb0c0168180 0x7fb0c00b6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:50.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.341+0000 7fb0c6800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0c00d3550 0x7fb0c00d39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:50.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.341+0000 7fb0c6800640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fb0c00d3ef0 con 0x7fb0c0168180 2024-05-11T12:14:50.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.341+0000 7fb0c6800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fb0c006ec20 con 0x7fb0c0163730 2024-05-11T12:14:50.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.341+0000 7fb0c6800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb0c00b6cd0 con 0x7fb0c00d3550 2024-05-11T12:14:50.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.341+0000 7fac70a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fac6c15f410 0x7fac6c0d2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:50.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.341+0000 7fac70a00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fac6c0d4720 0x7fac6c0b6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:50.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.341+0000 7fac70a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fac6c0b71c0 0x7fac6c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:50.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.341+0000 7fac70a00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fac6c0b7630 con 0x7fac6c0d4720 2024-05-11T12:14:50.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.341+0000 7fac70a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fac6c15f7f0 con 0x7fac6c15f410 2024-05-11T12:14:50.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.341+0000 7fac70a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fac6c0d3620 con 0x7fac6c0b71c0 2024-05-11T12:14:50.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.341+0000 7eff0a400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7eff040d3ed0 0x7eff040cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:50.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.341+0000 7eff0a400640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff040d01d0 con 0x7eff04162920 2024-05-11T12:14:50.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.341+0000 7eff0a400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff040d04b0 con 0x7eff040d39d0 2024-05-11T12:14:50.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.341+0000 7eff0a400640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7eff040d0de0 con 0x7eff040d3ed0 2024-05-11T12:14:50.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.342+0000 7fb0bea00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb0c0168180 0x7fb0c00b6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:50.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.342+0000 7fac6a800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fac6c0b71c0 0x7fac6c06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:50.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.342+0000 7fac6a800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fac6c0b71c0 0x7fac6c06d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:39248/0 (socket says 172.21.15.167:39248) 2024-05-11T12:14:50.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.342+0000 7fac6a800640 1 -- 172.21.15.167:0/3009019226 learned_addr learned my addr 172.21.15.167:0/3009019226 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:50.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.342+0000 7fb0bf400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb0c0163730 0x7fb0c006e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:50.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.342+0000 7fb0bf400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb0c0163730 0x7fb0c006e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.167:42556/0 (socket says 172.21.15.167:42556) 2024-05-11T12:14:50.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.342+0000 7fb0bf400640 1 -- 172.21.15.167:0/3870248872 learned_addr learned my addr 172.21.15.167:0/3870248872 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:50.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.342+0000 7fac69400640 1 --2- 172.21.15.167:0/3009019226 >> v2:172.21.15.167:3300/0 conn(0x7fac6c0d4720 0x7fac6c0b6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:50.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.342+0000 7fb0bfe00640 1 --2- 172.21.15.167:0/3870248872 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0c00d3550 0x7fb0c00d39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:50.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.342+0000 7fac69e00640 1 --2- 172.21.15.167:0/3009019226 >> v2:172.21.15.204:3300/0 conn(0x7fac6c15f410 0x7fac6c0d2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:50.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.342+0000 7f65b3200640 1 -- 172.21.15.167:0/3914910747 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65b4069650 con 0x7f65b40d3bf0 2024-05-11T12:14:50.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.342+0000 7eff09000640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7eff04162920 0x7eff040d32c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:43222/0 (socket says 172.21.15.167:43222) 2024-05-11T12:14:50.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.342+0000 7eff09000640 1 -- 172.21.15.167:0/1381174424 learned_addr learned my addr 172.21.15.167:0/1381174424 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:14:50.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.343+0000 7eff03e00640 1 --2- 172.21.15.167:0/1381174424 >> v2:172.21.15.167:3300/0 conn(0x7eff040d3ed0 0x7eff040cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:50.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.343+0000 7fac6a800640 1 -- 172.21.15.167:0/3009019226 >> v2:172.21.15.204:3300/0 conn(0x7fac6c15f410 msgr2=0x7fac6c0d2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:50.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.343+0000 7fac6a800640 1 --2- 172.21.15.167:0/3009019226 >> v2:172.21.15.204:3300/0 conn(0x7fac6c15f410 0x7fac6c0d2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:50.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.343+0000 7fac6a800640 1 -- 172.21.15.167:0/3009019226 >> v2:172.21.15.167:3300/0 conn(0x7fac6c0d4720 msgr2=0x7fac6c0b6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:50.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.343+0000 7fac6a800640 1 --2- 172.21.15.167:0/3009019226 >> v2:172.21.15.167:3300/0 conn(0x7fac6c0d4720 0x7fac6c0b6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:50.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.343+0000 7fac6a800640 1 -- 172.21.15.167:0/3009019226 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac6c0d3900 con 0x7fac6c0b71c0 2024-05-11T12:14:50.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.343+0000 7fb0bfe00640 1 -- 172.21.15.167:0/3870248872 >> v2:172.21.15.204:3300/0 conn(0x7fb0c0163730 msgr2=0x7fb0c006e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:50.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.343+0000 7fb0bfe00640 1 --2- 172.21.15.167:0/3870248872 >> v2:172.21.15.204:3300/0 conn(0x7fb0c0163730 0x7fb0c006e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:50.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.343+0000 7fb0bfe00640 1 -- 172.21.15.167:0/3870248872 >> v2:172.21.15.167:3300/0 conn(0x7fb0c0168180 msgr2=0x7fb0c00b6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:50.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.343+0000 7fb0bfe00640 1 --2- 172.21.15.167:0/3870248872 >> v2:172.21.15.167:3300/0 conn(0x7fb0c0168180 0x7fb0c00b6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:50.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.343+0000 7fb0bfe00640 1 -- 172.21.15.167:0/3870248872 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0c006dc00 con 0x7fb0c00d3550 2024-05-11T12:14:50.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.343+0000 7eff03e00640 1 -- 172.21.15.167:0/1381174424 >> v1:172.21.15.204:6789/0 conn(0x7eff040d39d0 legacy=0x7eff040b1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:50.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.343+0000 7eff03e00640 1 -- 172.21.15.167:0/1381174424 >> v1:172.21.15.167:6789/0 conn(0x7eff04162920 legacy=0x7eff040d32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:50.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.344+0000 7eff03e00640 1 -- 172.21.15.167:0/1381174424 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff040d0790 con 0x7eff040d3ed0 2024-05-11T12:14:50.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.344+0000 7f65aa000640 1 -- 172.21.15.167:0/3914910747 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f65a0044070 con 0x7f65b40d3bf0 2024-05-11T12:14:50.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.345+0000 7fac6a800640 1 --2- 172.21.15.167:0/3009019226 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fac6c0b71c0 0x7fac6c06d480 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fac6c0d28b0 tx=0x7fac6402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc211e27f4061b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:50.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.345+0000 7fb0bfe00640 1 --2- 172.21.15.167:0/3870248872 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0c00d3550 0x7fb0c00d39b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb0c00b6300 tx=0x7fb0b802a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d29ab7cc0cce8b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:50.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.345+0000 7fac68a00640 1 -- 172.21.15.167:0/3009019226 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac64037070 con 0x7fac6c0b71c0 2024-05-11T12:14:50.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.345+0000 7fac68a00640 1 -- 172.21.15.167:0/3009019226 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac64032440 con 0x7fac6c0b71c0 2024-05-11T12:14:50.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.345+0000 7fac68a00640 1 -- 172.21.15.167:0/3009019226 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac64032810 con 0x7fac6c0b71c0 2024-05-11T12:14:50.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.345+0000 7eff03e00640 1 --2- 172.21.15.167:0/1381174424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff040d3ed0 0x7eff040cfb80 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7efef00097a0 tx=0x7efef002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6a3e4bc896f4c1c server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:50.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.346+0000 7f65aa000640 1 --2- 172.21.15.167:0/3914910747 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f65940762d0 0x7f6594078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:50.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.346+0000 7f65aa000640 1 -- 172.21.15.167:0/3914910747 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6101+0+0 (secure 0 0 0) 0x7f65a00b84c0 con 0x7f65b40d3bf0 2024-05-11T12:14:50.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.346+0000 7fb0be000640 1 -- 172.21.15.167:0/3870248872 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b8031600 con 0x7fb0c00d3550 2024-05-11T12:14:50.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.346+0000 7fb0be000640 1 -- 172.21.15.167:0/3870248872 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0b80317a0 con 0x7fb0c00d3550 2024-05-11T12:14:50.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.346+0000 7fb0be000640 1 -- 172.21.15.167:0/3870248872 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b8031b70 con 0x7fb0c00d3550 2024-05-11T12:14:50.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.346+0000 7f65b1e00640 1 --2- 172.21.15.167:0/3914910747 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f65940762d0 0x7f6594078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:50.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.346+0000 7fac70a00640 1 -- 172.21.15.167:0/3009019226 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fac6c0b71c0 msgr2=0x7fac6c06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:50.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.346+0000 7fac70a00640 1 --2- 172.21.15.167:0/3009019226 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fac6c0b71c0 0x7fac6c06d480 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fac6c0d28b0 tx=0x7fac6402a270 comp rx=0 tx=0).stop 2024-05-11T12:14:50.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.346+0000 7fac70a00640 1 -- 172.21.15.167:0/3009019226 shutdown_connections 2024-05-11T12:14:50.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.346+0000 7fac70a00640 1 --2- 172.21.15.167:0/3009019226 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fac6c0b71c0 0x7fac6c06d480 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:50.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.346+0000 7fac70a00640 1 --2- 172.21.15.167:0/3009019226 >> v2:172.21.15.167:3300/0 conn(0x7fac6c0d4720 0x7fac6c0b6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:50.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.346+0000 7fac70a00640 1 --2- 172.21.15.167:0/3009019226 >> v2:172.21.15.204:3300/0 conn(0x7fac6c15f410 0x7fac6c0d2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:50.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.346+0000 7fac70a00640 1 -- 172.21.15.167:0/3009019226 >> 172.21.15.167:0/3009019226 conn(0x7fac6c0687f0 msgr2=0x7fac6c0d1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:50.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.347+0000 7fac70a00640 1 -- 172.21.15.167:0/3009019226 shutdown_connections 2024-05-11T12:14:50.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.347+0000 7fac70a00640 1 -- 172.21.15.167:0/3009019226 wait complete. 2024-05-11T12:14:50.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.347+0000 7fb0c6800640 1 -- 172.21.15.167:0/3870248872 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0c00d3550 msgr2=0x7fb0c00d39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:50.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.347+0000 7fb0c6800640 1 --2- 172.21.15.167:0/3870248872 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0c00d3550 0x7fb0c00d39b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb0c00b6300 tx=0x7fb0b802a5d0 comp rx=0 tx=0).stop 2024-05-11T12:14:50.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.347+0000 7fb0c6800640 1 -- 172.21.15.167:0/3870248872 shutdown_connections 2024-05-11T12:14:50.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.347+0000 7fb0c6800640 1 --2- 172.21.15.167:0/3870248872 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0c00d3550 0x7fb0c00d39b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:50.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.347+0000 7fb0c6800640 1 --2- 172.21.15.167:0/3870248872 >> v2:172.21.15.167:3300/0 conn(0x7fb0c0168180 0x7fb0c00b6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:50.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.347+0000 7fb0c6800640 1 --2- 172.21.15.167:0/3870248872 >> v2:172.21.15.204:3300/0 conn(0x7fb0c0163730 0x7fb0c006e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:50.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.347+0000 7fb0c6800640 1 -- 172.21.15.167:0/3870248872 >> 172.21.15.167:0/3870248872 conn(0x7fb0c00685e0 msgr2=0x7fb0c00689f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:50.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.347+0000 7f65b1e00640 1 --2- 172.21.15.167:0/3914910747 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f65940762d0 0x7f6594078790 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f65b41f86f0 tx=0x7f65a4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:50.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.347+0000 7fb0c6800640 1 -- 172.21.15.167:0/3870248872 shutdown_connections 2024-05-11T12:14:50.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.347+0000 7fb0c6800640 1 -- 172.21.15.167:0/3870248872 wait complete. 2024-05-11T12:14:50.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.349+0000 7fac70a00640 1 Processor -- start 2024-05-11T12:14:50.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.349+0000 7fac70a00640 1 -- start start 2024-05-11T12:14:50.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.349+0000 7fb0c6800640 1 Processor -- start 2024-05-11T12:14:50.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.349+0000 7fb0c6800640 1 -- start start 2024-05-11T12:14:50.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.350+0000 7fac70a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fac6c0b71c0 0x7fac6c14b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:50.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.350+0000 7fac70a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fac6c0d4720 0x7fac6c155410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:50.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.350+0000 7fac70a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac6c15f410 0x7fac6c155950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:50.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.350+0000 7fac70a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fac6c0d41a0 con 0x7fac6c15f410 2024-05-11T12:14:50.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.350+0000 7fac70a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fac6c0d3ec0 con 0x7fac6c0b71c0 2024-05-11T12:14:50.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.350+0000 7fac70a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fac6c0d3310 con 0x7fac6c0d4720 2024-05-11T12:14:50.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.350+0000 7fb0c6800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0c00d3550 0x7fb0c00d08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:50.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.350+0000 7fb0c6800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0c0163730 0x7fb0c00d0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:50.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.350+0000 7fb0c6800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb0c0168180 0x7fb0c00d1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:50.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.350+0000 7fb0c6800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c00b6870 con 0x7fb0c0168180 2024-05-11T12:14:50.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.350+0000 7fb0c6800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c006e1c0 con 0x7fb0c0163730 2024-05-11T12:14:50.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.350+0000 7fb0c6800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb0c00b69f0 con 0x7fb0c00d3550 2024-05-11T12:14:50.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.351+0000 7fac6a800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac6c15f410 0x7fac6c155950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:50.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.351+0000 7fac6a800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac6c15f410 0x7fac6c155950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:52338/0 (socket says 172.21.15.167:52338) 2024-05-11T12:14:50.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.351+0000 7fac6a800640 1 -- 172.21.15.167:0/325750323 learned_addr learned my addr 172.21.15.167:0/325750323 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:50.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.351+0000 7fb0bfe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb0c0168180 0x7fb0c00d1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:50.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.351+0000 7fb0bfe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb0c0168180 0x7fb0c00d1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:52348/0 (socket says 172.21.15.167:52348) 2024-05-11T12:14:50.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.351+0000 7fb0bfe00640 1 -- 172.21.15.167:0/3368208020 learned_addr learned my addr 172.21.15.167:0/3368208020 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:50.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.351+0000 7fac69400640 1 --2- 172.21.15.167:0/325750323 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fac6c0d4720 0x7fac6c155410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:50.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.351+0000 7fac69e00640 1 --2- 172.21.15.167:0/325750323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fac6c0b71c0 0x7fac6c14b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:50.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.351+0000 7fb0bf400640 1 --2- 172.21.15.167:0/3368208020 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0c00d3550 0x7fb0c00d08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:50.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.351+0000 7fb0bea00640 1 --2- 172.21.15.167:0/3368208020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0c0163730 0x7fb0c00d0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:50.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.351+0000 7fac6a800640 1 -- 172.21.15.167:0/325750323 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fac6c0d4720 msgr2=0x7fac6c155410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:50.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.351+0000 7fac6a800640 1 --2- 172.21.15.167:0/325750323 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fac6c0d4720 0x7fac6c155410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:50.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.351+0000 7fac6a800640 1 -- 172.21.15.167:0/325750323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fac6c0b71c0 msgr2=0x7fac6c14b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:50.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.351+0000 7fac6a800640 1 --2- 172.21.15.167:0/325750323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fac6c0b71c0 0x7fac6c14b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:50.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.351+0000 7fac6a800640 1 -- 172.21.15.167:0/325750323 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac6c160a50 con 0x7fac6c15f410 2024-05-11T12:14:50.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.352+0000 7fb0bfe00640 1 -- 172.21.15.167:0/3368208020 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0c00d3550 msgr2=0x7fb0c00d08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:50.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.352+0000 7fb0bfe00640 1 --2- 172.21.15.167:0/3368208020 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0c00d3550 0x7fb0c00d08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:50.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.352+0000 7fb0bfe00640 1 -- 172.21.15.167:0/3368208020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0c0163730 msgr2=0x7fb0c00d0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:50.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.352+0000 7fb0bfe00640 1 --2- 172.21.15.167:0/3368208020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0c0163730 0x7fb0c00d0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:50.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.352+0000 7fb0bfe00640 1 -- 172.21.15.167:0/3368208020 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0c0167090 con 0x7fb0c0168180 2024-05-11T12:14:50.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.352+0000 7fac6a800640 1 --2- 172.21.15.167:0/325750323 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac6c15f410 0x7fac6c155950 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fac640090b0 tx=0x7fac640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:50.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.352+0000 7fb0bfe00640 1 --2- 172.21.15.167:0/3368208020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb0c0168180 0x7fb0c00d1320 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fb0b802abc0 tx=0x7fb0b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:50.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.352+0000 7eff03400640 1 -- 172.21.15.167:0/1381174424 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef0037070 con 0x7eff040d3ed0 2024-05-11T12:14:50.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.352+0000 7eff03400640 1 -- 172.21.15.167:0/1381174424 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efef0032440 con 0x7eff040d3ed0 2024-05-11T12:14:50.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.352+0000 7eff03400640 1 -- 172.21.15.167:0/1381174424 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef0032810 con 0x7eff040d3ed0 2024-05-11T12:14:50.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.352+0000 7fac62000640 1 -- 172.21.15.167:0/325750323 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac64041020 con 0x7fac6c15f410 2024-05-11T12:14:50.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.352+0000 7fac62000640 1 -- 172.21.15.167:0/325750323 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac6403d640 con 0x7fac6c15f410 2024-05-11T12:14:50.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.352+0000 7fac62000640 1 -- 172.21.15.167:0/325750323 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac64037040 con 0x7fac6c15f410 2024-05-11T12:14:50.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.352+0000 7f65aa000640 1 -- 172.21.15.167:0/3914910747 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f65a0081b70 con 0x7f65b40d3bf0 2024-05-11T12:14:50.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.352+0000 7fb0b7400640 1 -- 172.21.15.167:0/3368208020 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b803f070 con 0x7fb0c0168180 2024-05-11T12:14:50.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.352+0000 7fb0b7400640 1 -- 172.21.15.167:0/3368208020 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0b803b8e0 con 0x7fb0c0168180 2024-05-11T12:14:50.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.352+0000 7fb0b7400640 1 -- 172.21.15.167:0/3368208020 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b803bcb0 con 0x7fb0c0168180 2024-05-11T12:14:50.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.353+0000 7fac70a00640 1 -- 172.21.15.167:0/325750323 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac6c0ce5b0 con 0x7fac6c15f410 2024-05-11T12:14:50.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.353+0000 7fb0c6800640 1 -- 172.21.15.167:0/3368208020 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0c0166db0 con 0x7fb0c0168180 2024-05-11T12:14:50.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.353+0000 7fac70a00640 1 -- 172.21.15.167:0/325750323 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac6c0ceb70 con 0x7fac6c15f410 2024-05-11T12:14:50.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.353+0000 7fb0c6800640 1 -- 172.21.15.167:0/3368208020 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0c014c4a0 con 0x7fb0c0168180 2024-05-11T12:14:50.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.354+0000 7eff0a400640 1 -- 172.21.15.167:0/1381174424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff040d3ed0 msgr2=0x7eff040cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:50.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.354+0000 7eff0a400640 1 --2- 172.21.15.167:0/1381174424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff040d3ed0 0x7eff040cfb80 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7efef00097a0 tx=0x7efef002a230 comp rx=0 tx=0).stop 2024-05-11T12:14:50.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.354+0000 7fb0c6800640 1 -- 172.21.15.167:0/3368208020 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0c0069820 con 0x7fb0c0168180 2024-05-11T12:14:50.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.354+0000 7eff0a400640 1 -- 172.21.15.167:0/1381174424 shutdown_connections 2024-05-11T12:14:50.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.354+0000 7eff0a400640 1 --2- 172.21.15.167:0/1381174424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff040d3ed0 0x7eff040cfb80 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:50.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.354+0000 7eff0a400640 1 -- 172.21.15.167:0/1381174424 >> 172.21.15.167:0/1381174424 conn(0x7eff0409e350 msgr2=0x7eff040d5020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:50.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.355+0000 7eff0a400640 1 -- 172.21.15.167:0/1381174424 shutdown_connections 2024-05-11T12:14:50.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.355+0000 7fac70a00640 1 -- 172.21.15.167:0/325750323 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac6c0746e0 con 0x7fac6c15f410 2024-05-11T12:14:50.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.355+0000 7eff0a400640 1 -- 172.21.15.167:0/1381174424 wait complete. 2024-05-11T12:14:50.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.356+0000 7fac62000640 1 -- 172.21.15.167:0/325750323 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fac64044020 con 0x7fac6c15f410 2024-05-11T12:14:50.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.357+0000 7fb0b7400640 1 -- 172.21.15.167:0/3368208020 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb0b803a3a0 con 0x7fb0c0168180 2024-05-11T12:14:50.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.357+0000 7eff0a400640 1 Processor -- start 2024-05-11T12:14:50.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.357+0000 7eff0a400640 1 -- start start 2024-05-11T12:14:50.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.358+0000 7fb0b7400640 1 --2- 172.21.15.167:0/3368208020 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb0a00762d0 0x7fb0a0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:50.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.358+0000 7fac62000640 1 --2- 172.21.15.167:0/325750323 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fac4c0762d0 0x7fac4c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:50.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.358+0000 7fb0b7400640 1 -- 172.21.15.167:0/3368208020 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6101+0+0 (secure 0 0 0) 0x7fb0b80b8ba0 con 0x7fb0c0168180 2024-05-11T12:14:50.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.358+0000 7fac62000640 1 -- 172.21.15.167:0/325750323 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6101+0+0 (secure 0 0 0) 0x7fac640b7e70 con 0x7fac6c15f410 2024-05-11T12:14:50.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.358+0000 7eff0a400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff040d39d0 0x7eff0415c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:50.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.358+0000 7eff0a400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff040d3ed0 0x7eff040b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:50.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.358+0000 7eff0a400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff04162920 0x7eff04161340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:50.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.358+0000 7eff0a400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7eff040b3dd0 con 0x7eff04162920 2024-05-11T12:14:50.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.358+0000 7eff0a400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7eff04162d00 con 0x7eff040d3ed0 2024-05-11T12:14:50.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.358+0000 7eff0a400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7eff040b3f50 con 0x7eff040d39d0 2024-05-11T12:14:50.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.358+0000 7fb0bf400640 1 --2- 172.21.15.167:0/3368208020 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb0a00762d0 0x7fb0a0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:50.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.359+0000 7fac69e00640 1 --2- 172.21.15.167:0/325750323 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fac4c0762d0 0x7fac4c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:50.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.359+0000 7eff09a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff04162920 0x7eff04161340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:50.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.359+0000 7eff09a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff04162920 0x7eff04161340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:52354/0 (socket says 172.21.15.167:52354) 2024-05-11T12:14:50.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.359+0000 7eff09a00640 1 -- 172.21.15.167:0/1881266171 learned_addr learned my addr 172.21.15.167:0/1881266171 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:50.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.359+0000 7eff09000640 1 --2- 172.21.15.167:0/1881266171 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff040d39d0 0x7eff0415c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:50.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.360+0000 7fb0bf400640 1 --2- 172.21.15.167:0/3368208020 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb0a00762d0 0x7fb0a0078790 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0009b30 tx=0x7fb0b0009240 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:50.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.360+0000 7fac69e00640 1 --2- 172.21.15.167:0/325750323 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fac4c0762d0 0x7fac4c078790 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fac5c009940 tx=0x7fac5c003cd0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:50.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.359+0000 7eff03e00640 1 --2- 172.21.15.167:0/1881266171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff040d3ed0 0x7eff040b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:50.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.360+0000 7eff09a00640 1 -- 172.21.15.167:0/1881266171 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff040d39d0 msgr2=0x7eff0415c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:50.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.360+0000 7eff09a00640 1 --2- 172.21.15.167:0/1881266171 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff040d39d0 0x7eff0415c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:50.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.360+0000 7eff09a00640 1 -- 172.21.15.167:0/1881266171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff040d3ed0 msgr2=0x7eff040b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:50.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.360+0000 7eff09a00640 1 --2- 172.21.15.167:0/1881266171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff040d3ed0 0x7eff040b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:50.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.360+0000 7eff09a00640 1 -- 172.21.15.167:0/1881266171 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff040c85d0 con 0x7eff04162920 2024-05-11T12:14:50.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.360+0000 7eff09a00640 1 --2- 172.21.15.167:0/1881266171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff04162920 0x7eff04161340 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7efefc00c710 tx=0x7efefc00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:50.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.362+0000 7fb0b7400640 1 -- 172.21.15.167:0/3368208020 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb0b80b8e20 con 0x7fb0c0168180 2024-05-11T12:14:50.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.367+0000 7eff00c00640 1 -- 172.21.15.167:0/1881266171 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efefc015d70 con 0x7eff04162920 2024-05-11T12:14:50.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.367+0000 7eff00c00640 1 -- 172.21.15.167:0/1881266171 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efefc00ce20 con 0x7eff04162920 2024-05-11T12:14:50.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.367+0000 7eff00c00640 1 -- 172.21.15.167:0/1881266171 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efefc003b70 con 0x7eff04162920 2024-05-11T12:14:50.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.367+0000 7eff0a400640 1 -- 172.21.15.167:0/1881266171 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff040c88b0 con 0x7eff04162920 2024-05-11T12:14:50.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.367+0000 7fac62000640 1 -- 172.21.15.167:0/325750323 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fac6c0746e0 con 0x7fac6c15f410 2024-05-11T12:14:50.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.367+0000 7eff0a400640 1 -- 172.21.15.167:0/1881266171 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff040c8e70 con 0x7eff04162920 2024-05-11T12:14:50.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.368+0000 7eff0a400640 1 -- 172.21.15.167:0/1881266171 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff04069650 con 0x7eff04162920 2024-05-11T12:14:50.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.370+0000 7eff00c00640 1 -- 172.21.15.167:0/1881266171 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7efefc003d10 con 0x7eff04162920 2024-05-11T12:14:50.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.371+0000 7eff00c00640 1 --2- 172.21.15.167:0/1881266171 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7efee40762d0 0x7efee4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:50.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.371+0000 7eff00c00640 1 -- 172.21.15.167:0/1881266171 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6101+0+0 (secure 0 0 0) 0x7efefc09fa30 con 0x7eff04162920 2024-05-11T12:14:50.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.371+0000 7eff09000640 1 --2- 172.21.15.167:0/1881266171 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7efee40762d0 0x7efee4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:50.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.372+0000 7eff09000640 1 --2- 172.21.15.167:0/1881266171 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7efee40762d0 0x7efee4078790 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7efef8000f60 tx=0x7efef8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:50.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.376+0000 7eff00c00640 1 -- 172.21.15.167:0/1881266171 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7efefc0691f0 con 0x7eff04162920 2024-05-11T12:14:50.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.657+0000 7fb0c6800640 1 -- 172.21.15.167:0/3368208020 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "a"} v 0) -- 0x7fb0c0072e00 con 0x7fb0c0168180 2024-05-11T12:14:50.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.658+0000 7f65b3200640 1 -- 172.21.15.167:0/3914910747 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "b"} v 0) -- 0x7f65b40d0d90 con 0x7f65b40d3bf0 2024-05-11T12:14:50.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.667+0000 7fac70a00640 1 -- 172.21.15.167:0/325750323 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "c"} v 0) -- 0x7fac6c069820 con 0x7fac6c15f410 2024-05-11T12:14:50.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.679+0000 7eff0a400640 1 -- 172.21.15.167:0/1881266171 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "d"} v 0) -- 0x7eff04075340 con 0x7eff04162920 2024-05-11T12:14:50.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.714+0000 7fb0b7400640 1 -- 172.21.15.167:0/3368208020 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "a"}]=0 failed mds gid 6636 v63) ==== 95+0+0 (secure 0 0 0) 0x7fb0b80823e0 con 0x7fb0c0168180 2024-05-11T12:14:50.714 INFO:teuthology.orchestra.run.smithi167.stderr:failed mds gid 6636 2024-05-11T12:14:50.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.717+0000 7fb0c6800640 1 -- 172.21.15.167:0/3368208020 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb0a00762d0 msgr2=0x7fb0a0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:50.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.717+0000 7fb0c6800640 1 --2- 172.21.15.167:0/3368208020 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb0a00762d0 0x7fb0a0078790 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0009b30 tx=0x7fb0b0009240 comp rx=0 tx=0).stop 2024-05-11T12:14:50.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.717+0000 7fb0c6800640 1 -- 172.21.15.167:0/3368208020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb0c0168180 msgr2=0x7fb0c00d1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:50.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.717+0000 7fb0c6800640 1 --2- 172.21.15.167:0/3368208020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb0c0168180 0x7fb0c00d1320 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fb0b802abc0 tx=0x7fb0b8004a40 comp rx=0 tx=0).stop 2024-05-11T12:14:50.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.717+0000 7fb0c6800640 1 -- 172.21.15.167:0/3368208020 shutdown_connections 2024-05-11T12:14:50.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.717+0000 7fb0c6800640 1 --2- 172.21.15.167:0/3368208020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb0c0168180 0x7fb0c00d1320 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:50.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.717+0000 7fb0c6800640 1 --2- 172.21.15.167:0/3368208020 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb0a00762d0 0x7fb0a0078790 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:50.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.717+0000 7fb0c6800640 1 --2- 172.21.15.167:0/3368208020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0c0163730 0x7fb0c00d0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:50.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.717+0000 7fb0c6800640 1 --2- 172.21.15.167:0/3368208020 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0c00d3550 0x7fb0c00d08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:50.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.717+0000 7fb0c6800640 1 -- 172.21.15.167:0/3368208020 >> 172.21.15.167:0/3368208020 conn(0x7fb0c00685e0 msgr2=0x7fb0c01627e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:50.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.717+0000 7fb0c6800640 1 -- 172.21.15.167:0/3368208020 shutdown_connections 2024-05-11T12:14:50.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.717+0000 7fb0c6800640 1 -- 172.21.15.167:0/3368208020 wait complete. 2024-05-11T12:14:50.725 INFO:tasks.ceph.mon.a.smithi167.stderr:2024-05-11T12:14:50.725+0000 7f3aa8e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-11T12:14:50.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.730+0000 7f65aa000640 1 -- 172.21.15.167:0/3914910747 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "b"}]=0 failed mds gid 5033 v64) ==== 95+0+0 (secure 0 0 0) 0x7f65a0081470 con 0x7f65b40d3bf0 2024-05-11T12:14:50.730 INFO:teuthology.orchestra.run.smithi167.stderr:failed mds gid 5033 2024-05-11T12:14:50.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.730+0000 7fac62000640 1 -- 172.21.15.167:0/325750323 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "c"}]=0 failed mds gid 6639 v64) ==== 95+0+0 (secure 0 0 0) 0x7fac6404b030 con 0x7fac6c15f410 2024-05-11T12:14:50.731 INFO:teuthology.orchestra.run.smithi167.stderr:failed mds gid 6639 2024-05-11T12:14:50.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.733+0000 7fac70a00640 1 -- 172.21.15.167:0/325750323 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fac4c0762d0 msgr2=0x7fac4c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:50.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.733+0000 7fac70a00640 1 --2- 172.21.15.167:0/325750323 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fac4c0762d0 0x7fac4c078790 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fac5c009940 tx=0x7fac5c003cd0 comp rx=0 tx=0).stop 2024-05-11T12:14:50.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.733+0000 7fac70a00640 1 -- 172.21.15.167:0/325750323 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac6c15f410 msgr2=0x7fac6c155950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:50.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.733+0000 7fac70a00640 1 --2- 172.21.15.167:0/325750323 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac6c15f410 0x7fac6c155950 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fac640090b0 tx=0x7fac640046c0 comp rx=0 tx=0).stop 2024-05-11T12:14:50.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.733+0000 7f65b3200640 1 -- 172.21.15.167:0/3914910747 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f65940762d0 msgr2=0x7f6594078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:50.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.733+0000 7f65b3200640 1 --2- 172.21.15.167:0/3914910747 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f65940762d0 0x7f6594078790 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f65b41f86f0 tx=0x7f65a4009040 comp rx=0 tx=0).stop 2024-05-11T12:14:50.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.733+0000 7f65b3200640 1 -- 172.21.15.167:0/3914910747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65b40d3bf0 msgr2=0x7f65b41605b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:50.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.734+0000 7fac70a00640 1 -- 172.21.15.167:0/325750323 shutdown_connections 2024-05-11T12:14:50.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.734+0000 7fac70a00640 1 --2- 172.21.15.167:0/325750323 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac6c15f410 0x7fac6c155950 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:50.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.734+0000 7fac70a00640 1 --2- 172.21.15.167:0/325750323 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fac6c0d4720 0x7fac6c155410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:50.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.734+0000 7fac70a00640 1 --2- 172.21.15.167:0/325750323 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fac4c0762d0 0x7fac4c078790 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:50.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.734+0000 7fac70a00640 1 --2- 172.21.15.167:0/325750323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fac6c0b71c0 0x7fac6c14b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:50.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.734+0000 7fac70a00640 1 -- 172.21.15.167:0/325750323 >> 172.21.15.167:0/325750323 conn(0x7fac6c0687f0 msgr2=0x7fac6c0b87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:50.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.734+0000 7f65b3200640 1 --2- 172.21.15.167:0/3914910747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65b40d3bf0 0x7f65b41605b0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f65a0009870 tx=0x7f65a00046c0 comp rx=0 tx=0).stop 2024-05-11T12:14:50.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.734+0000 7f65b3200640 1 -- 172.21.15.167:0/3914910747 shutdown_connections 2024-05-11T12:14:50.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.734+0000 7f65b3200640 1 --2- 172.21.15.167:0/3914910747 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f65940762d0 0x7f6594078790 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:50.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.734+0000 7f65b3200640 1 --2- 172.21.15.167:0/3914910747 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65b415f4d0 0x7f65b40a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:50.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.734+0000 7f65b3200640 1 --2- 172.21.15.167:0/3914910747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65b40d3bf0 0x7f65b41605b0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:50.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.734+0000 7f65b3200640 1 --2- 172.21.15.167:0/3914910747 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65b40d15a0 0x7f65b414f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:50.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.734+0000 7f65b3200640 1 -- 172.21.15.167:0/3914910747 >> 172.21.15.167:0/3914910747 conn(0x7f65b40d4f90 msgr2=0x7f65b40b85b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:50.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.734+0000 7fac70a00640 1 -- 172.21.15.167:0/325750323 shutdown_connections 2024-05-11T12:14:50.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.734+0000 7fac70a00640 1 -- 172.21.15.167:0/325750323 wait complete. 2024-05-11T12:14:50.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.734+0000 7f65b3200640 1 -- 172.21.15.167:0/3914910747 shutdown_connections 2024-05-11T12:14:50.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:50.734+0000 7f65b3200640 1 -- 172.21.15.167:0/3914910747 wait complete. 2024-05-11T12:14:50.740 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-11T12:14:50.741 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-11T12:14:50.746 INFO:tasks.ceph.mds.a:Started 2024-05-11T12:14:50.746 DEBUG:teuthology.parallel:result is None 2024-05-11T12:14:50.756 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-11T12:14:50.756 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-11T12:14:50.757 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-11T12:14:50.758 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-11T12:14:50.765 INFO:tasks.ceph.mds.b:Started 2024-05-11T12:14:50.765 DEBUG:teuthology.parallel:result is None 2024-05-11T12:14:50.801 INFO:tasks.ceph.mds.c:Started 2024-05-11T12:14:50.801 DEBUG:teuthology.parallel:result is None 2024-05-11T12:14:50.876 INFO:tasks.ceph.mds.a.smithi167.stdout:starting mds.a at 2024-05-11T12:14:50.927 INFO:tasks.ceph.mds.b.smithi204.stdout:starting mds.b at 2024-05-11T12:14:50.940 INFO:tasks.ceph.mds.c.smithi167.stdout:starting mds.c at 2024-05-11T12:14:51.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.734+0000 7eff00c00640 1 -- 172.21.15.167:0/1881266171 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "d"}]=0 failed mds gid 5036 v65) ==== 95+0+0 (secure 0 0 0) 0x7efefc0711e0 con 0x7eff04162920 2024-05-11T12:14:51.735 INFO:teuthology.orchestra.run.smithi167.stderr:failed mds gid 5036 2024-05-11T12:14:51.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.739+0000 7eff0a400640 1 -- 172.21.15.167:0/1881266171 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7efee40762d0 msgr2=0x7efee4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:51.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.739+0000 7eff0a400640 1 --2- 172.21.15.167:0/1881266171 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7efee40762d0 0x7efee4078790 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7efef8000f60 tx=0x7efef8009040 comp rx=0 tx=0).stop 2024-05-11T12:14:51.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.739+0000 7eff0a400640 1 -- 172.21.15.167:0/1881266171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff04162920 msgr2=0x7eff04161340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:51.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.739+0000 7eff0a400640 1 --2- 172.21.15.167:0/1881266171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff04162920 0x7eff04161340 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7efefc00c710 tx=0x7efefc00cbd0 comp rx=0 tx=0).stop 2024-05-11T12:14:51.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.739+0000 7eff0a400640 1 -- 172.21.15.167:0/1881266171 shutdown_connections 2024-05-11T12:14:51.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.739+0000 7eff0a400640 1 --2- 172.21.15.167:0/1881266171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff04162920 0x7eff04161340 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:51.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.739+0000 7eff0a400640 1 --2- 172.21.15.167:0/1881266171 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7efee40762d0 0x7efee4078790 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:51.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.739+0000 7eff0a400640 1 --2- 172.21.15.167:0/1881266171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff040d3ed0 0x7eff040b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:51.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.739+0000 7eff0a400640 1 --2- 172.21.15.167:0/1881266171 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff040d39d0 0x7eff0415c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:51.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.739+0000 7eff0a400640 1 -- 172.21.15.167:0/1881266171 >> 172.21.15.167:0/1881266171 conn(0x7eff0409e350 msgr2=0x7eff040b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:51.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.739+0000 7eff0a400640 1 -- 172.21.15.167:0/1881266171 shutdown_connections 2024-05-11T12:14:51.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.739+0000 7eff0a400640 1 -- 172.21.15.167:0/1881266171 wait complete. 2024-05-11T12:14:51.764 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-11T12:14:51.764 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-11T12:14:51.772 INFO:tasks.ceph.mds.d:Started 2024-05-11T12:14:51.772 DEBUG:teuthology.parallel:result is None 2024-05-11T12:14:51.773 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:14:51.938 INFO:tasks.ceph.mds.d.smithi204.stdout:starting mds.d at 2024-05-11T12:14:51.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.990+0000 7f1072200640 1 Processor -- start 2024-05-11T12:14:51.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.990+0000 7f1072200640 1 -- start start 2024-05-11T12:14:51.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.992+0000 7f1072200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f106c145ca0 0x7f106c0b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:51.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.992+0000 7f1072200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f106c146400 0x7f106c0b5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:51.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.992+0000 7f1072200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f106c146950 0x7f106c06f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:51.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.992+0000 7f1072200640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f106c0b3e60 con 0x7f106c146400 2024-05-11T12:14:51.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.992+0000 7f1072200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f106c0b4140 con 0x7f106c146950 2024-05-11T12:14:51.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.992+0000 7f1072200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f106c161900 con 0x7f106c145ca0 2024-05-11T12:14:51.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.992+0000 7f106be00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f106c146400 0x7f106c0b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:51.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.992+0000 7f106be00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f106c146400 0x7f106c0b5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:52400/0 (socket says 172.21.15.167:52400) 2024-05-11T12:14:51.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.992+0000 7f106be00640 1 -- 172.21.15.167:0/2544011343 learned_addr learned my addr 172.21.15.167:0/2544011343 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:51.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.993+0000 7f1071800640 1 --2- 172.21.15.167:0/2544011343 >> v2:172.21.15.204:3300/0 conn(0x7f106c146950 0x7f106c06f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:51.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.993+0000 7f106be00640 1 -- 172.21.15.167:0/2544011343 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f106c145ca0 msgr2=0x7f106c0b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:51.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.993+0000 7f1070e00640 1 --2- 172.21.15.167:0/2544011343 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f106c145ca0 0x7f106c0b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:51.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.993+0000 7f106be00640 1 --2- 172.21.15.167:0/2544011343 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f106c145ca0 0x7f106c0b4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:51.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.993+0000 7f106be00640 1 -- 172.21.15.167:0/2544011343 >> v2:172.21.15.204:3300/0 conn(0x7f106c146950 msgr2=0x7f106c06f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:51.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.993+0000 7f106be00640 1 --2- 172.21.15.167:0/2544011343 >> v2:172.21.15.204:3300/0 conn(0x7f106c146950 0x7f106c06f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:51.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.993+0000 7f106be00640 1 -- 172.21.15.167:0/2544011343 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f106c161be0 con 0x7f106c146400 2024-05-11T12:14:51.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.993+0000 7f1071800640 1 --2- 172.21.15.167:0/2544011343 >> v2:172.21.15.204:3300/0 conn(0x7f106c146950 0x7f106c06f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:51.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.993+0000 7f1070e00640 1 --2- 172.21.15.167:0/2544011343 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f106c145ca0 0x7f106c0b4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:51.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.994+0000 7f106be00640 1 --2- 172.21.15.167:0/2544011343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f106c146400 0x7f106c0b5020 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f106c1468e0 tx=0x7f105802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab3805d1292e5e3b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:51.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.994+0000 7f106b400640 1 -- 172.21.15.167:0/2544011343 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1058037070 con 0x7f106c146400 2024-05-11T12:14:51.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.994+0000 7f106b400640 1 -- 172.21.15.167:0/2544011343 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1058032440 con 0x7f106c146400 2024-05-11T12:14:51.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.994+0000 7f106b400640 1 -- 172.21.15.167:0/2544011343 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1058032810 con 0x7f106c146400 2024-05-11T12:14:51.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.995+0000 7f1072200640 1 -- 172.21.15.167:0/2544011343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f106c146400 msgr2=0x7f106c0b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:51.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.995+0000 7f1072200640 1 --2- 172.21.15.167:0/2544011343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f106c146400 0x7f106c0b5020 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f106c1468e0 tx=0x7f105802a1e0 comp rx=0 tx=0).stop 2024-05-11T12:14:51.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.995+0000 7f1072200640 1 -- 172.21.15.167:0/2544011343 shutdown_connections 2024-05-11T12:14:51.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.995+0000 7f1072200640 1 --2- 172.21.15.167:0/2544011343 >> v2:172.21.15.204:3300/0 conn(0x7f106c146950 0x7f106c06f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:51.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.995+0000 7f1072200640 1 --2- 172.21.15.167:0/2544011343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f106c146400 0x7f106c0b5020 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:51.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.995+0000 7f1072200640 1 --2- 172.21.15.167:0/2544011343 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f106c145ca0 0x7f106c0b4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:51.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.995+0000 7f1072200640 1 -- 172.21.15.167:0/2544011343 >> 172.21.15.167:0/2544011343 conn(0x7f106c0b2b60 msgr2=0x7f106c0b2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:51.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.995+0000 7f1072200640 1 -- 172.21.15.167:0/2544011343 shutdown_connections 2024-05-11T12:14:51.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.995+0000 7f1072200640 1 -- 172.21.15.167:0/2544011343 wait complete. 2024-05-11T12:14:51.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.997+0000 7f1072200640 1 Processor -- start 2024-05-11T12:14:51.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.997+0000 7f1072200640 1 -- start start 2024-05-11T12:14:51.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.998+0000 7f1072200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f106c145ca0 0x7f106c1562d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:51.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.998+0000 7f1072200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f106c146400 0x7f106c0b45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:51.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.998+0000 7f1072200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f106c146950 0x7f106c1410e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:51.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.998+0000 7f1072200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f106c06ef40 con 0x7f106c145ca0 2024-05-11T12:14:51.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.998+0000 7f1072200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f106c06edc0 con 0x7f106c146950 2024-05-11T12:14:52.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.998+0000 7f1072200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f106c06f730 con 0x7f106c146400 2024-05-11T12:14:52.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.998+0000 7f1070e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f106c145ca0 0x7f106c1562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:52.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.998+0000 7f1070e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f106c145ca0 0x7f106c1562d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:52402/0 (socket says 172.21.15.167:52402) 2024-05-11T12:14:52.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.998+0000 7f1070e00640 1 -- 172.21.15.167:0/4164574169 learned_addr learned my addr 172.21.15.167:0/4164574169 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:52.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.998+0000 7f106be00640 1 --2- 172.21.15.167:0/4164574169 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f106c146400 0x7f106c0b45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:52.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.998+0000 7f1070e00640 1 -- 172.21.15.167:0/4164574169 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f106c146400 msgr2=0x7f106c0b45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:52.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.998+0000 7f1070e00640 1 --2- 172.21.15.167:0/4164574169 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f106c146400 0x7f106c0b45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:52.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.998+0000 7f1070e00640 1 -- 172.21.15.167:0/4164574169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f106c146950 msgr2=0x7f106c1410e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:52.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.998+0000 7f1070e00640 1 --2- 172.21.15.167:0/4164574169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f106c146950 0x7f106c1410e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:52.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.998+0000 7f1070e00640 1 -- 172.21.15.167:0/4164574169 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f106c144940 con 0x7f106c145ca0 2024-05-11T12:14:52.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.999+0000 7f1070e00640 1 --2- 172.21.15.167:0/4164574169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f106c145ca0 0x7f106c1562d0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f106000c7c0 tx=0x7f106000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:52.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.999+0000 7f1068c00640 1 -- 172.21.15.167:0/4164574169 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1060015070 con 0x7f106c145ca0 2024-05-11T12:14:52.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.999+0000 7f1068c00640 1 -- 172.21.15.167:0/4164574169 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10600079d0 con 0x7f106c145ca0 2024-05-11T12:14:52.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.999+0000 7f1068c00640 1 -- 172.21.15.167:0/4164574169 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1060007da0 con 0x7f106c145ca0 2024-05-11T12:14:52.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.999+0000 7f1072200640 1 -- 172.21.15.167:0/4164574169 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f106c156c00 con 0x7f106c145ca0 2024-05-11T12:14:52.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:51.999+0000 7f1072200640 1 -- 172.21.15.167:0/4164574169 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f106c144ee0 con 0x7f106c145ca0 2024-05-11T12:14:52.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:52.000+0000 7f1072200640 1 -- 172.21.15.167:0/4164574169 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f106c069650 con 0x7f106c145ca0 2024-05-11T12:14:52.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:52.001+0000 7f1068c00640 1 -- 172.21.15.167:0/4164574169 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1060018070 con 0x7f106c145ca0 2024-05-11T12:14:52.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:52.003+0000 7f1068c00640 1 --2- 172.21.15.167:0/4164574169 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f104c0762d0 0x7f104c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:52.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:52.003+0000 7f1068c00640 1 -- 172.21.15.167:0/4164574169 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6187+0+0 (secure 0 0 0) 0x7f10600976a0 con 0x7f106c145ca0 2024-05-11T12:14:52.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:52.003+0000 7f106be00640 1 --2- 172.21.15.167:0/4164574169 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f104c0762d0 0x7f104c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:52.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:52.003+0000 7f106be00640 1 --2- 172.21.15.167:0/4164574169 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f104c0762d0 0x7f104c078790 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1058034000 tx=0x7f105802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:52.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:52.007+0000 7f1068c00640 1 -- 172.21.15.167:0/4164574169 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1060060e10 con 0x7f106c145ca0 2024-05-11T12:14:52.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:52.260+0000 7f1072200640 1 -- 172.21.15.167:0/4164574169 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f106c06eac0 con 0x7f106c145ca0 2024-05-11T12:14:52.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:52.262+0000 7f1068c00640 1 -- 172.21.15.167:0/4164574169 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+4362 (secure 0 0 0) 0x7f1060060710 con 0x7f106c145ca0 2024-05-11T12:14:52.262 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:52.262 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":66,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5401,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/2213070102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2213070102},{"type":"v1","addr":"172.21.15.204:6833","nonce":2213070102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6870,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3735016882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3735016882},{"type":"v1","addr":"172.21.15.167:6835","nonce":3735016882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:14:33.067226+0000","modified":"2024-05-11T12:14:51.739703+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6885},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6885":{"gid":6885,"name":"c","rank":0,"incarnation":66,"state":"up:replay","state_seq":1,"addr":"172.21.15.167:6837/2901182170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2901182170},{"type":"v1","addr":"172.21.15.167:6837","nonce":2901182170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-05-11T12:14:52.262 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 66 2024-05-11T12:14:52.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:52.264+0000 7f1072200640 1 -- 172.21.15.167:0/4164574169 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f104c0762d0 msgr2=0x7f104c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:52.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:52.264+0000 7f1072200640 1 --2- 172.21.15.167:0/4164574169 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f104c0762d0 0x7f104c078790 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1058034000 tx=0x7f105802e000 comp rx=0 tx=0).stop 2024-05-11T12:14:52.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:52.264+0000 7f1072200640 1 -- 172.21.15.167:0/4164574169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f106c145ca0 msgr2=0x7f106c1562d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:52.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:52.264+0000 7f1072200640 1 --2- 172.21.15.167:0/4164574169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f106c145ca0 0x7f106c1562d0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f106000c7c0 tx=0x7f106000cc80 comp rx=0 tx=0).stop 2024-05-11T12:14:52.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:52.264+0000 7f1072200640 1 -- 172.21.15.167:0/4164574169 shutdown_connections 2024-05-11T12:14:52.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:52.264+0000 7f1072200640 1 --2- 172.21.15.167:0/4164574169 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f104c0762d0 0x7f104c078790 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:52.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:52.264+0000 7f1072200640 1 --2- 172.21.15.167:0/4164574169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f106c146950 0x7f106c1410e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:52.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:52.264+0000 7f1072200640 1 --2- 172.21.15.167:0/4164574169 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f106c146400 0x7f106c0b45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:52.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:52.264+0000 7f1072200640 1 --2- 172.21.15.167:0/4164574169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f106c145ca0 0x7f106c1562d0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:52.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:52.264+0000 7f1072200640 1 -- 172.21.15.167:0/4164574169 >> 172.21.15.167:0/4164574169 conn(0x7f106c0b2b60 msgr2=0x7f106c15fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:52.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:52.265+0000 7f1072200640 1 -- 172.21.15.167:0/4164574169 shutdown_connections 2024-05-11T12:14:52.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:52.265+0000 7f1072200640 1 -- 172.21.15.167:0/4164574169 wait complete. 2024-05-11T12:14:52.685 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 66, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-11T12:14:33.067226+0000', 'modified': '2024-05-11T12:14:51.739703+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 100, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6885}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6885': {'gid': 6885, 'name': 'c', 'rank': 0, 'incarnation': 66, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.167:6837/2901182170', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6836', 'nonce': 2901182170}, {'type': 'v1', 'addr': '172.21.15.167:6837', 'nonce': 2901182170}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-11T12:14:52.686 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6885:up:replay 2024-05-11T12:14:53.687 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:14:53.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.898+0000 7f30c2c00640 1 Processor -- start 2024-05-11T12:14:53.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.898+0000 7f30c2c00640 1 -- start start 2024-05-11T12:14:53.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.899+0000 7f30c2c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f30bc15f150 0x7f30bc0d0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:53.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.900+0000 7f30c2c00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f30bc0d5c50 0x7f30bc0d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:53.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.900+0000 7f30c2c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30bc0cfb00 con 0x7f30bc15f530 2024-05-11T12:14:53.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.900+0000 7f30c2c00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f30bc0d3d50 con 0x7f30bc0d5c50 2024-05-11T12:14:53.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.900+0000 7f30c2c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f30bc0cfda0 con 0x7f30bc15f150 2024-05-11T12:14:53.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.900+0000 7f30c2200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f30bc0d5c50 0x7f30bc0d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:53.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.900+0000 7f30c2200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f30bc0d5c50 0x7f30bc0d1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:52418/0 (socket says 172.21.15.167:52418) 2024-05-11T12:14:53.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.900+0000 7f30c2200640 1 -- 172.21.15.167:0/3785377925 learned_addr learned my addr 172.21.15.167:0/3785377925 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:53.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.900+0000 7f30c2200640 1 -- 172.21.15.167:0/3785377925 >> v2:172.21.15.204:3300/0 conn(0x7f30bc15f150 msgr2=0x7f30bc0d0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:53.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.900+0000 7f30c2200640 1 --2- 172.21.15.167:0/3785377925 >> v2:172.21.15.204:3300/0 conn(0x7f30bc15f150 0x7f30bc0d0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:53.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.900+0000 7f30c2200640 1 -- 172.21.15.167:0/3785377925 >> v1:172.21.15.204:6789/0 conn(0x7f30bc15f530 legacy=0x7f30bc0d3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:53.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.901+0000 7f30c2200640 1 -- 172.21.15.167:0/3785377925 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30bc0d0080 con 0x7f30bc0d5c50 2024-05-11T12:14:53.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.902+0000 7f30c2200640 1 --2- 172.21.15.167:0/3785377925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30bc0d5c50 0x7f30bc0d1a50 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f30bc0d10a0 tx=0x7f30b402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9d0c6930bf96b48 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:53.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.902+0000 7f30bbe00640 1 -- 172.21.15.167:0/3785377925 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b4036070 con 0x7f30bc0d5c50 2024-05-11T12:14:53.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.902+0000 7f30bbe00640 1 -- 172.21.15.167:0/3785377925 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30b4031400 con 0x7f30bc0d5c50 2024-05-11T12:14:53.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.902+0000 7f30bbe00640 1 -- 172.21.15.167:0/3785377925 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b40317d0 con 0x7f30bc0d5c50 2024-05-11T12:14:53.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.902+0000 7f30c2c00640 1 -- 172.21.15.167:0/3785377925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30bc0d5c50 msgr2=0x7f30bc0d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:53.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.902+0000 7f30c2c00640 1 --2- 172.21.15.167:0/3785377925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30bc0d5c50 0x7f30bc0d1a50 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f30bc0d10a0 tx=0x7f30b402a5d0 comp rx=0 tx=0).stop 2024-05-11T12:14:53.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.903+0000 7f30c2c00640 1 -- 172.21.15.167:0/3785377925 shutdown_connections 2024-05-11T12:14:53.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.903+0000 7f30c2c00640 1 --2- 172.21.15.167:0/3785377925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30bc0d5c50 0x7f30bc0d1a50 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:53.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.903+0000 7f30c2c00640 1 --2- 172.21.15.167:0/3785377925 >> v2:172.21.15.204:3300/0 conn(0x7f30bc15f150 0x7f30bc0d0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:53.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.903+0000 7f30c2c00640 1 -- 172.21.15.167:0/3785377925 >> 172.21.15.167:0/3785377925 conn(0x7f30bc09e3a0 msgr2=0x7f30bc0d5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:53.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.903+0000 7f30c2c00640 1 -- 172.21.15.167:0/3785377925 shutdown_connections 2024-05-11T12:14:53.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.903+0000 7f30c2c00640 1 -- 172.21.15.167:0/3785377925 wait complete. 2024-05-11T12:14:53.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.904+0000 7f30c2c00640 1 Processor -- start 2024-05-11T12:14:53.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.904+0000 7f30c2c00640 1 -- start start 2024-05-11T12:14:53.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.905+0000 7f30c2c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30bc0d5c50 0x7f30bc0a2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:53.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.905+0000 7f30c2c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30bc15f150 0x7f30bc0ce860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:53.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.905+0000 7f30c2c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f30bc15f530 0x7f30bc0b8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:53.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.905+0000 7f30c2c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f30bc0d1f90 con 0x7f30bc0d5c50 2024-05-11T12:14:53.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.905+0000 7f30c2c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f30bc0d0640 con 0x7f30bc15f150 2024-05-11T12:14:53.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.905+0000 7f30c2c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f30bc0d2110 con 0x7f30bc15f530 2024-05-11T12:14:53.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.906+0000 7f30c1800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30bc0d5c50 0x7f30bc0a2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:53.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.906+0000 7f30c1800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30bc0d5c50 0x7f30bc0a2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:52420/0 (socket says 172.21.15.167:52420) 2024-05-11T12:14:53.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.906+0000 7f30c1800640 1 -- 172.21.15.167:0/1202389354 learned_addr learned my addr 172.21.15.167:0/1202389354 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:53.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.906+0000 7f30c1800640 1 -- 172.21.15.167:0/1202389354 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f30bc15f530 msgr2=0x7f30bc0b8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:53.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.906+0000 7f30c1800640 1 --2- 172.21.15.167:0/1202389354 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f30bc15f530 0x7f30bc0b8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:53.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.906+0000 7f30c1800640 1 -- 172.21.15.167:0/1202389354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30bc15f150 msgr2=0x7f30bc0ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:53.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.906+0000 7f30c1800640 1 --2- 172.21.15.167:0/1202389354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30bc15f150 0x7f30bc0ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:53.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.906+0000 7f30c1800640 1 -- 172.21.15.167:0/1202389354 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30bc09ee10 con 0x7f30bc0d5c50 2024-05-11T12:14:53.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.906+0000 7f30c1800640 1 --2- 172.21.15.167:0/1202389354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30bc0d5c50 0x7f30bc0a2930 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f30b000b4a0 tx=0x7f30b000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:53.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.906+0000 7f30b9600640 1 -- 172.21.15.167:0/1202389354 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b0014070 con 0x7f30bc0d5c50 2024-05-11T12:14:53.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.907+0000 7f30b9600640 1 -- 172.21.15.167:0/1202389354 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30b0010610 con 0x7f30bc0d5c50 2024-05-11T12:14:53.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.907+0000 7f30c2c00640 1 -- 172.21.15.167:0/1202389354 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30bc0ceda0 con 0x7f30bc0d5c50 2024-05-11T12:14:53.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.907+0000 7f30b9600640 1 -- 172.21.15.167:0/1202389354 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b0010a00 con 0x7f30bc0d5c50 2024-05-11T12:14:53.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.907+0000 7f30c2c00640 1 -- 172.21.15.167:0/1202389354 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30bc1f8780 con 0x7f30bc0d5c50 2024-05-11T12:14:53.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.908+0000 7f30c2c00640 1 -- 172.21.15.167:0/1202389354 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30bc069650 con 0x7f30bc0d5c50 2024-05-11T12:14:53.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.909+0000 7f30b9600640 1 -- 172.21.15.167:0/1202389354 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f30b0017070 con 0x7f30bc0d5c50 2024-05-11T12:14:53.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.910+0000 7f30b9600640 1 --2- 172.21.15.167:0/1202389354 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f309c0762d0 0x7f309c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:53.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.910+0000 7f30b9600640 1 -- 172.21.15.167:0/1202389354 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6187+0+0 (secure 0 0 0) 0x7f30b00971f0 con 0x7f30bc0d5c50 2024-05-11T12:14:53.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.910+0000 7f30c0e00640 1 --2- 172.21.15.167:0/1202389354 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f309c0762d0 0x7f309c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:53.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.911+0000 7f30c0e00640 1 --2- 172.21.15.167:0/1202389354 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f309c0762d0 0x7f309c078790 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f30a8006fd0 tx=0x7f30a8006bc0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:53.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:53.914+0000 7f30b9600640 1 -- 172.21.15.167:0/1202389354 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f30b0060960 con 0x7f30bc0d5c50 2024-05-11T12:14:54.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:54.160+0000 7f30c2c00640 1 -- 172.21.15.167:0/1202389354 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30bc06d7e0 con 0x7f30bc0d5c50 2024-05-11T12:14:54.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:54.161+0000 7f30b9600640 1 -- 172.21.15.167:0/1202389354 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5190 (secure 0 0 0) 0x7f30b0060260 con 0x7f30bc0d5c50 2024-05-11T12:14:54.161 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:54.161 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":68,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5401,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/2213070102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2213070102},{"type":"v1","addr":"172.21.15.204:6833","nonce":2213070102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5410,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1480900470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1480900470},{"type":"v1","addr":"172.21.15.204:6835","nonce":1480900470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67},{"gid":6870,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3735016882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3735016882},{"type":"v1","addr":"172.21.15.167:6835","nonce":3735016882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:14:33.067226+0000","modified":"2024-05-11T12:14:52.757909+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6885},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6885":{"gid":6885,"name":"c","rank":0,"incarnation":66,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.167:6837/2901182170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2901182170},{"type":"v1","addr":"172.21.15.167:6837","nonce":2901182170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-05-11T12:14:54.162 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 68 2024-05-11T12:14:54.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:54.163+0000 7f30c2c00640 1 -- 172.21.15.167:0/1202389354 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f309c0762d0 msgr2=0x7f309c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:54.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:54.163+0000 7f30c2c00640 1 --2- 172.21.15.167:0/1202389354 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f309c0762d0 0x7f309c078790 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f30a8006fd0 tx=0x7f30a8006bc0 comp rx=0 tx=0).stop 2024-05-11T12:14:54.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:54.163+0000 7f30c2c00640 1 -- 172.21.15.167:0/1202389354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30bc0d5c50 msgr2=0x7f30bc0a2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:54.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:54.163+0000 7f30c2c00640 1 --2- 172.21.15.167:0/1202389354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30bc0d5c50 0x7f30bc0a2930 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f30b000b4a0 tx=0x7f30b000b960 comp rx=0 tx=0).stop 2024-05-11T12:14:54.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:54.164+0000 7f30c2c00640 1 -- 172.21.15.167:0/1202389354 shutdown_connections 2024-05-11T12:14:54.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:54.164+0000 7f30c2c00640 1 --2- 172.21.15.167:0/1202389354 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f30bc15f530 0x7f30bc0b8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:54.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:54.164+0000 7f30c2c00640 1 --2- 172.21.15.167:0/1202389354 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f309c0762d0 0x7f309c078790 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:54.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:54.164+0000 7f30c2c00640 1 --2- 172.21.15.167:0/1202389354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30bc15f150 0x7f30bc0ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:54.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:54.164+0000 7f30c2c00640 1 --2- 172.21.15.167:0/1202389354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30bc0d5c50 0x7f30bc0a2930 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:54.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:54.164+0000 7f30c2c00640 1 -- 172.21.15.167:0/1202389354 >> 172.21.15.167:0/1202389354 conn(0x7f30bc09e3a0 msgr2=0x7f30bc0ba1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:54.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:54.164+0000 7f30c2c00640 1 -- 172.21.15.167:0/1202389354 shutdown_connections 2024-05-11T12:14:54.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:54.164+0000 7f30c2c00640 1 -- 172.21.15.167:0/1202389354 wait complete. 2024-05-11T12:14:54.586 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 68, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-11T12:14:33.067226+0000', 'modified': '2024-05-11T12:14:52.757909+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 100, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6885}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6885': {'gid': 6885, 'name': 'c', 'rank': 0, 'incarnation': 66, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.167:6837/2901182170', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6836', 'nonce': 2901182170}, {'type': 'v1', 'addr': '172.21.15.167:6837', 'nonce': 2901182170}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-11T12:14:54.586 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6885:up:rejoin 2024-05-11T12:14:55.587 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:14:55.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.798+0000 7efdfce00640 1 Processor -- start 2024-05-11T12:14:55.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.798+0000 7efdfce00640 1 -- start start 2024-05-11T12:14:55.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.800+0000 7efdfce00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7efdf8073a00 0x7efdf8073e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:55.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.800+0000 7efdfce00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdf815e880 0x7efdf815ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:55.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.800+0000 7efdfce00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdf8072520 con 0x7efdf80b62c0 2024-05-11T12:14:55.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.800+0000 7efdfce00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7efdf80732e0 con 0x7efdf8073a00 2024-05-11T12:14:55.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.800+0000 7efdfce00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7efdf80722d0 con 0x7efdf815e880 2024-05-11T12:14:55.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.800+0000 7efdf6800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdf815e880 0x7efdf815ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:55.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.800+0000 7efdf5e00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7efdf80b62c0 0x7efdf80b66a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.167:58326/0 (socket says 172.21.15.167:58326) 2024-05-11T12:14:55.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.800+0000 7efdf5e00640 1 -- 172.21.15.167:0/2112374377 learned_addr learned my addr 172.21.15.167:0/2112374377 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:14:55.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.801+0000 7efdf5400640 1 --2- 172.21.15.167:0/2112374377 >> v2:172.21.15.204:3300/0 conn(0x7efdf8073a00 0x7efdf8073e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:55.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.801+0000 7efdf6800640 1 -- 172.21.15.167:0/2112374377 >> v2:172.21.15.204:3300/0 conn(0x7efdf8073a00 msgr2=0x7efdf8073e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:55.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.801+0000 7efdf4a00640 1 -- 172.21.15.167:0/2112374377 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 583798616 0 0) 0x7efdf8072520 con 0x7efdf80b62c0 2024-05-11T12:14:55.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.801+0000 7efdf6800640 1 --2- 172.21.15.167:0/2112374377 >> v2:172.21.15.204:3300/0 conn(0x7efdf8073a00 0x7efdf8073e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:55.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.801+0000 7efdf6800640 1 -- 172.21.15.167:0/2112374377 >> v1:172.21.15.204:6789/0 conn(0x7efdf80b62c0 legacy=0x7efdf80b66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:55.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.801+0000 7efdf6800640 1 -- 172.21.15.167:0/2112374377 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdf8072800 con 0x7efdf815e880 2024-05-11T12:14:55.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.803+0000 7efdf6800640 1 --2- 172.21.15.167:0/2112374377 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdf815e880 0x7efdf815ed10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efdf00097a0 tx=0x7efdf002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3176162039757b48 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:55.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.803+0000 7efdf4a00640 1 -- 172.21.15.167:0/2112374377 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf0037070 con 0x7efdf815e880 2024-05-11T12:14:55.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.803+0000 7efdf4a00640 1 -- 172.21.15.167:0/2112374377 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdf0032440 con 0x7efdf815e880 2024-05-11T12:14:55.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.803+0000 7efdf4a00640 1 -- 172.21.15.167:0/2112374377 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf0032810 con 0x7efdf815e880 2024-05-11T12:14:55.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.804+0000 7efdfce00640 1 -- 172.21.15.167:0/2112374377 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdf815e880 msgr2=0x7efdf815ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:55.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.804+0000 7efdfce00640 1 --2- 172.21.15.167:0/2112374377 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdf815e880 0x7efdf815ed10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efdf00097a0 tx=0x7efdf002a180 comp rx=0 tx=0).stop 2024-05-11T12:14:55.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.804+0000 7efdfce00640 1 -- 172.21.15.167:0/2112374377 shutdown_connections 2024-05-11T12:14:55.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.804+0000 7efdfce00640 1 --2- 172.21.15.167:0/2112374377 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdf815e880 0x7efdf815ed10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:55.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.804+0000 7efdfce00640 1 --2- 172.21.15.167:0/2112374377 >> v2:172.21.15.204:3300/0 conn(0x7efdf8073a00 0x7efdf8073e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:55.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.804+0000 7efdfce00640 1 -- 172.21.15.167:0/2112374377 >> 172.21.15.167:0/2112374377 conn(0x7efdf80b2cb0 msgr2=0x7efdf80b3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:55.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.804+0000 7efdfce00640 1 -- 172.21.15.167:0/2112374377 shutdown_connections 2024-05-11T12:14:55.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.804+0000 7efdfce00640 1 -- 172.21.15.167:0/2112374377 wait complete. 2024-05-11T12:14:55.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.806+0000 7efdfce00640 1 Processor -- start 2024-05-11T12:14:55.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.806+0000 7efdfce00640 1 -- start start 2024-05-11T12:14:55.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.807+0000 7efdfce00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdf8073a00 0x7efdf815c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:55.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.807+0000 7efdfce00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efdf80b62c0 0x7efdf814f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:55.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.807+0000 7efdfce00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efdf815e880 0x7efdf8151930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:55.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.807+0000 7efdfce00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efdf80743c0 con 0x7efdf815e880 2024-05-11T12:14:55.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.807+0000 7efdfce00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7efdf8072dc0 con 0x7efdf80b62c0 2024-05-11T12:14:55.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.807+0000 7efdfce00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7efdf81579c0 con 0x7efdf8073a00 2024-05-11T12:14:55.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.807+0000 7efdf6800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efdf815e880 0x7efdf8151930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:55.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.807+0000 7efdf6800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efdf815e880 0x7efdf8151930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:52428/0 (socket says 172.21.15.167:52428) 2024-05-11T12:14:55.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.807+0000 7efdf6800640 1 -- 172.21.15.167:0/993858920 learned_addr learned my addr 172.21.15.167:0/993858920 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:55.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.807+0000 7efdf5e00640 1 --2- 172.21.15.167:0/993858920 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdf8073a00 0x7efdf815c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:55.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.807+0000 7efdf5400640 1 --2- 172.21.15.167:0/993858920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efdf80b62c0 0x7efdf814f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:55.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.807+0000 7efdf6800640 1 -- 172.21.15.167:0/993858920 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdf8073a00 msgr2=0x7efdf815c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:55.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.807+0000 7efdf6800640 1 --2- 172.21.15.167:0/993858920 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdf8073a00 0x7efdf815c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:55.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.807+0000 7efdf6800640 1 -- 172.21.15.167:0/993858920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efdf80b62c0 msgr2=0x7efdf814f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:55.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.807+0000 7efdf6800640 1 --2- 172.21.15.167:0/993858920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efdf80b62c0 0x7efdf814f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:55.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.807+0000 7efdf6800640 1 -- 172.21.15.167:0/993858920 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdf80b3d40 con 0x7efdf815e880 2024-05-11T12:14:55.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.807+0000 7efdf5e00640 1 --2- 172.21.15.167:0/993858920 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdf8073a00 0x7efdf815c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:55.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.808+0000 7efdf6800640 1 --2- 172.21.15.167:0/993858920 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efdf815e880 0x7efdf8151930 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7efdf002a1c0 tx=0x7efdf00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:55.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.808+0000 7efdee000640 1 -- 172.21.15.167:0/993858920 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf0037040 con 0x7efdf815e880 2024-05-11T12:14:55.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.808+0000 7efdee000640 1 -- 172.21.15.167:0/993858920 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdf0042070 con 0x7efdf815e880 2024-05-11T12:14:55.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.808+0000 7efdee000640 1 -- 172.21.15.167:0/993858920 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf00308a0 con 0x7efdf815e880 2024-05-11T12:14:55.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.808+0000 7efdfce00640 1 -- 172.21.15.167:0/993858920 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdf80b4020 con 0x7efdf815e880 2024-05-11T12:14:55.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.808+0000 7efdfce00640 1 -- 172.21.15.167:0/993858920 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdf80b45e0 con 0x7efdf815e880 2024-05-11T12:14:55.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.810+0000 7efdfce00640 1 -- 172.21.15.167:0/993858920 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdf8069820 con 0x7efdf815e880 2024-05-11T12:14:55.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.811+0000 7efdee000640 1 -- 172.21.15.167:0/993858920 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7efdf0030a40 con 0x7efdf815e880 2024-05-11T12:14:55.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.812+0000 7efdee000640 1 --2- 172.21.15.167:0/993858920 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7efdd80762d0 0x7efdd8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:55.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.812+0000 7efdee000640 1 -- 172.21.15.167:0/993858920 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6187+0+0 (secure 0 0 0) 0x7efdf00b8280 con 0x7efdf815e880 2024-05-11T12:14:55.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.812+0000 7efdf5e00640 1 --2- 172.21.15.167:0/993858920 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7efdd80762d0 0x7efdd8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:55.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.813+0000 7efdf5e00640 1 --2- 172.21.15.167:0/993858920 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7efdd80762d0 0x7efdd8078790 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7efde80012e0 tx=0x7efde8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:55.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:55.817+0000 7efdee000640 1 -- 172.21.15.167:0/993858920 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7efdf00818d0 con 0x7efdf815e880 2024-05-11T12:14:56.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.063+0000 7efdfce00640 1 -- 172.21.15.167:0/993858920 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efdf8156420 con 0x7efdf815e880 2024-05-11T12:14:56.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.064+0000 7efdee000640 1 -- 172.21.15.167:0/993858920 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+5197 (secure 0 0 0) 0x7efdf00811d0 con 0x7efdf815e880 2024-05-11T12:14:56.065 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:14:56.065 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":69,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5401,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/2213070102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2213070102},{"type":"v1","addr":"172.21.15.204:6833","nonce":2213070102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5410,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1480900470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1480900470},{"type":"v1","addr":"172.21.15.204:6835","nonce":1480900470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67},{"gid":6870,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3735016882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3735016882},{"type":"v1","addr":"172.21.15.167:6835","nonce":3735016882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:14:33.067226+0000","modified":"2024-05-11T12:14:54.768036+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6885},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6885":{"gid":6885,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6837/2901182170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2901182170},{"type":"v1","addr":"172.21.15.167:6837","nonce":2901182170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6885,"qdb_cluster":[6885]},"id":7}]} 2024-05-11T12:14:56.065 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 69 2024-05-11T12:14:56.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.067+0000 7efdfce00640 1 -- 172.21.15.167:0/993858920 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7efdd80762d0 msgr2=0x7efdd8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:56.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.067+0000 7efdfce00640 1 --2- 172.21.15.167:0/993858920 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7efdd80762d0 0x7efdd8078790 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7efde80012e0 tx=0x7efde8009040 comp rx=0 tx=0).stop 2024-05-11T12:14:56.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.067+0000 7efdfce00640 1 -- 172.21.15.167:0/993858920 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efdf815e880 msgr2=0x7efdf8151930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:56.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.067+0000 7efdfce00640 1 --2- 172.21.15.167:0/993858920 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efdf815e880 0x7efdf8151930 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7efdf002a1c0 tx=0x7efdf00046c0 comp rx=0 tx=0).stop 2024-05-11T12:14:56.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.067+0000 7efdfce00640 1 -- 172.21.15.167:0/993858920 shutdown_connections 2024-05-11T12:14:56.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.067+0000 7efdfce00640 1 --2- 172.21.15.167:0/993858920 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efdf815e880 0x7efdf8151930 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:56.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.067+0000 7efdfce00640 1 --2- 172.21.15.167:0/993858920 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7efdd80762d0 0x7efdd8078790 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:56.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.067+0000 7efdfce00640 1 --2- 172.21.15.167:0/993858920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efdf80b62c0 0x7efdf814f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:56.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.067+0000 7efdfce00640 1 --2- 172.21.15.167:0/993858920 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdf8073a00 0x7efdf815c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:56.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.067+0000 7efdfce00640 1 -- 172.21.15.167:0/993858920 >> 172.21.15.167:0/993858920 conn(0x7efdf80b2cb0 msgr2=0x7efdf8156c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:56.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.067+0000 7efdfce00640 1 -- 172.21.15.167:0/993858920 shutdown_connections 2024-05-11T12:14:56.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.067+0000 7efdfce00640 1 -- 172.21.15.167:0/993858920 wait complete. 2024-05-11T12:14:56.489 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 69, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-11T12:14:33.067226+0000', 'modified': '2024-05-11T12:14:54.768036+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 100, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6885}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6885': {'gid': 6885, 'name': 'c', 'rank': 0, 'incarnation': 66, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.167:6837/2901182170', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6836', 'nonce': 2901182170}, {'type': 'v1', 'addr': '172.21.15.167:6837', 'nonce': 2901182170}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6885, 'qdb_cluster': [6885]} 2024-05-11T12:14:56.490 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-11T12:14:56.490 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-11T12:14:56.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.704+0000 7f0f86200640 1 Processor -- start 2024-05-11T12:14:56.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.705+0000 7f0f86200640 1 -- start start 2024-05-11T12:14:56.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.706+0000 7f0f86200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f8013eee0 0x7f0f80073c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:56.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.706+0000 7f0f86200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0f800d0020 0x7f0f8013fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:56.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.706+0000 7f0f86200640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f8013f6c0 con 0x7f0f800741b0 2024-05-11T12:14:56.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.706+0000 7f0f86200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f0f80074630 con 0x7f0f800d0020 2024-05-11T12:14:56.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.706+0000 7f0f86200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0f8015c960 con 0x7f0f8013eee0 2024-05-11T12:14:56.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.706+0000 7f0f7ea00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f0f800741b0 0x7f0f800cf910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:43248/0 (socket says 172.21.15.167:43248) 2024-05-11T12:14:56.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.707+0000 7f0f7ea00640 1 -- 172.21.15.167:0/80315212 learned_addr learned my addr 172.21.15.167:0/80315212 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:14:56.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.707+0000 7f0f7fe00640 1 --2- 172.21.15.167:0/80315212 >> v2:172.21.15.204:3300/0 conn(0x7f0f800d0020 0x7f0f8013fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:56.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.707+0000 7f0f7f400640 1 --2- 172.21.15.167:0/80315212 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f8013eee0 0x7f0f80073c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:56.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.707+0000 7f0f7e000640 1 -- 172.21.15.167:0/80315212 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4187977116 0 0) 0x7f0f8013f6c0 con 0x7f0f800741b0 2024-05-11T12:14:56.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.707+0000 7f0f7e000640 1 -- 172.21.15.167:0/80315212 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f60003520 con 0x7f0f800741b0 2024-05-11T12:14:56.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.707+0000 7f0f7e000640 1 -- 172.21.15.167:0/80315212 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 25667429 0 0) 0x7f0f60003520 con 0x7f0f800741b0 2024-05-11T12:14:56.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.707+0000 7f0f7e000640 1 -- 172.21.15.167:0/80315212 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f8013eee0 msgr2=0x7f0f80073c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:56.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.707+0000 7f0f7e000640 1 --2- 172.21.15.167:0/80315212 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f8013eee0 0x7f0f80073c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:56.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.707+0000 7f0f7e000640 1 -- 172.21.15.167:0/80315212 >> v2:172.21.15.204:3300/0 conn(0x7f0f800d0020 msgr2=0x7f0f8013fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:56.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.707+0000 7f0f7e000640 1 --2- 172.21.15.167:0/80315212 >> v2:172.21.15.204:3300/0 conn(0x7f0f800d0020 0x7f0f8013fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:56.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.707+0000 7f0f7e000640 1 -- 172.21.15.167:0/80315212 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f800d0620 con 0x7f0f800741b0 2024-05-11T12:14:56.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.707+0000 7f0f7e000640 1 -- 172.21.15.167:0/80315212 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f0f68001a90 con 0x7f0f800741b0 2024-05-11T12:14:56.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.708+0000 7f0f7e000640 1 -- 172.21.15.167:0/80315212 >> v1:172.21.15.167:6789/0 conn(0x7f0f800741b0 legacy=0x7f0f800cf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:56.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.708+0000 7f0f7e000640 1 --2- 172.21.15.167:0/80315212 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f8013eee0 0x7f0f60003a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:56.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.708+0000 7f0f7e000640 1 --2- 172.21.15.167:0/80315212 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f60004020 0x7f0f60004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:56.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.708+0000 7f0f7e000640 1 --2- 172.21.15.167:0/80315212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f60004d40 0x7f0f600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:56.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.708+0000 7f0f7e000640 1 -- 172.21.15.167:0/80315212 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0f80074630 con 0x7f0f60004020 2024-05-11T12:14:56.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.708+0000 7f0f7e000640 1 -- 172.21.15.167:0/80315212 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8015c960 con 0x7f0f60004d40 2024-05-11T12:14:56.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.708+0000 7f0f7e000640 1 -- 172.21.15.167:0/80315212 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0f6000ba70 con 0x7f0f8013eee0 2024-05-11T12:14:56.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.708+0000 7f0f7ea00640 1 --2- 172.21.15.167:0/80315212 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f60004020 0x7f0f60004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:56.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.708+0000 7f0f7f400640 1 --2- 172.21.15.167:0/80315212 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f8013eee0 0x7f0f60003a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:56.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.708+0000 7f0f7ea00640 1 -- 172.21.15.167:0/80315212 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f8013eee0 msgr2=0x7f0f60003a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:56.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.708+0000 7f0f7ea00640 1 --2- 172.21.15.167:0/80315212 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f8013eee0 0x7f0f60003a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:56.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.708+0000 7f0f7ea00640 1 -- 172.21.15.167:0/80315212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f60004d40 msgr2=0x7f0f600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:56.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.708+0000 7f0f7ea00640 1 --2- 172.21.15.167:0/80315212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f60004d40 0x7f0f600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:56.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.708+0000 7f0f7ea00640 1 -- 172.21.15.167:0/80315212 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f60003520 con 0x7f0f60004020 2024-05-11T12:14:56.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.708+0000 7f0f7f400640 1 --2- 172.21.15.167:0/80315212 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f8013eee0 0x7f0f60003a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:56.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.709+0000 7f0f7ea00640 1 --2- 172.21.15.167:0/80315212 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f60004020 0x7f0f60004800 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f0f6800a8f0 tx=0x7f0f60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc277b8cb7a5fe03 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:56.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.710+0000 7f0f7e000640 1 -- 172.21.15.167:0/80315212 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6800a4c0 con 0x7f0f60004020 2024-05-11T12:14:56.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.710+0000 7f0f7e000640 1 -- 172.21.15.167:0/80315212 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f6802f070 con 0x7f0f60004020 2024-05-11T12:14:56.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.710+0000 7f0f7e000640 1 -- 172.21.15.167:0/80315212 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6802bc20 con 0x7f0f60004020 2024-05-11T12:14:56.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.710+0000 7f0f86200640 1 -- 172.21.15.167:0/80315212 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f60004020 msgr2=0x7f0f60004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:56.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.710+0000 7f0f86200640 1 --2- 172.21.15.167:0/80315212 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f60004020 0x7f0f60004800 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f0f6800a8f0 tx=0x7f0f60005830 comp rx=0 tx=0).stop 2024-05-11T12:14:56.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.710+0000 7f0f7fe00640 1 -- 172.21.15.167:0/80315212 reap_dead start 2024-05-11T12:14:56.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.710+0000 7f0f86200640 1 -- 172.21.15.167:0/80315212 shutdown_connections 2024-05-11T12:14:56.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.710+0000 7f0f86200640 1 -- 172.21.15.167:0/80315212 >> 172.21.15.167:0/80315212 conn(0x7f0f800b2cb0 msgr2=0x7f0f800b3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:56.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.710+0000 7f0f86200640 1 -- 172.21.15.167:0/80315212 shutdown_connections 2024-05-11T12:14:56.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.710+0000 7f0f86200640 1 -- 172.21.15.167:0/80315212 wait complete. 2024-05-11T12:14:56.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.712+0000 7f0f86200640 1 Processor -- start 2024-05-11T12:14:56.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.712+0000 7f0f86200640 1 -- start start 2024-05-11T12:14:56.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.713+0000 7f0f86200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f60004020 0x7f0f80138430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:56.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.713+0000 7f0f86200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f80136e20 0x7f0f801335c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:56.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.713+0000 7f0f86200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f801532b0 0x7f0f80133e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:56.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.713+0000 7f0f86200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0f800d0be0 con 0x7f0f80136e20 2024-05-11T12:14:56.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.713+0000 7f0f86200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8013f9f0 con 0x7f0f801532b0 2024-05-11T12:14:56.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.713+0000 7f0f86200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0f8013f2c0 con 0x7f0f60004020 2024-05-11T12:14:56.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.713+0000 7f0f7ea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f80136e20 0x7f0f801335c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:56.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.714+0000 7f0f7ea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f80136e20 0x7f0f801335c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:52444/0 (socket says 172.21.15.167:52444) 2024-05-11T12:14:56.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.714+0000 7f0f7ea00640 1 -- 172.21.15.167:0/2328665077 learned_addr learned my addr 172.21.15.167:0/2328665077 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:56.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.714+0000 7f0f7ea00640 1 -- 172.21.15.167:0/2328665077 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f60004020 msgr2=0x7f0f80138430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:56.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.714+0000 7f0f7fe00640 1 --2- 172.21.15.167:0/2328665077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f801532b0 0x7f0f80133e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:56.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.714+0000 7f0f7ea00640 1 --2- 172.21.15.167:0/2328665077 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f60004020 0x7f0f80138430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:56.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.714+0000 7f0f7ea00640 1 -- 172.21.15.167:0/2328665077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f801532b0 msgr2=0x7f0f80133e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:56.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.714+0000 7f0f7ea00640 1 --2- 172.21.15.167:0/2328665077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f801532b0 0x7f0f80133e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:56.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.714+0000 7f0f7ea00640 1 -- 172.21.15.167:0/2328665077 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f80137390 con 0x7f0f80136e20 2024-05-11T12:14:56.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.714+0000 7f0f7ea00640 1 --2- 172.21.15.167:0/2328665077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f80136e20 0x7f0f801335c0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f0f6802b470 tx=0x7f0f8013f960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:56.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.714+0000 7f0f77400640 1 -- 172.21.15.167:0/2328665077 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f68001a80 con 0x7f0f80136e20 2024-05-11T12:14:56.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.715+0000 7f0f77400640 1 -- 172.21.15.167:0/2328665077 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f6802bd80 con 0x7f0f80136e20 2024-05-11T12:14:56.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.715+0000 7f0f77400640 1 -- 172.21.15.167:0/2328665077 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f68004da0 con 0x7f0f80136e20 2024-05-11T12:14:56.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.715+0000 7f0f86200640 1 -- 172.21.15.167:0/2328665077 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f80137670 con 0x7f0f80136e20 2024-05-11T12:14:56.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.715+0000 7f0f86200640 1 -- 172.21.15.167:0/2328665077 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f80137c30 con 0x7f0f80136e20 2024-05-11T12:14:56.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.717+0000 7f0f77400640 1 -- 172.21.15.167:0/2328665077 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0f68034470 con 0x7f0f80136e20 2024-05-11T12:14:56.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.719+0000 7f0f77400640 1 --2- 172.21.15.167:0/2328665077 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0f5c076470 0x7f0f5c078930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:56.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.719+0000 7f0f77400640 1 -- 172.21.15.167:0/2328665077 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6187+0+0 (secure 0 0 0) 0x7f0f680b8390 con 0x7f0f80136e20 2024-05-11T12:14:56.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.719+0000 7f0f8902e740 1 Processor -- start 2024-05-11T12:14:56.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.719+0000 7f0f7f400640 1 --2- 172.21.15.167:0/2328665077 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0f5c076470 0x7f0f5c078930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:56.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.719+0000 7f0f8902e740 1 -- start start 2024-05-11T12:14:56.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.720+0000 7f0f7f400640 1 --2- 172.21.15.167:0/2328665077 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0f5c076470 0x7f0f5c078930 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0f70009910 tx=0x7f0f70008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:56.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.720+0000 7f0f8902e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c8a0c7840 0x562c8a0c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:56.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.720+0000 7f0f8902e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8a0c26d0 0x562c8a0c6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:56.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.720+0000 7f0f8902e740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x562c8a0c6980 0x562c8a0c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:56.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.720+0000 7f0f8902e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562c8a02b790 con 0x562c8a0c26d0 2024-05-11T12:14:56.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.720+0000 7f0f8902e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562c89fd0f20 con 0x562c8a0c7840 2024-05-11T12:14:56.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.720+0000 7f0f8902e740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x562c8a027750 con 0x562c8a0c6980 2024-05-11T12:14:56.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.720+0000 7f0f7ea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8a0c26d0 0x562c8a0c6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:56.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.721+0000 7f0f7ea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8a0c26d0 0x562c8a0c6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:52446/0 (socket says 172.21.15.167:52446) 2024-05-11T12:14:56.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.721+0000 7f0f7ea00640 1 -- 172.21.15.167:0/1965463238 learned_addr learned my addr 172.21.15.167:0/1965463238 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:56.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.721+0000 7f0f7ea00640 1 -- 172.21.15.167:0/1965463238 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x562c8a0c6980 msgr2=0x562c8a0c6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:56.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.721+0000 7f0f7ea00640 1 --2- 172.21.15.167:0/1965463238 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x562c8a0c6980 0x562c8a0c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:56.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.721+0000 7f0f7ea00640 1 -- 172.21.15.167:0/1965463238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c8a0c7840 msgr2=0x562c8a0c7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:56.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.721+0000 7f0f7ea00640 1 --2- 172.21.15.167:0/1965463238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c8a0c7840 0x562c8a0c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:56.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.721+0000 7f0f7ea00640 1 -- 172.21.15.167:0/1965463238 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c89d3d2d0 con 0x562c8a0c26d0 2024-05-11T12:14:56.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.721+0000 7f0f7ea00640 1 --2- 172.21.15.167:0/1965463238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8a0c26d0 0x562c8a0c6440 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f0f6807e670 tx=0x7f0f68080b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4cff2ea0d6db27b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:56.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.721+0000 7f0f6f400640 1 -- 172.21.15.167:0/1965463238 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f68081b00 con 0x562c8a0c26d0 2024-05-11T12:14:56.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.721+0000 7f0f6f400640 1 -- 172.21.15.167:0/1965463238 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f6802f070 con 0x562c8a0c26d0 2024-05-11T12:14:56.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.721+0000 7f0f6f400640 1 -- 172.21.15.167:0/1965463238 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f68081400 con 0x562c8a0c26d0 2024-05-11T12:14:56.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.722+0000 7f0f8902e740 1 -- 172.21.15.167:0/1965463238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8a0c26d0 msgr2=0x562c8a0c6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:56.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.722+0000 7f0f8902e740 1 --2- 172.21.15.167:0/1965463238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8a0c26d0 0x562c8a0c6440 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f0f6807e670 tx=0x7f0f68080b70 comp rx=0 tx=0).stop 2024-05-11T12:14:56.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.722+0000 7f0f8902e740 1 -- 172.21.15.167:0/1965463238 shutdown_connections 2024-05-11T12:14:56.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.722+0000 7f0f8902e740 1 --2- 172.21.15.167:0/1965463238 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x562c8a0c6980 0x562c8a0c6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:56.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.722+0000 7f0f8902e740 1 --2- 172.21.15.167:0/1965463238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8a0c26d0 0x562c8a0c6440 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:56.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.722+0000 7f0f8902e740 1 --2- 172.21.15.167:0/1965463238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c8a0c7840 0x562c8a0c7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:56.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.722+0000 7f0f8902e740 1 -- 172.21.15.167:0/1965463238 >> 172.21.15.167:0/1965463238 conn(0x562c8a0c4940 msgr2=0x562c8a10d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:56.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.722+0000 7f0f8902e740 1 -- 172.21.15.167:0/1965463238 shutdown_connections 2024-05-11T12:14:56.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.722+0000 7f0f8902e740 1 -- 172.21.15.167:0/1965463238 wait complete. 2024-05-11T12:14:56.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.723+0000 7f0f8902e740 1 -- start start 2024-05-11T12:14:56.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.723+0000 7f0f8902e740 1 Processor -- start 2024-05-11T12:14:56.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.724+0000 7f0f8902e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8a0c6980 0x562c8a0eaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:56.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.724+0000 7f0f8902e740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x562c8a0c7840 0x562c8a115640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:56.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.724+0000 7f0f8902e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c8a0eb7c0 0x562c8a1142c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:56.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.724+0000 7f0f8902e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562c8a027750 con 0x562c8a0c6980 2024-05-11T12:14:56.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.724+0000 7f0f8902e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562c89fe49c0 con 0x562c8a0eb7c0 2024-05-11T12:14:56.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.724+0000 7f0f8902e740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x562c8a02b790 con 0x562c8a0c7840 2024-05-11T12:14:56.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.724+0000 7f0f8902e740 10 client.0 _ll_register_callbacks cb 0x562c89fb1120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:14:56.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.725+0000 7f0f7f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8a0c6980 0x562c8a0eaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:56.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.725+0000 7f0f7f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8a0c6980 0x562c8a0eaae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:52454/0 (socket says 172.21.15.167:52454) 2024-05-11T12:14:56.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.725+0000 7f0f7f400640 1 -- 172.21.15.167:0/2894450262 learned_addr learned my addr 172.21.15.167:0/2894450262 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:56.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.725+0000 7f0f7f400640 1 -- 172.21.15.167:0/2894450262 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x562c8a0c7840 msgr2=0x562c8a115640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:56.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.725+0000 7f0f7f400640 1 --2- 172.21.15.167:0/2894450262 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x562c8a0c7840 0x562c8a115640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:56.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.725+0000 7f0f7f400640 1 -- 172.21.15.167:0/2894450262 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c8a0eb7c0 msgr2=0x562c8a1142c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:56.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.725+0000 7f0f7f400640 1 --2- 172.21.15.167:0/2894450262 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c8a0eb7c0 0x562c8a1142c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:56.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.725+0000 7f0f7f400640 1 -- 172.21.15.167:0/2894450262 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c89e076b0 con 0x562c8a0c6980 2024-05-11T12:14:56.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.725+0000 7f0f7f400640 1 --2- 172.21.15.167:0/2894450262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8a0c6980 0x562c8a0eaae0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f0f700156f0 tx=0x7f0f70015af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6daf32c55ba17658 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:56.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.725+0000 7f0f67e00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-11T12:14:56.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.726+0000 7f0f67e00640 1 -- 172.21.15.167:0/2894450262 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f70026060 con 0x562c8a0c6980 2024-05-11T12:14:56.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.726+0000 7f0f67e00640 1 -- 172.21.15.167:0/2894450262 <== mon.0 v2:172.21.15.167:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 6187+0+0 (secure 0 0 0) 0x7f0f700169b0 con 0x562c8a0c6980 2024-05-11T12:14:56.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.726+0000 7f0f8902e740 1 -- 172.21.15.167:0/2894450262 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f6802f070 con 0x562c8a0c6980 2024-05-11T12:14:56.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.726+0000 7f0f67e00640 1 -- 172.21.15.167:0/2894450262 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f0f7002c050 con 0x562c8a0c6980 2024-05-11T12:14:56.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.726+0000 7f0f8902e740 10 client.6912 fetch_fsmap learned FSMap version 69 2024-05-11T12:14:56.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.726+0000 7f0f8902e740 1 -- 172.21.15.167:0/2894450262 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) -- 0x562c8a113ac0 con 0x562c8a0c6980 2024-05-11T12:14:56.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.726+0000 7f0f67e00640 1 -- 172.21.15.167:0/2894450262 <== mon.0 v2:172.21.15.167:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f6802f070 con 0x562c8a0c6980 2024-05-11T12:14:56.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.726+0000 7f0f67e00640 1 -- 172.21.15.167:0/2894450262 <== mon.0 v2:172.21.15.167:3300/0 5 ==== fsmap(e 69) ==== 3494+0+0 (secure 0 0 0) 0x7f0f7002f020 con 0x562c8a0c6980 2024-05-11T12:14:56.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.726+0000 7f0f8902e740 10 client.6912 fetch_fsmap finished waiting for FSMap version 69 2024-05-11T12:14:56.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.726+0000 7f0f8902e740 10 client.6912 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-11T12:14:56.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.726+0000 7f0f8902e740 20 client.6912 populate_metadata read hostname 'smithi167' 2024-05-11T12:14:56.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.726+0000 7f0f8902e740 1 --2- 172.21.15.167:0/2894450262 >> [v2:172.21.15.167:6836/2901182170,v1:172.21.15.167:6837/2901182170] conn(0x562c8a111eb0 0x562c8a1122b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:56.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.726+0000 7f0f8902e740 4 client.6912 mds_command: new command op to 6885 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:14:56.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.726+0000 7f0f8902e740 1 -- 172.21.15.167:0/2894450262 --> [v2:172.21.15.167:6836/2901182170,v1:172.21.15.167:6837/2901182170] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c8a0393a0 con 0x562c8a111eb0 2024-05-11T12:14:56.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.727+0000 7f0f7ea00640 1 --2- 172.21.15.167:0/2894450262 >> [v2:172.21.15.167:6836/2901182170,v1:172.21.15.167:6837/2901182170] conn(0x562c8a111eb0 0x562c8a1122b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:56.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.727+0000 7f0f7ea00640 1 --2- 172.21.15.167:0/2894450262 >> [v2:172.21.15.167:6836/2901182170,v1:172.21.15.167:6837/2901182170] conn(0x562c8a111eb0 0x562c8a1122b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d196a37aab365862 server_cookie=6812a704a0dd47cc in_seq=0 out_seq=0 2024-05-11T12:14:56.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.727+0000 7f0f67e00640 10 client.6912 ms_handle_connect on v2:172.21.15.167:6836/2901182170 2024-05-11T12:14:56.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.728+0000 7f0f67e00640 1 -- 172.21.15.167:0/2894450262 <== mds.0 v2:172.21.15.167:6836/2901182170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562c8a0393a0 con 0x562c8a111eb0 2024-05-11T12:14:56.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.728+0000 7f0f67e00640 10 client.6912 handle_command_reply: tid=0 2024-05-11T12:14:56.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.728+0000 7f0f8902e740 1 client.6912 shutdown 2024-05-11T12:14:56.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.728+0000 7f0f8902e740 1 -- 172.21.15.167:0/2894450262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8a0c6980 msgr2=0x562c8a0eaae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:56.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.728+0000 7f0f8902e740 1 --2- 172.21.15.167:0/2894450262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8a0c6980 0x562c8a0eaae0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f0f700156f0 tx=0x7f0f70015af0 comp rx=0 tx=0).stop 2024-05-11T12:14:56.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.728+0000 7f0f8902e740 1 -- 172.21.15.167:0/2894450262 shutdown_connections 2024-05-11T12:14:56.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.728+0000 7f0f8902e740 1 --2- 172.21.15.167:0/2894450262 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c8a0eb7c0 0x562c8a1142c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:56.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.728+0000 7f0f8902e740 1 --2- 172.21.15.167:0/2894450262 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x562c8a0c7840 0x562c8a115640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:56.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.728+0000 7f0f8902e740 1 --2- 172.21.15.167:0/2894450262 >> [v2:172.21.15.167:6836/2901182170,v1:172.21.15.167:6837/2901182170] conn(0x562c8a111eb0 0x562c8a1122b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:56.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.729+0000 7f0f8902e740 1 --2- 172.21.15.167:0/2894450262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8a0c6980 0x562c8a0eaae0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:56.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.729+0000 7f0f8902e740 1 -- 172.21.15.167:0/2894450262 >> 172.21.15.167:0/2894450262 conn(0x562c8a0c26d0 msgr2=0x562c8a0c1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:56.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.729+0000 7f0f67e00640 0 client.6912 ms_handle_reset on v2:172.21.15.167:6836/2901182170 2024-05-11T12:14:56.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.729+0000 7f0f8902e740 1 -- 172.21.15.167:0/2894450262 shutdown_connections 2024-05-11T12:14:56.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.729+0000 7f0f8902e740 1 -- 172.21.15.167:0/2894450262 wait complete. 2024-05-11T12:14:56.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.729+0000 7f0f8902e740 20 client.6912 trim_cache size 0 max 16384 2024-05-11T12:14:56.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.750+0000 7f0f8902e740 1 Processor -- start 2024-05-11T12:14:56.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.750+0000 7f0f8902e740 1 -- start start 2024-05-11T12:14:56.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.751+0000 7f0f8902e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c8a0c6980 0x562c8a103830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:56.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.751+0000 7f0f8902e740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x562c8a0c7840 0x562c8a0c1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:56.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.751+0000 7f0f8902e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8a0eb7c0 0x562c8a0eaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:56.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.751+0000 7f0f8902e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0f802018f0 con 0x562c8a0eb7c0 2024-05-11T12:14:56.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.751+0000 7f0f8902e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0f80200c50 con 0x562c8a0c6980 2024-05-11T12:14:56.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.751+0000 7f0f8902e740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0f802012a0 con 0x562c8a0c7840 2024-05-11T12:14:56.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.751+0000 7f0f7fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8a0eb7c0 0x562c8a0eaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:56.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.751+0000 7f0f7fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8a0eb7c0 0x562c8a0eaae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:52456/0 (socket says 172.21.15.167:52456) 2024-05-11T12:14:56.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.751+0000 7f0f7fe00640 1 -- 172.21.15.167:0/3235766269 learned_addr learned my addr 172.21.15.167:0/3235766269 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:56.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.751+0000 7f0f7ea00640 1 --2- 172.21.15.167:0/3235766269 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x562c8a0c7840 0x562c8a0c1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:56.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.752+0000 7f0f7f400640 1 --2- 172.21.15.167:0/3235766269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c8a0c6980 0x562c8a103830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:56.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.752+0000 7f0f7fe00640 1 -- 172.21.15.167:0/3235766269 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x562c8a0c7840 msgr2=0x562c8a0c1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:56.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.752+0000 7f0f7fe00640 1 --2- 172.21.15.167:0/3235766269 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x562c8a0c7840 0x562c8a0c1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:56.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.752+0000 7f0f7fe00640 1 -- 172.21.15.167:0/3235766269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c8a0c6980 msgr2=0x562c8a103830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:56.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.752+0000 7f0f7fe00640 1 --2- 172.21.15.167:0/3235766269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c8a0c6980 0x562c8a103830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:56.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.752+0000 7f0f7fe00640 1 -- 172.21.15.167:0/3235766269 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c8a112dd0 con 0x562c8a0eb7c0 2024-05-11T12:14:56.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.752+0000 7f0f7fe00640 1 --2- 172.21.15.167:0/3235766269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8a0eb7c0 0x562c8a0eaae0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f0f780097a0 tx=0x7f0f78009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=673da662e3f873c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:56.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.752+0000 7f0f6ea00640 1 -- 172.21.15.167:0/3235766269 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f78018070 con 0x562c8a0eb7c0 2024-05-11T12:14:56.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.752+0000 7f0f6ea00640 1 -- 172.21.15.167:0/3235766269 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f78010b60 con 0x562c8a0eb7c0 2024-05-11T12:14:56.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.752+0000 7f0f6ea00640 1 -- 172.21.15.167:0/3235766269 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7800f040 con 0x562c8a0eb7c0 2024-05-11T12:14:56.754 INFO:tasks.ceph.mds.c.smithi167.stderr:2024-05-11T12:14:56.752+0000 7f9de7000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-11T12:14:56.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.753+0000 7f0f8902e740 1 -- 172.21.15.167:0/3235766269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8a0eb7c0 msgr2=0x562c8a0eaae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:56.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.753+0000 7f0f8902e740 1 --2- 172.21.15.167:0/3235766269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8a0eb7c0 0x562c8a0eaae0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f0f780097a0 tx=0x7f0f78009c60 comp rx=0 tx=0).stop 2024-05-11T12:14:56.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.753+0000 7f0f8902e740 1 -- 172.21.15.167:0/3235766269 shutdown_connections 2024-05-11T12:14:56.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.753+0000 7f0f8902e740 1 --2- 172.21.15.167:0/3235766269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8a0eb7c0 0x562c8a0eaae0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:56.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.753+0000 7f0f8902e740 1 --2- 172.21.15.167:0/3235766269 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x562c8a0c7840 0x562c8a0c1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:56.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.753+0000 7f0f8902e740 1 --2- 172.21.15.167:0/3235766269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c8a0c6980 0x562c8a103830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:56.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.753+0000 7f0f8902e740 1 -- 172.21.15.167:0/3235766269 >> 172.21.15.167:0/3235766269 conn(0x562c8a0c26d0 msgr2=0x562c8a0c6110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:56.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.753+0000 7f0f8902e740 1 -- 172.21.15.167:0/3235766269 shutdown_connections 2024-05-11T12:14:56.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.753+0000 7f0f8902e740 1 -- 172.21.15.167:0/3235766269 wait complete. 2024-05-11T12:14:56.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.754+0000 7f0f8902e740 1 -- start start 2024-05-11T12:14:56.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.754+0000 7f0f8902e740 1 Processor -- start 2024-05-11T12:14:56.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.755+0000 7f0f8902e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c8a0c6980 0x562c8a18bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:56.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.755+0000 7f0f8902e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8a0c7840 0x562c8a10c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:56.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.755+0000 7f0f8902e740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x562c8a0eb7c0 0x562c8a11bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:56.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.755+0000 7f0f8902e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562c8a0ed900 con 0x562c8a0c7840 2024-05-11T12:14:56.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.755+0000 7f0f8902e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562c89fd0f20 con 0x562c8a0c6980 2024-05-11T12:14:56.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.755+0000 7f0f8902e740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x562c8a1103a0 con 0x562c8a0eb7c0 2024-05-11T12:14:56.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.755+0000 7f0f8902e740 10 client.0 _ll_register_callbacks cb 0x7f0f700095c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:14:56.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.755+0000 7f0f7ea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8a0c7840 0x562c8a10c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:56.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.756+0000 7f0f7ea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8a0c7840 0x562c8a10c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:52468/0 (socket says 172.21.15.167:52468) 2024-05-11T12:14:56.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.756+0000 7f0f7ea00640 1 -- 172.21.15.167:0/1507390991 learned_addr learned my addr 172.21.15.167:0/1507390991 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:56.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.756+0000 7f0f7fe00640 1 --2- 172.21.15.167:0/1507390991 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x562c8a0eb7c0 0x562c8a11bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:56.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.756+0000 7f0f7ea00640 1 -- 172.21.15.167:0/1507390991 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x562c8a0eb7c0 msgr2=0x562c8a11bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:56.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.756+0000 7f0f7ea00640 1 --2- 172.21.15.167:0/1507390991 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x562c8a0eb7c0 0x562c8a11bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:56.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.756+0000 7f0f7ea00640 1 -- 172.21.15.167:0/1507390991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c8a0c6980 msgr2=0x562c8a18bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:56.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.756+0000 7f0f7f400640 1 --2- 172.21.15.167:0/1507390991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c8a0c6980 0x562c8a18bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:56.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.756+0000 7f0f7ea00640 1 --2- 172.21.15.167:0/1507390991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c8a0c6980 0x562c8a18bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:56.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.756+0000 7f0f7ea00640 1 -- 172.21.15.167:0/1507390991 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c8a0eaff0 con 0x562c8a0c7840 2024-05-11T12:14:56.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.756+0000 7f0f7ea00640 1 --2- 172.21.15.167:0/1507390991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8a0c7840 0x562c8a10c4f0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f0f6802bfd0 tx=0x7f0f68097000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a38b71b3a1806d55 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:56.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.756+0000 7f0f67e00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-11T12:14:56.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.756+0000 7f0f67e00640 1 -- 172.21.15.167:0/1507390991 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6807da50 con 0x562c8a0c7840 2024-05-11T12:14:56.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.757+0000 7f0f67e00640 1 -- 172.21.15.167:0/1507390991 <== mon.0 v2:172.21.15.167:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 6187+0+0 (secure 0 0 0) 0x7f0f68042cd0 con 0x562c8a0c7840 2024-05-11T12:14:56.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.757+0000 7f0f8902e740 1 -- 172.21.15.167:0/1507390991 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f78010b60 con 0x562c8a0c7840 2024-05-11T12:14:56.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.757+0000 7f0f67e00640 1 -- 172.21.15.167:0/1507390991 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f0f680857b0 con 0x562c8a0c7840 2024-05-11T12:14:56.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.757+0000 7f0f8902e740 10 client.6918 fetch_fsmap learned FSMap version 69 2024-05-11T12:14:56.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.757+0000 7f0f8902e740 1 -- 172.21.15.167:0/1507390991 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) -- 0x562c8a10cf50 con 0x562c8a0c7840 2024-05-11T12:14:56.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.757+0000 7f0f67e00640 1 -- 172.21.15.167:0/1507390991 <== mon.0 v2:172.21.15.167:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f78010b60 con 0x562c8a0c7840 2024-05-11T12:14:56.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.757+0000 7f0f67e00640 1 -- 172.21.15.167:0/1507390991 <== mon.0 v2:172.21.15.167:3300/0 5 ==== fsmap(e 69) ==== 3494+0+0 (secure 0 0 0) 0x7f0f6809ab60 con 0x562c8a0c7840 2024-05-11T12:14:56.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.757+0000 7f0f8902e740 10 client.6918 fetch_fsmap finished waiting for FSMap version 69 2024-05-11T12:14:56.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.757+0000 7f0f8902e740 10 client.6918 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-11T12:14:56.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.757+0000 7f0f8902e740 20 client.6918 populate_metadata read hostname 'smithi167' 2024-05-11T12:14:56.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.757+0000 7f0f8902e740 1 --2- 172.21.15.167:0/1507390991 >> [v2:172.21.15.167:6836/2901182170,v1:172.21.15.167:6837/2901182170] conn(0x562c8a11c2b0 0x562c8a18e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:56.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.757+0000 7f0f8902e740 4 client.6918 mds_command: new command op to 6885 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-11T12:14:56.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.757+0000 7f0f8902e740 1 -- 172.21.15.167:0/1507390991 --> [v2:172.21.15.167:6836/2901182170,v1:172.21.15.167:6837/2901182170] -- command(tid 0: {"prefix": "status"}) -- 0x562c8a033210 con 0x562c8a11c2b0 2024-05-11T12:14:56.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.757+0000 7f0f7f400640 1 --2- 172.21.15.167:0/1507390991 >> [v2:172.21.15.167:6836/2901182170,v1:172.21.15.167:6837/2901182170] conn(0x562c8a11c2b0 0x562c8a18e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:56.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.758+0000 7f0f7f400640 1 --2- 172.21.15.167:0/1507390991 >> [v2:172.21.15.167:6836/2901182170,v1:172.21.15.167:6837/2901182170] conn(0x562c8a11c2b0 0x562c8a18e6d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82fc6ed789b2a2b2 server_cookie=54ac9d664e725095 in_seq=0 out_seq=0 2024-05-11T12:14:56.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.758+0000 7f0f67e00640 10 client.6918 ms_handle_connect on v2:172.21.15.167:6836/2901182170 2024-05-11T12:14:56.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.758+0000 7f0f67e00640 1 -- 172.21.15.167:0/1507390991 <== mds.0 v2:172.21.15.167:6836/2901182170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x562c8a033210 con 0x562c8a11c2b0 2024-05-11T12:14:56.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.758+0000 7f0f67e00640 10 client.6918 handle_command_reply: tid=0 2024-05-11T12:14:56.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.758+0000 7f0f8902e740 1 client.6918 shutdown 2024-05-11T12:14:56.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.759+0000 7f0f8902e740 1 -- 172.21.15.167:0/1507390991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8a0c7840 msgr2=0x562c8a10c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:56.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.759+0000 7f0f8902e740 1 --2- 172.21.15.167:0/1507390991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8a0c7840 0x562c8a10c4f0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f0f6802bfd0 tx=0x7f0f68097000 comp rx=0 tx=0).stop 2024-05-11T12:14:56.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.759+0000 7f0f8902e740 1 -- 172.21.15.167:0/1507390991 shutdown_connections 2024-05-11T12:14:56.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.759+0000 7f0f8902e740 1 --2- 172.21.15.167:0/1507390991 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x562c8a0eb7c0 0x562c8a11bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:56.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.759+0000 7f0f8902e740 1 --2- 172.21.15.167:0/1507390991 >> [v2:172.21.15.167:6836/2901182170,v1:172.21.15.167:6837/2901182170] conn(0x562c8a11c2b0 0x562c8a18e6d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:56.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.759+0000 7f0f8902e740 1 --2- 172.21.15.167:0/1507390991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8a0c7840 0x562c8a10c4f0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:56.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.759+0000 7f0f8902e740 1 --2- 172.21.15.167:0/1507390991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c8a0c6980 0x562c8a18bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:56.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.759+0000 7f0f67e00640 0 client.6918 ms_handle_reset on v2:172.21.15.167:6836/2901182170 2024-05-11T12:14:56.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.759+0000 7f0f8902e740 1 -- 172.21.15.167:0/1507390991 >> 172.21.15.167:0/1507390991 conn(0x562c8a0c26d0 msgr2=0x562c8a0c1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:56.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.759+0000 7f0f8902e740 1 -- 172.21.15.167:0/1507390991 shutdown_connections 2024-05-11T12:14:56.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.759+0000 7f0f8902e740 1 -- 172.21.15.167:0/1507390991 wait complete. 2024-05-11T12:14:56.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.759+0000 7f0f8902e740 20 client.6918 trim_cache size 0 max 16384 2024-05-11T12:14:56.763 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T12:14:56.763 INFO:teuthology.orchestra.run.smithi167.stdout: "cluster_fsid": "5f32d344-7289-4c27-881b-1533419ab678", 2024-05-11T12:14:56.764 INFO:teuthology.orchestra.run.smithi167.stdout: "whoami": 0, 2024-05-11T12:14:56.764 INFO:teuthology.orchestra.run.smithi167.stdout: "id": 6885, 2024-05-11T12:14:56.764 INFO:teuthology.orchestra.run.smithi167.stdout: "want_state": "up:active", 2024-05-11T12:14:56.764 INFO:teuthology.orchestra.run.smithi167.stdout: "state": "up:active", 2024-05-11T12:14:56.764 INFO:teuthology.orchestra.run.smithi167.stdout: "fs_name": "cephfs", 2024-05-11T12:14:56.764 INFO:teuthology.orchestra.run.smithi167.stdout: "rank_uptime": 5.0166640300000003, 2024-05-11T12:14:56.764 INFO:teuthology.orchestra.run.smithi167.stdout: "mdsmap_epoch": 69, 2024-05-11T12:14:56.764 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_epoch": 101, 2024-05-11T12:14:56.764 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_epoch_barrier": 101, 2024-05-11T12:14:56.764 INFO:teuthology.orchestra.run.smithi167.stdout: "uptime": 5.8142197830000004 2024-05-11T12:14:56.764 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T12:14:56.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.760+0000 7f0f86200640 1 -- 172.21.15.167:0/2328665077 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0f5c076470 msgr2=0x7f0f5c078930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:56.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.760+0000 7f0f86200640 1 --2- 172.21.15.167:0/2328665077 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0f5c076470 0x7f0f5c078930 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0f70009910 tx=0x7f0f70008040 comp rx=0 tx=0).stop 2024-05-11T12:14:56.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.760+0000 7f0f86200640 1 -- 172.21.15.167:0/2328665077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f80136e20 msgr2=0x7f0f801335c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:56.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.760+0000 7f0f86200640 1 --2- 172.21.15.167:0/2328665077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f80136e20 0x7f0f801335c0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f0f6802b470 tx=0x7f0f8013f960 comp rx=0 tx=0).stop 2024-05-11T12:14:56.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.760+0000 7f0f86200640 1 -- 172.21.15.167:0/2328665077 shutdown_connections 2024-05-11T12:14:56.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.760+0000 7f0f86200640 1 --2- 172.21.15.167:0/2328665077 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0f5c076470 0x7f0f5c078930 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:56.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.760+0000 7f0f86200640 1 --2- 172.21.15.167:0/2328665077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f801532b0 0x7f0f80133e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:56.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.760+0000 7f0f86200640 1 --2- 172.21.15.167:0/2328665077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f80136e20 0x7f0f801335c0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:56.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.760+0000 7f0f86200640 1 --2- 172.21.15.167:0/2328665077 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f60004020 0x7f0f80138430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:56.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.760+0000 7f0f86200640 1 -- 172.21.15.167:0/2328665077 >> 172.21.15.167:0/2328665077 conn(0x7f0f800b2cb0 msgr2=0x7f0f8015e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:56.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.760+0000 7f0f86200640 1 -- 172.21.15.167:0/2328665077 shutdown_connections 2024-05-11T12:14:56.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.760+0000 7f0f86200640 1 -- 172.21.15.167:0/2328665077 wait complete. 2024-05-11T12:14:56.777 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-11T12:14:56.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.964+0000 7f3d95400640 1 Processor -- start 2024-05-11T12:14:56.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.965+0000 7f3d95400640 1 -- start start 2024-05-11T12:14:56.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.966+0000 7f3d95400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3d900b62c0 0x7f3d900b66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:56.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.966+0000 7f3d95400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d90073a00 0x7f3d90073e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:56.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.966+0000 7f3d95400640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d900b6be0 con 0x7f3d900ce530 2024-05-11T12:14:56.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.966+0000 7f3d95400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f3d900732e0 con 0x7f3d900b62c0 2024-05-11T12:14:56.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.966+0000 7f3d95400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3d900722d0 con 0x7f3d90073a00 2024-05-11T12:14:56.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.967+0000 7f3d8f200640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f3d900ce530 0x7f3d900ce9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:43254/0 (socket says 172.21.15.167:43254) 2024-05-11T12:14:56.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.967+0000 7f3d8f200640 1 -- 172.21.15.167:0/2046589546 learned_addr learned my addr 172.21.15.167:0/2046589546 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:14:56.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.967+0000 7f3d8e800640 1 --2- 172.21.15.167:0/2046589546 >> v2:172.21.15.204:3300/0 conn(0x7f3d900b62c0 0x7f3d900b66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:56.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.967+0000 7f3d8de00640 1 --2- 172.21.15.167:0/2046589546 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d90073a00 0x7f3d90073e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:56.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.967+0000 7f3d8d400640 1 -- 172.21.15.167:0/2046589546 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2903213185 0 0) 0x7f3d900b6be0 con 0x7f3d900ce530 2024-05-11T12:14:56.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.967+0000 7f3d8d400640 1 -- 172.21.15.167:0/2046589546 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d74003520 con 0x7f3d900ce530 2024-05-11T12:14:56.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.967+0000 7f3d8d400640 1 -- 172.21.15.167:0/2046589546 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3851355130 0 0) 0x7f3d74003520 con 0x7f3d900ce530 2024-05-11T12:14:56.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.967+0000 7f3d8d400640 1 -- 172.21.15.167:0/2046589546 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d90073a00 msgr2=0x7f3d90073e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:56.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.967+0000 7f3d8d400640 1 --2- 172.21.15.167:0/2046589546 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d90073a00 0x7f3d90073e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:56.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.968+0000 7f3d8d400640 1 -- 172.21.15.167:0/2046589546 >> v2:172.21.15.204:3300/0 conn(0x7f3d900b62c0 msgr2=0x7f3d900b66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:56.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.968+0000 7f3d8d400640 1 --2- 172.21.15.167:0/2046589546 >> v2:172.21.15.204:3300/0 conn(0x7f3d900b62c0 0x7f3d900b66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:56.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.968+0000 7f3d8d400640 1 -- 172.21.15.167:0/2046589546 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d90072520 con 0x7f3d900ce530 2024-05-11T12:14:56.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.968+0000 7f3d8d400640 1 -- 172.21.15.167:0/2046589546 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f3d88001a70 con 0x7f3d900ce530 2024-05-11T12:14:56.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.968+0000 7f3d8d400640 1 -- 172.21.15.167:0/2046589546 >> v1:172.21.15.167:6789/0 conn(0x7f3d900ce530 legacy=0x7f3d900ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:56.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.968+0000 7f3d8d400640 1 --2- 172.21.15.167:0/2046589546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d74003a50 0x7f3d74003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:56.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.968+0000 7f3d8d400640 1 --2- 172.21.15.167:0/2046589546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d74004800 0x7f3d74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:56.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.968+0000 7f3d8d400640 1 --2- 172.21.15.167:0/2046589546 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d90073a00 0x7f3d740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:56.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.968+0000 7f3d8d400640 1 -- 172.21.15.167:0/2046589546 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3d900732e0 con 0x7f3d74004800 2024-05-11T12:14:56.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.968+0000 7f3d8d400640 1 -- 172.21.15.167:0/2046589546 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3d900722d0 con 0x7f3d74003a50 2024-05-11T12:14:56.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.968+0000 7f3d8d400640 1 -- 172.21.15.167:0/2046589546 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3d7400ba10 con 0x7f3d90073a00 2024-05-11T12:14:56.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.968+0000 7f3d8de00640 1 --2- 172.21.15.167:0/2046589546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d74004800 0x7f3d74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:56.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.968+0000 7f3d8f200640 1 --2- 172.21.15.167:0/2046589546 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d90073a00 0x7f3d740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:56.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.968+0000 7f3d8e800640 1 --2- 172.21.15.167:0/2046589546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d74003a50 0x7f3d74003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:56.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.968+0000 7f3d8de00640 1 -- 172.21.15.167:0/2046589546 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d90073a00 msgr2=0x7f3d740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:56.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.968+0000 7f3d8de00640 1 --2- 172.21.15.167:0/2046589546 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d90073a00 0x7f3d740051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:56.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.968+0000 7f3d8de00640 1 -- 172.21.15.167:0/2046589546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d74003a50 msgr2=0x7f3d74003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:56.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.968+0000 7f3d8de00640 1 --2- 172.21.15.167:0/2046589546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d74003a50 0x7f3d74003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:56.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.968+0000 7f3d8de00640 1 -- 172.21.15.167:0/2046589546 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d74003520 con 0x7f3d74004800 2024-05-11T12:14:56.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.968+0000 7f3d8f200640 1 --2- 172.21.15.167:0/2046589546 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d90073a00 0x7f3d740051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:56.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.968+0000 7f3d8e800640 1 --2- 172.21.15.167:0/2046589546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d74003a50 0x7f3d74003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:56.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.969+0000 7f3d8de00640 1 --2- 172.21.15.167:0/2046589546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d74004800 0x7f3d74004c80 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c009af0 tx=0x7f3d74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34efd00d84c9c600 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:56.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.970+0000 7f3d8d400640 1 -- 172.21.15.167:0/2046589546 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c036070 con 0x7f3d74004800 2024-05-11T12:14:56.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.970+0000 7f3d8d400640 1 -- 172.21.15.167:0/2046589546 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d7c031400 con 0x7f3d74004800 2024-05-11T12:14:56.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.970+0000 7f3d8d400640 1 -- 172.21.15.167:0/2046589546 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c009690 con 0x7f3d74004800 2024-05-11T12:14:56.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.970+0000 7f3d95400640 1 -- 172.21.15.167:0/2046589546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d74004800 msgr2=0x7f3d74004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:56.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.970+0000 7f3d95400640 1 --2- 172.21.15.167:0/2046589546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d74004800 0x7f3d74004c80 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c009af0 tx=0x7f3d74005830 comp rx=0 tx=0).stop 2024-05-11T12:14:56.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.970+0000 7f3d8f200640 1 -- 172.21.15.167:0/2046589546 reap_dead start 2024-05-11T12:14:56.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.970+0000 7f3d95400640 1 -- 172.21.15.167:0/2046589546 shutdown_connections 2024-05-11T12:14:56.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.970+0000 7f3d95400640 1 -- 172.21.15.167:0/2046589546 >> 172.21.15.167:0/2046589546 conn(0x7f3d900b2cb0 msgr2=0x7f3d900b3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:56.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.971+0000 7f3d95400640 1 -- 172.21.15.167:0/2046589546 shutdown_connections 2024-05-11T12:14:56.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.971+0000 7f3d95400640 1 -- 172.21.15.167:0/2046589546 wait complete. 2024-05-11T12:14:56.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.972+0000 7f3d95400640 1 Processor -- start 2024-05-11T12:14:56.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.972+0000 7f3d95400640 1 -- start start 2024-05-11T12:14:56.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.973+0000 7f3d95400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d74004800 0x7f3d9013da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:56.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.973+0000 7f3d95400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d90150090 0x7f3d9013d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:56.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.973+0000 7f3d95400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d90155c80 0x7f3d90156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:56.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.973+0000 7f3d95400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3d90072dc0 con 0x7f3d90155c80 2024-05-11T12:14:56.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.973+0000 7f3d95400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3d90072ae0 con 0x7f3d74004800 2024-05-11T12:14:56.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.973+0000 7f3d95400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3d9015f890 con 0x7f3d90150090 2024-05-11T12:14:56.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.974+0000 7f3d8f200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d90155c80 0x7f3d90156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:56.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.974+0000 7f3d8f200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d90155c80 0x7f3d90156100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:52478/0 (socket says 172.21.15.167:52478) 2024-05-11T12:14:56.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.974+0000 7f3d8f200640 1 -- 172.21.15.167:0/1659328070 learned_addr learned my addr 172.21.15.167:0/1659328070 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:56.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.974+0000 7f3d8de00640 1 --2- 172.21.15.167:0/1659328070 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d90150090 0x7f3d9013d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:56.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.974+0000 7f3d8e800640 1 --2- 172.21.15.167:0/1659328070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d74004800 0x7f3d9013da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:56.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.974+0000 7f3d8f200640 1 -- 172.21.15.167:0/1659328070 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d90150090 msgr2=0x7f3d9013d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:56.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.974+0000 7f3d8f200640 1 --2- 172.21.15.167:0/1659328070 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d90150090 0x7f3d9013d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:56.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.974+0000 7f3d8f200640 1 -- 172.21.15.167:0/1659328070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d74004800 msgr2=0x7f3d9013da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:56.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.974+0000 7f3d8f200640 1 --2- 172.21.15.167:0/1659328070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d74004800 0x7f3d9013da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:56.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.974+0000 7f3d8f200640 1 -- 172.21.15.167:0/1659328070 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d90154ec0 con 0x7f3d90155c80 2024-05-11T12:14:56.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.974+0000 7f3d8de00640 1 --2- 172.21.15.167:0/1659328070 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d90150090 0x7f3d9013d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:56.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.974+0000 7f3d8e800640 1 --2- 172.21.15.167:0/1659328070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d74004800 0x7f3d9013da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:56.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.974+0000 7f3d8f200640 1 --2- 172.21.15.167:0/1659328070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d90155c80 0x7f3d90156100 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f3d900ce910 tx=0x7f3d8800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:56.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.975+0000 7f3d86a00640 1 -- 172.21.15.167:0/1659328070 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d88004d60 con 0x7f3d90155c80 2024-05-11T12:14:56.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.975+0000 7f3d86a00640 1 -- 172.21.15.167:0/1659328070 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d8800bb80 con 0x7f3d90155c80 2024-05-11T12:14:56.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.975+0000 7f3d86a00640 1 -- 172.21.15.167:0/1659328070 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d88007070 con 0x7f3d90155c80 2024-05-11T12:14:56.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.975+0000 7f3d95400640 1 -- 172.21.15.167:0/1659328070 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d901551a0 con 0x7f3d90155c80 2024-05-11T12:14:56.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.975+0000 7f3d95400640 1 -- 172.21.15.167:0/1659328070 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d90155760 con 0x7f3d90155c80 2024-05-11T12:14:56.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.976+0000 7f3d95400640 1 -- 172.21.15.167:0/1659328070 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d90155960 con 0x7f3d90155c80 2024-05-11T12:14:56.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.977+0000 7f3d86a00640 1 -- 172.21.15.167:0/1659328070 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3d8801b020 con 0x7f3d90155c80 2024-05-11T12:14:56.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.978+0000 7f3d86a00640 1 --2- 172.21.15.167:0/1659328070 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3d70076300 0x7f3d700787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:56.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.978+0000 7f3d86a00640 1 -- 172.21.15.167:0/1659328070 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6187+0+0 (secure 0 0 0) 0x7f3d88097980 con 0x7f3d90155c80 2024-05-11T12:14:56.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.978+0000 7f3d8e800640 1 --2- 172.21.15.167:0/1659328070 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3d70076300 0x7f3d700787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:56.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.979+0000 7f3d8e800640 1 --2- 172.21.15.167:0/1659328070 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3d70076300 0x7f3d700787c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3d80009b20 tx=0x7f3d80008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:56.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:56.983+0000 7f3d86a00640 1 -- 172.21.15.167:0/1659328070 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3d90155960 con 0x7f3d90155c80 2024-05-11T12:14:57.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:57.230+0000 7f3d95400640 1 -- 172.21.15.167:0/1659328070 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3d90069820 con 0x7f3d90155c80 2024-05-11T12:14:57.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:57.231+0000 7f3d86a00640 1 -- 172.21.15.167:0/1659328070 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v69) ==== 53+0+73 (secure 0 0 0) 0x7f3d88061060 con 0x7f3d90155c80 2024-05-11T12:14:57.231 INFO:teuthology.orchestra.run.smithi167.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-11T12:14:57.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:57.233+0000 7f3d95400640 1 -- 172.21.15.167:0/1659328070 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3d70076300 msgr2=0x7f3d700787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:57.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:57.233+0000 7f3d95400640 1 --2- 172.21.15.167:0/1659328070 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3d70076300 0x7f3d700787c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3d80009b20 tx=0x7f3d80008040 comp rx=0 tx=0).stop 2024-05-11T12:14:57.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:57.233+0000 7f3d95400640 1 -- 172.21.15.167:0/1659328070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d90155c80 msgr2=0x7f3d90156100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:57.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:57.233+0000 7f3d95400640 1 --2- 172.21.15.167:0/1659328070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d90155c80 0x7f3d90156100 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f3d900ce910 tx=0x7f3d8800b4a0 comp rx=0 tx=0).stop 2024-05-11T12:14:57.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:57.233+0000 7f3d95400640 1 -- 172.21.15.167:0/1659328070 shutdown_connections 2024-05-11T12:14:57.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:57.233+0000 7f3d95400640 1 --2- 172.21.15.167:0/1659328070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d90155c80 0x7f3d90156100 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:57.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:57.233+0000 7f3d95400640 1 --2- 172.21.15.167:0/1659328070 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d90150090 0x7f3d9013d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:57.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:57.233+0000 7f3d95400640 1 --2- 172.21.15.167:0/1659328070 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3d70076300 0x7f3d700787c0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:57.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:57.233+0000 7f3d95400640 1 --2- 172.21.15.167:0/1659328070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d74004800 0x7f3d9013da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:57.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:57.233+0000 7f3d95400640 1 -- 172.21.15.167:0/1659328070 >> 172.21.15.167:0/1659328070 conn(0x7f3d900b2cb0 msgr2=0x7f3d9015eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:57.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:57.234+0000 7f3d95400640 1 -- 172.21.15.167:0/1659328070 shutdown_connections 2024-05-11T12:14:57.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:57.234+0000 7f3d95400640 1 -- 172.21.15.167:0/1659328070 wait complete. 2024-05-11T12:14:57.253 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-11T12:14:57.253 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-11T12:14:57.253 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi167.front.sepia.ceph.com 2024-05-11T12:14:57.253 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-11T12:14:57.253 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-11T12:14:57.254 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-11T12:14:57.254 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-11T12:14:57.254 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-11T12:14:57.254 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-11T12:14:57.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:57.254 DEBUG:teuthology.orchestra.run.smithi167:> ip addr 2024-05-11T12:14:57.301 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-11T12:14:57.301 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-11T12:14:57.301 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2024-05-11T12:14:57.301 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:14:57.301 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2024-05-11T12:14:57.302 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:14:57.302 INFO:teuthology.orchestra.run.smithi167.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:14:57.302 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e6 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:14:57.302 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f0 2024-05-11T12:14:57.302 INFO:teuthology.orchestra.run.smithi167.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:14:57.302 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e7 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:14:57.302 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f1 2024-05-11T12:14:57.302 INFO:teuthology.orchestra.run.smithi167.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:14:57.302 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:08 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:14:57.302 INFO:teuthology.orchestra.run.smithi167.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-11T12:14:57.302 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:09 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:14:57.302 INFO:teuthology.orchestra.run.smithi167.stdout: inet 172.21.15.167/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-11T12:14:57.302 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft 42372sec preferred_lft 42372sec 2024-05-11T12:14:57.302 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::ec4:7aff:fe88:7309/64 scope link 2024-05-11T12:14:57.302 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:14:57.303 INFO:teuthology.orchestra.run.smithi167.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-11T12:14:57.303 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 1a:a8:8d:b3:f4:32 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:14:57.303 INFO:teuthology.orchestra.run.smithi167.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-11T12:14:57.303 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:14:57.303 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::b017:48ff:fea0:7183/64 scope link 2024-05-11T12:14:57.303 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:14:57.303 INFO:teuthology.orchestra.run.smithi167.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-11T12:14:57.303 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 1a:a8:8d:b3:f4:32 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-11T12:14:57.303 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::18a8:8dff:feb3:f432/64 scope link 2024-05-11T12:14:57.303 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:14:57.303 INFO:teuthology.orchestra.run.smithi167.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-11T12:14:57.303 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 7e:79:1f:cb:d2:83 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-11T12:14:57.303 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::7c79:1fff:fecb:d283/64 scope link 2024-05-11T12:14:57.303 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:14:57.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:57.304 DEBUG:teuthology.orchestra.run.smithi167:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-11T12:14:57.372 INFO:teuthology.orchestra.run.smithi167.stdout:1 2024-05-11T12:14:57.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:57.375 DEBUG:teuthology.orchestra.run.smithi167:> route 2024-05-11T12:14:57.425 INFO:teuthology.orchestra.run.smithi167.stdout:Kernel IP routing table 2024-05-11T12:14:57.425 INFO:teuthology.orchestra.run.smithi167.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-11T12:14:57.425 INFO:teuthology.orchestra.run.smithi167.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-11T12:14:57.426 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-11T12:14:57.426 INFO:teuthology.orchestra.run.smithi167.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-11T12:14:57.426 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T12:14:57.426 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-05-11T12:14:57.426 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-11T12:14:57.426 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-11T12:14:57.426 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-11T12:14:57.426 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-05-11T12:14:57.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T12:14:57.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:14:57.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:57.637 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2024-05-11T12:14:57.686 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-11T12:14:57.686 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-11T12:14:57.687 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-11T12:14:57.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:57.687 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set brx.0 up 2024-05-11T12:14:57.769 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T12:14:57.770 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-11T12:14:57.821 INFO:teuthology.orchestra.run.smithi167.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-11T12:14:57.821 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T12:14:57.821 DEBUG:teuthology.orchestra.run.smithi167:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-11T12:14:57.867 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-05-11T12:14:57.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:57.868 DEBUG:teuthology.orchestra.run.smithi167:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:14:57.992 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:14:57.993 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-11T12:14:58.028 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:14:58.028 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-11T12:14:58.028 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.167:3300,172.21.15.204:3300,172.21.15.204:3301" 2024-05-11T12:14:58.028 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: trying mount with new device syntax: 0@5f32d344-7289-4c27-881b-1533419ab678.cephfs=/ 2024-05-11T12:14:58.028 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.167:3300/172.21.15.204:3300/172.21.15.204:3301" will pass to kernel 2024-05-11T12:14:58.030 INFO:teuthology.orchestra.run.smithi167.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-11T12:14:58.030 INFO:teuthology.orchestra.run.smithi167.stdout: You just mounted a file system that supports labels which does not 2024-05-11T12:14:58.030 INFO:teuthology.orchestra.run.smithi167.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-11T12:14:58.030 INFO:teuthology.orchestra.run.smithi167.stdout: applications will generate AVC messages and not be allowed access to 2024-05-11T12:14:58.030 INFO:teuthology.orchestra.run.smithi167.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-11T12:14:58.036 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-11T12:14:58.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:58.037 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-11T12:14:58.110 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-11T12:14:58.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-11T12:14:58.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:14:58.227 INFO:teuthology.orchestra.run.smithi167.stdout:client5129 2024-05-11T12:14:58.227 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:14:58.227 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:14:58.279 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-11T12:14:58.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.489+0000 7f3514c00640 1 Processor -- start 2024-05-11T12:14:58.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.490+0000 7f3514c00640 1 -- start start 2024-05-11T12:14:58.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.491+0000 7f3514c00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f351015f150 0x7f35100d0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:58.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.491+0000 7f3514c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35100d5c50 0x7f35100d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:58.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.491+0000 7f3514c00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35100cfb00 con 0x7f351015f530 2024-05-11T12:14:58.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.491+0000 7f3514c00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f35100d3d50 con 0x7f351015f150 2024-05-11T12:14:58.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.491+0000 7f3514c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f35100cfda0 con 0x7f35100d5c50 2024-05-11T12:14:58.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.491+0000 7f350f400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f351015f150 0x7f35100d0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:58.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.491+0000 7f350f400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f351015f150 0x7f35100d0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:52486/0 (socket says 172.21.15.167:52486) 2024-05-11T12:14:58.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.491+0000 7f350f400640 1 -- 172.21.15.167:0/3766770459 learned_addr learned my addr 172.21.15.167:0/3766770459 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:58.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.492+0000 7f350f400640 1 -- 172.21.15.167:0/3766770459 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35100d5c50 msgr2=0x7f35100d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:14:58.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.492+0000 7f350fe00640 1 --2- 172.21.15.167:0/3766770459 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35100d5c50 0x7f35100d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:58.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.492+0000 7f350f400640 1 --2- 172.21.15.167:0/3766770459 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35100d5c50 0x7f35100d1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:58.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.492+0000 7f350f400640 1 -- 172.21.15.167:0/3766770459 >> v1:172.21.15.167:6789/0 conn(0x7f351015f530 legacy=0x7f35100d3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:58.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.492+0000 7f350f400640 1 -- 172.21.15.167:0/3766770459 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35100d0080 con 0x7f351015f150 2024-05-11T12:14:58.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.493+0000 7f350f400640 1 --2- 172.21.15.167:0/3766770459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f351015f150 0x7f35100d0b60 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f3500009870 tx=0x7f350002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2346037b06059a09 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:58.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.493+0000 7f350ea00640 1 -- 172.21.15.167:0/3766770459 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3500037070 con 0x7f351015f150 2024-05-11T12:14:58.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.493+0000 7f350ea00640 1 -- 172.21.15.167:0/3766770459 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3500032440 con 0x7f351015f150 2024-05-11T12:14:58.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.493+0000 7f350ea00640 1 -- 172.21.15.167:0/3766770459 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3500032810 con 0x7f351015f150 2024-05-11T12:14:58.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.494+0000 7f3514c00640 1 -- 172.21.15.167:0/3766770459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f351015f150 msgr2=0x7f35100d0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:58.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.494+0000 7f3514c00640 1 --2- 172.21.15.167:0/3766770459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f351015f150 0x7f35100d0b60 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f3500009870 tx=0x7f350002a1c0 comp rx=0 tx=0).stop 2024-05-11T12:14:58.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.494+0000 7f3514c00640 1 -- 172.21.15.167:0/3766770459 shutdown_connections 2024-05-11T12:14:58.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.494+0000 7f3514c00640 1 --2- 172.21.15.167:0/3766770459 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35100d5c50 0x7f35100d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:58.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.494+0000 7f3514c00640 1 --2- 172.21.15.167:0/3766770459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f351015f150 0x7f35100d0b60 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:58.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.494+0000 7f3514c00640 1 -- 172.21.15.167:0/3766770459 >> 172.21.15.167:0/3766770459 conn(0x7f351009e3a0 msgr2=0x7f35100d5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:58.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.494+0000 7f3514c00640 1 -- 172.21.15.167:0/3766770459 shutdown_connections 2024-05-11T12:14:58.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.494+0000 7f3514c00640 1 -- 172.21.15.167:0/3766770459 wait complete. 2024-05-11T12:14:58.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.496+0000 7f3514c00640 1 Processor -- start 2024-05-11T12:14:58.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.496+0000 7f3514c00640 1 -- start start 2024-05-11T12:14:58.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.497+0000 7f3514c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35100d5c50 0x7f3510156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:58.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.497+0000 7f3514c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f351015f150 0x7f351006dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:58.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.497+0000 7f3514c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f351015f530 0x7f351006e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:58.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.497+0000 7f3514c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f35100d1f90 con 0x7f351015f530 2024-05-11T12:14:58.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.497+0000 7f3514c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f35100d0640 con 0x7f35100d5c50 2024-05-11T12:14:58.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.497+0000 7f3514c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f35100d2110 con 0x7f351015f150 2024-05-11T12:14:58.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.497+0000 7f350fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f351015f530 0x7f351006e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:58.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.497+0000 7f350fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f351015f530 0x7f351006e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:52494/0 (socket says 172.21.15.167:52494) 2024-05-11T12:14:58.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.497+0000 7f350fe00640 1 -- 172.21.15.167:0/2186548399 learned_addr learned my addr 172.21.15.167:0/2186548399 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:14:58.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.498+0000 7f350f400640 1 --2- 172.21.15.167:0/2186548399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35100d5c50 0x7f3510156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:58.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.498+0000 7f3506a00640 1 --2- 172.21.15.167:0/2186548399 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f351015f150 0x7f351006dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:58.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.498+0000 7f350fe00640 1 -- 172.21.15.167:0/2186548399 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f351015f150 msgr2=0x7f351006dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:58.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.498+0000 7f350fe00640 1 --2- 172.21.15.167:0/2186548399 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f351015f150 0x7f351006dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:58.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.498+0000 7f350fe00640 1 -- 172.21.15.167:0/2186548399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35100d5c50 msgr2=0x7f3510156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:58.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.498+0000 7f350fe00640 1 --2- 172.21.15.167:0/2186548399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35100d5c50 0x7f3510156b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:58.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.498+0000 7f350fe00640 1 -- 172.21.15.167:0/2186548399 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35100a29c0 con 0x7f351015f530 2024-05-11T12:14:58.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.498+0000 7f3506a00640 1 --2- 172.21.15.167:0/2186548399 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f351015f150 0x7f351006dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:14:58.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.498+0000 7f350fe00640 1 --2- 172.21.15.167:0/2186548399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f351015f530 0x7f351006e270 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f350800cc30 tx=0x7f350800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:58.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.498+0000 7f3507e00640 1 -- 172.21.15.167:0/2186548399 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350800ce40 con 0x7f351015f530 2024-05-11T12:14:58.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.498+0000 7f3507e00640 1 -- 172.21.15.167:0/2186548399 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3508003990 con 0x7f351015f530 2024-05-11T12:14:58.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.498+0000 7f3507e00640 1 -- 172.21.15.167:0/2186548399 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3508003d40 con 0x7f351015f530 2024-05-11T12:14:58.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.498+0000 7f3514c00640 1 -- 172.21.15.167:0/2186548399 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35100a2ca0 con 0x7f351015f530 2024-05-11T12:14:58.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.498+0000 7f3514c00640 1 -- 172.21.15.167:0/2186548399 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35100b4520 con 0x7f351015f530 2024-05-11T12:14:58.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.500+0000 7f3514c00640 1 -- 172.21.15.167:0/2186548399 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35100b4720 con 0x7f351015f530 2024-05-11T12:14:58.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.501+0000 7f3507e00640 1 -- 172.21.15.167:0/2186548399 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3508003ee0 con 0x7f351015f530 2024-05-11T12:14:58.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.502+0000 7f3507e00640 1 --2- 172.21.15.167:0/2186548399 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f34ec0762d0 0x7f34ec078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:58.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.502+0000 7f3507e00640 1 -- 172.21.15.167:0/2186548399 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6187+0+0 (secure 0 0 0) 0x7f3508097c90 con 0x7f351015f530 2024-05-11T12:14:58.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.502+0000 7f350f400640 1 --2- 172.21.15.167:0/2186548399 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f34ec0762d0 0x7f34ec078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:58.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.503+0000 7f350f400640 1 --2- 172.21.15.167:0/2186548399 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f34ec0762d0 0x7f34ec078790 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3500034000 tx=0x7f350002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:58.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.506+0000 7f3507e00640 1 -- 172.21.15.167:0/2186548399 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f35100b4720 con 0x7f351015f530 2024-05-11T12:14:58.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.754+0000 7f3514c00640 1 -- 172.21.15.167:0/2186548399 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f35100d29f0 con 0x7f351015f530 2024-05-11T12:14:58.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.754+0000 7f3507e00640 1 -- 172.21.15.167:0/2186548399 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v69) ==== 53+0+73 (secure 0 0 0) 0x7f3508014030 con 0x7f351015f530 2024-05-11T12:14:58.755 INFO:teuthology.orchestra.run.smithi167.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-11T12:14:58.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.757+0000 7f3514c00640 1 -- 172.21.15.167:0/2186548399 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f34ec0762d0 msgr2=0x7f34ec078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:58.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.757+0000 7f3514c00640 1 --2- 172.21.15.167:0/2186548399 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f34ec0762d0 0x7f34ec078790 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3500034000 tx=0x7f350002e000 comp rx=0 tx=0).stop 2024-05-11T12:14:58.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.757+0000 7f3514c00640 1 -- 172.21.15.167:0/2186548399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f351015f530 msgr2=0x7f351006e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:58.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.757+0000 7f3514c00640 1 --2- 172.21.15.167:0/2186548399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f351015f530 0x7f351006e270 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f350800cc30 tx=0x7f350800a5c0 comp rx=0 tx=0).stop 2024-05-11T12:14:58.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.757+0000 7f3514c00640 1 -- 172.21.15.167:0/2186548399 shutdown_connections 2024-05-11T12:14:58.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.757+0000 7f3514c00640 1 --2- 172.21.15.167:0/2186548399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f351015f530 0x7f351006e270 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:58.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.757+0000 7f3514c00640 1 --2- 172.21.15.167:0/2186548399 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f351015f150 0x7f351006dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:58.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.757+0000 7f3514c00640 1 --2- 172.21.15.167:0/2186548399 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f34ec0762d0 0x7f34ec078790 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:58.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.757+0000 7f3514c00640 1 --2- 172.21.15.167:0/2186548399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35100d5c50 0x7f3510156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:58.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.757+0000 7f3514c00640 1 -- 172.21.15.167:0/2186548399 >> 172.21.15.167:0/2186548399 conn(0x7f351009e3a0 msgr2=0x7f35100b92e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:58.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.757+0000 7f3514c00640 1 -- 172.21.15.167:0/2186548399 shutdown_connections 2024-05-11T12:14:58.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:14:58.757+0000 7f3514c00640 1 -- 172.21.15.167:0/2186548399 wait complete. 2024-05-11T12:14:58.776 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-11T12:14:58.777 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-11T12:14:58.777 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi204.front.sepia.ceph.com 2024-05-11T12:14:58.777 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-11T12:14:58.777 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-11T12:14:58.777 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-11T12:14:58.777 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-11T12:14:58.777 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-11T12:14:58.777 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-11T12:14:58.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:58.777 DEBUG:teuthology.orchestra.run.smithi204:> ip addr 2024-05-11T12:14:58.837 INFO:teuthology.orchestra.run.smithi204.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-11T12:14:58.837 INFO:teuthology.orchestra.run.smithi204.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-11T12:14:58.837 INFO:teuthology.orchestra.run.smithi204.stdout: inet 127.0.0.1/8 scope host lo 2024-05-11T12:14:58.837 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:14:58.837 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 ::1/128 scope host 2024-05-11T12:14:58.837 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:14:58.838 INFO:teuthology.orchestra.run.smithi204.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:14:58.838 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2a brd ff:ff:ff:ff:ff:ff 2024-05-11T12:14:58.838 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f0 2024-05-11T12:14:58.838 INFO:teuthology.orchestra.run.smithi204.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:14:58.838 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2b brd ff:ff:ff:ff:ff:ff 2024-05-11T12:14:58.838 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f1 2024-05-11T12:14:58.838 INFO:teuthology.orchestra.run.smithi204.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:14:58.838 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:aa brd ff:ff:ff:ff:ff:ff 2024-05-11T12:14:58.838 INFO:teuthology.orchestra.run.smithi204.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-11T12:14:58.838 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:ab brd ff:ff:ff:ff:ff:ff 2024-05-11T12:14:58.838 INFO:teuthology.orchestra.run.smithi204.stdout: inet 172.21.15.204/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-11T12:14:58.838 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft 42366sec preferred_lft 42366sec 2024-05-11T12:14:58.838 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::ec4:7aff:fe88:77ab/64 scope link 2024-05-11T12:14:58.839 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:14:58.839 INFO:teuthology.orchestra.run.smithi204.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-11T12:14:58.839 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether e2:9e:5b:47:45:5d brd ff:ff:ff:ff:ff:ff 2024-05-11T12:14:58.839 INFO:teuthology.orchestra.run.smithi204.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-11T12:14:58.839 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:14:58.839 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::a498:7aff:fe50:48c3/64 scope link 2024-05-11T12:14:58.839 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:14:58.839 INFO:teuthology.orchestra.run.smithi204.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-11T12:14:58.839 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether fa:0a:7f:e4:30:35 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-11T12:14:58.839 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::f80a:7fff:fee4:3035/64 scope link 2024-05-11T12:14:58.839 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:14:58.839 INFO:teuthology.orchestra.run.smithi204.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-11T12:14:58.839 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether e2:9e:5b:47:45:5d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-11T12:14:58.840 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::e09e:5bff:fe47:455d/64 scope link 2024-05-11T12:14:58.840 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:14:58.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:58.840 DEBUG:teuthology.orchestra.run.smithi204:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-11T12:14:58.916 INFO:teuthology.orchestra.run.smithi204.stdout:1 2024-05-11T12:14:58.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:58.921 DEBUG:teuthology.orchestra.run.smithi204:> route 2024-05-11T12:14:58.975 INFO:teuthology.orchestra.run.smithi204.stdout:Kernel IP routing table 2024-05-11T12:14:58.975 INFO:teuthology.orchestra.run.smithi204.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-11T12:14:58.975 INFO:teuthology.orchestra.run.smithi204.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-11T12:14:58.975 INFO:teuthology.orchestra.run.smithi204.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-11T12:14:58.975 INFO:teuthology.orchestra.run.smithi204.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-11T12:14:58.976 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T12:14:58.976 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-05-11T12:14:58.977 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-11T12:14:58.977 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-11T12:14:58.977 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-11T12:14:58.977 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-05-11T12:14:59.080 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:14:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T12:14:59.174 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:14:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:14:59.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:59.184 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list 2024-05-11T12:14:59.230 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-11T12:14:59.230 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-11T12:14:59.231 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-11T12:14:59.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:59.231 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link set brx.0 up 2024-05-11T12:14:59.359 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T12:14:59.360 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-11T12:14:59.413 INFO:teuthology.orchestra.run.smithi204.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-11T12:14:59.413 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T12:14:59.413 DEBUG:teuthology.orchestra.run.smithi204:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-11T12:14:59.463 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-05-11T12:14:59.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:59.464 DEBUG:teuthology.orchestra.run.smithi204:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:14:59.591 INFO:teuthology.orchestra.run.smithi204.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:14:59.591 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-11T12:14:59.625 INFO:teuthology.orchestra.run.smithi204.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-11T12:14:59.625 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-11T12:14:59.625 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: resolved to: "172.21.15.167:3300,172.21.15.204:3300,172.21.15.204:3301" 2024-05-11T12:14:59.625 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: trying mount with new device syntax: 1@5f32d344-7289-4c27-881b-1533419ab678.cephfs=/ 2024-05-11T12:14:59.625 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.167:3300/172.21.15.204:3300/172.21.15.204:3301" will pass to kernel 2024-05-11T12:14:59.626 INFO:teuthology.orchestra.run.smithi204.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-05-11T12:14:59.627 INFO:teuthology.orchestra.run.smithi204.stdout: You just mounted a file system that supports labels which does not 2024-05-11T12:14:59.627 INFO:teuthology.orchestra.run.smithi204.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-11T12:14:59.627 INFO:teuthology.orchestra.run.smithi204.stdout: applications will generate AVC messages and not be allowed access to 2024-05-11T12:14:59.627 INFO:teuthology.orchestra.run.smithi204.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-11T12:14:59.632 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-11T12:14:59.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:14:59.632 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-11T12:14:59.701 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-11T12:14:59.796 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:14:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-11T12:14:59.802 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T12:14:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:14:59.811 INFO:teuthology.orchestra.run.smithi204.stdout:client5425 2024-05-11T12:14:59.811 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:14:59.811 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:14:59.864 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-05-11T12:15:00.093 INFO:teuthology.orchestra.run.smithi167.stdout:[{"id":5425,"entity":{"name":{"type":"client","num":5425},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":1530078856}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":3,"uptime":0.45999209899999999,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.99894929720776904,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5425 192.168.0.1:0/1530078856","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi204","kernel_version":"6.9.0-rc3-gfb9a6a44107e","root":"/"}},{"id":5129,"entity":{"name":{"type":"client","num":5129},"addr":{"type":"any","addr":"172.21.15.167:0","nonce":3784505284}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":3,"uptime":2.0559646869999999,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.99527017573097631,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5129 172.21.15.167:0/3784505284","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi167","kernel_version":"6.9.0-rc3-gfb9a6a44107e","root":"/"}}] 2024-05-11T12:15:00.094 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "1", "hostname": "smithi204", "kernel_version": "6.9.0-rc3-gfb9a6a44107e", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1530078856, "type": "any" }, "name": { "num": 5425, "type": "client" } }, "id": 5425, "inst": "client.5425 192.168.0.1:0/1530078856", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 3, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.998949297207769 }, "state": "open", "uptime": 0.459992099 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "0", "hostname": "smithi167", "kernel_version": "6.9.0-rc3-gfb9a6a44107e", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.167:0", "nonce": 3784505284, "type": "any" }, "name": { "num": 5129, "type": "client" } }, "id": 5129, "inst": "client.5129 172.21.15.167:0/3784505284", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 3, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.9952701757309763 }, "state": "open", "uptime": 2.055964687 } ] 2024-05-11T12:15:00.095 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-05-11T12:15:00.342 INFO:teuthology.orchestra.run.smithi167.stdout:{"message":"","return_code":0} 2024-05-11T12:15:00.343 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-11T12:15:00.343 DEBUG:teuthology.orchestra.run.smithi167:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 0 show session 2024-05-11T12:15:00.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.437+0000 7f22ef11b280 1 -- start start 2024-05-11T12:15:00.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.438+0000 7f22ef11b280 1 Processor -- start 2024-05-11T12:15:00.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.438+0000 7f22ef11b280 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5562e1ce6c00 0x5562e2ac0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:00.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.438+0000 7f22ef11b280 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562e1ce7800 0x5562e2ac0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:00.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.438+0000 7f22ef11b280 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562e1ce7c00 0x5562e2ac0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:00.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.438+0000 7f22ef11b280 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5562e2acc600 con 0x5562e1ce7c00 2024-05-11T12:15:00.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.438+0000 7f22ef11b280 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5562e2acc480 con 0x5562e1ce7800 2024-05-11T12:15:00.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.438+0000 7f22ef11b280 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x5562e2acc300 con 0x5562e1ce6c00 2024-05-11T12:15:00.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.439+0000 7f22ed000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562e1ce7c00 0x5562e2ac0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:00.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.439+0000 7f22ed000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562e1ce7c00 0x5562e2ac0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:38912/0 (socket says 172.21.15.167:38912) 2024-05-11T12:15:00.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.439+0000 7f22ed000640 1 -- 172.21.15.167:0/1335132001 learned_addr learned my addr 172.21.15.167:0/1335132001 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:00.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.439+0000 7f22ed000640 1 -- 172.21.15.167:0/1335132001 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5562e1ce6c00 msgr2=0x5562e2ac0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:00.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.439+0000 7f22ed000640 1 --2- 172.21.15.167:0/1335132001 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5562e1ce6c00 0x5562e2ac0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:00.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.439+0000 7f22ed000640 1 -- 172.21.15.167:0/1335132001 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562e1ce7800 msgr2=0x5562e2ac0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:00.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.439+0000 7f22ed000640 1 --2- 172.21.15.167:0/1335132001 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562e1ce7800 0x5562e2ac0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:00.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.439+0000 7f22ed000640 1 -- 172.21.15.167:0/1335132001 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562e1d1a780 con 0x5562e1ce7c00 2024-05-11T12:15:00.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.440+0000 7f22ed000640 1 --2- 172.21.15.167:0/1335132001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562e1ce7c00 0x5562e2ac0b00 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x5562e2b132f0 tx=0x5562e2a86780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ee0e82cc3d3ee66 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:00.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.440+0000 7f22e9400640 1 -- 172.21.15.167:0/1335132001 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5562e1d0c680 con 0x5562e1ce7c00 2024-05-11T12:15:00.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.440+0000 7f22e9400640 1 -- 172.21.15.167:0/1335132001 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5562e1d12a80 con 0x5562e1ce7c00 2024-05-11T12:15:00.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.440+0000 7f22e9400640 1 -- 172.21.15.167:0/1335132001 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5562e1d0cb60 con 0x5562e1ce7c00 2024-05-11T12:15:00.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.440+0000 7f22ef11b280 1 -- 172.21.15.167:0/1335132001 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562e1d1a3c0 con 0x5562e1ce7c00 2024-05-11T12:15:00.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.440+0000 7f22ef11b280 1 -- 172.21.15.167:0/1335132001 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5562e1d1b680 con 0x5562e1ce7c00 2024-05-11T12:15:00.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.440+0000 7f22ef11b280 4 waiting for MDS map... 2024-05-11T12:15:00.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.440+0000 7f22e9400640 1 -- 172.21.15.167:0/1335132001 <== mon.0 v2:172.21.15.167:3300/0 4 ==== osd_map(101..101 src has 1..101) ==== 6187+0+0 (secure 0 0 0) 0x5562e2bb8600 con 0x5562e1ce7c00 2024-05-11T12:15:00.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.440+0000 7f22e9400640 1 -- 172.21.15.167:0/1335132001 <== mon.0 v2:172.21.15.167:3300/0 5 ==== fsmap(e 69) ==== 3494+0+0 (secure 0 0 0) 0x5562e1d41180 con 0x5562e1ce7c00 2024-05-11T12:15:00.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.440+0000 7f22ef11b280 4 Got MDS map 69 2024-05-11T12:15:00.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.440+0000 7f22ef11b280 10 main: main 2024-05-11T12:15:00.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.441+0000 7f22ef11b280 4 main: connecting to RADOS... 2024-05-11T12:15:00.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.441+0000 7f22ef11b280 1 Processor -- start 2024-05-11T12:15:00.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.441+0000 7f22ef11b280 1 -- start start 2024-05-11T12:15:00.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.442+0000 7f22ef11b280 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562e2bbb400 0x5562e2ac1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:00.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.442+0000 7f22ef11b280 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562e2bbb800 0x5562e2ac1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:00.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.442+0000 7f22ef11b280 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5562e2bbbc00 0x5562e2ac1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:00.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.442+0000 7f22ef11b280 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5562e2acc180 con 0x5562e2bbb400 2024-05-11T12:15:00.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.442+0000 7f22ef11b280 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5562e1d48480 con 0x5562e2bbb800 2024-05-11T12:15:00.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.442+0000 7f22ef11b280 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x5562e1d48900 con 0x5562e2bbbc00 2024-05-11T12:15:00.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.442+0000 7f22ebc00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562e2bbb400 0x5562e2ac1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:00.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.442+0000 7f22ebc00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562e2bbb400 0x5562e2ac1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:38918/0 (socket says 172.21.15.167:38918) 2024-05-11T12:15:00.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.442+0000 7f22ebc00640 1 -- 172.21.15.167:0/4269318880 learned_addr learned my addr 172.21.15.167:0/4269318880 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:00.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.443+0000 7f22ebc00640 1 -- 172.21.15.167:0/4269318880 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5562e2bbbc00 msgr2=0x5562e2ac1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:00.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.443+0000 7f22ebc00640 1 --2- 172.21.15.167:0/4269318880 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5562e2bbbc00 0x5562e2ac1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:00.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.443+0000 7f22ebc00640 1 -- 172.21.15.167:0/4269318880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562e2bbb800 msgr2=0x5562e2ac1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:00.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.443+0000 7f22ebc00640 1 --2- 172.21.15.167:0/4269318880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562e2bbb800 0x5562e2ac1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:00.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.443+0000 7f22ebc00640 1 -- 172.21.15.167:0/4269318880 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562e1d1b4a0 con 0x5562e2bbb400 2024-05-11T12:15:00.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.443+0000 7f22ebc00640 1 --2- 172.21.15.167:0/4269318880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562e2bbb400 0x5562e2ac1080 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x5562e2bd65d0 tx=0x5562e2bd8d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa706c4bebf367a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:00.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.443+0000 7f22e6c00640 1 -- 172.21.15.167:0/4269318880 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5562e1d0c9c0 con 0x5562e2bbb400 2024-05-11T12:15:00.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.443+0000 7f22e6c00640 1 -- 172.21.15.167:0/4269318880 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5562e1d12c40 con 0x5562e2bbb400 2024-05-11T12:15:00.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.443+0000 7f22e6c00640 1 -- 172.21.15.167:0/4269318880 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5562e1d0c680 con 0x5562e2bbb400 2024-05-11T12:15:00.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.444+0000 7f22ef11b280 1 -- 172.21.15.167:0/4269318880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562e2bbb400 msgr2=0x5562e2ac1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:00.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.444+0000 7f22ef11b280 1 --2- 172.21.15.167:0/4269318880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562e2bbb400 0x5562e2ac1080 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x5562e2bd65d0 tx=0x5562e2bd8d20 comp rx=0 tx=0).stop 2024-05-11T12:15:00.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.444+0000 7f22ef11b280 1 -- 172.21.15.167:0/4269318880 shutdown_connections 2024-05-11T12:15:00.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.444+0000 7f22ef11b280 1 --2- 172.21.15.167:0/4269318880 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5562e2bbbc00 0x5562e2ac1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:00.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.444+0000 7f22ef11b280 1 --2- 172.21.15.167:0/4269318880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562e2bbb800 0x5562e2ac1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:00.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.444+0000 7f22ef11b280 1 --2- 172.21.15.167:0/4269318880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562e2bbb400 0x5562e2ac1080 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:00.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.444+0000 7f22ef11b280 1 -- 172.21.15.167:0/4269318880 >> 172.21.15.167:0/4269318880 conn(0x5562e2bbb000 msgr2=0x5562e1dcc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:00.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.444+0000 7f22ef11b280 1 -- 172.21.15.167:0/4269318880 shutdown_connections 2024-05-11T12:15:00.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.444+0000 7f22ef11b280 1 -- 172.21.15.167:0/4269318880 wait complete. 2024-05-11T12:15:00.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.445+0000 7f22ef11b280 1 Processor -- start 2024-05-11T12:15:00.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.445+0000 7f22ef11b280 1 -- start start 2024-05-11T12:15:00.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.445+0000 7f22ef11b280 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5562e2bbb400 0x5562e2ac1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:00.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.445+0000 7f22ef11b280 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562e2bbb800 0x5562e2ac1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:00.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.445+0000 7f22ef11b280 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562e2bbbc00 0x5562e2ac1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:00.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.445+0000 7f22ef11b280 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5562e2acd980 con 0x5562e2bbb800 2024-05-11T12:15:00.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.445+0000 7f22ef11b280 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5562e2acc180 con 0x5562e2bbbc00 2024-05-11T12:15:00.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.445+0000 7f22ef11b280 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x5562e2acd800 con 0x5562e2bbb400 2024-05-11T12:15:00.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.446+0000 7f22ec600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562e2bbb800 0x5562e2ac1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:00.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.446+0000 7f22ec600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562e2bbb800 0x5562e2ac1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:38928/0 (socket says 172.21.15.167:38928) 2024-05-11T12:15:00.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.446+0000 7f22ec600640 1 -- 172.21.15.167:0/1569627338 learned_addr learned my addr 172.21.15.167:0/1569627338 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:00.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.446+0000 7f22ebc00640 1 --2- 172.21.15.167:0/1569627338 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5562e2bbb400 0x5562e2ac1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:00.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.446+0000 7f22ec600640 1 -- 172.21.15.167:0/1569627338 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5562e2bbb400 msgr2=0x5562e2ac1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:00.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.446+0000 7f22ec600640 1 --2- 172.21.15.167:0/1569627338 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5562e2bbb400 0x5562e2ac1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:00.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.446+0000 7f22ebc00640 1 --2- 172.21.15.167:0/1569627338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562e2bbbc00 0x5562e2ac1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:00.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.446+0000 7f22ec600640 1 -- 172.21.15.167:0/1569627338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562e2bbbc00 msgr2=0x5562e2ac1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:00.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.446+0000 7f22ec600640 1 --2- 172.21.15.167:0/1569627338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562e2bbbc00 0x5562e2ac1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:00.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.446+0000 7f22ec600640 1 -- 172.21.15.167:0/1569627338 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562e1d1a780 con 0x5562e2bbb800 2024-05-11T12:15:00.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.446+0000 7f22ec600640 1 --2- 172.21.15.167:0/1569627338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562e2bbb800 0x5562e2ac1600 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x5562e2bd7410 tx=0x5562e2bd9020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:00.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.447+0000 7f22e4e00640 1 -- 172.21.15.167:0/1569627338 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5562e1d0c9c0 con 0x5562e2bbb800 2024-05-11T12:15:00.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.447+0000 7f22e4e00640 1 -- 172.21.15.167:0/1569627338 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5562e1d12700 con 0x5562e2bbb800 2024-05-11T12:15:00.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.447+0000 7f22e4e00640 1 -- 172.21.15.167:0/1569627338 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5562e1d0c680 con 0x5562e2bbb800 2024-05-11T12:15:00.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.447+0000 7f22ef11b280 1 -- 172.21.15.167:0/1569627338 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5562e1d1a3c0 con 0x5562e2bbb800 2024-05-11T12:15:00.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.447+0000 7f22ef11b280 1 -- 172.21.15.167:0/1569627338 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562e1d1b680 con 0x5562e2bbb800 2024-05-11T12:15:00.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.447+0000 7f22ef11b280 4 main: resolving pool 15 2024-05-11T12:15:00.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.449+0000 7f22e4e00640 1 -- 172.21.15.167:0/1569627338 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x5562e2c28c00 con 0x5562e2bbb800 2024-05-11T12:15:00.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.449+0000 7f22e4e00640 1 --2- 172.21.15.167:0/1569627338 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x5562e2c34c00 0x5562e2ac2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:00.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.449+0000 7f22e4e00640 1 -- 172.21.15.167:0/1569627338 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6187+0+0 (secure 0 0 0) 0x5562e2c36800 con 0x5562e2bbb800 2024-05-11T12:15:00.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.449+0000 7f22ef11b280 4 main: creating IoCtx.. 2024-05-11T12:15:00.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.450+0000 7f22ef11b280 1 --2- 172.21.15.167:0/1569627338 >> [v2:172.21.15.167:6818/1902208221,v1:172.21.15.167:6820/1902208221] conn(0x5562e2c35000 0x5562e2ac2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:00.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.450+0000 7f22ef11b280 1 -- 172.21.15.167:0/1569627338 --> [v2:172.21.15.167:6818/1902208221,v1:172.21.15.167:6820/1902208221] -- osd_op(unknown.0.0:1 15.b 15:d0630e4c:::mds0_sessionmap:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101) -- 0x5562e2c35400 con 0x5562e2c35000 2024-05-11T12:15:00.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.450+0000 7f22ebc00640 1 --2- 172.21.15.167:0/1569627338 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x5562e2c34c00 0x5562e2ac2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:00.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.450+0000 7f22ebc00640 1 --2- 172.21.15.167:0/1569627338 >> [v2:172.21.15.167:6818/1902208221,v1:172.21.15.167:6820/1902208221] conn(0x5562e2c35000 0x5562e2ac2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:00.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.450+0000 7f22ebc00640 1 --2- 172.21.15.167:0/1569627338 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x5562e2c34c00 0x5562e2ac2100 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x5562e2bd67e0 tx=0x5562e2bd8e40 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:00.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.450+0000 7f22ebc00640 1 --2- 172.21.15.167:0/1569627338 >> [v2:172.21.15.167:6818/1902208221,v1:172.21.15.167:6820/1902208221] conn(0x5562e2c35000 0x5562e2ac2680 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:00.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.451+0000 7f22ebc00640 1 -- 172.21.15.167:0/1569627338 <== osd.2 v2:172.21.15.167:6818/1902208221 1 ==== osd_op_reply(1 mds0_sessionmap [omap-get-header out=14b] v0'0 uv5 ondisk = 0) ==== 159+0+14 (crc 0 0 0) 0x5562e2a74000 con 0x5562e2c35000 2024-05-11T12:15:00.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.451+0000 7f22ef11b280 1 -- 172.21.15.167:0/1569627338 --> [v2:172.21.15.167:6818/1902208221,v1:172.21.15.167:6820/1902208221] -- osd_op(unknown.0.0:2 15.b 15:d0630e4c:::mds0_sessionmap:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101) -- 0x5562e2c35800 con 0x5562e2c35000 2024-05-11T12:15:00.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.451+0000 7f22ebc00640 1 -- 172.21.15.167:0/1569627338 <== osd.2 v2:172.21.15.167:6818/1902208221 2 ==== osd_op_reply(2 mds0_sessionmap [omap-get-vals out=475b] v0'0 uv5 ondisk = 0) ==== 159+0+475 (crc 0 0 0) 0x5562e2a74000 con 0x5562e2c35000 2024-05-11T12:15:00.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.451+0000 7f22ef11b280 1 -- 172.21.15.167:0/1569627338 --> [v2:172.21.15.167:6818/1902208221,v1:172.21.15.167:6820/1902208221] -- osd_op(unknown.0.0:3 15.b 15:d0630e4c:::mds0_sessionmap:head [omap-get-vals in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101) -- 0x5562e2c35c00 con 0x5562e2c35000 2024-05-11T12:15:00.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.452+0000 7f22ebc00640 1 -- 172.21.15.167:0/1569627338 <== osd.2 v2:172.21.15.167:6818/1902208221 3 ==== osd_op_reply(3 mds0_sessionmap [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) ==== 159+0+5 (crc 0 0 0) 0x5562e2a74000 con 0x5562e2c35000 2024-05-11T12:15:00.454 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T12:15:00.454 INFO:teuthology.orchestra.run.smithi167.stdout: "0": { 2024-05-11T12:15:00.454 INFO:teuthology.orchestra.run.smithi167.stdout: "data": { 2024-05-11T12:15:00.454 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions": [ 2024-05-11T12:15:00.455 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:15:00.455 INFO:teuthology.orchestra.run.smithi167.stdout: "id": 5425, 2024-05-11T12:15:00.455 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": { 2024-05-11T12:15:00.455 INFO:teuthology.orchestra.run.smithi167.stdout: "name": { 2024-05-11T12:15:00.455 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "client", 2024-05-11T12:15:00.455 INFO:teuthology.orchestra.run.smithi167.stdout: "num": 5425 2024-05-11T12:15:00.455 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:15:00.455 INFO:teuthology.orchestra.run.smithi167.stdout: "addr": { 2024-05-11T12:15:00.455 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "any", 2024-05-11T12:15:00.455 INFO:teuthology.orchestra.run.smithi167.stdout: "addr": "192.168.0.1:0", 2024-05-11T12:15:00.455 INFO:teuthology.orchestra.run.smithi167.stdout: "nonce": 1530078856 2024-05-11T12:15:00.455 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:15:00.455 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:15:00.455 INFO:teuthology.orchestra.run.smithi167.stdout: "state": "open", 2024-05-11T12:15:00.456 INFO:teuthology.orchestra.run.smithi167.stdout: "num_leases": 0, 2024-05-11T12:15:00.456 INFO:teuthology.orchestra.run.smithi167.stdout: "num_caps": 0, 2024-05-11T12:15:00.456 INFO:teuthology.orchestra.run.smithi167.stdout: "request_load_avg": 0, 2024-05-11T12:15:00.456 INFO:teuthology.orchestra.run.smithi167.stdout: "uptime": 0.00099998300000000008, 2024-05-11T12:15:00.456 INFO:teuthology.orchestra.run.smithi167.stdout: "requests_in_flight": 0, 2024-05-11T12:15:00.456 INFO:teuthology.orchestra.run.smithi167.stdout: "num_completed_requests": 0, 2024-05-11T12:15:00.456 INFO:teuthology.orchestra.run.smithi167.stdout: "num_completed_flushes": 0, 2024-05-11T12:15:00.456 INFO:teuthology.orchestra.run.smithi167.stdout: "reconnecting": false, 2024-05-11T12:15:00.456 INFO:teuthology.orchestra.run.smithi167.stdout: "recall_caps": { 2024-05-11T12:15:00.456 INFO:teuthology.orchestra.run.smithi167.stdout: "value": 0, 2024-05-11T12:15:00.456 INFO:teuthology.orchestra.run.smithi167.stdout: "halflife": 60 2024-05-11T12:15:00.456 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:15:00.456 INFO:teuthology.orchestra.run.smithi167.stdout: "release_caps": { 2024-05-11T12:15:00.456 INFO:teuthology.orchestra.run.smithi167.stdout: "value": 0, 2024-05-11T12:15:00.456 INFO:teuthology.orchestra.run.smithi167.stdout: "halflife": 60 2024-05-11T12:15:00.457 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:15:00.457 INFO:teuthology.orchestra.run.smithi167.stdout: "recall_caps_throttle": { 2024-05-11T12:15:00.457 INFO:teuthology.orchestra.run.smithi167.stdout: "value": 0, 2024-05-11T12:15:00.457 INFO:teuthology.orchestra.run.smithi167.stdout: "halflife": 1.5 2024-05-11T12:15:00.457 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:15:00.457 INFO:teuthology.orchestra.run.smithi167.stdout: "recall_caps_throttle2o": { 2024-05-11T12:15:00.457 INFO:teuthology.orchestra.run.smithi167.stdout: "value": 0, 2024-05-11T12:15:00.457 INFO:teuthology.orchestra.run.smithi167.stdout: "halflife": 0.5 2024-05-11T12:15:00.457 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:15:00.457 INFO:teuthology.orchestra.run.smithi167.stdout: "session_cache_liveness": { 2024-05-11T12:15:00.457 INFO:teuthology.orchestra.run.smithi167.stdout: "value": 0, 2024-05-11T12:15:00.457 INFO:teuthology.orchestra.run.smithi167.stdout: "halflife": 300 2024-05-11T12:15:00.457 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:15:00.457 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_acquisition": { 2024-05-11T12:15:00.457 INFO:teuthology.orchestra.run.smithi167.stdout: "value": 0, 2024-05-11T12:15:00.457 INFO:teuthology.orchestra.run.smithi167.stdout: "halflife": 30 2024-05-11T12:15:00.458 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:15:00.458 INFO:teuthology.orchestra.run.smithi167.stdout: "last_trim_completed_requests_tid": 0, 2024-05-11T12:15:00.458 INFO:teuthology.orchestra.run.smithi167.stdout: "last_trim_completed_flushes_tid": 0, 2024-05-11T12:15:00.458 INFO:teuthology.orchestra.run.smithi167.stdout: "delegated_inos": [], 2024-05-11T12:15:00.458 INFO:teuthology.orchestra.run.smithi167.stdout: "inst": "client.5425 192.168.0.1:0/1530078856", 2024-05-11T12:15:00.458 INFO:teuthology.orchestra.run.smithi167.stdout: "completed_requests": [], 2024-05-11T12:15:00.458 INFO:teuthology.orchestra.run.smithi167.stdout: "prealloc_inos": [], 2024-05-11T12:15:00.458 INFO:teuthology.orchestra.run.smithi167.stdout: "client_metadata": { 2024-05-11T12:15:00.458 INFO:teuthology.orchestra.run.smithi167.stdout: "client_features": { 2024-05-11T12:15:00.458 INFO:teuthology.orchestra.run.smithi167.stdout: "feature_bits": "0x000000000037fbff" 2024-05-11T12:15:00.458 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:15:00.458 INFO:teuthology.orchestra.run.smithi167.stdout: "metric_spec": { 2024-05-11T12:15:00.458 INFO:teuthology.orchestra.run.smithi167.stdout: "metric_flags": { 2024-05-11T12:15:00.458 INFO:teuthology.orchestra.run.smithi167.stdout: "feature_bits": "0x000000000080ffff" 2024-05-11T12:15:00.459 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:15:00.459 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:15:00.459 INFO:teuthology.orchestra.run.smithi167.stdout: "entity_id": "1", 2024-05-11T12:15:00.459 INFO:teuthology.orchestra.run.smithi167.stdout: "hostname": "smithi204", 2024-05-11T12:15:00.459 INFO:teuthology.orchestra.run.smithi167.stdout: "kernel_version": "6.9.0-rc3-gfb9a6a44107e", 2024-05-11T12:15:00.459 INFO:teuthology.orchestra.run.smithi167.stdout: "root": "/" 2024-05-11T12:15:00.459 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:15:00.459 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:15:00.459 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T12:15:00.459 INFO:teuthology.orchestra.run.smithi167.stdout: "id": 5129, 2024-05-11T12:15:00.459 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": { 2024-05-11T12:15:00.459 INFO:teuthology.orchestra.run.smithi167.stdout: "name": { 2024-05-11T12:15:00.459 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "client", 2024-05-11T12:15:00.460 INFO:teuthology.orchestra.run.smithi167.stdout: "num": 5129 2024-05-11T12:15:00.460 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:15:00.460 INFO:teuthology.orchestra.run.smithi167.stdout: "addr": { 2024-05-11T12:15:00.460 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "any", 2024-05-11T12:15:00.460 INFO:teuthology.orchestra.run.smithi167.stdout: "addr": "172.21.15.167:0", 2024-05-11T12:15:00.460 INFO:teuthology.orchestra.run.smithi167.stdout: "nonce": 3784505284 2024-05-11T12:15:00.460 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:15:00.460 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:15:00.460 INFO:teuthology.orchestra.run.smithi167.stdout: "state": "open", 2024-05-11T12:15:00.460 INFO:teuthology.orchestra.run.smithi167.stdout: "num_leases": 0, 2024-05-11T12:15:00.460 INFO:teuthology.orchestra.run.smithi167.stdout: "num_caps": 0, 2024-05-11T12:15:00.460 INFO:teuthology.orchestra.run.smithi167.stdout: "request_load_avg": 0, 2024-05-11T12:15:00.460 INFO:teuthology.orchestra.run.smithi167.stdout: "uptime": 0.00099998300000000008, 2024-05-11T12:15:00.460 INFO:teuthology.orchestra.run.smithi167.stdout: "requests_in_flight": 0, 2024-05-11T12:15:00.460 INFO:teuthology.orchestra.run.smithi167.stdout: "num_completed_requests": 0, 2024-05-11T12:15:00.460 INFO:teuthology.orchestra.run.smithi167.stdout: "num_completed_flushes": 0, 2024-05-11T12:15:00.461 INFO:teuthology.orchestra.run.smithi167.stdout: "reconnecting": false, 2024-05-11T12:15:00.461 INFO:teuthology.orchestra.run.smithi167.stdout: "recall_caps": { 2024-05-11T12:15:00.461 INFO:teuthology.orchestra.run.smithi167.stdout: "value": 0, 2024-05-11T12:15:00.461 INFO:teuthology.orchestra.run.smithi167.stdout: "halflife": 60 2024-05-11T12:15:00.461 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:15:00.461 INFO:teuthology.orchestra.run.smithi167.stdout: "release_caps": { 2024-05-11T12:15:00.461 INFO:teuthology.orchestra.run.smithi167.stdout: "value": 0, 2024-05-11T12:15:00.461 INFO:teuthology.orchestra.run.smithi167.stdout: "halflife": 60 2024-05-11T12:15:00.461 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:15:00.461 INFO:teuthology.orchestra.run.smithi167.stdout: "recall_caps_throttle": { 2024-05-11T12:15:00.461 INFO:teuthology.orchestra.run.smithi167.stdout: "value": 0, 2024-05-11T12:15:00.461 INFO:teuthology.orchestra.run.smithi167.stdout: "halflife": 1.5 2024-05-11T12:15:00.461 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:15:00.461 INFO:teuthology.orchestra.run.smithi167.stdout: "recall_caps_throttle2o": { 2024-05-11T12:15:00.461 INFO:teuthology.orchestra.run.smithi167.stdout: "value": 0, 2024-05-11T12:15:00.462 INFO:teuthology.orchestra.run.smithi167.stdout: "halflife": 0.5 2024-05-11T12:15:00.462 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:15:00.462 INFO:teuthology.orchestra.run.smithi167.stdout: "session_cache_liveness": { 2024-05-11T12:15:00.462 INFO:teuthology.orchestra.run.smithi167.stdout: "value": 0, 2024-05-11T12:15:00.462 INFO:teuthology.orchestra.run.smithi167.stdout: "halflife": 300 2024-05-11T12:15:00.462 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:15:00.462 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_acquisition": { 2024-05-11T12:15:00.462 INFO:teuthology.orchestra.run.smithi167.stdout: "value": 0, 2024-05-11T12:15:00.462 INFO:teuthology.orchestra.run.smithi167.stdout: "halflife": 30 2024-05-11T12:15:00.462 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:15:00.462 INFO:teuthology.orchestra.run.smithi167.stdout: "last_trim_completed_requests_tid": 0, 2024-05-11T12:15:00.462 INFO:teuthology.orchestra.run.smithi167.stdout: "last_trim_completed_flushes_tid": 0, 2024-05-11T12:15:00.462 INFO:teuthology.orchestra.run.smithi167.stdout: "delegated_inos": [], 2024-05-11T12:15:00.463 INFO:teuthology.orchestra.run.smithi167.stdout: "inst": "client.5129 172.21.15.167:0/3784505284", 2024-05-11T12:15:00.463 INFO:teuthology.orchestra.run.smithi167.stdout: "completed_requests": [], 2024-05-11T12:15:00.463 INFO:teuthology.orchestra.run.smithi167.stdout: "prealloc_inos": [], 2024-05-11T12:15:00.463 INFO:teuthology.orchestra.run.smithi167.stdout: "client_metadata": { 2024-05-11T12:15:00.463 INFO:teuthology.orchestra.run.smithi167.stdout: "client_features": { 2024-05-11T12:15:00.463 INFO:teuthology.orchestra.run.smithi167.stdout: "feature_bits": "0x000000000037fbff" 2024-05-11T12:15:00.463 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:15:00.463 INFO:teuthology.orchestra.run.smithi167.stdout: "metric_spec": { 2024-05-11T12:15:00.463 INFO:teuthology.orchestra.run.smithi167.stdout: "metric_flags": { 2024-05-11T12:15:00.463 INFO:teuthology.orchestra.run.smithi167.stdout: "feature_bits": "0x000000000080ffff" 2024-05-11T12:15:00.463 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:15:00.463 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:15:00.463 INFO:teuthology.orchestra.run.smithi167.stdout: "entity_id": "0", 2024-05-11T12:15:00.463 INFO:teuthology.orchestra.run.smithi167.stdout: "hostname": "smithi167", 2024-05-11T12:15:00.464 INFO:teuthology.orchestra.run.smithi167.stdout: "kernel_version": "6.9.0-rc3-gfb9a6a44107e", 2024-05-11T12:15:00.464 INFO:teuthology.orchestra.run.smithi167.stdout: "root": "/" 2024-05-11T12:15:00.464 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:15:00.464 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:15:00.464 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-05-11T12:15:00.464 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:15:00.464 INFO:teuthology.orchestra.run.smithi167.stdout: "result": 0 2024-05-11T12:15:00.464 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:15:00.464 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T12:15:00.464 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:15:00.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.452+0000 7f22ef11b280 1 -- 172.21.15.167:0/1569627338 >> [v2:172.21.15.167:6818/1902208221,v1:172.21.15.167:6820/1902208221] conn(0x5562e2c35000 msgr2=0x5562e2ac2680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:00.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.452+0000 7f22ef11b280 1 --2- 172.21.15.167:0/1569627338 >> [v2:172.21.15.167:6818/1902208221,v1:172.21.15.167:6820/1902208221] conn(0x5562e2c35000 0x5562e2ac2680 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:00.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.452+0000 7f22ef11b280 1 -- 172.21.15.167:0/1569627338 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x5562e2c34c00 msgr2=0x5562e2ac2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:00.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.452+0000 7f22ef11b280 1 --2- 172.21.15.167:0/1569627338 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x5562e2c34c00 0x5562e2ac2100 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x5562e2bd67e0 tx=0x5562e2bd8e40 comp rx=0 tx=0).stop 2024-05-11T12:15:00.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.452+0000 7f22ef11b280 1 -- 172.21.15.167:0/1569627338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562e2bbb800 msgr2=0x5562e2ac1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:00.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.452+0000 7f22ef11b280 1 --2- 172.21.15.167:0/1569627338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562e2bbb800 0x5562e2ac1600 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x5562e2bd7410 tx=0x5562e2bd9020 comp rx=0 tx=0).stop 2024-05-11T12:15:00.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.452+0000 7f22ec600640 1 -- 172.21.15.167:0/1569627338 reap_dead start 2024-05-11T12:15:00.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.452+0000 7f22ef11b280 1 -- 172.21.15.167:0/1569627338 shutdown_connections 2024-05-11T12:15:00.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.452+0000 7f22ef11b280 1 -- 172.21.15.167:0/1569627338 >> 172.21.15.167:0/1569627338 conn(0x5562e2bbb000 msgr2=0x5562e1dcc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:00.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.452+0000 7f22ef11b280 1 -- 172.21.15.167:0/1569627338 shutdown_connections 2024-05-11T12:15:00.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.452+0000 7f22ef11b280 1 -- 172.21.15.167:0/1569627338 wait complete. 2024-05-11T12:15:00.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.453+0000 7f22ef11b280 1 -- 172.21.15.167:0/1335132001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562e1ce7c00 msgr2=0x5562e2ac0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:00.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.453+0000 7f22ef11b280 1 --2- 172.21.15.167:0/1335132001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562e1ce7c00 0x5562e2ac0b00 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x5562e2b132f0 tx=0x5562e2a86780 comp rx=0 tx=0).stop 2024-05-11T12:15:00.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.453+0000 7f22ef11b280 1 -- 172.21.15.167:0/1335132001 shutdown_connections 2024-05-11T12:15:00.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.453+0000 7f22ef11b280 1 --2- 172.21.15.167:0/1335132001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562e1ce7c00 0x5562e2ac0b00 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:00.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.453+0000 7f22ef11b280 1 --2- 172.21.15.167:0/1335132001 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562e1ce7800 0x5562e2ac0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:00.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.453+0000 7f22ef11b280 1 --2- 172.21.15.167:0/1335132001 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5562e1ce6c00 0x5562e2ac0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:00.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.453+0000 7f22ef11b280 1 -- 172.21.15.167:0/1335132001 >> 172.21.15.167:0/1335132001 conn(0x5562e1ce7000 msgr2=0x5562e1dcd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:00.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.453+0000 7f22ef11b280 1 -- 172.21.15.167:0/1335132001 shutdown_connections 2024-05-11T12:15:00.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.453+0000 7f22ef11b280 1 -- 172.21.15.167:0/1335132001 wait complete. 2024-05-11T12:15:00.468 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '0', 'show', 'session'] in time 0:00:00.124844, result: { "0": { "data": { "sessions": [ { "id": 5425, "entity": { "name": { "type": "client", "num": 5425 }, "addr": { "type": "any", "addr": "192.168.0.1:0", "nonce": 1530078856 } }, "state": "open", "num_leases": 0, "num_caps": 0, "request_load_avg": 0, "uptime": 0.00099998300000000008, "requests_in_flight": 0, "num_completed_requests": 0, "num_completed_flushes": 0, "reconnecting": false, "recall_caps": { "value": 0, "halflife": 60 }, "release_caps": { "value": 0, "halflife": 60 }, "recall_caps_throttle": { "value": 0, "halflife": 1.5 }, "recall_caps_throttle2o": { "value": 0, "halflife": 0.5 }, "session_cache_liveness": { "value": 0, "halflife": 300 }, "cap_acquisition": { "value": 0, "halflife": 30 }, "last_trim_completed_requests_tid": 0, "last_trim_completed_flushes_tid": 0, "delegated_inos": [], "inst": "client.5425 192.168.0.1:0/1530078856", "completed_requests": [], "prealloc_inos": [], "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "entity_id": "1", "hostname": "smithi204", "kernel_version": "6.9.0-rc3-gfb9a6a44107e", "root": "/" } }, { "id": 5129, "entity": { "name": { "type": "client", "num": 5129 }, "addr": { "type": "any", "addr": "172.21.15.167:0", "nonce": 3784505284 } }, "state": "open", "num_leases": 0, "num_caps": 0, "request_load_avg": 0, "uptime": 0.00099998300000000008, "requests_in_flight": 0, "num_completed_requests": 0, "num_completed_flushes": 0, "reconnecting": false, "recall_caps": { "value": 0, "halflife": 60 }, "release_caps": { "value": 0, "halflife": 60 }, "recall_caps_throttle": { "value": 0, "halflife": 1.5 }, "recall_caps_throttle2o": { "value": 0, "halflife": 0.5 }, "session_cache_liveness": { "value": 0, "halflife": 300 }, "cap_acquisition": { "value": 0, "halflife": 30 }, "last_trim_completed_requests_tid": 0, "last_trim_completed_flushes_tid": 0, "delegated_inos": [], "inst": "client.5129 172.21.15.167:0/3784505284", "completed_requests": [], "prealloc_inos": [], "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "entity_id": "0", "hostname": "smithi167", "kernel_version": "6.9.0-rc3-gfb9a6a44107e", "root": "/" } } ] }, "result": 0 } } 2024-05-11T12:15:00.468 INFO:tasks.cephfs.test_sessionmap:SessionMap: { "0": { "data": { "sessions": [ { "id": 5425, "entity": { "name": { "type": "client", "num": 5425 }, "addr": { "type": "any", "addr": "192.168.0.1:0", "nonce": 1530078856 } }, "state": "open", "num_leases": 0, "num_caps": 0, "request_load_avg": 0, "uptime": 0.000999983, "requests_in_flight": 0, "num_completed_requests": 0, "num_completed_flushes": 0, "reconnecting": false, "recall_caps": { "value": 0, "halflife": 60 }, "release_caps": { "value": 0, "halflife": 60 }, "recall_caps_throttle": { "value": 0, "halflife": 1.5 }, "recall_caps_throttle2o": { "value": 0, "halflife": 0.5 }, "session_cache_liveness": { "value": 0, "halflife": 300 }, "cap_acquisition": { "value": 0, "halflife": 30 }, "last_trim_completed_requests_tid": 0, "last_trim_completed_flushes_tid": 0, "delegated_inos": [], "inst": "client.5425 192.168.0.1:0/1530078856", "completed_requests": [], "prealloc_inos": [], "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "entity_id": "1", "hostname": "smithi204", "kernel_version": "6.9.0-rc3-gfb9a6a44107e", "root": "/" } }, { "id": 5129, "entity": { "name": { "type": "client", "num": 5129 }, "addr": { "type": "any", "addr": "172.21.15.167:0", "nonce": 3784505284 } }, "state": "open", "num_leases": 0, "num_caps": 0, "request_load_avg": 0, "uptime": 0.000999983, "requests_in_flight": 0, "num_completed_requests": 0, "num_completed_flushes": 0, "reconnecting": false, "recall_caps": { "value": 0, "halflife": 60 }, "release_caps": { "value": 0, "halflife": 60 }, "recall_caps_throttle": { "value": 0, "halflife": 1.5 }, "recall_caps_throttle2o": { "value": 0, "halflife": 0.5 }, "session_cache_liveness": { "value": 0, "halflife": 300 }, "cap_acquisition": { "value": 0, "halflife": 30 }, "last_trim_completed_requests_tid": 0, "last_trim_completed_flushes_tid": 0, "delegated_inos": [], "inst": "client.5129 172.21.15.167:0/3784505284", "completed_requests": [], "prealloc_inos": [], "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "entity_id": "0", "hostname": "smithi167", "kernel_version": "6.9.0-rc3-gfb9a6a44107e", "root": "/" } } ] }, "result": 0 } } 2024-05-11T12:15:00.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:15:00.469 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir bravo) 2024-05-11T12:15:00.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:15:00.529 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch bravo/file_a) 2024-05-11T12:15:00.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:15:00.586 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest/mnt.1 && exec touch bravo/file_b) 2024-05-11T12:15:00.658 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 2 2024-05-11T12:15:00.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.862+0000 7f3d72e00640 1 Processor -- start 2024-05-11T12:15:00.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.862+0000 7f3d72e00640 1 -- start start 2024-05-11T12:15:00.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.864+0000 7f3d72e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3d6c0b0ca0 0x7f3d6c06fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:00.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.864+0000 7f3d72e00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d6c0aee80 con 0x7f3d6c15d030 2024-05-11T12:15:00.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.864+0000 7f3d72e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d6c0af160 con 0x7f3d6c0b0780 2024-05-11T12:15:00.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.864+0000 7f3d72e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f3d6c0af440 con 0x7f3d6c0b0ca0 2024-05-11T12:15:00.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.864+0000 7f3d6be00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f3d6c15d030 0x7f3d6c0b0070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:51852/0 (socket says 172.21.15.167:51852) 2024-05-11T12:15:00.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.864+0000 7f3d6be00640 1 -- 172.21.15.167:0/3720023719 learned_addr learned my addr 172.21.15.167:0/3720023719 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:15:00.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.865+0000 7f3d6aa00640 1 -- 172.21.15.167:0/3720023719 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2050809466 0 0) 0x7f3d6c0aee80 con 0x7f3d6c15d030 2024-05-11T12:15:00.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.865+0000 7f3d6aa00640 1 -- 172.21.15.167:0/3720023719 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d50003520 con 0x7f3d6c15d030 2024-05-11T12:15:00.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.865+0000 7f3d6b400640 1 --2- 172.21.15.167:0/3720023719 >> v2:172.21.15.204:3300/0 conn(0x7f3d6c0b0ca0 0x7f3d6c06fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:00.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.865+0000 7f3d6aa00640 1 -- 172.21.15.167:0/3720023719 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1318942606 0 0) 0x7f3d50003520 con 0x7f3d6c15d030 2024-05-11T12:15:00.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.865+0000 7f3d6aa00640 1 -- 172.21.15.167:0/3720023719 >> v2:172.21.15.204:3300/0 conn(0x7f3d6c0b0ca0 msgr2=0x7f3d6c06fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:00.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.865+0000 7f3d6aa00640 1 --2- 172.21.15.167:0/3720023719 >> v2:172.21.15.204:3300/0 conn(0x7f3d6c0b0ca0 0x7f3d6c06fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:00.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.865+0000 7f3d6aa00640 1 -- 172.21.15.167:0/3720023719 >> v1:172.21.15.204:6789/0 conn(0x7f3d6c0b0780 legacy=0x7f3d6c06fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:00.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.865+0000 7f3d6aa00640 1 -- 172.21.15.167:0/3720023719 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d6c0af720 con 0x7f3d6c15d030 2024-05-11T12:15:00.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.865+0000 7f3d6aa00640 1 -- 172.21.15.167:0/3720023719 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f3d5c0016f0 con 0x7f3d6c15d030 2024-05-11T12:15:00.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.865+0000 7f3d6aa00640 1 -- 172.21.15.167:0/3720023719 >> v1:172.21.15.167:6789/0 conn(0x7f3d6c15d030 legacy=0x7f3d6c0b0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:00.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.865+0000 7f3d6aa00640 1 --2- 172.21.15.167:0/3720023719 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d50003a00 0x7f3d50003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:00.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.865+0000 7f3d6aa00640 1 --2- 172.21.15.167:0/3720023719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d50004800 0x7f3d50004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:00.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.865+0000 7f3d6aa00640 1 --2- 172.21.15.167:0/3720023719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d50005290 0x7f3d5000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:00.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.865+0000 7f3d6aa00640 1 -- 172.21.15.167:0/3720023719 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6c0af440 con 0x7f3d50005290 2024-05-11T12:15:00.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.865+0000 7f3d6aa00640 1 -- 172.21.15.167:0/3720023719 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5000c1e0 con 0x7f3d50004800 2024-05-11T12:15:00.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.865+0000 7f3d6aa00640 1 -- 172.21.15.167:0/3720023719 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3d5000c3e0 con 0x7f3d50003a00 2024-05-11T12:15:00.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.866+0000 7f3d70e00640 1 --2- 172.21.15.167:0/3720023719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d50005290 0x7f3d5000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:00.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.866+0000 7f3d6b400640 1 --2- 172.21.15.167:0/3720023719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d50004800 0x7f3d50004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:00.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.866+0000 7f3d70e00640 1 -- 172.21.15.167:0/3720023719 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d50003a00 msgr2=0x7f3d50003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:00.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.866+0000 7f3d70e00640 1 --2- 172.21.15.167:0/3720023719 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d50003a00 0x7f3d50003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:00.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.866+0000 7f3d70e00640 1 -- 172.21.15.167:0/3720023719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d50004800 msgr2=0x7f3d50004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:00.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.866+0000 7f3d70e00640 1 --2- 172.21.15.167:0/3720023719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d50004800 0x7f3d50004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:00.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.866+0000 7f3d70e00640 1 -- 172.21.15.167:0/3720023719 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d50003520 con 0x7f3d50005290 2024-05-11T12:15:00.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.867+0000 7f3d70e00640 1 -- 172.21.15.167:0/3720023719 reap_dead start 2024-05-11T12:15:00.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.867+0000 7f3d70e00640 1 --2- 172.21.15.167:0/3720023719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d50005290 0x7f3d5000b9c0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f3d640044b0 tx=0x7f3d50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9b39a97960901f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:00.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.868+0000 7f3d6aa00640 1 -- 172.21.15.167:0/3720023719 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d64004080 con 0x7f3d50005290 2024-05-11T12:15:00.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.868+0000 7f3d6aa00640 1 -- 172.21.15.167:0/3720023719 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d6402ada0 con 0x7f3d50005290 2024-05-11T12:15:00.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.868+0000 7f3d6aa00640 1 -- 172.21.15.167:0/3720023719 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d64006d90 con 0x7f3d50005290 2024-05-11T12:15:00.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.868+0000 7f3d72e00640 1 -- 172.21.15.167:0/3720023719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d50005290 msgr2=0x7f3d5000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:00.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.868+0000 7f3d72e00640 1 --2- 172.21.15.167:0/3720023719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d50005290 0x7f3d5000b9c0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f3d640044b0 tx=0x7f3d50005830 comp rx=0 tx=0).stop 2024-05-11T12:15:00.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.868+0000 7f3d72e00640 1 -- 172.21.15.167:0/3720023719 shutdown_connections 2024-05-11T12:15:00.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.868+0000 7f3d72e00640 1 --2- 172.21.15.167:0/3720023719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d50005290 0x7f3d5000b9c0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:00.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.868+0000 7f3d72e00640 1 -- 172.21.15.167:0/3720023719 >> 172.21.15.167:0/3720023719 conn(0x7f3d6c073a10 msgr2=0x7f3d6c06c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:00.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.868+0000 7f3d72e00640 1 -- 172.21.15.167:0/3720023719 shutdown_connections 2024-05-11T12:15:00.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.868+0000 7f3d72e00640 1 -- 172.21.15.167:0/3720023719 wait complete. 2024-05-11T12:15:00.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.870+0000 7f3d72e00640 1 Processor -- start 2024-05-11T12:15:00.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.870+0000 7f3d72e00640 1 -- start start 2024-05-11T12:15:00.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.871+0000 7f3d72e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d50005290 0x7f3d6c0b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:00.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.871+0000 7f3d72e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d6c146500 0x7f3d6c15b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:00.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.871+0000 7f3d72e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d6c1f3dd0 0x7f3d6c1f4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:00.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.871+0000 7f3d72e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6c15fad0 con 0x7f3d50005290 2024-05-11T12:15:00.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.871+0000 7f3d72e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6c15f950 con 0x7f3d6c146500 2024-05-11T12:15:00.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.871+0000 7f3d72e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3d6c15fc50 con 0x7f3d6c1f3dd0 2024-05-11T12:15:00.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.871+0000 7f3d6be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d50005290 0x7f3d6c0b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:00.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.871+0000 7f3d6be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d50005290 0x7f3d6c0b5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:38950/0 (socket says 172.21.15.167:38950) 2024-05-11T12:15:00.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.871+0000 7f3d6be00640 1 -- 172.21.15.167:0/3224113375 learned_addr learned my addr 172.21.15.167:0/3224113375 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:00.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.872+0000 7f3d6be00640 1 -- 172.21.15.167:0/3224113375 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d6c1f3dd0 msgr2=0x7f3d6c1f4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:00.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.872+0000 7f3d6be00640 1 --2- 172.21.15.167:0/3224113375 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d6c1f3dd0 0x7f3d6c1f4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:00.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.872+0000 7f3d6be00640 1 -- 172.21.15.167:0/3224113375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d6c146500 msgr2=0x7f3d6c15b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:00.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.872+0000 7f3d6be00640 1 --2- 172.21.15.167:0/3224113375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d6c146500 0x7f3d6c15b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:00.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.872+0000 7f3d6be00640 1 -- 172.21.15.167:0/3224113375 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d6c0b5540 con 0x7f3d50005290 2024-05-11T12:15:00.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.872+0000 7f3d6be00640 1 --2- 172.21.15.167:0/3224113375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d50005290 0x7f3d6c0b5000 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c00c720 tx=0x7f3d5c00cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:00.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.872+0000 7f3d63e00640 1 -- 172.21.15.167:0/3224113375 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c0074e0 con 0x7f3d50005290 2024-05-11T12:15:00.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.872+0000 7f3d63e00640 1 -- 172.21.15.167:0/3224113375 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d5c00ce30 con 0x7f3d50005290 2024-05-11T12:15:00.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.872+0000 7f3d63e00640 1 -- 172.21.15.167:0/3224113375 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c0049e0 con 0x7f3d50005290 2024-05-11T12:15:00.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.872+0000 7f3d72e00640 1 -- 172.21.15.167:0/3224113375 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d6c0b57e0 con 0x7f3d50005290 2024-05-11T12:15:00.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.873+0000 7f3d72e00640 1 -- 172.21.15.167:0/3224113375 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d6c0b5da0 con 0x7f3d50005290 2024-05-11T12:15:00.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.874+0000 7f3d72e00640 1 -- 172.21.15.167:0/3224113375 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d6c0b5f70 con 0x7f3d50005290 2024-05-11T12:15:00.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.875+0000 7f3d63e00640 1 -- 172.21.15.167:0/3224113375 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3d5c011070 con 0x7f3d50005290 2024-05-11T12:15:00.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.876+0000 7f3d63e00640 1 --2- 172.21.15.167:0/3224113375 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3d480762d0 0x7f3d48078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:00.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.876+0000 7f3d63e00640 1 -- 172.21.15.167:0/3224113375 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6187+0+0 (secure 0 0 0) 0x7f3d5c0974d0 con 0x7f3d50005290 2024-05-11T12:15:00.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.876+0000 7f3d6b400640 1 --2- 172.21.15.167:0/3224113375 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3d480762d0 0x7f3d48078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:00.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.877+0000 7f3d6b400640 1 --2- 172.21.15.167:0/3224113375 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3d480762d0 0x7f3d48078790 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c15b3d0 tx=0x7f3d5400d280 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:00.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:00.881+0000 7f3d63e00640 1 -- 172.21.15.167:0/3224113375 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3d5c060c50 con 0x7f3d50005290 2024-05-11T12:15:01.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:01.130+0000 7f3d72e00640 1 -- 172.21.15.167:0/3224113375 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) -- 0x7f3d6c0684d0 con 0x7f3d50005290 2024-05-11T12:15:01.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:01.813+0000 7f3d63e00640 1 -- 172.21.15.167:0/3224113375 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v70) ==== 105+0+0 (secure 0 0 0) 0x7f3d5c060550 con 0x7f3d50005290 2024-05-11T12:15:01.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:01.817+0000 7f3d72e00640 1 -- 172.21.15.167:0/3224113375 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3d480762d0 msgr2=0x7f3d48078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:01.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:01.817+0000 7f3d72e00640 1 --2- 172.21.15.167:0/3224113375 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3d480762d0 0x7f3d48078790 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c15b3d0 tx=0x7f3d5400d280 comp rx=0 tx=0).stop 2024-05-11T12:15:01.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:01.818+0000 7f3d72e00640 1 -- 172.21.15.167:0/3224113375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d50005290 msgr2=0x7f3d6c0b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:01.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:01.818+0000 7f3d72e00640 1 --2- 172.21.15.167:0/3224113375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d50005290 0x7f3d6c0b5000 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c00c720 tx=0x7f3d5c00cbe0 comp rx=0 tx=0).stop 2024-05-11T12:15:01.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:01.818+0000 7f3d72e00640 1 -- 172.21.15.167:0/3224113375 shutdown_connections 2024-05-11T12:15:01.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:01.818+0000 7f3d72e00640 1 --2- 172.21.15.167:0/3224113375 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d6c1f3dd0 0x7f3d6c1f4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:01.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:01.818+0000 7f3d72e00640 1 --2- 172.21.15.167:0/3224113375 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3d480762d0 0x7f3d48078790 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:01.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:01.818+0000 7f3d72e00640 1 --2- 172.21.15.167:0/3224113375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d6c146500 0x7f3d6c15b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:01.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:01.818+0000 7f3d72e00640 1 --2- 172.21.15.167:0/3224113375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d50005290 0x7f3d6c0b5000 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:01.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:01.818+0000 7f3d72e00640 1 -- 172.21.15.167:0/3224113375 >> 172.21.15.167:0/3224113375 conn(0x7f3d6c073a10 msgr2=0x7f3d6c15fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:01.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:01.818+0000 7f3d72e00640 1 -- 172.21.15.167:0/3224113375 shutdown_connections 2024-05-11T12:15:01.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:01.818+0000 7f3d72e00640 1 -- 172.21.15.167:0/3224113375 wait complete. 2024-05-11T12:15:01.847 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:15:02.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.028+0000 7f140b800640 1 Processor -- start 2024-05-11T12:15:02.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.028+0000 7f140b800640 1 -- start start 2024-05-11T12:15:02.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.029+0000 7f140b800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1404073a00 0x7f1404073e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:02.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.030+0000 7f140b800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f140415e880 0x7f140415ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:02.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.030+0000 7f140b800640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1404072520 con 0x7f14040b62c0 2024-05-11T12:15:02.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.030+0000 7f140b800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f14040732e0 con 0x7f1404073a00 2024-05-11T12:15:02.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.030+0000 7f140b800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f14040722d0 con 0x7f140415e880 2024-05-11T12:15:02.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.030+0000 7f1408e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f14040b62c0 0x7f14040b66a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:51864/0 (socket says 172.21.15.167:51864) 2024-05-11T12:15:02.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.030+0000 7f1408e00640 1 -- 172.21.15.167:0/2024767475 learned_addr learned my addr 172.21.15.167:0/2024767475 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:15:02.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.030+0000 7f1403e00640 1 --2- 172.21.15.167:0/2024767475 >> v2:172.21.15.204:3300/0 conn(0x7f1404073a00 0x7f1404073e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:02.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.030+0000 7f1409800640 1 --2- 172.21.15.167:0/2024767475 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f140415e880 0x7f140415ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:02.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.031+0000 7f1403400640 1 -- 172.21.15.167:0/2024767475 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3556335200 0 0) 0x7f1404072520 con 0x7f14040b62c0 2024-05-11T12:15:02.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.031+0000 7f1403400640 1 -- 172.21.15.167:0/2024767475 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f13ec003520 con 0x7f14040b62c0 2024-05-11T12:15:02.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.031+0000 7f1403400640 1 -- 172.21.15.167:0/2024767475 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2735552225 0 0) 0x7f13ec003520 con 0x7f14040b62c0 2024-05-11T12:15:02.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.031+0000 7f1403400640 1 -- 172.21.15.167:0/2024767475 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f140415e880 msgr2=0x7f140415ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:02.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.031+0000 7f1403400640 1 --2- 172.21.15.167:0/2024767475 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f140415e880 0x7f140415ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:02.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.031+0000 7f1403400640 1 -- 172.21.15.167:0/2024767475 >> v2:172.21.15.204:3300/0 conn(0x7f1404073a00 msgr2=0x7f1404073e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:02.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.031+0000 7f1403400640 1 --2- 172.21.15.167:0/2024767475 >> v2:172.21.15.204:3300/0 conn(0x7f1404073a00 0x7f1404073e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:02.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.031+0000 7f1403400640 1 -- 172.21.15.167:0/2024767475 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1404072800 con 0x7f14040b62c0 2024-05-11T12:15:02.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.031+0000 7f1403400640 1 -- 172.21.15.167:0/2024767475 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f13f80016f0 con 0x7f14040b62c0 2024-05-11T12:15:02.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.031+0000 7f1403400640 1 -- 172.21.15.167:0/2024767475 >> v1:172.21.15.167:6789/0 conn(0x7f14040b62c0 legacy=0x7f14040b66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:02.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.031+0000 7f1403400640 1 --2- 172.21.15.167:0/2024767475 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13ec003a00 0x7f13ec003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:02.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.031+0000 7f1403400640 1 --2- 172.21.15.167:0/2024767475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13ec004800 0x7f13ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:02.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.031+0000 7f1403400640 1 --2- 172.21.15.167:0/2024767475 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f140415e880 0x7f13ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:02.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.031+0000 7f1403400640 1 -- 172.21.15.167:0/2024767475 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f14040732e0 con 0x7f13ec004800 2024-05-11T12:15:02.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.031+0000 7f1403400640 1 -- 172.21.15.167:0/2024767475 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f14040722d0 con 0x7f13ec003a00 2024-05-11T12:15:02.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.031+0000 7f1403400640 1 -- 172.21.15.167:0/2024767475 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f13ec00b9c0 con 0x7f140415e880 2024-05-11T12:15:02.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.031+0000 7f1403e00640 1 --2- 172.21.15.167:0/2024767475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13ec004800 0x7f13ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:02.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.032+0000 7f1408e00640 1 --2- 172.21.15.167:0/2024767475 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13ec003a00 0x7f13ec003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:02.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.032+0000 7f1403e00640 1 -- 172.21.15.167:0/2024767475 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f140415e880 msgr2=0x7f13ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:02.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.032+0000 7f1409800640 1 --2- 172.21.15.167:0/2024767475 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f140415e880 0x7f13ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:02.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.032+0000 7f1403e00640 1 --2- 172.21.15.167:0/2024767475 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f140415e880 0x7f13ec0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:02.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.032+0000 7f1403e00640 1 -- 172.21.15.167:0/2024767475 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13ec003a00 msgr2=0x7f13ec003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:02.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.032+0000 7f1403e00640 1 --2- 172.21.15.167:0/2024767475 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13ec003a00 0x7f13ec003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:02.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.032+0000 7f1403e00640 1 -- 172.21.15.167:0/2024767475 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13ec003520 con 0x7f13ec004800 2024-05-11T12:15:02.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.033+0000 7f1403e00640 1 --2- 172.21.15.167:0/2024767475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13ec004800 0x7f13ec004c80 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f13f0009af0 tx=0x7f13ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ffa48e151a6e553 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:02.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.034+0000 7f1403400640 1 -- 172.21.15.167:0/2024767475 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f0036070 con 0x7f13ec004800 2024-05-11T12:15:02.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.034+0000 7f1403400640 1 -- 172.21.15.167:0/2024767475 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13f0031400 con 0x7f13ec004800 2024-05-11T12:15:02.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.034+0000 7f1403400640 1 -- 172.21.15.167:0/2024767475 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f0009690 con 0x7f13ec004800 2024-05-11T12:15:02.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.034+0000 7f140b800640 1 -- 172.21.15.167:0/2024767475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13ec004800 msgr2=0x7f13ec004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:02.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.034+0000 7f140b800640 1 --2- 172.21.15.167:0/2024767475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13ec004800 0x7f13ec004c80 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f13f0009af0 tx=0x7f13ec005830 comp rx=0 tx=0).stop 2024-05-11T12:15:02.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.034+0000 7f1409800640 1 -- 172.21.15.167:0/2024767475 reap_dead start 2024-05-11T12:15:02.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.034+0000 7f140b800640 1 -- 172.21.15.167:0/2024767475 shutdown_connections 2024-05-11T12:15:02.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.034+0000 7f140b800640 1 -- 172.21.15.167:0/2024767475 >> 172.21.15.167:0/2024767475 conn(0x7f14040b2cb0 msgr2=0x7f14040b3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:02.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.034+0000 7f140b800640 1 -- 172.21.15.167:0/2024767475 shutdown_connections 2024-05-11T12:15:02.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.035+0000 7f140b800640 1 -- 172.21.15.167:0/2024767475 wait complete. 2024-05-11T12:15:02.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.036+0000 7f140b800640 1 Processor -- start 2024-05-11T12:15:02.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.036+0000 7f140b800640 1 -- start start 2024-05-11T12:15:02.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.037+0000 7f140b800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13ec004800 0x7f1404157d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:02.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.037+0000 7f140b800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f140414ba30 0x7f1404144800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:02.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.037+0000 7f140b800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1404158a30 0x7f1404158ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:02.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.037+0000 7f140b800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f14040743c0 con 0x7f140414ba30 2024-05-11T12:15:02.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.037+0000 7f140b800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1404072dc0 con 0x7f13ec004800 2024-05-11T12:15:02.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.037+0000 7f140b800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f14041579c0 con 0x7f1404158a30 2024-05-11T12:15:02.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.038+0000 7f1403e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f140414ba30 0x7f1404144800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:02.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.038+0000 7f1403e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f140414ba30 0x7f1404144800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:38970/0 (socket says 172.21.15.167:38970) 2024-05-11T12:15:02.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.038+0000 7f1403e00640 1 -- 172.21.15.167:0/3830610375 learned_addr learned my addr 172.21.15.167:0/3830610375 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:02.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.038+0000 7f1408e00640 1 --2- 172.21.15.167:0/3830610375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13ec004800 0x7f1404157d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:02.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.038+0000 7f1403e00640 1 -- 172.21.15.167:0/3830610375 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1404158a30 msgr2=0x7f1404158ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:02.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.038+0000 7f1403e00640 1 --2- 172.21.15.167:0/3830610375 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1404158a30 0x7f1404158ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:02.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.038+0000 7f1403e00640 1 -- 172.21.15.167:0/3830610375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13ec004800 msgr2=0x7f1404157d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:02.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.038+0000 7f1403e00640 1 --2- 172.21.15.167:0/3830610375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13ec004800 0x7f1404157d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:02.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.038+0000 7f1403e00640 1 -- 172.21.15.167:0/3830610375 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f140415cc20 con 0x7f140414ba30 2024-05-11T12:15:02.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.038+0000 7f1403e00640 1 --2- 172.21.15.167:0/3830610375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f140414ba30 0x7f1404144800 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f13f0034040 tx=0x7f13f0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:02.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.039+0000 7f1400c00640 1 -- 172.21.15.167:0/3830610375 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f0040020 con 0x7f140414ba30 2024-05-11T12:15:02.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.039+0000 7f1400c00640 1 -- 172.21.15.167:0/3830610375 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13f0003810 con 0x7f140414ba30 2024-05-11T12:15:02.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.039+0000 7f1400c00640 1 -- 172.21.15.167:0/3830610375 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f0036040 con 0x7f140414ba30 2024-05-11T12:15:02.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.039+0000 7f140b800640 1 -- 172.21.15.167:0/3830610375 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f140415be60 con 0x7f140414ba30 2024-05-11T12:15:02.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.039+0000 7f140b800640 1 -- 172.21.15.167:0/3830610375 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f140415c420 con 0x7f140414ba30 2024-05-11T12:15:02.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.040+0000 7f140b800640 1 -- 172.21.15.167:0/3830610375 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1404069820 con 0x7f140414ba30 2024-05-11T12:15:02.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.041+0000 7f1400c00640 1 -- 172.21.15.167:0/3830610375 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f13f0043020 con 0x7f140414ba30 2024-05-11T12:15:02.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.042+0000 7f1400c00640 1 --2- 172.21.15.167:0/3830610375 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f13e40762d0 0x7f13e4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:02.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.042+0000 7f1400c00640 1 -- 172.21.15.167:0/3830610375 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6187+0+0 (secure 0 0 0) 0x7f13f00b79c0 con 0x7f140414ba30 2024-05-11T12:15:02.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.043+0000 7f1408e00640 1 --2- 172.21.15.167:0/3830610375 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f13e40762d0 0x7f13e4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:02.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.043+0000 7f1408e00640 1 --2- 172.21.15.167:0/3830610375 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f13e40762d0 0x7f13e4078790 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f13f8001240 tx=0x7f13f8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:02.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.047+0000 7f1400c00640 1 -- 172.21.15.167:0/3830610375 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f13f004a030 con 0x7f140414ba30 2024-05-11T12:15:02.295 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.295+0000 7f140b800640 1 -- 172.21.15.167:0/3830610375 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f14041544f0 con 0x7f140414ba30 2024-05-11T12:15:02.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.297+0000 7f1400c00640 1 -- 172.21.15.167:0/3830610375 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+5213 (secure 0 0 0) 0x7f13f00044e0 con 0x7f140414ba30 2024-05-11T12:15:02.297 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:15:02.297 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":71,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5401,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/2213070102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2213070102},{"type":"v1","addr":"172.21.15.204:6833","nonce":2213070102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5410,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1480900470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1480900470},{"type":"v1","addr":"172.21.15.204:6835","nonce":1480900470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:14:33.067226+0000","modified":"2024-05-11T12:15:01.821691+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6885,"mds_1":6870},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6870":{"gid":6870,"name":"a","rank":1,"incarnation":71,"state":"up:creating","state_seq":1,"addr":"172.21.15.167:6835/3735016882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3735016882},{"type":"v1","addr":"172.21.15.167:6835","nonce":3735016882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6885":{"gid":6885,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6837/2901182170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2901182170},{"type":"v1","addr":"172.21.15.167:6837","nonce":2901182170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6885,"qdb_cluster":[6885]},"id":7}]} 2024-05-11T12:15:02.297 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 71 2024-05-11T12:15:02.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.299+0000 7f140b800640 1 -- 172.21.15.167:0/3830610375 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f13e40762d0 msgr2=0x7f13e4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:02.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.299+0000 7f140b800640 1 --2- 172.21.15.167:0/3830610375 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f13e40762d0 0x7f13e4078790 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f13f8001240 tx=0x7f13f8009040 comp rx=0 tx=0).stop 2024-05-11T12:15:02.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.299+0000 7f140b800640 1 -- 172.21.15.167:0/3830610375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f140414ba30 msgr2=0x7f1404144800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:02.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.299+0000 7f140b800640 1 --2- 172.21.15.167:0/3830610375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f140414ba30 0x7f1404144800 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f13f0034040 tx=0x7f13f0004730 comp rx=0 tx=0).stop 2024-05-11T12:15:02.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.299+0000 7f140b800640 1 -- 172.21.15.167:0/3830610375 shutdown_connections 2024-05-11T12:15:02.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.299+0000 7f140b800640 1 --2- 172.21.15.167:0/3830610375 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1404158a30 0x7f1404158ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:02.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.299+0000 7f140b800640 1 --2- 172.21.15.167:0/3830610375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f140414ba30 0x7f1404144800 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:02.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.299+0000 7f140b800640 1 --2- 172.21.15.167:0/3830610375 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f13e40762d0 0x7f13e4078790 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:02.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.299+0000 7f140b800640 1 --2- 172.21.15.167:0/3830610375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13ec004800 0x7f1404157d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:02.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.299+0000 7f140b800640 1 -- 172.21.15.167:0/3830610375 >> 172.21.15.167:0/3830610375 conn(0x7f14040b2cb0 msgr2=0x7f140415daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:02.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.300+0000 7f140b800640 1 -- 172.21.15.167:0/3830610375 shutdown_connections 2024-05-11T12:15:02.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:02.300+0000 7f140b800640 1 -- 172.21.15.167:0/3830610375 wait complete. 2024-05-11T12:15:02.725 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 71, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-11T12:14:33.067226+0000', 'modified': '2024-05-11T12:15:01.821691+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 100, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 6885, 'mds_1': 6870}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6870': {'gid': 6870, 'name': 'a', 'rank': 1, 'incarnation': 71, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.167:6835/3735016882', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6834', 'nonce': 3735016882}, {'type': 'v1', 'addr': '172.21.15.167:6835', 'nonce': 3735016882}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_6885': {'gid': 6885, 'name': 'c', 'rank': 0, 'incarnation': 66, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.167:6837/2901182170', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6836', 'nonce': 2901182170}, {'type': 'v1', 'addr': '172.21.15.167:6837', 'nonce': 2901182170}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6885, 'qdb_cluster': [6885]} 2024-05-11T12:15:02.725 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6870:up:creating 2024-05-11T12:15:03.726 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:15:03.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.937+0000 7f5936800640 1 Processor -- start 2024-05-11T12:15:03.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.938+0000 7f5936800640 1 -- start start 2024-05-11T12:15:03.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.939+0000 7f5936800640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f59300b62c0 0x7f59300b66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:03.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.939+0000 7f5936800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f593015e880 0x7f593015ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:03.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.939+0000 7f5936800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59300b6be0 con 0x7f5930073a00 2024-05-11T12:15:03.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.939+0000 7f5936800640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f59300732e0 con 0x7f59300b62c0 2024-05-11T12:15:03.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.939+0000 7f5936800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f59300722d0 con 0x7f593015e880 2024-05-11T12:15:03.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.940+0000 7f592f400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f59300b62c0 0x7f59300b66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:03.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.940+0000 7f592f400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f59300b62c0 0x7f59300b66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:38980/0 (socket says 172.21.15.167:38980) 2024-05-11T12:15:03.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.940+0000 7f592f400640 1 -- 172.21.15.167:0/1055784109 learned_addr learned my addr 172.21.15.167:0/1055784109 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:03.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.940+0000 7f592f400640 1 -- 172.21.15.167:0/1055784109 >> v2:172.21.15.204:3300/0 conn(0x7f593015e880 msgr2=0x7f593015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:03.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.940+0000 7f592fe00640 1 --2- 172.21.15.167:0/1055784109 >> v2:172.21.15.204:3300/0 conn(0x7f593015e880 0x7f593015ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:03.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.940+0000 7f592f400640 1 --2- 172.21.15.167:0/1055784109 >> v2:172.21.15.204:3300/0 conn(0x7f593015e880 0x7f593015ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:03.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.940+0000 7f592f400640 1 -- 172.21.15.167:0/1055784109 >> v1:172.21.15.204:6789/0 conn(0x7f5930073a00 legacy=0x7f5930073e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:03.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.940+0000 7f592f400640 1 -- 172.21.15.167:0/1055784109 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5930072520 con 0x7f59300b62c0 2024-05-11T12:15:03.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.940+0000 7f592fe00640 1 --2- 172.21.15.167:0/1055784109 >> v2:172.21.15.204:3300/0 conn(0x7f593015e880 0x7f593015ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:15:03.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.941+0000 7f592f400640 1 --2- 172.21.15.167:0/1055784109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59300b62c0 0x7f59300b66a0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f593015b290 tx=0x7f592002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7982111b2e5bc955 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:03.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.942+0000 7f592ea00640 1 -- 172.21.15.167:0/1055784109 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5920037070 con 0x7f59300b62c0 2024-05-11T12:15:03.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.942+0000 7f592ea00640 1 -- 172.21.15.167:0/1055784109 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5920032440 con 0x7f59300b62c0 2024-05-11T12:15:03.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.942+0000 7f592ea00640 1 -- 172.21.15.167:0/1055784109 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5920032810 con 0x7f59300b62c0 2024-05-11T12:15:03.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.942+0000 7f5936800640 1 -- 172.21.15.167:0/1055784109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59300b62c0 msgr2=0x7f59300b66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:03.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.942+0000 7f5936800640 1 --2- 172.21.15.167:0/1055784109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59300b62c0 0x7f59300b66a0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f593015b290 tx=0x7f592002a140 comp rx=0 tx=0).stop 2024-05-11T12:15:03.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.942+0000 7f5936800640 1 -- 172.21.15.167:0/1055784109 shutdown_connections 2024-05-11T12:15:03.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.942+0000 7f5936800640 1 --2- 172.21.15.167:0/1055784109 >> v2:172.21.15.204:3300/0 conn(0x7f593015e880 0x7f593015ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:03.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.942+0000 7f5936800640 1 --2- 172.21.15.167:0/1055784109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59300b62c0 0x7f59300b66a0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:03.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.942+0000 7f5936800640 1 -- 172.21.15.167:0/1055784109 >> 172.21.15.167:0/1055784109 conn(0x7f59300b2cb0 msgr2=0x7f59300b3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:03.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.943+0000 7f5936800640 1 -- 172.21.15.167:0/1055784109 shutdown_connections 2024-05-11T12:15:03.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.943+0000 7f5936800640 1 -- 172.21.15.167:0/1055784109 wait complete. 2024-05-11T12:15:03.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.944+0000 7f5936800640 1 Processor -- start 2024-05-11T12:15:03.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.944+0000 7f5936800640 1 -- start start 2024-05-11T12:15:03.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.945+0000 7f5936800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5930073a00 0x7f59301406b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:03.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.945+0000 7f5936800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59300b62c0 0x7f59300aa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:03.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.945+0000 7f5936800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f593015e880 0x7f593015c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:03.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.945+0000 7f5936800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5930072dc0 con 0x7f59300b62c0 2024-05-11T12:15:03.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.945+0000 7f5936800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5930072ae0 con 0x7f5930073a00 2024-05-11T12:15:03.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.945+0000 7f5936800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f59301579c0 con 0x7f593015e880 2024-05-11T12:15:03.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.946+0000 7f5927e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59300b62c0 0x7f59300aa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:03.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.946+0000 7f5927e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59300b62c0 0x7f59300aa4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:38990/0 (socket says 172.21.15.167:38990) 2024-05-11T12:15:03.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.946+0000 7f5927e00640 1 -- 172.21.15.167:0/982037476 learned_addr learned my addr 172.21.15.167:0/982037476 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:03.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.946+0000 7f592f400640 1 --2- 172.21.15.167:0/982037476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5930073a00 0x7f59301406b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:03.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.946+0000 7f5927e00640 1 -- 172.21.15.167:0/982037476 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f593015e880 msgr2=0x7f593015c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:03.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.946+0000 7f5927e00640 1 --2- 172.21.15.167:0/982037476 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f593015e880 0x7f593015c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:03.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.946+0000 7f5927e00640 1 -- 172.21.15.167:0/982037476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5930073a00 msgr2=0x7f59301406b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:03.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.946+0000 7f5927e00640 1 --2- 172.21.15.167:0/982037476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5930073a00 0x7f59301406b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:03.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.946+0000 7f5927e00640 1 -- 172.21.15.167:0/982037476 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59300aaf70 con 0x7f59300b62c0 2024-05-11T12:15:03.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.947+0000 7f5927e00640 1 --2- 172.21.15.167:0/982037476 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59300b62c0 0x7f59300aa4b0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f591800b420 tx=0x7f591800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:03.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.947+0000 7f5927400640 1 -- 172.21.15.167:0/982037476 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5918011830 con 0x7f59300b62c0 2024-05-11T12:15:03.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.947+0000 7f5927400640 1 -- 172.21.15.167:0/982037476 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5918016070 con 0x7f59300b62c0 2024-05-11T12:15:03.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.947+0000 7f5936800640 1 -- 172.21.15.167:0/982037476 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59300ab250 con 0x7f59300b62c0 2024-05-11T12:15:03.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.947+0000 7f5927400640 1 -- 172.21.15.167:0/982037476 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5918011c10 con 0x7f59300b62c0 2024-05-11T12:15:03.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.947+0000 7f5936800640 1 -- 172.21.15.167:0/982037476 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f593014ccf0 con 0x7f59300b62c0 2024-05-11T12:15:03.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.948+0000 7f5936800640 1 -- 172.21.15.167:0/982037476 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5930069820 con 0x7f59300b62c0 2024-05-11T12:15:03.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.949+0000 7f5927400640 1 -- 172.21.15.167:0/982037476 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f591801b460 con 0x7f59300b62c0 2024-05-11T12:15:03.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.951+0000 7f5927400640 1 --2- 172.21.15.167:0/982037476 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f590c0762d0 0x7f590c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:03.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.951+0000 7f5927400640 1 -- 172.21.15.167:0/982037476 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6187+0+0 (secure 0 0 0) 0x7f59180972b0 con 0x7f59300b62c0 2024-05-11T12:15:03.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.951+0000 7f592f400640 1 --2- 172.21.15.167:0/982037476 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f590c0762d0 0x7f590c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:03.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.952+0000 7f592f400640 1 --2- 172.21.15.167:0/982037476 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f590c0762d0 0x7f590c078790 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f5920034000 tx=0x7f592002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:03.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:03.956+0000 7f5927400640 1 -- 172.21.15.167:0/982037476 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5918020020 con 0x7f59300b62c0 2024-05-11T12:15:04.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.205+0000 7f5936800640 1 -- 172.21.15.167:0/982037476 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f59300b3e60 con 0x7f59300b62c0 2024-05-11T12:15:04.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.206+0000 7f5927400640 1 -- 172.21.15.167:0/982037476 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5216 (secure 0 0 0) 0x7f5918060a20 con 0x7f59300b62c0 2024-05-11T12:15:04.207 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:15:04.207 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":72,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5401,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/2213070102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2213070102},{"type":"v1","addr":"172.21.15.204:6833","nonce":2213070102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5410,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1480900470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1480900470},{"type":"v1","addr":"172.21.15.204:6835","nonce":1480900470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:14:33.067226+0000","modified":"2024-05-11T12:15:02.824895+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6885,"mds_1":6870},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6870":{"gid":6870,"name":"a","rank":1,"incarnation":71,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6835/3735016882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3735016882},{"type":"v1","addr":"172.21.15.167:6835","nonce":3735016882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6885":{"gid":6885,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6837/2901182170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2901182170},{"type":"v1","addr":"172.21.15.167:6837","nonce":2901182170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6885,"qdb_cluster":[6870,6885]},"id":7}]} 2024-05-11T12:15:04.207 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 72 2024-05-11T12:15:04.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.209+0000 7f5936800640 1 -- 172.21.15.167:0/982037476 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f590c0762d0 msgr2=0x7f590c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:04.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.209+0000 7f5936800640 1 --2- 172.21.15.167:0/982037476 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f590c0762d0 0x7f590c078790 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f5920034000 tx=0x7f592002e000 comp rx=0 tx=0).stop 2024-05-11T12:15:04.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.209+0000 7f5936800640 1 -- 172.21.15.167:0/982037476 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59300b62c0 msgr2=0x7f59300aa4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:04.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.209+0000 7f5936800640 1 --2- 172.21.15.167:0/982037476 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59300b62c0 0x7f59300aa4b0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f591800b420 tx=0x7f591800b8e0 comp rx=0 tx=0).stop 2024-05-11T12:15:04.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.209+0000 7f5936800640 1 -- 172.21.15.167:0/982037476 shutdown_connections 2024-05-11T12:15:04.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.209+0000 7f5936800640 1 --2- 172.21.15.167:0/982037476 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f593015e880 0x7f593015c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:04.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.209+0000 7f5936800640 1 --2- 172.21.15.167:0/982037476 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59300b62c0 0x7f59300aa4b0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:04.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.209+0000 7f5936800640 1 --2- 172.21.15.167:0/982037476 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f590c0762d0 0x7f590c078790 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:04.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.209+0000 7f5936800640 1 --2- 172.21.15.167:0/982037476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5930073a00 0x7f59301406b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:04.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.209+0000 7f5936800640 1 -- 172.21.15.167:0/982037476 >> 172.21.15.167:0/982037476 conn(0x7f59300b2cb0 msgr2=0x7f5930073e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:04.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.209+0000 7f5936800640 1 -- 172.21.15.167:0/982037476 shutdown_connections 2024-05-11T12:15:04.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.209+0000 7f5936800640 1 -- 172.21.15.167:0/982037476 wait complete. 2024-05-11T12:15:04.629 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 72, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-11T12:14:33.067226+0000', 'modified': '2024-05-11T12:15:02.824895+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 100, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 6885, 'mds_1': 6870}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6870': {'gid': 6870, 'name': 'a', 'rank': 1, 'incarnation': 71, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.167:6835/3735016882', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6834', 'nonce': 3735016882}, {'type': 'v1', 'addr': '172.21.15.167:6835', 'nonce': 3735016882}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_6885': {'gid': 6885, 'name': 'c', 'rank': 0, 'incarnation': 66, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.167:6837/2901182170', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6836', 'nonce': 2901182170}, {'type': 'v1', 'addr': '172.21.15.167:6837', 'nonce': 2901182170}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6885, 'qdb_cluster': [6870, 6885]} 2024-05-11T12:15:04.630 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 2/2 2024-05-11T12:15:04.630 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-11T12:15:04.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.839+0000 7f98a0600640 1 Processor -- start 2024-05-11T12:15:04.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.839+0000 7f98a0600640 1 -- start start 2024-05-11T12:15:04.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.841+0000 7f98a0600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9898148ed0 0x7f98981639a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:04.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.841+0000 7f98a0600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f989815f9a0 0x7f9898070840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:04.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.841+0000 7f98a0600640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f989815fe20 con 0x7f9898070d80 2024-05-11T12:15:04.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.841+0000 7f98a0600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f98981492b0 con 0x7f9898148ed0 2024-05-11T12:15:04.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.841+0000 7f98a0600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f98980b64e0 con 0x7f989815f9a0 2024-05-11T12:15:04.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.841+0000 7f989e600640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f9898070d80 0x7f989806df50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:51870/0 (socket says 172.21.15.167:51870) 2024-05-11T12:15:04.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.841+0000 7f989e600640 1 -- 172.21.15.167:0/3799502496 learned_addr learned my addr 172.21.15.167:0/3799502496 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:15:04.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.842+0000 7f989dc00640 1 --2- 172.21.15.167:0/3799502496 >> v2:172.21.15.204:3300/0 conn(0x7f9898148ed0 0x7f98981639a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:04.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.842+0000 7f989d200640 1 --2- 172.21.15.167:0/3799502496 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f989815f9a0 0x7f9898070840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:04.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.842+0000 7f9897e00640 1 -- 172.21.15.167:0/3799502496 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2535699804 0 0) 0x7f989815fe20 con 0x7f9898070d80 2024-05-11T12:15:04.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.842+0000 7f9897e00640 1 -- 172.21.15.167:0/3799502496 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9880003520 con 0x7f9898070d80 2024-05-11T12:15:04.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.843+0000 7f989d200640 1 -- 172.21.15.167:0/3799502496 >> v2:172.21.15.204:3300/0 conn(0x7f9898148ed0 msgr2=0x7f98981639a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:04.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.843+0000 7f989d200640 1 --2- 172.21.15.167:0/3799502496 >> v2:172.21.15.204:3300/0 conn(0x7f9898148ed0 0x7f98981639a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:04.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.843+0000 7f989d200640 1 -- 172.21.15.167:0/3799502496 >> v1:172.21.15.167:6789/0 conn(0x7f9898070d80 legacy=0x7f989806df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:04.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.843+0000 7f989d200640 1 -- 172.21.15.167:0/3799502496 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98980b67c0 con 0x7f989815f9a0 2024-05-11T12:15:04.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.844+0000 7f989d200640 1 --2- 172.21.15.167:0/3799502496 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f989815f9a0 0x7f9898070840 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f98840097a0 tx=0x7f988402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffb1b834daa636bc server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:04.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.847+0000 7f9897e00640 1 -- 172.21.15.167:0/3799502496 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9884037070 con 0x7f989815f9a0 2024-05-11T12:15:04.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.847+0000 7f9897e00640 1 -- 172.21.15.167:0/3799502496 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9884032440 con 0x7f989815f9a0 2024-05-11T12:15:04.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.847+0000 7f9897e00640 1 -- 172.21.15.167:0/3799502496 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9884032810 con 0x7f989815f9a0 2024-05-11T12:15:04.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.847+0000 7f98a0600640 1 -- 172.21.15.167:0/3799502496 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f989815f9a0 msgr2=0x7f9898070840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:04.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.847+0000 7f98a0600640 1 --2- 172.21.15.167:0/3799502496 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f989815f9a0 0x7f9898070840 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f98840097a0 tx=0x7f988402a180 comp rx=0 tx=0).stop 2024-05-11T12:15:04.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.847+0000 7f98a0600640 1 -- 172.21.15.167:0/3799502496 shutdown_connections 2024-05-11T12:15:04.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.847+0000 7f98a0600640 1 --2- 172.21.15.167:0/3799502496 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f989815f9a0 0x7f9898070840 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:04.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.847+0000 7f98a0600640 1 --2- 172.21.15.167:0/3799502496 >> v2:172.21.15.204:3300/0 conn(0x7f9898148ed0 0x7f98981639a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:04.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.847+0000 7f98a0600640 1 -- 172.21.15.167:0/3799502496 >> 172.21.15.167:0/3799502496 conn(0x7f9898072790 msgr2=0x7f989806c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:04.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.848+0000 7f98a0600640 1 -- 172.21.15.167:0/3799502496 shutdown_connections 2024-05-11T12:15:04.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.848+0000 7f98a0600640 1 -- 172.21.15.167:0/3799502496 wait complete. 2024-05-11T12:15:04.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.849+0000 7f98a0600640 1 Processor -- start 2024-05-11T12:15:04.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.850+0000 7f98a0600640 1 -- start start 2024-05-11T12:15:04.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.850+0000 7f98a0600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9898070d80 0x7f98981f5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:04.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.850+0000 7f98a0600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9898148ed0 0x7f9898142a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:04.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.850+0000 7f98a0600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f989815f9a0 0x7f9898142fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:04.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.850+0000 7f98a0600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f98980b6d80 con 0x7f9898148ed0 2024-05-11T12:15:04.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.850+0000 7f98a0600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f989806da30 con 0x7f9898070d80 2024-05-11T12:15:04.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.850+0000 7f98a0600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f98980d3af0 con 0x7f989815f9a0 2024-05-11T12:15:04.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.851+0000 7f989d200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9898148ed0 0x7f9898142a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:04.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.851+0000 7f989d200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9898148ed0 0x7f9898142a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:38996/0 (socket says 172.21.15.167:38996) 2024-05-11T12:15:04.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.851+0000 7f989d200640 1 -- 172.21.15.167:0/3395660833 learned_addr learned my addr 172.21.15.167:0/3395660833 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:04.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.851+0000 7f989dc00640 1 --2- 172.21.15.167:0/3395660833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9898070d80 0x7f98981f5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:04.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.851+0000 7f989e600640 1 --2- 172.21.15.167:0/3395660833 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f989815f9a0 0x7f9898142fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:04.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.851+0000 7f989d200640 1 -- 172.21.15.167:0/3395660833 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f989815f9a0 msgr2=0x7f9898142fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:04.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.851+0000 7f989d200640 1 --2- 172.21.15.167:0/3395660833 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f989815f9a0 0x7f9898142fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:04.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.851+0000 7f989d200640 1 -- 172.21.15.167:0/3395660833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9898070d80 msgr2=0x7f98981f5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:04.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.851+0000 7f989d200640 1 --2- 172.21.15.167:0/3395660833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9898070d80 0x7f98981f5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:04.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.851+0000 7f989d200640 1 -- 172.21.15.167:0/3395660833 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98980b30e0 con 0x7f9898148ed0 2024-05-11T12:15:04.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.852+0000 7f989d200640 1 --2- 172.21.15.167:0/3395660833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9898148ed0 0x7f9898142a80 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f988402a770 tx=0x7f98840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:04.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.852+0000 7f9895600640 1 -- 172.21.15.167:0/3395660833 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9884037050 con 0x7f9898148ed0 2024-05-11T12:15:04.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.852+0000 7f9895600640 1 -- 172.21.15.167:0/3395660833 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9884043070 con 0x7f9898148ed0 2024-05-11T12:15:04.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.852+0000 7f9895600640 1 -- 172.21.15.167:0/3395660833 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988403cb30 con 0x7f9898148ed0 2024-05-11T12:15:04.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.852+0000 7f98a0600640 1 -- 172.21.15.167:0/3395660833 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98980b33c0 con 0x7f9898148ed0 2024-05-11T12:15:04.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.852+0000 7f98a0600640 1 -- 172.21.15.167:0/3395660833 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98980b3980 con 0x7f9898148ed0 2024-05-11T12:15:04.856 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.856+0000 7f9895600640 1 -- 172.21.15.167:0/3395660833 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f988403ccd0 con 0x7f9898148ed0 2024-05-11T12:15:04.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.857+0000 7f9895600640 1 --2- 172.21.15.167:0/3395660833 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f98780762d0 0x7f9878078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:04.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.857+0000 7f9895600640 1 -- 172.21.15.167:0/3395660833 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6187+0+0 (secure 0 0 0) 0x7f98840b8440 con 0x7f9898148ed0 2024-05-11T12:15:04.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.857+0000 7f989dc00640 1 --2- 172.21.15.167:0/3395660833 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f98780762d0 0x7f9878078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:04.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.857+0000 7f98a358b740 1 Processor -- start 2024-05-11T12:15:04.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.857+0000 7f98a358b740 1 -- start start 2024-05-11T12:15:04.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.858+0000 7f989dc00640 1 --2- 172.21.15.167:0/3395660833 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f98780762d0 0x7f9878078790 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9898143de0 tx=0x7f988c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:04.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.858+0000 7f98a358b740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55dc65cb2840 0x55dc65cb2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:04.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.858+0000 7f98a358b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc65cad6d0 0x55dc65cb1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:04.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.858+0000 7f98a358b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc65cb1980 0x55dc65cb1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:04.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.858+0000 7f98a358b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55dc65c16790 con 0x55dc65cb1980 2024-05-11T12:15:04.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.858+0000 7f98a358b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55dc65bbbf20 con 0x55dc65cad6d0 2024-05-11T12:15:04.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.858+0000 7f98a358b740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55dc65c12750 con 0x55dc65cb2840 2024-05-11T12:15:04.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.859+0000 7f989e600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc65cb1980 0x55dc65cb1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:04.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.859+0000 7f989e600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc65cb1980 0x55dc65cb1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39012/0 (socket says 172.21.15.167:39012) 2024-05-11T12:15:04.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.859+0000 7f989e600640 1 -- 172.21.15.167:0/1525757599 learned_addr learned my addr 172.21.15.167:0/1525757599 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:04.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.859+0000 7f989d200640 1 --2- 172.21.15.167:0/1525757599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc65cad6d0 0x55dc65cb1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:04.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.859+0000 7f989dc00640 1 --2- 172.21.15.167:0/1525757599 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55dc65cb2840 0x55dc65cb2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:04.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.859+0000 7f989e600640 1 -- 172.21.15.167:0/1525757599 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55dc65cb2840 msgr2=0x55dc65cb2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:04.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.859+0000 7f989e600640 1 --2- 172.21.15.167:0/1525757599 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55dc65cb2840 0x55dc65cb2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:04.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.859+0000 7f989e600640 1 -- 172.21.15.167:0/1525757599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc65cad6d0 msgr2=0x55dc65cb1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:04.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.859+0000 7f989e600640 1 --2- 172.21.15.167:0/1525757599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc65cad6d0 0x55dc65cb1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:04.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.859+0000 7f989e600640 1 -- 172.21.15.167:0/1525757599 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc659282d0 con 0x55dc65cb1980 2024-05-11T12:15:04.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.860+0000 7f989e600640 1 --2- 172.21.15.167:0/1525757599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc65cb1980 0x55dc65cb1dc0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f9890002140 tx=0x7f989000d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0d5f073f7e54e0d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:04.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.860+0000 7f988a000640 1 -- 172.21.15.167:0/1525757599 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9890018070 con 0x55dc65cb1980 2024-05-11T12:15:04.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.860+0000 7f988a000640 1 -- 172.21.15.167:0/1525757599 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9890013670 con 0x55dc65cb1980 2024-05-11T12:15:04.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.860+0000 7f988a000640 1 -- 172.21.15.167:0/1525757599 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9890013a60 con 0x55dc65cb1980 2024-05-11T12:15:04.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.861+0000 7f98a358b740 1 -- 172.21.15.167:0/1525757599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc65cb1980 msgr2=0x55dc65cb1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:04.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.861+0000 7f98a358b740 1 --2- 172.21.15.167:0/1525757599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc65cb1980 0x55dc65cb1dc0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f9890002140 tx=0x7f989000d520 comp rx=0 tx=0).stop 2024-05-11T12:15:04.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.861+0000 7f98a358b740 1 -- 172.21.15.167:0/1525757599 shutdown_connections 2024-05-11T12:15:04.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.861+0000 7f98a358b740 1 --2- 172.21.15.167:0/1525757599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc65cb1980 0x55dc65cb1dc0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:04.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.861+0000 7f98a358b740 1 --2- 172.21.15.167:0/1525757599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc65cad6d0 0x55dc65cb1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:04.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.861+0000 7f98a358b740 1 --2- 172.21.15.167:0/1525757599 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55dc65cb2840 0x55dc65cb2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:04.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.861+0000 7f98a358b740 1 -- 172.21.15.167:0/1525757599 >> 172.21.15.167:0/1525757599 conn(0x55dc65caf940 msgr2=0x55dc65cf8400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:04.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.861+0000 7f98a358b740 1 -- 172.21.15.167:0/1525757599 shutdown_connections 2024-05-11T12:15:04.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.861+0000 7f98a358b740 1 -- 172.21.15.167:0/1525757599 wait complete. 2024-05-11T12:15:04.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.862+0000 7f98a358b740 1 -- start start 2024-05-11T12:15:04.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.862+0000 7f98a358b740 1 Processor -- start 2024-05-11T12:15:04.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.863+0000 7f98a358b740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55dc65cb1980 0x55dc65ccadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:04.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.863+0000 7f98a358b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc65cb2840 0x55dc65ccbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:04.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.863+0000 7f98a358b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc65ccdf70 0x55dc65ccc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:04.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.863+0000 7f98a358b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55dc65cde3d0 con 0x55dc65cb2840 2024-05-11T12:15:04.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.863+0000 7f98a358b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55dc65bcf9c0 con 0x55dc65ccdf70 2024-05-11T12:15:04.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.863+0000 7f98a358b740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55dc65cdd850 con 0x55dc65cb1980 2024-05-11T12:15:04.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.863+0000 7f98a358b740 10 client.0 _ll_register_callbacks cb 0x55dc65b9c120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:15:04.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.863+0000 7f989d200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc65cb2840 0x55dc65ccbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:04.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.863+0000 7f989d200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc65cb2840 0x55dc65ccbed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39014/0 (socket says 172.21.15.167:39014) 2024-05-11T12:15:04.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.863+0000 7f989d200640 1 -- 172.21.15.167:0/1873157148 learned_addr learned my addr 172.21.15.167:0/1873157148 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:04.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.863+0000 7f989d200640 1 -- 172.21.15.167:0/1873157148 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55dc65cb1980 msgr2=0x55dc65ccadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:04.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.863+0000 7f989d200640 1 --2- 172.21.15.167:0/1873157148 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55dc65cb1980 0x55dc65ccadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:04.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.863+0000 7f989d200640 1 -- 172.21.15.167:0/1873157148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc65ccdf70 msgr2=0x55dc65ccc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:04.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.864+0000 7f989e600640 1 --2- 172.21.15.167:0/1873157148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc65ccdf70 0x55dc65ccc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:04.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.864+0000 7f989d200640 1 --2- 172.21.15.167:0/1873157148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc65ccdf70 0x55dc65ccc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:04.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.864+0000 7f989d200640 1 -- 172.21.15.167:0/1873157148 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc659f26b0 con 0x55dc65cb2840 2024-05-11T12:15:04.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.864+0000 7f989d200640 1 --2- 172.21.15.167:0/1873157148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc65cb2840 0x55dc65ccbed0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f9884081440 tx=0x7f9884081280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=407e4a9f44b237b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:04.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.864+0000 7f987ea00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-11T12:15:04.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.864+0000 7f987ea00640 1 -- 172.21.15.167:0/1873157148 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988409a020 con 0x55dc65cb2840 2024-05-11T12:15:04.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.864+0000 7f987ea00640 1 -- 172.21.15.167:0/1873157148 <== mon.0 v2:172.21.15.167:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 6187+0+0 (secure 0 0 0) 0x7f9884088b50 con 0x55dc65cb2840 2024-05-11T12:15:04.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.864+0000 7f98a358b740 1 -- 172.21.15.167:0/1873157148 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9890013670 con 0x55dc65cb2840 2024-05-11T12:15:04.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.864+0000 7f987ea00640 1 -- 172.21.15.167:0/1873157148 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f9884080fc0 con 0x55dc65cb2840 2024-05-11T12:15:04.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.864+0000 7f98a358b740 10 client.6978 fetch_fsmap learned FSMap version 72 2024-05-11T12:15:04.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.864+0000 7f98a358b740 1 -- 172.21.15.167:0/1873157148 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) -- 0x55dc65cfaf80 con 0x55dc65cb2840 2024-05-11T12:15:04.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.865+0000 7f987ea00640 1 -- 172.21.15.167:0/1873157148 <== mon.0 v2:172.21.15.167:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9890013670 con 0x55dc65cb2840 2024-05-11T12:15:04.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.865+0000 7f987ea00640 1 -- 172.21.15.167:0/1873157148 <== mon.0 v2:172.21.15.167:3300/0 5 ==== fsmap(e 72) ==== 3514+0+0 (secure 0 0 0) 0x7f9884080710 con 0x55dc65cb2840 2024-05-11T12:15:04.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.865+0000 7f98a358b740 10 client.6978 fetch_fsmap finished waiting for FSMap version 72 2024-05-11T12:15:04.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.865+0000 7f98a358b740 10 client.6978 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-11T12:15:04.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.865+0000 7f98a358b740 20 client.6978 populate_metadata read hostname 'smithi167' 2024-05-11T12:15:04.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.865+0000 7f98a358b740 1 --2- 172.21.15.167:0/1873157148 >> [v2:172.21.15.167:6834/3735016882,v1:172.21.15.167:6835/3735016882] conn(0x55dc65cd0a80 0x55dc65cd0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:04.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.865+0000 7f98a358b740 4 client.6978 mds_command: new command op to 6870 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:15:04.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.865+0000 7f98a358b740 1 -- 172.21.15.167:0/1873157148 --> [v2:172.21.15.167:6834/3735016882,v1:172.21.15.167:6835/3735016882] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc65c243a0 con 0x55dc65cd0a80 2024-05-11T12:15:04.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.865+0000 7f989dc00640 1 --2- 172.21.15.167:0/1873157148 >> [v2:172.21.15.167:6834/3735016882,v1:172.21.15.167:6835/3735016882] conn(0x55dc65cd0a80 0x55dc65cd0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:04.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.866+0000 7f989dc00640 1 --2- 172.21.15.167:0/1873157148 >> [v2:172.21.15.167:6834/3735016882,v1:172.21.15.167:6835/3735016882] conn(0x55dc65cd0a80 0x55dc65cd0e80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a58abebdd9b8d245 server_cookie=7df16d2372a48031 in_seq=0 out_seq=0 2024-05-11T12:15:04.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.866+0000 7f987ea00640 10 client.6978 ms_handle_connect on v2:172.21.15.167:6834/3735016882 2024-05-11T12:15:04.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.866+0000 7f987ea00640 1 -- 172.21.15.167:0/1873157148 <== mds.1 v2:172.21.15.167:6834/3735016882 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dc65c243a0 con 0x55dc65cd0a80 2024-05-11T12:15:04.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.866+0000 7f987ea00640 10 client.6978 handle_command_reply: tid=0 2024-05-11T12:15:04.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.867+0000 7f98a358b740 1 client.6978 shutdown 2024-05-11T12:15:04.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.867+0000 7f98a358b740 1 -- 172.21.15.167:0/1873157148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc65cb2840 msgr2=0x55dc65ccbed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:04.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.867+0000 7f98a358b740 1 --2- 172.21.15.167:0/1873157148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc65cb2840 0x55dc65ccbed0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f9884081440 tx=0x7f9884081280 comp rx=0 tx=0).stop 2024-05-11T12:15:04.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.867+0000 7f98a358b740 1 -- 172.21.15.167:0/1873157148 shutdown_connections 2024-05-11T12:15:04.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.867+0000 7f98a358b740 1 --2- 172.21.15.167:0/1873157148 >> [v2:172.21.15.167:6834/3735016882,v1:172.21.15.167:6835/3735016882] conn(0x55dc65cd0a80 0x55dc65cd0e80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:04.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.867+0000 7f98a358b740 1 --2- 172.21.15.167:0/1873157148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc65ccdf70 0x55dc65ccc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:04.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.867+0000 7f98a358b740 1 --2- 172.21.15.167:0/1873157148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc65cb2840 0x55dc65ccbed0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:04.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.867+0000 7f987ea00640 0 client.6978 ms_handle_reset on v2:172.21.15.167:6834/3735016882 2024-05-11T12:15:04.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.867+0000 7f98a358b740 1 --2- 172.21.15.167:0/1873157148 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55dc65cb1980 0x55dc65ccadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:04.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.867+0000 7f98a358b740 1 -- 172.21.15.167:0/1873157148 >> 172.21.15.167:0/1873157148 conn(0x55dc65cad6d0 msgr2=0x55dc65cacdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:04.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.867+0000 7f98a358b740 1 -- 172.21.15.167:0/1873157148 shutdown_connections 2024-05-11T12:15:04.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.867+0000 7f98a358b740 1 -- 172.21.15.167:0/1873157148 wait complete. 2024-05-11T12:15:04.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.867+0000 7f98a358b740 20 client.6978 trim_cache size 0 max 16384 2024-05-11T12:15:04.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.889+0000 7f98a358b740 1 Processor -- start 2024-05-11T12:15:04.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.890+0000 7f98a358b740 1 -- start start 2024-05-11T12:15:04.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.890+0000 7f98a358b740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55dc65cb1980 0x55dc65d1c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:04.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.890+0000 7f98a358b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc65cb2840 0x55dc65cdda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:04.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.891+0000 7f98a358b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc65cddf80 0x55dc65d71810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:04.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.891+0000 7f98a358b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9898201f40 con 0x55dc65cddf80 2024-05-11T12:15:04.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.891+0000 7f98a358b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9898200340 con 0x55dc65cb2840 2024-05-11T12:15:04.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.891+0000 7f98a358b740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f9898200940 con 0x55dc65cb1980 2024-05-11T12:15:04.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.891+0000 7f989e600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc65cddf80 0x55dc65d71810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:04.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.891+0000 7f989e600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc65cddf80 0x55dc65d71810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39016/0 (socket says 172.21.15.167:39016) 2024-05-11T12:15:04.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.891+0000 7f989e600640 1 -- 172.21.15.167:0/2152615517 learned_addr learned my addr 172.21.15.167:0/2152615517 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:04.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.891+0000 7f989d200640 1 --2- 172.21.15.167:0/2152615517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc65cb2840 0x55dc65cdda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:04.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.891+0000 7f989dc00640 1 --2- 172.21.15.167:0/2152615517 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55dc65cb1980 0x55dc65d1c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:04.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.891+0000 7f989e600640 1 -- 172.21.15.167:0/2152615517 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55dc65cb1980 msgr2=0x55dc65d1c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:04.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.891+0000 7f989e600640 1 --2- 172.21.15.167:0/2152615517 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55dc65cb1980 0x55dc65d1c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:04.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.891+0000 7f989e600640 1 -- 172.21.15.167:0/2152615517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc65cb2840 msgr2=0x55dc65cdda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:04.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.891+0000 7f989e600640 1 --2- 172.21.15.167:0/2152615517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc65cb2840 0x55dc65cdda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:04.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.891+0000 7f989e600640 1 -- 172.21.15.167:0/2152615517 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc65cef240 con 0x55dc65cddf80 2024-05-11T12:15:04.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.892+0000 7f989d200640 1 --2- 172.21.15.167:0/2152615517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc65cb2840 0x55dc65cdda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:15:04.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.892+0000 7f989e600640 1 --2- 172.21.15.167:0/2152615517 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc65cddf80 0x55dc65d71810 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f9890009010 tx=0x7f989000b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6cdd2ea4a70120d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:04.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.892+0000 7f9889600640 1 -- 172.21.15.167:0/2152615517 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9890022070 con 0x55dc65cddf80 2024-05-11T12:15:04.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.892+0000 7f9889600640 1 -- 172.21.15.167:0/2152615517 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9890013470 con 0x55dc65cddf80 2024-05-11T12:15:04.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.892+0000 7f9889600640 1 -- 172.21.15.167:0/2152615517 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9890018040 con 0x55dc65cddf80 2024-05-11T12:15:04.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.893+0000 7f98a358b740 1 -- 172.21.15.167:0/2152615517 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc65cddf80 msgr2=0x55dc65d71810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:04.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.893+0000 7f98a358b740 1 --2- 172.21.15.167:0/2152615517 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc65cddf80 0x55dc65d71810 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f9890009010 tx=0x7f989000b000 comp rx=0 tx=0).stop 2024-05-11T12:15:04.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.893+0000 7f98a358b740 1 -- 172.21.15.167:0/2152615517 shutdown_connections 2024-05-11T12:15:04.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.893+0000 7f98a358b740 1 --2- 172.21.15.167:0/2152615517 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc65cddf80 0x55dc65d71810 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:04.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.893+0000 7f98a358b740 1 --2- 172.21.15.167:0/2152615517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc65cb2840 0x55dc65cdda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:04.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.893+0000 7f98a358b740 1 --2- 172.21.15.167:0/2152615517 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55dc65cb1980 0x55dc65d1c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:04.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.893+0000 7f98a358b740 1 -- 172.21.15.167:0/2152615517 >> 172.21.15.167:0/2152615517 conn(0x55dc65cad6d0 msgr2=0x55dc65ca8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:04.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.893+0000 7f98a358b740 1 -- 172.21.15.167:0/2152615517 shutdown_connections 2024-05-11T12:15:04.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.893+0000 7f98a358b740 1 -- 172.21.15.167:0/2152615517 wait complete. 2024-05-11T12:15:04.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.894+0000 7f98a358b740 1 -- start start 2024-05-11T12:15:04.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.894+0000 7f98a358b740 1 Processor -- start 2024-05-11T12:15:04.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.895+0000 7f98a358b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc65cb1980 0x55dc65cfbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:04.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.895+0000 7f98a358b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc65cb2840 0x55dc65d60690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:04.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.895+0000 7f98a358b740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55dc65cddf80 0x55dc65cacf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:04.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.895+0000 7f98a358b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55dc65ccf1c0 con 0x55dc65cb2840 2024-05-11T12:15:04.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.895+0000 7f98a358b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55dc65cde3d0 con 0x55dc65cb1980 2024-05-11T12:15:04.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.895+0000 7f98a358b740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55dc65cfa580 con 0x55dc65cddf80 2024-05-11T12:15:04.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.895+0000 7f98a358b740 10 client.0 _ll_register_callbacks cb 0x7f9884009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:15:04.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.895+0000 7f989d200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc65cb2840 0x55dc65d60690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:04.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.895+0000 7f989d200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc65cb2840 0x55dc65d60690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39020/0 (socket says 172.21.15.167:39020) 2024-05-11T12:15:04.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.895+0000 7f989d200640 1 -- 172.21.15.167:0/1516850410 learned_addr learned my addr 172.21.15.167:0/1516850410 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:04.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.895+0000 7f989dc00640 1 --2- 172.21.15.167:0/1516850410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc65cb1980 0x55dc65cfbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:04.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.895+0000 7f989e600640 1 --2- 172.21.15.167:0/1516850410 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55dc65cddf80 0x55dc65cacf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:04.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.896+0000 7f989d200640 1 -- 172.21.15.167:0/1516850410 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55dc65cddf80 msgr2=0x55dc65cacf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:04.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.896+0000 7f989d200640 1 --2- 172.21.15.167:0/1516850410 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55dc65cddf80 0x55dc65cacf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:04.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.896+0000 7f989d200640 1 -- 172.21.15.167:0/1516850410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc65cb1980 msgr2=0x55dc65cfbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:04.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.896+0000 7f989d200640 1 --2- 172.21.15.167:0/1516850410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc65cb1980 0x55dc65cfbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:04.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.896+0000 7f989d200640 1 -- 172.21.15.167:0/1516850410 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc65cad4b0 con 0x55dc65cb2840 2024-05-11T12:15:04.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.896+0000 7f989dc00640 1 --2- 172.21.15.167:0/1516850410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc65cb1980 0x55dc65cfbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:15:04.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.896+0000 7f989d200640 1 --2- 172.21.15.167:0/1516850410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc65cb2840 0x55dc65d60690 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f988403d040 tx=0x7f98840b6650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c19303c32f6f366a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:04.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.896+0000 7f987ea00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-11T12:15:04.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.896+0000 7f987ea00640 1 -- 172.21.15.167:0/1516850410 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988409a070 con 0x55dc65cb2840 2024-05-11T12:15:04.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.896+0000 7f987ea00640 1 -- 172.21.15.167:0/1516850410 <== mon.0 v2:172.21.15.167:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 6187+0+0 (secure 0 0 0) 0x7f98840a2b90 con 0x55dc65cb2840 2024-05-11T12:15:04.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.896+0000 7f98a358b740 1 -- 172.21.15.167:0/1516850410 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9890013470 con 0x55dc65cb2840 2024-05-11T12:15:04.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.896+0000 7f987ea00640 1 -- 172.21.15.167:0/1516850410 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f988408f050 con 0x55dc65cb2840 2024-05-11T12:15:04.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.896+0000 7f98a358b740 10 client.6984 fetch_fsmap learned FSMap version 72 2024-05-11T12:15:04.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.896+0000 7f98a358b740 1 -- 172.21.15.167:0/1516850410 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) -- 0x55dc65cd5cc0 con 0x55dc65cb2840 2024-05-11T12:15:04.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.897+0000 7f987ea00640 1 -- 172.21.15.167:0/1516850410 <== mon.0 v2:172.21.15.167:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9890013470 con 0x55dc65cb2840 2024-05-11T12:15:04.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.897+0000 7f987ea00640 1 -- 172.21.15.167:0/1516850410 <== mon.0 v2:172.21.15.167:3300/0 5 ==== fsmap(e 72) ==== 3514+0+0 (secure 0 0 0) 0x7f9884092020 con 0x55dc65cb2840 2024-05-11T12:15:04.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.897+0000 7f98a358b740 10 client.6984 fetch_fsmap finished waiting for FSMap version 72 2024-05-11T12:15:04.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.897+0000 7f98a358b740 10 client.6984 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-11T12:15:04.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.897+0000 7f98a358b740 20 client.6984 populate_metadata read hostname 'smithi167' 2024-05-11T12:15:04.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.897+0000 7f98a358b740 1 --2- 172.21.15.167:0/1516850410 >> [v2:172.21.15.167:6834/3735016882,v1:172.21.15.167:6835/3735016882] conn(0x55dc65ccfa10 0x55dc65cf81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:04.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.897+0000 7f98a358b740 4 client.6984 mds_command: new command op to 6870 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-11T12:15:04.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.897+0000 7f98a358b740 1 -- 172.21.15.167:0/1516850410 --> [v2:172.21.15.167:6834/3735016882,v1:172.21.15.167:6835/3735016882] -- command(tid 0: {"prefix": "status"}) -- 0x55dc65c1e210 con 0x55dc65ccfa10 2024-05-11T12:15:04.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.897+0000 7f989dc00640 1 --2- 172.21.15.167:0/1516850410 >> [v2:172.21.15.167:6834/3735016882,v1:172.21.15.167:6835/3735016882] conn(0x55dc65ccfa10 0x55dc65cf81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:04.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.898+0000 7f989dc00640 1 --2- 172.21.15.167:0/1516850410 >> [v2:172.21.15.167:6834/3735016882,v1:172.21.15.167:6835/3735016882] conn(0x55dc65ccfa10 0x55dc65cf81b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f42dcef8062a884a server_cookie=dc4ed848877b10ed in_seq=0 out_seq=0 2024-05-11T12:15:04.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.898+0000 7f987ea00640 10 client.6984 ms_handle_connect on v2:172.21.15.167:6834/3735016882 2024-05-11T12:15:04.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.898+0000 7f987ea00640 1 -- 172.21.15.167:0/1516850410 <== mds.1 v2:172.21.15.167:6834/3735016882 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55dc65c1e210 con 0x55dc65ccfa10 2024-05-11T12:15:04.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.898+0000 7f987ea00640 10 client.6984 handle_command_reply: tid=0 2024-05-11T12:15:04.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.898+0000 7f98a358b740 1 client.6984 shutdown 2024-05-11T12:15:04.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.898+0000 7f98a358b740 1 -- 172.21.15.167:0/1516850410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc65cb2840 msgr2=0x55dc65d60690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:04.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.898+0000 7f98a358b740 1 --2- 172.21.15.167:0/1516850410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc65cb2840 0x55dc65d60690 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f988403d040 tx=0x7f98840b6650 comp rx=0 tx=0).stop 2024-05-11T12:15:04.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.898+0000 7f98a358b740 1 -- 172.21.15.167:0/1516850410 shutdown_connections 2024-05-11T12:15:04.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.898+0000 7f98a358b740 1 --2- 172.21.15.167:0/1516850410 >> [v2:172.21.15.167:6834/3735016882,v1:172.21.15.167:6835/3735016882] conn(0x55dc65ccfa10 0x55dc65cf81b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:04.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.898+0000 7f98a358b740 1 --2- 172.21.15.167:0/1516850410 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55dc65cddf80 0x55dc65cacf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:04.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.898+0000 7f98a358b740 1 --2- 172.21.15.167:0/1516850410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc65cb2840 0x55dc65d60690 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:04.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.898+0000 7f987ea00640 0 client.6984 ms_handle_reset on v2:172.21.15.167:6834/3735016882 2024-05-11T12:15:04.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.898+0000 7f98a358b740 1 --2- 172.21.15.167:0/1516850410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc65cb1980 0x55dc65cfbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:04.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.898+0000 7f98a358b740 1 -- 172.21.15.167:0/1516850410 >> 172.21.15.167:0/1516850410 conn(0x55dc65cad6d0 msgr2=0x55dc65cfb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:04.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.898+0000 7f98a358b740 1 -- 172.21.15.167:0/1516850410 shutdown_connections 2024-05-11T12:15:04.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.898+0000 7f98a358b740 1 -- 172.21.15.167:0/1516850410 wait complete. 2024-05-11T12:15:04.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.898+0000 7f98a358b740 20 client.6984 trim_cache size 0 max 16384 2024-05-11T12:15:04.903 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T12:15:04.903 INFO:teuthology.orchestra.run.smithi167.stdout: "cluster_fsid": "5f32d344-7289-4c27-881b-1533419ab678", 2024-05-11T12:15:04.904 INFO:teuthology.orchestra.run.smithi167.stdout: "whoami": 1, 2024-05-11T12:15:04.904 INFO:teuthology.orchestra.run.smithi167.stdout: "id": 6870, 2024-05-11T12:15:04.904 INFO:teuthology.orchestra.run.smithi167.stdout: "want_state": "up:active", 2024-05-11T12:15:04.904 INFO:teuthology.orchestra.run.smithi167.stdout: "state": "up:active", 2024-05-11T12:15:04.904 INFO:teuthology.orchestra.run.smithi167.stdout: "fs_name": "cephfs", 2024-05-11T12:15:04.904 INFO:teuthology.orchestra.run.smithi167.stdout: "rank_uptime": 3.0745227740000001, 2024-05-11T12:15:04.904 INFO:teuthology.orchestra.run.smithi167.stdout: "mdsmap_epoch": 72, 2024-05-11T12:15:04.904 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_epoch": 101, 2024-05-11T12:15:04.904 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_epoch_barrier": 101, 2024-05-11T12:15:04.904 INFO:teuthology.orchestra.run.smithi167.stdout: "uptime": 14.018677024 2024-05-11T12:15:04.904 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T12:15:04.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.899+0000 7f98a0600640 1 -- 172.21.15.167:0/3395660833 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f98780762d0 msgr2=0x7f9878078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:04.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.899+0000 7f98a0600640 1 --2- 172.21.15.167:0/3395660833 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f98780762d0 0x7f9878078790 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9898143de0 tx=0x7f988c008040 comp rx=0 tx=0).stop 2024-05-11T12:15:04.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.899+0000 7f98a0600640 1 -- 172.21.15.167:0/3395660833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9898148ed0 msgr2=0x7f9898142a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:04.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.899+0000 7f98a0600640 1 --2- 172.21.15.167:0/3395660833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9898148ed0 0x7f9898142a80 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f988402a770 tx=0x7f98840046c0 comp rx=0 tx=0).stop 2024-05-11T12:15:04.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.899+0000 7f98a0600640 1 -- 172.21.15.167:0/3395660833 shutdown_connections 2024-05-11T12:15:04.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.899+0000 7f98a0600640 1 --2- 172.21.15.167:0/3395660833 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f989815f9a0 0x7f9898142fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:04.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.899+0000 7f98a0600640 1 --2- 172.21.15.167:0/3395660833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9898148ed0 0x7f9898142a80 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:04.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.899+0000 7f98a0600640 1 --2- 172.21.15.167:0/3395660833 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f98780762d0 0x7f9878078790 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:04.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.899+0000 7f98a0600640 1 --2- 172.21.15.167:0/3395660833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9898070d80 0x7f98981f5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:04.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.899+0000 7f98a0600640 1 -- 172.21.15.167:0/3395660833 >> 172.21.15.167:0/3395660833 conn(0x7f9898072790 msgr2=0x7f9898148200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:04.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.899+0000 7f98a0600640 1 -- 172.21.15.167:0/3395660833 shutdown_connections 2024-05-11T12:15:04.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:04.899+0000 7f98a0600640 1 -- 172.21.15.167:0/3395660833 wait complete. 2024-05-11T12:15:04.916 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-11T12:15:05.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.102+0000 7efd45000640 1 Processor -- start 2024-05-11T12:15:05.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.102+0000 7efd45000640 1 -- start start 2024-05-11T12:15:05.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.104+0000 7efd45000640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7efd4015b4d0 0x7efd400ade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:05.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.104+0000 7efd45000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7efd400b47b0 0x7efd40074050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:05.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.104+0000 7efd45000640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd40072420 con 0x7efd40072b10 2024-05-11T12:15:05.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.104+0000 7efd45000640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7efd400b5010 con 0x7efd4015b4d0 2024-05-11T12:15:05.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.104+0000 7efd45000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7efd400b4d30 con 0x7efd400b47b0 2024-05-11T12:15:05.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.104+0000 7efd3f400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7efd4015b4d0 0x7efd400ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:05.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.104+0000 7efd3f400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7efd4015b4d0 0x7efd400ade30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39032/0 (socket says 172.21.15.167:39032) 2024-05-11T12:15:05.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.104+0000 7efd3f400640 1 -- 172.21.15.167:0/1691470093 learned_addr learned my addr 172.21.15.167:0/1691470093 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:05.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.105+0000 7efd37e00640 1 --2- 172.21.15.167:0/1691470093 >> v2:172.21.15.204:3300/0 conn(0x7efd400b47b0 0x7efd40074050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:05.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.105+0000 7efd3f400640 1 -- 172.21.15.167:0/1691470093 >> v2:172.21.15.204:3300/0 conn(0x7efd400b47b0 msgr2=0x7efd40074050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:05.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.105+0000 7efd3f400640 1 --2- 172.21.15.167:0/1691470093 >> v2:172.21.15.204:3300/0 conn(0x7efd400b47b0 0x7efd40074050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:05.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.105+0000 7efd3f400640 1 -- 172.21.15.167:0/1691470093 >> v1:172.21.15.167:6789/0 conn(0x7efd40072b10 legacy=0x7efd4015f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:05.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.105+0000 7efd3f400640 1 -- 172.21.15.167:0/1691470093 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd4006e040 con 0x7efd4015b4d0 2024-05-11T12:15:05.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.106+0000 7efd3f400640 1 --2- 172.21.15.167:0/1691470093 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd4015b4d0 0x7efd400ade30 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7efd4006d150 tx=0x7efd3002c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a34dca0e02fef23 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:05.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.106+0000 7efd3ea00640 1 -- 172.21.15.167:0/1691470093 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd30036020 con 0x7efd4015b4d0 2024-05-11T12:15:05.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.106+0000 7efd3ea00640 1 -- 172.21.15.167:0/1691470093 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd30032640 con 0x7efd4015b4d0 2024-05-11T12:15:05.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.106+0000 7efd3ea00640 1 -- 172.21.15.167:0/1691470093 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd30032a10 con 0x7efd4015b4d0 2024-05-11T12:15:05.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.107+0000 7efd45000640 1 -- 172.21.15.167:0/1691470093 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd4015b4d0 msgr2=0x7efd400ade30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:05.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.107+0000 7efd45000640 1 --2- 172.21.15.167:0/1691470093 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd4015b4d0 0x7efd400ade30 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7efd4006d150 tx=0x7efd3002c940 comp rx=0 tx=0).stop 2024-05-11T12:15:05.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.107+0000 7efd45000640 1 -- 172.21.15.167:0/1691470093 shutdown_connections 2024-05-11T12:15:05.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.107+0000 7efd45000640 1 --2- 172.21.15.167:0/1691470093 >> v2:172.21.15.204:3300/0 conn(0x7efd400b47b0 0x7efd40074050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:05.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.107+0000 7efd45000640 1 --2- 172.21.15.167:0/1691470093 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd4015b4d0 0x7efd400ade30 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:05.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.107+0000 7efd45000640 1 -- 172.21.15.167:0/1691470093 >> 172.21.15.167:0/1691470093 conn(0x7efd400b2df0 msgr2=0x7efd400b3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:05.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.107+0000 7efd45000640 1 -- 172.21.15.167:0/1691470093 shutdown_connections 2024-05-11T12:15:05.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.107+0000 7efd45000640 1 -- 172.21.15.167:0/1691470093 wait complete. 2024-05-11T12:15:05.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.109+0000 7efd45000640 1 Processor -- start 2024-05-11T12:15:05.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.109+0000 7efd45000640 1 -- start start 2024-05-11T12:15:05.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.110+0000 7efd45000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd40072b10 0x7efd400cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:05.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.110+0000 7efd45000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd400b47b0 0x7efd4013aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:05.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.110+0000 7efd45000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd4015b4d0 0x7efd401563d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:05.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.110+0000 7efd45000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efd4015b8b0 con 0x7efd4015b4d0 2024-05-11T12:15:05.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.110+0000 7efd45000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7efd4006e600 con 0x7efd400b47b0 2024-05-11T12:15:05.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.110+0000 7efd45000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7efd4015ba30 con 0x7efd40072b10 2024-05-11T12:15:05.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.110+0000 7efd3fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd4015b4d0 0x7efd401563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:05.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.110+0000 7efd3fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd4015b4d0 0x7efd401563d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39046/0 (socket says 172.21.15.167:39046) 2024-05-11T12:15:05.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.110+0000 7efd3fe00640 1 -- 172.21.15.167:0/2347880752 learned_addr learned my addr 172.21.15.167:0/2347880752 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:05.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.111+0000 7efd37e00640 1 --2- 172.21.15.167:0/2347880752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd400b47b0 0x7efd4013aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:05.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.111+0000 7efd3fe00640 1 -- 172.21.15.167:0/2347880752 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd40072b10 msgr2=0x7efd400cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:05.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.111+0000 7efd3f400640 1 --2- 172.21.15.167:0/2347880752 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd40072b10 0x7efd400cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:05.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.111+0000 7efd3fe00640 1 --2- 172.21.15.167:0/2347880752 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd40072b10 0x7efd400cd5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:05.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.111+0000 7efd3fe00640 1 -- 172.21.15.167:0/2347880752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd400b47b0 msgr2=0x7efd4013aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:05.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.111+0000 7efd3fe00640 1 --2- 172.21.15.167:0/2347880752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd400b47b0 0x7efd4013aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:05.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.111+0000 7efd3fe00640 1 -- 172.21.15.167:0/2347880752 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd400cf3b0 con 0x7efd4015b4d0 2024-05-11T12:15:05.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.111+0000 7efd37e00640 1 --2- 172.21.15.167:0/2347880752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd400b47b0 0x7efd4013aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:15:05.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.111+0000 7efd3fe00640 1 --2- 172.21.15.167:0/2347880752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd4015b4d0 0x7efd401563d0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7efd3800c7a0 tx=0x7efd3800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:05.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.111+0000 7efd37400640 1 -- 172.21.15.167:0/2347880752 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd38003b70 con 0x7efd4015b4d0 2024-05-11T12:15:05.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.111+0000 7efd37400640 1 -- 172.21.15.167:0/2347880752 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd38003d10 con 0x7efd4015b4d0 2024-05-11T12:15:05.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.111+0000 7efd37400640 1 -- 172.21.15.167:0/2347880752 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3800aa70 con 0x7efd4015b4d0 2024-05-11T12:15:05.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.111+0000 7efd45000640 1 -- 172.21.15.167:0/2347880752 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd400cf690 con 0x7efd4015b4d0 2024-05-11T12:15:05.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.112+0000 7efd45000640 1 -- 172.21.15.167:0/2347880752 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd400cff30 con 0x7efd4015b4d0 2024-05-11T12:15:05.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.114+0000 7efd37400640 1 -- 172.21.15.167:0/2347880752 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7efd3800ac10 con 0x7efd4015b4d0 2024-05-11T12:15:05.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.115+0000 7efd37400640 1 --2- 172.21.15.167:0/2347880752 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7efd1c076300 0x7efd1c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:05.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.115+0000 7efd37400640 1 -- 172.21.15.167:0/2347880752 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6187+0+0 (secure 0 0 0) 0x7efd38097810 con 0x7efd4015b4d0 2024-05-11T12:15:05.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.115+0000 7efd3f400640 1 --2- 172.21.15.167:0/2347880752 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7efd1c076300 0x7efd1c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:05.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.115+0000 7efd4940b740 1 Processor -- start 2024-05-11T12:15:05.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.115+0000 7efd4940b740 1 -- start start 2024-05-11T12:15:05.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.116+0000 7efd3f400640 1 --2- 172.21.15.167:0/2347880752 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7efd1c076300 0x7efd1c0787c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7efd30037000 tx=0x7efd30008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:05.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.116+0000 7efd4940b740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55ef74820a10 0x55ef74820df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:05.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.116+0000 7efd4940b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef7481b8a0 0x55ef7481f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:05.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.116+0000 7efd4940b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef7481fb50 0x55ef7481ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:05.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.116+0000 7efd4940b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ef74784790 con 0x55ef7481fb50 2024-05-11T12:15:05.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.116+0000 7efd4940b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ef74729f20 con 0x55ef7481b8a0 2024-05-11T12:15:05.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.116+0000 7efd4940b740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55ef74780750 con 0x55ef74820a10 2024-05-11T12:15:05.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.117+0000 7efd3fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef7481fb50 0x55ef7481ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:05.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.117+0000 7efd3fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef7481fb50 0x55ef7481ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39052/0 (socket says 172.21.15.167:39052) 2024-05-11T12:15:05.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.117+0000 7efd3fe00640 1 -- 172.21.15.167:0/14031950 learned_addr learned my addr 172.21.15.167:0/14031950 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:05.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.117+0000 7efd37e00640 1 --2- 172.21.15.167:0/14031950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef7481b8a0 0x55ef7481f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:05.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.117+0000 7efd3f400640 1 --2- 172.21.15.167:0/14031950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55ef74820a10 0x55ef74820df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:05.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.117+0000 7efd3fe00640 1 -- 172.21.15.167:0/14031950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55ef74820a10 msgr2=0x55ef74820df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:05.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.117+0000 7efd3fe00640 1 --2- 172.21.15.167:0/14031950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55ef74820a10 0x55ef74820df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:05.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.117+0000 7efd3fe00640 1 -- 172.21.15.167:0/14031950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef7481b8a0 msgr2=0x55ef7481f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:05.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.117+0000 7efd3fe00640 1 --2- 172.21.15.167:0/14031950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef7481b8a0 0x55ef7481f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:05.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.117+0000 7efd3fe00640 1 -- 172.21.15.167:0/14031950 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef744962d0 con 0x55ef7481fb50 2024-05-11T12:15:05.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.117+0000 7efd3fe00640 1 --2- 172.21.15.167:0/14031950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef7481fb50 0x55ef7481ff90 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7efd3805cd50 tx=0x7efd38060170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a5104d8d122f0f server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:05.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.118+0000 7efd2fe00640 1 -- 172.21.15.167:0/14031950 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd38060760 con 0x55ef7481fb50 2024-05-11T12:15:05.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.118+0000 7efd2fe00640 1 -- 172.21.15.167:0/14031950 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd38064b10 con 0x55ef7481fb50 2024-05-11T12:15:05.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.118+0000 7efd2fe00640 1 -- 172.21.15.167:0/14031950 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd38075df0 con 0x55ef7481fb50 2024-05-11T12:15:05.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.118+0000 7efd4940b740 1 -- 172.21.15.167:0/14031950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef7481fb50 msgr2=0x55ef7481ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:05.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.118+0000 7efd4940b740 1 --2- 172.21.15.167:0/14031950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef7481fb50 0x55ef7481ff90 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7efd3805cd50 tx=0x7efd38060170 comp rx=0 tx=0).stop 2024-05-11T12:15:05.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.118+0000 7efd4940b740 1 -- 172.21.15.167:0/14031950 shutdown_connections 2024-05-11T12:15:05.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.118+0000 7efd4940b740 1 --2- 172.21.15.167:0/14031950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef7481fb50 0x55ef7481ff90 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:05.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.118+0000 7efd4940b740 1 --2- 172.21.15.167:0/14031950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef7481b8a0 0x55ef7481f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:05.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.118+0000 7efd4940b740 1 --2- 172.21.15.167:0/14031950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55ef74820a10 0x55ef74820df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:05.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.118+0000 7efd4940b740 1 -- 172.21.15.167:0/14031950 >> 172.21.15.167:0/14031950 conn(0x55ef7481db10 msgr2=0x55ef748665d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:05.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.118+0000 7efd4940b740 1 -- 172.21.15.167:0/14031950 shutdown_connections 2024-05-11T12:15:05.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.119+0000 7efd4940b740 1 -- 172.21.15.167:0/14031950 wait complete. 2024-05-11T12:15:05.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.119+0000 7efd4940b740 1 -- start start 2024-05-11T12:15:05.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.119+0000 7efd4940b740 1 Processor -- start 2024-05-11T12:15:05.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.120+0000 7efd4940b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef7481fb50 0x55ef748386b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:05.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.120+0000 7efd4940b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef74820a10 0x55ef748398c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:05.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.120+0000 7efd4940b740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55ef7483dba0 0x55ef74867f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:05.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.120+0000 7efd4940b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ef7484c5a0 con 0x55ef74820a10 2024-05-11T12:15:05.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.120+0000 7efd4940b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ef7473d9c0 con 0x55ef7481fb50 2024-05-11T12:15:05.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.120+0000 7efd4940b740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55ef7484ba20 con 0x55ef7483dba0 2024-05-11T12:15:05.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.121+0000 7efd4940b740 10 client.0 _ll_register_callbacks cb 0x55ef7470a120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:15:05.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.121+0000 7efd37e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef74820a10 0x55ef748398c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:05.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.121+0000 7efd37e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef74820a10 0x55ef748398c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39068/0 (socket says 172.21.15.167:39068) 2024-05-11T12:15:05.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.121+0000 7efd37e00640 1 -- 172.21.15.167:0/3269114552 learned_addr learned my addr 172.21.15.167:0/3269114552 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:05.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.121+0000 7efd3f400640 1 --2- 172.21.15.167:0/3269114552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef7481fb50 0x55ef748386b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:05.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.121+0000 7efd37e00640 1 -- 172.21.15.167:0/3269114552 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55ef7483dba0 msgr2=0x55ef74867f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:05.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.121+0000 7efd37e00640 1 --2- 172.21.15.167:0/3269114552 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55ef7483dba0 0x55ef74867f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:05.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.121+0000 7efd37e00640 1 -- 172.21.15.167:0/3269114552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef7481fb50 msgr2=0x55ef748386b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:05.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.121+0000 7efd37e00640 1 --2- 172.21.15.167:0/3269114552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef7481fb50 0x55ef748386b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:05.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.121+0000 7efd37e00640 1 -- 172.21.15.167:0/3269114552 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef745606b0 con 0x55ef74820a10 2024-05-11T12:15:05.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.121+0000 7efd37e00640 1 --2- 172.21.15.167:0/3269114552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef74820a10 0x55ef748398c0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7efd2800df40 tx=0x7efd2800f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcffc8e505c01040 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:05.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.121+0000 7efd2cc00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-11T12:15:05.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.122+0000 7efd2cc00640 1 -- 172.21.15.167:0/3269114552 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd28018020 con 0x55ef74820a10 2024-05-11T12:15:05.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.122+0000 7efd2cc00640 1 -- 172.21.15.167:0/3269114552 <== mon.0 v2:172.21.15.167:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 6187+0+0 (secure 0 0 0) 0x7efd28004240 con 0x55ef74820a10 2024-05-11T12:15:05.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.122+0000 7efd4940b740 1 -- 172.21.15.167:0/3269114552 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd38064b10 con 0x55ef74820a10 2024-05-11T12:15:05.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.122+0000 7efd2cc00640 1 -- 172.21.15.167:0/3269114552 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7efd28022050 con 0x55ef74820a10 2024-05-11T12:15:05.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.122+0000 7efd4940b740 10 client.6996 fetch_fsmap learned FSMap version 72 2024-05-11T12:15:05.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.122+0000 7efd4940b740 1 -- 172.21.15.167:0/3269114552 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) -- 0x55ef74839f00 con 0x55ef74820a10 2024-05-11T12:15:05.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.122+0000 7efd2cc00640 1 -- 172.21.15.167:0/3269114552 <== mon.0 v2:172.21.15.167:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd38064b10 con 0x55ef74820a10 2024-05-11T12:15:05.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.122+0000 7efd2cc00640 1 -- 172.21.15.167:0/3269114552 <== mon.0 v2:172.21.15.167:3300/0 5 ==== fsmap(e 72) ==== 3514+0+0 (secure 0 0 0) 0x7efd28025020 con 0x55ef74820a10 2024-05-11T12:15:05.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.122+0000 7efd4940b740 10 client.6996 fetch_fsmap finished waiting for FSMap version 72 2024-05-11T12:15:05.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.122+0000 7efd4940b740 10 client.6996 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-11T12:15:05.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.122+0000 7efd4940b740 20 client.6996 populate_metadata read hostname 'smithi167' 2024-05-11T12:15:05.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.122+0000 7efd4940b740 1 --2- 172.21.15.167:0/3269114552 >> [v2:172.21.15.167:6836/2901182170,v1:172.21.15.167:6837/2901182170] conn(0x55ef74840940 0x55ef7483cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:05.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.122+0000 7efd4940b740 4 client.6996 mds_command: new command op to 6885 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:15:05.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.122+0000 7efd4940b740 1 -- 172.21.15.167:0/3269114552 --> [v2:172.21.15.167:6836/2901182170,v1:172.21.15.167:6837/2901182170] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef747923a0 con 0x55ef74840940 2024-05-11T12:15:05.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.123+0000 7efd3f400640 1 --2- 172.21.15.167:0/3269114552 >> [v2:172.21.15.167:6836/2901182170,v1:172.21.15.167:6837/2901182170] conn(0x55ef74840940 0x55ef7483cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:05.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.123+0000 7efd3f400640 1 --2- 172.21.15.167:0/3269114552 >> [v2:172.21.15.167:6836/2901182170,v1:172.21.15.167:6837/2901182170] conn(0x55ef74840940 0x55ef7483cfb0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f7f82e3fad8bfe server_cookie=d957204be8b9a965 in_seq=0 out_seq=0 2024-05-11T12:15:05.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.123+0000 7efd2cc00640 10 client.6996 ms_handle_connect on v2:172.21.15.167:6836/2901182170 2024-05-11T12:15:05.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.124+0000 7efd2cc00640 1 -- 172.21.15.167:0/3269114552 <== mds.0 v2:172.21.15.167:6836/2901182170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ef747923a0 con 0x55ef74840940 2024-05-11T12:15:05.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.124+0000 7efd2cc00640 10 client.6996 handle_command_reply: tid=0 2024-05-11T12:15:05.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.124+0000 7efd4940b740 1 client.6996 shutdown 2024-05-11T12:15:05.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.124+0000 7efd4940b740 1 -- 172.21.15.167:0/3269114552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef74820a10 msgr2=0x55ef748398c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:05.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.124+0000 7efd4940b740 1 --2- 172.21.15.167:0/3269114552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef74820a10 0x55ef748398c0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7efd2800df40 tx=0x7efd2800f670 comp rx=0 tx=0).stop 2024-05-11T12:15:05.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.124+0000 7efd4940b740 1 -- 172.21.15.167:0/3269114552 shutdown_connections 2024-05-11T12:15:05.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.124+0000 7efd4940b740 1 --2- 172.21.15.167:0/3269114552 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55ef7483dba0 0x55ef74867f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:05.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.124+0000 7efd4940b740 1 --2- 172.21.15.167:0/3269114552 >> [v2:172.21.15.167:6836/2901182170,v1:172.21.15.167:6837/2901182170] conn(0x55ef74840940 0x55ef7483cfb0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:05.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.124+0000 7efd4940b740 1 --2- 172.21.15.167:0/3269114552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef74820a10 0x55ef748398c0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:05.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.124+0000 7efd4940b740 1 --2- 172.21.15.167:0/3269114552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef7481fb50 0x55ef748386b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:05.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.124+0000 7efd4940b740 1 -- 172.21.15.167:0/3269114552 >> 172.21.15.167:0/3269114552 conn(0x55ef7481b8a0 msgr2=0x55ef7481ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:05.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.124+0000 7efd2cc00640 0 client.6996 ms_handle_reset on v2:172.21.15.167:6836/2901182170 2024-05-11T12:15:05.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.125+0000 7efd4940b740 1 -- 172.21.15.167:0/3269114552 shutdown_connections 2024-05-11T12:15:05.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.125+0000 7efd4940b740 1 -- 172.21.15.167:0/3269114552 wait complete. 2024-05-11T12:15:05.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.125+0000 7efd4940b740 20 client.6996 trim_cache size 0 max 16384 2024-05-11T12:15:05.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.145+0000 7efd4940b740 1 Processor -- start 2024-05-11T12:15:05.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.145+0000 7efd4940b740 1 -- start start 2024-05-11T12:15:05.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.146+0000 7efd4940b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef7481fb50 0x55ef7483cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:05.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.146+0000 7efd4940b740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55ef74820a10 0x55ef7483d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:05.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.146+0000 7efd4940b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef7483dba0 0x55ef7483ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:05.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.146+0000 7efd4940b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efd40201ed0 con 0x55ef7481fb50 2024-05-11T12:15:05.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.146+0000 7efd4940b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7efd4013c290 con 0x55ef7483dba0 2024-05-11T12:15:05.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.146+0000 7efd4940b740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7efd4013c8e0 con 0x55ef74820a10 2024-05-11T12:15:05.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.146+0000 7efd3f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef7481fb50 0x55ef7483cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:05.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.146+0000 7efd3f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef7481fb50 0x55ef7483cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39082/0 (socket says 172.21.15.167:39082) 2024-05-11T12:15:05.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.146+0000 7efd3f400640 1 -- 172.21.15.167:0/620983208 learned_addr learned my addr 172.21.15.167:0/620983208 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:05.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.147+0000 7efd3f400640 1 -- 172.21.15.167:0/620983208 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55ef74820a10 msgr2=0x55ef7483d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:05.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.147+0000 7efd3f400640 1 --2- 172.21.15.167:0/620983208 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55ef74820a10 0x55ef7483d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:05.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.147+0000 7efd3f400640 1 -- 172.21.15.167:0/620983208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef7483dba0 msgr2=0x55ef7483ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:05.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.147+0000 7efd3f400640 1 --2- 172.21.15.167:0/620983208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef7483dba0 0x55ef7483ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:05.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.147+0000 7efd3f400640 1 -- 172.21.15.167:0/620983208 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef74867ae0 con 0x55ef7481fb50 2024-05-11T12:15:05.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.147+0000 7efd3f400640 1 --2- 172.21.15.167:0/620983208 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef7481fb50 0x55ef7483cf80 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7efd3000b2c0 tx=0x7efd30031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2cee44beea35483 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:05.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.147+0000 7efd2f400640 1 -- 172.21.15.167:0/620983208 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd30036040 con 0x55ef7481fb50 2024-05-11T12:15:05.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.147+0000 7efd2f400640 1 -- 172.21.15.167:0/620983208 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd30004230 con 0x55ef7481fb50 2024-05-11T12:15:05.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.147+0000 7efd2f400640 1 -- 172.21.15.167:0/620983208 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3003b8f0 con 0x55ef7481fb50 2024-05-11T12:15:05.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.148+0000 7efd4940b740 1 -- 172.21.15.167:0/620983208 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef7481fb50 msgr2=0x55ef7483cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:05.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.148+0000 7efd4940b740 1 --2- 172.21.15.167:0/620983208 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef7481fb50 0x55ef7483cf80 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7efd3000b2c0 tx=0x7efd30031000 comp rx=0 tx=0).stop 2024-05-11T12:15:05.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.148+0000 7efd4940b740 1 -- 172.21.15.167:0/620983208 shutdown_connections 2024-05-11T12:15:05.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.148+0000 7efd4940b740 1 --2- 172.21.15.167:0/620983208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef7483dba0 0x55ef7483ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:05.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.148+0000 7efd4940b740 1 --2- 172.21.15.167:0/620983208 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55ef74820a10 0x55ef7483d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:05.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.148+0000 7efd4940b740 1 --2- 172.21.15.167:0/620983208 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef7481fb50 0x55ef7483cf80 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:05.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.148+0000 7efd4940b740 1 -- 172.21.15.167:0/620983208 >> 172.21.15.167:0/620983208 conn(0x55ef7481b8a0 msgr2=0x55ef74878cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:05.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.148+0000 7efd4940b740 1 -- 172.21.15.167:0/620983208 shutdown_connections 2024-05-11T12:15:05.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.148+0000 7efd4940b740 1 -- 172.21.15.167:0/620983208 wait complete. 2024-05-11T12:15:05.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.149+0000 7efd4940b740 1 -- start start 2024-05-11T12:15:05.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.149+0000 7efd4940b740 1 Processor -- start 2024-05-11T12:15:05.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.150+0000 7efd4940b740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55ef7481fb50 0x55ef748d2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:05.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.150+0000 7efd4940b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef74820a10 0x55ef74837220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:05.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.150+0000 7efd4940b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef7483dba0 0x55ef7483cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:05.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.150+0000 7efd4940b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ef748407c0 con 0x55ef7483dba0 2024-05-11T12:15:05.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.150+0000 7efd4940b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ef748684d0 con 0x55ef74820a10 2024-05-11T12:15:05.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.150+0000 7efd4940b740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55ef74840e10 con 0x55ef7481fb50 2024-05-11T12:15:05.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.150+0000 7efd4940b740 10 client.0 _ll_register_callbacks cb 0x7efd2800d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:15:05.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.150+0000 7efd3fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef7483dba0 0x55ef7483cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:05.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.150+0000 7efd3fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef7483dba0 0x55ef7483cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39088/0 (socket says 172.21.15.167:39088) 2024-05-11T12:15:05.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.150+0000 7efd3fe00640 1 -- 172.21.15.167:0/1485477509 learned_addr learned my addr 172.21.15.167:0/1485477509 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:05.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.151+0000 7efd3fe00640 1 -- 172.21.15.167:0/1485477509 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55ef7481fb50 msgr2=0x55ef748d2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:05.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.151+0000 7efd3fe00640 1 --2- 172.21.15.167:0/1485477509 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55ef7481fb50 0x55ef748d2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:05.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.151+0000 7efd3fe00640 1 -- 172.21.15.167:0/1485477509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef74820a10 msgr2=0x55ef74837220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:05.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.151+0000 7efd37e00640 1 --2- 172.21.15.167:0/1485477509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef74820a10 0x55ef74837220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:05.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.151+0000 7efd3fe00640 1 --2- 172.21.15.167:0/1485477509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef74820a10 0x55ef74837220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:05.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.151+0000 7efd3fe00640 1 -- 172.21.15.167:0/1485477509 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef7486a820 con 0x55ef7483dba0 2024-05-11T12:15:05.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.151+0000 7efd3fe00640 1 --2- 172.21.15.167:0/1485477509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef7483dba0 0x55ef7483cf10 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7efd3805cd50 tx=0x7efd3805cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67b624f42f913d28 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:05.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.151+0000 7efd2cc00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-11T12:15:05.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.151+0000 7efd2cc00640 1 -- 172.21.15.167:0/1485477509 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd38093020 con 0x55ef7483dba0 2024-05-11T12:15:05.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.151+0000 7efd2cc00640 1 -- 172.21.15.167:0/1485477509 <== mon.0 v2:172.21.15.167:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 6187+0+0 (secure 0 0 0) 0x7efd3805f9c0 con 0x55ef7483dba0 2024-05-11T12:15:05.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.151+0000 7efd4940b740 1 -- 172.21.15.167:0/1485477509 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd30004230 con 0x55ef7483dba0 2024-05-11T12:15:05.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.152+0000 7efd2cc00640 1 -- 172.21.15.167:0/1485477509 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7efd3805cdb0 con 0x55ef7483dba0 2024-05-11T12:15:05.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.152+0000 7efd4940b740 10 client.7002 fetch_fsmap learned FSMap version 72 2024-05-11T12:15:05.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.152+0000 7efd4940b740 1 -- 172.21.15.167:0/1485477509 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) -- 0x55ef748ecbb0 con 0x55ef7483dba0 2024-05-11T12:15:05.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.152+0000 7efd2cc00640 1 -- 172.21.15.167:0/1485477509 <== mon.0 v2:172.21.15.167:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd30004230 con 0x55ef7483dba0 2024-05-11T12:15:05.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.152+0000 7efd2cc00640 1 -- 172.21.15.167:0/1485477509 <== mon.0 v2:172.21.15.167:3300/0 5 ==== fsmap(e 72) ==== 3514+0+0 (secure 0 0 0) 0x7efd3805c0d0 con 0x55ef7483dba0 2024-05-11T12:15:05.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.152+0000 7efd4940b740 10 client.7002 fetch_fsmap finished waiting for FSMap version 72 2024-05-11T12:15:05.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.152+0000 7efd4940b740 10 client.7002 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-11T12:15:05.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.152+0000 7efd4940b740 20 client.7002 populate_metadata read hostname 'smithi167' 2024-05-11T12:15:05.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.152+0000 7efd4940b740 1 --2- 172.21.15.167:0/1485477509 >> [v2:172.21.15.167:6836/2901182170,v1:172.21.15.167:6837/2901182170] conn(0x55ef748f0520 0x55ef748f0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:05.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.152+0000 7efd4940b740 4 client.7002 mds_command: new command op to 6885 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-11T12:15:05.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.152+0000 7efd4940b740 1 -- 172.21.15.167:0/1485477509 --> [v2:172.21.15.167:6836/2901182170,v1:172.21.15.167:6837/2901182170] -- command(tid 0: {"prefix": "status"}) -- 0x55ef7478c210 con 0x55ef748f0520 2024-05-11T12:15:05.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.152+0000 7efd3f400640 1 --2- 172.21.15.167:0/1485477509 >> [v2:172.21.15.167:6836/2901182170,v1:172.21.15.167:6837/2901182170] conn(0x55ef748f0520 0x55ef748f0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:05.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.153+0000 7efd3f400640 1 --2- 172.21.15.167:0/1485477509 >> [v2:172.21.15.167:6836/2901182170,v1:172.21.15.167:6837/2901182170] conn(0x55ef748f0520 0x55ef748f0900 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29e3c443a947bfe server_cookie=22b108f39ec1d7c5 in_seq=0 out_seq=0 2024-05-11T12:15:05.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.153+0000 7efd2cc00640 10 client.7002 ms_handle_connect on v2:172.21.15.167:6836/2901182170 2024-05-11T12:15:05.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.153+0000 7efd2cc00640 1 -- 172.21.15.167:0/1485477509 <== mds.0 v2:172.21.15.167:6836/2901182170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x55ef7478c210 con 0x55ef748f0520 2024-05-11T12:15:05.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.153+0000 7efd2cc00640 10 client.7002 handle_command_reply: tid=0 2024-05-11T12:15:05.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.153+0000 7efd4940b740 1 client.7002 shutdown 2024-05-11T12:15:05.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.153+0000 7efd4940b740 1 -- 172.21.15.167:0/1485477509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef7483dba0 msgr2=0x55ef7483cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:05.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.153+0000 7efd4940b740 1 --2- 172.21.15.167:0/1485477509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef7483dba0 0x55ef7483cf10 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7efd3805cd50 tx=0x7efd3805cc00 comp rx=0 tx=0).stop 2024-05-11T12:15:05.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.153+0000 7efd4940b740 1 -- 172.21.15.167:0/1485477509 shutdown_connections 2024-05-11T12:15:05.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.153+0000 7efd4940b740 1 --2- 172.21.15.167:0/1485477509 >> [v2:172.21.15.167:6836/2901182170,v1:172.21.15.167:6837/2901182170] conn(0x55ef748f0520 0x55ef748f0900 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:05.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.153+0000 7efd4940b740 1 --2- 172.21.15.167:0/1485477509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef7483dba0 0x55ef7483cf10 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:05.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.153+0000 7efd4940b740 1 --2- 172.21.15.167:0/1485477509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ef74820a10 0x55ef74837220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:05.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.153+0000 7efd4940b740 1 --2- 172.21.15.167:0/1485477509 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55ef7481fb50 0x55ef748d2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:05.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.153+0000 7efd2cc00640 0 client.7002 ms_handle_reset on v2:172.21.15.167:6836/2901182170 2024-05-11T12:15:05.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.153+0000 7efd4940b740 1 -- 172.21.15.167:0/1485477509 >> 172.21.15.167:0/1485477509 conn(0x55ef7481b8a0 msgr2=0x55ef748384c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:05.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.153+0000 7efd4940b740 1 -- 172.21.15.167:0/1485477509 shutdown_connections 2024-05-11T12:15:05.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.153+0000 7efd4940b740 1 -- 172.21.15.167:0/1485477509 wait complete. 2024-05-11T12:15:05.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.154+0000 7efd4940b740 20 client.7002 trim_cache size 0 max 16384 2024-05-11T12:15:05.157 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T12:15:05.157 INFO:teuthology.orchestra.run.smithi167.stdout: "cluster_fsid": "5f32d344-7289-4c27-881b-1533419ab678", 2024-05-11T12:15:05.157 INFO:teuthology.orchestra.run.smithi167.stdout: "whoami": 0, 2024-05-11T12:15:05.157 INFO:teuthology.orchestra.run.smithi167.stdout: "id": 6885, 2024-05-11T12:15:05.157 INFO:teuthology.orchestra.run.smithi167.stdout: "want_state": "up:active", 2024-05-11T12:15:05.158 INFO:teuthology.orchestra.run.smithi167.stdout: "state": "up:active", 2024-05-11T12:15:05.158 INFO:teuthology.orchestra.run.smithi167.stdout: "fs_name": "cephfs", 2024-05-11T12:15:05.158 INFO:teuthology.orchestra.run.smithi167.stdout: "rank_uptime": 13.411303551, 2024-05-11T12:15:05.158 INFO:teuthology.orchestra.run.smithi167.stdout: "mdsmap_epoch": 72, 2024-05-11T12:15:05.158 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_epoch": 101, 2024-05-11T12:15:05.158 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_epoch_barrier": 101, 2024-05-11T12:15:05.158 INFO:teuthology.orchestra.run.smithi167.stdout: "uptime": 14.208859742 2024-05-11T12:15:05.158 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T12:15:05.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.154+0000 7efd45000640 1 -- 172.21.15.167:0/2347880752 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7efd1c076300 msgr2=0x7efd1c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:05.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.154+0000 7efd45000640 1 --2- 172.21.15.167:0/2347880752 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7efd1c076300 0x7efd1c0787c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7efd30037000 tx=0x7efd30008040 comp rx=0 tx=0).stop 2024-05-11T12:15:05.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.154+0000 7efd45000640 1 -- 172.21.15.167:0/2347880752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd4015b4d0 msgr2=0x7efd401563d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:05.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.154+0000 7efd45000640 1 --2- 172.21.15.167:0/2347880752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd4015b4d0 0x7efd401563d0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7efd3800c7a0 tx=0x7efd3800cc60 comp rx=0 tx=0).stop 2024-05-11T12:15:05.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.154+0000 7efd45000640 1 -- 172.21.15.167:0/2347880752 shutdown_connections 2024-05-11T12:15:05.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.154+0000 7efd45000640 1 --2- 172.21.15.167:0/2347880752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd4015b4d0 0x7efd401563d0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:05.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.154+0000 7efd45000640 1 --2- 172.21.15.167:0/2347880752 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7efd1c076300 0x7efd1c0787c0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:05.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.154+0000 7efd45000640 1 --2- 172.21.15.167:0/2347880752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd400b47b0 0x7efd4013aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:05.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.154+0000 7efd45000640 1 --2- 172.21.15.167:0/2347880752 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd40072b10 0x7efd400cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:05.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.154+0000 7efd45000640 1 -- 172.21.15.167:0/2347880752 >> 172.21.15.167:0/2347880752 conn(0x7efd400b2df0 msgr2=0x7efd400b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:05.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.154+0000 7efd45000640 1 -- 172.21.15.167:0/2347880752 shutdown_connections 2024-05-11T12:15:05.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:05.154+0000 7efd45000640 1 -- 172.21.15.167:0/2347880752 wait complete. 2024-05-11T12:15:05.173 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-05-11T12:15:05.401 INFO:teuthology.orchestra.run.smithi167.stdout:{"message":"","return_code":0} 2024-05-11T12:15:05.401 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-11T12:15:05.402 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json perf dump objecter 2024-05-11T12:15:05.624 INFO:teuthology.orchestra.run.smithi167.stdout:{"objecter":{"op_active":0,"op_laggy":0,"op_send":40,"op_send_bytes":11562,"op_resend":32,"op_reply":40,"op_latency":{"avgcount":40,"sum":0.494991491,"avgtime":0.012374787},"op_inflight":0,"oplen_avg":{"avgcount":40,"sum":62},"op":40,"op_r":0,"op_w":40,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":10,"osdop_read":0,"osdop_write":2,"osdop_writefull":8,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":20,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":0,"map_full":0,"map_inc":0,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":15,"omap_rd":0,"omap_del":1}} 2024-05-11T12:15:05.624 DEBUG:tasks.cephfs.filesystem:_json_asok output { "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 0, "map_full": 0, "map_inc": 0, "omap_del": 1, "omap_rd": 0, "omap_wr": 15, "op": 40, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 40, "avgtime": 0.012374787, "sum": 0.494991491 }, "op_pg": 0, "op_r": 0, "op_reply": 40, "op_resend": 32, "op_rmw": 0, "op_send": 40, "op_send_bytes": 11562, "op_w": 40, "oplen_avg": { "avgcount": 40, "sum": 62 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 10, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 20, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 2, "osdop_writefull": 8, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 } } 2024-05-11T12:15:05.625 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json export dir /bravo 1 2024-05-11T12:15:05.846 INFO:teuthology.orchestra.run.smithi167.stdout:{"return_code":0} 2024-05-11T12:15:05.847 DEBUG:tasks.cephfs.filesystem:_json_asok output { "return_code": 0 } 2024-05-11T12:15:05.847 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json perf dump objecter 2024-05-11T12:15:06.071 INFO:teuthology.orchestra.run.smithi167.stdout:{"objecter":{"op_active":0,"op_laggy":0,"op_send":40,"op_send_bytes":11562,"op_resend":32,"op_reply":40,"op_latency":{"avgcount":40,"sum":0.494991491,"avgtime":0.012374787},"op_inflight":0,"oplen_avg":{"avgcount":40,"sum":62},"op":40,"op_r":0,"op_w":40,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":10,"osdop_read":0,"osdop_write":2,"osdop_writefull":8,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":20,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":0,"map_full":0,"map_inc":0,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":15,"omap_rd":0,"omap_del":1}} 2024-05-11T12:15:06.071 DEBUG:tasks.cephfs.filesystem:_json_asok output { "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 0, "map_full": 0, "map_inc": 0, "omap_del": 1, "omap_rd": 0, "omap_wr": 15, "op": 40, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 40, "avgtime": 0.012374787, "sum": 0.494991491 }, "op_pg": 0, "op_r": 0, "op_reply": 40, "op_resend": 32, "op_rmw": 0, "op_send": 40, "op_send_bytes": 11562, "op_w": 40, "oplen_avg": { "avgcount": 40, "sum": 62 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 10, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 20, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 2, "osdop_writefull": 8, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 } } 2024-05-11T12:15:06.071 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-05-11T12:15:07.794 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T12:15:07.802 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T12:15:11.073 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json perf dump objecter 2024-05-11T12:15:11.304 INFO:teuthology.orchestra.run.smithi167.stdout:{"objecter":{"op_active":0,"op_laggy":0,"op_send":44,"op_send_bytes":15215,"op_resend":32,"op_reply":44,"op_latency":{"avgcount":44,"sum":0.512991181,"avgtime":0.011658890},"op_inflight":0,"oplen_avg":{"avgcount":44,"sum":68},"op":44,"op_r":0,"op_w":44,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":10,"osdop_read":0,"osdop_write":4,"osdop_writefull":8,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":20,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":0,"map_full":0,"map_inc":0,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":19,"omap_rd":0,"omap_del":1}} 2024-05-11T12:15:11.304 DEBUG:tasks.cephfs.filesystem:_json_asok output { "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 0, "map_full": 0, "map_inc": 0, "omap_del": 1, "omap_rd": 0, "omap_wr": 19, "op": 44, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 44, "avgtime": 0.01165889, "sum": 0.512991181 }, "op_pg": 0, "op_r": 0, "op_reply": 44, "op_resend": 32, "op_rmw": 0, "op_send": 44, "op_send_bytes": 15215, "op_w": 44, "oplen_avg": { "avgcount": 44, "sum": 68 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 10, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 20, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 4, "osdop_writefull": 8, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 } } 2024-05-11T12:15:11.305 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-05-11T12:15:11.305 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:15:11.305 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:15:11.355 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:15:11.355 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:15:11.448 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-11T12:15:11.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.640+0000 7f631b200640 1 Processor -- start 2024-05-11T12:15:11.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.640+0000 7f631b200640 1 -- start start 2024-05-11T12:15:11.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.642+0000 7f631b200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f631c15aaa0 0x7f631c06dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:11.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.642+0000 7f631b200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f631c06cfb0 0x7f631c0b4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:11.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.642+0000 7f631b200640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f631c15d8d0 con 0x7f631c0b02f0 2024-05-11T12:15:11.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.642+0000 7f631b200640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f631c15ae80 con 0x7f631c15aaa0 2024-05-11T12:15:11.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.642+0000 7f631b200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f631c15dbb0 con 0x7f631c06cfb0 2024-05-11T12:15:11.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.643+0000 7f6319e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f631c15aaa0 0x7f631c06dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:11.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.643+0000 7f6319e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f631c15aaa0 0x7f631c06dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35150/0 (socket says 172.21.15.167:35150) 2024-05-11T12:15:11.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.643+0000 7f6319e00640 1 -- 172.21.15.167:0/1377293940 learned_addr learned my addr 172.21.15.167:0/1377293940 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:11.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.643+0000 7f631a800640 1 --2- 172.21.15.167:0/1377293940 >> v2:172.21.15.204:3300/0 conn(0x7f631c06cfb0 0x7f631c0b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:11.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.643+0000 7f6319e00640 1 -- 172.21.15.167:0/1377293940 >> v2:172.21.15.204:3300/0 conn(0x7f631c06cfb0 msgr2=0x7f631c0b4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:11.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.643+0000 7f6319e00640 1 --2- 172.21.15.167:0/1377293940 >> v2:172.21.15.204:3300/0 conn(0x7f631c06cfb0 0x7f631c0b4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:11.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.643+0000 7f6319e00640 1 -- 172.21.15.167:0/1377293940 >> v1:172.21.15.167:6789/0 conn(0x7f631c0b02f0 legacy=0x7f631c06c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:11.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.643+0000 7f6319e00640 1 -- 172.21.15.167:0/1377293940 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f631c15de90 con 0x7f631c15aaa0 2024-05-11T12:15:11.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.644+0000 7f6319e00640 1 --2- 172.21.15.167:0/1377293940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f631c15aaa0 0x7f631c06dec0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f631c09f0a0 tx=0x7f630c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6b12fbf5aac149 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:11.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.645+0000 7f6319400640 1 -- 172.21.15.167:0/1377293940 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630c037070 con 0x7f631c15aaa0 2024-05-11T12:15:11.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.645+0000 7f6319400640 1 -- 172.21.15.167:0/1377293940 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f630c032440 con 0x7f631c15aaa0 2024-05-11T12:15:11.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.645+0000 7f6319400640 1 -- 172.21.15.167:0/1377293940 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630c032810 con 0x7f631c15aaa0 2024-05-11T12:15:11.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.645+0000 7f631b200640 1 -- 172.21.15.167:0/1377293940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f631c15aaa0 msgr2=0x7f631c06dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:11.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.645+0000 7f631b200640 1 --2- 172.21.15.167:0/1377293940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f631c15aaa0 0x7f631c06dec0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f631c09f0a0 tx=0x7f630c02a270 comp rx=0 tx=0).stop 2024-05-11T12:15:11.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.645+0000 7f631b200640 1 -- 172.21.15.167:0/1377293940 shutdown_connections 2024-05-11T12:15:11.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.645+0000 7f631b200640 1 --2- 172.21.15.167:0/1377293940 >> v2:172.21.15.204:3300/0 conn(0x7f631c06cfb0 0x7f631c0b4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:11.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.645+0000 7f631b200640 1 --2- 172.21.15.167:0/1377293940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f631c15aaa0 0x7f631c06dec0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:11.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.645+0000 7f631b200640 1 -- 172.21.15.167:0/1377293940 >> 172.21.15.167:0/1377293940 conn(0x7f631c0b2df0 msgr2=0x7f631c0b3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:11.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.645+0000 7f631b200640 1 -- 172.21.15.167:0/1377293940 shutdown_connections 2024-05-11T12:15:11.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.645+0000 7f631b200640 1 -- 172.21.15.167:0/1377293940 wait complete. 2024-05-11T12:15:11.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.647+0000 7f631b200640 1 Processor -- start 2024-05-11T12:15:11.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.647+0000 7f631b200640 1 -- start start 2024-05-11T12:15:11.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.648+0000 7f631b200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f631c06cfb0 0x7f631c0b5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:11.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.648+0000 7f631b200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f631c0b02f0 0x7f631c155700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:11.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.648+0000 7f631b200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f631c15aaa0 0x7f631c1f5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:11.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.648+0000 7f631b200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f631c15d480 con 0x7f631c0b02f0 2024-05-11T12:15:11.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.648+0000 7f631b200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f631c06e400 con 0x7f631c15aaa0 2024-05-11T12:15:11.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.648+0000 7f631b200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f631c15d600 con 0x7f631c06cfb0 2024-05-11T12:15:11.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.649+0000 7f6311400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f631c0b02f0 0x7f631c155700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:11.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.649+0000 7f6311400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f631c0b02f0 0x7f631c155700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35162/0 (socket says 172.21.15.167:35162) 2024-05-11T12:15:11.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.649+0000 7f6311400640 1 -- 172.21.15.167:0/1053381250 learned_addr learned my addr 172.21.15.167:0/1053381250 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:11.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.649+0000 7f6319e00640 1 --2- 172.21.15.167:0/1053381250 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f631c06cfb0 0x7f631c0b5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:11.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.649+0000 7f631a800640 1 --2- 172.21.15.167:0/1053381250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f631c15aaa0 0x7f631c1f5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:11.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.649+0000 7f6311400640 1 -- 172.21.15.167:0/1053381250 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f631c06cfb0 msgr2=0x7f631c0b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:11.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.649+0000 7f6311400640 1 --2- 172.21.15.167:0/1053381250 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f631c06cfb0 0x7f631c0b5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:11.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.649+0000 7f6311400640 1 -- 172.21.15.167:0/1053381250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f631c15aaa0 msgr2=0x7f631c1f5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:11.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.649+0000 7f6311400640 1 --2- 172.21.15.167:0/1053381250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f631c15aaa0 0x7f631c1f5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:11.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.649+0000 7f6311400640 1 -- 172.21.15.167:0/1053381250 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f631c152190 con 0x7f631c0b02f0 2024-05-11T12:15:11.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.650+0000 7f6311400640 1 --2- 172.21.15.167:0/1053381250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f631c0b02f0 0x7f631c155700 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f630400c450 tx=0x7f630400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:11.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.650+0000 7f6312a00640 1 -- 172.21.15.167:0/1053381250 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63040052d0 con 0x7f631c0b02f0 2024-05-11T12:15:11.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.650+0000 7f6312a00640 1 -- 172.21.15.167:0/1053381250 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6304005470 con 0x7f631c0b02f0 2024-05-11T12:15:11.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.650+0000 7f6312a00640 1 -- 172.21.15.167:0/1053381250 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6304012870 con 0x7f631c0b02f0 2024-05-11T12:15:11.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.650+0000 7f631b200640 1 -- 172.21.15.167:0/1053381250 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f631c152470 con 0x7f631c0b02f0 2024-05-11T12:15:11.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.650+0000 7f631b200640 1 -- 172.21.15.167:0/1053381250 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f631c1f4f70 con 0x7f631c0b02f0 2024-05-11T12:15:11.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.651+0000 7f631b200640 1 -- 172.21.15.167:0/1053381250 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f631c069650 con 0x7f631c0b02f0 2024-05-11T12:15:11.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.652+0000 7f6312a00640 1 -- 172.21.15.167:0/1053381250 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6304012a10 con 0x7f631c0b02f0 2024-05-11T12:15:11.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.653+0000 7f6312a00640 1 --2- 172.21.15.167:0/1053381250 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f62f80762d0 0x7f62f8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:11.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.653+0000 7f6312a00640 1 -- 172.21.15.167:0/1053381250 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6187+0+0 (secure 0 0 0) 0x7f6304097460 con 0x7f631c0b02f0 2024-05-11T12:15:11.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.654+0000 7f6319e00640 1 --2- 172.21.15.167:0/1053381250 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f62f80762d0 0x7f62f8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:11.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.654+0000 7f6319e00640 1 --2- 172.21.15.167:0/1053381250 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f62f80762d0 0x7f62f8078790 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f630c034000 tx=0x7f630c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:11.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.658+0000 7f6312a00640 1 -- 172.21.15.167:0/1053381250 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6304060b40 con 0x7f631c0b02f0 2024-05-11T12:15:11.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.882+0000 7f631b200640 1 -- 172.21.15.167:0/1053381250 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f631c1f5140 con 0x7f631c0b02f0 2024-05-11T12:15:11.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.885+0000 7f6312a00640 1 -- 172.21.15.167:0/1053381250 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v101) ==== 80+0+124 (secure 0 0 0) 0x7f6304060440 con 0x7f631c0b02f0 2024-05-11T12:15:11.886 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.15.167:6837/3493868276 2024-05-12T12:14:50.725899+0000 2024-05-11T12:15:11.886 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.15.167:6836/3493868276 2024-05-12T12:14:50.725899+0000 2024-05-11T12:15:11.886 INFO:teuthology.orchestra.run.smithi167.stderr:listed 2 entries 2024-05-11T12:15:11.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.888+0000 7f631b200640 1 -- 172.21.15.167:0/1053381250 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f62f80762d0 msgr2=0x7f62f8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:11.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.888+0000 7f631b200640 1 --2- 172.21.15.167:0/1053381250 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f62f80762d0 0x7f62f8078790 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f630c034000 tx=0x7f630c02e000 comp rx=0 tx=0).stop 2024-05-11T12:15:11.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.888+0000 7f631b200640 1 -- 172.21.15.167:0/1053381250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f631c0b02f0 msgr2=0x7f631c155700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:11.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.888+0000 7f631b200640 1 --2- 172.21.15.167:0/1053381250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f631c0b02f0 0x7f631c155700 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f630400c450 tx=0x7f630400c910 comp rx=0 tx=0).stop 2024-05-11T12:15:11.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.888+0000 7f631b200640 1 -- 172.21.15.167:0/1053381250 shutdown_connections 2024-05-11T12:15:11.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.888+0000 7f631b200640 1 --2- 172.21.15.167:0/1053381250 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f62f80762d0 0x7f62f8078790 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:11.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.888+0000 7f631b200640 1 --2- 172.21.15.167:0/1053381250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f631c15aaa0 0x7f631c1f5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:11.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.888+0000 7f631b200640 1 --2- 172.21.15.167:0/1053381250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f631c0b02f0 0x7f631c155700 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:11.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.888+0000 7f631b200640 1 --2- 172.21.15.167:0/1053381250 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f631c06cfb0 0x7f631c0b5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:11.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.888+0000 7f631b200640 1 -- 172.21.15.167:0/1053381250 >> 172.21.15.167:0/1053381250 conn(0x7f631c0b2df0 msgr2=0x7f631c0b0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:11.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.888+0000 7f631b200640 1 -- 172.21.15.167:0/1053381250 shutdown_connections 2024-05-11T12:15:11.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:11.889+0000 7f631b200640 1 -- 172.21.15.167:0/1053381250 wait complete. 2024-05-11T12:15:11.908 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-11T12:15:11.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:15:11.908 DEBUG:teuthology.orchestra.run.smithi167:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-11T12:15:12.017 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2024-05-11T12:15:12.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:15:12.017 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-11T12:15:12.080 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:15:12.080 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:15:12.121 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:15:12.121 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:15:12.206 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-11T12:15:12.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.409+0000 7fbba1e00640 1 Processor -- start 2024-05-11T12:15:12.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.409+0000 7fbba1e00640 1 -- start start 2024-05-11T12:15:12.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.410+0000 7fbba1e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fbb9c15aaa0 0x7fbb9c06dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:12.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.410+0000 7fbba1e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb9c0b02f0 0x7fbb9c06c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:12.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.411+0000 7fbba1e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb9c15d8d0 con 0x7fbb9c0add80 2024-05-11T12:15:12.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.411+0000 7fbba1e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fbb9c15ae80 con 0x7fbb9c15aaa0 2024-05-11T12:15:12.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.411+0000 7fbba1e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbb9c15dbb0 con 0x7fbb9c0b02f0 2024-05-11T12:15:12.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.411+0000 7fbba0a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fbb9c15aaa0 0x7fbb9c06dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:12.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.411+0000 7fbb93e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb9c0b02f0 0x7fbb9c06c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:12.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.412+0000 7fbb93e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb9c0b02f0 0x7fbb9c06c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:48098/0 (socket says 172.21.15.167:48098) 2024-05-11T12:15:12.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.412+0000 7fbb93e00640 1 -- 172.21.15.167:0/1290440102 learned_addr learned my addr 172.21.15.167:0/1290440102 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:12.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.412+0000 7fbb93e00640 1 -- 172.21.15.167:0/1290440102 >> v2:172.21.15.204:3300/0 conn(0x7fbb9c15aaa0 msgr2=0x7fbb9c06dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:12.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.412+0000 7fbb93e00640 1 --2- 172.21.15.167:0/1290440102 >> v2:172.21.15.204:3300/0 conn(0x7fbb9c15aaa0 0x7fbb9c06dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:12.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.412+0000 7fbb93e00640 1 -- 172.21.15.167:0/1290440102 >> v1:172.21.15.204:6789/0 conn(0x7fbb9c0add80 legacy=0x7fbb9c0b4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:12.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.412+0000 7fbb93e00640 1 -- 172.21.15.167:0/1290440102 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb9c15de90 con 0x7fbb9c0b02f0 2024-05-11T12:15:12.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.414+0000 7fbb93e00640 1 --2- 172.21.15.167:0/1290440102 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb9c0b02f0 0x7fbb9c06c8a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c009870 tx=0x7fbb8c02a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e0800f6e6d06582 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:12.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.414+0000 7fbb9be00640 1 -- 172.21.15.167:0/1290440102 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c037070 con 0x7fbb9c0b02f0 2024-05-11T12:15:12.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.414+0000 7fbb9be00640 1 -- 172.21.15.167:0/1290440102 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb8c032440 con 0x7fbb9c0b02f0 2024-05-11T12:15:12.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.414+0000 7fbb9be00640 1 -- 172.21.15.167:0/1290440102 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c032810 con 0x7fbb9c0b02f0 2024-05-11T12:15:12.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.415+0000 7fbba1e00640 1 -- 172.21.15.167:0/1290440102 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb9c0b02f0 msgr2=0x7fbb9c06c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:12.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.415+0000 7fbba1e00640 1 --2- 172.21.15.167:0/1290440102 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb9c0b02f0 0x7fbb9c06c8a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c009870 tx=0x7fbb8c02a250 comp rx=0 tx=0).stop 2024-05-11T12:15:12.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.415+0000 7fbba1e00640 1 -- 172.21.15.167:0/1290440102 shutdown_connections 2024-05-11T12:15:12.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.415+0000 7fbba1e00640 1 --2- 172.21.15.167:0/1290440102 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb9c0b02f0 0x7fbb9c06c8a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:12.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.415+0000 7fbba1e00640 1 --2- 172.21.15.167:0/1290440102 >> v2:172.21.15.204:3300/0 conn(0x7fbb9c15aaa0 0x7fbb9c06dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:12.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.415+0000 7fbba1e00640 1 -- 172.21.15.167:0/1290440102 >> 172.21.15.167:0/1290440102 conn(0x7fbb9c0b2df0 msgr2=0x7fbb9c0b3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:12.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.415+0000 7fbba1e00640 1 -- 172.21.15.167:0/1290440102 shutdown_connections 2024-05-11T12:15:12.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.415+0000 7fbba1e00640 1 -- 172.21.15.167:0/1290440102 wait complete. 2024-05-11T12:15:12.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.417+0000 7fbba1e00640 1 Processor -- start 2024-05-11T12:15:12.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.417+0000 7fbba1e00640 1 -- start start 2024-05-11T12:15:12.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.418+0000 7fbba1e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb9c0add80 0x7fbb9c09e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:12.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.418+0000 7fbba1e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb9c0b02f0 0x7fbb9c09c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:12.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.418+0000 7fbba1e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb9c15aaa0 0x7fbb9c14e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:12.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.418+0000 7fbba1e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9c15d480 con 0x7fbb9c0add80 2024-05-11T12:15:12.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.418+0000 7fbba1e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9c06e400 con 0x7fbb9c0b02f0 2024-05-11T12:15:12.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.418+0000 7fbba1e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbb9c15d600 con 0x7fbb9c15aaa0 2024-05-11T12:15:12.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.418+0000 7fbba0a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb9c0add80 0x7fbb9c09e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:12.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.418+0000 7fbba0a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb9c0add80 0x7fbb9c09e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35176/0 (socket says 172.21.15.167:35176) 2024-05-11T12:15:12.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.418+0000 7fbba0a00640 1 -- 172.21.15.167:0/1098372629 learned_addr learned my addr 172.21.15.167:0/1098372629 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:12.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.418+0000 7fbba1400640 1 --2- 172.21.15.167:0/1098372629 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb9c15aaa0 0x7fbb9c14e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:12.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.418+0000 7fbb93e00640 1 --2- 172.21.15.167:0/1098372629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb9c0b02f0 0x7fbb9c09c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:12.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.419+0000 7fbba0a00640 1 -- 172.21.15.167:0/1098372629 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb9c15aaa0 msgr2=0x7fbb9c14e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:12.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.419+0000 7fbba0a00640 1 --2- 172.21.15.167:0/1098372629 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb9c15aaa0 0x7fbb9c14e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:12.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.419+0000 7fbba0a00640 1 -- 172.21.15.167:0/1098372629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb9c0b02f0 msgr2=0x7fbb9c09c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:12.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.419+0000 7fbba0a00640 1 --2- 172.21.15.167:0/1098372629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb9c0b02f0 0x7fbb9c09c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:12.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.419+0000 7fbba0a00640 1 -- 172.21.15.167:0/1098372629 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb9c1f81b0 con 0x7fbb9c0add80 2024-05-11T12:15:12.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.419+0000 7fbb93e00640 1 --2- 172.21.15.167:0/1098372629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb9c0b02f0 0x7fbb9c09c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:15:12.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.419+0000 7fbba0a00640 1 --2- 172.21.15.167:0/1098372629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb9c0add80 0x7fbb9c09e180 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fbb8400e870 tx=0x7fbb8400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:12.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.419+0000 7fbb99600640 1 -- 172.21.15.167:0/1098372629 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb84017070 con 0x7fbb9c0add80 2024-05-11T12:15:12.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.419+0000 7fbb99600640 1 -- 172.21.15.167:0/1098372629 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb8400c940 con 0x7fbb9c0add80 2024-05-11T12:15:12.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.419+0000 7fbba1e00640 1 -- 172.21.15.167:0/1098372629 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb9c1f8490 con 0x7fbb9c0add80 2024-05-11T12:15:12.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.419+0000 7fbb99600640 1 -- 172.21.15.167:0/1098372629 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8400cd30 con 0x7fbb9c0add80 2024-05-11T12:15:12.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.419+0000 7fbba1e00640 1 -- 172.21.15.167:0/1098372629 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb9c1f8a50 con 0x7fbb9c0add80 2024-05-11T12:15:12.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.421+0000 7fbba1e00640 1 -- 172.21.15.167:0/1098372629 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb9c1f8e90 con 0x7fbb9c0add80 2024-05-11T12:15:12.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.422+0000 7fbb99600640 1 -- 172.21.15.167:0/1098372629 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbb8401a070 con 0x7fbb9c0add80 2024-05-11T12:15:12.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.423+0000 7fbb99600640 1 --2- 172.21.15.167:0/1098372629 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbb780762d0 0x7fbb78078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:12.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.423+0000 7fbb99600640 1 -- 172.21.15.167:0/1098372629 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6187+0+0 (secure 0 0 0) 0x7fbb84097a30 con 0x7fbb9c0add80 2024-05-11T12:15:12.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.424+0000 7fbb93e00640 1 --2- 172.21.15.167:0/1098372629 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbb780762d0 0x7fbb78078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:12.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.424+0000 7fbb93e00640 1 --2- 172.21.15.167:0/1098372629 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbb780762d0 0x7fbb78078790 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c034000 tx=0x7fbb8c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:12.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.428+0000 7fbb99600640 1 -- 172.21.15.167:0/1098372629 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fbb840611a0 con 0x7fbb9c0add80 2024-05-11T12:15:12.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.651+0000 7fbba1e00640 1 -- 172.21.15.167:0/1098372629 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbb9c069650 con 0x7fbb9c0add80 2024-05-11T12:15:12.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.652+0000 7fbb99600640 1 -- 172.21.15.167:0/1098372629 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v101) ==== 80+0+124 (secure 0 0 0) 0x7fbb84060aa0 con 0x7fbb9c0add80 2024-05-11T12:15:12.652 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.15.167:6837/3493868276 2024-05-12T12:14:50.725899+0000 2024-05-11T12:15:12.652 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.15.167:6836/3493868276 2024-05-12T12:14:50.725899+0000 2024-05-11T12:15:12.652 INFO:teuthology.orchestra.run.smithi167.stderr:listed 2 entries 2024-05-11T12:15:12.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.654+0000 7fbba1e00640 1 -- 172.21.15.167:0/1098372629 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbb780762d0 msgr2=0x7fbb78078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:12.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.654+0000 7fbba1e00640 1 --2- 172.21.15.167:0/1098372629 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbb780762d0 0x7fbb78078790 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c034000 tx=0x7fbb8c02e000 comp rx=0 tx=0).stop 2024-05-11T12:15:12.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.654+0000 7fbba1e00640 1 -- 172.21.15.167:0/1098372629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb9c0add80 msgr2=0x7fbb9c09e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:12.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.654+0000 7fbba1e00640 1 --2- 172.21.15.167:0/1098372629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb9c0add80 0x7fbb9c09e180 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fbb8400e870 tx=0x7fbb8400ed30 comp rx=0 tx=0).stop 2024-05-11T12:15:12.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.654+0000 7fbba1e00640 1 -- 172.21.15.167:0/1098372629 shutdown_connections 2024-05-11T12:15:12.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.654+0000 7fbba1e00640 1 --2- 172.21.15.167:0/1098372629 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb9c15aaa0 0x7fbb9c14e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:12.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.655+0000 7fbba1e00640 1 --2- 172.21.15.167:0/1098372629 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbb780762d0 0x7fbb78078790 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:12.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.655+0000 7fbba1e00640 1 --2- 172.21.15.167:0/1098372629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb9c0b02f0 0x7fbb9c09c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:12.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.655+0000 7fbba1e00640 1 --2- 172.21.15.167:0/1098372629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb9c0add80 0x7fbb9c09e180 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:12.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.655+0000 7fbba1e00640 1 -- 172.21.15.167:0/1098372629 >> 172.21.15.167:0/1098372629 conn(0x7fbb9c0b2df0 msgr2=0x7fbb9c0d0530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:12.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.655+0000 7fbba1e00640 1 -- 172.21.15.167:0/1098372629 shutdown_connections 2024-05-11T12:15:12.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:12.655+0000 7fbba1e00640 1 -- 172.21.15.167:0/1098372629 wait complete. 2024-05-11T12:15:12.674 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-11T12:15:12.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:15:12.675 DEBUG:teuthology.orchestra.run.smithi204:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-05-11T12:15:12.781 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi204.front.sepia.ceph.com 2024-05-11T12:15:12.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:15:12.782 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-11T12:15:12.833 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-05-11T12:15:13.065 INFO:teuthology.orchestra.run.smithi167.stdout:[] 2024-05-11T12:15:13.066 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2024-05-11T12:15:13.066 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-05-11T12:15:13.301 INFO:teuthology.orchestra.run.smithi167.stdout:{"message":"","return_code":0} 2024-05-11T12:15:13.301 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-11T12:15:13.301 DEBUG:teuthology.orchestra.run.smithi167:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 0 show session 2024-05-11T12:15:13.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.399+0000 7f48a191e280 1 -- start start 2024-05-11T12:15:13.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.399+0000 7f48a191e280 1 Processor -- start 2024-05-11T12:15:13.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.400+0000 7f48a191e280 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585b9fbac00 0x5585bad22000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:13.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.400+0000 7f48a191e280 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5585b9fbb800 0x5585bad22580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:13.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.400+0000 7f48a191e280 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585b9fbbc00 0x5585bad22b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:13.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.400+0000 7f48a191e280 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5585badaa180 con 0x5585b9fbac00 2024-05-11T12:15:13.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.400+0000 7f48a191e280 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5585badaa300 con 0x5585b9fbbc00 2024-05-11T12:15:13.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.400+0000 7f48a191e280 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x5585badaa480 con 0x5585b9fbb800 2024-05-11T12:15:13.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.400+0000 7f489ee00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585b9fbac00 0x5585bad22000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:13.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.400+0000 7f489ee00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585b9fbac00 0x5585bad22000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35198/0 (socket says 172.21.15.167:35198) 2024-05-11T12:15:13.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.400+0000 7f489ee00640 1 -- 172.21.15.167:0/1599139692 learned_addr learned my addr 172.21.15.167:0/1599139692 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:13.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.401+0000 7f489f800640 1 --2- 172.21.15.167:0/1599139692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585b9fbbc00 0x5585bad22b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:13.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.401+0000 7f489e400640 1 --2- 172.21.15.167:0/1599139692 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5585b9fbb800 0x5585bad22580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:13.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.401+0000 7f489ee00640 1 -- 172.21.15.167:0/1599139692 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5585b9fbb800 msgr2=0x5585bad22580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:13.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.401+0000 7f489ee00640 1 --2- 172.21.15.167:0/1599139692 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5585b9fbb800 0x5585bad22580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:13.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.401+0000 7f489ee00640 1 -- 172.21.15.167:0/1599139692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585b9fbbc00 msgr2=0x5585bad22b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:13.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.401+0000 7f489ee00640 1 --2- 172.21.15.167:0/1599139692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585b9fbbc00 0x5585bad22b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:13.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.401+0000 7f489ee00640 1 -- 172.21.15.167:0/1599139692 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585b9fee780 con 0x5585b9fbac00 2024-05-11T12:15:13.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.401+0000 7f489f800640 1 --2- 172.21.15.167:0/1599139692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585b9fbbc00 0x5585bad22b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:15:13.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.401+0000 7f489ee00640 1 --2- 172.21.15.167:0/1599139692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585b9fbac00 0x5585bad22000 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x5585bade54a0 tx=0x5585bad5cfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e4367a91d89cf52 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:13.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.401+0000 7f489bc00640 1 -- 172.21.15.167:0/1599139692 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5585b9fe0680 con 0x5585b9fbac00 2024-05-11T12:15:13.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.401+0000 7f489bc00640 1 -- 172.21.15.167:0/1599139692 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5585b9fe6700 con 0x5585b9fbac00 2024-05-11T12:15:13.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.401+0000 7f489bc00640 1 -- 172.21.15.167:0/1599139692 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5585b9fe0b60 con 0x5585b9fbac00 2024-05-11T12:15:13.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.401+0000 7f48a191e280 1 -- 172.21.15.167:0/1599139692 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585b9fef4a0 con 0x5585b9fbac00 2024-05-11T12:15:13.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.401+0000 7f48a191e280 1 -- 172.21.15.167:0/1599139692 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5585b9fee000 con 0x5585b9fbac00 2024-05-11T12:15:13.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.401+0000 7f48a191e280 4 waiting for MDS map... 2024-05-11T12:15:13.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.402+0000 7f489bc00640 1 -- 172.21.15.167:0/1599139692 <== mon.0 v2:172.21.15.167:3300/0 4 ==== osd_map(101..101 src has 1..101) ==== 6187+0+0 (secure 0 0 0) 0x5585bae90600 con 0x5585b9fbac00 2024-05-11T12:15:13.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.402+0000 7f489bc00640 1 -- 172.21.15.167:0/1599139692 <== mon.0 v2:172.21.15.167:3300/0 5 ==== fsmap(e 73) ==== 3518+0+0 (secure 0 0 0) 0x5585ba015180 con 0x5585b9fbac00 2024-05-11T12:15:13.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.402+0000 7f48a191e280 4 Got MDS map 73 2024-05-11T12:15:13.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.402+0000 7f48a191e280 10 main: main 2024-05-11T12:15:13.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.402+0000 7f48a191e280 4 main: connecting to RADOS... 2024-05-11T12:15:13.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.403+0000 7f48a191e280 1 Processor -- start 2024-05-11T12:15:13.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.403+0000 7f48a191e280 1 -- start start 2024-05-11T12:15:13.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.404+0000 7f48a191e280 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585bae93400 0x5585bad23080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:13.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.404+0000 7f48a191e280 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5585bae93800 0x5585bad23600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:13.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.404+0000 7f48a191e280 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585bae93c00 0x5585bad23b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:13.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.404+0000 7f48a191e280 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5585badaa600 con 0x5585bae93400 2024-05-11T12:15:13.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.404+0000 7f48a191e280 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5585badaa780 con 0x5585bae93c00 2024-05-11T12:15:13.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.404+0000 7f48a191e280 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x5585badaa900 con 0x5585bae93800 2024-05-11T12:15:13.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.404+0000 7f489f800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585bae93400 0x5585bad23080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:13.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.404+0000 7f489f800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585bae93400 0x5585bad23080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35214/0 (socket says 172.21.15.167:35214) 2024-05-11T12:15:13.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.404+0000 7f489f800640 1 -- 172.21.15.167:0/1877859282 learned_addr learned my addr 172.21.15.167:0/1877859282 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:13.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.404+0000 7f489ee00640 1 --2- 172.21.15.167:0/1877859282 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5585bae93800 0x5585bad23600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:13.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.404+0000 7f489f800640 1 -- 172.21.15.167:0/1877859282 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5585bae93800 msgr2=0x5585bad23600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:13.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.404+0000 7f489f800640 1 --2- 172.21.15.167:0/1877859282 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5585bae93800 0x5585bad23600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:13.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.404+0000 7f489e400640 1 --2- 172.21.15.167:0/1877859282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585bae93c00 0x5585bad23b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:13.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.404+0000 7f489f800640 1 -- 172.21.15.167:0/1877859282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585bae93c00 msgr2=0x5585bad23b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:13.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.404+0000 7f489f800640 1 --2- 172.21.15.167:0/1877859282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585bae93c00 0x5585bad23b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:13.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.404+0000 7f489f800640 1 -- 172.21.15.167:0/1877859282 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585b9fee3c0 con 0x5585bae93400 2024-05-11T12:15:13.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.405+0000 7f489f800640 1 --2- 172.21.15.167:0/1877859282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585bae93400 0x5585bad23080 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x5585baeae480 tx=0x5585baeb0cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30624a84488c0cbb server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:13.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.405+0000 7f4899400640 1 -- 172.21.15.167:0/1877859282 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5585b9fe0680 con 0x5585bae93400 2024-05-11T12:15:13.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.405+0000 7f4899400640 1 -- 172.21.15.167:0/1877859282 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5585b9fe6fc0 con 0x5585bae93400 2024-05-11T12:15:13.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.405+0000 7f4899400640 1 -- 172.21.15.167:0/1877859282 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5585b9fe0b60 con 0x5585bae93400 2024-05-11T12:15:13.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.405+0000 7f48a191e280 1 -- 172.21.15.167:0/1877859282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585bae93400 msgr2=0x5585bad23080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:13.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.405+0000 7f48a191e280 1 --2- 172.21.15.167:0/1877859282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585bae93400 0x5585bad23080 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x5585baeae480 tx=0x5585baeb0cc0 comp rx=0 tx=0).stop 2024-05-11T12:15:13.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.405+0000 7f48a191e280 1 -- 172.21.15.167:0/1877859282 shutdown_connections 2024-05-11T12:15:13.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.405+0000 7f48a191e280 1 --2- 172.21.15.167:0/1877859282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585bae93c00 0x5585bad23b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:13.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.405+0000 7f48a191e280 1 --2- 172.21.15.167:0/1877859282 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5585bae93800 0x5585bad23600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:13.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.405+0000 7f48a191e280 1 --2- 172.21.15.167:0/1877859282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585bae93400 0x5585bad23080 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:13.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.405+0000 7f48a191e280 1 -- 172.21.15.167:0/1877859282 >> 172.21.15.167:0/1877859282 conn(0x5585bae93000 msgr2=0x5585ba0a5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:13.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.406+0000 7f48a191e280 1 -- 172.21.15.167:0/1877859282 shutdown_connections 2024-05-11T12:15:13.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.406+0000 7f48a191e280 1 -- 172.21.15.167:0/1877859282 wait complete. 2024-05-11T12:15:13.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.406+0000 7f48a191e280 1 Processor -- start 2024-05-11T12:15:13.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.406+0000 7f48a191e280 1 -- start start 2024-05-11T12:15:13.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.407+0000 7f48a191e280 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585bae93400 0x5585bad23080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:13.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.407+0000 7f48a191e280 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585bae93800 0x5585bad23600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:13.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.407+0000 7f48a191e280 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5585bae93c00 0x5585bad23b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:13.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.407+0000 7f48a191e280 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5585badaa900 con 0x5585bae93800 2024-05-11T12:15:13.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.407+0000 7f48a191e280 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5585badaaa80 con 0x5585bae93400 2024-05-11T12:15:13.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.407+0000 7f48a191e280 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x5585badaa780 con 0x5585bae93c00 2024-05-11T12:15:13.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.407+0000 7f489ee00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585bae93800 0x5585bad23600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:13.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.407+0000 7f489ee00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585bae93800 0x5585bad23600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35222/0 (socket says 172.21.15.167:35222) 2024-05-11T12:15:13.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.407+0000 7f489ee00640 1 -- 172.21.15.167:0/1075642813 learned_addr learned my addr 172.21.15.167:0/1075642813 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:13.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.408+0000 7f489ee00640 1 -- 172.21.15.167:0/1075642813 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5585bae93c00 msgr2=0x5585bad23b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:13.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.408+0000 7f489ee00640 1 --2- 172.21.15.167:0/1075642813 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5585bae93c00 0x5585bad23b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:13.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.408+0000 7f489ee00640 1 -- 172.21.15.167:0/1075642813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585bae93400 msgr2=0x5585bad23080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:13.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.408+0000 7f489ee00640 1 --2- 172.21.15.167:0/1075642813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585bae93400 0x5585bad23080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:13.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.408+0000 7f489ee00640 1 -- 172.21.15.167:0/1075642813 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585b9fee780 con 0x5585bae93800 2024-05-11T12:15:13.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.408+0000 7f489ee00640 1 --2- 172.21.15.167:0/1075642813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585bae93800 0x5585bad23600 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x5585baeae360 tx=0x5585bad5c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:13.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.408+0000 7f4897600640 1 -- 172.21.15.167:0/1075642813 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5585b9fe0820 con 0x5585bae93800 2024-05-11T12:15:13.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.408+0000 7f4897600640 1 -- 172.21.15.167:0/1075642813 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5585b9fe6e00 con 0x5585bae93800 2024-05-11T12:15:13.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.408+0000 7f4897600640 1 -- 172.21.15.167:0/1075642813 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5585b9fe0680 con 0x5585bae93800 2024-05-11T12:15:13.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.408+0000 7f48a191e280 1 -- 172.21.15.167:0/1075642813 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5585b9fef4a0 con 0x5585bae93800 2024-05-11T12:15:13.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.408+0000 7f48a191e280 1 -- 172.21.15.167:0/1075642813 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585b9fee960 con 0x5585bae93800 2024-05-11T12:15:13.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.409+0000 7f48a191e280 4 main: resolving pool 15 2024-05-11T12:15:13.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.410+0000 7f4897600640 1 -- 172.21.15.167:0/1075642813 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x5585baf14c00 con 0x5585bae93800 2024-05-11T12:15:13.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.411+0000 7f4897600640 1 --2- 172.21.15.167:0/1075642813 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x5585baf0dc00 0x5585bad24100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:13.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.411+0000 7f4897600640 1 -- 172.21.15.167:0/1075642813 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6187+0+0 (secure 0 0 0) 0x5585baf0ec00 con 0x5585bae93800 2024-05-11T12:15:13.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.411+0000 7f48a191e280 4 main: creating IoCtx.. 2024-05-11T12:15:13.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.411+0000 7f48a191e280 1 --2- 172.21.15.167:0/1075642813 >> [v2:172.21.15.167:6818/1902208221,v1:172.21.15.167:6820/1902208221] conn(0x5585ba360000 0x5585bad24680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:13.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.411+0000 7f48a191e280 1 -- 172.21.15.167:0/1075642813 --> [v2:172.21.15.167:6818/1902208221,v1:172.21.15.167:6820/1902208221] -- osd_op(unknown.0.0:1 15.b 15:d0630e4c:::mds0_sessionmap:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101) -- 0x5585ba360400 con 0x5585ba360000 2024-05-11T12:15:13.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.411+0000 7f489f800640 1 --2- 172.21.15.167:0/1075642813 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x5585baf0dc00 0x5585bad24100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:13.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.412+0000 7f489e400640 1 --2- 172.21.15.167:0/1075642813 >> [v2:172.21.15.167:6818/1902208221,v1:172.21.15.167:6820/1902208221] conn(0x5585ba360000 0x5585bad24680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:13.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.412+0000 7f489f800640 1 --2- 172.21.15.167:0/1075642813 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x5585baf0dc00 0x5585bad24100 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x5585baeae480 tx=0x5585baeb0c00 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:13.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.412+0000 7f489e400640 1 --2- 172.21.15.167:0/1075642813 >> [v2:172.21.15.167:6818/1902208221,v1:172.21.15.167:6820/1902208221] conn(0x5585ba360000 0x5585bad24680 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:13.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.413+0000 7f489e400640 1 -- 172.21.15.167:0/1075642813 <== osd.2 v2:172.21.15.167:6818/1902208221 1 ==== osd_op_reply(1 mds0_sessionmap [omap-get-header out=14b] v0'0 uv10 ondisk = 0) ==== 159+0+14 (crc 0 0 0) 0x5585bad48000 con 0x5585ba360000 2024-05-11T12:15:13.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.413+0000 7f48a191e280 1 -- 172.21.15.167:0/1075642813 --> [v2:172.21.15.167:6818/1902208221,v1:172.21.15.167:6820/1902208221] -- osd_op(unknown.0.0:2 15.b 15:d0630e4c:::mds0_sessionmap:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101) -- 0x5585ba360800 con 0x5585ba360000 2024-05-11T12:15:13.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.413+0000 7f489e400640 1 -- 172.21.15.167:0/1075642813 <== osd.2 v2:172.21.15.167:6818/1902208221 2 ==== osd_op_reply(2 mds0_sessionmap [omap-get-vals out=5b] v0'0 uv10 ondisk = 0) ==== 159+0+5 (crc 0 0 0) 0x5585bad48000 con 0x5585ba360000 2024-05-11T12:15:13.415 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T12:15:13.415 INFO:teuthology.orchestra.run.smithi167.stdout: "0": { 2024-05-11T12:15:13.416 INFO:teuthology.orchestra.run.smithi167.stdout: "data": { 2024-05-11T12:15:13.416 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions": [] 2024-05-11T12:15:13.416 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T12:15:13.416 INFO:teuthology.orchestra.run.smithi167.stdout: "result": 0 2024-05-11T12:15:13.416 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T12:15:13.416 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T12:15:13.416 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:15:13.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.413+0000 7f48a191e280 1 -- 172.21.15.167:0/1075642813 >> [v2:172.21.15.167:6818/1902208221,v1:172.21.15.167:6820/1902208221] conn(0x5585ba360000 msgr2=0x5585bad24680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:13.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.413+0000 7f48a191e280 1 --2- 172.21.15.167:0/1075642813 >> [v2:172.21.15.167:6818/1902208221,v1:172.21.15.167:6820/1902208221] conn(0x5585ba360000 0x5585bad24680 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:13.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.413+0000 7f48a191e280 1 -- 172.21.15.167:0/1075642813 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x5585baf0dc00 msgr2=0x5585bad24100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:13.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.413+0000 7f48a191e280 1 --2- 172.21.15.167:0/1075642813 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x5585baf0dc00 0x5585bad24100 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x5585baeae480 tx=0x5585baeb0c00 comp rx=0 tx=0).stop 2024-05-11T12:15:13.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.413+0000 7f48a191e280 1 -- 172.21.15.167:0/1075642813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585bae93800 msgr2=0x5585bad23600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:13.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.413+0000 7f48a191e280 1 --2- 172.21.15.167:0/1075642813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585bae93800 0x5585bad23600 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x5585baeae360 tx=0x5585bad5c000 comp rx=0 tx=0).stop 2024-05-11T12:15:13.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.413+0000 7f489e400640 1 -- 172.21.15.167:0/1075642813 reap_dead start 2024-05-11T12:15:13.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.413+0000 7f48a191e280 1 -- 172.21.15.167:0/1075642813 shutdown_connections 2024-05-11T12:15:13.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.413+0000 7f48a191e280 1 -- 172.21.15.167:0/1075642813 >> 172.21.15.167:0/1075642813 conn(0x5585bae93000 msgr2=0x5585ba0a5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:13.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.413+0000 7f48a191e280 1 -- 172.21.15.167:0/1075642813 shutdown_connections 2024-05-11T12:15:13.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.414+0000 7f48a191e280 1 -- 172.21.15.167:0/1075642813 wait complete. 2024-05-11T12:15:13.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.414+0000 7f48a191e280 1 -- 172.21.15.167:0/1599139692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585b9fbac00 msgr2=0x5585bad22000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:13.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.414+0000 7f48a191e280 1 --2- 172.21.15.167:0/1599139692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585b9fbac00 0x5585bad22000 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x5585bade54a0 tx=0x5585bad5cfc0 comp rx=0 tx=0).stop 2024-05-11T12:15:13.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.414+0000 7f48a191e280 1 -- 172.21.15.167:0/1599139692 shutdown_connections 2024-05-11T12:15:13.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.414+0000 7f48a191e280 1 --2- 172.21.15.167:0/1599139692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585b9fbbc00 0x5585bad22b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:13.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.414+0000 7f48a191e280 1 --2- 172.21.15.167:0/1599139692 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5585b9fbb800 0x5585bad22580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:13.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.414+0000 7f48a191e280 1 --2- 172.21.15.167:0/1599139692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585b9fbac00 0x5585bad22000 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:13.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.414+0000 7f48a191e280 1 -- 172.21.15.167:0/1599139692 >> 172.21.15.167:0/1599139692 conn(0x5585b9fbb000 msgr2=0x5585ba0a5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:13.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.414+0000 7f48a191e280 1 -- 172.21.15.167:0/1599139692 shutdown_connections 2024-05-11T12:15:13.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:13.414+0000 7f48a191e280 1 -- 172.21.15.167:0/1599139692 wait complete. 2024-05-11T12:15:13.420 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '0', 'show', 'session'] in time 0:00:00.119335, result: { "0": { "data": { "sessions": [] }, "result": 0 } } 2024-05-11T12:15:13.421 INFO:tasks.cephfs.test_sessionmap:SessionMap: { "0": { "data": { "sessions": [] }, "result": 0 } } 2024-05-11T12:15:13.421 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-11T12:15:13.421 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-11T12:15:13.515 DEBUG:teuthology.orchestra.run.smithi204:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-11T12:15:13.621 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-11T12:15:13.621 INFO:tasks.cephfs.mount:Terminating background process 2024-05-11T12:15:13.621 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:15:13.621 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:15:13.678 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-11T12:15:13.678 INFO:tasks.cephfs.mount:Terminating background process 2024-05-11T12:15:13.678 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:15:13.678 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:15:13.735 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052713,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-11T12:15:13.735 INFO:tasks.cephfs.mount:Terminating background process 2024-05-11T12:15:13.736 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:15:13.736 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:15:13.792 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-11T12:15:13.792 INFO:tasks.cephfs.mount:Terminating background process 2024-05-11T12:15:13.792 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:15:13.792 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:15:13.848 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052713,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-11T12:15:13.848 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:15:14.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.051+0000 7ffb9b600640 1 Processor -- start 2024-05-11T12:15:14.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.051+0000 7ffb9b600640 1 -- start start 2024-05-11T12:15:14.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.053+0000 7ffb9b600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ffb94073a00 0x7ffb94073e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:14.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.053+0000 7ffb9b600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb9415e880 0x7ffb9415ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:14.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.053+0000 7ffb9b600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb94072520 con 0x7ffb940b62c0 2024-05-11T12:15:14.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.053+0000 7ffb9b600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7ffb940732e0 con 0x7ffb94073a00 2024-05-11T12:15:14.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.053+0000 7ffb9b600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ffb940722d0 con 0x7ffb9415e880 2024-05-11T12:15:14.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.054+0000 7ffb99600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb9415e880 0x7ffb9415ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:14.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.054+0000 7ffb99600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb9415e880 0x7ffb9415ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:48158/0 (socket says 172.21.15.167:48158) 2024-05-11T12:15:14.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.054+0000 7ffb99600640 1 -- 172.21.15.167:0/882651103 learned_addr learned my addr 172.21.15.167:0/882651103 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:14.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.054+0000 7ffb93e00640 1 --2- 172.21.15.167:0/882651103 >> v2:172.21.15.204:3300/0 conn(0x7ffb94073a00 0x7ffb94073e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:14.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.055+0000 7ffb99600640 1 -- 172.21.15.167:0/882651103 >> v2:172.21.15.204:3300/0 conn(0x7ffb94073a00 msgr2=0x7ffb94073e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:14.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.055+0000 7ffb99600640 1 --2- 172.21.15.167:0/882651103 >> v2:172.21.15.204:3300/0 conn(0x7ffb94073a00 0x7ffb94073e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:14.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.055+0000 7ffb99600640 1 -- 172.21.15.167:0/882651103 >> v1:172.21.15.204:6789/0 conn(0x7ffb940b62c0 legacy=0x7ffb940b66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:14.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.055+0000 7ffb99600640 1 -- 172.21.15.167:0/882651103 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb94072800 con 0x7ffb9415e880 2024-05-11T12:15:14.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.056+0000 7ffb99600640 1 --2- 172.21.15.167:0/882651103 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb9415e880 0x7ffb9415ed10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c009870 tx=0x7ffb8c02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b7e8febd63d14ba server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:14.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.057+0000 7ffb93400640 1 -- 172.21.15.167:0/882651103 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c037070 con 0x7ffb9415e880 2024-05-11T12:15:14.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.057+0000 7ffb93400640 1 -- 172.21.15.167:0/882651103 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb8c032440 con 0x7ffb9415e880 2024-05-11T12:15:14.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.057+0000 7ffb93400640 1 -- 172.21.15.167:0/882651103 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c032810 con 0x7ffb9415e880 2024-05-11T12:15:14.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.057+0000 7ffb9b600640 1 -- 172.21.15.167:0/882651103 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb9415e880 msgr2=0x7ffb9415ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:14.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.057+0000 7ffb9b600640 1 --2- 172.21.15.167:0/882651103 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb9415e880 0x7ffb9415ed10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c009870 tx=0x7ffb8c02a270 comp rx=0 tx=0).stop 2024-05-11T12:15:14.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.058+0000 7ffb9b600640 1 -- 172.21.15.167:0/882651103 shutdown_connections 2024-05-11T12:15:14.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.058+0000 7ffb9b600640 1 --2- 172.21.15.167:0/882651103 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb9415e880 0x7ffb9415ed10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:14.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.058+0000 7ffb9b600640 1 --2- 172.21.15.167:0/882651103 >> v2:172.21.15.204:3300/0 conn(0x7ffb94073a00 0x7ffb94073e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:14.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.058+0000 7ffb9b600640 1 -- 172.21.15.167:0/882651103 >> 172.21.15.167:0/882651103 conn(0x7ffb940b2cb0 msgr2=0x7ffb940b3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:14.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.058+0000 7ffb9b600640 1 -- 172.21.15.167:0/882651103 shutdown_connections 2024-05-11T12:15:14.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.058+0000 7ffb9b600640 1 -- 172.21.15.167:0/882651103 wait complete. 2024-05-11T12:15:14.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.060+0000 7ffb9b600640 1 Processor -- start 2024-05-11T12:15:14.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.060+0000 7ffb9b600640 1 -- start start 2024-05-11T12:15:14.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.061+0000 7ffb9b600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb94073a00 0x7ffb941f56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:14.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.061+0000 7ffb9b600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb940b62c0 0x7ffb94151f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:14.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.061+0000 7ffb9b600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb9415e880 0x7ffb941f5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:14.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.061+0000 7ffb9b600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ffb940743c0 con 0x7ffb940b62c0 2024-05-11T12:15:14.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.061+0000 7ffb9b600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ffb94072dc0 con 0x7ffb94073a00 2024-05-11T12:15:14.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.061+0000 7ffb9b600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ffb941579c0 con 0x7ffb9415e880 2024-05-11T12:15:14.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.062+0000 7ffb93e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb940b62c0 0x7ffb94151f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:14.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.062+0000 7ffb93e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb940b62c0 0x7ffb94151f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35226/0 (socket says 172.21.15.167:35226) 2024-05-11T12:15:14.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.062+0000 7ffb93e00640 1 -- 172.21.15.167:0/2674148165 learned_addr learned my addr 172.21.15.167:0/2674148165 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:14.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.062+0000 7ffb98c00640 1 --2- 172.21.15.167:0/2674148165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb94073a00 0x7ffb941f56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:14.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.062+0000 7ffb99600640 1 --2- 172.21.15.167:0/2674148165 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb9415e880 0x7ffb941f5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:14.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.062+0000 7ffb93e00640 1 -- 172.21.15.167:0/2674148165 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb9415e880 msgr2=0x7ffb941f5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:14.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.062+0000 7ffb93e00640 1 --2- 172.21.15.167:0/2674148165 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb9415e880 0x7ffb941f5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:14.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.062+0000 7ffb93e00640 1 -- 172.21.15.167:0/2674148165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb94073a00 msgr2=0x7ffb941f56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:14.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.062+0000 7ffb93e00640 1 --2- 172.21.15.167:0/2674148165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb94073a00 0x7ffb941f56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:14.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.062+0000 7ffb93e00640 1 -- 172.21.15.167:0/2674148165 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb941f5be0 con 0x7ffb940b62c0 2024-05-11T12:15:14.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.062+0000 7ffb99600640 1 --2- 172.21.15.167:0/2674148165 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb9415e880 0x7ffb941f5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:15:14.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.062+0000 7ffb93e00640 1 --2- 172.21.15.167:0/2674148165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb940b62c0 0x7ffb94151f50 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c00d830 tx=0x7ffb7c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:14.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.063+0000 7ffb90c00640 1 -- 172.21.15.167:0/2674148165 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c00b6c0 con 0x7ffb940b62c0 2024-05-11T12:15:14.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.063+0000 7ffb90c00640 1 -- 172.21.15.167:0/2674148165 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb7c019070 con 0x7ffb940b62c0 2024-05-11T12:15:14.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.063+0000 7ffb90c00640 1 -- 172.21.15.167:0/2674148165 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c00ba80 con 0x7ffb940b62c0 2024-05-11T12:15:14.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.063+0000 7ffb9b600640 1 -- 172.21.15.167:0/2674148165 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb940b45d0 con 0x7ffb940b62c0 2024-05-11T12:15:14.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.063+0000 7ffb9b600640 1 -- 172.21.15.167:0/2674148165 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb940b4e10 con 0x7ffb940b62c0 2024-05-11T12:15:14.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.064+0000 7ffb9b600640 1 -- 172.21.15.167:0/2674148165 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb94069820 con 0x7ffb940b62c0 2024-05-11T12:15:14.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.065+0000 7ffb90c00640 1 -- 172.21.15.167:0/2674148165 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ffb7c01e460 con 0x7ffb940b62c0 2024-05-11T12:15:14.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.066+0000 7ffb90c00640 1 --2- 172.21.15.167:0/2674148165 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ffb700762d0 0x7ffb70078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:14.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.066+0000 7ffb90c00640 1 -- 172.21.15.167:0/2674148165 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6187+0+0 (secure 0 0 0) 0x7ffb7c097e30 con 0x7ffb940b62c0 2024-05-11T12:15:14.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.066+0000 7ffb98c00640 1 --2- 172.21.15.167:0/2674148165 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ffb700762d0 0x7ffb70078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:14.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.067+0000 7ffb98c00640 1 --2- 172.21.15.167:0/2674148165 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ffb700762d0 0x7ffb70078790 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ffb84007fd0 tx=0x7ffb84009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:14.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.070+0000 7ffb90c00640 1 -- 172.21.15.167:0/2674148165 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ffb940b45d0 con 0x7ffb940b62c0 2024-05-11T12:15:14.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.317+0000 7ffb9b600640 1 -- 172.21.15.167:0/2674148165 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb94156700 con 0x7ffb940b62c0 2024-05-11T12:15:14.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.318+0000 7ffb90c00640 1 -- 172.21.15.167:0/2674148165 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5217 (secure 0 0 0) 0x7ffb941f5be0 con 0x7ffb940b62c0 2024-05-11T12:15:14.319 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:15:14.319 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":73,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5401,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/2213070102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2213070102},{"type":"v1","addr":"172.21.15.204:6833","nonce":2213070102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5410,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1480900470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1480900470},{"type":"v1","addr":"172.21.15.204:6835","nonce":1480900470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:14:33.067226+0000","modified":"2024-05-11T12:15:05.949679+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6885,"mds_1":6870},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6870":{"gid":6870,"name":"a","rank":1,"incarnation":71,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6835/3735016882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3735016882},{"type":"v1","addr":"172.21.15.167:6835","nonce":3735016882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6885":{"gid":6885,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6837/2901182170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2901182170},{"type":"v1","addr":"172.21.15.167:6837","nonce":2901182170}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6885,"qdb_cluster":[6885,6870]},"id":7}]} 2024-05-11T12:15:14.319 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 73 2024-05-11T12:15:14.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.321+0000 7ffb9b600640 1 -- 172.21.15.167:0/2674148165 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ffb700762d0 msgr2=0x7ffb70078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:14.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.321+0000 7ffb9b600640 1 --2- 172.21.15.167:0/2674148165 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ffb700762d0 0x7ffb70078790 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ffb84007fd0 tx=0x7ffb84009040 comp rx=0 tx=0).stop 2024-05-11T12:15:14.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.321+0000 7ffb9b600640 1 -- 172.21.15.167:0/2674148165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb940b62c0 msgr2=0x7ffb94151f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:14.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.321+0000 7ffb9b600640 1 --2- 172.21.15.167:0/2674148165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb940b62c0 0x7ffb94151f50 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c00d830 tx=0x7ffb7c00dcf0 comp rx=0 tx=0).stop 2024-05-11T12:15:14.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.321+0000 7ffb9b600640 1 -- 172.21.15.167:0/2674148165 shutdown_connections 2024-05-11T12:15:14.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.321+0000 7ffb9b600640 1 --2- 172.21.15.167:0/2674148165 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb9415e880 0x7ffb941f5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:14.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.321+0000 7ffb9b600640 1 --2- 172.21.15.167:0/2674148165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb940b62c0 0x7ffb94151f50 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:14.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.321+0000 7ffb9b600640 1 --2- 172.21.15.167:0/2674148165 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ffb700762d0 0x7ffb70078790 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:14.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.321+0000 7ffb9b600640 1 --2- 172.21.15.167:0/2674148165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb94073a00 0x7ffb941f56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:14.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.321+0000 7ffb9b600640 1 -- 172.21.15.167:0/2674148165 >> 172.21.15.167:0/2674148165 conn(0x7ffb940b2cb0 msgr2=0x7ffb940ceb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:14.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.321+0000 7ffb9b600640 1 -- 172.21.15.167:0/2674148165 shutdown_connections 2024-05-11T12:15:14.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.321+0000 7ffb9b600640 1 -- 172.21.15.167:0/2674148165 wait complete. 2024-05-11T12:15:14.340 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:15:14.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.539+0000 7f8ec9600640 1 Processor -- start 2024-05-11T12:15:14.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.539+0000 7f8ec9600640 1 -- start start 2024-05-11T12:15:14.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.541+0000 7f8ec9600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8ec415b6d0 0x7f8ec40cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:14.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.541+0000 7f8ec9600640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ec4072c00 con 0x7f8ec40b6c40 2024-05-11T12:15:14.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.541+0000 7f8ec9600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ec4072ee0 con 0x7f8ec40b6070 2024-05-11T12:15:14.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.541+0000 7f8ec9600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f8ec40b43c0 con 0x7f8ec415b6d0 2024-05-11T12:15:14.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.541+0000 7f8ec3400640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f8ec40b6c40 0x7f8ec40739c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:41084/0 (socket says 172.21.15.167:41084) 2024-05-11T12:15:14.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.541+0000 7f8ec3400640 1 -- 172.21.15.167:0/339425776 learned_addr learned my addr 172.21.15.167:0/339425776 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:15:14.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.541+0000 7f8ec2a00640 1 --2- 172.21.15.167:0/339425776 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8ec415b6d0 0x7f8ec40cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:14.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.542+0000 7f8ec1600640 1 -- 172.21.15.167:0/339425776 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1222892406 0 0) 0x7f8ec4072c00 con 0x7f8ec40b6c40 2024-05-11T12:15:14.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.542+0000 7f8ec1600640 1 -- 172.21.15.167:0/339425776 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ea8004630 con 0x7f8ec40b6c40 2024-05-11T12:15:14.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.542+0000 7f8ec1600640 1 -- 172.21.15.167:0/339425776 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4244586925 0 0) 0x7f8ea8004630 con 0x7f8ec40b6c40 2024-05-11T12:15:14.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.542+0000 7f8ec1600640 1 -- 172.21.15.167:0/339425776 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8ec415b6d0 msgr2=0x7f8ec40cefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:14.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.542+0000 7f8ec1600640 1 --2- 172.21.15.167:0/339425776 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8ec415b6d0 0x7f8ec40cefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:14.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.542+0000 7f8ec1600640 1 -- 172.21.15.167:0/339425776 >> v1:172.21.15.204:6789/0 conn(0x7f8ec40b6070 legacy=0x7f8ec40b6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:14.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.542+0000 7f8ec1600640 1 -- 172.21.15.167:0/339425776 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ec40731c0 con 0x7f8ec40b6c40 2024-05-11T12:15:14.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.542+0000 7f8ec1600640 1 -- 172.21.15.167:0/339425776 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f8ebc001a70 con 0x7f8ec40b6c40 2024-05-11T12:15:14.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.542+0000 7f8ec1600640 1 -- 172.21.15.167:0/339425776 >> v1:172.21.15.167:6789/0 conn(0x7f8ec40b6c40 legacy=0x7f8ec40739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:14.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.542+0000 7f8ec1600640 1 --2- 172.21.15.167:0/339425776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ea8004b40 0x7f8ea8004f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:14.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.542+0000 7f8ec1600640 1 --2- 172.21.15.167:0/339425776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ea8005910 0x7f8ea8005d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:14.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.542+0000 7f8ec1600640 1 --2- 172.21.15.167:0/339425776 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8ec415b6d0 0x7f8ea8006310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:14.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.542+0000 7f8ec1600640 1 -- 172.21.15.167:0/339425776 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec40b43c0 con 0x7f8ea8004b40 2024-05-11T12:15:14.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.542+0000 7f8ec1600640 1 -- 172.21.15.167:0/339425776 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea800cc00 con 0x7f8ea8005910 2024-05-11T12:15:14.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.542+0000 7f8ec1600640 1 -- 172.21.15.167:0/339425776 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f8ea800ce00 con 0x7f8ec415b6d0 2024-05-11T12:15:14.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.542+0000 7f8ec2a00640 1 --2- 172.21.15.167:0/339425776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ea8004b40 0x7f8ea8004f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:14.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.543+0000 7f8ec2000640 1 --2- 172.21.15.167:0/339425776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ea8005910 0x7f8ea8005d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:14.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.543+0000 7f8ec3400640 1 --2- 172.21.15.167:0/339425776 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8ec415b6d0 0x7f8ea8006310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:14.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.543+0000 7f8ec2a00640 1 -- 172.21.15.167:0/339425776 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8ec415b6d0 msgr2=0x7f8ea8006310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:14.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.543+0000 7f8ec2a00640 1 --2- 172.21.15.167:0/339425776 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8ec415b6d0 0x7f8ea8006310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:14.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.543+0000 7f8ec2a00640 1 -- 172.21.15.167:0/339425776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ea8005910 msgr2=0x7f8ea8005d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:14.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.543+0000 7f8ec2a00640 1 --2- 172.21.15.167:0/339425776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ea8005910 0x7f8ea8005d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:14.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.543+0000 7f8ec2a00640 1 -- 172.21.15.167:0/339425776 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ea8004630 con 0x7f8ea8004b40 2024-05-11T12:15:14.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.544+0000 7f8ec2a00640 1 --2- 172.21.15.167:0/339425776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ea8004b40 0x7f8ea8004f60 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4009ac0 tx=0x7f8ea8006940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b38873fa38cd7b42 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:14.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.544+0000 7f8ec1600640 1 -- 172.21.15.167:0/339425776 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb4036070 con 0x7f8ea8004b40 2024-05-11T12:15:14.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.544+0000 7f8ec1600640 1 -- 172.21.15.167:0/339425776 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eb4031400 con 0x7f8ea8004b40 2024-05-11T12:15:14.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.544+0000 7f8ec1600640 1 -- 172.21.15.167:0/339425776 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb4009690 con 0x7f8ea8004b40 2024-05-11T12:15:14.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.545+0000 7f8ec9600640 1 -- 172.21.15.167:0/339425776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ea8004b40 msgr2=0x7f8ea8004f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:14.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.545+0000 7f8ec9600640 1 --2- 172.21.15.167:0/339425776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ea8004b40 0x7f8ea8004f60 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4009ac0 tx=0x7f8ea8006940 comp rx=0 tx=0).stop 2024-05-11T12:15:14.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.545+0000 7f8ec3400640 1 -- 172.21.15.167:0/339425776 reap_dead start 2024-05-11T12:15:14.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.545+0000 7f8ec9600640 1 -- 172.21.15.167:0/339425776 shutdown_connections 2024-05-11T12:15:14.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.545+0000 7f8ec9600640 1 -- 172.21.15.167:0/339425776 >> 172.21.15.167:0/339425776 conn(0x7f8ec40b2cb0 msgr2=0x7f8ec40b3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:14.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.545+0000 7f8ec9600640 1 -- 172.21.15.167:0/339425776 shutdown_connections 2024-05-11T12:15:14.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.545+0000 7f8ec9600640 1 -- 172.21.15.167:0/339425776 wait complete. 2024-05-11T12:15:14.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.547+0000 7f8ec9600640 1 Processor -- start 2024-05-11T12:15:14.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.547+0000 7f8ec9600640 1 -- start start 2024-05-11T12:15:14.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.548+0000 7f8ec9600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ea8004b40 0x7f8ec4144e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:14.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.548+0000 7f8ec9600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8ec414bc40 0x7f8ec4146680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:14.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.548+0000 7f8ec9600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ec4148a80 0x7f8ec41478e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:14.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.548+0000 7f8ec9600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec40b4610 con 0x7f8ec4148a80 2024-05-11T12:15:14.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.548+0000 7f8ec9600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec415bab0 con 0x7f8ea8004b40 2024-05-11T12:15:14.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.548+0000 7f8ec9600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f8ec40b0720 con 0x7f8ec414bc40 2024-05-11T12:15:14.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.548+0000 7f8ec3400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ec4148a80 0x7f8ec41478e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:14.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.548+0000 7f8ec3400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ec4148a80 0x7f8ec41478e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35238/0 (socket says 172.21.15.167:35238) 2024-05-11T12:15:14.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.548+0000 7f8ec3400640 1 -- 172.21.15.167:0/27723989 learned_addr learned my addr 172.21.15.167:0/27723989 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:14.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.548+0000 7f8ec2000640 1 --2- 172.21.15.167:0/27723989 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8ec414bc40 0x7f8ec4146680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:14.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.548+0000 7f8ec2a00640 1 --2- 172.21.15.167:0/27723989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ea8004b40 0x7f8ec4144e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:14.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.548+0000 7f8ec3400640 1 -- 172.21.15.167:0/27723989 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8ec414bc40 msgr2=0x7f8ec4146680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:14.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.548+0000 7f8ec3400640 1 --2- 172.21.15.167:0/27723989 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8ec414bc40 0x7f8ec4146680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:14.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.548+0000 7f8ec3400640 1 -- 172.21.15.167:0/27723989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ea8004b40 msgr2=0x7f8ec4144e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:14.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.548+0000 7f8ec3400640 1 --2- 172.21.15.167:0/27723989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ea8004b40 0x7f8ec4144e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:14.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.548+0000 7f8ec3400640 1 -- 172.21.15.167:0/27723989 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ec4147590 con 0x7f8ec4148a80 2024-05-11T12:15:14.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.549+0000 7f8ec2a00640 1 --2- 172.21.15.167:0/27723989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ea8004b40 0x7f8ec4144e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:15:14.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.549+0000 7f8ec3400640 1 --2- 172.21.15.167:0/27723989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ec4148a80 0x7f8ec41478e0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4074920 tx=0x7f8ebc00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:14.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.549+0000 7f8ebaa00640 1 -- 172.21.15.167:0/27723989 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc00bb80 con 0x7f8ec4148a80 2024-05-11T12:15:14.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.549+0000 7f8ebaa00640 1 -- 172.21.15.167:0/27723989 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ebc00bd20 con 0x7f8ec4148a80 2024-05-11T12:15:14.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.549+0000 7f8ebaa00640 1 -- 172.21.15.167:0/27723989 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc007330 con 0x7f8ec4148a80 2024-05-11T12:15:14.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.549+0000 7f8ec9600640 1 -- 172.21.15.167:0/27723989 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ec4146ff0 con 0x7f8ec4148a80 2024-05-11T12:15:14.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.549+0000 7f8ec9600640 1 -- 172.21.15.167:0/27723989 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ec414f910 con 0x7f8ec4148a80 2024-05-11T12:15:14.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.550+0000 7f8ec9600640 1 -- 172.21.15.167:0/27723989 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ec4069820 con 0x7f8ec4148a80 2024-05-11T12:15:14.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.552+0000 7f8ebaa00640 1 -- 172.21.15.167:0/27723989 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8ebc01b020 con 0x7f8ec4148a80 2024-05-11T12:15:14.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.553+0000 7f8ebaa00640 1 --2- 172.21.15.167:0/27723989 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f8ea4076300 0x7f8ea40787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:14.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.553+0000 7f8ebaa00640 1 -- 172.21.15.167:0/27723989 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6187+0+0 (secure 0 0 0) 0x7f8ebc097a90 con 0x7f8ec4148a80 2024-05-11T12:15:14.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.553+0000 7f8ec2a00640 1 --2- 172.21.15.167:0/27723989 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f8ea4076300 0x7f8ea40787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:14.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.554+0000 7f8ec2a00640 1 --2- 172.21.15.167:0/27723989 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f8ea4076300 0x7f8ea40787c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4033000 tx=0x7f8eb4008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:14.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.558+0000 7f8ebaa00640 1 -- 172.21.15.167:0/27723989 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8ebc061200 con 0x7f8ec4148a80 2024-05-11T12:15:14.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.803+0000 7f8ec9600640 1 -- 172.21.15.167:0/27723989 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ec4166ea0 con 0x7f8ec4148a80 2024-05-11T12:15:14.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.804+0000 7f8ebaa00640 1 -- 172.21.15.167:0/27723989 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5217 (secure 0 0 0) 0x7f8ebc060b00 con 0x7f8ec4148a80 2024-05-11T12:15:14.805 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:15:14.805 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":73,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5401,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/2213070102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2213070102},{"type":"v1","addr":"172.21.15.204:6833","nonce":2213070102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5410,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1480900470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1480900470},{"type":"v1","addr":"172.21.15.204:6835","nonce":1480900470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:14:33.067226+0000","modified":"2024-05-11T12:15:05.949679+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6885,"mds_1":6870},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6870":{"gid":6870,"name":"a","rank":1,"incarnation":71,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6835/3735016882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3735016882},{"type":"v1","addr":"172.21.15.167:6835","nonce":3735016882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6885":{"gid":6885,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6837/2901182170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2901182170},{"type":"v1","addr":"172.21.15.167:6837","nonce":2901182170}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6885,"qdb_cluster":[6885,6870]},"id":7}]} 2024-05-11T12:15:14.805 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 73 2024-05-11T12:15:14.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.807+0000 7f8ec9600640 1 -- 172.21.15.167:0/27723989 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f8ea4076300 msgr2=0x7f8ea40787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:14.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.807+0000 7f8ec9600640 1 --2- 172.21.15.167:0/27723989 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f8ea4076300 0x7f8ea40787c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4033000 tx=0x7f8eb4008030 comp rx=0 tx=0).stop 2024-05-11T12:15:14.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.807+0000 7f8ec9600640 1 -- 172.21.15.167:0/27723989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ec4148a80 msgr2=0x7f8ec41478e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:14.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.807+0000 7f8ec9600640 1 --2- 172.21.15.167:0/27723989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ec4148a80 0x7f8ec41478e0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4074920 tx=0x7f8ebc00b4a0 comp rx=0 tx=0).stop 2024-05-11T12:15:14.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.807+0000 7f8ec9600640 1 -- 172.21.15.167:0/27723989 shutdown_connections 2024-05-11T12:15:14.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.807+0000 7f8ec9600640 1 --2- 172.21.15.167:0/27723989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ec4148a80 0x7f8ec41478e0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:14.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.807+0000 7f8ec9600640 1 --2- 172.21.15.167:0/27723989 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8ec414bc40 0x7f8ec4146680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:14.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.807+0000 7f8ec9600640 1 --2- 172.21.15.167:0/27723989 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f8ea4076300 0x7f8ea40787c0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:14.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.807+0000 7f8ec9600640 1 --2- 172.21.15.167:0/27723989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ea8004b40 0x7f8ec4144e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:14.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.807+0000 7f8ec9600640 1 -- 172.21.15.167:0/27723989 >> 172.21.15.167:0/27723989 conn(0x7f8ec40b2cb0 msgr2=0x7f8ec41525b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:14.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.807+0000 7f8ec9600640 1 -- 172.21.15.167:0/27723989 shutdown_connections 2024-05-11T12:15:14.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:14.807+0000 7f8ec9600640 1 -- 172.21.15.167:0/27723989 wait complete. 2024-05-11T12:15:14.826 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:15:15.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.025+0000 7fd7ffe00640 1 Processor -- start 2024-05-11T12:15:15.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.025+0000 7fd7ffe00640 1 -- start start 2024-05-11T12:15:15.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.027+0000 7fd7ffe00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd7f80b62c0 0x7fd7f80b66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:15.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.027+0000 7fd7ffe00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd7f8073a00 0x7fd7f8073e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:15.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.027+0000 7fd7ffe00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7f80b6be0 con 0x7fd7f80ce530 2024-05-11T12:15:15.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.027+0000 7fd7ffe00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fd7f80732e0 con 0x7fd7f8073a00 2024-05-11T12:15:15.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.027+0000 7fd7ffe00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fd7f80722d0 con 0x7fd7f80b62c0 2024-05-11T12:15:15.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.027+0000 7fd7fca00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd7f8073a00 0x7fd7f8073e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:15.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.027+0000 7fd7fca00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd7f8073a00 0x7fd7f8073e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35254/0 (socket says 172.21.15.167:35254) 2024-05-11T12:15:15.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.027+0000 7fd7fca00640 1 -- 172.21.15.167:0/1320251067 learned_addr learned my addr 172.21.15.167:0/1320251067 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:15.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.028+0000 7fd7fd400640 1 --2- 172.21.15.167:0/1320251067 >> v2:172.21.15.204:3300/0 conn(0x7fd7f80b62c0 0x7fd7f80b66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:15.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.028+0000 7fd7fca00640 1 -- 172.21.15.167:0/1320251067 >> v2:172.21.15.204:3300/0 conn(0x7fd7f80b62c0 msgr2=0x7fd7f80b66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:15.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.028+0000 7fd7fca00640 1 --2- 172.21.15.167:0/1320251067 >> v2:172.21.15.204:3300/0 conn(0x7fd7f80b62c0 0x7fd7f80b66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:15.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.028+0000 7fd7fca00640 1 -- 172.21.15.167:0/1320251067 >> v1:172.21.15.167:6789/0 conn(0x7fd7f80ce530 legacy=0x7fd7f80ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:15.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.028+0000 7fd7fca00640 1 -- 172.21.15.167:0/1320251067 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f8072520 con 0x7fd7f8073a00 2024-05-11T12:15:15.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.029+0000 7fd7fca00640 1 --2- 172.21.15.167:0/1320251067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7f8073a00 0x7fd7f8073e80 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fd7f806dce0 tx=0x7fd7e402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d7fcb30ebec35ab server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:15.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.029+0000 7fd7f7e00640 1 -- 172.21.15.167:0/1320251067 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e4037070 con 0x7fd7f8073a00 2024-05-11T12:15:15.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.029+0000 7fd7f7e00640 1 -- 172.21.15.167:0/1320251067 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7e4032440 con 0x7fd7f8073a00 2024-05-11T12:15:15.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.029+0000 7fd7f7e00640 1 -- 172.21.15.167:0/1320251067 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e4032810 con 0x7fd7f8073a00 2024-05-11T12:15:15.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.030+0000 7fd7ffe00640 1 -- 172.21.15.167:0/1320251067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7f8073a00 msgr2=0x7fd7f8073e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:15.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.030+0000 7fd7ffe00640 1 --2- 172.21.15.167:0/1320251067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7f8073a00 0x7fd7f8073e80 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fd7f806dce0 tx=0x7fd7e402a270 comp rx=0 tx=0).stop 2024-05-11T12:15:15.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.030+0000 7fd7ffe00640 1 -- 172.21.15.167:0/1320251067 shutdown_connections 2024-05-11T12:15:15.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.030+0000 7fd7ffe00640 1 --2- 172.21.15.167:0/1320251067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7f8073a00 0x7fd7f8073e80 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:15.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.030+0000 7fd7ffe00640 1 --2- 172.21.15.167:0/1320251067 >> v2:172.21.15.204:3300/0 conn(0x7fd7f80b62c0 0x7fd7f80b66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:15.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.030+0000 7fd7ffe00640 1 -- 172.21.15.167:0/1320251067 >> 172.21.15.167:0/1320251067 conn(0x7fd7f80b2cb0 msgr2=0x7fd7f80b3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:15.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.030+0000 7fd7ffe00640 1 -- 172.21.15.167:0/1320251067 shutdown_connections 2024-05-11T12:15:15.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.030+0000 7fd7ffe00640 1 -- 172.21.15.167:0/1320251067 wait complete. 2024-05-11T12:15:15.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.032+0000 7fd7ffe00640 1 Processor -- start 2024-05-11T12:15:15.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.032+0000 7fd7ffe00640 1 -- start start 2024-05-11T12:15:15.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.033+0000 7fd7ffe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7f8073a00 0x7fd7f8145670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:15.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.033+0000 7fd7ffe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7f80b62c0 0x7fd7f80b5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:15.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.033+0000 7fd7ffe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7f80ce530 0x7fd7f80cad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:15.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.033+0000 7fd7ffe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f8072dc0 con 0x7fd7f80b62c0 2024-05-11T12:15:15.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.033+0000 7fd7ffe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f8072ae0 con 0x7fd7f80ce530 2024-05-11T12:15:15.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.033+0000 7fd7ffe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd7f80743c0 con 0x7fd7f8073a00 2024-05-11T12:15:15.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.033+0000 7fd7fca00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7f80b62c0 0x7fd7f80b5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:15.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.033+0000 7fd7fca00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7f80b62c0 0x7fd7f80b5830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35264/0 (socket says 172.21.15.167:35264) 2024-05-11T12:15:15.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.033+0000 7fd7fca00640 1 -- 172.21.15.167:0/2797780744 learned_addr learned my addr 172.21.15.167:0/2797780744 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:15.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.034+0000 7fd7fca00640 1 -- 172.21.15.167:0/2797780744 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7f8073a00 msgr2=0x7fd7f8145670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:15.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.034+0000 7fd7fca00640 1 --2- 172.21.15.167:0/2797780744 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7f8073a00 0x7fd7f8145670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:15.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.034+0000 7fd7fca00640 1 -- 172.21.15.167:0/2797780744 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7f80ce530 msgr2=0x7fd7f80cad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:15.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.034+0000 7fd7fca00640 1 --2- 172.21.15.167:0/2797780744 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7f80ce530 0x7fd7f80cad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:15.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.034+0000 7fd7fca00640 1 -- 172.21.15.167:0/2797780744 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f80b44b0 con 0x7fd7f80b62c0 2024-05-11T12:15:15.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.034+0000 7fd7fca00640 1 --2- 172.21.15.167:0/2797780744 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7f80b62c0 0x7fd7f80b5830 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fd7e402a2b0 tx=0x7fd7e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:15.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.034+0000 7fd7f5600640 1 -- 172.21.15.167:0/2797780744 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e4041020 con 0x7fd7f80b62c0 2024-05-11T12:15:15.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.034+0000 7fd7f5600640 1 -- 172.21.15.167:0/2797780744 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7e40306c0 con 0x7fd7f80b62c0 2024-05-11T12:15:15.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.034+0000 7fd7ffe00640 1 -- 172.21.15.167:0/2797780744 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7f80b4790 con 0x7fd7f80b62c0 2024-05-11T12:15:15.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.034+0000 7fd7f5600640 1 -- 172.21.15.167:0/2797780744 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e4037040 con 0x7fd7f80b62c0 2024-05-11T12:15:15.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.034+0000 7fd7ffe00640 1 -- 172.21.15.167:0/2797780744 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7f80b4d50 con 0x7fd7f80b62c0 2024-05-11T12:15:15.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.035+0000 7fd7ffe00640 1 -- 172.21.15.167:0/2797780744 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7f8069820 con 0x7fd7f80b62c0 2024-05-11T12:15:15.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.037+0000 7fd7f5600640 1 -- 172.21.15.167:0/2797780744 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd7e4044020 con 0x7fd7f80b62c0 2024-05-11T12:15:15.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.038+0000 7fd7f5600640 1 --2- 172.21.15.167:0/2797780744 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd7d80762d0 0x7fd7d8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:15.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.038+0000 7fd7f5600640 1 -- 172.21.15.167:0/2797780744 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6187+0+0 (secure 0 0 0) 0x7fd7e40b7fd0 con 0x7fd7f80b62c0 2024-05-11T12:15:15.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.038+0000 7fd7fd400640 1 --2- 172.21.15.167:0/2797780744 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd7d80762d0 0x7fd7d8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:15.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.039+0000 7fd7fd400640 1 --2- 172.21.15.167:0/2797780744 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd7d80762d0 0x7fd7d8078790 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec0098f0 tx=0x7fd7ec008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:15.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.043+0000 7fd7f5600640 1 -- 172.21.15.167:0/2797780744 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd7e404b030 con 0x7fd7f80b62c0 2024-05-11T12:15:15.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.242+0000 7fd7ffe00640 1 -- 172.21.15.167:0/2797780744 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd7f8073ef0 con 0x7fd7f80b62c0 2024-05-11T12:15:15.245 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.245+0000 7fd7f5600640 1 -- 172.21.15.167:0/2797780744 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+17053 (secure 0 0 0) 0x7fd7e40817e0 con 0x7fd7f80b62c0 2024-05-11T12:15:15.246 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:15:15.246 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":101,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:14:50.738224+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:14:30.972447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-11T12:14:32.037637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.167:6837/3493868276":"2024-05-12T12:14:50.725899+0000","172.21.15.167:6836/3493868276":"2024-05-12T12:14:50.725899+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:15:15.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.248+0000 7fd7ffe00640 1 -- 172.21.15.167:0/2797780744 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd7d80762d0 msgr2=0x7fd7d8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:15.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.248+0000 7fd7ffe00640 1 --2- 172.21.15.167:0/2797780744 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd7d80762d0 0x7fd7d8078790 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec0098f0 tx=0x7fd7ec008040 comp rx=0 tx=0).stop 2024-05-11T12:15:15.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.248+0000 7fd7ffe00640 1 -- 172.21.15.167:0/2797780744 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7f80b62c0 msgr2=0x7fd7f80b5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:15.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.248+0000 7fd7ffe00640 1 --2- 172.21.15.167:0/2797780744 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7f80b62c0 0x7fd7f80b5830 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fd7e402a2b0 tx=0x7fd7e40046c0 comp rx=0 tx=0).stop 2024-05-11T12:15:15.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.248+0000 7fd7ffe00640 1 -- 172.21.15.167:0/2797780744 shutdown_connections 2024-05-11T12:15:15.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.248+0000 7fd7ffe00640 1 --2- 172.21.15.167:0/2797780744 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd7d80762d0 0x7fd7d8078790 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:15.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.248+0000 7fd7ffe00640 1 --2- 172.21.15.167:0/2797780744 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7f80ce530 0x7fd7f80cad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:15.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.248+0000 7fd7ffe00640 1 --2- 172.21.15.167:0/2797780744 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7f80b62c0 0x7fd7f80b5830 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:15.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.248+0000 7fd7ffe00640 1 --2- 172.21.15.167:0/2797780744 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7f8073a00 0x7fd7f8145670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:15.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.248+0000 7fd7ffe00640 1 -- 172.21.15.167:0/2797780744 >> 172.21.15.167:0/2797780744 conn(0x7fd7f80b2cb0 msgr2=0x7fd7f815ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:15.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.249+0000 7fd7ffe00640 1 -- 172.21.15.167:0/2797780744 shutdown_connections 2024-05-11T12:15:15.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.249+0000 7fd7ffe00640 1 -- 172.21.15.167:0/2797780744 wait complete. 2024-05-11T12:15:15.268 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-11T12:15:15.268 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:15:15.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.455+0000 7f2427600640 1 Processor -- start 2024-05-11T12:15:15.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.455+0000 7f2427600640 1 -- start start 2024-05-11T12:15:15.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.457+0000 7f2427600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f24200b62c0 0x7f24200b66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:15.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.457+0000 7f2427600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2420073a00 0x7f2420073e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:15.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.457+0000 7f2427600640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f24200ce530 0x7f24200ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:15.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.457+0000 7f2427600640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f24200732e0 con 0x7f24200ce530 2024-05-11T12:15:15.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.457+0000 7f2427600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f24200722d0 con 0x7f24200b62c0 2024-05-11T12:15:15.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.457+0000 7f2427600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f24200b6be0 con 0x7f2420073a00 2024-05-11T12:15:15.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.457+0000 7f2425600640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f24200ce530 0x7f24200ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:15.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.457+0000 7f2425600640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f24200ce530 0x7f24200ce9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35268/0 (socket says 172.21.15.167:35268) 2024-05-11T12:15:15.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.457+0000 7f2425600640 1 -- 172.21.15.167:0/1006601549 learned_addr learned my addr 172.21.15.167:0/1006601549 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:15.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.458+0000 7f2425600640 1 -- 172.21.15.167:0/1006601549 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2420073a00 msgr2=0x7f2420073e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:15.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.458+0000 7f2425600640 1 --2- 172.21.15.167:0/1006601549 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2420073a00 0x7f2420073e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:15.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.458+0000 7f2425600640 1 -- 172.21.15.167:0/1006601549 >> v2:172.21.15.204:3300/0 conn(0x7f24200b62c0 msgr2=0x7f24200b66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:15.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.458+0000 7f2425600640 1 --2- 172.21.15.167:0/1006601549 >> v2:172.21.15.204:3300/0 conn(0x7f24200b62c0 0x7f24200b66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:15.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.458+0000 7f2425600640 1 -- 172.21.15.167:0/1006601549 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2420072520 con 0x7f24200ce530 2024-05-11T12:15:15.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.459+0000 7f2425600640 1 --2- 172.21.15.167:0/1006601549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24200ce530 0x7f24200ce9c0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f242006dce0 tx=0x7f241802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa2ffe46e113dd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:15.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.459+0000 7f241f400640 1 -- 172.21.15.167:0/1006601549 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2418036070 con 0x7f24200ce530 2024-05-11T12:15:15.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.459+0000 7f241f400640 1 -- 172.21.15.167:0/1006601549 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2418031600 con 0x7f24200ce530 2024-05-11T12:15:15.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.459+0000 7f241f400640 1 -- 172.21.15.167:0/1006601549 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24180319d0 con 0x7f24200ce530 2024-05-11T12:15:15.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.459+0000 7f2427600640 1 -- 172.21.15.167:0/1006601549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24200ce530 msgr2=0x7f24200ce9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:15.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.459+0000 7f2427600640 1 --2- 172.21.15.167:0/1006601549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24200ce530 0x7f24200ce9c0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f242006dce0 tx=0x7f241802a6a0 comp rx=0 tx=0).stop 2024-05-11T12:15:15.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.459+0000 7f2427600640 1 -- 172.21.15.167:0/1006601549 shutdown_connections 2024-05-11T12:15:15.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.459+0000 7f2427600640 1 --2- 172.21.15.167:0/1006601549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24200ce530 0x7f24200ce9c0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:15.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.459+0000 7f2427600640 1 --2- 172.21.15.167:0/1006601549 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2420073a00 0x7f2420073e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:15.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.459+0000 7f2427600640 1 --2- 172.21.15.167:0/1006601549 >> v2:172.21.15.204:3300/0 conn(0x7f24200b62c0 0x7f24200b66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:15.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.459+0000 7f2427600640 1 -- 172.21.15.167:0/1006601549 >> 172.21.15.167:0/1006601549 conn(0x7f24200b2cb0 msgr2=0x7f24200b3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:15.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.460+0000 7f2427600640 1 -- 172.21.15.167:0/1006601549 shutdown_connections 2024-05-11T12:15:15.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.460+0000 7f2427600640 1 -- 172.21.15.167:0/1006601549 wait complete. 2024-05-11T12:15:15.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.461+0000 7f2427600640 1 Processor -- start 2024-05-11T12:15:15.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.461+0000 7f2427600640 1 -- start start 2024-05-11T12:15:15.464 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.462+0000 7f2427600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2420073a00 0x7f24201f8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:15.464 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.462+0000 7f2427600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24200b62c0 0x7f242015e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:15.464 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.462+0000 7f2427600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24200ce530 0x7f24200b5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:15.464 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.462+0000 7f2427600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2420072dc0 con 0x7f24200ce530 2024-05-11T12:15:15.464 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.462+0000 7f2427600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2420072ae0 con 0x7f24200b62c0 2024-05-11T12:15:15.464 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.462+0000 7f2427600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f24200743c0 con 0x7f2420073a00 2024-05-11T12:15:15.464 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.463+0000 7f2425600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24200ce530 0x7f24200b5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:15.464 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.463+0000 7f2425600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24200ce530 0x7f24200b5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35284/0 (socket says 172.21.15.167:35284) 2024-05-11T12:15:15.464 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.463+0000 7f2425600640 1 -- 172.21.15.167:0/1235019309 learned_addr learned my addr 172.21.15.167:0/1235019309 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:15.464 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.463+0000 7f2424c00640 1 --2- 172.21.15.167:0/1235019309 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2420073a00 0x7f24201f8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:15.464 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.463+0000 7f2425600640 1 -- 172.21.15.167:0/1235019309 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2420073a00 msgr2=0x7f24201f8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:15.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.463+0000 7f241fe00640 1 --2- 172.21.15.167:0/1235019309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24200b62c0 0x7f242015e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:15.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.463+0000 7f2425600640 1 --2- 172.21.15.167:0/1235019309 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2420073a00 0x7f24201f8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:15.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.463+0000 7f2425600640 1 -- 172.21.15.167:0/1235019309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24200b62c0 msgr2=0x7f242015e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:15.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.463+0000 7f2425600640 1 --2- 172.21.15.167:0/1235019309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24200b62c0 0x7f242015e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:15.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.463+0000 7f2425600640 1 -- 172.21.15.167:0/1235019309 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24201f8740 con 0x7f24200ce530 2024-05-11T12:15:15.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.463+0000 7f241fe00640 1 --2- 172.21.15.167:0/1235019309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24200b62c0 0x7f242015e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:15:15.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.463+0000 7f2425600640 1 --2- 172.21.15.167:0/1235019309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24200ce530 0x7f24200b5340 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f2418009f40 tx=0x7f2418004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:15.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.464+0000 7f241cc00640 1 -- 172.21.15.167:0/1235019309 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2418041020 con 0x7f24200ce530 2024-05-11T12:15:15.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.464+0000 7f241cc00640 1 -- 172.21.15.167:0/1235019309 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2418007780 con 0x7f24200ce530 2024-05-11T12:15:15.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.464+0000 7f241cc00640 1 -- 172.21.15.167:0/1235019309 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2418036040 con 0x7f24200ce530 2024-05-11T12:15:15.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.464+0000 7f2427600640 1 -- 172.21.15.167:0/1235019309 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f242015d8b0 con 0x7f24200ce530 2024-05-11T12:15:15.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.464+0000 7f2427600640 1 -- 172.21.15.167:0/1235019309 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f242015de30 con 0x7f24200ce530 2024-05-11T12:15:15.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.465+0000 7f2427600640 1 -- 172.21.15.167:0/1235019309 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24200767b0 con 0x7f24200ce530 2024-05-11T12:15:15.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.466+0000 7f241cc00640 1 -- 172.21.15.167:0/1235019309 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2418044020 con 0x7f24200ce530 2024-05-11T12:15:15.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.467+0000 7f241cc00640 1 --2- 172.21.15.167:0/1235019309 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f24000762d0 0x7f2400078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:15.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.467+0000 7f241cc00640 1 -- 172.21.15.167:0/1235019309 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6187+0+0 (secure 0 0 0) 0x7f24180b8550 con 0x7f24200ce530 2024-05-11T12:15:15.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.467+0000 7f2424c00640 1 --2- 172.21.15.167:0/1235019309 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f24000762d0 0x7f2400078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:15.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.468+0000 7f2424c00640 1 --2- 172.21.15.167:0/1235019309 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f24000762d0 0x7f2400078790 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f241000a810 tx=0x7f2410008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:15.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.473+0000 7f241cc00640 1 -- 172.21.15.167:0/1235019309 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f24200767b0 con 0x7f24200ce530 2024-05-11T12:15:15.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.718+0000 7f2427600640 1 -- 172.21.15.167:0/1235019309 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2420069820 con 0x7f24200ce530 2024-05-11T12:15:15.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.719+0000 7f241cc00640 1 -- 172.21.15.167:0/1235019309 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5217 (secure 0 0 0) 0x7f241804b030 con 0x7f24200ce530 2024-05-11T12:15:15.719 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:15:15.720 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":73,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5401,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/2213070102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2213070102},{"type":"v1","addr":"172.21.15.204:6833","nonce":2213070102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5410,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1480900470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1480900470},{"type":"v1","addr":"172.21.15.204:6835","nonce":1480900470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:14:33.067226+0000","modified":"2024-05-11T12:15:05.949679+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6885,"mds_1":6870},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6870":{"gid":6870,"name":"a","rank":1,"incarnation":71,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6835/3735016882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3735016882},{"type":"v1","addr":"172.21.15.167:6835","nonce":3735016882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6885":{"gid":6885,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6837/2901182170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2901182170},{"type":"v1","addr":"172.21.15.167:6837","nonce":2901182170}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6885,"qdb_cluster":[6885,6870]},"id":7}]} 2024-05-11T12:15:15.720 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 73 2024-05-11T12:15:15.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.722+0000 7f2427600640 1 -- 172.21.15.167:0/1235019309 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f24000762d0 msgr2=0x7f2400078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:15.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.722+0000 7f2427600640 1 --2- 172.21.15.167:0/1235019309 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f24000762d0 0x7f2400078790 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f241000a810 tx=0x7f2410008040 comp rx=0 tx=0).stop 2024-05-11T12:15:15.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.722+0000 7f2427600640 1 -- 172.21.15.167:0/1235019309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24200ce530 msgr2=0x7f24200b5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:15.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.722+0000 7f2427600640 1 --2- 172.21.15.167:0/1235019309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24200ce530 0x7f24200b5340 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f2418009f40 tx=0x7f2418004a40 comp rx=0 tx=0).stop 2024-05-11T12:15:15.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.722+0000 7f2427600640 1 -- 172.21.15.167:0/1235019309 shutdown_connections 2024-05-11T12:15:15.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.722+0000 7f2427600640 1 --2- 172.21.15.167:0/1235019309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24200ce530 0x7f24200b5340 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:15.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.722+0000 7f2427600640 1 --2- 172.21.15.167:0/1235019309 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f24000762d0 0x7f2400078790 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:15.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.722+0000 7f2427600640 1 --2- 172.21.15.167:0/1235019309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24200b62c0 0x7f242015e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:15.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.722+0000 7f2427600640 1 --2- 172.21.15.167:0/1235019309 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2420073a00 0x7f24201f8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:15.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.722+0000 7f2427600640 1 -- 172.21.15.167:0/1235019309 >> 172.21.15.167:0/1235019309 conn(0x7f24200b2cb0 msgr2=0x7f2420155fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:15.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.722+0000 7f2427600640 1 -- 172.21.15.167:0/1235019309 shutdown_connections 2024-05-11T12:15:15.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.722+0000 7f2427600640 1 -- 172.21.15.167:0/1235019309 wait complete. 2024-05-11T12:15:15.739 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:15:15.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.942+0000 7fd0bf000640 1 Processor -- start 2024-05-11T12:15:15.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.942+0000 7fd0bf000640 1 -- start start 2024-05-11T12:15:15.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.944+0000 7fd0bf000640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd0b80b62c0 0x7fd0b80b66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:15.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.944+0000 7fd0bf000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0b815e880 0x7fd0b815ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:15.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.944+0000 7fd0bf000640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0b80b6be0 con 0x7fd0b8073a00 2024-05-11T12:15:15.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.944+0000 7fd0bf000640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fd0b80732e0 con 0x7fd0b80b62c0 2024-05-11T12:15:15.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.944+0000 7fd0bf000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd0b80722d0 con 0x7fd0b815e880 2024-05-11T12:15:15.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.944+0000 7fd0b7e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd0b80b62c0 0x7fd0b80b66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:15.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.944+0000 7fd0b7e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd0b80b62c0 0x7fd0b80b66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35286/0 (socket says 172.21.15.167:35286) 2024-05-11T12:15:15.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.944+0000 7fd0b7e00640 1 -- 172.21.15.167:0/3029468306 learned_addr learned my addr 172.21.15.167:0/3029468306 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:15.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.945+0000 7fd0b7e00640 1 -- 172.21.15.167:0/3029468306 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0b815e880 msgr2=0x7fd0b815ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:15.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.945+0000 7fd0bd000640 1 --2- 172.21.15.167:0/3029468306 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0b815e880 0x7fd0b815ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:15.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.945+0000 7fd0b7e00640 1 --2- 172.21.15.167:0/3029468306 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0b815e880 0x7fd0b815ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:15.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.945+0000 7fd0b7e00640 1 -- 172.21.15.167:0/3029468306 >> v1:172.21.15.167:6789/0 conn(0x7fd0b8073a00 legacy=0x7fd0b8073e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:15.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.945+0000 7fd0b7e00640 1 -- 172.21.15.167:0/3029468306 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0b8072520 con 0x7fd0b80b62c0 2024-05-11T12:15:15.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.946+0000 7fd0b7e00640 1 --2- 172.21.15.167:0/3029468306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0b80b62c0 0x7fd0b80b66a0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8009870 tx=0x7fd0a802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb643df3f6b091f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:15.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.946+0000 7fd0b6a00640 1 -- 172.21.15.167:0/3029468306 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a8037070 con 0x7fd0b80b62c0 2024-05-11T12:15:15.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.946+0000 7fd0b6a00640 1 -- 172.21.15.167:0/3029468306 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0a8032440 con 0x7fd0b80b62c0 2024-05-11T12:15:15.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.946+0000 7fd0b6a00640 1 -- 172.21.15.167:0/3029468306 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a8032810 con 0x7fd0b80b62c0 2024-05-11T12:15:15.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.947+0000 7fd0bf000640 1 -- 172.21.15.167:0/3029468306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0b80b62c0 msgr2=0x7fd0b80b66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:15.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.947+0000 7fd0bf000640 1 --2- 172.21.15.167:0/3029468306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0b80b62c0 0x7fd0b80b66a0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8009870 tx=0x7fd0a802a1c0 comp rx=0 tx=0).stop 2024-05-11T12:15:15.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.947+0000 7fd0bf000640 1 -- 172.21.15.167:0/3029468306 shutdown_connections 2024-05-11T12:15:15.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.947+0000 7fd0bf000640 1 --2- 172.21.15.167:0/3029468306 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0b815e880 0x7fd0b815ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:15.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.947+0000 7fd0bf000640 1 --2- 172.21.15.167:0/3029468306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0b80b62c0 0x7fd0b80b66a0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:15.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.947+0000 7fd0bf000640 1 -- 172.21.15.167:0/3029468306 >> 172.21.15.167:0/3029468306 conn(0x7fd0b80b2cb0 msgr2=0x7fd0b80b3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:15.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.947+0000 7fd0bf000640 1 -- 172.21.15.167:0/3029468306 shutdown_connections 2024-05-11T12:15:15.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.947+0000 7fd0bf000640 1 -- 172.21.15.167:0/3029468306 wait complete. 2024-05-11T12:15:15.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.949+0000 7fd0bf000640 1 Processor -- start 2024-05-11T12:15:15.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.949+0000 7fd0bf000640 1 -- start start 2024-05-11T12:15:15.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.950+0000 7fd0bf000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0b8073a00 0x7fd0b80aa970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:15.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.950+0000 7fd0bf000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0b80b62c0 0x7fd0b815cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:15.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.950+0000 7fd0bf000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0b815e880 0x7fd0b81f83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:15.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.950+0000 7fd0bf000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b8072dc0 con 0x7fd0b8073a00 2024-05-11T12:15:15.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.950+0000 7fd0bf000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b8072ae0 con 0x7fd0b815e880 2024-05-11T12:15:15.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.950+0000 7fd0bf000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd0b81579c0 con 0x7fd0b80b62c0 2024-05-11T12:15:15.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.950+0000 7fd0b7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0b8073a00 0x7fd0b80aa970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:15.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.950+0000 7fd0b7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0b8073a00 0x7fd0b80aa970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35302/0 (socket says 172.21.15.167:35302) 2024-05-11T12:15:15.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.950+0000 7fd0b7e00640 1 -- 172.21.15.167:0/885515368 learned_addr learned my addr 172.21.15.167:0/885515368 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:15.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.950+0000 7fd0b7e00640 1 -- 172.21.15.167:0/885515368 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0b80b62c0 msgr2=0x7fd0b815cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:15.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.950+0000 7fd0b7e00640 1 --2- 172.21.15.167:0/885515368 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0b80b62c0 0x7fd0b815cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:15.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.950+0000 7fd0b7e00640 1 -- 172.21.15.167:0/885515368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0b815e880 msgr2=0x7fd0b81f83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:15.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.950+0000 7fd0b7e00640 1 --2- 172.21.15.167:0/885515368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0b815e880 0x7fd0b81f83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:15.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.950+0000 7fd0b7e00640 1 -- 172.21.15.167:0/885515368 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0b815d4b0 con 0x7fd0b8073a00 2024-05-11T12:15:15.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.951+0000 7fd0b7e00640 1 --2- 172.21.15.167:0/885515368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0b8073a00 0x7fd0b80aa970 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8000c00 tx=0x7fd0a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:15.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.951+0000 7fd0afe00640 1 -- 172.21.15.167:0/885515368 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a8037040 con 0x7fd0b8073a00 2024-05-11T12:15:15.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.951+0000 7fd0afe00640 1 -- 172.21.15.167:0/885515368 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0a8042070 con 0x7fd0b8073a00 2024-05-11T12:15:15.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.951+0000 7fd0afe00640 1 -- 172.21.15.167:0/885515368 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a8030bd0 con 0x7fd0b8073a00 2024-05-11T12:15:15.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.951+0000 7fd0bf000640 1 -- 172.21.15.167:0/885515368 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0b815d790 con 0x7fd0b8073a00 2024-05-11T12:15:15.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.951+0000 7fd0bf000640 1 -- 172.21.15.167:0/885515368 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0b81f88f0 con 0x7fd0b8073a00 2024-05-11T12:15:15.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.953+0000 7fd0bf000640 1 -- 172.21.15.167:0/885515368 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0b8141400 con 0x7fd0b8073a00 2024-05-11T12:15:15.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.954+0000 7fd0afe00640 1 -- 172.21.15.167:0/885515368 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd0a802ad10 con 0x7fd0b8073a00 2024-05-11T12:15:15.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.955+0000 7fd0afe00640 1 --2- 172.21.15.167:0/885515368 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd0980762d0 0x7fd098078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:15.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.955+0000 7fd0afe00640 1 -- 172.21.15.167:0/885515368 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6187+0+0 (secure 0 0 0) 0x7fd0a80b8050 con 0x7fd0b8073a00 2024-05-11T12:15:15.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.955+0000 7fd0b7400640 1 --2- 172.21.15.167:0/885515368 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd0980762d0 0x7fd098078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:15.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.956+0000 7fd0b7400640 1 --2- 172.21.15.167:0/885515368 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd0980762d0 0x7fd098078790 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd0a4000f60 tx=0x7fd0a40079b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:15.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:15.962+0000 7fd0afe00640 1 -- 172.21.15.167:0/885515368 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd0b8141400 con 0x7fd0b8073a00 2024-05-11T12:15:16.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.209+0000 7fd0bf000640 1 -- 172.21.15.167:0/885515368 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0b8069820 con 0x7fd0b8073a00 2024-05-11T12:15:16.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.210+0000 7fd0afe00640 1 -- 172.21.15.167:0/885515368 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5217 (secure 0 0 0) 0x7fd0a8081860 con 0x7fd0b8073a00 2024-05-11T12:15:16.210 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:15:16.210 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":73,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5401,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/2213070102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2213070102},{"type":"v1","addr":"172.21.15.204:6833","nonce":2213070102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5410,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1480900470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1480900470},{"type":"v1","addr":"172.21.15.204:6835","nonce":1480900470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T12:14:33.067226+0000","modified":"2024-05-11T12:15:05.949679+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6885,"mds_1":6870},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6870":{"gid":6870,"name":"a","rank":1,"incarnation":71,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6835/3735016882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3735016882},{"type":"v1","addr":"172.21.15.167:6835","nonce":3735016882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6885":{"gid":6885,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6837/2901182170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2901182170},{"type":"v1","addr":"172.21.15.167:6837","nonce":2901182170}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6885,"qdb_cluster":[6885,6870]},"id":7}]} 2024-05-11T12:15:16.211 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 73 2024-05-11T12:15:16.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.212+0000 7fd0bf000640 1 -- 172.21.15.167:0/885515368 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd0980762d0 msgr2=0x7fd098078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:16.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.212+0000 7fd0bf000640 1 --2- 172.21.15.167:0/885515368 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd0980762d0 0x7fd098078790 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd0a4000f60 tx=0x7fd0a40079b0 comp rx=0 tx=0).stop 2024-05-11T12:15:16.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.212+0000 7fd0bf000640 1 -- 172.21.15.167:0/885515368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0b8073a00 msgr2=0x7fd0b80aa970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:16.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.212+0000 7fd0bf000640 1 --2- 172.21.15.167:0/885515368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0b8073a00 0x7fd0b80aa970 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8000c00 tx=0x7fd0a80046c0 comp rx=0 tx=0).stop 2024-05-11T12:15:16.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.213+0000 7fd0bf000640 1 -- 172.21.15.167:0/885515368 shutdown_connections 2024-05-11T12:15:16.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.213+0000 7fd0bf000640 1 --2- 172.21.15.167:0/885515368 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd0980762d0 0x7fd098078790 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:16.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.213+0000 7fd0bf000640 1 --2- 172.21.15.167:0/885515368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0b815e880 0x7fd0b81f83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:16.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.213+0000 7fd0bf000640 1 --2- 172.21.15.167:0/885515368 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0b80b62c0 0x7fd0b815cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:16.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.213+0000 7fd0bf000640 1 --2- 172.21.15.167:0/885515368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0b8073a00 0x7fd0b80aa970 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:16.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.213+0000 7fd0bf000640 1 -- 172.21.15.167:0/885515368 >> 172.21.15.167:0/885515368 conn(0x7fd0b80b2cb0 msgr2=0x7fd0b8155aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:16.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.213+0000 7fd0bf000640 1 -- 172.21.15.167:0/885515368 shutdown_connections 2024-05-11T12:15:16.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.213+0000 7fd0bf000640 1 -- 172.21.15.167:0/885515368 wait complete. 2024-05-11T12:15:16.230 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:15:16.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.426+0000 7f0bcf200640 1 Processor -- start 2024-05-11T12:15:16.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.426+0000 7f0bcf200640 1 -- start start 2024-05-11T12:15:16.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.427+0000 7f0bcf200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0bc8145ca0 0x7f0bc80b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:16.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.428+0000 7f0bcf200640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bc8070010 con 0x7f0bc8146400 2024-05-11T12:15:16.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.428+0000 7f0bcf200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bc80b3e60 con 0x7f0bc80cffa0 2024-05-11T12:15:16.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.428+0000 7f0bcf200640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f0bc80b4140 con 0x7f0bc8145ca0 2024-05-11T12:15:16.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.428+0000 7f0bcde00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0bc8145ca0 0x7f0bc80b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:16.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.428+0000 7f0bcde00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0bc8145ca0 0x7f0bc80b4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35316/0 (socket says 172.21.15.167:35316) 2024-05-11T12:15:16.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.428+0000 7f0bcd400640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f0bc8146400 0x7f0bc806f1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:41102/0 (socket says 172.21.15.167:41102) 2024-05-11T12:15:16.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.428+0000 7f0bcde00640 1 -- 172.21.15.167:0/1145168285 learned_addr learned my addr 172.21.15.167:0/1145168285 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:16.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.428+0000 7f0bcde00640 1 -- 172.21.15.167:0/1145168285 >> v1:172.21.15.204:6789/0 conn(0x7f0bc80cffa0 legacy=0x7f0bc806f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:16.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.429+0000 7f0bcde00640 1 -- 172.21.15.167:0/1145168285 >> v1:172.21.15.167:6789/0 conn(0x7f0bc8146400 legacy=0x7f0bc806f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:16.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.429+0000 7f0bcde00640 1 -- 172.21.15.167:0/1145168285 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc8161900 con 0x7f0bc8145ca0 2024-05-11T12:15:16.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.429+0000 7f0bcca00640 1 -- 172.21.15.167:0/1145168285 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2608642957 0 0) 0x7f0bc8070010 con 0x7f0bc8146400 2024-05-11T12:15:16.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.430+0000 7f0bcde00640 1 --2- 172.21.15.167:0/1145168285 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bc8145ca0 0x7f0bc80b4ae0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc009870 tx=0x7f0bbc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e08c8304cfc68a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:16.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.430+0000 7f0bcca00640 1 -- 172.21.15.167:0/1145168285 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc037070 con 0x7f0bc8145ca0 2024-05-11T12:15:16.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.430+0000 7f0bcca00640 1 -- 172.21.15.167:0/1145168285 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bbc032440 con 0x7f0bc8145ca0 2024-05-11T12:15:16.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.430+0000 7f0bcca00640 1 -- 172.21.15.167:0/1145168285 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc032810 con 0x7f0bc8145ca0 2024-05-11T12:15:16.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.430+0000 7f0bcf200640 1 -- 172.21.15.167:0/1145168285 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bc8145ca0 msgr2=0x7f0bc80b4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:16.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.430+0000 7f0bcf200640 1 --2- 172.21.15.167:0/1145168285 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bc8145ca0 0x7f0bc80b4ae0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc009870 tx=0x7f0bbc02a270 comp rx=0 tx=0).stop 2024-05-11T12:15:16.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.430+0000 7f0bcf200640 1 -- 172.21.15.167:0/1145168285 shutdown_connections 2024-05-11T12:15:16.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.430+0000 7f0bcf200640 1 --2- 172.21.15.167:0/1145168285 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bc8145ca0 0x7f0bc80b4ae0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:16.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.431+0000 7f0bcf200640 1 -- 172.21.15.167:0/1145168285 >> 172.21.15.167:0/1145168285 conn(0x7f0bc80b2b60 msgr2=0x7f0bc80b2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:16.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.431+0000 7f0bcf200640 1 -- 172.21.15.167:0/1145168285 shutdown_connections 2024-05-11T12:15:16.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.431+0000 7f0bcf200640 1 -- 172.21.15.167:0/1145168285 wait complete. 2024-05-11T12:15:16.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.432+0000 7f0bcf200640 1 Processor -- start 2024-05-11T12:15:16.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.432+0000 7f0bcf200640 1 -- start start 2024-05-11T12:15:16.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.433+0000 7f0bcf200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bc80cffa0 0x7f0bc815ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:16.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.433+0000 7f0bcf200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bc8145ca0 0x7f0bc8073070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:16.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.433+0000 7f0bcf200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0bc8146400 0x7f0bc80c8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:16.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.433+0000 7f0bcf200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc8146200 con 0x7f0bc8145ca0 2024-05-11T12:15:16.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.433+0000 7f0bcf200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc8146080 con 0x7f0bc80cffa0 2024-05-11T12:15:16.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.433+0000 7f0bcf200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0bc806edc0 con 0x7f0bc8146400 2024-05-11T12:15:16.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.434+0000 7f0bcd400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bc8145ca0 0x7f0bc8073070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:16.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.434+0000 7f0bcd400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bc8145ca0 0x7f0bc8073070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35322/0 (socket says 172.21.15.167:35322) 2024-05-11T12:15:16.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.434+0000 7f0bcd400640 1 -- 172.21.15.167:0/808493505 learned_addr learned my addr 172.21.15.167:0/808493505 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:16.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.434+0000 7f0bcd400640 1 -- 172.21.15.167:0/808493505 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0bc8146400 msgr2=0x7f0bc80c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:16.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.434+0000 7f0bcd400640 1 --2- 172.21.15.167:0/808493505 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0bc8146400 0x7f0bc80c8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:16.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.434+0000 7f0bcd400640 1 -- 172.21.15.167:0/808493505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bc80cffa0 msgr2=0x7f0bc815ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:16.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.434+0000 7f0bcd400640 1 --2- 172.21.15.167:0/808493505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bc80cffa0 0x7f0bc815ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:16.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.434+0000 7f0bcd400640 1 -- 172.21.15.167:0/808493505 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc8073b30 con 0x7f0bc8145ca0 2024-05-11T12:15:16.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.434+0000 7f0bcd400640 1 --2- 172.21.15.167:0/808493505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bc8145ca0 0x7f0bc8073070 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f0bb800c3c0 tx=0x7f0bb800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:16.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.435+0000 7f0bc2000640 1 -- 172.21.15.167:0/808493505 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb80074e0 con 0x7f0bc8145ca0 2024-05-11T12:15:16.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.435+0000 7f0bc2000640 1 -- 172.21.15.167:0/808493505 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bb80047e0 con 0x7f0bc8145ca0 2024-05-11T12:15:16.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.435+0000 7f0bcf200640 1 -- 172.21.15.167:0/808493505 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bc80cacb0 con 0x7f0bc8145ca0 2024-05-11T12:15:16.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.435+0000 7f0bc2000640 1 -- 172.21.15.167:0/808493505 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb8004c10 con 0x7f0bc8145ca0 2024-05-11T12:15:16.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.435+0000 7f0bcf200640 1 -- 172.21.15.167:0/808493505 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bc80cb550 con 0x7f0bc8145ca0 2024-05-11T12:15:16.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.436+0000 7f0bcf200640 1 -- 172.21.15.167:0/808493505 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bc80cb750 con 0x7f0bc8145ca0 2024-05-11T12:15:16.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.437+0000 7f0bc2000640 1 -- 172.21.15.167:0/808493505 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0bb8011070 con 0x7f0bc8145ca0 2024-05-11T12:15:16.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.438+0000 7f0bc2000640 1 --2- 172.21.15.167:0/808493505 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0bac0762d0 0x7f0bac078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:16.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.438+0000 7f0bc2000640 1 -- 172.21.15.167:0/808493505 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6187+0+0 (secure 0 0 0) 0x7f0bb8097370 con 0x7f0bc8145ca0 2024-05-11T12:15:16.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.438+0000 7f0bcde00640 1 --2- 172.21.15.167:0/808493505 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0bac0762d0 0x7f0bac078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:16.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.439+0000 7f0bcde00640 1 --2- 172.21.15.167:0/808493505 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0bac0762d0 0x7f0bac078790 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc034000 tx=0x7f0bbc02c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:16.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.443+0000 7f0bc2000640 1 -- 172.21.15.167:0/808493505 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0bc80cb750 con 0x7f0bc8145ca0 2024-05-11T12:15:16.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.651+0000 7f0bcf200640 1 -- 172.21.15.167:0/808493505 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0bc8069650 con 0x7f0bc8145ca0 2024-05-11T12:15:16.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.652+0000 7f0bc2000640 1 -- 172.21.15.167:0/808493505 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+17053 (secure 0 0 0) 0x7f0bb80604a0 con 0x7f0bc8145ca0 2024-05-11T12:15:16.653 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:15:16.653 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":101,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:14:50.738224+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:14:30.972447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-11T12:14:32.037637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.167:6837/3493868276":"2024-05-12T12:14:50.725899+0000","172.21.15.167:6836/3493868276":"2024-05-12T12:14:50.725899+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:15:16.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.656+0000 7f0bcf200640 1 -- 172.21.15.167:0/808493505 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0bac0762d0 msgr2=0x7f0bac078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:16.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.656+0000 7f0bcf200640 1 --2- 172.21.15.167:0/808493505 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0bac0762d0 0x7f0bac078790 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc034000 tx=0x7f0bbc02c000 comp rx=0 tx=0).stop 2024-05-11T12:15:16.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.656+0000 7f0bcf200640 1 -- 172.21.15.167:0/808493505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bc8145ca0 msgr2=0x7f0bc8073070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:16.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.656+0000 7f0bcf200640 1 --2- 172.21.15.167:0/808493505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bc8145ca0 0x7f0bc8073070 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f0bb800c3c0 tx=0x7f0bb800c880 comp rx=0 tx=0).stop 2024-05-11T12:15:16.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.656+0000 7f0bcf200640 1 -- 172.21.15.167:0/808493505 shutdown_connections 2024-05-11T12:15:16.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.656+0000 7f0bcf200640 1 --2- 172.21.15.167:0/808493505 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0bc8146400 0x7f0bc80c8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:16.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.656+0000 7f0bcf200640 1 --2- 172.21.15.167:0/808493505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bc8145ca0 0x7f0bc8073070 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:16.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.656+0000 7f0bcf200640 1 --2- 172.21.15.167:0/808493505 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0bac0762d0 0x7f0bac078790 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:16.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.656+0000 7f0bcf200640 1 --2- 172.21.15.167:0/808493505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bc80cffa0 0x7f0bc815ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:16.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.656+0000 7f0bcf200640 1 -- 172.21.15.167:0/808493505 >> 172.21.15.167:0/808493505 conn(0x7f0bc80b2b60 msgr2=0x7f0bc8142a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:16.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.656+0000 7f0bcf200640 1 -- 172.21.15.167:0/808493505 shutdown_connections 2024-05-11T12:15:16.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.656+0000 7f0bcf200640 1 -- 172.21.15.167:0/808493505 wait complete. 2024-05-11T12:15:16.676 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-11T12:15:16.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.875+0000 7f1eaca00640 1 Processor -- start 2024-05-11T12:15:16.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.876+0000 7f1eaca00640 1 -- start start 2024-05-11T12:15:16.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.877+0000 7f1eaca00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1ea80b02f0 0x7f1ea806c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:16.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.877+0000 7f1eaca00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1ea80add80 0x7f1ea806cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:16.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.877+0000 7f1eaca00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ea815d8d0 con 0x7f1ea815aaa0 2024-05-11T12:15:16.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.877+0000 7f1eaca00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f1ea815ae80 con 0x7f1ea80b02f0 2024-05-11T12:15:16.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.877+0000 7f1eaca00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1ea815dbb0 con 0x7f1ea80add80 2024-05-11T12:15:16.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.878+0000 7f1e9fe00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1ea80b02f0 0x7f1ea806c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:16.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.878+0000 7f1e9fe00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1ea80b02f0 0x7f1ea806c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35336/0 (socket says 172.21.15.167:35336) 2024-05-11T12:15:16.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.878+0000 7f1e9fe00640 1 -- 172.21.15.167:0/2022807731 learned_addr learned my addr 172.21.15.167:0/2022807731 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:16.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.878+0000 7f1ea7e00640 1 --2- 172.21.15.167:0/2022807731 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1ea80add80 0x7f1ea806cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:16.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.878+0000 7f1e9fe00640 1 -- 172.21.15.167:0/2022807731 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1ea80add80 msgr2=0x7f1ea806cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:16.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.878+0000 7f1e9fe00640 1 --2- 172.21.15.167:0/2022807731 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1ea80add80 0x7f1ea806cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:16.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.878+0000 7f1e9fe00640 1 -- 172.21.15.167:0/2022807731 >> v1:172.21.15.167:6789/0 conn(0x7f1ea815aaa0 legacy=0x7f1ea806dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:16.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.878+0000 7f1e9fe00640 1 -- 172.21.15.167:0/2022807731 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ea815de90 con 0x7f1ea80b02f0 2024-05-11T12:15:16.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.879+0000 7f1e9fe00640 1 --2- 172.21.15.167:0/2022807731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ea80b02f0 0x7f1ea806c8a0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f1e90009870 tx=0x7f1e9002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e40f68bfcf314c server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:16.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.880+0000 7f1ea6a00640 1 -- 172.21.15.167:0/2022807731 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e90037070 con 0x7f1ea80b02f0 2024-05-11T12:15:16.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.880+0000 7f1ea6a00640 1 -- 172.21.15.167:0/2022807731 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e90032440 con 0x7f1ea80b02f0 2024-05-11T12:15:16.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.880+0000 7f1ea6a00640 1 -- 172.21.15.167:0/2022807731 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e90032810 con 0x7f1ea80b02f0 2024-05-11T12:15:16.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.880+0000 7f1eaca00640 1 -- 172.21.15.167:0/2022807731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ea80b02f0 msgr2=0x7f1ea806c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:16.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.880+0000 7f1eaca00640 1 --2- 172.21.15.167:0/2022807731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ea80b02f0 0x7f1ea806c8a0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f1e90009870 tx=0x7f1e9002a250 comp rx=0 tx=0).stop 2024-05-11T12:15:16.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.880+0000 7f1eaca00640 1 -- 172.21.15.167:0/2022807731 shutdown_connections 2024-05-11T12:15:16.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.880+0000 7f1eaca00640 1 --2- 172.21.15.167:0/2022807731 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1ea80add80 0x7f1ea806cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:16.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.880+0000 7f1eaca00640 1 --2- 172.21.15.167:0/2022807731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ea80b02f0 0x7f1ea806c8a0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:16.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.880+0000 7f1eaca00640 1 -- 172.21.15.167:0/2022807731 >> 172.21.15.167:0/2022807731 conn(0x7f1ea80b2df0 msgr2=0x7f1ea80b3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:16.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.880+0000 7f1eaca00640 1 -- 172.21.15.167:0/2022807731 shutdown_connections 2024-05-11T12:15:16.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.880+0000 7f1eaca00640 1 -- 172.21.15.167:0/2022807731 wait complete. 2024-05-11T12:15:16.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.882+0000 7f1eaca00640 1 Processor -- start 2024-05-11T12:15:16.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.882+0000 7f1eaca00640 1 -- start start 2024-05-11T12:15:16.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.883+0000 7f1eaca00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1ea80add80 0x7f1ea809ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:16.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.883+0000 7f1eaca00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ea80b02f0 0x7f1ea81543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:16.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.883+0000 7f1eaca00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ea815aaa0 0x7f1ea80b0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:16.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.883+0000 7f1eaca00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea815d600 con 0x7f1ea80b02f0 2024-05-11T12:15:16.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.883+0000 7f1eaca00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea815d480 con 0x7f1ea815aaa0 2024-05-11T12:15:16.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.883+0000 7f1eaca00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1ea80b4080 con 0x7f1ea80add80 2024-05-11T12:15:16.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.883+0000 7f1e9fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ea80b02f0 0x7f1ea81543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:16.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.883+0000 7f1e9fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ea80b02f0 0x7f1ea81543d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35348/0 (socket says 172.21.15.167:35348) 2024-05-11T12:15:16.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.883+0000 7f1e9fe00640 1 -- 172.21.15.167:0/1738706825 learned_addr learned my addr 172.21.15.167:0/1738706825 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:16.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.884+0000 7f1e9fe00640 1 -- 172.21.15.167:0/1738706825 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1ea80add80 msgr2=0x7f1ea809ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:16.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.884+0000 7f1ea7400640 1 --2- 172.21.15.167:0/1738706825 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1ea80add80 0x7f1ea809ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:16.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.884+0000 7f1e9fe00640 1 --2- 172.21.15.167:0/1738706825 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1ea80add80 0x7f1ea809ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:16.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.884+0000 7f1e9fe00640 1 -- 172.21.15.167:0/1738706825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ea815aaa0 msgr2=0x7f1ea80b0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:16.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.884+0000 7f1e9fe00640 1 --2- 172.21.15.167:0/1738706825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ea815aaa0 0x7f1ea80b0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:16.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.884+0000 7f1e9fe00640 1 -- 172.21.15.167:0/1738706825 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ea809d8d0 con 0x7f1ea80b02f0 2024-05-11T12:15:16.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.884+0000 7f1ea7400640 1 --2- 172.21.15.167:0/1738706825 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1ea80add80 0x7f1ea809ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:15:16.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.884+0000 7f1e9fe00640 1 --2- 172.21.15.167:0/1738706825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ea80b02f0 0x7f1ea81543d0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f1e90000c00 tx=0x7f1e900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:16.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.884+0000 7f1e9f400640 1 -- 172.21.15.167:0/1738706825 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e90041020 con 0x7f1ea80b02f0 2024-05-11T12:15:16.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.884+0000 7f1e9f400640 1 -- 172.21.15.167:0/1738706825 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e900306c0 con 0x7f1ea80b02f0 2024-05-11T12:15:16.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.884+0000 7f1eaca00640 1 -- 172.21.15.167:0/1738706825 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ea809dbb0 con 0x7f1ea80b02f0 2024-05-11T12:15:16.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.885+0000 7f1e9f400640 1 -- 172.21.15.167:0/1738706825 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e90037040 con 0x7f1ea80b02f0 2024-05-11T12:15:16.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.885+0000 7f1eaca00640 1 -- 172.21.15.167:0/1738706825 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ea809e170 con 0x7f1ea80b02f0 2024-05-11T12:15:16.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.886+0000 7f1eaca00640 1 -- 172.21.15.167:0/1738706825 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ea8069650 con 0x7f1ea80b02f0 2024-05-11T12:15:16.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.887+0000 7f1e9f400640 1 -- 172.21.15.167:0/1738706825 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1e90044020 con 0x7f1ea80b02f0 2024-05-11T12:15:16.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.888+0000 7f1e9f400640 1 --2- 172.21.15.167:0/1738706825 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f1e840762d0 0x7f1e84078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:16.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.888+0000 7f1e9f400640 1 -- 172.21.15.167:0/1738706825 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6187+0+0 (secure 0 0 0) 0x7f1e900b80c0 con 0x7f1ea80b02f0 2024-05-11T12:15:16.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.888+0000 7f1ea7400640 1 --2- 172.21.15.167:0/1738706825 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f1e840762d0 0x7f1e84078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:16.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.889+0000 7f1ea7400640 1 --2- 172.21.15.167:0/1738706825 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f1e840762d0 0x7f1e84078790 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1e980016a0 tx=0x7f1e98009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:16.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:16.892+0000 7f1e9f400640 1 -- 172.21.15.167:0/1738706825 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1e9004b030 con 0x7f1ea80b02f0 2024-05-11T12:15:17.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.142+0000 7f1eaca00640 1 -- 172.21.15.167:0/1738706825 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f1ea80d0070 con 0x7f1ea80b02f0 2024-05-11T12:15:17.143 INFO:tasks.ceph.mon.a.smithi167.stderr:2024-05-11T12:15:17.143+0000 7f3aac000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-11T12:15:17.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.146+0000 7f1e9f400640 1 -- 172.21.15.167:0/1738706825 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v74) ==== 161+0+0 (secure 0 0 0) 0x7f1e900817a0 con 0x7f1ea80b02f0 2024-05-11T12:15:17.146 INFO:teuthology.orchestra.run.smithi167.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-11T12:15:17.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.149+0000 7f1eaca00640 1 -- 172.21.15.167:0/1738706825 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f1e840762d0 msgr2=0x7f1e84078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:17.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.149+0000 7f1eaca00640 1 --2- 172.21.15.167:0/1738706825 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f1e840762d0 0x7f1e84078790 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1e980016a0 tx=0x7f1e98009040 comp rx=0 tx=0).stop 2024-05-11T12:15:17.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.149+0000 7f1eaca00640 1 -- 172.21.15.167:0/1738706825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ea80b02f0 msgr2=0x7f1ea81543d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:17.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.149+0000 7f1eaca00640 1 --2- 172.21.15.167:0/1738706825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ea80b02f0 0x7f1ea81543d0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f1e90000c00 tx=0x7f1e900046c0 comp rx=0 tx=0).stop 2024-05-11T12:15:17.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.150+0000 7f1eaca00640 1 -- 172.21.15.167:0/1738706825 shutdown_connections 2024-05-11T12:15:17.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.150+0000 7f1eaca00640 1 --2- 172.21.15.167:0/1738706825 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f1e840762d0 0x7f1e84078790 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:17.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.150+0000 7f1eaca00640 1 --2- 172.21.15.167:0/1738706825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ea815aaa0 0x7f1ea80b0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:17.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.150+0000 7f1eaca00640 1 --2- 172.21.15.167:0/1738706825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ea80b02f0 0x7f1ea81543d0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:17.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.150+0000 7f1eaca00640 1 --2- 172.21.15.167:0/1738706825 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1ea80add80 0x7f1ea809ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:17.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.150+0000 7f1eaca00640 1 -- 172.21.15.167:0/1738706825 >> 172.21.15.167:0/1738706825 conn(0x7f1ea80b2df0 msgr2=0x7f1ea815b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:17.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.152+0000 7f1eaca00640 1 -- 172.21.15.167:0/1738706825 shutdown_connections 2024-05-11T12:15:17.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.152+0000 7f1eaca00640 1 -- 172.21.15.167:0/1738706825 wait complete. 2024-05-11T12:15:17.172 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-11T12:15:17.173 INFO:tasks.ceph.mds.c.smithi167.stderr: -4752> 2024-05-11T12:14:56.752+0000 7f9de7000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-11T12:15:17.213 INFO:tasks.ceph.mds.a.smithi167.stdout:starting mds.a at 2024-05-11T12:15:17.234 INFO:tasks.ceph.mds.c.smithi167.stdout:starting mds.c at 2024-05-11T12:15:17.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.348+0000 7f4f78a00640 1 Processor -- start 2024-05-11T12:15:17.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.348+0000 7f4f78a00640 1 -- start start 2024-05-11T12:15:17.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.349+0000 7f4f78a00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4f74073a00 0x7f4f74073e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:17.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.349+0000 7f4f78a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4f7415e880 0x7f4f7415ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:17.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.350+0000 7f4f78a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f74072520 con 0x7f4f740b62c0 2024-05-11T12:15:17.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.350+0000 7f4f78a00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f4f740732e0 con 0x7f4f74073a00 2024-05-11T12:15:17.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.350+0000 7f4f78a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f4f740722d0 con 0x7f4f7415e880 2024-05-11T12:15:17.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.350+0000 7f4f71400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4f74073a00 0x7f4f74073e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:17.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.350+0000 7f4f71400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4f74073a00 0x7f4f74073e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35394/0 (socket says 172.21.15.167:35394) 2024-05-11T12:15:17.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.350+0000 7f4f71400640 1 -- 172.21.15.167:0/1632465592 learned_addr learned my addr 172.21.15.167:0/1632465592 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:17.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.350+0000 7f4f71400640 1 -- 172.21.15.167:0/1632465592 >> v2:172.21.15.204:3300/0 conn(0x7f4f7415e880 msgr2=0x7f4f7415ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:17.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.350+0000 7f4f72800640 1 --2- 172.21.15.167:0/1632465592 >> v2:172.21.15.204:3300/0 conn(0x7f4f7415e880 0x7f4f7415ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:17.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.350+0000 7f4f71400640 1 --2- 172.21.15.167:0/1632465592 >> v2:172.21.15.204:3300/0 conn(0x7f4f7415e880 0x7f4f7415ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:17.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.350+0000 7f4f71400640 1 -- 172.21.15.167:0/1632465592 >> v1:172.21.15.204:6789/0 conn(0x7f4f740b62c0 legacy=0x7f4f740b66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:17.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.351+0000 7f4f71400640 1 -- 172.21.15.167:0/1632465592 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f74072800 con 0x7f4f74073a00 2024-05-11T12:15:17.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.352+0000 7f4f71400640 1 --2- 172.21.15.167:0/1632465592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f74073a00 0x7f4f74073e80 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f4f74072450 tx=0x7f4f6002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d60ea3595ddb566 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:17.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.352+0000 7f4f70a00640 1 -- 172.21.15.167:0/1632465592 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f60037070 con 0x7f4f74073a00 2024-05-11T12:15:17.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.352+0000 7f4f70a00640 1 -- 172.21.15.167:0/1632465592 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f60032440 con 0x7f4f74073a00 2024-05-11T12:15:17.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.352+0000 7f4f70a00640 1 -- 172.21.15.167:0/1632465592 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f60032810 con 0x7f4f74073a00 2024-05-11T12:15:17.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.352+0000 7f4f78a00640 1 -- 172.21.15.167:0/1632465592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f74073a00 msgr2=0x7f4f74073e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:17.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.352+0000 7f4f78a00640 1 --2- 172.21.15.167:0/1632465592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f74073a00 0x7f4f74073e80 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f4f74072450 tx=0x7f4f6002a1c0 comp rx=0 tx=0).stop 2024-05-11T12:15:17.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.353+0000 7f4f78a00640 1 -- 172.21.15.167:0/1632465592 shutdown_connections 2024-05-11T12:15:17.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.353+0000 7f4f78a00640 1 --2- 172.21.15.167:0/1632465592 >> v2:172.21.15.204:3300/0 conn(0x7f4f7415e880 0x7f4f7415ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:17.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.353+0000 7f4f78a00640 1 --2- 172.21.15.167:0/1632465592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f74073a00 0x7f4f74073e80 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:17.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.353+0000 7f4f78a00640 1 -- 172.21.15.167:0/1632465592 >> 172.21.15.167:0/1632465592 conn(0x7f4f740b2cb0 msgr2=0x7f4f740b3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:17.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.353+0000 7f4f78a00640 1 -- 172.21.15.167:0/1632465592 shutdown_connections 2024-05-11T12:15:17.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.353+0000 7f4f78a00640 1 -- 172.21.15.167:0/1632465592 wait complete. 2024-05-11T12:15:17.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.354+0000 7f4f78a00640 1 Processor -- start 2024-05-11T12:15:17.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.354+0000 7f4f78a00640 1 -- start start 2024-05-11T12:15:17.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.355+0000 7f4f78a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f74073a00 0x7f4f7415b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:17.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.355+0000 7f4f78a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f740b62c0 0x7f4f741515d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:17.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.355+0000 7f4f78a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f7415e880 0x7f4f74151b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:17.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.355+0000 7f4f78a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4f740743c0 con 0x7f4f740b62c0 2024-05-11T12:15:17.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.355+0000 7f4f78a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4f74072dc0 con 0x7f4f74073a00 2024-05-11T12:15:17.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.355+0000 7f4f78a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4f741579c0 con 0x7f4f7415e880 2024-05-11T12:15:17.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.355+0000 7f4f71400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f740b62c0 0x7f4f741515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:17.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.356+0000 7f4f71400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f740b62c0 0x7f4f741515d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35404/0 (socket says 172.21.15.167:35404) 2024-05-11T12:15:17.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.356+0000 7f4f71400640 1 -- 172.21.15.167:0/2751755117 learned_addr learned my addr 172.21.15.167:0/2751755117 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:17.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.356+0000 7f4f71400640 1 -- 172.21.15.167:0/2751755117 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f7415e880 msgr2=0x7f4f74151b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:17.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.356+0000 7f4f71400640 1 --2- 172.21.15.167:0/2751755117 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f7415e880 0x7f4f74151b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:17.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.356+0000 7f4f71400640 1 -- 172.21.15.167:0/2751755117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f74073a00 msgr2=0x7f4f7415b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:17.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.356+0000 7f4f71400640 1 --2- 172.21.15.167:0/2751755117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f74073a00 0x7f4f7415b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:17.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.356+0000 7f4f71400640 1 -- 172.21.15.167:0/2751755117 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f7414f230 con 0x7f4f740b62c0 2024-05-11T12:15:17.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.356+0000 7f4f71400640 1 --2- 172.21.15.167:0/2751755117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f740b62c0 0x7f4f741515d0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f4f6002a200 tx=0x7f4f600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:17.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.357+0000 7f4f6a000640 1 -- 172.21.15.167:0/2751755117 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f60037040 con 0x7f4f740b62c0 2024-05-11T12:15:17.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.357+0000 7f4f6a000640 1 -- 172.21.15.167:0/2751755117 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f60042070 con 0x7f4f740b62c0 2024-05-11T12:15:17.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.357+0000 7f4f78a00640 1 -- 172.21.15.167:0/2751755117 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f740b01b0 con 0x7f4f740b62c0 2024-05-11T12:15:17.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.357+0000 7f4f6a000640 1 -- 172.21.15.167:0/2751755117 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f60030bb0 con 0x7f4f740b62c0 2024-05-11T12:15:17.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.357+0000 7f4f78a00640 1 -- 172.21.15.167:0/2751755117 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f74150af0 con 0x7f4f740b62c0 2024-05-11T12:15:17.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.358+0000 7f4f78a00640 1 -- 172.21.15.167:0/2751755117 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f74069820 con 0x7f4f740b62c0 2024-05-11T12:15:17.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.359+0000 7f4f6a000640 1 -- 172.21.15.167:0/2751755117 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4f6002ad10 con 0x7f4f740b62c0 2024-05-11T12:15:17.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.360+0000 7f4f6a000640 1 --2- 172.21.15.167:0/2751755117 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4f540762d0 0x7f4f54078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:17.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.360+0000 7f4f6a000640 1 -- 172.21.15.167:0/2751755117 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6359+0+0 (secure 0 0 0) 0x7f4f600b8310 con 0x7f4f740b62c0 2024-05-11T12:15:17.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.361+0000 7f4f71e00640 1 --2- 172.21.15.167:0/2751755117 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4f540762d0 0x7f4f54078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:17.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.361+0000 7f4f71e00640 1 --2- 172.21.15.167:0/2751755117 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4f540762d0 0x7f4f54078790 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f4f64003cc0 tx=0x7f4f640079b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:17.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.365+0000 7f4f6a000640 1 -- 172.21.15.167:0/2751755117 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4f60081940 con 0x7f4f740b62c0 2024-05-11T12:15:17.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:17.610+0000 7f4f78a00640 1 -- 172.21.15.167:0/2751755117 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f4f74151190 con 0x7f4f740b62c0 2024-05-11T12:15:18.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.149+0000 7f4f6a000640 1 -- 172.21.15.167:0/2751755117 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v75) ==== 104+0+0 (secure 0 0 0) 0x7f4f60081240 con 0x7f4f740b62c0 2024-05-11T12:15:18.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.153+0000 7f4f78a00640 1 -- 172.21.15.167:0/2751755117 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4f540762d0 msgr2=0x7f4f54078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:18.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.153+0000 7f4f78a00640 1 --2- 172.21.15.167:0/2751755117 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4f540762d0 0x7f4f54078790 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f4f64003cc0 tx=0x7f4f640079b0 comp rx=0 tx=0).stop 2024-05-11T12:15:18.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.153+0000 7f4f78a00640 1 -- 172.21.15.167:0/2751755117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f740b62c0 msgr2=0x7f4f741515d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:18.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.153+0000 7f4f78a00640 1 --2- 172.21.15.167:0/2751755117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f740b62c0 0x7f4f741515d0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f4f6002a200 tx=0x7f4f600046c0 comp rx=0 tx=0).stop 2024-05-11T12:15:18.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.153+0000 7f4f78a00640 1 -- 172.21.15.167:0/2751755117 shutdown_connections 2024-05-11T12:15:18.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.153+0000 7f4f78a00640 1 --2- 172.21.15.167:0/2751755117 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f7415e880 0x7f4f74151b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:18.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.153+0000 7f4f78a00640 1 --2- 172.21.15.167:0/2751755117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f740b62c0 0x7f4f741515d0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:18.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.153+0000 7f4f78a00640 1 --2- 172.21.15.167:0/2751755117 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4f540762d0 0x7f4f54078790 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:18.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.153+0000 7f4f78a00640 1 --2- 172.21.15.167:0/2751755117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f74073a00 0x7f4f7415b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:18.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.153+0000 7f4f78a00640 1 -- 172.21.15.167:0/2751755117 >> 172.21.15.167:0/2751755117 conn(0x7f4f740b2cb0 msgr2=0x7f4f7415eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:18.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.153+0000 7f4f78a00640 1 -- 172.21.15.167:0/2751755117 shutdown_connections 2024-05-11T12:15:18.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.153+0000 7f4f78a00640 1 -- 172.21.15.167:0/2751755117 wait complete. 2024-05-11T12:15:18.177 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-11T12:15:18.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.426+0000 7f2ef2400640 1 Processor -- start 2024-05-11T12:15:18.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.427+0000 7f2ef2400640 1 -- start start 2024-05-11T12:15:18.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.428+0000 7f2ef2400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2eec0b3600 0x7f2eec15fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:18.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.428+0000 7f2ef2400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2eec1602e0 0x7f2eec0cee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:18.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.428+0000 7f2ef2400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2eec15d550 con 0x7f2eec13eee0 2024-05-11T12:15:18.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.428+0000 7f2ef2400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f2eec160b10 con 0x7f2eec1602e0 2024-05-11T12:15:18.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.428+0000 7f2ef2400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2eec15c790 con 0x7f2eec0b3600 2024-05-11T12:15:18.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.429+0000 7f2ee3e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2eec0b3600 0x7f2eec15fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:18.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.429+0000 7f2ee3e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2eec0b3600 0x7f2eec15fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:48320/0 (socket says 172.21.15.167:48320) 2024-05-11T12:15:18.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.429+0000 7f2ee3e00640 1 -- 172.21.15.167:0/3017987662 learned_addr learned my addr 172.21.15.167:0/3017987662 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:18.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.429+0000 7f2eebe00640 1 --2- 172.21.15.167:0/3017987662 >> v2:172.21.15.204:3300/0 conn(0x7f2eec1602e0 0x7f2eec0cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:18.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.430+0000 7f2ee3e00640 1 -- 172.21.15.167:0/3017987662 >> v2:172.21.15.204:3300/0 conn(0x7f2eec1602e0 msgr2=0x7f2eec0cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:18.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.430+0000 7f2ee3e00640 1 --2- 172.21.15.167:0/3017987662 >> v2:172.21.15.204:3300/0 conn(0x7f2eec1602e0 0x7f2eec0cee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:18.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.430+0000 7f2ee3e00640 1 -- 172.21.15.167:0/3017987662 >> v1:172.21.15.204:6789/0 conn(0x7f2eec13eee0 legacy=0x7f2eec0b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:18.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.430+0000 7f2ee3e00640 1 -- 172.21.15.167:0/3017987662 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eec15ca70 con 0x7f2eec0b3600 2024-05-11T12:15:18.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.431+0000 7f2ee3e00640 1 --2- 172.21.15.167:0/3017987662 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2eec0b3600 0x7f2eec15fda0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2ed400bb10 tx=0x7f2ed402c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c75864cddb25a8de server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:18.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.431+0000 7f2eeaa00640 1 -- 172.21.15.167:0/3017987662 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4036020 con 0x7f2eec0b3600 2024-05-11T12:15:18.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.432+0000 7f2eeaa00640 1 -- 172.21.15.167:0/3017987662 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ed4032820 con 0x7f2eec0b3600 2024-05-11T12:15:18.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.432+0000 7f2eeaa00640 1 -- 172.21.15.167:0/3017987662 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4032bf0 con 0x7f2eec0b3600 2024-05-11T12:15:18.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.432+0000 7f2ef2400640 1 -- 172.21.15.167:0/3017987662 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2eec0b3600 msgr2=0x7f2eec15fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:18.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.432+0000 7f2ef2400640 1 --2- 172.21.15.167:0/3017987662 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2eec0b3600 0x7f2eec15fda0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2ed400bb10 tx=0x7f2ed402c920 comp rx=0 tx=0).stop 2024-05-11T12:15:18.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.432+0000 7f2ef2400640 1 -- 172.21.15.167:0/3017987662 shutdown_connections 2024-05-11T12:15:18.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.432+0000 7f2ef2400640 1 --2- 172.21.15.167:0/3017987662 >> v2:172.21.15.204:3300/0 conn(0x7f2eec1602e0 0x7f2eec0cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:18.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.432+0000 7f2ef2400640 1 --2- 172.21.15.167:0/3017987662 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2eec0b3600 0x7f2eec15fda0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:18.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.432+0000 7f2ef2400640 1 -- 172.21.15.167:0/3017987662 >> 172.21.15.167:0/3017987662 conn(0x7f2eec06c660 msgr2=0x7f2eec06ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:18.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.433+0000 7f2ef2400640 1 -- 172.21.15.167:0/3017987662 shutdown_connections 2024-05-11T12:15:18.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.433+0000 7f2ef2400640 1 -- 172.21.15.167:0/3017987662 wait complete. 2024-05-11T12:15:18.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.435+0000 7f2ef2400640 1 Processor -- start 2024-05-11T12:15:18.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.435+0000 7f2ef2400640 1 -- start start 2024-05-11T12:15:18.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.436+0000 7f2ef2400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2eec0b3600 0x7f2eec152420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:18.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.436+0000 7f2ef2400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2eec13eee0 0x7f2eec158a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:18.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.436+0000 7f2ef2400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2eec1602e0 0x7f2eec156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:18.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.436+0000 7f2ef2400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec13f2c0 con 0x7f2eec13eee0 2024-05-11T12:15:18.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.436+0000 7f2ef2400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec15d030 con 0x7f2eec0b3600 2024-05-11T12:15:18.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.436+0000 7f2ef2400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2eec13f440 con 0x7f2eec1602e0 2024-05-11T12:15:18.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.437+0000 7f2ee3e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2eec13eee0 0x7f2eec158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:18.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.437+0000 7f2ee3e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2eec13eee0 0x7f2eec158a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35416/0 (socket says 172.21.15.167:35416) 2024-05-11T12:15:18.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.437+0000 7f2ee3e00640 1 -- 172.21.15.167:0/3217748397 learned_addr learned my addr 172.21.15.167:0/3217748397 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:18.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.437+0000 7f2eeb400640 1 --2- 172.21.15.167:0/3217748397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2eec0b3600 0x7f2eec152420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:18.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.437+0000 7f2eebe00640 1 --2- 172.21.15.167:0/3217748397 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2eec1602e0 0x7f2eec156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:18.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.437+0000 7f2ee3e00640 1 -- 172.21.15.167:0/3217748397 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2eec1602e0 msgr2=0x7f2eec156ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:18.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.437+0000 7f2ee3e00640 1 --2- 172.21.15.167:0/3217748397 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2eec1602e0 0x7f2eec156ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:18.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.437+0000 7f2ee3e00640 1 -- 172.21.15.167:0/3217748397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2eec0b3600 msgr2=0x7f2eec152420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:18.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.437+0000 7f2ee3e00640 1 --2- 172.21.15.167:0/3217748397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2eec0b3600 0x7f2eec152420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:18.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.437+0000 7f2ee3e00640 1 -- 172.21.15.167:0/3217748397 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eec157980 con 0x7f2eec13eee0 2024-05-11T12:15:18.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.437+0000 7f2eebe00640 1 --2- 172.21.15.167:0/3217748397 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2eec1602e0 0x7f2eec156ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:15:18.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.438+0000 7f2ee3e00640 1 --2- 172.21.15.167:0/3217748397 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2eec13eee0 0x7f2eec158a20 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f2ed400b2c0 tx=0x7f2ed4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:18.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.438+0000 7f2ee3400640 1 -- 172.21.15.167:0/3217748397 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4042070 con 0x7f2eec13eee0 2024-05-11T12:15:18.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.438+0000 7f2ee3400640 1 -- 172.21.15.167:0/3217748397 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ed4031cb0 con 0x7f2eec13eee0 2024-05-11T12:15:18.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.438+0000 7f2ee3400640 1 -- 172.21.15.167:0/3217748397 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4036040 con 0x7f2eec13eee0 2024-05-11T12:15:18.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.438+0000 7f2ef2400640 1 -- 172.21.15.167:0/3217748397 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eec157c60 con 0x7f2eec13eee0 2024-05-11T12:15:18.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.438+0000 7f2ef2400640 1 -- 172.21.15.167:0/3217748397 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eec158220 con 0x7f2eec13eee0 2024-05-11T12:15:18.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.439+0000 7f2ef2400640 1 -- 172.21.15.167:0/3217748397 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2eec160710 con 0x7f2eec13eee0 2024-05-11T12:15:18.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.440+0000 7f2ee3400640 1 -- 172.21.15.167:0/3217748397 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2ed4045070 con 0x7f2eec13eee0 2024-05-11T12:15:18.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.441+0000 7f2ee3400640 1 --2- 172.21.15.167:0/3217748397 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2ec8076300 0x7f2ec80787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:18.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.441+0000 7f2ee3400640 1 -- 172.21.15.167:0/3217748397 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6359+0+0 (secure 0 0 0) 0x7f2ed40b82c0 con 0x7f2eec13eee0 2024-05-11T12:15:18.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.441+0000 7f2eeb400640 1 --2- 172.21.15.167:0/3217748397 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2ec8076300 0x7f2ec80787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:18.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.442+0000 7f2eeb400640 1 --2- 172.21.15.167:0/3217748397 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2ec8076300 0x7f2ec80787c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2edc000f60 tx=0x7f2edc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:18.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.446+0000 7f2ee3400640 1 -- 172.21.15.167:0/3217748397 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2eec160710 con 0x7f2eec13eee0 2024-05-11T12:15:18.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:18.644+0000 7f2ef2400640 1 -- 172.21.15.167:0/3217748397 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2eec069820 con 0x7f2eec13eee0 2024-05-11T12:15:19.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.162+0000 7f2ee3400640 1 -- 172.21.15.167:0/3217748397 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v103) ==== 181+0+0 (secure 0 0 0) 0x7f2ed402f070 con 0x7f2eec13eee0 2024-05-11T12:15:19.163 INFO:teuthology.orchestra.run.smithi167.stderr:pool 'cephfs_metadata' removed 2024-05-11T12:15:19.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.167+0000 7f2ef2400640 1 -- 172.21.15.167:0/3217748397 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2ec8076300 msgr2=0x7f2ec80787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:19.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.167+0000 7f2ef2400640 1 --2- 172.21.15.167:0/3217748397 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2ec8076300 0x7f2ec80787c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2edc000f60 tx=0x7f2edc009040 comp rx=0 tx=0).stop 2024-05-11T12:15:19.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.167+0000 7f2ef2400640 1 -- 172.21.15.167:0/3217748397 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2eec13eee0 msgr2=0x7f2eec158a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:19.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.167+0000 7f2ef2400640 1 --2- 172.21.15.167:0/3217748397 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2eec13eee0 0x7f2eec158a20 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f2ed400b2c0 tx=0x7f2ed4004730 comp rx=0 tx=0).stop 2024-05-11T12:15:19.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.167+0000 7f2ef2400640 1 -- 172.21.15.167:0/3217748397 shutdown_connections 2024-05-11T12:15:19.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.167+0000 7f2ef2400640 1 --2- 172.21.15.167:0/3217748397 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2eec1602e0 0x7f2eec156ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:19.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.167+0000 7f2ef2400640 1 --2- 172.21.15.167:0/3217748397 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2eec13eee0 0x7f2eec158a20 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:19.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.167+0000 7f2ef2400640 1 --2- 172.21.15.167:0/3217748397 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f2ec8076300 0x7f2ec80787c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:19.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.167+0000 7f2ef2400640 1 --2- 172.21.15.167:0/3217748397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2eec0b3600 0x7f2eec152420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:19.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.167+0000 7f2ef2400640 1 -- 172.21.15.167:0/3217748397 >> 172.21.15.167:0/3217748397 conn(0x7f2eec06c660 msgr2=0x7f2eec15f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:19.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.167+0000 7f2ef2400640 1 -- 172.21.15.167:0/3217748397 shutdown_connections 2024-05-11T12:15:19.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.168+0000 7f2ef2400640 1 -- 172.21.15.167:0/3217748397 wait complete. 2024-05-11T12:15:19.192 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-11T12:15:19.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.400+0000 7f5880800640 1 Processor -- start 2024-05-11T12:15:19.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.400+0000 7f5880800640 1 -- start start 2024-05-11T12:15:19.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.402+0000 7f5880800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5878160870 0x7f587815ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:19.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.402+0000 7f5880800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f58780b56f0 0x7f58780b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:19.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.402+0000 7f5880800640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5878160c50 con 0x7f58780b3fb0 2024-05-11T12:15:19.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.402+0000 7f5880800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f587815de10 con 0x7f58780b56f0 2024-05-11T12:15:19.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.402+0000 7f5880800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f587815e0f0 con 0x7f5878160870 2024-05-11T12:15:19.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.403+0000 7f587e800640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f58780b3fb0 0x7f58781612e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:33192/0 (socket says 172.21.15.167:33192) 2024-05-11T12:15:19.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.403+0000 7f587e800640 1 -- 172.21.15.167:0/421019582 learned_addr learned my addr 172.21.15.167:0/421019582 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:15:19.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.403+0000 7f587d400640 1 --2- 172.21.15.167:0/421019582 >> v2:172.21.15.204:3300/0 conn(0x7f58780b56f0 0x7f58780b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:19.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.403+0000 7f587de00640 1 --2- 172.21.15.167:0/421019582 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5878160870 0x7f587815ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:19.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.403+0000 7f587ca00640 1 -- 172.21.15.167:0/421019582 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2145575836 0 0) 0x7f5878160c50 con 0x7f58780b3fb0 2024-05-11T12:15:19.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.403+0000 7f587ca00640 1 -- 172.21.15.167:0/421019582 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5860003520 con 0x7f58780b3fb0 2024-05-11T12:15:19.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.403+0000 7f587ca00640 1 -- 172.21.15.167:0/421019582 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3756556512 0 0) 0x7f5860003520 con 0x7f58780b3fb0 2024-05-11T12:15:19.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.403+0000 7f587ca00640 1 -- 172.21.15.167:0/421019582 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5878160870 msgr2=0x7f587815ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:19.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.403+0000 7f587ca00640 1 --2- 172.21.15.167:0/421019582 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5878160870 0x7f587815ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:19.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.403+0000 7f587ca00640 1 -- 172.21.15.167:0/421019582 >> v2:172.21.15.204:3300/0 conn(0x7f58780b56f0 msgr2=0x7f58780b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:19.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.403+0000 7f587ca00640 1 --2- 172.21.15.167:0/421019582 >> v2:172.21.15.204:3300/0 conn(0x7f58780b56f0 0x7f58780b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:19.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.403+0000 7f587ca00640 1 -- 172.21.15.167:0/421019582 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f587815e3d0 con 0x7f58780b3fb0 2024-05-11T12:15:19.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.404+0000 7f587ca00640 1 -- 172.21.15.167:0/421019582 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f5874001a70 con 0x7f58780b3fb0 2024-05-11T12:15:19.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.404+0000 7f587ca00640 1 -- 172.21.15.167:0/421019582 >> v1:172.21.15.167:6789/0 conn(0x7f58780b3fb0 legacy=0x7f58781612e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:19.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.404+0000 7f587ca00640 1 --2- 172.21.15.167:0/421019582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5860003a50 0x7f5860003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:19.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.404+0000 7f587ca00640 1 --2- 172.21.15.167:0/421019582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5860004800 0x7f5860004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:19.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.404+0000 7f587ca00640 1 --2- 172.21.15.167:0/421019582 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5878160870 0x7f5860005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:19.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.404+0000 7f587ca00640 1 -- 172.21.15.167:0/421019582 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f587815de10 con 0x7f5860003a50 2024-05-11T12:15:19.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.404+0000 7f587ca00640 1 -- 172.21.15.167:0/421019582 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f587815e0f0 con 0x7f5860004800 2024-05-11T12:15:19.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.404+0000 7f587ca00640 1 -- 172.21.15.167:0/421019582 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f586000bc00 con 0x7f5878160870 2024-05-11T12:15:19.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.404+0000 7f587de00640 1 --2- 172.21.15.167:0/421019582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5860003a50 0x7f5860003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:19.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.404+0000 7f587d400640 1 --2- 172.21.15.167:0/421019582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5860004800 0x7f5860004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:19.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.404+0000 7f587de00640 1 -- 172.21.15.167:0/421019582 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5878160870 msgr2=0x7f5860005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:19.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.404+0000 7f587e800640 1 --2- 172.21.15.167:0/421019582 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5878160870 0x7f5860005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:19.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.405+0000 7f587de00640 1 --2- 172.21.15.167:0/421019582 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5878160870 0x7f5860005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:19.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.405+0000 7f587de00640 1 -- 172.21.15.167:0/421019582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5860004800 msgr2=0x7f5860004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:19.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.405+0000 7f587de00640 1 --2- 172.21.15.167:0/421019582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5860004800 0x7f5860004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:19.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.405+0000 7f587de00640 1 -- 172.21.15.167:0/421019582 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5860003520 con 0x7f5860003a50 2024-05-11T12:15:19.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.405+0000 7f587e800640 1 --2- 172.21.15.167:0/421019582 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5878160870 0x7f5860005270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:15:19.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.405+0000 7f587d400640 1 --2- 172.21.15.167:0/421019582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5860004800 0x7f5860004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:15:19.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.406+0000 7f587de00640 1 --2- 172.21.15.167:0/421019582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5860003a50 0x7f5860003e70 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f586c00ba30 tx=0x7f5860005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b2e15db7082017b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:19.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.406+0000 7f587ca00640 1 -- 172.21.15.167:0/421019582 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586c030070 con 0x7f5860003a50 2024-05-11T12:15:19.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.406+0000 7f587ca00640 1 -- 172.21.15.167:0/421019582 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f586c02ade0 con 0x7f5860003a50 2024-05-11T12:15:19.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.406+0000 7f587ca00640 1 -- 172.21.15.167:0/421019582 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586c00b600 con 0x7f5860003a50 2024-05-11T12:15:19.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.407+0000 7f5880800640 1 -- 172.21.15.167:0/421019582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5860003a50 msgr2=0x7f5860003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:19.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.407+0000 7f5880800640 1 --2- 172.21.15.167:0/421019582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5860003a50 0x7f5860003e70 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f586c00ba30 tx=0x7f5860005830 comp rx=0 tx=0).stop 2024-05-11T12:15:19.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.407+0000 7f587e800640 1 -- 172.21.15.167:0/421019582 reap_dead start 2024-05-11T12:15:19.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.407+0000 7f5880800640 1 -- 172.21.15.167:0/421019582 shutdown_connections 2024-05-11T12:15:19.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.407+0000 7f5880800640 1 -- 172.21.15.167:0/421019582 >> 172.21.15.167:0/421019582 conn(0x7f587806c660 msgr2=0x7f587806ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:19.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.407+0000 7f5880800640 1 -- 172.21.15.167:0/421019582 shutdown_connections 2024-05-11T12:15:19.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.407+0000 7f5880800640 1 -- 172.21.15.167:0/421019582 wait complete. 2024-05-11T12:15:19.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.409+0000 7f5880800640 1 Processor -- start 2024-05-11T12:15:19.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.409+0000 7f5880800640 1 -- start start 2024-05-11T12:15:19.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.410+0000 7f5880800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5860003a50 0x7f58780af180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:19.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.410+0000 7f5880800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f587813d840 0x7f587815f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:19.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.410+0000 7f5880800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f587815f930 0x7f58780ad460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:19.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.410+0000 7f5880800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f587815d9d0 con 0x7f587813d840 2024-05-11T12:15:19.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.410+0000 7f5880800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f587815d850 con 0x7f587815f930 2024-05-11T12:15:19.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.410+0000 7f5880800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f587815db50 con 0x7f5860003a50 2024-05-11T12:15:19.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.410+0000 7f587d400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f587813d840 0x7f587815f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:19.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.410+0000 7f587d400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f587813d840 0x7f587815f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:46864/0 (socket says 172.21.15.167:46864) 2024-05-11T12:15:19.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.410+0000 7f587d400640 1 -- 172.21.15.167:0/181803878 learned_addr learned my addr 172.21.15.167:0/181803878 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:19.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.411+0000 7f587de00640 1 --2- 172.21.15.167:0/181803878 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5860003a50 0x7f58780af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:19.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.411+0000 7f587d400640 1 -- 172.21.15.167:0/181803878 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5860003a50 msgr2=0x7f58780af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:19.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.411+0000 7f587e800640 1 --2- 172.21.15.167:0/181803878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f587815f930 0x7f58780ad460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:19.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.411+0000 7f587d400640 1 --2- 172.21.15.167:0/181803878 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5860003a50 0x7f58780af180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:19.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.411+0000 7f587d400640 1 -- 172.21.15.167:0/181803878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f587815f930 msgr2=0x7f58780ad460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:19.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.411+0000 7f587d400640 1 --2- 172.21.15.167:0/181803878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f587815f930 0x7f58780ad460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:19.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.411+0000 7f587d400640 1 -- 172.21.15.167:0/181803878 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58780ae0e0 con 0x7f587813d840 2024-05-11T12:15:19.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.411+0000 7f587de00640 1 --2- 172.21.15.167:0/181803878 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5860003a50 0x7f58780af180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:15:19.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.411+0000 7f587e800640 1 --2- 172.21.15.167:0/181803878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f587815f930 0x7f58780ad460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:15:19.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.411+0000 7f587d400640 1 --2- 172.21.15.167:0/181803878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f587813d840 0x7f587815f3f0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f58680109d0 tx=0x7f5868010e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:19.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.411+0000 7f5872000640 1 -- 172.21.15.167:0/181803878 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586800f040 con 0x7f587813d840 2024-05-11T12:15:19.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.411+0000 7f5872000640 1 -- 172.21.15.167:0/181803878 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f586800ebc0 con 0x7f587813d840 2024-05-11T12:15:19.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.411+0000 7f5872000640 1 -- 172.21.15.167:0/181803878 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586801c400 con 0x7f587813d840 2024-05-11T12:15:19.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.411+0000 7f5880800640 1 -- 172.21.15.167:0/181803878 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58780ae3c0 con 0x7f587813d840 2024-05-11T12:15:19.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.412+0000 7f5880800640 1 -- 172.21.15.167:0/181803878 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58780ae980 con 0x7f587813d840 2024-05-11T12:15:19.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.413+0000 7f5880800640 1 -- 172.21.15.167:0/181803878 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5878069820 con 0x7f587813d840 2024-05-11T12:15:19.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.414+0000 7f5872000640 1 -- 172.21.15.167:0/181803878 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f586801c5a0 con 0x7f587813d840 2024-05-11T12:15:19.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.415+0000 7f5872000640 1 --2- 172.21.15.167:0/181803878 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f585c0762d0 0x7f585c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:19.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.415+0000 7f5872000640 1 -- 172.21.15.167:0/181803878 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5921+0+0 (secure 0 0 0) 0x7f5868097e00 con 0x7f587813d840 2024-05-11T12:15:19.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.415+0000 7f587de00640 1 --2- 172.21.15.167:0/181803878 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f585c0762d0 0x7f585c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:19.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.416+0000 7f587de00640 1 --2- 172.21.15.167:0/181803878 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f585c0762d0 0x7f585c078790 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f586c00ba00 tx=0x7f586c030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:19.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.419+0000 7f5872000640 1 -- 172.21.15.167:0/181803878 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5868018030 con 0x7f587813d840 2024-05-11T12:15:19.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:19.628+0000 7f5880800640 1 -- 172.21.15.167:0/181803878 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5878069f80 con 0x7f587813d840 2024-05-11T12:15:20.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.172+0000 7f5872000640 1 -- 172.21.15.167:0/181803878 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v104) ==== 169+0+0 (secure 0 0 0) 0x7f5868061550 con 0x7f587813d840 2024-05-11T12:15:20.173 INFO:teuthology.orchestra.run.smithi167.stderr:pool 'cephfs_data' removed 2024-05-11T12:15:20.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.177+0000 7f5880800640 1 -- 172.21.15.167:0/181803878 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f585c0762d0 msgr2=0x7f585c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:20.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.177+0000 7f5880800640 1 --2- 172.21.15.167:0/181803878 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f585c0762d0 0x7f585c078790 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f586c00ba00 tx=0x7f586c030000 comp rx=0 tx=0).stop 2024-05-11T12:15:20.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.177+0000 7f5880800640 1 -- 172.21.15.167:0/181803878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f587813d840 msgr2=0x7f587815f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:20.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.177+0000 7f5880800640 1 --2- 172.21.15.167:0/181803878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f587813d840 0x7f587815f3f0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f58680109d0 tx=0x7f5868010e90 comp rx=0 tx=0).stop 2024-05-11T12:15:20.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.177+0000 7f5880800640 1 -- 172.21.15.167:0/181803878 shutdown_connections 2024-05-11T12:15:20.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.177+0000 7f5880800640 1 --2- 172.21.15.167:0/181803878 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f585c0762d0 0x7f585c078790 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:20.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.177+0000 7f5880800640 1 --2- 172.21.15.167:0/181803878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f587815f930 0x7f58780ad460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:20.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.177+0000 7f5880800640 1 --2- 172.21.15.167:0/181803878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f587813d840 0x7f587815f3f0 secure :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f58680109d0 tx=0x7f5868010e90 comp rx=0 tx=0).stop 2024-05-11T12:15:20.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.177+0000 7f5880800640 1 --2- 172.21.15.167:0/181803878 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5860003a50 0x7f58780af180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:20.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.177+0000 7f5880800640 1 -- 172.21.15.167:0/181803878 >> 172.21.15.167:0/181803878 conn(0x7f587806c660 msgr2=0x7f58780b2ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:20.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.179+0000 7f5880800640 1 -- 172.21.15.167:0/181803878 shutdown_connections 2024-05-11T12:15:20.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.179+0000 7f5880800640 1 -- 172.21.15.167:0/181803878 wait complete. 2024-05-11T12:15:20.204 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-11T12:15:20.208 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-11T12:15:20.328 INFO:teuthology.orchestra.run.smithi167.stdout:[global] 2024-05-11T12:15:20.328 INFO:teuthology.orchestra.run.smithi167.stdout: chdir = "" 2024-05-11T12:15:20.328 INFO:teuthology.orchestra.run.smithi167.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-11T12:15:20.328 INFO:teuthology.orchestra.run.smithi167.stdout: auth supported = cephx 2024-05-11T12:15:20.328 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:15:20.328 INFO:teuthology.orchestra.run.smithi167.stdout: filestore xattr use omap = true 2024-05-11T12:15:20.328 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:15:20.328 INFO:teuthology.orchestra.run.smithi167.stdout: mon clock drift allowed = 1.000 2024-05-11T12:15:20.328 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:15:20.328 INFO:teuthology.orchestra.run.smithi167.stdout: osd crush chooseleaf type = 0 2024-05-11T12:15:20.328 INFO:teuthology.orchestra.run.smithi167.stdout: auth debug = true 2024-05-11T12:15:20.328 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:15:20.329 INFO:teuthology.orchestra.run.smithi167.stdout: ms die on old message = true 2024-05-11T12:15:20.329 INFO:teuthology.orchestra.run.smithi167.stdout: ms die on bug = true 2024-05-11T12:15:20.329 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:15:20.329 INFO:teuthology.orchestra.run.smithi167.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-11T12:15:20.329 INFO:teuthology.orchestra.run.smithi167.stdout: mon pg warn max object skew = 0 2024-05-11T12:15:20.329 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:15:20.329 INFO:teuthology.orchestra.run.smithi167.stdout: # disable pg_autoscaler by default for new pools 2024-05-11T12:15:20.329 INFO:teuthology.orchestra.run.smithi167.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-11T12:15:20.329 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:15:20.329 INFO:teuthology.orchestra.run.smithi167.stdout: osd pool default size = 2 2024-05-11T12:15:20.329 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:15:20.329 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd allow primary affinity = true 2024-05-11T12:15:20.329 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd allow pg remap = true 2024-05-11T12:15:20.329 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on legacy crush tunables = false 2024-05-11T12:15:20.329 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on crush straw calc version zero = false 2024-05-11T12:15:20.330 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on no sortbitwise = false 2024-05-11T12:15:20.330 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on osd down out interval zero = false 2024-05-11T12:15:20.330 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on too few osds = false 2024-05-11T12:15:20.330 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-11T12:15:20.330 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-11T12:15:20.330 INFO:teuthology.orchestra.run.smithi167.stdout: mon_allow_pool_size_one = true 2024-05-11T12:15:20.330 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:15:20.330 INFO:teuthology.orchestra.run.smithi167.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-11T12:15:20.330 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:15:20.330 INFO:teuthology.orchestra.run.smithi167.stdout: osd default data pool replay window = 5 2024-05-11T12:15:20.330 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:15:20.330 INFO:teuthology.orchestra.run.smithi167.stdout: mon allow pool delete = true 2024-05-11T12:15:20.330 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:15:20.330 INFO:teuthology.orchestra.run.smithi167.stdout: mon cluster log file level = debug 2024-05-11T12:15:20.331 INFO:teuthology.orchestra.run.smithi167.stdout: debug asserts on shutdown = true 2024-05-11T12:15:20.331 INFO:teuthology.orchestra.run.smithi167.stdout: mon health detail to clog = false 2024-05-11T12:15:20.331 INFO:teuthology.orchestra.run.smithi167.stdout: mon host = "172.21.15.167,172.21.15.204,[v2:172.21.15.204:3301,v1:172.21.15.204:6790]" 2024-05-11T12:15:20.331 INFO:teuthology.orchestra.run.smithi167.stdout: ms die on skipped message = False 2024-05-11T12:15:20.331 INFO:teuthology.orchestra.run.smithi167.stdout: fsid = 5f32d344-7289-4c27-881b-1533419ab678 2024-05-11T12:15:20.331 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:15:20.331 INFO:teuthology.orchestra.run.smithi167.stdout:[osd] 2024-05-11T12:15:20.331 INFO:teuthology.orchestra.run.smithi167.stdout: osd journal size = 100 2024-05-11T12:15:20.331 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:15:20.331 INFO:teuthology.orchestra.run.smithi167.stdout: osd scrub load threshold = 5.0 2024-05-11T12:15:20.331 INFO:teuthology.orchestra.run.smithi167.stdout: osd scrub max interval = 600 2024-05-11T12:15:20.331 INFO:teuthology.orchestra.run.smithi167.stdout: osd mclock profile = high_recovery_ops 2024-05-11T12:15:20.331 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:15:20.331 INFO:teuthology.orchestra.run.smithi167.stdout: osd recover clone overlap = true 2024-05-11T12:15:20.331 INFO:teuthology.orchestra.run.smithi167.stdout: osd recovery max chunk = 1048576 2024-05-11T12:15:20.332 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:15:20.332 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug shutdown = true 2024-05-11T12:15:20.332 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug op order = true 2024-05-11T12:15:20.332 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug verify stray on activate = true 2024-05-11T12:15:20.332 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug trim objects = true 2024-05-11T12:15:20.332 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:15:20.332 INFO:teuthology.orchestra.run.smithi167.stdout: osd open classes on start = true 2024-05-11T12:15:20.332 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug pg log writeout = true 2024-05-11T12:15:20.332 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:15:20.332 INFO:teuthology.orchestra.run.smithi167.stdout: osd deep scrub update digest min age = 30 2024-05-11T12:15:20.332 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:15:20.332 INFO:teuthology.orchestra.run.smithi167.stdout: osd map max advance = 10 2024-05-11T12:15:20.332 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:15:20.332 INFO:teuthology.orchestra.run.smithi167.stdout: journal zero on create = true 2024-05-11T12:15:20.332 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:15:20.333 INFO:teuthology.orchestra.run.smithi167.stdout: filestore ondisk finisher threads = 3 2024-05-11T12:15:20.333 INFO:teuthology.orchestra.run.smithi167.stdout: filestore apply finisher threads = 3 2024-05-11T12:15:20.333 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:15:20.333 INFO:teuthology.orchestra.run.smithi167.stdout: bdev debug aio = true 2024-05-11T12:15:20.333 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug misdirected ops = true 2024-05-11T12:15:20.333 INFO:teuthology.orchestra.run.smithi167.stdout: bluestore block size = 96636764160 2024-05-11T12:15:20.333 INFO:teuthology.orchestra.run.smithi167.stdout: bluestore fsck on mount = True 2024-05-11T12:15:20.333 INFO:teuthology.orchestra.run.smithi167.stdout: debug bluefs = 20 2024-05-11T12:15:20.333 INFO:teuthology.orchestra.run.smithi167.stdout: debug bluestore = 20 2024-05-11T12:15:20.333 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2024-05-11T12:15:20.333 INFO:teuthology.orchestra.run.smithi167.stdout: debug osd = 20 2024-05-11T12:15:20.333 INFO:teuthology.orchestra.run.smithi167.stdout: debug rocksdb = 10 2024-05-11T12:15:20.333 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd backfillfull_ratio = 0.85 2024-05-11T12:15:20.333 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd full ratio = 0.9 2024-05-11T12:15:20.334 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd nearfull ratio = 0.8 2024-05-11T12:15:20.334 INFO:teuthology.orchestra.run.smithi167.stdout: osd failsafe full ratio = 0.95 2024-05-11T12:15:20.334 INFO:teuthology.orchestra.run.smithi167.stdout: osd objectstore = bluestore 2024-05-11T12:15:20.334 INFO:teuthology.orchestra.run.smithi167.stdout: osd op complaint time = 180 2024-05-11T12:15:20.334 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:15:20.334 INFO:teuthology.orchestra.run.smithi167.stdout:[mgr] 2024-05-11T12:15:20.334 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2024-05-11T12:15:20.334 INFO:teuthology.orchestra.run.smithi167.stdout: debug mgr = 20 2024-05-11T12:15:20.334 INFO:teuthology.orchestra.run.smithi167.stdout: debug mon = 20 2024-05-11T12:15:20.334 INFO:teuthology.orchestra.run.smithi167.stdout: debug auth = 20 2024-05-11T12:15:20.334 INFO:teuthology.orchestra.run.smithi167.stdout: mon reweight min pgs per osd = 4 2024-05-11T12:15:20.334 INFO:teuthology.orchestra.run.smithi167.stdout: mon reweight min bytes per osd = 10 2024-05-11T12:15:20.334 INFO:teuthology.orchestra.run.smithi167.stdout: mgr/telemetry/nag = false 2024-05-11T12:15:20.334 INFO:teuthology.orchestra.run.smithi167.stdout: debug client = 20 2024-05-11T12:15:20.334 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:15:20.335 INFO:teuthology.orchestra.run.smithi167.stdout:[mon] 2024-05-11T12:15:20.335 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2024-05-11T12:15:20.335 INFO:teuthology.orchestra.run.smithi167.stdout: debug mon = 20 2024-05-11T12:15:20.335 INFO:teuthology.orchestra.run.smithi167.stdout: debug paxos = 20 2024-05-11T12:15:20.335 INFO:teuthology.orchestra.run.smithi167.stdout: debug auth = 20 2024-05-11T12:15:20.335 INFO:teuthology.orchestra.run.smithi167.stdout: mon data avail warn = 5 2024-05-11T12:15:20.335 INFO:teuthology.orchestra.run.smithi167.stdout: mon mgr mkfs grace = 240 2024-05-11T12:15:20.335 INFO:teuthology.orchestra.run.smithi167.stdout: mon reweight min pgs per osd = 4 2024-05-11T12:15:20.335 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd reporter subtree level = osd 2024-05-11T12:15:20.335 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd prime pg temp = true 2024-05-11T12:15:20.335 INFO:teuthology.orchestra.run.smithi167.stdout: mon reweight min bytes per osd = 10 2024-05-11T12:15:20.335 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:15:20.335 INFO:teuthology.orchestra.run.smithi167.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-11T12:15:20.335 INFO:teuthology.orchestra.run.smithi167.stdout: auth mon ticket ttl = 660 # 11m 2024-05-11T12:15:20.335 INFO:teuthology.orchestra.run.smithi167.stdout: auth service ticket ttl = 240 # 4m 2024-05-11T12:15:20.336 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:15:20.336 INFO:teuthology.orchestra.run.smithi167.stdout: # don't complain about insecure global_id in the test suite 2024-05-11T12:15:20.336 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-11T12:15:20.336 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-11T12:15:20.336 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:15:20.336 INFO:teuthology.orchestra.run.smithi167.stdout: # 1m isn't quite enough 2024-05-11T12:15:20.336 INFO:teuthology.orchestra.run.smithi167.stdout: mon_down_mkfs_grace = 2m 2024-05-11T12:15:20.336 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:15:20.336 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_filestore_osds = false 2024-05-11T12:15:20.336 INFO:teuthology.orchestra.run.smithi167.stdout: mon down mkfs grace = 300 2024-05-11T12:15:20.336 INFO:teuthology.orchestra.run.smithi167.stdout: mon op complaint time = 120 2024-05-11T12:15:20.336 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:15:20.336 INFO:teuthology.orchestra.run.smithi167.stdout:[client] 2024-05-11T12:15:20.336 INFO:teuthology.orchestra.run.smithi167.stdout: rgw cache enabled = true 2024-05-11T12:15:20.336 INFO:teuthology.orchestra.run.smithi167.stdout: rgw enable ops log = true 2024-05-11T12:15:20.337 INFO:teuthology.orchestra.run.smithi167.stdout: rgw enable usage log = true 2024-05-11T12:15:20.337 INFO:teuthology.orchestra.run.smithi167.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-11T12:15:20.337 INFO:teuthology.orchestra.run.smithi167.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-11T12:15:20.337 INFO:teuthology.orchestra.run.smithi167.stdout: client mount timeout = 600 2024-05-11T12:15:20.337 INFO:teuthology.orchestra.run.smithi167.stdout: debug client = 20 2024-05-11T12:15:20.337 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2024-05-11T12:15:20.337 INFO:teuthology.orchestra.run.smithi167.stdout: pid file = "" 2024-05-11T12:15:20.337 INFO:teuthology.orchestra.run.smithi167.stdout: rados mon op timeout = 900 2024-05-11T12:15:20.337 INFO:teuthology.orchestra.run.smithi167.stdout: rados osd op timeout = 900 2024-05-11T12:15:20.337 INFO:teuthology.orchestra.run.smithi167.stdout:[mon.a] 2024-05-11T12:15:20.337 INFO:teuthology.orchestra.run.smithi167.stdout:[mon.b] 2024-05-11T12:15:20.337 INFO:teuthology.orchestra.run.smithi167.stdout:[mon.c] 2024-05-11T12:15:20.337 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.a] 2024-05-11T12:15:20.337 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.c] 2024-05-11T12:15:20.337 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.b] 2024-05-11T12:15:20.338 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.d] 2024-05-11T12:15:20.338 INFO:teuthology.orchestra.run.smithi167.stdout:[mds] 2024-05-11T12:15:20.338 INFO:teuthology.orchestra.run.smithi167.stdout: debug mds = 20 2024-05-11T12:15:20.338 INFO:teuthology.orchestra.run.smithi167.stdout: debug mds balancer = 20 2024-05-11T12:15:20.338 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2024-05-11T12:15:20.338 INFO:teuthology.orchestra.run.smithi167.stdout: mds debug frag = True 2024-05-11T12:15:20.338 INFO:teuthology.orchestra.run.smithi167.stdout: mds debug scatterstat = True 2024-05-11T12:15:20.338 INFO:teuthology.orchestra.run.smithi167.stdout: mds op complaint time = 180 2024-05-11T12:15:20.338 INFO:teuthology.orchestra.run.smithi167.stdout: mds verify scatter = True 2024-05-11T12:15:20.338 INFO:teuthology.orchestra.run.smithi167.stdout: osd op complaint time = 180 2024-05-11T12:15:20.338 INFO:teuthology.orchestra.run.smithi167.stdout: rados mon op timeout = 900 2024-05-11T12:15:20.338 INFO:teuthology.orchestra.run.smithi167.stdout: rados osd op timeout = 900 2024-05-11T12:15:20.339 INFO:teuthology.orchestra.run.smithi204.stdout:[global] 2024-05-11T12:15:20.339 INFO:teuthology.orchestra.run.smithi204.stdout: chdir = "" 2024-05-11T12:15:20.339 INFO:teuthology.orchestra.run.smithi204.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-11T12:15:20.339 INFO:teuthology.orchestra.run.smithi204.stdout: auth supported = cephx 2024-05-11T12:15:20.339 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:15:20.339 INFO:teuthology.orchestra.run.smithi204.stdout: filestore xattr use omap = true 2024-05-11T12:15:20.339 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:15:20.339 INFO:teuthology.orchestra.run.smithi204.stdout: mon clock drift allowed = 1.000 2024-05-11T12:15:20.339 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:15:20.339 INFO:teuthology.orchestra.run.smithi204.stdout: osd crush chooseleaf type = 0 2024-05-11T12:15:20.339 INFO:teuthology.orchestra.run.smithi204.stdout: auth debug = true 2024-05-11T12:15:20.340 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:15:20.340 INFO:teuthology.orchestra.run.smithi204.stdout: ms die on old message = true 2024-05-11T12:15:20.340 INFO:teuthology.orchestra.run.smithi204.stdout: ms die on bug = true 2024-05-11T12:15:20.340 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:15:20.340 INFO:teuthology.orchestra.run.smithi204.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-11T12:15:20.340 INFO:teuthology.orchestra.run.smithi204.stdout: mon pg warn max object skew = 0 2024-05-11T12:15:20.340 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:15:20.340 INFO:teuthology.orchestra.run.smithi204.stdout: # disable pg_autoscaler by default for new pools 2024-05-11T12:15:20.340 INFO:teuthology.orchestra.run.smithi204.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-11T12:15:20.340 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:15:20.340 INFO:teuthology.orchestra.run.smithi204.stdout: osd pool default size = 2 2024-05-11T12:15:20.340 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:15:20.340 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd allow primary affinity = true 2024-05-11T12:15:20.340 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd allow pg remap = true 2024-05-11T12:15:20.340 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on legacy crush tunables = false 2024-05-11T12:15:20.341 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on crush straw calc version zero = false 2024-05-11T12:15:20.341 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on no sortbitwise = false 2024-05-11T12:15:20.341 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on osd down out interval zero = false 2024-05-11T12:15:20.341 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on too few osds = false 2024-05-11T12:15:20.341 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-11T12:15:20.341 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-11T12:15:20.341 INFO:teuthology.orchestra.run.smithi204.stdout: mon_allow_pool_size_one = true 2024-05-11T12:15:20.341 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:15:20.341 INFO:teuthology.orchestra.run.smithi204.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-11T12:15:20.341 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:15:20.341 INFO:teuthology.orchestra.run.smithi204.stdout: osd default data pool replay window = 5 2024-05-11T12:15:20.341 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:15:20.341 INFO:teuthology.orchestra.run.smithi204.stdout: mon allow pool delete = true 2024-05-11T12:15:20.341 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:15:20.341 INFO:teuthology.orchestra.run.smithi204.stdout: mon cluster log file level = debug 2024-05-11T12:15:20.341 INFO:teuthology.orchestra.run.smithi204.stdout: debug asserts on shutdown = true 2024-05-11T12:15:20.342 INFO:teuthology.orchestra.run.smithi204.stdout: mon health detail to clog = false 2024-05-11T12:15:20.342 INFO:teuthology.orchestra.run.smithi204.stdout: mon host = "172.21.15.167,172.21.15.204,[v2:172.21.15.204:3301,v1:172.21.15.204:6790]" 2024-05-11T12:15:20.342 INFO:teuthology.orchestra.run.smithi204.stdout: ms die on skipped message = False 2024-05-11T12:15:20.342 INFO:teuthology.orchestra.run.smithi204.stdout: fsid = 5f32d344-7289-4c27-881b-1533419ab678 2024-05-11T12:15:20.342 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:15:20.342 INFO:teuthology.orchestra.run.smithi204.stdout:[osd] 2024-05-11T12:15:20.342 INFO:teuthology.orchestra.run.smithi204.stdout: osd journal size = 100 2024-05-11T12:15:20.342 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:15:20.342 INFO:teuthology.orchestra.run.smithi204.stdout: osd scrub load threshold = 5.0 2024-05-11T12:15:20.342 INFO:teuthology.orchestra.run.smithi204.stdout: osd scrub max interval = 600 2024-05-11T12:15:20.342 INFO:teuthology.orchestra.run.smithi204.stdout: osd mclock profile = high_recovery_ops 2024-05-11T12:15:20.342 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:15:20.342 INFO:teuthology.orchestra.run.smithi204.stdout: osd recover clone overlap = true 2024-05-11T12:15:20.342 INFO:teuthology.orchestra.run.smithi204.stdout: osd recovery max chunk = 1048576 2024-05-11T12:15:20.343 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:15:20.343 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug shutdown = true 2024-05-11T12:15:20.343 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug op order = true 2024-05-11T12:15:20.343 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug verify stray on activate = true 2024-05-11T12:15:20.343 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug trim objects = true 2024-05-11T12:15:20.343 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:15:20.343 INFO:teuthology.orchestra.run.smithi204.stdout: osd open classes on start = true 2024-05-11T12:15:20.343 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug pg log writeout = true 2024-05-11T12:15:20.343 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:15:20.343 INFO:teuthology.orchestra.run.smithi204.stdout: osd deep scrub update digest min age = 30 2024-05-11T12:15:20.343 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:15:20.343 INFO:teuthology.orchestra.run.smithi204.stdout: osd map max advance = 10 2024-05-11T12:15:20.343 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:15:20.343 INFO:teuthology.orchestra.run.smithi204.stdout: journal zero on create = true 2024-05-11T12:15:20.343 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:15:20.343 INFO:teuthology.orchestra.run.smithi204.stdout: filestore ondisk finisher threads = 3 2024-05-11T12:15:20.344 INFO:teuthology.orchestra.run.smithi204.stdout: filestore apply finisher threads = 3 2024-05-11T12:15:20.344 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:15:20.344 INFO:teuthology.orchestra.run.smithi204.stdout: bdev debug aio = true 2024-05-11T12:15:20.344 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug misdirected ops = true 2024-05-11T12:15:20.344 INFO:teuthology.orchestra.run.smithi204.stdout: bluestore block size = 96636764160 2024-05-11T12:15:20.344 INFO:teuthology.orchestra.run.smithi204.stdout: bluestore fsck on mount = True 2024-05-11T12:15:20.344 INFO:teuthology.orchestra.run.smithi204.stdout: debug bluefs = 20 2024-05-11T12:15:20.344 INFO:teuthology.orchestra.run.smithi204.stdout: debug bluestore = 20 2024-05-11T12:15:20.344 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-05-11T12:15:20.344 INFO:teuthology.orchestra.run.smithi204.stdout: debug osd = 20 2024-05-11T12:15:20.344 INFO:teuthology.orchestra.run.smithi204.stdout: debug rocksdb = 10 2024-05-11T12:15:20.344 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd backfillfull_ratio = 0.85 2024-05-11T12:15:20.344 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd full ratio = 0.9 2024-05-11T12:15:20.344 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd nearfull ratio = 0.8 2024-05-11T12:15:20.345 INFO:teuthology.orchestra.run.smithi204.stdout: osd failsafe full ratio = 0.95 2024-05-11T12:15:20.345 INFO:teuthology.orchestra.run.smithi204.stdout: osd objectstore = bluestore 2024-05-11T12:15:20.345 INFO:teuthology.orchestra.run.smithi204.stdout: osd op complaint time = 180 2024-05-11T12:15:20.345 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:15:20.345 INFO:teuthology.orchestra.run.smithi204.stdout:[mgr] 2024-05-11T12:15:20.345 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-05-11T12:15:20.345 INFO:teuthology.orchestra.run.smithi204.stdout: debug mgr = 20 2024-05-11T12:15:20.345 INFO:teuthology.orchestra.run.smithi204.stdout: debug mon = 20 2024-05-11T12:15:20.345 INFO:teuthology.orchestra.run.smithi204.stdout: debug auth = 20 2024-05-11T12:15:20.345 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min pgs per osd = 4 2024-05-11T12:15:20.345 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min bytes per osd = 10 2024-05-11T12:15:20.345 INFO:teuthology.orchestra.run.smithi204.stdout: mgr/telemetry/nag = false 2024-05-11T12:15:20.345 INFO:teuthology.orchestra.run.smithi204.stdout: debug client = 20 2024-05-11T12:15:20.345 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:15:20.345 INFO:teuthology.orchestra.run.smithi204.stdout:[mon] 2024-05-11T12:15:20.345 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-05-11T12:15:20.346 INFO:teuthology.orchestra.run.smithi204.stdout: debug mon = 20 2024-05-11T12:15:20.346 INFO:teuthology.orchestra.run.smithi204.stdout: debug paxos = 20 2024-05-11T12:15:20.346 INFO:teuthology.orchestra.run.smithi204.stdout: debug auth = 20 2024-05-11T12:15:20.346 INFO:teuthology.orchestra.run.smithi204.stdout: mon data avail warn = 5 2024-05-11T12:15:20.346 INFO:teuthology.orchestra.run.smithi204.stdout: mon mgr mkfs grace = 240 2024-05-11T12:15:20.346 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min pgs per osd = 4 2024-05-11T12:15:20.346 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd reporter subtree level = osd 2024-05-11T12:15:20.346 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd prime pg temp = true 2024-05-11T12:15:20.346 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min bytes per osd = 10 2024-05-11T12:15:20.346 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:15:20.346 INFO:teuthology.orchestra.run.smithi204.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-11T12:15:20.346 INFO:teuthology.orchestra.run.smithi204.stdout: auth mon ticket ttl = 660 # 11m 2024-05-11T12:15:20.346 INFO:teuthology.orchestra.run.smithi204.stdout: auth service ticket ttl = 240 # 4m 2024-05-11T12:15:20.346 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:15:20.347 INFO:teuthology.orchestra.run.smithi204.stdout: # don't complain about insecure global_id in the test suite 2024-05-11T12:15:20.347 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-11T12:15:20.347 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-11T12:15:20.347 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:15:20.347 INFO:teuthology.orchestra.run.smithi204.stdout: # 1m isn't quite enough 2024-05-11T12:15:20.347 INFO:teuthology.orchestra.run.smithi204.stdout: mon_down_mkfs_grace = 2m 2024-05-11T12:15:20.347 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:15:20.347 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_filestore_osds = false 2024-05-11T12:15:20.347 INFO:teuthology.orchestra.run.smithi204.stdout: mon down mkfs grace = 300 2024-05-11T12:15:20.347 INFO:teuthology.orchestra.run.smithi204.stdout: mon op complaint time = 120 2024-05-11T12:15:20.347 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T12:15:20.347 INFO:teuthology.orchestra.run.smithi204.stdout:[client] 2024-05-11T12:15:20.347 INFO:teuthology.orchestra.run.smithi204.stdout: rgw cache enabled = true 2024-05-11T12:15:20.347 INFO:teuthology.orchestra.run.smithi204.stdout: rgw enable ops log = true 2024-05-11T12:15:20.347 INFO:teuthology.orchestra.run.smithi204.stdout: rgw enable usage log = true 2024-05-11T12:15:20.347 INFO:teuthology.orchestra.run.smithi204.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-11T12:15:20.348 INFO:teuthology.orchestra.run.smithi204.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-11T12:15:20.348 INFO:teuthology.orchestra.run.smithi204.stdout: client mount timeout = 600 2024-05-11T12:15:20.348 INFO:teuthology.orchestra.run.smithi204.stdout: debug client = 20 2024-05-11T12:15:20.348 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-05-11T12:15:20.348 INFO:teuthology.orchestra.run.smithi204.stdout: pid file = "" 2024-05-11T12:15:20.348 INFO:teuthology.orchestra.run.smithi204.stdout: rados mon op timeout = 900 2024-05-11T12:15:20.348 INFO:teuthology.orchestra.run.smithi204.stdout: rados osd op timeout = 900 2024-05-11T12:15:20.348 INFO:teuthology.orchestra.run.smithi204.stdout:[mon.a] 2024-05-11T12:15:20.348 INFO:teuthology.orchestra.run.smithi204.stdout:[mon.b] 2024-05-11T12:15:20.348 INFO:teuthology.orchestra.run.smithi204.stdout:[mon.c] 2024-05-11T12:15:20.348 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.a] 2024-05-11T12:15:20.348 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.c] 2024-05-11T12:15:20.348 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.b] 2024-05-11T12:15:20.348 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.d] 2024-05-11T12:15:20.349 INFO:teuthology.orchestra.run.smithi204.stdout:[mds] 2024-05-11T12:15:20.349 INFO:teuthology.orchestra.run.smithi204.stdout: debug mds = 20 2024-05-11T12:15:20.349 INFO:teuthology.orchestra.run.smithi204.stdout: debug mds balancer = 20 2024-05-11T12:15:20.349 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-05-11T12:15:20.349 INFO:teuthology.orchestra.run.smithi204.stdout: mds debug frag = True 2024-05-11T12:15:20.349 INFO:teuthology.orchestra.run.smithi204.stdout: mds debug scatterstat = True 2024-05-11T12:15:20.349 INFO:teuthology.orchestra.run.smithi204.stdout: mds op complaint time = 180 2024-05-11T12:15:20.349 INFO:teuthology.orchestra.run.smithi204.stdout: mds verify scatter = True 2024-05-11T12:15:20.349 INFO:teuthology.orchestra.run.smithi204.stdout: osd op complaint time = 180 2024-05-11T12:15:20.349 INFO:teuthology.orchestra.run.smithi204.stdout: rados mon op timeout = 900 2024-05-11T12:15:20.349 INFO:teuthology.orchestra.run.smithi204.stdout: rados osd op timeout = 900 2024-05-11T12:15:20.366 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-11T12:15:20.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.574+0000 7fc404c00640 1 Processor -- start 2024-05-11T12:15:20.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.574+0000 7fc404c00640 1 -- start start 2024-05-11T12:15:20.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.575+0000 7fc404c00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc400147ff0 0x7fc4000d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:20.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.577+0000 7fc404c00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4000d1430 con 0x7fc4000d2e80 2024-05-11T12:15:20.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.577+0000 7fc404c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4000d1710 con 0x7fc400148cf0 2024-05-11T12:15:20.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.577+0000 7fc404c00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fc4000b64f0 con 0x7fc400147ff0 2024-05-11T12:15:20.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.577+0000 7fc3fde00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc400147ff0 0x7fc4000d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:20.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.577+0000 7fc3fe800640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fc4000d2e80 0x7fc40015f950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:33198/0 (socket says 172.21.15.167:33198) 2024-05-11T12:15:20.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.577+0000 7fc3fe800640 1 -- 172.21.15.167:0/306336022 learned_addr learned my addr 172.21.15.167:0/306336022 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:15:20.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.577+0000 7fc3fde00640 1 -- 172.21.15.167:0/306336022 >> v1:172.21.15.204:6789/0 conn(0x7fc400148cf0 legacy=0x7fc4000b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:20.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.577+0000 7fc3fde00640 1 -- 172.21.15.167:0/306336022 >> v1:172.21.15.167:6789/0 conn(0x7fc4000d2e80 legacy=0x7fc40015f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:20.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.577+0000 7fc3fca00640 1 -- 172.21.15.167:0/306336022 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1164730238 0 0) 0x7fc4000d1430 con 0x7fc4000d2e80 2024-05-11T12:15:20.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.577+0000 7fc3fde00640 1 -- 172.21.15.167:0/306336022 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4000d19f0 con 0x7fc400147ff0 2024-05-11T12:15:20.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.578+0000 7fc3fde00640 1 --2- 172.21.15.167:0/306336022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc400147ff0 0x7fc4000d24d0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fc3f00097a0 tx=0x7fc3f002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d376c58d676feccd server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:20.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.578+0000 7fc3fca00640 1 -- 172.21.15.167:0/306336022 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0037070 con 0x7fc400147ff0 2024-05-11T12:15:20.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.578+0000 7fc3fca00640 1 -- 172.21.15.167:0/306336022 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3f0032440 con 0x7fc400147ff0 2024-05-11T12:15:20.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.578+0000 7fc3fca00640 1 -- 172.21.15.167:0/306336022 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0032810 con 0x7fc400147ff0 2024-05-11T12:15:20.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.578+0000 7fc404c00640 1 -- 172.21.15.167:0/306336022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc400147ff0 msgr2=0x7fc4000d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:20.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.578+0000 7fc404c00640 1 --2- 172.21.15.167:0/306336022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc400147ff0 0x7fc4000d24d0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fc3f00097a0 tx=0x7fc3f002a180 comp rx=0 tx=0).stop 2024-05-11T12:15:20.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.579+0000 7fc404c00640 1 -- 172.21.15.167:0/306336022 shutdown_connections 2024-05-11T12:15:20.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.579+0000 7fc404c00640 1 --2- 172.21.15.167:0/306336022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc400147ff0 0x7fc4000d24d0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:20.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.579+0000 7fc404c00640 1 -- 172.21.15.167:0/306336022 >> 172.21.15.167:0/306336022 conn(0x7fc4000b8300 msgr2=0x7fc4000b86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:20.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.579+0000 7fc404c00640 1 -- 172.21.15.167:0/306336022 shutdown_connections 2024-05-11T12:15:20.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.579+0000 7fc404c00640 1 -- 172.21.15.167:0/306336022 wait complete. 2024-05-11T12:15:20.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.580+0000 7fc404c00640 1 Processor -- start 2024-05-11T12:15:20.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.581+0000 7fc404c00640 1 -- start start 2024-05-11T12:15:20.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.581+0000 7fc404c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4000d2e80 0x7fc40015ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:20.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.581+0000 7fc404c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc400147ff0 0x7fc4000cf000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:20.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.581+0000 7fc404c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc400148cf0 0x7fc40013ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:20.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.581+0000 7fc404c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc4001483d0 con 0x7fc4000d2e80 2024-05-11T12:15:20.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.581+0000 7fc404c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc4000d2040 con 0x7fc400148cf0 2024-05-11T12:15:20.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.581+0000 7fc404c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc4000d2a10 con 0x7fc400147ff0 2024-05-11T12:15:20.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.582+0000 7fc3fde00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4000d2e80 0x7fc40015ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:20.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.582+0000 7fc3fde00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4000d2e80 0x7fc40015ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:46880/0 (socket says 172.21.15.167:46880) 2024-05-11T12:15:20.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.582+0000 7fc3fde00640 1 -- 172.21.15.167:0/2821831171 learned_addr learned my addr 172.21.15.167:0/2821831171 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:20.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.582+0000 7fc3fde00640 1 -- 172.21.15.167:0/2821831171 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc400147ff0 msgr2=0x7fc4000cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:20.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.582+0000 7fc3fde00640 1 --2- 172.21.15.167:0/2821831171 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc400147ff0 0x7fc4000cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:20.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.582+0000 7fc3fe800640 1 --2- 172.21.15.167:0/2821831171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc400148cf0 0x7fc40013ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:20.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.582+0000 7fc3fde00640 1 -- 172.21.15.167:0/2821831171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc400148cf0 msgr2=0x7fc40013ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:20.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.582+0000 7fc3fde00640 1 --2- 172.21.15.167:0/2821831171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc400148cf0 0x7fc40013ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:20.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.582+0000 7fc3fde00640 1 -- 172.21.15.167:0/2821831171 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4000ccf10 con 0x7fc4000d2e80 2024-05-11T12:15:20.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.582+0000 7fc3fde00640 1 --2- 172.21.15.167:0/2821831171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4000d2e80 0x7fc40015ea80 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fc3f00098c0 tx=0x7fc3f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:20.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.583+0000 7fc3f6000640 1 -- 172.21.15.167:0/2821831171 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0041070 con 0x7fc4000d2e80 2024-05-11T12:15:20.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.583+0000 7fc3f6000640 1 -- 172.21.15.167:0/2821831171 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3f003c870 con 0x7fc4000d2e80 2024-05-11T12:15:20.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.583+0000 7fc3f6000640 1 -- 172.21.15.167:0/2821831171 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0037040 con 0x7fc4000d2e80 2024-05-11T12:15:20.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.583+0000 7fc404c00640 1 -- 172.21.15.167:0/2821831171 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4000cd1f0 con 0x7fc4000d2e80 2024-05-11T12:15:20.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.583+0000 7fc404c00640 1 -- 172.21.15.167:0/2821831171 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4000cd7b0 con 0x7fc4000d2e80 2024-05-11T12:15:20.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.584+0000 7fc404c00640 1 -- 172.21.15.167:0/2821831171 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc400069520 con 0x7fc4000d2e80 2024-05-11T12:15:20.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.585+0000 7fc3f6000640 1 -- 172.21.15.167:0/2821831171 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc3f0044070 con 0x7fc4000d2e80 2024-05-11T12:15:20.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.586+0000 7fc3f6000640 1 --2- 172.21.15.167:0/2821831171 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc3dc0762d0 0x7fc3dc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:20.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.586+0000 7fc3f6000640 1 -- 172.21.15.167:0/2821831171 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5507+0+0 (secure 0 0 0) 0x7fc3f00b8070 con 0x7fc4000d2e80 2024-05-11T12:15:20.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.586+0000 7fc3fd400640 1 --2- 172.21.15.167:0/2821831171 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc3dc0762d0 0x7fc3dc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:20.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.587+0000 7fc3fd400640 1 --2- 172.21.15.167:0/2821831171 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc3dc0762d0 0x7fc3dc078790 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8006fd0 tx=0x7fc3e8006bc0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:20.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.591+0000 7fc3f6000640 1 -- 172.21.15.167:0/2821831171 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc3f0081a80 con 0x7fc4000d2e80 2024-05-11T12:15:20.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.813+0000 7fc404c00640 1 -- 172.21.15.167:0/2821831171 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fc4001420a0 con 0x7fc4000d2e80 2024-05-11T12:15:20.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.820+0000 7fc3f6000640 1 -- 172.21.15.167:0/2821831171 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) ==== 104+0+0 (secure 0 0 0) 0x7fc3f0081380 con 0x7fc4000d2e80 2024-05-11T12:15:20.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.822+0000 7fc404c00640 1 -- 172.21.15.167:0/2821831171 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc3dc0762d0 msgr2=0x7fc3dc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:20.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.822+0000 7fc404c00640 1 --2- 172.21.15.167:0/2821831171 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc3dc0762d0 0x7fc3dc078790 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8006fd0 tx=0x7fc3e8006bc0 comp rx=0 tx=0).stop 2024-05-11T12:15:20.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.822+0000 7fc404c00640 1 -- 172.21.15.167:0/2821831171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4000d2e80 msgr2=0x7fc40015ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:20.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.822+0000 7fc404c00640 1 --2- 172.21.15.167:0/2821831171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4000d2e80 0x7fc40015ea80 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fc3f00098c0 tx=0x7fc3f00046c0 comp rx=0 tx=0).stop 2024-05-11T12:15:20.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.823+0000 7fc404c00640 1 -- 172.21.15.167:0/2821831171 shutdown_connections 2024-05-11T12:15:20.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.823+0000 7fc404c00640 1 --2- 172.21.15.167:0/2821831171 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fc3dc0762d0 0x7fc3dc078790 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:20.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.823+0000 7fc404c00640 1 --2- 172.21.15.167:0/2821831171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc400148cf0 0x7fc40013ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:20.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.823+0000 7fc404c00640 1 --2- 172.21.15.167:0/2821831171 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc400147ff0 0x7fc4000cf000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:20.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.823+0000 7fc404c00640 1 --2- 172.21.15.167:0/2821831171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4000d2e80 0x7fc40015ea80 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:20.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.823+0000 7fc404c00640 1 -- 172.21.15.167:0/2821831171 >> 172.21.15.167:0/2821831171 conn(0x7fc4000b8300 msgr2=0x7fc4000d01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:20.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.823+0000 7fc404c00640 1 -- 172.21.15.167:0/2821831171 shutdown_connections 2024-05-11T12:15:20.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:20.823+0000 7fc404c00640 1 -- 172.21.15.167:0/2821831171 wait complete. 2024-05-11T12:15:20.843 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_version_splitting' 2024-05-11T12:15:21.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.031+0000 7f318d200640 1 Processor -- start 2024-05-11T12:15:21.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.031+0000 7f318d200640 1 -- start start 2024-05-11T12:15:21.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.033+0000 7f318d200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31880d42c0 0x7f31880d0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:21.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.033+0000 7f318d200640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31880cf990 con 0x7f318806cc40 2024-05-11T12:15:21.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.033+0000 7f318d200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31880cfc70 con 0x7f318815f460 2024-05-11T12:15:21.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.033+0000 7f318d200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f31880cff50 con 0x7f31880d42c0 2024-05-11T12:15:21.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.033+0000 7f3186a00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f318806cc40 0x7f31880d3bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:33204/0 (socket says 172.21.15.167:33204) 2024-05-11T12:15:21.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.033+0000 7f3186a00640 1 -- 172.21.15.167:0/2625854259 learned_addr learned my addr 172.21.15.167:0/2625854259 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:15:21.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.033+0000 7f3187e00640 1 --2- 172.21.15.167:0/2625854259 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31880d42c0 0x7f31880d0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:21.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.033+0000 7f3186000640 1 -- 172.21.15.167:0/2625854259 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4262314280 0 0) 0x7f31880cf990 con 0x7f318806cc40 2024-05-11T12:15:21.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.033+0000 7f3186000640 1 -- 172.21.15.167:0/2625854259 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3168003520 con 0x7f318806cc40 2024-05-11T12:15:21.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.033+0000 7f3186000640 1 -- 172.21.15.167:0/2625854259 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2834139356 0 0) 0x7f3168003520 con 0x7f318806cc40 2024-05-11T12:15:21.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.033+0000 7f3186000640 1 -- 172.21.15.167:0/2625854259 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31880d42c0 msgr2=0x7f31880d0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:21.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.033+0000 7f3186000640 1 --2- 172.21.15.167:0/2625854259 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31880d42c0 0x7f31880d0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:21.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.033+0000 7f3186000640 1 -- 172.21.15.167:0/2625854259 >> v1:172.21.15.204:6789/0 conn(0x7f318815f460 legacy=0x7f31880b83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:21.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.033+0000 7f3186000640 1 -- 172.21.15.167:0/2625854259 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31880d0230 con 0x7f318806cc40 2024-05-11T12:15:21.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.033+0000 7f3186000640 1 -- 172.21.15.167:0/2625854259 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f3170001a70 con 0x7f318806cc40 2024-05-11T12:15:21.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.034+0000 7f3186000640 1 -- 172.21.15.167:0/2625854259 >> v1:172.21.15.167:6789/0 conn(0x7f318806cc40 legacy=0x7f31880d3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:21.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.034+0000 7f3186000640 1 --2- 172.21.15.167:0/2625854259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3168003a30 0x7f3168003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:21.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.034+0000 7f3186000640 1 --2- 172.21.15.167:0/2625854259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3168004800 0x7f3168004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:21.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.034+0000 7f3186000640 1 --2- 172.21.15.167:0/2625854259 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31880d42c0 0x7f3168005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:21.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.034+0000 7f3186000640 1 -- 172.21.15.167:0/2625854259 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f31880cff50 con 0x7f3168004800 2024-05-11T12:15:21.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.034+0000 7f3186000640 1 -- 172.21.15.167:0/2625854259 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f316800bb20 con 0x7f3168003a30 2024-05-11T12:15:21.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.034+0000 7f3186000640 1 -- 172.21.15.167:0/2625854259 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f316800bd20 con 0x7f31880d42c0 2024-05-11T12:15:21.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.034+0000 7f3186a00640 1 --2- 172.21.15.167:0/2625854259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3168004800 0x7f3168004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:21.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.034+0000 7f3187400640 1 --2- 172.21.15.167:0/2625854259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3168003a30 0x7f3168003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:21.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.034+0000 7f3187e00640 1 --2- 172.21.15.167:0/2625854259 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31880d42c0 0x7f3168005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:21.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.035+0000 7f3186a00640 1 -- 172.21.15.167:0/2625854259 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31880d42c0 msgr2=0x7f3168005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:21.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.035+0000 7f3186a00640 1 --2- 172.21.15.167:0/2625854259 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31880d42c0 0x7f3168005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:21.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.035+0000 7f3186a00640 1 -- 172.21.15.167:0/2625854259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3168003a30 msgr2=0x7f3168003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:21.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.035+0000 7f3186a00640 1 --2- 172.21.15.167:0/2625854259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3168003a30 0x7f3168003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:21.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.035+0000 7f3186a00640 1 -- 172.21.15.167:0/2625854259 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3168003520 con 0x7f3168004800 2024-05-11T12:15:21.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.035+0000 7f3187400640 1 --2- 172.21.15.167:0/2625854259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3168003a30 0x7f3168003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:15:21.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.036+0000 7f3186a00640 1 --2- 172.21.15.167:0/2625854259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3168004800 0x7f3168004c80 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f317000a8b0 tx=0x7f3168005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bd7d8077ff0a6e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:21.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.036+0000 7f3186000640 1 -- 172.21.15.167:0/2625854259 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317000a500 con 0x7f3168004800 2024-05-11T12:15:21.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.036+0000 7f3186000640 1 -- 172.21.15.167:0/2625854259 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f317002f070 con 0x7f3168004800 2024-05-11T12:15:21.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.036+0000 7f3186000640 1 -- 172.21.15.167:0/2625854259 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317002bc50 con 0x7f3168004800 2024-05-11T12:15:21.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.037+0000 7f318d200640 1 -- 172.21.15.167:0/2625854259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3168004800 msgr2=0x7f3168004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:21.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.037+0000 7f318d200640 1 --2- 172.21.15.167:0/2625854259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3168004800 0x7f3168004c80 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f317000a8b0 tx=0x7f3168005830 comp rx=0 tx=0).stop 2024-05-11T12:15:21.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.037+0000 7f3187e00640 1 -- 172.21.15.167:0/2625854259 reap_dead start 2024-05-11T12:15:21.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.037+0000 7f318d200640 1 -- 172.21.15.167:0/2625854259 shutdown_connections 2024-05-11T12:15:21.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.037+0000 7f318d200640 1 -- 172.21.15.167:0/2625854259 >> 172.21.15.167:0/2625854259 conn(0x7f31880d1a50 msgr2=0x7f31880d1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:21.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.037+0000 7f318d200640 1 -- 172.21.15.167:0/2625854259 shutdown_connections 2024-05-11T12:15:21.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.037+0000 7f318d200640 1 -- 172.21.15.167:0/2625854259 wait complete. 2024-05-11T12:15:21.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.040+0000 7f318d200640 1 Processor -- start 2024-05-11T12:15:21.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.040+0000 7f318d200640 1 -- start start 2024-05-11T12:15:21.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.041+0000 7f318d200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3168004800 0x7f31880cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:21.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.041+0000 7f318d200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f318815d150 0x7f318815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:21.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.041+0000 7f318d200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31881618f0 0x7f3188161d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:21.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.041+0000 7f318d200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f31880cf620 con 0x7f3168004800 2024-05-11T12:15:21.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.041+0000 7f318d200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f31880b8ab0 con 0x7f318815d150 2024-05-11T12:15:21.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.041+0000 7f318d200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f318806db20 con 0x7f31881618f0 2024-05-11T12:15:21.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.041+0000 7f3187400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3168004800 0x7f31880cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:21.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.041+0000 7f3187400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3168004800 0x7f31880cb4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:46896/0 (socket says 172.21.15.167:46896) 2024-05-11T12:15:21.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.041+0000 7f3187400640 1 -- 172.21.15.167:0/2929280764 learned_addr learned my addr 172.21.15.167:0/2929280764 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:21.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.041+0000 7f3187400640 1 -- 172.21.15.167:0/2929280764 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31881618f0 msgr2=0x7f3188161d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:21.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.041+0000 7f3187e00640 1 --2- 172.21.15.167:0/2929280764 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31881618f0 0x7f3188161d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:21.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.041+0000 7f3186a00640 1 --2- 172.21.15.167:0/2929280764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f318815d150 0x7f318815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:21.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.041+0000 7f3187400640 1 --2- 172.21.15.167:0/2929280764 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31881618f0 0x7f3188161d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:21.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.041+0000 7f3187400640 1 -- 172.21.15.167:0/2929280764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f318815d150 msgr2=0x7f318815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:21.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.041+0000 7f3187400640 1 --2- 172.21.15.167:0/2929280764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f318815d150 0x7f318815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:21.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.041+0000 7f3187400640 1 -- 172.21.15.167:0/2929280764 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3188162680 con 0x7f3168004800 2024-05-11T12:15:21.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.041+0000 7f3187400640 1 --2- 172.21.15.167:0/2929280764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3168004800 0x7f31880cb4c0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f317800e550 tx=0x7f317800ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:21.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.041+0000 7f317f400640 1 -- 172.21.15.167:0/2929280764 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31780052e0 con 0x7f3168004800 2024-05-11T12:15:21.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.041+0000 7f317f400640 1 -- 172.21.15.167:0/2929280764 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3178005480 con 0x7f3168004800 2024-05-11T12:15:21.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.041+0000 7f317f400640 1 -- 172.21.15.167:0/2929280764 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3178007020 con 0x7f3168004800 2024-05-11T12:15:21.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.042+0000 7f318d200640 1 -- 172.21.15.167:0/2929280764 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3188162960 con 0x7f3168004800 2024-05-11T12:15:21.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.042+0000 7f318d200640 1 -- 172.21.15.167:0/2929280764 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31881426d0 con 0x7f3168004800 2024-05-11T12:15:21.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.043+0000 7f318d200640 1 -- 172.21.15.167:0/2929280764 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31880694b0 con 0x7f3168004800 2024-05-11T12:15:21.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.044+0000 7f317f400640 1 -- 172.21.15.167:0/2929280764 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f317801a020 con 0x7f3168004800 2024-05-11T12:15:21.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.047+0000 7f317f400640 1 --2- 172.21.15.167:0/2929280764 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3164076300 0x7f31640787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:21.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.047+0000 7f317f400640 1 -- 172.21.15.167:0/2929280764 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5507+0+0 (secure 0 0 0) 0x7f3178096c60 con 0x7f3168004800 2024-05-11T12:15:21.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.047+0000 7f3186a00640 1 --2- 172.21.15.167:0/2929280764 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3164076300 0x7f31640787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:21.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.047+0000 7f3186a00640 1 --2- 172.21.15.167:0/2929280764 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3164076300 0x7f31640787c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3170033000 tx=0x7f3170038040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:21.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.051+0000 7f317f400640 1 -- 172.21.15.167:0/2929280764 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3178060560 con 0x7f3168004800 2024-05-11T12:15:21.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:21.323+0000 7f318d200640 1 -- 172.21.15.167:0/2929280764 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_version_splitting"]} v 0) -- 0x7f31880cf390 con 0x7f3168004800 2024-05-11T12:15:22.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:22.179+0000 7f317f400640 1 -- 172.21.15.167:0/2929280764 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_version_splitting"]}]=0 v271) ==== 145+0+0 (secure 0 0 0) 0x7f317805fe60 con 0x7f3168004800 2024-05-11T12:15:22.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:22.183+0000 7f318d200640 1 -- 172.21.15.167:0/2929280764 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3164076300 msgr2=0x7f31640787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:22.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:22.183+0000 7f318d200640 1 --2- 172.21.15.167:0/2929280764 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3164076300 0x7f31640787c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3170033000 tx=0x7f3170038040 comp rx=0 tx=0).stop 2024-05-11T12:15:22.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:22.183+0000 7f318d200640 1 -- 172.21.15.167:0/2929280764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3168004800 msgr2=0x7f31880cb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:22.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:22.183+0000 7f318d200640 1 --2- 172.21.15.167:0/2929280764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3168004800 0x7f31880cb4c0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f317800e550 tx=0x7f317800ea10 comp rx=0 tx=0).stop 2024-05-11T12:15:22.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:22.183+0000 7f318d200640 1 -- 172.21.15.167:0/2929280764 shutdown_connections 2024-05-11T12:15:22.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:22.183+0000 7f318d200640 1 --2- 172.21.15.167:0/2929280764 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31881618f0 0x7f3188161d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:22.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:22.183+0000 7f318d200640 1 --2- 172.21.15.167:0/2929280764 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3164076300 0x7f31640787c0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:22.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:22.183+0000 7f318d200640 1 --2- 172.21.15.167:0/2929280764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f318815d150 0x7f318815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:22.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:22.183+0000 7f318d200640 1 --2- 172.21.15.167:0/2929280764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3168004800 0x7f31880cb4c0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:22.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:22.183+0000 7f318d200640 1 -- 172.21.15.167:0/2929280764 >> 172.21.15.167:0/2929280764 conn(0x7f31880d1a50 msgr2=0x7f31880b63b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:22.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:22.183+0000 7f318d200640 1 -- 172.21.15.167:0/2929280764 shutdown_connections 2024-05-11T12:15:22.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:22.183+0000 7f318d200640 1 -- 172.21.15.167:0/2929280764 wait complete. 2024-05-11T12:15:22.204 INFO:tasks.cephfs_test_runner:That when many sessions are updated, they are correctly ... ok 2024-05-11T12:15:22.204 INFO:tasks.cephfs_test_runner: 2024-05-11T12:15:22.204 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-05-11T12:15:22.205 INFO:tasks.cephfs_test_runner:Ran 6 tests in 227.411s 2024-05-11T12:15:22.205 INFO:tasks.cephfs_test_runner: 2024-05-11T12:15:22.205 INFO:tasks.cephfs_test_runner:OK (skipped=2) 2024-05-11T12:15:22.205 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-05-11T12:15:22.214 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-11T12:15:22.256 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-11T12:15:22.257 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:15:22.257 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:15:22.314 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-11T12:15:22.314 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T12:15:22.314 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:15:22.366 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-11T12:15:22.367 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:15:22.367 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:15:22.424 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052713,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-11T12:15:22.424 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T12:15:22.424 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:15:22.473 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052713,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-11T12:15:22.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:15:22.474 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2024-05-11T12:15:22.530 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-11T12:15:22.530 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-11T12:15:22.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:15:22.531 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-11T12:15:22.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:15:22.615 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-11T12:15:22.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:15:22.688 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link delete ceph-brx 2024-05-11T12:15:22.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:15:22.817 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2024-05-11T12:15:22.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:15:22.860 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link delete ceph-brx 2024-05-11T12:15:22.978 INFO:teuthology.orchestra.run.smithi167.stderr:Cannot find device "ceph-brx" 2024-05-11T12:15:22.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T12:15:22.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:15:22.981 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list 2024-05-11T12:15:23.038 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-11T12:15:23.038 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-11T12:15:23.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:15:23.039 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-11T12:15:23.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:15:23.124 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-11T12:15:23.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:15:23.199 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link delete ceph-brx 2024-05-11T12:15:23.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:15:23.303 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list 2024-05-11T12:15:23.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:15:23.344 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link delete ceph-brx 2024-05-11T12:15:23.458 INFO:teuthology.orchestra.run.smithi204.stderr:Cannot find device "ceph-brx" 2024-05-11T12:15:23.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T12:15:23.462 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-05-11T12:15:23.475 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-11T12:15:23.475 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T12:15:23.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.687+0000 7f78ad800640 1 Processor -- start 2024-05-11T12:15:23.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.687+0000 7f78ad800640 1 -- start start 2024-05-11T12:15:23.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.689+0000 7f78ad800640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f78a8145b00 0x7f78a806f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:23.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.689+0000 7f78ad800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78a8146770 0x7f78a80ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:23.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.689+0000 7f78ad800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78a806fbc0 con 0x7f78a80cef10 2024-05-11T12:15:23.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.689+0000 7f78ad800640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f78a80b4520 con 0x7f78a8145b00 2024-05-11T12:15:23.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.689+0000 7f78ad800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f78a80b4800 con 0x7f78a8146770 2024-05-11T12:15:23.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.689+0000 7f78a7e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f78a8145b00 0x7f78a806f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:23.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.689+0000 7f78a7e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f78a8145b00 0x7f78a806f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:46904/0 (socket says 172.21.15.167:46904) 2024-05-11T12:15:23.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.689+0000 7f78a7e00640 1 -- 172.21.15.167:0/1284752193 learned_addr learned my addr 172.21.15.167:0/1284752193 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:23.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.690+0000 7f78a7400640 1 --2- 172.21.15.167:0/1284752193 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78a8146770 0x7f78a80ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:23.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.690+0000 7f78a7e00640 1 -- 172.21.15.167:0/1284752193 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78a8146770 msgr2=0x7f78a80ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:23.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.690+0000 7f78a7e00640 1 --2- 172.21.15.167:0/1284752193 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78a8146770 0x7f78a80ce9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:23.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.690+0000 7f78a7e00640 1 -- 172.21.15.167:0/1284752193 >> v1:172.21.15.204:6789/0 conn(0x7f78a80cef10 legacy=0x7f78a806c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:23.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.690+0000 7f78a7e00640 1 -- 172.21.15.167:0/1284752193 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78a80b4ae0 con 0x7f78a8145b00 2024-05-11T12:15:23.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.691+0000 7f78a7e00640 1 --2- 172.21.15.167:0/1284752193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78a8145b00 0x7f78a806f2c0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f7898009870 tx=0x7f789802a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52420cd093a7c0be server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:23.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.691+0000 7f78a6a00640 1 -- 172.21.15.167:0/1284752193 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7898037070 con 0x7f78a8145b00 2024-05-11T12:15:23.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.691+0000 7f78a6a00640 1 -- 172.21.15.167:0/1284752193 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7898032440 con 0x7f78a8145b00 2024-05-11T12:15:23.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.691+0000 7f78a6a00640 1 -- 172.21.15.167:0/1284752193 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7898032810 con 0x7f78a8145b00 2024-05-11T12:15:23.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.692+0000 7f78ad800640 1 -- 172.21.15.167:0/1284752193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78a8145b00 msgr2=0x7f78a806f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:23.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.692+0000 7f78ad800640 1 --2- 172.21.15.167:0/1284752193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78a8145b00 0x7f78a806f2c0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f7898009870 tx=0x7f789802a130 comp rx=0 tx=0).stop 2024-05-11T12:15:23.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.692+0000 7f78ad800640 1 -- 172.21.15.167:0/1284752193 shutdown_connections 2024-05-11T12:15:23.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.692+0000 7f78ad800640 1 --2- 172.21.15.167:0/1284752193 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78a8146770 0x7f78a80ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:23.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.692+0000 7f78ad800640 1 --2- 172.21.15.167:0/1284752193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78a8145b00 0x7f78a806f2c0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:23.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.692+0000 7f78ad800640 1 -- 172.21.15.167:0/1284752193 >> 172.21.15.167:0/1284752193 conn(0x7f78a80b3460 msgr2=0x7f78a80d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:23.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.692+0000 7f78ad800640 1 -- 172.21.15.167:0/1284752193 shutdown_connections 2024-05-11T12:15:23.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.692+0000 7f78ad800640 1 -- 172.21.15.167:0/1284752193 wait complete. 2024-05-11T12:15:23.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.694+0000 7f78ad800640 1 Processor -- start 2024-05-11T12:15:23.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.694+0000 7f78ad800640 1 -- start start 2024-05-11T12:15:23.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.695+0000 7f78ad800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78a80cef10 0x7f78a80ae820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:23.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.695+0000 7f78ad800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78a8145b00 0x7f78a8071f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:23.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.695+0000 7f78ad800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78a8146770 0x7f78a8072470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:23.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.695+0000 7f78ad800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f78a806f980 con 0x7f78a8146770 2024-05-11T12:15:23.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.695+0000 7f78ad800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f78a806f800 con 0x7f78a8145b00 2024-05-11T12:15:23.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.695+0000 7f78ad800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f78a806cd70 con 0x7f78a80cef10 2024-05-11T12:15:23.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.696+0000 7f78ace00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78a8146770 0x7f78a8072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:23.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.696+0000 7f78ace00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78a8146770 0x7f78a8072470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:46918/0 (socket says 172.21.15.167:46918) 2024-05-11T12:15:23.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.696+0000 7f78ace00640 1 -- 172.21.15.167:0/475302277 learned_addr learned my addr 172.21.15.167:0/475302277 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:23.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.696+0000 7f78ace00640 1 -- 172.21.15.167:0/475302277 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78a80cef10 msgr2=0x7f78a80ae820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:23.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.696+0000 7f78ace00640 1 --2- 172.21.15.167:0/475302277 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78a80cef10 0x7f78a80ae820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:23.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.696+0000 7f78ace00640 1 -- 172.21.15.167:0/475302277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78a8145b00 msgr2=0x7f78a8071f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:23.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.696+0000 7f78a7400640 1 --2- 172.21.15.167:0/475302277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78a8145b00 0x7f78a8071f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:23.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.696+0000 7f78ace00640 1 --2- 172.21.15.167:0/475302277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78a8145b00 0x7f78a8071f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:23.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.696+0000 7f78ace00640 1 -- 172.21.15.167:0/475302277 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78a80af270 con 0x7f78a8146770 2024-05-11T12:15:23.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.696+0000 7f78ace00640 1 --2- 172.21.15.167:0/475302277 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78a8146770 0x7f78a8072470 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f78a000c780 tx=0x7f78a000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:23.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.697+0000 7f789fe00640 1 -- 172.21.15.167:0/475302277 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a0004b60 con 0x7f78a8146770 2024-05-11T12:15:23.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.697+0000 7f789fe00640 1 -- 172.21.15.167:0/475302277 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78a0004d00 con 0x7f78a8146770 2024-05-11T12:15:23.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.697+0000 7f78ad800640 1 -- 172.21.15.167:0/475302277 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78a80af550 con 0x7f78a8146770 2024-05-11T12:15:23.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.697+0000 7f789fe00640 1 -- 172.21.15.167:0/475302277 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a000aa90 con 0x7f78a8146770 2024-05-11T12:15:23.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.697+0000 7f78ad800640 1 -- 172.21.15.167:0/475302277 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78a80afb10 con 0x7f78a8146770 2024-05-11T12:15:23.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.698+0000 7f78ad800640 1 -- 172.21.15.167:0/475302277 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78a80694b0 con 0x7f78a8146770 2024-05-11T12:15:23.698 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.699+0000 7f789fe00640 1 -- 172.21.15.167:0/475302277 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f78a000ac30 con 0x7f78a8146770 2024-05-11T12:15:23.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.700+0000 7f789fe00640 1 --2- 172.21.15.167:0/475302277 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f78880762d0 0x7f7888078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:23.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.700+0000 7f789fe00640 1 -- 172.21.15.167:0/475302277 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5507+0+0 (secure 0 0 0) 0x7f78a0097860 con 0x7f78a8146770 2024-05-11T12:15:23.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.700+0000 7f78a7e00640 1 --2- 172.21.15.167:0/475302277 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f78880762d0 0x7f7888078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:23.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.701+0000 7f78a7e00640 1 --2- 172.21.15.167:0/475302277 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f78880762d0 0x7f7888078790 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f7898034000 tx=0x7f789802c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:23.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.704+0000 7f789fe00640 1 -- 172.21.15.167:0/475302277 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f78a0020070 con 0x7f78a8146770 2024-05-11T12:15:23.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.913+0000 7f78ad800640 1 -- 172.21.15.167:0/475302277 --> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f78a8160ea0 con 0x7f78880762d0 2024-05-11T12:15:23.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.914+0000 7f789fe00640 1 -- 172.21.15.167:0/475302277 <== mgr.4106 v2:172.21.15.167:6832/578352559 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13497 (secure 0 0 0) 0x7f78a8160ea0 con 0x7f78880762d0 2024-05-11T12:15:23.914 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:15:23.914 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2024-05-11T12:15:23.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.918+0000 7f78ad800640 1 -- 172.21.15.167:0/475302277 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f78880762d0 msgr2=0x7f7888078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:23.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.918+0000 7f78ad800640 1 --2- 172.21.15.167:0/475302277 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f78880762d0 0x7f7888078790 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f7898034000 tx=0x7f789802c000 comp rx=0 tx=0).stop 2024-05-11T12:15:23.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.918+0000 7f78ad800640 1 -- 172.21.15.167:0/475302277 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78a8146770 msgr2=0x7f78a8072470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:23.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.918+0000 7f78ad800640 1 --2- 172.21.15.167:0/475302277 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78a8146770 0x7f78a8072470 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f78a000c780 tx=0x7f78a000cc40 comp rx=0 tx=0).stop 2024-05-11T12:15:23.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.918+0000 7f78ad800640 1 -- 172.21.15.167:0/475302277 shutdown_connections 2024-05-11T12:15:23.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.918+0000 7f78ad800640 1 --2- 172.21.15.167:0/475302277 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78a8146770 0x7f78a8072470 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:23.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.918+0000 7f78ad800640 1 --2- 172.21.15.167:0/475302277 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f78880762d0 0x7f7888078790 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:23.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.918+0000 7f78ad800640 1 --2- 172.21.15.167:0/475302277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78a8145b00 0x7f78a8071f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:23.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.918+0000 7f78ad800640 1 --2- 172.21.15.167:0/475302277 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78a80cef10 0x7f78a80ae820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:23.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.918+0000 7f78ad800640 1 -- 172.21.15.167:0/475302277 >> 172.21.15.167:0/475302277 conn(0x7f78a80b3460 msgr2=0x7f78a80cf310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:23.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.918+0000 7f78ad800640 1 -- 172.21.15.167:0/475302277 shutdown_connections 2024-05-11T12:15:23.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:23.918+0000 7f78ad800640 1 -- 172.21.15.167:0/475302277 wait complete. 2024-05-11T12:15:23.935 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":243,"stamp":"2024-05-11T12:15:23.323999+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":124,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":260648,"kb_used_data":11192,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714072,"statfs":{"total":773094113280,"available":772827209728,"internally_reserved":0,"allocated":11460608,"data_stored":5871287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24635,"num_objects":-42,"num_object_clones":0,"num_object_copies":-84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-42,"num_whiteouts":0,"num_read":-94,"num_read_kb":-89,"num_write":-161,"num_write_kb":-84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.175696"},"pg_stats":[{"pgid":"1.0","version":"19'72","reported_seq":260,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-11T12:15:20.740225+0000","last_change":"2024-05-11T12:10:35.066673+0000","last_active":"2024-05-11T12:15:20.740225+0000","last_peered":"2024-05-11T12:15:20.740225+0000","last_clean":"2024-05-11T12:15:20.740225+0000","last_became_active":"2024-05-11T12:10:35.065942+0000","last_became_peered":"2024-05-11T12:10:35.065942+0000","last_unstale":"2024-05-11T12:15:20.740225+0000","last_undegraded":"2024-05-11T12:15:20.740225+0000","last_fullsized":"2024-05-11T12:15:20.740225+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_clean_scrub_stamp":"2024-05-11T12:10:33.601136+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:12:02.133182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476797,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33016,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338824,"statfs":{"total":96636764160,"available":96602955776,"internally_reserved":0,"allocated":1880064,"data_stored":1177453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476798,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32436,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1286144,"data_stored":587085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476797,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32440,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1286144,"data_stored":587085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476799,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32440,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1286144,"data_stored":587085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509502,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32440,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1286144,"data_stored":587085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509503,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32440,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1286144,"data_stored":587085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509502,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33016,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338824,"statfs":{"total":96636764160,"available":96602955776,"internally_reserved":0,"allocated":1880064,"data_stored":1177453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542206,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32420,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339420,"statfs":{"total":96636764160,"available":96603566080,"internally_reserved":0,"allocated":1269760,"data_stored":580956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T12:15:23.936 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T12:15:24.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.125+0000 7efdb4a00640 1 Processor -- start 2024-05-11T12:15:24.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.125+0000 7efdb4a00640 1 -- start start 2024-05-11T12:15:24.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.127+0000 7efdb4a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdb00718e0 0x7efdb0160e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:24.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.127+0000 7efdb4a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7efdb01573a0 0x7efdb0157810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:24.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.127+0000 7efdb4a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdb0161390 con 0x7efdb0071cc0 2024-05-11T12:15:24.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.127+0000 7efdb4a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7efdb00d1d20 con 0x7efdb01573a0 2024-05-11T12:15:24.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.127+0000 7efdb4a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7efdb00d0ec0 con 0x7efdb00718e0 2024-05-11T12:15:24.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.127+0000 7efdade00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdb00718e0 0x7efdb0160e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:24.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.127+0000 7efdade00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdb00718e0 0x7efdb0160e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:42178/0 (socket says 172.21.15.167:42178) 2024-05-11T12:15:24.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.128+0000 7efdade00640 1 -- 172.21.15.167:0/693693687 learned_addr learned my addr 172.21.15.167:0/693693687 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:24.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.128+0000 7efdae800640 1 --2- 172.21.15.167:0/693693687 >> v2:172.21.15.204:3300/0 conn(0x7efdb01573a0 0x7efdb0157810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:24.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.128+0000 7efdade00640 1 -- 172.21.15.167:0/693693687 >> v2:172.21.15.204:3300/0 conn(0x7efdb01573a0 msgr2=0x7efdb0157810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:24.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.128+0000 7efdade00640 1 --2- 172.21.15.167:0/693693687 >> v2:172.21.15.204:3300/0 conn(0x7efdb01573a0 0x7efdb0157810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:24.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.128+0000 7efdade00640 1 -- 172.21.15.167:0/693693687 >> v1:172.21.15.204:6789/0 conn(0x7efdb0071cc0 legacy=0x7efdb00cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:24.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.128+0000 7efdade00640 1 -- 172.21.15.167:0/693693687 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdb0157d50 con 0x7efdb00718e0 2024-05-11T12:15:24.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.130+0000 7efdade00640 1 --2- 172.21.15.167:0/693693687 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdb00718e0 0x7efdb0160e50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7efda0009870 tx=0x7efda002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5373e983700126b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:24.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.130+0000 7efdaca00640 1 -- 172.21.15.167:0/693693687 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda0037070 con 0x7efdb00718e0 2024-05-11T12:15:24.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.130+0000 7efdaca00640 1 -- 172.21.15.167:0/693693687 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efda0032440 con 0x7efdb00718e0 2024-05-11T12:15:24.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.130+0000 7efdaca00640 1 -- 172.21.15.167:0/693693687 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda0032810 con 0x7efdb00718e0 2024-05-11T12:15:24.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.131+0000 7efdb4a00640 1 -- 172.21.15.167:0/693693687 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdb00718e0 msgr2=0x7efdb0160e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:24.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.131+0000 7efdb4a00640 1 --2- 172.21.15.167:0/693693687 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdb00718e0 0x7efdb0160e50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7efda0009870 tx=0x7efda002a270 comp rx=0 tx=0).stop 2024-05-11T12:15:24.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.131+0000 7efdb4a00640 1 -- 172.21.15.167:0/693693687 shutdown_connections 2024-05-11T12:15:24.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.131+0000 7efdb4a00640 1 --2- 172.21.15.167:0/693693687 >> v2:172.21.15.204:3300/0 conn(0x7efdb01573a0 0x7efdb0157810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:24.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.131+0000 7efdb4a00640 1 --2- 172.21.15.167:0/693693687 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdb00718e0 0x7efdb0160e50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:24.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.131+0000 7efdb4a00640 1 -- 172.21.15.167:0/693693687 >> 172.21.15.167:0/693693687 conn(0x7efdb0072730 msgr2=0x7efdb00d2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:24.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.132+0000 7efdb4a00640 1 -- 172.21.15.167:0/693693687 shutdown_connections 2024-05-11T12:15:24.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.132+0000 7efdb4a00640 1 -- 172.21.15.167:0/693693687 wait complete. 2024-05-11T12:15:24.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.134+0000 7efdb4a00640 1 Processor -- start 2024-05-11T12:15:24.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.134+0000 7efdb4a00640 1 -- start start 2024-05-11T12:15:24.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.135+0000 7efdb4a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efdb00718e0 0x7efdb0155a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:24.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.135+0000 7efdb4a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efdb0071cc0 0x7efdb00cc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:24.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.135+0000 7efdb4a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdb01573a0 0x7efdb00ca090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:24.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.135+0000 7efdb4a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efdb00d1760 con 0x7efdb0071cc0 2024-05-11T12:15:24.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.135+0000 7efdb4a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7efdb00d1480 con 0x7efdb00718e0 2024-05-11T12:15:24.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.135+0000 7efdb4a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7efdb00d1a40 con 0x7efdb01573a0 2024-05-11T12:15:24.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.135+0000 7efdad400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efdb0071cc0 0x7efdb00cc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:24.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.136+0000 7efdad400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efdb0071cc0 0x7efdb00cc9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:46924/0 (socket says 172.21.15.167:46924) 2024-05-11T12:15:24.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.136+0000 7efdad400640 1 -- 172.21.15.167:0/626680887 learned_addr learned my addr 172.21.15.167:0/626680887 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:24.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.136+0000 7efdae800640 1 --2- 172.21.15.167:0/626680887 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdb01573a0 0x7efdb00ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:24.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.136+0000 7efdad400640 1 -- 172.21.15.167:0/626680887 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdb01573a0 msgr2=0x7efdb00ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:24.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.136+0000 7efdad400640 1 --2- 172.21.15.167:0/626680887 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdb01573a0 0x7efdb00ca090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:24.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.136+0000 7efdade00640 1 --2- 172.21.15.167:0/626680887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efdb00718e0 0x7efdb0155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:24.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.136+0000 7efdad400640 1 -- 172.21.15.167:0/626680887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efdb00718e0 msgr2=0x7efdb0155a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:24.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.136+0000 7efdad400640 1 --2- 172.21.15.167:0/626680887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efdb00718e0 0x7efdb0155a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:24.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.136+0000 7efdad400640 1 -- 172.21.15.167:0/626680887 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdb00c8ca0 con 0x7efdb0071cc0 2024-05-11T12:15:24.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.136+0000 7efdad400640 1 --2- 172.21.15.167:0/626680887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efdb0071cc0 0x7efdb00cc9d0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7efd9800c500 tx=0x7efd9800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:24.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.137+0000 7efda6000640 1 -- 172.21.15.167:0/626680887 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd980052d0 con 0x7efdb0071cc0 2024-05-11T12:15:24.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.137+0000 7efda6000640 1 -- 172.21.15.167:0/626680887 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd98005470 con 0x7efdb0071cc0 2024-05-11T12:15:24.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.137+0000 7efda6000640 1 -- 172.21.15.167:0/626680887 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd98012870 con 0x7efdb0071cc0 2024-05-11T12:15:24.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.137+0000 7efdb4a00640 1 -- 172.21.15.167:0/626680887 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdb00c8f80 con 0x7efdb0071cc0 2024-05-11T12:15:24.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.137+0000 7efdb4a00640 1 -- 172.21.15.167:0/626680887 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdb00c9540 con 0x7efdb0071cc0 2024-05-11T12:15:24.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.138+0000 7efdb4a00640 1 -- 172.21.15.167:0/626680887 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdb0069590 con 0x7efdb0071cc0 2024-05-11T12:15:24.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.139+0000 7efda6000640 1 -- 172.21.15.167:0/626680887 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7efd98012a10 con 0x7efdb0071cc0 2024-05-11T12:15:24.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.140+0000 7efda6000640 1 --2- 172.21.15.167:0/626680887 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7efd8c076300 0x7efd8c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:24.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.140+0000 7efda6000640 1 -- 172.21.15.167:0/626680887 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5507+0+0 (secure 0 0 0) 0x7efd980971c0 con 0x7efdb0071cc0 2024-05-11T12:15:24.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.141+0000 7efdade00640 1 --2- 172.21.15.167:0/626680887 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7efd8c076300 0x7efd8c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:24.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.142+0000 7efdade00640 1 --2- 172.21.15.167:0/626680887 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7efd8c076300 0x7efd8c0787c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7efda0034000 tx=0x7efda002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:24.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.146+0000 7efda6000640 1 -- 172.21.15.167:0/626680887 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7efd98060b50 con 0x7efdb0071cc0 2024-05-11T12:15:24.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.354+0000 7efdb4a00640 1 -- 172.21.15.167:0/626680887 --> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efdb00d25e0 con 0x7efd8c076300 2024-05-11T12:15:24.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.356+0000 7efda6000640 1 -- 172.21.15.167:0/626680887 <== mgr.4106 v2:172.21.15.167:6832/578352559 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13497 (secure 0 0 0) 0x7efdb00d25e0 con 0x7efd8c076300 2024-05-11T12:15:24.356 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:15:24.356 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2024-05-11T12:15:24.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.359+0000 7efdb4a00640 1 -- 172.21.15.167:0/626680887 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7efd8c076300 msgr2=0x7efd8c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:24.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.359+0000 7efdb4a00640 1 --2- 172.21.15.167:0/626680887 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7efd8c076300 0x7efd8c0787c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7efda0034000 tx=0x7efda002e000 comp rx=0 tx=0).stop 2024-05-11T12:15:24.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.359+0000 7efdb4a00640 1 -- 172.21.15.167:0/626680887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efdb0071cc0 msgr2=0x7efdb00cc9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:24.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.359+0000 7efdb4a00640 1 --2- 172.21.15.167:0/626680887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efdb0071cc0 0x7efdb00cc9d0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7efd9800c500 tx=0x7efd9800c9c0 comp rx=0 tx=0).stop 2024-05-11T12:15:24.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.359+0000 7efdb4a00640 1 -- 172.21.15.167:0/626680887 shutdown_connections 2024-05-11T12:15:24.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.359+0000 7efdb4a00640 1 --2- 172.21.15.167:0/626680887 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdb01573a0 0x7efdb00ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:24.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.359+0000 7efdb4a00640 1 --2- 172.21.15.167:0/626680887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efdb0071cc0 0x7efdb00cc9d0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:24.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.359+0000 7efdb4a00640 1 --2- 172.21.15.167:0/626680887 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7efd8c076300 0x7efd8c0787c0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:24.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.359+0000 7efdb4a00640 1 --2- 172.21.15.167:0/626680887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efdb00718e0 0x7efdb0155a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:24.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.359+0000 7efdb4a00640 1 -- 172.21.15.167:0/626680887 >> 172.21.15.167:0/626680887 conn(0x7efdb0072730 msgr2=0x7efdb0159f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:24.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.359+0000 7efdb4a00640 1 -- 172.21.15.167:0/626680887 shutdown_connections 2024-05-11T12:15:24.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.359+0000 7efdb4a00640 1 -- 172.21.15.167:0/626680887 wait complete. 2024-05-11T12:15:24.378 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":243,"stamp":"2024-05-11T12:15:23.323999+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":124,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":260648,"kb_used_data":11192,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714072,"statfs":{"total":773094113280,"available":772827209728,"internally_reserved":0,"allocated":11460608,"data_stored":5871287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24635,"num_objects":-42,"num_object_clones":0,"num_object_copies":-84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-42,"num_whiteouts":0,"num_read":-94,"num_read_kb":-89,"num_write":-161,"num_write_kb":-84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.175696"},"pg_stats":[{"pgid":"1.0","version":"19'72","reported_seq":260,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-11T12:15:20.740225+0000","last_change":"2024-05-11T12:10:35.066673+0000","last_active":"2024-05-11T12:15:20.740225+0000","last_peered":"2024-05-11T12:15:20.740225+0000","last_clean":"2024-05-11T12:15:20.740225+0000","last_became_active":"2024-05-11T12:10:35.065942+0000","last_became_peered":"2024-05-11T12:10:35.065942+0000","last_unstale":"2024-05-11T12:15:20.740225+0000","last_undegraded":"2024-05-11T12:15:20.740225+0000","last_fullsized":"2024-05-11T12:15:20.740225+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_clean_scrub_stamp":"2024-05-11T12:10:33.601136+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:12:02.133182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476797,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33016,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338824,"statfs":{"total":96636764160,"available":96602955776,"internally_reserved":0,"allocated":1880064,"data_stored":1177453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476798,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32436,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1286144,"data_stored":587085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476797,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32440,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1286144,"data_stored":587085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476799,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32440,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1286144,"data_stored":587085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509502,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32440,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1286144,"data_stored":587085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509503,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32440,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1286144,"data_stored":587085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509502,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33016,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338824,"statfs":{"total":96636764160,"available":96602955776,"internally_reserved":0,"allocated":1880064,"data_stored":1177453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542206,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32420,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339420,"statfs":{"total":96636764160,"available":96603566080,"internally_reserved":0,"allocated":1269760,"data_stored":580956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T12:15:24.379 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-11T12:15:24.379 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T12:15:24.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.577+0000 7fbeabe00640 1 Processor -- start 2024-05-11T12:15:24.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.577+0000 7fbeabe00640 1 -- start start 2024-05-11T12:15:24.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.579+0000 7fbeabe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbea40b50a0 0x7fbea4073cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:24.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.579+0000 7fbeabe00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbea415fb90 con 0x7fbea413eeb0 2024-05-11T12:15:24.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.579+0000 7fbeabe00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbea415fe70 con 0x7fbea413f290 2024-05-11T12:15:24.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.579+0000 7fbeabe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbea415d5b0 con 0x7fbea40b50a0 2024-05-11T12:15:24.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.579+0000 7fbea9400640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fbea413eeb0 0x7fbea406f480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:33214/0 (socket says 172.21.15.167:33214) 2024-05-11T12:15:24.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.579+0000 7fbea9400640 1 -- 172.21.15.167:0/772846705 learned_addr learned my addr 172.21.15.167:0/772846705 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:15:24.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.579+0000 7fbea9e00640 1 --2- 172.21.15.167:0/772846705 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbea40b50a0 0x7fbea4073cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:24.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.579+0000 7fbea3e00640 1 -- 172.21.15.167:0/772846705 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3768098881 0 0) 0x7fbea415fb90 con 0x7fbea413eeb0 2024-05-11T12:15:24.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.580+0000 7fbea3e00640 1 -- 172.21.15.167:0/772846705 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe8c003520 con 0x7fbea413eeb0 2024-05-11T12:15:24.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.580+0000 7fbea3e00640 1 -- 172.21.15.167:0/772846705 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1922537437 0 0) 0x7fbe8c003520 con 0x7fbea413eeb0 2024-05-11T12:15:24.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.580+0000 7fbea3e00640 1 -- 172.21.15.167:0/772846705 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbea40b50a0 msgr2=0x7fbea4073cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:24.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.580+0000 7fbea3e00640 1 --2- 172.21.15.167:0/772846705 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbea40b50a0 0x7fbea4073cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:24.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.580+0000 7fbea3e00640 1 -- 172.21.15.167:0/772846705 >> v1:172.21.15.204:6789/0 conn(0x7fbea413f290 legacy=0x7fbea40735c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:24.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.580+0000 7fbea3e00640 1 -- 172.21.15.167:0/772846705 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbea4160150 con 0x7fbea413eeb0 2024-05-11T12:15:24.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.580+0000 7fbea3e00640 1 -- 172.21.15.167:0/772846705 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7fbe980016f0 con 0x7fbea413eeb0 2024-05-11T12:15:24.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.580+0000 7fbea3e00640 1 -- 172.21.15.167:0/772846705 >> v1:172.21.15.167:6789/0 conn(0x7fbea413eeb0 legacy=0x7fbea406f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:24.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.580+0000 7fbea3e00640 1 --2- 172.21.15.167:0/772846705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe8c003a30 0x7fbe8c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:24.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.580+0000 7fbea3e00640 1 --2- 172.21.15.167:0/772846705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe8c004800 0x7fbe8c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:24.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.580+0000 7fbea3e00640 1 --2- 172.21.15.167:0/772846705 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbea40b50a0 0x7fbe8c005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:24.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.580+0000 7fbea3e00640 1 -- 172.21.15.167:0/772846705 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8c00bb80 con 0x7fbe8c004800 2024-05-11T12:15:24.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.580+0000 7fbea3e00640 1 -- 172.21.15.167:0/772846705 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8c00bd30 con 0x7fbe8c003a30 2024-05-11T12:15:24.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.580+0000 7fbea3e00640 1 -- 172.21.15.167:0/772846705 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbe8c00bf30 con 0x7fbea40b50a0 2024-05-11T12:15:24.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.580+0000 7fbea8a00640 1 --2- 172.21.15.167:0/772846705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe8c004800 0x7fbe8c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:24.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.581+0000 7fbea9400640 1 --2- 172.21.15.167:0/772846705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe8c003a30 0x7fbe8c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:24.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.581+0000 7fbea9e00640 1 --2- 172.21.15.167:0/772846705 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbea40b50a0 0x7fbe8c005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:24.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.581+0000 7fbea8a00640 1 -- 172.21.15.167:0/772846705 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbea40b50a0 msgr2=0x7fbe8c005290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:24.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.581+0000 7fbea8a00640 1 --2- 172.21.15.167:0/772846705 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbea40b50a0 0x7fbe8c005290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:24.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.581+0000 7fbea8a00640 1 -- 172.21.15.167:0/772846705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe8c003a30 msgr2=0x7fbe8c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:24.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.581+0000 7fbea8a00640 1 --2- 172.21.15.167:0/772846705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe8c003a30 0x7fbe8c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:24.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.581+0000 7fbea8a00640 1 -- 172.21.15.167:0/772846705 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe8c003520 con 0x7fbe8c004800 2024-05-11T12:15:24.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.581+0000 7fbea9e00640 1 --2- 172.21.15.167:0/772846705 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbea40b50a0 0x7fbe8c005290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:15:24.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.582+0000 7fbea8a00640 1 --2- 172.21.15.167:0/772846705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe8c004800 0x7fbe8c004c80 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fbe90001790 tx=0x7fbe8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ee3fe3318fc36ba server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:24.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.582+0000 7fbea3e00640 1 -- 172.21.15.167:0/772846705 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe900013b0 con 0x7fbe8c004800 2024-05-11T12:15:24.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.582+0000 7fbea3e00640 1 -- 172.21.15.167:0/772846705 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe900052d0 con 0x7fbe8c004800 2024-05-11T12:15:24.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.582+0000 7fbea3e00640 1 -- 172.21.15.167:0/772846705 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9002aae0 con 0x7fbe8c004800 2024-05-11T12:15:24.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.583+0000 7fbeabe00640 1 -- 172.21.15.167:0/772846705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe8c004800 msgr2=0x7fbe8c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:24.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.583+0000 7fbeabe00640 1 --2- 172.21.15.167:0/772846705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe8c004800 0x7fbe8c004c80 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fbe90001790 tx=0x7fbe8c005830 comp rx=0 tx=0).stop 2024-05-11T12:15:24.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.583+0000 7fbea9e00640 1 -- 172.21.15.167:0/772846705 reap_dead start 2024-05-11T12:15:24.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.583+0000 7fbeabe00640 1 -- 172.21.15.167:0/772846705 shutdown_connections 2024-05-11T12:15:24.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.583+0000 7fbeabe00640 1 -- 172.21.15.167:0/772846705 >> 172.21.15.167:0/772846705 conn(0x7fbea40cd8f0 msgr2=0x7fbea40cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:24.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.583+0000 7fbeabe00640 1 -- 172.21.15.167:0/772846705 shutdown_connections 2024-05-11T12:15:24.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.583+0000 7fbeabe00640 1 -- 172.21.15.167:0/772846705 wait complete. 2024-05-11T12:15:24.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.585+0000 7fbeabe00640 1 Processor -- start 2024-05-11T12:15:24.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.585+0000 7fbeabe00640 1 -- start start 2024-05-11T12:15:24.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.586+0000 7fbeabe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe8c004800 0x7fbea413a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:24.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.586+0000 7fbeabe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbea414fa00 0x7fbea4139e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:24.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.586+0000 7fbeabe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbea415e7a0 0x7fbea415ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:24.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.586+0000 7fbeabe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbea406fb90 con 0x7fbea414fa00 2024-05-11T12:15:24.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.586+0000 7fbeabe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbea4160710 con 0x7fbe8c004800 2024-05-11T12:15:24.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.586+0000 7fbeabe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbea406fd10 con 0x7fbea415e7a0 2024-05-11T12:15:24.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.586+0000 7fbea8a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbea414fa00 0x7fbea4139e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:24.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.586+0000 7fbea8a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbea414fa00 0x7fbea4139e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:46940/0 (socket says 172.21.15.167:46940) 2024-05-11T12:15:24.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.586+0000 7fbea8a00640 1 -- 172.21.15.167:0/11301920 learned_addr learned my addr 172.21.15.167:0/11301920 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:24.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.587+0000 7fbea9e00640 1 --2- 172.21.15.167:0/11301920 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbea415e7a0 0x7fbea415ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:24.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.587+0000 7fbea8a00640 1 -- 172.21.15.167:0/11301920 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbea415e7a0 msgr2=0x7fbea415ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:24.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.587+0000 7fbea9400640 1 --2- 172.21.15.167:0/11301920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe8c004800 0x7fbea413a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:24.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.587+0000 7fbea8a00640 1 --2- 172.21.15.167:0/11301920 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbea415e7a0 0x7fbea415ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:24.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.587+0000 7fbea8a00640 1 -- 172.21.15.167:0/11301920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe8c004800 msgr2=0x7fbea413a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:24.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.587+0000 7fbea8a00640 1 --2- 172.21.15.167:0/11301920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe8c004800 0x7fbea413a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:24.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.587+0000 7fbea8a00640 1 -- 172.21.15.167:0/11301920 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbea413c7e0 con 0x7fbea414fa00 2024-05-11T12:15:24.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.587+0000 7fbea8a00640 1 --2- 172.21.15.167:0/11301920 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbea414fa00 0x7fbea4139e60 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fbe90001790 tx=0x7fbe90001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:24.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.587+0000 7fbea1600640 1 -- 172.21.15.167:0/11301920 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9003f020 con 0x7fbea414fa00 2024-05-11T12:15:24.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.587+0000 7fbea1600640 1 -- 172.21.15.167:0/11301920 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe90005480 con 0x7fbea414fa00 2024-05-11T12:15:24.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.587+0000 7fbea1600640 1 -- 172.21.15.167:0/11301920 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe90037040 con 0x7fbea414fa00 2024-05-11T12:15:24.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.587+0000 7fbeabe00640 1 -- 172.21.15.167:0/11301920 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbea4150950 con 0x7fbea414fa00 2024-05-11T12:15:24.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.587+0000 7fbeabe00640 1 -- 172.21.15.167:0/11301920 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbea413ca50 con 0x7fbea414fa00 2024-05-11T12:15:24.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.589+0000 7fbeabe00640 1 -- 172.21.15.167:0/11301920 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbea4069590 con 0x7fbea414fa00 2024-05-11T12:15:24.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.590+0000 7fbea1600640 1 -- 172.21.15.167:0/11301920 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbe90042020 con 0x7fbea414fa00 2024-05-11T12:15:24.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.591+0000 7fbea1600640 1 --2- 172.21.15.167:0/11301920 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbe840762d0 0x7fbe84078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:24.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.591+0000 7fbea1600640 1 -- 172.21.15.167:0/11301920 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5507+0+0 (secure 0 0 0) 0x7fbe900b6fe0 con 0x7fbea414fa00 2024-05-11T12:15:24.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.591+0000 7fbea9400640 1 --2- 172.21.15.167:0/11301920 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbe840762d0 0x7fbe84078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:24.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.592+0000 7fbea9400640 1 --2- 172.21.15.167:0/11301920 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbe840762d0 0x7fbe84078790 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fbe98001eb0 tx=0x7fbe98000ed0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:24.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.596+0000 7fbea1600640 1 -- 172.21.15.167:0/11301920 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fbe90049030 con 0x7fbea414fa00 2024-05-11T12:15:24.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.799+0000 7fbeabe00640 1 -- 172.21.15.167:0/11301920 --> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbea406cbd0 con 0x7fbe840762d0 2024-05-11T12:15:24.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.800+0000 7fbea1600640 1 -- 172.21.15.167:0/11301920 <== mgr.4106 v2:172.21.15.167:6832/578352559 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13497 (secure 0 0 0) 0x7fbea406cbd0 con 0x7fbe840762d0 2024-05-11T12:15:24.800 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:15:24.800 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2024-05-11T12:15:24.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.803+0000 7fbeabe00640 1 -- 172.21.15.167:0/11301920 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbe840762d0 msgr2=0x7fbe84078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:24.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.803+0000 7fbeabe00640 1 --2- 172.21.15.167:0/11301920 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbe840762d0 0x7fbe84078790 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fbe98001eb0 tx=0x7fbe98000ed0 comp rx=0 tx=0).stop 2024-05-11T12:15:24.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.803+0000 7fbeabe00640 1 -- 172.21.15.167:0/11301920 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbea414fa00 msgr2=0x7fbea4139e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:24.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.803+0000 7fbeabe00640 1 --2- 172.21.15.167:0/11301920 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbea414fa00 0x7fbea4139e60 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fbe90001790 tx=0x7fbe90001120 comp rx=0 tx=0).stop 2024-05-11T12:15:24.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.803+0000 7fbeabe00640 1 -- 172.21.15.167:0/11301920 shutdown_connections 2024-05-11T12:15:24.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.803+0000 7fbeabe00640 1 --2- 172.21.15.167:0/11301920 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbea415e7a0 0x7fbea415ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:24.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.803+0000 7fbeabe00640 1 --2- 172.21.15.167:0/11301920 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbea414fa00 0x7fbea4139e60 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:24.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.803+0000 7fbeabe00640 1 --2- 172.21.15.167:0/11301920 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fbe840762d0 0x7fbe84078790 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:24.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.803+0000 7fbeabe00640 1 --2- 172.21.15.167:0/11301920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe8c004800 0x7fbea413a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:24.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.803+0000 7fbeabe00640 1 -- 172.21.15.167:0/11301920 >> 172.21.15.167:0/11301920 conn(0x7fbea40cd8f0 msgr2=0x7fbea406e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:24.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.804+0000 7fbeabe00640 1 -- 172.21.15.167:0/11301920 shutdown_connections 2024-05-11T12:15:24.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:24.804+0000 7fbeabe00640 1 -- 172.21.15.167:0/11301920 wait complete. 2024-05-11T12:15:24.822 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":243,"stamp":"2024-05-11T12:15:23.323999+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":124,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":260648,"kb_used_data":11192,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714072,"statfs":{"total":773094113280,"available":772827209728,"internally_reserved":0,"allocated":11460608,"data_stored":5871287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24635,"num_objects":-42,"num_object_clones":0,"num_object_copies":-84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-42,"num_whiteouts":0,"num_read":-94,"num_read_kb":-89,"num_write":-161,"num_write_kb":-84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.175696"},"pg_stats":[{"pgid":"1.0","version":"19'72","reported_seq":260,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-11T12:15:20.740225+0000","last_change":"2024-05-11T12:10:35.066673+0000","last_active":"2024-05-11T12:15:20.740225+0000","last_peered":"2024-05-11T12:15:20.740225+0000","last_clean":"2024-05-11T12:15:20.740225+0000","last_became_active":"2024-05-11T12:10:35.065942+0000","last_became_peered":"2024-05-11T12:10:35.065942+0000","last_unstale":"2024-05-11T12:15:20.740225+0000","last_undegraded":"2024-05-11T12:15:20.740225+0000","last_fullsized":"2024-05-11T12:15:20.740225+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_clean_scrub_stamp":"2024-05-11T12:10:33.601136+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:12:02.133182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476797,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33016,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338824,"statfs":{"total":96636764160,"available":96602955776,"internally_reserved":0,"allocated":1880064,"data_stored":1177453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476798,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32436,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1286144,"data_stored":587085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476797,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32440,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1286144,"data_stored":587085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476799,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32440,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1286144,"data_stored":587085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509502,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32440,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1286144,"data_stored":587085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509503,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32440,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1286144,"data_stored":587085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509502,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33016,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338824,"statfs":{"total":96636764160,"available":96602955776,"internally_reserved":0,"allocated":1880064,"data_stored":1177453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542206,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32420,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339420,"statfs":{"total":96636764160,"available":96603566080,"internally_reserved":0,"allocated":1269760,"data_stored":580956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T12:15:24.823 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:15:25.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.024+0000 7fa4cb200640 1 Processor -- start 2024-05-11T12:15:25.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.024+0000 7fa4cb200640 1 -- start start 2024-05-11T12:15:25.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.026+0000 7fa4cb200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa4cc0d04a0 0x7fa4cc1601f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:25.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.026+0000 7fa4cb200640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4cc0b4560 con 0x7fa4cc0afcd0 2024-05-11T12:15:25.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.026+0000 7fa4cb200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4cc0b4800 con 0x7fa4cc0cf970 2024-05-11T12:15:25.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.026+0000 7fa4cb200640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fa4cc0b5130 con 0x7fa4cc0d04a0 2024-05-11T12:15:25.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.026+0000 7fa4c9e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fa4cc0afcd0 0x7fa4cc0cf260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:33222/0 (socket says 172.21.15.167:33222) 2024-05-11T12:15:25.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.026+0000 7fa4c9e00640 1 -- 172.21.15.167:0/3872641064 learned_addr learned my addr 172.21.15.167:0/3872641064 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:15:25.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.026+0000 7fa4ca800640 1 --2- 172.21.15.167:0/3872641064 >> v2:172.21.15.167:3300/0 conn(0x7fa4cc0d04a0 0x7fa4cc1601f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:25.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.027+0000 7fa4c8a00640 1 -- 172.21.15.167:0/3872641064 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1727671420 0 0) 0x7fa4cc0b4560 con 0x7fa4cc0afcd0 2024-05-11T12:15:25.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.027+0000 7fa4c8a00640 1 -- 172.21.15.167:0/3872641064 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa4b0003520 con 0x7fa4cc0afcd0 2024-05-11T12:15:25.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.027+0000 7fa4ca800640 1 -- 172.21.15.167:0/3872641064 >> v1:172.21.15.204:6789/0 conn(0x7fa4cc0cf970 legacy=0x7fa4cc0721d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:25.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.027+0000 7fa4ca800640 1 -- 172.21.15.167:0/3872641064 >> v1:172.21.15.167:6789/0 conn(0x7fa4cc0afcd0 legacy=0x7fa4cc0cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:25.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.027+0000 7fa4ca800640 1 -- 172.21.15.167:0/3872641064 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4cc0b4ae0 con 0x7fa4cc0d04a0 2024-05-11T12:15:25.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.028+0000 7fa4ca800640 1 --2- 172.21.15.167:0/3872641064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4cc0d04a0 0x7fa4cc1601f0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4009d90 tx=0x7fa4c402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99f21d3de4f59760 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:25.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.028+0000 7fa4c8a00640 1 -- 172.21.15.167:0/3872641064 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4007740 con 0x7fa4cc0d04a0 2024-05-11T12:15:25.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.028+0000 7fa4c8a00640 1 -- 172.21.15.167:0/3872641064 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4c40078e0 con 0x7fa4cc0d04a0 2024-05-11T12:15:25.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.028+0000 7fa4c8a00640 1 -- 172.21.15.167:0/3872641064 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4007c90 con 0x7fa4cc0d04a0 2024-05-11T12:15:25.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.029+0000 7fa4cb200640 1 -- 172.21.15.167:0/3872641064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4cc0d04a0 msgr2=0x7fa4cc1601f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:25.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.029+0000 7fa4cb200640 1 --2- 172.21.15.167:0/3872641064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4cc0d04a0 0x7fa4cc1601f0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4009d90 tx=0x7fa4c402a660 comp rx=0 tx=0).stop 2024-05-11T12:15:25.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.029+0000 7fa4cb200640 1 -- 172.21.15.167:0/3872641064 shutdown_connections 2024-05-11T12:15:25.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.029+0000 7fa4cb200640 1 --2- 172.21.15.167:0/3872641064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4cc0d04a0 0x7fa4cc1601f0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:25.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.029+0000 7fa4cb200640 1 -- 172.21.15.167:0/3872641064 >> 172.21.15.167:0/3872641064 conn(0x7fa4cc0b3460 msgr2=0x7fa4cc0d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:25.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.029+0000 7fa4cb200640 1 -- 172.21.15.167:0/3872641064 shutdown_connections 2024-05-11T12:15:25.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.029+0000 7fa4cb200640 1 -- 172.21.15.167:0/3872641064 wait complete. 2024-05-11T12:15:25.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.031+0000 7fa4cb200640 1 Processor -- start 2024-05-11T12:15:25.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.031+0000 7fa4cb200640 1 -- start start 2024-05-11T12:15:25.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.032+0000 7fa4cb200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4cc0afcd0 0x7fa4cc15a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:25.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.032+0000 7fa4cb200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4cc0cf970 0x7fa4cc0725e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:25.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.032+0000 7fa4cb200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4cc0d04a0 0x7fa4cc1469c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:25.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.032+0000 7fa4cb200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa4cc0cff40 con 0x7fa4cc0d04a0 2024-05-11T12:15:25.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.032+0000 7fa4cb200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa4cc0b00b0 con 0x7fa4cc0afcd0 2024-05-11T12:15:25.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.032+0000 7fa4cb200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa4cc0d00c0 con 0x7fa4cc0cf970 2024-05-11T12:15:25.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.032+0000 7fa4ca800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4cc0d04a0 0x7fa4cc1469c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:25.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.032+0000 7fa4ca800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4cc0d04a0 0x7fa4cc1469c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:46966/0 (socket says 172.21.15.167:46966) 2024-05-11T12:15:25.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.032+0000 7fa4ca800640 1 -- 172.21.15.167:0/4004184518 learned_addr learned my addr 172.21.15.167:0/4004184518 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:25.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.032+0000 7fa4c9e00640 1 --2- 172.21.15.167:0/4004184518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4cc0afcd0 0x7fa4cc15a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:25.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.032+0000 7fa4c9400640 1 --2- 172.21.15.167:0/4004184518 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4cc0cf970 0x7fa4cc0725e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:25.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.033+0000 7fa4ca800640 1 -- 172.21.15.167:0/4004184518 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4cc0cf970 msgr2=0x7fa4cc0725e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:25.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.033+0000 7fa4ca800640 1 --2- 172.21.15.167:0/4004184518 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4cc0cf970 0x7fa4cc0725e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:25.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.033+0000 7fa4ca800640 1 -- 172.21.15.167:0/4004184518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4cc0afcd0 msgr2=0x7fa4cc15a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:25.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.033+0000 7fa4ca800640 1 --2- 172.21.15.167:0/4004184518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4cc0afcd0 0x7fa4cc15a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:25.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.033+0000 7fa4ca800640 1 -- 172.21.15.167:0/4004184518 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4cc147740 con 0x7fa4cc0d04a0 2024-05-11T12:15:25.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.033+0000 7fa4c9400640 1 --2- 172.21.15.167:0/4004184518 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4cc0cf970 0x7fa4cc0725e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:15:25.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.033+0000 7fa4ca800640 1 --2- 172.21.15.167:0/4004184518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4cc0d04a0 0x7fa4cc1469c0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4009d90 tx=0x7fa4c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:25.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.033+0000 7fa4c2000640 1 -- 172.21.15.167:0/4004184518 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c403f070 con 0x7fa4cc0d04a0 2024-05-11T12:15:25.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.033+0000 7fa4c2000640 1 -- 172.21.15.167:0/4004184518 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4c4031c40 con 0x7fa4cc0d04a0 2024-05-11T12:15:25.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.033+0000 7fa4c2000640 1 -- 172.21.15.167:0/4004184518 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4009450 con 0x7fa4cc0d04a0 2024-05-11T12:15:25.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.033+0000 7fa4cb200640 1 -- 172.21.15.167:0/4004184518 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4cc147a20 con 0x7fa4cc0d04a0 2024-05-11T12:15:25.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.033+0000 7fa4cb200640 1 -- 172.21.15.167:0/4004184518 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4cc147fe0 con 0x7fa4cc0d04a0 2024-05-11T12:15:25.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.034+0000 7fa4cb200640 1 -- 172.21.15.167:0/4004184518 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4cc0694b0 con 0x7fa4cc0d04a0 2024-05-11T12:15:25.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.036+0000 7fa4c2000640 1 -- 172.21.15.167:0/4004184518 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa4c4003990 con 0x7fa4cc0d04a0 2024-05-11T12:15:25.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.037+0000 7fa4c2000640 1 --2- 172.21.15.167:0/4004184518 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa4ac0762d0 0x7fa4ac078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:25.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.037+0000 7fa4c2000640 1 -- 172.21.15.167:0/4004184518 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5507+0+0 (secure 0 0 0) 0x7fa4c40b82d0 con 0x7fa4cc0d04a0 2024-05-11T12:15:25.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.037+0000 7fa4c9e00640 1 --2- 172.21.15.167:0/4004184518 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa4ac0762d0 0x7fa4ac078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:25.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.038+0000 7fa4c9e00640 1 --2- 172.21.15.167:0/4004184518 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa4ac0762d0 0x7fa4ac078790 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc007a60 tx=0x7fa4bc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:25.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.041+0000 7fa4c2000640 1 -- 172.21.15.167:0/4004184518 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa4c4081ce0 con 0x7fa4cc0d04a0 2024-05-11T12:15:25.253 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.254+0000 7fa4cb200640 1 -- 172.21.15.167:0/4004184518 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa4cc0cf040 con 0x7fa4cc0d04a0 2024-05-11T12:15:25.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.255+0000 7fa4c2000640 1 -- 172.21.15.167:0/4004184518 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) ==== 74+0+13609 (secure 0 0 0) 0x7fa4c40815e0 con 0x7fa4cc0d04a0 2024-05-11T12:15:25.255 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:15:25.255 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":104,"fsid":"5f32d344-7289-4c27-881b-1533419ab678","created":"2024-05-11T12:10:20.035498+0000","modified":"2024-05-11T12:15:20.164008+0000","last_up_change":"2024-05-11T12:10:32.596203+0000","last_in_change":"2024-05-11T12:10:23.292769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:10:33.260270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f30405d9-0563-4bb3-be12-ec51d5932be7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6809","nonce":1777085638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6811","nonce":1777085638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6815","nonce":1777085638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1777085638},{"type":"v1","addr":"172.21.15.167:6813","nonce":1777085638}]},"public_addr":"172.21.15.167:6809/1777085638","cluster_addr":"172.21.15.167:6811/1777085638","heartbeat_back_addr":"172.21.15.167:6815/1777085638","heartbeat_front_addr":"172.21.15.167:6813/1777085638","state":["exists","up"]},{"osd":1,"uuid":"139c6ac5-9615-41f4-8ec1-f409b6b68f6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6801","nonce":1480182821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6803","nonce":1480182821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6807","nonce":1480182821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1480182821},{"type":"v1","addr":"172.21.15.167:6805","nonce":1480182821}]},"public_addr":"172.21.15.167:6801/1480182821","cluster_addr":"172.21.15.167:6803/1480182821","heartbeat_back_addr":"172.21.15.167:6807/1480182821","heartbeat_front_addr":"172.21.15.167:6805/1480182821","state":["exists","up"]},{"osd":2,"uuid":"f1c5d46a-2397-41ee-a5ee-4eb956dc6ca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6820","nonce":1902208221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6823","nonce":1902208221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6829","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6830","nonce":1902208221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6825","nonce":1902208221},{"type":"v1","addr":"172.21.15.167:6827","nonce":1902208221}]},"public_addr":"172.21.15.167:6820/1902208221","cluster_addr":"172.21.15.167:6823/1902208221","heartbeat_back_addr":"172.21.15.167:6830/1902208221","heartbeat_front_addr":"172.21.15.167:6827/1902208221","state":["exists","up"]},{"osd":3,"uuid":"2e95ae7d-d784-47ab-a14a-e74331ca8821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6817","nonce":1903389836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6819","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6821","nonce":1903389836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6831","nonce":1903389836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1903389836},{"type":"v1","addr":"172.21.15.167:6826","nonce":1903389836}]},"public_addr":"172.21.15.167:6817/1903389836","cluster_addr":"172.21.15.167:6821/1903389836","heartbeat_back_addr":"172.21.15.167:6831/1903389836","heartbeat_front_addr":"172.21.15.167:6826/1903389836","state":["exists","up"]},{"osd":4,"uuid":"3a063a36-d2ae-4f90-87b8-dc9f35a2f3cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6817","nonce":2030699881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6819","nonce":2030699881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6823","nonce":2030699881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2030699881},{"type":"v1","addr":"172.21.15.204:6821","nonce":2030699881}]},"public_addr":"172.21.15.204:6817/2030699881","cluster_addr":"172.21.15.204:6819/2030699881","heartbeat_back_addr":"172.21.15.204:6823/2030699881","heartbeat_front_addr":"172.21.15.204:6821/2030699881","state":["exists","up"]},{"osd":5,"uuid":"12519311-12a1-4b83-b835-9af10f397c8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6825","nonce":1178824319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6827","nonce":1178824319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6831","nonce":1178824319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1178824319},{"type":"v1","addr":"172.21.15.204:6829","nonce":1178824319}]},"public_addr":"172.21.15.204:6825/1178824319","cluster_addr":"172.21.15.204:6827/1178824319","heartbeat_back_addr":"172.21.15.204:6831/1178824319","heartbeat_front_addr":"172.21.15.204:6829/1178824319","state":["exists","up"]},{"osd":6,"uuid":"1ed30e71-408f-4586-869b-ce90d9bc76c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6801","nonce":3396850806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6803","nonce":3396850806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6807","nonce":3396850806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3396850806},{"type":"v1","addr":"172.21.15.204:6805","nonce":3396850806}]},"public_addr":"172.21.15.204:6801/3396850806","cluster_addr":"172.21.15.204:6803/3396850806","heartbeat_back_addr":"172.21.15.204:6807/3396850806","heartbeat_front_addr":"172.21.15.204:6805/3396850806","state":["exists","up"]},{"osd":7,"uuid":"37a4a8a8-ff19-4659-8a2f-dbd1b2d743b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6809","nonce":223313265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6811","nonce":223313265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6816","nonce":223313265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":223313265},{"type":"v1","addr":"172.21.15.204:6813","nonce":223313265}]},"public_addr":"172.21.15.204:6809/223313265","cluster_addr":"172.21.15.204:6811/223313265","heartbeat_back_addr":"172.21.15.204:6816/223313265","heartbeat_front_addr":"172.21.15.204:6813/223313265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.900801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.898837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.073306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.700897+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.945324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.749819+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.666975+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:10:28.979168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.167:6837/2901182170":"2024-05-12T12:15:17.144165+0000","172.21.15.167:6836/2901182170":"2024-05-12T12:15:17.144165+0000","172.21.15.167:6835/3735016882":"2024-05-12T12:15:17.144144+0000","172.21.15.167:6834/3735016882":"2024-05-12T12:15:17.144144+0000","172.21.15.167:6837/3493868276":"2024-05-12T12:14:50.725899+0000","172.21.15.167:6836/3493868276":"2024-05-12T12:14:50.725899+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:15:25.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.258+0000 7fa4cb200640 1 -- 172.21.15.167:0/4004184518 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa4ac0762d0 msgr2=0x7fa4ac078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:25.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.258+0000 7fa4cb200640 1 --2- 172.21.15.167:0/4004184518 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa4ac0762d0 0x7fa4ac078790 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc007a60 tx=0x7fa4bc009040 comp rx=0 tx=0).stop 2024-05-11T12:15:25.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.258+0000 7fa4cb200640 1 -- 172.21.15.167:0/4004184518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4cc0d04a0 msgr2=0x7fa4cc1469c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:25.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.258+0000 7fa4cb200640 1 --2- 172.21.15.167:0/4004184518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4cc0d04a0 0x7fa4cc1469c0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4009d90 tx=0x7fa4c4004a40 comp rx=0 tx=0).stop 2024-05-11T12:15:25.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.258+0000 7fa4cb200640 1 -- 172.21.15.167:0/4004184518 shutdown_connections 2024-05-11T12:15:25.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.258+0000 7fa4cb200640 1 --2- 172.21.15.167:0/4004184518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4cc0d04a0 0x7fa4cc1469c0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:25.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.258+0000 7fa4cb200640 1 --2- 172.21.15.167:0/4004184518 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4cc0cf970 0x7fa4cc0725e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:25.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.258+0000 7fa4cb200640 1 --2- 172.21.15.167:0/4004184518 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fa4ac0762d0 0x7fa4ac078790 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:25.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.258+0000 7fa4cb200640 1 --2- 172.21.15.167:0/4004184518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4cc0afcd0 0x7fa4cc15a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:25.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.258+0000 7fa4cb200640 1 -- 172.21.15.167:0/4004184518 >> 172.21.15.167:0/4004184518 conn(0x7fa4cc0b3460 msgr2=0x7fa4cc0cf260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:25.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.258+0000 7fa4cb200640 1 -- 172.21.15.167:0/4004184518 shutdown_connections 2024-05-11T12:15:25.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:25.258+0000 7fa4cb200640 1 -- 172.21.15.167:0/4004184518 wait complete. 2024-05-11T12:15:26.278 INFO:tasks.ceph:Scrubbing osd.0 2024-05-11T12:15:26.278 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-05-11T12:15:26.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.496+0000 7f4a83200640 1 Processor -- start 2024-05-11T12:15:26.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.497+0000 7f4a83200640 1 -- start start 2024-05-11T12:15:26.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.498+0000 7f4a83200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a8415f260 0x7f4a841637e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:26.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.498+0000 7f4a83200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4a840cfd90 0x7f4a840d3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:26.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.498+0000 7f4a83200640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a84163d20 con 0x7f4a840d4310 2024-05-11T12:15:26.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.498+0000 7f4a83200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f4a840d3940 con 0x7f4a840cfd90 2024-05-11T12:15:26.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.498+0000 7f4a83200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4a8415f640 con 0x7f4a8415f260 2024-05-11T12:15:26.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.498+0000 7f4a82800640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f4a840d4310 0x7f4a8406d060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:33236/0 (socket says 172.21.15.167:33236) 2024-05-11T12:15:26.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.498+0000 7f4a82800640 1 -- 172.21.15.167:0/1214808827 learned_addr learned my addr 172.21.15.167:0/1214808827 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:15:26.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.499+0000 7f4a81e00640 1 --2- 172.21.15.167:0/1214808827 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a8415f260 0x7f4a841637e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:26.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.499+0000 7f4a81400640 1 --2- 172.21.15.167:0/1214808827 >> v2:172.21.15.204:3300/0 conn(0x7f4a840cfd90 0x7f4a840d3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:26.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.499+0000 7f4a80a00640 1 -- 172.21.15.167:0/1214808827 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 321876741 0 0) 0x7f4a74001110 con 0x7f4a840d4310 2024-05-11T12:15:26.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.499+0000 7f4a80a00640 1 -- 172.21.15.167:0/1214808827 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a68003520 con 0x7f4a840d4310 2024-05-11T12:15:26.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.499+0000 7f4a80a00640 1 -- 172.21.15.167:0/1214808827 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2234083130 0 0) 0x7f4a68003520 con 0x7f4a840d4310 2024-05-11T12:15:26.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.499+0000 7f4a80a00640 1 -- 172.21.15.167:0/1214808827 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a8415f260 msgr2=0x7f4a841637e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:26.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.499+0000 7f4a80a00640 1 --2- 172.21.15.167:0/1214808827 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a8415f260 0x7f4a841637e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:26.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.499+0000 7f4a80a00640 1 -- 172.21.15.167:0/1214808827 >> v2:172.21.15.204:3300/0 conn(0x7f4a840cfd90 msgr2=0x7f4a840d3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:26.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.499+0000 7f4a80a00640 1 --2- 172.21.15.167:0/1214808827 >> v2:172.21.15.204:3300/0 conn(0x7f4a840cfd90 0x7f4a840d3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:26.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.499+0000 7f4a80a00640 1 -- 172.21.15.167:0/1214808827 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a840d2d30 con 0x7f4a840d4310 2024-05-11T12:15:26.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.499+0000 7f4a80a00640 1 -- 172.21.15.167:0/1214808827 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f4a740018c0 con 0x7f4a840d4310 2024-05-11T12:15:26.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.499+0000 7f4a80a00640 1 -- 172.21.15.167:0/1214808827 >> v1:172.21.15.167:6789/0 conn(0x7f4a840d4310 legacy=0x7f4a8406d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:26.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.499+0000 7f4a80a00640 1 --2- 172.21.15.167:0/1214808827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a68003a00 0x7f4a68003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:26.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.499+0000 7f4a80a00640 1 --2- 172.21.15.167:0/1214808827 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a8415f260 0x7f4a68004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:26.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.499+0000 7f4a80a00640 1 --2- 172.21.15.167:0/1214808827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a68004d40 0x7f4a680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:26.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.499+0000 7f4a80a00640 1 -- 172.21.15.167:0/1214808827 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4a840d3940 con 0x7f4a68003a00 2024-05-11T12:15:26.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.499+0000 7f4a80a00640 1 -- 172.21.15.167:0/1214808827 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8415f640 con 0x7f4a68004d40 2024-05-11T12:15:26.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.499+0000 7f4a80a00640 1 -- 172.21.15.167:0/1214808827 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4a6800b9c0 con 0x7f4a8415f260 2024-05-11T12:15:26.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.500+0000 7f4a81e00640 1 --2- 172.21.15.167:0/1214808827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a68003a00 0x7f4a68003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:26.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.500+0000 7f4a81e00640 1 -- 172.21.15.167:0/1214808827 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a8415f260 msgr2=0x7f4a68004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:26.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.500+0000 7f4a81e00640 1 --2- 172.21.15.167:0/1214808827 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a8415f260 0x7f4a68004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:26.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.500+0000 7f4a81e00640 1 -- 172.21.15.167:0/1214808827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a68004d40 msgr2=0x7f4a680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:26.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.500+0000 7f4a81e00640 1 --2- 172.21.15.167:0/1214808827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a68004d40 0x7f4a680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:26.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.500+0000 7f4a81e00640 1 -- 172.21.15.167:0/1214808827 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a68003520 con 0x7f4a68003a00 2024-05-11T12:15:26.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.501+0000 7f4a81e00640 1 --2- 172.21.15.167:0/1214808827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a68003a00 0x7f4a68003e20 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c009fc0 tx=0x7f4a68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d256887694db7dff server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:26.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.501+0000 7f4a80a00640 1 -- 172.21.15.167:0/1214808827 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c009c10 con 0x7f4a68003a00 2024-05-11T12:15:26.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.501+0000 7f4a80a00640 1 -- 172.21.15.167:0/1214808827 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a7c036070 con 0x7f4a68003a00 2024-05-11T12:15:26.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.501+0000 7f4a80a00640 1 -- 172.21.15.167:0/1214808827 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c02d070 con 0x7f4a68003a00 2024-05-11T12:15:26.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.502+0000 7f4a83200640 1 -- 172.21.15.167:0/1214808827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a68003a00 msgr2=0x7f4a68003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:26.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.502+0000 7f4a83200640 1 --2- 172.21.15.167:0/1214808827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a68003a00 0x7f4a68003e20 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c009fc0 tx=0x7f4a68005830 comp rx=0 tx=0).stop 2024-05-11T12:15:26.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.502+0000 7f4a82800640 1 -- 172.21.15.167:0/1214808827 reap_dead start 2024-05-11T12:15:26.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.502+0000 7f4a83200640 1 -- 172.21.15.167:0/1214808827 shutdown_connections 2024-05-11T12:15:26.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.502+0000 7f4a83200640 1 -- 172.21.15.167:0/1214808827 >> 172.21.15.167:0/1214808827 conn(0x7f4a840b6cd0 msgr2=0x7f4a840b70e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:26.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.502+0000 7f4a83200640 1 -- 172.21.15.167:0/1214808827 shutdown_connections 2024-05-11T12:15:26.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.502+0000 7f4a83200640 1 -- 172.21.15.167:0/1214808827 wait complete. 2024-05-11T12:15:26.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.504+0000 7f4a83200640 1 Processor -- start 2024-05-11T12:15:26.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.504+0000 7f4a83200640 1 -- start start 2024-05-11T12:15:26.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.505+0000 7f4a83200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a68003a00 0x7f4a84155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:26.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.505+0000 7f4a83200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a84156270 0x7f4a841577c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:26.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.505+0000 7f4a83200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a84160ef0 0x7f4a84161370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:26.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.505+0000 7f4a83200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4a840d35d0 con 0x7f4a84156270 2024-05-11T12:15:26.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.505+0000 7f4a83200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4a840d32f0 con 0x7f4a68003a00 2024-05-11T12:15:26.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.505+0000 7f4a83200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4a8406d770 con 0x7f4a84160ef0 2024-05-11T12:15:26.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.505+0000 7f4a81400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a84156270 0x7f4a841577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:26.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.505+0000 7f4a81400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a84156270 0x7f4a841577c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:46988/0 (socket says 172.21.15.167:46988) 2024-05-11T12:15:26.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.505+0000 7f4a81400640 1 -- 172.21.15.167:0/1330354828 learned_addr learned my addr 172.21.15.167:0/1330354828 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:26.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.506+0000 7f4a82800640 1 --2- 172.21.15.167:0/1330354828 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a84160ef0 0x7f4a84161370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:26.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.506+0000 7f4a81400640 1 -- 172.21.15.167:0/1330354828 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a84160ef0 msgr2=0x7f4a84161370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:26.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.506+0000 7f4a81400640 1 --2- 172.21.15.167:0/1330354828 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a84160ef0 0x7f4a84161370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:26.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.506+0000 7f4a81e00640 1 --2- 172.21.15.167:0/1330354828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a68003a00 0x7f4a84155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:26.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.506+0000 7f4a81400640 1 -- 172.21.15.167:0/1330354828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a68003a00 msgr2=0x7f4a84155b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:26.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.506+0000 7f4a81400640 1 --2- 172.21.15.167:0/1330354828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a68003a00 0x7f4a84155b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:26.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.506+0000 7f4a81400640 1 -- 172.21.15.167:0/1330354828 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a84156080 con 0x7f4a84156270 2024-05-11T12:15:26.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.506+0000 7f4a82800640 1 --2- 172.21.15.167:0/1330354828 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a84160ef0 0x7f4a84161370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:15:26.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.506+0000 7f4a81400640 1 --2- 172.21.15.167:0/1330354828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a84156270 0x7f4a841577c0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c00e7c0 tx=0x7f4a6c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:26.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.506+0000 7f4a7a000640 1 -- 172.21.15.167:0/1330354828 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c017070 con 0x7f4a84156270 2024-05-11T12:15:26.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.506+0000 7f4a7a000640 1 -- 172.21.15.167:0/1330354828 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a6c00c8e0 con 0x7f4a84156270 2024-05-11T12:15:26.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.506+0000 7f4a83200640 1 -- 172.21.15.167:0/1330354828 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a84158010 con 0x7f4a84156270 2024-05-11T12:15:26.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.506+0000 7f4a7a000640 1 -- 172.21.15.167:0/1330354828 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c00ccd0 con 0x7f4a84156270 2024-05-11T12:15:26.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.507+0000 7f4a83200640 1 -- 172.21.15.167:0/1330354828 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a840cd540 con 0x7f4a84156270 2024-05-11T12:15:26.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.507+0000 7f4a83200640 1 -- 172.21.15.167:0/1330354828 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4a840694b0 con 0x7f4a84156270 2024-05-11T12:15:26.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.509+0000 7f4a7a000640 1 -- 172.21.15.167:0/1330354828 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4a6c01a070 con 0x7f4a84156270 2024-05-11T12:15:26.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.510+0000 7f4a7a000640 1 --2- 172.21.15.167:0/1330354828 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4a600762d0 0x7f4a60078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:26.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.510+0000 7f4a7a000640 1 -- 172.21.15.167:0/1330354828 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5507+0+0 (secure 0 0 0) 0x7f4a6c0973a0 con 0x7f4a84156270 2024-05-11T12:15:26.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.510+0000 7f4a7a000640 1 --2- 172.21.15.167:0/1330354828 >> [v2:172.21.15.167:6808/1777085638,v1:172.21.15.167:6809/1777085638] conn(0x7f4a6007c8b0 0x7f4a6007ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:26.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.510+0000 7f4a7a000640 1 -- 172.21.15.167:0/1330354828 --> [v2:172.21.15.167:6808/1777085638,v1:172.21.15.167:6809/1777085638] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4a6c021ec0 con 0x7f4a6007c8b0 2024-05-11T12:15:26.510 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.510+0000 7f4a7a000640 1 -- 172.21.15.167:0/1330354828 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=104) ==== 24+0+0 (secure 0 0 0) 0x7f4a6c09e050 con 0x7f4a84156270 2024-05-11T12:15:26.510 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.510+0000 7f4a81e00640 1 --2- 172.21.15.167:0/1330354828 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4a600762d0 0x7f4a60078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:26.510 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.510+0000 7f4a82800640 1 --2- 172.21.15.167:0/1330354828 >> [v2:172.21.15.167:6808/1777085638,v1:172.21.15.167:6809/1777085638] conn(0x7f4a6007c8b0 0x7f4a6007ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:26.510 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.511+0000 7f4a81e00640 1 --2- 172.21.15.167:0/1330354828 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4a600762d0 0x7f4a60078790 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c035000 tx=0x7f4a7c02f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:26.510 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.511+0000 7f4a82800640 1 --2- 172.21.15.167:0/1330354828 >> [v2:172.21.15.167:6808/1777085638,v1:172.21.15.167:6809/1777085638] conn(0x7f4a6007c8b0 0x7f4a6007ecf0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:26.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.512+0000 7f4a7a000640 1 -- 172.21.15.167:0/1330354828 <== osd.0 v2:172.21.15.167:6808/1777085638 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4a6c021ec0 con 0x7f4a6007c8b0 2024-05-11T12:15:26.525 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.526+0000 7f4a83200640 1 -- 172.21.15.167:0/1330354828 --> [v2:172.21.15.167:6808/1777085638,v1:172.21.15.167:6809/1777085638] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f4a840ba720 con 0x7f4a6007c8b0 2024-05-11T12:15:26.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.527+0000 7f4a7a000640 1 -- 172.21.15.167:0/1330354828 <== osd.0 v2:172.21.15.167:6808/1777085638 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f4a840ba720 con 0x7f4a6007c8b0 2024-05-11T12:15:26.526 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T12:15:26.526 INFO:teuthology.orchestra.run.smithi167.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-11T12:15:26.526 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T12:15:26.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.527+0000 7f4a83200640 1 -- 172.21.15.167:0/1330354828 >> [v2:172.21.15.167:6808/1777085638,v1:172.21.15.167:6809/1777085638] conn(0x7f4a6007c8b0 msgr2=0x7f4a6007ecf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:26.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.527+0000 7f4a83200640 1 --2- 172.21.15.167:0/1330354828 >> [v2:172.21.15.167:6808/1777085638,v1:172.21.15.167:6809/1777085638] conn(0x7f4a6007c8b0 0x7f4a6007ecf0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:26.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.528+0000 7f4a83200640 1 -- 172.21.15.167:0/1330354828 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4a600762d0 msgr2=0x7f4a60078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:26.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.528+0000 7f4a83200640 1 --2- 172.21.15.167:0/1330354828 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f4a600762d0 0x7f4a60078790 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c035000 tx=0x7f4a7c02f000 comp rx=0 tx=0).stop 2024-05-11T12:15:26.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.528+0000 7f4a83200640 1 -- 172.21.15.167:0/1330354828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a84156270 msgr2=0x7f4a841577c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:26.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.528+0000 7f4a83200640 1 --2- 172.21.15.167:0/1330354828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a84156270 0x7f4a841577c0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c00e7c0 tx=0x7f4a6c00ec80 comp rx=0 tx=0).stop 2024-05-11T12:15:26.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.528+0000 7f4a82800640 1 -- 172.21.15.167:0/1330354828 reap_dead start 2024-05-11T12:15:26.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.528+0000 7f4a83200640 1 -- 172.21.15.167:0/1330354828 shutdown_connections 2024-05-11T12:15:26.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.528+0000 7f4a83200640 1 -- 172.21.15.167:0/1330354828 >> 172.21.15.167:0/1330354828 conn(0x7f4a840b6cd0 msgr2=0x7f4a8415e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:26.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.528+0000 7f4a83200640 1 -- 172.21.15.167:0/1330354828 shutdown_connections 2024-05-11T12:15:26.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.528+0000 7f4a83200640 1 -- 172.21.15.167:0/1330354828 wait complete. 2024-05-11T12:15:26.543 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-05-11T12:15:26.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.745+0000 7f9c53c00640 1 Processor -- start 2024-05-11T12:15:26.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.746+0000 7f9c53c00640 1 -- start start 2024-05-11T12:15:26.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.747+0000 7f9c53c00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9c4c073270 0x7f9c4c15d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:26.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.747+0000 7f9c53c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9c4c0ce8f0 0x7f9c4c0ced60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:26.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.747+0000 7f9c53c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c4c15c6d0 con 0x7f9c4c073650 2024-05-11T12:15:26.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.747+0000 7f9c53c00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f9c4c06ead0 con 0x7f9c4c073270 2024-05-11T12:15:26.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.747+0000 7f9c53c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f9c4c160b00 con 0x7f9c4c0ce8f0 2024-05-11T12:15:26.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.747+0000 7f9c51200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9c4c073270 0x7f9c4c15d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:26.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.748+0000 7f9c51200640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9c4c073270 0x7f9c4c15d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:46992/0 (socket says 172.21.15.167:46992) 2024-05-11T12:15:26.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.748+0000 7f9c51200640 1 -- 172.21.15.167:0/1716517786 learned_addr learned my addr 172.21.15.167:0/1716517786 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:26.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.748+0000 7f9c51200640 1 -- 172.21.15.167:0/1716517786 >> v2:172.21.15.204:3300/0 conn(0x7f9c4c0ce8f0 msgr2=0x7f9c4c0ced60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:26.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.748+0000 7f9c51200640 1 --2- 172.21.15.167:0/1716517786 >> v2:172.21.15.204:3300/0 conn(0x7f9c4c0ce8f0 0x7f9c4c0ced60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:26.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.748+0000 7f9c51200640 1 -- 172.21.15.167:0/1716517786 >> v1:172.21.15.204:6789/0 conn(0x7f9c4c073650 legacy=0x7f9c4c06e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:26.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.748+0000 7f9c51200640 1 -- 172.21.15.167:0/1716517786 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c4c0700b0 con 0x7f9c4c073270 2024-05-11T12:15:26.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.749+0000 7f9c51200640 1 --2- 172.21.15.167:0/1716517786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c4c073270 0x7f9c4c15d170 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c009e50 tx=0x7f9c3c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=412a6b7e8b461782 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:26.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.749+0000 7f9c4b400640 1 -- 172.21.15.167:0/1716517786 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c036070 con 0x7f9c4c073270 2024-05-11T12:15:26.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.749+0000 7f9c4b400640 1 -- 172.21.15.167:0/1716517786 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c3c031600 con 0x7f9c4c073270 2024-05-11T12:15:26.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.749+0000 7f9c4b400640 1 -- 172.21.15.167:0/1716517786 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c0319d0 con 0x7f9c4c073270 2024-05-11T12:15:26.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.750+0000 7f9c53c00640 1 -- 172.21.15.167:0/1716517786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c4c073270 msgr2=0x7f9c4c15d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:26.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.750+0000 7f9c53c00640 1 --2- 172.21.15.167:0/1716517786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c4c073270 0x7f9c4c15d170 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c009e50 tx=0x7f9c3c02a6c0 comp rx=0 tx=0).stop 2024-05-11T12:15:26.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.750+0000 7f9c53c00640 1 -- 172.21.15.167:0/1716517786 shutdown_connections 2024-05-11T12:15:26.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.750+0000 7f9c53c00640 1 --2- 172.21.15.167:0/1716517786 >> v2:172.21.15.204:3300/0 conn(0x7f9c4c0ce8f0 0x7f9c4c0ced60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:26.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.750+0000 7f9c53c00640 1 --2- 172.21.15.167:0/1716517786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c4c073270 0x7f9c4c15d170 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:26.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.750+0000 7f9c53c00640 1 -- 172.21.15.167:0/1716517786 >> 172.21.15.167:0/1716517786 conn(0x7f9c4c06c450 msgr2=0x7f9c4c06c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:26.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.750+0000 7f9c53c00640 1 -- 172.21.15.167:0/1716517786 shutdown_connections 2024-05-11T12:15:26.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.750+0000 7f9c53c00640 1 -- 172.21.15.167:0/1716517786 wait complete. 2024-05-11T12:15:26.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.752+0000 7f9c53c00640 1 Processor -- start 2024-05-11T12:15:26.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.752+0000 7f9c53c00640 1 -- start start 2024-05-11T12:15:26.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.753+0000 7f9c53c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9c4c073270 0x7f9c4c15b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:26.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.753+0000 7f9c53c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c4c073650 0x7f9c4c0ae820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:26.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.753+0000 7f9c53c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c4c0ce8f0 0x7f9c4c0cf600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:26.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.753+0000 7f9c53c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4c070390 con 0x7f9c4c073650 2024-05-11T12:15:26.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.753+0000 7f9c53c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4c15cce0 con 0x7f9c4c0ce8f0 2024-05-11T12:15:26.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.753+0000 7f9c53c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f9c4c073b00 con 0x7f9c4c073270 2024-05-11T12:15:26.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.753+0000 7f9c4be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c4c073650 0x7f9c4c0ae820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:26.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.753+0000 7f9c4be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c4c073650 0x7f9c4c0ae820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:46994/0 (socket says 172.21.15.167:46994) 2024-05-11T12:15:26.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.753+0000 7f9c4be00640 1 -- 172.21.15.167:0/499986706 learned_addr learned my addr 172.21.15.167:0/499986706 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:26.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.754+0000 7f9c4be00640 1 -- 172.21.15.167:0/499986706 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9c4c073270 msgr2=0x7f9c4c15b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:26.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.754+0000 7f9c4be00640 1 --2- 172.21.15.167:0/499986706 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9c4c073270 0x7f9c4c15b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:26.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.754+0000 7f9c4be00640 1 -- 172.21.15.167:0/499986706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c4c0ce8f0 msgr2=0x7f9c4c0cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:26.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.754+0000 7f9c4be00640 1 --2- 172.21.15.167:0/499986706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c4c0ce8f0 0x7f9c4c0cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:26.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.754+0000 7f9c4be00640 1 -- 172.21.15.167:0/499986706 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c4c0b0230 con 0x7f9c4c073650 2024-05-11T12:15:26.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.754+0000 7f9c4be00640 1 --2- 172.21.15.167:0/499986706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c4c073650 0x7f9c4c0ae820 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f9c3400c460 tx=0x7f9c3400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:26.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.754+0000 7f9c48c00640 1 -- 172.21.15.167:0/499986706 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c34016020 con 0x7f9c4c073650 2024-05-11T12:15:26.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.754+0000 7f9c48c00640 1 -- 172.21.15.167:0/499986706 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c34012870 con 0x7f9c4c073650 2024-05-11T12:15:26.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.754+0000 7f9c48c00640 1 -- 172.21.15.167:0/499986706 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c34012c60 con 0x7f9c4c073650 2024-05-11T12:15:26.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.754+0000 7f9c53c00640 1 -- 172.21.15.167:0/499986706 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c4c0af190 con 0x7f9c4c073650 2024-05-11T12:15:26.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.754+0000 7f9c53c00640 1 -- 172.21.15.167:0/499986706 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c4c0af750 con 0x7f9c4c073650 2024-05-11T12:15:26.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.756+0000 7f9c53c00640 1 -- 172.21.15.167:0/499986706 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c4c0694b0 con 0x7f9c4c073650 2024-05-11T12:15:26.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.757+0000 7f9c48c00640 1 -- 172.21.15.167:0/499986706 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9c34019020 con 0x7f9c4c073650 2024-05-11T12:15:26.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.758+0000 7f9c48c00640 1 --2- 172.21.15.167:0/499986706 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f9c280762d0 0x7f9c28078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:26.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.758+0000 7f9c48c00640 1 -- 172.21.15.167:0/499986706 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5507+0+0 (secure 0 0 0) 0x7f9c34097310 con 0x7f9c4c073650 2024-05-11T12:15:26.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.758+0000 7f9c51200640 1 --2- 172.21.15.167:0/499986706 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f9c280762d0 0x7f9c28078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:26.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.759+0000 7f9c51200640 1 --2- 172.21.15.167:0/499986706 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f9c280762d0 0x7f9c28078790 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c033000 tx=0x7f9c3c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:26.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.763+0000 7f9c48c00640 1 -- 172.21.15.167:0/499986706 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9c34020030 con 0x7f9c4c073650 2024-05-11T12:15:26.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.970+0000 7f9c53c00640 1 -- 172.21.15.167:0/499986706 --> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f9c4c0d0a60 con 0x7f9c280762d0 2024-05-11T12:15:26.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.971+0000 7f9c48c00640 1 -- 172.21.15.167:0/499986706 <== mgr.4106 v2:172.21.15.167:6832/578352559 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f9c4c0d0a60 con 0x7f9c280762d0 2024-05-11T12:15:26.970 INFO:teuthology.orchestra.run.smithi167.stderr:instructed osd(s) 0 to deep-scrub 2024-05-11T12:15:26.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.973+0000 7f9c53c00640 1 -- 172.21.15.167:0/499986706 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f9c280762d0 msgr2=0x7f9c28078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:26.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.973+0000 7f9c53c00640 1 --2- 172.21.15.167:0/499986706 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f9c280762d0 0x7f9c28078790 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c033000 tx=0x7f9c3c02e000 comp rx=0 tx=0).stop 2024-05-11T12:15:26.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.973+0000 7f9c53c00640 1 -- 172.21.15.167:0/499986706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c4c073650 msgr2=0x7f9c4c0ae820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:26.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.973+0000 7f9c53c00640 1 --2- 172.21.15.167:0/499986706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c4c073650 0x7f9c4c0ae820 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f9c3400c460 tx=0x7f9c3400c920 comp rx=0 tx=0).stop 2024-05-11T12:15:26.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.974+0000 7f9c53c00640 1 -- 172.21.15.167:0/499986706 shutdown_connections 2024-05-11T12:15:26.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.974+0000 7f9c53c00640 1 --2- 172.21.15.167:0/499986706 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f9c280762d0 0x7f9c28078790 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:26.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.974+0000 7f9c53c00640 1 --2- 172.21.15.167:0/499986706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c4c0ce8f0 0x7f9c4c0cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:26.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.974+0000 7f9c53c00640 1 --2- 172.21.15.167:0/499986706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c4c073650 0x7f9c4c0ae820 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:26.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.974+0000 7f9c53c00640 1 --2- 172.21.15.167:0/499986706 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9c4c073270 0x7f9c4c15b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:26.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.974+0000 7f9c53c00640 1 -- 172.21.15.167:0/499986706 >> 172.21.15.167:0/499986706 conn(0x7f9c4c06c450 msgr2=0x7f9c4c0b2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:26.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.974+0000 7f9c53c00640 1 -- 172.21.15.167:0/499986706 shutdown_connections 2024-05-11T12:15:26.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:26.974+0000 7f9c53c00640 1 -- 172.21.15.167:0/499986706 wait complete. 2024-05-11T12:15:26.991 INFO:tasks.ceph:Scrubbing osd.1 2024-05-11T12:15:26.992 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-05-11T12:15:27.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.201+0000 7f20c7e00640 1 Processor -- start 2024-05-11T12:15:27.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.201+0000 7f20c7e00640 1 -- start start 2024-05-11T12:15:27.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.203+0000 7f20c7e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f20c815f260 0x7f20c81637e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:27.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.203+0000 7f20c7e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f20c80cfd90 0x7f20c80d3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:27.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.203+0000 7f20c7e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20c8163d20 con 0x7f20c80d4310 2024-05-11T12:15:27.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.203+0000 7f20c7e00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f20c80d3940 con 0x7f20c80cfd90 2024-05-11T12:15:27.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.203+0000 7f20c7e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f20c815f640 con 0x7f20c815f260 2024-05-11T12:15:27.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.204+0000 7f20be000640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f20c80cfd90 0x7f20c80d3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:27.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.204+0000 7f20be000640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f20c80cfd90 0x7f20c80d3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:47002/0 (socket says 172.21.15.167:47002) 2024-05-11T12:15:27.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.204+0000 7f20be000640 1 -- 172.21.15.167:0/2802044296 learned_addr learned my addr 172.21.15.167:0/2802044296 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:27.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.204+0000 7f20be000640 1 -- 172.21.15.167:0/2802044296 >> v2:172.21.15.204:3300/0 conn(0x7f20c815f260 msgr2=0x7f20c81637e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:27.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.204+0000 7f20be000640 1 --2- 172.21.15.167:0/2802044296 >> v2:172.21.15.204:3300/0 conn(0x7f20c815f260 0x7f20c81637e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:27.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.204+0000 7f20be000640 1 -- 172.21.15.167:0/2802044296 >> v1:172.21.15.204:6789/0 conn(0x7f20c80d4310 legacy=0x7f20c806d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:27.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.204+0000 7f20be000640 1 -- 172.21.15.167:0/2802044296 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20c80d2d30 con 0x7f20c80cfd90 2024-05-11T12:15:27.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.205+0000 7f20be000640 1 --2- 172.21.15.167:0/2802044296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20c80cfd90 0x7f20c80d3dd0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f20c808d4e0 tx=0x7f20b002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c92cc1140dbfe76 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:27.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.205+0000 7f20c6000640 1 -- 172.21.15.167:0/2802044296 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b0037070 con 0x7f20c80cfd90 2024-05-11T12:15:27.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.206+0000 7f20c6000640 1 -- 172.21.15.167:0/2802044296 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20b0032440 con 0x7f20c80cfd90 2024-05-11T12:15:27.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.206+0000 7f20c6000640 1 -- 172.21.15.167:0/2802044296 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b0032810 con 0x7f20c80cfd90 2024-05-11T12:15:27.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.206+0000 7f20c7e00640 1 -- 172.21.15.167:0/2802044296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20c80cfd90 msgr2=0x7f20c80d3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:27.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.206+0000 7f20c7e00640 1 --2- 172.21.15.167:0/2802044296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20c80cfd90 0x7f20c80d3dd0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f20c808d4e0 tx=0x7f20b002a1e0 comp rx=0 tx=0).stop 2024-05-11T12:15:27.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.206+0000 7f20c7e00640 1 -- 172.21.15.167:0/2802044296 shutdown_connections 2024-05-11T12:15:27.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.206+0000 7f20c7e00640 1 --2- 172.21.15.167:0/2802044296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20c80cfd90 0x7f20c80d3dd0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:27.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.206+0000 7f20c7e00640 1 --2- 172.21.15.167:0/2802044296 >> v2:172.21.15.204:3300/0 conn(0x7f20c815f260 0x7f20c81637e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:27.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.206+0000 7f20c7e00640 1 -- 172.21.15.167:0/2802044296 >> 172.21.15.167:0/2802044296 conn(0x7f20c80b6cd0 msgr2=0x7f20c80b70e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:27.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.206+0000 7f20c7e00640 1 -- 172.21.15.167:0/2802044296 shutdown_connections 2024-05-11T12:15:27.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.206+0000 7f20c7e00640 1 -- 172.21.15.167:0/2802044296 wait complete. 2024-05-11T12:15:27.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.208+0000 7f20c7e00640 1 Processor -- start 2024-05-11T12:15:27.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.208+0000 7f20c7e00640 1 -- start start 2024-05-11T12:15:27.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.209+0000 7f20c7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20c80cfd90 0x7f20c80ae650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:27.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.209+0000 7f20c7e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20c80d4310 0x7f20c814d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:27.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.209+0000 7f20c7e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20c815f260 0x7f20c80b97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:27.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.209+0000 7f20c7e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f20c80d35d0 con 0x7f20c80cfd90 2024-05-11T12:15:27.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.209+0000 7f20c7e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f20c80d32f0 con 0x7f20c815f260 2024-05-11T12:15:27.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.209+0000 7f20c7e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f20c806d770 con 0x7f20c80d4310 2024-05-11T12:15:27.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.209+0000 7f20c6a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20c80cfd90 0x7f20c80ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:27.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.210+0000 7f20c6a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20c80cfd90 0x7f20c80ae650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:47012/0 (socket says 172.21.15.167:47012) 2024-05-11T12:15:27.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.210+0000 7f20c6a00640 1 -- 172.21.15.167:0/2862878097 learned_addr learned my addr 172.21.15.167:0/2862878097 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:27.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.210+0000 7f20c6a00640 1 -- 172.21.15.167:0/2862878097 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20c80d4310 msgr2=0x7f20c814d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:27.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.210+0000 7f20c6a00640 1 --2- 172.21.15.167:0/2862878097 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20c80d4310 0x7f20c814d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:27.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.210+0000 7f20c6a00640 1 -- 172.21.15.167:0/2862878097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20c815f260 msgr2=0x7f20c80b97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:27.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.210+0000 7f20c7400640 1 --2- 172.21.15.167:0/2862878097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20c815f260 0x7f20c80b97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:27.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.210+0000 7f20c6a00640 1 --2- 172.21.15.167:0/2862878097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20c815f260 0x7f20c80b97a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:27.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.210+0000 7f20c6a00640 1 -- 172.21.15.167:0/2862878097 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20c80aeb90 con 0x7f20c80cfd90 2024-05-11T12:15:27.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.210+0000 7f20c6a00640 1 --2- 172.21.15.167:0/2862878097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20c80cfd90 0x7f20c80ae650 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f20b800b440 tx=0x7f20b800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:27.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.211+0000 7f20bf400640 1 -- 172.21.15.167:0/2862878097 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b8015020 con 0x7f20c80cfd90 2024-05-11T12:15:27.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.211+0000 7f20bf400640 1 -- 172.21.15.167:0/2862878097 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20b8011890 con 0x7f20c80cfd90 2024-05-11T12:15:27.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.211+0000 7f20bf400640 1 -- 172.21.15.167:0/2862878097 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b8011c80 con 0x7f20c80cfd90 2024-05-11T12:15:27.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.211+0000 7f20c7e00640 1 -- 172.21.15.167:0/2862878097 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20c814e200 con 0x7f20c80cfd90 2024-05-11T12:15:27.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.211+0000 7f20c7e00640 1 -- 172.21.15.167:0/2862878097 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20c814e7c0 con 0x7f20c80cfd90 2024-05-11T12:15:27.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.212+0000 7f20c7e00640 1 -- 172.21.15.167:0/2862878097 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f20c80694b0 con 0x7f20c80cfd90 2024-05-11T12:15:27.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.213+0000 7f20bf400640 1 -- 172.21.15.167:0/2862878097 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f20b8018020 con 0x7f20c80cfd90 2024-05-11T12:15:27.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.214+0000 7f20bf400640 1 --2- 172.21.15.167:0/2862878097 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f20a40762d0 0x7f20a4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:27.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.215+0000 7f20bf400640 1 -- 172.21.15.167:0/2862878097 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5507+0+0 (secure 0 0 0) 0x7f20b8097d40 con 0x7f20c80cfd90 2024-05-11T12:15:27.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.215+0000 7f20be000640 1 --2- 172.21.15.167:0/2862878097 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f20a40762d0 0x7f20a4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:27.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.215+0000 7f20bf400640 1 --2- 172.21.15.167:0/2862878097 >> [v2:172.21.15.167:6800/1480182821,v1:172.21.15.167:6801/1480182821] conn(0x7f20a407c8b0 0x7f20a407ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:27.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.215+0000 7f20bf400640 1 -- 172.21.15.167:0/2862878097 --> [v2:172.21.15.167:6800/1480182821,v1:172.21.15.167:6801/1480182821] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f20a407f3c0 con 0x7f20a407c8b0 2024-05-11T12:15:27.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.215+0000 7f20bf400640 1 -- 172.21.15.167:0/2862878097 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=104) ==== 24+0+0 (secure 0 0 0) 0x7f20b80980f0 con 0x7f20c80cfd90 2024-05-11T12:15:27.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.215+0000 7f20c7400640 1 --2- 172.21.15.167:0/2862878097 >> [v2:172.21.15.167:6800/1480182821,v1:172.21.15.167:6801/1480182821] conn(0x7f20a407c8b0 0x7f20a407ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:27.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.216+0000 7f20be000640 1 --2- 172.21.15.167:0/2862878097 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f20a40762d0 0x7f20a4078790 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f20b0034000 tx=0x7f20b002c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:27.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.216+0000 7f20c7400640 1 --2- 172.21.15.167:0/2862878097 >> [v2:172.21.15.167:6800/1480182821,v1:172.21.15.167:6801/1480182821] conn(0x7f20a407c8b0 0x7f20a407ecf0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:27.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.217+0000 7f20bf400640 1 -- 172.21.15.167:0/2862878097 <== osd.1 v2:172.21.15.167:6800/1480182821 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f20a407f3c0 con 0x7f20a407c8b0 2024-05-11T12:15:27.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.231+0000 7f20c7e00640 1 -- 172.21.15.167:0/2862878097 --> [v2:172.21.15.167:6800/1480182821,v1:172.21.15.167:6801/1480182821] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f20c815cc20 con 0x7f20a407c8b0 2024-05-11T12:15:27.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.232+0000 7f20bf400640 1 -- 172.21.15.167:0/2862878097 <== osd.1 v2:172.21.15.167:6800/1480182821 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f20c815cc20 con 0x7f20a407c8b0 2024-05-11T12:15:27.231 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T12:15:27.232 INFO:teuthology.orchestra.run.smithi167.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-11T12:15:27.232 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T12:15:27.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.232+0000 7f20c7e00640 1 -- 172.21.15.167:0/2862878097 >> [v2:172.21.15.167:6800/1480182821,v1:172.21.15.167:6801/1480182821] conn(0x7f20a407c8b0 msgr2=0x7f20a407ecf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:27.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.232+0000 7f20c7e00640 1 --2- 172.21.15.167:0/2862878097 >> [v2:172.21.15.167:6800/1480182821,v1:172.21.15.167:6801/1480182821] conn(0x7f20a407c8b0 0x7f20a407ecf0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:27.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.233+0000 7f20c7e00640 1 -- 172.21.15.167:0/2862878097 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f20a40762d0 msgr2=0x7f20a4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:27.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.233+0000 7f20c7e00640 1 --2- 172.21.15.167:0/2862878097 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f20a40762d0 0x7f20a4078790 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f20b0034000 tx=0x7f20b002c000 comp rx=0 tx=0).stop 2024-05-11T12:15:27.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.233+0000 7f20c7e00640 1 -- 172.21.15.167:0/2862878097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20c80cfd90 msgr2=0x7f20c80ae650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:27.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.233+0000 7f20c7e00640 1 --2- 172.21.15.167:0/2862878097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20c80cfd90 0x7f20c80ae650 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f20b800b440 tx=0x7f20b800b900 comp rx=0 tx=0).stop 2024-05-11T12:15:27.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.233+0000 7f20c7400640 1 -- 172.21.15.167:0/2862878097 reap_dead start 2024-05-11T12:15:27.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.233+0000 7f20c7e00640 1 -- 172.21.15.167:0/2862878097 shutdown_connections 2024-05-11T12:15:27.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.233+0000 7f20c7e00640 1 -- 172.21.15.167:0/2862878097 >> 172.21.15.167:0/2862878097 conn(0x7f20c80b6cd0 msgr2=0x7f20c815e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:27.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.233+0000 7f20c7e00640 1 -- 172.21.15.167:0/2862878097 shutdown_connections 2024-05-11T12:15:27.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.233+0000 7f20c7e00640 1 -- 172.21.15.167:0/2862878097 wait complete. 2024-05-11T12:15:27.249 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-05-11T12:15:27.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.447+0000 7f3c32000640 1 Processor -- start 2024-05-11T12:15:27.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.447+0000 7f3c32000640 1 -- start start 2024-05-11T12:15:27.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.449+0000 7f3c32000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3c2c161380 0x7f3c2c074260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:27.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.449+0000 7f3c32000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3c2c0b3d50 0x7f3c2c072950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:27.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.449+0000 7f3c32000640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c2c0b65e0 con 0x7f3c2c15ac80 2024-05-11T12:15:27.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.449+0000 7f3c32000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f3c2c06e3c0 con 0x7f3c2c161380 2024-05-11T12:15:27.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.449+0000 7f3c32000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3c2c15b060 con 0x7f3c2c0b3d50 2024-05-11T12:15:27.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.449+0000 7f3c2b400640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f3c2c15ac80 0x7f3c2c160c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:33252/0 (socket says 172.21.15.167:33252) 2024-05-11T12:15:27.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.449+0000 7f3c2b400640 1 -- 172.21.15.167:0/1621444092 learned_addr learned my addr 172.21.15.167:0/1621444092 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:15:27.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.450+0000 7f3c2be00640 1 --2- 172.21.15.167:0/1621444092 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3c2c0b3d50 0x7f3c2c072950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:27.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.450+0000 7f3c2aa00640 1 --2- 172.21.15.167:0/1621444092 >> v2:172.21.15.204:3300/0 conn(0x7f3c2c161380 0x7f3c2c074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:27.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.450+0000 7f3c2a000640 1 -- 172.21.15.167:0/1621444092 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3403859895 0 0) 0x7f3c2c0b65e0 con 0x7f3c2c15ac80 2024-05-11T12:15:27.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.450+0000 7f3c2a000640 1 -- 172.21.15.167:0/1621444092 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c10003520 con 0x7f3c2c15ac80 2024-05-11T12:15:27.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.450+0000 7f3c2a000640 1 -- 172.21.15.167:0/1621444092 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 115702207 0 0) 0x7f3c10003520 con 0x7f3c2c15ac80 2024-05-11T12:15:27.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.450+0000 7f3c2a000640 1 -- 172.21.15.167:0/1621444092 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3c2c0b3d50 msgr2=0x7f3c2c072950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:27.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.450+0000 7f3c2a000640 1 --2- 172.21.15.167:0/1621444092 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3c2c0b3d50 0x7f3c2c072950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:27.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.450+0000 7f3c2a000640 1 -- 172.21.15.167:0/1621444092 >> v2:172.21.15.204:3300/0 conn(0x7f3c2c161380 msgr2=0x7f3c2c074260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:27.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.450+0000 7f3c2a000640 1 --2- 172.21.15.167:0/1621444092 >> v2:172.21.15.204:3300/0 conn(0x7f3c2c161380 0x7f3c2c074260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:27.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.450+0000 7f3c2a000640 1 -- 172.21.15.167:0/1621444092 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c2c0b68c0 con 0x7f3c2c15ac80 2024-05-11T12:15:27.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.450+0000 7f3c2a000640 1 -- 172.21.15.167:0/1621444092 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f3c1c001a70 con 0x7f3c2c15ac80 2024-05-11T12:15:27.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.450+0000 7f3c2a000640 1 -- 172.21.15.167:0/1621444092 >> v1:172.21.15.167:6789/0 conn(0x7f3c2c15ac80 legacy=0x7f3c2c160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:27.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.450+0000 7f3c2a000640 1 --2- 172.21.15.167:0/1621444092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c10003a00 0x7f3c10003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:27.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.450+0000 7f3c2a000640 1 --2- 172.21.15.167:0/1621444092 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3c10004800 0x7f3c10004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:27.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.450+0000 7f3c2a000640 1 --2- 172.21.15.167:0/1621444092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c100051c0 0x7f3c1000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:27.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.450+0000 7f3c2a000640 1 -- 172.21.15.167:0/1621444092 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2c15b060 con 0x7f3c10003a00 2024-05-11T12:15:27.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.450+0000 7f3c2a000640 1 -- 172.21.15.167:0/1621444092 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1000c010 con 0x7f3c100051c0 2024-05-11T12:15:27.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.450+0000 7f3c2a000640 1 -- 172.21.15.167:0/1621444092 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3c1000c210 con 0x7f3c10004800 2024-05-11T12:15:27.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.450+0000 7f3c2be00640 1 --2- 172.21.15.167:0/1621444092 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3c2c0b3d50 0x7f3c2c072950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:15:27.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.450+0000 7f3c2aa00640 1 --2- 172.21.15.167:0/1621444092 >> v2:172.21.15.204:3300/0 conn(0x7f3c2c161380 0x7f3c2c074260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:15:27.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.451+0000 7f3c2b400640 1 --2- 172.21.15.167:0/1621444092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c10003a00 0x7f3c10003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:27.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.451+0000 7f3c2b400640 1 -- 172.21.15.167:0/1621444092 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3c10004800 msgr2=0x7f3c10004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:27.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.451+0000 7f3c2b400640 1 --2- 172.21.15.167:0/1621444092 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3c10004800 0x7f3c10004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:27.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.451+0000 7f3c2b400640 1 -- 172.21.15.167:0/1621444092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c100051c0 msgr2=0x7f3c1000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:27.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.451+0000 7f3c2b400640 1 --2- 172.21.15.167:0/1621444092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c100051c0 0x7f3c1000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:27.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.451+0000 7f3c2b400640 1 -- 172.21.15.167:0/1621444092 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c10003520 con 0x7f3c10003a00 2024-05-11T12:15:27.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.452+0000 7f3c2b400640 1 --2- 172.21.15.167:0/1621444092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c10003a00 0x7f3c10003e20 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c00a850 tx=0x7f3c10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a846ff0c790bd8ea server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:27.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.452+0000 7f3c2a000640 1 -- 172.21.15.167:0/1621444092 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c00a4a0 con 0x7f3c10003a00 2024-05-11T12:15:27.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.452+0000 7f3c2a000640 1 -- 172.21.15.167:0/1621444092 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c1c02f070 con 0x7f3c10003a00 2024-05-11T12:15:27.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.452+0000 7f3c2a000640 1 -- 172.21.15.167:0/1621444092 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c02bc30 con 0x7f3c10003a00 2024-05-11T12:15:27.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.453+0000 7f3c32000640 1 -- 172.21.15.167:0/1621444092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c10003a00 msgr2=0x7f3c10003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:27.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.453+0000 7f3c32000640 1 --2- 172.21.15.167:0/1621444092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c10003a00 0x7f3c10003e20 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c00a850 tx=0x7f3c10005830 comp rx=0 tx=0).stop 2024-05-11T12:15:27.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.453+0000 7f3c2be00640 1 -- 172.21.15.167:0/1621444092 reap_dead start 2024-05-11T12:15:27.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.453+0000 7f3c32000640 1 -- 172.21.15.167:0/1621444092 shutdown_connections 2024-05-11T12:15:27.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.453+0000 7f3c32000640 1 -- 172.21.15.167:0/1621444092 >> 172.21.15.167:0/1621444092 conn(0x7f3c2c06c450 msgr2=0x7f3c2c06c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:27.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.453+0000 7f3c32000640 1 -- 172.21.15.167:0/1621444092 shutdown_connections 2024-05-11T12:15:27.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.453+0000 7f3c32000640 1 -- 172.21.15.167:0/1621444092 wait complete. 2024-05-11T12:15:27.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.455+0000 7f3c32000640 1 Processor -- start 2024-05-11T12:15:27.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.455+0000 7f3c32000640 1 -- start start 2024-05-11T12:15:27.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.456+0000 7f3c32000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3c10003a00 0x7f3c2c13e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:27.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.456+0000 7f3c32000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c2c13ef30 0x7f3c2c13fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:27.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.456+0000 7f3c32000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c2c140410 0x7f3c2c0b17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:27.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.456+0000 7f3c32000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2c0b44f0 con 0x7f3c2c140410 2024-05-11T12:15:27.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.456+0000 7f3c32000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2c0b41a0 con 0x7f3c2c13ef30 2024-05-11T12:15:27.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.456+0000 7f3c32000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3c2c072950 con 0x7f3c10003a00 2024-05-11T12:15:27.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.457+0000 7f3c2be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c2c140410 0x7f3c2c0b17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:27.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.457+0000 7f3c2be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c2c140410 0x7f3c2c0b17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:47042/0 (socket says 172.21.15.167:47042) 2024-05-11T12:15:27.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.457+0000 7f3c2be00640 1 -- 172.21.15.167:0/2827204600 learned_addr learned my addr 172.21.15.167:0/2827204600 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:27.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.457+0000 7f3c2aa00640 1 --2- 172.21.15.167:0/2827204600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c2c13ef30 0x7f3c2c13fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:27.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.457+0000 7f3c2b400640 1 --2- 172.21.15.167:0/2827204600 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3c10003a00 0x7f3c2c13e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:27.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.457+0000 7f3c2be00640 1 -- 172.21.15.167:0/2827204600 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3c10003a00 msgr2=0x7f3c2c13e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:27.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.457+0000 7f3c2be00640 1 --2- 172.21.15.167:0/2827204600 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3c10003a00 0x7f3c2c13e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:27.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.457+0000 7f3c2be00640 1 -- 172.21.15.167:0/2827204600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c2c13ef30 msgr2=0x7f3c2c13fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:27.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.457+0000 7f3c2be00640 1 --2- 172.21.15.167:0/2827204600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c2c13ef30 0x7f3c2c13fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:27.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.457+0000 7f3c2be00640 1 -- 172.21.15.167:0/2827204600 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c2c142a10 con 0x7f3c2c140410 2024-05-11T12:15:27.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.457+0000 7f3c2be00640 1 --2- 172.21.15.167:0/2827204600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c2c140410 0x7f3c2c0b17b0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f3c24006fd0 tx=0x7f3c2400e830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:27.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.458+0000 7f3c23400640 1 -- 172.21.15.167:0/2827204600 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2401a070 con 0x7f3c2c140410 2024-05-11T12:15:27.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.458+0000 7f3c23400640 1 -- 172.21.15.167:0/2827204600 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c24004230 con 0x7f3c2c140410 2024-05-11T12:15:27.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.458+0000 7f3c23400640 1 -- 172.21.15.167:0/2827204600 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2400f040 con 0x7f3c2c140410 2024-05-11T12:15:27.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.458+0000 7f3c32000640 1 -- 172.21.15.167:0/2827204600 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c2c1f4ed0 con 0x7f3c2c140410 2024-05-11T12:15:27.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.458+0000 7f3c32000640 1 -- 172.21.15.167:0/2827204600 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c2c1f5490 con 0x7f3c2c140410 2024-05-11T12:15:27.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.459+0000 7f3c32000640 1 -- 172.21.15.167:0/2827204600 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c2c0694b0 con 0x7f3c2c140410 2024-05-11T12:15:27.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.460+0000 7f3c23400640 1 -- 172.21.15.167:0/2827204600 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3c2401d070 con 0x7f3c2c140410 2024-05-11T12:15:27.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.462+0000 7f3c23400640 1 --2- 172.21.15.167:0/2827204600 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3c08076300 0x7f3c080787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:27.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.462+0000 7f3c23400640 1 -- 172.21.15.167:0/2827204600 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5507+0+0 (secure 0 0 0) 0x7f3c24096340 con 0x7f3c2c140410 2024-05-11T12:15:27.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.462+0000 7f3c2b400640 1 --2- 172.21.15.167:0/2827204600 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3c08076300 0x7f3c080787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:27.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.463+0000 7f3c2b400640 1 --2- 172.21.15.167:0/2827204600 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3c08076300 0x7f3c080787c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c033000 tx=0x7f3c1c007000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:27.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.466+0000 7f3c23400640 1 -- 172.21.15.167:0/2827204600 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3c24052990 con 0x7f3c2c140410 2024-05-11T12:15:27.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.678+0000 7f3c32000640 1 -- 172.21.15.167:0/2827204600 --> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f3c2c070120 con 0x7f3c08076300 2024-05-11T12:15:27.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.679+0000 7f3c23400640 1 -- 172.21.15.167:0/2827204600 <== mgr.4106 v2:172.21.15.167:6832/578352559 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f3c2c070120 con 0x7f3c08076300 2024-05-11T12:15:27.678 INFO:teuthology.orchestra.run.smithi167.stderr:instructed osd(s) 1 to deep-scrub 2024-05-11T12:15:27.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.681+0000 7f3c32000640 1 -- 172.21.15.167:0/2827204600 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3c08076300 msgr2=0x7f3c080787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:27.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.681+0000 7f3c32000640 1 --2- 172.21.15.167:0/2827204600 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3c08076300 0x7f3c080787c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c033000 tx=0x7f3c1c007000 comp rx=0 tx=0).stop 2024-05-11T12:15:27.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.681+0000 7f3c32000640 1 -- 172.21.15.167:0/2827204600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c2c140410 msgr2=0x7f3c2c0b17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:27.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.681+0000 7f3c32000640 1 --2- 172.21.15.167:0/2827204600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c2c140410 0x7f3c2c0b17b0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f3c24006fd0 tx=0x7f3c2400e830 comp rx=0 tx=0).stop 2024-05-11T12:15:27.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.682+0000 7f3c32000640 1 -- 172.21.15.167:0/2827204600 shutdown_connections 2024-05-11T12:15:27.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.682+0000 7f3c32000640 1 --2- 172.21.15.167:0/2827204600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c2c140410 0x7f3c2c0b17b0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:27.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.682+0000 7f3c32000640 1 --2- 172.21.15.167:0/2827204600 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3c08076300 0x7f3c080787c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:27.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.682+0000 7f3c32000640 1 --2- 172.21.15.167:0/2827204600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c2c13ef30 0x7f3c2c13fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:27.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.682+0000 7f3c32000640 1 --2- 172.21.15.167:0/2827204600 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3c10003a00 0x7f3c2c13e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:27.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.682+0000 7f3c32000640 1 -- 172.21.15.167:0/2827204600 >> 172.21.15.167:0/2827204600 conn(0x7f3c2c06c450 msgr2=0x7f3c2c06e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:27.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.682+0000 7f3c32000640 1 -- 172.21.15.167:0/2827204600 shutdown_connections 2024-05-11T12:15:27.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.682+0000 7f3c32000640 1 -- 172.21.15.167:0/2827204600 wait complete. 2024-05-11T12:15:27.701 INFO:tasks.ceph:Scrubbing osd.2 2024-05-11T12:15:27.701 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-05-11T12:15:27.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.910+0000 7fcc37000640 1 Processor -- start 2024-05-11T12:15:27.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.910+0000 7fcc37000640 1 -- start start 2024-05-11T12:15:27.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.912+0000 7fcc37000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc30142740 0x7fcc300b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:27.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.912+0000 7fcc37000640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fcc300b4ca0 0x7fcc3006fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:27.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.912+0000 7fcc37000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc3006eb90 con 0x7fcc30070170 2024-05-11T12:15:27.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.912+0000 7fcc37000640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fcc3006f7a0 con 0x7fcc300b4ca0 2024-05-11T12:15:27.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.912+0000 7fcc37000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fcc3006ee70 con 0x7fcc30142740 2024-05-11T12:15:27.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.912+0000 7fcc2f400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fcc300b4ca0 0x7fcc3006fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:27.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.912+0000 7fcc2f400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fcc300b4ca0 0x7fcc3006fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:47046/0 (socket says 172.21.15.167:47046) 2024-05-11T12:15:27.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.912+0000 7fcc2f400640 1 -- 172.21.15.167:0/971186894 learned_addr learned my addr 172.21.15.167:0/971186894 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:27.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.913+0000 7fcc2f400640 1 -- 172.21.15.167:0/971186894 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc30142740 msgr2=0x7fcc300b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:27.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.913+0000 7fcc2f400640 1 --2- 172.21.15.167:0/971186894 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc30142740 0x7fcc300b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:27.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.913+0000 7fcc2f400640 1 -- 172.21.15.167:0/971186894 >> v1:172.21.15.204:6789/0 conn(0x7fcc30070170 legacy=0x7fcc30072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:27.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.913+0000 7fcc2f400640 1 -- 172.21.15.167:0/971186894 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc3006f150 con 0x7fcc300b4ca0 2024-05-11T12:15:27.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.914+0000 7fcc2f400640 1 --2- 172.21.15.167:0/971186894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc300b4ca0 0x7fcc3006fc30 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fcc18009870 tx=0x7fcc1802a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8284f37a430820b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:27.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.914+0000 7fcc2ea00640 1 -- 172.21.15.167:0/971186894 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc18037070 con 0x7fcc300b4ca0 2024-05-11T12:15:27.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.914+0000 7fcc2ea00640 1 -- 172.21.15.167:0/971186894 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc18032440 con 0x7fcc300b4ca0 2024-05-11T12:15:27.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.914+0000 7fcc2ea00640 1 -- 172.21.15.167:0/971186894 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc18032810 con 0x7fcc300b4ca0 2024-05-11T12:15:27.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.915+0000 7fcc37000640 1 -- 172.21.15.167:0/971186894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc300b4ca0 msgr2=0x7fcc3006fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:27.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.915+0000 7fcc37000640 1 --2- 172.21.15.167:0/971186894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc300b4ca0 0x7fcc3006fc30 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fcc18009870 tx=0x7fcc1802a130 comp rx=0 tx=0).stop 2024-05-11T12:15:27.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.915+0000 7fcc37000640 1 -- 172.21.15.167:0/971186894 shutdown_connections 2024-05-11T12:15:27.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.915+0000 7fcc37000640 1 --2- 172.21.15.167:0/971186894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc300b4ca0 0x7fcc3006fc30 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:27.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.915+0000 7fcc37000640 1 --2- 172.21.15.167:0/971186894 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc30142740 0x7fcc300b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:27.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.915+0000 7fcc37000640 1 -- 172.21.15.167:0/971186894 >> 172.21.15.167:0/971186894 conn(0x7fcc300d0c70 msgr2=0x7fcc300d1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:27.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.915+0000 7fcc37000640 1 -- 172.21.15.167:0/971186894 shutdown_connections 2024-05-11T12:15:27.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.915+0000 7fcc37000640 1 -- 172.21.15.167:0/971186894 wait complete. 2024-05-11T12:15:27.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.916+0000 7fcc37000640 1 Processor -- start 2024-05-11T12:15:27.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.917+0000 7fcc37000640 1 -- start start 2024-05-11T12:15:27.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.918+0000 7fcc37000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc30070170 0x7fcc3013bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:27.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.918+0000 7fcc37000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc300b4ca0 0x7fcc3006cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:27.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.918+0000 7fcc37000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc30142740 0x7fcc3006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:27.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.918+0000 7fcc37000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcc301609a0 con 0x7fcc30070170 2024-05-11T12:15:27.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.918+0000 7fcc37000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcc30142b20 con 0x7fcc30142740 2024-05-11T12:15:27.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.918+0000 7fcc37000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fcc30160b20 con 0x7fcc300b4ca0 2024-05-11T12:15:27.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.918+0000 7fcc2fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc30070170 0x7fcc3013bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:27.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.918+0000 7fcc2fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc30070170 0x7fcc3013bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:47062/0 (socket says 172.21.15.167:47062) 2024-05-11T12:15:27.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.918+0000 7fcc2fe00640 1 -- 172.21.15.167:0/2857617410 learned_addr learned my addr 172.21.15.167:0/2857617410 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:27.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.918+0000 7fcc2f400640 1 --2- 172.21.15.167:0/2857617410 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc300b4ca0 0x7fcc3006cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:27.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.918+0000 7fcc35000640 1 --2- 172.21.15.167:0/2857617410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc30142740 0x7fcc3006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:27.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.918+0000 7fcc2fe00640 1 -- 172.21.15.167:0/2857617410 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc300b4ca0 msgr2=0x7fcc3006cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:27.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.918+0000 7fcc2fe00640 1 --2- 172.21.15.167:0/2857617410 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc300b4ca0 0x7fcc3006cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:27.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.918+0000 7fcc2fe00640 1 -- 172.21.15.167:0/2857617410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc30142740 msgr2=0x7fcc3006d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:27.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.918+0000 7fcc2fe00640 1 --2- 172.21.15.167:0/2857617410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc30142740 0x7fcc3006d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:27.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.918+0000 7fcc2fe00640 1 -- 172.21.15.167:0/2857617410 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc30072150 con 0x7fcc30070170 2024-05-11T12:15:27.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.919+0000 7fcc2fe00640 1 --2- 172.21.15.167:0/2857617410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc30070170 0x7fcc3013bd60 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fcc2800ba50 tx=0x7fcc2800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:27.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.919+0000 7fcc27e00640 1 -- 172.21.15.167:0/2857617410 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc28019070 con 0x7fcc30070170 2024-05-11T12:15:27.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.919+0000 7fcc27e00640 1 -- 172.21.15.167:0/2857617410 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc2800cc00 con 0x7fcc30070170 2024-05-11T12:15:27.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.919+0000 7fcc27e00640 1 -- 172.21.15.167:0/2857617410 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc28014440 con 0x7fcc30070170 2024-05-11T12:15:27.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.919+0000 7fcc37000640 1 -- 172.21.15.167:0/2857617410 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc30072430 con 0x7fcc30070170 2024-05-11T12:15:27.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.919+0000 7fcc37000640 1 -- 172.21.15.167:0/2857617410 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc300729f0 con 0x7fcc30070170 2024-05-11T12:15:27.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.920+0000 7fcc37000640 1 -- 172.21.15.167:0/2857617410 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcc300730b0 con 0x7fcc30070170 2024-05-11T12:15:27.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.922+0000 7fcc27e00640 1 -- 172.21.15.167:0/2857617410 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fcc2801c070 con 0x7fcc30070170 2024-05-11T12:15:27.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.923+0000 7fcc27e00640 1 --2- 172.21.15.167:0/2857617410 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fcc0c0762d0 0x7fcc0c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:27.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.923+0000 7fcc27e00640 1 -- 172.21.15.167:0/2857617410 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5507+0+0 (secure 0 0 0) 0x7fcc28097390 con 0x7fcc30070170 2024-05-11T12:15:27.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.923+0000 7fcc2f400640 1 --2- 172.21.15.167:0/2857617410 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fcc0c0762d0 0x7fcc0c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:27.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.923+0000 7fcc27e00640 1 --2- 172.21.15.167:0/2857617410 >> [v2:172.21.15.167:6818/1902208221,v1:172.21.15.167:6820/1902208221] conn(0x7fcc0c07c8b0 0x7fcc0c07ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:27.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.923+0000 7fcc27e00640 1 -- 172.21.15.167:0/2857617410 --> [v2:172.21.15.167:6818/1902208221,v1:172.21.15.167:6820/1902208221] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcc28018ec0 con 0x7fcc0c07c8b0 2024-05-11T12:15:27.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.923+0000 7fcc27e00640 1 -- 172.21.15.167:0/2857617410 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=104) ==== 24+0+0 (secure 0 0 0) 0x7fcc2809e050 con 0x7fcc30070170 2024-05-11T12:15:27.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.923+0000 7fcc35000640 1 --2- 172.21.15.167:0/2857617410 >> [v2:172.21.15.167:6818/1902208221,v1:172.21.15.167:6820/1902208221] conn(0x7fcc0c07c8b0 0x7fcc0c07ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:27.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.924+0000 7fcc2f400640 1 --2- 172.21.15.167:0/2857617410 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fcc0c0762d0 0x7fcc0c078790 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fcc18034000 tx=0x7fcc1802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:27.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.924+0000 7fcc35000640 1 --2- 172.21.15.167:0/2857617410 >> [v2:172.21.15.167:6818/1902208221,v1:172.21.15.167:6820/1902208221] conn(0x7fcc0c07c8b0 0x7fcc0c07ecf0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:27.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.925+0000 7fcc27e00640 1 -- 172.21.15.167:0/2857617410 <== osd.2 v2:172.21.15.167:6818/1902208221 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcc28018ec0 con 0x7fcc0c07c8b0 2024-05-11T12:15:27.939 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.940+0000 7fcc37000640 1 -- 172.21.15.167:0/2857617410 --> [v2:172.21.15.167:6818/1902208221,v1:172.21.15.167:6820/1902208221] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fcc30069590 con 0x7fcc0c07c8b0 2024-05-11T12:15:27.940 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.940+0000 7fcc27e00640 1 -- 172.21.15.167:0/2857617410 <== osd.2 v2:172.21.15.167:6818/1902208221 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fcc30069590 con 0x7fcc0c07c8b0 2024-05-11T12:15:27.940 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T12:15:27.940 INFO:teuthology.orchestra.run.smithi167.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-11T12:15:27.940 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T12:15:27.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.941+0000 7fcc37000640 1 -- 172.21.15.167:0/2857617410 >> [v2:172.21.15.167:6818/1902208221,v1:172.21.15.167:6820/1902208221] conn(0x7fcc0c07c8b0 msgr2=0x7fcc0c07ecf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:27.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.941+0000 7fcc37000640 1 --2- 172.21.15.167:0/2857617410 >> [v2:172.21.15.167:6818/1902208221,v1:172.21.15.167:6820/1902208221] conn(0x7fcc0c07c8b0 0x7fcc0c07ecf0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:27.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.941+0000 7fcc37000640 1 -- 172.21.15.167:0/2857617410 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fcc0c0762d0 msgr2=0x7fcc0c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:27.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.941+0000 7fcc37000640 1 --2- 172.21.15.167:0/2857617410 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fcc0c0762d0 0x7fcc0c078790 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fcc18034000 tx=0x7fcc1802e000 comp rx=0 tx=0).stop 2024-05-11T12:15:27.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.941+0000 7fcc37000640 1 -- 172.21.15.167:0/2857617410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc30070170 msgr2=0x7fcc3013bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:27.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.941+0000 7fcc37000640 1 --2- 172.21.15.167:0/2857617410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc30070170 0x7fcc3013bd60 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fcc2800ba50 tx=0x7fcc2800bf10 comp rx=0 tx=0).stop 2024-05-11T12:15:27.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.941+0000 7fcc35000640 1 -- 172.21.15.167:0/2857617410 reap_dead start 2024-05-11T12:15:27.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.941+0000 7fcc37000640 1 -- 172.21.15.167:0/2857617410 shutdown_connections 2024-05-11T12:15:27.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.941+0000 7fcc37000640 1 -- 172.21.15.167:0/2857617410 >> 172.21.15.167:0/2857617410 conn(0x7fcc300d0c70 msgr2=0x7fcc300b1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:27.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.942+0000 7fcc37000640 1 -- 172.21.15.167:0/2857617410 shutdown_connections 2024-05-11T12:15:27.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:27.942+0000 7fcc37000640 1 -- 172.21.15.167:0/2857617410 wait complete. 2024-05-11T12:15:27.959 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-05-11T12:15:28.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.162+0000 7ff0be600640 1 Processor -- start 2024-05-11T12:15:28.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.163+0000 7ff0be600640 1 -- start start 2024-05-11T12:15:28.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.164+0000 7ff0be600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0b8073270 0x7ff0b815d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:28.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.164+0000 7ff0be600640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff0b8073650 0x7ff0b806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:28.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.164+0000 7ff0be600640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0b815c6d0 con 0x7ff0b80ce8f0 2024-05-11T12:15:28.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.164+0000 7ff0be600640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7ff0b8160b00 con 0x7ff0b8073650 2024-05-11T12:15:28.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.164+0000 7ff0be600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff0b815cce0 con 0x7ff0b8073270 2024-05-11T12:15:28.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.165+0000 7ff0aea00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff0b8073650 0x7ff0b806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:28.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.165+0000 7ff0b7e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7ff0b80ce8f0 0x7ff0b80ced80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:33258/0 (socket says 172.21.15.167:33258) 2024-05-11T12:15:28.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.165+0000 7ff0b7e00640 1 -- 172.21.15.167:0/1337478561 learned_addr learned my addr 172.21.15.167:0/1337478561 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:15:28.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.165+0000 7ff0b7400640 1 --2- 172.21.15.167:0/1337478561 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0b8073270 0x7ff0b815d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:28.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.165+0000 7ff0aea00640 1 -- 172.21.15.167:0/1337478561 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0b8073270 msgr2=0x7ff0b815d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:28.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.165+0000 7ff0aea00640 1 --2- 172.21.15.167:0/1337478561 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0b8073270 0x7ff0b815d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:28.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.165+0000 7ff0aea00640 1 -- 172.21.15.167:0/1337478561 >> v1:172.21.15.167:6789/0 conn(0x7ff0b80ce8f0 legacy=0x7ff0b80ced80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:28.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.165+0000 7ff0aea00640 1 -- 172.21.15.167:0/1337478561 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0b80700b0 con 0x7ff0b8073650 2024-05-11T12:15:28.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.165+0000 7ff0b6a00640 1 -- 172.21.15.167:0/1337478561 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 86479824 0 0) 0x7ff0b815c6d0 con 0x7ff0b80ce8f0 2024-05-11T12:15:28.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.165+0000 7ff0b7400640 1 --2- 172.21.15.167:0/1337478561 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0b8073270 0x7ff0b815d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:15:28.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.166+0000 7ff0aea00640 1 --2- 172.21.15.167:0/1337478561 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0b8073650 0x7ff0b806e3c0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7ff0a00097a0 tx=0x7ff0a002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65cd5b1582fa0562 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:28.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.167+0000 7ff0b6a00640 1 -- 172.21.15.167:0/1337478561 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a0037070 con 0x7ff0b8073650 2024-05-11T12:15:28.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.167+0000 7ff0b6a00640 1 -- 172.21.15.167:0/1337478561 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0a0032440 con 0x7ff0b8073650 2024-05-11T12:15:28.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.167+0000 7ff0b6a00640 1 -- 172.21.15.167:0/1337478561 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a0032810 con 0x7ff0b8073650 2024-05-11T12:15:28.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.167+0000 7ff0be600640 1 -- 172.21.15.167:0/1337478561 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0b8073650 msgr2=0x7ff0b806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:28.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.167+0000 7ff0be600640 1 --2- 172.21.15.167:0/1337478561 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0b8073650 0x7ff0b806e3c0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7ff0a00097a0 tx=0x7ff0a002a180 comp rx=0 tx=0).stop 2024-05-11T12:15:28.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.167+0000 7ff0be600640 1 -- 172.21.15.167:0/1337478561 shutdown_connections 2024-05-11T12:15:28.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.167+0000 7ff0be600640 1 --2- 172.21.15.167:0/1337478561 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0b8073650 0x7ff0b806e3c0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:28.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.167+0000 7ff0be600640 1 --2- 172.21.15.167:0/1337478561 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0b8073270 0x7ff0b815d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:28.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.167+0000 7ff0be600640 1 -- 172.21.15.167:0/1337478561 >> 172.21.15.167:0/1337478561 conn(0x7ff0b806c450 msgr2=0x7ff0b806c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:28.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.167+0000 7ff0be600640 1 -- 172.21.15.167:0/1337478561 shutdown_connections 2024-05-11T12:15:28.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.168+0000 7ff0be600640 1 -- 172.21.15.167:0/1337478561 wait complete. 2024-05-11T12:15:28.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.169+0000 7ff0be600640 1 Processor -- start 2024-05-11T12:15:28.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.170+0000 7ff0be600640 1 -- start start 2024-05-11T12:15:28.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.170+0000 7ff0be600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0b8073270 0x7ff0b813d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:28.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.170+0000 7ff0be600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0b8073650 0x7ff0b8144900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:28.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.171+0000 7ff0be600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0b80ce8f0 0x7ff0b81f8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:28.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.171+0000 7ff0be600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b806fd60 con 0x7ff0b8073650 2024-05-11T12:15:28.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.171+0000 7ff0be600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b8070390 con 0x7ff0b80ce8f0 2024-05-11T12:15:28.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.171+0000 7ff0be600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff0b8073ad0 con 0x7ff0b8073270 2024-05-11T12:15:28.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.171+0000 7ff0aea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0b8073650 0x7ff0b8144900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:28.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.171+0000 7ff0aea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0b8073650 0x7ff0b8144900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:47076/0 (socket says 172.21.15.167:47076) 2024-05-11T12:15:28.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.171+0000 7ff0aea00640 1 -- 172.21.15.167:0/1506880395 learned_addr learned my addr 172.21.15.167:0/1506880395 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:28.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.171+0000 7ff0b7400640 1 --2- 172.21.15.167:0/1506880395 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0b8073270 0x7ff0b813d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:28.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.171+0000 7ff0b7e00640 1 --2- 172.21.15.167:0/1506880395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0b80ce8f0 0x7ff0b81f8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:28.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.171+0000 7ff0aea00640 1 -- 172.21.15.167:0/1506880395 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0b8073270 msgr2=0x7ff0b813d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:28.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.171+0000 7ff0aea00640 1 --2- 172.21.15.167:0/1506880395 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0b8073270 0x7ff0b813d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:28.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.171+0000 7ff0aea00640 1 -- 172.21.15.167:0/1506880395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0b80ce8f0 msgr2=0x7ff0b81f8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:28.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.171+0000 7ff0aea00640 1 --2- 172.21.15.167:0/1506880395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0b80ce8f0 0x7ff0b81f8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:28.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.171+0000 7ff0aea00640 1 -- 172.21.15.167:0/1506880395 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0b813dcc0 con 0x7ff0b8073650 2024-05-11T12:15:28.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.171+0000 7ff0b7400640 1 --2- 172.21.15.167:0/1506880395 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0b8073270 0x7ff0b813d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:15:28.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.172+0000 7ff0aea00640 1 --2- 172.21.15.167:0/1506880395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0b8073650 0x7ff0b8144900 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7ff0a00097a0 tx=0x7ff0a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:28.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.172+0000 7ff0afe00640 1 -- 172.21.15.167:0/1506880395 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a0037040 con 0x7ff0b8073650 2024-05-11T12:15:28.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.172+0000 7ff0afe00640 1 -- 172.21.15.167:0/1506880395 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0a0004250 con 0x7ff0b8073650 2024-05-11T12:15:28.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.172+0000 7ff0afe00640 1 -- 172.21.15.167:0/1506880395 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a0032e20 con 0x7ff0b8073650 2024-05-11T12:15:28.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.172+0000 7ff0be600640 1 -- 172.21.15.167:0/1506880395 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0b8144040 con 0x7ff0b8073650 2024-05-11T12:15:28.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.172+0000 7ff0be600640 1 -- 172.21.15.167:0/1506880395 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0b8144600 con 0x7ff0b8073650 2024-05-11T12:15:28.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.173+0000 7ff0be600640 1 -- 172.21.15.167:0/1506880395 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0b80694b0 con 0x7ff0b8073650 2024-05-11T12:15:28.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.175+0000 7ff0afe00640 1 -- 172.21.15.167:0/1506880395 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff0a0030770 con 0x7ff0b8073650 2024-05-11T12:15:28.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.176+0000 7ff0afe00640 1 --2- 172.21.15.167:0/1506880395 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff0940762d0 0x7ff094078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:28.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.176+0000 7ff0afe00640 1 -- 172.21.15.167:0/1506880395 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5507+0+0 (secure 0 0 0) 0x7ff0a00b87b0 con 0x7ff0b8073650 2024-05-11T12:15:28.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.176+0000 7ff0b7400640 1 --2- 172.21.15.167:0/1506880395 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff0940762d0 0x7ff094078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:28.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.177+0000 7ff0b7400640 1 --2- 172.21.15.167:0/1506880395 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff0940762d0 0x7ff094078790 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8006fd0 tx=0x7ff0a8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:28.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.181+0000 7ff0afe00640 1 -- 172.21.15.167:0/1506880395 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff0a00821d0 con 0x7ff0b8073650 2024-05-11T12:15:28.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.392+0000 7ff0be600640 1 -- 172.21.15.167:0/1506880395 --> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7ff0b80b68b0 con 0x7ff0940762d0 2024-05-11T12:15:28.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.393+0000 7ff0afe00640 1 -- 172.21.15.167:0/1506880395 <== mgr.4106 v2:172.21.15.167:6832/578352559 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ff0b80b68b0 con 0x7ff0940762d0 2024-05-11T12:15:28.393 INFO:teuthology.orchestra.run.smithi167.stderr:instructed osd(s) 2 to deep-scrub 2024-05-11T12:15:28.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.396+0000 7ff0be600640 1 -- 172.21.15.167:0/1506880395 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff0940762d0 msgr2=0x7ff094078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:28.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.396+0000 7ff0be600640 1 --2- 172.21.15.167:0/1506880395 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff0940762d0 0x7ff094078790 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8006fd0 tx=0x7ff0a8008040 comp rx=0 tx=0).stop 2024-05-11T12:15:28.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.396+0000 7ff0be600640 1 -- 172.21.15.167:0/1506880395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0b8073650 msgr2=0x7ff0b8144900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:28.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.396+0000 7ff0be600640 1 --2- 172.21.15.167:0/1506880395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0b8073650 0x7ff0b8144900 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7ff0a00097a0 tx=0x7ff0a00046c0 comp rx=0 tx=0).stop 2024-05-11T12:15:28.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.396+0000 7ff0be600640 1 -- 172.21.15.167:0/1506880395 shutdown_connections 2024-05-11T12:15:28.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.396+0000 7ff0be600640 1 --2- 172.21.15.167:0/1506880395 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff0940762d0 0x7ff094078790 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:28.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.396+0000 7ff0be600640 1 --2- 172.21.15.167:0/1506880395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0b80ce8f0 0x7ff0b81f8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:28.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.396+0000 7ff0be600640 1 --2- 172.21.15.167:0/1506880395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0b8073650 0x7ff0b8144900 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:28.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.396+0000 7ff0be600640 1 --2- 172.21.15.167:0/1506880395 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0b8073270 0x7ff0b813d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:28.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.396+0000 7ff0be600640 1 -- 172.21.15.167:0/1506880395 >> 172.21.15.167:0/1506880395 conn(0x7ff0b806c450 msgr2=0x7ff0b8141c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:28.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.396+0000 7ff0be600640 1 -- 172.21.15.167:0/1506880395 shutdown_connections 2024-05-11T12:15:28.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.396+0000 7ff0be600640 1 -- 172.21.15.167:0/1506880395 wait complete. 2024-05-11T12:15:28.414 INFO:tasks.ceph:Scrubbing osd.3 2024-05-11T12:15:28.414 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-05-11T12:15:28.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.615+0000 7f25f5600640 1 Processor -- start 2024-05-11T12:15:28.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.615+0000 7f25f5600640 1 -- start start 2024-05-11T12:15:28.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.617+0000 7f25f5600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25f0145910 0x7f25f00d11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:28.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.617+0000 7f25f5600640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f25f00b54f0 0x7f25f00b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:28.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.617+0000 7f25f5600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25f0145cf0 con 0x7f25f00b6370 2024-05-11T12:15:28.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.617+0000 7f25f5600640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f25f00d16e0 con 0x7f25f00b54f0 2024-05-11T12:15:28.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.617+0000 7f25f5600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f25f00d0100 con 0x7f25f0145910 2024-05-11T12:15:28.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.617+0000 7f25e7e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f25f00b54f0 0x7f25f00b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:28.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.618+0000 7f25e7e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f25f00b54f0 0x7f25f00b70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:47082/0 (socket says 172.21.15.167:47082) 2024-05-11T12:15:28.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.618+0000 7f25e7e00640 1 -- 172.21.15.167:0/2826554145 learned_addr learned my addr 172.21.15.167:0/2826554145 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:28.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.618+0000 7f25eea00640 1 --2- 172.21.15.167:0/2826554145 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25f0145910 0x7f25f00d11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:28.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.618+0000 7f25e7e00640 1 -- 172.21.15.167:0/2826554145 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25f0145910 msgr2=0x7f25f00d11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:28.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.618+0000 7f25e7e00640 1 --2- 172.21.15.167:0/2826554145 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25f0145910 0x7f25f00d11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:28.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.618+0000 7f25e7e00640 1 -- 172.21.15.167:0/2826554145 >> v1:172.21.15.204:6789/0 conn(0x7f25f00b6370 legacy=0x7f25f006ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:28.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.618+0000 7f25e7e00640 1 -- 172.21.15.167:0/2826554145 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25f00d03e0 con 0x7f25f00b54f0 2024-05-11T12:15:28.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.619+0000 7f25e7e00640 1 --2- 172.21.15.167:0/2826554145 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25f00b54f0 0x7f25f00b70d0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f25d8009870 tx=0x7f25d802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d079e27ea23e7a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:28.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.619+0000 7f25ee000640 1 -- 172.21.15.167:0/2826554145 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d8037070 con 0x7f25f00b54f0 2024-05-11T12:15:28.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.619+0000 7f25ee000640 1 -- 172.21.15.167:0/2826554145 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25d8032440 con 0x7f25f00b54f0 2024-05-11T12:15:28.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.619+0000 7f25ee000640 1 -- 172.21.15.167:0/2826554145 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d8032810 con 0x7f25f00b54f0 2024-05-11T12:15:28.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.620+0000 7f25f5600640 1 -- 172.21.15.167:0/2826554145 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25f00b54f0 msgr2=0x7f25f00b70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:28.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.620+0000 7f25f5600640 1 --2- 172.21.15.167:0/2826554145 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25f00b54f0 0x7f25f00b70d0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f25d8009870 tx=0x7f25d802a1c0 comp rx=0 tx=0).stop 2024-05-11T12:15:28.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.620+0000 7f25f5600640 1 -- 172.21.15.167:0/2826554145 shutdown_connections 2024-05-11T12:15:28.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.620+0000 7f25f5600640 1 --2- 172.21.15.167:0/2826554145 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25f00b54f0 0x7f25f00b70d0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:28.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.620+0000 7f25f5600640 1 --2- 172.21.15.167:0/2826554145 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25f0145910 0x7f25f00d11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:28.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.620+0000 7f25f5600640 1 -- 172.21.15.167:0/2826554145 >> 172.21.15.167:0/2826554145 conn(0x7f25f00ce900 msgr2=0x7f25f00ced10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:28.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.620+0000 7f25f5600640 1 -- 172.21.15.167:0/2826554145 shutdown_connections 2024-05-11T12:15:28.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.620+0000 7f25f5600640 1 -- 172.21.15.167:0/2826554145 wait complete. 2024-05-11T12:15:28.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.622+0000 7f25f5600640 1 Processor -- start 2024-05-11T12:15:28.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.622+0000 7f25f5600640 1 -- start start 2024-05-11T12:15:28.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.623+0000 7f25f5600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25f00b54f0 0x7f25f01e8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:28.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.623+0000 7f25f5600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25f00b6370 0x7f25f014b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:28.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.623+0000 7f25f5600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25f0145910 0x7f25f014cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:28.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.623+0000 7f25f5600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f25f00d0c80 con 0x7f25f0145910 2024-05-11T12:15:28.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.623+0000 7f25f5600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f25f00d09a0 con 0x7f25f00b6370 2024-05-11T12:15:28.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.623+0000 7f25f5600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f25f00cfd20 con 0x7f25f00b54f0 2024-05-11T12:15:28.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.623+0000 7f25ef400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25f0145910 0x7f25f014cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:28.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.623+0000 7f25ef400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25f0145910 0x7f25f014cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:47090/0 (socket says 172.21.15.167:47090) 2024-05-11T12:15:28.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.623+0000 7f25ef400640 1 -- 172.21.15.167:0/3985637645 learned_addr learned my addr 172.21.15.167:0/3985637645 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:28.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.624+0000 7f25ef400640 1 -- 172.21.15.167:0/3985637645 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25f00b54f0 msgr2=0x7f25f01e8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:28.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.624+0000 7f25ef400640 1 --2- 172.21.15.167:0/3985637645 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25f00b54f0 0x7f25f01e8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:28.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.624+0000 7f25ef400640 1 -- 172.21.15.167:0/3985637645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25f00b6370 msgr2=0x7f25f014b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:28.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.624+0000 7f25ef400640 1 --2- 172.21.15.167:0/3985637645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25f00b6370 0x7f25f014b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:28.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.624+0000 7f25ef400640 1 -- 172.21.15.167:0/3985637645 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25f014bc10 con 0x7f25f0145910 2024-05-11T12:15:28.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.624+0000 7f25ef400640 1 --2- 172.21.15.167:0/3985637645 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25f0145910 0x7f25f014cf80 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f25e800c780 tx=0x7f25e800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:28.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.624+0000 7f25e6a00640 1 -- 172.21.15.167:0/3985637645 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e8004b60 con 0x7f25f0145910 2024-05-11T12:15:28.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.624+0000 7f25e6a00640 1 -- 172.21.15.167:0/3985637645 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25e8004d00 con 0x7f25f0145910 2024-05-11T12:15:28.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.624+0000 7f25e6a00640 1 -- 172.21.15.167:0/3985637645 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e800aa90 con 0x7f25f0145910 2024-05-11T12:15:28.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.624+0000 7f25f5600640 1 -- 172.21.15.167:0/3985637645 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25f014d4c0 con 0x7f25f0145910 2024-05-11T12:15:28.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.624+0000 7f25f5600640 1 -- 172.21.15.167:0/3985637645 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25f014c1c0 con 0x7f25f0145910 2024-05-11T12:15:28.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.625+0000 7f25f5600640 1 -- 172.21.15.167:0/3985637645 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f25f00d1860 con 0x7f25f0145910 2024-05-11T12:15:28.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.627+0000 7f25e6a00640 1 -- 172.21.15.167:0/3985637645 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f25e800ac30 con 0x7f25f0145910 2024-05-11T12:15:28.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.628+0000 7f25e6a00640 1 --2- 172.21.15.167:0/3985637645 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f25cc0762d0 0x7f25cc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:28.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.628+0000 7f25e6a00640 1 -- 172.21.15.167:0/3985637645 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5507+0+0 (secure 0 0 0) 0x7f25e8097790 con 0x7f25f0145910 2024-05-11T12:15:28.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.628+0000 7f25e6a00640 1 --2- 172.21.15.167:0/3985637645 >> [v2:172.21.15.167:6816/1903389836,v1:172.21.15.167:6817/1903389836] conn(0x7f25cc07c8b0 0x7f25cc07ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:28.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.628+0000 7f25eea00640 1 --2- 172.21.15.167:0/3985637645 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f25cc0762d0 0x7f25cc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:28.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.628+0000 7f25e6a00640 1 -- 172.21.15.167:0/3985637645 --> [v2:172.21.15.167:6816/1903389836,v1:172.21.15.167:6817/1903389836] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f25e80020b0 con 0x7f25cc07c8b0 2024-05-11T12:15:28.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.628+0000 7f25e6a00640 1 -- 172.21.15.167:0/3985637645 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=104) ==== 24+0+0 (secure 0 0 0) 0x7f25e809e050 con 0x7f25f0145910 2024-05-11T12:15:28.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.628+0000 7f25e7e00640 1 --2- 172.21.15.167:0/3985637645 >> [v2:172.21.15.167:6816/1903389836,v1:172.21.15.167:6817/1903389836] conn(0x7f25cc07c8b0 0x7f25cc07ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:28.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.629+0000 7f25eea00640 1 --2- 172.21.15.167:0/3985637645 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f25cc0762d0 0x7f25cc078790 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f25e000a810 tx=0x7f25e0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:28.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.629+0000 7f25e7e00640 1 --2- 172.21.15.167:0/3985637645 >> [v2:172.21.15.167:6816/1903389836,v1:172.21.15.167:6817/1903389836] conn(0x7f25cc07c8b0 0x7f25cc07ecf0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:28.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.630+0000 7f25e6a00640 1 -- 172.21.15.167:0/3985637645 <== osd.3 v2:172.21.15.167:6816/1903389836 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f25e80020b0 con 0x7f25cc07c8b0 2024-05-11T12:15:28.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.645+0000 7f25f5600640 1 -- 172.21.15.167:0/3985637645 --> [v2:172.21.15.167:6816/1903389836,v1:172.21.15.167:6817/1903389836] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f25f0069bc0 con 0x7f25cc07c8b0 2024-05-11T12:15:28.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.645+0000 7f25e6a00640 1 -- 172.21.15.167:0/3985637645 <== osd.3 v2:172.21.15.167:6816/1903389836 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f25f0069bc0 con 0x7f25cc07c8b0 2024-05-11T12:15:28.645 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T12:15:28.645 INFO:teuthology.orchestra.run.smithi167.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-11T12:15:28.645 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T12:15:28.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.646+0000 7f25f5600640 1 -- 172.21.15.167:0/3985637645 >> [v2:172.21.15.167:6816/1903389836,v1:172.21.15.167:6817/1903389836] conn(0x7f25cc07c8b0 msgr2=0x7f25cc07ecf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:28.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.646+0000 7f25f5600640 1 --2- 172.21.15.167:0/3985637645 >> [v2:172.21.15.167:6816/1903389836,v1:172.21.15.167:6817/1903389836] conn(0x7f25cc07c8b0 0x7f25cc07ecf0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:28.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.646+0000 7f25f5600640 1 -- 172.21.15.167:0/3985637645 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f25cc0762d0 msgr2=0x7f25cc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:28.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.646+0000 7f25f5600640 1 --2- 172.21.15.167:0/3985637645 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f25cc0762d0 0x7f25cc078790 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f25e000a810 tx=0x7f25e0008040 comp rx=0 tx=0).stop 2024-05-11T12:15:28.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.646+0000 7f25f5600640 1 -- 172.21.15.167:0/3985637645 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25f0145910 msgr2=0x7f25f014cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:28.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.646+0000 7f25f5600640 1 --2- 172.21.15.167:0/3985637645 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25f0145910 0x7f25f014cf80 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f25e800c780 tx=0x7f25e800cc40 comp rx=0 tx=0).stop 2024-05-11T12:15:28.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.646+0000 7f25ef400640 1 -- 172.21.15.167:0/3985637645 reap_dead start 2024-05-11T12:15:28.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.646+0000 7f25f5600640 1 -- 172.21.15.167:0/3985637645 shutdown_connections 2024-05-11T12:15:28.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.646+0000 7f25f5600640 1 -- 172.21.15.167:0/3985637645 >> 172.21.15.167:0/3985637645 conn(0x7f25f00ce900 msgr2=0x7f25f00b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:28.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.646+0000 7f25f5600640 1 -- 172.21.15.167:0/3985637645 shutdown_connections 2024-05-11T12:15:28.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.646+0000 7f25f5600640 1 -- 172.21.15.167:0/3985637645 wait complete. 2024-05-11T12:15:28.663 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-05-11T12:15:28.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.870+0000 7fe573a00640 1 Processor -- start 2024-05-11T12:15:28.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.871+0000 7fe573a00640 1 -- start start 2024-05-11T12:15:28.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.872+0000 7fe573a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe56c15ac80 0x7fe56c074260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:28.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.872+0000 7fe573a00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe56c0b3d50 0x7fe56c160c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:28.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.872+0000 7fe573a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe56c1611b0 0x7fe56c15d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:28.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.872+0000 7fe573a00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fe56c06e3c0 con 0x7fe56c0b3d50 2024-05-11T12:15:28.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.872+0000 7fe573a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fe56c15b060 con 0x7fe56c1611b0 2024-05-11T12:15:28.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.872+0000 7fe573a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe56c0b65e0 con 0x7fe56c15ac80 2024-05-11T12:15:28.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.872+0000 7fe563e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe56c0b3d50 0x7fe56c160c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:28.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.873+0000 7fe563e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe56c0b3d50 0x7fe56c160c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58062/0 (socket says 172.21.15.167:58062) 2024-05-11T12:15:28.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.873+0000 7fe563e00640 1 -- 172.21.15.167:0/20725900 learned_addr learned my addr 172.21.15.167:0/20725900 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:28.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.873+0000 7fe571000640 1 --2- 172.21.15.167:0/20725900 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe56c15ac80 0x7fe56c074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:28.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.873+0000 7fe563e00640 1 -- 172.21.15.167:0/20725900 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe56c15ac80 msgr2=0x7fe56c074260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:28.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.873+0000 7fe563e00640 1 --2- 172.21.15.167:0/20725900 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe56c15ac80 0x7fe56c074260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:28.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.873+0000 7fe563e00640 1 -- 172.21.15.167:0/20725900 >> v2:172.21.15.204:3300/0 conn(0x7fe56c1611b0 msgr2=0x7fe56c15d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:28.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.873+0000 7fe571a00640 1 --2- 172.21.15.167:0/20725900 >> v2:172.21.15.204:3300/0 conn(0x7fe56c1611b0 0x7fe56c15d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:28.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.873+0000 7fe563e00640 1 --2- 172.21.15.167:0/20725900 >> v2:172.21.15.204:3300/0 conn(0x7fe56c1611b0 0x7fe56c15d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:28.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.873+0000 7fe563e00640 1 -- 172.21.15.167:0/20725900 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe56c0b68c0 con 0x7fe56c0b3d50 2024-05-11T12:15:28.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.873+0000 7fe571a00640 1 --2- 172.21.15.167:0/20725900 >> v2:172.21.15.204:3300/0 conn(0x7fe56c1611b0 0x7fe56c15d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:15:28.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.874+0000 7fe563e00640 1 --2- 172.21.15.167:0/20725900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe56c0b3d50 0x7fe56c160c70 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fe56c0717b0 tx=0x7fe55402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60c24ca365633b9c server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:28.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.875+0000 7fe56be00640 1 -- 172.21.15.167:0/20725900 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe554037070 con 0x7fe56c0b3d50 2024-05-11T12:15:28.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.875+0000 7fe56be00640 1 -- 172.21.15.167:0/20725900 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe554032440 con 0x7fe56c0b3d50 2024-05-11T12:15:28.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.875+0000 7fe56be00640 1 -- 172.21.15.167:0/20725900 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe554032810 con 0x7fe56c0b3d50 2024-05-11T12:15:28.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.875+0000 7fe573a00640 1 -- 172.21.15.167:0/20725900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe56c0b3d50 msgr2=0x7fe56c160c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:28.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.875+0000 7fe573a00640 1 --2- 172.21.15.167:0/20725900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe56c0b3d50 0x7fe56c160c70 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fe56c0717b0 tx=0x7fe55402a1e0 comp rx=0 tx=0).stop 2024-05-11T12:15:28.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.875+0000 7fe573a00640 1 -- 172.21.15.167:0/20725900 shutdown_connections 2024-05-11T12:15:28.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.875+0000 7fe573a00640 1 --2- 172.21.15.167:0/20725900 >> v2:172.21.15.204:3300/0 conn(0x7fe56c1611b0 0x7fe56c15d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:28.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.875+0000 7fe573a00640 1 --2- 172.21.15.167:0/20725900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe56c0b3d50 0x7fe56c160c70 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:28.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.875+0000 7fe573a00640 1 --2- 172.21.15.167:0/20725900 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe56c15ac80 0x7fe56c074260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:28.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.875+0000 7fe573a00640 1 -- 172.21.15.167:0/20725900 >> 172.21.15.167:0/20725900 conn(0x7fe56c06c450 msgr2=0x7fe56c06c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:28.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.875+0000 7fe573a00640 1 -- 172.21.15.167:0/20725900 shutdown_connections 2024-05-11T12:15:28.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.875+0000 7fe573a00640 1 -- 172.21.15.167:0/20725900 wait complete. 2024-05-11T12:15:28.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.877+0000 7fe573a00640 1 Processor -- start 2024-05-11T12:15:28.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.877+0000 7fe573a00640 1 -- start start 2024-05-11T12:15:28.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.878+0000 7fe573a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe56c0b3d50 0x7fe56c0cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:28.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.878+0000 7fe573a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe56c15ac80 0x7fe56c0c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:28.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.878+0000 7fe573a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe56c1611b0 0x7fe56c1441a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:28.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.878+0000 7fe573a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe56c0b44f0 con 0x7fe56c1611b0 2024-05-11T12:15:28.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.878+0000 7fe573a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe56c0b41b0 con 0x7fe56c15ac80 2024-05-11T12:15:28.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.878+0000 7fe573a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe56c15dbc0 con 0x7fe56c0b3d50 2024-05-11T12:15:28.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.878+0000 7fe571a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe56c1611b0 0x7fe56c1441a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:28.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.879+0000 7fe571a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe56c1611b0 0x7fe56c1441a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58074/0 (socket says 172.21.15.167:58074) 2024-05-11T12:15:28.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.879+0000 7fe571a00640 1 -- 172.21.15.167:0/1795449662 learned_addr learned my addr 172.21.15.167:0/1795449662 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:28.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.879+0000 7fe571000640 1 --2- 172.21.15.167:0/1795449662 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe56c0b3d50 0x7fe56c0cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:28.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.879+0000 7fe563e00640 1 --2- 172.21.15.167:0/1795449662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe56c15ac80 0x7fe56c0c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:28.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.879+0000 7fe571a00640 1 -- 172.21.15.167:0/1795449662 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe56c0b3d50 msgr2=0x7fe56c0cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:28.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.879+0000 7fe571a00640 1 --2- 172.21.15.167:0/1795449662 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe56c0b3d50 0x7fe56c0cfd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:28.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.879+0000 7fe571a00640 1 -- 172.21.15.167:0/1795449662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe56c15ac80 msgr2=0x7fe56c0c5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:28.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.879+0000 7fe571a00640 1 --2- 172.21.15.167:0/1795449662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe56c15ac80 0x7fe56c0c5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:28.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.879+0000 7fe571a00640 1 -- 172.21.15.167:0/1795449662 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe56c0cf2d0 con 0x7fe56c1611b0 2024-05-11T12:15:28.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.879+0000 7fe563e00640 1 --2- 172.21.15.167:0/1795449662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe56c15ac80 0x7fe56c0c5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:15:28.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.879+0000 7fe571a00640 1 --2- 172.21.15.167:0/1795449662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe56c1611b0 0x7fe56c1441a0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fe56400c7c0 tx=0x7fe56400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:28.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.880+0000 7fe569600640 1 -- 172.21.15.167:0/1795449662 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe564015020 con 0x7fe56c1611b0 2024-05-11T12:15:28.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.880+0000 7fe569600640 1 -- 172.21.15.167:0/1795449662 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe564004230 con 0x7fe56c1611b0 2024-05-11T12:15:28.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.880+0000 7fe569600640 1 -- 172.21.15.167:0/1795449662 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe564007620 con 0x7fe56c1611b0 2024-05-11T12:15:28.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.880+0000 7fe573a00640 1 -- 172.21.15.167:0/1795449662 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe56c0afc10 con 0x7fe56c1611b0 2024-05-11T12:15:28.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.880+0000 7fe573a00640 1 -- 172.21.15.167:0/1795449662 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe56c0b01d0 con 0x7fe56c1611b0 2024-05-11T12:15:28.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.881+0000 7fe573a00640 1 -- 172.21.15.167:0/1795449662 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe56c0694b0 con 0x7fe56c1611b0 2024-05-11T12:15:28.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.882+0000 7fe569600640 1 -- 172.21.15.167:0/1795449662 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe564018020 con 0x7fe56c1611b0 2024-05-11T12:15:28.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.884+0000 7fe569600640 1 --2- 172.21.15.167:0/1795449662 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe548076300 0x7fe5480787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:28.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.884+0000 7fe569600640 1 -- 172.21.15.167:0/1795449662 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5507+0+0 (secure 0 0 0) 0x7fe5640974f0 con 0x7fe56c1611b0 2024-05-11T12:15:28.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.884+0000 7fe571000640 1 --2- 172.21.15.167:0/1795449662 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe548076300 0x7fe5480787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:28.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.885+0000 7fe571000640 1 --2- 172.21.15.167:0/1795449662 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe548076300 0x7fe5480787c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe55c009eb0 tx=0x7fe55c00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:28.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:28.889+0000 7fe569600640 1 -- 172.21.15.167:0/1795449662 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe56401f020 con 0x7fe56c1611b0 2024-05-11T12:15:29.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.094+0000 7fe573a00640 1 -- 172.21.15.167:0/1795449662 --> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fe56c070120 con 0x7fe548076300 2024-05-11T12:15:29.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.095+0000 7fe569600640 1 -- 172.21.15.167:0/1795449662 <== mgr.4106 v2:172.21.15.167:6832/578352559 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe56c070120 con 0x7fe548076300 2024-05-11T12:15:29.094 INFO:teuthology.orchestra.run.smithi167.stderr:instructed osd(s) 3 to deep-scrub 2024-05-11T12:15:29.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.098+0000 7fe573a00640 1 -- 172.21.15.167:0/1795449662 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe548076300 msgr2=0x7fe5480787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:29.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.098+0000 7fe573a00640 1 --2- 172.21.15.167:0/1795449662 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe548076300 0x7fe5480787c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe55c009eb0 tx=0x7fe55c00b040 comp rx=0 tx=0).stop 2024-05-11T12:15:29.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.098+0000 7fe573a00640 1 -- 172.21.15.167:0/1795449662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe56c1611b0 msgr2=0x7fe56c1441a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:29.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.098+0000 7fe573a00640 1 --2- 172.21.15.167:0/1795449662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe56c1611b0 0x7fe56c1441a0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fe56400c7c0 tx=0x7fe56400cc80 comp rx=0 tx=0).stop 2024-05-11T12:15:29.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.098+0000 7fe573a00640 1 -- 172.21.15.167:0/1795449662 shutdown_connections 2024-05-11T12:15:29.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.098+0000 7fe573a00640 1 --2- 172.21.15.167:0/1795449662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe56c1611b0 0x7fe56c1441a0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:29.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.098+0000 7fe573a00640 1 --2- 172.21.15.167:0/1795449662 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe548076300 0x7fe5480787c0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:29.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.098+0000 7fe573a00640 1 --2- 172.21.15.167:0/1795449662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe56c15ac80 0x7fe56c0c5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:29.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.098+0000 7fe573a00640 1 --2- 172.21.15.167:0/1795449662 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe56c0b3d50 0x7fe56c0cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:29.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.098+0000 7fe573a00640 1 -- 172.21.15.167:0/1795449662 >> 172.21.15.167:0/1795449662 conn(0x7fe56c06c450 msgr2=0x7fe56c06f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:29.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.098+0000 7fe573a00640 1 -- 172.21.15.167:0/1795449662 shutdown_connections 2024-05-11T12:15:29.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.098+0000 7fe573a00640 1 -- 172.21.15.167:0/1795449662 wait complete. 2024-05-11T12:15:29.116 INFO:tasks.ceph:Scrubbing osd.4 2024-05-11T12:15:29.116 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-05-11T12:15:29.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.324+0000 7fb8f7e00640 1 Processor -- start 2024-05-11T12:15:29.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.325+0000 7fb8f7e00640 1 -- start start 2024-05-11T12:15:29.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.326+0000 7fb8f7e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8f815f260 0x7fb8f81637e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:29.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.326+0000 7fb8f7e00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8f8163d20 con 0x7fb8f80cfd90 2024-05-11T12:15:29.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.326+0000 7fb8f7e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8f815f640 con 0x7fb8f806d060 2024-05-11T12:15:29.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.326+0000 7fb8f7e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb8f80d3940 con 0x7fb8f815f260 2024-05-11T12:15:29.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.326+0000 7fb8f6000640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fb8f80cfd90 0x7fb8f80d3dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:59374/0 (socket says 172.21.15.167:59374) 2024-05-11T12:15:29.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.327+0000 7fb8f6000640 1 -- 172.21.15.167:0/4169443602 learned_addr learned my addr 172.21.15.167:0/4169443602 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:15:29.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.327+0000 7fb8f6a00640 1 --2- 172.21.15.167:0/4169443602 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8f815f260 0x7fb8f81637e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:29.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.327+0000 7fb8f5600640 1 -- 172.21.15.167:0/4169443602 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1270055233 0 0) 0x7fb8e0001110 con 0x7fb8f80cfd90 2024-05-11T12:15:29.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.327+0000 7fb8f5600640 1 -- 172.21.15.167:0/4169443602 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb8d8003520 con 0x7fb8f80cfd90 2024-05-11T12:15:29.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.327+0000 7fb8f5600640 1 -- 172.21.15.167:0/4169443602 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 663457834 0 0) 0x7fb8d8003520 con 0x7fb8f80cfd90 2024-05-11T12:15:29.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.327+0000 7fb8f5600640 1 -- 172.21.15.167:0/4169443602 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8f815f260 msgr2=0x7fb8f81637e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:29.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.327+0000 7fb8f5600640 1 --2- 172.21.15.167:0/4169443602 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8f815f260 0x7fb8f81637e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:29.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.327+0000 7fb8f5600640 1 -- 172.21.15.167:0/4169443602 >> v1:172.21.15.204:6789/0 conn(0x7fb8f806d060 legacy=0x7fb8f80b8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:29.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.327+0000 7fb8f5600640 1 -- 172.21.15.167:0/4169443602 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8f80d2d30 con 0x7fb8f80cfd90 2024-05-11T12:15:29.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.327+0000 7fb8f5600640 1 -- 172.21.15.167:0/4169443602 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7fb8e00018c0 con 0x7fb8f80cfd90 2024-05-11T12:15:29.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.328+0000 7fb8f5600640 1 -- 172.21.15.167:0/4169443602 >> v1:172.21.15.167:6789/0 conn(0x7fb8f80cfd90 legacy=0x7fb8f80d3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:29.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.328+0000 7fb8f5600640 1 --2- 172.21.15.167:0/4169443602 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8f815f260 0x7fb8d8003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:29.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.328+0000 7fb8f5600640 1 --2- 172.21.15.167:0/4169443602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8d8004000 0x7fb8d8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:29.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.328+0000 7fb8f5600640 1 --2- 172.21.15.167:0/4169443602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8d8004d40 0x7fb8d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:29.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.328+0000 7fb8f5600640 1 -- 172.21.15.167:0/4169443602 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f80d3940 con 0x7fb8d8004000 2024-05-11T12:15:29.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.328+0000 7fb8f5600640 1 -- 172.21.15.167:0/4169443602 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d800bb50 con 0x7fb8d8004d40 2024-05-11T12:15:29.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.328+0000 7fb8f5600640 1 -- 172.21.15.167:0/4169443602 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb8d800bd50 con 0x7fb8f815f260 2024-05-11T12:15:29.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.328+0000 7fb8f6000640 1 --2- 172.21.15.167:0/4169443602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8d8004000 0x7fb8d8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:29.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.328+0000 7fb8f7400640 1 --2- 172.21.15.167:0/4169443602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8d8004d40 0x7fb8d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:29.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.328+0000 7fb8f6a00640 1 --2- 172.21.15.167:0/4169443602 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8f815f260 0x7fb8d8003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:29.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.328+0000 7fb8f6000640 1 -- 172.21.15.167:0/4169443602 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8f815f260 msgr2=0x7fb8d8003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:29.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.328+0000 7fb8f6000640 1 --2- 172.21.15.167:0/4169443602 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8f815f260 0x7fb8d8003a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:29.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.328+0000 7fb8f6000640 1 -- 172.21.15.167:0/4169443602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8d8004d40 msgr2=0x7fb8d80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:29.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.328+0000 7fb8f6000640 1 --2- 172.21.15.167:0/4169443602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8d8004d40 0x7fb8d80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:29.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.328+0000 7fb8f6000640 1 -- 172.21.15.167:0/4169443602 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8d8003520 con 0x7fb8d8004000 2024-05-11T12:15:29.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.329+0000 7fb8f6000640 1 --2- 172.21.15.167:0/4169443602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8d8004000 0x7fb8d8004800 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fb8e000c490 tx=0x7fb8d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d97136f6c9ddb7ee server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:29.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.330+0000 7fb8f5600640 1 -- 172.21.15.167:0/4169443602 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0035070 con 0x7fb8d8004000 2024-05-11T12:15:29.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.330+0000 7fb8f5600640 1 -- 172.21.15.167:0/4169443602 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8e0005ad0 con 0x7fb8d8004000 2024-05-11T12:15:29.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.330+0000 7fb8f5600640 1 -- 172.21.15.167:0/4169443602 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e000c0e0 con 0x7fb8d8004000 2024-05-11T12:15:29.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.330+0000 7fb8f7e00640 1 -- 172.21.15.167:0/4169443602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8d8004000 msgr2=0x7fb8d8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:29.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.330+0000 7fb8f7e00640 1 --2- 172.21.15.167:0/4169443602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8d8004000 0x7fb8d8004800 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fb8e000c490 tx=0x7fb8d8005830 comp rx=0 tx=0).stop 2024-05-11T12:15:29.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.330+0000 7fb8f7400640 1 -- 172.21.15.167:0/4169443602 reap_dead start 2024-05-11T12:15:29.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.330+0000 7fb8f7e00640 1 -- 172.21.15.167:0/4169443602 shutdown_connections 2024-05-11T12:15:29.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.330+0000 7fb8f7e00640 1 -- 172.21.15.167:0/4169443602 >> 172.21.15.167:0/4169443602 conn(0x7fb8f80b6cd0 msgr2=0x7fb8f80b70e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:29.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.331+0000 7fb8f7e00640 1 -- 172.21.15.167:0/4169443602 shutdown_connections 2024-05-11T12:15:29.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.331+0000 7fb8f7e00640 1 -- 172.21.15.167:0/4169443602 wait complete. 2024-05-11T12:15:29.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.332+0000 7fb8f7e00640 1 Processor -- start 2024-05-11T12:15:29.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.332+0000 7fb8f7e00640 1 -- start start 2024-05-11T12:15:29.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.333+0000 7fb8f7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8d8004000 0x7fb8f81f9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:29.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.333+0000 7fb8f7e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8f80b8600 0x7fb8f80b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:29.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.333+0000 7fb8f7e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8f8158010 0x7fb8f8158490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:29.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.333+0000 7fb8f7e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f80d35d0 con 0x7fb8d8004000 2024-05-11T12:15:29.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.333+0000 7fb8f7e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f80d32f0 con 0x7fb8f8158010 2024-05-11T12:15:29.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.333+0000 7fb8f7e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb8f80d44e0 con 0x7fb8f80b8600 2024-05-11T12:15:29.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.334+0000 7fb8f6a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8d8004000 0x7fb8f81f9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:29.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.334+0000 7fb8f6a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8d8004000 0x7fb8f81f9410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58100/0 (socket says 172.21.15.167:58100) 2024-05-11T12:15:29.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.334+0000 7fb8f6a00640 1 -- 172.21.15.167:0/372684484 learned_addr learned my addr 172.21.15.167:0/372684484 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:29.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.334+0000 7fb8f6a00640 1 -- 172.21.15.167:0/372684484 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8f80b8600 msgr2=0x7fb8f80b8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:29.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.334+0000 7fb8f6a00640 1 --2- 172.21.15.167:0/372684484 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8f80b8600 0x7fb8f80b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:29.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.334+0000 7fb8f6a00640 1 -- 172.21.15.167:0/372684484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8f8158010 msgr2=0x7fb8f8158490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:29.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.334+0000 7fb8f6a00640 1 --2- 172.21.15.167:0/372684484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8f8158010 0x7fb8f8158490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:29.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.334+0000 7fb8f6a00640 1 -- 172.21.15.167:0/372684484 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8f81589d0 con 0x7fb8d8004000 2024-05-11T12:15:29.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.334+0000 7fb8f6a00640 1 --2- 172.21.15.167:0/372684484 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8d8004000 0x7fb8f81f9410 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8010860 tx=0x7fb8e8010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:29.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.335+0000 7fb8eea00640 1 -- 172.21.15.167:0/372684484 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e800f040 con 0x7fb8d8004000 2024-05-11T12:15:29.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.335+0000 7fb8eea00640 1 -- 172.21.15.167:0/372684484 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8e800eb00 con 0x7fb8d8004000 2024-05-11T12:15:29.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.335+0000 7fb8f7e00640 1 -- 172.21.15.167:0/372684484 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8f80cc300 con 0x7fb8d8004000 2024-05-11T12:15:29.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.335+0000 7fb8eea00640 1 -- 172.21.15.167:0/372684484 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e801c400 con 0x7fb8d8004000 2024-05-11T12:15:29.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.335+0000 7fb8f7e00640 1 -- 172.21.15.167:0/372684484 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8f80cc8c0 con 0x7fb8d8004000 2024-05-11T12:15:29.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.336+0000 7fb8f7e00640 1 -- 172.21.15.167:0/372684484 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb8f80cc0b0 con 0x7fb8d8004000 2024-05-11T12:15:29.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.337+0000 7fb8eea00640 1 -- 172.21.15.167:0/372684484 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb8e801c5a0 con 0x7fb8d8004000 2024-05-11T12:15:29.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.338+0000 7fb8eea00640 1 --2- 172.21.15.167:0/372684484 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb8d4076300 0x7fb8d40787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:29.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.338+0000 7fb8eea00640 1 -- 172.21.15.167:0/372684484 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5507+0+0 (secure 0 0 0) 0x7fb8e8097ac0 con 0x7fb8d8004000 2024-05-11T12:15:29.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.338+0000 7fb8eea00640 1 --2- 172.21.15.167:0/372684484 >> [v2:172.21.15.204:6815/2030699881,v1:172.21.15.204:6817/2030699881] conn(0x7fb8d407c8e0 0x7fb8d407ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:29.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.338+0000 7fb8eea00640 1 -- 172.21.15.167:0/372684484 --> [v2:172.21.15.204:6815/2030699881,v1:172.21.15.204:6817/2030699881] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb8d407f3f0 con 0x7fb8d407c8e0 2024-05-11T12:15:29.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.338+0000 7fb8f6000640 1 --2- 172.21.15.167:0/372684484 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb8d4076300 0x7fb8d40787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:29.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.338+0000 7fb8eea00640 1 -- 172.21.15.167:0/372684484 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=104) ==== 24+0+0 (secure 0 0 0) 0x7fb8e8099e60 con 0x7fb8d8004000 2024-05-11T12:15:29.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.339+0000 7fb8f7400640 1 --2- 172.21.15.167:0/372684484 >> [v2:172.21.15.204:6815/2030699881,v1:172.21.15.204:6817/2030699881] conn(0x7fb8d407c8e0 0x7fb8d407ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:29.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.339+0000 7fb8f6000640 1 --2- 172.21.15.167:0/372684484 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb8d4076300 0x7fb8d40787c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb8e000c490 tx=0x7fb8e0007000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:29.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.340+0000 7fb8f7400640 1 --2- 172.21.15.167:0/372684484 >> [v2:172.21.15.204:6815/2030699881,v1:172.21.15.204:6817/2030699881] conn(0x7fb8d407c8e0 0x7fb8d407ed20 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:29.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.342+0000 7fb8eea00640 1 -- 172.21.15.167:0/372684484 <== osd.4 v2:172.21.15.204:6815/2030699881 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb8d407f3f0 con 0x7fb8d407c8e0 2024-05-11T12:15:29.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.357+0000 7fb8f7e00640 1 -- 172.21.15.167:0/372684484 --> [v2:172.21.15.204:6815/2030699881,v1:172.21.15.204:6817/2030699881] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fb8f80694b0 con 0x7fb8d407c8e0 2024-05-11T12:15:29.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.358+0000 7fb8eea00640 1 -- 172.21.15.167:0/372684484 <== osd.4 v2:172.21.15.204:6815/2030699881 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fb8f80694b0 con 0x7fb8d407c8e0 2024-05-11T12:15:29.357 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T12:15:29.357 INFO:teuthology.orchestra.run.smithi167.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-11T12:15:29.357 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T12:15:29.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.358+0000 7fb8f7e00640 1 -- 172.21.15.167:0/372684484 >> [v2:172.21.15.204:6815/2030699881,v1:172.21.15.204:6817/2030699881] conn(0x7fb8d407c8e0 msgr2=0x7fb8d407ed20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:29.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.358+0000 7fb8f7e00640 1 --2- 172.21.15.167:0/372684484 >> [v2:172.21.15.204:6815/2030699881,v1:172.21.15.204:6817/2030699881] conn(0x7fb8d407c8e0 0x7fb8d407ed20 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:29.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.358+0000 7fb8f7e00640 1 -- 172.21.15.167:0/372684484 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb8d4076300 msgr2=0x7fb8d40787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:29.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.358+0000 7fb8f7e00640 1 --2- 172.21.15.167:0/372684484 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fb8d4076300 0x7fb8d40787c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb8e000c490 tx=0x7fb8e0007000 comp rx=0 tx=0).stop 2024-05-11T12:15:29.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.358+0000 7fb8f7e00640 1 -- 172.21.15.167:0/372684484 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8d8004000 msgr2=0x7fb8f81f9410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:29.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.358+0000 7fb8f7e00640 1 --2- 172.21.15.167:0/372684484 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8d8004000 0x7fb8f81f9410 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8010860 tx=0x7fb8e8010d20 comp rx=0 tx=0).stop 2024-05-11T12:15:29.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.359+0000 7fb8f7400640 1 -- 172.21.15.167:0/372684484 reap_dead start 2024-05-11T12:15:29.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.359+0000 7fb8f7e00640 1 -- 172.21.15.167:0/372684484 shutdown_connections 2024-05-11T12:15:29.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.359+0000 7fb8f7e00640 1 -- 172.21.15.167:0/372684484 >> 172.21.15.167:0/372684484 conn(0x7fb8f80b6cd0 msgr2=0x7fb8f815cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:29.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.359+0000 7fb8f7e00640 1 -- 172.21.15.167:0/372684484 shutdown_connections 2024-05-11T12:15:29.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.359+0000 7fb8f7e00640 1 -- 172.21.15.167:0/372684484 wait complete. 2024-05-11T12:15:29.378 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-05-11T12:15:29.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.574+0000 7f3b38e00640 1 Processor -- start 2024-05-11T12:15:29.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.575+0000 7f3b38e00640 1 -- start start 2024-05-11T12:15:29.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.576+0000 7f3b38e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b3413fc30 0x7f3b34160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:29.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.576+0000 7f3b38e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3b34140010 0x7f3b34161020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:29.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.576+0000 7f3b38e00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b340d0220 con 0x7f3b340cf580 2024-05-11T12:15:29.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.576+0000 7f3b38e00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f3b341619d0 con 0x7f3b34140010 2024-05-11T12:15:29.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.576+0000 7f3b38e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3b340d0ac0 con 0x7f3b3413fc30 2024-05-11T12:15:29.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.577+0000 7f3b2aa00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3b34140010 0x7f3b34161020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:29.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.577+0000 7f3b33e00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f3b340cf580 0x7f3b340cf9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:59386/0 (socket says 172.21.15.167:59386) 2024-05-11T12:15:29.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.577+0000 7f3b33e00640 1 -- 172.21.15.167:0/3287836915 learned_addr learned my addr 172.21.15.167:0/3287836915 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:15:29.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.577+0000 7f3b33400640 1 --2- 172.21.15.167:0/3287836915 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b3413fc30 0x7f3b34160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:29.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.577+0000 7f3b2aa00640 1 -- 172.21.15.167:0/3287836915 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b3413fc30 msgr2=0x7f3b34160ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:29.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.577+0000 7f3b2aa00640 1 --2- 172.21.15.167:0/3287836915 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b3413fc30 0x7f3b34160ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:29.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.577+0000 7f3b2aa00640 1 -- 172.21.15.167:0/3287836915 >> v1:172.21.15.167:6789/0 conn(0x7f3b340cf580 legacy=0x7f3b340cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:29.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.577+0000 7f3b2aa00640 1 -- 172.21.15.167:0/3287836915 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b340d0500 con 0x7f3b34140010 2024-05-11T12:15:29.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.577+0000 7f3b32a00640 1 -- 172.21.15.167:0/3287836915 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 831337901 0 0) 0x7f3b340d0220 con 0x7f3b340cf580 2024-05-11T12:15:29.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.577+0000 7f3b33400640 1 --2- 172.21.15.167:0/3287836915 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b3413fc30 0x7f3b34160ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:15:29.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.578+0000 7f3b2aa00640 1 --2- 172.21.15.167:0/3287836915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b34140010 0x7f3b34161020 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c0097a0 tx=0x7f3b1c02a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb2d65843615a786 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:29.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.579+0000 7f3b32a00640 1 -- 172.21.15.167:0/3287836915 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c037070 con 0x7f3b34140010 2024-05-11T12:15:29.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.579+0000 7f3b32a00640 1 -- 172.21.15.167:0/3287836915 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b1c02adb0 con 0x7f3b34140010 2024-05-11T12:15:29.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.579+0000 7f3b32a00640 1 -- 172.21.15.167:0/3287836915 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c032660 con 0x7f3b34140010 2024-05-11T12:15:29.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.579+0000 7f3b38e00640 1 -- 172.21.15.167:0/3287836915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b34140010 msgr2=0x7f3b34161020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:29.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.579+0000 7f3b38e00640 1 --2- 172.21.15.167:0/3287836915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b34140010 0x7f3b34161020 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c0097a0 tx=0x7f3b1c02a060 comp rx=0 tx=0).stop 2024-05-11T12:15:29.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.579+0000 7f3b38e00640 1 -- 172.21.15.167:0/3287836915 shutdown_connections 2024-05-11T12:15:29.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.579+0000 7f3b38e00640 1 --2- 172.21.15.167:0/3287836915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b34140010 0x7f3b34161020 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:29.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.580+0000 7f3b38e00640 1 --2- 172.21.15.167:0/3287836915 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b3413fc30 0x7f3b34160ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:29.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.580+0000 7f3b38e00640 1 -- 172.21.15.167:0/3287836915 >> 172.21.15.167:0/3287836915 conn(0x7f3b3406c450 msgr2=0x7f3b3406c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:29.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.580+0000 7f3b38e00640 1 -- 172.21.15.167:0/3287836915 shutdown_connections 2024-05-11T12:15:29.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.580+0000 7f3b38e00640 1 -- 172.21.15.167:0/3287836915 wait complete. 2024-05-11T12:15:29.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.581+0000 7f3b38e00640 1 Processor -- start 2024-05-11T12:15:29.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.582+0000 7f3b38e00640 1 -- start start 2024-05-11T12:15:29.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.583+0000 7f3b38e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b340cf580 0x7f3b341f4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:29.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.583+0000 7f3b38e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b3413fc30 0x7f3b34139d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:29.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.583+0000 7f3b38e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b34140010 0x7f3b3415bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:29.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.583+0000 7f3b38e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3406eab0 con 0x7f3b340cf580 2024-05-11T12:15:29.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.583+0000 7f3b38e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3406e590 con 0x7f3b3413fc30 2024-05-11T12:15:29.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.583+0000 7f3b38e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3b3406ec30 con 0x7f3b34140010 2024-05-11T12:15:29.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.583+0000 7f3b33400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b340cf580 0x7f3b341f4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:29.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.583+0000 7f3b33400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b340cf580 0x7f3b341f4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58120/0 (socket says 172.21.15.167:58120) 2024-05-11T12:15:29.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.583+0000 7f3b33400640 1 -- 172.21.15.167:0/3089061806 learned_addr learned my addr 172.21.15.167:0/3089061806 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:29.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.583+0000 7f3b33400640 1 -- 172.21.15.167:0/3089061806 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b34140010 msgr2=0x7f3b3415bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:29.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.583+0000 7f3b33400640 1 --2- 172.21.15.167:0/3089061806 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b34140010 0x7f3b3415bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:29.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.583+0000 7f3b33400640 1 -- 172.21.15.167:0/3089061806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b3413fc30 msgr2=0x7f3b34139d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:29.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.583+0000 7f3b33400640 1 --2- 172.21.15.167:0/3089061806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b3413fc30 0x7f3b34139d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:29.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.583+0000 7f3b33400640 1 -- 172.21.15.167:0/3089061806 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b340729b0 con 0x7f3b340cf580 2024-05-11T12:15:29.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.584+0000 7f3b33400640 1 --2- 172.21.15.167:0/3089061806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b340cf580 0x7f3b341f4ea0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f3b2400c7f0 tx=0x7f3b2400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:29.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.584+0000 7f3b2be00640 1 -- 172.21.15.167:0/3089061806 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b24015070 con 0x7f3b340cf580 2024-05-11T12:15:29.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.584+0000 7f3b2be00640 1 -- 172.21.15.167:0/3089061806 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b24007a30 con 0x7f3b340cf580 2024-05-11T12:15:29.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.584+0000 7f3b2be00640 1 -- 172.21.15.167:0/3089061806 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b24007de0 con 0x7f3b340cf580 2024-05-11T12:15:29.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.584+0000 7f3b38e00640 1 -- 172.21.15.167:0/3089061806 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b34072c90 con 0x7f3b340cf580 2024-05-11T12:15:29.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.584+0000 7f3b38e00640 1 -- 172.21.15.167:0/3089061806 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b34073250 con 0x7f3b340cf580 2024-05-11T12:15:29.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.585+0000 7f3b38e00640 1 -- 172.21.15.167:0/3089061806 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b340b10d0 con 0x7f3b340cf580 2024-05-11T12:15:29.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.586+0000 7f3b2be00640 1 -- 172.21.15.167:0/3089061806 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3b24018070 con 0x7f3b340cf580 2024-05-11T12:15:29.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.587+0000 7f3b2be00640 1 --2- 172.21.15.167:0/3089061806 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3b100762d0 0x7f3b10078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:29.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.587+0000 7f3b2be00640 1 -- 172.21.15.167:0/3089061806 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5507+0+0 (secure 0 0 0) 0x7f3b240974e0 con 0x7f3b340cf580 2024-05-11T12:15:29.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.587+0000 7f3b2aa00640 1 --2- 172.21.15.167:0/3089061806 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3b100762d0 0x7f3b10078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:29.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.588+0000 7f3b2aa00640 1 --2- 172.21.15.167:0/3089061806 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3b100762d0 0x7f3b10078790 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c034000 tx=0x7f3b1c02c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:29.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.592+0000 7f3b2be00640 1 -- 172.21.15.167:0/3089061806 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3b340b10d0 con 0x7f3b340cf580 2024-05-11T12:15:29.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.796+0000 7f3b38e00640 1 -- 172.21.15.167:0/3089061806 --> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f3b340ce040 con 0x7f3b100762d0 2024-05-11T12:15:29.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.797+0000 7f3b2be00640 1 -- 172.21.15.167:0/3089061806 <== mgr.4106 v2:172.21.15.167:6832/578352559 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f3b340ce040 con 0x7f3b100762d0 2024-05-11T12:15:29.797 INFO:teuthology.orchestra.run.smithi167.stderr:instructed osd(s) 4 to deep-scrub 2024-05-11T12:15:29.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.800+0000 7f3b38e00640 1 -- 172.21.15.167:0/3089061806 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3b100762d0 msgr2=0x7f3b10078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:29.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.800+0000 7f3b38e00640 1 --2- 172.21.15.167:0/3089061806 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3b100762d0 0x7f3b10078790 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c034000 tx=0x7f3b1c02c000 comp rx=0 tx=0).stop 2024-05-11T12:15:29.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.800+0000 7f3b38e00640 1 -- 172.21.15.167:0/3089061806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b340cf580 msgr2=0x7f3b341f4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:29.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.800+0000 7f3b38e00640 1 --2- 172.21.15.167:0/3089061806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b340cf580 0x7f3b341f4ea0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f3b2400c7f0 tx=0x7f3b2400cbf0 comp rx=0 tx=0).stop 2024-05-11T12:15:29.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.800+0000 7f3b38e00640 1 -- 172.21.15.167:0/3089061806 shutdown_connections 2024-05-11T12:15:29.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.800+0000 7f3b38e00640 1 --2- 172.21.15.167:0/3089061806 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b34140010 0x7f3b3415bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:29.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.800+0000 7f3b38e00640 1 --2- 172.21.15.167:0/3089061806 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f3b100762d0 0x7f3b10078790 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:29.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.800+0000 7f3b38e00640 1 --2- 172.21.15.167:0/3089061806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b3413fc30 0x7f3b34139d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:29.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.800+0000 7f3b38e00640 1 --2- 172.21.15.167:0/3089061806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b340cf580 0x7f3b341f4ea0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:29.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.800+0000 7f3b38e00640 1 -- 172.21.15.167:0/3089061806 >> 172.21.15.167:0/3089061806 conn(0x7f3b3406c450 msgr2=0x7f3b3413ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:29.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.800+0000 7f3b38e00640 1 -- 172.21.15.167:0/3089061806 shutdown_connections 2024-05-11T12:15:29.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:29.800+0000 7f3b38e00640 1 -- 172.21.15.167:0/3089061806 wait complete. 2024-05-11T12:15:29.818 INFO:tasks.ceph:Scrubbing osd.5 2024-05-11T12:15:29.818 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-05-11T12:15:30.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.029+0000 7f022fe00640 1 Processor -- start 2024-05-11T12:15:30.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.029+0000 7f022fe00640 1 -- start start 2024-05-11T12:15:30.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.030+0000 7f022fe00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f023013ffd0 0x7f0230143170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:30.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.031+0000 7f022fe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f023006db70 0x7f02300cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:30.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.031+0000 7f022fe00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02301436b0 con 0x7f02301428d0 2024-05-11T12:15:30.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.031+0000 7f022fe00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f0230160c40 con 0x7f023013ffd0 2024-05-11T12:15:30.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.031+0000 7f022fe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0230161aa0 con 0x7f023006db70 2024-05-11T12:15:30.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.031+0000 7f022e000640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f02301428d0 0x7f023006d460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:59392/0 (socket says 172.21.15.167:59392) 2024-05-11T12:15:30.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.031+0000 7f022e000640 1 -- 172.21.15.167:0/3810976009 learned_addr learned my addr 172.21.15.167:0/3810976009 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:15:30.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.031+0000 7f022ea00640 1 --2- 172.21.15.167:0/3810976009 >> v2:172.21.15.204:3300/0 conn(0x7f023013ffd0 0x7f0230143170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:30.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.031+0000 7f022f400640 1 --2- 172.21.15.167:0/3810976009 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f023006db70 0x7f02300cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:30.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.031+0000 7f022d600640 1 -- 172.21.15.167:0/3810976009 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4110999793 0 0) 0x7f021c001110 con 0x7f02301428d0 2024-05-11T12:15:30.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.032+0000 7f022d600640 1 -- 172.21.15.167:0/3810976009 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0214003520 con 0x7f02301428d0 2024-05-11T12:15:30.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.032+0000 7f022d600640 1 -- 172.21.15.167:0/3810976009 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3967127701 0 0) 0x7f0214003520 con 0x7f02301428d0 2024-05-11T12:15:30.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.032+0000 7f022d600640 1 -- 172.21.15.167:0/3810976009 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f023006db70 msgr2=0x7f02300cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:30.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.032+0000 7f022d600640 1 --2- 172.21.15.167:0/3810976009 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f023006db70 0x7f02300cf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:30.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.032+0000 7f022d600640 1 -- 172.21.15.167:0/3810976009 >> v2:172.21.15.204:3300/0 conn(0x7f023013ffd0 msgr2=0x7f0230143170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:30.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.032+0000 7f022d600640 1 --2- 172.21.15.167:0/3810976009 >> v2:172.21.15.204:3300/0 conn(0x7f023013ffd0 0x7f0230143170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:30.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.032+0000 7f022d600640 1 -- 172.21.15.167:0/3810976009 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0230160e90 con 0x7f02301428d0 2024-05-11T12:15:30.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.032+0000 7f022d600640 1 -- 172.21.15.167:0/3810976009 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7f021c0018c0 con 0x7f02301428d0 2024-05-11T12:15:30.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.032+0000 7f022d600640 1 -- 172.21.15.167:0/3810976009 >> v1:172.21.15.167:6789/0 conn(0x7f02301428d0 legacy=0x7f023006d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:30.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.032+0000 7f022d600640 1 --2- 172.21.15.167:0/3810976009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0214003a50 0x7f0214003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:30.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.032+0000 7f022d600640 1 --2- 172.21.15.167:0/3810976009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0214004800 0x7f0214004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:30.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.032+0000 7f022d600640 1 --2- 172.21.15.167:0/3810976009 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f023006db70 0x7f02140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:30.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.032+0000 7f022d600640 1 -- 172.21.15.167:0/3810976009 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0230160c40 con 0x7f0214003a50 2024-05-11T12:15:30.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.032+0000 7f022d600640 1 -- 172.21.15.167:0/3810976009 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0230161aa0 con 0x7f0214004800 2024-05-11T12:15:30.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.032+0000 7f022d600640 1 -- 172.21.15.167:0/3810976009 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f021400ba10 con 0x7f023006db70 2024-05-11T12:15:30.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.032+0000 7f022ea00640 1 --2- 172.21.15.167:0/3810976009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0214003a50 0x7f0214003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:30.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.033+0000 7f022e000640 1 --2- 172.21.15.167:0/3810976009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0214004800 0x7f0214004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:30.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.033+0000 7f022f400640 1 --2- 172.21.15.167:0/3810976009 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f023006db70 0x7f02140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:30.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.033+0000 7f022ea00640 1 -- 172.21.15.167:0/3810976009 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f023006db70 msgr2=0x7f02140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:30.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.033+0000 7f022ea00640 1 --2- 172.21.15.167:0/3810976009 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f023006db70 0x7f02140051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:30.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.033+0000 7f022ea00640 1 -- 172.21.15.167:0/3810976009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0214004800 msgr2=0x7f0214004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:30.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.033+0000 7f022ea00640 1 --2- 172.21.15.167:0/3810976009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0214004800 0x7f0214004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:30.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.033+0000 7f022ea00640 1 -- 172.21.15.167:0/3810976009 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0214003520 con 0x7f0214003a50 2024-05-11T12:15:30.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.034+0000 7f022ea00640 1 --2- 172.21.15.167:0/3810976009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0214003a50 0x7f0214003e70 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f0220009af0 tx=0x7f0214005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07b6d626caa6c50 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:30.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.034+0000 7f022d600640 1 -- 172.21.15.167:0/3810976009 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0220036070 con 0x7f0214003a50 2024-05-11T12:15:30.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.034+0000 7f022d600640 1 -- 172.21.15.167:0/3810976009 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0220031400 con 0x7f0214003a50 2024-05-11T12:15:30.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.034+0000 7f022d600640 1 -- 172.21.15.167:0/3810976009 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0220009690 con 0x7f0214003a50 2024-05-11T12:15:30.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.035+0000 7f022fe00640 1 -- 172.21.15.167:0/3810976009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0214003a50 msgr2=0x7f0214003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:30.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.035+0000 7f022fe00640 1 --2- 172.21.15.167:0/3810976009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0214003a50 0x7f0214003e70 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f0220009af0 tx=0x7f0214005830 comp rx=0 tx=0).stop 2024-05-11T12:15:30.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.035+0000 7f022f400640 1 -- 172.21.15.167:0/3810976009 reap_dead start 2024-05-11T12:15:30.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.035+0000 7f022fe00640 1 -- 172.21.15.167:0/3810976009 shutdown_connections 2024-05-11T12:15:30.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.035+0000 7f022fe00640 1 -- 172.21.15.167:0/3810976009 >> 172.21.15.167:0/3810976009 conn(0x7f02300b32a0 msgr2=0x7f02300d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:30.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.035+0000 7f022fe00640 1 -- 172.21.15.167:0/3810976009 shutdown_connections 2024-05-11T12:15:30.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.035+0000 7f022fe00640 1 -- 172.21.15.167:0/3810976009 wait complete. 2024-05-11T12:15:30.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.037+0000 7f022fe00640 1 Processor -- start 2024-05-11T12:15:30.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.037+0000 7f022fe00640 1 -- start start 2024-05-11T12:15:30.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.038+0000 7f022fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0214003a50 0x7f0230072260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:30.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.038+0000 7f022fe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f02300af230 0x7f023013f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:30.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.038+0000 7f022fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02301e8e50 0x7f02301e9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:30.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.038+0000 7f022fe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0230161730 con 0x7f02301e8e50 2024-05-11T12:15:30.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.038+0000 7f022fe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0230161450 con 0x7f0214003a50 2024-05-11T12:15:30.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.038+0000 7f022fe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f02301403b0 con 0x7f02300af230 2024-05-11T12:15:30.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.039+0000 7f022f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02301e8e50 0x7f02301e9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:30.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.039+0000 7f022f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02301e8e50 0x7f02301e9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58146/0 (socket says 172.21.15.167:58146) 2024-05-11T12:15:30.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.039+0000 7f022f400640 1 -- 172.21.15.167:0/1817476202 learned_addr learned my addr 172.21.15.167:0/1817476202 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:30.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.039+0000 7f022e000640 1 --2- 172.21.15.167:0/1817476202 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f02300af230 0x7f023013f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:30.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.039+0000 7f022ea00640 1 --2- 172.21.15.167:0/1817476202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0214003a50 0x7f0230072260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:30.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.039+0000 7f022f400640 1 -- 172.21.15.167:0/1817476202 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f02300af230 msgr2=0x7f023013f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:30.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.039+0000 7f022f400640 1 --2- 172.21.15.167:0/1817476202 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f02300af230 0x7f023013f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:30.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.039+0000 7f022f400640 1 -- 172.21.15.167:0/1817476202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0214003a50 msgr2=0x7f0230072260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:30.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.039+0000 7f022f400640 1 --2- 172.21.15.167:0/1817476202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0214003a50 0x7f0230072260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:30.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.039+0000 7f022f400640 1 -- 172.21.15.167:0/1817476202 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02300728d0 con 0x7f02301e8e50 2024-05-11T12:15:30.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.039+0000 7f022f400640 1 --2- 172.21.15.167:0/1817476202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02301e8e50 0x7f02301e9300 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f023015b220 tx=0x7f022800f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:30.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.040+0000 7f0226a00640 1 -- 172.21.15.167:0/1817476202 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022801c070 con 0x7f02301e8e50 2024-05-11T12:15:30.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.040+0000 7f0226a00640 1 -- 172.21.15.167:0/1817476202 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0228017400 con 0x7f02301e8e50 2024-05-11T12:15:30.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.040+0000 7f0226a00640 1 -- 172.21.15.167:0/1817476202 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02280177f0 con 0x7f02301e8e50 2024-05-11T12:15:30.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.040+0000 7f022fe00640 1 -- 172.21.15.167:0/1817476202 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0230072bb0 con 0x7f02301e8e50 2024-05-11T12:15:30.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.040+0000 7f022fe00640 1 -- 172.21.15.167:0/1817476202 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0230073170 con 0x7f02301e8e50 2024-05-11T12:15:30.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.041+0000 7f022fe00640 1 -- 172.21.15.167:0/1817476202 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f02300694b0 con 0x7f02301e8e50 2024-05-11T12:15:30.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.042+0000 7f0226a00640 1 -- 172.21.15.167:0/1817476202 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0228005d10 con 0x7f02301e8e50 2024-05-11T12:15:30.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.043+0000 7f0226a00640 1 --2- 172.21.15.167:0/1817476202 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f02100762d0 0x7f0210078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:30.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.043+0000 7f0226a00640 1 -- 172.21.15.167:0/1817476202 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5507+0+0 (secure 0 0 0) 0x7f0228098460 con 0x7f02301e8e50 2024-05-11T12:15:30.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.044+0000 7f022ea00640 1 --2- 172.21.15.167:0/1817476202 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f02100762d0 0x7f0210078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:30.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.044+0000 7f0226a00640 1 --2- 172.21.15.167:0/1817476202 >> [v2:172.21.15.204:6824/1178824319,v1:172.21.15.204:6825/1178824319] conn(0x7f021007c8b0 0x7f021007ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:30.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.044+0000 7f0226a00640 1 -- 172.21.15.167:0/1817476202 --> [v2:172.21.15.204:6824/1178824319,v1:172.21.15.204:6825/1178824319] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0228010e50 con 0x7f021007c8b0 2024-05-11T12:15:30.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.044+0000 7f0226a00640 1 -- 172.21.15.167:0/1817476202 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=104) ==== 24+0+0 (secure 0 0 0) 0x7f02280988f0 con 0x7f02301e8e50 2024-05-11T12:15:30.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.044+0000 7f022ea00640 1 --2- 172.21.15.167:0/1817476202 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f02100762d0 0x7f0210078790 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0220033000 tx=0x7f0220008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:30.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.044+0000 7f022e000640 1 --2- 172.21.15.167:0/1817476202 >> [v2:172.21.15.204:6824/1178824319,v1:172.21.15.204:6825/1178824319] conn(0x7f021007c8b0 0x7f021007ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:30.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.045+0000 7f022e000640 1 --2- 172.21.15.167:0/1817476202 >> [v2:172.21.15.204:6824/1178824319,v1:172.21.15.204:6825/1178824319] conn(0x7f021007c8b0 0x7f021007ecf0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:30.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.047+0000 7f0226a00640 1 -- 172.21.15.167:0/1817476202 <== osd.5 v2:172.21.15.204:6824/1178824319 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0228010e50 con 0x7f021007c8b0 2024-05-11T12:15:30.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.062+0000 7f022fe00640 1 -- 172.21.15.167:0/1817476202 --> [v2:172.21.15.204:6824/1178824319,v1:172.21.15.204:6825/1178824319] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f023006f900 con 0x7f021007c8b0 2024-05-11T12:15:30.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.063+0000 7f0226a00640 1 -- 172.21.15.167:0/1817476202 <== osd.5 v2:172.21.15.204:6824/1178824319 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f023006f900 con 0x7f021007c8b0 2024-05-11T12:15:30.063 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T12:15:30.063 INFO:teuthology.orchestra.run.smithi167.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-11T12:15:30.063 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T12:15:30.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.064+0000 7f022fe00640 1 -- 172.21.15.167:0/1817476202 >> [v2:172.21.15.204:6824/1178824319,v1:172.21.15.204:6825/1178824319] conn(0x7f021007c8b0 msgr2=0x7f021007ecf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:30.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.064+0000 7f022fe00640 1 --2- 172.21.15.167:0/1817476202 >> [v2:172.21.15.204:6824/1178824319,v1:172.21.15.204:6825/1178824319] conn(0x7f021007c8b0 0x7f021007ecf0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:30.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.064+0000 7f022fe00640 1 -- 172.21.15.167:0/1817476202 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f02100762d0 msgr2=0x7f0210078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:30.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.064+0000 7f022fe00640 1 --2- 172.21.15.167:0/1817476202 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f02100762d0 0x7f0210078790 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0220033000 tx=0x7f0220008030 comp rx=0 tx=0).stop 2024-05-11T12:15:30.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.064+0000 7f022fe00640 1 -- 172.21.15.167:0/1817476202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02301e8e50 msgr2=0x7f02301e9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:30.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.064+0000 7f022fe00640 1 --2- 172.21.15.167:0/1817476202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02301e8e50 0x7f02301e9300 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f023015b220 tx=0x7f022800f4a0 comp rx=0 tx=0).stop 2024-05-11T12:15:30.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.064+0000 7f022f400640 1 -- 172.21.15.167:0/1817476202 reap_dead start 2024-05-11T12:15:30.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.064+0000 7f022fe00640 1 -- 172.21.15.167:0/1817476202 shutdown_connections 2024-05-11T12:15:30.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.064+0000 7f022fe00640 1 -- 172.21.15.167:0/1817476202 >> 172.21.15.167:0/1817476202 conn(0x7f02300b32a0 msgr2=0x7f023006c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:30.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.064+0000 7f022fe00640 1 -- 172.21.15.167:0/1817476202 shutdown_connections 2024-05-11T12:15:30.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.064+0000 7f022fe00640 1 -- 172.21.15.167:0/1817476202 wait complete. 2024-05-11T12:15:30.082 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-05-11T12:15:30.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.282+0000 7fe16f000640 1 Processor -- start 2024-05-11T12:15:30.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.283+0000 7fe16f000640 1 -- start start 2024-05-11T12:15:30.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.284+0000 7fe16f000640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe168160440 0x7fe1680729f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:30.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.284+0000 7fe16f000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1680cf9c0 0x7fe1680aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:30.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.284+0000 7fe16f000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1680cffa0 con 0x7fe168072f30 2024-05-11T12:15:30.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.284+0000 7fe16f000640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fe1680d0280 con 0x7fe168160440 2024-05-11T12:15:30.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.284+0000 7fe16f000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe1680d0560 con 0x7fe1680cf9c0 2024-05-11T12:15:30.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.285+0000 7fe167e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe168160440 0x7fe1680729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:30.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.285+0000 7fe167e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe168160440 0x7fe1680729f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58152/0 (socket says 172.21.15.167:58152) 2024-05-11T12:15:30.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.285+0000 7fe167e00640 1 -- 172.21.15.167:0/777932796 learned_addr learned my addr 172.21.15.167:0/777932796 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:30.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.285+0000 7fe16d000640 1 --2- 172.21.15.167:0/777932796 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1680cf9c0 0x7fe1680aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:30.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.285+0000 7fe167e00640 1 -- 172.21.15.167:0/777932796 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1680cf9c0 msgr2=0x7fe1680aebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:30.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.285+0000 7fe167e00640 1 --2- 172.21.15.167:0/777932796 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1680cf9c0 0x7fe1680aebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:30.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.285+0000 7fe167e00640 1 -- 172.21.15.167:0/777932796 >> v1:172.21.15.204:6789/0 conn(0x7fe168072f30 legacy=0x7fe1680cf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:30.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.285+0000 7fe167e00640 1 -- 172.21.15.167:0/777932796 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1680d0840 con 0x7fe168160440 2024-05-11T12:15:30.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.286+0000 7fe167e00640 1 --2- 172.21.15.167:0/777932796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe168160440 0x7fe1680729f0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fe1580098a0 tx=0x7fe15802a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=129b2faaca42ec1 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:30.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.287+0000 7fe166a00640 1 -- 172.21.15.167:0/777932796 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe158037070 con 0x7fe168160440 2024-05-11T12:15:30.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.287+0000 7fe166a00640 1 -- 172.21.15.167:0/777932796 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe158032440 con 0x7fe168160440 2024-05-11T12:15:30.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.287+0000 7fe166a00640 1 -- 172.21.15.167:0/777932796 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe158032810 con 0x7fe168160440 2024-05-11T12:15:30.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.287+0000 7fe16f000640 1 -- 172.21.15.167:0/777932796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe168160440 msgr2=0x7fe1680729f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:30.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.287+0000 7fe16f000640 1 --2- 172.21.15.167:0/777932796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe168160440 0x7fe1680729f0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fe1580098a0 tx=0x7fe15802a200 comp rx=0 tx=0).stop 2024-05-11T12:15:30.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.287+0000 7fe16f000640 1 -- 172.21.15.167:0/777932796 shutdown_connections 2024-05-11T12:15:30.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.287+0000 7fe16f000640 1 --2- 172.21.15.167:0/777932796 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1680cf9c0 0x7fe1680aebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:30.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.287+0000 7fe16f000640 1 --2- 172.21.15.167:0/777932796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe168160440 0x7fe1680729f0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:30.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.287+0000 7fe16f000640 1 -- 172.21.15.167:0/777932796 >> 172.21.15.167:0/777932796 conn(0x7fe16806c450 msgr2=0x7fe16806c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:30.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.288+0000 7fe16f000640 1 -- 172.21.15.167:0/777932796 shutdown_connections 2024-05-11T12:15:30.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.288+0000 7fe16f000640 1 -- 172.21.15.167:0/777932796 wait complete. 2024-05-11T12:15:30.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.289+0000 7fe16f000640 1 Processor -- start 2024-05-11T12:15:30.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.289+0000 7fe16f000640 1 -- start start 2024-05-11T12:15:30.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.290+0000 7fe16f000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe168072f30 0x7fe16813f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:30.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.290+0000 7fe16f000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1680cf9c0 0x7fe1681f51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:30.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.290+0000 7fe16f000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe168160440 0x7fe1680b4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:30.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.290+0000 7fe16f000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe1680af4b0 con 0x7fe168072f30 2024-05-11T12:15:30.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.290+0000 7fe16f000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe1680af330 con 0x7fe1680cf9c0 2024-05-11T12:15:30.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.290+0000 7fe16f000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe1680af630 con 0x7fe168160440 2024-05-11T12:15:30.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.291+0000 7fe167e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe168072f30 0x7fe16813f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:30.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.291+0000 7fe167e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe168072f30 0x7fe16813f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58158/0 (socket says 172.21.15.167:58158) 2024-05-11T12:15:30.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.291+0000 7fe167e00640 1 -- 172.21.15.167:0/768876394 learned_addr learned my addr 172.21.15.167:0/768876394 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:30.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.291+0000 7fe167400640 1 --2- 172.21.15.167:0/768876394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1680cf9c0 0x7fe1681f51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:30.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.291+0000 7fe16d000640 1 --2- 172.21.15.167:0/768876394 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe168160440 0x7fe1680b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:30.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.291+0000 7fe167e00640 1 -- 172.21.15.167:0/768876394 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe168160440 msgr2=0x7fe1680b4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:30.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.291+0000 7fe167e00640 1 --2- 172.21.15.167:0/768876394 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe168160440 0x7fe1680b4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:30.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.291+0000 7fe167e00640 1 -- 172.21.15.167:0/768876394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1680cf9c0 msgr2=0x7fe1681f51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:30.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.291+0000 7fe167e00640 1 --2- 172.21.15.167:0/768876394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1680cf9c0 0x7fe1681f51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:30.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.291+0000 7fe167e00640 1 -- 172.21.15.167:0/768876394 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1681f5b80 con 0x7fe168072f30 2024-05-11T12:15:30.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.291+0000 7fe16d000640 1 --2- 172.21.15.167:0/768876394 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe168160440 0x7fe1680b4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:15:30.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.291+0000 7fe167400640 1 --2- 172.21.15.167:0/768876394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1680cf9c0 0x7fe1681f51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:15:30.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.291+0000 7fe167e00640 1 --2- 172.21.15.167:0/768876394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe168072f30 0x7fe16813f6a0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fe158000c00 tx=0x7fe1580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:30.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.292+0000 7fe15fe00640 1 -- 172.21.15.167:0/768876394 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe158037040 con 0x7fe168072f30 2024-05-11T12:15:30.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.292+0000 7fe15fe00640 1 -- 172.21.15.167:0/768876394 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe158042070 con 0x7fe168072f30 2024-05-11T12:15:30.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.292+0000 7fe16f000640 1 -- 172.21.15.167:0/768876394 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1681f5e60 con 0x7fe168072f30 2024-05-11T12:15:30.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.292+0000 7fe15fe00640 1 -- 172.21.15.167:0/768876394 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe158030bd0 con 0x7fe168072f30 2024-05-11T12:15:30.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.292+0000 7fe16f000640 1 -- 172.21.15.167:0/768876394 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1680b3f50 con 0x7fe168072f30 2024-05-11T12:15:30.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.293+0000 7fe16f000640 1 -- 172.21.15.167:0/768876394 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1680694b0 con 0x7fe168072f30 2024-05-11T12:15:30.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.294+0000 7fe15fe00640 1 -- 172.21.15.167:0/768876394 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe15802ad10 con 0x7fe168072f30 2024-05-11T12:15:30.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.295+0000 7fe15fe00640 1 --2- 172.21.15.167:0/768876394 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe1480762d0 0x7fe148078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:30.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.295+0000 7fe15fe00640 1 -- 172.21.15.167:0/768876394 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5507+0+0 (secure 0 0 0) 0x7fe1580b7e50 con 0x7fe168072f30 2024-05-11T12:15:30.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.295+0000 7fe167400640 1 --2- 172.21.15.167:0/768876394 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe1480762d0 0x7fe148078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:30.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.296+0000 7fe167400640 1 --2- 172.21.15.167:0/768876394 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe1480762d0 0x7fe148078790 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe154007fd0 tx=0x7fe154009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:30.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.301+0000 7fe15fe00640 1 -- 172.21.15.167:0/768876394 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe158081860 con 0x7fe168072f30 2024-05-11T12:15:30.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.507+0000 7fe16f000640 1 -- 172.21.15.167:0/768876394 --> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fe1680b4190 con 0x7fe1480762d0 2024-05-11T12:15:30.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.508+0000 7fe15fe00640 1 -- 172.21.15.167:0/768876394 <== mgr.4106 v2:172.21.15.167:6832/578352559 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe1540013a0 con 0x7fe1480762d0 2024-05-11T12:15:30.507 INFO:teuthology.orchestra.run.smithi167.stderr:instructed osd(s) 5 to deep-scrub 2024-05-11T12:15:30.510 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.511+0000 7fe16f000640 1 -- 172.21.15.167:0/768876394 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe1480762d0 msgr2=0x7fe148078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:30.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.511+0000 7fe16f000640 1 --2- 172.21.15.167:0/768876394 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe1480762d0 0x7fe148078790 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe154007fd0 tx=0x7fe154009040 comp rx=0 tx=0).stop 2024-05-11T12:15:30.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.511+0000 7fe16f000640 1 -- 172.21.15.167:0/768876394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe168072f30 msgr2=0x7fe16813f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:30.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.511+0000 7fe16f000640 1 --2- 172.21.15.167:0/768876394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe168072f30 0x7fe16813f6a0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fe158000c00 tx=0x7fe1580046c0 comp rx=0 tx=0).stop 2024-05-11T12:15:30.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.511+0000 7fe16f000640 1 -- 172.21.15.167:0/768876394 shutdown_connections 2024-05-11T12:15:30.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.511+0000 7fe16f000640 1 --2- 172.21.15.167:0/768876394 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe168160440 0x7fe1680b4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:30.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.511+0000 7fe16f000640 1 --2- 172.21.15.167:0/768876394 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fe1480762d0 0x7fe148078790 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:30.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.511+0000 7fe16f000640 1 --2- 172.21.15.167:0/768876394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1680cf9c0 0x7fe1681f51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:30.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.511+0000 7fe16f000640 1 --2- 172.21.15.167:0/768876394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe168072f30 0x7fe16813f6a0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:30.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.511+0000 7fe16f000640 1 -- 172.21.15.167:0/768876394 >> 172.21.15.167:0/768876394 conn(0x7fe16806c450 msgr2=0x7fe168161510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:30.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.511+0000 7fe16f000640 1 -- 172.21.15.167:0/768876394 shutdown_connections 2024-05-11T12:15:30.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.511+0000 7fe16f000640 1 -- 172.21.15.167:0/768876394 wait complete. 2024-05-11T12:15:30.528 INFO:tasks.ceph:Scrubbing osd.6 2024-05-11T12:15:30.528 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-05-11T12:15:30.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.734+0000 7f41b9600640 1 Processor -- start 2024-05-11T12:15:30.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.734+0000 7f41b9600640 1 -- start start 2024-05-11T12:15:30.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.736+0000 7f41b9600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41b415cda0 0x7f41b40b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:30.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.736+0000 7f41b9600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f41b4070140 0x7f41b40b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:30.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.736+0000 7f41b9600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41b415f970 con 0x7f41b40b5eb0 2024-05-11T12:15:30.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.736+0000 7f41b9600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f41b415fc50 con 0x7f41b4070140 2024-05-11T12:15:30.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.736+0000 7f41b9600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f41b415ff30 con 0x7f41b415cda0 2024-05-11T12:15:30.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.737+0000 7f41b2a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41b415cda0 0x7f41b40b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:30.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.737+0000 7f41b3400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f41b4070140 0x7f41b40b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:30.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.737+0000 7f41b2a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41b415cda0 0x7f41b40b5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:60146/0 (socket says 172.21.15.167:60146) 2024-05-11T12:15:30.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.737+0000 7f41b3400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f41b4070140 0x7f41b40b3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.167:51604/0 (socket says 172.21.15.167:51604) 2024-05-11T12:15:30.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.737+0000 7f41b2a00640 1 -- 172.21.15.167:0/1677486041 learned_addr learned my addr 172.21.15.167:0/1677486041 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:30.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.738+0000 7f41b3400640 1 -- 172.21.15.167:0/1677486041 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41b415cda0 msgr2=0x7f41b40b5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:30.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.738+0000 7f41b3400640 1 --2- 172.21.15.167:0/1677486041 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41b415cda0 0x7f41b40b5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:30.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.738+0000 7f41b3400640 1 -- 172.21.15.167:0/1677486041 >> v1:172.21.15.204:6789/0 conn(0x7f41b40b5eb0 legacy=0x7f41b406fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:30.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.738+0000 7f41b3400640 1 -- 172.21.15.167:0/1677486041 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41b4160210 con 0x7f41b4070140 2024-05-11T12:15:30.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.739+0000 7f41b3400640 1 --2- 172.21.15.167:0/1677486041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41b4070140 0x7f41b40b3de0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f41ac009d90 tx=0x7f41ac02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0bff39eb1293965 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:30.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.740+0000 7f41b2000640 1 -- 172.21.15.167:0/1677486041 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ac007740 con 0x7f41b4070140 2024-05-11T12:15:30.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.740+0000 7f41b2000640 1 -- 172.21.15.167:0/1677486041 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41ac0078e0 con 0x7f41b4070140 2024-05-11T12:15:30.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.740+0000 7f41b2000640 1 -- 172.21.15.167:0/1677486041 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ac007cb0 con 0x7f41b4070140 2024-05-11T12:15:30.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.740+0000 7f41b9600640 1 -- 172.21.15.167:0/1677486041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41b4070140 msgr2=0x7f41b40b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:30.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.740+0000 7f41b9600640 1 --2- 172.21.15.167:0/1677486041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41b4070140 0x7f41b40b3de0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f41ac009d90 tx=0x7f41ac02a660 comp rx=0 tx=0).stop 2024-05-11T12:15:30.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.740+0000 7f41b9600640 1 -- 172.21.15.167:0/1677486041 shutdown_connections 2024-05-11T12:15:30.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.740+0000 7f41b9600640 1 --2- 172.21.15.167:0/1677486041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41b4070140 0x7f41b40b3de0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:30.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.740+0000 7f41b9600640 1 --2- 172.21.15.167:0/1677486041 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41b415cda0 0x7f41b40b5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:30.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.740+0000 7f41b9600640 1 -- 172.21.15.167:0/1677486041 >> 172.21.15.167:0/1677486041 conn(0x7f41b40d0c70 msgr2=0x7f41b40d1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:30.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.740+0000 7f41b9600640 1 -- 172.21.15.167:0/1677486041 shutdown_connections 2024-05-11T12:15:30.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.740+0000 7f41b9600640 1 -- 172.21.15.167:0/1677486041 wait complete. 2024-05-11T12:15:30.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.742+0000 7f41b9600640 1 Processor -- start 2024-05-11T12:15:30.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.742+0000 7f41b9600640 1 -- start start 2024-05-11T12:15:30.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.743+0000 7f41b9600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41b4070140 0x7f41b4154040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:30.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.743+0000 7f41b9600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41b40b5eb0 0x7f41b414b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:30.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.743+0000 7f41b9600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41b415cda0 0x7f41b40b1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:30.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.743+0000 7f41b9600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f41b40af6c0 con 0x7f41b4070140 2024-05-11T12:15:30.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.743+0000 7f41b9600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f41b415d180 con 0x7f41b415cda0 2024-05-11T12:15:30.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.743+0000 7f41b9600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f41b40b4320 con 0x7f41b40b5eb0 2024-05-11T12:15:30.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.744+0000 7f41b2a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41b4070140 0x7f41b4154040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:30.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.744+0000 7f41b2a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41b4070140 0x7f41b4154040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58160/0 (socket says 172.21.15.167:58160) 2024-05-11T12:15:30.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.744+0000 7f41b2a00640 1 -- 172.21.15.167:0/2096513296 learned_addr learned my addr 172.21.15.167:0/2096513296 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:30.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.744+0000 7f41abe00640 1 --2- 172.21.15.167:0/2096513296 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41b40b5eb0 0x7f41b414b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:30.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.744+0000 7f41b3400640 1 --2- 172.21.15.167:0/2096513296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41b415cda0 0x7f41b40b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:30.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.744+0000 7f41b2a00640 1 -- 172.21.15.167:0/2096513296 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41b40b5eb0 msgr2=0x7f41b414b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:30.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.744+0000 7f41b2a00640 1 --2- 172.21.15.167:0/2096513296 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41b40b5eb0 0x7f41b414b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:30.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.744+0000 7f41b2a00640 1 -- 172.21.15.167:0/2096513296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41b415cda0 msgr2=0x7f41b40b1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:30.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.744+0000 7f41b2a00640 1 --2- 172.21.15.167:0/2096513296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41b415cda0 0x7f41b40b1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:30.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.744+0000 7f41b2a00640 1 -- 172.21.15.167:0/2096513296 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41b40b1ea0 con 0x7f41b4070140 2024-05-11T12:15:30.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.744+0000 7f41abe00640 1 --2- 172.21.15.167:0/2096513296 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41b40b5eb0 0x7f41b414b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:15:30.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.745+0000 7f41b2a00640 1 --2- 172.21.15.167:0/2096513296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41b4070140 0x7f41b4154040 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f41a400e870 tx=0x7f41a400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:30.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.745+0000 7f41aaa00640 1 -- 172.21.15.167:0/2096513296 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a4017070 con 0x7f41b4070140 2024-05-11T12:15:30.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.745+0000 7f41aaa00640 1 -- 172.21.15.167:0/2096513296 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41a400c8e0 con 0x7f41b4070140 2024-05-11T12:15:30.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.745+0000 7f41aaa00640 1 -- 172.21.15.167:0/2096513296 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a400ccd0 con 0x7f41b4070140 2024-05-11T12:15:30.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.745+0000 7f41b9600640 1 -- 172.21.15.167:0/2096513296 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41b4154700 con 0x7f41b4070140 2024-05-11T12:15:30.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.745+0000 7f41b9600640 1 -- 172.21.15.167:0/2096513296 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41b415d550 con 0x7f41b4070140 2024-05-11T12:15:30.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.746+0000 7f41b9600640 1 -- 172.21.15.167:0/2096513296 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f41b4069590 con 0x7f41b4070140 2024-05-11T12:15:30.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.748+0000 7f41aaa00640 1 -- 172.21.15.167:0/2096513296 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f41a401a070 con 0x7f41b4070140 2024-05-11T12:15:30.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.749+0000 7f41aaa00640 1 --2- 172.21.15.167:0/2096513296 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f41900762d0 0x7f4190078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:30.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.749+0000 7f41aaa00640 1 -- 172.21.15.167:0/2096513296 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5507+0+0 (secure 0 0 0) 0x7f41a4097310 con 0x7f41b4070140 2024-05-11T12:15:30.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.749+0000 7f41abe00640 1 --2- 172.21.15.167:0/2096513296 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f41900762d0 0x7f4190078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:30.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.749+0000 7f41aaa00640 1 --2- 172.21.15.167:0/2096513296 >> [v2:172.21.15.204:6800/3396850806,v1:172.21.15.204:6801/3396850806] conn(0x7f419007c8b0 0x7f419007ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:30.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.749+0000 7f41aaa00640 1 -- 172.21.15.167:0/2096513296 --> [v2:172.21.15.204:6800/3396850806,v1:172.21.15.204:6801/3396850806] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f419007f3c0 con 0x7f419007c8b0 2024-05-11T12:15:30.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.749+0000 7f41aaa00640 1 -- 172.21.15.167:0/2096513296 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=104) ==== 24+0+0 (secure 0 0 0) 0x7f41a409e050 con 0x7f41b4070140 2024-05-11T12:15:30.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.750+0000 7f41abe00640 1 --2- 172.21.15.167:0/2096513296 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f41900762d0 0x7f4190078790 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f41b40b2670 tx=0x7f419c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:30.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.750+0000 7f41b3400640 1 --2- 172.21.15.167:0/2096513296 >> [v2:172.21.15.204:6800/3396850806,v1:172.21.15.204:6801/3396850806] conn(0x7f419007c8b0 0x7f419007ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:30.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.751+0000 7f41b3400640 1 --2- 172.21.15.167:0/2096513296 >> [v2:172.21.15.204:6800/3396850806,v1:172.21.15.204:6801/3396850806] conn(0x7f419007c8b0 0x7f419007ecf0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:30.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.753+0000 7f41aaa00640 1 -- 172.21.15.167:0/2096513296 <== osd.6 v2:172.21.15.204:6800/3396850806 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f419007f3c0 con 0x7f419007c8b0 2024-05-11T12:15:30.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.769+0000 7f41b9600640 1 -- 172.21.15.167:0/2096513296 --> [v2:172.21.15.204:6800/3396850806,v1:172.21.15.204:6801/3396850806] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f41b4069e70 con 0x7f419007c8b0 2024-05-11T12:15:30.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.770+0000 7f41aaa00640 1 -- 172.21.15.167:0/2096513296 <== osd.6 v2:172.21.15.204:6800/3396850806 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f41b4069e70 con 0x7f419007c8b0 2024-05-11T12:15:30.769 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T12:15:30.769 INFO:teuthology.orchestra.run.smithi167.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-11T12:15:30.769 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T12:15:30.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.770+0000 7f41b9600640 1 -- 172.21.15.167:0/2096513296 >> [v2:172.21.15.204:6800/3396850806,v1:172.21.15.204:6801/3396850806] conn(0x7f419007c8b0 msgr2=0x7f419007ecf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:30.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.770+0000 7f41b9600640 1 --2- 172.21.15.167:0/2096513296 >> [v2:172.21.15.204:6800/3396850806,v1:172.21.15.204:6801/3396850806] conn(0x7f419007c8b0 0x7f419007ecf0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:30.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.771+0000 7f41b9600640 1 -- 172.21.15.167:0/2096513296 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f41900762d0 msgr2=0x7f4190078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:30.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.771+0000 7f41b9600640 1 --2- 172.21.15.167:0/2096513296 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f41900762d0 0x7f4190078790 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f41b40b2670 tx=0x7f419c009040 comp rx=0 tx=0).stop 2024-05-11T12:15:30.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.771+0000 7f41b9600640 1 -- 172.21.15.167:0/2096513296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41b4070140 msgr2=0x7f41b4154040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:30.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.771+0000 7f41b9600640 1 --2- 172.21.15.167:0/2096513296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41b4070140 0x7f41b4154040 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f41a400e870 tx=0x7f41a400ed30 comp rx=0 tx=0).stop 2024-05-11T12:15:30.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.771+0000 7f41b3400640 1 -- 172.21.15.167:0/2096513296 reap_dead start 2024-05-11T12:15:30.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.771+0000 7f41b9600640 1 -- 172.21.15.167:0/2096513296 shutdown_connections 2024-05-11T12:15:30.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.771+0000 7f41b9600640 1 -- 172.21.15.167:0/2096513296 >> 172.21.15.167:0/2096513296 conn(0x7f41b40d0c70 msgr2=0x7f41b406ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:30.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.771+0000 7f41b9600640 1 -- 172.21.15.167:0/2096513296 shutdown_connections 2024-05-11T12:15:30.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.771+0000 7f41b9600640 1 -- 172.21.15.167:0/2096513296 wait complete. 2024-05-11T12:15:30.787 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-05-11T12:15:30.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.986+0000 7fd55a000640 1 Processor -- start 2024-05-11T12:15:30.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.987+0000 7fd55a000640 1 -- start start 2024-05-11T12:15:30.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.988+0000 7fd55a000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5541561c0 0x7fd5540b0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:30.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.988+0000 7fd55a000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd554160440 0x7fd5540b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:30.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.988+0000 7fd55a000640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5541565a0 con 0x7fd554073c60 2024-05-11T12:15:30.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.988+0000 7fd55a000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fd5540b6840 con 0x7fd554160440 2024-05-11T12:15:30.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.988+0000 7fd55a000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd5540afd00 con 0x7fd5541561c0 2024-05-11T12:15:30.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.989+0000 7fd559600640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fd554073c60 0x7fd554162cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:59402/0 (socket says 172.21.15.167:59402) 2024-05-11T12:15:30.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.989+0000 7fd559600640 1 -- 172.21.15.167:0/1436663421 learned_addr learned my addr 172.21.15.167:0/1436663421 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:15:30.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.989+0000 7fd558c00640 1 --2- 172.21.15.167:0/1436663421 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5541561c0 0x7fd5540b0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:30.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.989+0000 7fd553400640 1 -- 172.21.15.167:0/1436663421 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4042080273 0 0) 0x7fd5541565a0 con 0x7fd554073c60 2024-05-11T12:15:30.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.989+0000 7fd553e00640 1 --2- 172.21.15.167:0/1436663421 >> v2:172.21.15.204:3300/0 conn(0x7fd554160440 0x7fd5540b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:30.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.989+0000 7fd553400640 1 -- 172.21.15.167:0/1436663421 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd538003520 con 0x7fd554073c60 2024-05-11T12:15:30.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.989+0000 7fd553400640 1 -- 172.21.15.167:0/1436663421 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 930219852 0 0) 0x7fd538003520 con 0x7fd554073c60 2024-05-11T12:15:30.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.989+0000 7fd553400640 1 -- 172.21.15.167:0/1436663421 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5541561c0 msgr2=0x7fd5540b0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:30.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.989+0000 7fd553400640 1 --2- 172.21.15.167:0/1436663421 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5541561c0 0x7fd5540b0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:30.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.989+0000 7fd553400640 1 -- 172.21.15.167:0/1436663421 >> v2:172.21.15.204:3300/0 conn(0x7fd554160440 msgr2=0x7fd5540b0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:30.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.989+0000 7fd553400640 1 --2- 172.21.15.167:0/1436663421 >> v2:172.21.15.204:3300/0 conn(0x7fd554160440 0x7fd5540b0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:30.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.989+0000 7fd553400640 1 -- 172.21.15.167:0/1436663421 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5540b1400 con 0x7fd554073c60 2024-05-11T12:15:30.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.989+0000 7fd553400640 1 -- 172.21.15.167:0/1436663421 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 904266740 0 0) 0x7fd54c0016f0 con 0x7fd554073c60 2024-05-11T12:15:30.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.989+0000 7fd553400640 1 -- 172.21.15.167:0/1436663421 >> v1:172.21.15.167:6789/0 conn(0x7fd554073c60 legacy=0x7fd554162cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:30.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.989+0000 7fd553400640 1 --2- 172.21.15.167:0/1436663421 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd538003ac0 0x7fd538003ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:30.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.990+0000 7fd553400640 1 --2- 172.21.15.167:0/1436663421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd538004800 0x7fd538004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:30.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.990+0000 7fd553400640 1 --2- 172.21.15.167:0/1436663421 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd538005210 0x7fd53800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:30.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.990+0000 7fd553400640 1 -- 172.21.15.167:0/1436663421 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd5540b6840 con 0x7fd538004800 2024-05-11T12:15:30.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.990+0000 7fd553400640 1 -- 172.21.15.167:0/1436663421 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd5540afd00 con 0x7fd538003ac0 2024-05-11T12:15:30.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.990+0000 7fd553400640 1 -- 172.21.15.167:0/1436663421 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd53800c140 con 0x7fd538005210 2024-05-11T12:15:30.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.990+0000 7fd558c00640 1 --2- 172.21.15.167:0/1436663421 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5541561c0 0x7fd5540b0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:15:30.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.990+0000 7fd553e00640 1 --2- 172.21.15.167:0/1436663421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd538004800 0x7fd538004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:30.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.990+0000 7fd559600640 1 --2- 172.21.15.167:0/1436663421 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd538005210 0x7fd53800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:30.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.990+0000 7fd553e00640 1 -- 172.21.15.167:0/1436663421 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd538005210 msgr2=0x7fd53800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:30.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.990+0000 7fd553e00640 1 --2- 172.21.15.167:0/1436663421 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd538005210 0x7fd53800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:30.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.990+0000 7fd553e00640 1 -- 172.21.15.167:0/1436663421 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd538003ac0 msgr2=0x7fd538003ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:30.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.990+0000 7fd553e00640 1 --2- 172.21.15.167:0/1436663421 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd538003ac0 0x7fd538003ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:30.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.990+0000 7fd553e00640 1 -- 172.21.15.167:0/1436663421 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd538003520 con 0x7fd538004800 2024-05-11T12:15:30.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.992+0000 7fd553e00640 1 --2- 172.21.15.167:0/1436663421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd538004800 0x7fd538004c80 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fd544007360 tx=0x7fd538005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c929eca39a397098 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:30.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.992+0000 7fd553400640 1 -- 172.21.15.167:0/1436663421 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd544009be0 con 0x7fd538004800 2024-05-11T12:15:30.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.992+0000 7fd553400640 1 -- 172.21.15.167:0/1436663421 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd54402e620 con 0x7fd538004800 2024-05-11T12:15:30.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.992+0000 7fd553400640 1 -- 172.21.15.167:0/1436663421 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54402e9f0 con 0x7fd538004800 2024-05-11T12:15:30.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.992+0000 7fd55a000640 1 -- 172.21.15.167:0/1436663421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd538004800 msgr2=0x7fd538004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:30.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.992+0000 7fd55a000640 1 --2- 172.21.15.167:0/1436663421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd538004800 0x7fd538004c80 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fd544007360 tx=0x7fd538005830 comp rx=0 tx=0).stop 2024-05-11T12:15:30.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.992+0000 7fd559600640 1 -- 172.21.15.167:0/1436663421 reap_dead start 2024-05-11T12:15:30.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.993+0000 7fd55a000640 1 -- 172.21.15.167:0/1436663421 shutdown_connections 2024-05-11T12:15:30.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.993+0000 7fd55a000640 1 -- 172.21.15.167:0/1436663421 >> 172.21.15.167:0/1436663421 conn(0x7fd554073140 msgr2=0x7fd554073550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:30.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.993+0000 7fd55a000640 1 -- 172.21.15.167:0/1436663421 shutdown_connections 2024-05-11T12:15:30.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.993+0000 7fd55a000640 1 -- 172.21.15.167:0/1436663421 wait complete. 2024-05-11T12:15:30.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.995+0000 7fd55a000640 1 Processor -- start 2024-05-11T12:15:30.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.995+0000 7fd55a000640 1 -- start start 2024-05-11T12:15:30.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.996+0000 7fd55a000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd538004800 0x7fd554158af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:30.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.996+0000 7fd55a000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd554150d30 0x7fd5541f89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:30.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.996+0000 7fd55a000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5541f8f10 0x7fd5541f93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:30.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.996+0000 7fd55a000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd5540b64d0 con 0x7fd538004800 2024-05-11T12:15:30.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.996+0000 7fd55a000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd5540b61f0 con 0x7fd554150d30 2024-05-11T12:15:30.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.996+0000 7fd55a000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd5540b6cd0 con 0x7fd5541f8f10 2024-05-11T12:15:30.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.996+0000 7fd558c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd538004800 0x7fd554158af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:30.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.996+0000 7fd558c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd538004800 0x7fd554158af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58176/0 (socket says 172.21.15.167:58176) 2024-05-11T12:15:30.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.996+0000 7fd558c00640 1 -- 172.21.15.167:0/3434498562 learned_addr learned my addr 172.21.15.167:0/3434498562 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:30.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.996+0000 7fd553e00640 1 --2- 172.21.15.167:0/3434498562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd554150d30 0x7fd5541f89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:30.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.996+0000 7fd559600640 1 --2- 172.21.15.167:0/3434498562 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5541f8f10 0x7fd5541f93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:30.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.996+0000 7fd558c00640 1 -- 172.21.15.167:0/3434498562 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5541f8f10 msgr2=0x7fd5541f93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:30.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.996+0000 7fd558c00640 1 --2- 172.21.15.167:0/3434498562 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5541f8f10 0x7fd5541f93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:30.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.996+0000 7fd558c00640 1 -- 172.21.15.167:0/3434498562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd554150d30 msgr2=0x7fd5541f89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:30.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.996+0000 7fd558c00640 1 --2- 172.21.15.167:0/3434498562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd554150d30 0x7fd5541f89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:30.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.996+0000 7fd558c00640 1 -- 172.21.15.167:0/3434498562 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd554159180 con 0x7fd538004800 2024-05-11T12:15:30.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.997+0000 7fd558c00640 1 --2- 172.21.15.167:0/3434498562 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd538004800 0x7fd554158af0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fd53c00e570 tx=0x7fd53c00ea30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:30.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.997+0000 7fd550c00640 1 -- 172.21.15.167:0/3434498562 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53c00f040 con 0x7fd538004800 2024-05-11T12:15:30.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.997+0000 7fd550c00640 1 -- 172.21.15.167:0/3434498562 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd53c004250 con 0x7fd538004800 2024-05-11T12:15:30.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.997+0000 7fd550c00640 1 -- 172.21.15.167:0/3434498562 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53c003610 con 0x7fd538004800 2024-05-11T12:15:30.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.997+0000 7fd55a000640 1 -- 172.21.15.167:0/3434498562 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd554159460 con 0x7fd538004800 2024-05-11T12:15:30.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.997+0000 7fd55a000640 1 -- 172.21.15.167:0/3434498562 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd554159a20 con 0x7fd538004800 2024-05-11T12:15:30.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.998+0000 7fd55a000640 1 -- 172.21.15.167:0/3434498562 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd554069bc0 con 0x7fd538004800 2024-05-11T12:15:30.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:30.999+0000 7fd550c00640 1 -- 172.21.15.167:0/3434498562 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd53c0037b0 con 0x7fd538004800 2024-05-11T12:15:31.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.000+0000 7fd550c00640 1 --2- 172.21.15.167:0/3434498562 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd530076300 0x7fd5300787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:31.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.001+0000 7fd550c00640 1 -- 172.21.15.167:0/3434498562 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5507+0+0 (secure 0 0 0) 0x7fd53c0970c0 con 0x7fd538004800 2024-05-11T12:15:31.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.001+0000 7fd553e00640 1 --2- 172.21.15.167:0/3434498562 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd530076300 0x7fd5300787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:31.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.001+0000 7fd553e00640 1 --2- 172.21.15.167:0/3434498562 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd530076300 0x7fd5300787c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd544035000 tx=0x7fd54400b030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:31.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.005+0000 7fd550c00640 1 -- 172.21.15.167:0/3434498562 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd53c017020 con 0x7fd538004800 2024-05-11T12:15:31.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.214+0000 7fd55a000640 1 -- 172.21.15.167:0/3434498562 --> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7fd5540b7970 con 0x7fd530076300 2024-05-11T12:15:31.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.215+0000 7fd550c00640 1 -- 172.21.15.167:0/3434498562 <== mgr.4106 v2:172.21.15.167:6832/578352559 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fd5540b7970 con 0x7fd530076300 2024-05-11T12:15:31.215 INFO:teuthology.orchestra.run.smithi167.stderr:instructed osd(s) 6 to deep-scrub 2024-05-11T12:15:31.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.218+0000 7fd55a000640 1 -- 172.21.15.167:0/3434498562 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd530076300 msgr2=0x7fd5300787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:31.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.218+0000 7fd55a000640 1 --2- 172.21.15.167:0/3434498562 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd530076300 0x7fd5300787c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd544035000 tx=0x7fd54400b030 comp rx=0 tx=0).stop 2024-05-11T12:15:31.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.218+0000 7fd55a000640 1 -- 172.21.15.167:0/3434498562 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd538004800 msgr2=0x7fd554158af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:31.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.218+0000 7fd55a000640 1 --2- 172.21.15.167:0/3434498562 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd538004800 0x7fd554158af0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fd53c00e570 tx=0x7fd53c00ea30 comp rx=0 tx=0).stop 2024-05-11T12:15:31.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.218+0000 7fd55a000640 1 -- 172.21.15.167:0/3434498562 shutdown_connections 2024-05-11T12:15:31.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.218+0000 7fd55a000640 1 --2- 172.21.15.167:0/3434498562 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5541f8f10 0x7fd5541f93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:31.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.218+0000 7fd55a000640 1 --2- 172.21.15.167:0/3434498562 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7fd530076300 0x7fd5300787c0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:31.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.218+0000 7fd55a000640 1 --2- 172.21.15.167:0/3434498562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd554150d30 0x7fd5541f89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:31.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.218+0000 7fd55a000640 1 --2- 172.21.15.167:0/3434498562 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd538004800 0x7fd554158af0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:31.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.218+0000 7fd55a000640 1 -- 172.21.15.167:0/3434498562 >> 172.21.15.167:0/3434498562 conn(0x7fd554073140 msgr2=0x7fd554073550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:31.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.218+0000 7fd55a000640 1 -- 172.21.15.167:0/3434498562 shutdown_connections 2024-05-11T12:15:31.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.219+0000 7fd55a000640 1 -- 172.21.15.167:0/3434498562 wait complete. 2024-05-11T12:15:31.237 INFO:tasks.ceph:Scrubbing osd.7 2024-05-11T12:15:31.237 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-05-11T12:15:31.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.442+0000 7f0dd5e00640 1 Processor -- start 2024-05-11T12:15:31.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.442+0000 7f0dd5e00640 1 -- start start 2024-05-11T12:15:31.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.443+0000 7f0dd5e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0dd00cf820 0x7f0dd00b52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:31.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.443+0000 7f0dd5e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0dd00b5820 0x7f0dd0073220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:31.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.443+0000 7f0dd5e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0dd00b3f30 con 0x7f0dd01439a0 2024-05-11T12:15:31.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.443+0000 7f0dd5e00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f0dd00b5c90 con 0x7f0dd00b5820 2024-05-11T12:15:31.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.443+0000 7f0dd5e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f0dd00b3ce0 con 0x7f0dd00cf820 2024-05-11T12:15:31.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.444+0000 7f0dcf400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0dd00b5820 0x7f0dd0073220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:31.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.444+0000 7f0dcf400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0dd00b5820 0x7f0dd0073220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58178/0 (socket says 172.21.15.167:58178) 2024-05-11T12:15:31.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.444+0000 7f0dcf400640 1 -- 172.21.15.167:0/319550296 learned_addr learned my addr 172.21.15.167:0/319550296 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:31.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.444+0000 7f0dce000640 1 --2- 172.21.15.167:0/319550296 >> v2:172.21.15.204:3300/0 conn(0x7f0dd00cf820 0x7f0dd00b52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:31.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.444+0000 7f0dcf400640 1 -- 172.21.15.167:0/319550296 >> v2:172.21.15.204:3300/0 conn(0x7f0dd00cf820 msgr2=0x7f0dd00b52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:31.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.444+0000 7f0dcf400640 1 --2- 172.21.15.167:0/319550296 >> v2:172.21.15.204:3300/0 conn(0x7f0dd00cf820 0x7f0dd00b52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:31.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.444+0000 7f0dcf400640 1 -- 172.21.15.167:0/319550296 >> v1:172.21.15.204:6789/0 conn(0x7f0dd01439a0 legacy=0x7f0dd00cfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:31.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.444+0000 7f0dcf400640 1 -- 172.21.15.167:0/319550296 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dd00b4210 con 0x7f0dd00b5820 2024-05-11T12:15:31.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.444+0000 7f0dce000640 1 --2- 172.21.15.167:0/319550296 >> v2:172.21.15.204:3300/0 conn(0x7f0dd00cf820 0x7f0dd00b52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:15:31.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.445+0000 7f0dcf400640 1 --2- 172.21.15.167:0/319550296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dd00b5820 0x7f0dd0073220 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f0dd00d19f0 tx=0x7f0dc802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c032588555a238 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:31.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.446+0000 7f0dcd600640 1 -- 172.21.15.167:0/319550296 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8036070 con 0x7f0dd00b5820 2024-05-11T12:15:31.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.446+0000 7f0dcd600640 1 -- 172.21.15.167:0/319550296 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dc8031600 con 0x7f0dd00b5820 2024-05-11T12:15:31.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.446+0000 7f0dcd600640 1 -- 172.21.15.167:0/319550296 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc80319d0 con 0x7f0dd00b5820 2024-05-11T12:15:31.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.446+0000 7f0dd5e00640 1 -- 172.21.15.167:0/319550296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dd00b5820 msgr2=0x7f0dd0073220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:31.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.446+0000 7f0dd5e00640 1 --2- 172.21.15.167:0/319550296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dd00b5820 0x7f0dd0073220 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f0dd00d19f0 tx=0x7f0dc802a6a0 comp rx=0 tx=0).stop 2024-05-11T12:15:31.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.446+0000 7f0dd5e00640 1 -- 172.21.15.167:0/319550296 shutdown_connections 2024-05-11T12:15:31.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.446+0000 7f0dd5e00640 1 --2- 172.21.15.167:0/319550296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dd00b5820 0x7f0dd0073220 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:31.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.446+0000 7f0dd5e00640 1 --2- 172.21.15.167:0/319550296 >> v2:172.21.15.204:3300/0 conn(0x7f0dd00cf820 0x7f0dd00b52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:31.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.446+0000 7f0dd5e00640 1 -- 172.21.15.167:0/319550296 >> 172.21.15.167:0/319550296 conn(0x7f0dd00d0c70 msgr2=0x7f0dd00d1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:31.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.447+0000 7f0dd5e00640 1 -- 172.21.15.167:0/319550296 shutdown_connections 2024-05-11T12:15:31.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.447+0000 7f0dd5e00640 1 -- 172.21.15.167:0/319550296 wait complete. 2024-05-11T12:15:31.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.448+0000 7f0dd5e00640 1 Processor -- start 2024-05-11T12:15:31.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.448+0000 7f0dd5e00640 1 -- start start 2024-05-11T12:15:31.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.449+0000 7f0dd5e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dd00b5820 0x7f0dd01383a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:31.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.449+0000 7f0dd5e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0dd00cf820 0x7f0dd013ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:31.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.449+0000 7f0dd5e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dd01439a0 0x7f0dd01f57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:31.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.449+0000 7f0dd5e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd00b4ab0 con 0x7f0dd00b5820 2024-05-11T12:15:31.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.449+0000 7f0dd5e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd00b47d0 con 0x7f0dd01439a0 2024-05-11T12:15:31.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.449+0000 7f0dd5e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0dd0073760 con 0x7f0dd00cf820 2024-05-11T12:15:31.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.450+0000 7f0dcea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dd00b5820 0x7f0dd01383a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:31.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.450+0000 7f0dcea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dd00b5820 0x7f0dd01383a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58188/0 (socket says 172.21.15.167:58188) 2024-05-11T12:15:31.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.450+0000 7f0dcea00640 1 -- 172.21.15.167:0/1263947953 learned_addr learned my addr 172.21.15.167:0/1263947953 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:31.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.450+0000 7f0dce000640 1 --2- 172.21.15.167:0/1263947953 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0dd00cf820 0x7f0dd013ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:31.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.450+0000 7f0dcea00640 1 -- 172.21.15.167:0/1263947953 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0dd00cf820 msgr2=0x7f0dd013ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:31.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.450+0000 7f0dcea00640 1 --2- 172.21.15.167:0/1263947953 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0dd00cf820 0x7f0dd013ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:31.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.450+0000 7f0dcea00640 1 -- 172.21.15.167:0/1263947953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dd01439a0 msgr2=0x7f0dd01f57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:31.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.450+0000 7f0dcea00640 1 --2- 172.21.15.167:0/1263947953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dd01439a0 0x7f0dd01f57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:31.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.450+0000 7f0dcea00640 1 -- 172.21.15.167:0/1263947953 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dd0137160 con 0x7f0dd00b5820 2024-05-11T12:15:31.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.451+0000 7f0dcea00640 1 --2- 172.21.15.167:0/1263947953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dd00b5820 0x7f0dd01383a0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f0dc000b3f0 tx=0x7f0dc000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:31.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.451+0000 7f0dc7400640 1 -- 172.21.15.167:0/1263947953 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc0015020 con 0x7f0dd00b5820 2024-05-11T12:15:31.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.451+0000 7f0dc7400640 1 -- 172.21.15.167:0/1263947953 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dc0011830 con 0x7f0dd00b5820 2024-05-11T12:15:31.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.451+0000 7f0dd5e00640 1 -- 172.21.15.167:0/1263947953 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dd0137440 con 0x7f0dd00b5820 2024-05-11T12:15:31.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.451+0000 7f0dc7400640 1 -- 172.21.15.167:0/1263947953 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc0011c20 con 0x7f0dd00b5820 2024-05-11T12:15:31.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.451+0000 7f0dd5e00640 1 -- 172.21.15.167:0/1263947953 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dd0137a00 con 0x7f0dd00b5820 2024-05-11T12:15:31.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.452+0000 7f0dd5e00640 1 -- 172.21.15.167:0/1263947953 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0dd0136f10 con 0x7f0dd00b5820 2024-05-11T12:15:31.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.453+0000 7f0dc7400640 1 -- 172.21.15.167:0/1263947953 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0dc0018020 con 0x7f0dd00b5820 2024-05-11T12:15:31.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.454+0000 7f0dc7400640 1 --2- 172.21.15.167:0/1263947953 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0db0076300 0x7f0db00787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:31.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.454+0000 7f0dc7400640 1 -- 172.21.15.167:0/1263947953 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5507+0+0 (secure 0 0 0) 0x7f0dc0097180 con 0x7f0dd00b5820 2024-05-11T12:15:31.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.454+0000 7f0dce000640 1 --2- 172.21.15.167:0/1263947953 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0db0076300 0x7f0db00787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:31.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.454+0000 7f0dc7400640 1 --2- 172.21.15.167:0/1263947953 >> [v2:172.21.15.204:6808/223313265,v1:172.21.15.204:6809/223313265] conn(0x7f0db007c8e0 0x7f0db007ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:31.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.454+0000 7f0dc7400640 1 -- 172.21.15.167:0/1263947953 --> [v2:172.21.15.204:6808/223313265,v1:172.21.15.204:6809/223313265] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0dc000de50 con 0x7f0db007c8e0 2024-05-11T12:15:31.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.454+0000 7f0dc7400640 1 -- 172.21.15.167:0/1263947953 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=104) ==== 24+0+0 (secure 0 0 0) 0x7f0dc009e050 con 0x7f0dd00b5820 2024-05-11T12:15:31.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.455+0000 7f0dcf400640 1 --2- 172.21.15.167:0/1263947953 >> [v2:172.21.15.204:6808/223313265,v1:172.21.15.204:6809/223313265] conn(0x7f0db007c8e0 0x7f0db007ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:31.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.455+0000 7f0dce000640 1 --2- 172.21.15.167:0/1263947953 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0db0076300 0x7f0db00787c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0dd00d0730 tx=0x7f0dbc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:31.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.456+0000 7f0dcf400640 1 --2- 172.21.15.167:0/1263947953 >> [v2:172.21.15.204:6808/223313265,v1:172.21.15.204:6809/223313265] conn(0x7f0db007c8e0 0x7f0db007ed20 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:31.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.458+0000 7f0dc7400640 1 -- 172.21.15.167:0/1263947953 <== osd.7 v2:172.21.15.204:6808/223313265 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0dc000de50 con 0x7f0db007c8e0 2024-05-11T12:15:31.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.473+0000 7f0dd5e00640 1 -- 172.21.15.167:0/1263947953 --> [v2:172.21.15.204:6808/223313265,v1:172.21.15.204:6809/223313265] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f0dd01431c0 con 0x7f0db007c8e0 2024-05-11T12:15:31.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.474+0000 7f0dc7400640 1 -- 172.21.15.167:0/1263947953 <== osd.7 v2:172.21.15.204:6808/223313265 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f0dd01431c0 con 0x7f0db007c8e0 2024-05-11T12:15:31.474 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T12:15:31.474 INFO:teuthology.orchestra.run.smithi167.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-11T12:15:31.474 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T12:15:31.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.475+0000 7f0dd5e00640 1 -- 172.21.15.167:0/1263947953 >> [v2:172.21.15.204:6808/223313265,v1:172.21.15.204:6809/223313265] conn(0x7f0db007c8e0 msgr2=0x7f0db007ed20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:31.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.475+0000 7f0dd5e00640 1 --2- 172.21.15.167:0/1263947953 >> [v2:172.21.15.204:6808/223313265,v1:172.21.15.204:6809/223313265] conn(0x7f0db007c8e0 0x7f0db007ed20 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:31.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.475+0000 7f0dd5e00640 1 -- 172.21.15.167:0/1263947953 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0db0076300 msgr2=0x7f0db00787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:31.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.475+0000 7f0dd5e00640 1 --2- 172.21.15.167:0/1263947953 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f0db0076300 0x7f0db00787c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0dd00d0730 tx=0x7f0dbc008040 comp rx=0 tx=0).stop 2024-05-11T12:15:31.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.475+0000 7f0dd5e00640 1 -- 172.21.15.167:0/1263947953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dd00b5820 msgr2=0x7f0dd01383a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:31.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.475+0000 7f0dd5e00640 1 --2- 172.21.15.167:0/1263947953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dd00b5820 0x7f0dd01383a0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f0dc000b3f0 tx=0x7f0dc000b8b0 comp rx=0 tx=0).stop 2024-05-11T12:15:31.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.475+0000 7f0dcf400640 1 -- 172.21.15.167:0/1263947953 reap_dead start 2024-05-11T12:15:31.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.476+0000 7f0dd5e00640 1 -- 172.21.15.167:0/1263947953 shutdown_connections 2024-05-11T12:15:31.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.476+0000 7f0dd5e00640 1 -- 172.21.15.167:0/1263947953 >> 172.21.15.167:0/1263947953 conn(0x7f0dd00d0c70 msgr2=0x7f0dd006c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:31.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.476+0000 7f0dd5e00640 1 -- 172.21.15.167:0/1263947953 shutdown_connections 2024-05-11T12:15:31.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.476+0000 7f0dd5e00640 1 -- 172.21.15.167:0/1263947953 wait complete. 2024-05-11T12:15:31.495 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-05-11T12:15:31.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.691+0000 7f657c600640 1 Processor -- start 2024-05-11T12:15:31.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.691+0000 7f657c600640 1 -- start start 2024-05-11T12:15:31.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.693+0000 7f657c600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6574161380 0x7f6574074260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:31.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.693+0000 7f657c600640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f65740b3d50 0x7f6574072950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:31.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.693+0000 7f657c600640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65740b65e0 con 0x7f657415ac80 2024-05-11T12:15:31.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.693+0000 7f657c600640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f657406e3c0 con 0x7f65740b3d50 2024-05-11T12:15:31.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.693+0000 7f657c600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f657415b060 con 0x7f6574161380 2024-05-11T12:15:31.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.693+0000 7f6579c00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f657415ac80 0x7f6574160c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:59414/0 (socket says 172.21.15.167:59414) 2024-05-11T12:15:31.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.693+0000 7f6579c00640 1 -- 172.21.15.167:0/3797201314 learned_addr learned my addr 172.21.15.167:0/3797201314 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:15:31.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.693+0000 7f657a600640 1 --2- 172.21.15.167:0/3797201314 >> v2:172.21.15.167:3300/0 conn(0x7f65740b3d50 0x7f6574072950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:31.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.694+0000 7f6579200640 1 --2- 172.21.15.167:0/3797201314 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6574161380 0x7f6574074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:31.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.694+0000 7f657a600640 1 -- 172.21.15.167:0/3797201314 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6574161380 msgr2=0x7f6574074260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:31.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.694+0000 7f657a600640 1 --2- 172.21.15.167:0/3797201314 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6574161380 0x7f6574074260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:31.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.694+0000 7f657a600640 1 -- 172.21.15.167:0/3797201314 >> v1:172.21.15.167:6789/0 conn(0x7f657415ac80 legacy=0x7f6574160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:31.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.694+0000 7f657a600640 1 -- 172.21.15.167:0/3797201314 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65740b68c0 con 0x7f65740b3d50 2024-05-11T12:15:31.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.695+0000 7f657a600640 1 --2- 172.21.15.167:0/3797201314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65740b3d50 0x7f6574072950 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f656c009d90 tx=0x7f656c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4caa719f616acbd8 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:31.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.695+0000 7f6573e00640 1 -- 172.21.15.167:0/3797201314 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656c007740 con 0x7f65740b3d50 2024-05-11T12:15:31.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.695+0000 7f6573e00640 1 -- 172.21.15.167:0/3797201314 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f656c0078e0 con 0x7f65740b3d50 2024-05-11T12:15:31.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.695+0000 7f6573e00640 1 -- 172.21.15.167:0/3797201314 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656c007cb0 con 0x7f65740b3d50 2024-05-11T12:15:31.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.696+0000 7f657c600640 1 -- 172.21.15.167:0/3797201314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65740b3d50 msgr2=0x7f6574072950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:31.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.696+0000 7f657c600640 1 --2- 172.21.15.167:0/3797201314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65740b3d50 0x7f6574072950 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f656c009d90 tx=0x7f656c02a660 comp rx=0 tx=0).stop 2024-05-11T12:15:31.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.696+0000 7f657c600640 1 -- 172.21.15.167:0/3797201314 shutdown_connections 2024-05-11T12:15:31.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.696+0000 7f657c600640 1 --2- 172.21.15.167:0/3797201314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65740b3d50 0x7f6574072950 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:31.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.696+0000 7f657c600640 1 --2- 172.21.15.167:0/3797201314 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6574161380 0x7f6574074260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:31.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.696+0000 7f657c600640 1 -- 172.21.15.167:0/3797201314 >> 172.21.15.167:0/3797201314 conn(0x7f657406c450 msgr2=0x7f657406c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:31.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.696+0000 7f657c600640 1 -- 172.21.15.167:0/3797201314 shutdown_connections 2024-05-11T12:15:31.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.696+0000 7f657c600640 1 -- 172.21.15.167:0/3797201314 wait complete. 2024-05-11T12:15:31.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.698+0000 7f657c600640 1 Processor -- start 2024-05-11T12:15:31.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.698+0000 7f657c600640 1 -- start start 2024-05-11T12:15:31.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.699+0000 7f657c600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65740b3d50 0x7f65741446f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:31.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.699+0000 7f657c600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f657415ac80 0x7f65741f6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:31.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.699+0000 7f657c600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6574161380 0x7f657415b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:31.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.699+0000 7f657c600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f65740b44f0 con 0x7f6574161380 2024-05-11T12:15:31.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.699+0000 7f657c600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f65740b41e0 con 0x7f657415ac80 2024-05-11T12:15:31.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.699+0000 7f657c600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6574072e90 con 0x7f65740b3d50 2024-05-11T12:15:31.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.699+0000 7f657a600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6574161380 0x7f657415b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:31.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.699+0000 7f657a600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6574161380 0x7f657415b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58198/0 (socket says 172.21.15.167:58198) 2024-05-11T12:15:31.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.699+0000 7f657a600640 1 -- 172.21.15.167:0/3784902535 learned_addr learned my addr 172.21.15.167:0/3784902535 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:31.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.700+0000 7f6579c00640 1 --2- 172.21.15.167:0/3784902535 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65740b3d50 0x7f65741446f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:31.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.700+0000 7f657a600640 1 -- 172.21.15.167:0/3784902535 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65740b3d50 msgr2=0x7f65741446f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:31.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.700+0000 7f657a600640 1 --2- 172.21.15.167:0/3784902535 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65740b3d50 0x7f65741446f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:31.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.700+0000 7f657a600640 1 -- 172.21.15.167:0/3784902535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f657415ac80 msgr2=0x7f65741f6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:31.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.700+0000 7f6579200640 1 --2- 172.21.15.167:0/3784902535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f657415ac80 0x7f65741f6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:31.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.700+0000 7f657a600640 1 --2- 172.21.15.167:0/3784902535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f657415ac80 0x7f65741f6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:31.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.700+0000 7f657a600640 1 -- 172.21.15.167:0/3784902535 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65741f3e60 con 0x7f6574161380 2024-05-11T12:15:31.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.700+0000 7f6579c00640 1 --2- 172.21.15.167:0/3784902535 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65740b3d50 0x7f65741446f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:15:31.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.700+0000 7f657a600640 1 --2- 172.21.15.167:0/3784902535 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6574161380 0x7f657415b210 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f656c009eb0 tx=0x7f656c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:31.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.700+0000 7f6571600640 1 -- 172.21.15.167:0/3784902535 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656c040070 con 0x7f6574161380 2024-05-11T12:15:31.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.701+0000 7f6571600640 1 -- 172.21.15.167:0/3784902535 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f656c031bc0 con 0x7f6574161380 2024-05-11T12:15:31.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.701+0000 7f6571600640 1 -- 172.21.15.167:0/3784902535 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656c045460 con 0x7f6574161380 2024-05-11T12:15:31.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.701+0000 7f657c600640 1 -- 172.21.15.167:0/3784902535 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65741f6d90 con 0x7f6574161380 2024-05-11T12:15:31.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.701+0000 7f657c600640 1 -- 172.21.15.167:0/3784902535 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65741f40d0 con 0x7f6574161380 2024-05-11T12:15:31.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.702+0000 7f657c600640 1 -- 172.21.15.167:0/3784902535 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65740694b0 con 0x7f6574161380 2024-05-11T12:15:31.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.703+0000 7f6571600640 1 -- 172.21.15.167:0/3784902535 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f656c039040 con 0x7f6574161380 2024-05-11T12:15:31.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.704+0000 7f6571600640 1 --2- 172.21.15.167:0/3784902535 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f65540762d0 0x7f6554078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:31.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.704+0000 7f6571600640 1 -- 172.21.15.167:0/3784902535 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5507+0+0 (secure 0 0 0) 0x7f656c0b8e50 con 0x7f6574161380 2024-05-11T12:15:31.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.704+0000 7f6579c00640 1 --2- 172.21.15.167:0/3784902535 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f65540762d0 0x7f6554078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:31.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.705+0000 7f6579c00640 1 --2- 172.21.15.167:0/3784902535 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f65540762d0 0x7f6554078790 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6568000f60 tx=0x7f6568009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:31.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.709+0000 7f6571600640 1 -- 172.21.15.167:0/3784902535 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f656c082860 con 0x7f6574161380 2024-05-11T12:15:31.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.928+0000 7f657c600640 1 -- 172.21.15.167:0/3784902535 --> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f65740b0b60 con 0x7f65540762d0 2024-05-11T12:15:31.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.928+0000 7f6571600640 1 -- 172.21.15.167:0/3784902535 <== mgr.4106 v2:172.21.15.167:6832/578352559 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f65740b0b60 con 0x7f65540762d0 2024-05-11T12:15:31.928 INFO:teuthology.orchestra.run.smithi167.stderr:instructed osd(s) 7 to deep-scrub 2024-05-11T12:15:31.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.931+0000 7f657c600640 1 -- 172.21.15.167:0/3784902535 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f65540762d0 msgr2=0x7f6554078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:31.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.931+0000 7f657c600640 1 --2- 172.21.15.167:0/3784902535 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f65540762d0 0x7f6554078790 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6568000f60 tx=0x7f6568009040 comp rx=0 tx=0).stop 2024-05-11T12:15:31.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.931+0000 7f657c600640 1 -- 172.21.15.167:0/3784902535 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6574161380 msgr2=0x7f657415b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:31.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.931+0000 7f657c600640 1 --2- 172.21.15.167:0/3784902535 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6574161380 0x7f657415b210 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f656c009eb0 tx=0x7f656c004a40 comp rx=0 tx=0).stop 2024-05-11T12:15:31.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.932+0000 7f657c600640 1 -- 172.21.15.167:0/3784902535 shutdown_connections 2024-05-11T12:15:31.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.932+0000 7f657c600640 1 --2- 172.21.15.167:0/3784902535 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6574161380 0x7f657415b210 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:31.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.932+0000 7f657c600640 1 --2- 172.21.15.167:0/3784902535 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f65540762d0 0x7f6554078790 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:31.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.932+0000 7f657c600640 1 --2- 172.21.15.167:0/3784902535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f657415ac80 0x7f65741f6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:31.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.932+0000 7f657c600640 1 --2- 172.21.15.167:0/3784902535 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65740b3d50 0x7f65741446f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:31.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.932+0000 7f657c600640 1 -- 172.21.15.167:0/3784902535 >> 172.21.15.167:0/3784902535 conn(0x7f657406c450 msgr2=0x7f6574159b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:31.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.932+0000 7f657c600640 1 -- 172.21.15.167:0/3784902535 shutdown_connections 2024-05-11T12:15:31.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:31.932+0000 7f657c600640 1 -- 172.21.15.167:0/3784902535 wait complete. 2024-05-11T12:15:31.950 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T12:15:32.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.141+0000 7f7f33a00640 1 Processor -- start 2024-05-11T12:15:32.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.141+0000 7f7f33a00640 1 -- start start 2024-05-11T12:15:32.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.143+0000 7f7f33a00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7f2c0b3ea0 0x7f7f2c161490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:32.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.143+0000 7f7f33a00640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f2c15df30 con 0x7f7f2c1619d0 2024-05-11T12:15:32.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.143+0000 7f7f33a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f2c15e210 con 0x7f7f2c157750 2024-05-11T12:15:32.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.143+0000 7f7f33a00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f7f2c161000 con 0x7f7f2c0b3ea0 2024-05-11T12:15:32.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.143+0000 7f7f2be00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7f2c0b3ea0 0x7f7f2c161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:32.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.143+0000 7f7f31a00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f7f2c1619d0 0x7f7f2c06d460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:59416/0 (socket says 172.21.15.167:59416) 2024-05-11T12:15:32.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.143+0000 7f7f2be00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7f2c0b3ea0 0x7f7f2c161490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58208/0 (socket says 172.21.15.167:58208) 2024-05-11T12:15:32.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.143+0000 7f7f31a00640 1 -- 172.21.15.167:0/967819526 learned_addr learned my addr 172.21.15.167:0/967819526 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T12:15:32.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.143+0000 7f7f2be00640 1 -- 172.21.15.167:0/967819526 >> v1:172.21.15.204:6789/0 conn(0x7f7f2c157750 legacy=0x7f7f2c15d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:32.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.144+0000 7f7f2be00640 1 -- 172.21.15.167:0/967819526 >> v1:172.21.15.167:6789/0 conn(0x7f7f2c1619d0 legacy=0x7f7f2c06d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:32.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.144+0000 7f7f2be00640 1 -- 172.21.15.167:0/967819526 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f2c15e4f0 con 0x7f7f2c0b3ea0 2024-05-11T12:15:32.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.145+0000 7f7f2be00640 1 --2- 172.21.15.167:0/967819526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f2c0b3ea0 0x7f7f2c161490 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f7f180097a0 tx=0x7f7f1802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df42218fe04dd91f server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:32.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.145+0000 7f7f2b400640 1 -- 172.21.15.167:0/967819526 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f18037070 con 0x7f7f2c0b3ea0 2024-05-11T12:15:32.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.145+0000 7f7f2b400640 1 -- 172.21.15.167:0/967819526 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f18032440 con 0x7f7f2c0b3ea0 2024-05-11T12:15:32.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.145+0000 7f7f2b400640 1 -- 172.21.15.167:0/967819526 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f18032810 con 0x7f7f2c0b3ea0 2024-05-11T12:15:32.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.146+0000 7f7f33a00640 1 -- 172.21.15.167:0/967819526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f2c0b3ea0 msgr2=0x7f7f2c161490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:32.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.146+0000 7f7f33a00640 1 --2- 172.21.15.167:0/967819526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f2c0b3ea0 0x7f7f2c161490 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f7f180097a0 tx=0x7f7f1802a180 comp rx=0 tx=0).stop 2024-05-11T12:15:32.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.146+0000 7f7f33a00640 1 -- 172.21.15.167:0/967819526 shutdown_connections 2024-05-11T12:15:32.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.146+0000 7f7f33a00640 1 --2- 172.21.15.167:0/967819526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f2c0b3ea0 0x7f7f2c161490 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:32.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.146+0000 7f7f33a00640 1 -- 172.21.15.167:0/967819526 >> 172.21.15.167:0/967819526 conn(0x7f7f2c0cd8f0 msgr2=0x7f7f2c0cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:32.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.146+0000 7f7f33a00640 1 -- 172.21.15.167:0/967819526 shutdown_connections 2024-05-11T12:15:32.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.146+0000 7f7f33a00640 1 -- 172.21.15.167:0/967819526 wait complete. 2024-05-11T12:15:32.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.148+0000 7f7f33a00640 1 Processor -- start 2024-05-11T12:15:32.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.148+0000 7f7f33a00640 1 -- start start 2024-05-11T12:15:32.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.149+0000 7f7f33a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7f2c0b3ea0 0x7f7f2c158470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:32.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.149+0000 7f7f33a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f2c157750 0x7f7f2c1e6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:32.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.149+0000 7f7f33a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f2c1619d0 0x7f7f2c1e70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:32.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.149+0000 7f7f33a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c06dcf0 con 0x7f7f2c157750 2024-05-11T12:15:32.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.149+0000 7f7f33a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c06db70 con 0x7f7f2c1619d0 2024-05-11T12:15:32.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.149+0000 7f7f33a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f7f2c0cf1c0 con 0x7f7f2c0b3ea0 2024-05-11T12:15:32.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.150+0000 7f7f2be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f2c157750 0x7f7f2c1e6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:32.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.150+0000 7f7f2be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f2c157750 0x7f7f2c1e6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58222/0 (socket says 172.21.15.167:58222) 2024-05-11T12:15:32.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.150+0000 7f7f2be00640 1 -- 172.21.15.167:0/555476065 learned_addr learned my addr 172.21.15.167:0/555476065 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:32.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.150+0000 7f7f2be00640 1 -- 172.21.15.167:0/555476065 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7f2c0b3ea0 msgr2=0x7f7f2c158470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:32.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.150+0000 7f7f2be00640 1 --2- 172.21.15.167:0/555476065 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7f2c0b3ea0 0x7f7f2c158470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:32.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.150+0000 7f7f2be00640 1 -- 172.21.15.167:0/555476065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f2c1619d0 msgr2=0x7f7f2c1e70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:32.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.150+0000 7f7f2be00640 1 --2- 172.21.15.167:0/555476065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f2c1619d0 0x7f7f2c1e70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:32.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.150+0000 7f7f2be00640 1 -- 172.21.15.167:0/555476065 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f2c135340 con 0x7f7f2c157750 2024-05-11T12:15:32.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.150+0000 7f7f2be00640 1 --2- 172.21.15.167:0/555476065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f2c157750 0x7f7f2c1e6b80 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f7f18000c00 tx=0x7f7f180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:32.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.151+0000 7f7f28c00640 1 -- 172.21.15.167:0/555476065 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f18037040 con 0x7f7f2c157750 2024-05-11T12:15:32.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.151+0000 7f7f28c00640 1 -- 172.21.15.167:0/555476065 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f18042070 con 0x7f7f2c157750 2024-05-11T12:15:32.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.151+0000 7f7f28c00640 1 -- 172.21.15.167:0/555476065 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f18030a70 con 0x7f7f2c157750 2024-05-11T12:15:32.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.151+0000 7f7f33a00640 1 -- 172.21.15.167:0/555476065 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f2c1355b0 con 0x7f7f2c157750 2024-05-11T12:15:32.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.151+0000 7f7f33a00640 1 -- 172.21.15.167:0/555476065 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f2c135b70 con 0x7f7f2c157750 2024-05-11T12:15:32.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.152+0000 7f7f33a00640 1 -- 172.21.15.167:0/555476065 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f2c069590 con 0x7f7f2c157750 2024-05-11T12:15:32.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.153+0000 7f7f28c00640 1 -- 172.21.15.167:0/555476065 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7f18030c10 con 0x7f7f2c157750 2024-05-11T12:15:32.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.154+0000 7f7f28c00640 1 --2- 172.21.15.167:0/555476065 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f7f0c0762d0 0x7f7f0c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:32.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.154+0000 7f7f28c00640 1 -- 172.21.15.167:0/555476065 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5507+0+0 (secure 0 0 0) 0x7f7f180b7fa0 con 0x7f7f2c157750 2024-05-11T12:15:32.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.155+0000 7f7f31000640 1 --2- 172.21.15.167:0/555476065 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f7f0c0762d0 0x7f7f0c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:32.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.155+0000 7f7f31000640 1 --2- 172.21.15.167:0/555476065 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f7f0c0762d0 0x7f7f0c078790 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7f20003cc0 tx=0x7f7f200079b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:32.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.159+0000 7f7f28c00640 1 -- 172.21.15.167:0/555476065 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7f180819b0 con 0x7f7f2c157750 2024-05-11T12:15:32.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.366+0000 7f7f33a00640 1 -- 172.21.15.167:0/555476065 --> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7f2c15b450 con 0x7f7f0c0762d0 2024-05-11T12:15:32.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.367+0000 7f7f28c00640 1 -- 172.21.15.167:0/555476065 <== mgr.4106 v2:172.21.15.167:6832/578352559 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13357 (secure 0 0 0) 0x7f7f2c15b450 con 0x7f7f0c0762d0 2024-05-11T12:15:32.367 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:15:32.367 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2024-05-11T12:15:32.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.370+0000 7f7f33a00640 1 -- 172.21.15.167:0/555476065 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f7f0c0762d0 msgr2=0x7f7f0c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:32.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.370+0000 7f7f33a00640 1 --2- 172.21.15.167:0/555476065 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f7f0c0762d0 0x7f7f0c078790 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7f20003cc0 tx=0x7f7f200079b0 comp rx=0 tx=0).stop 2024-05-11T12:15:32.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.370+0000 7f7f33a00640 1 -- 172.21.15.167:0/555476065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f2c157750 msgr2=0x7f7f2c1e6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:32.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.370+0000 7f7f33a00640 1 --2- 172.21.15.167:0/555476065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f2c157750 0x7f7f2c1e6b80 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f7f18000c00 tx=0x7f7f180046c0 comp rx=0 tx=0).stop 2024-05-11T12:15:32.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.370+0000 7f7f33a00640 1 -- 172.21.15.167:0/555476065 shutdown_connections 2024-05-11T12:15:32.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.370+0000 7f7f33a00640 1 --2- 172.21.15.167:0/555476065 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f7f0c0762d0 0x7f7f0c078790 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:32.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.370+0000 7f7f33a00640 1 --2- 172.21.15.167:0/555476065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f2c1619d0 0x7f7f2c1e70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:32.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.370+0000 7f7f33a00640 1 --2- 172.21.15.167:0/555476065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f2c157750 0x7f7f2c1e6b80 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:32.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.370+0000 7f7f33a00640 1 --2- 172.21.15.167:0/555476065 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7f2c0b3ea0 0x7f7f2c158470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:32.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.370+0000 7f7f33a00640 1 -- 172.21.15.167:0/555476065 >> 172.21.15.167:0/555476065 conn(0x7f7f2c0cd8f0 msgr2=0x7f7f2c15f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:32.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.371+0000 7f7f33a00640 1 -- 172.21.15.167:0/555476065 shutdown_connections 2024-05-11T12:15:32.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:32.371+0000 7f7f33a00640 1 -- 172.21.15.167:0/555476065 wait complete. 2024-05-11T12:15:32.390 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":247,"stamp":"2024-05-11T12:15:31.325967+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":260668,"kb_used_data":11208,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714052,"statfs":{"total":773094113280,"available":772827189248,"internally_reserved":0,"allocated":11476992,"data_stored":5877416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.150964"},"pg_stats":[{"pgid":"1.0","version":"19'72","reported_seq":260,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-11T12:15:20.740225+0000","last_change":"2024-05-11T12:10:35.066673+0000","last_active":"2024-05-11T12:15:20.740225+0000","last_peered":"2024-05-11T12:15:20.740225+0000","last_clean":"2024-05-11T12:15:20.740225+0000","last_became_active":"2024-05-11T12:10:35.065942+0000","last_became_peered":"2024-05-11T12:10:35.065942+0000","last_unstale":"2024-05-11T12:15:20.740225+0000","last_undegraded":"2024-05-11T12:15:20.740225+0000","last_fullsized":"2024-05-11T12:15:20.740225+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:10:33.601136+0000","last_clean_scrub_stamp":"2024-05-11T12:10:33.601136+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:12:02.133182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476798,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33016,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338824,"statfs":{"total":96636764160,"available":96602955776,"internally_reserved":0,"allocated":1880064,"data_stored":1177453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476799,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32436,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1286144,"data_stored":587085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476798,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32440,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1286144,"data_stored":587085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476800,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32440,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1286144,"data_stored":587085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509503,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32440,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1286144,"data_stored":587085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509504,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32440,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1286144,"data_stored":587085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509503,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33016,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338824,"statfs":{"total":96636764160,"available":96602955776,"internally_reserved":0,"allocated":1880064,"data_stored":1177453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542208,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32440,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1286144,"data_stored":587085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T12:15:32.393 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-05-11T12:10:33.601136+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=12, tm_min=10, tm_sec=33, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=12, tm_min=15, tm_sec=25, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T12:15:32.393 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-11T12:15:37.943 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T12:15:37.951 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T12:15:52.393 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T12:15:52.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.611+0000 7ff076a00640 1 Processor -- start 2024-05-11T12:15:52.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.611+0000 7ff076a00640 1 -- start start 2024-05-11T12:15:52.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.613+0000 7ff076a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff0700af1f0 0x7ff0700cf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:52.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.613+0000 7ff076a00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff0700af5d0 0x7ff070073ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:52.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.613+0000 7ff076a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff07006d460 0x7ff07006d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:52.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.613+0000 7ff076a00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7ff0701608d0 con 0x7ff0700af5d0 2024-05-11T12:15:52.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.613+0000 7ff076a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7ff07006de10 con 0x7ff0700af1f0 2024-05-11T12:15:52.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.613+0000 7ff076a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff0700cf700 con 0x7ff07006d460 2024-05-11T12:15:52.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.613+0000 7ff067400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff0700af5d0 0x7ff070073ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:52.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.613+0000 7ff067400640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff0700af5d0 0x7ff070073ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:37294/0 (socket says 172.21.15.167:37294) 2024-05-11T12:15:52.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.613+0000 7ff067400640 1 -- 172.21.15.167:0/1262093325 learned_addr learned my addr 172.21.15.167:0/1262093325 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:52.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.614+0000 7ff074a00640 1 --2- 172.21.15.167:0/1262093325 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff07006d460 0x7ff07006d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:52.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.614+0000 7ff06fe00640 1 --2- 172.21.15.167:0/1262093325 >> v2:172.21.15.204:3300/0 conn(0x7ff0700af1f0 0x7ff0700cf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:52.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.614+0000 7ff067400640 1 -- 172.21.15.167:0/1262093325 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff07006d460 msgr2=0x7ff07006d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:52.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.614+0000 7ff067400640 1 --2- 172.21.15.167:0/1262093325 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff07006d460 0x7ff07006d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:52.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.614+0000 7ff067400640 1 -- 172.21.15.167:0/1262093325 >> v2:172.21.15.204:3300/0 conn(0x7ff0700af1f0 msgr2=0x7ff0700cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:52.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.614+0000 7ff067400640 1 --2- 172.21.15.167:0/1262093325 >> v2:172.21.15.204:3300/0 conn(0x7ff0700af1f0 0x7ff0700cf1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:52.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.614+0000 7ff067400640 1 -- 172.21.15.167:0/1262093325 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff07006fe10 con 0x7ff0700af5d0 2024-05-11T12:15:52.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.615+0000 7ff067400640 1 --2- 172.21.15.167:0/1262093325 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0700af5d0 0x7ff070073ce0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7ff0700ce8d0 tx=0x7ff05802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=256f64f2d2dbef4b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:52.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.615+0000 7ff06f400640 1 -- 172.21.15.167:0/1262093325 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff058037070 con 0x7ff0700af5d0 2024-05-11T12:15:52.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.615+0000 7ff06f400640 1 -- 172.21.15.167:0/1262093325 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff058032440 con 0x7ff0700af5d0 2024-05-11T12:15:52.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.615+0000 7ff06f400640 1 -- 172.21.15.167:0/1262093325 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff058032810 con 0x7ff0700af5d0 2024-05-11T12:15:52.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.616+0000 7ff076a00640 1 -- 172.21.15.167:0/1262093325 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0700af5d0 msgr2=0x7ff070073ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:52.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.616+0000 7ff076a00640 1 --2- 172.21.15.167:0/1262093325 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0700af5d0 0x7ff070073ce0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7ff0700ce8d0 tx=0x7ff05802a180 comp rx=0 tx=0).stop 2024-05-11T12:15:52.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.616+0000 7ff076a00640 1 -- 172.21.15.167:0/1262093325 shutdown_connections 2024-05-11T12:15:52.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.616+0000 7ff076a00640 1 --2- 172.21.15.167:0/1262093325 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff07006d460 0x7ff07006d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:52.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.616+0000 7ff076a00640 1 --2- 172.21.15.167:0/1262093325 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0700af5d0 0x7ff070073ce0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:52.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.616+0000 7ff076a00640 1 --2- 172.21.15.167:0/1262093325 >> v2:172.21.15.204:3300/0 conn(0x7ff0700af1f0 0x7ff0700cf1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:52.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.616+0000 7ff076a00640 1 -- 172.21.15.167:0/1262093325 >> 172.21.15.167:0/1262093325 conn(0x7ff0700cd8f0 msgr2=0x7ff0700cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:52.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.616+0000 7ff076a00640 1 -- 172.21.15.167:0/1262093325 shutdown_connections 2024-05-11T12:15:52.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.616+0000 7ff076a00640 1 -- 172.21.15.167:0/1262093325 wait complete. 2024-05-11T12:15:52.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.618+0000 7ff076a00640 1 Processor -- start 2024-05-11T12:15:52.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.618+0000 7ff076a00640 1 -- start start 2024-05-11T12:15:52.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.619+0000 7ff076a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff07006d460 0x7ff070163440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:52.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.619+0000 7ff076a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0700af1f0 0x7ff0701496d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:52.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.619+0000 7ff076a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0700af5d0 0x7ff0700b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:52.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.619+0000 7ff076a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff070160280 con 0x7ff0700af5d0 2024-05-11T12:15:52.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.619+0000 7ff076a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff07015ffa0 con 0x7ff07006d460 2024-05-11T12:15:52.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.619+0000 7ff076a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff070160560 con 0x7ff0700af1f0 2024-05-11T12:15:52.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.620+0000 7ff074a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0700af5d0 0x7ff0700b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:52.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.620+0000 7ff074a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0700af5d0 0x7ff0700b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:37296/0 (socket says 172.21.15.167:37296) 2024-05-11T12:15:52.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.620+0000 7ff074a00640 1 -- 172.21.15.167:0/3551844753 learned_addr learned my addr 172.21.15.167:0/3551844753 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:52.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.620+0000 7ff06fe00640 1 --2- 172.21.15.167:0/3551844753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff07006d460 0x7ff070163440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:52.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.620+0000 7ff067400640 1 --2- 172.21.15.167:0/3551844753 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0700af1f0 0x7ff0701496d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:52.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.620+0000 7ff074a00640 1 -- 172.21.15.167:0/3551844753 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0700af1f0 msgr2=0x7ff0701496d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:52.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.620+0000 7ff074a00640 1 --2- 172.21.15.167:0/3551844753 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0700af1f0 0x7ff0701496d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:52.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.620+0000 7ff074a00640 1 -- 172.21.15.167:0/3551844753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff07006d460 msgr2=0x7ff070163440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:52.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.620+0000 7ff074a00640 1 --2- 172.21.15.167:0/3551844753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff07006d460 0x7ff070163440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:52.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.620+0000 7ff074a00640 1 -- 172.21.15.167:0/3551844753 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0701f9110 con 0x7ff0700af5d0 2024-05-11T12:15:52.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.621+0000 7ff074a00640 1 --2- 172.21.15.167:0/3551844753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0700af5d0 0x7ff0700b1b40 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7ff068007e40 tx=0x7ff06800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:52.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.621+0000 7ff06cc00640 1 -- 172.21.15.167:0/3551844753 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff068014620 con 0x7ff0700af5d0 2024-05-11T12:15:52.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.621+0000 7ff06cc00640 1 -- 172.21.15.167:0/3551844753 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff068019070 con 0x7ff0700af5d0 2024-05-11T12:15:52.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.621+0000 7ff06cc00640 1 -- 172.21.15.167:0/3551844753 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff068014a20 con 0x7ff0700af5d0 2024-05-11T12:15:52.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.621+0000 7ff076a00640 1 -- 172.21.15.167:0/3551844753 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0700b3d90 con 0x7ff0700af5d0 2024-05-11T12:15:52.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.621+0000 7ff076a00640 1 -- 172.21.15.167:0/3551844753 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0700b4350 con 0x7ff0700af5d0 2024-05-11T12:15:52.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.622+0000 7ff076a00640 1 -- 172.21.15.167:0/3551844753 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff07015f920 con 0x7ff0700af5d0 2024-05-11T12:15:52.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.623+0000 7ff06cc00640 1 -- 172.21.15.167:0/3551844753 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff068014bc0 con 0x7ff0700af5d0 2024-05-11T12:15:52.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.624+0000 7ff06cc00640 1 --2- 172.21.15.167:0/3551844753 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff04c0762d0 0x7ff04c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:52.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.624+0000 7ff06cc00640 1 -- 172.21.15.167:0/3551844753 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5507+0+0 (secure 0 0 0) 0x7ff068097e70 con 0x7ff0700af5d0 2024-05-11T12:15:52.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.625+0000 7ff06fe00640 1 --2- 172.21.15.167:0/3551844753 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff04c0762d0 0x7ff04c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:52.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.625+0000 7ff06fe00640 1 --2- 172.21.15.167:0/3551844753 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff04c0762d0 0x7ff04c078790 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff060006fd0 tx=0x7ff060008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:52.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.629+0000 7ff06cc00640 1 -- 172.21.15.167:0/3551844753 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff07015f920 con 0x7ff0700af5d0 2024-05-11T12:15:52.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.838+0000 7ff076a00640 1 -- 172.21.15.167:0/3551844753 --> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff0700b3b80 con 0x7ff04c0762d0 2024-05-11T12:15:52.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.840+0000 7ff06cc00640 1 -- 172.21.15.167:0/3551844753 <== mgr.4106 v2:172.21.15.167:6832/578352559 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13364 (secure 0 0 0) 0x7ff0700b3b80 con 0x7ff04c0762d0 2024-05-11T12:15:52.841 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:15:52.841 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2024-05-11T12:15:52.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.843+0000 7ff076a00640 1 -- 172.21.15.167:0/3551844753 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff04c0762d0 msgr2=0x7ff04c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:52.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.843+0000 7ff076a00640 1 --2- 172.21.15.167:0/3551844753 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff04c0762d0 0x7ff04c078790 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff060006fd0 tx=0x7ff060008040 comp rx=0 tx=0).stop 2024-05-11T12:15:52.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.843+0000 7ff076a00640 1 -- 172.21.15.167:0/3551844753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0700af5d0 msgr2=0x7ff0700b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:52.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.843+0000 7ff076a00640 1 --2- 172.21.15.167:0/3551844753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0700af5d0 0x7ff0700b1b40 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7ff068007e40 tx=0x7ff06800e4b0 comp rx=0 tx=0).stop 2024-05-11T12:15:52.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.843+0000 7ff076a00640 1 -- 172.21.15.167:0/3551844753 shutdown_connections 2024-05-11T12:15:52.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.843+0000 7ff076a00640 1 --2- 172.21.15.167:0/3551844753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0700af5d0 0x7ff0700b1b40 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:52.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.843+0000 7ff076a00640 1 --2- 172.21.15.167:0/3551844753 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0700af1f0 0x7ff0701496d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:52.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.843+0000 7ff076a00640 1 --2- 172.21.15.167:0/3551844753 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7ff04c0762d0 0x7ff04c078790 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:52.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.843+0000 7ff076a00640 1 --2- 172.21.15.167:0/3551844753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff07006d460 0x7ff070163440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:52.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.843+0000 7ff076a00640 1 -- 172.21.15.167:0/3551844753 >> 172.21.15.167:0/3551844753 conn(0x7ff0700cd8f0 msgr2=0x7ff070073900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:52.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.843+0000 7ff076a00640 1 -- 172.21.15.167:0/3551844753 shutdown_connections 2024-05-11T12:15:52.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:52.843+0000 7ff076a00640 1 -- 172.21.15.167:0/3551844753 wait complete. 2024-05-11T12:15:52.862 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":257,"stamp":"2024-05-11T12:15:51.331635+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":260660,"kb_used_data":11208,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714060,"statfs":{"total":773094113280,"available":772827197440,"internally_reserved":0,"allocated":11476992,"data_stored":5877416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003528"},"pg_stats":[{"pgid":"1.0","version":"104'74","reported_seq":271,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-11T12:15:31.995180+0000","last_change":"2024-05-11T12:15:31.995249+0000","last_active":"2024-05-11T12:15:31.995180+0000","last_peered":"2024-05-11T12:15:31.995180+0000","last_clean":"2024-05-11T12:15:31.995180+0000","last_became_active":"2024-05-11T12:10:35.065942+0000","last_became_peered":"2024-05-11T12:10:35.065942+0000","last_unstale":"2024-05-11T12:15:31.995180+0000","last_undegraded":"2024-05-11T12:15:31.995180+0000","last_fullsized":"2024-05-11T12:15:31.995180+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"104'74","last_scrub_stamp":"2024-05-11T12:15:31.995010+0000","last_deep_scrub":"104'74","last_deep_scrub_stamp":"2024-05-11T12:15:31.995010+0000","last_clean_scrub_stamp":"2024-05-11T12:15:31.995010+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:47:10.598826+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476802,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33012,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338828,"statfs":{"total":96636764160,"available":96602959872,"internally_reserved":0,"allocated":1880064,"data_stored":1177453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476803,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32436,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1286144,"data_stored":587085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476802,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32440,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1286144,"data_stored":587085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476804,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32440,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1286144,"data_stored":587085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509507,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32440,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1286144,"data_stored":587085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509508,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32440,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1286144,"data_stored":587085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509507,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33012,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338828,"statfs":{"total":96636764160,"available":96602959872,"internally_reserved":0,"allocated":1880064,"data_stored":1177453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542212,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32440,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1286144,"data_stored":587085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T12:15:52.863 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-05-11T12:15:53.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.052+0000 7f5136c00640 1 Processor -- start 2024-05-11T12:15:53.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.053+0000 7f5136c00640 1 -- start start 2024-05-11T12:15:53.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.054+0000 7f5136c00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5130161380 0x7f5130074260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:53.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.054+0000 7f5136c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51300b3d50 0x7f5130072950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:53.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.054+0000 7f5136c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51300b65e0 con 0x7f513015ac80 2024-05-11T12:15:53.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.054+0000 7f5136c00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f513006e3c0 con 0x7f5130161380 2024-05-11T12:15:53.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.054+0000 7f5136c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f513015b060 con 0x7f51300b3d50 2024-05-11T12:15:53.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.055+0000 7f5127e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5130161380 0x7f5130074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:53.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.055+0000 7f5127e00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5130161380 0x7f5130074260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:37306/0 (socket says 172.21.15.167:37306) 2024-05-11T12:15:53.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.055+0000 7f5127e00640 1 -- 172.21.15.167:0/1442693100 learned_addr learned my addr 172.21.15.167:0/1442693100 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:53.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.055+0000 7f5134c00640 1 --2- 172.21.15.167:0/1442693100 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51300b3d50 0x7f5130072950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:53.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.055+0000 7f5127e00640 1 -- 172.21.15.167:0/1442693100 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51300b3d50 msgr2=0x7f5130072950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:53.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.055+0000 7f5127e00640 1 --2- 172.21.15.167:0/1442693100 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51300b3d50 0x7f5130072950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:53.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.055+0000 7f5127e00640 1 -- 172.21.15.167:0/1442693100 >> v1:172.21.15.204:6789/0 conn(0x7f513015ac80 legacy=0x7f5130160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:53.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.055+0000 7f5127e00640 1 -- 172.21.15.167:0/1442693100 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51300b68c0 con 0x7f5130161380 2024-05-11T12:15:53.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.055+0000 7f5134c00640 1 --2- 172.21.15.167:0/1442693100 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51300b3d50 0x7f5130072950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:15:53.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.056+0000 7f5127e00640 1 --2- 172.21.15.167:0/1442693100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5130161380 0x7f5130074260 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f5118009870 tx=0x7f511802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d553e06f962d8f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:53.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.057+0000 7f512f400640 1 -- 172.21.15.167:0/1442693100 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5118037070 con 0x7f5130161380 2024-05-11T12:15:53.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.057+0000 7f512f400640 1 -- 172.21.15.167:0/1442693100 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5118032440 con 0x7f5130161380 2024-05-11T12:15:53.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.057+0000 7f512f400640 1 -- 172.21.15.167:0/1442693100 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5118032810 con 0x7f5130161380 2024-05-11T12:15:53.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.057+0000 7f5136c00640 1 -- 172.21.15.167:0/1442693100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5130161380 msgr2=0x7f5130074260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:53.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.057+0000 7f5136c00640 1 --2- 172.21.15.167:0/1442693100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5130161380 0x7f5130074260 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f5118009870 tx=0x7f511802a1c0 comp rx=0 tx=0).stop 2024-05-11T12:15:53.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.057+0000 7f5136c00640 1 -- 172.21.15.167:0/1442693100 shutdown_connections 2024-05-11T12:15:53.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.057+0000 7f5136c00640 1 --2- 172.21.15.167:0/1442693100 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51300b3d50 0x7f5130072950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:53.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.057+0000 7f5136c00640 1 --2- 172.21.15.167:0/1442693100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5130161380 0x7f5130074260 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:53.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.057+0000 7f5136c00640 1 -- 172.21.15.167:0/1442693100 >> 172.21.15.167:0/1442693100 conn(0x7f513006c450 msgr2=0x7f513006c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:53.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.057+0000 7f5136c00640 1 -- 172.21.15.167:0/1442693100 shutdown_connections 2024-05-11T12:15:53.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.058+0000 7f5136c00640 1 -- 172.21.15.167:0/1442693100 wait complete. 2024-05-11T12:15:53.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.059+0000 7f5136c00640 1 Processor -- start 2024-05-11T12:15:53.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.059+0000 7f5136c00640 1 -- start start 2024-05-11T12:15:53.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.060+0000 7f5136c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51300b3d50 0x7f5130143a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:53.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.060+0000 7f5136c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f513015ac80 0x7f5130143f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:53.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.060+0000 7f5136c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5130161380 0x7f51300ae820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:53.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.060+0000 7f5136c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f51300b44f0 con 0x7f513015ac80 2024-05-11T12:15:53.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.060+0000 7f5136c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f51300b41a0 con 0x7f5130161380 2024-05-11T12:15:53.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.060+0000 7f5136c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5130072e90 con 0x7f51300b3d50 2024-05-11T12:15:53.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.060+0000 7f5127e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f513015ac80 0x7f5130143f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:53.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.061+0000 7f5127e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f513015ac80 0x7f5130143f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:37320/0 (socket says 172.21.15.167:37320) 2024-05-11T12:15:53.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.061+0000 7f5127e00640 1 -- 172.21.15.167:0/162299914 learned_addr learned my addr 172.21.15.167:0/162299914 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:53.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.061+0000 7f512fe00640 1 --2- 172.21.15.167:0/162299914 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51300b3d50 0x7f5130143a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:53.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.061+0000 7f5127e00640 1 -- 172.21.15.167:0/162299914 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51300b3d50 msgr2=0x7f5130143a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:53.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.061+0000 7f5127e00640 1 --2- 172.21.15.167:0/162299914 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51300b3d50 0x7f5130143a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:53.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.061+0000 7f5134c00640 1 --2- 172.21.15.167:0/162299914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5130161380 0x7f51300ae820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:53.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.061+0000 7f5127e00640 1 -- 172.21.15.167:0/162299914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5130161380 msgr2=0x7f51300ae820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:53.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.061+0000 7f5127e00640 1 --2- 172.21.15.167:0/162299914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5130161380 0x7f51300ae820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:53.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.061+0000 7f5127e00640 1 -- 172.21.15.167:0/162299914 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f513015b210 con 0x7f513015ac80 2024-05-11T12:15:53.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.061+0000 7f5134c00640 1 --2- 172.21.15.167:0/162299914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5130161380 0x7f51300ae820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:15:53.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.061+0000 7f5127e00640 1 --2- 172.21.15.167:0/162299914 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f513015ac80 0x7f5130143f40 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f5118000c00 tx=0x7f51180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:53.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.062+0000 7f512cc00640 1 -- 172.21.15.167:0/162299914 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5118037040 con 0x7f513015ac80 2024-05-11T12:15:53.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.062+0000 7f512cc00640 1 -- 172.21.15.167:0/162299914 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5118042070 con 0x7f513015ac80 2024-05-11T12:15:53.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.062+0000 7f512cc00640 1 -- 172.21.15.167:0/162299914 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5118030bd0 con 0x7f513015ac80 2024-05-11T12:15:53.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.062+0000 7f5136c00640 1 -- 172.21.15.167:0/162299914 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51300afd80 con 0x7f513015ac80 2024-05-11T12:15:53.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.062+0000 7f5136c00640 1 -- 172.21.15.167:0/162299914 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f513015b4d0 con 0x7f513015ac80 2024-05-11T12:15:53.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.063+0000 7f5136c00640 1 -- 172.21.15.167:0/162299914 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51300cff20 con 0x7f513015ac80 2024-05-11T12:15:53.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.064+0000 7f512cc00640 1 -- 172.21.15.167:0/162299914 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f511802ad10 con 0x7f513015ac80 2024-05-11T12:15:53.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.065+0000 7f512cc00640 1 --2- 172.21.15.167:0/162299914 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f510c0762d0 0x7f510c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:53.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.065+0000 7f512cc00640 1 -- 172.21.15.167:0/162299914 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5507+0+0 (secure 0 0 0) 0x7f51180b7f70 con 0x7f513015ac80 2024-05-11T12:15:53.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.065+0000 7f512fe00640 1 --2- 172.21.15.167:0/162299914 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f510c0762d0 0x7f510c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:53.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.066+0000 7f512fe00640 1 --2- 172.21.15.167:0/162299914 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f510c0762d0 0x7f510c078790 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5120003d90 tx=0x7f5120009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:53.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.070+0000 7f512cc00640 1 -- 172.21.15.167:0/162299914 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f51180818f0 con 0x7f513015ac80 2024-05-11T12:15:53.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.278+0000 7f5136c00640 1 -- 172.21.15.167:0/162299914 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f5130070120 con 0x7f513015ac80 2024-05-11T12:15:53.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.283+0000 7f512cc00640 1 -- 172.21.15.167:0/162299914 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v26)=0 v26) ==== 123+0+0 (secure 0 0 0) 0x7f51180811f0 con 0x7f513015ac80 2024-05-11T12:15:53.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.284+0000 7f512cc00640 1 -- 172.21.15.167:0/162299914 <== mon.0 v2:172.21.15.167:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f5118080d00 con 0x7f513015ac80 2024-05-11T12:15:53.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.286+0000 7f5136c00640 1 -- 172.21.15.167:0/162299914 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f510c0762d0 msgr2=0x7f510c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:53.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.286+0000 7f5136c00640 1 --2- 172.21.15.167:0/162299914 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f510c0762d0 0x7f510c078790 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5120003d90 tx=0x7f5120009040 comp rx=0 tx=0).stop 2024-05-11T12:15:53.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.286+0000 7f5136c00640 1 -- 172.21.15.167:0/162299914 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f513015ac80 msgr2=0x7f5130143f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:53.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.286+0000 7f5136c00640 1 --2- 172.21.15.167:0/162299914 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f513015ac80 0x7f5130143f40 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f5118000c00 tx=0x7f51180046c0 comp rx=0 tx=0).stop 2024-05-11T12:15:53.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.287+0000 7f5136c00640 1 -- 172.21.15.167:0/162299914 shutdown_connections 2024-05-11T12:15:53.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.287+0000 7f5136c00640 1 --2- 172.21.15.167:0/162299914 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f510c0762d0 0x7f510c078790 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:53.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.287+0000 7f5136c00640 1 --2- 172.21.15.167:0/162299914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5130161380 0x7f51300ae820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:53.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.287+0000 7f5136c00640 1 --2- 172.21.15.167:0/162299914 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f513015ac80 0x7f5130143f40 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:53.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.287+0000 7f5136c00640 1 --2- 172.21.15.167:0/162299914 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51300b3d50 0x7f5130143a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:53.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.287+0000 7f5136c00640 1 -- 172.21.15.167:0/162299914 >> 172.21.15.167:0/162299914 conn(0x7f513006c450 msgr2=0x7f51300732c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:53.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.287+0000 7f5136c00640 1 -- 172.21.15.167:0/162299914 shutdown_connections 2024-05-11T12:15:53.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.287+0000 7f5136c00640 1 -- 172.21.15.167:0/162299914 wait complete. 2024-05-11T12:15:53.306 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-11T12:15:53.306 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T12:15:53.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.501+0000 7f294da00640 1 Processor -- start 2024-05-11T12:15:53.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.502+0000 7f294da00640 1 -- start start 2024-05-11T12:15:53.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.503+0000 7f294da00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f29480cd610 0x7f2948163be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:53.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.503+0000 7f294da00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f29480cd9f0 0x7f2948160230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:53.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.503+0000 7f294da00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29480b6300 con 0x7f2948160770 2024-05-11T12:15:53.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.503+0000 7f294da00640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f29480b65e0 con 0x7f29480cd610 2024-05-11T12:15:53.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.503+0000 7f294da00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f2948163100 con 0x7f29480cd9f0 2024-05-11T12:15:53.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.504+0000 7f2946a00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f29480cd610 0x7f2948163be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:53.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.504+0000 7f2946a00640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f29480cd610 0x7f2948163be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:37328/0 (socket says 172.21.15.167:37328) 2024-05-11T12:15:53.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.504+0000 7f2946a00640 1 -- 172.21.15.167:0/2889989413 learned_addr learned my addr 172.21.15.167:0/2889989413 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:53.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.504+0000 7f2946a00640 1 -- 172.21.15.167:0/2889989413 >> v2:172.21.15.204:3300/0 conn(0x7f29480cd9f0 msgr2=0x7f2948160230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:53.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.504+0000 7f2946a00640 1 --2- 172.21.15.167:0/2889989413 >> v2:172.21.15.204:3300/0 conn(0x7f29480cd9f0 0x7f2948160230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:53.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.504+0000 7f2946a00640 1 -- 172.21.15.167:0/2889989413 >> v1:172.21.15.204:6789/0 conn(0x7f2948160770 legacy=0x7f29480d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:15:53.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.504+0000 7f2946a00640 1 -- 172.21.15.167:0/2889989413 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29481633e0 con 0x7f29480cd610 2024-05-11T12:15:53.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.506+0000 7f2946a00640 1 --2- 172.21.15.167:0/2889989413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29480cd610 0x7f2948163be0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f29480d05d0 tx=0x7f293802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d89e7f1db91769 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:53.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.506+0000 7f2946000640 1 -- 172.21.15.167:0/2889989413 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2938037070 con 0x7f29480cd610 2024-05-11T12:15:53.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.506+0000 7f2946000640 1 -- 172.21.15.167:0/2889989413 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f2938032440 con 0x7f29480cd610 2024-05-11T12:15:53.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.506+0000 7f2946000640 1 -- 172.21.15.167:0/2889989413 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2938032830 con 0x7f29480cd610 2024-05-11T12:15:53.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.507+0000 7f294da00640 1 -- 172.21.15.167:0/2889989413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29480cd610 msgr2=0x7f2948163be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:53.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.507+0000 7f294da00640 1 --2- 172.21.15.167:0/2889989413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29480cd610 0x7f2948163be0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f29480d05d0 tx=0x7f293802a1e0 comp rx=0 tx=0).stop 2024-05-11T12:15:53.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.507+0000 7f294da00640 1 -- 172.21.15.167:0/2889989413 shutdown_connections 2024-05-11T12:15:53.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.507+0000 7f294da00640 1 --2- 172.21.15.167:0/2889989413 >> v2:172.21.15.204:3300/0 conn(0x7f29480cd9f0 0x7f2948160230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:53.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.507+0000 7f294da00640 1 --2- 172.21.15.167:0/2889989413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29480cd610 0x7f2948163be0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:53.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.507+0000 7f294da00640 1 -- 172.21.15.167:0/2889989413 >> 172.21.15.167:0/2889989413 conn(0x7f2948072730 msgr2=0x7f29480d2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:53.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.507+0000 7f294da00640 1 -- 172.21.15.167:0/2889989413 shutdown_connections 2024-05-11T12:15:53.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.507+0000 7f294da00640 1 -- 172.21.15.167:0/2889989413 wait complete. 2024-05-11T12:15:53.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.509+0000 7f294da00640 1 Processor -- start 2024-05-11T12:15:53.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.509+0000 7f294da00640 1 -- start start 2024-05-11T12:15:53.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.510+0000 7f294da00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29480cd610 0x7f29480cabd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:53.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.510+0000 7f294da00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29480cd9f0 0x7f294815ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:53.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.510+0000 7f294da00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2948160770 0x7f294815c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:53.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.510+0000 7f294da00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f29480cde50 con 0x7f29480cd610 2024-05-11T12:15:53.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.510+0000 7f294da00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f29480d2a80 con 0x7f2948160770 2024-05-11T12:15:53.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.510+0000 7f294da00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f29480b5e90 con 0x7f29480cd9f0 2024-05-11T12:15:53.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.510+0000 7f2946a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29480cd610 0x7f29480cabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:53.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.510+0000 7f2946a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29480cd610 0x7f29480cabd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:37334/0 (socket says 172.21.15.167:37334) 2024-05-11T12:15:53.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.510+0000 7f2946a00640 1 -- 172.21.15.167:0/3184366247 learned_addr learned my addr 172.21.15.167:0/3184366247 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T12:15:53.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.510+0000 7f293e000640 1 --2- 172.21.15.167:0/3184366247 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29480cd9f0 0x7f294815ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:53.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.510+0000 7f2946a00640 1 -- 172.21.15.167:0/3184366247 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29480cd9f0 msgr2=0x7f294815ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:53.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.511+0000 7f2946a00640 1 --2- 172.21.15.167:0/3184366247 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29480cd9f0 0x7f294815ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:53.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.511+0000 7f2946a00640 1 -- 172.21.15.167:0/3184366247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2948160770 msgr2=0x7f294815c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:53.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.511+0000 7f2946a00640 1 --2- 172.21.15.167:0/3184366247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2948160770 0x7f294815c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:53.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.511+0000 7f2946a00640 1 -- 172.21.15.167:0/3184366247 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29480ca130 con 0x7f29480cd610 2024-05-11T12:15:53.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.511+0000 7f2946a00640 1 --2- 172.21.15.167:0/3184366247 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29480cd610 0x7f29480cabd0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f293802a220 tx=0x7f29380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:53.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.511+0000 7f293f400640 1 -- 172.21.15.167:0/3184366247 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2938041020 con 0x7f29480cd610 2024-05-11T12:15:53.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.511+0000 7f293f400640 1 -- 172.21.15.167:0/3184366247 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f2938030970 con 0x7f29480cd610 2024-05-11T12:15:53.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.511+0000 7f294da00640 1 -- 172.21.15.167:0/3184366247 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f294815d040 con 0x7f29480cd610 2024-05-11T12:15:53.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.511+0000 7f293f400640 1 -- 172.21.15.167:0/3184366247 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2938037040 con 0x7f29480cd610 2024-05-11T12:15:53.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.512+0000 7f294da00640 1 -- 172.21.15.167:0/3184366247 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29480ca6b0 con 0x7f29480cd610 2024-05-11T12:15:53.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.513+0000 7f294da00640 1 -- 172.21.15.167:0/3184366247 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29480b4820 con 0x7f29480cd610 2024-05-11T12:15:53.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.514+0000 7f293f400640 1 -- 172.21.15.167:0/3184366247 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2938044020 con 0x7f29480cd610 2024-05-11T12:15:53.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.515+0000 7f293f400640 1 --2- 172.21.15.167:0/3184366247 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f29240762d0 0x7f2924078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:53.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.515+0000 7f293f400640 1 -- 172.21.15.167:0/3184366247 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5507+0+0 (secure 0 0 0) 0x7f29380b80d0 con 0x7f29480cd610 2024-05-11T12:15:53.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.516+0000 7f293e000640 1 --2- 172.21.15.167:0/3184366247 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f29240762d0 0x7f2924078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:53.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.516+0000 7f293e000640 1 --2- 172.21.15.167:0/3184366247 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f29240762d0 0x7f2924078790 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2948159e20 tx=0x7f2930008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:53.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.520+0000 7f293f400640 1 -- 172.21.15.167:0/3184366247 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f29480b4820 con 0x7f29480cd610 2024-05-11T12:15:53.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.773+0000 7f294da00640 1 -- 172.21.15.167:0/3184366247 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f294815fae0 con 0x7f29480cd610 2024-05-11T12:15:53.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.774+0000 7f293f400640 1 -- 172.21.15.167:0/3184366247 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+3881 (secure 0 0 0) 0x7f294815fae0 con 0x7f29480cd610 2024-05-11T12:15:53.774 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T12:15:53.775 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":75,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5401,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/2213070102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2213070102},{"type":"v1","addr":"172.21.15.204:6833","nonce":2213070102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5410,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1480900470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1480900470},{"type":"v1","addr":"172.21.15.204:6835","nonce":1480900470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67},{"gid":7080,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3608517801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3608517801},{"type":"v1","addr":"172.21.15.167:6835","nonce":3608517801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":7086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2359447353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2359447353},{"type":"v1","addr":"172.21.15.167:6837","nonce":2359447353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75}],"filesystems":[]} 2024-05-11T12:15:53.775 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 75 2024-05-11T12:15:53.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.777+0000 7f294da00640 1 -- 172.21.15.167:0/3184366247 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f29240762d0 msgr2=0x7f2924078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:53.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.777+0000 7f294da00640 1 --2- 172.21.15.167:0/3184366247 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f29240762d0 0x7f2924078790 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2948159e20 tx=0x7f2930008040 comp rx=0 tx=0).stop 2024-05-11T12:15:53.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.777+0000 7f294da00640 1 -- 172.21.15.167:0/3184366247 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29480cd610 msgr2=0x7f29480cabd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:53.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.777+0000 7f294da00640 1 --2- 172.21.15.167:0/3184366247 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29480cd610 0x7f29480cabd0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f293802a220 tx=0x7f29380046c0 comp rx=0 tx=0).stop 2024-05-11T12:15:53.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.777+0000 7f294da00640 1 -- 172.21.15.167:0/3184366247 shutdown_connections 2024-05-11T12:15:53.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.777+0000 7f294da00640 1 --2- 172.21.15.167:0/3184366247 >> [v2:172.21.15.167:6832/578352559,v1:172.21.15.167:6833/578352559] conn(0x7f29240762d0 0x7f2924078790 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:53.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.777+0000 7f294da00640 1 --2- 172.21.15.167:0/3184366247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2948160770 0x7f294815c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:53.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.777+0000 7f294da00640 1 --2- 172.21.15.167:0/3184366247 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29480cd9f0 0x7f294815ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:53.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.777+0000 7f294da00640 1 --2- 172.21.15.167:0/3184366247 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29480cd610 0x7f29480cabd0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:53.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.777+0000 7f294da00640 1 -- 172.21.15.167:0/3184366247 >> 172.21.15.167:0/3184366247 conn(0x7f2948072730 msgr2=0x7f29480b4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:53.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.777+0000 7f294da00640 1 -- 172.21.15.167:0/3184366247 shutdown_connections 2024-05-11T12:15:53.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T12:15:53.777+0000 7f294da00640 1 -- 172.21.15.167:0/3184366247 wait complete. 2024-05-11T12:15:53.796 DEBUG:tasks.cephfs.filesystem:already dead... 2024-05-11T12:15:53.796 INFO:teuthology.misc:Shutting down mds daemons... 2024-05-11T12:15:53.796 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-11T12:15:53.796 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T12:15:53.816 INFO:tasks.ceph.mds.a:Stopped 2024-05-11T12:15:53.816 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-11T12:15:53.816 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T12:15:53.888 INFO:tasks.ceph.mds.c:Stopped 2024-05-11T12:15:53.888 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-11T12:15:53.889 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T12:15:53.919 INFO:tasks.ceph.mds.b:Stopped 2024-05-11T12:15:53.920 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-11T12:15:53.920 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T12:15:53.992 INFO:tasks.ceph.mds.d:Stopped 2024-05-11T12:15:53.993 INFO:teuthology.misc:Shutting down osd daemons... 2024-05-11T12:15:53.993 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-05-11T12:15:53.993 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T12:15:54.067 INFO:tasks.ceph.osd.0:Stopped 2024-05-11T12:15:54.067 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-05-11T12:15:54.067 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T12:15:54.127 INFO:tasks.ceph.osd.1:Stopped 2024-05-11T12:15:54.128 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-05-11T12:15:54.128 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T12:15:54.187 INFO:tasks.ceph.osd.2:Stopped 2024-05-11T12:15:54.187 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-05-11T12:15:54.187 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T12:15:54.242 INFO:tasks.ceph.osd.3:Stopped 2024-05-11T12:15:54.243 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-05-11T12:15:54.243 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T12:15:54.317 INFO:tasks.ceph.osd.4:Stopped 2024-05-11T12:15:54.317 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-05-11T12:15:54.317 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T12:15:54.370 INFO:tasks.ceph.osd.5:Stopped 2024-05-11T12:15:54.370 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-05-11T12:15:54.370 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T12:15:54.444 INFO:tasks.ceph.osd.6:Stopped 2024-05-11T12:15:54.444 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-05-11T12:15:54.444 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T12:15:54.517 INFO:tasks.ceph.osd.7:Stopped 2024-05-11T12:15:54.518 INFO:teuthology.misc:Shutting down mgr daemons... 2024-05-11T12:15:54.518 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-05-11T12:15:54.518 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T12:15:54.590 INFO:tasks.ceph.mgr.y:Stopped 2024-05-11T12:15:54.590 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-05-11T12:15:54.590 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T12:15:54.683 INFO:tasks.ceph.mgr.x:Stopped 2024-05-11T12:15:54.683 INFO:teuthology.misc:Shutting down mon daemons... 2024-05-11T12:15:54.683 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-11T12:15:54.683 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T12:15:54.684 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~0s 2024-05-11T12:15:54.684 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~0s 2024-05-11T12:15:54.714 INFO:tasks.ceph.mon.a:Stopped 2024-05-11T12:15:54.714 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-05-11T12:15:54.714 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T12:15:54.738 INFO:tasks.ceph.mon.b:Stopped 2024-05-11T12:15:54.738 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-11T12:15:54.738 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T12:15:54.826 INFO:tasks.ceph.mon.c:Stopped 2024-05-11T12:15:54.826 INFO:tasks.ceph:Checking cluster log for badness... 2024-05-11T12:15:54.827 DEBUG:teuthology.orchestra.run.smithi167:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'client session with non-allowable root' | head -n 1 2024-05-11T12:15:54.930 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi167.front.sepia.ceph.com 2024-05-11T12:15:54.930 DEBUG:teuthology.orchestra.run.smithi167:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-05-11T12:15:55.210 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi167.front.sepia.ceph.com 2024-05-11T12:15:55.211 DEBUG:teuthology.orchestra.run.smithi167:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-05-11T12:15:55.338 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi167.front.sepia.ceph.com 2024-05-11T12:15:55.338 DEBUG:teuthology.orchestra.run.smithi167:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-05-11T12:15:55.458 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi167.front.sepia.ceph.com 2024-05-11T12:15:55.458 DEBUG:teuthology.orchestra.run.smithi167:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-05-11T12:15:55.612 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi204.front.sepia.ceph.com 2024-05-11T12:15:55.612 DEBUG:teuthology.orchestra.run.smithi204:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-05-11T12:15:55.879 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi204.front.sepia.ceph.com 2024-05-11T12:15:55.879 DEBUG:teuthology.orchestra.run.smithi204:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-05-11T12:15:56.019 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi204.front.sepia.ceph.com 2024-05-11T12:15:56.019 DEBUG:teuthology.orchestra.run.smithi204:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-05-11T12:15:56.180 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi204.front.sepia.ceph.com 2024-05-11T12:15:56.180 DEBUG:teuthology.orchestra.run.smithi204:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-05-11T12:15:56.350 INFO:tasks.ceph:Archiving mon data... 2024-05-11T12:15:56.352 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702355/data/mon.a.tgz 2024-05-11T12:15:56.352 DEBUG:teuthology.orchestra.run.smithi167:> mktemp 2024-05-11T12:15:56.409 INFO:teuthology.orchestra.run.smithi167.stdout:/tmp/tmp.FhpocgqOuj 2024-05-11T12:15:56.409 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.FhpocgqOuj 2024-05-11T12:15:56.623 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 0666 /tmp/tmp.FhpocgqOuj 2024-05-11T12:15:56.766 DEBUG:teuthology.orchestra.remote:smithi167:/tmp/tmp.FhpocgqOuj is 510KB 2024-05-11T12:15:56.877 DEBUG:teuthology.orchestra.run.smithi167:> rm -fr /tmp/tmp.FhpocgqOuj 2024-05-11T12:15:56.934 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702355/data/mon.b.tgz 2024-05-11T12:15:56.935 DEBUG:teuthology.orchestra.run.smithi204:> mktemp 2024-05-11T12:15:56.991 INFO:teuthology.orchestra.run.smithi204.stdout:/tmp/tmp.Ll32Pdb46x 2024-05-11T12:15:56.991 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.Ll32Pdb46x 2024-05-11T12:15:57.238 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 0666 /tmp/tmp.Ll32Pdb46x 2024-05-11T12:15:57.363 DEBUG:teuthology.orchestra.remote:smithi204:/tmp/tmp.Ll32Pdb46x is 651KB 2024-05-11T12:15:57.475 DEBUG:teuthology.orchestra.run.smithi204:> rm -fr /tmp/tmp.Ll32Pdb46x 2024-05-11T12:15:57.530 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702355/data/mon.c.tgz 2024-05-11T12:15:57.530 DEBUG:teuthology.orchestra.run.smithi204:> mktemp 2024-05-11T12:15:57.582 INFO:teuthology.orchestra.run.smithi204.stdout:/tmp/tmp.xrYwDcH6K5 2024-05-11T12:15:57.582 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.xrYwDcH6K5 2024-05-11T12:15:57.816 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 0666 /tmp/tmp.xrYwDcH6K5 2024-05-11T12:15:57.953 DEBUG:teuthology.orchestra.remote:smithi204:/tmp/tmp.xrYwDcH6K5 is 651KB 2024-05-11T12:15:58.066 DEBUG:teuthology.orchestra.run.smithi204:> rm -fr /tmp/tmp.xrYwDcH6K5 2024-05-11T12:15:58.123 INFO:tasks.ceph:Cleaning ceph cluster... 2024-05-11T12:15:58.123 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-11T12:15:58.131 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-11T12:15:58.266 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-11T12:15:58.323 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-11T12:15:58.323 INFO:tasks.ceph:Archiving crash dumps... 2024-05-11T12:15:58.325 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702355/remote/smithi167/crash 2024-05-11T12:15:58.326 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-11T12:15:58.414 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702355/remote/smithi204/crash 2024-05-11T12:15:58.415 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-11T12:15:58.499 INFO:tasks.ceph:Shutting down logrotate 2024-05-11T12:15:58.499 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T12:15:58.507 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T12:15:58.627 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-11T12:15:58.712 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-11T12:15:58.801 INFO:tasks.ceph:Compressing logs... 2024-05-11T12:15:58.801 DEBUG:teuthology.orchestra.run.smithi167:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-11T12:15:58.809 DEBUG:teuthology.orchestra.run.smithi204:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-11T12:15:58.885 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58418.log 2024-05-11T12:15:58.886 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56182.log 2024-05-11T12:15:58.888 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.58418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54653.log 2024-05-11T12:15:58.888 INFO:teuthology.orchestra.run.smithi167.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.58418.log.gz 2024-05-11T12:15:58.889 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.56182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57836.log 2024-05-11T12:15:58.889 INFO:teuthology.orchestra.run.smithi167.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56182.log.gz 2024-05-11T12:15:58.890 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.54653.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54653.log.gz 2024-05-11T12:15:58.890 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56973.log 2024-05-11T12:15:58.891 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.57836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40424.log 2024-05-11T12:15:58.892 INFO:teuthology.orchestra.run.smithi167.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57836.log.gz 2024-05-11T12:15:58.892 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35191.log 2024-05-11T12:15:58.893 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.56973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57099.log 2024-05-11T12:15:58.893 INFO:teuthology.orchestra.run.smithi167.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56973.log.gz 2024-05-11T12:15:58.893 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43856.log 2024-05-11T12:15:58.894 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.40424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50873.log 2024-05-11T12:15:58.894 INFO:teuthology.orchestra.run.smithi167.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40424.log.gz 2024-05-11T12:15:58.895 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.57099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55302.log 2024-05-11T12:15:58.896 INFO:teuthology.orchestra.run.smithi167.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57099.log.gz 2024-05-11T12:15:58.896 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.50873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51961.log 2024-05-11T12:15:58.896 INFO:teuthology.orchestra.run.smithi167.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50873.log.gz 2024-05-11T12:15:58.896 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35191.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-11T12:15:58.896 INFO:teuthology.orchestra.run.smithi204.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35191.log.gz 2024-05-11T12:15:58.896 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.1.43856.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43856.log.gz 2024-05-11T12:15:58.897 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.41206.log 2024-05-11T12:15:58.897 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35240.log 2024-05-11T12:15:58.898 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.55302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60198.log 2024-05-11T12:15:58.898 INFO:teuthology.orchestra.run.smithi167.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55302.log.gz 2024-05-11T12:15:58.898 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.0.51961.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.51961.log.gz 2024-05-11T12:15:58.898 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61537.log 2024-05-11T12:15:58.900 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.60198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.44232.log 2024-05-11T12:15:58.900 INFO:teuthology.orchestra.run.smithi167.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60198.log.gz 2024-05-11T12:15:58.900 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.61537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55008.log 2024-05-11T12:15:58.900 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.3.41206.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.41206.log.gz 2024-05-11T12:15:58.900 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35400.log 2024-05-11T12:15:58.901 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35240.log: 92.4% 87.4% -- replaced with /var/log/ceph/ceph.log.gz -- replaced with /var/log/ceph/ceph-client.admin.35240.log.gz 2024-05-11T12:15:58.901 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-05-11T12:15:58.901 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44350.log 2024-05-11T12:15:58.901 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35400.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-11T12:15:58.901 INFO:teuthology.orchestra.run.smithi204.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35400.log.gz 2024-05-11T12:15:58.902 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.1.44350.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.44350.log.gz 2024-05-11T12:15:58.902 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-05-11T12:15:58.903 INFO:teuthology.orchestra.run.smithi167.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.61537.log.gz 2024-05-11T12:15:58.903 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.2.44232.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.44232.log.gz 2024-05-11T12:15:58.903 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50533.log 2024-05-11T12:15:58.903 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.55008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46036.log 2024-05-11T12:15:58.903 INFO:teuthology.orchestra.run.smithi167.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55008.log.gz 2024-05-11T12:15:58.903 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.50533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41642.log 2024-05-11T12:15:58.903 INFO:teuthology.orchestra.run.smithi167.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50533.log.gz 2024-05-11T12:15:58.905 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.46036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42437.log 2024-05-11T12:15:58.905 INFO:teuthology.orchestra.run.smithi167.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46036.log.gz 2024-05-11T12:15:58.905 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.41642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48357.log 2024-05-11T12:15:58.906 INFO:teuthology.orchestra.run.smithi167.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41642.log.gz 2024-05-11T12:15:58.906 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-05-11T12:15:58.906 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-05-11T12:15:58.906 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35320.log 2024-05-11T12:15:58.906 INFO:teuthology.orchestra.run.smithi204.stderr: 94.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-11T12:15:58.906 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-05-11T12:15:58.907 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.42437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41852.log 2024-05-11T12:15:58.907 INFO:teuthology.orchestra.run.smithi167.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42437.log.gz 2024-05-11T12:15:58.907 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.48357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40466.log 2024-05-11T12:15:58.907 INFO:teuthology.orchestra.run.smithi167.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48357.log.gz 2024-05-11T12:15:58.908 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41832.log 2024-05-11T12:15:58.909 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-osd.4.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.35320.log.gz 2024-05-11T12:15:58.909 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.41852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59523.log 2024-05-11T12:15:58.909 INFO:teuthology.orchestra.run.smithi167.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41852.log.gz 2024-05-11T12:15:58.909 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-05-11T12:15:58.910 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.40466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51181.log 2024-05-11T12:15:58.910 INFO:teuthology.orchestra.run.smithi167.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.40466.log.gz 2024-05-11T12:15:58.911 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.59523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41600.log 2024-05-11T12:15:58.911 INFO:teuthology.orchestra.run.smithi167.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59523.log.gz 2024-05-11T12:15:58.911 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.51181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40634.log 2024-05-11T12:15:58.911 INFO:teuthology.orchestra.run.smithi167.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51181.log.gz 2024-05-11T12:15:58.912 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.1.41832.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-05-11T12:15:58.912 INFO:teuthology.orchestra.run.smithi204.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.41832.log.gz 2024-05-11T12:15:58.912 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-05-11T12:15:58.913 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.41600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57183.log 2024-05-11T12:15:58.913 INFO:teuthology.orchestra.run.smithi167.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41600.log.gz 2024-05-11T12:15:58.913 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.40634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53094.log 2024-05-11T12:15:58.914 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43361.log 2024-05-11T12:15:58.914 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.57183.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.40634.log.gz 2024-05-11T12:15:58.914 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53014.log 2024-05-11T12:15:58.914 INFO:teuthology.orchestra.run.smithi167.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57183.log.gz 2024-05-11T12:15:58.915 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.53094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43280.log 2024-05-11T12:15:58.915 INFO:teuthology.orchestra.run.smithi167.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53094.log.gz 2024-05-11T12:15:58.916 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-osd.6.log: /var/log/ceph/ceph-client.1.43361.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43361.log.gz 2024-05-11T12:15:58.917 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.53014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47937.log 2024-05-11T12:15:58.917 INFO:teuthology.orchestra.run.smithi167.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53014.log.gz 2024-05-11T12:15:58.917 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.43280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53757.log 2024-05-11T12:15:58.918 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.40791.log 2024-05-11T12:15:58.918 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.47937.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.43280.log.gz 2024-05-11T12:15:58.918 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42102.log 2024-05-11T12:15:58.918 INFO:teuthology.orchestra.run.smithi167.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47937.log.gz 2024-05-11T12:15:58.920 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.53757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45994.log 2024-05-11T12:15:58.920 INFO:teuthology.orchestra.run.smithi167.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53757.log.gz 2024-05-11T12:15:58.921 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.1.40791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35280.log 2024-05-11T12:15:58.921 INFO:teuthology.orchestra.run.smithi204.stderr: 93.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-05-11T12:15:58.921 INFO:teuthology.orchestra.run.smithi204.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.40791.log.gz 2024-05-11T12:15:58.921 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35360.log 2024-05-11T12:15:58.922 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.42102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57496.log 2024-05-11T12:15:58.922 INFO:teuthology.orchestra.run.smithi167.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42102.log.gz 2024-05-11T12:15:58.922 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.45994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52270.log 2024-05-11T12:15:58.922 INFO:teuthology.orchestra.run.smithi167.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45994.log.gz 2024-05-11T12:15:58.923 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.57496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61621.log 2024-05-11T12:15:58.923 INFO:teuthology.orchestra.run.smithi167.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57496.log.gz 2024-05-11T12:15:58.924 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.52270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55866.log 2024-05-11T12:15:58.925 INFO:teuthology.orchestra.run.smithi167.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52270.log.gz 2024-05-11T12:15:58.925 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.61621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40254.log 2024-05-11T12:15:58.925 INFO:teuthology.orchestra.run.smithi167.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.61621.log.gz 2024-05-11T12:15:58.926 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35440.log 2024-05-11T12:15:58.926 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35360.log: 88.3% 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35280.log.gz 2024-05-11T12:15:58.926 INFO:teuthology.orchestra.run.smithi204.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35360.log.gz 2024-05-11T12:15:58.926 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44748.log 2024-05-11T12:15:58.927 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.55866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62398.log 2024-05-11T12:15:58.927 INFO:teuthology.orchestra.run.smithi167.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55866.log.gz 2024-05-11T12:15:58.928 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.40254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41768.log 2024-05-11T12:15:58.928 INFO:teuthology.orchestra.run.smithi167.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40254.log.gz 2024-05-11T12:15:58.928 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.62398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48105.log 2024-05-11T12:15:58.928 INFO:teuthology.orchestra.run.smithi167.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.62398.log.gz 2024-05-11T12:15:58.930 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.41768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53589.log 2024-05-11T12:15:58.930 INFO:teuthology.orchestra.run.smithi167.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41768.log.gz 2024-05-11T12:15:58.930 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.48105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57580.log 2024-05-11T12:15:58.930 INFO:teuthology.orchestra.run.smithi167.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48105.log.gz 2024-05-11T12:15:58.931 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35440.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35440.log.gz 2024-05-11T12:15:58.931 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.1.44748.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.44748.log.gz 2024-05-11T12:15:58.931 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42405.log 2024-05-11T12:15:58.931 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42883.log 2024-05-11T12:15:58.931 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.1.42405.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42405.log.gz 2024-05-11T12:15:58.932 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.53589.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53589.log.gz 2024-05-11T12:15:58.933 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48782.log 2024-05-11T12:15:58.933 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.57580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42783.log 2024-05-11T12:15:58.933 INFO:teuthology.orchestra.run.smithi167.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57580.log.gz 2024-05-11T12:15:58.933 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.48782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62336.log 2024-05-11T12:15:58.933 INFO:teuthology.orchestra.run.smithi167.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48782.log.gz 2024-05-11T12:15:58.934 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.42783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47026.log 2024-05-11T12:15:58.934 INFO:teuthology.orchestra.run.smithi167.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42783.log.gz 2024-05-11T12:15:58.935 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.1.42883.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42883.log.gz 2024-05-11T12:15:58.935 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-05-11T12:15:58.935 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-05-11T12:15:58.936 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.62336.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.62336.log.gz 2024-05-11T12:15:58.936 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41684.log 2024-05-11T12:15:58.936 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.0.47026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45202.log 2024-05-11T12:15:58.936 INFO:teuthology.orchestra.run.smithi167.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47026.log.gz 2024-05-11T12:15:58.937 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.41684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48740.log 2024-05-11T12:15:58.937 INFO:teuthology.orchestra.run.smithi167.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41684.log.gz 2024-05-11T12:15:58.938 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.45202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48578.log 2024-05-11T12:15:58.938 INFO:teuthology.orchestra.run.smithi167.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45202.log.gz 2024-05-11T12:15:58.939 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.48740.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.28794.log 2024-05-11T12:15:58.940 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35149.log 2024-05-11T12:15:58.940 INFO:teuthology.orchestra.run.smithi167.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48740.log.gz 2024-05-11T12:15:58.940 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.48578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46688.log 2024-05-11T12:15:58.940 INFO:teuthology.orchestra.run.smithi167.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48578.log.gz 2024-05-11T12:15:58.941 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph.tmp-client.admin.28794.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.28794.log.gz 2024-05-11T12:15:58.941 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62041.log 2024-05-11T12:15:58.942 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35149.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35149.log.gz 2024-05-11T12:15:58.942 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.46688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34824.log 2024-05-11T12:15:58.943 INFO:teuthology.orchestra.run.smithi167.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46688.log.gz 2024-05-11T12:15:58.943 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.62041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45910.log 2024-05-11T12:15:58.943 INFO:teuthology.orchestra.run.smithi167.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.62041.log.gz 2024-05-11T12:15:58.944 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.34824.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-11T12:15:58.944 INFO:teuthology.orchestra.run.smithi167.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.34824.log.gz 2024-05-11T12:15:58.945 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.45910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43070.log 2024-05-11T12:15:58.946 INFO:teuthology.orchestra.run.smithi167.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45910.log.gz 2024-05-11T12:15:58.946 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60903.log 2024-05-11T12:15:58.947 INFO:teuthology.orchestra.run.smithi204.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-05-11T12:15:58.948 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.43070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55092.log 2024-05-11T12:15:58.948 INFO:teuthology.orchestra.run.smithi167.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43070.log.gz 2024-05-11T12:15:58.948 INFO:teuthology.orchestra.run.smithi167.stderr: 86.1% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-11T12:15:58.949 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.60903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48658.log 2024-05-11T12:15:58.949 INFO:teuthology.orchestra.run.smithi167.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.60903.log.gz 2024-05-11T12:15:58.950 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.55092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45454.log 2024-05-11T12:15:58.950 INFO:teuthology.orchestra.run.smithi167.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55092.log.gz 2024-05-11T12:15:58.951 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.48658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61151.log 2024-05-11T12:15:58.951 INFO:teuthology.orchestra.run.smithi167.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48658.log.gz 2024-05-11T12:15:58.952 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.45454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41362.log 2024-05-11T12:15:58.952 INFO:teuthology.orchestra.run.smithi167.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45454.log.gz 2024-05-11T12:15:58.953 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.61151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52480.log 2024-05-11T12:15:58.953 INFO:teuthology.orchestra.run.smithi167.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61151.log.gz 2024-05-11T12:15:58.954 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.41362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53673.log 2024-05-11T12:15:58.954 INFO:teuthology.orchestra.run.smithi167.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41362.log.gz 2024-05-11T12:15:58.955 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.52480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58924.log 2024-05-11T12:15:58.955 INFO:teuthology.orchestra.run.smithi167.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52480.log.gz 2024-05-11T12:15:58.956 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.53673.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53673.log.gz 2024-05-11T12:15:58.957 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62083.log 2024-05-11T12:15:58.957 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.0.58924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42755.log 2024-05-11T12:15:58.957 INFO:teuthology.orchestra.run.smithi167.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.58924.log.gz 2024-05-11T12:15:58.959 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.62083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44831.log 2024-05-11T12:15:58.959 INFO:teuthology.orchestra.run.smithi167.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.62083.log.gz 2024-05-11T12:15:58.960 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.42755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60987.log 2024-05-11T12:15:58.960 INFO:teuthology.orchestra.run.smithi167.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42755.log.gz 2024-05-11T12:15:58.961 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.44831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54821.log 2024-05-11T12:15:58.961 INFO:teuthology.orchestra.run.smithi167.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44831.log.gz 2024-05-11T12:15:58.962 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.60987.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-05-11T12:15:58.962 INFO:teuthology.orchestra.run.smithi167.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60987.log.gz 2024-05-11T12:15:58.963 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.54821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40124.log 2024-05-11T12:15:58.963 INFO:teuthology.orchestra.run.smithi167.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54821.log.gz 2024-05-11T12:15:58.963 INFO:teuthology.orchestra.run.smithi204.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-05-11T12:15:58.964 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61789.log 2024-05-11T12:15:58.965 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.40124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45538.log 2024-05-11T12:15:58.965 INFO:teuthology.orchestra.run.smithi167.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40124.log.gz 2024-05-11T12:15:58.966 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.61789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52751.log 2024-05-11T12:15:58.966 INFO:teuthology.orchestra.run.smithi167.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61789.log.gz 2024-05-11T12:15:58.967 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.45538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46376.log 2024-05-11T12:15:58.967 INFO:teuthology.orchestra.run.smithi167.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45538.log.gz 2024-05-11T12:15:58.968 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.52751.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52751.log.gz 2024-05-11T12:15:58.968 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50094.log 2024-05-11T12:15:58.969 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.46376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62439.log 2024-05-11T12:15:58.969 INFO:teuthology.orchestra.run.smithi167.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46376.log.gz 2024-05-11T12:15:58.970 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.50094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51455.log 2024-05-11T12:15:58.971 INFO:teuthology.orchestra.run.smithi167.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50094.log.gz 2024-05-11T12:15:58.971 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.62439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46166.log 2024-05-11T12:15:58.972 INFO:teuthology.orchestra.run.smithi167.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62439.log.gz 2024-05-11T12:15:58.973 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.51455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54290.log 2024-05-11T12:15:58.973 INFO:teuthology.orchestra.run.smithi167.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51455.log.gz 2024-05-11T12:15:58.973 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.46166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62251.log 2024-05-11T12:15:58.974 INFO:teuthology.orchestra.run.smithi167.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46166.log.gz 2024-05-11T12:15:58.974 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.54290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57921.log 2024-05-11T12:15:58.975 INFO:teuthology.orchestra.run.smithi167.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.54290.log.gz 2024-05-11T12:15:58.976 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.62251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57412.log 2024-05-11T12:15:58.976 INFO:teuthology.orchestra.run.smithi167.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.62251.log.gz 2024-05-11T12:15:58.977 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.57921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44999.log 2024-05-11T12:15:58.977 INFO:teuthology.orchestra.run.smithi167.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57921.log.gz 2024-05-11T12:15:58.978 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.57412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56708.log 2024-05-11T12:15:58.978 INFO:teuthology.orchestra.run.smithi167.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57412.log.gz 2024-05-11T12:15:58.979 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.44999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40700.log 2024-05-11T12:15:58.979 INFO:teuthology.orchestra.run.smithi167.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44999.log.gz 2024-05-11T12:15:58.980 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.56708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47895.log 2024-05-11T12:15:58.980 INFO:teuthology.orchestra.run.smithi167.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56708.log.gz 2024-05-11T12:15:58.981 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.40700.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-11T12:15:58.981 INFO:teuthology.orchestra.run.smithi167.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.40700.log.gz 2024-05-11T12:15:58.982 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.47895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58144.log 2024-05-11T12:15:58.983 INFO:teuthology.orchestra.run.smithi167.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47895.log.gz 2024-05-11T12:15:58.983 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53175.log 2024-05-11T12:15:58.985 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.58144.log: gzip 87.9% -- replaced with /var/log/ceph/ceph-client.admin.58144.log.gz 2024-05-11T12:15:58.985 INFO:teuthology.orchestra.run.smithi167.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.41894.log 2024-05-11T12:15:58.985 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.53175.log: 89.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-11T12:15:58.986 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43238.log 2024-05-11T12:15:58.986 INFO:teuthology.orchestra.run.smithi167.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53175.log.gz 2024-05-11T12:15:58.987 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.41894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61873.log 2024-05-11T12:15:58.987 INFO:teuthology.orchestra.run.smithi167.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41894.log.gz 2024-05-11T12:15:58.988 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.43238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45370.log 2024-05-11T12:15:58.988 INFO:teuthology.orchestra.run.smithi167.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43238.log.gz 2024-05-11T12:15:58.989 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.61873.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.61873.log.gz 2024-05-11T12:15:58.989 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45118.log 2024-05-11T12:15:58.990 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.45370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46460.log 2024-05-11T12:15:58.991 INFO:teuthology.orchestra.run.smithi167.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45370.log.gz 2024-05-11T12:15:58.991 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.45118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59154.log 2024-05-11T12:15:58.992 INFO:teuthology.orchestra.run.smithi167.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45118.log.gz 2024-05-11T12:15:58.992 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.46460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48189.log 2024-05-11T12:15:58.993 INFO:teuthology.orchestra.run.smithi167.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46460.log.gz 2024-05-11T12:15:58.994 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.59154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40082.log 2024-05-11T12:15:58.994 INFO:teuthology.orchestra.run.smithi167.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59154.log.gz 2024-05-11T12:15:58.994 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.48189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57454.log 2024-05-11T12:15:58.995 INFO:teuthology.orchestra.run.smithi167.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48189.log.gz 2024-05-11T12:15:58.996 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.40082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50958.log 2024-05-11T12:15:58.996 INFO:teuthology.orchestra.run.smithi167.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40082.log.gz 2024-05-11T12:15:58.997 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.57454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55920.log 2024-05-11T12:15:58.997 INFO:teuthology.orchestra.run.smithi167.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57454.log.gz 2024-05-11T12:15:58.998 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.50958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45328.log 2024-05-11T12:15:58.998 INFO:teuthology.orchestra.run.smithi167.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50958.log.gz 2024-05-11T12:15:58.999 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.55920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62167.log 2024-05-11T12:15:58.999 INFO:teuthology.orchestra.run.smithi167.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55920.log.gz 2024-05-11T12:15:59.000 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.45328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42115.log 2024-05-11T12:15:59.000 INFO:teuthology.orchestra.run.smithi167.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45328.log.gz 2024-05-11T12:15:59.001 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.62167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56631.log 2024-05-11T12:15:59.001 INFO:teuthology.orchestra.run.smithi167.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.62167.log.gz 2024-05-11T12:15:59.002 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.42115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55134.log 2024-05-11T12:15:59.002 INFO:teuthology.orchestra.run.smithi167.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42115.log.gz 2024-05-11T12:15:59.003 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.56631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58376.log 2024-05-11T12:15:59.003 INFO:teuthology.orchestra.run.smithi167.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56631.log.gz 2024-05-11T12:15:59.004 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.55134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40168.log 2024-05-11T12:15:59.004 INFO:teuthology.orchestra.run.smithi167.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55134.log.gz 2024-05-11T12:15:59.005 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.58376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46772.log 2024-05-11T12:15:59.005 INFO:teuthology.orchestra.run.smithi167.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58376.log.gz 2024-05-11T12:15:59.006 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.40168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46208.log 2024-05-11T12:15:59.006 INFO:teuthology.orchestra.run.smithi167.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40168.log.gz 2024-05-11T12:15:59.007 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.46772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49321.log 2024-05-11T12:15:59.008 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.46208.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.46772.log.gz 2024-05-11T12:15:59.008 INFO:teuthology.orchestra.run.smithi167.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46208.log.gz 2024-05-11T12:15:59.009 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58460.log 2024-05-11T12:15:59.009 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.49321.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-05-11T12:15:59.010 INFO:teuthology.orchestra.run.smithi167.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49321.log.gz 2024-05-11T12:15:59.010 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.58460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56098.log 2024-05-11T12:15:59.011 INFO:teuthology.orchestra.run.smithi167.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.58460.log.gz 2024-05-11T12:15:59.011 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47299.log 2024-05-11T12:15:59.013 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.56098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42118.log 2024-05-11T12:15:59.013 INFO:teuthology.orchestra.run.smithi167.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56098.log.gz 2024-05-11T12:15:59.014 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.47299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56557.log 2024-05-11T12:15:59.014 INFO:teuthology.orchestra.run.smithi167.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47299.log.gz 2024-05-11T12:15:59.015 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.42118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50575.log 2024-05-11T12:15:59.015 INFO:teuthology.orchestra.run.smithi167.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42118.log.gz 2024-05-11T12:15:59.016 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.56557.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56557.log.gz 2024-05-11T12:15:59.016 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40592.log 2024-05-11T12:15:59.017 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.50575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46838.log 2024-05-11T12:15:59.017 INFO:teuthology.orchestra.run.smithi167.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50575.log.gz 2024-05-11T12:15:59.018 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.40592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58009.log 2024-05-11T12:15:59.018 INFO:teuthology.orchestra.run.smithi167.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40592.log.gz 2024-05-11T12:15:59.020 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.46838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46586.log 2024-05-11T12:15:59.020 INFO:teuthology.orchestra.run.smithi167.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46838.log.gz 2024-05-11T12:15:59.021 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.58009.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58009.log.gz 2024-05-11T12:15:59.021 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52212.log 2024-05-11T12:15:59.022 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.46586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58502.log 2024-05-11T12:15:59.023 INFO:teuthology.orchestra.run.smithi167.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46586.log.gz 2024-05-11T12:15:59.024 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.52212.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52212.log.gz 2024-05-11T12:15:59.024 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54921.log 2024-05-11T12:15:59.024 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.58502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62293.log 2024-05-11T12:15:59.024 INFO:teuthology.orchestra.run.smithi167.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.58502.log.gz 2024-05-11T12:15:59.025 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.54921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52312.log 88.9% 2024-05-11T12:15:59.026 INFO:teuthology.orchestra.run.smithi167.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54921.log.gz 2024-05-11T12:15:59.027 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.62293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40212.log 2024-05-11T12:15:59.027 INFO:teuthology.orchestra.run.smithi167.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.62293.log.gz 2024-05-11T12:15:59.028 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.52312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49833.log 2024-05-11T12:15:59.028 INFO:teuthology.orchestra.run.smithi167.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52312.log.gz 2024-05-11T12:15:59.029 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.40212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47571.log 2024-05-11T12:15:59.029 INFO:teuthology.orchestra.run.smithi167.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40212.log.gz 2024-05-11T12:15:59.030 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.49833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55558.log 2024-05-11T12:15:59.030 INFO:teuthology.orchestra.run.smithi167.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49833.log.gz 2024-05-11T12:15:59.031 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.0.47571.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47571.log.gz 2024-05-11T12:15:59.031 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-05-11T12:15:59.032 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.55558.log: 87.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.55558.log.gz 2024-05-11T12:15:59.032 INFO:teuthology.orchestra.run.smithi167.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.58736.log 2024-05-11T12:15:59.033 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42449.log 2024-05-11T12:15:59.034 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.58736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49195.log 2024-05-11T12:15:59.035 INFO:teuthology.orchestra.run.smithi167.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58736.log.gz 2024-05-11T12:15:59.036 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.42449.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42449.log.gz 2024-05-11T12:15:59.036 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42786.log 2024-05-11T12:15:59.037 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.49195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55687.log 2024-05-11T12:15:59.038 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.42786.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49195.log.gz 2024-05-11T12:15:59.038 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46124.log 2024-05-11T12:15:59.038 INFO:teuthology.orchestra.run.smithi167.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42786.log.gz 2024-05-11T12:15:59.039 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.55687.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55687.log.gz 2024-05-11T12:15:59.040 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43154.log 2024-05-11T12:15:59.041 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.46124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41726.log 2024-05-11T12:15:59.041 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.43154.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46124.log.gz 2024-05-11T12:15:59.042 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48963.log 88.1% 2024-05-11T12:15:59.042 INFO:teuthology.orchestra.run.smithi167.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43154.log.gz 2024-05-11T12:15:59.043 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.41726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42105.log 2024-05-11T12:15:59.044 INFO:teuthology.orchestra.run.smithi167.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41726.log.gz 2024-05-11T12:15:59.044 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.48963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46080.log 2024-05-11T12:15:59.045 INFO:teuthology.orchestra.run.smithi167.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48963.log.gz 2024-05-11T12:15:59.046 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.42105.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-05-11T12:15:59.046 INFO:teuthology.orchestra.run.smithi167.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42105.log.gz 2024-05-11T12:15:59.047 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.46080.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-05-11T12:15:59.047 INFO:teuthology.orchestra.run.smithi167.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46080.log.gz 2024-05-11T12:15:59.048 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46418.log 2024-05-11T12:15:59.049 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35192.log 2024-05-11T12:15:59.051 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.46418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61915.log 2024-05-11T12:15:59.051 INFO:teuthology.orchestra.run.smithi167.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46418.log.gz 2024-05-11T12:15:59.052 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.35192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41810.log 2024-05-11T12:15:59.053 INFO:teuthology.orchestra.run.smithi167.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35192.log.gz 2024-05-11T12:15:59.055 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.61915.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.61915.log.gz 2024-05-11T12:15:59.055 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5/var/log/ceph/ceph-client.admin.41810.log: --verbose -- /var/log/ceph/ceph-client.admin.46292.log 2024-05-11T12:15:59.055 INFO:teuthology.orchestra.run.smithi167.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41810.log.gz 2024-05-11T12:15:59.056 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40040.log 2024-05-11T12:15:59.058 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.46292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48483.log 2024-05-11T12:15:59.058 INFO:teuthology.orchestra.run.smithi167.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46292.log.gz 2024-05-11T12:15:59.059 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.40040.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40040.log.gz 2024-05-11T12:15:59.060 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.48483.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48483.log.gz 2024-05-11T12:15:59.061 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60102.log 2024-05-11T12:15:59.062 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-05-11T12:15:59.064 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.60102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61495.log 2024-05-11T12:15:59.064 INFO:teuthology.orchestra.run.smithi167.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60102.log.gz 2024-05-11T12:15:59.067 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-osd.0.log: /var/log/ceph/ceph-client.admin.61495.log: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54966.log 2024-05-11T12:15:59.067 INFO:teuthology.orchestra.run.smithi167.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61495.log.gz 2024-05-11T12:15:59.068 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50178.log 2024-05-11T12:15:59.070 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.54966.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54966.log.gz 2024-05-11T12:15:59.070 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43028.log 2024-05-11T12:15:59.071 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.50178.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50178.log.gz 2024-05-11T12:15:59.073 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.43028.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43028.log.gz 2024-05-11T12:15:59.073 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51497.log 2024-05-11T12:15:59.075 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44408.log 2024-05-11T12:15:59.076 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.51497.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51497.log.gz 2024-05-11T12:15:59.078 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.44408.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44408.log.gz 2024-05-11T12:15:59.078 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59277.log 2024-05-11T12:15:59.080 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54737.log 2024-05-11T12:15:59.081 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.59277.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59277.log.gz 2024-05-11T12:15:59.083 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.54737.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54737.log.gz 2024-05-11T12:15:59.083 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57286.log 2024-05-11T12:15:59.085 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57328.log 2024-05-11T12:15:59.086 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.57286.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57286.log.gz 2024-05-11T12:15:59.088 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.57328.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57328.log.gz 2024-05-11T12:15:59.088 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50449.log 2024-05-11T12:15:59.090 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58102.log 2024-05-11T12:15:59.091 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.50449.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50449.log.gz 2024-05-11T12:15:59.093 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.58102.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58102.log.gz 2024-05-11T12:15:59.093 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40508.log 2024-05-11T12:15:59.095 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48912.log 2024-05-11T12:15:59.096 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.40508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42709.log 2024-05-11T12:15:59.096 INFO:teuthology.orchestra.run.smithi167.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40508.log.gz 2024-05-11T12:15:59.098 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.48912.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48912.log.gz 2024-05-11T12:15:59.099 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.42709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53841.log 2024-05-11T12:15:59.100 INFO:teuthology.orchestra.run.smithi167.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42709.log.gz 2024-05-11T12:15:59.101 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45244.log 2024-05-11T12:15:59.103 INFO:teuthology.orchestra.run.smithi204.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-05-11T12:15:59.104 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.53841.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.53841.log.gz 2024-05-11T12:15:59.104 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.45244.log: 92.0% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-05-11T12:15:59.104 INFO:teuthology.orchestra.run.smithi167.stderr:gzip 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45244.log.gz -5 2024-05-11T12:15:59.104 INFO:teuthology.orchestra.run.smithi167.stderr: --verbose -- /var/log/ceph/ceph-client.admin.60819.log 2024-05-11T12:15:59.105 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53399.log 2024-05-11T12:15:59.106 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.60819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45496.log 2024-05-11T12:15:59.106 INFO:teuthology.orchestra.run.smithi167.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60819.log.gz 2024-05-11T12:15:59.107 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.53399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59677.log 2024-05-11T12:15:59.108 INFO:teuthology.orchestra.run.smithi167.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53399.log.gz 2024-05-11T12:15:59.109 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.45496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45826.log 2024-05-11T12:15:59.109 INFO:teuthology.orchestra.run.smithi167.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45496.log.gz 2024-05-11T12:15:59.110 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.59677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47341.log 2024-05-11T12:15:59.110 INFO:teuthology.orchestra.run.smithi167.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.59677.log.gz 2024-05-11T12:15:59.112 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.45826.log: 88.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.45826.log.gz -5 2024-05-11T12:15:59.112 INFO:teuthology.orchestra.run.smithi167.stderr: --verbose -- /var/log/ceph/ceph-client.admin.49910.log 2024-05-11T12:15:59.112 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.47341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52396.log 2024-05-11T12:15:59.113 INFO:teuthology.orchestra.run.smithi167.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47341.log.gz/var/log/ceph/ceph-client.admin.49910.log: 2024-05-11T12:15:59.114 INFO:teuthology.orchestra.run.smithi167.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49910.log.gz 2024-05-11T12:15:59.114 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61193.log 2024-05-11T12:15:59.115 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.52396.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52396.log.gz 2024-05-11T12:15:59.115 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40296.log 2024-05-11T12:15:59.117 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42744.log 2024-05-11T12:15:59.117 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.61193.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.61193.log.gz 2024-05-11T12:15:59.118 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.40296.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40296.log.gz 2024-05-11T12:15:59.119 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50136.log 2024-05-11T12:15:59.119 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.42744.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42744.log.gz 2024-05-11T12:15:59.120 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52625.log 2024-05-11T12:15:59.121 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46642.log 2024-05-11T12:15:59.121 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.50136.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50136.log.gz 2024-05-11T12:15:59.122 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.52625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57965.log 2024-05-11T12:15:59.122 INFO:teuthology.orchestra.run.smithi167.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52625.log.gz 2024-05-11T12:15:59.124 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.46642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47979.log 2024-05-11T12:15:59.124 INFO:teuthology.orchestra.run.smithi167.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46642.log.gz 2024-05-11T12:15:59.125 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.57965.log: gzip 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57965.log.gz 2024-05-11T12:15:59.125 INFO:teuthology.orchestra.run.smithi167.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.55731.log 2024-05-11T12:15:59.126 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.47979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40382.log 2024-05-11T12:15:59.126 INFO:teuthology.orchestra.run.smithi167.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47979.log.gz 2024-05-11T12:15:59.128 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.55731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49659.log 2024-05-11T12:15:59.128 INFO:teuthology.orchestra.run.smithi167.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55731.log.gz 2024-05-11T12:15:59.129 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.40382.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40382.log.gz 2024-05-11T12:15:59.129 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59865.log 2024-05-11T12:15:59.130 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.0.49659.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42450.log 2024-05-11T12:15:59.130 INFO:teuthology.orchestra.run.smithi167.stderr: -- replaced with /var/log/ceph/ceph-client.0.49659.log.gz 2024-05-11T12:15:59.131 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.0.59865.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.59865.log.gz 2024-05-11T12:15:59.132 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42439.log 2024-05-11T12:15:59.133 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.42450.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42450.log.gz 2024-05-11T12:15:59.133 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53715.log 2024-05-11T12:15:59.134 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.42439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49111.log 2024-05-11T12:15:59.135 INFO:teuthology.orchestra.run.smithi167.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42439.log.gz 2024-05-11T12:15:59.136 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.53715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51413.log 2024-05-11T12:15:59.136 INFO:teuthology.orchestra.run.smithi167.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53715.log.gz 2024-05-11T12:15:59.137 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.49111.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49111.log.gz 2024-05-11T12:15:59.138 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.51413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50491.log 2024-05-11T12:15:59.138 INFO:teuthology.orchestra.run.smithi167.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51413.log.gz 2024-05-11T12:15:59.139 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41137.log 2024-05-11T12:15:59.140 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.50491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42112.log 2024-05-11T12:15:59.140 INFO:teuthology.orchestra.run.smithi167.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50491.log.gz 2024-05-11T12:15:59.141 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.41137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55397.log 2024-05-11T12:15:59.143 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.42112.log: 91.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-05-11T12:15:59.143 INFO:teuthology.orchestra.run.smithi167.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42112.log.gz 2024-05-11T12:15:59.146 INFO:teuthology.orchestra.run.smithi167.stderr: 92.0%/var/log/ceph/ceph-client.admin.55397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62209.log 2024-05-11T12:15:59.146 INFO:teuthology.orchestra.run.smithi167.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41137.log.gz 2024-05-11T12:15:59.146 INFO:teuthology.orchestra.run.smithi167.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.55397.log.gz 2024-05-11T12:15:59.146 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43927.log 2024-05-11T12:15:59.146 INFO:teuthology.orchestra.run.smithi167.stderr:gzip/var/log/ceph/ceph-client.admin.62209.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.51002.log 2024-05-11T12:15:59.147 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.43927.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62209.log.gz 2024-05-11T12:15:59.147 INFO:teuthology.orchestra.run.smithi167.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43927.log.gz 2024-05-11T12:15:59.147 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50407.log 2024-05-11T12:15:59.148 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.51002.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-05-11T12:15:59.148 INFO:teuthology.orchestra.run.smithi167.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51002.log.gz 2024-05-11T12:15:59.149 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.50407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42108.log 2024-05-11T12:15:59.149 INFO:teuthology.orchestra.run.smithi167.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50407.log.gz 2024-05-11T12:15:59.150 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44366.log 2024-05-11T12:15:59.152 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.42108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59282.log 2024-05-11T12:15:59.152 INFO:teuthology.orchestra.run.smithi167.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42108.log.gz 2024-05-11T12:15:59.153 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.44366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42454.log 2024-05-11T12:15:59.154 INFO:teuthology.orchestra.run.smithi167.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44366.log.gz 2024-05-11T12:15:59.155 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.59282.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.59282.log.gz 2024-05-11T12:15:59.156 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.42454.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42454.log.gz 2024-05-11T12:15:59.157 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59283.log 2024-05-11T12:15:59.158 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51623.log 2024-05-11T12:15:59.159 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.59283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58586.log 2024-05-11T12:15:59.159 INFO:teuthology.orchestra.run.smithi167.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.59283.log.gz 2024-05-11T12:15:59.161 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.51623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43444.log 2024-05-11T12:15:59.161 INFO:teuthology.orchestra.run.smithi167.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51623.log.gz 2024-05-11T12:15:59.162 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.58586.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.58586.log.gz 2024-05-11T12:15:59.163 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47811.log 2024-05-11T12:15:59.163 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.43444.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43444.log.gz 2024-05-11T12:15:59.164 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53483.log 2024-05-11T12:15:59.165 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.47811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53907.log 2024-05-11T12:15:59.165 INFO:teuthology.orchestra.run.smithi167.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47811.log.gz 2024-05-11T12:15:59.167 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.53483.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53483.log.gz 2024-05-11T12:15:59.167 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41516.log 2024-05-11T12:15:59.167 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.53907.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.53907.log.gz 2024-05-11T12:15:59.169 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61305.log 2024-05-11T12:15:59.170 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.41516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47221.log 2024-05-11T12:15:59.170 INFO:teuthology.orchestra.run.smithi167.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41516.log.gz 2024-05-11T12:15:59.171 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.61305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51773.log 2024-05-11T12:15:59.171 INFO:teuthology.orchestra.run.smithi167.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61305.log.gz 2024-05-11T12:15:59.172 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.47221.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47221.log.gz 2024-05-11T12:15:59.173 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45740.log 2024-05-11T12:15:59.174 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.51773.log: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60240.log 2024-05-11T12:15:59.174 INFO:teuthology.orchestra.run.smithi167.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51773.log.gz 2024-05-11T12:15:59.175 INFO:teuthology.orchestra.run.smithi204.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-05-11T12:15:59.176 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.45740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40550.log 2024-05-11T12:15:59.176 INFO:teuthology.orchestra.run.smithi167.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45740.log.gz 2024-05-11T12:15:59.177 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.60240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56931.log 2024-05-11T12:15:59.177 INFO:teuthology.orchestra.run.smithi167.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.60240.log.gz 2024-05-11T12:15:59.178 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.40550.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40550.log.gz 2024-05-11T12:15:59.179 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.56931.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56931.log.gz 2024-05-11T12:15:59.180 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45076.log 2024-05-11T12:15:59.181 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59077.log 2024-05-11T12:15:59.183 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.45076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45698.log 2024-05-11T12:15:59.183 INFO:teuthology.orchestra.run.smithi167.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45076.log.gz 2024-05-11T12:15:59.184 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.59077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53799.log 2024-05-11T12:15:59.184 INFO:teuthology.orchestra.run.smithi167.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59077.log.gz 2024-05-11T12:15:59.186 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.45698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55218.log 2024-05-11T12:15:59.186 INFO:teuthology.orchestra.run.smithi167.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45698.log.gz 2024-05-11T12:15:59.187 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.53799.log: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59944.log 2024-05-11T12:15:59.187 INFO:teuthology.orchestra.run.smithi167.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53799.log.gz 2024-05-11T12:15:59.189 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.55218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47383.log 2024-05-11T12:15:59.189 INFO:teuthology.orchestra.run.smithi167.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55218.log.gz 2024-05-11T12:15:59.190 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.59944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52793.log 2024-05-11T12:15:59.190 INFO:teuthology.orchestra.run.smithi167.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59944.log.gz 2024-05-11T12:15:59.191 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.47383.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47383.log.gz 2024-05-11T12:15:59.193 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.52793.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52793.log.gz 2024-05-11T12:15:59.193 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50915.log 2024-05-11T12:15:59.195 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59284.log 2024-05-11T12:15:59.196 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.50915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56750.log 2024-05-11T12:15:59.196 INFO:teuthology.orchestra.run.smithi167.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50915.log.gz 2024-05-11T12:15:59.197 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.59284.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.59284.log.gz 2024-05-11T12:15:59.198 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.56750.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56750.log.gz 2024-05-11T12:15:59.199 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52835.log 2024-05-11T12:15:59.200 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45412.log 2024-05-11T12:15:59.202 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.52835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46502.log 2024-05-11T12:15:59.202 INFO:teuthology.orchestra.run.smithi167.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52835.log.gz 2024-05-11T12:15:59.203 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.45412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48868.log 2024-05-11T12:15:59.203 INFO:teuthology.orchestra.run.smithi167.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45412.log.gz 2024-05-11T12:15:59.204 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.46502.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46502.log.gz 2024-05-11T12:15:59.205 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48315.log 2024-05-11T12:15:59.205 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.48868.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48868.log.gz 2024-05-11T12:15:59.206 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61579.log 2024-05-11T12:15:59.207 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.48315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55600.log 2024-05-11T12:15:59.208 INFO:teuthology.orchestra.run.smithi167.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48315.log.gz 2024-05-11T12:15:59.210 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.61579.log: gzip -5 --verbose -- 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61579.log.gz 2024-05-11T12:15:59.210 INFO:teuthology.orchestra.run.smithi167.stderr: /var/log/ceph/ceph-client.admin.49968.log 2024-05-11T12:15:59.210 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.55600.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55600.log.gz 2024-05-11T12:15:59.211 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47681.log 2024-05-11T12:15:59.212 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.49968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45618.log 2024-05-11T12:15:59.212 INFO:teuthology.orchestra.run.smithi167.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49968.log.gz 2024-05-11T12:15:59.213 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.47681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45160.log 2024-05-11T12:15:59.214 INFO:teuthology.orchestra.run.smithi167.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47681.log.gz 2024-05-11T12:15:59.215 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.45618.log: gzip -5 --verbose -- 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45618.log.gz 2024-05-11T12:15:59.215 INFO:teuthology.orchestra.run.smithi167.stderr: /var/log/ceph/ceph-client.admin.54368.log 2024-05-11T12:15:59.216 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.45160.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45160.log.gz 2024-05-11T12:15:59.216 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58628.log 2024-05-11T12:15:59.218 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.54368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59003.log 2024-05-11T12:15:59.218 INFO:teuthology.orchestra.run.smithi167.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.54368.log.gz 2024-05-11T12:15:59.219 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.58628.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58628.log.gz 2024-05-11T12:15:59.220 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51665.log 2024-05-11T12:15:59.220 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.59003.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59003.log.gz 2024-05-11T12:15:59.221 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42453.log 2024-05-11T12:15:59.222 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.51665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54695.log 2024-05-11T12:15:59.223 INFO:teuthology.orchestra.run.smithi167.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51665.log.gz 2024-05-11T12:15:59.224 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.42453.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42453.log.gz 2024-05-11T12:15:59.225 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.54695.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54695.log.gz 2024-05-11T12:15:59.226 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57370.log 2024-05-11T12:15:59.227 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52040.log 2024-05-11T12:15:59.229 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.57370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44534.log 2024-05-11T12:15:59.229 INFO:teuthology.orchestra.run.smithi167.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57370.log.gz 2024-05-11T12:15:59.230 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.52040.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.52040.log.gz 2024-05-11T12:15:59.230 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48021.log 2024-05-11T12:15:59.231 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.44534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44909.log 2024-05-11T12:15:59.231 INFO:teuthology.orchestra.run.smithi167.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44534.log.gz 2024-05-11T12:15:59.233 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.48021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48824.log 2024-05-11T12:15:59.233 INFO:teuthology.orchestra.run.smithi167.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48021.log.gz 2024-05-11T12:15:59.235 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.44909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57141.log 87.8% 2024-05-11T12:15:59.235 INFO:teuthology.orchestra.run.smithi167.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44909.log.gz 2024-05-11T12:15:59.236 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.48824.log: gzip 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48824.log.gz -5 2024-05-11T12:15:59.236 INFO:teuthology.orchestra.run.smithi167.stderr: --verbose -- /var/log/ceph/ceph-client.admin.56290.log 2024-05-11T12:15:59.237 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.57141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48399.log 2024-05-11T12:15:59.237 INFO:teuthology.orchestra.run.smithi167.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57141.log.gz 2024-05-11T12:15:59.239 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.56290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58328.log 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56290.log.gz 2024-05-11T12:15:59.239 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-05-11T12:15:59.240 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.48399.log: gzip 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48399.log.gz -5 2024-05-11T12:15:59.240 INFO:teuthology.orchestra.run.smithi167.stderr: --verbose -- /var/log/ceph/ceph-client.admin.56140.log 2024-05-11T12:15:59.242 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.58328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46544.log 2024-05-11T12:15:59.242 INFO:teuthology.orchestra.run.smithi167.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58328.log.gz 2024-05-11T12:15:59.243 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.56140.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56140.log.gz 2024-05-11T12:15:59.244 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42986.log 2024-05-11T12:15:59.245 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.46544.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42116.log 2024-05-11T12:15:59.245 INFO:teuthology.orchestra.run.smithi167.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46544.log.gz 2024-05-11T12:15:59.246 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.42986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51097.log 2024-05-11T12:15:59.246 INFO:teuthology.orchestra.run.smithi167.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42986.log.gz 2024-05-11T12:15:59.247 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.42116.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42116.log.gz 2024-05-11T12:15:59.248 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60600.log 2024-05-11T12:15:59.249 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.51097.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51097.log.gz 2024-05-11T12:15:59.249 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61705.log 2024-05-11T12:15:59.251 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.60600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55260.log 2024-05-11T12:15:59.251 INFO:teuthology.orchestra.run.smithi167.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60600.log.gz 2024-05-11T12:15:59.252 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.61705.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.61705.log.gz 2024-05-11T12:15:59.253 INFO:teuthology.orchestra.run.smithi167.stderr:gzip/var/log/ceph/ceph-client.admin.55260.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.53260.log 2024-05-11T12:15:59.253 INFO:teuthology.orchestra.run.smithi167.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55260.log.gz 2024-05-11T12:15:59.254 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44618.log 2024-05-11T12:15:59.256 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.53260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50365.log 2024-05-11T12:15:59.256 INFO:teuthology.orchestra.run.smithi167.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53260.log.gz 2024-05-11T12:15:59.257 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.44618.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44618.log.gz 2024-05-11T12:15:59.257 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56056.log 2024-05-11T12:15:59.259 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.50365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47147.log 2024-05-11T12:15:59.259 INFO:teuthology.orchestra.run.smithi167.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50365.log.gz 2024-05-11T12:15:59.260 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.56056.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56056.log.gz 2024-05-11T12:15:59.261 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61747.log 2024-05-11T12:15:59.262 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.47147.log: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56224.log 2024-05-11T12:15:59.262 INFO:teuthology.orchestra.run.smithi167.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47147.log.gz 2024-05-11T12:15:59.263 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.61747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48231.log 2024-05-11T12:15:59.263 INFO:teuthology.orchestra.run.smithi167.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.61747.log.gz 2024-05-11T12:15:59.265 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.56224.log: /var/log/ceph/ceph-client.admin.48231.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.56224.log.gz 2024-05-11T12:15:59.266 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55050.log 2024-05-11T12:15:59.266 INFO:teuthology.orchestra.run.smithi167.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48231.log.gz 2024-05-11T12:15:59.267 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41420.log 2024-05-11T12:15:59.268 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.55050.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55050.log.gz 2024-05-11T12:15:59.269 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48273.log 2024-05-11T12:15:59.270 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.41420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62481.log 2024-05-11T12:15:59.270 INFO:teuthology.orchestra.run.smithi167.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41420.log.gz 2024-05-11T12:15:59.272 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.48273.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48273.log.gz 2024-05-11T12:15:59.273 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61105.log 2024-05-11T12:15:59.273 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.62481.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.62481.log.gz 2024-05-11T12:15:59.274 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50280.log 2024-05-11T12:15:59.275 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.61105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61663.log 2024-05-11T12:15:59.275 INFO:teuthology.orchestra.run.smithi167.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.61105.log.gz 2024-05-11T12:15:59.276 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.50280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55643.log 2024-05-11T12:15:59.277 INFO:teuthology.orchestra.run.smithi167.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50280.log.gz 2024-05-11T12:15:59.278 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.61663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50220.log 2024-05-11T12:15:59.278 INFO:teuthology.orchestra.run.smithi167.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61663.log.gz 2024-05-11T12:15:59.279 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.55643.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55643.log.gz 2024-05-11T12:15:59.280 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52709.log 2024-05-11T12:15:59.281 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.50220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60522.log 2024-05-11T12:15:59.281 INFO:teuthology.orchestra.run.smithi167.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50220.log.gz 2024-05-11T12:15:59.282 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.52709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52135.log 2024-05-11T12:15:59.282 INFO:teuthology.orchestra.run.smithi167.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52709.log.gz 2024-05-11T12:15:59.283 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.60522.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60522.log.gz 2024-05-11T12:15:59.284 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39958.log 2024-05-11T12:15:59.285 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.52135.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52135.log.gz 2024-05-11T12:15:59.285 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50052.log 2024-05-11T12:15:59.287 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.39958.log: gzip 88.4% -5 --verbose -- /var/log/ceph/ceph-client.admin.56792.log 2024-05-11T12:15:59.287 INFO:teuthology.orchestra.run.smithi167.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39958.log.gz 2024-05-11T12:15:59.288 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.50052.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50052.log.gz 2024-05-11T12:15:59.288 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52354.log 2024-05-11T12:15:59.290 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.56792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50010.log 2024-05-11T12:15:59.290 INFO:teuthology.orchestra.run.smithi167.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.56792.log.gz 2024-05-11T12:15:59.291 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.52354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61263.log 2024-05-11T12:15:59.291 INFO:teuthology.orchestra.run.smithi167.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52354.log.gz 2024-05-11T12:15:59.292 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.50010.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50010.log.gz 2024-05-11T12:15:59.293 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54174.log 2024-05-11T12:15:59.294 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.61263.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61263.log.gz 2024-05-11T12:15:59.294 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52522.log 2024-05-11T12:15:59.295 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.54174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42446.log 2024-05-11T12:15:59.296 INFO:teuthology.orchestra.run.smithi167.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54174.log.gz 2024-05-11T12:15:59.297 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.52522.log: gzip -5 --verbose 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52522.log.gz 2024-05-11T12:15:59.297 INFO:teuthology.orchestra.run.smithi167.stderr: -- /var/log/ceph/ceph-client.admin.45784.log 2024-05-11T12:15:59.298 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.42446.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42446.log.gz 2024-05-11T12:15:59.299 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42442.log 2024-05-11T12:15:59.300 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.45784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58544.log 2024-05-11T12:15:59.300 INFO:teuthology.orchestra.run.smithi167.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45784.log.gz 2024-05-11T12:15:59.301 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.42442.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42442.log.gz 2024-05-11T12:15:59.303 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.58544.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58544.log.gz 2024-05-11T12:15:59.303 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54779.log 2024-05-11T12:15:59.304 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46730.log 2024-05-11T12:15:59.306 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.54779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44678.log 2024-05-11T12:15:59.306 INFO:teuthology.orchestra.run.smithi167.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54779.log.gz 2024-05-11T12:15:59.308 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.46730.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46730.log.gz 2024-05-11T12:15:59.308 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60153.log 2024-05-11T12:15:59.309 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.44678.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44678.log.gz 2024-05-11T12:15:59.309 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53547.log 2024-05-11T12:15:59.311 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.60153.log: 89.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.60153.log.gz -5 2024-05-11T12:15:59.311 INFO:teuthology.orchestra.run.smithi167.stderr: --verbose -- /var/log/ceph/ceph-client.admin.60663.log 2024-05-11T12:15:59.312 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.53547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46250.log 2024-05-11T12:15:59.312 INFO:teuthology.orchestra.run.smithi167.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53547.log.gz 2024-05-11T12:15:59.314 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.60663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52438.log 2024-05-11T12:15:59.314 INFO:teuthology.orchestra.run.smithi167.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.60663.log.gz 2024-05-11T12:15:59.315 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.46250.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46250.log.gz 2024-05-11T12:15:59.316 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.52438.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52438.log.gz 2024-05-11T12:15:59.316 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58232.log 2024-05-11T12:15:59.318 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61999.log 2024-05-11T12:15:59.320 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.58232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48063.log 2024-05-11T12:15:59.320 INFO:teuthology.orchestra.run.smithi167.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.58232.log.gz 2024-05-11T12:15:59.321 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.61999.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.61999.log.gz 2024-05-11T12:15:59.321 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43764.log 2024-05-11T12:15:59.322 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.48063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57243.log 2024-05-11T12:15:59.322 INFO:teuthology.orchestra.run.smithi167.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48063.log.gz 2024-05-11T12:15:59.323 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.0.43764.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.43764.log.gz 2024-05-11T12:15:59.324 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40340.log 2024-05-11T12:15:59.325 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.57243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53348.log 2024-05-11T12:15:59.325 INFO:teuthology.orchestra.run.smithi167.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57243.log.gz 2024-05-11T12:15:59.327 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.40340.log: gzip -5 --verbose -- 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40340.log.gz 2024-05-11T12:15:59.327 INFO:teuthology.orchestra.run.smithi167.stderr: /var/log/ceph/ceph-client.admin.47105.log 2024-05-11T12:15:59.328 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.53348.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53348.log.gz 2024-05-11T12:15:59.329 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45952.log 2024-05-11T12:15:59.330 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.47105.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.47105.log.gz 2024-05-11T12:15:59.331 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53217.log 2024-05-11T12:15:59.332 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.45952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54216.log 2024-05-11T12:15:59.332 INFO:teuthology.orchestra.run.smithi167.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45952.log.gz 2024-05-11T12:15:59.333 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.53217.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53217.log.gz 2024-05-11T12:15:59.334 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49279.log 2024-05-11T12:15:59.335 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.54216.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54216.log.gz 2024-05-11T12:15:59.335 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50712.log 2024-05-11T12:15:59.337 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.49279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55824.log 2024-05-11T12:15:59.337 INFO:teuthology.orchestra.run.smithi167.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49279.log.gz 2024-05-11T12:15:59.338 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.50712.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50712.log.gz 2024-05-11T12:15:59.338 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56478.log 2024-05-11T12:15:59.340 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.55824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61957.log 2024-05-11T12:15:59.340 INFO:teuthology.orchestra.run.smithi167.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.55824.log.gz 2024-05-11T12:15:59.340 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.0.56478.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.56478.log.gz 2024-05-11T12:15:59.341 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41936.log 2024-05-11T12:15:59.342 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.61957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53441.log 2024-05-11T12:15:59.343 INFO:teuthology.orchestra.run.smithi167.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.61957.log.gz 2024-05-11T12:15:59.344 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.41936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52919.log 2024-05-11T12:15:59.344 INFO:teuthology.orchestra.run.smithi167.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41936.log.gz 2024-05-11T12:15:59.349 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.53441.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53441.log.gz 2024-05-11T12:15:59.349 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.52919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54472.log 2024-05-11T12:15:59.349 INFO:teuthology.orchestra.run.smithi167.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52919.log.gz 2024-05-11T12:15:59.351 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58274.log 2024-05-11T12:15:59.351 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.54472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55972.log 2024-05-11T12:15:59.351 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.58274.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.54472.log.gz 2024-05-11T12:15:59.351 INFO:teuthology.orchestra.run.smithi167.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58274.log.gz 2024-05-11T12:15:59.351 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51227.log 2024-05-11T12:15:59.353 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.55972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46334.log 2024-05-11T12:15:59.353 INFO:teuthology.orchestra.run.smithi167.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55972.log.gz 2024-05-11T12:15:59.354 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.51227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45286.log 2024-05-11T12:15:59.354 INFO:teuthology.orchestra.run.smithi167.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51227.log.gz 2024-05-11T12:15:59.355 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.46334.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46334.log.gz 2024-05-11T12:15:59.356 INFO:teuthology.orchestra.run.smithi167.stderr:gzip/var/log/ceph/ceph-client.admin.45286.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.52667.log 2024-05-11T12:15:59.357 INFO:teuthology.orchestra.run.smithi167.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45286.log.gz 2024-05-11T12:15:59.357 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61029.log 2024-05-11T12:15:59.359 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.52667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59481.log 2024-05-11T12:15:59.360 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.61029.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52667.log.gz 2024-05-11T12:15:59.361 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56014.log 2024-05-11T12:15:59.361 INFO:teuthology.orchestra.run.smithi167.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.61029.log.gz 2024-05-11T12:15:59.362 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.59481.log: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51046.log 2024-05-11T12:15:59.362 INFO:teuthology.orchestra.run.smithi167.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59481.log.gz 2024-05-11T12:15:59.363 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.56014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42117.log 2024-05-11T12:15:59.364 INFO:teuthology.orchestra.run.smithi167.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56014.log.gz 2024-05-11T12:15:59.365 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.51046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60861.log 2024-05-11T12:15:59.365 INFO:teuthology.orchestra.run.smithi167.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51046.log.gz 2024-05-11T12:15:59.366 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.42117.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42117.log.gz 2024-05-11T12:15:59.367 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.60861.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60861.log.gz 2024-05-11T12:15:59.368 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44450.log 2024-05-11T12:15:59.369 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51707.log 2024-05-11T12:15:59.371 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.44450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52582.log 2024-05-11T12:15:59.371 INFO:teuthology.orchestra.run.smithi167.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44450.log.gz 2024-05-11T12:15:59.373 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.51707.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.51707.log.gz 2024-05-11T12:15:59.373 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.52582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49047.log 2024-05-11T12:15:59.374 INFO:teuthology.orchestra.run.smithi167.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52582.log.gz 2024-05-11T12:15:59.374 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49471.log 2024-05-11T12:15:59.376 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.49047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49237.log 2024-05-11T12:15:59.376 INFO:teuthology.orchestra.run.smithi167.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49047.log.gz 2024-05-11T12:15:59.377 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.49471.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49471.log.gz 2024-05-11T12:15:59.378 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51539.log 2024-05-11T12:15:59.379 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.49237.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49237.log.gz 2024-05-11T12:15:59.379 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45868.log 2024-05-11T12:15:59.380 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.51539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59569.log 2024-05-11T12:15:59.381 INFO:teuthology.orchestra.run.smithi167.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51539.log.gz 2024-05-11T12:15:59.382 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.45868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62125.log 2024-05-11T12:15:59.382 INFO:teuthology.orchestra.run.smithi167.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45868.log.gz 2024-05-11T12:15:59.383 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.59569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42740.log 2024-05-11T12:15:59.383 INFO:teuthology.orchestra.run.smithi167.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59569.log.gz 2024-05-11T12:15:59.385 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.62125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54863.log 88.5% -- replaced with /var/log/ceph/ceph-client.admin.62125.log.gz 2024-05-11T12:15:59.385 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-05-11T12:15:59.386 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.42740.log: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55176.log 2024-05-11T12:15:59.386 INFO:teuthology.orchestra.run.smithi167.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42740.log.gz 2024-05-11T12:15:59.388 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.54863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59611.log 2024-05-11T12:15:59.388 INFO:teuthology.orchestra.run.smithi167.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54863.log.gz 2024-05-11T12:15:59.389 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.55176.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55176.log.gz 2024-05-11T12:15:59.390 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-05-11T12:15:59.391 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.59611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44576.log 2024-05-11T12:15:59.391 INFO:teuthology.orchestra.run.smithi167.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.59611.log.gz 2024-05-11T12:15:59.392 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48147.log 2024-05-11T12:15:59.394 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.44576.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44576.log.gz 2024-05-11T12:15:59.394 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.48147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58670.log 2024-05-11T12:15:59.395 INFO:teuthology.orchestra.run.smithi167.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48147.log.gz 2024-05-11T12:15:59.398 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.58670.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.58670.log.gz 2024-05-11T12:15:59.398 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49738.log 2024-05-11T12:15:59.400 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51311.log 2024-05-11T12:15:59.401 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.49738.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49738.log.gz 2024-05-11T12:15:59.402 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49153.log 2024-05-11T12:15:59.404 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.51311.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51311.log.gz 2024-05-11T12:15:59.404 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54611.log 2024-05-11T12:15:59.405 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.49153.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49153.log.gz 2024-05-11T12:15:59.406 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57057.log 2024-05-11T12:15:59.407 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.54611.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54611.log.gz 2024-05-11T12:15:59.407 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39752.log 2024-05-11T12:15:59.409 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.57057.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57057.log.gz 2024-05-11T12:15:59.410 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.39752.log: -- /var/log/ceph/ceph-client.admin.55782.log 2024-05-11T12:15:59.410 INFO:teuthology.orchestra.run.smithi167.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39752.log.gz 2024-05-11T12:15:59.411 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54095.log 2024-05-11T12:15:59.413 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.55782.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55782.log.gz 2024-05-11T12:15:59.414 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.0.54095.log: -- /var/log/ceph/ceph-client.admin.53631.log 2024-05-11T12:15:59.414 INFO:teuthology.orchestra.run.smithi167.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.54095.log.gz 2024-05-11T12:15:59.416 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58060.log 2024-05-11T12:15:59.417 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.53631.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53631.log.gz 2024-05-11T12:15:59.418 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60306.log/var/log/ceph/ceph-client.admin.58060.log: 2024-05-11T12:15:59.418 INFO:teuthology.orchestra.run.smithi167.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.58060.log.gz 2024-05-11T12:15:59.419 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55477.log 2024-05-11T12:15:59.423 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.60306.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.60306.log.gz 2024-05-11T12:15:59.423 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.55477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57675.log 89.5% -- replaced with /var/log/ceph/ceph-client.admin.55477.log.gz 2024-05-11T12:15:59.423 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-05-11T12:15:59.424 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51269.log 2024-05-11T12:15:59.426 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.57675.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57675.log.gz 2024-05-11T12:15:59.426 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60007.log 2024-05-11T12:15:59.427 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.51269.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51269.log.gz 2024-05-11T12:15:59.428 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35091.log 2024-05-11T12:15:59.431 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.60007.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.60007.log.gz 2024-05-11T12:15:59.431 INFO:teuthology.orchestra.run.smithi167.stderr:gzip/var/log/ceph/ceph-client.admin.35091.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.49405.log 2024-05-11T12:15:59.431 INFO:teuthology.orchestra.run.smithi167.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35091.log.gz 2024-05-11T12:15:59.432 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57755.log 2024-05-11T12:15:59.435 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.49405.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.49405.log.gz 2024-05-11T12:15:59.435 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.57755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60777.log 2024-05-11T12:15:59.435 INFO:teuthology.orchestra.run.smithi167.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57755.log.gz 2024-05-11T12:15:59.436 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39862.log 2024-05-11T12:15:59.438 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.60777.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60777.log.gz 2024-05-11T12:15:59.439 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50792.log 2024-05-11T12:15:59.439 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.39862.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39862.log.gz 2024-05-11T12:15:59.440 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48441.log 2024-05-11T12:15:59.441 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.50792.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50792.log.gz 2024-05-11T12:15:59.443 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57878.log 2024-05-11T12:15:59.444 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.48441.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48441.log.gz 2024-05-11T12:15:59.445 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.57878.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.57878.log.gz 2024-05-11T12:15:59.446 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41558.log 2024-05-11T12:15:59.447 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57015.log 2024-05-11T12:15:59.449 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.41558.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41558.log.gz 2024-05-11T12:15:59.449 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49363.log 2024-05-11T12:15:59.450 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.57015.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.57015.log.gz 2024-05-11T12:15:59.452 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.49363.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49363.log.gz 2024-05-11T12:15:59.452 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44492.log 2024-05-11T12:15:59.454 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49005.log 2024-05-11T12:15:59.456 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.44492.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44492.log.gz 2024-05-11T12:15:59.456 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50617.log 2024-05-11T12:15:59.457 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.49005.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49005.log.gz 2024-05-11T12:15:59.458 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53304.log 2024-05-11T12:15:59.459 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.50617.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50617.log.gz 2024-05-11T12:15:59.460 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43196.log 2024-05-11T12:15:59.462 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.53304.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53304.log.gz 2024-05-11T12:15:59.462 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50323.log 2024-05-11T12:15:59.463 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.43196.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43196.log.gz 2024-05-11T12:15:59.464 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51581.log 2024-05-11T12:15:59.465 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.50323.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50323.log.gz 2024-05-11T12:15:59.466 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54410.log 2024-05-11T12:15:59.467 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.51581.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51581.log.gz 2024-05-11T12:15:59.468 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61831.log 2024-05-11T12:15:59.468 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.54410.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54410.log.gz 2024-05-11T12:15:59.470 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42800.log 2024-05-11T12:15:59.471 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.61831.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.61831.log.gz 2024-05-11T12:15:59.471 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51353.log 2024-05-11T12:15:59.472 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.42800.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42800.log.gz 2024-05-11T12:15:59.474 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.51353.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51353.log.gz 2024-05-11T12:15:59.475 INFO:teuthology.orchestra.run.smithi167.stderr:gzip 92.0% -- replaced with /var/log/ceph/ceph-mds.a.log.gz -5 2024-05-11T12:15:59.475 INFO:teuthology.orchestra.run.smithi167.stderr: --verbose -- /var/log/ceph/ceph-client.admin.52877.log 2024-05-11T12:15:59.476 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44720.log 2024-05-11T12:15:59.478 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.52877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47853.log 2024-05-11T12:15:59.478 INFO:teuthology.orchestra.run.smithi167.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52877.log.gz 2024-05-11T12:15:59.479 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.44720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60945.log 2024-05-11T12:15:59.479 INFO:teuthology.orchestra.run.smithi167.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44720.log.gz 2024-05-11T12:15:59.480 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.47853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51139.log 2024-05-11T12:15:59.481 INFO:teuthology.orchestra.run.smithi167.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47853.log.gz 2024-05-11T12:15:59.482 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.60945.log: gzip 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60945.log.gz 2024-05-11T12:15:59.482 INFO:teuthology.orchestra.run.smithi167.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.58190.log 2024-05-11T12:15:59.483 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.51139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57538.log 2024-05-11T12:15:59.483 INFO:teuthology.orchestra.run.smithi167.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51139.log.gz 2024-05-11T12:15:59.485 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.58190.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.58190.log.gz 2024-05-11T12:15:59.486 INFO:teuthology.orchestra.run.smithi167.stderr:gzip/var/log/ceph/ceph-client.admin.57538.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.42760.log 2024-05-11T12:15:59.486 INFO:teuthology.orchestra.run.smithi167.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57538.log.gz 2024-05-11T12:15:59.487 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43112.log 2024-05-11T12:15:59.489 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.42760.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42760.log.gz 2024-05-11T12:15:59.489 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.43112.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43112.log.gz 2024-05-11T12:15:59.646 INFO:teuthology.orchestra.run.smithi167.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-05-11T12:15:59.685 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y has been restored 2024-05-11T12:15:59.685 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x has been restored 2024-05-11T12:15:59.861 INFO:teuthology.orchestra.run.smithi204.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-05-11T12:15:59.863 INFO:teuthology.orchestra.run.smithi204.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-05-11T12:15:59.906 INFO:teuthology.orchestra.run.smithi204.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-05-11T12:15:59.907 INFO:teuthology.orchestra.run.smithi204.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-05-11T12:15:59.910 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-05-11T12:15:59.910 INFO:teuthology.orchestra.run.smithi204.stderr:real 0m1.048s 2024-05-11T12:15:59.911 INFO:teuthology.orchestra.run.smithi204.stderr:user 0m4.109s 2024-05-11T12:15:59.911 INFO:teuthology.orchestra.run.smithi204.stderr:sys 0m0.448s 2024-05-11T12:16:00.064 INFO:teuthology.orchestra.run.smithi167.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-05-11T12:16:00.067 INFO:teuthology.orchestra.run.smithi167.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-05-11T12:16:00.150 INFO:teuthology.orchestra.run.smithi167.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-05-11T12:16:00.199 INFO:teuthology.orchestra.run.smithi167.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-05-11T12:16:00.203 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-05-11T12:16:00.203 INFO:teuthology.orchestra.run.smithi167.stderr:real 0m1.348s 2024-05-11T12:16:00.203 INFO:teuthology.orchestra.run.smithi167.stderr:user 0m4.957s 2024-05-11T12:16:00.203 INFO:teuthology.orchestra.run.smithi167.stderr:sys 0m1.498s 2024-05-11T12:16:00.204 INFO:tasks.ceph:Archiving logs... 2024-05-11T12:16:00.204 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702355/remote/smithi167/log 2024-05-11T12:16:00.205 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-11T12:16:01.206 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702355/remote/smithi204/log 2024-05-11T12:16:01.207 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-11T12:16:01.909 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-11T12:16:01.919 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-11T12:16:01.920 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-11T12:16:01.928 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-11T12:16:02.074 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-11T12:16:02.084 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-11T12:16:02.084 DEBUG:teuthology.orchestra.run.smithi167:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-11T12:16:02.092 DEBUG:teuthology.orchestra.run.smithi204:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-11T12:16:02.139 INFO:teuthology.orchestra.run.smithi167.stderr:bash: line 1: ntpq: command not found 2024-05-11T12:16:02.147 INFO:teuthology.orchestra.run.smithi204.stderr:bash: line 1: ntpq: command not found 2024-05-11T12:16:02.149 INFO:teuthology.orchestra.run.smithi167.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-11T12:16:02.149 INFO:teuthology.orchestra.run.smithi167.stdout:=============================================================================== 2024-05-11T12:16:02.149 INFO:teuthology.orchestra.run.smithi167.stdout:^* hv01.front.sepia.ceph.com 3 6 377 1 +983us[ +983us] +/- 41ms 2024-05-11T12:16:02.150 INFO:teuthology.orchestra.run.smithi167.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 1 -1254us[-1249us] +/- 67ms 2024-05-11T12:16:02.150 INFO:teuthology.orchestra.run.smithi167.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 1 -251us[ -250us] +/- 68ms 2024-05-11T12:16:02.150 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T12:16:02.157 INFO:teuthology.orchestra.run.smithi204.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-11T12:16:02.157 INFO:teuthology.orchestra.run.smithi204.stdout:=============================================================================== 2024-05-11T12:16:02.157 INFO:teuthology.orchestra.run.smithi204.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 1 +1003us[+1003us] +/- 41ms 2024-05-11T12:16:02.157 INFO:teuthology.orchestra.run.smithi204.stdout:^* hv02.front.sepia.ceph.com 3 6 377 2 -1240us[-1228us] +/- 67ms 2024-05-11T12:16:02.157 INFO:teuthology.orchestra.run.smithi204.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 0 -240us[ -240us] +/- 68ms 2024-05-11T12:16:02.157 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T12:16:02.158 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-11T12:16:02.177 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-11T12:16:02.179 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-11T12:16:02.194 DEBUG:teuthology.orchestra.run.smithi167:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-11T12:16:02.414 DEBUG:teuthology.orchestra.run.smithi204:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-11T12:16:02.611 DEBUG:teuthology.orchestra.run.smithi167:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-11T12:16:02.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T12:16:02.707 DEBUG:teuthology.orchestra.run.smithi204:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-11T12:16:02.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T12:16:02.800 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-11T12:16:02.810 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-11T12:16:02.854 INFO:teuthology.task.internal:Duration was 866.316739 seconds 2024-05-11T12:16:02.854 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-11T12:16:02.887 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-11T12:16:02.887 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-11T12:16:02.895 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-11T12:16:03.029 INFO:teuthology.orchestra.run.smithi204.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-11T12:16:03.036 INFO:teuthology.orchestra.run.smithi167.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-11T12:16:03.455 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-11T12:16:03.455 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi167.front.sepia.ceph.com 2024-05-11T12:16:03.455 DEBUG:teuthology.orchestra.run.smithi167:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-11T12:16:03.517 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi204.front.sepia.ceph.com 2024-05-11T12:16:03.517 DEBUG:teuthology.orchestra.run.smithi204:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-11T12:16:03.593 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-11T12:16:03.594 DEBUG:teuthology.orchestra.run.smithi167:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-11T12:16:03.601 DEBUG:teuthology.orchestra.run.smithi204:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-11T12:16:03.684 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-11T12:16:03.684 DEBUG:teuthology.orchestra.run.smithi167:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-11T12:16:03.690 DEBUG:teuthology.orchestra.run.smithi204:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-11T12:16:03.873 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-11T12:16:03.889 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-11T12:16:03.889 DEBUG:teuthology.orchestra.run.smithi167:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-11T12:16:03.963 DEBUG:teuthology.orchestra.run.smithi204:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-11T12:16:04.053 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-11T12:16:04.065 DEBUG:teuthology.orchestra.run.smithi167:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-11T12:16:04.073 DEBUG:teuthology.orchestra.run.smithi204:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-11T12:16:04.167 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern = core 2024-05-11T12:16:04.168 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern = core 2024-05-11T12:16:04.208 DEBUG:teuthology.orchestra.run.smithi167:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-11T12:16:04.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T12:16:04.254 DEBUG:teuthology.orchestra.run.smithi204:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-11T12:16:04.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T12:16:04.310 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-11T12:16:04.321 INFO:teuthology.task.internal:Transferring archived files... 2024-05-11T12:16:04.321 DEBUG:teuthology.misc:Transferring archived files from smithi167:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702355/remote/smithi167 2024-05-11T12:16:04.322 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-11T12:16:04.425 DEBUG:teuthology.misc:Transferring archived files from smithi204:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702355/remote/smithi204 2024-05-11T12:16:04.425 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-11T12:16:04.523 INFO:teuthology.task.internal:Removing archive directory... 2024-05-11T12:16:04.524 DEBUG:teuthology.orchestra.run.smithi167:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-11T12:16:04.531 DEBUG:teuthology.orchestra.run.smithi204:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-11T12:16:04.588 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-11T12:16:04.600 INFO:teuthology.task.internal:Not uploading archives. 2024-05-11T12:16:04.600 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-11T12:16:04.621 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-11T12:16:04.621 DEBUG:teuthology.orchestra.run.smithi167:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-11T12:16:04.627 DEBUG:teuthology.orchestra.run.smithi204:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-11T12:16:04.673 INFO:teuthology.orchestra.run.smithi167.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 11 12:16 /home/ubuntu/cephtest 2024-05-11T12:16:04.684 INFO:teuthology.orchestra.run.smithi204.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 11 12:16 /home/ubuntu/cephtest 2024-05-11T12:16:04.687 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-11T12:16:04.698 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-11T12:16:04.760 INFO:teuthology.run:Summary data: client.0-kernel-sha1: fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e client.1-kernel-sha1: fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/sessionmap} duration: 866.3167390823364 flavor: default owner: scheduled_rishabh@teuthology success: true 2024-05-11T12:16:04.760 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-11T12:16:04.844 INFO:teuthology.run:pass